MySQL Series: How to Detect UTF-8 and Multi-byte Characters

Vance Lucas — Wed, 24 Mar 2010 15:44:39 +0000

Multi-byte characters can cause quite a few headaches for the unsuspecting webmaster. Sometimes all you need to do to figure out how to fix the problem is detect which database records have UTF-8 data in them and which ones do not. If you’ve been scanning records manually, stop now. Here’s a quick query to cure your ales:

Return all the rows with multi-byte characters in table posts on field title:

1
2
3

SELECT *
FROM posts
WHERE LENGTH(title) <> CHAR_LENGTH(title)

This does pretty much what it looks like: returns all the rows in the posts table where the title’s character length does not match the title’s length. This works because the LENGTH function returns the number of bytes in the string, while the CHAR_LENGTH function returns the number of characters in the string. If the string contains multi-byte characters (individual characters that are made up of more than one byte) like international UTF-8 characters, the two functions will return different numbers and will not be equal, thus including that row in your results.

MySQL Series: Return NULL Values First With Descending Order

Vance Lucas — Wed, 16 Dec 2009 21:28:55 +0000

Sometimes there are unique situations where you need to order query results by a particular field in descending order, but also need NULL values first. The default (and logical) behavior of MySQL in this case is to return NULL values last, because in descending order they have the lowest value (none). But what if you really need to reverse this and force NULL values to the top of the result set?

I recently ran into this situation with movie results. The business requirement was that the movies be displayed in descending order by release date, so that the newest ones appeared first. Okay, pretty simple – until I looked at the results. Turns out, we had data on movies that were currently in production, but did not yet have a release date set. I accounted for this earlier by allowing the release date column to be NULL in the table schema, but forgot about the ramifications of that on my queries. These movies were obviously newer, but were always displayed last on the results page because of their NULL release dates. Sounds like the perfect scenario for a conditional statement.

Using the MySQL Control Flow Functions, I quickly crafted an IF() statement in the SELECT portion of the query that looked like this:

1	IF(m.date_released IS NULL OR m.date_released = '0000-00-00', 1, 0) AS in_production

So if the release date has a NULL value or has been inserted as a blank string and formatted to ‘0000-00-00′, it is given a value of 1, and 0 otherwise. We then sort descending by our new ‘in_production’ alias in the ORDER BY clause to get the NULL or empty values on top (assigned a 1 value), and then by release date and other criteria second.

The whole query looks something like this:

1
2
3

SELECT m.*, IF(m.date_released IS NULL OR m.date_released = '0000-00-00', 1, 0) AS in_production
FROM movies AS m
ORDER BY in_production DESC, m.date_released DESC

It’s an easy way to manipulate the result set in MySQL without slowing down the query or issuing multiple queries.

Vance Lucas » MySQL

MySQL Series: How to Detect UTF-8 and Multi-byte Characters

MySQL Series: Return NULL Values First With Descending Order