12.5.2 Regular Expressions

A regular expression is a powerful way of specifying a pattern
for a complex search.

MySQL uses Henry Spencer's implementation of regular
expressions, which is aimed at conformance with POSIX 1003.2.
MySQL uses the extended version to support pattern-matching
operations performed with the
REGEXP operator in SQL statements.

This section summarizes, with examples, the special characters
and constructs that can be used in MySQL for
REGEXP operations. It does not
contain all the details that can be found in Henry Spencer's
regex(7) manual page. That manual page is
included in MySQL source distributions, in the
regex.7 file under the
regex directory. See also
Section 3.3.4.7, “Pattern Matching”.

Regular Expression Operators

Performs a pattern match of a string expression
expr against a pattern
pat. The pattern can be an
extended regular expression, the syntax for which is
discussed later in this section. Returns
1 if expr
matches pat; otherwise it returns
0. If either
expr or
pat is NULL,
the result is NULL.
RLIKE is a
synonym for REGEXP, provided
for mSQL compatibility.

The pattern need not be a literal string. For example, it
can be specified as a string expression or table column.

Note

Because MySQL uses the C escape syntax in strings (for
example, “\n” to represent
the newline character), you must double any
“\” that you use in your
REGEXP strings.

REGEXP and
RLIKE use
the character set and collations of the arguments when
deciding the type of a character and performing the
comparison. If the arguments have different character sets
or collations, coercibility rules apply as described in
Section 10.1.7.5, “Collation of Expressions”.

Warning

The REGEXP and
RLIKE
operators work in byte-wise fashion, so they are not
multibyte safe and may produce unexpected results with
multibyte character sets. In addition, these operators
compare characters by their byte values and accented
characters may not compare as equal even if a given
collation treats them as equal.

Syntax of Regular Expressions

A regular expression describes a set of strings. The simplest
regular expression is one that has no special characters in it.
For example, the regular expression hello
matches hello and nothing else.

Nontrivial regular expressions use certain special constructs so
that they can match more than one string. For example, the
regular expression hello|word matches either
the string hello or the string
word.

As a more complex example, the regular expression
B[an]*s matches any of the strings
Bananas, Baaaaas,
Bs, and any other string starting with a
B, ending with an s, and
containing any number of a or
n characters in between.

A regular expression for the REGEXP
operator may use any of the following special characters and
constructs:

Matches any character that is (or is not, if ^ is used)
either a, b,
c, d or
X. A - character
between two other characters forms a range that matches all
characters from the first character to the second. For
example, [0-9] matches any decimal digit.
To include a literal ] character, it must
immediately follow the opening bracket [.
To include a literal - character, it must
be written first or last. Any character that does not have a
defined special meaning inside a [] pair
matches only itself.

Within a bracket expression (written using
[ and ]), matches the
sequence of characters of that collating element.
characters is either a single character
or a character name like newline. The
following table lists the permissible character names.

The following table shows the permissible character names
and the characters that they match. For characters given as
numeric values, the values are represented in octal.

Within a bracket expression (written using
[ and ]),
[=character_class=] represents an
equivalence class. It matches all characters with the same
collation value, including itself. For example, if
o and (+) are the
members of an equivalence class, [[=o=]],
[[=(+)=]], and [o(+)]
are all synonymous. An equivalence class may not be used as
an endpoint of a range.

[:character_class:]

Within a bracket expression (written using
[ and ]),
[:character_class:] represents a
character class that matches all characters belonging to
that class. The following table lists the standard class
names. These names stand for the character classes defined
in the ctype(3) manual page. A particular
locale may provide other class names. A character class may
not be used as an endpoint of a range.

These markers stand for word boundaries. They match the
beginning and end of words, respectively. A word is a
sequence of word characters that is not preceded by or
followed by word characters. A word character is an
alphanumeric character in the alnum class
or an underscore (_).

To use a literal instance of a special character in a regular
expression, precede it by two backslash (\) characters. The
MySQL parser interprets one of the backslashes, and the regular
expression library interprets the other. For example, to match
the string 1+2 that contains the special
+ character, only the last of the following
regular expressions is the correct one:

The regular expression support in MySQL seems to be based on traditional-style regex (like ereg() in PHP), not the more sophisticated regular expression matching found in Perl or PHP's preg_match(). And in case the above doesn't make it clear (being mostly SELECT statements using the function directly to return a 1 or 0), you typically would use the REGEXP function in a WHERE clause like this:

SELECT * FROM foo WHERE bar REGEXP "baz"

To match a "special" character such as $, you need to prefix it with the backslash \ character. So \$ matches an actual dollar sign. However, in almost any programming language that claims a "C-like" syntax, that backslash is likely to get picked up on as a special character. So you may need to use an extra backslash. Also, at least in Perl and PHP, the $ is a special character itself, because it indicates that what follows is a variable name - so it will need a backslash too.

In Perl or PHP, you probably will write something like this to match on a line starting with a $ sign:

Now if you print $query, it will have the valueSELECT * FROM `foo` WHERE `bar` REGEXP "^\$"which is what you really want, and how you would type it into the mysql command line. Remember also that PHPMyAdmin expects you to put a backslash before a backslash or apostrophe. So in PHPMyAdmin you would enterSELECT * FROM `foo` WHERE `bar` REGEXP "^\\$"

I guess if you only want to use . and .* regular expressions, you may as well stick to using LIKE with the _ and % wildcards, as that is probably a bit faster. Finally, when using regular expressions in *any* language you need to watch out, because it is very easy to write ones that will always match, and almost as easy to write ones that will never match anything. So do check!

Regexp's are pretty complicated. If you need anything more complicated than what's shown above, a good site to learn how to use them is http://www.regular-expressions.info. The sites listed in the second comment are probably a good idea too.

Also, don't get in the habit of using character classes to escape metacharacters (like using [(] to match a literal parenthese). The open parentheses '(' and ')' have no special meaning inside a character class, but others like $ retain their meaning. Using [$] won't match the character '$', it'll still match the beginning of the string.

So escape them properly with backslashes.

So what if your PHP code has something that looks like...

mysql_query("select * from a where Name rlike '(^|//////|)example'")

...just to match a string that starts with "example" or contains the string "|example". It looks terrible and it works.

For those of you struggling to escape special characters with long sequences of backslashes (see Vaz's post), I have to ask: why bother? Why not just use the dot-character syntax mentioned on this very page? For example, I needed to find all the ID attributes in some HTML. I tried escaping single and double-quotes for about 30 seconds, then I just switched to this:

SELECT * FROM site WHERE html REGEXP "id=[[.apostrophe.][.quotation-mark.]]archives[[.apostrophe.][.quotation-mark.]]";

created tables eg. the collate 'latin1_swedish_ci' have to be changed, if you want to use case sensitive REGEXPs like [[:upper:]] or [[:lower:]]!ALTER TABLE <name> CONVERT TO CHARACTER SET latin1 COLLATE latin_general_cs

The above post by Guido Dieterich (about collation and case sensitivity) is a good point. However, there is a way to match in a case-sensitive manner without having to change the collation of your existing tables: use the "BINARY" keyword.

Here's an extended example (based on the one previously posted):

SELECT('WORD' REGEXP '[[:upper:]]{4}') AS `upper_match`, # this will be a 1('WORD' REGEXP '[[:lower:]]{4}') AS `lower_match`, # this will be a 1 on an "*_ci" collation# -BINARY- matches below(BINARY 'WORD' REGEXP '[[:upper:]]{4}') AS `bin_upper_match`, # this will be a 1(BINARY 'WORD' REGEXP '[[:lower:]]{4}') AS `bin_lower_match` # this will be a 0 even on an "*_ci" collation

-place the fixed/bigger part first or "fail fast", for ex: instead of (e|abcd) use (abcd|e)-extract similitudes, ex: instead of (abcd|abef) use ab(cd|ef)-prefere LIKE with % and _ , or string functions if the match is simple-try to create less cycles(regex use a lot backtracking)!!!

I can't believe I'm the first to post this solution, since the pain of MySQL's REGEXP not working with multibyte character sets has been expressed all over the Web for years, I see.

Here's my simple workaround, for a database, server, and current connection that are entirely in UTF-8. Of course this only helps the majority of us developers who are in fact dealing with stored data that could just as well have be expressed in latin1:

Maybe this will be useful for others.While creating a search function with syntax-highlighting for content which include bulletin board codes ([code], [ul], [li], etc.) I was stuck for a few hours about how to exclude the contents of the BBC tags. For example, if someone search for "ode", the [code] tags won't needed in the result list, not to mention that the syntax highlighting destroyed the html code as well :)

I was only able to do it this way ($search is the search string, passed from php):

I struggled with the [:character_type:] element of the regexp. I had a problem where users were entering artist names with various degrees of spacing in the name... for example "Loudog" and "Lou Dog".

To get around this, I changed the string being searched for, in code, to: "[:space:]*L[:space:]*o[:space:]*u[:space:]*D[:space:]*o[:space:]*g[:space:]*" but this wasn't working.

Eventually someone spotted that I was essentially searching for zero-or-more instances of the characters :, s, p, a, c or e. This search should instead have been: "[[:space:]]*L[[:space:]]*o[[:space:]]*u[[:space:]]*D[[:space:]]*o[[:space:]]*g[[:space:]]*"

It's not clear in the examples above that this is what you should be searching for. This is roughly equivelent to "/\s+L\s+o\s+u\s+D\s+o\s+g\s+/i"

Using ORed decompositions is fine for my simple example, but in actual data, the length of the decompositions may be well beyond the point where they would be worth doing over just using two fields... or better yet if MySQL would implement the negation operator ?!.

Also, decompositions are much less clear in what they are doing. It's pretty obvious what the negation operator is doing. When I have to revisit a regex 2 years from now to add something to it, I'm going to want to smack someone that used a decomp.