iconv

Description

Performs a character set conversion on the string
str from in_charset
to out_charset.

Parameters

in_charset

The input charset.

out_charset

The output charset.

If you append the string //TRANSLIT to
out_charset transliteration is activated. This
means that when a character can't be represented in the target charset,
it can be approximated through one or several similarly looking
characters. If you append the string //IGNORE,
characters that cannot be represented in the target charset are silently
discarded. Otherwise, E_NOTICE is generated and the function
will return FALSE.

Caution

If and how //TRANSLIT works exactly depends on the
system's iconv() implementation (cf. ICONV_IMPL).
Some implementations are known to ignore //TRANSLIT,
so the conversion is likely to fail for characters which are illegal for
the out_charset.

str

The string to be converted.

Return Values

Returns the converted string or FALSE on failure.

Changelog

Version

Description

5.4.0

Since this version, the function returns FALSE on illegal characters,
unless //IGNORE is specified in output charset.
Before, it returned partial output string.

Original : This is the Euro symbol '€'.
TRANSLIT : This is the Euro symbol 'EUR'.
IGNORE : This is the Euro symbol ''.
Plain :
Notice: iconv(): Detected an illegal character in input string in .\iconv-example.php on line 7

User Contributed Notes 47 notes

The "//ignore" option doesn't work with recent versions of the iconv library. So if you're having trouble with that option, you aren't alone.

That means you can't currently use this function to filter invalid characters. Instead it silently fails and returns an empty string (or you'll get a notice but only if you have E_NOTICE enabled).

This has been a known bug with a known solution for at least since 2009 years but no one seems to be willing to fix it (PHP must pass the -c option to iconv). It's still broken as of the latest release 5.4.3.

That will strip invalid characters from UTF-8 strings (so that you can insert it into a database, etc.). Instead of "none" you can also use the value 32 if you want it to insert spaces in place of the invalid characters.

Please note that iconv('UTF-8', 'ASCII//TRANSLIT', ...) doesn't work properly when locale category LC_CTYPE is set to C or POSIX. You must choose another locale otherwise all non-ASCII characters will be replaced with question marks. This is at least true with glibc 2.5.

On some systems there may be no such function as iconv(); this is due to the following reason: a constant is defined named `iconv` with the value `libiconv`. So, the string PHP_FUNCTION(iconv) transforms to PHP_FUNCTION(libiconv), and you have to call libiconv() function instead of iconv().I had seen this on FreeBSD, but I am sure that was a rather special build.If you'd want not to be dependent on this behaviour, add the following to your script:<?phpif (!function_exists('iconv') && function_exists('libiconv')) { function iconv($input_encoding, $output_encoding, $string) { return libiconv($input_encoding, $output_encoding, $string); }}?>Thanks to tony2001 at phpclub.net for explaining this behaviour.

Like many other people, I have encountered massive problems when using iconv() to convert between encodings (from UTF-8 to ISO-8859-15 in my case), especially on large strings.

The main problem here is that when your string contains illegal UTF-8 characters, there is no really straight forward way to handle those. iconv() simply (and silently!) terminates the string when encountering the problematic characters (also if using //IGNORE), returning a clipped string. The

I have found a lot of hints, suggestions and alternative methods (it's scary and in my opinion no good sign how many ways PHP natively provides to convert the encoding of strings), but none of them really worked, except for this one:

There may be situations when a new version of a web site, all in UTF-8, has to display some old data remaining in the database with ISO-8859-1 accents. The problem is iconv("ISO-8859-1", "UTF-8", $string) should not be applied if $string is already UTF-8 encoded.

As orrd101 said, there is a bug with //IGNORE in recent PHP versions (we use 5.6.5) where we couldn't convert some strings (i.e. "∙" from UTF8 to CP1251 with //IGNORE). But we have found a workaround and now we use both //TRANSLIT and //IGNORE flags:$text="∙";iconv("UTF8", "CP1251//TRANSLIT//IGNORE", $text);

So, as iconv() does not always work correctly, in most cases, much easier to use htmlentities(). Example: <?php $content=htmlentities(file_get_contents("incoming.txt"), ENT_QUOTES, "Windows-1252"); file_put_contents("outbound.txt", html_entity_decode($content, ENT_QUOTES , "utf-8")); ?>

I have used iconv to convert from cp1251 into UTF-8. I spent a day to investigate why a string with Russian capital 'Р' (sounds similar to 'r') at the end cannot be inserted into a database.

The problem is not in iconv. But 'Р' in cp1251 is chr(208) and 'Р' in UTF-8 is chr(208).chr(106). chr(106) is one of the space symbol which match '\s' in regex. So, it can be taken by a greedy '+' or '*' operator. In that case, you loose 'Р' in your string.

On my system, according to tests, and also as reported by other people elsewhere, you can combine TRANSLIT and IGNORE only by appending

//IGNORE//TRANSLIT

strictly in that order, but NOT by appending //TRANSLIT//IGNORE, which would lead to //IGNORE being ignored ( :) ).

Anyway, it's hard to understand how one could devise a system of passing options that does not allow to couple both options in a neat manner, and also to understand why the default behaviour should be the less useful and most dangerous one (throwing away most of your data at the first unexpected character). Software design FAIL :-/

Many mail servers don't handle utf-8 correctly as they assume iso-8859-x encodings, so you would want to convert the headers, subject and body of an email prior to sending it out.

If iconv() and mb_convert_encoding() are missing the following function can be used to convert UTF8 to iso-8859-7 encoding. It discards all characters that are not 2-byte greek characters or single-byte (ascii).

Be aware that iconv in PHP uses system implementations of locales and languages, what works under linux, normally doesn't in windows.

Also, you may notice that recent versions of linux (debian, ubuntu, centos, etc) the //TRANSLIT option doesn't work. since most distros doesn't include the intl packages (example: php5-intl and icuxx (where xx is a number) in debian) by default. And this because the intl package conflicts with another package needed for international DNS resolution.

Problem is that configuration is dependent of the sysadmin of the machine where you're hosted, so iconv is pretty much useless by default, depending on what configuration is used by your distro or the machine's admin.

To strip bogus characters from your input (such as data from an unsanitized or other source which you can't trust to necessarily give you strings encoded according to their advertised encoding set), use the same character set as both the input and the output, with //IGNORE on the output charcter set.<?php// assuming '†' is actually UTF8, htmlentities will assume it's iso-8859 // since we did not specify in the 3rd argument of htmlentities.// This generates "&acirc;[bad utf-8 character]"// If passed to any libxml, it will generate a fatal error.$badUTF8 = htmlentities('†');

// iconv() can ignore characters which cannot be encoded in the target character set$goodUTF8 = iconv("utf-8", "utf-8//IGNORE", $badUTF8);?>The result of the example does not give you back the dagger character which was the original input (it got lost when htmlentities was misused to encode it incorrectly, though this is common from people not accustomed to dealing with extended character sets), but it does at least give you data which is sane in your target character set.

Note an important difference between iconv() and mb_convert_encoding() - if you're working with strings, as opposed to files, you most likely want mb_convert_encoding() and not iconv(), because iconv() will add a byte-order marker to the beginning of (for example) a UTF-32 string when converting from e.g. ISO-8859-1, which can throw off all your subsequent calculations and operations on the resulting string.

In other words, iconv() appears to be intended for use when converting the contents of files - whereas mb_convert_encoding() is intended for use when juggling strings internally, e.g. strings that aren't being read/written to/from files, but exchanged with some other media.

You can use iconv in order to access filenames or paths that containt Japanese (and probably also Koreans and Chinese) characters that would normally not be processed correctly by 'copy', 'fopen', 'rename', etc...

Please keep in mind that iconv is not your magic stick by which you can convert anything to your desired encoding sheme.

Even if there is encoding declaration for text you're about to manipulate in PHP, there is always chances that there are some characters or character sequences that are in wrong encoding scheme and despite that your browser (for example) is displaying text to you properly.

These are situations with so-called mixed encoding. You have to:

detect current encoding for your string,

repair broken encoding sheme (here I'm struggling with different methods and third-party libraries such as forceutf8 on GitHub),

If you get this error message: "Notice: iconv(): Detected an illegal character in input string in file.php on line x", and your text or database is likely to contain text copied from Microsoft Word documents, it's very likely that the error is because of the evil 0x96 "long dash" character. MS Word as default converts all double hyphens into this illegal character. The solution is either to convert 0x96 (dash) into the regular 0x2d (hyphen/minus), or to append the //TRANSLIT or //IGNORE parameters (se above).

If you need to strip as many national characters from UTF-8 as possible and keep the rest of input unchanged (i.e. convert whatever can be converted to ASCII and leave the rest), you can do it like this: