Here's my problem. I have a plugin I've written for Movable Type, which takes the page MT generates and splits it up into a bunch of subpages. The problem is that in doing this all the UTF8 characters get boned and converted into US-ASCII or something.

For instance, ß, which in UTF8 is 0xC39F, turns into 0xDF.

How do I get it to stop doing this? For the short term, I have inserted calls to utf8::encode from the UTF8 module before I write the string out, to convert them back to UTF8, however I'd really like to fix the thing so it doesn't convert the strings to US-ASCII in the first place.

From what I can tell, Perl does something to mark strings as UTF8 somehow, so I should maybe do something along those lines? I don't know.

Adding "use utf8;" into the script doesn't seem to make a difference. I gather from reading the documentation that it's purely to tell perl that the script itself is in UTF8, not the variables it's manipulating.

Your locale may be set to US-ASCII, thus defaulting to that when you write out the other pages. Check out this page for some more info. Try explicitly setting the locale to UTF8 and see if that fixes it.

Thanks for commenting. I had a try of that, specifically I added "setlocale(LC_CTYPE, "en_US.UTF-8");" to the point in my code before it does all the processing, but as far as I can tell, it didn't make any difference. :(