In Mathematica v8 i am using this convoluted way to read the contents of a unicode file saved in utf-8 format
txt = Import["file.txt",CharacterEncoding -> "UTF-8"]
w = ReadList[StringToStream[txt], Record, RecordLists -> True]
the output like this:
{{unicode chars},{unicode chars},{unicode chars}}
the letters displayed correctly even if i don't use CharacterEncoding -> "UTF-8"
but using
ReadList["file.txt", Record]
will return the file as a garbage characters , and setting $SystemCharacterEncoding = "UTF-8"
$CharacterEncoding = $SystemCharacterEncoding
does not cure the problem since ReadList can't accept CharacterEncoding -> "UTF-8" in its syntax unlike Import.
are there some cure to this phenomena.
thanks
eros