I run the script, but the error raised: UnicodeDecodeError: 'ascii' codec can't decode byte...... However, I tried the code in interactive python command line, there are no errors raised, could you give me the clue?

2 Answers
2

Your call to unicode() is working correctly. It is the concatenation, which is adding a unicode object to a byte string, that is causing trouble. If you change the first line to u'''<tr><td>''', (or u'<tr><td>') it should work fine.

Edit: This means your error lies in the data that is already in html by the time python reaches this snippet:

You should always deal with unicode objects when possible. Decode input to unicode as soon as you can. Encode output to a specific encoding (utf-8, for instance) as late as you can. Use codecs.open() instead of the built-in open(), whenever possible.
–
jcdyerMay 25 '10 at 15:14

If html is not already a unicode object but a normal string, it is converted to unicode when it is concatenated with the converted version of rs[0]. If html now contains special characters you can get a unicode error.

So the other contents of html also need to be correctly decoded to unicode. If the special characters come from string literals, you could use unicode literals (like u"abcä") instead.