…when trying to load a XML document using one of the .NET XML API objects like XmlReader, XmlDocument, or XDocument? Was “0x[whatever]” by chance one of these characters?

0x00

0x01

0x02

0x03

0x04

0x05

0x06

0x07

0x08

0x0B

0x0C

0x0E

0x0F

0x10

0x11

0x12

0x13

0x14

0x15

0x1A

0x1B

0x1C

0x1D

0x1E

0x1F

0x16

0x17

0x18

0x19

0x7F

The problem that causes these XmlExceptions is that the data being read or loaded contains characters that are illegal according to the XML specifications. Almost always, these characters are in the ASCII control character range (think whacky characters like null, bell, backspace, etc). These aren’t characters that have any business being in XML data; they’re illegal characters that should be removed, usually having found their way into the data from file format conversions, like when someone tries to create an XML file from Excel data, or export their data to XML from a format that may be stored as binary.

The decimal range for ASCII control characters is 0 – 31, and 127. Or, in hex, 0x00 – 0x1F. (The control character 0x7F is not disallowed, but its use is “discouraged” to avoid compatibility issues.) If any character in the string or stream that contains the XML data contains one of these control characters, an XmlException will be thrown by whatever System.Xml or System.Xml.Linq class (e.g. XmlReader, XmlDocument, XDocument) is trying to load the XML data. In fact, if XML data contains the character ‘\b’ (bell), your motherboard will actually make the bell sound before the XmlException is thrown.

There are a few exceptions though: the formatting characters ‘\n’, ‘\r’, and ‘\t’ are not illegal in XML, per the 1.0 and 1.1 specifications, and therefore do not cause this XmlException. Thus, if you’re encountering XML data that is causing an XmlException because the data “contains invalid characters”, the feeds you’re processing need to be sanitized of illegal XML characters per the XML 1.0 specification (which is what System.Xml conforms to—not XML 1.1) should be removed. The methods below will accomplish this:

Useful as these methods are, don’t go off pasting them into your code anywhere. Create a class instead. Here’s why: let’s say you use the routine to sanitize a string in one section of code. Then another section of code uses that same string that has been sanitized. How does the other section positively know that the string doesn’t contain any control characters anymore, without checking? It doesn’t. Who knows where that string has been (if it’s been sanitized) before it gets to a different routine, further down the processing pipeline. Program defensive and agnostically. If the sanitized string isn’t a string and is instead a different type that represents sanitized strings, you can guarantee that the string doesn’t contain illegal characters.

Now, if the strings that need to be sanitized are being retrieved from a Stream, via a TextReader, for example, we can create a custom StreamReader class that will skip over illegal characters. Let’s say that you’re retrieving XML like so:

But creating a class that inherits from StreamReader and avoiding the costly string-building operation performed by SanitizeXmlString() is much more efficient. The class will have to override a couple methods when it’s finished, but when it is, a Stream could be consumed and sanitized like this instead:

To get this XmlSanitizingStream working correctly, we’ll first need to override two methods integral to the StreamReader: Peek(), and Read(). The Read method should only return legal XML characters, and Peek() should skip past a character if it’s not legal.

Next, we’ll need to override the other Read* methods (Read, ReadToEnd, ReadLine, ReadBlock). These all use Peek() and Read() to derive their returns. If they are not overridden, calling them on XmlSanitizingStream will invoke them on the underlying base StreamReader. That StreamReader will then use its Peek() and Read() methods, not the XmlSanitizingStream’s, resulting in unsanitized characters making their way through.

To make life easy and avoid writing these other Read* methods from scratch, we can disassemble the TextReader class using Reflector, and copy its versions of the other Read* methods, without having to change more than a few lines of code related to ArgumentExceptions.

The complete version of XmlSanitizingStream can be downloaded here. Rename the file extension to “.cs” from “.doc” after downloading.