Bug of the day: How ConfigParser ruined my calculations

December 7, 2011

For my thesis, I have implemented a series of numerical applications. The core application (written in C++) uses simple INI-like configuration files for reading system parameters. For example:

[system]
name=example2
r1=1.0
r2=1.5
V1=-0.05
V2=0.1
[...]

Nothing exciting there, reading this is a breeze with QSettings. Now I needed to do some symbolic calculations, for which I chose Python and the excellent SymPy module. To read the system configuration, I chose the standard ConfigParser module.

This code gets the parameters in the system-specific code. Nothing exciting here. Except, stuff doesn’t work. The results of the following calculations suddenly don’t match my expectations (or the predictions made by my other programs).

Since the code after the snippet above is about 30 lines of SymPy magic (accumulating expressions that print dozens of lines onto my terminal), I suspected that some error occurred there. After about two days of searching for the problem there, and calculating everything by hand, something occurred to me when I looked at the debug output:

DEBUG V1 = -0.1

Didn’t the configuration file say “V1=-0.05″? Let’s have a look at the parameter dict:

{'v1': -0.05, 'v2': 0.1, 'r1': 1.0, 'r2': 1.5}

See the problem? “v1″ has a lower-case “v”, so params.get("V1", -0.1) fails and returns the default value (-0.1). One glimpse at the documentation says that

parser.optionxform = str

solves the problem. Gah!

Share this:

Related

6 Responses to “Bug of the day: How ConfigParser ruined my calculations”

Your “solution” could be quite dangerous, since you convert Unicode-Strings into Byte-Strings (in Python 2.x). So it would be better to use the proposed solution in the documentation:
“parser.optionxform = lambda option: option“
That way you do not transform anything.

I used exactly the proposed solution from the documentation. That aside, the difference between byte and Unicode strings is irrelevant for me in this case, since all configuration files are written by hand, by me.

This is why in industry we test modules separately. That way you know your algorithm is working (or not), even if the load/store fails. Unit tests work wonders.

For all the nay-sayers, a unit test may not have caught this particular bug. However it would have narrowed the place where the bug could exist closer to the right spot, and therefore make it easier to fix.