This is probably exclusively of interest to my coworkers, but MongoDB has a new fail points framework. Fail points make it easier to test things that are hard to fake, like page faults or network errors. Basically, you create a glorified boolean called a fail point, which you can turn on and off while mongod is running.

To show how this works, I’ll modify the humble “ping” command. “ping” is fairly simple:

> db.runCommand({"ping" : 1})
{ "ok" : 1 }

I’d like to make the response include a "pong" : 1 field, on command.

The ping command is defined in src/mongo/db/dbcommands_generic.cpp. To add a fail point, we first have to include the fail points header at the top of the file:

Now recompile the database. By default, mongod doesn’t allow failpoints to be run. To even allow the possibility of fail points being triggered, you have to run mongod with the --setParameter enableTestCommands=1 option.

$ ./mongod --setParameter enableTestCommands=1

Note: as of this writing, you cannot enable failpoints with the setParameter command, you must start the database with this option.

The failpoint still isn’t turned on, so if you run db.runCommand({ping:1}), you can see that there’s still just the “ok” field. You can enable the fail point with the configureFailPoint command:

Possible modes are "alwaysOn", "off", and {"times" : 37} (which would be on for the next 37 times the fail point is hit… obviously the value for “times” is configurable).

This is a derpy example, but I’ve found it super helpful for debugging concurrency issues where I need to force a thread to block until another thread has done something. You can do that with something like:

while (MONGO_FAIL_POINT(looper)) {
sleep(0);
}

If you wanted to merely delay something, say, immitate a slow connection, you can use MONGO_FAIL_POINT_BLOCK to pass in information: