Whether you believe in small or big government, one thing is for certain – it has some very big data on its hands. Some of this is freely available, such as the census data that was recently released. They have so much “big data” in fact, that the US government even went so far as to set up an entire subcommittee on the topic.

Tokutek has been privileged to be in the middle of many of these conversations. This includes discussions at the state level, where Tokutek was the sole database company recently invited to an international business discussion at the Massachusetts statehouse. Several of our recent talks and product enhancements have also squarely hit upon federal government needs, including:

Big Data presents a wealth of information; however harnessing it can be a struggle. One area for this is machine-generated data, such as sensor networks, can provide a fire hose of useful data, but can also be burdensome to work with. Earlier this year, our CTO spoke at the Morrelly Homeland Security Center about how to handle data ingestion rates, a critical area for defense systems.

The US government certainly has some sizable challenges these days, with large deficits and rancorous politics. Some of these issues almost seem intractable. But one area where technology has a clear way to help the government is with its big data problems. We’ve been grateful to be able to work on some of the most interesting challenges and learn quite a bit from the community.