Sample Datasets for Benchmarking and Testing

Sometimes you just need some data to test and stress things. But randomly generated data is awful — it doesn’t have realistic distributions, and it isn’t easy to understand whether your results are meaningful and correct. Real or quasi-real data is best. Whether you’re looking for a couple of megabytes or many terabytes, the following sources of data might help you benchmark and test under more realistic conditions.

Related

Author

Share this post

Comments (21)

It should be noted that imdb has some restrictive licensing that would prevent anyone from making a data set publicly available. While the data is quite good and interesting, it would be of limited utility since it isn’t in database-ready form (IIRC).

using the data in a training course doesn’t seem like personal or non-commercial use.

“IMDb grants you a limited license to access and make personal use of this site and not to download (other than page caching) or modify it, or any portion of it, except with express written consent of IMDb. This site or any portion of this site may not be reproduced, duplicated, copied, sold, resold, visited, or otherwise exploited for any commercial purpose without express written consent of IMDb. This license does not include any resale or commercial use of this site or its contents or any derivative use of this site or its contents. ”

I’m not trying to be a pain, but rather to educate myself on these issues. When I was at MySQL/Sun/Oracle, we were expressly forbidden from trying to create any test datasets based on the data by their fleet of lawyers

Hmm. If we were to ask a lawyer right now, I bet they’d scare us and tell us whatever we do, don’t admit wrongdoing. I think the best thing to do is say that we should probably find or generate a dataset whose licensing clearly permits using it for examples in our courses, and thank you for your feedback, Patrick.

For relational work, the csv dumps are probably the quickest bet, but it does contain multivalued attributes. The quadruples dump is a normalized format. I haven’t gotten round to building a normalized format from that one.

Where are the freely available literature citation datasets? DBLP seems to only cover compsci mostly. I need a REALLY big citation dataset over many disciplines and publishing houses (ACM, IEEE, Springer, etc.). Anyone?

A big problem with these data sets are that they are small, trivial cases, which limits the amount and kind of testing you can do. Large data sets exist but they are often implausibly large to move around over the Internet. You can use the listed data sets to easily test basic correctness but you can’t use them to test scaling behaviors.

Synthetic data sets are not interesting but neither are they random or unrealistic if built by a competent designer. The great thing about synthetic data set generators beyond producing data of unbounded size is that you can configure arbitrary distributions and properties of the data that test a broad range of characteristics not possible with real-world data sets. For example, if I want to simulate several types of skew and bias that move logically in space and time over the properties of the data set, it is pretty simple to do that. A good example of this is location and sensing data, which has fairly complex skew patterns in reality that will break most spatial indexing systems at scale — it is hard to get a data set that demonstrates this, but it is fairly easy to generate a synthetic data set that generates the same bulk behavior.

Purely synthetic data set generators can very accurately model real-world data patterns, it just requires the ability to generate complex skew behaviors and interactions in the data that do not rise above the noise floor in trivial samples. What would be useful, possibly more useful than sample data sets, is building a collection of synthetic workload generators with parameter sets that exactly match the distribution, dynamics, and skew of real-world data sets. The results are not interesting per se but they allow you to characterize runtime behaviors under all sorts of assumptions at arbitrary scales. For complex data sets like real-time spatial and graphs, synthetic is really the only way to get an accurate measure of a system.

Real-world data sets would be preferable in theory but in practice you cannot test a lot of things that matter using them. I would not suggest using badly designed randomly generated data sets but a good synthetic data set generator can be an excellent tool.

Thanks for this post. I am currently searching for a dataset of blogs or forums. I need to use them to test different spam detection techniques (as part of my studies )but I did not manage to find any available data . Ideally it should be realistic data that contains both spam comments and realistic comments.

Do you have an idea of where I can find such datasets please :s as I’ve been searching for a long time now and I did not find anything useful.

Thanks for this list, some of them are really useful especially the airline data, that is very difficult to generate. For other stuff like e-mail addresses etc. ,I used to use a tool like generatedata.com. Nice and free, but very limited, so about a year ago I decided to write my own generator that can be used directly in your application code.

http://www.data.gov also appears to have a lot of potentially good data sets (census, earthquakes, etc.) so you don’t have to search individual places like the census bureau or national earthquake information center.

I hv 24 tables in the schema, and needs to use DML and select operations on these tables for 24 tables to check the db performance testing. Can someone give the idea how can I generate the load on db??