I'm planning to move one of my development database servers to its own machine. I'm currently running it on my laptop, which is not very convenient.

DB Servel is PostgreSQL 9.0.1,

OS is Gentoo.

There is only one development server attached to the db, so there's not many queries. The problem is the amount of data: I need to store a copy of planet.osm, which is over 170 Gb at the moment, and increasing.

I'm looking for a solution that doesn't require me to buy expensive hardware, as this is not a production server anyway. I might even be able to reuse some low-end machines I have, with the only exception being their hard drive.

So my question is, what kind of hard drive should I be looking for?

I am currently using a relatively cheap 1.5 Tb external USB drive with a single btrfs partition on it, but I'm not sure that would be fast enough for a db server to find a needle in a 170+ Gb haystack on it.

What tools can I use to measure hdd performance on Linux?

Note that this question is somewhat related to this one, with the main difference being I'm not building a production server.

Thank you very much for the hard drive suggestions. I think I'll go with a RAID10, although I'll need to read more on the subject. But these are good advice for starters, thanks.
–
Attila O.Oct 22 '10 at 16:15

You need to remember though that the f2 layout is a bit slower than the n2 (the typical configuration) with write, it's nothing major: about 10-20% slower for streaming writes, random will be even slower, but I don't know the specific numbers.
–
Hubert KarioOct 22 '10 at 16:34

I would not use a USB drive -- The performance will be suck-tastic. Invest some money in a machine with moderately fast SATA drives. Consider RAID if you think you'll need the redundancy.
You can get a 1U system that's suitable for development for around $1000 or less (think Iron Systems or really low end Dell -- Remember that performance is not paramount in development environments, it just needs to be "good enough").

Another option would be spending $6000-10000 on a VMWare host & virtualizing your whole environment (this would be my suggestion -- It makes life a lot easier & you get the flexibility of being able to spawn copies of your development environment to try out changes or do pre-release testing. Downside here is virtualized performance is almost always worse than a dedicated box, but like I said above performance in Dev isn't always the ultimate goal.

Re: your benchmarking question, this google search is probably a good place to start, but honestly I wouldn't worry about it for a development environment. Benchmarking disk is really most useful when you're comparing systems...

Yep, while the USB drives are handy for a laptop, it seems I'll need to go with SATA drives. I don't plan on virtualizing mu whole environment right now, although that might be an option to consider for the future. Thanks for your help.
–
Attila O.Oct 22 '10 at 16:12

I would recommend a pair of Samsung 840 SSDs in RAID-10. You will need to tweak Postgresql to get best performance, but for small random reads you should expect a performance boost of something like 15x compared to a hard drive.