Performance Problems - High IO Wait

05-14-2008, 12:26 PM

So my database has been running great for a while, and all of a sudden I'm seeing a lot of long running queries (that have been optimized and have been running fine) and my servers io wait starts growing to ~30%. My database is 22 GB, and all but one of my tables are InnoDB. The DB is doing around 1,000 qps.

My DB traffic stays pretty consistent so increased traffic doesn't seem to cause the high io wait times. The only thing I can think of that has changed is the number of rows in my tables. I've got 2 tables that are around 30 million rows.

I'm not a dba. I have done quite a bit of reading on Mysql performance, though, and have tried to tune the Mysql as best I can. I assume our problems are coming from something being configured incorrectly.

Comment

I used to have my buffer pool size set to 12GB, but mysql wouldn't start - I'm pretty sure I ran out of memory. I bumped it back down to 8GB, but I could probably get it back up to around 10GB. But that may only buy me a little time with my database growing.

I guess my question is: since my DB is 22GB+, and I only have 15GB of RAM on the server, does that mean I need to start splitting my database up, or are there other ways to keep performance up with a DB larger than the amount of RAM on the server?