I have been facing the same issue with over 16000 posts and have seen the performance of my site degrade. But I just put in a solution that did the trick. My initial load query is down from 6.8s to 0.4 seconds and the heaviest category lookup has come down from 4.3 seconds to .4 seconds as well.

I tried your code and it does improve my wordpress performance alot. However, when I click on next page, the code breaks. My blog has about 600 posts and 1000 comments. Currently, I have to set very long timeout for WP-Cache plugin so that the slow performance of my blog won't affect too many users. However, I hope somebody could tell me how to fix this problem.
I run wordpress 2.0.2. Do I have to upgrade it to get better performance?