Who needs intended XHTML? The only XHTML formatting needed to keep perfect readability are line breaks after all non-pair elements and block pair elements.

The slowest server-side stuff is probably function page_rows(); On one of my websites it runs 60 seconds and than it dies:
Fatal error: Maximum execution time of 60 seconds exceeded in C:\...\wp-includes\query.php on line 182

Bad pages management prevent WordPress from being a No.1 CMS out there.

WordPress cannot (in theory) be made much faster, unless entire pages are made static with a caching system and it's core is not even loaded for the visitor. It's simply a limit of WordPress's architecture and would require an entire core rewrite to change. If you have mysql caching turned on and an opcode cache (apc, eaccelerator, xcache, etc) that's the most you can do before solutions like wp-cache.

Speed improvements are good but you have to realize that you cannot have a zillion plugins and then expect things to run smoothly. There is always a trade off between functionality and speed. Look at operating systems, the more stuff you have to preload the longer it takes.