It takes 80% of the time to find the issue with not performing application and non scalable application. And 20% of the time to fix it.

Practices to find the issues with the code :

1. Monitoring the system/application
2. Always do performance test your code after each release in the production.
3. Keep the historical data of the application of releases to find the culprit in the release, if performance goes down.
4. Modularize the components – very very important. Infact it is the key if u want a scaleable application.
5. Monitor the application – HypericHQ a monitoring tool : Wide variety of data JVM, JMS, OS statistics.
CTRL-BREAK in windows
Kill-3 on Unix/Linux

Most of the time database query take time. It is quite necessary you query data on indexes only. Secondly its good not to do atleast query live data and create job to create replica and query on statistics data. In this way db query will not block.