Server Memory.

I have 6 GB RAM memory in the database Server. But for the most of the time it is using only 2 GB and rest 4 GB is used. Could you Please help how to make database to use that additional 4 GB and improve the speed.