The vowel u is sometimes not pronounced.
The most common are: between k and s (eg gakusei, 'student', sounds like gaksei.)It's also dropped in the verb endings -desu and -masu (ii desu, 'its's good', sounds like ii des).

The only time I know that the vowels are silent are actually 「です」and 「ます」.

The reason you hear missing vowels is actually due to the fact that the "u" is not properly pronounced during fast speech, which is very common.

For example, try repeatedly saying 「がくせい」really really fast.
You will realise that you will begin to slur and you will begin to pronounce it like "gak-sei". Many other words would also have the same effect when spoken quickly.

Actually, if you want to sound more polite, you can pronounce the "u" in 「です」. You don't have to emphasize on the "u", just pronounce it lightly.