When I actually fall asleep while the episode is playing that's when I know it's time to move on, if I don't understand the series, the series simply doesn't interest me and when the plot is completely horrible it makes me want to pluck my eyeballs out.
Mangas...pretty much the same too.

If I can get through the first episode of an anime, I'll usually watch it the rest of the way. There are some things that will make me drop even a promising anime right away though. 'Adult' content is the fastest. Light treatment of religious subjects is the second.

If an anime passes those tests, the only reason I've ever dropped something is for going to filler. Some animes can get away with a lot of filler before I drop it, but others are just too painfully divergent from the manga story, and the show suffers. I skipped nearly a year of Naruto. Pandora Hearts is another that failed in filler - if it had gone on any longer I would have dropped it (but since it ended it's not exactly 'dropping' now). 07 Ghost I only got about 10 episodes in before the 90% filler was too much to bear.

I usually drop the anime when it gets to boring and dragging.. Also when i can't really relate to any of the characters mindset. When i can't understand the character's reasons/views fully on the things he does in the anime, i get really annoyed. I guess it boils down to how the characters are presented in the first episode so you can pinpoint their motivations for their actions in the stories to come. Too much fillers is also one of the reasons.

Those are the 9 anime i have dropped, I dropped yozakura, Sensei, Ga-rei and DNA as i just couldnt be fucked
I dropped Skip Beat as it was the worst shit ever
Strawberry panic just sucked
Bleach = boring fillers, plot never seemed to be getting anywhere
Akane = boring
Haruhi shit = I watched the last 5 episodes. I guess you could say that was the entire series though, i hated it anyway and the only thing entertaining about it was when Kyon went to pimp slap Haruhi