Unofficial news and tips about Google

May 23, 2010

Gmail's Spam Filter No Longer Effective?

Yahoo has recently quoted a study of the Fraunhofer Institute which concluded that Yahoo Mail and Hotmail have better spam filters than Gmail.

"The Fraunhofer Institute, an independent research firm, found that Yahoo! Mail users saw the least amount of spam out of the five providers tested, with nearly 40% less spam than Hotmail and 55% less spam than Gmail – meaning Gmail users in the study saw more than twice as much spam as Yahoo! Mail users."

If the results are accurate, then either Gmail's spam detection technology is no longer state of the art or Yahoo and Microsoft have dramatically improved their spam filters. Last time I checked, most of the messages from the inbox of my Hotmail account were spam.

John Mueller, from Google, says that Gmail's spam filter is quite effective: "When I first got my Gmail account (it seems like such a long time ago), I purposely signed up to all kinds of mailing lists to try out the filter. It seemed like a fun idea -- but then I noticed that I actually wanted to keep my Gmail account :). About one spammy message comes through each month, but the most important part is that I haven't lost anything that I was waiting for."

I rarely get spam messages that aren't flagged by Gmail, but the spam filter is not perfect. Sometimes Gmail flags legitimate messages as spam, so the algorithms are too aggressive.

What do you think? Is Gmail worse than Yahoo Mail or Hotmail when it comes to filtering spam?

Update 2: Apparently, the study was sponsored by Microsoft and Yahoo's conclusions are misleading. The "prestigious" Fraunhofer Institute counted all the spam messages received by six users and Gmail users happened to receive more spam messages than Yahoo Mail and Hotmail users. The study admits that Yahoo Mail and Gmail detected correctly all the spam messages, while Hotmail couldn't detect 23% of the messages.

Yahoo concluded that "independent empirical studies done by the prestigious Fraunhofer Institute show that Yahoo! Mail is #1 in blocking malware and spam from reaching mailboxes. It's official. No one fights spam harder, smarter, or better than Yahoo! Mail." But so is Gmail, according to Fraunhofer's irrelevant study.

Gmail's spam filter is definetely better than Hotmail's, as you said in your post. About Yahoo, I have a friend who switch from Yahoo Mail to Gmail 2 weeks ago because she couldn't keep up with all the spam she was getting. And it works very well for her.

I had to create several filters to automaticaly get rid of russian mails ( they didn't seem to get flagged as spam )Never used Yahoo and on my hotmail everything that's not from a trusted sender is marked as spam.

Gmail spam filter is the best in the world at present. I don't recall if I have ever had any spam messages. My primary email was yahoo which is full of crappy spam messages. So I decided to auto forward all my emails to gmail which, in fact, did a better job at getting rid of spam and keeping the important emails. This way I never have to open yahoo ever again.

I do not use Hotmail of Yahoo, but it is hard to imagine that any spam filter can beat Gmail which is almost perfect... I do not remember last time I got a spam message and no wanted messages are misclassified. So the study essentially reports that Yahoo users get 55% less spam than zero spam Gmail users get! How did they calculate 55% of zero? :)

Used Google translate on the link above. Didn't really say much more than what was already posted.

An idea though. I could be wrong, but it looks like the study was looking at email in the German language. Google's spam filter might work a whole lot better for English than German. I would find it surprising if Hotmail and Yahoo were actually better, but the language switch might be a factor.

I translated the PDF file using Google Translate, but it's not very clear. From what I've read, six people created email accounts at Yahoo, Hotmail, Gmail, GMX and Web.de and then waited to see how much spam messages they get.

GMX - 116 Gmail - 18 Hotmail - 13 Web.de - 57 Yahoo Mail - 8

"The metric M9 consider what portion of received spam messages arrive in the inbox. In the subjects considered here have Google Mail and Yahoo! 0% arising from the peak, ie no Spam message has been received in Inbox. Hotmail have been in 23% of received spam messages in their inbox is sorted according- but it was purely of emails from Hotmail himself is- were sent." (machine translation)

So all the spam messages were correctly flagged by Yahoo Mail and Gmail. What a misleading study!

I do not trust any German studies including any product from Google. Germans are Apple lovers, and Google haters, And this has gotten worse since Android Phones start to push Apples iPhone market down...I also have gmail, yahoo and hotmail accounts. Inbox in google gets a spam mail like once in a monce, and yahoo and hotmail are full of spam-mails.. there is no improvement on them at all.

I can only speak from my own experience but i conclude that gmail fights spam really well now, comparing with what was happening let's say 2 years ago. Maybe yahoo isn't too bad, but their products including mail, look and act like an old man.

I use all three email services, GMail and Hotmail for my important personal mail and Yahoo gets my mailing lists and other assorted hobby mail. Both GMail and Hotmail seem to do excellent jobs on keeping the spam out, I never see spam in the Inbox or spam folders, whereas my Yahoo account gets well over 200 spam mails filtered into the spam folder each week.

They did a study with only 6 users? I would think you need a bigger audience before you draw conclusions on what mail system does the best job filtering spam.

Well, my gmail inbox tends to pass maybe 2-3 pieces of spam PER YEAR. I don't know if it's the best, but it's certainly more than good enough for me.

Less than a week ago, I also just signed up for hotmail, in anticipation of checking out the new features coming out later. It already has caught 14 spam messages, and let one through. I'm not impressed, especially since I haven't given out my hotmail address to anyone.

How desperate of Yahoo! to press release such a pathetic study. I've used gmail from the every early days, and have never had a mail mislabelled as spam since the very, very early days. And I get no more than a couple of spams in my inbox per month. Last time I looked, Hotmail was thigh deep in spam - though I've heard rumours M$ has improved. Yahoo! used to frequently mislabel my emails, but I've not used Yahoo! Mail for a couple of years.

on hotmail all i receive in the inbox is spam. on gmail most of the spam is filtered. on yahoo some of the good messages get filtered too, and i had to train it filter some of the messages that were actually spam. so for my accounts the order would be 1. gmail 2. yahoo 3. hotmail. that's one of the reasons i stopped using hotmail actually. i only kept gmail and yahoo.

Lately, You have been posting interesting posts comparing Gmail, Hotmail and Yahoo Mail.I just would like everyone to remember that Gmail is the only email provider that allows you to play mp3 files you receive by email, online!It works since long ago and it works very good.This is an example showing the generosity and the efficiency of Google services.

I've had one good site flagged as spam until I white-listed it (I forgave Gmail - those emails do look like spam). Only one or two real spam slip thru each month, while the spam folder gets 100-150 new spam/day. Anyone who considers that a problem shouldn't be allowed online.

In English: "It is remarkable that at all email providers no external spam message made it into the inbox."

To summarize:- All providers' spam filters worked equally and perfectly by filtering out 100% of external spam.

And:- The spam numbers in the above comments and post update (23% for hotmail) are actually internal emails by the providers since these were counted as spam, too.- The 73 page report does a great job in hiding the actual numbers of false negatives by adding up spam messages in the inbox and spam box for most part of the report.

I see from the comments that the study was severely flawed, but regardless, I thought I'd make the point that Gmail's spam filtering system seems to do a much better job if you make sure to report spam that DOES get through. It appears that Google uses the corpus of your spam vs non spam email to analyse the likelihood that an incoming email is spam. Remember, what is spam to one person may not be spam to another -- and I think Google's system takes this into account.

I've had my gmail account since it was first released and I am a huge user.. lots of newsletters and prescriptions, forums, etc.. and I rarely get a piece of spam. In fact, I think the last time I did was months ago.

Gmail is the best (definitely the coolest, too) and all the others are just posers.

I take most of them from Postini, a Google-owned product that focuses on spam filtering among other things. Postini does a good job.

Oddly enough, messages that make it through Postini still get filtered as spam within Gmail. Never figured that one out.

As for the comment about legitimate messages getting filtered as spam (i.e. false positives) this can be due to content, due to other people flagging the message as spam (in the case of mass sends), due to the reputation of the sending IP address, or even other factors.

All in all I'm impressed by spam filters at all of the major ISPs. It was just a few short years ago that A LOT of spam made it through. Now it is the exception instead of the norm.

My experience of using both Yahoo Mail and GMail for the past several years is that Yahoo let's through a great deal more spam than does GMail. Rarely a piece of spam gets into my GMail inbox and I've had no legitimate emails go into the spam box. Quite the contrary with Yahoo. This "study" is just more BS from Yahoo and Microsquish.

Well, GMail was worse like..1 year ago or something. Spam got to my inbox and it was really annoying. Eventually, they somehow made them stay in Spam folder and not reach my Inbox anymore. But, the problem with Yahoo and Hotmail is that you get a lot of false positives, and it happened very often to me to have normal e-mails in my Spam folder..Anyway, I'm still sticking with Google..

well I have both Gmail and Yahoo mail plus a pobox.com forwarding account that redirects everything to both accounts - and this allows me to see what spam is sourced directly to Yahoo and Gmail and I can say without any equivocation that Gmail does a better job.

I almost never get in my Gmail inbox - spam but when I check the SPAM folder it's chock full of crap.

About a year or two ago. this was not the case and there was about the same amount on Gmail as I now see on Yahoo.. Gmail has gotten better.

I'm sorry but i need to say that gmail is the best.I have those three account. But i have never seen spam mail in inbox or lost any mail in span box since gmail open beta.On the other hand, i always receive lots of spam in inbox from hotmail and yahoo.I think no company better than gmail, it is too powerful

I still wonder how a newly created email account whose address you haven't revealed to anybody can get spam. It happened to me a number of years ago and someone in this thread mention it's still happening. Does anybody here know?

@anon: Could be spammers enumerating possible accounts, trying usernames that already exist at another provider. You might not have revealed the address publicly but still shared it with people with malware. Your own provider might do non kosher things with your address.

I have definitely noticed gmail's spam filter become worse over the past month or two. I previously got maybe 1 spam every 3 months to my inbox, and usually 1 or 2 false positives every month to my spam folder.

Lately, I've been getting 5-10 spam messages in my inbox every day, while still getting the occassional false positive in the spam folder.

I agree with the last two postings.Until recently, I received hardly any spam on my gmail acount.Yesterday (June 4th 2010) I received a whole bunch of 14 spam emails.One of them says I've won the australian lottery :-)

I've been getting many spam emails recently as well, most of them from 'hotmail' accounts. The only hotmail account I use is through a .edu for messages from school.Why can't we set our gmail account to get mail only from trusted senders? Why does there need to be a 'spam' folder at all.I noticed that Gmail does not delete all spam after 30 days either, and even if it did, why would I compromise my account by not deleting spam right away?

The same here. If asked two days ago as to whether someone should pick Gmail over Hotmail or Yahoo mail, I would have said, "Hands down, YES!" But, the last couple of days have produced the most fowl spam that I have every received in my history with Gmail. Please, Google... return to the glory days!

Gmail filter sucks for now :((( I was very happy with it for last few years :) But from last few weeks its start flooding of spam shit from all Internet right to my gmail box! :((((((((((((((( VERY DISAPPOINTED.Google, fix it please!ps: maybe we need to make new design for e-mail system for Internet (globally. like IPv6). Old scheme is too old and have poor design.

I might have thaught that the above comments are from microcrap or yahoo users ,...but yea suddenly gmail spam filter is going nuts. I have getting messages like lottery spam and most spams are from gooogle groups.But still cant think about switching to another webmail.

I'm having the same problem. My inbox is flooded every day with spam, most of which has porn in the title. I don't get why GMail can't see this is spam, especially as there is so much of it and I mark it as spam every single day.

Gmail FILTER not working...PENIS PENIS STILL COMES VIAGRA VIAGRA STILL COMESVICODIN VICODIN STILL COMESLARGER PENIS STILL COMESNOT WORDS IN THE FILTER NOT WORKINGI DO NOT HAVE TIME TO GO THOUGH OVER 100 SPAM GMAILSGMAIL NOT ANY GOODI HAVE (WASTED) HOURS ON HOURS FOOLING WITH TRASH GMAIL(ME TO ME) GMAILS...GMAIL KNOW I DO NOT SEND MYSELF ANY EMAILS...SO HOW TO STOP (SAME) EMAILS OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER...I NEED TO SAY THAT OVER AND OVERGMAIL SPAM FILTER NOT WORKING...GMAIL NEEDS TO MAKE SOME CHANGES...STOP THE (SAME SPAMERS)

You know why there are so many holes in the Gmail's spam filter? Because they make money selling to spammers. Why do you think there's no option to turn off the spam folder so you can't look at it? They are lying to you. After so many empty promises about their great spam filtering and "flexibility" options, why else do you think they keep letting the spam through? Otherwise they would have given us the option to just turn off the spam folder.

Just switched over from Yahoo Mail Plus (paid) to Gmail and there's no comparison - WAY less false positives and false negatives with Gmail. Actually, in 2 weeks, there have been no good emails going into the spam folder and no spam in my inbox. That's vs. an average of 4 or 5 mis-flags with Yahoo a week.

I was using Yahoo mail for a loooong time and what drove me away was that I'd regularly get various spam (mostly WoW related) and when I tried to forward one of those messages to the WoW maker (suggested solution on their site), I had to solve 5-6 captchas! And I guess my account got flagged as spammer's, because the next time, I wrote an email to my friend with the word 'spick' in it (I'm not a native english speaker) I got captchas crammed down my throat again.

Gmail's spam filter has been losing quality for quite some time. I don't blame them, Google is in the advertising business, and a spam-blocker in Gmail is a conflict of interest. I still use Gmail, but I just manually set filters that aren't working automatically anymore.

considering a vast amount of scammers/spammers use yahoo accounts for spamming i personally find the findings that gmail are not good enough at blocking spam a joke! come on yahoo step up the Anti and get your hackers/spammers/scammers off yahoo and let the world have some respite from reading blah.blah from yahoo wants my bank and address details.

Seems lately the GMail spam is not working so well. My spam bucket used to only get 2 to 3 a day and now i get more than 50 spam emails a day. Used to be they never got through to me, seems gmal is slipping

I have also noticed that Gmail's spam blocking has become far less effective. Even when I specifically report a sender as spam, I still get message from that same sender (exact same email address in many cases, although some spammers use different variations of address or domain to fake out spam filters). Up until a few months ago, Gmail worked nearly flawlessly. Now, it feels like it's about 50% as effective.