The power of spam filters

March 29, 2010

I read an interesting article on the BBC website today about how a long running Canadian magazine had decided to produce an online version but had seen poor results due to spam filters. They have decided to change their name from "Beaver" to "Canada's History". I can only imagine the costs and time to rebrand a 90 year old magazine and I'm sure they didn't take the decision lightly.

Now, I suspect this is not purely the fault of email spam filters but more likely an issue with web filters and parental controls made available by many ISP's and used by many schools. They would also have an issue with search engines. Google, for example, has a SafeSearch function that will remove explicit search results before they are returned.

It would be a very poor spam filter that would mark an email as spam because of the existence of one word in the content. Although this does still happen with spam filters that are old and not maintained. Nowadays we have message digest signatures, baysian filters and other tools to help identify spam rather than simple content filtering.

Back in 1996 and the great Scunthorpe scandal I can understand the problem. Email was used considerably less than it is now and spam filters didn't exist. AOL and other ISP's used simple filters to search for explicit phrases within form submissions.

While reading around this topic, I noticed Wikipedia had a great example of an email being blocked for no reason other than a badly configured spam filter.

"In May 2006 Ray Kennedy from Manchester in the UK found that e-mails that he had written to his local council to complain about a planning application had been blocked as they contained the word erection when referring to a structure."