Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

An anonymous reader writes "Commtouch released its Email Threats Trend Report based on the automated analysis of billions of email messages weekly. The report examines the appearance of new kinds of attachment spamsuch as PDF spam and Excel spam together with the decline of image spam, as well as the growing threat of innocent appearing spam containing links to malicious web sites. Image spam declined to a level of less than 5% of all spam, down from 30% in the first quarter of 2007; also, image pump-and-dump spam has all but disappeared, with pornographic images taking its place."

The link referenced in the posting goes to a summary page that is a little light on details. At the bottom of that page is a link to the PDF-formatted report [commtouch.com]. There's a lot more information there, including some screenshots of example SPAM and malware sites, trends in attack vectors, zombie systems, etc.. Interesting stuff.

Seriously, though, I had no idea spam could be so colourful and attractive looking. All I get is random ascii. If I'm lucky, I may see something like (altered to protect the click-happy) the following:

You can pick up your postcard at the following web address:[1]http://xm190.internetdsl.tpnet.pl/

Sometimes the editors miss the important stuff and post links to sites with less than meaningful content. If the actual data behind the story is more informative, then yes. Seriously. I wish some other sites had that kind of feature. I hate clicking through four sites just to get to the source of the information because someone wants to get hits.

Gmail's spam filters have definitely improved. When I first got my account (in '03?) I foolishly posted something to Usenet via DejaGoogle (required my @gmail.com account) and the spam just started rolling in. I still get lots of spam, but almost all of it is properly routed to the spam folder, and thanks to the CustomizeGoogle extension, I don't even see the spam count.

... here's a report from a company that specialises in anti-virus and other security
products.

While I'm not denying spam etc. is an annoyance and does cause a lot of people some
problems, do we really want to accept at face value some words from an organisation
that could well have a vested interest in making the problem appear more threatening
than it really is?

Personally I'd prefer to teach people how to avoid spam/virus infection - in the same
way we teach people how to avoid clinical infection, than to go around wailing about
how bad the problem is.

FWIW, about 90% of our e-mail has been spam... and we've seen a solid 50% increase in traffic over the past quarter. The numbers aren't that out of whack.
quote: Personally I'd prefer to teach people how to avoid spam/virus infection...
Good luck with that. Particularly with the avoiding spam part. If you come up with a foolproof method that actually involves using e-mail... I'm sure you'll be a lot richer than I am.

Good luck with that. Particularly with the avoiding spam part. If you come up with a foolproof method that actually involves using e-mail... I'm sure you'll be a lot richer than I am.

I have a modest proposal: Hitmen. And Hitwomen. It's simple enough. Everybody using email who are frustrated with spam donates a buck or so a year. The millions of dollars are used to hire teams of investigators who track down those sending spam, then you hire somebody to dispose of them.

Clearly they are not counting the volume of email within companies, but even so, if this number is even within 15% accuracy, we're about 10 years away from abandoning email as a communication medium altogether. Can you imagine any other form of communication that was 95% inefficient?

Can you imagine any other form of communication that was 95% inefficient?

I don't think the 95% figure is very important. What's matters is the balance of power in the arms race between spamming and filtering, i.e. how many spam actually show up in your inbox. Is the average user actually seeing any more spam than 10 years ago? I'm not.

Can you imagine any other form of communication that was 95% inefficient?

Flirting.

Let us pick some text randomly off a googled link and exercise our imagination.

"First for Emailing - UK's only Emailing Academy

We are offering you two free e-courses value $45 each. One is our new success emailing communication programme and the other is our popular lifestyle coaching programme

SUCCESS EMAILING Communication Tips - series of 4 communication tips modules. Designed to get you connecting and interacting more easily and effectively plus monthly success emailing newsletter with tips, quotes and news..."

When there is a large industry which advertises itself in terms like that instead of the original [flirtzone.com] then perhaps there would be a point to be made that email communications are unusually inefficient. In the meantime, well, sure looks to me like anyone who has ever interacted with the opposite sex should have no problem imagining a form of communication in which 5% efficiency would be a striking -- well nigh unbelievable actually -- increase, and somehow that communication medium has not died out in several millions of years.

Are you saying flirting is 95% ineffective? You have got to be kidding.

Well, let's assume I am, shall we?

In that case, explain the existence of the site I faux-quoted and its ilk.

Methinks that if I was indeed kidding, there would not exist the market which this class of business caters to. (Or, for that matter, the porn/prostitution/yadayadayada classes of business.) However, since they do exist, we can deduce that the market that they are addressing does indeed exist, and it would appear to further be a re

Why pay money when the amount of 'mail recipients' is down to 5% because filters have become so efficient?

That empties the possible pool of suckers out there so you might as well give up and find some other scam. (Remember, there zipper-heads want to get your money for free. If they can't... Well fuck it...)

Checking my mail stats, since 4 am this morning, I've received 51985 emails, 51909 of which were filtered as spam. That's 99%. Checking the bandwidth monitor, the spam has consumed a steady 100Kbit/s since 4 am, despite being mostly blocked in SMTP envelope via SPF and reputation (SPF blocks forgeries, reputation blocks spammers with the balls to use their own domain).

Geez, are you writing your email address on bathroom walls with a "For a good time email...." next to it? That's an insane amount of emails in such a short time. Wouldn't it be easier to abandon an obviously tainted email address and start fresh with 2 new ones (1 real, 1 decoy/spam depository)?

The statistics for CERN yesterday: 90% rejected, 7% (manually) moved to spam folder, 3% good mails. And that's not even including those that are just deleted without being moved to the spam folder. Scary tendency.

So far these numbers are right on for what we see here at my company.Last year we were running about 80% spamIn July 07 we were at 90-92%August-07 we reached 95%Looking at the numbers this morning we hit 96% for the week.Numbers rounded to the thousandsWe run a cluster of 4 Eprism 2000 Appliances for inbound mailThis week we received 21,490,000 total inbound messagesWe rejected 15,757,000 on RBL and Block lists6,591,000 were passed through for spam filtering.858,000 were passed as clean.

Symantec specializes in security products too, but their threat report only puts spam at 70% of email. From my own experience, the figure has a lot of variance -- corporate email accounts send a lot more legitimate email around than the average ISP account, and their addresses are less likely to be scraped from public sites, so their overall percentage will be lower (with the exception of role accounts: webmaster@ gets so much spam, it's just a spamtrap where I work)And yeah, I'd like to see a lot more emp

It's certainly high. Looking at my spam filter for the last week, 99% of email sent to me is spam - I'm now getting in excess of 250 spam emails a day, but generally only a couple of legitimate email messages per day. Fortunately, SpamAssassin filters nearly all of it.

Most of the subjects are as follows:(filtered for privacy)
Courses next term
[Course name here] Grades
IMPORTANT: Calculus Final Exam Time
Hello from [Relative name here]
[Subscribe newsletter here]
Funny pictures

If the financial incentive is removed the problem should go away. The spammer is not the root cause, the entity hiring the spammer and benefiting from the people responding to the advertisement appears to be the root cause and is easier to identify.

The entity initiating the process is identifiable ( the contact information must be accurate in order to effect the sale ) unlike the spammer that can utilize many techniques to avoid identification.

I say major fines backed up by labor camps if they can't pay. CIA kidnappings and visits to Gitmo for major out of country spammers. Maybe contract with Russia for one of their old siberian prison camps.

Then again, I might be a tad irrational with my hatred of mass spammers.

Is this any different then the stats of the dead tree style of spam that appears in my mailbox every day?
And we have seen the huge (cough) progress made in removing that snail mail spam from the system.

Honestly, there seems to have been more progress in weeding out the digital spam then the paper sort.
Even vague sort of laws and protections and such.

Probably minor variants in the same name or address. Speaking of which, non profits could save themselves a few bucks if they took the time to clean up their fund raising databases. I frequently get doubles from them (minor variations in address).

Is this any different then the stats of the dead tree style of spam that appears in my mailbox every day?

Good point. I'd estimate it at ~75% for me, defining SPAM as 'unsolicited commercial package that I'm not interested in'. Near 0% for fraudulent letters, which I got about a half dozen of when I was a teen. Near 90% of the spam caught by my filters is fraudulent and illegal in nature.

This is hardy new. Anyone with an old (>10 years) domain name is on every spam address list in the galaxy and likely gets 99.99% spam. All my mail server does is run spam assassin and clamav and a few times per day, actually delivers a real message.

I get a fair bit of solicited and genuine email, and a moderate amount of spam. Thunderbird's and Gmail's filters seem to do almost all the filtering perfectly these days, but even checking the size of my inbox against my junk-boxes, I have to say that I'm getting nothing like 95%. Not even 50%.

Anecdotally, I don't think mine is an unusual scenario, which causes me to wonder: how many people are getting 96-100% spam, in order for this average to hold true? I mean, are there folks out there being inundated

My gmail account had 535 spams in the last 30 days against 20 genuine emails. So that's roughly 96%. My gmail account name hasn't been splashed anywhere public, I only use it for companies that require an email for registration and with friends. The account name is 10 characters long and won't appear in a dictionary anywhere.

So I would guess that my usage patterns put me somewhere near the average and I'm seeing the spam levels that they talk about. You might be very lucky:)

The reason you're not seeing the 95% is because the study doesn't take the content per individual mailbox. It counts the 95% based upon what mail servers (and their anti-spam systems) see.Honestly, if you were seeing 95% of the content in your mailbox as SPAM, you would have ditched e-mail by now.

Usually, the 95% does include e-mail sent to a bad recipient. The logic used by most anti-spam solutions is if that an e-mail got sent to mail server where the sender didn't know your e-mail address, the e-mail get

My account on my own domain doesn't get much spam, that is because the username is fairly unusual. HOWEVER the amount of spam the server gets is rather larger. It is offcourse rejected as it silently drops email for a non-existing account. Now it all depends on how well known your domain is, I had obscure ones that barely got touched and popular ones were I needed a seperate machine to just deal with it all. I don't even bother reading the admin email, you should as this is the official way to get in touch

Since most slashdotters are libertarians for some reason (and I could argue even I am to some degree) my question is: where's the technological efficient solution to this.

We've seen some "free market" solutions which basically required that you pay a fee to every mail provider so they don't trash your email. And this didn't particularly help spam either.

I come to the conclusion that spam as an issue is one of two things, or both of those things:

1) Not that big of a problem (hard to believe if you are a mail provider / ISP yourself)

2) Impossible to solve by means of free market solutions, and requires cooperation and standardization of new technology.

Point 2 is hard to happen since every little startup that comes with a mini solution, trumpet it on their own and hence they are only a nuissance to deal with in the big picture (due to lack of a single standard, it's impossible to have clients which make the process of whitelisting easier and even half automatic).

Here are couple of solution which would get us half-there, but are only quarter-implemented right now:

1) Whitelist SMTP servers by talking back to the supposed mail of origin and comparing IP-s. The SMTP may return list of IP-s this host responds from. This is then cached and used for further authentication on this domain. It *may* lead to DoS if many hosts do a first-time check simultaneously, but it's unlikely (and less problematic, given we're eliminating 95% of bad emails this way).

2) Test-for-human-intelligence in your first email to a new email. Such as, I don't know, some sort of CAPTCHA you fill-in? Once this is done, communication can proceed without further tests between those two emails. The receiver still has the option to block you, lest you employ a mechanical turk.

Those solutions are boring, they're incomplete in a way, they introduce hassle, but if we *all* agree on those, they can be made less of a hassle, and still not lose their efficacy.

That would require the likes of AOL, Hotmail, Gmail and so on free mail providers to cooperate with the likes of Microsoft, Apple, Linux devs and so on, to implement this on both the clients and servers.

Right now, I could see Hotmail cooperating with Microsoft (.. wink, wink..:P ), but that's where it ends.

Established technology can be replaced if a technology for another use becomes popular enough, and can be adapted to replace the existing technology, and has some advantage. For email, two candidate technologies are notification feeds like RSS or Atom, and social web sites like Myspace and Facebook.

Both have an advantage over email in that you can control who you receive messages from because the sender identity cannot be faked. In RSS, you poll to get updates, so you know with certainty who you are polli

I'd say that the real skew on the free market comes from the armies of spambots. They can spew essentially infinite spam, which gives the spammers a huge thumb on the scales in any free competition between us and them.I'd say the agreement that needs to be made between us is to start shunning ISPs who behave so impolitely. Email is a commons, and subject to the tragedy of the commons. The solution to the tragedy of the commons is politeness.

We can't stop it because we aren't addressing the real problem. Spam is an economic problem. People send out spam because they make money off of it. And they will therefore continue to send out spam as long as they make money off of it.

If you want to stop spam, you have to remove the economic incentive. To do that, you need to cut off the co-conspirators that are allowing the spamvertised domains to be established and hosted. If you can either prevent them from getting a cut off the action, or punish them severely for taking their cut, then you can stop spam.

Until then, if all we do is try to filter spam out, we'll just continue to see the costs of inaction. Beyond that, we're ignoring the fact that filtering has real costs, as well. Filtering doesn't prevent the spam from traversing the internet, and furthermore it requires human time to update as the spammers change their tactics.

We can't stop it because we aren't addressing the real problem. Spam is an economic problem. People send out spam because they make money off of it. And they will therefore continue to send out spam as long as they make money off of it.

If you want to stop spam, you have to remove the economic incentive. To do that, you need to cut off the co-conspirators

You're right, but for the wrong (IMO) reason. Spam has economic incentive because all the costs of email are borne by the recipient. Botnets have made it even cheaper. You must remove that if you want to really fix the problem.

If you do not remove the economic incentive, nothing will work because it will just be an arms race and the "good guys" will necessarily always be on the defensive side.

As email asymptotically reachs 100% spam, we will have essentially created a mechanism whose sole goal is to deliver us undesired ads and scams.
Talking about spam detectors and blockers and blacklists is irrelevant. Why devote all of this energy to ensure that maybe 5, 10, or 20 people can contact you or your business a day? Or even 20,000, which only highlights the issue that separating spam from valid emails is just bad juju.
Simply put, there is no solution to asynchronous communication that is not too tedious or too restrictive. We'd be a lot better off if we blew up all the email servers, and put all of the energy and cost savings into developing encrypted telepathy.
You think I'm kidding.

and put all of the energy and cost savings into developing encrypted telepathyIt will never work. Considering the trash in my brain, I must conclude that it has already been done, and it has already been compromised.

...before it reaches the level of spam I get in the mailbox in front of my house. I swear, if we want to save the trees, we need to start by arresting the people putting all those unwanted 20-100 page sales catalogs in everyone's mailbox every day.

I hate to bring up anecdotal evidence, but, while I still get spam, my flood has gone down to a relative trickle simply by plugging postgrey into postfix. I could probably reduce it to zero with a bayesian filter, but I won't bother. Scanning through my logs, my server rejects literally thousands of spams every day, and I'm just one guy with two email addresses and a handful of aliases.

So, it would come as no surprise to me that spam volume is that high, I just never see it. I almost want to turn off my filter for a day just to see what would happen.

I knew somebody would bring up greylisting.:) During the business day[1], I work for a company that produces several widely-used anti-spam appliances and a service-based filter as well. We see about 2,000 networks a week, and get a pretty good feel for spam trends and countermeasure effectiveness. I can say with all honesty that in my experiences, greylisting hurts more than it helps for most organizations.

Basically, greylisting is putting an email transaction on hold to see if the sender will retry. The idea is that if the sender is illigitimate, they won't bother resending. However, spammers have been onto this method for as long as it's existed, much moreso lately. All they have to do is take greylisted hosts and move them to the end of their script for later processing. The second time around, the spam gets through anyway. Even with its meager benefits, most organizations want email to come through as quickly as possible, and greylisting delays email by its very nature. It's also much less effective than existing technology that won't hinder most legitimate mail like DNSBL and/or SPF, spamwords+OCR (for image spam), and blocking on unknown recipients.

To summate, if greylisting makes you happy, then don't let me dissuade you from using it. it does indeed stop some spam. But please don't give the false impression that it's a magic bullet; most of the complaints we receive are from clients who've enabled greylisting and can't figure out why their mail is delayed.

[1] I am also a consultant to another firm who hosts manged email with spam filtering. Due to the complaints above, we have also disabled greylisting there. It was only effective at stopping about 5% of spam reliably, but a delay is put on all mail that isn't otherwise whitelisted. There are plenty of other methods which are both more effective and don't slow down the mailflow or tie up much resources on the MTA.

I think you miss the true point of greylisting. See, the delay is only half of the whole equation. Sure, the host may try again, but I'm also subscribed to a few relatively non-strict DNSBL lists. Now, imagine the combination:

I think there's more to the spam problem then the usual people we blame for it.I have a personal email address on my own domain that used to NEVER get spam. I moved into my own apartment a month ago and I signed up a new phone number with Bell Canada and a new account with my local city utility company. I gave that email address to both without thinking- usually I give one of my alternates. Well, now that address is getting tons of spam of the worst kind.

I keep seeing statements, including one in the PDF report from TFA, that Win+IE users can get their machines infected with malware just by visiting a web site, without even clicking their mouse on anything in the site. However, these statements always seem to come from people who make money in the security business, and they never seem to say anything about what the actual IE vulnerabilities are. I'm very skeptical, although I haven't run Windows in a decade, so maybe I'm just naive. Can any slashdotters wi

and what goes on the business card, the press release and other similar locations? or you think you can run a business that has no email address and ignores emails sent blindly to sales@ info@ and webmaster@ not to mention support@ ?

Most ISPs just block outgoing port 25. I heard a few viruses have started trying to get the ISP login id/pass so they can send through the ISP smtp servers, but that won't work really because the ISP will notice that fast and shut you down since it would get their servers on blacklists pretty fast. It's the ISPs that don't do a damn thing that are the problem - anyone know of any major ones that still don't block port 25?

ISPs are in the perfect position to sniff traffic and identify infected machines that are part of botnets. It's obviously technically possible since the government does it at AT&T. You don't even need to sniff ALL traffic, SYN packets are enough. Most tech savvy businesses already sniff all their traffic with IDS systems, it's not a big leap.ISPs should also be blocking outbound port 25 traffic from dynamic addresses (and if you need to use an external mail relay, use a tunnel or port 587.) Some ISPs do

"The solution that stops 90%+ spam is out there, but it costs a little money to implement. It's still less money than what we currently are spending fighting spam. What are they waiting for - government mandates? Fines? Lawsuits? Getting their netblocks in 2,000,000 private blacklists that they have no chance in hell of getting out of?"

I can actually understand the ISPs on this one. Yes, spam costs a huge amount of money to the economy as a whole, however it's not such a major cost to the ISPs themselves.

Either ISPs are common carriers (the postal system is a prime example, I get lots of 'junk mail' for every legitimate piece of mail, but at least they're getting paid to deliver the crap,) or they aren't (and NOBODY wants that.)

Its like the telephone itself.

Its NOT the phone company's problem if people call you in the middle of the night and threaten to cut off your balls.

People's phones don't get hacked to start calling random numbers with automated voice messages threatening to cut off your balls. Also, you can easily trace phone calls through phone company records or even caller ID most of the time, where emails are MUCH harder to trace.

ISPs are not common carriers. They are their own category, "data service providers", which fall somewhere inbetween. They are under no threat of losing such a status if they implement blocking measures.

Other have pointed out other problems with your post. I'll point out that most spam does not originate in US ISPs. Even if the spammers themselves are in the US, they use ISPs in places where there are virtually no legal checks on what you do with your computer to generate the traffic (or spread the botnets to generate the traffic).

To all the whiners that don't like the port 25 blocking: Dynamic IP space is already "damaged goods", and you have multiple workarounds available to you. Any sane mail admin (including many large ISPs) already blacklist dynamic space therefor you can't effectively run a mail server on dynamic IP space.

When I signed up for a DSL account with SBC/ATT, I asked for static addresses and and got my delegation request for tiny/29 netblock processed a few days later. All good, right? With everything setup on my end, I send out a few test messages to my personal ATT email account (hosted by the folks at Yahoo), and it gets the 'YahooFiltered: Bulk' treatment 4 out 5 tries.

In my experience, the current blacklists treat anything at the end of a DSL line, static or dynamic as a dynamic address nowadays. I've hosted my email on my domains on a DSL line for ages but I'm now looking at alternate solutions (among which possibly just routing the stuff through my ISP for problematic domains).

Some ISPs are doing a little - mostly with inbound. Occasionally with outbound. What kills me are the ISPs that reject inbound mail from dynamic IP space but don't block direct outbound (port 25) mail from dynamic IP space. Hello!?! Can I beat you over the head with a clue-by-four?Looking at my corporate mail servers, it's obvious that many of the major ISPs are not filtering. In the US, Comcast is one of the WORST offenders, but Verizon, Road Runner, and others are pretty damn bad too. It's a world-wide pr

ISP's transmit data, I really don't want them to be starting to be clever. What next, RIAA requests that people are limited to X posts to usenet so they can't post large binaeries? Limit P2P traffic? Sniff traffic in general for undesired elements?

In a way, my PC becoming a spam zombie is part of the price of freedom. Do you really want the internet to be regulated?

Oh sure, you can start light, but in the end sooner or later someone will abuse it and push for ever more stringent restriction, all in the na