Inside the precision hack

There’s a scene toward the end of the book Contact by Carl Sagan, where the protagonist Ellie Arroway finds a Message embedded deep in the digits of PI. The Message is perhaps an artifact of an extremely advanced intelligence that apparently manipulated one of the fundamental constants of the universe as a testament to their power as they wove space and time. I’m reminded of this scene by the Time.com 100 Poll where millions have voted on who are the world’s most influential people in government, science, technology and the arts. Just as Ellie found a Message embedded in PI, we find a Message embedded in the results of this poll. Looking at the first letters of each of the top 21 leading names in the poll we find the message “marblecake, also the game”. The poll announces (perhaps subtly) to the world, that the most influential are not the Obamas, Britneys or the Rick Warrens of the world, the most influential are an extremely advanced intelligence: the hackers.

At 4AM this morning I received an email inviting me to an IRC chatroom where someone would explain to me exactly how the Time.com 100 Poll was precision hacked. Naturally, I was a bit suspicious. Anyone could claim to be responsible for the hack – but I ventured onto the IRC channel (feeling a bit like a Woodward or Bernstein meeting Deep Throat in a parking garage). After talking to ‘Zombocom’ (not his real nick) for a few minutes, it was clear that Zombocom was a key player in the hack. He explained how it all works.

The Beginning

Zombocom told me that it all started out when the folks that hang out on the random board of 4chan (sometimes known as /b/) became aware that Time.com had enlisted moot (the founder of 4chan) as one of the candidates in the Time.com 100 poll. A little investigation showed that a poll vote could be submitted just by doing an HTTP get on the URL:

where ID is a number associated with the person being voted for (in this case 1883924 is Rain’s ID).

Soon afterward, several people crafted ‘autovoters’ that would use the simple voting URL protocol to vote for moot. These simple autovoters could be triggered by an easily embeddable ‘spam URL’. The autovoters were very flexible allowing the rating to be set for any poll candidate. For example, the URL

http://fun.qinip.com/gen.php?id=1883924
&rating=1&amount=160

could be used to push 160 ratings of 1 (the worst rating) for the artist Rain to the Time.com poll.

In early stages of the poll, Time.com didn’t have any authentication or validation – the door was wide open to any client that wanted to stuff the ballot box. Soon these autovoting spam urls were sprinkled around the web voting up moot. If you were a fan of Rain, it is likely that when you visited a Rain forum, you were really voting for moot via one of these spam urls.

Soon afterward, it was discovered that the Time.com Poll didn’t even range check its parameters to ensure that the ratings fell within the 1 to 100 range. The autovoters were adapted to take advantage of this loophole, which resulted in the Time.com poll showing moot with a 300% rating, while all other candidates had ratings far below zero. Time.com apparently noticed this and intervened by eliminating millions of votes for moot and restoring the poll to a previous state (presumably) from a backup. Shortly afterward, Time.com changed the protocol to attempt to authenticate votes by requiring that a key be appended to the poll submission URL that consisted of an MD5 hash of the URL + a secret word (AKA ‘the salt’).

“Needless to say, we were enraged” says Zombocom. /b/ responded by getting organized – they created an IRC channel (#time_vote) devoted to the hack, and started to recruit. Shortly afterward, one of the members discovered that the ‘salt’, the key to authenticating requests, was poorly hidden in Time.com’s voting flash application and could be extracted. With the salt in hand – the autovoters were back online, rocking the vote.

Another challenge faced by the autovoters was that if you voted for the same person more often than once every 13 seconds, your IP would be banned from voting. However, it was noticed that you could cycle through votes for other candidates during those 13 seconds. The autovoters quickly adapted to take advantage of this loophole interleaving up-votes for moot with down-votes for the competition ensuring that no candidate received a vote more frequently than once every 13 seconds, while maximizing the voting leverage.

One of the first autovoters was MOOTHATTAN. This is a simple moot up-voter that will vote for moot about 100 times per minute. (Warning, just by visiting that site, you’ll invoke the autovoter – so if you don’t want to hack the vote, you should probably skip the visit).

Here’s a screenshot of another autovoter, a program called Mooter, developed by rdn:

Mooter is a Delphi app (windows only) that can submit about 300 votes per minute from a single IP address. It will also take advantage of any proxies and cycle through them so that the votes appear to be coming from multiple IP addresses. rdn, the author of Mooter, has used Mooter to submit 20 thousand votes in a single 15 minute period. In the last two weeks, (when rdn started keeping track) Mooter alone has submitted 10,000,000 votes (about 3.3% of the total number of poll votes).

From the screenshot you can see that Mooter is quite a sophisticated application. It allows fine grained control over who receives votes, what type of rating they get, voting frequency, the proxy cycle, along with charts and graphs showing all sorts of nifty data.

In addition to highly configurable autovoting apps, the loose collective of #time_vote maintains charts and graphs of the various candidate voting histories. Here’s a voting graph that shows the per-minute frequency of votes for boxer Manny Pacquiao.

So with the charts, graphs, spam URLs and autovoters #time_vote had things well in hand. Moot would easily cruise to a victory. Although they still had some annoying competition, especially from fans of the boxer Manny Paquoia. Zombocom says that “it can take upwards of 4.5K votes a minute to keep Manny in his place”. Despite the Manny problem, the #time_vote collective had complete dominance of the poll.

The Ultimate Precision Hack

At this point Zombocom was starting to get bored and so he started fiddling with his voting scripts. Much to his surprise, he found that no matter what he did, he was never getting banned by Time.com. Zombocom suspects that his ban immunity may be because he’s running an ipv6 stack which may be confusing Time.com’s IP blocker. With no 13 second rate limit to worry about, he was able to crank out votes as fast as his computer would let him – about 5,000 votes a minute (and soon he’ll have a new server online that should give him up to 50,000 votes a minute.) With this new found power, Zombocom was able to take the hack to the next level.

Zombocom joked to one of his friends “it would be funny to troll Time.com and put us up as most influential, but since we are not explicitly on the list we’ll have to spell it out. ” His friend thought it was impossible. But two weeks later, “marblecake’ was indeed spelled out for all to see at the top of the Time.com poll.

So what is the significance of ‘marblecake’? Zombocom says: ” Marblecake was an irc channel where the “Message to Scientology” video originated. Many believe we are “dead” or only doing hugraids etc, so I thought it would also be a way of saying : we’re still around and we don’t just do only “moralfag” stuff .

To actually manipulate the poll, Zombocom wrote two perl scripts. The first one, auto.pl is pretty simple. It finds the highest rated person in the poll that is not in the desired top 21 (recall, there are 21 characters in the Message) and down-votes them (you can view this as eliminating the riff-raff). The second perl script, the_game.pl is responsible for maintaining the proper order of the top 21 by inspecting the rating of a particular person and comparing that rating to what it should be to maintain the proper order and then up-voting or down-voting as necessary to get the desired rating. With these two scripts, (less than 200 lines of perl) Zombocom can put the poll in any order he wants.

Ultimately, this hack involved lots of work and a little bit of luck. Someone figured out the voting URL protocol. A bunch of folks wrote various autovoters, which were then used by a thousand or more to stack the vote in moots favor. Others, sprinkled the spam urls throughout the forums tricking the ‘competition’ into voting for moot. When Time.com responded by trying to close the door on the hacks, the loose collective rallied and a member discovered the ‘salt’ that would re-open the poll to the autovoters. The lucky bit was when Zombocom discovered that no matter what he did, he wouldn’t get banned. This opened the door to the fine grained manipulation that led to the embedding of the Message.

At the core of the hack is the work of a dozen or so, backed by an army of a thousand who downloaded and ran the autovoters and also backed by an untold number of others that unwittingly fell prey to the spam url autovoters. So why do they do it? Why do they write code, build complex applications, publish graphs – why do they organize a team that is more effective than most startup companies? Says Zombocom: “For the lulz”.

This also points out to the incredible mis-managment of the voting server, as well as the total lack of QA for the voting application.
I’m not downplaying the technical feats, but really what a show of we_do_not_know_what_we_are_doing ™.
Though a comment or two in the general vicinity of “too much time on their hands” comes to mind ;-)

@pruneau – this relates directly to the music machinery business. If a poll run by a large media company (that should know how to run a poll) is so easily hacked, and with such finesse, by a loose collective, for fun, imagine what an organized group, motivated by money could do to a social music recommender. The next time you get a recommendation that says something like ‘people who like XXX also like Coldplay’ think hard about how that recommendation came about.

That’s the most idiotic mental leap I’ve ever seen. How do you get from geeks messing around to hacking elections? Yeah, shit from companies like Diebold is insecure, but it’s not able to be hacked just by sending URLs.

This is an interesting article…I’m fascinated by the applications that were built for this poll. I honestly thought it was just scripts that were being run on websites that were activated when users visited them(like the MOOTHATTAN).

the actual request send a key variable too, containing the md5 of the salt phrase plus the rest of the URL. So the key for the above entry is md5(lego-rulespollName=time100_2009&id=1883924&rating=1).

Without the key, or with a bad key, the vote is identifiable as fake. Even if you take the exact request made, and then spam it for 100 rating everytime, you can’t vary the rating unless you have either manually obtained the key, or could generate it.

The article makes no mention of key-caching, nor does it mention the method for key generation, yet the program seems to offer 8 choices for values, including 3 for each of 200+ potential votes. that’s 600 keys, and without those, they can go back the day the contest ends and wipe out all the garbage votes.

Including the ones with bad keys.
Including the LOIC blasts, many requests, but all with blank useragents.

I guess I’m just a bit jealous because I actually -did- reverse engineer the entire system and built a foolproof autovoter, and the brute-force machine here gets all the credit.

For cripe’s sake, all they need is a decent firewall with stateful packet inspection. Hell, I could write you a rule in Check Point’s SmartView Tracker to take care of that kind of problem in a few minutes. Drop any IP that makes a request to range x, protocol https, more than x times per x amount of time, expire after x hours.

you dumbass. this could easily block legitimate traffic. web pages often require numerous hits in rapid succession for things like scripts, css files, and images.

sounds like you should be working for AT&T with that kind of shit logic.

also, it’s sad that this is called a hack. to supplement my income, i offer services to _cheat_ at online contests, mostly battle of the bands type stuff. what’s interesting is most of these shitty battle of the bands contests are way better designed than this TIME poll. They require email validation, or at the very least limit by IP (which has its own set of problems). TIME should be embarrassed. but my point was that this is cheating, not hacking. determining a GET url, or pulling a string out of a flash file is not hacking. and no hacker uses Delphi, that’s just a joke.

Forget Coldplay…How about thinking about the vote manipulation next time you see a poll that says the public oh I don’t know lets say favors invading a foreign country or giving a couple billion to Goldman Sachs or … Great job on the hack and in particular thanks for making it public.

However, it seems that you either didn’t watch Contact, or didn’t pay attention to it, because you just drove a truck over one of my favorite movies.

Ellie doesn’t find a message “embedded in the digits of pi,” as you say. She finds a message embedded in *radio waves* coming from the Vega constellation. The message is encoded in a frequency that is the numerical value of the atomic weight of Hydrogen multiplied by π. The message was coming in bursts, with each set of bursts corresponding to a number in the Fibonacci sequence.

The idea was that intelligent beings somewhat like ourselves — not universe-creators, but simple ordinary run-of-the-mill aliens who happened to be older and more technologically advanced than us — sent a message in search for sentient beings like themselves. They sent a message that could only be interpreted and understood by intelligent life, and would be unmistakably the product of intelligent life. Using H*π and the Fibonacci sequence was a way to ensure that anyone *hearing* the message would not confuse it with a natural event.

If there was a message to be found in π, presumably someone would have found it by now, without needing to spend so much time listening to radio static from the sky.

Isaac – I’m talking about the book, not the movie. In the book ellis asks the aliens what they wonder about. They hint to ellis about a greater race that engineered the universe. They hint that there might be messages hidden in the constants like e or pi which leads ellis to start her next search for patterns in pi. Read the book. It is really quite good.

I agree that Time seems to have no idea when it comes to their online poll, but maybe this just gives us an idea of how seriously they take the poll. It’s as if they said “Look at my delicious sandwich!” and, as we all struggled to find ways to steal the sandwich, Time decided it wasn’t really hungry and went bowling instead.

Did you read the BOOK? After all the stuff with the messages from Vega, Ellie Arroway (oh, and in the book she wasn’t the only one who went into the machine) learns during her voyage that the beings who sent the signal have also been looking for signals themselves, signals from beings in some higher plane of existence that could have encoded messages into the fundamental constants and numbers of the Universe itself.

When Ellie (and the others) return, people don’t believe their story, and they’re forced to keep quiet by blackmail. But in secret they do their own studies and eventually find messages encoded in pi itself, giving them the evidence they need to reveal to the world what they had seen and that they didn’t just imagine it all.

This whole section at the end offers a more optimistic ending, I think, but it was cut from the movie because it was felt that moviegoing audiences wouldn’t get it.

So yeah, Paul didn’t misinterpret the movie — you should read the book.

Marblecake rules. We are amused.
Thank you for a very informative and fascinating article. You’ve solved a mystery for me.
I am, however, a little annoyed. You could have protected us all by simply mentioning the acrostic of Marblecake. Thanks to you, I’ve just lost the game.

> @ plamere Says:
> April 15, 2009 at 4:59 pm
>
>@pruneau – this relates directly to the music machinery >business. If a poll run by a large media company (that should >know how to run a poll) is so easily hacked, and with such >finesse (…)

I stand corrected. And thank you for taking your time to do and write about such an investigation.

But really, from my point of view (i.e. someone with some experience configuring networks and/or securing device and/or developping network software), the polling company did _NOT_ do due diligence.

The simple fact that someone using the same IPv6 source address can still do more than x votes per second shoot their whole “security” system downs right there.

They should review this, and appraise the fact that today internet is not a nice place by any means, or just admit that their polls can be rigged and be done with it. And use the results accordingly.

What makes this particular case special is that the rigging was done with neither malign, nor stealth intentions: it’s obvious for everybody to see.

Thanks and kudos to the “acrostic” hacker group for teaching us a valuable lesson in security. I hope the poll company gets it, tough.

@pruneau – the fact that the company that messed up so royally happens to be one of the largest media companies in the world is predictive of the type security we are likely to encounter at many other online companies. For most, I suspect that security is an afterthought. We are increasingly relying on social systems to help us pick our music, our tv, our news, our political candidates. The fact that they are so vulnerable to hacking has potentially large impact on our lives.

I think after the past couple of years TIME.com doesn’t really care who spends thousands of hours and leverages hundreds of fan sites to manipulate the poll. It’s not represented as a scientific undertaking and they are attempting good journalism while idle hands play around with their toys. Incidentally, NASA’s poll was recently hacked in the very same way and probably for the very same reason.

Lastly, TIME.com probably doesn’t have a vested interest in canceling those bot votes – since they are now counting that as user activity and will be able to justify higher ad rates to their adverts. Maybe, just maybe, TIME.com got the upper hand in this one and leveraged the power of the rabid fans to artificially increase their value!

Wow, I am so far from techie, but that was interesting – and maybe slightly frightening. But, yeah, definitely fascinating. A whole ‘nother world that we’re all connected to – whether we know it or not. Thanks!

Actually, we don’t know if every digit is in pi. Just because it’s irrational doesn’t mean it’s normal. You see, a normal number is one in which every digit has a uniform distribution. There doesn’t exist a proof that pi is normal. (For that matter, neither is there a proof for euler’s number) Therefore, seeing such a message (by itself) doesn’t mean anything but it can lead supporting evidence to a conclusion that a greater power was manipulating certain things.

I have no idea whether the book goes over normal numbers, though, as I haven’t read it either.

So why do they do it? Why do they write code, build complex applications, publish graphs – why do they organize a team that is more effective than most startup companies? Says Zombocom: “For the lulz”.

I’m not sure that the calculation of pi contains all messages (like the infinite number of monkeys at typewriters generating the complete works of Shakespeare ({B Newhart, [Infinite Proceedings]}), but pi can’t be altered even by God (who in the bible didn’t even work it out to the nearest unit: 10 cubits x pi = 30). I know that Carl Sagan as author wanted to throw a little cosmic easter-egg to his abused heroine, who had just had the alien equivalent of the e-mail from Nigeria (aka “The Spanish Prisoner”), but I’m sorry that he chose that one. I would rather have had a straight line of pulsars in the sky signalling HANG IN THERE JODIE.

Alternatively, he could have left the question open for a little longer of whether all along she really -was- in a state of schizophrenic delusion. Most people who receive alien messages are.

Incidentally, there’s a Star Trek book where there are supercomputers whose data processing power is underestimated around 99 per cent because they spend most of the time goofing off, and for fun they have calculated that pi does finish up as a repeating decimal. But it’s a throwaway joke line. I can’t produce the proof that pi is not a fraction (a rational number), which is what a repeating number is, but I trust the people who can.

I can nearly imagine a universe whose physical rules are so different that although pi exists it is not significant, and the measurement of configurations is dominated by the constant called zelda, which I probably just invented. that is as close as I would contemplate to changing the actual value of pi. Also, the decimal (digits 0 to 9) expression of pi probably partly depends on the fact that we have ten “digits” on our hands, whereas other critters don’t (Yakuza). But that is a variable that does not allow for much expression. Actually, pi calculates really nicely in hexadecimal, and I think that serious pi nuts use that.

The decimal value of another “irrational” maths number, e, starts off with,

2.71828 18284 59045 23536…

Look at the hundredths to hundred-thousandths digits. And look at the millionths to the billionths. Now how about that?

And again the mainstream media misses every relevant point. And a blogger has to do the work. Time Warner will learn a lesson from this, but not the lesson they need, because they won’t ever read this article and take meaning from it.

I used actually work for a subsidiary of Time and I was at least partly responsible for making sure that there were no glaring security holes in the code that was written before it went live.

One of the biggest problems was that there wasn’t enough buy-in from management in this regard and hence the ratio of developers to security auditors was far too high. It simply wasn’t possible to read every line of code manually so we used automated tools and random sampling. Clearly, this leaves some room for improvement. Automated tools can pick up a lot but will always miss some things that a competent auditor will see.

Another problem was the quick turnaround required on some code. For instance, the poll can be announced in the magazine or even on the website by the editors and no mention of this made to the developers until a couple of days before it is due to go live. This shouldn’t happen, but it does and because the developers pull an all-nighter to get it up and running, the editors never have to take the blame for it not working. Hence, no incentive to modify their behaviour.

The last problem I will mention here was the varying nature of the developers. Some had good security coding practices and others didn’t. Team managers should have been fixing this when auditing code but, as I said earlier, not enough auditing was done.

I believe things have improved a little in the department where I worked but obviously the problem is not a solved one just yet. From my observations at other places I have worked and/or audited, these problems are not unique to Time.

For those who think this is just an odd case out: I do web development professionally. I spend a lot of time trying to convince clients to pay for security features and I get denied a lot. They think that no one is going to ever hack their application. It’s a bit disconcerting.

the article says the first draft allowed you to revisit a url to cast a vote.

the final draft had “the salt” hard coded in the flash swf file, which there are many descriptors for.

unless I or the article missed something, they didn’t do any more “hacking” than the average joe who puts up an unreachable score on a flash web game scoreboard. There are many ways to secure a poll better than this.

frankly I’m a little surprised they went into depth of writing a few if else statements to spell something out in the poll… I think the real surprise is that Time 100 didn’t use any real security, even after noticing a threat.

amen. this isn’t hacking. they cheated an online poll using some really simple techniques. flash decompilers are a dime a dozen, and the string “lego-rules” or whatever it was would stick out like a sore thumb. writing a gui in delphi with fancy plots just drives home that this was a fun little poll manipulation (something the /b/ guys seem to do a lot of), not a sophisticated, or even unsophisticated hack. they just have too much time on their hands (and i’m jealous) and they command a really large flock of sheep.

Could you you please ask Time.com to comment on this story? They have just announced that moot is the winner while denying any tomfoolery and not mentioning the ‘marblecake’ business at all. I find it hard to believe they dont know anything about this; its at least shoddy journalism and at worst blatant lies. Wheres their integrity? SOMEBODY please expose them.

Interesting article. The hack is specktacular in that it is to Time mag, and lulz, but technicaly straight forward to any competant and experienced web programer who has the balls and inclination. Due to it’s hameless nature to such a powerful media organisation I admire Mr Moot.

This was certainly VERY interesting, and until I read the very last part ‘lulz’ this type of stuff always makes me want to learn how to script. But ‘for the lulz’! that kind of behavior is disgusting, it can hurt SO many people, just for the laughs? What happens when some teenager decides to try and hack something really important, just for the kicks? The CoS (Church of Scientology) does deserve whatever happens to it, I agree with people hacking and whatever them, but only because it is for a reason, a very good reason! For fun is almost never a good reason

wait… this is hacking? i thought hacking required way more 1337 skills than what is described above. at least thats what the movie hackers paints it out to be. i mean this is fairly simple scripting… its not like they de-rainbowed the md5 or anything crazy

Hacking at its core, is simple exploitation. No matter how open that exploit is.. From item duping, to draining millions of dollars from banks. Although Hackers, are known for their skills, and ability to find these exploits by applying past knowledge. Script-kiddies on the other hand, use other peoples works, in the intent they are meant for, rarely discovering un-trotted territories. They also rarely know how a particular exploit works, they just know that it does. Just because you use someone else’s tools doesn’t make you less intelligent. In fact few hackers out there actually have their own OS they coded and designed, which means few hackers can say they stand on their own 2 feet completely.

Oh and “for the Lulz” is a great reason to do something, because if you ever need to do it again in the future, you now have the experience to make it go much smoother the second time around.

There is nothing more dangerous than a self proclaimed genius, especially when they have all the information of the internet at their disposal

If you believe you are to stupid to do something, you probably are.

4chan itself is more than just a zest pool, its a breeding ground for true creativity. Thousands of people, all trying to be able to say they are the creator of something popular, while simultaneously remaining anonymous, destroying all proof that the work was ever theirs. Lets divide 4chan’s populace into 4 categories, noting people can transcend, and regress, any or all of them at times.

NewFags – the bread and butter behind 4chan, usually without any photoshoping skills, they hang around perpetuating their favorite memes.

Trolls – these cheeky bastards, are here on a psychological level. They lead the NewFags around like sheep. They are manipulative and controlling, taking pleasure in their ability to predict, and control others.

OldFags – Players who consider themselves Senior members of 4chan, they have often been around for a long time, but not always. They usually have atleast moderate photomanipulation skills, and have a deeper understanding of what 4chan is about. Most of the meme’s come from this level, although some do come from the trolls.

Underground – This would be the elite’s with varying skills, and to much time on their hands. I havnt been around long enough to know that much about this catagory, and all I know are rumors

Referring to the whole “worrying about how serious media companies take their security” thing:

I challenge anyone with the skill level of those who conducted this “hack” to try to extract a single dime from a system that Matters to Owners™. Seriously, the whole “manipulated by some collective” angle originating with the first post is nonsense. Anything that matters to someone on this planet, online or offline, is secured as it should be. If security seems lax it’s because nobody gives a damn about a TIME.com online poll, including TIME.com itself.

If you ever saw a successful, real hack on a valuable target know that the perpetrators are no /b/tards and have been spending valuable time on actual technical learning rather than “trolling the internetz.” And they won’t do it “for the lulz” since they are well aware of the consequences (and yes, an actual act of “hacking” has dire consequences if one gets caught).

In the end, this boils down to “lulz:” cheap, short-lived, insignificant, disaffected-youth-type enjoyment. Have a lollipop.

P.S. It’s tough, rough, and cold out here. Stick to your armchairs so long as respective basement owners haven’t applied the Final Solution to your little den.

Has anyone considered the possibility that this whole thing was just a way to get the internet generation interested in and talking about Time magazine? Why did they put moot in the poll in the first place?

Someone mentioned 4 categories of 4chan users, but they’re wrong. There are 2 categories: cancer and not-cancer. And considering how much child porn is on /b/ I’d say cancer wins the numbers game.

Being myself a follower of Pharyngula, I can agree that the polls will go crazy. I mean really crazy weighted in one direction. But this…I cannot imagine the degree of organization that went into this. Truly fantastic.