Conservative bloggers have arrived at the "anger" stage of grieving, going so far as to accuse contract developers of sabotaging the Mitt Romney campaign's "get out the vote" efforts. But Romney campaign Digital Director Zac Moffatt told Ars that Project Orca was relatively successful—and that it was not a determining factor in Romney's election-night loss.

Project Orca was a Web-based app developed to help the Romney campaign track which supporters had voted and to help poll watchers report any potential voter suppression, fraud, or other irregularities back to the campaign for follow-up by its legal team. Volunteers at polls across the country were to access Orca from their smartphones and feed all data back to the main Romney campaign site inside the Boston Garden. But, as we reported, Orca caused widespread frustration after user credentials were issued improperly. At one point during election day, Comcast cut off inbound traffic to Orca's servers because it was mistaken for a denial of service attack. And even while the system was working, the high volume of data being sent to the server caused such slow response that it appeared to some users that the system had crashed.

Here's how Moffatt described the problems a few days ago to CNET: "The primary issue was we beta-tested in a different environment than the Garden [Boston Garden, where the 800 campaign staffers were working]. There was so much data coming in—1200 records or more per minute—it shut down the system for a time. Users were frustrated by lag, and some people dropped off and we experienced attrition as a result."

When I spoke with him today, Moffatt didn't call Orca a success, and he acknowledged frustration with the system. But he also insisted things weren't quite as dire as some accounts from frustrated volunteers suggested. "I can tell you that data from 91 percent of counties in the targeted states came in," Moffatt said, "and that we had 14.5 million people who were marked as having voted. And there were 4,397 reports of incidents that we were able to pass to our legal department."

A compressed schedule played a role in Orca's difficulties, but there were other factors. While it tapped into data collected by the Web and mobile efforts of Moffatt's digital campaign operations, a separate team developed Orca. According to multiple sources familiar with Orca, the project was overseen by Romney campaign Director of Voter Contact Dan Centinello and did not involve developers working for Moffatt's operations, such as devs from Targeted Victory—the company co-founded by Moffatt—and from mobile developer Rockfish Interactive. Moffatt said he didn't even see the Orca project's code until it went live on election day.

Listen all y'all, this is sabotage?

"For whatever reasons, the conservative bloggers have latched onto Orca as the reason it all fell apart," Moffatt added. Those bloggers have suggested that developers with Democratic sympathies somehow acted as a fifth column within the Romney camp. Targeted Victory was singled out by some bloggers because a few of its developers worked for Al Gore in the past.

"Why do they have Al Gore's dev working on Romney's social media development?!" blogger Catherine Ann Fitzpatrick asked. "Truly, how can they expect dedication?" She also singled out another developer who is African-American and "who has a 96 percent chance of being an Obama voter… I will be accused of 'racism' for even flagging. But it's the truth."

Moffatt said that this kind of thinking puts too much faith in a single piece of software. "Anyone who knows campaigns knows this was all baked in before that day—there was no magic, Orca wasn't a silver bullet," he said. "But on the flip side, when you're on a campaign that has only six months of infrastructure time, sometimes you have to throw a Hail Mary. It's really hard to go up against someone who has four years of lead time."

As far as Orca drawing away volunteers who could have been involved in less high-tech get-out-the-vote efforts, Moffatt also disagreed with the bloggers. "Some people are saying if those 37,000 people went out and each brought 20 more voters to the polls, it would have changed the result," he said. "But that would assume we knew the 20 people who weren't going out and voting."

What about the decision to outsource Orca instead of assigning it to a group like Targeted Victory? Moffatt says that too was the right decision. "We don't build things like that; we're not a firm that would build it," he said. "If we had built it, it would have broken—we know our limitations." Nor was Orca a good fit for the other companies that the Romney campaign relied on for voter engagement apps and Web development.

Ars attempted to contact members of the Orca development team for further comment through various channels, without success.

Ah, always delightful to see how fast the misdirection and accusations start flying, you'd almost think that Romney was new at this whole politics and management thing...wasnt he supposed to be America's CEO or something?

But he also insisted things weren't quite as dire as some accounts from frustrated volunteers suggested.

Bad news, sirrah. Volunteers are the lifeblood of campaigns. You could have saved a lot of face by apologizing and preparing Orca 2, this time with user feedback and advance testing, but instead you do this. Please do us all a favor and just stop. You're hurting america. Just... stop.

To me the idea of sabotage is a bit silly. Sure, its likely that the developers were Obama supporters, but really, does that make a difference? Just because they support Obama doesn't mean they were willing to risk their job and their career just to make a flimsy political statement.

What I'd like to know is why America(n conservatives) isn't completely embarrassed by the far right media?

I live in the UK, but watching fox news is like watching a comedy channel....Rush Limbaugh, Sean Hannity....Glenn Beck.......its horrible. Edit: and how could I forget Bill O' flippin Reilly?....and Sarah Palin O_o

How many times did Romney have to lie to Americans during primaries/debates/ads/conferences before someone in almost any part of "big media" just came out and called the man a damn liar?

How does a presidential candidate who ranked 47th out of 50 states in job creation while governor of Massachusetts, run a campaign citing himself as job creator in chief for over a year and half of America believes that guy is "the guy" for job creation?

It makes half of America look no better than extremists who I'm sure don't sign up to the madness the media presents the majority of "the right" to be, but are represented by what I can only call maniacs.

Call me a naive developer, but why did they only had six months to prep this IT infrastructure? They would need it whoever was the nominee, right? And they need this stuff at every election, each two years, so they could have created it long ago and only enhance the system incrementally for each election, maybe rewrite it when it's too old and creaky and there are new revolutionary technologies to embrace (but do that way in advance and have the previous system as a fallback), etc. Having each candidate create his own campaign IT infrastructure from scratch, at every election, is stupid.

I'm very surprised that this could possibly be blamed for the election result. As I see it, they're attempting to imply that seemingly sporadic faults caused by phantom saboteurs prevented the Romney camp from reacting to real time data and coordinating get out the vote efforts across tens of thousands volunteers that would have otherwise encouraged almost a million voters across 9 swing states to get out and vote. All in the space of a voting window of some 10-12 hours depending on the state. Had they succeeded it sounds like this would have been one of the finest examples of organisation and execution in the history of mankind.

Call me a naive developer, but why did they only had six months to prep this IT infrastructure? They would need it whoever was the nominee, right? And they need this stuff at every election, each two years, so they could have created it long ago and only enhance the system incrementally for each election, maybe rewrite it when it's too old and creaky and there are new revolutionary technologies to embrace (but do that way in advance and have the previous system as a fallback), etc. Having each candidate create his own campaign IT infrastructure from scratch, at every election, is stupid.

That's not how Republicans roll. Every man for himself, otherwise it's communism.

"Why do they have Al Gore's dev working on Romney's social media development?!" blogger Catherine Ann Fitzpatrick asked. "Truly, how can they expect dedication?" She also singled out another developer who is African-American and "who has a 96 percent chance of being an Obama voter… I will be accused of 'racism' for even flagging. But it's the truth."

Call me a naive developer, but why did they only had six months to prep this IT infrastructure? They would need it whoever was the nominee, right? And they need this stuff at every election, each two years, so they could have created it long ago and only enhance the system incrementally for each election, maybe rewrite it when it's too old and creaky and there are new revolutionary technologies to embrace (but do that way in advance and have the previous system as a fallback), etc. Having each candidate create his own campaign IT infrastructure from scratch, at every election, is stupid.

Because you could never trust an app that your mortal enemies might have used. Those dirty communist liberal hippies put in backdoors and spyware to steal your data and steal the election from the god-fearing people who are fighting for all that is good in the world. /conspiracy

Personally, I agree that both sides could use apps like this. It sounds like a great idea and use of technology. It's not something that should wait until 6 months before the election to be developed.

I'd really like to see this Orca thing and get to know what it exactly does and how it is supposed to work.

A mere 1200 requests per minute make the server choke? Was that a repurposed netbook with a spinning drive or what did it do upon a request that makes it slow down?And in a country with that many people, how could you not expect to get a crazily high peak traffic?

This looks to me like the book example of an IT project gone wrong on the planning and organizing side of things. "We tested in a different environment [...] there was so much data coming in [...]".No proper load testing? Works fine with the dozen people around just now and their smartphones so it's ok?

And sabotage?I'll stick with Hanlon's razor here. But yeah, conspiracies are more flattering than failing due to your own faults.

Quote:

She also singled out another developer who is African-American and "who has a 96 percent chance of being an Obama voter… I will be accused of 'racism' for even flagging. But it's the truth."

Oh my god.Around elections I always get the impression that the entire US population is straight out of the loony bin. I'm glad I know enough US citizens to know that this is not true and cases such as this are not the norm.

Call me a naive developer, but why did they only had six months to prep this IT infrastructure? They would need it whoever was the nominee, right? And they need this stuff at every election, each two years, so they could have created it long ago and only enhance the system incrementally for each election, maybe rewrite it when it's too old and creaky and there are new revolutionary technologies to embrace (but do that way in advance and have the previous system as a fallback), etc. Having each candidate create his own campaign IT infrastructure from scratch, at every election, is stupid.

That's not how Republicans roll. Every man for himself, otherwise it's communism.

I would argue that Republicans were incredibly well organized until a few years ago, when in the span of a few years the entire country experienced a revolution in social media and they got left behind. At the same time, the party's carefully planned conservative strategy fell afoul of economic implosion, a pair of wars, and G. W. Bush's incompetence.

At the same time, the democrats, which had largely been denigrated as being liberal hippy nutjobs became popular with a huge segment of americans because they were tech savvy enough to reach out to their constituents through social media, and to reach out effectively. On top of that, the democrats knew how to effectively read the data, while the republicans didn't.

On the other hand, Obama's campaign effectively leveraged a massive database to mobilize volunteers and voters, but how much of that can the democratic party take credit for? How much of Obama's database was made accessible to the party at large, to assist democrats running for lesser offices? I would argue that the "every man for himself" mentality is simply American mentality, and it's bipartisan enough that Obama's massive IT project was for him alone, and almost none of his project's success was spread to the rest of the party, except incidentally. That may change, as both sides realize how important metrics are to effectively leveraging support. Expect to see these massive IT projects to be integral to both parties, and not just presidents, four years from now.

I live in the UK, but watching fox news is like watching a comedy channel....Rush Limbaugh, Sean Hannity....Glenn Beck....... and how could I forget Bill O' flippin Reilly?....and Sarah Palin O_o............its horrible.

I'll copy/pasta something I posted on BoingBoing a few weeks ago about a post about Ann Coulter calling Obama a retard, because it's a comment which applies to everybody you listed above, and everybody who participates in sensationalist journalism-

'I worked in talk radio for more than a decade (news, politics, sports, current issues) with some shock jock types. I can tell you that the vast majority of them didn't actually believe most of the outlandish, ridiculous crap that came out of their mouths. They said it because they knew how to get a rise out of their audience; how to get their blood boiling. That's all talk radio is and that's all this is. It's just bait to provoke the less-enlightened listeners who actually think the hosts believe what they say.

Personally, I agree that both sides could use apps like this. It sounds like a great idea and use of technology. It's not something that should wait until 6 months before the election to be developed.

True, if I were the DNC I'd want the Obama campaign's software and databases. Since he cant run again for President, it would be best if all the databases, software, etc. get turned over to the party. Then they can continue to work on it between now and when the nominee is selected in 2016.

I'd really like to see this Orca thing and get to know what it exactly does and how it is supposed to work.

A mere 1200 requests per minute make the server choke? Was that a repurposed netbook with a spinning drive or what did it do upon a request that makes it slow down?And in a country with that many people, how could you not expect to get a crazily high peak traffic?

This looks to me like the book example of an IT project gone wrong on the planning and organizing side of things. "We tested in a different environment [...] there was so much data coming in [...]".No proper load testing? Works fine with the dozen people around just now and their smartphones so it's ok?

And sabotage?I'll stick with Hanlon's razor here. But yeah, conspiracies are more flattering than failing due to your own faults.

Quote:

She also singled out another developer who is African-American and "who has a 96 percent chance of being an Obama voter… I will be accused of 'racism' for even flagging. But it's the truth."

Oh my god.Around elections I always get the impression that the entire US population is straight out of the loony bin. I'm glad I know enough US citizens to know that this is not true and cases such as this are not the norm.

The first article by Ars describes it as a single web server at the Garden where they were setup. It wasn't even in a proper datacenter.

"Why do they have Al Gore's dev working on Romney's social media development?!" blogger Catherine Ann Fitzpatrick asked. "Truly, how can they expect dedication?" She also singled out another developer who is African-American and "who has a 96 percent chance of being an Obama voter… I will be accused of 'racism' for even flagging. But it's the truth."

Truly one of the great double-edged swords of the internet. It's fantastic that anyone is allowed to say what they want. It's horrible that anyone is allowed to say what they want.

A web app causing a president candidate to lose? Didn't they have contingencies? Maybe Obama had a better native app. Maybe Orca spent to much time around those liberal penguins called linux.

A much more likely story:A developer didn't do their jobs, they didn't run it on the cloud.People just didn't vote for Romney and a web app wouldn't help anyways.

Yep. After failing to encourage people to vote after months of campaigning and millions in advertisements, phone calls and rallies it all came down to being unable to find out that a person wasn't voting on the day and getting someone to their place of work or residence to convince them to vote over the space of 12 hours.

This must be why apple's stock is well back from their high, clearly the election showed where the most effective RDF actually is /joke

joematt wrote:

Comcast throttled them. That's all.

Which is rather ironic, after all, don't companies like Comcast generally favor purported 'pro business' candidates like Romney? Obviously i don't think it was an intentional throttle, but just a bit funny.

I find the concept behind orca pretty good (though its a shame narwal gets glossed over so often...since it did work :x ). Will be interesting to see if they refine it for future elections or just shelve it.

"Why do they have Al Gore's dev working on Romney's social media development?!" blogger Catherine Ann Fitzpatrick asked. "Truly, how can they expect dedication?" She also singled out another developer who is African-American and "who has a 96 percent chance of being an Obama voter… I will be accused of 'racism' for even flagging. But it's the truth."

Truly one of the great double-edged swords of the internet. It's fantastic that anyone is allowed to say what they want. It's horrible that anyone is allowed to say what they want.

I'd be fascinated to learn how many Nazis said, wrote or otherwise claimed that their views on Jews and other undesirables were simply "the truth".

Saying that a racist comment is the truth doesn't make it so, but it does make you racist even when attached to disclaimers.

I thought it was obvious why Romney lost... In order to placate the worst amongst us, he sold his political soul and alienated the middle. Had he held firm in the primaries, and (obviously) won... He'd be our next President.

Instead, he spent a year trying to be the biggest a-hole on the block... And succeeded...

"There was so much data coming in—1200 records or more per minute—it shut down the system for a time."

Was this running on a toaster?

Yeah I noticed that too... just goes to show they're trying to save face with even more lies.

I had to re-read that twice as well to make sure I wasn't imaging things. Had to have been a toaster.

Regarding the bias of developers: I can't speak for them, but I know it would seriously be my ass if I ever intentionally sabotaged something I was working on. Not only would I be fired from my current job, I would then have to explain why I was fired to every future employer. It's seriously not worth it. I actually had a somewhat similar experience happen to me as well some time ago: I was asked to do some work for a Christian evangelism application, despite the fact that I'm an atheist. I might have grumbled a bit, but I *certainly* didn't sabotage the app.

Sean Gallagher / Sean is Ars Technica's IT Editor. A former Navy officer, systems administrator, and network systems integrator with 20 years of IT journalism experience, he lives and works in Baltimore, Maryland.