What Is Taking a Sip From the Twitter Firehose Going to Cost?

Twitter on Monday gave seven real-time search and discovery companies ranging from “funded startups to part-time, one-man operations” access to 100 percent of its tweets. Twitter’s so-called “firehose” is a valuable asset; the company has made partners like Google (s GOOG), Microsoft (s MSFT) and most recently Yahoo (s YHOO) pay to use it in their own real-time search. The move is part of a new, yet-to-be-standardized initiative of metered access for people and companies that build on Twitter.

From what we can gather, the startups aren’t yet paying much if anything for what they’re calling a “commercial licensing agreement” to Twitter’s firehose. Which is fair — you can imagine that someone like Ellerdale wouldn’t really be in the same category of demands on Twitter’s infrastructure as someone like Google. What the deals represent is an effort towards formalization of Twitter’s developer community, which now operates more than 50,000 applications.

Advertisement

New Twitter communications head Sean Garrett told us that while Twitter isn’t disclosing the terms of the current partnerships, it plans to make them readily available in the future. As he put it:

“As the agreements standardize, we hope to make the terms well known so developers know if the firehose is right for their business. Additionally, for current partners, we would like to help them plan for the future (as licensing costs increase with their business’ maturity).”

Twitter had previously given firehose access to startups including Summize (which it later bought) and FriendFeed. Garrett today called those arrangements “a dalliance” that was “short-lived” when Twitter shifted focus to its core service in light of extreme growth. He said the new deals are, by contrast, “sustainable and scalable.”

The startups that were just given access are Ellerdale, Collecta, Kosmix, Scoopler, twazzup, CrowdEye, and Chainn Search. Previously they’d only had access to a limited rate of tweets — clearly a handicap when you’re trying to respond to search queries and track trends in real time. In blog posts and tweets (except for Chainn Search, which doesn’t appear to have a web presence yet), the companies said they were grateful for the integration. Kosmix, for one, said it’s not “ready to showcase or demo the integration just yet.”

CrowdEye’s Ken Moss had a more personal response, saying he was grateful Twitter delivered on relationships with developers after it had previously shown preferential treatment to paying companies.

Twitter isn’t formalizing these relationships out of the good of its heart; it’s gearing up for a major monetization effort and will clearly expect developers to contribute a tithing. It’s in the company’s best interest for its developers to grow the ecosystem and contribute back.

All Twitter accounts have Streaming API access and can receive various slices of the Firehose, including following a list of users, tracking a keyword, finding all Geotagged tweets in a location, and also a random sampling of Tweets.

This is the key question. I also want to know if non-firehose will become metered? What if I only want access to low volume? To a specific list? It seems like that will remain free. But it is important for start-ups building their platform to know.

It’s funny that there’s been an explosion of “Big Data” stories floating around lately. First in the Economist, and now GigaOm. Not to mention the steady flood all over Tweets…

At Drawn to Scale (http://www.drawntoscalehq.com), we’ve built a platform to solve the challenges of dealing with data like the Twitter Firehose puts out. We make it easy to process, store, and query/search all this data, in real-time.