The Next Big Thing You Missed: Recreate Live Events with Twitter and Instagram

Tarikh Korula, co-founder and CEO of Seen, on a recent visit to San Francisco. Photo: Josh Valcarcel/WIRED

Tarikh Korula’s startup, known as Seen, was born in a moment of frustration. He had organized a hackathon for the popular blog TechCrunch and he wanted to put together a post congratulating the winner, but he couldn’t find an appropriate photo — even through the winner’s demo had been photographed time and again by the gathered geeks.

“There were probably like 10,000 photos snapped,” Korula says, “and I don’t know where they are. It’s nuts that we can’t find this stuff.”

To solve the problem once and for all, Korula teamed up with Mor Namaan, an information science professor, to create Seen, a web service that automatically recreates live events by crawling the web for relevant photos, videos, and first person accounts. If Seen had existed when Korula was looking for his hackathon photo, he wouldn’t have spent all that time trawling Flickr, scrolling through blog posts, and typing search terms into Google. Seen would have already created a special collection of photos and videos for the TechCrunch event, and human curators would have highlighted the most interesting visuals — including, most likely, that elusive shot of the winner.

Korula had a long history building systems that bridge the physical and virtual worlds. One, created for Yahoo, involved a fleet of bicycles that snapped photos as they rolled down the street, geotagged these images, and uploaded them to Flickr. Namaan had worked on the same project, drawing on doctoral work at Stanford that involved organizing large sets of geotagged photos.

Between Namaan’s subsequent work at Rutgers, where he led a social media lab, and Korula’s work at a development studio known as Uncommon Projects, Seen remained on the backburner after work began in 2010. But then a year ago, Korula and Namaan fully committed to the project. They raised a friends-and-family funding round, entered the New York Times’s “timeSpace” incubator program, and ultimately arranged an investment from the newspaper company (which is now in the process of closing).

Seen staffed up to its current six employees, who work out of an office in New York’s SoHo neighborhood. Currently, the company’s service mines Twitter and Instagram to recreate major real-world events, like concerts, conferences, and other newsworthy occurrences. Recently, for instance, it built a virtual version of a New York visit from British street artist Banksy, which brought Seen a spike of new traffic. Events are recreated using a combination of software — which builds an “automatic timeline” by analyzing metadata like tags, titles, and other associated text — and human curators, who help surface the most interesting moments from the timeline.

Seen’s plan is to expand its base of social media networks and further refine its software algorithms for tying media to events. One recent innovation, for example, fingerprints the audio tracks attached to YouTube videos and compares them to a large array of other audio fingerprints. Videos with moments of identical background noise are assumed to be from the same event.

Korula did his graduate work in the physical computing program at New York University. He frames Seen alongside other internet startups, like check-in outfit Foursquare, that are rooted in real-world events at specific locations (Foursquare’s founder Dennis Crowley graduated from the same NYU program as Korula). After struggling for years with technological limitations, such startups are finally poised to proliferate and grow, Korula says. Ten years ago, he worked on a project, sponsored by the designer Tommy Hilfiger, that tracked the progress of a racing yacht using a map and a series of blog posts. Today, the humblest America tourist can carry comparable technology in his pocket.

“We’ve been watching the trend for a decade,” Korula says, “and right now, it’s really obvious to us that there has been a paradigm shift in the amount of media and type of media being shared. We feel like there’s a huge opportunity to organize this shift, and nobody’s doing it … Not everybody sees that shift as much as we do, because they sort of haven’t been immersed in how little media there was five years ago.”