Our Love of the New

Analysing the rise and fall of websites is the perfect way to shed light on the old debate over whether talent or experience matters most, say mathematicians.

The question crops up everywhere, from job interviews to presidential races, says Vwani Roychowdhury, but it’s hard to examine the problem using hard figures.

However, the same way of thinking can be applied to websites, which also succeed or fail based on many millions of human decisions. In fact, the web may be one of the few places it is possible to quantify the balance between the two, say the researchers.

Roychowdhury and his colleagues Joseph Kong at the University of California in Los Angeles and Nima Sarshar from the University of Regina, Canada, used “web crawlers” to visit some 22 million web pages once a month for a year. Each time they visited a page they recorded the number of other pages that link to it: its “in-degree”.

After the year was up, the team found that the pages they deemed “winners” – those with an in-degree value more than 1000 – were not all old, well-established pages that started the year with a high in-degree and high traffic, as they had expected.

Web pages that hadn’t existed when the year began accounted for just under half of the winners – displacing an equal number of older, more established pages in the process. That proportion held even when the bar for success was raised, suggesting there is a general tendency for young websites to out-compete established websites half the time.

Roychowdhury says the new pages could only pull themselves up from nowhere because of the quality of their content. They were pages with “talent”, able to compete against those with more “experience”.
Political parallel

“Talent versus experience is difficult to document in a society,” Roychowdhury told New Scientist. “But what we show is that on the web it can be documented in terms of page popularity – and newborn pages become more popular than older established pages on a regular basis.”

He adds that the web’s success may be down to the roughly 50:50 balance between experience and talent that allows constant renewal without anarchy. Roychowdhury even sees similarities in US politics. “The fact that Obama got the nomination over Clinton is a testimony to the way a society rarely but consistently gives a thumbs up to talent over experience.”

More practically, the new style of analysis could help search engines improve results, says Roychowdhury. Since Google introduced the idea, many engines use the page in-degree, also known as “page rank”, to order search results.

But using in-degree alone ignores the importance of new emerging websites. Ranking pages by the rate of in-degree growth might better reflect the ever-changing web, he suggests.