Rich reward for data miners

Glenn Mulcaster

The dream of a Melbourne economist to turn statistical data mining into a rich spectator sport became reality on Friday when the first Australian winner in a two-year global $3 million data prediction contest shared in the first milestone purse.

The worldwide contest is being staged on Kaggle, the brainchild of former Treasury and Reserve Bank of Australia staffer Anthony Goldbloom on behalf of the Heritage Health Network of hospitals in Los Angeles.

Related Content

Melbourne engineer Phil Brierley, who works for IBM as a data miner and sells his own data mining software Tiberius, originally entered the contest independently, but last month pooled resources with Florida fund manager and stock analyst, David Vogel, and Virginia health industry consultant, Randy Axelrod, forming the Market Makers team.

Market Makers claimed a share in the first incentive prize of $50,000 split with an independent competitor, Dutch software engineer William Mestrom.

Advertisement

Brierley previously worked for the National Australia Bank analysing customer credit risk.

Doctors will be able to use recommendations stemming from content's live incoming data to treat patients sooner and prevent costly emergency hospitalisation.

The contest has been running for six months, hosted on a website run by Kaggle, with a live leaderboard that reflects the performance of the algorithms submitted for testing by more than 500 competing teams.

The first incentive prizes were awarded to the top two teams at the O'Reilly Media's Strataconf 2011 data analysis conference in New York on Friday. Leading teams had to agree to reveal their methods to help improve the algorithms that sift through the statistics.

More incentive prizes will be awarded every six months to maintain competitive interest and lift the accuracy of predictions. The major purse in the prize is 18 months away.

Three other Australian teams have spent time in the top 10 this month but missed out on the incentive prizes.

Two Canberra researchers, James Petterson and Tiberio Caetano, from the Australian National University, are in second position overall.

A US corporate data analyst, Opera Solutions, jumped more than 200 places with its latest entry this week, to rank third.

Recent movers in and out of the top 10 include Planet Melbourne, a team led by Melbourne technology entrepreneurs Andrew Conway and Matthew Kwan.

Years ago it was a big-ticket item only available to companies with large software budgets and fast computers at their disposal.

Now cheap desktop computers, fast internet access and open-source software tools have made it relatively easy for individuals with an engineering, mathematical or software background to extract valuable information from datasets on modest budgets using off-the-shelf equipment.

Kaggle was formed to bring the internet phenomenon of crowd-sourcing to solve problems for business and academia.

Algorithms to predict credit worthiness of loan applicants, the risk profile of insurance policy holders and maintenance schedule of trains can save millions of dollars for industry and often yields an immediate and sometimes spectacular return on investment.

Already organisations such as Deloitte, NASA, the world chess federation, Wikipedia and universities in Australia and the US have staged contests on the Kaggle platform to solve problems, awarding cash prizes to winning contestants, some of whom have earned lucrative job offers because of their success. Current contests include an insurance claim prediction prize and another to predict the likelihood of people falling into financial hard times.

Kaggle recently opened an office in San Francisco, where Goldbloom is being courted by venture capitalists for an investment round that could help fund future expansion.