Rethinking how the inflation rate is calculated

Ever wonder how the government calculates inflation? Meet George Minichiello. He's searching through the meat aisle of a Brooklyn grocery store.

"Here it is," Minichiello announces, "pork roast. And the pork roast today is $3.99, I'm sorry, $3.49 per pound. Same as last month."

Next he looks for bacon, but can't find the price. He heads over to the cashier and gets the price from her.

"Okay," he notes, "the pound of bacon is priced differently. Last month it was on sale for $3.99, today it's the regular price of $6.99."

Minichiello's an economic assistant for the Bureau of Labor Statistics (BLS). He's out at different types of stores four days a week.

The BLS has hundreds of people like George. They go out to their local stores and write down the prices of certain items. The bureau then uses the prices to calculate inflation. It's quite a process.

"I have pharmacies, service organizations like dentists, opticians, grocery stores, large department stores, small bodegas," Minichiello says. "You know, I almost hate to shop after doing this all day."

But at up MIT in Boston, Alberto Cavallo's found another method for gathering inflation data. Using the Internet.

"Okay," he says, pulling up a list on his computer screen. "So on April 8th, the Coca Cola Cherry Coke is $1.99. The caffeine free Pepsi soda is $1.69."

Back in 2007, Cavallo started collecting costs of online goods. He has a computer program that scans websites' HTML codes, and scrapes out all the prices.

"In the U.S.," he explains, "the estimates of inflation we were

able to obtain with online data do in fact match very well the ones produced by the statistical offices in the country."

He gets the same inflation numbers using online methods as the BLS does sending out people like Minichiello to stores.

Cavallo's method evolved into The Billion Prices Project. The project now grabs about a half a million prices a day. And while the government bureau comes out with monthly inflation numbers, the Billion Prices Project gives the inflation rate daily. Using a lot fewer employees.

The BLS has taken notice. Michael Horrigan heads the office there in charge of inflation numbers. Lately, his bureau has started looking to Cavallo and the Billion Prices Project for guidance.

"We really wanted to learn what sort of what the process was, what the methodology was," Horrigan says. "But it also introduced us into the world of webscraping."

He sees online price gathering as a promising way the bureau can cut down there data collection costs.

"You know," he continues, "this is a day and age where we're trying to figure out ways in which we can deal with the current budget situation."

Both Horrigan and Cavallo say it's too soon for the bureau to only rely on web prices. The actual costs of things like services and real estate are still tough to gather online. But, Horrigan says the government agency will slowly be using more of the Billion Prices Project's methods.

And Alberto Cavallo at MIT? He's already thinking about the next index he'd like to re-imagine with big data: the gross domestic product.