Search

Why Economics Needs Data Mining

Cosma Shalizi explains eloquently the major issues surrounding the accuracy of economic modeling. He talks about how we don't know whether what happened in the past can be relevant for he future.

Also, listening between the lines (at approximately 4:30), please hear the plea for the idea that if you're going to do models, you need data to validate them.

Cosma Shalizi urges economists to stop doing what they are doing: Fitting large complex models to a small set of highly correlated time series data. Once you add enough variables, parameters, bells and whistles, your model can fit past data very well, and yet fail miserably in the future. Shalizi tells us how to separate the wheat from the chaff, how to compensate for overfitting and prevent models from memorizing noise. He introduces techniques from data mining and machine learning to economics -- this is new economic thinking. via ineteconomics.org