Wednesday, December 07, 2011

And Ode To Data

As an engineer who has hung around lots of test sections over the last few years, I have developed a mortal fear of a four letter word: Data.

Data is nature's way of keeping your ego in check; data is nature's way of proving that your understanding of the universe (or that miniscule fraction of the universe that you are studying) is entirely incorrect (or at least, needs some level of tweaking).

Data is the world's best physics teacher. The curious engineer relishes fitting physical models to incomprehensible data patterns. The high of fitting a model and making measurable predictions is one of the great highs of life. It is often worth the corresponding low when one realizes that there's a caveat to the model that we just developed.

My attitude towards data has evolved since I joined Intel about a year ago. In graduate school my life was centered around data. The emphasis back then was to fit an intellectually elegant physical model to the data - so as to make the results general. I believe there is a physicist within every engineer. It is this inner physicist within every engineer that likes to fit a physical model to the data. A shot at redemption for engineer who is, at some level ashamed of becoming a sell-out.\

Having made the switch from academia to the dark side (the corporate world), I am guilty of selling out all the more. While my loyalties do still lie with data (and at Intel, data is the undisputed king), the way of looking at data is completely different.It is less passionate; it is more dreary and mundane. Heartless sounding statistical methods are applied to the data to cull out main effects and interactions.

Which is not to say that physics is entirely forgotten. Physics is more like a philosophy here; it is what informs one's intuition. All the model fitting that is done here is done in one's head.