Open Source, Data Science, Startups, and Life

Welcome to the Age of Free Energy

Well, its official, the future is here. This week we were introduced to how we’ll power our homes, businesses, transportation, and industries for years to come. In Elon Musk’s keynote, he took us on a journey that started with a very simple problem statement. We are producing energy poorly and polluting the planet in doing so. Like any good engineering problem, the solution lies in breaking down the problem into its individual parts. The first part is where to get the energy. Well, it turns out “we have this handy fusion reactor in the sky called the sun” explains Musk. Unfortunately, the sun doesn’t shine at night, so the second part of the problem is how do we store the energy to use it at night and off peak hours (as our energy needs fluctuate). Turns out the answer is a battery pack. So there you have it, we now have a way to store the sun in our homes and power our lives not only in developed countries, but technically anywhere. How to make this economically feasible is a question left unanswered.

Copyright of Tesla Motors.

Curious how the battery pack works? You can actually view the patent here. In fact, Tesla went even further to remove its patents and open source the technology for anyone to use. With Tesla’s announcement this week, it is clear to me that they are not a product company trying to sell a few cars to the wealthy, but it is a technology company looking to make a societal revolution by allowing others to improve on these technologies without fear of litigation.

Why might you ask am I talking about energy technology on my analytics blog? To me, there are striking similarities to how data as a resource is consumed by the few who have the skills, technology, and resources to apply it in their every day lives. The problem statement in this case is that we continue to manage data poorly and are polluting the decision making process with bad insights that impact every aspect of our lives. Similar to the energy problem, there is no single solution, but multiple parts that we need to address. The first part of this problem is the pervasiveness of data exhaust from every digital thing that is poorly instrumented and difficult to work with. For this, we have a pretty good solution called “Hadoop” that we’ll need to continue to make easier for people to use. Hadoop clusters in your home anyone? Don’t believe me, check out the company BigBoards. Next, we’ll need to find a way to store and process data efficiently. For this, the front runner to me is Spark in its ability to crunch through data fast, applying sophisticated operations to find patterns in data, and then stream it into applications easy to use APIs. Last, we’ll need to learn how to engineer a universal way of consuming the “energy” or as it is commonly referred to in the analytics category, the “Insight” that comes out. For this we have not yet identified a universal solution, and represents the Data Science Last Mile I have written about before. Looking into the future, there is a lot to be optimistic about as I look at additional parts of society we’ll disrupt: