Is Hadoop Killing the EDW? Fair question since in it’s 11th year Hadoop is known as the innovative kid on the block for analyzing large data sets. If the Hadoop ecosystem can analyze large data sets will it kill the EDW?

The Enterprise Data Warehouse has ruled the data center for the past couple of decades. One of the biggest question big data question I get is what’s up with the EDW. Most database developers and architects want to know what is the future of the EDW.

In this video I will give my views on if Hadoop is killing the EDW!

Transcript

(forgive any errors text was transcribed by a machine)

Hi I’m Thomas Henson with thomashenson.com and today is another episode of big data big questions today’s question this made it a little bit controversial but it is big data killing the enterprise data warehouse let’s find out so is the death of universe data warehouse coming all because of big data the simple answer is in the short-term the medium-term no but it really is hampering the growth of those enterprise traditional data warehouses right and part of the reason is the deluge of all this unstructured data so 80% of all the data in the data center and in the world is all unstructured data and if you think about enterprise data warehouses they’re very structured and they’re very structured because they need to be fast right so they support our applications and I support our dashboards but when it comes to you know trying to analyze that data and trying to get that unstructured data into a structured version it really starts to blow up your storage requirements on your enterprise data warehouse and so part of the reason that the enterprise data warehouse growth is slow is because of 70% of that data that’s in those enterprise data warehouses is all really cold data so really you know only thirty percent of the data in your enterprise data warehouse is what’s used and normally that’s your newest date so that cold data is sitting there on some of your premium fast storage you know taking up that space that has your licensing fees for your enterprise data warehouse and also the premium storage and a premium hardware that is sitting on then couple that with the fact that we talked about 80% of all new data that’s coming in and data is created in the world is all in structured data right so they could clean up data from Facebook any kind of social media platforms but video and you know log files and some of your semi structured data that’s coming you know often for your Fitbit or any kind of those IOT or any of the new emerging technologies and so all this data if you’re trying to pack it into your presentation warehouse just going to explode that license fee and then also that hardware and then you don’t even know if this data has any value to it soon and that’s where big data in Hadoop and spark and that whole ecosystem comes because we can store that data that unstructured on local storage and be able to analyze that data before we need to you know put it into the dashboard or some kind of application that’s supporting it so in the long term I think that the enterprise data warehouse will start to sunset and we’re starting to see that right now but for the immediate term still you’re seeing a lot of people doing enterprise data warehouse uploads so they’re taking some of that 70% of that cold data the transfer in Hadoop environment to save on calls to the sable net licensee but also to marry that with this new data this new instruction data that’s coming in from whether it be from sensors social media or anywhere in the world and they’re marrying that data to see if they can pull any insights from it then once they have insights depending on the workload sometimes they’re pushing it back up to the enterprise data warehouse and sometimes they’re using some of the newer projects to actually use their new environment and they’re you know big data architecture to support those production you know type enterprise data warehouse applications so so that’s all we have for today if you have any questions make sure you submit up to big data big questions you can do that in the comments below or you can do it on my website thomashenson.com thanks and I’ll see you again next time.

Winning: The Ultimate Business How-To Book by Jack and Suzy Welch is a book about how to succeed in business the Jack Welch way. The book offers advice for your career from Jack’s experience.
Jack Welch was the famed GE CEO that come up from the ranks at GE. Welch’s tenor as CEO was from 1981 – 2001. During Jack’s time at GE a lot of people under him went on the become executives or CEO of other companies. Winning gives tips for navigating your career as a whole. Not a how to guide for your career but a wholesitc approach to career/business management.

In Winning Jack describes his philosophical management approach. Welch provides many examples from his career at GE and speaking engagments around the world since retirement. One story that stuck out was about Jack’s first big mistake. Early in his career Jack was running a factory at 28 and was responsible for creating an explosion at his plant. Jack was sure he was going to get chewed out by his leadership but instead was asked what he had learned. Imagine the impression this left on Jack, throughout his career Jack took the same approach with his staff. Success has cost and Jack is very open those costs. He admits to failing in the work life balance area and that he didn’t have a very good relationship with his kids while they were small. Part of this could be attributed to a generational mind set because it wasn’t seen as a priority in his era. The book overall covers how to succeed in the world of business and life the Jack Welch way.

With every book I read I like to pull out the points that I can apply in my business/life.

Winning’s 3 key themes

Optimistic outlook– One key theme Jack hits you over the head with is to never consider yourself a victim. Even in times where you are the victim it is does you no good to play the victim card. Instead he encourages readers to take a can do attitude with them into everything they do. Sitting around blaming others is not going to fix the situation, the most important thing is to move forward. Do not let setbacks derail you. Jack devotes a couple of chapters on career development and having a positive attitude is the first key to having a successful career. No matter what position Jack was hiring for he always placed a positive attitude as the first trait he was looking for in a candidate.

Candor – The most used word in the book because it is at the core of all the principles Jack teaches. None of the other principles will work unless you are in a candid environment. Welch differentiates between being candid versus mean spirited. Candor allows you to have honest and open evaluations in your organization. People know where they stand because of the candor in your organization. Think about how well you could benefit from open and honest feedback from your staff and leadership.

Executive Training Programs – During Jack Welch’s tenor as CEO at General Electric, GE was known as a hotbed of talent. Many Senior-level executives and future CEOs came from GE. While hiring great talent accounted for some of that success, the main reason was because of their executive training programs. Jack says the seed for the idea was from Pepsi but the success of the program was from the GE leadership. Management would identify potential future leaders in the company and enroll them in the executive training programs where they would be trained to become executives. In baseball terms Welch called this stacking his bench. He was always trying to ensure when they lost a great leader they always had someone ready to step in and replace him.

Conclusion

This was the second time I read this book and probably won’t be the last time I read it. It’s one of those books you need to read every so often to keep yourself motivated. Some good career tips about how to find the career you want and how to go about evaluating companies you want to work for. Hopefully you will read this book and apply some of the same strategies to your career. If you liked this article be sure to sign up for my email list.