Big Data Career Without Coding?

Do all career options in Big Data demand skills with coding or administration? Big Data projects are in high demand right now, but skill sets for these projects come from different backgrounds. If you are wanting to get involved with Big Data, but don’t have a technical background watch the video to learn your options.

Video – Non-Technical Careers in Big Data

Transcript

Hi, folks! Thomas Henson here with thomashenson.com. Today is another episode of Big Data Big Questions. Welcome back to the new year. Our first thing that we’re going to tackle today in our first episode of Big Data Big Questions for 2018 is going to be non-technical jobs or career options inside of big data.

It’s submitted in from one of our YouTube users. You can find out more right after this.

Today’s question comes in from YouTube. Remember, if you have any questions around big data or anything that you want to ask and you want me to answer, you can submit those in our YouTube comments below on any of the videos, or you can go to my website at thomashenson.com/bigdataquestions. You can submit any questions there, and I’ll answer them as best I can on air, and give you my advice on the Hadoop community, or big data, or data engineers, or any questions that you have.

Today’s question comes in from YouTube, and it’s from Shahzad Khan. He says, “I work as a change manager, and I don’t know anything about Java or Hadoop, but I want to learn this technology. Is it all right for me to learn, since I’m not into coding? Also, I’ve never been involved in a development team, please suggest.”

Great question. Thanks for the comments and thanks for watching. Continue to watch. My first thing when I look at this is, we’ve talked about the ability, and I’ve had a couple other videos that you’ve seen where we’ve talked about, that you don’t have to know Java to be involved in Hadoop. If you have any questions around that, you can check into that. Really, I think this question, I want to frame it a little bit different, and think about, just because you want to be involved in big data, and you want to be involved in the community and all the things that are happening, you don’t necessarily have to have a technical role to be involved in that.

There’s three roles that I want to talk about that are non-technical from the aspect of coding and Hadoop administration that you can do to still be involved in data or even big data. I’m going to put them together. These aren’t just specifically for big data. This can be around data analytics.

The first one is around data governance. When we talk about data governance, we talk about, what’s the flow of data? Where did the data originate? Everybody’s probably heard of the adage or the example of garbage in, garbage out. Where’s your data coming from? Can you trust, and can you automate, and trust the data that’s coming in? Data governance is about where that data comes from, but it’s also about, how timely is that data? You’re really involved with the sourcing of the data. You’re also looking at things around… I remember one of my first career options. I remember sitting around, and we have a couple different applications, and the heads of each application were together, and we were all there to talk about the different ways that we name things in our own databases. If you think about it, we were trying to merge everything into an enterprise data warehouse. This is a little more old school, but it still happens in big data, when we have these different data sources.

You might have an instance where data from one data set is named or has a different key than data in a separate data set, but you want to be able to merge those. Data governance is around, you can help find and help be a part of that, where the data’s coming from, so that’s one option. I would look into data governance if you still wanted to be involved in big data but didn’t have the technical skills or didn’t have desire to have the technical skills.

Another one is project management. We always need good project managers. Project managers, they’re the ones, the workhorses that really help bring the developers, bring the data scientists, bring the front-end developers, bring everybody together, and really gets that project going. Makes sure that we’re communicating. If you’re interested in project management, you can do that from a non-technical perspective. One of the things, though. I’ve got some stuff on my website where I went through and did the scrum master training. Think of agile development. Just like you would in traditional application development, big data needs agile developers or agile project managers as well.

Then, also look at the scrum master training, but also look at DevOps, and see where that is, if there’s any DevOps certifications, or anything that you can provide in that background to be able to help and manage these teams. Project management is a second one, and then the big one, the next one, compliance and security. We always need compliance and we always need security, especially now with the maturity of the Hadoop community and how much Hadoop is taking over and being used in the enterprise. There’s always compliance around it. You think of HIPAA, you think of some of the SEC compliance here in America. Then, you can also think of GDPR. GDPR, General Data Protection Requirements compliance, I would look at that regulation.

That’s something that’s really interesting to me, and if I was somebody non-technical, and I was interested in compliance or security, that is one area I would start to look at, because I think there’s going to be a growing need. Anytime there’s any kind of regulation, and this is a political statement in any way, but anytime there’s any kind of regulation or change in regulation, there’s a lot of things that go on behind the scenes as far as interpreting that and making sure that you’re in compliance with your enterprise, or if you’re working for some kind of public institution, you want to make sure you’re doing that. Anytime something like that, if you can become an expert and move to that, that would be huge as well.

For securing the data, too. It’s an ongoing, probably overused joke. How many data breaches have you heard about? There’s one every day. Big data is not, we’re not, immune to that. In fact, we’re larger, a larger target. Think about the three Vs.

Volume. How much data do we have in your Data Lake? Big data has big data, right? You need to be able to secure that. Those are the three areas I would look at for non-technical jobs if you still want to be involved in data. Data governance, project management, and compliance and security. That’s all for today. Thanks for tuning in. Make sure you subscribe, so you never miss an episode. I will see you again on Big Data Big Questions.

Data Scientist for the past few years has been named the sexiest job in IT. However the Data Engineer is a huge part of the Big Data movement. The Data Engineer is one the top paying jobs in IT. On average the Data Engineer can make anywhere from 90K – 150K a year.

Data Engineers are responsible for moving large amounts of data, administering the Hadoop/Streaming/Analytics Cluster, and writing MapReduce/Spark/Flink/Scala/etc. jobs.

With all this excitement for Data Analytics and Data Engineers, how can you get involved in this community?

Ready to learn tips to becoming a Data Engineer? Checkout this video for tips to becoming a Data Engineer.

Transcript

Hi Folks, I’m Thomas Henson, with thomashenson.com, and welcome back to another episode of Big Data, Big Questions. Today’s question is: What are some tips for learning to become a better data engineer? Find out more right after this.

So, today’s episode is all about tips for learning to become a better data engineer. So, if you’re watching this, you’re probably concerned with, one, how can I start out becoming a data engineer? What are some ways that I can learn to become better? Or maybe you’re just looking to answer one specific question. But all those are encompassed in what we call the data engineer.

A data engineer is somebody who’s concerned with moving data in and out of Hadoop ecosystem, being able to give status scientists and data analysts better views into the data. So, we’re involved with the day-to-day interactions of how that data is coming in. Is it in how we’re ingesting that data? How are we creating those applications and tuning those applications so that the data comes in faster? All to support those business analysts, those business decisions, and data scientists in creating better models and having just more data to put their hands on.

And so, a lot of times what we’re always doing is we’re asked to take on a couple terabytes of data here, maybe implement and do all the configuration for your hives. You know, your hive implementation or HBase or anything that’s in that big data ecosystem. Some of the tips that I’ve found for just getting started, so if you’re brand new to this and you don’t know where to start, the first thing I would recommend is, go out and just download the sandboxes.

So, download Cloudera’s sandbox, or download Hortonworks’ sandbox and just start playing with it. Go through some of the tutorials. Stand up on your local machine in a VM environment, and just start playing with moving some of the data around. Find some sample data, so go to data.world. Also, I have a post and a video on where to find some data sets, so take those data sets in, start ingesting those. I have a ton of resources and a ton of material on just some simple examples that you can walk through with Pig, and some around Hive. So, go there and find some of those. But, basically what I’m saying is, just get hands-on. Start creating applications. Start trying to do some simple things like, ingest some data in, put it into Hive, and be able to create a table and pull some of that data out, and just maybe some simple Hive queries. And do the same thing with Pig, and just kind of go around to some of those applications that you’re curious about, and start playing with them.

Another thing is, is once you start playing, and sampling, and testing that data, get involved. By getting involved, just ask some questions, create a blog post, try to find a way that you can contribute back to the community. I mean, that’s what I did when I was first starting out. I started off with a sandbox, and what I did was, I took and made sure that every day for 30 minutes, I was learning something new in the Hadoop ecosystem. And so, that’s another tip for you too, is to take and try to do this 30 minutes a day, every day. Even Saturdays, Sundays. Don’t take a day off. And it’s only 30 minutes. And if it’s something that you’re passionate about, and you like doing, that time is just going to fly by. But over time, that’s just really going to give you more and more time in the Hadoop ecosystem. So, whether you’re doing this for a project at work, whether you’re already in the ecosystem and you’re just trying to improve, that 30 minutes a day is really going to help. And it’s something that I’ve continued to do, and continued to do, now, even though I’ve been in part of the community for three or four years now. It’s how I just continue to learn, so I make sure I’m always kind of pushing.

Around Christmas I got the flu and had a lot of time on my hands. Sitting around for a couple of days I was able to finish Soft Skills by John Sonmez. Now John Sonmez is also a known software development expert, speaker, and author. One of the things John stresses in this book is to start a blog. In fact, John Sonmez attributes a lot of his success to starting a blog. Okay now that is two experts, authors, and speakers with the same advice. There must be something to starting a blog.

How to make it succesful

Having a blog is important but how can I make it successful?

How can I create content that will help the community?

I already had a blog but I wanted to make it successful. Creating a blog was one thing but making it successful was something else.

After a week of recovery I found a new email course on how to start a blog by John Sonmez. I was skeptical at first because I already had a blog. What I was looking for is how to make a successful blog. I signed up for the course and started receiving the emails over a 3 week period.

The course was great it outlined the strategy for creating great content and where to focus your energy. Now I’m not going to spoil the course for you but I will tell what hit home for me was the consistency part. All the times I wasn’t consistent was the reason for my unread blog. Sure my content, style, and ability to connect with readers matters too, but you cannot get better at those unless you are writing the blog post week after week.

So that is why I have been able to blog more already in 2015 that other years past. I realized at of things that matter most its my consistency that will help me achieve my goals.

Why you should learn ASP.NET

Developers get caught up in the which language is best to learn or the my language is better than yours mentality. The truth is there in no particular language that trumps them all. Odds are that over your career as a software developer you will have to learn many different programing languages and frameworks. For that reason I am going to explain why ASP.NET is the worlds best framework; not really but I do want to explain why you may want to learn ASP.NET.

Mature Framework

Have you ever spent weeks learning a new framework only to find out later that support has been dropped for that framework? Maybe the framework your working with hasn’t lost support, but have noticed releases are slowing down. Developers in ASP.NET don’t have that worry. ASP.NET is not a new framework in fact it was first released in 2002 by Microsoft. ASP.NET was released as an upgrade from the Active Server Pages (ASP). Since ASP.NET has been around since 2002 means it is a mature framework with many features and community support.

It’s developed by Microsoft

Did I mention that ASP.NET is a Microsoft product? Because Microsoft backs the ASP.NET framework developers can choose any .NET language that want to work with C# or VB.NET. If you have a development team working a Webform application and half the team wants to code in VB.NET while the other half want to use C# that’s okay because ASP.NET was built on a Common Language Runtime (CLR). Microsoft also bring with it a large set of tools that integrate well with ASP.NET, for example:

SQL Server – Database

Azure – Cloud Services Platform

Visual Studio – Integrated Development Environment

In 2012 I switched from developing in VB.NET to PHP, from a syntax perspective I didn’t really have any problems. PHP was a language I had experience with and found it easy to pick up, but the real issue from me was the IDE. I was so used to Visual Studio that is was hard to use a plain text editor like Sublime Text or Text Wrangler, but I did settle on using Netbeans. Netbeans is a great product, however I had spent over 2 years using Visual Studio and had a hard time adjusting. The same situation arose when I switched from SQL Server to MySQL, Microsoft just has these other tools beat.

Market Demand

I know Microsoft is seen as some evil corporation in developer circles but Microsoft knows how to write software. For that reason the ASP.NET framework is a top chose in government and financial institutions. These institutions want to their application running in a secure environment which happens to be a Windows environment. Careers in Government, Finance, Energy, and Medical industries make up a large part of the US economy. The good news is these industries are hiring and one of skills they are searching for is experience in ASP.NET. In simple terms, there are a lot more .NET openings then there are candidates to fill those positions. Take a look at the job sites:

Craiglist Nashville – In Nashville I have some contacts who have told me there is a market shortage of .NET developers. Nashville has a heavy presence in both the Medical and Financial Industry.

ASP.NET Pays the Bills

Okay here is what you really wanted to know. How much do ASP.NET developers make? In the 2013 salary survey conducted by Visual Studio the Magazine, the average ASP.NET Developer’s salary is $94,784. The average developer in this study typically had a 4 year degree and 12.5 years experience. Salaries for ASP.NET developers have steady risen over the past few years and have fared well during the recession. The average household brings in around $50K, which means the average ASP.NET almost makes twice as much as the average household. So no worries about making ends meat in ASP.NET.

ASP.NET is a mature framework with a very large community and a ton of support from Microsoft. Developers have a wide range of tools to leverage that are backed by the Microsoft brand. Career wise ASP.NET developer are in high demand with an average salary around $94,784. My career has mostly centered around ASP.NET so I might be biased but overall I would recommend it as a option for a wide range of products. What are your thoughts on the ASP.NET framework?