講師

Ilkay Altintas

Chief Data Science Officer

Amarnath Gupta

Director, Advanced Query Processing Lab

字幕

Cloud Computing: An Important Big Data Enabler. In our first lecture in this course, we mentioned the cloud as one of the two influences of the launch of the big data era. We called it on-demand computing, and we said that it enables us to compute any time any anywhere. Simply, whenever we demand it. In this video, we will see how we deploy the cloud to our benefit in our big data applications. The main idea behind cloud computing is to transform computing infrastructure into a commodity. So application developers can focus on solving application-specific challenges instead of trying to build infrastructure to run on. So how does this happen? We can simply define a cloud computing service, as a rental service for computing. You rent what you want, and return upon usage. Think about this, you wouldn't buy, or even build, a truck every time you have to move a piece of furniture. You would simply rent. Why build a computing cluster when you can rent? Especially if you are not using it all the time. Similarly, you can rent a car or a bike when you are on vacation. So you can bike anytime, anywhere. Let's dig into this question. What factors do you consider when you're developing a solution for your yourself or your client? Should you build a hardware and software resources yourself? Or should you rent these resources from the cloud? Let's look at in-house hardware and software resource building first. If you choose to develop in-house capabilities, you have to hire people and buy hardware that suits your requirements. These includes, but not limited to, buying networking hardware, storage disks, upgrading hardware when it becomes obsolete, and so on. Not to forget, the real estate cost of keeping the hardware. How do you estimate the size of your hardware needs? Do you make a five year estimate, or ten year? In today's fast changing world, it is becoming harder to estimate future demands. Getting the software that fits your needs is equally challenging. Most software installations require a lot of tweaking and manual intervention that require a lot of skills. You will need your engineers to do this. Compatibility issues bring problems that are hard to foresee. Most software is updated on a daily basis. You must ensure you're updated. This insures you avoid security risks and get the best. Over all, building your own data center or computing power house can be expensive. And it can be time consuming, maintaining it is a task by itself. This requires high initial capital investments and efficient operation of several departments in your business, which you might not have if you are a startup company. Most people forget to include the cost of disposing old hardware. Now lets see what the cloud can do for us. Cloud's benefits are similar to what you would get from a rental car company. You pay for what you use, which means a low capital investment. You don't need to go to the dealership, do a negotiation, get a bank loan, get insurance. That means quick implementation of your projects. Just like you don't need to buy a car if you only need a car for a limited use, deploying your application on a server that is geographically closer to your client can give you fast service and happy customers. For startup and small business, it can be challenging to do so. Cloud lets you do this with a click. You can be sitting at a coffeeshop or your home and starting your Internet business, without a huge capital investment, thanks to the cloud. And you don't need to have a five or ten year resource estimation plan. Adapt to your requirements faster if your business is growing faster than you thought. Cloud lets you forget about the resource management problems and lets you focus on your business's products or domain expertise with minimal cost. Just as you can rent a truck or a convertible at a rental car company, you can build your own custom machine on cloud. With a custom machine, we mean a commodity cluster made out of the right type of computing nodes for your application. You pick not only a CPU or a GPU, but pick from a whole menu of compute, memory and storage choices. It's a buffet on the cloud. Design machines to suit your application requirements, data size and analytics. Get what you want, and pay for what you use. Compare this with buying and maintaining all combinations of hardware that you possibly would use. That is so costly and not possible at all times. Thanks to all these advantages, there are many cloud server providers today. And the numbers are growing. Here we list some of the players in the cloud computing market. Take a moment to look at them. You will probably recognize some big names and some others you have not even heard of before. As a summary, cloud does the heavy lifting, so your team can extract value from data with getting bogged down in the infrastructure details. Cloud provides convenient and viable solutions for scaling your prototype to a full fledged application. You can leverage the experts to handle security, robustness, and let them handle the technical issues. Your team can work on utilizing your strengths to solve your domain specific problem.