Преподаватели

Google Cloud Training

Текст видео

Like any computer, and the cloud computer is a computer, you need compute processing units, CPUs. And the CPUs on the cloud are provided by a compute engine of virtual machines. And you need a place to store your input data, to store your output data, to store your intermediate data, things that are persistent, things that are temporary. And that storage on GCP is provided by cloud storage. And connecting the two, connecting the virtual machines or compute engine instances with these storage units or cloud storage is a private network. You think not to directly interact with this network but it's there and it is what allows you to have a global scale data and compute infrastructure. For the most part, when you work with GCP, you will not be working at the level at which we're going to be talking about in this chapter. So we're not going to be working at the level of individual virtual machines. No, you're not going to be spinning up VMs in order to do a job, we'll be working with things that are much higher level than that. But even if you need to work at this low level, in terms of infrastructure, the design goals of GCP remain the same. And the design goal is for working with cloud infrastructure to be as no-ops as possible. And no-ops here essentially means that we want to minimize a system administration overhead. And because we're talking about computing and storage, we want to basically also mention that we want this to be as flexible as possible. In such a way that you can change the type of virtual machine that you're running without paying any penalties, for example. So you're not reserving instances for long periods of time. In fact, we want to make it as flexible and easy for you to get your compute jobs done as possible. So when we talk about compute engine, the idea is in terms of flexibility. You can go ahead and get a compute engine that is say, N1 standard four and that's a very specific configuration of machines that you can have. But we would like you to stop thinking about it in terms of this very specific physical infrastructure, and instead start thinking in terms of more abstract concepts. So for example you might say, I want a virtual machine that has 8 CPUs and 30 gig of RAM. And it's the job of the Google cloud infrastructure to go ahead and fetch you a virtual machine that has 8 CPUs and 30 gigs of RAM. Regardless of the type of machine that you get, you will always get load balancing, advanced networking, monitoring, clustering, container support, etc. So there is no second class machine here. Every machine that you have, has all of these capabilities built in. At the same time we want to give you flexible compute. And you lose flexibility whenever you say that I have to go ahead and get a machine, and have to keep it running for months on end. Because face it, if you're running a machine for months on end, you have essentially bought the machine. And what we want is for you to work with machines on the order of minutes. However, there are always going to be workloads where you might find yourself having a machine and using it fully tilt for long periods of time. Rather than ask you to try to determine which of your workloads you are going to be running for long periods of time, GCP gives you a discount after the fact. So at the end of the month, if it turns out that you've used a machine for 60% of the month, you will automatically get a 15% discount. And this is something that happens on your bill after we've found that you've used it. So what this means is that you always get to retain your agility. So, for example, if you have a workload that's currently running on 8 CPUs and you decide that you need to increase it to 12 CPUs, for a few hours, well, go ahead and do that, right? You can move your workload to a different machine when you need to and move it back to a smaller machine when the peak loads go away. In addition to this whole idea of being able to change the machine type of stopped instances, you have another concept that's very, very, very useful, especially when it comes to jobs like Hadoop jobs. And this is the idea for preemptible virtual machine. The reason that GCP, one of the reasons that GCP can say, well, if you want an 8 CPU machine, 30 gig of RAM, we'll find it for you and we'll give to you, is because some of those machines that are currently being used are what are called preemptible. Whoever is using those machines has agreed that in return for a hefty 80% discount on the machine charge, they agree to give it up if someone comes along and is willing to pay full price for those machines. So that's what a preemptible machine is. So a preemptible machine is a machine that you get a great discount on in return for your flexibility, in letting go of of it when you don't need it. But why would you do that? Why would have a machine that you're willing to give up? Well, if you're running a workload like Hadoop, which is fault-tolerant, if a machine goes away, well, whatever that machine was doing, those jobs get basically distributed among the other workers. Then preemptible machines are a great strategy to reduce your overall cost. So you might say, for example, that you're creating a data proc cluster, a data proc is a Hadoop cluster on GCP, but we look at it in the next chapter. So you may say I'm going to create a data proc cluster, and in my data proc cluster, I'm going to have 10 standard VMs and 30 preemptible VMs. So now your job is going to get done four times faster. And at the same time, those extra 30 machines that you're using, are actually at 80% of the normal cost. So not only are you getting it done faster, you're also getting it done cheaper. So preemptible machines are good thing to incorporate into your strategy. With the idea that even if you don't get a preemptible machines, those standard machines are enough for you to get the job done in a timely manner. So you don't want to bank on a preemptible machine, being available when you need it, but if it is available and you happen to get it, you automatically gotten a huge discount on the total cost of your job.