Google App Engine Blog

News, notes, tips and tricks from the Google App Engine Team

Tuesday, May 10, 2011

Google App Engine has grown tremendously since it launched in Preview status in 2008. More than 100,000 developers use App Engine every month to deliver apps that dynamically scale with usage without the need to manage hardware or software. App Engine now hosts more than 200,000 active apps that serve over 1.5 billion site views daily. Over the last three years we’ve collected great feedback from our customers and have continued to improve our services. We are now confident that App Engine is almost ready to graduate from Preview status, and we are pleased to announce that graduation will be taking place later this year. Graduation indicates a long term commitment by Google to App Engine which in turn means a long term commitment by App Engine to our customers!

Our customers are really excited about many of the features of GAE4B: SLA, SSL for custom domains, SQL, etc

The per-user, per-app pricing is not appropriate for many customers that are excited about the GAE4B features because they are not always building applications focused on users within their organization.

Because App Engine is currently a Preview product, some businesses are reluctant to heavily invest in the platform.

App Engine graduating from Preview later this year!

Based on this feedback we’ve decided to make some fairly large changes:

We will be leaving Preview to become an official Google product! This is the top priority for our team and we are on schedule to graduate in the second half of this year. This does not mean we’ve stopped development of other much needed features, for example today we are releasing Backends.

Most of the GAE4B features are going to be rolled into App Engine, enabling all paying customers to take advantage of them. This includes: SLA, SSL for custom domains, SQL, Operational and Developer support, and the new business-oriented Terms of Service. The domain console is currently in trusted tester and will remain so for the time being. GAE4B will no longer be offered as a standalone product.

In order to become an official Google product we must restructure our pricing model to obtain sustainable revenue. Based on customer feedback this means focusing on usage-based pricing and placing per-user, per-app pricing on hold until further notice.

For App Engine, leaving Preview will include providing all paid users a 99.95% uptime service level agreement, operational and developer support, billing via invoice, a new Terms of Service agreement geared towards businesses, and a new, easier to understand usage-based pricing structure for App Engine that is in line with the value App Engine provides. It will also reaffirm our deprecation policy whereby we will support deprecated versions of product APIs for 3 years, allowing applications written to prior API specifications to continue to function.

New App Engine Pricing Model

Although the change won’t take place until later this year, we are announcing this today because it will impact all of our customers and we want to ensure they have enough lead time to understand and account for these changes. We encourage our customers to review the details of the new pricing model, but the major changes are:
Usage Types

Free Apps: Similar to the Free Apps of today but with more restrictive quotas

Paid Apps: Similar to Apps which have Billing Enabled today, except they will have a 99.95% SLA and cost $9/app/month in addition to usage fees. Customers will need to opt-in to this.

Premier Accounts: A new offering which will allow a company to not pay for individual Apps but rather use as many as they need. Premier Accounts will be eligible to receive Operational Support for $500/month in addition to usage fees.

Usage Fees

Instances: Due to customer feedback and to better service memory intensive applications, we will be eliminating CPU hours. Instead, our serving infrastructure will charge for the number of Instances running as a new unit called “Instance-hours (IH)” (1 instance running for 1 hour). These instances will be similar to the instances you can see in the Admin Console today with the exception that we will be improving our scheduler to ensure each instance has an appropriate level of utilization. IHs can either be pay-as-you-go ($0.08/IH) or you can commit to a minimum number of IHs over the course of a week and pay less for those reserved IHs ($0.05/IH).

APIs: All APIs which are currently charged as CPU hours will instead be charged per operation. This is intended to make it easier to predict and understand the cost of an application. Our website contains the details of the API pricing.

Datastore Storage: High Replication Datastore (HRD) storage is being reduced in price from $0.45/G/month to $0.24/G/month as of today. HRD has delivered essentially 99.999% uptime since we launched it in January. As of today HRD is the default option for new applications, and we strongly encourage users of the M/S Datastore to migrate to HRD. Master/Slave Datastore storage will increase to $0.24/G/month when App Engine leaves Preview. We are actively working on better tools to make this migration easier.

All of these changes (other than the HRD price change) will take effect later this year. We will be providing our customers samples of their new bills before the new pricing model is launched so that they will be able to see what will be changing.

Thank you!

We are excited to graduate from Preview and we want to let everyone know how much we appreciate the support, commitment, and feedback we have received. We look forward to working with you for years to come!

58 comments:

The thing I dislike about the new pricing model is that I need to pay in order to have scaling capability. Would it be possible to have your free app automatically go to paid status for a month if you break your free quota?

I think many people have seen GAE as a FREE hosting service, rather than a "You can develop your application for free" service. I hope these changes will begin to change this opinion and keep the Engine running for many years.

ps; I'm sure I speak for a huge number of people who can not wait to get SSL for our domains! All very exciting, good work guys.

I don't like the new pricing model at all. I invested so much time to tune my app to consume as less cpu as possible and will face a immoral price increase.

I have a dev app that uses currently about 40cpu hours a day, has ~16 active instances on average and serves ~7.5 requests/second. Right now I am paying ~120$ a month.

With the new pricing model I would have to pay for the instances alone ~921.6$ a month. For api usage about 200$ and the base fee of 9$ thats summerized 1130.6$ or in other words a price change of 942%. Thats immoral and illegal for a good reason in most european countries.

I beg you to offer an option to choose between cpu based pricing and instance based pricing and I am sure I am not the only person having this problem.

This is extremely disappointing news. Usage-based billing has been a big incentive to use App Engine instead of EC2. It's disheartening to see you guys force a flat-fee model on all your customers rather than making it an optional path for enterprise apps.

It's also disappointing to see such a significant change to the pricing model more than 2 full years after billing is introduced. It was not at all clear that the previous pricing model would be subject to such drastic change. Developers who chose App Engine based on a cost/benefit analysis now have to revisit that decision, and figure out if their existing investment in the platform was a waste of time.

I support your comments @roloma. I understand this only applies to GAE4B and not standard GAE, so if we want SLA's and support it should be more expensive, but not SO expensive. I hope SSL comes to regular GAE for a resonable price and Google keeps with the CPU pricing model there.

I'm disappointed as well. The thing I liked most about GAE was "the pay per usage" price model. I took for granted to be "forced" into the AppEngine platform.But with the new pricing model it makes more sense for me to look into e.g. CloudFoundry. I probably pay similar price but have more flexibility in terms of the platform.

I understand for businesses this pricing is not a problem, but I think many developers on GAE have chosen GAE for the "not pay per instance mode".

Well the free API has been exceedingly generous, so I am not surprised by this announcement, but I do think the pricing details page is short of detail.

It's unclear to me how the existing instances (which you can track in the console over 24 views) translate to Instance Hours. Will something be added to the console to allow estimation of this, ahead of the changes? It does state customers will be shown example bills in advance, but we'll have to see how this works.

Also the Free Pricing details page states "1G" for outgoing bandwidth and incoming bandwidth is included. Does that mean unsecured, secured or both? And how are the premium features going to be priced e.g. SSL for your own domains. Presumably only available if paying $9 a month, but how much extra on top.

I imagine Google would have known they would have to make changes like this for a long time - it would have been good for more transparency regarding this, rather than waiting until you've coded your app to use bespoke Google's APIs! Hardly the most transferable platform, but I tolerated that because of the pay per usage model.

Google has lost its major differentiation in its platform as a service offering - it used to be pay for what you use, now it's pay a subscription and what you use...

Hmm, my bill is going to jump from a few bucks a month to $2466.42 ... of course the revenue coming in from my Google gadgets will offset this somewhat, so my total out of pocket will only be about $2466.41

eek! also very worried about this. :( i hope it wasn't the plan all along to make the service super affordable and attractive to lure people in then jack the price once there were enough customers locked in. i hope the billing preview calms our fears and it turns out not to be too big a change to the current billing amounts.

This is very disturbing. I think that Google has been losing a lot of money with AppEngine and this is the only way they could keep it going. The thing is now I don't have much incentive not to go Azure which is more flexible and not that much more expensive.

I doubt it was a deliberate bait and switch, but it's certainly poor developer relations. An app that needs 2GB of datastore space suddenly costs $108/year instead of $4. The platform is suddenly hostile to any app that doesn't fit within the free quotas, but doesn't yet have enough traffic to effectively monetize.

Being able to outgrow the free quotas gradually as your site's traffic grows was absolutely what set App Engine apart. It's disappointing that you guys would so capriciously abandon such a key feature.

If you ever need someone to write an article about what a catastrophic mistake it was to pick Google App Engine for their web app's infrastructure I'd be happy to do the job. I could tell the tale of struggling to develop for the unconventional database and handler models only to be rewarded with mysterious outages, scheduled maintenance during prime time business hours, undeliverable mail service, and now a "whoops, guess your bill's going up 500%" total restructuring of service fees. Luckily, I can always migrate my app to a competing service...oh wait, no I can't.

If the changes are applied the way they are stated above, we'd likely pay an order of magnitude more for the service. At that point, we could host all of our services way more effectively on EC2, and we will very likely switch.

Against the many drawbacks of AppEngine, the pricing model was one of it's biggest features. The new pricing model eliminates that feature.

we chose to go w/ the HRS solution, paying for service is not the issue. our app relies heavily on datastore operations, our number of targeted users would exceed the 50k quota by far. simply unaffordable, API calls used to be unlimited. with this move, you/Google seem to turn away from startups and go enterprise. that's pretty sad to see, as without a correction of the limitations I don't see a common base to continue working with GAE.

"Instead, our serving infrastructure will charge for the number of Instances running"

As companies age, they start looking for ways to make free money without actual work. (Think of the big banks.) Sad to see signs Google is going that way. If this move results in charging even for instances sitting idly (while we don't even have direct control over the # of instances!) that would be a pretty big change from "no evil". My app has light load and is set to multithreaded yet AE keeps spawning new instances for no reason. I refuse to pay for those.

"These instances will be similar to the instances you can see in the Admin Console today with the exception that we will be improving our scheduler to ensure each instance has an appropriate level of utilization."

Make the scheduler calculate costs based on CPU usage and I might stay. If you try to charge me for idle CPU cycles (in whichever instance) I can't see any reason not to just rent a VM instead. That's the point when Google loses any advantage over VMs.

Wow. That's not much of a thank you to all the people who have been beta testing appengine for the past few years. Very unfortunate news. We had just managed to get a new version of our app built out that allows us to charge users for usage. I'm not sure if we'll be able to get enough income to cover these costs in time. We spent something like 3 years growing an audience on appengine and we're going to be billed off of the platform. I'll be looking at any Google offering with a very healthy sense of skepticism from now on. Really surprised by this.

We seriously hope Google will not implement the announced new pricing model.

We thought that GAE is mature enough (1-2 years old) and Google a reliable and trustworthy business partner. Therefore we made a cost/benefit decision for GAE based on the old quotes (especially the API quotes). However, with the drastic reduction of free API calls (50k operations for the data store API and of 1000 stanzas for the XMPP API), the new pricing model is a radical change from the old API quotes and would increase the costs of GAE by 1000-2000%...and thus kill our business case.

If GAE puts the pricing changes into practice, we will have no option, but to change the service. But even more disturbing: now being close to the launch of our app, we will then need another 2-3 months to adapt our app for the new service and loose all the time and money we put into GAE.

We are very concerned and hope that Google will reconsider a model that is at least closer to the old one. We are willing to pay for the service and also price increases are fine, but the announced radical pricing change would, frankly speaking, make us hate Google.

So we become early adopters, provide valuable feedback to better the service, recommend App Engine to colleagues and for what - an evil bait and switch at the eleventh hour?

I say this because many of us have taken the decision to design and implement our apps for App Engine, which is hardly the most portable of platforms, in the knowledge that the cost per usage model would prove to be financially viable.

If my calculations prove correct, my app which costs in the tens of dollars to run will now cost several hundred! How is this sustainable? It's ridiculous.

Sure, if you're a business which can cover these ludicrous cost increases it's fine but many of us are smaller independent developers. A 100-1000% cost increase will absolutely kill the developer community surrounding App Engine and just force us to migrate to other providers.

Way to go Google and congratulations for potentially ruining a great product.

I am a bit confused by all this. Particularly the talk of this requiring us to pay flat amounts for always-on service. Doesn't App Engine spin up and spin down instances on demand? I presume instances don't stay on much longer than they're serving, so there isn't a HUGE efficiency loss. I presume 'part-hours' of instance usage will be added together rather than rounding each part up to 1 full hour. Maybe I am totally wrong on this, it's been a while since I've kept up to date on AppEngine. If it was switching to a EC2 style model where instances cost you even when not in use, then that would be a very fundamental change, and a very disappointing one. AppEngine's big win - the scalability of the DB aside - was very granular CPU usage and cost.

the thing is that spin up times in java are very long, so we are forced to use the always-on option (now 2.1$ a week) and it's going to cost about 25$ a week (!). after so much efforts we put trying to struggle through GAE quirks I must admit this announcement is a very big disappointment for us. not to mention the draw backs that we are waiting forever to get fixed (emails from normal domain, https, next-gen queries, ...).

"Google stop instances when they aren't being used and only start new instances when necessary so the number if instance hours shouldn't be much higher than existing CPU hours.

[...]

I really don't understand why everyone is saying this will cost so much?"

It's very easy to imagine scenarios where the cost will skyrocket in the new system.

Let's say an app gets a few hits every few minutes on average. That's very little load, the CPU is 98% idle but you were only charged for the CPU you're actually using (which is what Google has been saying all along, until all that suddenly changed now.) Total CPU consumption: a few dozen minutes / day.

Now if Google goes rogue and charges for *instances*, those few hits every few minutes will keep one full instance going all day (scenario 1). An instance is kept alive the full 24 hours, maxing out the free quota, even though the app is the same and uses the processor only 1% of the day. No room to grow or scale when occasionally many hits happen to come in at once.

OR, scenario 2, if Google releases the instance after each incoming request every few minutes, what kind of mandatory startup delay are we talking about? 15 seconds? More? Not good either.

I've always thought that what AE is about is that the Google engineers are smart enough to give the other 98% of the CPU to other customers dynamically, without the apps affecting each other.

It also worries me that you guys couldn't think of the scenarios listed here, ones that will upset many apps, and had to ask.

Googlers, think of all the ways the new 24 IH / day limit will be abused.

Up until now developers were encouraged to use CPU cycles sparingly since payment was based on actual resource usage.

Now they can keep a whole instance 100% busy for a full day and still be within the free limit.

No need to optimize at all either. Response within 5ms vs 100 times more is hardly noticeable, and now there will be little reason to care.

"Due to customer feedback and to better service memory intensive applications, we will be eliminating CPU hours." Really, what customer requested THAT? Did someone trick you into offering a free instance for calculations & similar type loads? How's that going to help other AE users?

The biggest problem I can see is that with the single threaded python interpreter you need n instances for n simultaneous urlfetches. This will make any python app that relies on external APIs expensive (several simultaneous instances required all the time) or slow to respond to requests.

We use Appengine and EC2 together for our Appstack. Today at Appengine our biggest App consumes about 1,5 CPU Hours a day (this is in free Quota). We are into paid Quotas every about 7th day, but only with some cents.In the past year we developed heavily on appengine and worked hard to optimise our code for the funny inconveniences that appengine comes with. All this for having the advantage of CPU-Hour payment.The same app uses three to four instances in average. We have got a lot of very small requests so serve.I haven't yet calculated the new quota system for us. But CPU Hour vs. instance hour pricing is a dealbreaker by itself.I don't mind the 9$ - although it obviously is in conflict with what Google always said Appengine is about. And reasonable pricing changes are okay.My impression is, that IAAS gets cheaper and cheaper ... and now Appengine (which we used to love) takes the 2000% hammer.

I really hope, that someone from Google says something like "Hey we were only joking. Don't be evil! Of cause this was only ment to show you what Microsoft would do"

In the meantime we are starting to migrate our last Apps to Amazon. I didn't see this coming and I dont like what I see.

We have to give them credit, the service was cheap and actually led people to think about design and optimize. What I don't like nevertheless is the lock-in and now suddenly the unexplained effect of price changes to current apps.

So Google please post several real-world examples of the effect of your changes or we will seriously consider if our next project will base on GAE.

For near a year now, I invested myself a lot in the GAE specific plateform to create a web application relying on App Engine scalability.

Everything was on the rail and I hoped to launch a commercial version in September.

For months I operated a full scale test wich was very promising: it consumed a lot à instances (about 12 average on 24h) for a proportionnaly very few CPU hours (18h) thanks to heavy optimisations I spent countless nights to perform.

Everything was perfect until... I read this post.

If I'm right, my bill will increase between 1100 and 1700%, definitly ruining my business model.

How stupid I was...

How my colleagues and friends will be right to laught at me (after I have spent a year to break their ears with " GAE the best cloud ever !!! ")...

I'm sad, frustrated and shameful...

And I will take an Azure Pass : at least if Msft become crazy with their pricing, I will always have the oppotunity to move my hosting as Azure app can be hosted elsewhere without heavy modification.

After a year of developing and optimizing for the AE all of a sudden everything is turned up side down!

I don't mind paying for good service but this feels like a restaurant would charge for the time you sat down and not for what you ordered, which used to be the case with AE. All optimization down the drain..

It's hard to grasp how much the increase will add up to in my case but I don't have a good feeling about this. Other alternatives now look much more interesting.

Google, should we have seen this coming? Billing has been around for a long time. You didn't even leave us as much as a hint. When will it change again? Into what?

Please don't get greedy. It will kill my business and in the end maybe yours too.

I was looking forward to the Google I/O 2011 for some awesome App Engine announcements.

The 'Backends' part fooled me for a second. Then I continued reading and thought to myself "Something's not right... I must have misunderstood what I just read." Then I read again and again. Alas. Now I feel anxious and unsure of my app's future in App Engine.

AppEngine team, I know you are brilliant programmers, and many don't care much for the business side of things. But, can we get at least some of your thoughts about PRICING and the change.

For me, as someone who have been evangelizing AppEngine in person and in forums, it's like a slap in the face. I do realize that you must make money from this endeavor, but can we hear you say in this. A post dedicated to just this issue would be appreciated.

I looks like this goes very much against you motto (don't be evil). Even if you miscalculated pricing when you started AppEngine (maybe you set your pricing too low then), we want your say.

I chose the Channel-API of GAE because there was quite a large free quota and it looked promising...now when I have 500K users, I will need to pay 60000$ a month just for the channel API?you call these easy scaling? you must be kidding...

Lucky for my company we are still in an early phase where we can stop using the GAE and move to Amazon

I understand the need to ensure the long-term survival of GAE by ensuring it's profitability. I always felt that the free quotas were generously high, and probably couldn't be sustainable. That said, these latest pricing terms have left me with many doubts and fears. The post about the new pricing seems to be a bit vague, but what I have been able to understand is that GAE's greatest strengths in comparison to its competitors seems to have disappeared. It's very important that GAE is able to scale for very small users who might only consume a few gigabytes of data month to the larger customers. The new pricing doesn't seem to be very accommodating in that regard. For a base price of $9/month, it's possible to get great (non-cloud) hosting elsewhere. The option for dedicated hosts in addition to modified server software solutions means that many users (including myself) will question the worth of investing extra time in GAE when comparable solutions can be found elsewhere. Please, Google, reconsider your userbase. It might seem more profitable to cater towards the already accomplished websites, but every website has small beginnings. In addition, it might be worth considering that many of us have been dedicated beta tests over the course of these past two years, and it would be greatly appreciated if some accommodation could be made for better price scaling for smaller websites.