Amazons Site Outage Raises Scale Questions

Opinion: When Amazon.com crashed for almost 2 hours on Monday, it might have meant little were it not for the fact that a major industry report had predicted it just three days earlier.

Amazon.com has built a reputation for being an aggressive, take-no-prisoners kind of company, but it showed its more cooperative side this week.
Late last week, Internet traffic monitoring firm Keynote Systems issued a report that said many of the largest e-commerce players will face major load-handling challenges for the holidays if they dont make big changes by the fourth quarter.
However, after so many years of holiday shopping seasons, some in the industry scoffed that the majors could be caught so short. To help out, Amazon generously knocked out its own servers for almost 2 hours on Aug. 21.

OK, OK. Amazon probably didnt crash just to help Keynote make a point. But its timing was impeccable nonetheless. Its interesting to note that within a few days this month, three of the industrys most powerful retail forcesWal-Mart, Amazon and Googleall suffered problems that, one way or the other, can be classified as scalability-related.
Wal-Mart started last week, when it reported its first quarterly profit decline in 10 years. Even a magnificently profitable and well-run company is going to have a periodic profit decline. Indeed, from a statistical perspective, its hard to have that not happen, so lets not make too much out of the decline. But it adds support to the argument that there comes a point when Wal-Marts business model cannot grow any more. Can it become a trillion-dollar company? Maybe $10 trillion?
IT systems are the same way. The concept that bigger is more efficient has very definite limits, especially when one gets to Wal-Marts size.
Then we move to Amazon. Like Wal-Mart, Amazon seems open to the idea of selling anything and everything to anyone. The larger the audience and the bigger the universe of products to bring in, stock and ship, the larger the margin of error becomes.
Its already difficult to accurately project how many people will visit your site on a given day. The only way to do that is to look at historical patterns, increase the number by however much your budget allows and hope for the best. Many e-commerce players have gotten quite good at it, often accurately projecting traffic within a few percentage points.
But as the size of the audience and the number of products offered soar, that "few percent" error can get to be a huge figure. Suddenly, buying enough bandwidth to handle traffic within XX percent of the historical figure can become extremely expensive. As Amazon grows, its going to find defending against periodic traffic fluctuations more and more difficult.
The only e-commerce giant to stumble this week was Google. To be fair, from an IT perspective, it didnt stumble at all. It got hit with a series of bad-news developments relating to retailer resistance to Google Checkout, a pair of traffic monitoring services reporting a drop in Googles search market share, a drop in Googles stock price andbelieve it or notpressure from Brazilian authorities to surrender more information that involves child pornography and neo-Nazis. (Note to Google PR: You typically want to never have your brand mentioned in the same sentence with child pornography and neo-Nazis.)
But the Google question has some similarities to the Wal-Mart question. Is the drop simply seasonal, as some have suggested, or is it another scalability red flag? Googles servers may be able to handle some ludicrously large number of petabytes of storage, but can it be consistently better than free alternatives?
The free nature of Web searching truly puts all of the pressure on pure quality. As Google has grown, it has gotten quite creative in expanding into new businesses. But how creative has it been cleverly making its own searches more accurate and comprehensive? When was the last announcement of something that simply improved its searches?
Have the Goliath roles been reversed? Will some smaller and more creative pureplay search engine move in and disrupt the core of Googles business? Before it gets too much larger, it might serve Google well if a lot of its managers reread some of the key business disruption books adorning Amazons virtual shelves. That is, of course, if the site is up.
Evan Schuman is retail editor for Ziff Davis Internets Enterprise Edit group. He has tracked high-tech issues since 1987, has been opinionated long before that and doesnt plan to stop anytime soon. He can be reached at Evan_Schuman@ziffdavis.com.Check out eWEEK.coms for the latest news, views and analysis on technologys impact on retail.

Evan Schuman is the editor of CIOInsight.com's Retail industry center. He has covered retail technology issues since 1988 for Ziff-Davis, CMP Media, IDG, Penton, Lebhar-Friedman, VNU, BusinessWeek, Business 2.0 and United Press International, among others. He can be reached by e-mail at Evan.Schuman@ziffdavisenterprise.com.