“In the midst of chaos, there is also opportunity” – Sun Tzu, The Art of War

A typical eretailer gets nearly a quarter of their yearly revenue between Thanksgiving and Christmas. For many online stores, the two busiest days of the year are Black Friday (the day after Thanksgiving) and the following Cyber Monday, which sees an overwhelming number of ecommerce sales. While those days account for a huge spike in sales, they also represent huge spikes in traffic and the associated stress and uncertainty about site performance and uptime.

Shoppers demand high performance and a flawless experience from your site. If your page load time is reduced by just one second due to an influx of traffic, you can expect to see conversions reduced by 7%. And at peak traffic times, more than 75% of online consumers will leave for a competitor’s site rather than suffer delays (PDF).

Google search results are influenced by site performance, so you can create a virtuous circle: faster performance brings you more customers, through improved search results, and you get more conversions, as shoppers have a better experience. Use the tips below to help improve your website’s performance – not only at peak times, but every day of the year.

Getting Started: Know Where You Stand Against Your Competition

Performance improvement is potentially unending; how do you know what’s bad, good enough, or excellent? Competitive data helps you know where you stand.

Compare the page size and load time for key customer and competitive pages; consider including:

Home page response time for top competitors

Product search listing response time

Product page response time for similar products to yours

The length of the wait for confirmation after you hit the “Buy” button

You can do a test of key pages for half a dozen competitors, and your own site, in just a few hours. (Be sure to clear your browser cache by using shift-refresh before measuring download times.) Here are some things you can do with the results:

Index total performance. How long does a “soup to nuts” visit, search, and purchase process take on different sites? See where you stand.

Note feature differences. See if competitors add value to the shopping process through extra features, or things your site does that compare strongly.

Create a plan. If you’re well behind competitors in one or more areas of site performance, aim to meet their average response times; if you’re already competitive, work to become #1.

Note: builtwith is a web service that lets you check the Internet-facing web server for your favorite sites, including competing sites. (It does not show back-end architecture.) You can enter the URL you want to check on the builtwith site, or use the builtwith browser plug-in to compare sites as you surf the web. A Chrome extension, Wappalyzer, provides similar information. Using either tool, you’ll see that NGINX is the Internet-facing server software for a very high percentage of high-performance sites.

Increase Capacity With an HTTP Gateway

An HTTP gateway is known technically as a reverse proxy server. By either name, an HTTP gateway receives requests from user devices – clients – on the Web, then dispatches the requests to other servers over a local area network. Once a response is generated, the gateway returns it to the client that made the request.

Many sites have a simple architecture: a web server that runs application code to put together web pages, often with a database server supporting the application. More often than not, the application server is an Apache server.

Apache and many other web applications platforms are, however, limited in their ability to handle significant amounts of traffic. Every new browser session opens six to eight server connections, and Apache devotes significant memory to each connection. The connections stay open while pages are requested, put together, returned, and acknowledged.

If several thousand users or more are online at once, tens of thousands of connections are needed. But three things can happen:

The Apache server can run out of memory and start swapping sessions out to disk, greatly slowing performance. If connection requests continue to mount, swapping can increase, affecting more and more of the site’s users.

The Apache server can run out of connections. To save memory, the server software can limit the number of simultaneous connections, but the effect is similar to running out of memory: the available connections get used up, new connection requests have to wait, and users see slow performance – especially if connection requests back up.

The application server can run out of CPU capacity due to managing connections and running the application, again making users wait and causing requests to back up.

An HTTP gateway reverse proxy server can prevent these problems. The reverse proxy server receives requests from the web and feeds them over a local area network to the Apache server. The Apache server then runs application code and returns the needed file to the reverse proxy server, which acknowledges receipt quickly. The Apache server can run at close to its rated capacity because it doesn’t have to hold thousands of connections open waiting for acknowledgements from faraway clients.

NGINX open source software and NGINX Plus are very widely used for this purpose, with other functions added on as needed (see below). NGINX uses an event-driven architecture that can handle hundreds of thousands of connections concurrently, translating slow internet-side connections to high-speed local transactions.

By using NGINX as a reverse proxy:

An Apache server can be run at full speed, receiving requests and responding over the local area network.

Static files are easily cached on the NGINX server, so the load on the Apache server and, following onto that, the database server, is significantly reduced.

More Apache servers can be added, with the NGINX server handling load balancing among them (see below).

Cache and Microcache Your Site Content

Caching objects has a multitude of beneficial effects. The large majority of the data on a typical web page is made up of static objects – image files such as JPEGs and PNG files, code files such as JavaScript and CSS, and so on. (You can see how this would be the case on a typical page from an e-commerce site.) Caching these files can greatly speed performance. You can also briefly cache dynamically generated files, which is called microcaching.

There are three levels of caching for static files:

Local caching. You can support local caching of static assets through configuration settings and by repeating the same assets on multiple pages. Example configuration settings for NGINX are given below. (You are likely to get some local caching-type benefits from organizational and service provider caching.)

Content delivery network (CDN) caching. A CDN puts content closer to the user and onto dedicated machines that answer requests quickly. However, the impact of HTTP/2 on CDNs may reduce their benefits without careful optimization; see the NGINX HTTP/2 white paper (PDF).

Most analysis of caching’s benefits focuses on faster file retrieval. However, all forms of caching have an additional benefit that is harder to measure, but very important for high-performance websites: offloading traffic from web and application servers, which frees them up to deliver high performance, and prevents problems caused by overloading.

The idea of running a PHP (or similar) server is to serve freshly assembled content to every user. This is a simple and appealing approach, but it bogs down when traffic skyrockets and the application server can’t keep up.

Microcaching, as mentioned above, means caching a dynamically generated page for a brief period of time, perhaps as short as one second. This form of caching converts a dynamically generated object into a new, static object for a brief, but useful, period of time. For a web page that gets tens, or up to hundreds, of requests per second, most of the requests now come out of the cache with minimal effect on page freshness.

Like other PHP-driven frameworks, Magento is a good example of the benefits of microcaching. Magento is a flexible, powerful content management system that’s widely used for e-commerce. Magento is frequently implemented as a web and application server running PHP to assemble pages, calling a database server for content. This assemblage works great up to a point, then can die an ugly death when traffic exceeds the application server’s ability to cope – which can easily happen to an e-commerce site, especially during the holidays.

To prevent overload, put a reverse proxy server, such as an NGINX open source or NGINX Plus server, in front of a Magento application server, then use the NGINX server for microcaching. Putting the NGINX server in front offloads Internet traffic handling and caching from Magento, while cutting peak processing demands on the application server by one or two orders of magnitude. It also opens the door to expansion to multiple application servers, load balancing among them, and monitoring, all of which allow the site to scale without changing the core application logic at all.

Add Multiple Application Servers With Load Balancing

If you have a simple web server architecture, adding load balancing is the quickest step you can take to make it more flexible and resilient. (The NGINX Admin Guide has extensive details on load balancing.)

Load balancing assumes a few things that we should spell out, because each item can be either a gateway for better performance, or a performance bottleneck:

Web and/or application servers. A reverse proxy server can improve performance even if you have just one application server, as it offloads Internet traffic from the application server, can implement static caching, and provides the first building block for a more robust setup. However, the promise of a more robust setup is only fully realized when you use multiple web and/or application servers and load balance between them.

Smart load-balancing strategy. You need a load-balancing strategy that makes sense for your web application. “Next request goes to next server” — also known as round robin — is a good start. But taking steps to send traffic to a less-used server, to keep specific clients on the same server during a session, and to monitor processing as it occurs are all important elements of an effective load-balancing strategy. (Click for more information on the advanced load-balancing and monitoring features in NGINX Plus.)

Resource extensibility and flexibility. A smart load-balancing strategy should allow you to almost arbitrarily route traffic and add resources, whether running on physical servers or in the cloud, to meet spikes and long-term increases in demand.

Use of additional features. Using your reverse proxy server to support additional features such as content caching (as mentioned), SSL termination //link//, HTTP/2 termination //link//, and so on are important steps for maximizing your investment in a more complex site architecture and minimizing the impact of placing an intermediary between your application server and the Internet.

Note: For a deeper description of where a reverse proxy server fits in your application architecture, see our article on Layer 7 Load Balancing.

Limit Incoming Requests per User

A typical web page may have more than 100 assets, including the core HTML file, images, and code files. WIth the typical browser opening half a dozen or so connections per user session, a user moving quickly through your site may generate a small blizzard of requests.

However, both legitimate and illegitimate non-human users may generate many more requests than an actual visitor. During busy periods, you want to take extra care to preserve site capability for regular users.

Spiders for search engines are one example of a non-human, high-resource user; a spider can hammer a site, slowing performance for regular users. Use the HTTP gateway to restrict access, shutting out high-volume (generally, not human) users. Restrictions include the number of connections per IP address, the request rate, and the download speed allowed for a connection. You can also take additional steps to prevent distributed denial of service (DDoS) attacks.

Conclusion

Black Friday, Cyber Monday, new product releases, and other real-world events are an opportunity to examine website architecture for weak points that can lead to slow response times and downtime. But effective performance optimization is a continual process.

Set goals, review your architecture, then search for weak points and vulnerabilities. NGINX open source and NGINX Plus are widely known remedies for site performance problems, opening the door for many different optimization techniques.