API Best Practices: Analytics

How to measure the success of your APIs and API program

ArvindSai

Oct 19, 2016

Previously, we discussed the key features of developer portals and considerations for the organizations who use them. Here, we'll cover how analytics are key for the success of different stakeholders in an API program.

As an API provider, you need to measure, analyze, and act on metrics associated with your APIs and your API program. Most API programs typically involve four types of users with unique needs when it comes to analyzing API metrics.

API producers

API developers care about building APIs using best practices based on learnings derived from other API developers who are doing similar things (such as applying specific types of policies to their API proxies). In addition, API developers need visibility into the step-by-step behavior of all the APIs they build in order to diagnose latency problems and improve performance of those APIs.

Operations admins

Operations teams care about maintaining peak performance and availability of their APIs. They want to see the throughput, latency, and errors associated with those APIs. In addition, they expect to get alerted in near real-time to quickly identify and resolve any issues that affect the quality of service of those APIs. These teams also care about protecting their APIs against malicious bots that could compromise their data and services.

Product owners

Product managers are responsible for the success of API programs, and thus need to measure the adoption and usage of the published APIs across various dimensions such as products, developers, apps, channels, and locations. Product managers also want to measure the business impact and financial value of those APIs by capturing the transaction or business metrics related to them.

API consumers

App developers want to understand the volume of API traffic and the quality of service (success rate, response times, and response codes, for example) for the APIs they build their apps against. App developers also need to track business metrics (such as money exchanged with the API producer), based on the API product pricing plans.

Analytics solves different problems for each of the user types discussed above and leverages data related to APIs, app developers, applications, and end users.

How API developers optimize APIs

As an API developer, you apply a set of policies to your APIs to ensure seamless and robust app functionality, while protecting your back-end systems. You must ensure that once implemented, your APIs are functioning as expected and performing with minimal latencies. This is enabled by visibility into the step-by-step flow with timing information for each API request as it flows through the API proxy.

Here’s an example of a real-time trace capability that helps API developers diagnose their APIs:

Implement the wrong policy, and your API won't be used by app developers. For example, putting an OAuth policy in a product catalog API will force end-users to log in to the mobile app before getting generic information about the company’s products. This adds friction to that API’s adoption. So, by anonymously analyzing APIs across a wide population of customers, the analytics platform can provide insights into best practices on the most common policies implemented across a cross-section of APIs.

How API operations admins monitor APIs and SLAs

Once deployed, APIs become the conduit—and potentially the gating factor—for all user interactions that depend on information exchanged via those APIs. Therefore, your operations teams need the ability to monitor various traffic metrics in near-real time to ensure the desired operation of those APIs.

In addition to keeping track of total traffic volume and throughput for each API, the following additional metrics serve as first-level indicators for the overall health of the published APIs:

Response times for both the API proxy as well as the back-end systems at multiple call distribution levels (median, TP95, and TP99, for example)

Availability measurements based on error rates at each of the various tiers (client tier, API proxy, and the back-end systems)

Cache performance for measuring response times and hit rates for each API enabled with local cache

The diagram below shows the benefit of using a caching policy as part of the API where over 90% of the API calls were addressed from that cache. This resulted in a net improvement of over 3.5x in response time.

Another concern is identifying and blocking malicious users (typically automated bots) from hitting APIs to either steal valuable information or consume resources. Analyzing incoming traffic for patterns associated with API call frequency, location, and sequences can give operations teams the power to optimize operation of their APIs for all their consumers.

How product owners measure an API program’s success

To measure the success of any API program, product managers must be able to analyze the following types of metrics and reports:

API traffic trends broken down by products, app developers, and apps

Trends in signups of new app developers and apps registered for each product

Revenue or business value delivered for each published API

Revenue generated from app developers for subscribing to published APIs

Most prolific or highest-value developers

Developers who consistently exceeding their quotas

Developers who use APIs for free and are candidates for paid offerings

How API consumers see their apps’ API usage

App developers who subscribe to API products through an organization’s developer portal expect visibility into their API usage as well as the quality of service delivered for each of those APIs. Some of the metrics that app developers care about include:

Traffic volume, response times, and errors for each of the APIs called over time

Overall availability for each of the APIs for valid calls that don’t contain client-side errors

The diagram below shows an example of the total availability of the published APIs as seen by consumers, with a breakdown of each of the tiers (API proxy tier, gateway, and back-end systems) and their contribution toward the APIs’ availability.

If the app developer has subscribed to specific pricing plans for using those APIs, then it’s necessary to provide some of the following reports for those developers as part of the developer portal:

Revenue shared (if applicable) by the API publisher for calls made by the API subscriber’s apps

Organizations use API management platforms to provide various types of users fine-grained visibility into API usage and performance. As enterprises adopt modern software practices like microservices, multi-cloud, and platform-as-a-service, gaining deep visibility into how their APIs perform and how developers use them is critical to success.