As described in that blog post, enabling billing scenarios for the service providers is a key investment area for this release. Service providers cannot successfully monetize their services in the absence of a system that tracks and reports on tenant resource utilization. These services are offered on a subscription basis and therefore it is critical that the resource utilization is reported at the subscription granularity to assist in billing scenarios.

Overview of the Usage Metering System

The usage system is located alongside the Service Management API in the Windows Azure Pack (WAP) stack, enabling it to access tenant utilization data for all the services provided in the WAP stack and provide an REST API which is leveraged to integrate with the billing system of the provider, as illustrated in the figure below.

Service providers have invested a lot in their own billing system and it was critical that the 2012 R2 release be able to integrate with the existing systems in place. Therefore, we targeted our investments to ensure that 2012 R2 integrates easily with various billing providers and ITFM (IT Financial Management) products that are in the market.

It is important to note that there is no billing system being shipped in 2012 R2 release. Service providers have to create the billing integration module (also referred as “billing adaptor”) to provide data to the billing system they are using.

Now, lets go a little deeper to look at the building blocks of the usage metering system and how its architected.

The Usage Metering System has four main components. Three of these components, the Data Generator, The Data Collector and the Usage Database are internal to the system and the fourth component the Usage API is an external facing API that the billing adapter will interface with to extract the tenant resource utilization data.

Data Generator

The Data Generator tier represents the services (resource providers) registered as part of the system. They collect information specific to a subscription and expose it to the Usage Collector. The Usage Collector expects information to be made available following a specific data contract. This contract is the same across all the providers. All providers in the system adhere to this contract to provide information. IaaS metrics in Windows Azure Pack are provided by VM Clouds resource provider.

Data Collector

The Data Collector is an internal component that periodically collects usage information from all the registered Data Generators and stores it in the Usage Database.

Usage Database

The Usage Database is a transient store, which stores all the data from the various Data Generators for a period of 30-40 days. The expectation is that during this time, the billing system would have extracted the data from this database for billing purposes.

Usage API

This is a RESTful API and is the only way to extract the data from the Usage Database. Since Service Providers typically have a billing system which allows them to generate monthly bills to their subscribers. Customers can easily create an integration to their billing system by extracting data from the Usage Database through the Usage API. The component that customers develop to integrate with their billing system is called a “Billing Adapter”, which serves as a bridge between the Usage Metering system and the customer billing system.

In the figure below, in the red circles, you can see the VM Clouds resource provider, alongside other resource providers such as Service Bus, generating the IaaS resource utilization data, which is collected and stored in the Usage Database and made available through the Usage API.

The Usage API can be leveraged to create the billing adaptor and interface with the billing system within the provider data center. In the figure below, you can see that the role of the “billing adaptor” serving to integrate the Usage Metering System and the billing provider within the provider datacenter.

Interacting with the Usage System

This section explains the ways an external system can interact with the Usage Metering System. Two different types of information are available through the Usage API:

Tenant resource utilization for all subscriptions

Plan, add-on, subscription, and account information

The information is presented via two channels:

Usage API that queries all the historical data

Real time CRUD events via the Event Notification System.

The billing adaptor uses both these channels to be able to effectively create a billing reports while being able to respond in real time as plans, subscriptions and accounts get created and managed in the environment.

Usage API (Exposed on the Usage Endpoint)

Usage Data

The Usage endpoint exposes an API to return tenant resource utilization data pertaining to every subscription across services. The caller (“Billing Adapter”) needs to provide the “startid”. This parameter informs the Usage Metering System to return usage data, starting from that ID. The Billing Adapter advances the “startid” based on the number of records returned for the subsequent call.

Configuration

The administrator needs to ensure that the Usage Metering Service is configured correctly to authenticate the Billing Adaptor. That can be done by ensuring that the service is capable of accepting the correct credentials that will be used to authenticate. The steps below describe how to ensure that the credentials are set properly. Note: During the installation process, the password used is a random sequence and hence this step is necessary to establish connectivity.

On the WAP deployment launch the Management Service PowerShell Module on the Admin API server.

StartID is the record id of the first record you want to fetch and BatchSize is the maximum number of records you want to fetch.

Execute the API call and read Usage Data

Data Contracts can be used to de-serialize the response returned (as in Sample below)

Usage Data Model

The Usage Data Model is shown in the figure below and can be used to associate the data returned by the Usage API.

Event Notification System

The Service Management API keeps track of events within the Usage Metering System and sends notifications to any registered subscriber (e.g. a Billing Adaptor). Examples of the events are plan, addon, subscription creation\updates and account creation.

Notifications are sent as a Post call to an endpoint registered with the Usage Metering System. The Management Service PowerShell Module should be used to define the required notification end point. Note that the notificationEndPoint must end with a trailing slash.

The Billing Adaptor can be set up to handle the event in a blocking or a non-blocking manner. The SubscriberType BillingService & MandatoryService are both blocking. The only nonblocking option is OptionalService. If the Billing Adaptor is set up to be blocking, a plan creation event in the service management API should trigger a corresponding plan to be created in the billing system. If this operation is not successful, the plan creation at the service management API will fail. This enables consistency between the platform and the billing system.

Notification Data Contracts

Notifications sent to the billing adapter adhere to type - NotificationEvent<T> type. T could be replaced by the below objects.

Plan

PlanAddOn

AdminSubscription

ResourceProviderReference

PlanAddOnReference

PlanAddOnReference

When you download the WAP (Windows Azure Pack) the data contracts can be found under:

· \SampleBillingAdapter\DataContracts\*

Following are the two important properties of NotificationEvent

1. NotificationEvent Method could have following values:

1. Post to create a new account/subscription/addon/plan

2. Delete to delete an account/subscription/addon/plan

3. Post an update to a plan

2. NotificationEvent Entity sends an event when any of the above objects are created\updated\deleted.

Pricing APIs

The Pricing API is designed for billing system in the Service Provider data center to specify prices for Plans and Add-ons to flow into the 2012 R2 system. The billing adaptor can choose provide prices for each Plan, or Plan add-on in real-time. As part of implementing the notification subscriber, we have specifications for the below APIs that the billing service can implement to enable pricing data to flow back into the system. The implementation of these APIs is optional. If the below APIs are enabled the price values for the plans and add-ons will be visible in the WAP Tenant site at the time of addition of the Plan\Add-On.

This API is expected to return a string with pricing information. The 2012 R2 system will display this information alongside plans for the subscriber, but these are textual and not typed.

Detailed Description of the Sample Adapter Project Files

This section explains the content of the sample billing adapter (SampleBillingAdapter.sln). At a high level the billing adapter consists of the below parts:

1. SampleBillingAdapter.cs provides an example of the different calls to the Usage REST API

2. The set of Data Contracts that can be used to deserialize the API responses

SampleBillingAdapter.cs

This is the entry point for the application. The file contains the below:

1. Instantiation of a UserServiceHttpClient with the required configuration data.

2. This UsageServiceHttpClient is then used to query the usage service. There are seven types of calls that can be made for the Billing data. This data is deserialized into instances of the data contracts that are included in the DataContracts directory

3. The data is then printed to the console.

Example:

using Microsoft.WindowsAzurePack.Usage.DataContracts; using System; using System.Collections.Generic; using System.Globalization; using System.Linq; using System.Net.Http; using System.Net.Http.Headers; using System.Text; using System.Threading.Tasks;

Data Contracts

The DataContracts directory contains all the required Data Contracts to interact effectively with the Usage API.

VM Data Gathered from the Usage API

VMProvider

Measure

Unit

Description

MemoryAllocated-Min

MB

Lowest allocated memory size for a VM within an hour timespan

MemoryAllocated-Max

MB

Highest allocated memory size for a VM within an hour timespan

MemoryConsumed-Min

MB

Lowest consumed memory size for a VM within an hour timespan

MemoryConsumed-Max

MB

Highest consumed memory size for a VM within an hour timespan

MemoryConsumed-Median

MB

Median average consumed memory size for a VM within an hour timespan

CPUAllocationCount-Min

Each

Lowest number of CPU core allocated for a VM within an hour timespan

CPUAllocationCount-Max

Each

Highest number of CPU core allocated for a VM within an hour timespan

CPUPercentUtilization-Median

%

Median average in percentage of CPU consumption for a VM within an hour timespan

CrossDiskIOPerSecond-Min

MB

Lowest input/output per second (IOPS) across all attached disk for a VM within an hour timespan

CrossDiskIOPerSecond-Max

MB

Highest input/output per second (IOPS) across all attached disk for a VM within an hour timespan

CrossDiskIOPerSecond-Median

MB

Median average input/output per second (IOPS) across all attached disk for a VM within an hour timespan

CrossDiskSizeAllocated-Min

MB

Lowest allocated disk size across all attached disk for a VM within an hour timespan

CrossDiskSizeAllocated-Max

MB

Highest allocated disk size across all attached disk for a VM within an hour timespan

PerNICKBSentPerSecond-Min

MB

Lowest bytes sent per second on a network adapter attached to a VM within an hour timespan

PerNICKBSentPerSecond-Max

MB

Highest bytes sent per second on a network adapter attached to a VM within an hour timespan

PerNICKBSentPerSecond-Median

MB

Median average bytes sent per second on a network adapter attached to a VM within an hour timespan

PerNICKBSentPerSecond-Average

MB

Straight average bytes sent per second on a network adapter attached to a VM within an hour timespan

PerNICKBReceivedPerSecond-Min

MB

Lowest bytes received per second on a network adapter attached to a VM within an hour timespan

PerNICKBReceivedPerSecond-Max

MB

Highest bytes received per second on a network adapter attached to a VM within an hour timespan

PerNICKBReceivedPerSecond-Median

MB

Median average bytes received per second on a network adapter attached to a VM within an hour timespan

PerNICKBReceivedPerSecond-Average

MB

Straight average bytes received per second on a network adapter attached to a VM within an hour timespan

As you can see, this is a powerful API that allows bi-directional data flow. The usage data from the 2012 R2 stack to the billing adaptor and the pricing data (business logic decides the prices) and that data flows from the billing system into the 2012 R2 stack.

In subsequent blogs, we will provide more details as we hear more from our customers.