Required. Deployed on the PAS subnet, one job per Azure availability set.

Storage Accounts

PCF on Azure requires 5 standard storage accounts: BOSH, Ops Manager, and three PAS storage accounts. Each account comes with a set amount of disk. Reference architecture recommends using 5 storage accounts because Azure Storage Accounts have an IOPs limit of approximately 20k per account, which generally relates to a BOSH JOB/VM limit of approximately 20 VMs each.

Service Tiles

Deployed on the Services subnet. Each service tile is deployed to an availability set.

Alternative Network Layouts for Azure

This section describes the possible network layouts for PCF deployments as covered by the reference architecture of PCF on Azure.

At a high level, there are currently two possible ways of deploying PCF as described by the reference architecture:

Single resource group, or

Multiple resource groups.

The first scenario is outlined in Installing PCF on Azure. It models a single PCF deployment in a single Azure Resource Group.

Network Layout

This diagram illustrates the network topology of the base reference architecture for PCF on Azure. In this deployment, you expose only a minimal number of public IP addresses and deploy only one resource group.

Network Objects

The following table lists the network objects in PCF on Azure reference architecture.

Network Object

Notes

Estimated Number

External Public IP addresses

Use

global IP address for apps and system access

Ops Manager or optional jumpbox.

Optionally, you can use a public IP address for the ssh-proxy and tcp-router load balancers.

1-4

Virtual Network

One per deployment. Azure virtual network objects allow multiple subnets with multiple CIDRs, so a typical deployment of PCF will likely only ever require one Azure Virtual Network object.

1

Subnets

Separate subnets for

management (Ops Manager, BOSH Director, Jumpbox),

PAS,

and services.

Using separate subnets allows you to configure different firewall rules due to your needs.

4

Routes

Routes are typically created by Azure dynamically when subnets are created, but you may need to create additional routes to force outbound communication to dedicated SNAT nodes. These objects are required to deploy PCF without public IP addresses.

3+

Firewall Rules

Azure firewall rules are collected into a Network Security Group (NSG) and bound to a Virtual Network object and can be created to use IP ranges, subnets, or instance tags to match for source and destination fields in a rule. One NSG can be used for all firewall rules.

12

Load Balancers

Used to handle requests to Gorouters and infrastructure components. Azure uses 1 or more load balancers. The API and Apps load balancer is required. The TCP Router load balancer used for TCP routing feature and the SSH load balancer that allows SSH access to Diego apps are both optional. In addition, you can use a MySQL load balancer to provide high availability to MySQL. This is also optional.

1-4

Jumpbox

Optional. Provides a way of accessing different network components. For example, you can configure it with your own permissions and then set it up to access to Pivotal Network to download tiles. Using a jumpbox is particularly useful in IaaSes where Ops Manager does not have a public IP address. In these cases, you can SSH into Ops Manager or any other component through the jumpbox.

1

Multiple Resource Group Deployment

This diagram illustrates the case where you want to use additional resource groups in your PCF deployment on Azure.

Shared network resources may already exist in an Azure subscription. In this type of deployment, using multiple resource groups allows you to reuse existing resources instead of provisioning new ones.

To use multiple resource groups, you need to provide the BOSH Service Principal with access to the existing network resources.

Custom roles can be assigned to service principals with az role assignment create --role [ROLE] --assignee [SERVICE_PRINCIPAL_ID].

Multiple Resource Group and Multiple Load Balancer Deployment

This diagram illustrates the case where you want to deploy multiple load balancers with your multiple resource group deployment.

The key design points of this deployment are the following:

Two Azure load balancer (ALBs) to the Gorouter exist. The first ALB is for API access, which utilizes a private IP address. The system domain should resolve to this ALB. The second ALB is for application access, which can either use a public or private IP address. The apps domain should resolve to this ALB.

Azure does not allow the Gorouters to be members of more than one ALB member pool for the same ports, for example 80 and 443. This restriction requires an additional reverse proxy to front the Gorouters to allow hem to expose traffic on these ports for the system domain.

Log Analytics Integration

Azure Log Analytics is a service that helps you collect and analyze data generated by resources in your cloud and on-premises environments. The Microsoft Azure Log Analytics Nozzle for PCF receives logs and metrics from the Loggregator Firehose, filters and resolves the events, and then sends them to Log Analytics, where they appear on unified dashboards and can be correlated with and alerted on other Azure resources.

DNS Delegation

Azure DNS supports DNS delegation, allowing for sub-level domains to be hosted within Azure. This functionality is fully supported within PCF.

Pivotal recommends that use a sub-zone for your PCF deployment. For example, if your company’s domain is example.com, your PCF zone in Azure DNS would be pcf.example.com.

As Azure DNS does not support recursion, in order to properly configure Azure DNS, create an NS record with your registrar which points to the four name servers supplied by your Azure DNS Zone configuration. Once your NS records have been created, you can then create the required wildcard A records for the PCF application and system domains, as well as any other records desired for your PCF deployments.

You do not need to make any configuration changes in PCF to support Azure DNS.

Azure Blob Storage

Due to limitations in PCF, it is not possible to support a high-availability deployment of the backing NFS store needed for droplets, buildpacks, and other resources. Azure Blob Storage provides fully-redundant holt, cold, or archival storage in either local, regional, or global offerings. It is recommended to use Azure Blob Storage as the external File Storage to provide unlimited scaling and redundancy for high-availability deployments of PCF.

To enable Azure Blob Storage, do the following:

Create a Storage Account with the level of redundancy you require. The storage account does not need to be in the same Gesource Group as PCF.

Create Containers for the buildpacks, droplets, packages, and resources required by PCF.

Open Ops Manager.

Select Pivotal Application Services.

Click the Settings Tab.

Select File Storage.

Click External Azure Account.

Enter the Storage Account details.

In the Ops Manager main page, apply the changes when you are ready to reconfigure.

Note: There is no direct path to migrate previous objects to Azure Blob Storage. Contact Pivotal Support if you need assistance with this migration.

Load Balancer Migrations

The Azure Load Balancer has two SKUs: Standard and Basic. The Standard SKU Load Balancer is a new load balancer for all TCP and UDP applications with an expanded and more granular feature set over the Basic Load Balancer. Azure’s Standard SKU Load Balancer lets you scale your applications and create high availability in environments ranging from small scale deployments to large and complex multi-zone architectures. While the Basic SKU Load Balancer works within the scope of an availability set, a Standard Load Balancer covers an entire virtual network. Pivotal recommends using the Standard SKU Load Balancer instead of the Basic SKU Load Balancer. If you currently use the Basic SKU Load Balancer and wish to migrate to the the Standard SKU, please see the Migrate Basic SKU Load Balancer to Standard SKU Load Balancer documentation on GitHub.