Creating a Shared VPC with Deployment Manager (tutorial)

This tutorial guides you through the process of setting up
Shared VPC
using
Deployment Manager,
which provides a way to enforce strict network security rules across your
organization for access to GCP resources.

Introduction

In large organizations, you may need to put different departments or different
applications into different projects to separate budgeting, access control, and
so on. With Shared VPC,
Organization
administrators can give multiple projects permission to use a single, shared VPC
network and corresponding networking resources.

With Shared VPC, as an Organization administrator, you can allow the network and
security admins of your organization to manage a VPC network of
RFC 1918
IP spaces (and related features such as VPNs or firewall rules) that associated
projects can use. Administrators in associated projects can create virtual
machine (VM) instances in the shared VPC network space. You can apply and
enforce consistent policies across an organization.

Because Shared VPC is often used in large organizations, or in organizations
with strict security rules, being able to easily reproduce a Shared VPC setup is
important. You can use Deployment Manager, an Infrastructure as Code (IaC) tool,
to achieve this.

Note: Shared VPC was previously known as Cross-Project Networking (XPN).

About running the tutorial

This tutorial assumes that you are familiar with Organization resources in
Google Cloud Platform (GCP) and that you are the administrator of an
Organization resource. Understanding Shared VPCs and Deployment Manager will
help you follow this tutorial. Throughout this tutorial, the word deployment
refers to a deployment from Deployment Manager.

For simplicity, this tutorial uses a single IAM user—your existing identity as
an Organization resource administrator. (If you don't already have an
Organization resource, you can find instructions in the
Before you begin
section for creating one.) In a company, three different people are usually
involved in setting up the scenario illustrated by this tutorial: an
Organization resource administrator, a network administrator (who manages the
Shared VPC), and a user of the Shared VPC.

Deployment architecture

The following diagram shows the architecture of this solution:

In this tutorial, you create three of the projects from this diagram:

The Deployment Manager home project is managed by an Organization
resource administrator (in this tutorial we assume that's you) and is used
to create the other projects and the Shared VPC setup.

The host project is managed by network administrators and hosts the
Shared VPC. All the networking configuration will be done in this project.

The service project is managed by users of the Shared VPC. In this
project, resources can be created in the Shared VPC from the host project.

Before you begin

Later in this tutorial, you create two GCP projects with a deployment (that is,
a Deployment Manager deployment). You create this deployment in a dedicated
project with a specific configuration. The following steps guide you
through the initial setup of this project and its configuration.

Note: Some of the steps of this tutorial can be done in the
GCP Console, but the most important ones can't be. Therefore, we
recommend that you use the gcloud command-line tool throughout this tutorial.

Creating the home project for Deployment Manager

Create a new project in your Organization and set a billing account
for it. You must choose a unique ID for this project. For instance, you can
use a name like [YOUR_NAME]-[DATE]-dm-home.

Warning: This project is going to create and own the Deployment Manager
deployments. Because of the permissions granted in later steps, this project
should not be used for any purpose other than creating other projects.

gcloud

Display the organization list and make a note of the
organization ID:

gcloud organizations list

Set environment variables for values you will use repeatedly in
this tutorial. Replace [ORG_ID] with the value you copied
in the previous step, and replace [DM_HOME_PROJECT_ID]
with the ID of your home project.

export ORG_ID=[ORG_ID]
export DM_HOME_PROJECT=[DM_HOME_PROJECT_ID]

Create the new project:

gcloud projects create $DM_HOME_PROJECT \
--organization=$ORG_ID

Get a list of billing IDs for the project and make a note of
the billing ID:

gcloud beta billing accounts list

If you have access to several billing accounts, review your
internal policies for each of them and choose the appropriate one.

Set an environment variable to the ID of the billing account
that you got from the previous step. Replace
[BILLING_ACCOUNT_ID] with the billing ID you got from the
previous step.

Configuring the Organization resource policies

Shared VPC is an Organization-level feature. As such, it requires some
Organization-level policies to be configured—the service account used by
Deployment Manager needs specific roles at the Organization level.

gcloud

Set environment variables for the project number and service account
name. The following commands read the values from your project and then use
the values to set the environment variables.

console

In your project, go to IAM & admin > Settings.

Note the project number.

Go to IAM & admin > IAM.

Switch to your organization in the top project-selection menu.

Click Add at the top of the window.

Add the following roles to the Deployment Manager service
account. This service account name is
[PROJECT_NUMBER]@cloudservices.gserviceaccount.com, where
[PROJECT_NUMBER] is the value you noted earlier.

Resource Manager > Project Creator

Billing > Billing Account User

Compute Engine > Compute Shared VPC Admin

Note: In a production environment, you should protect the Shared VPC host
project against deletion. In order to do that, run the following command:

A unique name for your host project. The host project is where the VPC
will be created and managed. Note that you must also change this value in
the last line of the file, for a total of two replacements.

Example:

[YOUR_NAME]-[DATE]-host

SERVICE_PROJECT

A unique name for your service project. The service project is where
the VPC from the host project will be used.

Example:

[YOUR_NAME]-[DATE]-svc

ORG_ID

Your Organization ID. You should have this value from earlier steps.
If not, you can get it by running the following command:

gcloud organizations list

Use quotation marks around the Organization ID so that it's not
considered a numeric value in the YAML file.

BILLING_ACCOUNT_ID

The ID of your billing account. You should have this value from
earlier steps. If not, you can get it by running the following
command:

gcloud beta billing accounts list

EMAIL

Your email address. Note that there are a total of five places
where you need to set the email address.

Note: There are two resources of type project.py in the
config_shared_vpc.yaml file. The first is the host project for the Shared
VPC, and the second is the service project. You need to modify both with
your own values.

Save the file and exit nano with the following key sequence:

^O <Enter> ^X

If you used the GCP Console to create the Deployment Manager
home project, set an environment variable to the ID of your home project:

After a few minutes, the deployment is complete and your two new projects are
created. Although the service project is already linked to the host project, you
have not yet created the VPC that is going to be shared.

Troubleshooting

If problems occurred while the preview was being created, you can delete the
deployment (no resources have been created yet) and retry the process.

However, if the preview was created but the deployment failed, you might not be
able to re-create that deployment. Projects cannot be deleted and re-created
immediately; they are marked for deletion for a safety period of 30 days. If you
do need to re-create the deployment, change the values of the HOST_PROJECT and
SERVICE_PROJECT settings in the config_shared_vpc.yaml file before you try
again.

Creating the VPC

You now are going to use another Deployment Manager template to create the VPC
in the host project. This is typically an operation that would be done by a
network administrator. Because the Shared VPC configuration has already been set
up, the VPC is going to be available for use in the service project
immediately.

Deploying your networking configuration

Set environment variables to the names of the host project and service
project names that you set in the config_shared_vpc.yaml file:

Verification

You can now verify access to the host project's network from the service
project. You can access this network because you have the role
roles/compute.networkUser in the host project. For a production deployment,
you need to assign this role to every user who is going to use Shared VPC.

Create a test instance in the service project using a subnetwork from the
host project.

gcloud

The result of this command is the link to the VPC used by the
instance you created. You can see from this link that the VPC is from
the host project, even if the instance was created in the service
project (a result of using the --project flag in the
command).

Console

Select your instance in the instance list.

Verify that it is using the network from the host project.

Cleaning up

After you've finished the current tutorial, you can clean up the
resources you created on Google Cloud Platform so you won't be billed
for them in the future. The following sections describe how to delete
or turn off these resources.