Customer-supplied encryption keys

Overview

As an additional layer on top of Google-managed encryption keys,
you can choose to provide your own AES-256 key, encoded in
standard Base64. This key is known as a customer-supplied encryption
key. If you provide a customer-supplied encryption key, Cloud Storage
does not permanently store your key on Google's servers or otherwise
manage your key. Instead, you provide your key for each Cloud Storage
operation, and your key is purged from Google's servers after the operation
is complete. Cloud Storage stores only a cryptographic hash of the key so
that future requests can be validated against the hash. Your key cannot be
recovered from this hash, and the hash cannot be used to decrypt your data.

When is the key used?

When you apply a customer-supplied encryption key to an object, Cloud Storage
uses the key when encrypting:

The object's data.

The object's CRC32C checksum.

The object's MD5 hash.

Cloud Storage uses standard server-side keys to encrypt the remaining
metadata for the object, including the object's name. This allows you to
read and update general metadata, as well as list and delete objects,
without needing the customer-supplied encryption key. However, to perform any of
these actions, you must have sufficient permission to do so.

For example, if an object is encrypted with a customer-supplied encryption key,
the key must be used to perform operations on the object such as downloading or
moving it. If you attempt to read the object's metadata without providing the
key, you receive metadata such as the object name and Content-Type, but not
the object's CRC32C checksum or MD5 hash. If you do supply your key with the
request for the object metadata, the object's CRC32C checksum and MD5 hash are
included with the metadata.

Note: Because most metadata can be read regardless of encryption method, do
not include sensitive information in your object and bucket metadata or names.

HTTPS check

To protect your data as it travels over the Internet during read and write
operations, use Transport Layer Security, commonly known as TLS or HTTPS. TLS is
required when you provide an encryption key. If you accidentally use your
encryption key over an unencrypted (HTTP) connection, it is possible for an
attacker to intercept your key. Because of this possibility, the
Cloud Storage API returns an error message warning you that your key may be
compromised. If this occurs, you should immediately rotate your keys.

Restrictions

The following restrictions apply when using customer-supplied encryption keys:

You cannot use the Google Cloud Console to download objects that are encrypted
with a customer-supplied encryption key. Similarly, when you use the
Google Cloud Console to upload an object, you cannot encrypt the object with a
customer-supplied encryption key. You can perform these actions with
customer-managed encryption keys.

You can only set customer-supplied encryption keys on individual objects.
You cannot set a default customer-supplied encryption key for a bucket.

If you are performing a compose operation on objects encrypted by
customer-supplied encryption keys, the component objects must be
encrypted by the same key, and you need to provide the key with the compose
request. The resulting composite object is encrypted by the same key.

Tools for using encryption keys

There are multiple ways that you can use customer-supplied encryption keys
with Cloud Storage. These include:

Partner companies.

The JSON and XML REST APIs.

The gsutil command-line tool.

Encryption keys with partner companies

There are several third-party partner options for using customer-supplied
encryption keys. These partners make it easier to generate an encryption key
and to associate that key with an object in Cloud Storage. Partners that can
supply keys for Cloud Storage include Ionic Security and KeyNexus.

Encryption keys with REST APIs

When you use a customer-supplied encryption key and work directly with the
JSON or XML API, you must provide both the AES-256 key and a SHA256
hash of the key. You should store both the AES-256 key and the SHA256 hash of
the key securely. Google stores the SHA256 hash of your key in the object's
metadata, where you can retrieve it later. This SHA256 hash cannot be used by
Google (or anyone else) to decrypt your data. It is stored as a way to
uniquely identify the AES-256 key that was used to encrypt a particular object.

An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key.

If you are performing a rewrite operation with the JSON API, the
headers listed above are used for encrypting the destination object, and
the following headers are used for decrypting the source object:

An RFC 4648 Base64-encoded string of the SHA256 hash of the source object's encryption key.

Important: The rewrite operation makes a copy of an object. If you omit the
headers listed in the first table in a rewrite operation, the destination
object is encrypted using the default server-side encryption, not your
customer-supplied encryption key.

Response

JSON

When using the JSON API, the metadata for a customer-supplied encryption
key is returned in the response body, which includes the following
properties:

Property name

Value

Description

customerEncryption

object

Information about the encryption used for the request.

customerEncryption.encryptionAlgorithm

string

The encryption algorithm that was used. Always contains the value AES256.

customerEncryption.keySha256

string

An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key. You can use this SHA256 hash to uniquely identify the AES-256 encryption key required to decrypt the object, which you must store securely.

XML

When using the XML API, the response includes the following headers:

Header name

Value

Description

x-goog-encryption-algorithm

string

The encryption algorithm that was used. Always contains the value AES256.

x-goog-encryption-key-sha256

string

An RFC 4648 Base64-encoded string of the SHA256 hash of your encryption key. You can use this SHA256 hash to uniquely identify the AES-256 encryption key required to decrypt the object, which you must store securely.

You receive an HTTP 400 error in the following cases:

You upload an object using a customer-supplied encryption key, and you attempt
to perform another operation on the object (other than requesting or updating
most metadata or deleting the object) without providing the key.

You upload an object using a customer-supplied encryption key, and you attempt
to perform another operation on the object with an incorrect key.

You upload an object without providing a customer-supplied encryption key, and
you attempt to perform another operation on the object with a customer-supplied
encryption key.

You specify an encryption algorithm, key, or SHA256 hash that is not valid.

Note: gsutil automatically calculates the SHA256 hash of your AES-256 encryption
key, so there is no need to configure it in the .boto file.

You can optionally specify one or more decryption keys. While the
encryption_key option is used by gsutil as both an encryption and decryption
key, any decryption_key options you specify are used only to decrypt objects.
For details, see the gsutil documentation.

As long as you have encryption or decryption keys in your boto configuration
file, they are used for all gsutil commands. When decrypting, gsutil
calculates the SHA256 hash of the supplied encryption and decryption keys and
selects the correct key to use for a particular object by matching the
SHA256 hash in the object's metadata.

You receive an error if you upload an object using a customer-supplied
encryption key and then attempt to perform another operation on the object
(other than requesting or updating metadata or deleting the object)
without providing the key.

Encryption key rotation

If an object is encrypted using a customer-supplied encryption key, you can
rotate the object's key by rewriting the object. Rewrites are
supported through the JSON API, but not the XML API. See
Rotating an encryption key for examples of key rotation.

Note: If your bucket uses object versioning, be sure to delete
the noncurrent versions of your objects after you rotate the key. Noncurrent
versions are not deleted automatically and remain encrypted with the old key.