527 lines
19 KiB
Plaintext
527 lines
19 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Google Cloud - Secrets Engines
|
|
sidebar_title: Google Cloud
|
|
description: |-
|
|
The Google Cloud secrets engine for Vault dynamically generates Google Cloud
|
|
service account keys and OAuth tokens based on IAM policies.
|
|
---
|
|
|
|
# Google Cloud Secrets Engine
|
|
|
|
The Google Cloud Vault secrets engine dynamically generates Google Cloud service
|
|
account keys and OAuth tokens based on IAM policies. This enables users to gain
|
|
access to Google Cloud resources without needing to create or manage a dedicated
|
|
service account.
|
|
|
|
The benefits of using this secrets engine to manage Google Cloud IAM service accounts are:
|
|
|
|
- **Automatic cleanup of GCP IAM service account keys** - each Service Account
|
|
key is associated with a Vault lease. When the lease expires (either during
|
|
normal revocation or through early revocation), the service account key is
|
|
automatically revoked.
|
|
|
|
- **Quick, short-term access** - users do not need to create new GCP Service
|
|
Accounts for short-term or one-off access (such as batch jobs or quick
|
|
introspection).
|
|
|
|
- **Multi-cloud and hybrid cloud applications** - users authenticate to Vault
|
|
using a central identity service (such as LDAP) and generate GCP credentials
|
|
without the need to create or manage a new Service Account for that user.
|
|
|
|
~> **NOTE: Deprecation of `access_token` Leases**: In previous versions of this secrets engine
|
|
(released with Vault <= 0.11.1), a lease was generated with access tokens. If you're using
|
|
an old version of the plugin, please upgrade. Read more in the
|
|
[upgrade guide](#deprecation-of-access-token-leases)
|
|
|
|
## Setup
|
|
|
|
Most secrets engines must be configured in advance before they can perform their
|
|
functions. These steps are usually completed by an operator or configuration
|
|
management tool.
|
|
|
|
1. Enable the Google Cloud secrets engine:
|
|
|
|
```text
|
|
$ vault secrets enable gcp
|
|
Success! Enabled the gcp secrets engine at: gcp/
|
|
```
|
|
|
|
By default, the secrets engine will mount at the name of the engine. To
|
|
enable the secrets engine at a different path, use the `-path` argument.
|
|
|
|
1. Configure the secrets engine with account credentials, or leave blank or unwritten
|
|
to use Application Default Credentials.
|
|
|
|
```text
|
|
$ vault write gcp/config credentials=@my-credentials.json
|
|
Success! Data written to: gcp/config
|
|
```
|
|
|
|
If you are running Vault from inside [Google Compute Engine][gce] or [Google
|
|
Kubernetes Engine][gke], the instance or pod service account can be used in
|
|
place of specifying the credentials JSON file.
|
|
For more information on authentication, see the [authentication section](#authentication) below.
|
|
|
|
1. Configure a roleset. Rolesets determine the permissions that Service Account
|
|
credentials generated by Vault will have on GCP resources.
|
|
|
|
To configure a roleset that generates OAuth2 access tokens (preferred):
|
|
|
|
```text
|
|
$ vault write gcp/roleset/my-token-roleset \
|
|
project="my-project" \
|
|
secret_type="access_token" \
|
|
token_scopes="https://www.googleapis.com/auth/cloud-platform" \
|
|
bindings=-<<EOF
|
|
resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
|
|
roles = ["roles/viewer"]
|
|
}
|
|
EOF
|
|
```
|
|
|
|
To configure a roleset that generates GCP Service Account keys:
|
|
|
|
```text
|
|
$ vault write gcp/roleset/my-key-roleset \
|
|
project="my-project" \
|
|
secret_type="service_account_key" \
|
|
bindings=-<<EOF
|
|
resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
|
|
roles = ["roles/viewer"]
|
|
}
|
|
EOF
|
|
```
|
|
|
|
Alternatively, provide a file for the `bindings` argument like so:
|
|
|
|
```text
|
|
$ vault write gcp/roleset/my-roleset
|
|
bindings=@mybindings.hcl
|
|
...
|
|
```
|
|
|
|
For more information on role bindings and sample role bindings, please see
|
|
the [roleset bindings](#roleset-bindings) section below.
|
|
|
|
For more information on the differences between OAuth2 access tokens and
|
|
Service Account keys, see the [things to note](#things-to-note) section
|
|
below.
|
|
|
|
## Usage
|
|
|
|
After the secrets engine is configured and a user/machine has a Vault token with
|
|
the proper permission, it can generate credentials. Depending on how the roleset
|
|
was configured, you can generate OAuth2 tokens or service account keys.
|
|
|
|
### Access Tokens
|
|
|
|
To generate OAuth2 tokens, read from `gcp/token/...`. The roleset must have been
|
|
created as type `access_token`:
|
|
|
|
```shell-session
|
|
$ vault read gcp/token/my-token-roleset
|
|
|
|
Key Value
|
|
--- -----
|
|
expires_at_seconds 1537402548
|
|
token ya29.c.ElodBmNPwHUNY5gcBpnXcE4ywG4w1k...
|
|
token_ttl 3599
|
|
```
|
|
|
|
This endpoint generates a non-renewable, non-revocable static OAuth2 access token
|
|
with a lifetime of one hour, where `token_ttl` is given in seconds and the
|
|
`expires_at_seconds` is the expiry time for the token, given as a Unix timestamp.
|
|
The `token` value then can be used as a HTTP Authorization Bearer token in requests
|
|
to GCP APIs:
|
|
|
|
```shell-session
|
|
$ curl -H "Authorization: Bearer ya29.c.ElodBmNPwHUNY5gcBpnXcE4ywG4w1k..."
|
|
```
|
|
|
|
### Service Account Keys
|
|
|
|
To generate service account keys, read from `gcp/key/...`. The roleset must have
|
|
been created as type `service_account_key`:
|
|
|
|
```shell-session
|
|
$ vault read gcp/key/my-key-roleset
|
|
|
|
Key Value
|
|
--- -----
|
|
lease_id gcp/key/my-key-roleset/ce563a99-5e55-389b...
|
|
lease_duration 30m
|
|
lease_renewable true
|
|
key_algorithm KEY_ALG_RSA_2048
|
|
key_type TYPE_GOOGLE_CREDENTIALS_FILE
|
|
private_key_data ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsC...
|
|
```
|
|
|
|
This endpoint generates a new [GCP IAM service account key][iam-keys] associated
|
|
with the roleset's Service Account. When the lease expires (or is revoked
|
|
early), the Service Account key will be deleted.
|
|
|
|
**There is a default limit of 10 keys per Service Account.** For more
|
|
information on this limit and recommended mitigation, please see the [things to
|
|
note](#things-to-note) section below.
|
|
|
|
## Roleset Bindings
|
|
|
|
Roleset bindings define a list of resources and the associated IAM roles on that
|
|
resource. Roleset bindings are used as the `binding` argument when creating or
|
|
updating a roleset and are specified in the following format using HCL:
|
|
|
|
```hcl
|
|
resource NAME {
|
|
roles = [ROLE, [ROLE...]]
|
|
}
|
|
```
|
|
|
|
For example:
|
|
|
|
```hcl
|
|
resource "buckets/my-bucket" {
|
|
roles = [
|
|
"roles/storage.objectAdmin",
|
|
"roles/storage.legacyBucketReader",
|
|
]
|
|
}
|
|
|
|
# At instance level, using self-link
|
|
resource "https://www.googleapis.com/compute/v1/projects/my-project/zone/my-zone/instances/my-instance" {
|
|
roles = [
|
|
"roles/compute.instanceAdmin.v1"
|
|
]
|
|
}
|
|
|
|
# At project level
|
|
resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
|
|
roles = [
|
|
"roles/compute.instanceAdmin.v1",
|
|
"roles/iam.serviceAccountUser", # required if managing instances that run as service accounts
|
|
]
|
|
}
|
|
|
|
# At folder level
|
|
resource "//cloudresourcemanager.googleapis.com/folders/123456" {
|
|
roles = [
|
|
"roles/compute.viewer",
|
|
"roles/deploymentmanager.viewer",
|
|
]
|
|
}
|
|
|
|
```
|
|
|
|
The top-level `resource` block defines the resource or resource path for which
|
|
IAM policy information will be bound. The resource path may be specified in a
|
|
few different formats:
|
|
|
|
- **Project-level self-link** - a URI with scheme and host, generally
|
|
corresponding to the `self_link` attribute of a resource in GCP. This must
|
|
include the resource nested in the parent project.
|
|
|
|
```text
|
|
# compute alpha zone
|
|
https://www.googleapis.com/compute/alpha/projects/my-project/zones/us-central1-c
|
|
```
|
|
|
|
- **Full resource name** - a schema-less URI consisting of a DNS-compatible API
|
|
service name and resource path. See the [full resource name API
|
|
documentation][resource-name-full] for more information.
|
|
|
|
```text
|
|
# Compute snapshot
|
|
//compute.googleapis.com/project/my-project/snapshots/my-compute-snapshot
|
|
|
|
# Pubsub snapshot
|
|
//pubsub.googleapis.com/project/my-project/snapshots/my-pubsub-snapshot
|
|
|
|
# BigQuery dataset
|
|
//bigquery.googleapis.com/projects/my-project/datasets/mydataset
|
|
|
|
# Resource manager
|
|
//cloudresourcemanager.googleapis.com/projects/my-project"
|
|
```
|
|
|
|
- **Relative resource name** - A path-noscheme URI path, usually as accepted by
|
|
the API. Use this if the version or service are apparent from the resource
|
|
type. Please see the [relative resource name API
|
|
documentation][resource-name-relative] for more information.
|
|
|
|
```text
|
|
# Storage bucket objects
|
|
buckets/my-bucket
|
|
buckets/my-bucket/objects/my-object
|
|
|
|
# PubSub topics
|
|
projects/my-project/topics/my-pubsub-topic
|
|
```
|
|
|
|
The nested `roles` attribute is an array of strings names of [GCP IAM
|
|
roles][iam-roles]. The roles may be specified in the following formats:
|
|
|
|
- **Global role name** - these are global roles built into Google Cloud. For the
|
|
full list of available roles, please see the [list of predefined GCP
|
|
roles][predefined-roles].
|
|
|
|
```text
|
|
roles/viewer
|
|
roles/bigquery.user
|
|
roles/billing.admin
|
|
```
|
|
|
|
- **Organization-level custom role** - these are roles that are created at the
|
|
organization level by organization owners.
|
|
|
|
```text
|
|
organizations/my-organization/roles/my-custom-role
|
|
```
|
|
|
|
For more information, please see the documentation on [GCP custom
|
|
roles][custom-roles].
|
|
|
|
- **Project-level custom role** - these are roles that are created at a
|
|
per-project level by project owners.
|
|
|
|
```text
|
|
projects/my-project/roles/my-custom-role
|
|
```
|
|
|
|
For more information, please see the documentation on [GCP custom
|
|
roles][custom-roles].
|
|
|
|
## Authentication
|
|
|
|
The Google Cloud Vault secrets backend uses the official Google Cloud Golang
|
|
SDK. This means it supports the common ways of [providing credentials to Google
|
|
Cloud][cloud-creds]. In addition to specifying `credentials` directly via Vault
|
|
configuration, you can also get configuration from the following values **on the
|
|
Vault server**:
|
|
|
|
1. The environment variables `GOOGLE_APPLICATION_CREDENTIALS`. This is specified
|
|
as the **path** to a Google Cloud credentials file, typically for a service
|
|
account. If this environment variable is present, the resulting credentials are
|
|
used. If the credentials are invalid, an error is returned.
|
|
|
|
1. Default instance credentials. When no environment variable is present, the
|
|
default service account credentials are used. This is useful when running Vault
|
|
on [Google Compute Engine][gce] or [Google Kubernetes Engine][gke]
|
|
|
|
For more information on service accounts, please see the [Google Cloud Service
|
|
Accounts documentation][service-accounts].
|
|
|
|
To use this secrets engine, the service account must have the following
|
|
minimum scope(s):
|
|
|
|
```text
|
|
https://www.googleapis.com/auth/cloud-platform
|
|
```
|
|
|
|
### Required Permissions
|
|
|
|
The credentials given to Vault must have the following permissions:
|
|
|
|
```text
|
|
# Service Account + Key Admin
|
|
iam.serviceAccounts.create
|
|
iam.serviceAccounts.delete
|
|
iam.serviceAccounts.get
|
|
iam.serviceAccounts.list
|
|
iam.serviceAccounts.update
|
|
iam.serviceAccountKeys.create
|
|
iam.serviceAccountKeys.delete
|
|
iam.serviceAccountKeys.get
|
|
iam.serviceAccountKeys.list
|
|
|
|
# IAM Policy Changes
|
|
<service>.<resource>.getIamPolicy
|
|
<service>.<resource>.setIamPolicy
|
|
```
|
|
|
|
where `<service>` and `<resource>` correspond to permissions which will be
|
|
granted, for example:
|
|
|
|
```text
|
|
# Projects
|
|
resourcemanager.projects.getIamPolicy
|
|
resourcemanager.projects.setIamPolicy
|
|
|
|
# All compute
|
|
compute.*.getIamPolicy
|
|
compute.*.setIamPolicy
|
|
|
|
# BigQuery Datasets
|
|
bigquery.datasets.get
|
|
bigquery.datasets.update
|
|
```
|
|
|
|
You can either:
|
|
|
|
- Create a [custom role][custom-roles] using these permissions, and assign this
|
|
role at a project-level
|
|
|
|
- Assign the set of roles required to get resource-specific
|
|
`getIamPolicy/setIamPolicy` permissions. At a minimum you will need to assign
|
|
`roles/iam.serviceAccountAdmin` and `roles/iam.serviceAccountKeyAdmin` so
|
|
Vault can manage service accounts and keys.
|
|
|
|
- Notice that BigQuery requires different permissions than other resource. This is
|
|
because BigQuery currently uses legacy ACL instead of traditional IAM permissions.
|
|
This means to update access on the dataset, Vault must be able to update the dataset's
|
|
metadata.
|
|
|
|
### Root Credential Rotation
|
|
|
|
If the mount is configured with credentials directly, the credential's key may be
|
|
rotated to a Vault-generated value that is not accessible by the operator. For more
|
|
details on this operation, please see the
|
|
[Root Credential Rotation](/api/secret/gcp#rotate-root-credentials) API docs.
|
|
|
|
## Things to Note
|
|
|
|
### Access Tokens vs. Service Account Keys
|
|
|
|
Advantages of `access_tokens`:
|
|
|
|
- Can generate infinite number of tokens per roleset
|
|
|
|
Disadvantages of `access_tokens`:
|
|
|
|
- Cannot be used with some client libraries or tools
|
|
- Have a static life-time of 1 hr that cannot be modified, revoked, or extended.
|
|
|
|
Advantages of `service_account_keys`:
|
|
|
|
- Controllable life-time through Vault, allowing for longer access
|
|
- Can be used by all normal GCP tooling
|
|
|
|
Disadvantages of `service_account_keys`:
|
|
|
|
- Infinite lifetime in GCP (i.e. if they are not managed properly, leaked keys can live forever)
|
|
- Limited to 10 per roleset/service account.
|
|
|
|
When generating OAuth access tokens, Vault will still
|
|
generate a dedicated service account and key. This private key is stored in Vault
|
|
and is never accessible to other users, and the underlying key can
|
|
be rotated. See the [GCP API documentation][api] for more information on
|
|
rotation.
|
|
|
|
### Service Accounts are Tied to Rolesets
|
|
|
|
Service Accounts are created when the roleset is created (or updated) rather
|
|
than each time a secret is generated. This may be different from how other
|
|
secrets engines behave, but it is for good reasons:
|
|
|
|
- IAM Service Account creation and permission propagation can take up to 60
|
|
seconds to complete. By creating the Service Account in advance, we speed up
|
|
the timeliness of future operations and reduce the flakiness of automated
|
|
workflows.
|
|
|
|
- Each GCP project has a limit on the number of IAM Service Accounts. You can
|
|
[request additional quota][quotas]. The quota increase is processed by humans,
|
|
so it is best to request this additional quota in advance. This limit is
|
|
currently 100, **including system-managed Service Accounts**. If Service
|
|
Accounts were created per secret, this quota limit would reduce the number of
|
|
secrets that can be generated.
|
|
|
|
### Service Account Keys Quota Limits
|
|
|
|
GCP IAM has a hard limit (currently 10) on the number of Service Account keys.
|
|
Attempts to generate more keys will result in an error. If you find yourself
|
|
running into this limit, consider the following:
|
|
|
|
- Have shorter TTLs or revoke access earlier. If you are not using past Service
|
|
Account keys, consider rotating and freeing quota earlier.
|
|
|
|
- Create additional rolesets which share the same set of permissions. Additional
|
|
rolesets can be created with the same set of permissions. This will create a
|
|
new service account and increases the number of keys you can create.
|
|
|
|
- Where possible, use OAuth2 access tokens instead of Service Account keys.
|
|
|
|
### Resources Must Exist at Roleset Creation
|
|
|
|
Because the bindings for the Service Account are set during roleset creation,
|
|
resources that do not exist will fail the `getIamPolicy` API call.
|
|
|
|
### Roleset Creation May Partially Fail
|
|
|
|
Every Service Account creation, key creation, and IAM policy change is a GCP API
|
|
call per resource. If an API call to one of these resources fails, the roleset
|
|
creation fails and Vault will attempt to rollback.
|
|
|
|
These rollbacks are API calls, so they may also fail. The secrets engine uses a
|
|
WAL to ensure that unused bindings are cleaned up. In the case of quota limits,
|
|
you may need to clean these up manually.
|
|
|
|
### Do Not Modify Vault-owned IAM Accounts
|
|
|
|
While Vault will initially create and assign permissions to IAM service
|
|
accounts, it is possible that an external user deletes or modifies this service
|
|
account. These changes are difficult to detect, and it is best to prevent this
|
|
type of modification through IAM permissions.
|
|
|
|
Vault roleset Service Accounts will have emails in the format:
|
|
|
|
```
|
|
vault<roleset-prefix>-<creation-unix-timestamp>@...
|
|
```
|
|
|
|
Communicate with your teams (or use IAM permissions) to not modify these
|
|
resources.
|
|
|
|
## Help & Support
|
|
|
|
The Google Cloud Vault secrets engine is written as an external Vault plugin and
|
|
thus exists outside the main Vault repository. It is automatically bundled with
|
|
Vault releases, but the code is managed separately.
|
|
|
|
Please report issues, add feature requests, and submit contributions to the
|
|
[vault-plugin-secrets-gcp repo on GitHub][repo].
|
|
|
|
## API
|
|
|
|
The GCP secrets engine has a full HTTP API. Please see the [GCP secrets engine API docs][api]
|
|
for more details.
|
|
|
|
[api]: /api/secret/gcp
|
|
[cloud-creds]: https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application
|
|
[custom-roles]: https://cloud.google.com/iam/docs/creating-custom-roles
|
|
[gce]: https://cloud.google.com/compute/
|
|
[gke]: https://cloud.google.com/kubernetes-engine/
|
|
[iam-keys]: https://cloud.google.com/iam/docs/service-accounts#service_account_keys
|
|
[iam-roles]: https://cloud.google.com/iam/docs/understanding-roles
|
|
[predefined-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
|
|
[repo]: https://github.com/hashicorp/vault-plugin-secrets-gcp
|
|
[resource-name-full]: https://cloud.google.com/apis/design/resource_names#full_resource_name
|
|
[resource-name-relative]: https://cloud.google.com/apis/design/resource_names#relative_resource_name
|
|
[quotas]: https://cloud.google.com/compute/quotas
|
|
[service-accounts]: https://cloud.google.com/compute/docs/access/service-accounts
|
|
|
|
## Upgrade Guides
|
|
|
|
### Deprecation of Access Token Leases
|
|
|
|
~> **NOTE**: This only affects access tokens. There is no change to the `service_account_key` secret type
|
|
|
|
Previous versions of this secrets engine (Vault <= 0.11.1) created a lease for
|
|
each access token secret. We have removed them after discovering that these
|
|
tokens, specifically Google OAuth2 tokens for IAM service accounts, are
|
|
non-revocable and have a static 60 minute lifetime. To match the current
|
|
limitations of the GCP APIs, the secrets engine will no longer allow for
|
|
revocation or manage the token TTL - more specifically, **the access_token
|
|
response will no longer include `lease_id` or other lease information**. This
|
|
change does not reflect any change to the actual underlying OAuth tokens or GCP
|
|
service accounts.
|
|
|
|
To upgrade:
|
|
|
|
- Remove references from `lease_id`, `lease_duration` or other `lease_*`
|
|
attributes when reading responses for the access tokens secrets endpoint (i.e.
|
|
from `gcp/token/$roleset`). See the [documentation for access
|
|
tokens](#access-tokens) to see the new format for the response.
|
|
|
|
- Be aware of leftover leases from previous versions. While these old leases
|
|
will still be revocable, they will not actually invalidate their associated
|
|
access token, and that token will still be useable for up to one hour.
|