luxolus/open-vault
2
0
Fork 0
Code Issues 1 Pull requests Releases Activity
open-vault/website/pages/docs/secrets/gcp/index.mdx
Jeff Escalante fbfed414bd
Routing Fixes (#8703) 
* fix a couple of .html extension links
* remove trailing slash config
* fix redirects, broken links, incorrect anchor link formats
2020-04-09 13:56:59 -04:00

515 lines
18 KiB
Plaintext
Raw Blame History

---
layout: docs
page_title: Google Cloud - Secrets Engines
sidebar_title: Google Cloud
description: |-
The Google Cloud secrets engine for Vault dynamically generates Google Cloud
service account keys and OAuth tokens based on IAM policies.
---
# Google Cloud Secrets Engine
The Google Cloud Vault secrets engine dynamically generates Google Cloud service
account keys and OAuth tokens based on IAM policies. This enables users to gain
access to Google Cloud resources without needing to create or manage a dedicated
service account.
The benefits of using this secrets engine to manage Google Cloud IAM service accounts are:
- **Automatic cleanup of GCP IAM service account keys** - each Service Account
key is associated with a Vault lease. When the lease expires (either during
normal revocation or through early revocation), the service account key is
automatically revoked.
- **Quick, short-term access** - users do not need to create new GCP Service
Accounts for short-term or one-off access (such as batch jobs or quick
introspection).
- **Multi-cloud and hybrid cloud applications** - users authenticate to Vault
using a central identity service (such as LDAP) and generate GCP credentials
without the need to create or manage a new Service Account for that user.
~> **NOTE: Deprecation of `access_token` Leases**: In previous versions of this secrets engine
(released with Vault <= 0.11.1), a lease was generated with access tokens. If you're using
an old version of the plugin, please upgrade. Read more in the
[upgrade guide](#deprecation-of-access-token-leases)
## Setup
Most secrets engines must be configured in advance before they can perform their
functions. These steps are usually completed by an operator or configuration
management tool.
1. Enable the Google Cloud secrets engine:
```text
$ vault secrets enable gcp
Success! Enabled the gcp secrets engine at: gcp/
```
By default, the secrets engine will mount at the name of the engine. To
enable the secrets engine at a different path, use the `-path` argument.
1. Configure the secrets engine with account credentials, or leave blank or unwritten
to use Application Default Credentials.
```text
$ vault write gcp/config credentials=@my-credentials.json
Success! Data written to: gcp/config
```
If you are running Vault from inside [Google Compute Engine][gce] or [Google
Kubernetes Engine][gke], the instance or pod service account can be used in
place or specifying the credentials JSON file.
For more information on authentication, see the [authentication section](#authentication) below.
1. Configure a roleset. Rolesets determine the permissions that Service Account
credentials generated by Vault will have on GCP resources.
To configure a roleset that generates OAuth2 access tokens (preferred):
```text
$ vault write gcp/roleset/my-token-roleset \
project="my-project" \
secret_type="access_token" \
token_scopes="https://www.googleapis.com/auth/cloud-platform" \
bindings=-<<EOF
resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
roles = ["roles/viewer"]
}
EOF
```
To configure a roleset that generates GCP Service Account keys:
```text
$ vault write gcp/roleset/my-key-roleset \
project="my-project" \
secret_type="service_account_key" \
bindings=-<<EOF
resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
roles = ["roles/viewer"]
}
EOF
```
Alternatively, provide a file for the `bindings` argument like so:
```text
$ vault write gcp/roleset/my-roleset
bindings=@mybindings.hcl
...
```
For more information on role bindings and sample role bindings, please see
the [roleset bindings](#roleset-bindings) section below.
For more information on the differences between OAuth2 access tokens and
Service Account keys, see the [things to note](#things-to-note) section
below.
## Usage
After the secrets engine is configured and a user/machine has a Vault token with
the proper permission, it can generate credentials. Depending on how the roleset
was configured, you can generate OAuth2 tokens or service account keys.
### Access Tokens
To generate OAuth2 tokens, read from `gcp/token/...`. The roleset must have been
created as type `access_token`:
```text
$ vault read gcp/token/my-token-roleset
Key Value
--- -----
expires_at_seconds 1537402548
token ya29.c.ElodBmNPwHUNY5gcBpnXcE4ywG4w1k...
token_ttl 3599
```
This endpoint generates a non-renewable, non-revocable static OAuth2 access token
with a lifetime of one hour, where `token_ttl` is given in seconds and the
`expires_at_seconds` is the expiry time for the token, given as a Unix timestamp.
The `token` value then can be used as a HTTP Authorization Bearer token in requests
to GCP APIs:
```sh
$ curl -H "Authorization: Bearer ya29.c.ElodBmNPwHUNY5gcBpnXcE4ywG4w1k..."
```
### Service Account Keys
To generate service account keys, read from `gcp/key/...`. The roleset must have
been created as type `service_account_key`:
```text
$ vault read gcp/key/my-key-roleset
Key Value
--- -----
lease_id gcp/key/my-key-roleset/ce563a99-5e55-389b...
lease_duration 30m
lease_renewable true
key_algorithm KEY_ALG_RSA_2048
key_type TYPE_GOOGLE_CREDENTIALS_FILE
private_key_data ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsC...
```
This endpoint generates a new [GCP IAM service account key][iam-keys] associated
with the roleset's Service Account. When the lease expires (or is revoked
early), the Service Account key will be deleted.
**There is a default limit of 10 keys per Service Account.** For more
information on this limit and recommended mitigation, please see the [things to
note](#things-to-note) section below.
## Roleset Bindings
Roleset bindings define a list of resources and the associated IAM roles on that
resource. Roleset bindings are used as the `binding` argument when creating or
updating a roleset and are specified in the following format using HCL:
```hcl
resource NAME {
roles = [ROLE, [ROLE...]]
}
```
For example:
```hcl
resource "buckets/my-bucket" {
roles = [
"roles/storage.objectAdmin",
"roles/storage.legacyBucketReader",
]
}
# At instance level, using self-link
resource "https://www.googleapis.com/compute/v1/projects/my-project/zone/my-zone/instances/my-instance" {
roles = [
"roles/compute.instanceAdmin.v1"
]
}
# At project level
resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
roles = [
"roles/compute.instanceAdmin.v1",
"roles/iam.serviceAccountUser", # required if managing instances that run as service accounts
]
}
# At folder level
resource "//cloudresourcemanager.googleapis.com/folders/123456" {
roles = [
"roles/compute.viewer",
"roles/deploymentmanager.viewer",
]
}
```
The top-level `resource` block defines the resource or resource path for which
IAM policy information will be bound. The resource path may be specified in a
few different formats:
- **Project-level self-link** - a URI with scheme and host, generally
corresponding to the `self_link` attribute of a resource in GCP. This must
include the resource nested in the parent project.
```text
# compute alpha zone
https://www.googleapis.com/compute/alpha/projects/my-project/zones/us-central1-c
```
- **Full resource name** - a schema-less URI consisting of a DNS-compatible API
service name and resource path. See the [full resource name API
documentation][resource-name-full] for more information.
```text
# Compute snapshot
//compute.googleapis.com/project/my-project/snapshots/my-compute-snapshot
# Pubsub snapshot
//pubsub.googleapis.com/project/my-project/snapshots/my-pubsub-snapshot
# Resource manager
//cloudresourcemanager.googleapis.com/projects/my-project"
```
- **Relative resource name** - A path-noscheme URI path, usually as accepted by
the API. Use this if the version or service are apparent from the resource
type. Please see the [relative resource name API
documentation][resource-name-relative] for more information.
```text
# Storage bucket objects
buckets/my-bucket
buckets/my-bucket/objects/my-object
# PubSub topics
projects/my-project/topics/my-pubsub-topic
```
The nested `roles` attribute is an array of strings names of [GCP IAM
roles][iam-roles]. The roles may be specified in the following formats:
- **Global role name** - these are global roles built into Google Cloud. For the
full list of available roles, please see the [list of predefined GCP
roles][predefined-roles].
```text
roles/viewer
roles/bigquery.user
roles/billing.admin
```
- **Organization-level custom role** - these are roles that are created at the
organization level by organization owners.
```text
organizations/my-organization/roles/my-custom-role
```
For more information, please see the documentation on [GCP custom
roles][custom-roles].
- **Project-level custom role** - these are roles that are created at a
per-project level by project owners.
```text
projects/my-project/roles/my-custom-role
```
For more information, please see the documentation on [GCP custom
roles][custom-roles].
## Authentication
The Google Cloud Vault secrets backend uses the official Google Cloud Golang
SDK. This means it supports the common ways of [providing credentials to Google
Cloud][cloud-creds]. In addition to specifying `credentials` directly via Vault
configuration, you can also get configuration from the following values **on the
Vault server**:
1. The environment variables `GOOGLE_APPLICATION_CREDENTIALS`. This is specified
as the **path** to a Google Cloud credentials file, typically for a service
account. If this environment variable is present, the resulting credentials are
used. If the credentials are invalid, an error is returned.
1. Default instance credentials. When no environment variable is present, the
default service account credentials are used. This is useful when running Vault
on [Google Compute Engine][gce] or [Google Kubernetes Engine][gke]
For more information on service accounts, please see the [Google Cloud Service
Accounts documentation][service-accounts].
To use this storage backend, the service account must have the following
minimum scope(s):
```text
https://www.googleapis.com/auth/cloud-platform
```
### Required Permissions
The credentials given to Vault must have the following permissions:
```text
# Service Account + Key Admin
iam.serviceAccounts.create
iam.serviceAccounts.delete
iam.serviceAccounts.get
iam.serviceAccounts.list
iam.serviceAccounts.update
iam.serviceAccountKeys.create
iam.serviceAccountKeys.delete
iam.serviceAccountKeys.get
iam.serviceAccountKeys.list
# IAM Policy Changes
<service>.<resource>.getIamPolicy
<service>.<resource>.setIamPolicy
```
where `<service>` and `<resource>` correspond to permissions which will be
granted, for example:
```text
# Projects
resourcemanager.projects.getIamPolicy
resourcemanager.projects.setIamPolicy
# All compute
compute.*.getIamPolicy
compute.*.setIamPolicy
```
You can either:
- Create a [custom role][custom-roles] using these permissions, and assign this
role at a project-level
- Assign the set of roles required to get resource-specific
`getIamPolicy/setIamPolicy` permissions. At a minimum you will need to assign
`roles/iam.serviceAccountAdmin` and `roles/iam.serviceAccountKeyAdmin` so
Vault can manage service accounts and keys.
### Root Credential Rotation
If the mount is configured with credentials directly, the credential's key may be
rotated to a Vault-generated value that is not accessible by the operator. For more
details on this operation, please see the
[Root Credential Rotation](/api/secret/gcp#rotate-root-credentials) API docs.
## Things to Note
### Access Tokens vs. Service Account Keys
Advantages of `access_tokens`:
- Can generate infinite number of tokens per roleset
Disadvantages of `access_tokens`:
- Cannot be used with some client libraries or tools
- Have a static life-time of 1 hr that cannot be modified, revoked, or extended.
Advantages of `service_account_keys`:
- Controllable life-time through Vault, allowing for longer access
- Can be used by all normal GCP tooling
Disadvantages of `service_account_keys`:
- Infinite lifetime in GCP (i.e. if they are not managed properly, leaked keys can live forever)
- Limited to 10 per roleset/service account.
When generating OAuth access tokens, Vault will still
generate a dedicated service account and key. This private key is stored in Vault
and is never accessible to other users, and the underlying key can
be rotated. See the [GCP API documentation][api] for more information on
rotation.
### Service Accounts are Tied to Rolesets
Service Accounts are created when the roleset is created (or updated) rather
than each time a secret is generated. This may be different from how other
secrets engines behave, but it is for good reasons:
- IAM Service Account creation and permission propagation can take up to 60
seconds to complete. By creating the Service Account in advance, we speed up
the timeliness of future operations and reduce the flakiness of automated
workflows.
- Each GCP project has a limit on the number of IAM Service Accounts. You can
[request additional quota][quotas]. The quota increase is processed by humans,
so it is best to request this additional quota in advance. This limit is
currently 100, **including system-managed Service Accounts**. If Service
Accounts were created per secret, this quota limit would reduce the number of
secrets that can be generated.
### Service Account Keys Quota Limits
GCP IAM has a hard limit (currently 10) on the number of Service Account keys.
Attempts to generate more keys will result in an error. If you find yourself
running into this limit, consider the following:
- Have shorter TTLs or revoke access earlier. If you are not using past Service
Account keys, consider rotating and freeing quota earlier.
- Create additional rolesets which share the same set of permissions. Additional
rolesets can be created with the same set of permissions. This will create a
new service account and increases the number of keys you can create.
- Where possible, use OAuth2 access tokens instead of Service Account keys.
### Resources Must Exist at Roleset Creation
Because we the bindings for the Service Account are set during roleset creation,
resources that do not exist will fail the `getIamPolicy` API call.
### Roleset Creation May Partially Fail
Every Service Account creation, key creation, and IAM policy change is a GCP API
call per resource. If an API call to one of these resources fails, the roleset
creation fails and Vault will attempt to rollback.
These rollbacks are API calls, so they may also fail. The secrets engine uses a
WAL to ensure that unused bindings are cleaned up. In the case of quota limits,
you may need to clean these up manually.
### Do Not Modify Vault-owned IAM Accounts
While Vault will initially create and assign permissions to IAM service
accounts, it is possible that an external user deletes or modifies this service
account. These changes are difficult to detect, and it is best to prevent this
type of modification through IAM permissions.
Vault roleset Service Accounts will have emails in the format:
```
vault<roleset-prefix>-<creation-unix-timestamp>@...
```
Communicate with your teams (or use IAM permissions) to not modify these
resources.
## Help &amp; Support
The Google Cloud Vault secrets engine is written as an external Vault plugin and
thus exists outside the main Vault repository. It is automatically bundled with
Vault releases, but the code is managed separately.
Please report issues, add feature requests, and submit contributions to the
[vault-plugin-secrets-gcp repo on GitHub][repo].
## API
The GCP secrets engine has a full HTTP API. Please see the [GCP secrets engine API docs][api]
for more details.
[api]: /api/secret/gcp
[cloud-creds]: https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application
[custom-roles]: https://cloud.google.com/iam/docs/creating-custom-roles
[gce]: https://cloud.google.com/compute/
[gke]: https://cloud.google.com/kubernetes-engine/
[iam-keys]: https://cloud.google.com/iam/docs/service-accounts#service_account_keys
[iam-roles]: https://cloud.google.com/iam/docs/understanding-roles
[predefined-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
[repo]: https://github.com/hashicorp/vault-plugin-secrets-gcp
[resource-name-full]: https://cloud.google.com/apis/design/resource_names#full_resource_name
[resource-name-relative]: https://cloud.google.com/apis/design/resource_names#relative_resource_name
[quotas]: https://cloud.google.com/compute/quotas
[service-accounts]: https://cloud.google.com/compute/docs/access/service-accounts
## Upgrade Guides
### Deprecation of Access Token Leases
~> **NOTE**: This only affects access tokens. There is no change to the `service_account_key` secret type
Previous versions of this secrets engine (Vault <= 0.11.1) created a lease for
each access token secret. We have removed them after discovering that these
tokens, specifically Google OAuth2 tokens for IAM service accounts, are
non-revocable and have a static 60 minute lifetime. To match the current
limitations of the GCP APIs, the secrets engine will no longer allow for
revocation or manage the token TTL - more specifically, **the access_token
response will no longer include `lease_id` or other lease information**. This
change does not reflect any change to the actual underlying OAuth tokens or GCP
service accounts.
To upgrade:
- Remove references from `lease_id`, `lease_duration` or other `lease_*`
attributes when reading responses for the access tokens secrets endpoint (i.e.
from `gcp/token/$roleset`). See the [documentation for access
tokens](#access-tokens) to see the new format for the response.
- Be aware of leftover leases from previous versions. While these old leases
will still be revocable, they will not actually invalidate their associated
access token, and that token will still be useable for up to one hour.
Reference in a new issue View git blame Copy permalink