diff --git a/website/source/api/secret/gcp/index.html.md b/website/source/api/secret/gcp/index.html.md index 7979bd93b..db94ff6de 100644 --- a/website/source/api/secret/gcp/index.html.md +++ b/website/source/api/secret/gcp/index.html.md @@ -8,8 +8,8 @@ description: |- # GCP Secrets Engine (API) -This is the API documentation for the Vault GCP (Google Cloud Platform) -secrets engine. For general information about the usage and operation of +This is the API documentation for the Vault GCP (Google Cloud Platform) +secrets engine. For general information about the usage and operation of the GCP secrets engine, please see [these docs](/docs/secrets/gcp/index.html). This documentation assumes the GCP secrets engine is enabled at the `/gcp` path @@ -22,13 +22,13 @@ update your API calls accordingly. | :------- | :------------------------| :------------------------ | | `POST` | `/gcp/config` | `204 (empty body)` | -This endpoint configures shared information for the secrets engine. +This endpoint configures shared information for the secrets engine. ### Parameters - `credentials` (`string:""`) - JSON credentials (either file contents or '@path/to/file') - See docs for [alternative ways](/docs/secrets/gcp/index.html#passing-credentials-to-vault) - to pass in to this parameter, as well as the + See docs for [alternative ways](/docs/secrets/gcp/index.html#passing-credentials-to-vault) + to pass in to this parameter, as well as the [required permissions](/docs/secrets/gcp/index.html#required-permissions). - `ttl` (`int: 0 || string:"0s"`) – Specifies default config TTL for long-lived credentials @@ -36,7 +36,7 @@ This endpoint configures shared information for the secrets engine. - `max_ttl` (`int: 0 || string:"0s"`)– Specifies default config TTL for long-lived credentials (i.e. service account keys). Accepts integer number of seconds or Go duration format string.** - + ### Sample Payload ```json @@ -63,7 +63,7 @@ $ curl \ | :------- | :------------------------| :------------------------ | | `GET` | `/gcp/config` | `200 application/json` | -Credentials will be omitted from returned data. +Credentials will be omitted from returned data. ### Sample Request @@ -92,14 +92,17 @@ $ curl \ | `POST` | `/gcp/roleset/:name` | `204 (empty body)` | This method allows you to create a roleset or update an existing roleset. See [roleset docs](/docs/secrets/gcp/index.html#rolesets) for the GCP secrets backend -to learn more about what happens when you create or update a roleset. +to learn more about what happens when you create or update a roleset. +**If you update a roleset's bindings, this will effectively revoke any secrets +generated under this roleset.** + ### Parameters - `name` (`string: `): Required. Name of the role. Cannot be updated. - `secret_type` (`string: "access_token"`): Type of secret generated for this role set. Accepted values: `access_token`, `service_account_key`. Cannot be updated. -- `project` (`string: `): Name of the GCP project that this roleset's service account will belong to. Cannot be updated. +- `project` (`string: `): Name of the GCP project that this roleset's service account will belong to. Cannot be updated. - `bindings` (`string: `): Bindings configuration string (expects HCL or JSON format in raw or base64-encoded string) - `token_scopes` (`array: []`): List of OAuth scopes to assign to `access_token` secrets generated under this role set (`access_token` role sets only) @@ -110,8 +113,8 @@ to learn more about what happens when you create or update a roleset. "secret_type": "access_token", "project": "mygcpproject", "bindings": "...", - "token_scopes": [ - "https://www.googleapis.com/auth/cloud-platform", + "token_scopes": [ + "https://www.googleapis.com/auth/cloud-platform", "https://www.googleapis.com/auth/bigquery" ] } @@ -124,14 +127,14 @@ See [bindings format docs](/docs/secrets/gcp/index.html#roleset-bindings) for mo resource "project/mygcpproject" { roles = [ "roles/viewer" - ], + ], } resource "https://selflink/to/my/resource" { roles = [ "project/mygcpproject/roles/projcustomrole", "organizations/myorg/roles/orgcustomrole" - ], + ], } ``` @@ -151,10 +154,10 @@ $ curl \ | :------- | :--------------------------------| :--------------------- | | `POST` | `/gcp/roleset/:name/rotate` | `204 (empty body)`` | -This will rotate the service account this roleset uses to generate secrets. +This will rotate the service account this roleset uses to generate secrets. (this also replaces the key `access_token` roleset). This can be used to invalidate old secrets generated by the roleset or fix issues if a roleset's service account -(and/or keys) was changed outside of Vault (i.e. through GCP APIs/cloud console). +(and/or keys) was changed outside of Vault (i.e. through GCP APIs/cloud console). ### Sample Request @@ -210,7 +213,7 @@ $ curl \ "bindings": { "project/mygcpproject": [ "roles/viewer" - ], + ], "https://selflink/to/my/resource": [ "project/mygcpproject/roles/projcustomrole", "organizations/myorg/roles/orgcustomrole" @@ -264,9 +267,9 @@ $ curl \ | `GET` | `POST` | `/gcp/token/:roleset` | `200 application/json` | Generates an OAuth2 token with the scopes defined on the roleset. This OAuth access token can -be used in GCP API calls, e.g. `curl -H "Authorization: Bearer $TOKEN" ...` +be used in GCP API calls, e.g. `curl -H "Authorization: Bearer $TOKEN" ...` -Tokens are non-renewable and have a TTL of an hour by default. +Tokens are non-renewable and have a TTL of an hour by default. ### Parameters @@ -309,19 +312,19 @@ If using `GET` ('read'), the optional parameters will be set to their defaults. want to specify different values for these params. These keys are renewable and have TTL/max TTL as defined by either the backend config -or the system default if config was not defined. +or the system default if config was not defined. ### Parameters - `roleset` (`string:`): Name of an roleset with secret type `service_account_key` to generate key under. -- `key_algorithm` (`string:"KEY_ALG_RSA_2048"`): Key algorithm used to generate key. Defaults to 2k RSA key - You probably should not choose other values (i.e. 1k), but accepted values are +- `key_algorithm` (`string:"KEY_ALG_RSA_2048"`): Key algorithm used to generate key. Defaults to 2k RSA key + You probably should not choose other values (i.e. 1k), but accepted values are `enum(`[`ServiceAccountKeyAlgorithm`](https://cloud.google.com/iam/reference/rest/v1/projects.serviceAccounts.keys#ServiceAccountKeyAlgorithm)`)` - `key_type` (`string:"TYPE_GOOGLE_CREDENTIALS_FILE`): Private key type to generate. Defaults to JSON credentials file. Accepted values are `enum(`[`ServiceAccountPrivateKeyType`](https://cloud.google.com/iam/reference/rest/v1/projects.serviceAccounts.keys#ServiceAccountPrivateKeyType)`)` -### Sample Payload +### Sample Payload ```json { "key_algorithm": "TYPE_GOOGLE_CREDENTIALS_FILE", @@ -357,7 +360,7 @@ $ curl \ "data": { "private_key_data":"", "key_algorithm": "TYPE_GOOGLE_CREDENTIALS_FILE", - "key_type": "KEY_ALG_RSA_2048" + "key_type": "KEY_ALG_RSA_2048" }, "wrap_info": null, "warnings": null, @@ -367,4 +370,4 @@ $ curl \ ## Revoking/Renewing Secrets -See docs on how to [renew](/api/system/leases.html#renew-lease) and [revoke](/api/system/leases.html#revoke-lease) leases. \ No newline at end of file +See docs on how to [renew](/api/system/leases.html#renew-lease) and [revoke](/api/system/leases.html#revoke-lease) leases. diff --git a/website/source/docs/secrets/gcp/index.html.md b/website/source/docs/secrets/gcp/index.html.md index 4d95842a9..27fcad4aa 100644 --- a/website/source/docs/secrets/gcp/index.html.md +++ b/website/source/docs/secrets/gcp/index.html.md @@ -1,590 +1,429 @@ --- layout: "docs" -page_title: "GCP - Secrets Engines" +page_title: "Google Cloud - Secrets Engines" sidebar_current: "docs-secrets-gcp" description: |- - The GCP secrets engine for Vault generates service account keys and OAuth tokens dynamically based - on IAM policies. + The Google Cloud Vault secrets engine dynamically generates Google Cloud + service account keys and OAuth tokens based on IAM policies. --- -# GCP Secrets Engine +# Google Cloud Secrets Engine -The Google Cloud Platform secrets engine dynamically generates IAM service account -credentials based on IAM policies, allowing users to have access to GCP resources -without having to manage a new identity (service account). - -## Quick Links: -* **[Things to Note](#things-to-note): If you are running into unexpected errors - (or wish to avoid doing so), please read this section! In particular, if you have used other Vault secrets engines - (i.e. the AWS engine), this covers important differences in behavior.** -* Contributing/Issues: This engine was written as a Vault plugin and thus - exists outside the main Vault repo. Please report issues, request features, or - submit contributions to the [plugin-specific repo on Github](https://github.com/hashicorp/vault-plugin-secrets-gcp) - (all are welcome!) -* Docs Quick Links: - * [GCP secrets engine HTTP API](/api/secret/gcp/index.html) - * [Expected `roleset` bindings format](#roleset-bindings) +The Google Cloud Vault secrets engine dynamically generates Google Cloud service +account keys and OAuth tokens based on IAM policies. This enables users to gain +access to Google Cloud resources without needing to create or manage a dedicated +service account. +The benefits of using this secrets engine to manage Google Cloud IAM service accounts are: -## Background +- **Automatic cleanup of GCP IAM service account keys** - each Service Account + key is associated with a Vault lease. When the lease expires (either during + normal revocation or through early revocation), the service account key is + automatically revoked. -Benefits of using this Vault secrets engine to manage IAM service accounts and credentials: +- **Quick, short-term access** - users do not need to create new GCP Service + Accounts for short-term or one-off access (such as batch jobs or quick + introspection). -* Automatic cleanup of long-lived IAM service account keys - * Vault associates leases with generated IAM service account keys s.t. they will - be automatically revoked when the lease expires. -* Users don't need to create a new service-account per one-off or short-term access. -* Multi-cloud applications: - * Users authenticate to Vault using some central identity (LDAP, AppRole, etc) - and can generate GCP credentials without having to create and manage a new - service account for that user. +- **Multi-cloud and hybrid cloud applications** - users authenticate to Vault + using a central identity service (such as LDAP) and generate GCP credentials + without the need to create or manage a new Service Account for that user. +## Setup -## Overview +Most secrets engines must be configured in advance before they can perform their +functions. These steps are usually completed by an operator or configuration +management tool. -*Note*: The following docs assume that the secrets engine has been mounted at `gcp`, -the default mount path. Adjust your calls accordingly if you mount the engine at a -different path. All example calls in this doc will use the Vault CL tool - see [HTTP API](/api/secret/gcp/index.html) -for possible API calls. +1. Enable the Google Cloud secrets engine: -### Setup - -Initially, the secrets engine must be enabled in Vault. - -```sh -$ vault secrets enable gcp -Success! Enabled the gcp secrets engine at: gcp/ -``` - -You will need to set up the secrets engine with: - -* **[Config](#config)**: General config, including credentials that Vault will need to make calls - to GCP APIs (either explicitly or using Application Default Credentials), lease defaults, etc. - -* **[Rolesets](#rolesets)**: Generated credentials will need to be associated with sets of - [IAM roles](https://cloud.google.com/iam/docs/understanding-roles) on specific GCP resources. Rolesets - have an associated `secret_type` that determines a secret type that can be generated - under this role set. - -### Usage - Secret Generation - -Once the secrets engine has been set up, you can generate two types of secrets in Vault: - -* **[Access Tokens](#access-tokens)** (`secret_type`: `access_token`): - This endpoint generates non-renewable OAuth2 access tokens with a lifetime of an hour. - -* **[Service Account Keys](#service-account-keys)** (`secret_type`: `service_account_key`): This endpoint generates long-lived - [IAM service account keys](https://cloud.google.com/iam/docs/service-accounts#service_account_keys). - - -Each secret is associated with a [Vault lease](/docs/concepts/lease.html) that -can be revoked and possibly renewed (see lease docs for how to do so). On revoking -a lease, the secret is no longer guaranteed to work. - -Each secret is generated under a specified role set, which determine which permissions (IAM roles) -the generated credentials have on specific GCP resources. Note that a role set can only generate one -type of secret, specified at role set creation as `secret_type` - - -## Config - -The `config/` endpoint is used to configure any information shared by the -entire GCP secrets engine. You can - -Allowed Operations: `write`, `read` - -Examples: - -```bash -$ vault write gcp/config credentials="..." ttl=100 max_ttl=1000 -$ vault read gcp/config -``` - -Parameters For Write: - -* `credentials` (`string:""`) - JSON credentials (either file contents or '@path/to/file'). - See next sections to learn more about required permissions and alternative ways to provide these credentials. -* `ttl` (`int: 0 || string:"0s"`) – Specifies default config TTL for long-lived credentials - (i.e. service account keys). Accepts integer number of seconds or Go duration format string. -* `max_ttl` (`int: 0 || string:"0s"`)– Specifies default config TTL for long-lived credentials - (i.e. service account keys). Accepts integer number of seconds or Go duration format string. - -#### Passing Credentials To Vault -If you would rather not pass the IAM credentials using the payload `credentials` parameter, -there are multiple ways to pass IAM credentials to the Vault server. You can specify credentials -in the following ways (given in order of evaluation): - -1. Static credential JSON provided to the API as a payload (i.e. `credentials` parameter) -2. Credentials in the environment variables `GOOGLE_CREDENTIALS` or `GOOGLE_CLOUD_KEYFILE_JSON` -3. Parse JSON file ~/.gcp/credentials -4. [Google Application Default Credentials](https://cloud.google.com/docs/authentication/production) - - -#### Required Permissions - -At present, this endpoint does not confirm that the provided GCP credentials -are for existing IAM users or have valid permissions. However, they will need -the following permissions: - -``` -// Service Account + Key Admin - iam.serviceAccounts.create - iam.serviceAccounts.delete - iam.serviceAccounts.get - iam.serviceAccounts.list - iam.serviceAccountKeys.create - iam.serviceAccountKeys.delete - iam.serviceAccountKeys.get - iam.serviceAccountKeys.list - iam.serviceAccounts.update - -// IAM Policy Changes - $service.$resource.getIamPolicy - $service.$resource.setIamPolicy -``` - -where `$service.$resource` refers to the GCP service/resources you wish to -assign credentials permissions on, e.g. -`cloudresourcemanager.projects.get/setIamPolicy` for GCP projects - -You can either create a [custom role](https://cloud.google.com/iam/docs/creating-custom-roles) -with these permissions and assign it at a project-level to an IAM service account for Vault to use, or -assign `roles/iam.serviceAccountAdmin` and `roles/iam.serviceAccountKeyAdmin` to get the permissions other -than `get/setIamPolicy` permissions, and assign custom roles or global roles that include `get/setIamPolicy` -permissions for the resources that Vault will change policies for. - - -## Rolesets - -Rolesets determine the permissions that service account credentials (secrets) -generated by Vault will have on given GCP resources. - -Endpoint: `roleset/` (`rolesets/` for list) - -Allowed Operations: `write`, `read`, `list`, `delete`, `rotate`, `rotate-key` - -### Creating/Updating Rolesets -Each roleset will have an associated service account that is *created when the role set is created or updated*. - -(see [things to note](#things-to-note) for background) - -Parameters For Write: - -* `name` (`string: `): Required. Name of the role. Cannot be updated. Given as part of path. -* `secret_type` (`string: "access_token"`): Type of secret generated for this role set. - Accepted values: `access_token`, `service_account_key`. Cannot be updated. -* `project` (`string: `): Name of the GCP project that this roleset's service account will belong to. - Cannot be updated. -* `bindings` (`string: `): Bindings configuration string (expected HCL or JSON string, raw or base64-encoded) -* `token_scopes` (`array: []`): List of OAuth scopes to assign to `access_token` secrets generated under - this role set (`access_token` role sets only) - -Update is the same call but will error if non-updatable fields are given with different values. -If you update a roleset's bindings, note this will effectively revoke any secrets generated under -this roleset. - -Examples: - -```bash -# secret type `access_token` -$ vault write gcp/roleset/my-token-roleset \ - project="mygcpproject" \ - secret_type="access_token" \ - token_scopes="https://www.googleapis.com/auth/cloud-platform" - bindings=@binds.hcl \ - -# secret type `service_account_key` -$ vault write gcp/roleset/my-key-roleset \ - project="mygcpproject" \ - secret_type="service_account_key" \ - bindings="" -``` - -#### Roleset Bindings - -The rolesets `binding` argument accepts bindings in the following format: - -```hcl -resource "path/to/my/resource" { - roles = [ - "roles/viewer", - "projects/X/roles/myprojCustomRole" - ] -} - -resource "//service.googleapis.com/path/to/another/resource" { - roles = [ - "organizations/Y/roles/myprojCustomRole" - ] -} -``` - -The top-level blocks are `resource` blocks, defined as `resource "a/resource/name" {...}`. Each block -define IAM policy information to be bound to this resource. - -The following resource path formats are supported: - -1. Project-level Self-Link (Specify Service and Version) - A URI with scheme and host. Generally the `self_link` attribute of some resource. - Must be resource with parent project (i.e. relative resource name `projects/$PROJECT/...`). - Examples: - * Compute alpha zone: - - ``` - https://www.googleapis.com/compute/alpha/projects/my-project/zones/us-central1-c - ``` - -2. [Full resource name](https://cloud.google.com/apis/design/resource_names#full_resource_name) (Specify Service): - A scheme-less URI consisting of a DNS-compatible API service name and a resource path. - The resource path is the relative resource name (see next). Use to specify service but use - either the preferred service version or the only version for which this resource is IAM-enabled. - - Examples: - * Compute snapshot: - + ```text + $ vault secrets enable gcp + Success! Enabled the gcp secrets engine at: gcp/ ``` - //compute.googleapis.com/project/X/snapshots/Y - ``` - * Pubsub snapshot: - + + By default, the secrets engine will mount at the name of the engine. To + enable the secrets engine at a different path, use the `-path` argument. + +1. Configure the secrets engine with account credentials and optionally tune the +default TTLs: + + ```text + $ vault write gcp/config \ + credentials=@my-credentials.json \ + ttl=4h \ + max_ttl=30m + Success! Data written to: gcp/config ``` - //pubsub.googleapis.com/project/X/snapshots/Y + + If you are running Vault from inside [Google Compute Engine][gce] or [Google + Kubernetes Engine][gke], the instance or pod service account can be used in + place or specifying the credentials JSON file. For more information on authentication, see the [authentication section](#authentication) below. + +1. Configure a roleset. Rolesets determine the permissions that Service Account +credentials generated by Vault will have on GCP resources. + + To configure a roleset that generates OAuth2 access tokens (preferred): + + ```text + $ vault write gcp/roleset/my-token-roleset \ + project="my-project" \ + secret_type="access_token" \ + token_scopes="https://www.googleapis.com/auth/cloud-platform" \ + bindings=-<... +private_key_data ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsC... ``` -This endpoint generates [IAM service account keys](https://cloud.google.com/iam/docs/service-accounts#service_account_keys) -associated with the role set's service account. These keys are by default long-lived in GCP, but Vault -associates a lease (see output `lease_*` information). This lease can be renewed to extend the key lifetime, -or revoked. When the lease is revoked, the service account key will be deleted (thus freeing up quota). +This endpoint generates a new [GCP IAM service account key][iam-keys] associated +with the roleset's Service Account. When the lease expires (or is revoked +early), the Service Account key will be deleted. -As mentioned in [things to note](#things-to-note), there is a limit of 10 service account keys per -account and thus a limit of 10 secrets per `service_account_key` roleset. Read this section to learn about -how to get around this limit if you are running into issues. +**There is a default limit of 10 keys per Service Account.** For more +information on this limit and recommended mitigation, please see the [things to +note](#things-to-note) section below. -To learn how to use service account keys, see -[calling Google APIs](https://developers.google.com/identity/protocols/OAuth2ServiceAccount#callinganapi), -[how to authenticate in gcloud](https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account), or -[how to authenticate in Google Cloud client libraries](https://cloud.google.com/docs/authentication/getting-started). +## Roleset Bindings + +Roleset bindings define a list of resources and the associated IAM roles on that +resource. Roleset bindings are used as the `binding` argument when creating or +updating a rolset and are specified in the following format using HCL: + +```hcl +resource NAME { + roles = [ROLE, [ROLE...]] +} +``` + +For example: + +```hcl +resource "buckets/my-bucket" { + roles = [ + "roles/storage.objectAdmin", + "roles/storage.legacyBucketReader", + ] +} +``` + +The top-level `resource` block defines the resource or resource path for which +IAM policy information will be bound. The resource path may be specified in a +few different formats: + +- **Project-level self-link** - a URI with scheme and host, generally + corresponding to the `self_link` attribute of a resource in GCP. This must + include the resource nested in the parent project. + + ```text + # compute alpha zome + https://www.googleapis.com/compute/alpha/projects/my-project/zones/us-central1-c + ``` + +- **Full resource name** - a schema-less URI consisting of a DNS-compatible API + service name and resource path. See the [full resource name API + documentation][resource-name-full] for more information. + + ```text + # Compute snapshot + //compute.googleapis.com/project/my-project/snapshots/my-compute-snapshot + + # Pubsub snapshot + //pubsub.googleapis.com/project/my-project/snapshots/my-pubsub-snapshot + ``` + +- **Relative resource name** - A path-noscheme URI path, usually as accepted by + the API. Use this if the version or service are apparent from the resource + type. Please see the [relative resource name API + documentation][resource-name-relative] for more information. + + ```text + # Storage bucket objects + buckets/my-bucket + buckets/my-bucket/objects/my-object + + # PubSub topics + projects/my-project/topics/my-pubsub-topic + ``` + +The nested `roles` attribute is an array of strings names of [GCP IAM +roles][iam-roles]. The roles may be specified in the following formats: + +- **Global role name** - these are global roles built into Google Cloud. For the + full list of available roles, please see the [list of predefined GCP + roles][predefined-roles]. + + ```text + roles/viewer + roles/bigquery.user + roles/billing.admin + ``` + +- **Organization-level custom role** - these are roles that are created at the + organization level by organization owners. + + ```text + organizations/my-organization/roles/my-custom-role + ``` + + For more information, please see the documentation on [GCP custom + roles][custom-roles]. + +- **Project-level custom role** - these are roles that are created at a + per-project level by project owners. + + ```text + projects/my-project/roles/my-custom-role + ``` + + For more information, please see the documentation on [GCP custom + roles][custom-roles]. + + +## Authentication + +The Google Cloud Vault secrets backend uses the official Google Cloud Golang +SDK. This means it supports the common ways of [providing credentials to Google +Cloud][cloud-creds]. In addition to specifying `credentials` directly via Vault +configuration, you can also get configuration from the following values **on the +Vault server**: + +1. The environment variables `GOOGLE_APPLICATION_CREDENTIALS`. This is specified +as the **path** to a Google Cloud credentials file, typically for a service +account. If this environment variable is present, the resulting credentials are +used. If the credentials are invalid, an error is returned. + +1. Default instance credentials. When no environment variable is present, the +default service account credentials are used. This is useful when running Vault +on [Google Compute Engine][gce] or [Google Kubernetes Engine][gke] + +For more information on service accounts, please see the [Google Cloud Service +Accounts documentation][service-accounts]. + +To use this storage backend, the service account must have the following +minimum scope(s): + +```text +https://www.googleapis.com/auth/cloud-platform +https://www.googleapis.com/auth/iam +``` + +### Required Permissions + +The credentials given to Vault must have the following permissions: + +```text +# Service Account + Key Admin +iam.serviceAccounts.create +iam.serviceAccounts.delete +iam.serviceAccounts.get +iam.serviceAccounts.list +iam.serviceAccounts.update +iam.serviceAccountKeys.create +iam.serviceAccountKeys.delete +iam.serviceAccountKeys.get +iam.serviceAccountKeys.list + +# IAM Policy Changes +..getIamPolicy +..setIamPolicy +``` + +where `` and `` correspond to permissions which will be +granted, for example: + +```text +# Projects +resourcemanager.projects.getIamPolicy +resourcemanager.projects.setIamPolicy + +# All compute +compute.*.getIamPolicy +compute.*.setIamPolicy +``` + +You can either: + +- Create a [custom role][custom-roles] using these permissions, and assign this + role at a project-level + +- Assign the set of roles required to get resource-specific + `getIamPolicy/setIamPolicy` permissions. At a minimum you will need to assign + `roles/iam.serviceAccountAdmin` and `roles/iam.serviceAccountKeyAdmin` so + Vault can manage service accounts and keys. ## Things to Note -### Access Tokens vs Service Account Keys +### Access Tokens vs. Service Account Keys -While in general the IAM team prefers that you use short-term access tokens over long-lived, hard-to-manage service account keys -to authenticate calls to GCP APIs, this is currently impractical as several Google tools -require service account keys for authenticating calls (i.e. `gcloud` CLI tool, client libraries). -Thus, while we default to roleset `secret_type` = `access_token`, we also offer the option -to generate keys (set role set `secret_type` to `service_account_key`), but we caution that -these keys should still be carefully monitored even if they have been associated with -Vault leases. +OAuth2 access tokens are **strongly preferred** to Service Account keys, but +some Google services do not yet support OAuth2 access tokens. Due to the quotas +imposed on Service Account Keys, these keys should be strictly monitored even if +they have been associated with a Vault lease. -**NOTE**: `access_token` role sets will generate a service account key that is saved in Vault. Vault will be the only -place this key can be accessed (i.e. console or API users cannot access the private key data) and this data will -not be returned in output for reading the role set. However, **this key can and should be [rotated](#rotation) as -necessary to avoid possible theft.** Note that rotation does effectively invalidate all secrets previously generated -under the roleset. +Even when using Vault to generate OAuth2 access tokens, Vault will still +generate a dedicated Service Account key whose private key is stored in Vault. +This private key is never accessible to other users, and the underlying key can +be rotated. See the [GCP API documentation][api] for more information on +rotation. -### Service account are created **on roleset update/creation** rather than per secret. +### Service Accounts are Tied to Rolesets -(**NOTE: This is different than AWS!!**) +Service Accounts are created when the roleset is created (or updated) rather +than each time a secret is generated. This may be different from how other +secrets engines behave, but it is for good reasons: -There are a couple of reasons why we want service account creation to happen during role-set creation/updates (i.e. -during server set up) rather than on the fly, per secret generated: +- IAM Service Account creation and permission propagation can take up to 60 + seconds to complete. By creating the Service Account in advance, we speed up + the timeliness of future operations and reduce the flakiness of automated + workflows. -* IAM service account creation and permissions propagation can take up to a minute. Because the service account is - created and permissions are assigned during set-up, secrets can be used immediately instead of after a - possible delay. This can make automated workflows slightly less complicated/flaky. -* **Service Account Quotas**: GCP projects by default have a limit on the number of IAM service accounts you can create - (currently 100, *including system-managed service accounts*). You can - [request additional quota](https://cloud.google.com/compute/quotas), which is better done - during a set-up step. - * If service accounts are created per secret, this limit would instead be on the number of issued secrets. - - -However, there are a couple of caveats that come from generating per-role-set as well: +- Each GCP project has a limit on the number of IAM Service Accounts. You can + [request additional quota][quotas]. The quota increase is processed by humans, + so it is best to request this additional quota in advance. This limit is + currently 100, **including system-managed Service Accounts**. If Service + Accounts were created per secret, this quota limit would reduce the number of + secrets that can be generated. -* **Service account key limit**: - Because GCP IAM has set a hard limit (currently 10) on the number of service - account keys, attempts to generate more keys than the limit will result in an error - - Vault is notified *on the fly* when a user tries to create a new secret - and the GCP IAM service returns an error. Thus, if you are generating service account - keys, *for each `service_account_key` role set, there is a hard limit of of - 10 secrets (keys)** - If you find yourself running into this limit, consider: - * Having shorter TTLs or revoking access earlier: If you're not using service - account keys created earlier, consider rotating and freeing quota earlier. - * Creating more role sets with the same set of permissions: Additional role-sets can - always be created with the same set of permissions, which adds service accounts and - thus effectively increases the number of key secrets you can generate. - * Using `access_token` role sets: If you need several, very short-term accesses to GCP - resources, consider instead requesting `access_token` secrets which have no limit and - are naturally short-lived. - -* **Resources must exist at role set creation time.** Because we set the bindings for the service account at this point, - if the resource does not exist, calls to getIamPolicy will fail on the resource and the role set creation will fail. - -* **Role-set creation might take a while and partially fail.** Because every service account creation, key creation, and - IAM policy change per resource is a GCP API call and we can't do transactions through GCP, if one call fails, the roleset - creation fails and Vault will attempt to rollback all changes made. However, these rollback calls themselves are - API calls that also might fail. We add [WAL entries](#wal-cleanup) to ensure that any unused entities or bindings - get cleaned up eventually, but you may run into possible situations where you hit quota limits or have unused - IAM service accounts that have not been cleaned up. If you manually do this cleanup, the WAL entry will just - succeed without doing anything, so feel free to do this (carefully) if you need to immediately clean up issues. +### Service Account Keys Have Limits -### External Changes to Vault-owned IAM Accounts -While Vault will initially create -and assign permissions to IAM service accounts, it is possible that an external user deletes this service account -and/or Vault-managed keys (for `access_token` role sets), or adjusts this service account's permissions. -**We will deny secret generation in the first case until the role set has been rotated or updated, -but the second case is hard to detect on the fly.** +GCP IAM has a hard limit (currently 10) on the number of Service Account keys. +Attempts to generate more keys will result in an error. If you find yourself +running into this limit, consider the follwing: -* If your credentials has unexpected permissions, consider the cases that the service account has been - either assigned new external permissions or has possibly inherited permissions from a parent resource policy. - * Consider just rotating this service account periodically anyways to avoid tampering. -* In general, you should not be changing these service accounts via console/API if you are using Vault. - Please warn your GCP project owners to avoid accidentally changing these Vault roleset service accounts. - * Vault role set accounts have emails with the format `vault-@...`, - where `roleset-prefix` is the roleset name, possibly truncated to fit the character limit on rolesets. The - display name (description) will also have the full Vault roleset name. -* If the service account/key has been deleted, you will need to regenerate the role set account/key using the - `gcp/$roleset/rotate` or `gcp/$roleset/rotate-key` endpoints. Updates to the role set bindings - will also trigger service account recreation. - +- Have shorter TTLs or revoke access earlier. If you are not using past Service + Account keys, consider rotating and freeing quota earlier. -### Calling IAM Methods +- Create additional rolesets which share the same set of permissions. Additional + rolesets can be created with the same set of permissions. This will create a + new service account and increases the number of keys you can create. -An IAM-enabled resource (under an arbitrary GCP service) supports the following three IAM methods: +- Where possible, use OAuth2 access tokens instead of Service Account keys. -* `getIamPolicy` -* `setIamPolicy` -* `testIamPermissions` +### Resources Must Exist at Roleset Creation -In the case of this secrets engine, we need to call `getIamPolicy` and `setIamPolicy` on -an arbitrary resource under an arbitrary service, which would be difficult using -the [generated Go google APIs](https://github.com/google/google-api-go-client). Instead, -we autogenerated a library, using the [Google API Discovery Service](https://developers.google.com/discovery/) -to find IAM-enabled resources and configure HTTP calls on arbitrary services/resources for IAM. +Because we the bindings for the Service Account are set during roleset creation, +resources that do not exist will fail the `getIamPolicy` API call. -For each binding config resource block (with a resource name), we attempt to find the resource type based on the -relative resource name and match it to a service config as seen in this -[autogenerated config file](https://github.com/hashicorp/vault-plugin-secrets-gcp/blob/master/plugin/iamutil/iam_resources_generated.go) +### Roleset Creation May Partially Fail -To re-generate this file, run: +Every Service Account creation, key creation, and IAM policy change is a GCP API +call per resource. If an API call to one of these resources fails, the roleset +creation fails and Vault will attempt to rollback. + +These rollbacks are API calls, so they may also fail. The secrets engine uses a +WAL to ensure that unused bindings are cleaned up. In the case of quota limits, +you may need to clean these up manually. + +### Do Not Modify Vault-owned IAM Accounts + +While Vault will initially create and assign permissions to IAM service +accounts, it is possible that an external user deletes or modifies this service +account. These changes are difficult to detect, and it is best to prevent this +type of modification through IAM permissions. + +Vault roleset Service Accounts will have emails in the format: ``` -go generate github.com/hashicorp/vault-plugin-secrets-gcp/plugin/iamutil +vault-@... ``` +Communicate with your teams (or use IAM permissions) to not modify these +resources. -In general, we try to make it so you can specify the resource as given in the HTTP API URL -(between base API URL and get/setIamPolicy suffix). For some possibly non-standard APIs, we have also - added exceptions to try to reach something more standard; a notable current example is the Google Cloud Storage API, - whose methods look like `https://www.googleapis.com/storage/v1/b/bucket/o/object` where we accept either - `b/bucket/o/object` or `buckets/bucket/objects/object` as valid relative resource names. -If you are having trouble during role set creation with errors suggesting the resource format is invalid or API calls -are failing for a resource you know exists, please [report any issues](https://github.com/hashicorp/vault-plugin-secrets-gcp/issues) -you run into. It could be that the API is a non-standard form or we need to re-generate our config file. +## Help & Support + +The Google Cloud Vault secrets engine is written as an external Vault plugin and +thus exists outside the main Vault repository. It is automatically bundled with +Vault releases, but the code is managed separately. + +Please report issues, add feature requests, and submit contributions to the +[vault-plugin-secrets-gcp repo on GitHub][repo]. + ## API -The GCP secrets engine has a full HTTP API. Please see the [GCP secrets engine API docs](/api/secret/gcp/index.html) +The GCP secrets engine has a full HTTP API. Please see the [GCP secrets engine API docs][api] for more details. + +[api]: (/api/secret/gcp/index.html) +[cloud-creds]: https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application +[custom-roles]: https://cloud.google.com/iam/docs/creating-custom-roles +[gce]: https://cloud.google.com/compute/ +[gke]: https://cloud.google.com/kubernetes-engine/ +[iam-keys]: https://cloud.google.com/iam/docs/service-accounts#service_account_keys +[iam-roles]: https://cloud.google.com/iam/docs/understanding-roles +[predefined-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles +[repo]: https://github.com/hashicorp/vault-plugin-secrets-gcp +[resource-name-full]: https://cloud.google.com/apis/design/resource_names#full_resource_name +[resource-name-relative]: https://cloud.google.com/apis/design/resource_names#relative_resource_name +[quotas]: https://cloud.google.com/compute/quotas +[service-accounts]: https://cloud.google.com/compute/docs/access/service-accounts