--- layout: "docs" page_title: "GCP - Secrets Engines" sidebar_current: "docs-secrets-gcp" description: |- The GCP secrets engine for Vault generates service account keys and OAuth tokens dynamically based on IAM policies. --- # GCP Secrets Engine The Google Cloud Platform secrets engine dynamically generates IAM service account credentials based on IAM policies, allowing users to have access to GCP resources without having to manage a new identity (service account). ## Quick Links: * **[Things to Note](#things-to-note): If you are running into unexpected errors (or wish to avoid doing so), please read this section! In particular, if you have used other Vault secrets engines (i.e. the AWS engine), this covers important differences in behavior.** * Contributing/Issues: This engine was written as a Vault plugin and thus exists outside the main Vault repo. Please report issues, request features, or submit contributions to the [plugin-specific repo on Github](https://github.com/hashicorp/vault-plugin-secrets-gcp) (all are welcome!) * Docs Quick Links: * [GCP secrets engine HTTP API](/api/secret/gcp/index.html) * [Expected `roleset` bindings format](#roleset-bindings) ## Background Benefits of using this Vault secrets engine to manage IAM service accounts and credentials: * Automatic cleanup of long-lived IAM service account keys * Vault associates leases with generated IAM service account keys s.t. they will be automatically revoked when the lease expires. * Users don't need to create a new service-account per one-off or short-term access. * Multi-cloud applications: * Users authenticate to Vault using some central identity (LDAP, AppRole, etc) and can generate GCP credentials without having to create and manage a new service account for that user. ## Overview *Note*: The following docs assume that the secrets engine has been mounted at `gcp`, the default mount path. Adjust your calls accordingly if you mount the engine at a different path. All example calls in this doc will use the Vault CL tool - see [HTTP API](/api/secret/gcp/index.html) for possible API calls. ### Setup Initially, the secrets engine must be enabled in Vault. ```sh $ vault secrets enable gcp Success! Enabled the gcp secrets engine at: gcp/ ``` You will need to set up the secrets engine with: * **[Config](#config)**: General config, including credentials that Vault will need to make calls to GCP APIs (either explicitly or using Application Default Credentials), lease defaults, etc. Example: * **[Rolesets](#rolesets)**: Generated credentials will need to be associated with sets of [IAM roles](https://cloud.google.com/iam/docs/understanding-roles) on specific GCP resources. Rolesets have an associated `secret_type` that determines a secret type that can be generated under this role set. Example: ### Usage - Secret Generation Once the secrets engine has been set up, you can generate two types of secrets in Vault: * **[Access Tokens](#access-tokens)** (`secret_type`: `access_token`): This endpoint generates non-renewable OAuth2 access tokens with a lifetime of an hour. * **[Service Account Keys](#service-account-keys)** (`secret_type`: `service_account_key`): This endpoint generates long-lived [IAM service account keys](https://cloud.google.com/iam/docs/service-accounts#service_account_keys). Each secret is associated with a [Vault lease](/docs/concepts/lease.html) that can be revoked and possibly renewed (see lease docs for how to do so). On revoking a lease, the secret is no longer guaranteed to work. Each secret is generated under a specified role set, which determine which permissions (IAM roles) the generated credentials have on specific GCP resources. Note that a role set can only generate one type of secret, specified at role set creation as `secret_type` ## Config The `config/` endpoint is used to configure any information shared by the entire GCP secrets engine. You can Allowed Operations: `write`, `read` Examples: ```bash $ vault write gcp/config credentials="..." ttl=100 max_ttl=1000 $ vault read gcp/config ``` Parameters For Write: * `credentials` (`string:""`) - JSON credentials (either file contents or '@path/to/file'). See next sections to learn more about required permissions and alternative ways to provide these credentials. * `ttl` (`int: 0 || string:"0s"`) – Specifies default config TTL for long-lived credentials (i.e. service account keys). Accepts integer number of seconds or Go duration format string. * `max_ttl` (`int: 0 || string:"0s"`)– Specifies default config TTL for long-lived credentials (i.e. service account keys). Accepts integer number of seconds or Go duration format string. #### Passing Credentials To Vault If you would rather not pass the IAM credentials using the payload `credentials` parameter, there are multiple ways to pass IAM credentials to the Vault server. You can specify credentials in the following ways (given in order of evaluation): 1. Static credential JSON provided to the API as a payload (i.e. `credentials` parameter) 2. Credentials in the environment variables `GOOGLE_CREDENTIALS` or `GOOGLE_CLOUD_KEYFILE_JSON` 3. Parse JSON file ~/.gcp/credentials 4. [Google Application Default Credentials](https://cloud.google.com/docs/authentication/production) #### Required Permissions At present, this endpoint does not confirm that the provided GCP credentials are for existing IAM users or have valid permissions. However, they will need the following permissions: ``` // Service Account + Key Admin iam.serviceAccounts.create iam.serviceAccounts.delete iam.serviceAccounts.get iam.serviceAccounts.list iam.serviceAccountKeys.create iam.serviceAccountKeys.delete iam.serviceAccountKeys.get iam.serviceAccountKeys.list iam.serviceAccounts.update // IAM Policy Changes $service.$resource.getIamPolicy $service.$resource.setIamPolicy ``` where `$service.$resource` refers to the GCP service/resources you wish to assign credentials permissions on, e.g. `cloudresourcemanager.projects.get/setIamPolicy` for GCP projects You can either create a [custom role](https://cloud.google.com/iam/docs/creating-custom-roles) with these permissions and assign it at a project-level to an IAM service account for Vault to use, or assign `roles/iam.serviceAccountAdmin` and `roles/iam.serviceAccountKeyAdmin` to get the permissions other than `get/setIamPolicy` permissions, and assign custom roles or global roles that include `get/setIamPolicy` permissions for the resources that Vault will change policies for. ## Rolesets Rolesets determine the permissions that service account credentials (secrets) generated by Vault will have on given GCP resources. Endpoint: `roleset/` (`rolesets/` for list) Allowed Operations: `write`, `read`, `list`, `delete`, `rotate`, `rotate-key` ### Creating/Updating Rolesets Each roleset will have an associated service account that is *created when the role set is created or updated*. (see [things to note](#things-to-note) for background) Parameters For Write: * `name` (`string: `): Required. Name of the role. Cannot be updated. Given as part of path. * `secret_type` (`string: "access_token"`): Type of secret generated for this role set. Accepted values: `access_token`, `service_account_key`. Cannot be updated. * `project` (`string: `): Name of the GCP project that this roleset's service account will belong to. Cannot be updated. * `bindings` (`string: `): Bindings configuration string (expected HCL or JSON string, raw or base64-encoded) * `token_scopes` (`array: []`): List of OAuth scopes to assign to `access_token` secrets generated under this role set (`access_token` role sets only) Update is the same call but will error if non-updatable fields are given with different values. If you update a roleset's bindings, note this will effectively revoke any secrets generated under this roleset. Examples: ```bash # secret type `access_token` $ vault write gcp/roleset/my-token-roleset \ project="mygcpproject" \ secret_type="access_token" \ token_scopes="https://www.googleapis.com/auth/cloud-platform" bindings=@binds.hcl \ # secret type `service_account_key` $ vault write gcp/roleset/my-key-roleset \ project="mygcpproject" \ secret_type="service_account_key" \ bindings="" ``` #### Roleset Bindings The rolesets `binding` argument accepts bindings in the following format: ```hcl resource "path/to/my/resource" { roles = [ "roles/viewer", "projects/X/roles/myprojCustomRole" ] } resource "//service.googleapis.com/path/to/another/resource" { roles = [ "organizations/Y/roles/myprojCustomRole" ] } ``` The top-level blocks are `resource` blocks, defined as `resource "a/resource/name" {...}`. Each block define IAM policy information to be bound to this resource. The following resource path formats are supported: 1. Project-level Self-Link (Specify Service and Version) A URI with scheme and host. Generally the `self_link` attribute of some resource. Must be resource with parent project (i.e. relative resource name `projects/$PROJECT/...`). Examples: * Compute alpha zone: ``` https://www.googleapis.com/compute/alpha/projects/my-project/zones/us-central1-c ``` 2. [Full resource name](https://cloud.google.com/apis/design/resource_names#full_resource_name) (Specify Service): A scheme-less URI consisting of a DNS-compatible API service name and a resource path. The resource path is the relative resource name (see next). Use to specify service but use either the preferred service version or the only version for which this resource is IAM-enabled. Examples: * Compute snapshot: ``` //compute.googleapis.com/project/X/snapshots/Y ``` * Pubsub snapshot: ``` //pubsub.googleapis.com/project/X/snapshots/Y ``` 3. [Relative Resource Name](https://cloud.google.com/apis/design/resource_names#relative_resource_name): A [path-noscheme](https://tools.ietf.org/html/rfc3986#appendix-A) URI path. Use if version/service are apparent from resource type (or you want to use only the preferred version of the service). General format is resource name as accepted by the corresponding REST API, but we've added some exceptions (namely Storage). Examples: * Storage bucket object: ``` b/bucketname/o/objectname buckets/bucketname/o/objectname ``` * Pubsub Topic: ``` projects/X/topics/Y ``` Each `resource` block accepts the following arguments: * `roles`: An array of string names for [IAM roles](https://cloud.google.com/iam/docs/understanding-roles). Each string must be a global role name (`roles/roleFoo`), a project-level custom role (`projects/myproj/roles/roleFoo`) or an organization-level custom role (`organizations/myorg/roles/roleFoo`) You can provide this as a plaintext string blob, the base64-encoded version of this string, or using syntax `@path/to/bindings.hcl` to pass in a filename #### Creation Workflow: When an admin user creates a roleset or updates the role set bindings, Vault does the following: 1. Parses given bindings configuration file as described above. 2. Add [WAL](https://en.wikipedia.org/wiki/Write-ahead_logging) entries to clean-up: - Current service account, key, and bindings in roleset if updating and update succeeds - New service account, key, and bindings if create/update fails. 3. Attempt to create a new IAM service account. * The new service account email is `"vault$rolesetName-$timestamp@..."`, where the roleset name might be truncated to fit the IAM service account character limit) * The new service account display name will contain the Vault roleset it belongs to. 4. For each resource in the bindings a. call `getIamPolicy` method on the resource. b. For each role in the resource's role list, add a binding to this policy with the email of the new service account. { "role": "roles/theRoleToAssign", "member": "serviceAccount:$EMAIL" } c. call `setIamPolicy` method on the resource See [this section](#calling-iam-methods) to learn more about how we call IAM methods on various resources. **Note: Because the service account and policies are assigned now, the resources given must exist at roleset creation time.** 5. If the roleset generates access tokens (`secret_type = "access_token"`), create a new service account key for this service account. 6. Save the new role set. **Note:** If steps 2-6 fail, we return a error response and the new service account will not be saved. WAL entries will have been added s.t. any entities will eventually get cleaned up, but you may need to manually delete these if rollback fails and you want to free up quota immediately. After step 6, the create/update operation will return a success, but we have a last cleanup step: 7. Try to delete the old service account (and any keys) and remove its old IAM policy bindings. If any of these calls to GCP APIs fail, we added the WAL entries in step 2 so eventually these resources will get cleaned up. We will return a warning in the response. In addition, we will try to delete the WAL entries for the now in-use service account that we just created s.t. it does not attempt to clean up these entities later. #### WAL Cleanup As mentioned, Vault will create WAL entries, both for newly created service accounts/bindings, which may not get used if update fails, and for old service account bindings, which may need to be deleted if immediate cleanup fail. Each WAL entry contains the name of the role set that was under update/creation. The WAL cleanup functions work as follows: 1. Vault will attempt to get the role set (which may be missing because it was deleted or was never created). 2. For each type of WAL entry: * **Service Account**: We try to delete the service account saved in the WAL entry. * If the roleset is still using the service account: We do nothing and remove the WAL entry. This happens if we failed to delete a WAL entry for a successful operation, or we preemptively added a WAL entry to delete old service accounts for an update that ended up failing. * **Service Account Key** (for `access_token` rolesets) * If a key name is included in the WAL entry: * We are attempting to clean up a previously created and saved key after a role set update. * If the roleset still exists and uses the key, we do nothing and remove the WAL entry. Otherwise, we delete the key. * If no key name is included: * We are attempting to clean up newly created keys for failed roleset updates. Because the key name is generated by GCP, we do not know the key name when we create the WAL entry. * Instead of deleting one key, we list all the user-managed keys under this service account and delete any that are not being used by the current roleset, if it exists. Since this service account is being controlled by Vault and the secrets are access tokens, there should not be any other user-managed keys for this service account. * **IAM Resource-Policy Bindings**: * The entry contains the GCP resource name that may need policy bindings cleaned up, and the bound roles and service account email that needs to be cleaned up. We get the IAM policy, remove the necessary bindings, and set the IAM policy without these bindings. * If the roleset exists and is still using this service account email, we remove only IAM policy bindings not included in the current roleset's bindings. ### Other Roleset Operations #### Rotation If you want to reset the service account for a given roleset, or rotate the key used for `access_token` rolesets, the GCP secrets engine has two endpoints for this: Examples: ```bash $ vault write gcp/roleset/my-key-roleset/rotate \ $ vault write gcp/roleset/my-token-roleset/rotate $ vault write gcp/roleset/my-token-roleset/rotate-key # `access_token` only ``` The `rotate/` endpoint works similar to roleset update (i.e. see the create/update workflow above). It replaces the roleset's service account with a new service account and replaceds any bindings with the new email as the IAM policy binding member. This will in effect revoke any secrets generated under this roleset. The `rotate-key/` endpoint only applies to `access_token` rolesets and simply rotates the key saved in the roleset used to generate access tokens, while keeping the same service account. Note that rotating the service account (`/rotate`) for an `access_token` roleset will also effectively rotate the key. #### Read and Delete Examples: ```bash $ vault read gcp/roleset/my-token-roleset $ vault delete gcp/roleset/my-key-roleset ``` On read, you can get the associated service account, token scopes for `access_token` rolesets, and other information. On delete, the service account, bindings, and any keys associated with the roleset service account will be deleted. ## Generating Secrets ### Access Tokens You can only generate tokens under `access_token` rolesets. Example Call: ```bash $ vault read gcp/token/my-token-roleset Key Value --- ----- lease_id gcp/token/my-token-roleset/some-uuid lease_duration 59m59s lease_renewable false token ya29.c.Ell9... ``` This endpoint generates non-renewable OAuth2 access tokens with a lifetime of an hour. Vault will simply return an access token (as `token` in output) that can be used in calls to GCP APIs as part of an "Authorization: Bearer" header: ```bash $ curl -H "Authorization: Bearer $TOKEN" ... ``` ### Service Account Keys You can only generate tokens under `service_account_key` rolesets. Example Call: ```bash $ vault read gcp/key/my-key-roleset lease_id gcp/key/my-key-roleset/some-uuid lease_duration 768h lease_renewable true key_algorithm KEY_ALG_RSA_2048 key_type TYPE_GOOGLE_CREDENTIALS_FILE private_key_data ... ``` This endpoint generates [IAM service account keys](https://cloud.google.com/iam/docs/service-accounts#service_account_keys) associated with the role set's service account. These keys are by default long-lived in GCP, but Vault associates a lease (see output `lease_*` information). This lease can be renewed to extend the key lifetime, or revoked. When the lease is revoked, the service account key will be deleted (thus freeing up quota). As mentioned in [things to note](#things-to-note), there is a limit of 10 service account keys per account and thus a limit of 10 secrets per `service_account_key` roleset. Read this section to learn about how to get around this limit if you are running into issues. To learn how to use service account keys, see [calling Google APIs](https://developers.google.com/identity/protocols/OAuth2ServiceAccount#callinganapi), [how to authenticate in gcloud](https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account), or [how to authenticate in Google Cloud client libraries](https://cloud.google.com/docs/authentication/getting-started). ## Things to Note ### Access Tokens vs Service Account Keys While in general the IAM team prefers that you use short-term access tokens over long-lived, hard-to-manage service account keys to authenticate calls to GCP APIs, this is currently impractical as several Google tools require service account keys for authenticating calls (i.e. `gcloud` CLI tool, client libraries). Thus, while we default to roleset `secret_type` = `access_token`, we also offer the option to generate keys (set role set `secret_type` to `service_account_key`), but we caution that these keys should still be carefully monitored even if they have been associated with Vault leases. **NOTE**: `access_token` role sets will generate a service account key that is saved in Vault. Vault will be the only place this key can be accessed (i.e. console or API users cannot access the private key data) and this data will not be returned in output for reading the role set. However, **this key can and should be [rotated](#rotation) as necessary to avoid possible theft.** Note that rotation does effectively invalidate all secrets previously generated under the roleset. ### Service account are created **on roleset update/creation** rather than per secret. (**NOTE: This is different than AWS!!**) There are a couple of reasons why we want service account creation to happen during role-set creation/updates (i.e. during server set up) rather than on the fly, per secret generated: * IAM service account creation and permissions propagation can take up to a minute. Because the service account is created and permissions are assigned during set-up, secrets can be used immediately instead of after a possible delay. This can make automated workflows slightly less complicated/flaky. * **Service Account Quotas**: GCP projects by default have a limit on the number of IAM service accounts you can create (currently 100, *including system-managed service accounts*). You can [request additional quota](https://cloud.google.com/compute/quotas), which is better done during a set-up step. * If service accounts are created per secret, this limit would instead be on the number of issued secrets. However, there are a couple of caveats that come from generating per-role-set as well: * **Service account key limit**: Because GCP IAM has set a hard limit (currently 10) on the number of service account keys, attempts to generate more keys than the limit will result in an error - Vault is notified *on the fly* when a user tries to create a new secret and the GCP IAM service returns an error. Thus, if you are generating service account keys, *for each `service_account_key` role set, there is a hard limit of of 10 secrets (keys)** If you find yourself running into this limit, consider: * Having shorter TTLs or revoking access earlier: If you're not using service account keys created earlier, consider rotating and freeing quota earlier. * Creating more role sets with the same set of permissions: Additional role-sets can always be created with the same set of permissions, which adds service accounts and thus effectively increases the number of key secrets you can generate. * Using `access_token` role sets: If you need several, very short-term accesses to GCP resources, consider instead requesting `access_token` secrets which have no limit and are naturally short-lived. * **Resources must exist at role set creation time.** Because we set the bindings for the service account at this point, if the resource does not exist, calls to getIamPolicy will fail on the resource and the role set creation will fail. * **Role-set creation might take a while and partially fail.** Because every service account creation, key creation, and IAM policy change per resource is a GCP API call and we can't do transactions through GCP, if one call fails, the roleset creation fails and Vault will attempt to rollback all changes made. However, these rollback calls themselves are API calls that also might fail. We add [WAL entries](#wal-cleanup) to ensure that any unused entities or bindings get cleaned up eventually, but you may run into possible situations where you hit quota limits or have unused IAM service accounts that have not been cleaned up. If you manually do this cleanup, the WAL entry will just succeed without doing anything, so feel free to do this (carefully) if you need to immediately clean up issues. ### External Changes to Vault-owned IAM Accounts While Vault will initially create and assign permissions to IAM service accounts, it is possible that an external user deletes this service account and/or Vault-managed keys (for `access_token` role sets), or adjusts this service account's permissions. **We will deny secret generation in the first case until the role set has been rotated or updated, but the second case is hard to detect on the fly.** * If your credentials has unexpected permissions, consider the cases that the service account has been either assigned new external permissions or has possibly inherited permissions from a parent resource policy. * Consider just rotating this service account periodically anyways to avoid tampering. * In general, you should not be changing these service accounts via console/API if you are using Vault. Please warn your GCP project owners to avoid accidentally changing these Vault roleset service accounts. * Vault role set accounts have emails with the format `vault-@...`, where `roleset-prefix` is the roleset name, possibly truncated to fit the character limit on rolesets. The display name (description) will also have the full Vault roleset name. * If the service account/key has been deleted, you will need to regenerate the role set account/key using the `gcp/$roleset/rotate` or `gcp/$roleset/rotate-key` endpoints. Updates to the role set bindings will also trigger service account recreation. ### Calling IAM Methods An IAM-enabled resource (under an arbitrary GCP service) supports the following three IAM methods: * `getIamPolicy` * `setIamPolicy` * `testIamPermissions` In the case of this secrets engine, we need to call `getIamPolicy` and `setIamPolicy` on an arbitrary resource under an arbitrary service, which would be difficult using the [generated Go google APIs](https://github.com/google/google-api-go-client). Instead, we autogenerated a library, using the [Google API Discovery Service](https://developers.google.com/discovery/) to find IAM-enabled resources and configure HTTP calls on arbitrary services/resources for IAM. For each binding config resource block (with a resource name), we attempt to find the resource type based on the relative resource name and match it to a service config as seen in this [autogenerated config file](https://github.com/hashicorp/vault-plugin-secrets-gcp/blob/master/plugin/iamutil/iam_resources_generated.go) To re-generate this file, run: ``` go generate github.com/hashicorp/vault-plugin-secrets-gcp/plugin/iamutil ``` In general, we try to make it so you can specify the resource as given in the HTTP API URL (between base API URL and get/setIamPolicy suffix). For some possibly non-standard APIs, we have also added exceptions to try to reach something more standard; a notable current example is the Google Cloud Storage API, whose methods look like `https://www.googleapis.com/storage/v1/b/bucket/o/object` where we accept either `b/bucket/o/object` or `buckets/bucket/objects/object` as valid relative resource names. If you are having trouble during role set creation with errors suggesting the resource format is invalid or API calls are failing for a resource you know exists, please [report any issues](https://github.com/hashicorp/vault-plugin-secrets-gcp/issues) you run into. It could be that the API is a non-standard form or we need to re-generate our config file. ## API The GCP secrets engine has a full HTTP API. Please see the [GCP secrets engine API docs](/api/secret/gcp/index.html) for more details.