2017-03-08 02:47:23 +00:00
|
|
|
|
---
|
|
|
|
|
layout: "docs"
|
2018-03-30 16:36:37 +00:00
|
|
|
|
page_title: "Google Cloud Storage - Storage Backends - Configuration"
|
New Docs Website (#5535)
* conversion stage 1
* correct image paths
* add sidebar title to frontmatter
* docs/concepts and docs/internals
* configuration docs and multi-level nav corrections
* commands docs, index file corrections, small item nav correction
* secrets converted
* auth
* add enterprise and agent docs
* add extra dividers
* secret section, wip
* correct sidebar nav title in front matter for apu section, start working on api items
* auth and backend, a couple directory structure fixes
* remove old docs
* intro side nav converted
* reset sidebar styles, add hashi-global-styles
* basic styling for nav sidebar
* folder collapse functionality
* patch up border length on last list item
* wip restructure for content component
* taking middleman hacking to the extreme, but its working
* small css fix
* add new mega nav
* fix a small mistake from the rebase
* fix a content resolution issue with middleman
* title a couple missing docs pages
* update deps, remove temporary markup
* community page
* footer to layout, community page css adjustments
* wip downloads page
* deps updated, downloads page ready
* fix community page
* homepage progress
* add components, adjust spacing
* docs and api landing pages
* a bunch of fixes, add docs and api landing pages
* update deps, add deploy scripts
* add readme note
* update deploy command
* overview page, index title
* Update doc fields
Note this still requires the link fields to be populated -- this is solely related to copy on the description fields
* Update api_basic_categories.yml
Updated API category descriptions. Like the document descriptions you'll still need to update the link headers to the proper target pages.
* Add bottom hero, adjust CSS, responsive friendly
* Add mega nav title
* homepage adjustments, asset boosts
* small fixes
* docs page styling fixes
* meganav title
* some category link corrections
* Update API categories page
updated to reflect the second level headings for api categories
* Update docs_detailed_categories.yml
Updated to represent the existing docs structure
* Update docs_detailed_categories.yml
* docs page data fix, extra operator page remove
* api data fix
* fix makefile
* update deps, add product subnav to docs and api landing pages
* Rearrange non-hands-on guides to _docs_
Since there is no place for these on learn.hashicorp, we'll put them
under _docs_.
* WIP Redirects for guides to docs
* content and component updates
* font weight hotfix, redirects
* fix guides and intro sidenavs
* fix some redirects
* small style tweaks
* Redirects to learn and internally to docs
* Remove redirect to `/vault`
* Remove `.html` from destination on redirects
* fix incorrect index redirect
* final touchups
* address feedback from michell for makefile and product downloads
2018-10-19 15:40:11 +00:00
|
|
|
|
sidebar_title: "Google Cloud Storage"
|
2017-03-08 02:47:23 +00:00
|
|
|
|
sidebar_current: "docs-configuration-storage-google-cloud"
|
|
|
|
|
description: |-
|
2018-03-30 16:36:37 +00:00
|
|
|
|
The Google Cloud Storage storage backend is used to persist Vault's data in
|
|
|
|
|
Google Cloud Storage.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
---
|
|
|
|
|
|
2018-04-04 14:37:44 +00:00
|
|
|
|
# Google Cloud Storage Storage Backend
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
The Google Cloud Storage storage backend is used to persist Vault's data in
|
|
|
|
|
[Google Cloud Storage][gcs-docs].
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
- **High Availability** – the Google Cloud Storage storage backend supports high
|
|
|
|
|
availability. Because the Google Cloud Storage storage backend uses the system
|
|
|
|
|
time on the Vault node to acquire sessions, clock skew across Vault servers
|
|
|
|
|
can cause lock contention.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
- **Community Supported** – the Google Cloud Storage storage backend is
|
|
|
|
|
supported by the community. While it has undergone review by HashiCorp
|
|
|
|
|
employees, they may not be as knowledgeable about the technology. If you
|
|
|
|
|
encounter problems with them, you may be referred to the original author.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
|
|
|
|
```hcl
|
2017-03-08 14:17:00 +00:00
|
|
|
|
storage "gcs" {
|
2018-03-30 16:36:37 +00:00
|
|
|
|
bucket = "my-storage-bucket"
|
2017-03-08 02:47:23 +00:00
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
For more information on schemas or Google Cloud Storage, please see the [Google
|
|
|
|
|
Cloud Storage documentation][gcs-docs].
|
|
|
|
|
|
|
|
|
|
## `gcs` Setup
|
|
|
|
|
|
|
|
|
|
To use the Google Cloud Storage Vault storage backend, you must have a Google
|
|
|
|
|
Cloud Platform account with permissions to create Google Cloud Storage buckets.
|
|
|
|
|
|
|
|
|
|
To use the Google Cloud Storage Vault storage backend, you must have a Google
|
|
|
|
|
Cloud Platform account. Either using the API or web interface, create a bucket
|
|
|
|
|
using the [`gsutil`][cloud-sdk] command. Bucket names must be globally unique
|
|
|
|
|
across all of Google Cloud, so choose a unique name:
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
$ gsutil mb gs://mycompany-vault-data
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Even though the data is encrypted in transit and at rest, be sure to set the
|
|
|
|
|
appropriate permissions on the bucket to limit exposure. You may want to create
|
|
|
|
|
a service account that limits Vault's interactions with Google Cloud to objects
|
|
|
|
|
in the storage bucket using IAM permissions.
|
|
|
|
|
|
|
|
|
|
Here is a sample [Google Cloud IAM][iam] policy that grants the proper
|
|
|
|
|
permissions to a [service account][service-accounts]. Be sure to replace the
|
|
|
|
|
value with the value for your service account.
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"bindings": [
|
|
|
|
|
{
|
|
|
|
|
"role": "roles/storage.objectAdmin",
|
|
|
|
|
"members": [
|
|
|
|
|
"serviceAccount:my-vault@gserviceaccount.com"
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
]
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Then give Vault the service account's credential file as a configuration option.
|
|
|
|
|
|
|
|
|
|
For more information on schemas or Google Cloud Storage, please see the [Google
|
|
|
|
|
Cloud Storage documentation][gcs-docs].
|
|
|
|
|
|
|
|
|
|
## `gcs` Authentication
|
|
|
|
|
|
|
|
|
|
The Google Cloud Storage Vault storage backend uses the official Google Cloud
|
|
|
|
|
Golang SDK. This means it supports the common ways of [providing credentials to
|
|
|
|
|
Google Cloud][cloud-creds].
|
|
|
|
|
|
|
|
|
|
1. The environment variable `GOOGLE_APPLICATION_CREDENTIALS`. This is specified
|
|
|
|
|
as the **path** to a Google Cloud credentials file, typically for a service
|
|
|
|
|
account. If this environment variable is present, the resulting credentials are
|
|
|
|
|
used. If the credentials are invalid, an error is returned.
|
|
|
|
|
|
|
|
|
|
1. Default instance credentials. When no environment variable is present, the
|
|
|
|
|
default service account credentials are used.
|
|
|
|
|
|
|
|
|
|
For more information on service accounts, please see the [Google Cloud Service
|
|
|
|
|
Accounts documentation][service-accounts].
|
|
|
|
|
|
|
|
|
|
To use this storage backend, the service account must have the following
|
|
|
|
|
minimum scope(s):
|
|
|
|
|
|
|
|
|
|
```text
|
|
|
|
|
https://www.googleapis.com/auth/devstorage.read_write
|
|
|
|
|
```
|
|
|
|
|
|
2017-03-08 02:47:23 +00:00
|
|
|
|
## `gcs` Parameters
|
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
- `bucket` `(string: <required>)` – Specifies the name of the bucket to use for
|
|
|
|
|
storage.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
- `chunk_size` `(string: "8192")` – Specifies the maximum size (in kilobytes) to
|
|
|
|
|
send in a single request. If set to 0, it will attempt to send the whole
|
|
|
|
|
object at once, but will not retry any failures. If you are not storing large
|
|
|
|
|
objects in Vault, it is recommended to set this to a low value (minimum is
|
|
|
|
|
256) since it will reduce the amount of memory Vault uses.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
- `max_parallel` `(int: 128)` - Specifies the maximum number of parallel
|
|
|
|
|
operations to take place.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
### High Availability Parameters
|
|
|
|
|
|
|
|
|
|
- `ha_enabled` `(string: "false")` - Specifies if high availability mode is
|
|
|
|
|
enabled. This is a boolean value, but it is specified as a string like "true"
|
|
|
|
|
or "false".
|
2018-03-05 13:32:48 +00:00
|
|
|
|
|
2017-03-08 02:47:23 +00:00
|
|
|
|
## `gcs` Examples
|
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
### High Availability
|
|
|
|
|
|
|
|
|
|
This example shows configuring Google Cloud Storage with high availability
|
|
|
|
|
enabled.
|
|
|
|
|
|
|
|
|
|
```hcl
|
|
|
|
|
api_addr = "https://vault-leader.my-company.internal"
|
|
|
|
|
|
|
|
|
|
storage "gcs" {
|
|
|
|
|
bucket = "mycompany-vault-data"
|
|
|
|
|
ha_enabled = "true"
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Custom Chunk Size
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
This example shows setting a custom chunk size for uploads. When uploading large
|
|
|
|
|
data to Vault, setting a lower number can reduce Vault's memory consumption, but
|
|
|
|
|
will increase the number of outbound requests.
|
2017-03-08 02:47:23 +00:00
|
|
|
|
|
|
|
|
|
```hcl
|
2017-03-08 14:17:00 +00:00
|
|
|
|
storage "gcs" {
|
2018-03-30 16:36:37 +00:00
|
|
|
|
bucket = "mycompany-vault-data"
|
|
|
|
|
chunk_size = "512"
|
2017-03-08 02:47:23 +00:00
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
2018-03-30 16:36:37 +00:00
|
|
|
|
[cloud-creds]: https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application
|
|
|
|
|
[cloud-sdk]: https://cloud.google.com/sdk/downloads
|
|
|
|
|
[gcs-docs]: https://cloud.google.com/storage/docs/
|
|
|
|
|
[iam]: https://cloud.google.com/iam/docs/
|
|
|
|
|
[service-accounts]: https://cloud.google.com/compute/docs/access/service-accounts
|