2015-04-08 19:17:03 +00:00
|
|
|
---
|
2020-01-18 00:18:09 +00:00
|
|
|
layout: docs
|
|
|
|
page_title: Architecture
|
|
|
|
description: Learn about the internal architecture of Vault.
|
2015-04-08 19:17:03 +00:00
|
|
|
---
|
|
|
|
|
|
|
|
# Architecture
|
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
Vault is a complex system that has many different pieces. To help both users and
|
|
|
|
developers of Vault build a mental model of how it works, this page documents
|
|
|
|
the system architecture.
|
2015-04-08 19:17:03 +00:00
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
~> **Advanced Topic!** This page covers technical details of Vault. You don't
|
|
|
|
need to understand these details to effectively use Vault. The details are
|
|
|
|
documented here for those who wish to learn about them without having to go
|
|
|
|
spelunking through the source code. However, if you're an operator of Vault, we
|
|
|
|
recommend learning about the architecture due to the importance of Vault in an
|
|
|
|
environment.
|
2015-04-08 19:17:03 +00:00
|
|
|
|
2015-04-08 22:36:55 +00:00
|
|
|
# High-Level Overview
|
|
|
|
|
|
|
|
A very high level overview of Vault looks like this:
|
|
|
|
|
New Docs Website (#5535)
* conversion stage 1
* correct image paths
* add sidebar title to frontmatter
* docs/concepts and docs/internals
* configuration docs and multi-level nav corrections
* commands docs, index file corrections, small item nav correction
* secrets converted
* auth
* add enterprise and agent docs
* add extra dividers
* secret section, wip
* correct sidebar nav title in front matter for apu section, start working on api items
* auth and backend, a couple directory structure fixes
* remove old docs
* intro side nav converted
* reset sidebar styles, add hashi-global-styles
* basic styling for nav sidebar
* folder collapse functionality
* patch up border length on last list item
* wip restructure for content component
* taking middleman hacking to the extreme, but its working
* small css fix
* add new mega nav
* fix a small mistake from the rebase
* fix a content resolution issue with middleman
* title a couple missing docs pages
* update deps, remove temporary markup
* community page
* footer to layout, community page css adjustments
* wip downloads page
* deps updated, downloads page ready
* fix community page
* homepage progress
* add components, adjust spacing
* docs and api landing pages
* a bunch of fixes, add docs and api landing pages
* update deps, add deploy scripts
* add readme note
* update deploy command
* overview page, index title
* Update doc fields
Note this still requires the link fields to be populated -- this is solely related to copy on the description fields
* Update api_basic_categories.yml
Updated API category descriptions. Like the document descriptions you'll still need to update the link headers to the proper target pages.
* Add bottom hero, adjust CSS, responsive friendly
* Add mega nav title
* homepage adjustments, asset boosts
* small fixes
* docs page styling fixes
* meganav title
* some category link corrections
* Update API categories page
updated to reflect the second level headings for api categories
* Update docs_detailed_categories.yml
Updated to represent the existing docs structure
* Update docs_detailed_categories.yml
* docs page data fix, extra operator page remove
* api data fix
* fix makefile
* update deps, add product subnav to docs and api landing pages
* Rearrange non-hands-on guides to _docs_
Since there is no place for these on learn.hashicorp, we'll put them
under _docs_.
* WIP Redirects for guides to docs
* content and component updates
* font weight hotfix, redirects
* fix guides and intro sidenavs
* fix some redirects
* small style tweaks
* Redirects to learn and internally to docs
* Remove redirect to `/vault`
* Remove `.html` from destination on redirects
* fix incorrect index redirect
* final touchups
* address feedback from michell for makefile and product downloads
2018-10-19 15:40:11 +00:00
|
|
|
[![Architecture Overview](/img/layers.png)](/img/layers.png)
|
2015-04-08 22:36:55 +00:00
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
Let's begin to break down this picture. There is a clear separation of
|
|
|
|
components that are inside or outside of the security barrier. Only the storage
|
|
|
|
backend and the HTTP API are outside, all other components are inside the
|
|
|
|
barrier.
|
|
|
|
|
|
|
|
The storage backend is untrusted and is used to durably store encrypted data.
|
|
|
|
When the Vault server is started, it must be provided with a storage backend so
|
|
|
|
that data is available across restarts. The HTTP API similarly must be started
|
|
|
|
by the Vault server on start so that clients can interact with it.
|
|
|
|
|
|
|
|
Once started, the Vault is in a _sealed_ state. Before any operation can be
|
|
|
|
performed on the Vault it must be unsealed. This is done by providing the unseal
|
|
|
|
keys. When the Vault is initialized it generates an encryption key which is used
|
|
|
|
to protect all the data. That key is protected by a master key. By default,
|
|
|
|
Vault uses a technique known as [Shamir's secret sharing
|
|
|
|
algorithm](https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing) to split the
|
|
|
|
master key into 5 shares, any 3 of which are required to reconstruct the master
|
2015-04-08 22:36:55 +00:00
|
|
|
key.
|
|
|
|
|
New Docs Website (#5535)
* conversion stage 1
* correct image paths
* add sidebar title to frontmatter
* docs/concepts and docs/internals
* configuration docs and multi-level nav corrections
* commands docs, index file corrections, small item nav correction
* secrets converted
* auth
* add enterprise and agent docs
* add extra dividers
* secret section, wip
* correct sidebar nav title in front matter for apu section, start working on api items
* auth and backend, a couple directory structure fixes
* remove old docs
* intro side nav converted
* reset sidebar styles, add hashi-global-styles
* basic styling for nav sidebar
* folder collapse functionality
* patch up border length on last list item
* wip restructure for content component
* taking middleman hacking to the extreme, but its working
* small css fix
* add new mega nav
* fix a small mistake from the rebase
* fix a content resolution issue with middleman
* title a couple missing docs pages
* update deps, remove temporary markup
* community page
* footer to layout, community page css adjustments
* wip downloads page
* deps updated, downloads page ready
* fix community page
* homepage progress
* add components, adjust spacing
* docs and api landing pages
* a bunch of fixes, add docs and api landing pages
* update deps, add deploy scripts
* add readme note
* update deploy command
* overview page, index title
* Update doc fields
Note this still requires the link fields to be populated -- this is solely related to copy on the description fields
* Update api_basic_categories.yml
Updated API category descriptions. Like the document descriptions you'll still need to update the link headers to the proper target pages.
* Add bottom hero, adjust CSS, responsive friendly
* Add mega nav title
* homepage adjustments, asset boosts
* small fixes
* docs page styling fixes
* meganav title
* some category link corrections
* Update API categories page
updated to reflect the second level headings for api categories
* Update docs_detailed_categories.yml
Updated to represent the existing docs structure
* Update docs_detailed_categories.yml
* docs page data fix, extra operator page remove
* api data fix
* fix makefile
* update deps, add product subnav to docs and api landing pages
* Rearrange non-hands-on guides to _docs_
Since there is no place for these on learn.hashicorp, we'll put them
under _docs_.
* WIP Redirects for guides to docs
* content and component updates
* font weight hotfix, redirects
* fix guides and intro sidenavs
* fix some redirects
* small style tweaks
* Redirects to learn and internally to docs
* Remove redirect to `/vault`
* Remove `.html` from destination on redirects
* fix incorrect index redirect
* final touchups
* address feedback from michell for makefile and product downloads
2018-10-19 15:40:11 +00:00
|
|
|
[![Vault Shamir Secret Sharing Algorithm](/img/vault-shamir-secret-sharing.svg)](/img/vault-shamir-secret-sharing.svg)
|
2015-04-08 22:36:55 +00:00
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
The number of shares and the minimum threshold required can both be specified.
|
|
|
|
Shamir's technique can be disabled, and the master key used directly for
|
|
|
|
unsealing. Once Vault retrieves the encryption key, it is able to decrypt the
|
|
|
|
data in the storage backend, and enters the _unsealed_ state. Once unsealed,
|
|
|
|
Vault loads all of the configured audit devices, auth methods, and secrets
|
|
|
|
engines.
|
2015-04-08 22:36:55 +00:00
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
The configuration of those audit devices, auth methods, and secrets engines must
|
|
|
|
be stored in Vault since they are security sensitive. Only users with the
|
|
|
|
correct permissions should be able to modify them, meaning they cannot be
|
|
|
|
specified outside of the barrier. By storing them in Vault, any changes to them
|
|
|
|
are protected by the ACL system and tracked by audit logs.
|
2015-04-08 22:36:55 +00:00
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
After the Vault is unsealed, requests can be processed from the HTTP API to the
|
|
|
|
Core. The core is used to manage the flow of requests through the system,
|
|
|
|
enforce ACLs, and ensure audit logging is done.
|
2015-04-08 22:36:55 +00:00
|
|
|
|
|
|
|
When a client first connects to Vault, it needs to authenticate. Vault provides
|
2017-09-13 01:48:52 +00:00
|
|
|
configurable auth methods providing flexibility in the authentication mechanism
|
2015-04-08 22:36:55 +00:00
|
|
|
used. Human friendly mechanisms such as username/password or GitHub might be
|
2017-09-21 21:14:40 +00:00
|
|
|
used for operators, while applications may use public/private keys or tokens to
|
|
|
|
authenticate. An authentication request flows through core and into an auth
|
|
|
|
method, which determines if the request is valid and returns a list of
|
|
|
|
associated policies.
|
|
|
|
|
|
|
|
Policies are just a named ACL rule. For example, the "root" policy is built-in
|
|
|
|
and permits access to all resources. You can create any number of named policies
|
2021-05-29 23:14:21 +00:00
|
|
|
with fine-grained control over paths. Vault operates exclusively in a allowed-access
|
2017-09-21 21:14:40 +00:00
|
|
|
mode, meaning that unless access is explicitly granted via a policy, the action
|
|
|
|
is not allowed. Since a user may have multiple policies associated, an action is
|
|
|
|
allowed if any policy permits it. Policies are stored and managed by an internal
|
|
|
|
policy store. This internal store is manipulated through the system backend,
|
|
|
|
which is always mounted at `sys/`.
|
2015-04-08 22:36:55 +00:00
|
|
|
|
2017-09-13 01:48:52 +00:00
|
|
|
Once authentication takes place and an auth method provides a set of applicable
|
2017-09-21 21:14:40 +00:00
|
|
|
policies, a new client token is generated and managed by the token store. This
|
|
|
|
client token is sent back to the client, and is used to make future requests.
|
|
|
|
This is similar to a cookie sent by a website after a user logs in. The client
|
|
|
|
token may have a lease associated with it depending on the auth method
|
|
|
|
configuration. This means the client token may need to be periodically renewed
|
|
|
|
to avoid invalidation.
|
|
|
|
|
|
|
|
Once authenticated, requests are made providing the client token. The token is
|
|
|
|
used to verify the client is authorized and to load the relevant policies. The
|
|
|
|
policies are used to authorize the client request. The request is then routed to
|
|
|
|
the secrets engine, which is processed depending on its type. If the secrets
|
|
|
|
engine returns a secret, the core registers it with the expiration manager and
|
|
|
|
attaches a lease ID. The lease ID is used by clients to renew or revoke their
|
|
|
|
secret. If a client allows the lease to expire, the expiration manager
|
|
|
|
automatically revokes the secret.
|
|
|
|
|
|
|
|
The core handles logging of requests and responses to the audit broker, which
|
|
|
|
fans the request out to all the configured audit devices. Outside of the request
|
|
|
|
flow, the core performs certain background activity. Lease management is
|
|
|
|
critical, as it allows expired client tokens or secrets to be revoked
|
|
|
|
automatically. Additionally, Vault handles certain partial failure cases by
|
|
|
|
using write ahead logging with a rollback manager. This is managed transparently
|
|
|
|
within the core and is not user visible.
|
2015-04-08 22:36:55 +00:00
|
|
|
|
|
|
|
# Getting in Depth
|
|
|
|
|
|
|
|
This has been a brief high-level overview of the architecture of Vault. There
|
|
|
|
are more details available for each of the sub-systems.
|
|
|
|
|
2017-09-21 21:14:40 +00:00
|
|
|
For other details, either consult the code, ask in IRC or reach out to the
|
|
|
|
mailing list.
|