open-vault/website/content/docs/internals/architecture.mdx

113 lines
5.8 KiB
Plaintext
Raw Normal View History

2015-04-08 19:17:03 +00:00
---
layout: docs
page_title: Architecture
description: Learn about the internal architecture of Vault.
2015-04-08 19:17:03 +00:00
---
# Architecture
2017-09-21 21:14:40 +00:00
Vault is a complex system that has many different pieces. To help both users and
developers of Vault build a mental model of how it works, this page documents
the system architecture.
2015-04-08 19:17:03 +00:00
2017-09-21 21:14:40 +00:00
~> **Advanced Topic!** This page covers technical details of Vault. You don't
need to understand these details to effectively use Vault. The details are
documented here for those who wish to learn about them without having to go
spelunking through the source code. However, if you're an operator of Vault, we
recommend learning about the architecture due to the importance of Vault in an
environment.
2015-04-08 19:17:03 +00:00
2015-04-08 22:36:55 +00:00
# High-Level Overview
A very high level overview of Vault looks like this:
New Docs Website (#5535) * conversion stage 1 * correct image paths * add sidebar title to frontmatter * docs/concepts and docs/internals * configuration docs and multi-level nav corrections * commands docs, index file corrections, small item nav correction * secrets converted * auth * add enterprise and agent docs * add extra dividers * secret section, wip * correct sidebar nav title in front matter for apu section, start working on api items * auth and backend, a couple directory structure fixes * remove old docs * intro side nav converted * reset sidebar styles, add hashi-global-styles * basic styling for nav sidebar * folder collapse functionality * patch up border length on last list item * wip restructure for content component * taking middleman hacking to the extreme, but its working * small css fix * add new mega nav * fix a small mistake from the rebase * fix a content resolution issue with middleman * title a couple missing docs pages * update deps, remove temporary markup * community page * footer to layout, community page css adjustments * wip downloads page * deps updated, downloads page ready * fix community page * homepage progress * add components, adjust spacing * docs and api landing pages * a bunch of fixes, add docs and api landing pages * update deps, add deploy scripts * add readme note * update deploy command * overview page, index title * Update doc fields Note this still requires the link fields to be populated -- this is solely related to copy on the description fields * Update api_basic_categories.yml Updated API category descriptions. Like the document descriptions you'll still need to update the link headers to the proper target pages. * Add bottom hero, adjust CSS, responsive friendly * Add mega nav title * homepage adjustments, asset boosts * small fixes * docs page styling fixes * meganav title * some category link corrections * Update API categories page updated to reflect the second level headings for api categories * Update docs_detailed_categories.yml Updated to represent the existing docs structure * Update docs_detailed_categories.yml * docs page data fix, extra operator page remove * api data fix * fix makefile * update deps, add product subnav to docs and api landing pages * Rearrange non-hands-on guides to _docs_ Since there is no place for these on learn.hashicorp, we'll put them under _docs_. * WIP Redirects for guides to docs * content and component updates * font weight hotfix, redirects * fix guides and intro sidenavs * fix some redirects * small style tweaks * Redirects to learn and internally to docs * Remove redirect to `/vault` * Remove `.html` from destination on redirects * fix incorrect index redirect * final touchups * address feedback from michell for makefile and product downloads
2018-10-19 15:40:11 +00:00
[![Architecture Overview](/img/layers.png)](/img/layers.png)
2015-04-08 22:36:55 +00:00
2017-09-21 21:14:40 +00:00
Let's begin to break down this picture. There is a clear separation of
components that are inside or outside of the security barrier. Only the storage
backend and the HTTP API are outside, all other components are inside the
barrier.
The storage backend is untrusted and is used to durably store encrypted data.
When the Vault server is started, it must be provided with a storage backend so
that data is available across restarts. The HTTP API similarly must be started
by the Vault server on start so that clients can interact with it.
Once started, the Vault is in a _sealed_ state. Before any operation can be
performed on the Vault it must be unsealed. This is done by providing the unseal
keys. When the Vault is initialized it generates an encryption key which is used
to protect all the data. That key is protected by a master key. By default,
Vault uses a technique known as [Shamir's secret sharing
algorithm](https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing) to split the
master key into 5 shares, any 3 of which are required to reconstruct the master
2015-04-08 22:36:55 +00:00
key.
New Docs Website (#5535) * conversion stage 1 * correct image paths * add sidebar title to frontmatter * docs/concepts and docs/internals * configuration docs and multi-level nav corrections * commands docs, index file corrections, small item nav correction * secrets converted * auth * add enterprise and agent docs * add extra dividers * secret section, wip * correct sidebar nav title in front matter for apu section, start working on api items * auth and backend, a couple directory structure fixes * remove old docs * intro side nav converted * reset sidebar styles, add hashi-global-styles * basic styling for nav sidebar * folder collapse functionality * patch up border length on last list item * wip restructure for content component * taking middleman hacking to the extreme, but its working * small css fix * add new mega nav * fix a small mistake from the rebase * fix a content resolution issue with middleman * title a couple missing docs pages * update deps, remove temporary markup * community page * footer to layout, community page css adjustments * wip downloads page * deps updated, downloads page ready * fix community page * homepage progress * add components, adjust spacing * docs and api landing pages * a bunch of fixes, add docs and api landing pages * update deps, add deploy scripts * add readme note * update deploy command * overview page, index title * Update doc fields Note this still requires the link fields to be populated -- this is solely related to copy on the description fields * Update api_basic_categories.yml Updated API category descriptions. Like the document descriptions you'll still need to update the link headers to the proper target pages. * Add bottom hero, adjust CSS, responsive friendly * Add mega nav title * homepage adjustments, asset boosts * small fixes * docs page styling fixes * meganav title * some category link corrections * Update API categories page updated to reflect the second level headings for api categories * Update docs_detailed_categories.yml Updated to represent the existing docs structure * Update docs_detailed_categories.yml * docs page data fix, extra operator page remove * api data fix * fix makefile * update deps, add product subnav to docs and api landing pages * Rearrange non-hands-on guides to _docs_ Since there is no place for these on learn.hashicorp, we'll put them under _docs_. * WIP Redirects for guides to docs * content and component updates * font weight hotfix, redirects * fix guides and intro sidenavs * fix some redirects * small style tweaks * Redirects to learn and internally to docs * Remove redirect to `/vault` * Remove `.html` from destination on redirects * fix incorrect index redirect * final touchups * address feedback from michell for makefile and product downloads
2018-10-19 15:40:11 +00:00
[![Vault Shamir Secret Sharing Algorithm](/img/vault-shamir-secret-sharing.svg)](/img/vault-shamir-secret-sharing.svg)
2015-04-08 22:36:55 +00:00
2017-09-21 21:14:40 +00:00
The number of shares and the minimum threshold required can both be specified.
Shamir's technique can be disabled, and the master key used directly for
unsealing. Once Vault retrieves the encryption key, it is able to decrypt the
data in the storage backend, and enters the _unsealed_ state. Once unsealed,
Vault loads all of the configured audit devices, auth methods, and secrets
engines.
2015-04-08 22:36:55 +00:00
2017-09-21 21:14:40 +00:00
The configuration of those audit devices, auth methods, and secrets engines must
be stored in Vault since they are security sensitive. Only users with the
correct permissions should be able to modify them, meaning they cannot be
specified outside of the barrier. By storing them in Vault, any changes to them
are protected by the ACL system and tracked by audit logs.
2015-04-08 22:36:55 +00:00
2017-09-21 21:14:40 +00:00
After the Vault is unsealed, requests can be processed from the HTTP API to the
Core. The core is used to manage the flow of requests through the system,
enforce ACLs, and ensure audit logging is done.
2015-04-08 22:36:55 +00:00
When a client first connects to Vault, it needs to authenticate. Vault provides
configurable auth methods providing flexibility in the authentication mechanism
2015-04-08 22:36:55 +00:00
used. Human friendly mechanisms such as username/password or GitHub might be
2017-09-21 21:14:40 +00:00
used for operators, while applications may use public/private keys or tokens to
authenticate. An authentication request flows through core and into an auth
method, which determines if the request is valid and returns a list of
associated policies.
Policies are just a named ACL rule. For example, the "root" policy is built-in
and permits access to all resources. You can create any number of named policies
2021-05-29 23:14:21 +00:00
with fine-grained control over paths. Vault operates exclusively in a allowed-access
2017-09-21 21:14:40 +00:00
mode, meaning that unless access is explicitly granted via a policy, the action
is not allowed. Since a user may have multiple policies associated, an action is
allowed if any policy permits it. Policies are stored and managed by an internal
policy store. This internal store is manipulated through the system backend,
which is always mounted at `sys/`.
2015-04-08 22:36:55 +00:00
Once authentication takes place and an auth method provides a set of applicable
2017-09-21 21:14:40 +00:00
policies, a new client token is generated and managed by the token store. This
client token is sent back to the client, and is used to make future requests.
This is similar to a cookie sent by a website after a user logs in. The client
token may have a lease associated with it depending on the auth method
configuration. This means the client token may need to be periodically renewed
to avoid invalidation.
Once authenticated, requests are made providing the client token. The token is
used to verify the client is authorized and to load the relevant policies. The
policies are used to authorize the client request. The request is then routed to
the secrets engine, which is processed depending on its type. If the secrets
engine returns a secret, the core registers it with the expiration manager and
attaches a lease ID. The lease ID is used by clients to renew or revoke their
secret. If a client allows the lease to expire, the expiration manager
automatically revokes the secret.
The core handles logging of requests and responses to the audit broker, which
fans the request out to all the configured audit devices. Outside of the request
flow, the core performs certain background activity. Lease management is
critical, as it allows expired client tokens or secrets to be revoked
automatically. Additionally, Vault handles certain partial failure cases by
using write ahead logging with a rollback manager. This is managed transparently
within the core and is not user visible.
2015-04-08 22:36:55 +00:00
# Getting in Depth
This has been a brief high-level overview of the architecture of Vault. There
are more details available for each of the sub-systems.
2017-09-21 21:14:40 +00:00
For other details, either consult the code, ask in IRC or reach out to the
mailing list.