834ad52d68
* Upgrade guidance updates from VLT-172 Trying to clarify some upgrade questions. Learn update to follow in separate PR. * Apply suggestions from code review Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com> Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
151 lines
7.1 KiB
Plaintext
151 lines
7.1 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Upgrading Vault - Guides
|
|
description: |-
|
|
These are general upgrade instructions for Vault for both non-HA and HA
|
|
setups. Please ensure that you also read the version-specific upgrade notes.
|
|
---
|
|
|
|
# Upgrading Vault
|
|
|
|
These are general upgrade instructions for Vault for both non-HA and HA setups.
|
|
_Please ensure that you also read any version-specific upgrade notes which can be
|
|
found in the sidebar._
|
|
|
|
!> **IMPORTANT NOTE:** Always back up your data before upgrading! Vault does not
|
|
make backward-compatibility guarantees for its data store. Simply replacing the
|
|
newly-installed Vault binary with the previous version will not cleanly
|
|
downgrade Vault, as upgrades may perform changes to the underlying data
|
|
structure that make the data incompatible with a downgrade. If you need to roll
|
|
back to a previous version of Vault, you should roll back your data store as
|
|
well.
|
|
|
|
Vault upgrades are designed such that large jumps (ie 1.3.10 -> 1.7.x) are
|
|
supported. The upgrade notes for each intervening version must be reviewed. The
|
|
upgrade notes may describe additional steps or configuration to update before,
|
|
during, or after the upgrade.
|
|
|
|
## Learn
|
|
|
|
Refer to the [Vault Upgrade Standard Procedure](https://learn.hashicorp.com/tutorials/vault/sop-upgrade)
|
|
guide for step-by-step examples of upgrading Vault Enterprise.
|
|
|
|
## Agent
|
|
|
|
The Vault Agent is an API client of the Vault Server. Vault APIs are almost
|
|
always backwards compatible. When they are not, this is called out in the
|
|
upgrade guide for the new Vault version, and there is a lengthy deprecation
|
|
period. The Vault Agent version can lag behind the Vault Server version, though
|
|
we recommend keeping all Vault instances up to date with the most recent minor Vault version
|
|
to the extent possible.
|
|
|
|
## Testing the Upgrade
|
|
|
|
It's always a good idea to try to ensure that the upgrade will be successful in
|
|
your environment. The ideal way to do this is to take a snapshot of your data
|
|
and load it into a test cluster. However, if you are issuing secrets to third
|
|
party resources (cloud credentials, database credentials, etc.) ensure that you
|
|
do not allow external network connectivity during testing, in case credentials
|
|
expire. This prevents the test cluster from trying to revoke these resources
|
|
along with the non-test cluster.
|
|
|
|
## OSS to Enterprise Installations
|
|
|
|
Upgrading to Vault Enterprise installations follow the same steps as OSS upgrades except that the Vault Enterprise binary is to be used and the license file [applied](/api-docs/system/license#install-license), when applicable. The Enterprise binary and license file can be obtained through your HashiCorp sales team.
|
|
|
|
## Non-HA Installations
|
|
|
|
Upgrading non-HA installations of Vault is as simple as replacing the Vault
|
|
binary with the new version and restarting Vault. Any upgrade tasks that can be
|
|
performed for you will be taken care of when Vault is unsealed.
|
|
|
|
Always use `SIGINT` or `SIGTERM` to properly shut down Vault.
|
|
|
|
Be sure to also read and follow any instructions in the version-specific
|
|
upgrade notes.
|
|
|
|
## HA Installations
|
|
|
|
This is our recommended upgrade procedure, and the procedure we use internally
|
|
at HashiCorp. However, you should consider how to apply these steps to your
|
|
particular setup since HA setups can differ on whether a load balancer is in
|
|
use, what addresses clients are being given to connect to Vault (standby +
|
|
leader, leader-only, or discovered via service discovery), etc.
|
|
|
|
Whatever method you use, you should ensure that you never fail over from a
|
|
newer version of Vault to an older version. Our suggested procedure is designed
|
|
to prevent this.
|
|
|
|
Please note that Vault does not support true zero-downtime upgrades, but with
|
|
proper upgrade procedure the downtime should be very short (a few hundred
|
|
milliseconds to a second depending on how the speed of access to the storage
|
|
backend).
|
|
|
|
Perform these steps on each standby:
|
|
|
|
1. Properly shut down Vault on the standby node via `SIGINT` or `SIGTERM`
|
|
2. Replace the Vault binary with the new version; ensure that `mlock()` capability is added to the new binary with [setcap](/docs/configuration#disable_mlock)
|
|
3. Start the standby node
|
|
4. Unseal the standby node
|
|
5. Verify `vault status` shows correct Version and HA Mode is `standby`
|
|
6. Review the node's logs to ensure successful startup and unseal
|
|
|
|
At this point all standby nodes will be upgraded and ready to take over. The
|
|
upgrade will not be complete until one of the upgraded standby nodes takes over
|
|
active duty. To do this:
|
|
|
|
1. Properly shut down the remaining (active) node
|
|
|
|
~> **Note:** It is important that you shut the node down properly.
|
|
This will perform a step-down and release the HA lock, allowing a standby
|
|
node to take over with a very short delay.
|
|
If you kill Vault without letting it release the lock, a standby node will
|
|
not be able to take over until the lock's timeout period has expired. This
|
|
is backend-specific but could be ten seconds or more.
|
|
|
|
2. Replace the Vault binary with the new version; ensure that `mlock()` capability is added to the new binary with [setcap](/docs/configuration#disable_mlock)
|
|
4. Start the node
|
|
5. Unseal the node
|
|
5. Verify `vault status` shows correct Version and HA Mode is `standby`
|
|
7. Review the node's logs to ensure successful startup and unseal
|
|
|
|
Internal upgrade tasks will happen after one of the upgraded standby nodes
|
|
takes over active duty.
|
|
|
|
Be sure to also read and follow any instructions in the version-specific
|
|
upgrade notes.
|
|
|
|
## Replication Installations
|
|
|
|
-> **Note:** Prior to any upgrade, be sure to also read and follow any instructions in the version-specific
|
|
upgrade notes which are found in the navigation menu for this documentation.
|
|
|
|
Upgrading installations of Vault which participate in [Enterprise Replication](/docs/enterprise/replication) requires the following basic order of operations:
|
|
|
|
- **Upgrade the replication secondary instances first** using appropriate
|
|
guidance from the previous sections depending on whether each secondary
|
|
instance is Non-HA or HA
|
|
- Verify functionality of each secondary instance after upgrading
|
|
- When satisfied with functionality of upgraded secondary instances, upgrade
|
|
the primary instance
|
|
|
|
~> **Note:** It is not safe to replicate from a newer version of Vault to an
|
|
older version. When upgrading replicated clusters, ensure that upstream clusters
|
|
are always on older versions of Vault than downstream clusters.
|
|
|
|
Here is an example of upgrading four Vault replicated Vault clusters:
|
|
|
|
|
|
|
|
In the above scenario, the ideal upgrade procedure would be as follows,
|
|
verifying functionality after each cluster upgrade.
|
|
|
|
1. Upgrade Clusters B and D, using the HA upgrade process above. These clusters
|
|
have no downstream clusters, so they should be upgraded first, but the
|
|
ordering of B vs D does not matter.
|
|
2. Upgrade Cluster C, which now has an upgraded downstream cluster (Cluster D).
|
|
Because Cluster C is a cluster, it should also use the HA upgrade process.
|
|
3. Finally, upgrade Cluster A. All clusters downstream of A will already be
|
|
upgraded. It should be upgraded last, as it is a Performance Primary and a DR
|
|
Primary.
|