--- layout: docs page_title: Upgrading to Vault 1.12.x - Guides description: |- This page contains the list of deprecations and important or breaking changes for Vault 1.12.x. Please read it carefully. --- # Overview This page contains the list of deprecations and important or breaking changes for Vault 1.12.x compared to 1.11. Please read it carefully. ## Changes ### Supported Storage Backends Vault Enterprise will now perform a supported storage check at startup. There is no impact on open-source Vault users. @include 'ent-supported-storage.mdx' @include 'consul-dataplane-upgrade-note.mdx' ### External Plugin Loading Vault 1.12.0 introduced a change to how external plugins are loaded. Prior to Vault 1.12.0 plugins were lazy loaded on startup. This means that plugin processes were killed after a successful mount and then respawned when a request is routed to them. Vault 1.12.0 introduced auto mutual TLS for secrets/auth plugins so we do not lazy load them on startup anymore. ## Known Issues ### Pinning to builtin plugin versions may cause failure on upgrade 1.12.0 introduced plugin versions, and with it, the ability to explicitly specify the builtin version of a plugin when mounting an auth, database or secrets plugin. For example, `vault auth enable -plugin-version=v1.12.0+builtin.vault approle`. If there are any mounts where the _builtin_ version was explicitly specified in this way, Vault may fail to start on upgrading to 1.12.1 due to the specified version no longer being available. To check whether a mount path is affected, read the tune information, or the database config. The affected plugins are `snowflake-database-plugin@v0.6.0+builtin` and any plugins with `+builtin.vault` metadata in their version. In this example, the first two mounts are affected because `plugin_version` is explicitly set and is one of the affected versions. The third mount is not affected because it only has `+builtin` metadata, and is not the Snowflake database plugin. All mounts where the version is omitted, or the plugin is external (regardless of whether the version is specified) are unaffected. -> **NOTE:** Make sure you use Vault CLI 1.12.0 or later to check mounts. ```shell-session $ vault read sys/auth/approle/tune Key Value --- ----- ... plugin_version v1.12.0+builtin.vault $ vault read database/config/snowflake Key Value --- ----- ... plugin_name snowflake-database-plugin plugin_version v0.6.0+builtin $ vault read sys/auth/kubernetes/tune Key Value --- ----- ... plugin_version v0.14.0+builtin ``` As it is not currently possible to unset the plugin version, there are 3 possible remediations if you have any affected mounts: * Upgrade Vault directly to 1.12.2 once released * Upgrade to an external version of the plugin before upgrading to 1.12.1; * Using the [tune API](/vault/api-docs/system/auth#tune-auth-method) for auth methods * Using the [tune API](/vault/api-docs/system/mounts#tune-mount-configuration) for secrets plugins * Or using the [configure connection](/vault/api-docs/secret/databases#configure-connection) API for database plugins * Unmount and remount the path without a version specified before upgrading to 1.12.1. **Note:** This will delete all data and leases associated with the mount. The bug was introduced by commit https://github.com/hashicorp/vault/commit/c36330f4c713b886a8a23c08cbbd862a7c530fc8. #### Impacted Versions Affects upgrading from 1.12.0 to 1.12.1. All other upgrade paths are unaffected. 1.12.2 will introduce a fix that enables upgrades from affected deployments of 1.12.0. ### Mounts associated with deprecated builtin plugins will result in core shutdown on upgrade As of 1.12.0 Standalone (logical) DB Engines and the AppId Auth Method have been marked with the `Pending Removal` status. Any attempt to unseal Vault with mounts backed by one of these builtin plugins will result in an immediate shutdown of the Vault core. -> **NOTE** In the event that an external plugin with the same name and type as a deprecated builtin is deregistered, any subsequent unseal of Vault will also result in a core shutdown. ```shell-session $ vault plugin register -sha256=c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1 auth app-id Success! Registered plugin: app-id $ vault auth enable -plugin-name=app-id plugin Success! Enabled app-id auth method at: app-id/ $ vault auth list -detailed app-id/ app-id auth_app-id_3a8f2e24 system system default-service replicated false false map[] n/a 0018263c-0d64-7a70-fd5c-50e05c5f5dc3 n/a n/a c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1 n/a $ vault plugin deregister auth app-id Success! Deregistered plugin (if it was registered): app-id $ vault plugin list -detailed | grep "app-id" app-id auth v1.12.0+builtin.vault pending removal ``` The remediation for affected mounts is to set the [VAULT_ALLOW_PENDING_REMOVAL_MOUNTS](/vault/docs/commands/server#vault_allow_pending_removal_mounts) environment variable and replace any `Pending Removal` feature with the [preferred alternative feature](/vault/docs/deprecation/faq#q-what-should-i-do-if-i-use-mount-filters-appid-or-any-of-the-standalone-db-engines). For more information on the phases of deprecation, see the [Deprecation Notices FAQ](/vault/docs/deprecation/faq#q-what-are-the-phases-of-deprecation). #### Impacted Versions Affects upgrading from any version of Vault to 1.12.x. All other upgrade paths are unaffected. ### `vault plugin list` fails when audit logging is enabled If audit logging is enabled, Vault will fail to audit the response from any calls to the [`GET /v1/sys/plugins/catalog`](/vault/api-docs/system/plugins-catalog#list-plugins) endpoint, which causes the whole request to fail and return a 500 internal server error. From the CLI, this looks like the following: ```shell-session $ vault plugin list Error listing available plugins: data from server response is empty ``` It will produce errors in Vault Server's logs such as: ```text 2022-11-30T20:04:22.397Z [ERROR] audit: panic during logging: request_path=sys/plugins/catalog error="reflect: reflect.Value.Set using value obtained using unexported field" 2022-11-30T20:04:22.398Z [ERROR] core: failed to audit response: request_path=sys/plugins/catalog error= | 1 error occurred: | * panic generating audit log | ``` As a workaround, [listing plugins by type](/vault/api-docs/system/plugins-catalog#list-plugins-1) will succeed: * `vault list sys/plugins/catalog/auth` * `vault list sys/plugins/catalog/database` * `vault list sys/plugins/catalog/secret` The bug was introduced by commit https://github.com/hashicorp/vault/commit/76165052e54f884ed0aa2caa496083dc84ad1c19. #### Impacted Versions Affects versions 1.12.0, 1.12.1, and 1.12.2. A fix will be released in 1.12.3. ### PKI OCSP GET requests return malformed request responses If an OCSP GET request contains a '+' character, a malformed request response will be returned instead of properly processing the request due to a double decoding issue within the handler. As a workaround, OCSP POST requests can be used which are unaffected. #### Impacted Versions Affects version 1.12.3. A fix will be released in 1.12.4. @include 'tokenization-rotation-persistence.mdx' @include 'ocsp-redirect.mdx'