Commit graph

888 commits

Author SHA1 Message Date
Max Coulombe 6b2f4e5354
+ added redis elasticache as a built-in plugin (#17075)
* added redis elasticache as a built-in plugin
2022-09-09 16:16:30 -04:00
Mike Palmiotto 9849af8663
Add deprecation status to plugin api and cli (#17077)
* api: Add deprecation status to plugin endpoints

* cli: Add -detailed flag to `plugin list`

* docs: Update plugin list/info docs
2022-09-09 16:03:07 -04:00
Milena Zlaticanin 0977bd1ddc
Import Redis OSS database plugin into Vault (#17070)
* Import Redis OSS database plugin into Vault

* update the total number of db plugins

* small nit for testing

* adding changelog
2022-09-09 13:42:25 -05:00
Nick Cabatoff 3075c5bd65
Do not attempt to write a new TLS keyring at startup if raft is already setup (#17079) 2022-09-09 12:19:57 -04:00
claire bontempo 83fc61c16b
UI: OIDC Config for Vault as a provider (#17071)
* OIDC Config Routing (#16028)

* adds oidc config routes

* renames oidc applications route to clients

* UI/vault 6646/landing page (#16069)

* add to sidebar

* add landing image and text

* add permissions

* add permissions to permissions service

* remove comment

* fix.

* UI/OIDC models (#16091)

* add models and fix routing

* add ClientsCreate route

* remove form functions from client model

* update comment

* address comments, cleanup models

* add comment

* OIDC Adapters and Serializers (#16120)

* adds named-path base adapter

* adds oidc adapters with tests

* adds oidc serializers

* fixes issue with supported_scopes relationship in oidc provider model

* make radio card size flex (#16125)

* OIDC config details routes (#16126)

* adds details routes for oidc config resources

* adds details templates for oidc config resources

* OIDC parent route and index redirection (#16139)

* adds parent oidc route with header and adds redirection if clients have been created

* updates learn link

* adds findRecord override to named-path adapter (#16145)

* OIDC Scope Create/Edit View (#16174)

* adds oidc scope-form to create and edit views

* moves oidc header set logic from route to controller

* OIDC Scope Details View (#16191)

* adds oidc scope details view

* removes disabled arg from scope delete confirm action

* updates oidc scope template params link to use DocLink and adds success message on scope create success

* updates oidc scope delete confirm action copy

* adds oidc scopes list (#16196)

* UI/vault 6655/OIDC create view (#16331)

* setup header

* wip

* wip

* wip

* validations

* error validations

* cleanup

* wip

* fix error

* clean up

* handle modelValidations

* add documentation on the decorator

* remove spread attrs

* first test and some fixes

* halfway with test

* fix error where the data object was sending param entiyIds and not entity_ids

* validations or situation

* fix test

* small nit:

* test if this fixes the test

* fix

* cleanup

* nit

* Assignments Update/Edit View  (#16412)

* wip

* fix

* render search-select after promise is fulfilled

* add test coverage

Co-authored-by: clairebontempo@gmail.com <cbontempo@hashicorp.com>

* Added list view for keys (#16454)

* Added list view for providers (#16442)

* Added list view for providers

* Removed check for model data length

* Added new line at end of file

* Fixed linting issues causing ui tests to fail

* Added list view for application (#16469)

* UI/remove has many relationship (#16470)

* remove hasMany from models

* remove relationships from assignments create form

* update tests

* Assignment list view (#16340)

* inital setup

* handle default allow all

* add learn more link

* Fixed the default allow_all for assignment list view to match Figma design

* Fixed linting

* Fixed hbs file syntax

Co-authored-by: linda9379 <linda.jiang@hashicorp.com>

* configure mirage and helper (#16482)

* UI/OIDC client form (#16131)

* WIP client form

* wip

* still WIP

* fix form!;

* remove computeds, cache form attrs instead

* update scope form component name

* add white space validation

* add validations, cleanup

* add edit form

* fix link to in edit form

* disable edit form

* fix linkto

* wip/ search select filter

* WIP/search-select bug

* fix assignment save

* delete old modal js file

* glimmerize/create new search select modal component

* component cleanup

* fix bugginess

* fix search select and radio select action

* add tests

* revert some test changes

* oops, removed test tag

* add key list to response

* fix test

* move search select component to separate PR, revert changes

* one more revert

* remove oidc helper from this pr

* remove hasMany relationship

* minor cleanup

* update assignment form to use fallback

* fix allow_all appearing in dropdown on edit (#16508)

* UI/ OIDC Application (client) details view (#16507)

* fix test

* finish details page

* finish details view

* clean u[

* fix typo

* configure oidc mirage handler for tests

* remove params, add new route instead

* fix headers

* remove console.log

* remove controller/template reliance on tracked variable

* rename variable

* UI/Client route acceptance tests - fixed branch (#16654)

* WIP client route tests

* refactor client form so clientType is not edit-able

* fix ttl in client form

* wip// more acceptance tests and tags for hbs files

* fix typo

* fix syntax error

* finish tests

* fix client form test

* resolve commits

* update form test

* OIDC Assignments Details view. (#16511)

* setup

* cleanup

* view all fix

* wip setting up tabs

* wip

* revert to no queryParam or tabs

* add the read more component and styling

* rename folder

* cleanup

* fix

* UI/OIDC providers create/edit route (#16612)

* update to use DocLink component

* provider create form

* cleaup

* add formt est

* revert label text

* update doclink test

* disallow new scopes from ss

* fix test typo

* fix provider form flash message

* add period

* test new form field attr

* refactor form input

* fix edit portion of issuer field

* add test selector to new input field

* add comment

* Cleanup OIDC Config Mirage handler (#16674)

* cleaup mirage

* change to .then

* pull out into config file

* Scope acceptance tests (#16707)

* Started writing acceptance tests

* Added some more acceptance tests

* Added tags for hbs and more tests

* Modified variable names in scope form test

* Fixed tests and linting

* UI/OIDC Provider read view  (#16632)

* add providers/provider/client route

* provider details view

* add disabled button and tooltip for default

* add toolbar separators

* revert unrelated change

* query all client records and filter by allowed client id"

* refactor adapter to filter for clientId

* cleanup adapter method

* update test

* refactor test

* fix tests to accommodate for serializer change

* update empty state message

* fix linting

* metadata for client list view (#16725)

* Added metadata for list view in clients

* Fixed linting

* Fixed failing ui test

* fix scopes and clients tests (#16768)

* Initial fix of tests

* Fixed failing scopes and clients acceptance tests

* Fixed linting

* UI: Key create/edit form (#16729)

* add route models

* add forms

* add test

* remove helperText attr

* metadata for provider list view (#16738)

* Added meta-data for provider list view

* Added comment for serializer

* Fixed import path for scopes and clients acceptance test files

* UI/Add client ids to search select (#16744)

* WIP use clientID instead of name

* add client ids to search select

* remove provider form component changes

* fix search select on edit

* cleanup comments and method

* fix adapter query method

* clean up comments

* add test

* remove destructuring so linting passes

* fix tests

* add accidentally deleted param

* add clarifying comments

* cleanup

* change how shouldRenderName is set

* cleanup tests

* address comments

* OIDC Assignment Acceptance tests (#16741)

* test and fixes

* merge stuff

* fix

* fixes

* add waituntil

* inconsistent nav issue

* fixes

* blah

* UI/Key details view (#16776)

* add details view

* reformat model file

* todo for when listing applications

* add comment

* update key form with refactored search select

* add applications list

* update test

* update test

* add names to flash messages

* add rollbackAttributes to delete catch (#16796)

* UI: Checks if records exists before creating record when URL contains :name (#16823)

* check for record existing in createRecord

* use error banner instead of flash messages for forms

* add inline form message for validations

* add error count message to inlinealert

* add test for adapter

* add tests

* remove unused vars

* UI: Disable limiting clients when creating key, filter clients when editing (#16926)

* add tooltip to disabled radio button

* pass query object to search select

* update copy

* add comment

* cleanup console log and comment

* fix tests

* revert change because addressed in other pr

* fix diff

* fix test

* UI: Add redirect when last client is deleted  (#16927)

* afterModel redirect if no models exist

* fix test

* change space

* fix incorrect text

* UI:  Add InfoTooltip to selected 'ghost' client_ids (#16942)

* return option if undefined

* add info tooltip to search select

* change word

* add test

* UI: OIDC config keys acceptance tests (#16968)

* add keys test

* update other oidc tests

* remove-search select comment

* UI: Filter Client providers list view (#17027)

* pass param to adapter

* add test

* UI: OIDC Config Acceptance Tests (#17050)

* WIP/provider acceptance tests"

* WIP/this commit breaks lots of things

* fix tests

* update test selectors

* combine key and client tests

* cleanup clients and keys test

* finish tests

* small tidying

* UI: Remove trailing comma from scopes, provider details page (#17069)

* use info table row to cleanup scope logic

* infotableitemarray cleanup

* tidying

* add changelog

* teeny little empty state

* fix wildcard string helper not working

Co-authored-by: Jordan Reimer <zofskeez@gmail.com>
Co-authored-by: Angel Garbarino <Monkeychip@users.noreply.github.com>
Co-authored-by: Angel Garbarino <argarbarino@gmail.com>
Co-authored-by: linda9379 <57650314+linda9379@users.noreply.github.com>
Co-authored-by: linda9379 <linda.jiang@hashicorp.com>
2022-09-08 19:06:05 -06:00
Max Coulombe f9b5d1a563
Multiplexing opt out flag (#16972)
* added mplexing opt-out flag
2022-09-08 11:32:46 -04:00
Mike Palmiotto 403fdd77be
api: Add deprecation warnings to secrets/auth POST endpoints (#17058)
* api: Add deprecation warnings to endpoints
* Add changelog
2022-09-08 09:15:10 -04:00
akshya96 ab1e264f0b
updating go to 1.19.1 (#17053) 2022-09-07 16:13:33 -07:00
Jordan Reimer 5163bf2853
HCP Link Status (#16959)
* adds LinkStatus component to NavHeader to display banner with HCP link status

* adds changelog entry

* adds period to connected status message

* updates hcp link status to current cluster polling to automatically update state
2022-09-07 10:21:23 -06:00
Mike Palmiotto 991c574660
Mark database-specific secrets engines Pending Removal (#17038)
* plugins: Mark standalone database plugins Pending Removal
* Add changelog
2022-09-07 10:45:09 -04:00
Nick Cabatoff 7842b861b3
Make some activity log tests less flaky (#17028)
* OSS parts of ent #3157.  Some activity log tests were flaky because background workers could race with them; now we overload DisableTimers to stop some of them from running, and add some channels we can use to wait for others to complete before we start testing.

* Add CL
2022-09-07 09:06:15 -04:00
Mike Palmiotto bf744e3bde
Handle deprecated builtins (#17005)
* core: Handle deprecated mounts on enable and unseal
* changelog: Deprecation Status handling
* core: Add Pending Removal override var
* core: Add some documentation for Pending Removal override
2022-09-06 15:49:35 -04:00
Nick Cabatoff 5db952eada
autopilot: assume nodes we haven't received heartbeats from are running the same version as we are (#17019)
OSS parts of ent PR #3172: assume nodes we haven't received heartbeats from are running the same version as we are.  Failing to provide a version/upgrade_version will result in Autopilot (on ent) demoting those unversioned nodes to non-voters until we receive a heartbeat from them.
2022-09-06 14:49:04 -04:00
Max Coulombe 10121fed37
Prepare multiplexing support for database plugins (#16995)
* prepare multiplexing support for database plugins
2022-09-06 14:00:37 -04:00
Scott Miller 606edb66d6
Add support for a dedicated HMAC type in Transit. (#16668)
* Get import correct

* limits, docs

* changelog

* unit tests

* And fix import for hmac unit test

* typo

* Update website/content/api-docs/secret/transit.mdx

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

* Update builtin/logical/transit/path_keys.go

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

* Validate key sizes a bit more carefully

* Update sdk/helper/keysutil/policy.go

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
2022-09-06 10:17:58 -05:00
Violet Hynes 95b5449e8a
VAULT-6575 Vault agent respects retry config even with caching set (#16970)
* VAULT-6575 Vault agent respects retry config even with caching set

* VAULT-6575 Add changelog

* VAULT-6575 Change log levels
2022-09-06 10:35:54 -04:00
Remco Buddelmeijer b93d6e44e0
Add fields 'ttl' and 'num_uses' to SecretID generation. (#14474)
* Add fields 'ttl' and 'num_uses' to SecretID generation.

Add fields 'ttl' and 'num_uses' when generating/obtaining a SecretID.
Rather than just being able to use the Role's SecretID ttl and num uses. #14390

* Add secret_id_num_uses response field to generating SecretID

Add the response field secret_id_num_uses to the endpoints for generating
SecretIDs. Used in testing but also to supply the vendor with this variable.

* Add tests for new ttl and num_uses SecretID generation fields

Add tests to assert the new TTL and NumUses option in the SecretID entry.
Separate test for testing with just parameters vs a -force example.

* Patch up test for ttl and num_uses fields

* Add changelog entry for auth/approle 'ttl' and 'num_uses' fields

* Add fields to API Docs and AppRole Auth Docs example

* Correct error message for failing test on missing field.
Change the error message produced when a test fails due to a missing field.
Previous values did not map to correct fields.

* Remove unnecessary int cast to int "secret_id_num_uses" field.
Unnecessary cast to int where type already is int.

* Move numUses field check to after assignment.

* Remove metadata entry in sample payload to limit change to changes made.
Remove metadata entry in sample payload for custom-secret-id. The metadata was not
changed in the features pull request.

* Bind fields 'ttl' and 'num_uses' to role's configuration.

Rather than implicitly overriding, error when the ttl is lower than and the num
uses higher than the role's configuration. #14390

* Update changelog 14474 with a more detailed description.

More elaborate description for the changelog. Specifying the per-request based fields.

* Elaborate more on the bounds of the 'ttl' and 'num_uses' field.

Specify in both the api-docs and the CLI the limits of the fields.
Specify that the role's configuration is still the leading factor.

* Upper bound ttl with role secret id ttl

Upper bound ttl with role secret id ttl when creating a secret id
Adding test cases for infinite ttl and num uses
Adding test cases for negative ttl and num uses
Validation on infinite ttl and num uses

* Formatting issues. Removed unnecessary newline

* Update documentation for AppRole Secret ID and Role

Changed that TTL is not allowed to be shorter to longer

* Cleanup approle secret ID test and impl

* Define ttl and num_uses in every test

Define ttl and num_uses in every test despite them not being tested.
This is to ensure that no unexpected behaviour comes to mind.

* Rename test RoleSecretID -> RoleSecretIDWithoutFields

* Test secret id generation defaults to Role's config

Test secret id generation defaults to Role's configuration entries.

* Change finit -> finite

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>

* Rephrase comments to the correct validation check

* Rephrase role-secret-id option description

* Remove "default" incorrect statement about ttl

* Remove "default" incorrect statement about ttl for custom secret id

* Touch up approle.mdx to align more with path_role documentation

Co-authored-by: Remco Buddelmeijer <r.buddelmeijer@fullstaq.com>
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2022-09-02 09:29:59 -07:00
Austin Gebauer 1ea50db6c6
identity/oidc: adds claims_supported to discovery document (#16992)
* identity/oidc: adds claims_supported to discovery document

* adds changelog
2022-09-02 09:19:25 -07:00
Jordan Reimer d11b4995dd
LinkTo Transition Bug (#16983)
* removes on click modifier from LinkTo elements

* adds changelog

* reverts button changes and closes dropdown in next tick of runloop

* removes comment
2022-09-01 16:15:54 -06:00
Milena Zlaticanin 44d6a58eb5
update changelog from feature to improvement (#16986) 2022-09-01 15:50:51 -05:00
davidadeleon 24e346bf53
auth/token: Fix ignored parameter warnings for valid parameters on token create (#16938)
* Add fields to schema for relevant paths

* add changelog
2022-09-01 08:32:40 -04:00
Steven Clark cbbcffcc80
Update changelog for OCSP feature (#16969) 2022-08-31 16:55:22 -04:00
Mike Palmiotto 2c16be25e3
Add deprecation status to auth/secrets list (#16849)
* auth: Add Deprecation Status to auth list -detailed
* secrets: Add Deprecation Status to secrets list -detailed
* Add changelog entry for deprecation status list
2022-08-31 16:11:14 -04:00
Alexander Scheel f0a127487b
Add ability to cancel PKI tidy operations, pause between tidying certs (#16958)
* Allow tidy operations to be cancelled

When tidy operations take a long time to execute (and especially when
executing them automatically), having the ability to cancel them becomes
useful to reduce strain on Vault clusters (and let them be rescheduled
at a later time).

To this end, we add the /tidy-cancel write endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing auto-tidy synopsis / description

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add a pause duration between tidying certificates

By setting pause_duration, operators can have a little control over the
resource utilization of a tidy operation. While the list of certificates
remain in memory throughout the entire operation, a pause is added
between processing certificates and the revocation lock is released.
This allows other operations to occur during this gap and potentially
allows the tidy operation to consume less resources per unit of time
(due to the sleep -- though obviously consumes the same resources over
the time of the operation).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for cancellation, pause

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add API docs on pause_duration, /tidy-cancel

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add lock releasing around tidy pause

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Reset cancel guard, return errors

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-31 11:36:12 -07:00
Christopher Swenson 09ad6ab72c
Update mount table and CLI with plugin version for auth (#16856) 2022-08-31 19:23:05 +01:00
Hridoy Roy 8e7fec59ef
Load SSCT Generation Counter Upon DR Promotion [OSS] (#16956)
* port ssct bugfix to load epoch from storage

* changelog

* update changelog to be user-facing

* change 2 to two
2022-08-31 11:05:21 -07:00
Trishank Karthik Kuppusamy 303f59dce3
Allow configuring the possible salt lengths for RSA PSS signatures (#16549)
* accommodate salt lengths for RSA PSS

* address feedback

* generalise salt length to an int

* fix error reporting

* Revert "fix error reporting"

This reverts commit 8adfc15fe3303b8fdf9f094ea246945ab1364077.

* fix a faulty check

* check for min/max salt lengths

* stringly-typed HTTP param

* unit tests for sign/verify HTTP requests

also, add marshaling for both SDK and HTTP requests

* randomly sample valid salt length

* add changelog

* add documentation
2022-08-31 12:27:03 -04:00
Violet Hynes 2fb4ed211d
VAULT-7707 Add docs around making mass amounts of lease count quotas via automation (#16950)
* VAULT-7707 Add docs around making mass amounts of lease count quotas via automation

* VAULT-7707 Changelog

* VAULT-7707 add word

* VAULT-7707 Update some small wordings

* VAULT-7707 use a real em dash
2022-08-31 11:50:01 -04:00
Steven Clark b21e06b917
Add remove_roots_from_chain to sign and issue pki apis (#16935)
* Add remove_roots_from_chain flag to sign and issue pki apis

 - Add a new flag to allow end-users to control if we return the
   root/self-signed CA certificate within the list of certificates in
   ca_chain field on issue and sign api calls.

* Add cl

* PR feedback
2022-08-31 09:51:26 -04:00
Alexander Scheel a5fafd8163
Add ability to perform automatic tidy operations (#16900)
* Add ability to perform automatic tidy operations

This enables the PKI secrets engine to allow tidy to be started
periodically by the engine itself, avoiding the need for interaction.
This operation is disabled by default (to avoid load on clusters which
don't need tidy to be run) but can be enabled.

In particular, a default tidy configuration is written (via
/config/auto-tidy) which mirrors the options passed to /tidy. Two
additional parameters, enabled and interval, are accepted, allowing
auto-tidy to be enabled or disabled and controlling the interval
(between successful tidy runs) to attempt auto-tidy.

Notably, a manual execution of tidy will delay additional auto-tidy
operations. Status is reported via the existing /tidy-status endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on auto-tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for auto-tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prevent race during parallel testing

We modified the RollbackManager's execution window to allow more
faithful testing of the periodicFunc. However, the TestAutoRebuild and
the new TestAutoTidy would then race against each other for modifying
the period and creating their clusters (before resetting to the old
value).

This changeset adds a lock around this, preventing the races.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use tidyStatusLock to gate lastTidy time

This prevents a data race between the periodic func and the execution of
the running tidy.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add read lock around tidyStatus gauges

When reading from tidyStatus for computing gauges, since the underlying
values aren't atomics, we really should be gating these with a read lock
around the status access.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-30 15:45:54 -04:00
Ivan Buymov 99213c2616
Add retry policy and fix documentation for Cassandra storage backend (#10467)
* add simple_retry policy and initial_connection_timeout options,
fix docs for connection_timeout

* Cassandra: policy fix - added changelog.

Co-authored-by: Mehdi Ahmadi <aphorise@gmail.com>
2022-08-30 11:00:48 -07:00
Chelsea Shaw dd49b3bea3
UI: Use correct endpoint for force revoke prefix (#16930)
* Use correct endpoint for force revoke prefix

* Add changelog
2022-08-30 09:33:01 -07:00
John-Michael Faircloth b6c05fae33
feature: secrets/auth plugin multiplexing (#14946)
* enable registering backend muxed plugins in plugin catalog

* set the sysview on the pluginconfig to allow enabling secrets/auth plugins

* store backend instances in map

* store single implementations in the instances map

cleanup instance map and ensure we don't deadlock

* fix system backend unit tests

move GetMultiplexIDFromContext to pluginutil package

fix pluginutil test

fix dbplugin ut

* return error(s) if we can't get the plugin client

update comments

* refactor/move GetMultiplexIDFromContext test

* add changelog

* remove unnecessary field on pluginClient

* add unit tests to PluginCatalog for secrets/auth plugins

* fix comment

* return pluginClient from TestRunTestPlugin

* add multiplexed backend test

* honor metadatamode value in newbackend pluginconfig

* check that connection exists on cleanup

* add automtls to secrets/auth plugins

* don't remove apiclientmeta parsing

* use formatting directive for fmt.Errorf

* fix ut: remove tls provider func

* remove tlsproviderfunc from backend plugin tests

* use env var to prevent test plugin from running as a unit test

* WIP: remove lazy loading

* move non lazy loaded backend to new package

* use version wrapper for backend plugin factory

* remove backendVersionWrapper type

* implement getBackendPluginType for plugin catalog

* handle backend plugin v4 registration

* add plugin automtls env guard

* modify plugin factory to determine the backend to use

* remove old pluginsets from v5 and log pid in plugin catalog

* add reload mechanism via context

* readd v3 and v4 to pluginset

* call cleanup from reload if non-muxed

* move v5 backend code to new package

* use context reload for for ErrPluginShutdown case

* add wrapper on v5 backend

* fix run config UTs

* fix unit tests

- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests

* remove comment and update AutoMTLS field in test

* remove comment

* remove errwrap and unused context

* only support metadatamode false for v5 backend plugins

* update plugin catalog errors

* use const for env variables

* rename locks and remove unused

* remove unneeded nil check

* improvements based on staticcheck recommendations

* use const for single implementation string

* use const for context key

* use info default log level

* move pid to pluginClient struct

* remove v3 and v4 from multiplexed plugin set

* return from reload when non-multiplexed

* update automtls env string

* combine getBackend and getBrokeredClient

* update comments for plugin reload, Backend return val and log

* revert Backend return type

* allow non-muxed plugins to serve v5

* move v5 code to existing sdk plugin package

* do next export sdk fields now that we have removed extra plugin pkg

* set TLSProvider in ServeMultiplex for backwards compat

* use bool to flag multiplexing support on grpc backend server

* revert userpass main.go

* refactor plugin sdk

- update comments
- make use of multiplexing boolean and single implementation ID const

* update comment and use multierr

* attempt v4 if dispense fails on getPluginTypeForUnknown

* update comments on sdk plugin backend
2022-08-29 21:42:26 -05:00
Gabriel Santos ff5ff849fd
PKI - Honor header If-Modified-Since if present (#16249)
* honor header if-modified-since if present

* pathGetIssuerCRL first version

* check if modified since for CA endpoints

* fix date comparison for CA endpoints

* suggested changes and refactoring

* add writeIssuer to updateDefaultIssuerId and fix error

* Move methods out of storage.go into util.go

For the most part, these take a SC as param, but aren't directly storage
relevant operations. Move them out of storage.go as a result.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use UTC timezone for storage

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rework path_fetch for better if-modified-since handling

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Invalidate all issuers, CRLs on default write

When the default is updated, access under earlier timestamps will not
work as we're unclear if the timestamp is for this issuer or a previous
issuer. Thus, we need to invalidate the CRL and both issuers involved
(previous, next) by updating their LastModifiedTimes.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for If-Modified-Since

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly invalidate default issuer changes

When the default issuer changes, we'll have to mark the invalidation on
PR secondary clusters, so they know to update their CRL mapping as well.
The swapped issuers will have an updated modification time (which will
eventually replicate down and thus be correct), but the CRL modification
time is cluster-local information and thus won't be replicated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* make fmt

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor sendNotModifiedResponseIfNecessary

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on if-modified-since

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 15:28:47 -04:00
Alexander Scheel e03fb14be4
Support for generating Delta CRLs (#16773)
* Allow generation of up-to-date delta CRLs

While switching to periodic rebuilds of CRLs alleviates the constant
rebuild pressure on Vault during times of high revocation, the CRL
proper becomes stale. One response to this is to switch to OCSP, but not
every system has support for this. Additionally, OCSP usually requires
connectivity and isn't used to augment a pre-distributed CRL (and is
instead used independently).

By generating delta CRLs containing only new revocations, an existing
CRL can be supplemented with newer revocations without requiring Vault
to rebuild all complete CRLs. Admins can periodically fetch the delta
CRL and add it to the existing CRL and applications should be able to
support using serials from both.

Because delta CRLs are emptied when the next complete CRL is rebuilt, it
is important that applications fetch the delta CRL and correlate it to
their complete CRL; if their complete CRL is older than the delta CRL's
extension number, applications MUST fetch the newer complete CRL to
ensure they have a correct combination.

This modifies the revocation process and adds several new configuration
options, controlling whether Delta CRLs are enabled and when we'll
rebuild it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for delta CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on delta CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address review feedback: fix several bugs

Thanks Steve!

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly invoke periodic func on active nodes

We need to ensure we read the updated config (in case of OCSP request
handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL
re-building.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 11:37:09 -04:00
Violet Hynes 8c9c1d2b2a
VAULT-6433: Add namespace path to MFA read/list endpoints (#16911)
* VAULT-6433 Add namespace_path to MFA endpoints

* VAULT-6433 add changelog

* VAULT-6433 Return error in case of error

* VAULT-6433 Make logic a bit more concise
2022-08-29 09:11:25 -04:00
aphorise dbf762b488
Docs: API generate-recovery-token unhidden. (#16213) 2022-08-29 09:02:47 -04:00
aphorise 35650c9b96
Add systemd notify support to Agent. Resolves: #7028 (#9802)
Agent systemd notify added similar to Consul. Resolves: #7028
2022-08-29 08:18:47 -04:00
Alexander Scheel 43e722c69a
Let PKI tidy associate revoked certs with their issuers (#16871)
* Refactor tidy steps into two separate helpers

This refactors the tidy go routine into two separate helpers, making it
clear where the boundaries of each are: variables are passed into these
method and concerns are separated. As more operations are rolled into
tidy, we can continue adding more helpers as appropriate. Additionally,
as we move to make auto-tidy occur, we can use these as points to hook
into periodic tidying.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor revInfo checking to helper

This allows us to validate whether or not a revInfo entry contains a
presently valid issuer, from the existing mapping. Coupled with the
changeset to identify the issuer on revocation, we can begin adding
capabilities to tidy to update this association, decreasing CRL build
time and increasing the performance of OCSP.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor issuer fetching for revocation purposes

Revocation needs to gracefully handle using the old legacy cert bundle,
so fetching issuers (and parsing them) needs to be done slightly
differently than other places. Refactor this from revokeCert into a
common helper that can be used by tidy.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow tidy to associate revoked certs, issuers

When revoking a certificate, we need to associate the issuer that signed
its certificate back to the revInfo entry. Historically this was
performed during CRL building (and still remains so), but when running
without CRL building and with only OCSP, performance will degrade as the
issuer needs to be found each time.

Instead, allow the tidy operation to take over this role, allowing us to
increase the performance of OCSP and CRL in this scenario, by decoupling
issuer identification from CRL building in the ideal case.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for tidy updates

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on new tidy parameter, metrics

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor tidy config into shared struct

Finish adding metrics, status messages about new tidy operation.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-26 10:13:45 -07:00
claire bontempo 5fefab81f5
UI/OIDC auth bug for hcp namespace flag (#16886)
* revert to using paramsFor but add check for state having ns=

* revert to using paramsFor but add check for state having ns=

* cleanup hook

* add tests

* add changelog

* Test troubleshooting

* cleanup tests, use window stub correctly!

* add test for state param not existing at all

Co-authored-by: hashishaw <cshaw@hashicorp.com>
2022-08-26 10:04:01 -07:00
andrea-berling 7b7e590d3e
Update google-cloud-storage backend documentation (#14455)
* Update google-cloud-storage backend documentation

Add mentions the environment variables that can be used to configure the backend instead of using the stanza parameters

* Add changelog file

* Fix some typos

* Update website/content/docs/configuration/storage/google-cloud-storage.mdx

Commit suggestion #1

Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>

* Update website/content/docs/configuration/storage/google-cloud-storage.mdx

Commit suggestion #2

Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>

* Update website/content/docs/configuration/storage/google-cloud-storage.mdx

Commit suggestion #3

Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>

Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>
2022-08-26 09:59:40 -05:00
Jason O'Donnell e097e46a50
auth/kerberos: add config to include ldap groups in group alias (#16890) 2022-08-25 13:51:54 -07:00
Tom Proctor 4edf768f17
Version-aware plugin catalog (#16688)
Adds support for using semantic version information when registering
and managing plugins. New `detailed` field in the response data for listing
plugins and new `version` field in the response data for reading a
single plugin.
2022-08-25 21:31:42 +01:00
Mike Palmiotto c3de37e74b
Fix changelog type for registry deprecation status (#16889)
The previous changelog specified a type of `enhancement`, which is
invalid. It should be `improvement`.
2022-08-25 14:35:52 -04:00
Alexander Scheel f7bc1c8e3c
Cleanup changes around issuer revocation (#16874)
* Refactor CRL tests to use /sys/mounts

Thanks Steve for the approach! This also address nits from Kit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Skip CRL building steps when disabled

This skips a number of steps during CRL build when it is disabled (and
forceNew is not set). In particular, we avoid fetching issuers, we avoid
associating issuers with revocation entries (and building that in-memory
mapping), making CRL building more efficient.

This means that there'll again be very little overhead on clusters with
the CRL disabled.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prevent revoking roots from appearing on own CRLs

This change ensures that when marking a root as revoked, it no longer
appears on its own CRL. Very few clients support this event (as
generally only leaves/intermediates are checked for presence on a
parent's CRL) and it is technically undefined behavior (if the root is
revoked, its own CRL should be untrusted and thus including it on its
own CRL isn't a safe/correct distribution channel).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure stability of revInfo issuer identification

As mentioned by Kit, iterating through each revInfoEntry and associating
the first issuer which matches it can cause churn when many (equivalent)
issuers are in the system and issuers come and go (via CRLSigning usage,
which has been modified in this release as well). Because we'd not
include issuers without CRLSigning usage, we'd cause our verification
helper, isRevInfoIssuerValid, to think the issuer ID is no longer value
(when instead, it just lacks crlSigning bits).

We address this by pulling in all issuers we know of for the
identification. This allows us to keep valid-but-not-for-signing
issuers, and use other representatives of their identity set for
signing/building the CRL (if they are enabled for such usage).

As a side effect, we now no longer place these entries on the default
CRL in the event all issuers in the CRL set are without the usage.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

This is only for the last commit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-25 11:36:37 -04:00
Alexander Scheel 6089d2e247
Don't allow crl-signing issuer usage without CRLSign KeyUsage (#16865)
* Allow correct importing of certs without CRL KU

When Vault imports certificates without KU for CRLSign, we shouldn't
provision CRLUsage on the backing issuer; otherwise, we'll attempt to
build CRLs and Go will cause us to err out. This change makes it clear
(at issuer configuration time) that we can't possibly support this
operation and hopefully prevent users from running into the more cryptic
Go error.

Note that this does not apply for OCSP EKU: the EKU exists, per RFC 6960
Section 2.6 OCSP Signature Authority Delegation, to allow delegation of
OCSP signing to a child certificate. This EKU is not necessary on the
issuer itself, and generally assumes issuers are allowed to issue OCSP
responses regardless of KU/EKU.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add docs to clarify issue with import, CRL usage

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

* Add additional test assertion

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-24 07:45:54 -07:00
DevOps Rob 9e20e4128d
updated usage example (#9081)
* updated usage example

* Docs: updated examples with base64 - removed herestring for echo instead that's more simple.

* Docs: updated examples with base64 - removed herestring for echo instead that's more simple.

Co-authored-by: Mehdi Ahmadi <aphorise@gmail.com>
2022-08-24 09:03:30 -04:00
Hamid Ghaf c5df41e750
update to GO 1.18.5 (#16840)
* update to GO 1.18.5

* reverting a change

* feedback
2022-08-23 19:05:43 -04:00
davidadeleon 592949d7ba
command/debug: fix bug where monitor was not honoring configured duration (#16834)
* Use specific client with longer timeout for log debug target

* add changelog

* update changelog entry
2022-08-23 17:09:30 -04:00
Mike Palmiotto 6a438fd087
Vault 7133/registry status (#16846)
* plugins: Add Deprecation Status to builtinRegistry

* changelog: Deprecation Status method
2022-08-23 16:34:30 -04:00