* Regexp metacharacter `.` should be escaped when used literally
The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.
* Replace implementation of OpenAPI path translator with regexp AST-based one
* Add changelog
* Typo fix from PR review - thanks!
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Add comment based on review feedback
* Change style of error handling as suggested in code review
* Make a further tweak to the handling of the error case
* Add more tests, testing cases which fail with the previous implementation
* Resolve issue with a test, and improve comment
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Remove dynamic keys from SSH Secrets Engine
This removes the functionality of Vault creating keys and adding them to
the authorized keys file on hosts.
This functionality has been deprecated since Vault version 0.7.2.
The preferred alternative is to use the SSH CA method, which also allows
key generation but places limits on TTL and doesn't require Vault reach
out to provision each key on the specified host, making it much more
secure.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove dynamic ssh references from documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove dynamic key secret type entirely
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify changelog language
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add removal notice to the website
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Read total cert counts with atomic.LoadUint32(...)
When generating the tidy status, we read the values of two backend
atomics, b.certCount and b.revokedCertCount, without using the atomic
load operation. This resulted in a data race when the status was read
at the same time as an on-going tidy operation:
WARNING: DATA RACE
Write at 0x00c00c77680c by goroutine 90522:
sync/atomic.AddInt32()
/usr/local/go/src/runtime/race_amd64.s:281 +0xb
sync/atomic.AddUint32()
<autogenerated>:1 +0x1a
github.com/hashicorp/vault/builtin/logical/pki.(*backend).tidyStatusIncRevokedCertCount()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:1236 +0x107
github.com/hashicorp/vault/builtin/logical/pki.(*backend).doTidyRevocationStore()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:525 +0x1404
github.com/hashicorp/vault/builtin/logical/pki.(*backend).startTidyOperation.func1.1()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:290 +0x1a4
github.com/hashicorp/vault/builtin/logical/pki.(*backend).startTidyOperation.func1()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:342 +0x278
Previous read at 0x00c00c77680c by goroutine 90528:
reflect.Value.Uint()
/usr/local/go/src/reflect/value.go:2584 +0x195
encoding/json.uintEncoder()
/usr/local/go/src/encoding/json/encode.go:562 +0x45
encoding/json.ptrEncoder.encode()
/usr/local/go/src/encoding/json/encode.go:944 +0x3c2
encoding/json.ptrEncoder.encode-fm()
<autogenerated>:1 +0x90
encoding/json.(*encodeState).reflectValue()
/usr/local/go/src/encoding/json/encode.go:359 +0x88
encoding/json.interfaceEncoder()
/usr/local/go/src/encoding/json/encode.go:715 +0x17b
encoding/json.mapEncoder.encode()
/usr/local/go/src/encoding/json/encode.go:813 +0x854
... more stack trace pointing into JSON encoding and http
handler...
In particular, because the tidy status was directly reading the uint
value without resorting to the atomic side, the JSON serialization could
race with a later atomic update.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Also use atomic load in tests
Because no tidy operation is running here, it should be safe to read the
pointed value directly, but use the safer atomic.Load for consistency.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* wip
* Transit byok cli
* It works!
* changelog
* document return codes
* Update command/transit_import_key.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* make fmt
---------
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* The verify-sign command in it's cleanest existing form.
* Working state
* Updates to proper verification syntax
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* make fmt
* Git CI caught some stuff.
* Base functionality.
* make fmt; changelog
* pki issue command.
* Make fmt. Changelog.
* Error Handling Is Almost A Tutorial
* What I thought empty issuers response fix would be.
* Some tests
* PR-review updates.
* make fmt.
* Fix null response data for listing empty issuers causing a crash.
* Update command/pki_list_children_command.go
Fix double specifier
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add test for pki_list_children.
* Fix tests.
* Update descriptions for correctness based on PR reviews.
* make fmt.
* Updates based on PR feedback.
* Allow multiple arguements (space separated)
* Remove bad merge-thing.
* White-space hell fix change.
* Tests, and return information for issue ca
* Fix make fmt error introduced here: https://github.com/hashicorp/vault/pull/18876
* Update command/pki_issue_intermediate.go
Puncutation.
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove smart quotes for standard quotes.
* More information as part of the help text.
* Better help text.
* Add missing "/" into error message.
---------
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Move cert auth backend setup into initialize
In further review with new understanding after #18244, loading
configuration and CRLs within the backend's initialize function is the
ideal approach: Factory construction is strictly serial, resulting in
backend initialization blocking until config and CRLs are loaded.
By using an InitializeFunc(...), we delay loading until after all
backends are constructed (either right on startup in 1.12+, else during
the initial PeriodicFunc(...) invocation on 1.11 and earlier).
We also invoke initialize automatically on test Factory construction.
Resolves: #17847
Co-authored-by: valli_0x <personallune@mail.ru>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: valli_0x <personallune@mail.ru>
* Base functionality.
* make fmt; changelog
* What I thought empty issuers response fix would be.
* Fix null response data for listing empty issuers causing a crash.
* Update command/pki_list_children_command.go
Fix double specifier
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add test for pki_list_children.
* Fix tests.
* Update descriptions for correctness based on PR reviews.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* WIP/initial routing-ish
* refactor date dropdown to reuse in modal and allowe current month selection
* swap linter disable line
* refactor date-dropdown to return object
* refactor calendar widget, add tests
* change calendar start and end args to getters
* refactor dashboard to use date objects instead of array of year, month
* remove dashboard files for easier to follow git diff
* comment out dashboard tab until route name updated
* delete current tab and route
* fix undefined banner time
* cleanup version history serializer and upgrade data
* first pass of updating tests
* add changelog
* update client count util test
* validate end time is after start time
* update comment
* add current month to calendar widget
* add comments for code changes to make following API update
* Removed a modified file from pull request
* address comments/cleanup
* update variables to const
* update test const
* rename history -> dashboard, fix tests
* fix timestamps for attribution chart
* update release note
* refactor using backend start and end time params
* add test for adapter formatting time params
* fix tests
* cleanup adapter comment and query params
* change back history file name for diff
* rename file using cli
* revert filenames
* rename files via git cli
* revert route file name
* last cli rename
* refactor mirage
* hold off on running total changes
* update params in test
* refactor to remove conditional assertions
* finish tests
* fix firefox tooltip
* remove current-when
* refactor version history
* add timezone/UTC note
* final cleanup!!!!
* fix test
* fix client count date tests
* fix date-dropdown test
* clear datedropdown completely
* update date selectors to accommodate new year (#18586)
* Revert "hold off on running total changes"
This reverts commit 8dc79a626d549df83bc47e290392a556c670f98f.
* remove assumed 0 values
* update average helper to only calculate for array of objects
* remove passing in bar chart data, map in running totals component instead
* cleanup usage stat component
* clear ss filters for new queries
* update csv export, add explanation to modal
* update test copy
* consistently return null if no upgrade during activity (instead of empty array)
* update description, add clarifying comments
* update tes
* add more clarifying comments
* fix historic single month chart
* remove old test tag
* Update ui/app/components/clients/dashboard.js
* The verify-sign command in it's cleanest existing form.
* Working state
* Updates to proper verification syntax
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* make fmt
* Git CI caught some stuff.
* Some tests
* PR-review updates.
* make fmt.
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* The fields.
* UserID set, add to certificate
* Changelog.
* Fix test (set default).
* Add UserID constant to certutil, revert extension changes
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add user_ids as field for leaf signing
Presumably, this isn't necessary for CAs, given that CAs probably don't
have a user ID corresponding to them.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support setting multiple user_ids in Subject
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow any User ID with sign-verbatim
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for User IDs in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs about user_ids, allowed_user_ids
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Parallel migration (#18815)
* flagParallel sanity check
* Attempt to use ErrGroups
* Updated docs
* Allow 'start' and 'max-parallel' together
* parallel flag renamed to max-parallel
* tests for start + parallel
* Removed permit pool
* Updated docs to make it clearer that a high setting might not be honored based on storage backend setting
* System dependent max int size
* Default max-parallel 1 => 10
* Test folder/paths updated
Co-authored-by: Tomasz Pawelczak <10206601+gites@users.noreply.github.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
* wip
* wip
* Got it 'working', but not happy about cleanliness yet
* Switch to a dedicated defaultSeal with recovery keys
This is simpler than trying to hijack SealAccess as before. Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal. Then everything proceeds
as if you had a shamir seal to begin with.
* Handle recovery rekeying
* changelog
* Revert go.mod redirect
* revert multi-blob info
* Dumb nil unmarshal target
* More comments
* Update vault/seal.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Update changelog/18683.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* pr feedback
* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split
* Better comment on recovery seal during adjustSealMigration
* Make it possible to migrate from an auto-seal in recovery mode to shamir
* Fix sealMigrated to account for a recovery seal
* comments
* Update changelog/18683.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Address PR feedback
* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate
* Don't shortcut the reast of seal migration
* get rid of redundant transit server cleanup
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* named MFA method configurations
* fix a test
* CL
* fix an issue with same config name different ID and add a test
* feedback
* feedback on test
* consistent use of passcode for all MFA methods (#18611)
* make use of passcode factor consistent for all MFA types
* improved type for MFA factors
* add method name to login CLI
* minor refactoring
* only accept MFA method name with its namespace path in the login request MFA header
* fix a bug
* fixing an ErrorOrNil return value
* more informative error message
* Apply suggestions from code review
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* feedback
* test refactor a bit
* adding godoc for a test
* feedback
* remove sanitize method name
* guard a possbile nil ref
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* added audit-hash operations
* more audit paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added audit fields
* add changelog file
* dynamic fields should be nil
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* start to add test helper
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add tests for /sys/audit openapi paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <anton.averchenkov@hashicorp.com>
* Ember Engine for Kubernetes Secrets Engine (#17881)
* adds in-repo ember engine for kubernetes secrets engine
* updates kubernetes engine class name
* Kubernetes route plumbing (#17895)
* kubernetes route plumbing
* adds kubernetes role index route with redirect to details
* adds kubernetes as mountable and supported secrets engine (#17891)
* adds models, adapters and serializers for kubernetes secrets engine (#18010)
* adds mirage factories and handlers for kubernetes (#17943)
* Kubernetes Secrets Engine Configuration (#18093)
* moves RadioCard component to core addon
* adds kubernetes configuration view
* fixes tests using RadioCard after label for and input id changes
* adds confirm modal when editing kubernetes config
* addresses review comments
* Kubernetes Configuration View (#18147)
* removes configuration edit and index routes
* adds kubernetes configuration view
* Kubernetes Roles List (#18211)
* removes configuration edit and index routes
* adds kubernetes configuration view
* adds kubernetes secrets engine roles list view
* updates role details disabled state to explicitly check for false
* VAULT-9863 Kubernetes Overview Page (#18232)
* Add overview page view
* Add overview page tests
* Address feedback to update tests and minor changes
* Use template built in helper for conditionally showing num roles
* Set up roleOptions in constructor
* Set up models in tests and fix minor bug
* Kubernetes Secrets Engine Create/Edit Views (#18271)
* moves kv-object-editor to core addon
* moves json-editor to core addon
* adds kubernetes secrets engine create/edit views
* updates kubernetes/role adapter test
* addresses feedback
* fixes issue with overview route showing 404 page (#18303)
* Kubernetes Role Details View (#18294)
* moves format-duration helper to core addon
* adds kubernetes secrets engine role details view
* adds tests for role details page component
* adds capabilities checks for toolbar actions
* fixes list link for secrets in an ember engine (#18313)
* Manual Testing: Bug Fixes and Improvements (#18333)
* updates overview, configuration and roles components to pass args for individual model properties
* bug fixes and improvements
* adds top level index route to redirect to overview
* VAULT-9877 Kubernetes Credential Generate/View Pages (#18270)
* Add credentials route with create and view components
* Update mirage response for creds and add ajax post call for creds in adapter
* Move credentials create and view into one component
* Add test classes
* Remove files and update backend property name
* Code cleanup and add tests
* Put test helper in helper function
* Add one more test!
* Add code optimizations
* Fix model in route and add form
* Add onSubmit to form and preventDefault
* Fix tests
* Update mock data for test to be strong rather than record
* adds acceptance tests for kubernetes secrets engine roles (#18360)
* VAULT-11862 Kubernetes acceptance tests (#18431)
* VAULT-12185 overview acceptance tests
* VAULT-12298 credentials acceptance tests
* VAULT-12186 configuration acceptance tests
* VAULT-12127 Refactor breadcrumbs to use breadcrumb component (#18489)
* VAULT-12127 Refactor breadcrumbs to use Page::Breadcrumbs component
* Fix failing tests by adding breadcrumbs properties
* VAULT-12166 add jsdocs to kubernetes secrets engine pages (#18509)
* fixes incorrect merge conflict resolution
* updates kubernetes check env vars endpoint (#18588)
* hides kubernetes ca cert field if not defined in configuration view
* fixes loading substate handling issue (#18592)
* adds changelog entry
Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com>
* OpenAPI `generic_mount_paths` follow-up
An incremental improvement within larger context discussed in #18560.
* Following the revert in #18617, re-introduce the change from
`{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
otherwise paths from multiple plugins would clash - e.g. almost every
auth method would provide a conflicting definition for
`auth/{mountPath}/login`, and the last one written into the map would
win.
* Move the half of the functionality that was in `sdk/framework/` to
`vault/logical_system.go` with the rest; this is needed, as
`sdk/framework/` gets compiled in to externally built plugins, and
therefore there may be version skew between it and the Vault main
code. Implementing the `generic_mount_paths` feature entirely on one
side of this boundary frees us from problems caused by this.
* Update the special exception that recognizes `system` and `identity`
as singleton mounts to also include the other two singleton mounts,
`cubbyhole` and `auth/token`.
* Include a comment that documents to restricted circumstances in which
the `generic_mount_paths` option makes sense to use:
// Note that for this to actually be useful, you have to be using it with
// a Vault instance in which you have mounted one of each secrets engine
// and auth method of types you are interested in, at paths which identify
// their type, and for the KV secrets engine you will probably want to
// mount separate kv-v1 and kv-v2 mounts to include the documentation for
// each of those APIs.
* Fix tests
Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.
* Add changelog
* Update 18663.txt
Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:
- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`
The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):
> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.
Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.
Resolves: #18658
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role successful responses align with the declared response schema.
It also introduces a test helper in `sdk/helper/testhelpers`:
```go
func FindResponseSchema(t *testing.T, ...)
```
This test helper will be useful for all plugins that require similar response schema validation in tests.
### Background
This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
This pull request adds 3 functions (and corresponding tests):
`testhelpers/response_validation.go`:
- `ValidateResponse`
- `ValidateResponseData`
field_data.go:
- `ValidateStrict` (has the "strict" validation logic)
The functions are primarily meant to be used in tests to ensure that the responses are consistent with the defined response schema. An example of how the functions can be used in tests can be found in #18636.
### Background
This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
This PR relates to a feature request logged through HashiCorp commercial
support.
Vault lacks pagination in its APIs. As a result, certain list operations
can return **very** large responses. The user's chosen audit sinks may
experience difficulty consuming audit records that swell to tens of
megabytes of JSON.
In our case, one of the systems consuming audit log data could not cope,
and failed.
The responses of list operations are typically not very interesting, as
they are mostly lists of keys, or, even when they include a "key_info"
field, are not returning confidential information. They become even less
interesting once HMAC-ed by the audit system.
Some example Vault "list" operations that are prone to becoming very
large in an active Vault installation are:
auth/token/accessors/
identity/entity/id/
identity/entity-alias/id/
pki/certs/
In response, I've coded a new option that can be applied to audit
backends, `elide_list_responses`. When enabled, response data is elided
from audit logs, only when the operation type is "list".
For added safety, the elision only applies to the "keys" and "key_info"
fields within the response data - these are conventionally the only
fields present in a list response - see logical.ListResponse, and
logical.ListResponseWithInfo. However, other fields are technically
possible if a plugin author writes unusual code, and these will be
preserved in the audit log even with this option enabled.
The elision replaces the values of the "keys" and "key_info" fields with
an integer count of the number of entries. This allows even the elided
audit logs to still be useful for answering questions like "Was any data
returned?" or "How many records were listed?".
* add core state lockd eadlock detection config option v2
* add changelog
* split out NewTestCluster function to maintain build flag
* replace long func with constant
* remove line
* rename file, and move where detect deadlock flag is set
Anton Averchenkov