* Stub out initial health check command
This command will be used to generate health check results for the PKI
engine.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start common health check implementation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add common health check utilities
These utilities will collect helpers not specific to PKI health checks,
such as formatting longer durations more legibly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PKI health check common utils
Many health checks will need issuer and/or CRL information in order to
execute. We've centrally located these helpers to avoid particular
health checks from needing to reimplement them each time.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Adding ca_validity_period health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Begin using health-checks in PKI command
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow parsing raw requests afterwards
This shifts the last of the logic difference between Read(...) and
ReadRaw(...) to a new helper, allowing ReadRaw(...) requests to be
parsed into the same response structure afterwards as Read(...); this
allows API callers to fetch the raw secret and inspect the raw response
object in case something went wrong (error code &c) -- and when the
request succeeds, they can still get the api.Secret out.
This will be used with the PKI health check functionality, making both
LIST and READ operations use ReadRaw, and optionally parsing the secret
afterwards.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add crl_validity_period health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKI health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug in raw reading with contexts
When reading raw objects, don't manually call the context cancellation:
this causes timeouts and/or EOF errors when attempting to read or parse
the response body. See message in client.RawRequestWithContext(...) for
more information.
This was causing the test suite to randomly fail, due to the context
cancelling. The test suite's client usually had a default timeout,
whereas the CLI didn't, and thus didn't exhibit the same issue.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix typo in permissions message
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move %v->%w for errs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
This commit adds some logic to handle the case where a mount entry has a
non-builtin RunningVersion. This ensures that we only report deprecation
status for builtins.
* Started work on adding log-file support to Agent
* Allow log file to be picked up and appended
* Use NewLogFile everywhere
* Tried to pull out the config aggregation from Agent.Run
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Expose ssh algorithm_signer in web interface (#10114)
* Adds allowed values for algorithm_signer to ssh plugin API
* Adds algorithm_signer as field in UI
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
The current behaviour is to only add mount paths into the generated `opeanpi.json` spec if a `generic_mount_paths` flag is added to the request. This means that we would have to maintain two different `openapi.json` files, which is not ideal. The new solution in this PR is to add `{mount_path}` into every path with a default value specified:
```diff
-- "/auth/token/accessors/": {
++ "/auth/{mount_path}/accessors/": {
"parameters": [
{
"name": "mount_path",
"description": "....",
"in": "path",
"schema": {
"type": "string",
++ "default": "token"
}
}
],
```
Additionally, fixed the logic to generate the `operationId` (used to generate method names in the code generated from OpenAPI spec). It had a bug where the ID had `mountPath` in it. The new ID will look like this:
```diff
-- "operationId": "listAuthMountpathAccessors",
++ "operationId": "listTokenAccessors",
```
* Add automatic tidy of expired issuers
To aid PKI users like Consul, which periodically rotate intermediates,
and provided a little more consistency with older versions of Vault
which would silently (and dangerously!) replace the configured CA on
root/intermediate generation, we introduce an automatic tidy of expired
issuers.
This includes a longer safety buffer (1 year) and logging of the
relevant issuer information prior to deletion (certificate contents, key
ID, and issuer ID/name) to allow admins to recover this value if
desired, or perform further cleanup of keys.
From my PoV, removal of the issuer is thus a relatively safe operation
compared to keys (which I do not feel comfortable removing) as they can
always be re-imported if desired. Additionally, this is an opt-in tidy
operation, not enabled by default. Lastly, most major performance
penalties comes with lots of issuers within the mount, not as much
large numbers of keys (as only new issuer creation/import operations are
affected, unlike LIST /issuers which is a public, unauthenticated
endpoint).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs on tidy of issuers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Restructure logging
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing fields to expected tidy output
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly preserve other issuer config params
When setting a new default issuer, our helper function would overwrite
other parameters in the issuer configuration entry. However, up until
now, there were none.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new parameter to allow default to follow new
This parameter will allow operators to have the default issuer
automatically update when a new root is generated or a single issuer
with a key (potentially with others lacking key) is imported.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Storage migration tests fail on new members
These internal members shouldn't be tested by the storage migration
code, and so should be elided from the test results.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Follow new issuer on root generation, import
This updates the two places where issuers can be created (outside of
legacy CA bundle migration which already sets the default) to follow
newly created issuers when the config is set.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for new default-following behavior
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new API to PKI to list revoked certificates
- A new API that will return the list of serial numbers of
revoked certificates on the local cluster.
* Add cl
* PR feedback
* Ensure correct write ordering in rebuildIssuersChains
When troubleshooting a recent migration failure from 1.10->1.11, it was
noted that some PKI mounts had bad chain construction despite having
valid, chaining issuers. Due to the cluster's leadership trashing
between nodes, the migration logic was re-executed several times,
partially succeeding each time. While the legacy CA bundle migration
logic was written with this in mind, one shortcoming in the chain
building code lead us to truncate the ca_chain: by sorting the list of
issuers after including non-written issuers (with random IDs), these
issuers would occasionally be persisted prior to storage _prior_ to
existing CAs with modified chains.
The migration code carefully imported the active issuer prior to its
parents. However, due to this bug, there was a chance that, if write to
the pending parent succeeded but updating the active issuer didn't, the
active issuer's ca_chain field would only contain the self-reference and
not the parent's reference as well. Ultimately, a workaround of setting
and subsequently unsetting a manual chain would force a chain
regeneration.
In this patch, we simply fix the write ordering: because we need to
ensure a stable chain sorting, we leave the sort location in the same
place, but delay writing the provided referenceCert to the last
position. This is because the reference is meant to be the user-facing
action: without transactional write capabilities, other chains may
succeed, but if the last user-facing action fails, the user will
hopefully retry the action. This will also correct migration, by
ensuring the subsequent issuer import will be attempted again,
triggering another chain build and only persisting this issuer when
all other issuers have also been updated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remigrate ca_chains to fix any missing issuers
In the previous commit, we identified an issue that would occur on
legacy issuer migration to the new storage format. This is easy enough
to detect for any given mount (by an operator), but automating scanning
and remediating all PKI mounts in large deployments might be difficult.
Write a new storage migration version to regenerate all chains on
upgrade, once.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add issue to PKI considerations documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correct %v -> %w in chain building errs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* moves service worker message event listener from addon to raft-storage-overview component
* adds changelog entry
* adds raft-storage-overview test for downloading snapshot via service worker
By adding the link flags `-s -w` we can reduce the Vault binary size
from 204 MB to 167 MB (about 18% reduction in size).
This removes the DWARF section of the binary.
i.e., before:
```
$ objdump --section-headers vault-debug
vault-debug: file format mach-o arm64
Sections:
Idx Name Size VMA Type
0 __text 03a00340 0000000100001000 TEXT
1 __symbol_stub1 00000618 0000000103a01340 TEXT
2 __rodata 00c18088 0000000103a01960 DATA
3 __rodata 015aee18 000000010461c000 DATA
4 __typelink 0004616c 0000000105bcae20 DATA
5 __itablink 0000eb68 0000000105c10fa0 DATA
6 __gosymtab 00000000 0000000105c1fb08 DATA
7 __gopclntab 02a5b8e0 0000000105c1fb20 DATA
8 __go_buildinfo 00008c10 000000010867c000 DATA
9 __nl_symbol_ptr 00000410 0000000108684c10 DATA
10 __noptrdata 000fed00 0000000108685020 DATA
11 __data 0004e1f0 0000000108783d20 DATA
12 __bss 00052520 00000001087d1f20 BSS
13 __noptrbss 000151b0 0000000108824440 BSS
14 __zdebug_abbrev 00000129 000000010883c000 DATA, DEBUG
15 __zdebug_line 00651374 000000010883c129 DATA, DEBUG
16 __zdebug_frame 001e1de9 0000000108e8d49d DATA, DEBUG
17 __debug_gdb_scri 00000043 000000010906f286 DATA, DEBUG
18 __zdebug_info 00de2c09 000000010906f2c9 DATA, DEBUG
19 __zdebug_loc 00a619ea 0000000109e51ed2 DATA, DEBUG
20 __zdebug_ranges 001e94a6 000000010a8b38bc DATA, DEBUG
```
And after:
```
$ objdump --section-headers vault-no-debug
vault-no-debug: file format mach-o arm64
Sections:
Idx Name Size VMA Type
0 __text 03a00340 0000000100001000 TEXT
1 __symbol_stub1 00000618 0000000103a01340 TEXT
2 __rodata 00c18088 0000000103a01960 DATA
3 __rodata 015aee18 000000010461c000 DATA
4 __typelink 0004616c 0000000105bcae20 DATA
5 __itablink 0000eb68 0000000105c10fa0 DATA
6 __gosymtab 00000000 0000000105c1fb08 DATA
7 __gopclntab 02a5b8e0 0000000105c1fb20 DATA
8 __go_buildinfo 00008c20 000000010867c000 DATA
9 __nl_symbol_ptr 00000410 0000000108684c20 DATA
10 __noptrdata 000fed00 0000000108685040 DATA
11 __data 0004e1f0 0000000108783d40 DATA
12 __bss 00052520 00000001087d1f40 BSS
13 __noptrbss 000151b0 0000000108824460 BSS
```
The only side effect I have been able to find is that it is no longer
possible to use [delve](https://github.com/go-delve/delve) to run the
Vault binary.
Note, however, that running delve and other debuggers requires access
to the full source code, which isn't provided for the Enterprise, HSM,
etc. binaries, so it isn't possible to debug those anyway outside of
people who have the full source.
* panic traces
* `vault debug`
* error messages
* Despite what the documentation says, these flags do *not* delete the
function symbol table (so it is not the same as having a `strip`ped
binary).
It contains mappings between the compiled binary and functions,
paramters, and variables in the source code.
Using `llvm-dwarfdump`, it looks like:
```
0x011a6d85: DW_TAG_subprogram
DW_AT_name ("github.com/hashicorp/vault/api.(*replicationStateStore).recordState")
DW_AT_low_pc (0x0000000000a99300)
DW_AT_high_pc (0x0000000000a99419)
DW_AT_frame_base (DW_OP_call_frame_cfa)
DW_AT_decl_file ("/home/swenson/vault/api/client.go")
DW_AT_external (0x01)
0x011a6de1: DW_TAG_formal_parameter
DW_AT_name ("w")
DW_AT_variable_parameter (0x00)
DW_AT_decl_line (1735)
DW_AT_type (0x00000000001e834a "github.com/hashicorp/vault/api.replicationStateStore *")
DW_AT_location (0x009e832a:
[0x0000000000a99300, 0x0000000000a9933a): DW_OP_reg0 RAX
[0x0000000000a9933a, 0x0000000000a99419): DW_OP_call_frame_cfa)
0x011a6def: DW_TAG_formal_parameter
DW_AT_name ("resp")
DW_AT_variable_parameter (0x00)
DW_AT_decl_line (1735)
DW_AT_type (0x00000000001e82a2 "github.com/hashicorp/vault/api.Response *")
DW_AT_location (0x009e8370:
[0x0000000000a99300, 0x0000000000a9933a): DW_OP_reg3 RBX
[0x0000000000a9933a, 0x0000000000a99419): DW_OP_fbreg +8)
0x011a6e00: DW_TAG_variable
DW_AT_name ("newState")
DW_AT_decl_line (1738)
DW_AT_type (0x0000000000119f32 "string")
DW_AT_location (0x009e83b7:
[0x0000000000a99385, 0x0000000000a99385): DW_OP_reg0 RAX, DW_OP_piece 0x8, DW_OP_piece 0x8
[0x0000000000a99385, 0x0000000000a993a4): DW_OP_reg0 RAX, DW_OP_piece 0x8, DW_OP_reg3 RBX, DW_OP_piece 0x8
[0x0000000000a993a4, 0x0000000000a993a7): DW_OP_piece 0x8, DW_OP_reg3 RBX, DW_OP_piece 0x8)
```
This says that the particular binary section is the function
`github.com/hashicorp/vault/api.(*replicationStateStore).recordState`,
from the file `/home/swenson/vault/api/client.go`, containing
the `w` parameter on line 1735 mapped to certain registers and memory,
the `resp` paramter on line 1735 mapped to certain reigsters and memory,
and the `newState` variable on line 1738, mapped to certain registers,
and memory.
It's really only useful for a debugger.
Anyone running the code in a debugger will need full access the source
code anyway, so presumably they will be able to run `make dev` and build
the version with the DWARF sections intact, and then run their debugger.
* Return revocation info within existing certs/<serial> api
- The api already returned both the certificate and a revocation_time
field populated. Update the api to return revocation_time_rfc3339
as we do elsewhere and also the issuer id if it was revoked.
- This will allow callers to associate a revoked cert with an issuer
* Add cl
* PR feedback (docs update)
* Expose raw request from client.Logical()
Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.
Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add raw formatter for direct []byte data
As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for reading raw API endpoints
Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:
> Error reading pki/ca/pem: invalid character '-' in numeric literal
Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove panic
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add regression test for default CRL expiry
Also fixes a bug w.r.t. upgrading older entries and missing the Delta
Rebuild Interval field, setting it to the default.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog for earlier PR
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for PKCSv1_5_NoOID signatures
This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.
These are specified in RFC 3447 Section 9.2.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exclude new none type from PSS based tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKCS#1v1.5 signatures
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* adds error handling to auth-jwt component for missing roles and fixes bug where role wasn't being retained when using alternate oidc mount path at login
* fixes jwt login bug from auth mount tabs and adds test
* updates okta-number-challenge success value to arg in template
* adds changelog entry
* fixes issues logging in manually with jwt
* reverts mistaken change
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
* Add patch support to CLI
This is based off the existing write command, using the
JSONMergePatch(...) API client method rather than Write(...), allowing
us to update specific fields.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on PATCH support
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow passing AssociatedData factories in keysutil
This allows the high-level, algorithm-agnostic Encrypt/Decrypt with
Factory to pass in AssociatedData, and potentially take multiple
factories (to allow KMS keys to work). On AEAD ciphers with a relevant
factory, an AssociatedData factory will be used to populate the
AdditionalData field of the SymmetricOpts struct, using it in the AEAD
Seal process.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add associated_data to Transit Encrypt/Decrypt API
This allows passing the associated_data (the last AD in AEAD) to
Transit's encrypt/decrypt when using an AEAD cipher (currently
aes128-gcm96, aes256-gcm96, and chacha20-poly1305). We err if this
parameter is passed on non-AEAD ciphers presently.
This associated data can be safely transited in plaintext, without risk
of modifications. In the event of tampering with either the ciphertext
or the associated data, decryption will fail.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add to documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* runs ember-cli-update to 4.4.0
* updates yarn.lock
* updates dependencies causing runtime errors (#17135)
* Inject Store Service When Accessed Implicitly (#17345)
* adds codemod for injecting store service
* adds custom babylon parser with decorators-legacy plugin for jscodeshift transforms
* updates inject-store-service codemod to only look for .extend object expressions and adds recast options
* runs inject-store-service codemod on js files
* replace query-params helper with hash (#17404)
* Updates/removes dependencies throwing errors in Ember 4.4 (#17396)
* updates ember-responsive to latest
* updates ember-composable-helpers to latest and uses includes helper since contains was removed
* updates ember-concurrency to latest
* updates ember-cli-clipboard to latest
* temporary workaround for toolbar-link component throwing errors for using params arg with LinkTo
* adds missing store injection to auth configure route
* fixes issue with string-list component throwing error for accessing prop in same computation
* fixes non-iterable query params issue in mfa methods controller
* refactors field-to-attrs to handle belongsTo rather than fragments
* converts mount-config fragment to belongsTo on auth-method model
* removes ember-api-actions and adds tune method to auth-method adapter
* converts cluster replication attributes from fragment to relationship
* updates ember-data, removes ember-data-fragments and updates yarn to latest
* removes fragments from secret-engine model
* removes fragment from test-form-model
* removes commented out code
* minor change to inject-store-service codemod and runs again on js files
* Remove LinkTo positional params (#17421)
* updates ember-cli-page-object to latest version
* update toolbar-link to support link-to args and not positional params
* adds replace arg to toolbar-link component
* Clean up js lint errors (#17426)
* replaces assert.equal to assert.strictEqual
* update eslint no-console to error and disables invididual intended uses of console
* cleans up hbs lint warnings (#17432)
* Upgrade bug and test fixes (#17500)
* updates inject-service codemod to take arg for service name and runs for flashMessages service
* fixes hbs lint error after merging main
* fixes flash messages
* updates more deps
* bug fixes
* test fixes
* updates ember-cli-content-security-policy and prevents default form submission throwing errors
* more bug and test fixes
* removes commented out code
* fixes issue with code-mirror modifier sending change event on setup causing same computation error
* Upgrade Clean Up (#17543)
* updates deprecation workflow and filter
* cleans up build errors, removes unused ivy-codemirror and sass and updates ember-cli-sass and node-sass to latest
* fixes control groups test that was skipped after upgrade
* updates control group service tests
* addresses review feedback
* updates control group service handleError method to use router.currentURL rather that transition.intent.url
* adds changelog entry
* VAULT-8719 Support data array for alias clash error response so UI can understand error
* VAULT-8719 Changelog
* VAULT-8719 Update alias mount update logic
* VAULT-8719 Further restrict IsError()
The SSH secrets engine previously split the `validPrincipals` field
on comma, then if user templating is enabled, evaluated the
templates on each substring. This meant the identity template was only
ever allowed to return a single principal. There are use cases
where it would be helpful for identity metadata to contain a list
of valid principals and for the identity template to be able to inject
all of those as valid principals.
This change inverts the order of processing. First the template
is evaluated, and then the resulting string is split on commas.
This allows the identity template to return a single comma-separated
string with multiple permitted principals.
There is a potential security implication here, that if a user is
allowed to update their own identity metadata, they may be able to
elevate privileges where previously this was not possible.
Fixes#11038
* Fix tidy-status, tidy-cancel on PR Secondaries
PKI's tidy-status included a bug that prevented PR secondary nodes from
responding with the status of the running tidy operation: while the
operation constructor correctly forwarded the node on PR standby
instances, the handler itself forwarded also on PR secondary nodes.
This is incorrect as the PR secondary nodes are the active node in the
local PR cluster, and run tidy operations otherwise.
This meant that while auto-tidy and tidy operations would run, there was
no insight into the process.
When implementing tidy-cancel, tidy-status's handler logic was reused,
duplicating the bug there as well.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update go version to 1.19.2
This commit updates the default version of go to 1.19.2. This update
includes minor security fixes for archive/tar, net/http/httputil, and
regexp packages.
For more information on the release, see: https://go.dev/doc/devel/release#go1.19.2
* Update Docker versions in CI to 20.10.17
After updating Vault to go version 1.19.2, there were several SIGABRTs
in the vault tests. These were related to a missing `pthread_create`
syscall in Docker. Since CI was using a much older version of Docker,
the fix was to bump it to latest-1 (20.10.17).
While we're at it, add a note in the developer docs encouraging the use
of the latest Docker version.
* Fix for duplicate SANs in signed certificates when othernames are present in the CSR SAN extension and UseCSRValues is true.
When UseCSRValues is true (as is the case on the sign-verbatim endpoint), all extensions including Subject Alternative Names are copied from the CSR to the final certificate.
If the Subject Alternative Name in question contains any othernames (such as a Microsoft UPN) the SAN extension is added again as a workaround for an encoding issue (in function HandleOtherSANs).
Having duplicate x509v3 extensions is invalid and is rejected by openssl on Ubuntu 20.04, and also by Go since https://github.com/golang/go/issues/50988 (including in Go 1.19).
In this fix I do not add the extension from the CSR if it will be added during HandleOtherSANs.
* Added unittest and changelog entry.
Add some metrics helpful for monitoring raft cluster state.
Furthermore, we weren't emitting bolt metrics on regular (non-perf) standbys, and there were other metrics
in metricsLoop that would make sense to include in OSS but weren't. We now have an active-node-only func,
emitMetricsActiveNode. This runs metricsLoop on the active node. Standbys and perf-standbys run metricsLoop
from a goroutine managed by the runStandby rungroup.
Ensure that we don't try to access Core.perfStandby or Core.PerfStandby() from dynamicSystemView, which might be accessed with or without stateLock held.
This fixes a couple of references to loop variables in parallel tests
and deferred functions. When running a parallel test (calling
`t.Parallel()`) combined with the table-driven pattern, it's necessary
to copy the test case loop variable, otherwise only the last test case
is exercised. This is documented in the `testing` package:
https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks
`defer` statements that invoke a closure should also not reference a
loop variable directly as the referenced value will change in each
iteration of the loop.
Issues were automatically found with the `loopvarcapture` linter.
* PKI: Add support for signature_bits param to the intermediate/generate api
- Mainly to work properly with GCP backed managed keys, we need to
issue signatures that would match the GCP key algorithm.
- At this time due to https://github.com/golang/go/issues/45990 we
can't issue PSS signed CSRs, as the libraries in Go always request
a PKCS1v15.
- Add an extra check in intermediate/generate that validates the CSR's
signature before providing it back to the client in case we generated
a bad signature such as if an end-user used a GCP backed managed key
with a RSA PSS algorithm.
- GCP ignores the requested signature type and always signs with the
key's algorithm which can lead to a CSR that says it is signed with
a PKCS1v15 algorithm but is actually a RSA PSS signature
* Add cl
* PR feedback
* PKI: Do not load revoked certificates if CRL has been disabled
- Restore the prior behavior of not reading in all revoked certificates
if the CRL has been disabled as there might be performance issues
if a customer had or is still revoking a lot of certificates.
* Add cl
* Add failing test for when command != plugin name
* wrapFactoryCheckPerms uses pluginCatalog.Get to fetch the correct command
* Use filepath.Rel for consistency with plugin read API handler
* Added flag and env var which will disable client redirection
* Added changelog
* Docs fix for unsaved file, and test single request made
* Updated test for case when redirect is enabled, updated docs based on suggestions
* PKI: Fix managed key signatures when using specified signature_bits
- When calling sign-intermediate and other apis with signature_bits
value overridden with a backing managed key we did not use that
value as tests for the private key type were not working.
* Add cl
* Write explicit -help output to stdout
Per the consensus of most programs, and mirroring the GNU Coding
Standards for CLI design, when users request -help explicitly via the
CLI, this should be written to stdout to allow paging of output. stderr
is fine when an invalid usage triggers the help text however.
In our case, mitchellh/cli helpfully adds a HelpWriter that we
previously set to stderr explicitly. This writer is only called to print
user-requested help text; it is not called on error cases (e.g., bad
usage triggering additional help text to the user).
Thus it should safely be settable to stdout, enabling pagers without
additional redirects.
Notably, we do have to set ErrorWriter as it defaults to initializing to
the value of HelpWriter, which we want to set to stdout now, which is
less useful.
See also: https://www.gnu.org/prep/standards/html_node/_002d_002dhelp.htmlResolves: #17004
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* VAULT-6938 Remove license from being cache exempt
* VAULT-6938 Add changelog
* VAULT-6938 Typo in changelog name
* Update changelog/17265.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Basics of Cert-Count Telemetry, changelog, "best attempt" slice to capture (and test for) duplicates, Move sorting of possibleDoubleCountedRevokedSerials to after compare of entries. Add values to counter when still initializing.
Set lists to nil after use, Fix atomic2 import, Delay reporting metrics until after deduplication has completed,
The test works now, Move string slice to helper function; Add backendUUID to gauge name.
Break grabLockOrStop into two pieces to facilitate investigating deadlocks. Without this change, the "grab" goroutine looks the same regardless of who was calling grabLockOrStop, so there's no way to identify one of the deadlock parties.
* Fetch CRLs from a user defined CDP (PoC)
* Handle no param sent
* Move CRL fetch to a periodFunc. Use configured CA certs + system root as trusted certs for CRL fetch
* comments
* changelog
* Just use root trust
* cdp->url in api
* Store CRL and populate it initially in cdlWrite
* Update docs
* Update builtin/credential/cert/path_crls.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Handle pre-verification of a CRL url better
* just in case
* Fix crl write locking
* Add a CRL fetch unit test
* Remove unnecessary validity clear
* Better func name
* Don't exit early updating CRLs
* lock in updateCRLs
* gofumpt
* err-
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Add plugin version to GRPC interface
Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.
I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:
If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.
* Placeholder backend should be external
We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.
So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
* update modal copy to clarify when a user is unable to delete a specific version
* add tests
* cleanup tests, move console commands into helper function
* cleanup hbs
* add changelog
* OIDC Config Routing (#16028)
* adds oidc config routes
* renames oidc applications route to clients
* UI/vault 6646/landing page (#16069)
* add to sidebar
* add landing image and text
* add permissions
* add permissions to permissions service
* remove comment
* fix.
* UI/OIDC models (#16091)
* add models and fix routing
* add ClientsCreate route
* remove form functions from client model
* update comment
* address comments, cleanup models
* add comment
* OIDC Adapters and Serializers (#16120)
* adds named-path base adapter
* adds oidc adapters with tests
* adds oidc serializers
* fixes issue with supported_scopes relationship in oidc provider model
* make radio card size flex (#16125)
* OIDC config details routes (#16126)
* adds details routes for oidc config resources
* adds details templates for oidc config resources
* OIDC parent route and index redirection (#16139)
* adds parent oidc route with header and adds redirection if clients have been created
* updates learn link
* adds findRecord override to named-path adapter (#16145)
* OIDC Scope Create/Edit View (#16174)
* adds oidc scope-form to create and edit views
* moves oidc header set logic from route to controller
* OIDC Scope Details View (#16191)
* adds oidc scope details view
* removes disabled arg from scope delete confirm action
* updates oidc scope template params link to use DocLink and adds success message on scope create success
* updates oidc scope delete confirm action copy
* adds oidc scopes list (#16196)
* UI/vault 6655/OIDC create view (#16331)
* setup header
* wip
* wip
* wip
* validations
* error validations
* cleanup
* wip
* fix error
* clean up
* handle modelValidations
* add documentation on the decorator
* remove spread attrs
* first test and some fixes
* halfway with test
* fix error where the data object was sending param entiyIds and not entity_ids
* validations or situation
* fix test
* small nit:
* test if this fixes the test
* fix
* cleanup
* nit
* Assignments Update/Edit View (#16412)
* wip
* fix
* render search-select after promise is fulfilled
* add test coverage
Co-authored-by: clairebontempo@gmail.com <cbontempo@hashicorp.com>
* Added list view for keys (#16454)
* Added list view for providers (#16442)
* Added list view for providers
* Removed check for model data length
* Added new line at end of file
* Fixed linting issues causing ui tests to fail
* Added list view for application (#16469)
* UI/remove has many relationship (#16470)
* remove hasMany from models
* remove relationships from assignments create form
* update tests
* Assignment list view (#16340)
* inital setup
* handle default allow all
* add learn more link
* Fixed the default allow_all for assignment list view to match Figma design
* Fixed linting
* Fixed hbs file syntax
Co-authored-by: linda9379 <linda.jiang@hashicorp.com>
* configure mirage and helper (#16482)
* UI/OIDC client form (#16131)
* WIP client form
* wip
* still WIP
* fix form!;
* remove computeds, cache form attrs instead
* update scope form component name
* add white space validation
* add validations, cleanup
* add edit form
* fix link to in edit form
* disable edit form
* fix linkto
* wip/ search select filter
* WIP/search-select bug
* fix assignment save
* delete old modal js file
* glimmerize/create new search select modal component
* component cleanup
* fix bugginess
* fix search select and radio select action
* add tests
* revert some test changes
* oops, removed test tag
* add key list to response
* fix test
* move search select component to separate PR, revert changes
* one more revert
* remove oidc helper from this pr
* remove hasMany relationship
* minor cleanup
* update assignment form to use fallback
* fix allow_all appearing in dropdown on edit (#16508)
* UI/ OIDC Application (client) details view (#16507)
* fix test
* finish details page
* finish details view
* clean u[
* fix typo
* configure oidc mirage handler for tests
* remove params, add new route instead
* fix headers
* remove console.log
* remove controller/template reliance on tracked variable
* rename variable
* UI/Client route acceptance tests - fixed branch (#16654)
* WIP client route tests
* refactor client form so clientType is not edit-able
* fix ttl in client form
* wip// more acceptance tests and tags for hbs files
* fix typo
* fix syntax error
* finish tests
* fix client form test
* resolve commits
* update form test
* OIDC Assignments Details view. (#16511)
* setup
* cleanup
* view all fix
* wip setting up tabs
* wip
* revert to no queryParam or tabs
* add the read more component and styling
* rename folder
* cleanup
* fix
* UI/OIDC providers create/edit route (#16612)
* update to use DocLink component
* provider create form
* cleaup
* add formt est
* revert label text
* update doclink test
* disallow new scopes from ss
* fix test typo
* fix provider form flash message
* add period
* test new form field attr
* refactor form input
* fix edit portion of issuer field
* add test selector to new input field
* add comment
* Cleanup OIDC Config Mirage handler (#16674)
* cleaup mirage
* change to .then
* pull out into config file
* Scope acceptance tests (#16707)
* Started writing acceptance tests
* Added some more acceptance tests
* Added tags for hbs and more tests
* Modified variable names in scope form test
* Fixed tests and linting
* UI/OIDC Provider read view (#16632)
* add providers/provider/client route
* provider details view
* add disabled button and tooltip for default
* add toolbar separators
* revert unrelated change
* query all client records and filter by allowed client id"
* refactor adapter to filter for clientId
* cleanup adapter method
* update test
* refactor test
* fix tests to accommodate for serializer change
* update empty state message
* fix linting
* metadata for client list view (#16725)
* Added metadata for list view in clients
* Fixed linting
* Fixed failing ui test
* fix scopes and clients tests (#16768)
* Initial fix of tests
* Fixed failing scopes and clients acceptance tests
* Fixed linting
* UI: Key create/edit form (#16729)
* add route models
* add forms
* add test
* remove helperText attr
* metadata for provider list view (#16738)
* Added meta-data for provider list view
* Added comment for serializer
* Fixed import path for scopes and clients acceptance test files
* UI/Add client ids to search select (#16744)
* WIP use clientID instead of name
* add client ids to search select
* remove provider form component changes
* fix search select on edit
* cleanup comments and method
* fix adapter query method
* clean up comments
* add test
* remove destructuring so linting passes
* fix tests
* add accidentally deleted param
* add clarifying comments
* cleanup
* change how shouldRenderName is set
* cleanup tests
* address comments
* OIDC Assignment Acceptance tests (#16741)
* test and fixes
* merge stuff
* fix
* fixes
* add waituntil
* inconsistent nav issue
* fixes
* blah
* UI/Key details view (#16776)
* add details view
* reformat model file
* todo for when listing applications
* add comment
* update key form with refactored search select
* add applications list
* update test
* update test
* add names to flash messages
* add rollbackAttributes to delete catch (#16796)
* UI: Checks if records exists before creating record when URL contains :name (#16823)
* check for record existing in createRecord
* use error banner instead of flash messages for forms
* add inline form message for validations
* add error count message to inlinealert
* add test for adapter
* add tests
* remove unused vars
* UI: Disable limiting clients when creating key, filter clients when editing (#16926)
* add tooltip to disabled radio button
* pass query object to search select
* update copy
* add comment
* cleanup console log and comment
* fix tests
* revert change because addressed in other pr
* fix diff
* fix test
* UI: Add redirect when last client is deleted (#16927)
* afterModel redirect if no models exist
* fix test
* change space
* fix incorrect text
* UI: Add InfoTooltip to selected 'ghost' client_ids (#16942)
* return option if undefined
* add info tooltip to search select
* change word
* add test
* UI: OIDC config keys acceptance tests (#16968)
* add keys test
* update other oidc tests
* remove-search select comment
* UI: Filter Client providers list view (#17027)
* pass param to adapter
* add test
* UI: OIDC Config Acceptance Tests (#17050)
* WIP/provider acceptance tests"
* WIP/this commit breaks lots of things
* fix tests
* update test selectors
* combine key and client tests
* cleanup clients and keys test
* finish tests
* small tidying
* UI: Remove trailing comma from scopes, provider details page (#17069)
* use info table row to cleanup scope logic
* infotableitemarray cleanup
* tidying
* add changelog
* teeny little empty state
* fix wildcard string helper not working
Co-authored-by: Jordan Reimer <zofskeez@gmail.com>
Co-authored-by: Angel Garbarino <Monkeychip@users.noreply.github.com>
Co-authored-by: Angel Garbarino <argarbarino@gmail.com>
Co-authored-by: linda9379 <57650314+linda9379@users.noreply.github.com>
Co-authored-by: linda9379 <linda.jiang@hashicorp.com>
* adds LinkStatus component to NavHeader to display banner with HCP link status
* adds changelog entry
* adds period to connected status message
* updates hcp link status to current cluster polling to automatically update state
* OSS parts of ent #3157. Some activity log tests were flaky because background workers could race with them; now we overload DisableTimers to stop some of them from running, and add some channels we can use to wait for others to complete before we start testing.
* Add CL
* core: Handle deprecated mounts on enable and unseal
* changelog: Deprecation Status handling
* core: Add Pending Removal override var
* core: Add some documentation for Pending Removal override
OSS parts of ent PR #3172: assume nodes we haven't received heartbeats from are running the same version as we are. Failing to provide a version/upgrade_version will result in Autopilot (on ent) demoting those unversioned nodes to non-voters until we receive a heartbeat from them.
* Get import correct
* limits, docs
* changelog
* unit tests
* And fix import for hmac unit test
* typo
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Update builtin/logical/transit/path_keys.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Validate key sizes a bit more carefully
* Update sdk/helper/keysutil/policy.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Add fields 'ttl' and 'num_uses' to SecretID generation.
Add fields 'ttl' and 'num_uses' when generating/obtaining a SecretID.
Rather than just being able to use the Role's SecretID ttl and num uses. #14390
* Add secret_id_num_uses response field to generating SecretID
Add the response field secret_id_num_uses to the endpoints for generating
SecretIDs. Used in testing but also to supply the vendor with this variable.
* Add tests for new ttl and num_uses SecretID generation fields
Add tests to assert the new TTL and NumUses option in the SecretID entry.
Separate test for testing with just parameters vs a -force example.
* Patch up test for ttl and num_uses fields
* Add changelog entry for auth/approle 'ttl' and 'num_uses' fields
* Add fields to API Docs and AppRole Auth Docs example
* Correct error message for failing test on missing field.
Change the error message produced when a test fails due to a missing field.
Previous values did not map to correct fields.
* Remove unnecessary int cast to int "secret_id_num_uses" field.
Unnecessary cast to int where type already is int.
* Move numUses field check to after assignment.
* Remove metadata entry in sample payload to limit change to changes made.
Remove metadata entry in sample payload for custom-secret-id. The metadata was not
changed in the features pull request.
* Bind fields 'ttl' and 'num_uses' to role's configuration.
Rather than implicitly overriding, error when the ttl is lower than and the num
uses higher than the role's configuration. #14390
* Update changelog 14474 with a more detailed description.
More elaborate description for the changelog. Specifying the per-request based fields.
* Elaborate more on the bounds of the 'ttl' and 'num_uses' field.
Specify in both the api-docs and the CLI the limits of the fields.
Specify that the role's configuration is still the leading factor.
* Upper bound ttl with role secret id ttl
Upper bound ttl with role secret id ttl when creating a secret id
Adding test cases for infinite ttl and num uses
Adding test cases for negative ttl and num uses
Validation on infinite ttl and num uses
* Formatting issues. Removed unnecessary newline
* Update documentation for AppRole Secret ID and Role
Changed that TTL is not allowed to be shorter to longer
* Cleanup approle secret ID test and impl
* Define ttl and num_uses in every test
Define ttl and num_uses in every test despite them not being tested.
This is to ensure that no unexpected behaviour comes to mind.
* Rename test RoleSecretID -> RoleSecretIDWithoutFields
* Test secret id generation defaults to Role's config
Test secret id generation defaults to Role's configuration entries.
* Change finit -> finite
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* Rephrase comments to the correct validation check
* Rephrase role-secret-id option description
* Remove "default" incorrect statement about ttl
* Remove "default" incorrect statement about ttl for custom secret id
* Touch up approle.mdx to align more with path_role documentation
Co-authored-by: Remco Buddelmeijer <r.buddelmeijer@fullstaq.com>
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* removes on click modifier from LinkTo elements
* adds changelog
* reverts button changes and closes dropdown in next tick of runloop
* removes comment
* auth: Add Deprecation Status to auth list -detailed
* secrets: Add Deprecation Status to secrets list -detailed
* Add changelog entry for deprecation status list
* Allow tidy operations to be cancelled
When tidy operations take a long time to execute (and especially when
executing them automatically), having the ability to cancel them becomes
useful to reduce strain on Vault clusters (and let them be rescheduled
at a later time).
To this end, we add the /tidy-cancel write endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing auto-tidy synopsis / description
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add a pause duration between tidying certificates
By setting pause_duration, operators can have a little control over the
resource utilization of a tidy operation. While the list of certificates
remain in memory throughout the entire operation, a pause is added
between processing certificates and the revocation lock is released.
This allows other operations to occur during this gap and potentially
allows the tidy operation to consume less resources per unit of time
(due to the sleep -- though obviously consumes the same resources over
the time of the operation).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for cancellation, pause
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add API docs on pause_duration, /tidy-cancel
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add lock releasing around tidy pause
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Reset cancel guard, return errors
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* accommodate salt lengths for RSA PSS
* address feedback
* generalise salt length to an int
* fix error reporting
* Revert "fix error reporting"
This reverts commit 8adfc15fe3303b8fdf9f094ea246945ab1364077.
* fix a faulty check
* check for min/max salt lengths
* stringly-typed HTTP param
* unit tests for sign/verify HTTP requests
also, add marshaling for both SDK and HTTP requests
* randomly sample valid salt length
* add changelog
* add documentation
* VAULT-7707 Add docs around making mass amounts of lease count quotas via automation
* VAULT-7707 Changelog
* VAULT-7707 add word
* VAULT-7707 Update some small wordings
* VAULT-7707 use a real em dash
* Add remove_roots_from_chain flag to sign and issue pki apis
- Add a new flag to allow end-users to control if we return the
root/self-signed CA certificate within the list of certificates in
ca_chain field on issue and sign api calls.
* Add cl
* PR feedback
* Add ability to perform automatic tidy operations
This enables the PKI secrets engine to allow tidy to be started
periodically by the engine itself, avoiding the need for interaction.
This operation is disabled by default (to avoid load on clusters which
don't need tidy to be run) but can be enabled.
In particular, a default tidy configuration is written (via
/config/auto-tidy) which mirrors the options passed to /tidy. Two
additional parameters, enabled and interval, are accepted, allowing
auto-tidy to be enabled or disabled and controlling the interval
(between successful tidy runs) to attempt auto-tidy.
Notably, a manual execution of tidy will delay additional auto-tidy
operations. Status is reported via the existing /tidy-status endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on auto-tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for auto-tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prevent race during parallel testing
We modified the RollbackManager's execution window to allow more
faithful testing of the periodicFunc. However, the TestAutoRebuild and
the new TestAutoTidy would then race against each other for modifying
the period and creating their clusters (before resetting to the old
value).
This changeset adds a lock around this, preventing the races.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use tidyStatusLock to gate lastTidy time
This prevents a data race between the periodic func and the execution of
the running tidy.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add read lock around tidyStatus gauges
When reading from tidyStatus for computing gauges, since the underlying
values aren't atomics, we really should be gating these with a read lock
around the status access.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* enable registering backend muxed plugins in plugin catalog
* set the sysview on the pluginconfig to allow enabling secrets/auth plugins
* store backend instances in map
* store single implementations in the instances map
cleanup instance map and ensure we don't deadlock
* fix system backend unit tests
move GetMultiplexIDFromContext to pluginutil package
fix pluginutil test
fix dbplugin ut
* return error(s) if we can't get the plugin client
update comments
* refactor/move GetMultiplexIDFromContext test
* add changelog
* remove unnecessary field on pluginClient
* add unit tests to PluginCatalog for secrets/auth plugins
* fix comment
* return pluginClient from TestRunTestPlugin
* add multiplexed backend test
* honor metadatamode value in newbackend pluginconfig
* check that connection exists on cleanup
* add automtls to secrets/auth plugins
* don't remove apiclientmeta parsing
* use formatting directive for fmt.Errorf
* fix ut: remove tls provider func
* remove tlsproviderfunc from backend plugin tests
* use env var to prevent test plugin from running as a unit test
* WIP: remove lazy loading
* move non lazy loaded backend to new package
* use version wrapper for backend plugin factory
* remove backendVersionWrapper type
* implement getBackendPluginType for plugin catalog
* handle backend plugin v4 registration
* add plugin automtls env guard
* modify plugin factory to determine the backend to use
* remove old pluginsets from v5 and log pid in plugin catalog
* add reload mechanism via context
* readd v3 and v4 to pluginset
* call cleanup from reload if non-muxed
* move v5 backend code to new package
* use context reload for for ErrPluginShutdown case
* add wrapper on v5 backend
* fix run config UTs
* fix unit tests
- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests
* remove comment and update AutoMTLS field in test
* remove comment
* remove errwrap and unused context
* only support metadatamode false for v5 backend plugins
* update plugin catalog errors
* use const for env variables
* rename locks and remove unused
* remove unneeded nil check
* improvements based on staticcheck recommendations
* use const for single implementation string
* use const for context key
* use info default log level
* move pid to pluginClient struct
* remove v3 and v4 from multiplexed plugin set
* return from reload when non-multiplexed
* update automtls env string
* combine getBackend and getBrokeredClient
* update comments for plugin reload, Backend return val and log
* revert Backend return type
* allow non-muxed plugins to serve v5
* move v5 code to existing sdk plugin package
* do next export sdk fields now that we have removed extra plugin pkg
* set TLSProvider in ServeMultiplex for backwards compat
* use bool to flag multiplexing support on grpc backend server
* revert userpass main.go
* refactor plugin sdk
- update comments
- make use of multiplexing boolean and single implementation ID const
* update comment and use multierr
* attempt v4 if dispense fails on getPluginTypeForUnknown
* update comments on sdk plugin backend
* honor header if-modified-since if present
* pathGetIssuerCRL first version
* check if modified since for CA endpoints
* fix date comparison for CA endpoints
* suggested changes and refactoring
* add writeIssuer to updateDefaultIssuerId and fix error
* Move methods out of storage.go into util.go
For the most part, these take a SC as param, but aren't directly storage
relevant operations. Move them out of storage.go as a result.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use UTC timezone for storage
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rework path_fetch for better if-modified-since handling
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Invalidate all issuers, CRLs on default write
When the default is updated, access under earlier timestamps will not
work as we're unclear if the timestamp is for this issuer or a previous
issuer. Thus, we need to invalidate the CRL and both issuers involved
(previous, next) by updating their LastModifiedTimes.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for If-Modified-Since
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly invalidate default issuer changes
When the default issuer changes, we'll have to mark the invalidation on
PR secondary clusters, so they know to update their CRL mapping as well.
The swapped issuers will have an updated modification time (which will
eventually replicate down and thus be correct), but the CRL modification
time is cluster-local information and thus won't be replicated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* make fmt
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor sendNotModifiedResponseIfNecessary
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on if-modified-since
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow generation of up-to-date delta CRLs
While switching to periodic rebuilds of CRLs alleviates the constant
rebuild pressure on Vault during times of high revocation, the CRL
proper becomes stale. One response to this is to switch to OCSP, but not
every system has support for this. Additionally, OCSP usually requires
connectivity and isn't used to augment a pre-distributed CRL (and is
instead used independently).
By generating delta CRLs containing only new revocations, an existing
CRL can be supplemented with newer revocations without requiring Vault
to rebuild all complete CRLs. Admins can periodically fetch the delta
CRL and add it to the existing CRL and applications should be able to
support using serials from both.
Because delta CRLs are emptied when the next complete CRL is rebuilt, it
is important that applications fetch the delta CRL and correlate it to
their complete CRL; if their complete CRL is older than the delta CRL's
extension number, applications MUST fetch the newer complete CRL to
ensure they have a correct combination.
This modifies the revocation process and adds several new configuration
options, controlling whether Delta CRLs are enabled and when we'll
rebuild it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for delta CRLs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on delta CRLs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback: fix several bugs
Thanks Steve!
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly invoke periodic func on active nodes
We need to ensure we read the updated config (in case of OCSP request
handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL
re-building.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* VAULT-6433 Add namespace_path to MFA endpoints
* VAULT-6433 add changelog
* VAULT-6433 Return error in case of error
* VAULT-6433 Make logic a bit more concise
* Refactor tidy steps into two separate helpers
This refactors the tidy go routine into two separate helpers, making it
clear where the boundaries of each are: variables are passed into these
method and concerns are separated. As more operations are rolled into
tidy, we can continue adding more helpers as appropriate. Additionally,
as we move to make auto-tidy occur, we can use these as points to hook
into periodic tidying.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor revInfo checking to helper
This allows us to validate whether or not a revInfo entry contains a
presently valid issuer, from the existing mapping. Coupled with the
changeset to identify the issuer on revocation, we can begin adding
capabilities to tidy to update this association, decreasing CRL build
time and increasing the performance of OCSP.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor issuer fetching for revocation purposes
Revocation needs to gracefully handle using the old legacy cert bundle,
so fetching issuers (and parsing them) needs to be done slightly
differently than other places. Refactor this from revokeCert into a
common helper that can be used by tidy.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow tidy to associate revoked certs, issuers
When revoking a certificate, we need to associate the issuer that signed
its certificate back to the revInfo entry. Historically this was
performed during CRL building (and still remains so), but when running
without CRL building and with only OCSP, performance will degrade as the
issuer needs to be found each time.
Instead, allow the tidy operation to take over this role, allowing us to
increase the performance of OCSP and CRL in this scenario, by decoupling
issuer identification from CRL building in the ideal case.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for tidy updates
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on new tidy parameter, metrics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor tidy config into shared struct
Finish adding metrics, status messages about new tidy operation.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* revert to using paramsFor but add check for state having ns=
* revert to using paramsFor but add check for state having ns=
* cleanup hook
* add tests
* add changelog
* Test troubleshooting
* cleanup tests, use window stub correctly!
* add test for state param not existing at all
Co-authored-by: hashishaw <cshaw@hashicorp.com>
* Update google-cloud-storage backend documentation
Add mentions the environment variables that can be used to configure the backend instead of using the stanza parameters
* Add changelog file
* Fix some typos
* Update website/content/docs/configuration/storage/google-cloud-storage.mdx
Commit suggestion #1
Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>
* Update website/content/docs/configuration/storage/google-cloud-storage.mdx
Commit suggestion #2
Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>
* Update website/content/docs/configuration/storage/google-cloud-storage.mdx
Commit suggestion #3
Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>
Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>
Adds support for using semantic version information when registering
and managing plugins. New `detailed` field in the response data for listing
plugins and new `version` field in the response data for reading a
single plugin.
* Refactor CRL tests to use /sys/mounts
Thanks Steve for the approach! This also address nits from Kit.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Skip CRL building steps when disabled
This skips a number of steps during CRL build when it is disabled (and
forceNew is not set). In particular, we avoid fetching issuers, we avoid
associating issuers with revocation entries (and building that in-memory
mapping), making CRL building more efficient.
This means that there'll again be very little overhead on clusters with
the CRL disabled.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prevent revoking roots from appearing on own CRLs
This change ensures that when marking a root as revoked, it no longer
appears on its own CRL. Very few clients support this event (as
generally only leaves/intermediates are checked for presence on a
parent's CRL) and it is technically undefined behavior (if the root is
revoked, its own CRL should be untrusted and thus including it on its
own CRL isn't a safe/correct distribution channel).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure stability of revInfo issuer identification
As mentioned by Kit, iterating through each revInfoEntry and associating
the first issuer which matches it can cause churn when many (equivalent)
issuers are in the system and issuers come and go (via CRLSigning usage,
which has been modified in this release as well). Because we'd not
include issuers without CRLSigning usage, we'd cause our verification
helper, isRevInfoIssuerValid, to think the issuer ID is no longer value
(when instead, it just lacks crlSigning bits).
We address this by pulling in all issuers we know of for the
identification. This allows us to keep valid-but-not-for-signing
issuers, and use other representatives of their identity set for
signing/building the CRL (if they are enabled for such usage).
As a side effect, we now no longer place these entries on the default
CRL in the event all issuers in the CRL set are without the usage.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
This is only for the last commit.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow correct importing of certs without CRL KU
When Vault imports certificates without KU for CRLSign, we shouldn't
provision CRLUsage on the backing issuer; otherwise, we'll attempt to
build CRLs and Go will cause us to err out. This change makes it clear
(at issuer configuration time) that we can't possibly support this
operation and hopefully prevent users from running into the more cryptic
Go error.
Note that this does not apply for OCSP EKU: the EKU exists, per RFC 6960
Section 2.6 OCSP Signature Authority Delegation, to allow delegation of
OCSP signing to a child certificate. This EKU is not necessary on the
issuer itself, and generally assumes issuers are allowed to issue OCSP
responses regardless of KU/EKU.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs to clarify issue with import, CRL usage
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/pki.mdx
* Add additional test assertion
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* updated usage example
* Docs: updated examples with base64 - removed herestring for echo instead that's more simple.
* Docs: updated examples with base64 - removed herestring for echo instead that's more simple.
Co-authored-by: Mehdi Ahmadi <aphorise@gmail.com>
* auth/cert: Add metadata to identity-alias
Add the possibility to include certificate metadata in the created
logical.Alias (the identity alias), in addition to the metadata added
to logical.Auth. This is analogous to the behaviour of the ldap and
approle auth providers.
This possibility can be configured by the config endpoint of the
auth method mount and is disabled by default. We added the read
operation on this config endpoint as well.
Fixes: #14418
Signed-off-by: Peter Verraedt <peter.verraedt@kuleuven.be>
* Add changelog for #14751
Signed-off-by: Peter Verraedt <peter.verraedt@kuleuven.be>
* Test the usage of cert metadata in ACL policies
Signed-off-by: Peter Verraedt <peter@verraedt.be>
Signed-off-by: Peter Verraedt <peter.verraedt@kuleuven.be>
Signed-off-by: Peter Verraedt <peter@verraedt.be>
* Allow automatic rebuilding of CRLs
When enabled, periodic rebuilding of CRLs will improve PKI mounts in two
way:
1. Reduced load during periods of high (new) revocations, as the CRL
isn't rebuilt after each revocation but instead on a fixed schedule.
2. Ensuring the CRL is never stale as long as the cluster remains up,
by checking for next CRL expiry and regenerating CRLs before that
happens. This may increase cluster load when operators have large
CRLs that they'd prefer to let go stale, rather than regenerating
fresh copies.
In particular, we set a grace period before expiration of CRLs where,
when the periodic function triggers (about once a minute), we check
upcoming CRL expirations and check if we need to rebuild the CRLs.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on periodic rebuilding
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow modification of rollback period for testing
When testing backends that use the periodic func, and specifically,
testing the behavior of that periodic func, waiting for the usual 1m
interval can lead to excessively long test execution. By switching to a
shorter period--strictly for testing--we can make these tests execute
faster.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for auto-rebuilding of CRLs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove non-updating getConfig variant
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Avoid double reload of config
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Pull route paths out of cluster-route mixin
* Add redirect route and point there if authed and desired path is auth
* Cleanup test
* Use replaceWith instead of transitionTo
* Update tests
* Fix controller accessed by redirect route
* Add changelog
* Fix tests
* Refactor existing CRL function to storage getRevocationConfig
* Introduce ocsp_disable config option in config/crl
* Introduce OCSPSigning usage flag on issuer
* Add ocsp-request passthrough within lower layers of Vault
* Add OCSP responder to Vault PKI
* Add API documentation for OCSP
* Add cl
* Revert PKI storage migration modifications for OCSP
* Smaller PR feedback items
- pki.mdx doc update
- parens around logical.go comment to indicate DER encoded request is
related to OCSP and not the snapshots
- Use AllIssuers instead of writing them all out
- Drop zero initialization of crl config's Disable flag if not present
- Upgrade issuer on the fly instead of an initial migration
* Additional clean up backing out the writeRevocationConfig refactoring
* Remove Dirty issuer flag and update comment about not writing upgrade to
storage
* Address PR feedback and return Unknown response when mismatching issuer
* make fmt
* PR Feedback.
* More PR feedback
- Leverage ocsp response constant
- Remove duplicate errors regarding unknown issuers
* Migrate existing PKI mounts that only contains a key
- We missed testing a use-case of the migration that someone has a PKI
mount point that generated a CSR but never called set-signed back on
that mount point so it only contains a key.
* Add cl
* Add per-issuer AIA URI information
Per discussion on GitHub with @maxb, this allows issuers to have their
own copy of AIA URIs. Because each issuer has its own URLs (for CA and
CRL access), its necessary to mint their issued certs pointing to the
correct issuer and not to the global default issuer. For anyone using
multiple issuers within a mount, this change allows the issuer to point
back to itself via leaf's AIA info.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on per-issuer AIA info
Also add it to the considerations page as something to watch out for.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for per-issuer AIA information
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor AIA setting on the issuer
This introduces a common helper per Steve's suggestion.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify error messages w.r.t. AIA naming
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify error messages regarding AIA URLs
This clarifies which request parameter the invalid URL is contained
in, disambiguating the sometimes ambiguous usage of AIA, per suggestion
by Max.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename getURLs -> getGlobalAIAURLs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correct AIA acronym expansion word orders
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bad comment suggesting re-generating roots
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add two entries to URL tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow marking issuers as revoked
This allows PKI's issuers to be considered revoked and appear on each
others' CRLs. We disable issuance (via removing the usage) and prohibit
modifying the usage via the regular issuer management interface.
A separate endpoint is necessary because issuers (especially if signed
by a third-party CA using incremental serial numbers) might share a
serial number (e.g., an intermediate under cross-signing might share the
same number as an external root or an unrelated intermediate).
When the next CRL rebuild happens, this issuer will then appear on
others issuers CRLs, if they validate this issuer's certificate.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on revoking issuers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for issuer revocation semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Notate that CRLs will be rebuilt
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix timestamp field from _utc -> to _rfc3339
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure serial-based accesses shows as revoked
Thanks Kit!
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning when revoking default issuer
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* impr(ssh): fix bug with allowed_users_template and add allowed_domains_template field in SSH role configuration, closes#10943
* chore: add changelog entry
* Fix naming of permitted_dns_domains in webui
PKI has consistently used permitted_dns_domains since it was originally
introduced as a parameter. However, it appears the Web UI was updated to
add this field, but used an incorrect internal identifier
(permittedDnsNames rather than permittedDnsDomains).
This triggers a warning from the backend about an unknown parameter, and
the domain restriction isn't added:
> Endpoint ignored these unrecognized parameters: [permitted_dns_names]
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow Proof of Possession based revocation
Revocation by proof of possession ensures that we have a private key
matching the (provided or stored) certificate. This allows callers to
revoke certificate they own (as proven by holding the corresponding
private key), without having an admin create innumerable ACLs around
the serial_number parameter for every issuance/user.
We base this on Go TLS stack's verification of certificate<->key
matching, but extend it where applicable to ensure curves match, the
private key is indeed valid, and has the same structure as the
corresponding public key from the certificate.
This endpoint currently is authenticated, allowing operators to disable
the endpoint if it isn't desirable to use, via ACL policies.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify error message on ParseDERKey
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Leave revoke-with-key authenticated
After some discussion, given the potential for DoS (via submitting a lot
of keys/certs to validate, including invalid pairs), it seems best to
leave this as an authenticated endpoint. Presently in Vault, there's no
way to have an authenticated-but-unauthorized path (i.e., one which
bypasses ACL controls), so it is recommended (but not enforced) to make
this endpoint generally available by permissive ACL policies.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add API documentation on PoP
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add acceptance tests for Proof of Possession
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exercise negative cases in PoP tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ignore EC PARAMETER blocks during issuer import
While older versions of Vault supported sending this, we broke such
support in 1.11. Ignore them from the manage issuers endpoint (which is
aliased to the old /config/ca path) -- but keep erring in the import
keys paths. The latter is a new endpoint not aliased to anything and
only expects a single PEM block.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add regression test for EC PARAMs during import
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor serial creation to common helper
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add BYOC revocation to PKI mount
This allows operators to revoke certificates via a PEM blob passed to
Vault. In particular, Vault verifies the signature on the certificate
from an existing issuer within the mount, ensuring that one indeed
issued this certificate. The certificate is then added to storage and
its serial submitted for revocation.
This allows certificates generated with no_store=true to be submitted
for revocation afterwards, given a full copy of the certificate. As a
consequence, all roles can now safely move to no_store=true (if desired
for performance) and revocation can be done on a case-by-case basis.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs on BYOC revocation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PEM length check to BYOC import
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for BYOC
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Guard against legacy CA bundle usage
This prevents usage of the BYOC cert on a hybrid 1.10/1.12 cluster with
an non-upgraded CA issuer bundle.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Imported uuid library for initial commit to push a clean branch.
* Removed import statement in auth-form file since it was causing UI tests to fail as the import was not being used.
* Added nonce field to payload for okta sign in. (#16001)
* Added nonce field to payload for okta sign in.
* Added missing yarn package for uuid
* Fixed failing ui tests in cluster-test file to take into account of nonce field in the payload of okta login
* Removed uuid library and used crypto.randomUUID() to generate unique uuid values instead
* Fixed indent in package.json
* Removed uuid library since decided to use crypto.randomUUID() instead to generate unique uuid values
* Create polling function for correct answer in okta number challenge (#16070)
* Implemented polling function to get correct answer for okta number challenge.
* Disabled polling function for testing as it was causing acceptance test to fail in auth-test.js
* Changed API call to be the auth mount path instead of being static and created a variable to store the oktaNumberChallengeAnswer to be used later for the display screens
* Create component for okta number challenge screen (#16195)
* Implemented loading screen and display screen for correct answer for Okta Number Challenge
* Fixed linting issues on hbs files
* Added periods to parameter descriptions and made parameters optional
* Removed optional parameters from calling AuthForm component if authMethod is not Okta
* Implement error handling and screens for okta number challenge (#16276)
* Implemented loading screen and display screen for correct answer for Okta Number Challenge
* Fixed linting issues on hbs files
* Temporary changes to include error screen in okta number challenge
* Created error screen tests and made minor fixes
* Fixed error for wrong parameter name being passed in
* Fixed linting issues causing ui tests to fail
* Added periods at the end of param descriptions
* Imported uuid library for initial commit to push a clean branch.
* Removed import statement in auth-form file since it was causing UI tests to fail as the import was not being used.
* Removed uuid library since decided to use crypto.randomUUID() instead to generate unique uuid values
* Added nonce field to payload for okta sign in. (#16001)
* Added nonce field to payload for okta sign in.
* Added missing yarn package for uuid
* Fixed failing ui tests in cluster-test file to take into account of nonce field in the payload of okta login
* Removed uuid library and used crypto.randomUUID() to generate unique uuid values instead
* Fixed indent in package.json
* Create polling function for correct answer in okta number challenge (#16070)
* Implemented polling function to get correct answer for okta number challenge.
* Disabled polling function for testing as it was causing acceptance test to fail in auth-test.js
* Changed API call to be the auth mount path instead of being static and created a variable to store the oktaNumberChallengeAnswer to be used later for the display screens
* Create component for okta number challenge screen (#16195)
* Implemented loading screen and display screen for correct answer for Okta Number Challenge
* Fixed linting issues on hbs files
* Added periods to parameter descriptions and made parameters optional
* Removed optional parameters from calling AuthForm component if authMethod is not Okta
* Implement error handling and screens for okta number challenge (#16276)
* Implemented loading screen and display screen for correct answer for Okta Number Challenge
* Fixed linting issues on hbs files
* Temporary changes to include error screen in okta number challenge
* Created error screen tests and made minor fixes
* Fixed error for wrong parameter name being passed in
* Fixed linting issues causing ui tests to fail
* Added periods at the end of param descriptions
* UI/vault 7312/fix vault enterprise error for okta number challenge (#16568)
* Fixed bug with okta not working when selecting okta tab after being on other tab
* Fixed vault enterprise errors
* Fixed error when logging in with Okta in 'Other' tab
* Removed namespace parameter in option to use the default
* Added changelog
* VAULT-6818 delete unmerged entity aliases instead of orphaning them
* VAULT-6818 Prevent merge with clashing aliases, allow for resolution of clashing entity aliases
* VAULT-6818 Small updates
* VAULT-6818 Restrict to only one clash merge at once
* VAULT-6818 changelog
* VAULT-6818 use strutil package instead of slices
* VAULT-6818 Update variable names for clarity
* VAULT-6818 Update test
* VAULT-6818 update error message
* VAULT-6818 Use helper method
* VAULT-6818 validate entityIds
* VAULT-6818 group imports better
* VAULT-6818 use change instead of bug
* VAULT-6818 use multierror instead of custom struct
* VAULT-6818 Use multierror properly
* VAULT-6818 Small refactor based on feedback
* storage/raft: Fix cluster init with retry_join
Commit 8db66f4853abce3f432adcf1724b1f237b275415 introduced an error
wherein a join() would return nil (no error) with no information on its
channel if a joining node had been initialized. This was not handled
properly by the caller and resulted in a canceled `retry_join`.
Fix this by handling the `nil` channel respone by treating it as an
error and allowing the existing mechanics to work as intended.
* storage/raft: Improve retry_join go test
* storage/raft: Make VerifyRaftPeers pollable
* storage/raft: Add changelog entry for retry_join fix
* storage/raft: Add description to VerifyRaftPeers
* Add PSS signature support to Vault PKI engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use issuer's RevocationSigAlg for CRL signing
We introduce a new parameter on issuers, revocation_signature_algorithm
to control the signature algorithm used during CRL signing. This is
because the SignatureAlgorithm value from the certificate itself is
incorrect for this purpose: a RSA root could sign an ECDSA intermediate
with say, SHA256WithRSA, but when the intermediate goes to sign a CRL,
it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When
coupled with support for PSS-only keys, allowing the user to set the
signature algorithm value as desired seems like the best approach.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add use_pss, revocation_signature_algorithm docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PSS to signature role issuance test matrix
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow roots to self-identify revocation alg
When using PSS support with a managed key, sometimes the underlying
device will not support PKCS#1v1.5 signatures. This results in CRL
building failing, unless we update the entry's signature algorithm
prior to building the CRL for the new root.
With a RSA-type key and use_pss=true, we use the signature bits value to
decide which hash function to use for PSS support.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add clearer error message on failed import
When CRL building fails during cert/key import, due to PSS failures,
give a better indication to the user that import succeeded its just CRL
building that failed. This tells them the parameter to adjust on the
issuer and warns that CRL building will fail until this is fixed.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add case insensitive SigAlgo matching
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Convert UsePSS back to regular bool
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor PSS->certTemplate into helper function
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Proper string output on rev_sig_alg display
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Copy root's SignatureAlgorithm for CRL building
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow old certs to be cross-signed
In Vault 1.11, we introduced cross-signing support, but the earlier SKID
field change in Vault 1.10 causes problems: notably, certs created on
older versions of Vault (<=1.9) or outside of Vault (with a different
SKID method) cannot be cross-signed and validated in OpenSSL.
In particular, OpenSSL appears to be unique in requiring a SKID/AKID
match for chain building. If AKID and SKID are present on an otherwise
valid client/parent cert pair and the values are different, OpenSSL will
not build a valid path over those two, whereas most other chain
validation implementations will.
Regardless, to have proper cross-signing support, we really aught to
support copying an SKID. This adds such support to the sign-intermediate
endpoint. Support for the /issue endpoint is not added, as cross-signing
leaf certs isn't generally useful and can accept random SKIDs.
Resolves: #16461
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback, fix tests
Also adds a known-answer test using LE R3 CA's SKID.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback regarding separators
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Update Go to 1.18
From 1.17.12
1.18.5 was just released, but not all packages have been updated, so I
went with 1.18.4
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* identity/oidc: allow filtering the list providers response by an allowed_client_id
* adds changelog
* adds api documentation
* use identity store view in list provider test
Remove SHA1 for certs in prep for Go 1.18
* Remove certs with SHA1 from tests
* Use default SHA-256 with PKCS7 in AWS
* Update SHA1 deprecation note
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Added a small utility method to display warnings when parsing command arguments
Will print warning if flag is passed after arguments e.g.
vault <command> -a b -c
In this example -c will be interpreted as an argument which may be misleading
Add a new config option for Vault Agent's JWT auto auth
`remove_jwt_after_reading`, which defaults to true. Can stop
Agent from attempting to delete the file, which is useful in k8s
where the service account JWT is mounted as a read-only file
and so any attempt to delete it generates spammy error logs.
When leaving the JWT file in place, the read period for new
tokens is 1 minute instead of 500ms to reflect the assumption
that there will always be a file there, so finding a file does not
provide any signal that it needs to be re-read. Kubernetes
has a minimum TTL of 10 minutes for tokens, so a period of
1 minute gives Agent plenty of time to detect new tokens,
without leaving it too unresponsive. We may want to add a
config option to override these default periods in the future.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Remove gox in favor of go build.
`gox` hasn't had a release to update it in many years, so is missing
support for many modern systems, like `darwin/arm64`.
In any case, we only use it for dev builds, where we don't even use
the ability of it to build for multiple platforms. Release builds use
`go build` now.
So, this switches to `go build` everywhere.
I pulled this down and tested it in Windows as well. (Side note: I
couldn't get `gox` to work in Windows, so couldn't build before this
change.)
* docs(#16222): add documentation for changes in PR hashicorp/vault-plugin-secrets-kubernetes#10
* docs(#16222): add changelog entry
* docs(#16222): improve documentation to make the use case of setting both allowed_kubernetes_namespaces and allowed_kubernetes_namespace_selector parameters for role configuration
* use automtls for v5 secrets/auth plugins
* add automtls env guard
* start backend without metadata mode
* use PluginClientConfig for backend's NewPluginClient param
refactor
* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test
* add changelog
* prioritize automtls; improve comments
* user multierror; refactor pluginSet for v4 unit test
* add test cases for v4 and v5 plugin versions
* remove unnecessary call to AutoMTLSSupported
* update comment on pluginSets
* use runconfig directly in sdk newpluginclient
* use automtls without metadatamode for v5 backend plugin registration
* use multierror for plugin runconfig calls
* remove some unnecessary code
* raft: Ensure init before setting suffrage
As reported in https://hashicorp.atlassian.net/browse/VAULT-6773:
The /sys/storage/raft/join endpoint is intended to be unauthenticated. We rely
on the seal to manage trust.
It’s possible to use multiple join requests to switch nodes from voter to
non-voter. The screenshot shows a 3 node cluster where vault_2 is the leader,
and vault_3 and vault_4 are followers with non-voters set to false. sent two
requests to the raft join endpoint to have vault_3 and vault_4 join the cluster
with non_voters:true.
This commit fixes the issue by delaying the call to SetDesiredSuffrage until after
the initialization check, preventing unauthenticated mangling of voter status.
Tested locally using
https://github.com/hashicorp/vault-tools/blob/main/users/ncabatoff/cluster/raft.sh
and the reproducer outlined in VAULT-6773.
* raft: Return join err on failure
This is necessary to correctly distinguish errors returned from the Join
workflow. Previously, errors were being masked as timeouts.
* raft: Default autopilot parameters in teststorage
Change some defaults so we don't have to pass in parameters or set them
in the originating tests. These storage types are only used in two
places:
1) Raft HA testing
2) Seal migration testing
Both consumers have been tested and pass with this change.
* changelog: Unauthn voter status change bugfix
* Clarification for local mounts in the context of DR
The docs were unclear on this point, so @russparsloe and I looked into it.
Local mounts are indeed replicated to DR secondaries.
This is the opposite of what it says on https://developer.hashicorp.com/vault/tutorials/enterprise/performance-replication#disaster-recovery
> Local backend mounts are not replicated and their use will require existing DR mechanisms if DR is necessary in your implementation.
So that page will also need updating
* changelog
* fix changelog syntax for local mount with DR (#16218)
* pki: When a role sets key_type to any ignore key_bits value when signing
- Bypass the validation for the role's key_bits value when signing CSRs
if the key_type is set to any. We still validate the key is at least
2048 for RSA backed CSRs as we did in 1.9.x and lower.
* add func to set level for specific logger
* add endpoints to modify log level
* initialize base logger with IndependentLevels
* test to ensure other loggers remain unchanged
* add DELETE loggers endpoints to revert back to config
* add API docs page
* add changelog entry
* remove extraneous line
* add log level field to Core struct
* add godoc for getLogLevel
* add some loggers to c.allLoggers
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core
* Fix body handling
* Role resolution for rate limit quotas
* VAULT-6613 update precedence test
* Add changelog
* Handle body error
* VAULT-6613 Return early if error with json parsing
* Add signature_bits to sign-intermediate
This endpoint was lacking the signature_bits field like all the other
endpoints. Notably, in #15478, the ability to customize the intermediate
CSR's signature bits was removed without checking for the ability to
customize the final (root-signed) intermediate certificate's value.
This adds in that missing ability, bringing us parity with root
generation and role-based signing.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add signature_bits to sign-verbatim
This endpoint was also lacking the signature_bits field, preventing
other signature hash functions from being utilized here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Removed redundant checks for same env var in ReadEnvironment, extracted Unix domain socket logic to function, and made use of this logic in SetAddress. Adjusted unit tests to verify proper Unix domain socket handling.
* Adding case to revert from Unix domain socket dial function back to TCP
* Adding changelog file
* Only adjust DialContext if RoundTripper is an http.Transport
* Switching from read lock to normal lock
* only reset transport DialContext when setting different address type
* made ParseAddress a method on Config
* Adding additional tests to cover transitions to/from TCP to Unix
* Moved Config type method ParseAddress closer to type's other methods.
* make release note more end-user focused
* adopt review feedback to add comment about holding a lock
* ssh: Fix template regex test for defaultExtensions
- The regex to identify if our defaultExtensions contains a template was
a little too greedy, requiring the entire field to be just the regex. Allow
additional text within the value field to be added
* Add cl
* Support for rate limit path suffix quotas
* Support for rate limit path suffix quotas
* Precedence test for support for rate limit path suffix quotas
* Update clone method
* Fix mount determination
* Add changelog
* use constant for mounts
* Fix read endpoint, and remount/disable mount
* update godocs for queryquota
* Add cn_validations PKI Role parameter
This new parameter allows disabling all validations on a common name,
enabled by default on sign-verbatim and issuer generation options.
Presently, the default behavior is to allow either an email address
(denoted with an @ in the name) or a hostname to pass validation.
Operators can restrict roles to just a single option (e.g., for email
certs, limit CNs to have strictly email addresses and not hostnames).
By setting the value to `disabled`, CNs of other formats can be accepted
without validating their contents against our minimal correctness checks
for email/hostname/wildcard that we typically apply even when broad
permissions (allow_any_name=true, enforce_hostnames=false, and
allow_wildcard_certificates=true) are granted on the role.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update PKI tests for cn_validation support
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PKI API documentation on cn_validations
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow reading Nomad CA/Client cert configuration
In the Nomad secret engine, writing to /nomad/config/access allows users
to specify a CA certificate and client credential pair. However, these
values are not in the read of the endpoint, making it hard for operators
to see if these values were specified and if they need to be rotated.
Add `ca_cert` and `client_cert` parameters to the response, eliding the
`client_key` parameter as it is more sensitive (and should most likely
be replaced at the same time as `client_cert`).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix tests to expect additional fields
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test with existing CA/client cert+key
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Handle func
* Update - check if key_type and key_bits are allowed
* Update - fields
* Generating keys based on provided key_type and key_bits
* Returning signed key
* Refactor
* Refactor update to common logic function
* Descriptions
* Tests added
* Suggested changes and tests added and refactored
* Suggested changes and fmt run
* File refactoring
* Changelog file
* Update changelog/15561.txt
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Suggested changes - consistent returns and additional info to test messages
* ssh issue key pair documentation
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Add warning about EA in FIPS mode
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update path that metadata tab checks capabilities against
* Add changelog
* Update test to handle this case
* Fix tests url
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
* Add parsing for NSS-wrapped Ed25519 keys
NSS wraps Ed25519 using the PKCS#8 standard structure. The Go standard
library as of Go 1.18.x doesn't support parsing this key type with the
OID used by NSS; it requires the 1.3.101.112/RFC 8410 format, rather
than the RFC 5915-esque structure supported here.
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add integration test with NSS-created wrapped key
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
* Months after current are disabled, regardless of endTimeFromResponse
* move tracked values to getters for consistency
* months for widget are calculated in getter and then rendered
* Styling for current month is mix of hover and readonly
* Fix tests
* Add changelog
* Reset display year to endTimeFromResponse on toggle calendar
* update resetDisplayYear and naming
* Add test for displayYear when opened
* Support for CPS URLs in Custom Policy Identifiers.
* go fmt
* Add Changelog
* Fix panic in test-cases.
* Update builtin/logical/pki/path_roles.go
Fix intial nil identifiers.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Make valid policy OID so don't break ASN parse in test.
* Add test cases.
* go fmt.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Update AWS auth method certificates
Add tests that the `rsa2048` document can also be verified using the
`pkcs7` field for AWS auth.
Due to the use of SHA-1-based signatures for the `identity` and `pkcs7`
methods, we want to encourage moving toward using the RSA 2048 workflow,
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/verify-rsa2048.html
This doesn't require code changes for Vault necessarily, but adding in
the (many) certificates will help end users.
Also adds `rsa2048` option to API to fetch the RSA 2048 signature.
I will make a PR to update to the AWS auth docs to document the RSA 2048
flow soon after this.
* Removed red spellcheck underline for sensitive and secret KV values
* Added changelog file
* Moved spellcheck change into masked-input component file so that spellcheck does not apply for all sensitive fields
* VAULT-6131 OpenAPI schema now includes /auth/token endpoints when explicit permission has been granted
* VAULT-6131 add changelog
* VAULT-6131 Update changelog and fix related bug
* fix plugin reload mounts
* do not require sys/ prefix
* update plugin reload docs with examples
* fix unit test credential read path
* update docs to reflect correct cli usage
* allow sys/auth/foo or auth/foo
* append trailing slash if it doesn't exist in request
* add changelog
* use correct changelog number
* Add Read methods for KVClient
* KV write helper
* Add changelog
* Add Delete method
* Use extractVersionMetadata inside extractDataAndVersionMetadata
* Return nil, nil for v1 writes
* Add test for extracting version metadata
* Split kv client into v1 and v2-specific clients
* Add ability to set options on Put
* Add test for KV helpers
* Add custom metadata to top level and allow for getting versions as sorted slice
* Update tests
* Separate KV v1 and v2 into different files
* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity
* Move structs and godoc comments to more appropriate files
* Add more tests for extract methods
* Rework custom metadata helper to be more consistent with other helpers
* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method
* Return early for readability and make test value name less confusing
Add deprecation note about X.509/SHA-1
In preparation for moving to Go 1.18 in Vault 1.12.
Co-authored-by: Loann Le <84412881+taoism4504@users.noreply.github.com>
* Fixed unsupported revocation statements field display for DB roles
* Fixed linting
* Added changelog
* Fixed conditional to filter for only elasticsearch database and changed format of text in changelog
* Fixed conditional and added comment for bug fix
Updated documentation to describe the behavior when supplying `VAULT_HTTP_PROXY`. Also added support for `VAULT_PROXY_ADDR` as a 'better name' for `VAULT_HTTP_PROXY`.
* WIP replacing lib/pq
* change timezome param to be URI format
* add changelog
* add changelog for redshift
* update changelog
* add test for DSN style connection string
* more parseurl and quoteidentify to sdk; include copyright and license
* call dbutil.ParseURL instead, fix import ordering
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Allow role-based sign-verbatim with chosen issuer
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning with missing requested verbatim role
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pki/backend.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Fix handling of username_as_alias during LDAP authentication
There is a bug that was introduced in the LDAP authentication method by https://github.com/hashicorp/vault/pull/11000.
It was thought to be backward compatible but has broken a number of users. Later
a new parameter `username_as_alias` was introduced in https://github.com/hashicorp/vault/pull/14324
to make it possible for operators to restore the previous behavior.
The way it is currently working is not completely backward compatible thought
because when username_as_alias is set, a call to GetUserAliasAttributeValue() will
first be made, then this value is completely discarded in pathLogin() and replaced
by the username as expected.
This is an issue because it makes useless calls to the LDAP server and will break
backward compatibility if one of the constraints in GetUserAliasAttributeValue()
is not respected, even though the resulting value will be discarded anyway.
In order to maintain backward compatibility here we have to only call
GetUserAliasAttributeValue() if necessary.
Since this change of behavior was introduced in 1.9, this fix will need to be
backported to the 1.9, 1.10 and 1.11 branches.
* Add changelog
* Add tests
* Format code
* Update builtin/credential/ldap/backend.go
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Format and fix declaration
* Reword changelog
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Handle client count timezone
- Backend convert the timezone to UTC, to mitigate it's impact sending
start and end date other than 1. Chose 10 and 20 randomly.
* Added changelog
* Add a warning when Issuing Certificate set on a role does not resolve.
* Ivanka's requests - add a warning on deleting issuer or changing it's name.
* Fix nil checks; reduce number of roles to iterate through; only verify roles after migration.
* Fix semgrep failure, ignore roles deleted behind our back.
* Patch functionality for roles
* Make Patch Roles work again, add back patch issuers.
* Add changelog.
* Fix nil-reversion on empty response.
* Panics are bad. don't do that.
* Remove reference to stored license
- Stored license was deprecated in 1.8 and from 1.11 all licenses will be
auto loaded.
* Added changelog
* Remove test for stored license
* Add defensive check in serializer
* No default provider on create, add subText to service_account_file field
* Show empty state if no provider selected -- sorry for all the conditionals
* Button and distribution title styling on key edit
* Fix key distribute empty state permissions
* Don't try to fetch distribution if provider is permissionError
* Use search-select component for provider on distribute component
* Show distribution form errors on page rather than popup
* Add id, label, subtext to input-search for search-select fallback
* Remove created field from provider, default to querying for keys unless capabilities is false
* Fix link to provider from key-edit
* Search select label styling and add subText to fallback
* Refetch model after key rotate
* Create distribution method is task so we can load and disable button
* Move keymgmt to cloud group on mount options
* Key actions are tasks, fix tab active class
* Add isRunning attr to confirm-action which disables confirm button and replaces text with loader
* Fix provider active tab class
* Handle control groups on distribution
* Correctly handle error message on key-edit
* Show loading state on distribute, reload key after distribute
* Clear old validation errors if valid
* Fix tests
* Fix delete url
* Add changelog
* Address PR comments
* kick circle-ci
* Format go file breaking fmt
* Rename old changelog
* Remove resolved TODO
* Return signed ca as part of ca_chain field within sign-intermediate
- When signing a CA certificate we should include it along with the signing CA's CA chain in the response.
* PKI - Add not_before_duration API parameter to:
- Root CA generation
- Intermediate CA generation
- Intermediate CA signing
* Move not_before_duration to addCACommonFields
This gets applied on both root generation and intermediate signing,
which is the correct place to apply this.
Co-authored-by: guysv <sviryguy@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Resolves: #10631
Co-authored-by: guysv <sviryguy@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case for root/generate, sign-intermediate
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update path role description
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new not_before_duration to relevant docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: guysv <sviryguy@gmail.com>
* Add warning on missing AIA info fields
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog:
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Make sure that autopilot is disabled when we step down from active node state. Forward autopilot state requests to the active node. Avoid self-dialing due to stale advertisement.
* CLI: Add ability to display ListResponseWithInfos
The Vault Server API includes a ListResponseWithInfo call, allowing LIST
responses to contain additional information about their keys. This is in
a key=value mapping format (both for each key, to get the additional
metadata, as well as within each metadata).
Expand the `vault list` CLI command with a `-detailed` flag (and env var
VAULT_DETAILED_LISTS) to print this additional metadata. This looks
roughly like the following:
$ vault list -detailed pki/issuers
Keys issuer_name
---- -----------
0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7 n/a
35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0 n/a
382fad1e-e99c-9c54-e147-bb1faa8033d3 n/a
8bb4a793-2ad9-460c-9fa8-574c84a981f7 n/a
8bd231d7-20e2-f21f-ae1a-7aa3319715e7 n/a
9425d51f-cb81-426d-d6ad-5147d092094e n/a
ae679732-b497-ab0d-3220-806a2b9d81ed n/a
c5a44a1f-2ae4-2140-3acf-74b2609448cc utf8
d41d2419-efce-0e36-c96b-e91179a24dc1 something
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow detailed printing of LIST responses in JSON
When using the JSON formatter, only the absolute list of keys were
returned. Reuse the `-detailed` flag value for the `-format=json` list
response printer, allowing us to show the complete API response returned
by Vault.
This returns something like the following:
{
"request_id": "e9a25dcd-b67a-97d7-0f08-3670918ef3ff",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"key_info": {
"0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7": {
"issuer_name": ""
},
"35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0": {
"issuer_name": ""
},
"382fad1e-e99c-9c54-e147-bb1faa8033d3": {
"issuer_name": ""
},
"8bb4a793-2ad9-460c-9fa8-574c84a981f7": {
"issuer_name": ""
},
"8bd231d7-20e2-f21f-ae1a-7aa3319715e7": {
"issuer_name": ""
},
"9425d51f-cb81-426d-d6ad-5147d092094e": {
"issuer_name": ""
},
"ae679732-b497-ab0d-3220-806a2b9d81ed": {
"issuer_name": ""
},
"c5a44a1f-2ae4-2140-3acf-74b2609448cc": {
"issuer_name": "utf8"
},
"d41d2419-efce-0e36-c96b-e91179a24dc1": {
"issuer_name": "something"
}
},
"keys": [
"0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7",
"35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0",
"382fad1e-e99c-9c54-e147-bb1faa8033d3",
"8bb4a793-2ad9-460c-9fa8-574c84a981f7",
"8bd231d7-20e2-f21f-ae1a-7aa3319715e7",
"9425d51f-cb81-426d-d6ad-5147d092094e",
"ae679732-b497-ab0d-3220-806a2b9d81ed",
"c5a44a1f-2ae4-2140-3acf-74b2609448cc",
"d41d2419-efce-0e36-c96b-e91179a24dc1"
]
},
"warnings": null
}
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use field on UI rather than secret.Data
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Only include headers from visitable key_infos
Certain API endpoints return data from non-visitable key_infos, by
virtue of using a hand-rolled response. Limit our headers to those
from visitable key_infos. This means we won't return entire columns with
n/a entries, if no key matches the key_info key that includes that
header.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use setupEnv sourced detailed info
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix changelog environment variable
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix broken tests using setupEnv
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Warn on empty Subject field for issuers
When generating a root or signing an intermediate certificate, it is
possible to have Vault generate a certificate with an empty Subject.
These don't validate in most TLS implementations well, so add a warning.
Note that non-Common Name fields could be present to make a non-empty
subject, so simply requiring a CommonName isn't strictly the best.
For example:
$ vault write pki/root/generate/exported common_name=""
WARNING! The following warnings were returned from Vault:
* This issuer certificate was generated without a Subject; this makes
it likely that issuing leaf certs with this certificate will cause TLS
validation libraries to reject this certificate.
....
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove signature_bits on intermediate generate
This extraneous field wasn't respected during intermediate generation
and it isn't clear that it should be. Strictly, this field, if it were
to exist, would control the CSR's internal signature algorithm (certutil
defaults to the sane SHA-256 here). However, there's little value in
changing this as the signing authority can and probably will override
the final certificate's signature bits value, completely ignoring
whatever was in the provided CSR.
Removing this field will now cause warnings for those providing the
parameter (which already wasn't respected), which is the desired
behavior. No breakage should occur as a result of this change.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* TTL Picker convers to largest unit when value is number
* Initial value for transit auto-rotation period is 30d
* Add auto-rotation check to transit test
* Add changelog
* Add clarifying comment
* updating MFA to use official Okta SDK
* add changelog
* Update vault/login_mfa.go
Co-authored-by: swayne275 <swayne@hashicorp.com>
* cleanup query param building
* skip if not user factor
* updating struct tags to be more explicit
* fixing incorrect merge
* worrying that URL construction may change in the future, reimplementing GetFactorTransactionStatus
* adding some safety around url building
Co-authored-by: swayne275 <swayne@hashicorp.com>
* VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format
* VAULT-5885 Add changelog
* VAULT-5885 Update changelog as per PR comments
* VAULT-5885 Update changelog category to just 'auth'
* VAULT-5885 Hide useless token info in two-phase MFA case
* VAULT-5885 Update changelog to reflect token info now no longer present
* VAULT-5885 split up changelog into three blocks
* Always return non-nil CRL configuration
When using the default CRL configuration (as none has been set), return
the default configuration rather than inferring it in buildCRL. This
additionally allows us to return the default configuration on GET
operations to /config/crl.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Always return non-nil URL configuration
When using the default (empty) URL configuration as none has been set,
return the default configuration rather than inferring it inside of
fetchCAInfoByIssuerId or generateCert. This additionally allows us to
return the default configuration on GET operations to /config/urls.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use "not_before_duration" fiueld from role if above 0
* 'test' and update docs
* changelog file
* Requested changes - improved test and better description to changelog
* changelog description:
* update to ttl and not_before_duration API docs
* fill out nil response months in activity log query handle response based on requested month data
* changelog
* reverse month ordering for nil end months
* typo caught in ent test
* add import endpoint
* fix unlock
* add import_version
* refactor import endpoints and add tests
* add descriptions
* Update dependencies to include tink for Transit import operations. Convert Transit wrapping key endpoint to use shared wrapping key retrieval method. Disallow import of convergent keys to Transit via BYOK process.
* Include new 'hash_function' parameter on Transit import endpoints to specify OAEP random oracle hash function used to wrap ephemeral AES key.
* Add default values for Transit import endpoint fields. Prevent an OOB panic in Transit import. Proactively zero out ephemeral AES key used in Transit imports.
* Rename some Transit BYOK import variables. Ensure Transit BYOK ephemeral key is of the size specified byt the RFC.
* Add unit tests for Transit BYOK import endpoint.
* Simplify Transit BYOK import tests. Add a conditional on auto rotation to avoid errors on BYOK keys with allow_rotation=false.
* Added hash_function field to Transit import_version endpoint. Reworked Transit import unit tests. Added unit tests for Transit import_version endpoint.
* Add changelog entry for Transit BYOK.
* Transit BYOK formatting fixes.
* Omit 'convergent_encryption' field from Transit BYOK import endpoint, but reject with an error when the field is provided.
* Minor formatting fix in Transit import.
Co-authored-by: rculpepper <rculpepper@hashicorp.com>
* Add default timeout to legacy ssh.ClientConfig
When using the deprecated Dynamic SSH Keys method, Vault will make an
outbound SSH connection to an arbitrary remote host to place SSH keys.
We now set a timeout of 1 minute for this connection.
It is strongly recommended consumers of this SSH secrets engine feature
migrate to the more secure, and otherwise equivalent, SSH certificates
method.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix OIDC callback query params
- Value of namespace was getting stripped from the state query param
- Used native URL search param api to fetch the values
* Add changelog
* Remove unnecessary check for url encoding
* Extract ns value and pass as namespace param
* Update changelog
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.
Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.
I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
* POC of Okta Auth Number Challenge verification
* switch from callbacks to operations, forward validate to primary
* cleanup and nonce description update
* add changelog
* error on empty nonce, no forwarding, return correct_answer instead
* properly clean up verify goroutine
* add docs on new endpoint and parameters
* change polling frequency when WAITING to 1s
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* Starter PKI CA Storage API (#14796)
* Simple starting PKI storage api for CA rotation
* Add key and issuer storage apis
* Add listKeys and listIssuers storage implementations
* Add simple keys and issuers configuration storage api methods
* Handle resolving key, issuer references
The API context will usually have a user-specified reference to the key.
This is either the literal string "default" to select the default key,
an identifier of the key, or a slug name for the key. Here, we wish to
resolve this reference to an actual identifier that can be understood by
storage.
Also adds the missing Name field to keys.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add method to fetch an issuer's cert bundle
This adds a method to construct a certutil.CertBundle from the specified
issuer identifier, optionally loading its corresponding key for signing.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor certutil PrivateKey PEM handling
This refactors the parsing of PrivateKeys from PEM blobs into shared
methods (ParsePEMKey, ParseDERKey) that can be reused by the existing
Bundle parsing logic (ParsePEMBundle) or independently in the new
issuers/key-based PKI storage code.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add importKey, importCert to PKI storage
importKey is generally preferable to the low-level writeKey for adding
new entries. This takes only the contents of the private key (as a
string -- so a PEM bundle or a managed key handle) and checks if it
already exists in the storage.
If it does, it returns the existing key instance.
Otherwise, we create a new one. In the process, we detect any issuers
using this key and link them back to the new key entry.
The same holds for importCert over importKey, with the note that keys
are not modified when importing certificates.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for importing issuers, keys
This adds tests for importing keys and issuers into the new storage
layout, ensuring that identifiers are correctly inferred and linked.
Note that directly writing entries to storage (writeKey/writeissuer)
will take KeyID links from the parent entry and should not be used for
import; only existing entries should be updated with this info.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Implement PKI storage migration.
- Hook into the backend::initialize function, calling the migration on a primary only.
- Migrate an existing certificate bundle to the new issuers and key layout
* Make fetchCAInfo aware of new storage layout
This allows fetchCAInfo to fetch a specified issuer, via a reference
parameter provided by the user. We pass that into the storage layer and
have it return a cert bundle for us. Finally, we need to validate that
it truly has the key desired.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Begin /issuers API endpoints
This implements the fetch operations around issuers in the PKI Secrets
Engine. We implement the following operations:
- LIST /issuers - returns a list of known issuers' IDs and names.
- GET /issuer/:ref - returns a JSON blob with information about this
issuer.
- POST /issuer/:ref - allows configuring information about issuers,
presently just its name.
- DELETE /issuer/:ref - allows deleting the specified issuer.
- GET /issuer/:ref/{der,pem} - returns a raw API response with just
the DER (or PEM) of the issuer's certificate.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add import to PKI Issuers API
This adds the two core import code paths to the API:
/issuers/import/cert and /issuers/import/bundle. The former differs from
the latter in that the latter allows the import of keys. This allows
operators to restrict importing of keys to privileged roles, while
allowing more operators permission to import additional certificates
(not used for signing, but instead for path/chain building).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /issuer/:ref/sign-intermediate endpoint
This endpoint allows existing issuers to be used to sign intermediate
CA certificates. In the process, we've updated the existing
/root/sign-intermediate endpoint to be equivalent to a call to
/issuer/default/sign-intermediate.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /issuer/:ref/sign-self-issued endpoint
This endpoint allows existing issuers to be used to sign self-signed
certificates. In the process, we've updated the existing
/root/sign-self-issued endpoint to be equivalent to a call to
/issuer/default/sign-self-issued.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /issuer/:ref/sign-verbatim endpoint
This endpoint allows existing issuers to be used to directly sign CSRs.
In the process, we've updated the existing /sign-verbatim endpoint to be
equivalent to a call to /issuer/:ref/sign-verbatim.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow configuration of default issuers
Using the new updateDefaultIssuerId(...) from the storage migration PR
allows for easy implementation of configuring the default issuer. We
restrict callers from setting blank defaults and setting default to
default.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix fetching default issuers
After setting a default issuer, one should be able to use the old /ca,
/ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer
(and its chain). Update the fetchCertBySerial helper to no longer
support fetching the ca and prefer fetchCAInfo for that instead (as
we've already updated that to support fetching the new issuer location).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /issuer/:ref/{sign,issue}/:role
This updates the /sign and /issue endpoints, allowing them to take the
default issuer (if none is provided by a role) and adding
issuer-specific versions of them.
Note that at this point in time, the behavior isn't yet ideal (as
/sign/:role allows adding the ref=... parameter to override the default
issuer); a later change adding role-based issuer specification will fix
this incorrect behavior.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support root issuer generation
* Add support for issuer generate intermediate end-point
* Update issuer and key arguments to consistent values
- Update all new API endpoints to use the new agreed upon argument names.
- issuer_ref & key_ref to refer to existing
- issuer_name & key_name for new definitions
- Update returned values to always user issuer_id and key_id
* Add utility methods to fetch common ref and name arguments
- Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields.
- Centralize the logic to clean up these inputs and apply various validations to all of them.
* Rename common PKI backend handlers
- Use the buildPath convention for the function name instead of common...
* Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods
- PR feedback, move setting up the default configuration references within
the import methods instead of within the writeCaBundle method. This should
now cover all use cases of us setting up the defaults properly.
* Introduce constants for issuer_ref, rename isKeyDefaultSet...
* Fix legacy PKI sign-verbatim api path
- Addresses some test failures due to an incorrect refactoring of a legacy api
path /sign-verbatim within PKI
* Use import code to handle intermediate, config/ca
The existing bundle import code will satisfy the intermediate import;
use it instead of the old ca_bundle import logic. Additionally, update
/config/ca to use the new import code as well.
While testing, a panic was discovered:
> reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId
This was caused by returning a map with type issuerId->keyId; instead
switch to returning string->string maps so the audit log can properly
HMAC them.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify error message on missing defaults
When the default issuer and key are missing (and haven't yet been
specified), we should clarify that error message.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update test semantics for new changes
This makes two minor changes to the existing test suite:
1. Importing partial bundles should now succeed, where they'd
previously error.
2. fetchCertBySerial no longer handles CA certificates.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for deleting all keys, issuers
The old DELETE /root code must now delete all keys and issuers for
backwards compatibility. We strongly suggest calling individual delete
methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead,
for finer control.
In the process, we detect whether the deleted key/issuers was set as the
default. This will allow us to warn (from the single key/deletion issuer
code) whether or not the default was deleted (while allowing the
operation to succeed).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Introduce defaultRef constant within PKI
- Replace hardcoded "default" references with a constant to easily identify various usages.
- Use the addIssuerRefField function instead of redefining the field in various locations.
* Rework PKI test TestBackend_Root_Idempotency
- Validate that generate/root calls are no longer idempotent, but the bundle importing
does not generate new keys/issuers
- As before make sure that the delete root api resets everything
- Address a bug within the storage that we bombed when we had multiple different
key types within storage.
* Assign Name=current to migrated key and issuer
- Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer.
* Build CRL upon PKI intermediary set-signed api called
- Add a call to buildCRL if we created an issuer within pathImportIssuers
- Augment existing FullCAChain to verify we have a proper CRL post set-signed api call
- Remove a code block writing out "ca" storage entry that is no longer used.
* Identify which certificate or key failed
When importing complex chains, we should identify in which certificate
or key the failure occurred.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* PKI migration writes out empty migration log entry
- Since the elements of the struct were not exported we serialized an empty
migration log to disk and would re-run the migration
* Add chain-building logic to PKI issuers path
With the one-entry-per-issuer approach, CA Chains become implicitly
constructed from the pool of issuers. This roughly matches the existing
expectations from /config/ca (wherein a chain could be provided) and
/intemediate/set-signed (where a chain may be provided). However, in
both of those cases, we simply accepted a chain. Here, we need to be
able to reconstruct the chain from parts on disk.
However, with potential rotation of roots, we need to be aware of
disparate chains. Simply concating together all issuers isn't
sufficient. Thus we need to be able to parse a certificate's Issuer and
Subject field and reconstruct valid (and potentially parallel)
parent<->child mappings.
This attempts to handle roots, intermediates, cross-signed
intermediates, cross-signed roots, and rotated keys (wherein one might
not have a valid signature due to changed key material with the same
subject).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Return CA Chain when fetching issuers
This returns the CA Chain attribute of an issuer, showing its computed
chain based on other issuers in the database, when fetching a specific
issuer.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add testing for chain building
Using the issuance infrastructure, we generate new certificates (either
roots or intermediates), positing that this is roughly equivalent to
importing an external bundle (minus error handling during partial
imports). This allows us to incrementally construct complex chains,
creating reissuance cliques and cross-signing cycles.
By using ECDSA certificates, we avoid high signature verification and
key generation times.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow manual construction of issuer chain
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix handling of duplicate names
With the new issuer field (manual_chain), we can no longer err when a
name already exists: we might be updating the existing issuer (with the
same name), but changing its manual_chain field. Detect this error and
correctly handle it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for manual chain building
We break the clique, instead building these chains manually, ensuring
that the remaining chains do not change and only the modified certs
change. We then reset them (back to implicit chain building) and ensure
we get the same results as earlier.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add stricter verification of issuers PEM format
This ensures each issuer is only a single certificate entry (as
validated by count and parsing) without any trailing data.
We further ensure that each certificate PEM has leading and trailing
spaces removed with only a single trailing new line remaining.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix full chain building
Don't set the legacy IssuingCA field on the certificate bundle, as we
prefer the CAChain field over it.
Additionally, building the full chain could result in duplicate
certificates when the CAChain included the leaf certificate itself. When
building the full chain, ensure we don't include the bundle's
certificate twice.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add stricter tests for full chain construction
We wish to ensure that each desired certificate in the chain is only
present once.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename PKI types to avoid constant variable name collisions
keyId -> keyID
issuerId -> issuerID
key -> keyEntry
issuer -> issuerEntry
keyConfig -> keyConfigEntry
issuerConfig -> issuerConfigEntry
* Update CRL handling for multiple issuers
When building CRLs, we've gotta make sure certs issued by that issuer
land up on that issuer's CRL and not some other CRL. If no CRL is
found (matching a cert), we'll place it on the default CRL.
However, in the event of equivalent issuers (those with the same subject
AND the same key material) -- perhaps due to reissuance -- we'll only
create a single (unified) CRL for them.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow fetching updated CRL locations
This updates fetchCertBySerial to support querying the default issuer's
CRL.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove legacy CRL storage location test case
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update to CRLv2 Format to copy RawIssuer
When using the older Certificate.CreateCRL(...) call, Go's x509 library
copies the parsed pkix.Name version of the CRL Issuer's Subject field.
For certain constructed CAs, this fails since pkix.Name is not suitable
for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL.
In updating to the newer x509.CreateRevocationList(...) call, we can
construct the CRL in the CRLv2 format and correctly copy the issuer's
name. However, this requires holding an additional field per-CRL, the
CRLNumber field, which is required in Go's implementation of CRLv2
(though OPTIONAL in the spec). We store this on the new
LocalCRLConfigEntry object, per-CRL.
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add comment regarding CRL non-assignment in GOTO
In previous versions of Vault, it was possible to sign an empty CRL
(when the CRL was disabled and a force-rebuild was requested). Add a
comment about this case.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow fetching the specified issuer's CRL
We add a new API endpoint to fetch the specified issuer's CRL directly
(rather than the default issuer's CRL at /crl and /certs/crl). We also
add a new test to validate the CRL in a multi-root scenario and ensure
it is signed with the correct keys.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new PKI key prefix to seal wrapped storage (#15126)
* Refactor common backend initialization within backend_test
- Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage.
* Add ability to read legacy cert bundle if the migration has not occurred on secondaries.
- Track the migration state forbidding an issuer/key writing api call if we have not migrated
- For operations that just need to read the CA bundle, use the same tracking variable to
switch between reading the legacy bundle or use the new key/issuer storage.
- Add an invalidation function that will listen for updates to our log path to refresh the state
on secondary clusters.
* Always write migration entry to trigger secondary clusters to wake up
- Some PR feedback and handle a case in which the primary cluster does
not have a CA bundle within storage but somehow a secondary does.
* Update CA Chain to report entire chain
This merges the ca_chain JSON field (of the /certs/ca_chain path) with
the regular certificate field, returning the root of trust always. This
also affects the non-JSON (raw) endpoints as well.
We return the default issuer's chain here, rather than all known issuers
(as that may not form a strict chain).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow explicit issuer override on roles
When a role is used to generate a certificate (such as with the sign/
and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer
that issuer to the one on the request. This allows operators to set an
issuer (other than default) for requests to be issued against,
effectively making the change no different from the users' perspective
as it is "just" a different role name.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for role-based issuer selection
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expand NotAfter limit enforcement behavior
Vault previously strictly enforced NotAfter/ttl values on certificate
requests, erring if the requested TTL extended past the NotAfter date of
the issuer. In the event of issuing an intermediate, this behavior was
ignored, instead permitting the issuance.
Users generally do not think to check their issuer's NotAfter date when
requesting a certificate; thus this behavior was generally surprising.
Per RFC 5280 however, issuers need to maintain status information
throughout the life cycle of the issued cert. If this leaf cert were to
be issued for a longer duration than the parent issuer, the CA must
still maintain revocation information past its expiration.
Thus, we add an option to the issuer to change the desired behavior:
- err, to err out,
- permit, to permit the longer NotAfter date, or
- truncate, to silently truncate the expiration to the issuer's
NotAfter date.
Since expiration of certificates in the system's trust store are not
generally validated (when validating an arbitrary leaf, e.g., during TLS
validation), permit should generally only be used in that case. However,
browsers usually validate intermediate's validity periods, and thus
truncate should likely be used (as with permit, the leaf's chain will
not validate towards the end of the issuance period).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for expanded issuance behaviors
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning on keyless default issuer (#15178)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update PKI to new Operations framework (#15180)
The backend Framework has updated Callbacks (used extensively in PKI) to
become deprecated; Operations takes their place and clarifies forwarding
of requests.
We switch to the new format everywhere, updating some bad assumptions
about forwarding along the way. Anywhere writes are handled (that should
be propagated to all nodes in all clusters), we choose to forward the
request all the way up to the performance primary cluster's primary
node. This holds for issuers/keys, roles, and configs (such as CRL
config, which is globally set for all clusters despite all clusters
having their own separate CRL).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Kitography/vault 5474 rebase (#15150)
* These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.)
* Import and Generate API calls w/ needed additions to SDK.
* make fmt
* Add Help/Sync Text, fix some of internal/exported/kms code.
* Fix PEM/DER Encoding issue.
* make fmt
* Standardize keyIdParam, keyNameParam, keyTypeParam
* Add error response if key to be deleted is in use.
* replaces all instances of "default" in code with defaultRef
* Updates from Callbacks to Operations Function with explicit forwarding.
* Fixes a panic with names not being updated everywhere.
* add a logged error in addition to warning on deleting default key.
* Normalize whitespace upon importing keys.
Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Fix isKeyInUse functionality.
* Fixes tests associated with newline at end of key pem.
* Add alternative proposal PKI aliased paths (#15211)
* Add aliased path for root/rotate/:exported
This adds a user-friendly path name for generating a rotated root. We
automatically choose the name "next" for the newly generated root at
this path if it doesn't already exist.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add aliased path for intermediate/cross-sign
This allows cross-signatures to work.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add path for replacing the current root
This updates default to point to the value of the issuer with name
"next" rather than its current value.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove plural issuers/ in signing paths
These paths use a single issuer and thus shouldn't include the plural
issuers/ as a path prefix, instead using the singular issuer/ path
prefix.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Only warn if default issuer was imported
When the default issuer was not (re-)imported, we'd fail to find it,
causing an extraneous warning about missing keys, even though this
issuer indeed had a key.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing issuer sign/issue paths
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clean up various warnings within the PKI package (#15230)
* Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers
- Hook into the backend invalidation function so that secondaries are notified of
new/updated issuer or migrations occuring on the primary cluster. Upon notification
schedule a CRL rebuild to take place upon the next process to read/update the CRL
or within the periodic function if no request comes in.
* Schedule rebuilding PKI CRLs on active nodes only
- Address an issue that we were scheduling the rebuilding of a CRL on standby
nodes, which would not be able to write to storage.
- Fix an issue with standby nodes not correctly determining that a migration previously
occurred.
* Return legacy CRL storage path when no migration has occurred.
* Handle issuer, keys locking (#15227)
* Handle locking of issuers during writes
We need a write lock around writes to ensure serialization of
modifications. We use a single lock for both issuer and key
updates, in part because certain operations (like deletion) will
potentially affect both.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing b.useLegacyBundleCaStorage guards
Several locations needed to guard against early usage of the new issuers
endpoint pre-migration.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address PKI to properly support managed keys (#15256)
* Address codebase for managed key fixes
* Add proper public key comparison for better managed key support to importKeys
* Remove redundant public key fetching within PKI importKeys
* Correctly handle rebuilding remaining chains
When deleting a specific issuer, we might impact the chains. From a
consistency perspective, we need to ensure the remaining chains are
correct and don't refer to the since-deleted issuer, so trigger a full
rebuild here.
We don't need to call this in the delete-the-world (DELETE /root) code
path, as there shouldn't be any remaining issuers or chains to build.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove legacy CRL bundle on world deletion
When calling DELETE /root, we should remove the legacy CRL bundle, since
we're deleting the legacy CA issuer bundle as well.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove deleted issuers' CRL entries
Since CRLs are no longer resolvable after deletion (due to missing
issuer ID, which will cause resolution to fail regardless of if an ID or
a name/default reference was used), we should delete these CRLs from
storage to avoid leaking them.
In the event that this issuer comes back (with key material), we can
simply rebuild the CRL at that time (from the remaining revoked storage
entries).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add unauthed JSON fetching of CRLs, Issuers (#15253)
Default to fetching JSON CRL for consistency
This makes the bare issuer-specific CRL fetching endpoint return the
JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Add JSON-specific endpoint for fetching issuers
Unlike the unqualified /issuer/:ref endpoint (which also returns JSON),
we have a separate /issuer/:ref/json endpoint to return _only_ the
PEM-encoded certificate and the chain, mirroring the existing /cert/ca
endpoint but for a specific issuer. This allows us to make the endpoint
unauthenticated, whereas the bare endpoint would remain authenticated
and usually privileged.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Add tests for raw JSON endpoints
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add unauthenticated issuers endpoints to PKI table
This adds the unauthenticated issuers endpoints?
- LIST /issuers,
- Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and
- Fetching the CRL of this issuer (in JSON/DER/PEM form).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add issuer usage restrictions bitset
This allows issuers to have usage restrictions, limiting whether they
can be used to issue certificates or if they can generate CRLs. This
allows certain issuers to not generate a CRL (if the global config is
with the CRL enabled) or allows the issuer to not issue new certificates
(but potentially letting the CRL generation continue).
Setting both fields to false effectively forms a soft delete capability.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* PKI Pod rotation Add Base Changelog (#15283)
* PKI Pod rotation changelog.
* Use feature release-note formatting of changelog.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Co-authored-by: Kit Haines <kit.haines@hashicorp.com>
Co-authored-by: kitography <khaines@mit.edu>
Hamid Ghaf