Commit graph

15138 commits

Author SHA1 Message Date
Loann Le bbbb0bfc14
Vault documentation: updated unseal information (#15446)
* updated unseal info

* Update architecture.mdx

fixed spelling error

* updated based on feedback

* added new image

* Update website/content/docs/commands/operator/init.mdx

Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>

* Updates for accuracy

Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>
2022-05-16 14:44:23 -07:00
Chris Hoffman b14dc0d95d Remove duplicate policies when creating/updating identity groups (#15055)
* Remove duplicate policies for identity groups

* adding changelog

* test cleanup
2022-05-16 17:20:48 -04:00
Gabriel Santos 23e67be230
pki/sign-verbatim uses role not before duration (#15429)
* Use "not_before_duration" fiueld from role if above 0

* 'test' and update docs

* changelog file

* Requested changes - improved test and better description to changelog

* changelog description:

* update to ttl and not_before_duration API docs
2022-05-16 16:15:18 -04:00
Alexander Scheel 210045cd1f
Store migrated issuer, key in migration log (#15449)
If necessary, this will let us correlate migrated values afterwards.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 16:12:49 -04:00
Hridoy Roy 90538739bd
append nil months to query get to cover all requested months (OSS) (#15420)
* fill out nil response months in activity log query handle response based on requested month data

* changelog

* reverse month ordering for nil end months

* typo caught in ent test
2022-05-16 13:01:28 -07:00
AnPucel 390310409e
Add note about concurrency to plugin dev docs (#15357)
* Add note about concurrency

* Adding arrow syntax
2022-05-16 11:42:38 -07:00
Matt Schultz 3d1f40f850
Fix a keysutil policy lock (#15447)
* Defer an unlock call for keysutil policy imports.

* Make Transit import key type field case-insensitive for convenience.
2022-05-16 13:00:16 -05:00
Alexander Scheel 8750512f9f
Fix integer overflows with new parseutil (#15437)
* Use new parseutil helper: Safe variants

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update parseutil to v0.1.5

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix additional integer overflow in command/server

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:57:19 -04:00
Alexander Scheel 9b58e88efc
Update considerations for multiple issuers (#15442)
Also adds auditing section about suggested un-HMAC'd request/response
parameters.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:13:37 -04:00
Alexander Scheel dec6ee79d0
Fix ubi-docker-entrypoint.sh->docker-entrypoint.sh (#15443)
It is believed to break the helm chart; see also:

https://github.com/hashicorp/vault/pull/15272#discussion_r873927744

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:12:38 -04:00
Matt Schultz 611ab91e5a
Transit byok import endpoints (#15414)
* add import endpoint

* fix unlock

* add import_version

* refactor import endpoints and add tests

* add descriptions

* Update dependencies to include tink for Transit import operations. Convert Transit wrapping key endpoint to use shared wrapping key retrieval method. Disallow import of convergent keys to Transit via BYOK process.

* Include new 'hash_function' parameter on Transit import endpoints to specify OAEP random oracle hash function used to wrap ephemeral AES key.

* Add default values for Transit import endpoint fields. Prevent an OOB panic in Transit import. Proactively zero out ephemeral AES key used in Transit imports.

* Rename some Transit BYOK import variables. Ensure Transit BYOK ephemeral key is of the size specified byt the RFC.

* Add unit tests for Transit BYOK import endpoint.

* Simplify Transit BYOK import tests. Add a conditional on auto rotation to avoid errors on BYOK keys with allow_rotation=false.

* Added hash_function field to Transit import_version endpoint. Reworked Transit import unit tests. Added unit tests for Transit import_version endpoint.

* Add changelog entry for Transit BYOK.

* Transit BYOK formatting fixes.

* Omit 'convergent_encryption' field from Transit BYOK import endpoint, but reject with an error when the field is provided.

* Minor formatting fix in Transit import.

Co-authored-by: rculpepper <rculpepper@hashicorp.com>
2022-05-16 11:50:38 -05:00
Steven Clark 9607c5be97
Use backendUUID instead of mount points for managed keys (OSS) (#15441)
- Remove all references to mount point within PKI
 - Leverage the mount's backend UUID now instead of a mount point for all
   managed key lookups.
2022-05-16 12:48:54 -04:00
Alexander Scheel e1b1347e20
Set HOME=/home/vault in UBI Dockerfile (#15439)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 12:43:57 -04:00
Alexander Scheel 0ce7c3b331
Add default timeout to legacy ssh.ClientConfig (#15440)
* Add default timeout to legacy ssh.ClientConfig

When using the deprecated Dynamic SSH Keys method, Vault will make an
outbound SSH connection to an arbitrary remote host to place SSH keys.
We now set a timeout of 1 minute for this connection.

It is strongly recommended consumers of this SSH secrets engine feature
migrate to the more secure, and otherwise equivalent, SSH certificates
method.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 12:36:47 -04:00
Alexander Scheel 71372e4ea8
Include default information in LIST keys, issuers (#15435)
This shows whether the specified key or issuer is default, along with
the private key type in the case of a LIST /keys (authenticated) call.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 11:22:17 -04:00
Scott Miller b282bc4bd6
Plural typo in the Tokenization API docs for stores (#15424) 2022-05-13 14:00:33 -05:00
Tony Wittinger ba2984fa80
docs: updated monitor doc (#15358) 2022-05-13 11:19:51 -07:00
Arnav Palnitkar a0a67e671f
Fix OIDC callback query params (#15378)
* Fix OIDC callback query params

- Value of namespace was getting stripped from the state query param
- Used native URL search param api to fetch the values

* Add changelog

* Remove unnecessary check for url encoding

* Extract ns value and pass as namespace param

* Update changelog
2022-05-13 09:58:56 -07:00
Alexander Scheel d2bc5b5e3d
Restructure PKI Documentation Section (#15413)
* Rename pki.mdx -> pki/index.mdx

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off quick-start document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off considerations document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off intermediate CA setup document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off setup and usage document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Consistent quick-start doc naming

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add table of contents to index

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-13 11:22:07 -04:00
Alexander Scheel e6ad072542
UBI Containerfile - CRT Enablement (#15272)
* Copy UBI Dockerfile into Vault

This Dockerfile was modeled off of the existing Alpine Dockerfile (in
this repo) and the external Dockerfile from the docker-vault repo:

> https://github.com/hashicorp/docker-vault/blob/master/ubi/Dockerfile

We also import the UBI-specific docker-entrypoint.sh, as certain
RHEL/Alpine changes (like interpreter) require a separate entry script.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add UBI build to CRT pipeline

Also adds workflow_dispatch to the CRT pipeline, to allow manually
triggering CRT from PRs, when desired.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update Dockerfile

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>

* Update Dockerfile

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>

* Update Dockerfile

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>

* Update Dockerfile

* Update Dockerfile

* Update build.yml

Allow for both push to arbitrary branch plus workflow dispatch, per Newsletter article.

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>
2022-05-13 11:21:15 -04:00
Joe Rua 1e0ad4853a
Updated Docs For Specific Call Out (#14383)
The Parameter `cidr_list` is not support for Key_Type CA, customer was confused on this, so I feel we should specifically call this out to ensure there is no confusion
2022-05-13 10:42:20 -04:00
Alexander Scheel c64ac9d17a
Add tests for usage-based restrictions of issuers (#15411)
* Restructure leaf issuance test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add usage-based testing of issuing leaves, CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-13 09:57:58 -04:00
claire bontempo 125c13eb46
UI/remove chart check (#15407)
* remove chart check

* update assert number

* add settled

* more settled

* give up and remove flaky test entirely
2022-05-12 14:06:48 -07:00
Alexander Scheel f8b6077b8d
Fix unknown type parameter during CA generation (#15401)
* Fix unknown type parameter during CA generation

When generating a new PKI CA from the Web UI, the UI incorrectly sends
the type parameter in the POST body. The server will now warn on unknown
parameters, resulting in the UI surfacing that up to the caller. Since
the type is part of the URL, we don't need to duplicate it in the POST
body, so elide it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update ui/app/adapters/pki-ca-certificate.js

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
2022-05-12 16:28:47 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Chris Hoffman 63b31a13f9
Disabling client side rate limiting in Okta login MFA client (#15369)
* disabling client side rate limiting for MFA client

* add changelog
2022-05-12 15:55:33 -04:00
eoinoreilly30 6b82692f67
Fix typo (#15391) 2022-05-12 11:28:22 -07:00
Theron Voran 558e9a364b
docs: update for vault-k8s 0.16.0 (#15379) 2022-05-12 11:04:36 -07:00
swayne275 6f15ac255a
add mount accessor to audit request and responses (#15342)
* add mount accessor to audit request and responses

* add changelog

* cleanup todo
2022-05-12 11:28:00 -06:00
Christopher Swenson 0af0543bbe
VAULT-5935 agent: redact renew-self if using auto auth (#15380)
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.

Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.

I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
2022-05-12 09:25:55 -07:00
claire bontempo 8497010acd
UI/remove flaky client count tests (#15396)
* remove flaky tests

* and more flaky

* update assertions
2022-05-12 09:25:20 -07:00
Hridoy Roy 0c955eb743
report listener and storage types as found keys (#15383)
* report listener and storage types as found keys

* changelog
2022-05-12 09:04:56 -07:00
Alexander Scheel 01bd627ee9
Attempt to fix UI tests for PKI (#15374)
* ui/pki: Remove test for warning to re-generate root

With the multiple issuer feature merged from #15277, we can now allow
multiple root generations to occur in the mount. Remove the test for the
warning.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix test for empty CA chain

With Vault 1.11's #15277, we'll now always render the CAChain field on
response.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix formatting in role-ssh.js

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-12 11:20:50 -04:00
VAL d37f620591
Fix changelog formatting for feature (#15386) 2022-05-12 08:19:52 -07:00
Gabriel Santos 469ad6d09a
not_before_duration added to SSH (#15250)
* add-not-before-duration-to-ssh

* Missing field

* Adding tests

* changelog file

* Backend test

* Requested changes

* Update builtin/logical/ssh/path_roles.go

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
2022-05-12 08:50:40 -04:00
Calvin Leung Huang ca44b5a3e0
auth/okta: Add support for Okta number challenge (#15361)
* POC of Okta Auth Number Challenge verification

* switch from callbacks to operations, forward validate to primary

* cleanup and nonce description update

* add changelog

* error on empty nonce, no forwarding, return correct_answer instead

* properly clean up verify goroutine

* add docs on new endpoint and parameters

* change polling frequency when WAITING to 1s

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2022-05-11 17:09:29 -07:00
ldilalla-HC da7fe65aee
changelog++
* Update CHANGELOG.md

* Update CHANGELOG.md
2022-05-11 19:56:14 -04:00
Alexander Scheel b8142113bc
Add revocation check to chain building (#15371)
* Add CRL checking to chain building tests

This should ensure that, with our complex issuer setups, we can revoke
the issued certificates correctly and they'll show up on the correct
CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix related issuer detection in CRL building

When building our mapping of issuers, we incorrectly used the issuer's
RawIssuer field to construct the mapping, rather than the issuer's
RawSubject. This caused us to not correctly detect the cross-signed
issuers as having the same CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:49:20 -04:00
Steven Clark 24b168b250
Fix revoking certificates in pre-migration state within PKI (#15360)
* Address issues with revoke operations pre-migration of PKI issuers

 - Leverage the legacyBundleShimID though out the path of CRL building
   when legacy storage mode is active.
 - Instead of having multiple locations without a lock checking for the
   useLegacyBundleCaStorage flag is set, check it once and then use the
   same issuerId everywhere
 - Address some locking issues that might lead to a bad read/write when
   switching from legacy to non-legacy mode on startup and post-migration

* Add test suite for PKI apis pre-migration to new issuer storage format

 - Add tests that validate all apis work as expected in pre-migration mode
 - Add tests for apis that we don't expect to work, they should return a
   migration related error message
 - Add some missing validations on various new apis.
2022-05-11 13:33:04 -04:00
Christopher Swenson 4e4682247c
docs: fix typo in example custom db plugin (#15339) 2022-05-11 10:30:40 -07:00
Alexander Scheel de00d14a51
Benchmark chain building (#15315)
* Refactor chain building test cases to be shared

This will allow us to execute these test cases and then benchmark just
the chain building, separate from the certificate creation (and without
the consistency tests).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Benchmark chain building code

Using the existing test cases (and a few special ones), generate some
simple chains and benchmark how long chain building takes. We switch
from generating a cluster (slow) to directly calling
createBackendWithStorage(), which improves test execution time too:

$ go test -count=1 -run=Test_CAChainBuilding github.com/hashicorp/vault/builtin/logical/pki
ok  	github.com/hashicorp/vault/builtin/logical/pki	0.764s

(previously it was 5-10 seconds, for fewer tests).

Additionally, we now have benchmarks:

$ go test -v -run=BenchmarkChainBuilding -bench=. github.com/hashicorp/vault/builtin/logical/pki
goos: linux
goarch: amd64
pkg: github.com/hashicorp/vault/builtin/logical/pki
cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
BenchmarkChainBuilding
BenchmarkChainBuilding/test-case-0
BenchmarkChainBuilding/test-case-0-16         	     616	   1921783 ns/op
BenchmarkChainBuilding/test-case-1
BenchmarkChainBuilding/test-case-1-16         	    1191	    998201 ns/op
BenchmarkChainBuilding/test-case-2
BenchmarkChainBuilding/test-case-2-16         	     547	   2229810 ns/op
BenchmarkChainBuilding/test-case-3
BenchmarkChainBuilding/test-case-3-16         	     525	   2264951 ns/op
BenchmarkChainBuilding/test-case-4
BenchmarkChainBuilding/test-case-4-16         	    1732	    693686 ns/op
BenchmarkChainBuilding/test-case-5
BenchmarkChainBuilding/test-case-5-16         	   51700	     23230 ns/op
BenchmarkChainBuilding/test-case-6
BenchmarkChainBuilding/test-case-6-16         	    9343	    124523 ns/op
BenchmarkChainBuilding/test-case-7
BenchmarkChainBuilding/test-case-7-16         	    5106	    234902 ns/op
BenchmarkChainBuilding/test-case-8
BenchmarkChainBuilding/test-case-8-16         	    2334	    494382 ns/op
PASS
ok  	github.com/hashicorp/vault/builtin/logical/pki	12.707s

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:29:57 -04:00
Chelsea Shaw 0f8ef80336
UI: Fix version-history serializer and update mirage to reflect real API (#15362)
* Fix version-history serializer and update mirage to reflect real API

* add missing periods

* fix tests

Co-authored-by: Claire Bontempo <cbontempo@hashicorp.com>
2022-05-11 10:26:20 -07:00
Steven Clark 20a78a2118
Rework locking within the PKI CRLBuilder (#15325)
- Do not grab a lock within the requestRebuildIfActiveNode function
   to avoid issues being called from the invalidate function
 - Leverage more atmoic operations, and only grab the lock if we are
   going to perform the rebuild.
2022-05-11 13:14:21 -04:00
Alexander Scheel 48f3279c49
Return names for leaf_not_after_behavior responses (#15336)
Previously we'd return the raw enum value, which the entity accessing
the API wouldn't have any easy way of translating back into string
values. Return the string value directly instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:12:04 -04:00
Steven Clark 76cf103fb4
Use Get lookup within PKI resolveXXX functions instead of list iterations (#15322)
- Leverage a Get lookup operation to see if our reference field is a UUID
   instead of listing all key/issuers and iterating over the list.
 - This should be faster and we get a cached lookup possibly if it was a
   UUID entry that we previously loaded.
 - Address some small feedback about migration wording as well.
2022-05-11 13:11:56 -04:00
Alexander Scheel 8408c19115
Root issuers lack CA Chain + Chain Building Bug Fix (#15306)
* Return the ca_chain response from root issued cert api

* Fix parent selection in cert chain building

When building chains, we'd choose the next neighbor from Go's
unordered map. However, this doesn't necessarily result in the most
optimal path: we want to prefer to visit roots over other
intermediates, as this allows us to have a more consistent chain,
putting roots before their cross-signed equivalents rather than
potentially at the end.

We additionally now ensure chains are stable.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2022-05-11 13:09:18 -04:00
Steven Clark cb1cf36fcb
Add parameter validation to the new generate key PKI api (#15319)
- Validate the key_type and key_bits arguments that were provided and
   perform the same default processing of 0 as we used to do for the
   generateRoot/generateIntermediate apis
 - Add a test that validates the behaviour
 - Update the field description blurbs.
2022-05-11 13:07:18 -04:00
Steven Clark 92dad0e0c0
Compare issuer certificates using cert, signature algo and signature fields (#15285)
* Move existing test helpers into a new test_helpers.go file within PKI

* Compare issuer certificates by cert, signature algo and signature

 - Instead of comparing the strings of a certificate, instead leverage
   the Go Raw attribute within a parsed certificate to compare. The Raw
   attribute is a byte array of an ASN.1 DER containing the cert,
   signature algo and signature.
 - Rework a bit of the importIssuers function as well to fail checks on the
   inbound issuer earlier as well as load keys/issuers just before we need
   them
2022-05-11 13:04:54 -04:00
Alexander Scheel 575099d085
Prevent revoking all issuer certificates (#15289)
* Prevent revocation of issuers using revokeCert

Similar to the existing behavior, we'll prevent the revocation of
existing issuer certificates from the existing /revoke/:serial endpoint
for now. This is because a serial number alone is not enough information
(in the worst case) to precisely identify an issuer (as intermediates
signed by two separate external (e.g., OpenSSL) CAs using incremental
serial numbers might have the same serial number).

Additionally, we fix revoking certs on performance secondary clusters,
when they've not yet been migrated.

In a separate change, we'll open up a separate code path to revoke
issuers, ensuring we know exactly which issuer is revoked (and which CRL
it should belong on at time of revocation).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning when revoking expired cert

This prevents confusion when a nil response (with no revocation info) is
returned; requesters are informed that the specified certificate has
already expired.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:03:49 -04:00
Alexander Scheel 8695b6cca6
Add doc information about useLegacyBundleCaStorage (#15303)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:02:46 -04:00