Commit graph

15332 commits

Author SHA1 Message Date
Matt Schultz 3d1f40f850
Fix a keysutil policy lock (#15447)
* Defer an unlock call for keysutil policy imports.

* Make Transit import key type field case-insensitive for convenience.
2022-05-16 13:00:16 -05:00
Alexander Scheel 8750512f9f
Fix integer overflows with new parseutil (#15437)
* Use new parseutil helper: Safe variants

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update parseutil to v0.1.5

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix additional integer overflow in command/server

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:57:19 -04:00
Alexander Scheel 9b58e88efc
Update considerations for multiple issuers (#15442)
Also adds auditing section about suggested un-HMAC'd request/response
parameters.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:13:37 -04:00
Alexander Scheel dec6ee79d0
Fix ubi-docker-entrypoint.sh->docker-entrypoint.sh (#15443)
It is believed to break the helm chart; see also:

https://github.com/hashicorp/vault/pull/15272#discussion_r873927744

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:12:38 -04:00
Matt Schultz 611ab91e5a
Transit byok import endpoints (#15414)
* add import endpoint

* fix unlock

* add import_version

* refactor import endpoints and add tests

* add descriptions

* Update dependencies to include tink for Transit import operations. Convert Transit wrapping key endpoint to use shared wrapping key retrieval method. Disallow import of convergent keys to Transit via BYOK process.

* Include new 'hash_function' parameter on Transit import endpoints to specify OAEP random oracle hash function used to wrap ephemeral AES key.

* Add default values for Transit import endpoint fields. Prevent an OOB panic in Transit import. Proactively zero out ephemeral AES key used in Transit imports.

* Rename some Transit BYOK import variables. Ensure Transit BYOK ephemeral key is of the size specified byt the RFC.

* Add unit tests for Transit BYOK import endpoint.

* Simplify Transit BYOK import tests. Add a conditional on auto rotation to avoid errors on BYOK keys with allow_rotation=false.

* Added hash_function field to Transit import_version endpoint. Reworked Transit import unit tests. Added unit tests for Transit import_version endpoint.

* Add changelog entry for Transit BYOK.

* Transit BYOK formatting fixes.

* Omit 'convergent_encryption' field from Transit BYOK import endpoint, but reject with an error when the field is provided.

* Minor formatting fix in Transit import.

Co-authored-by: rculpepper <rculpepper@hashicorp.com>
2022-05-16 11:50:38 -05:00
Steven Clark 9607c5be97
Use backendUUID instead of mount points for managed keys (OSS) (#15441)
- Remove all references to mount point within PKI
 - Leverage the mount's backend UUID now instead of a mount point for all
   managed key lookups.
2022-05-16 12:48:54 -04:00
Alexander Scheel e1b1347e20
Set HOME=/home/vault in UBI Dockerfile (#15439)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 12:43:57 -04:00
Alexander Scheel 0ce7c3b331
Add default timeout to legacy ssh.ClientConfig (#15440)
* Add default timeout to legacy ssh.ClientConfig

When using the deprecated Dynamic SSH Keys method, Vault will make an
outbound SSH connection to an arbitrary remote host to place SSH keys.
We now set a timeout of 1 minute for this connection.

It is strongly recommended consumers of this SSH secrets engine feature
migrate to the more secure, and otherwise equivalent, SSH certificates
method.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 12:36:47 -04:00
Alexander Scheel 71372e4ea8
Include default information in LIST keys, issuers (#15435)
This shows whether the specified key or issuer is default, along with
the private key type in the case of a LIST /keys (authenticated) call.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 11:22:17 -04:00
Scott Miller b282bc4bd6
Plural typo in the Tokenization API docs for stores (#15424) 2022-05-13 14:00:33 -05:00
Tony Wittinger ba2984fa80
docs: updated monitor doc (#15358) 2022-05-13 11:19:51 -07:00
Arnav Palnitkar a0a67e671f
Fix OIDC callback query params (#15378)
* Fix OIDC callback query params

- Value of namespace was getting stripped from the state query param
- Used native URL search param api to fetch the values

* Add changelog

* Remove unnecessary check for url encoding

* Extract ns value and pass as namespace param

* Update changelog
2022-05-13 09:58:56 -07:00
Alexander Scheel d2bc5b5e3d
Restructure PKI Documentation Section (#15413)
* Rename pki.mdx -> pki/index.mdx

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off quick-start document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off considerations document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off intermediate CA setup document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Split off setup and usage document

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Consistent quick-start doc naming

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add table of contents to index

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-13 11:22:07 -04:00
Alexander Scheel e6ad072542
UBI Containerfile - CRT Enablement (#15272)
* Copy UBI Dockerfile into Vault

This Dockerfile was modeled off of the existing Alpine Dockerfile (in
this repo) and the external Dockerfile from the docker-vault repo:

> https://github.com/hashicorp/docker-vault/blob/master/ubi/Dockerfile

We also import the UBI-specific docker-entrypoint.sh, as certain
RHEL/Alpine changes (like interpreter) require a separate entry script.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add UBI build to CRT pipeline

Also adds workflow_dispatch to the CRT pipeline, to allow manually
triggering CRT from PRs, when desired.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update Dockerfile

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>

* Update Dockerfile

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>

* Update Dockerfile

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>

* Update Dockerfile

* Update Dockerfile

* Update build.yml

Allow for both push to arbitrary branch plus workflow dispatch, per Newsletter article.

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>
2022-05-13 11:21:15 -04:00
Joe Rua 1e0ad4853a
Updated Docs For Specific Call Out (#14383)
The Parameter `cidr_list` is not support for Key_Type CA, customer was confused on this, so I feel we should specifically call this out to ensure there is no confusion
2022-05-13 10:42:20 -04:00
Alexander Scheel c64ac9d17a
Add tests for usage-based restrictions of issuers (#15411)
* Restructure leaf issuance test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add usage-based testing of issuing leaves, CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-13 09:57:58 -04:00
claire bontempo 125c13eb46
UI/remove chart check (#15407)
* remove chart check

* update assert number

* add settled

* more settled

* give up and remove flaky test entirely
2022-05-12 14:06:48 -07:00
Alexander Scheel f8b6077b8d
Fix unknown type parameter during CA generation (#15401)
* Fix unknown type parameter during CA generation

When generating a new PKI CA from the Web UI, the UI incorrectly sends
the type parameter in the POST body. The server will now warn on unknown
parameters, resulting in the UI surfacing that up to the caller. Since
the type is part of the URL, we don't need to duplicate it in the POST
body, so elide it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update ui/app/adapters/pki-ca-certificate.js

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
2022-05-12 16:28:47 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Chris Hoffman 63b31a13f9
Disabling client side rate limiting in Okta login MFA client (#15369)
* disabling client side rate limiting for MFA client

* add changelog
2022-05-12 15:55:33 -04:00
eoinoreilly30 6b82692f67
Fix typo (#15391) 2022-05-12 11:28:22 -07:00
Theron Voran 558e9a364b
docs: update for vault-k8s 0.16.0 (#15379) 2022-05-12 11:04:36 -07:00
swayne275 6f15ac255a
add mount accessor to audit request and responses (#15342)
* add mount accessor to audit request and responses

* add changelog

* cleanup todo
2022-05-12 11:28:00 -06:00
Christopher Swenson 0af0543bbe
VAULT-5935 agent: redact renew-self if using auto auth (#15380)
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.

Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.

I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
2022-05-12 09:25:55 -07:00
claire bontempo 8497010acd
UI/remove flaky client count tests (#15396)
* remove flaky tests

* and more flaky

* update assertions
2022-05-12 09:25:20 -07:00
Hridoy Roy 0c955eb743
report listener and storage types as found keys (#15383)
* report listener and storage types as found keys

* changelog
2022-05-12 09:04:56 -07:00
Alexander Scheel 01bd627ee9
Attempt to fix UI tests for PKI (#15374)
* ui/pki: Remove test for warning to re-generate root

With the multiple issuer feature merged from #15277, we can now allow
multiple root generations to occur in the mount. Remove the test for the
warning.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix test for empty CA chain

With Vault 1.11's #15277, we'll now always render the CAChain field on
response.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix formatting in role-ssh.js

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-12 11:20:50 -04:00
VAL d37f620591
Fix changelog formatting for feature (#15386) 2022-05-12 08:19:52 -07:00
Gabriel Santos 469ad6d09a
not_before_duration added to SSH (#15250)
* add-not-before-duration-to-ssh

* Missing field

* Adding tests

* changelog file

* Backend test

* Requested changes

* Update builtin/logical/ssh/path_roles.go

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
2022-05-12 08:50:40 -04:00
Calvin Leung Huang ca44b5a3e0
auth/okta: Add support for Okta number challenge (#15361)
* POC of Okta Auth Number Challenge verification

* switch from callbacks to operations, forward validate to primary

* cleanup and nonce description update

* add changelog

* error on empty nonce, no forwarding, return correct_answer instead

* properly clean up verify goroutine

* add docs on new endpoint and parameters

* change polling frequency when WAITING to 1s

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2022-05-11 17:09:29 -07:00
ldilalla-HC da7fe65aee
changelog++
* Update CHANGELOG.md

* Update CHANGELOG.md
2022-05-11 19:56:14 -04:00
Alexander Scheel b8142113bc
Add revocation check to chain building (#15371)
* Add CRL checking to chain building tests

This should ensure that, with our complex issuer setups, we can revoke
the issued certificates correctly and they'll show up on the correct
CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix related issuer detection in CRL building

When building our mapping of issuers, we incorrectly used the issuer's
RawIssuer field to construct the mapping, rather than the issuer's
RawSubject. This caused us to not correctly detect the cross-signed
issuers as having the same CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:49:20 -04:00
Steven Clark 24b168b250
Fix revoking certificates in pre-migration state within PKI (#15360)
* Address issues with revoke operations pre-migration of PKI issuers

 - Leverage the legacyBundleShimID though out the path of CRL building
   when legacy storage mode is active.
 - Instead of having multiple locations without a lock checking for the
   useLegacyBundleCaStorage flag is set, check it once and then use the
   same issuerId everywhere
 - Address some locking issues that might lead to a bad read/write when
   switching from legacy to non-legacy mode on startup and post-migration

* Add test suite for PKI apis pre-migration to new issuer storage format

 - Add tests that validate all apis work as expected in pre-migration mode
 - Add tests for apis that we don't expect to work, they should return a
   migration related error message
 - Add some missing validations on various new apis.
2022-05-11 13:33:04 -04:00
Christopher Swenson 4e4682247c
docs: fix typo in example custom db plugin (#15339) 2022-05-11 10:30:40 -07:00
Alexander Scheel de00d14a51
Benchmark chain building (#15315)
* Refactor chain building test cases to be shared

This will allow us to execute these test cases and then benchmark just
the chain building, separate from the certificate creation (and without
the consistency tests).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Benchmark chain building code

Using the existing test cases (and a few special ones), generate some
simple chains and benchmark how long chain building takes. We switch
from generating a cluster (slow) to directly calling
createBackendWithStorage(), which improves test execution time too:

$ go test -count=1 -run=Test_CAChainBuilding github.com/hashicorp/vault/builtin/logical/pki
ok  	github.com/hashicorp/vault/builtin/logical/pki	0.764s

(previously it was 5-10 seconds, for fewer tests).

Additionally, we now have benchmarks:

$ go test -v -run=BenchmarkChainBuilding -bench=. github.com/hashicorp/vault/builtin/logical/pki
goos: linux
goarch: amd64
pkg: github.com/hashicorp/vault/builtin/logical/pki
cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
BenchmarkChainBuilding
BenchmarkChainBuilding/test-case-0
BenchmarkChainBuilding/test-case-0-16         	     616	   1921783 ns/op
BenchmarkChainBuilding/test-case-1
BenchmarkChainBuilding/test-case-1-16         	    1191	    998201 ns/op
BenchmarkChainBuilding/test-case-2
BenchmarkChainBuilding/test-case-2-16         	     547	   2229810 ns/op
BenchmarkChainBuilding/test-case-3
BenchmarkChainBuilding/test-case-3-16         	     525	   2264951 ns/op
BenchmarkChainBuilding/test-case-4
BenchmarkChainBuilding/test-case-4-16         	    1732	    693686 ns/op
BenchmarkChainBuilding/test-case-5
BenchmarkChainBuilding/test-case-5-16         	   51700	     23230 ns/op
BenchmarkChainBuilding/test-case-6
BenchmarkChainBuilding/test-case-6-16         	    9343	    124523 ns/op
BenchmarkChainBuilding/test-case-7
BenchmarkChainBuilding/test-case-7-16         	    5106	    234902 ns/op
BenchmarkChainBuilding/test-case-8
BenchmarkChainBuilding/test-case-8-16         	    2334	    494382 ns/op
PASS
ok  	github.com/hashicorp/vault/builtin/logical/pki	12.707s

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:29:57 -04:00
Chelsea Shaw 0f8ef80336
UI: Fix version-history serializer and update mirage to reflect real API (#15362)
* Fix version-history serializer and update mirage to reflect real API

* add missing periods

* fix tests

Co-authored-by: Claire Bontempo <cbontempo@hashicorp.com>
2022-05-11 10:26:20 -07:00
Steven Clark 20a78a2118
Rework locking within the PKI CRLBuilder (#15325)
- Do not grab a lock within the requestRebuildIfActiveNode function
   to avoid issues being called from the invalidate function
 - Leverage more atmoic operations, and only grab the lock if we are
   going to perform the rebuild.
2022-05-11 13:14:21 -04:00
Alexander Scheel 48f3279c49
Return names for leaf_not_after_behavior responses (#15336)
Previously we'd return the raw enum value, which the entity accessing
the API wouldn't have any easy way of translating back into string
values. Return the string value directly instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:12:04 -04:00
Steven Clark 76cf103fb4
Use Get lookup within PKI resolveXXX functions instead of list iterations (#15322)
- Leverage a Get lookup operation to see if our reference field is a UUID
   instead of listing all key/issuers and iterating over the list.
 - This should be faster and we get a cached lookup possibly if it was a
   UUID entry that we previously loaded.
 - Address some small feedback about migration wording as well.
2022-05-11 13:11:56 -04:00
Alexander Scheel 8408c19115
Root issuers lack CA Chain + Chain Building Bug Fix (#15306)
* Return the ca_chain response from root issued cert api

* Fix parent selection in cert chain building

When building chains, we'd choose the next neighbor from Go's
unordered map. However, this doesn't necessarily result in the most
optimal path: we want to prefer to visit roots over other
intermediates, as this allows us to have a more consistent chain,
putting roots before their cross-signed equivalents rather than
potentially at the end.

We additionally now ensure chains are stable.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2022-05-11 13:09:18 -04:00
Steven Clark cb1cf36fcb
Add parameter validation to the new generate key PKI api (#15319)
- Validate the key_type and key_bits arguments that were provided and
   perform the same default processing of 0 as we used to do for the
   generateRoot/generateIntermediate apis
 - Add a test that validates the behaviour
 - Update the field description blurbs.
2022-05-11 13:07:18 -04:00
Steven Clark 92dad0e0c0
Compare issuer certificates using cert, signature algo and signature fields (#15285)
* Move existing test helpers into a new test_helpers.go file within PKI

* Compare issuer certificates by cert, signature algo and signature

 - Instead of comparing the strings of a certificate, instead leverage
   the Go Raw attribute within a parsed certificate to compare. The Raw
   attribute is a byte array of an ASN.1 DER containing the cert,
   signature algo and signature.
 - Rework a bit of the importIssuers function as well to fail checks on the
   inbound issuer earlier as well as load keys/issuers just before we need
   them
2022-05-11 13:04:54 -04:00
Alexander Scheel 575099d085
Prevent revoking all issuer certificates (#15289)
* Prevent revocation of issuers using revokeCert

Similar to the existing behavior, we'll prevent the revocation of
existing issuer certificates from the existing /revoke/:serial endpoint
for now. This is because a serial number alone is not enough information
(in the worst case) to precisely identify an issuer (as intermediates
signed by two separate external (e.g., OpenSSL) CAs using incremental
serial numbers might have the same serial number).

Additionally, we fix revoking certs on performance secondary clusters,
when they've not yet been migrated.

In a separate change, we'll open up a separate code path to revoke
issuers, ensuring we know exactly which issuer is revoked (and which CRL
it should belong on at time of revocation).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning when revoking expired cert

This prevents confusion when a nil response (with no revocation info) is
returned; requesters are informed that the specified certificate has
already expired.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:03:49 -04:00
Alexander Scheel 8695b6cca6
Add doc information about useLegacyBundleCaStorage (#15303)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-11 13:02:46 -04:00
Alexander Scheel 435450ea6f
Update API docs for PKI multi-issuer functionality (#15238)
* Update API docs for multiple issuer functionality

This substantially restructures the PKI secret engine's docs for two
purposes:

 1. To provide an explicit grouping of APIs by user usage and roles,
 2. To add all of the new APIs, hopefully with as minimal duplication
    as possible.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add section on vault cli with DER/PEM response formats

 - Add [1] links next to the DER/PEM format entries within various PKI
   response tables. These link to a new section explaining that the vault
   cli does not support DER/PEM response formats
 - Remove repetition of vault cli blurb in various description fields.
 - Fix up some typos

* Restructure API docs and add missing sections

Also addresses minor nits in the content.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify some language in the API docs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2022-05-11 12:50:20 -04:00
Alexander Scheel 4f6c6ac317
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277)
* Starter PKI CA Storage API (#14796)

* Simple starting PKI storage api for CA rotation
* Add key and issuer storage apis
* Add listKeys and listIssuers storage implementations
* Add simple keys and issuers configuration storage api methods

* Handle resolving key, issuer references

The API context will usually have a user-specified reference to the key.
This is either the literal string "default" to select the default key,
an identifier of the key, or a slug name for the key. Here, we wish to
resolve this reference to an actual identifier that can be understood by
storage.

Also adds the missing Name field to keys.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add method to fetch an issuer's cert bundle

This adds a method to construct a certutil.CertBundle from the specified
issuer identifier, optionally loading its corresponding key for signing.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor certutil PrivateKey PEM handling

This refactors the parsing of PrivateKeys from PEM blobs into shared
methods (ParsePEMKey, ParseDERKey) that can be reused by the existing
Bundle parsing logic (ParsePEMBundle) or independently in the new
issuers/key-based PKI storage code.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add importKey, importCert to PKI storage

importKey is generally preferable to the low-level writeKey for adding
new entries. This takes only the contents of the private key (as a
string -- so a PEM bundle or a managed key handle) and checks if it
already exists in the storage.

If it does, it returns the existing key instance.

Otherwise, we create a new one. In the process, we detect any issuers
using this key and link them back to the new key entry.

The same holds for importCert over importKey, with the note that keys
are not modified when importing certificates.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for importing issuers, keys

This adds tests for importing keys and issuers into the new storage
layout, ensuring that identifiers are correctly inferred and linked.

Note that directly writing entries to storage (writeKey/writeissuer)
will take KeyID links from the parent entry and should not be used for
import; only existing entries should be updated with this info.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Implement PKI storage migration.

 - Hook into the backend::initialize function, calling the migration on a primary only.
 - Migrate an existing certificate bundle to the new issuers and key layout

* Make fetchCAInfo aware of new storage layout

This allows fetchCAInfo to fetch a specified issuer, via a reference
parameter provided by the user. We pass that into the storage layer and
have it return a cert bundle for us. Finally, we need to validate that
it truly has the key desired.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Begin /issuers API endpoints

This implements the fetch operations around issuers in the PKI Secrets
Engine. We implement the following operations:

 - LIST /issuers - returns a list of known issuers' IDs and names.
 - GET /issuer/:ref - returns a JSON blob with information about this
   issuer.
 - POST /issuer/:ref - allows configuring information about issuers,
   presently just its name.
 - DELETE /issuer/:ref - allows deleting the specified issuer.
 - GET /issuer/:ref/{der,pem} - returns a raw API response with just
   the DER (or PEM) of the issuer's certificate.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add import to PKI Issuers API

This adds the two core import code paths to the API:
/issuers/import/cert and /issuers/import/bundle. The former differs from
the latter in that the latter allows the import of keys. This allows
operators to restrict importing of keys to privileged roles, while
allowing more operators permission to import additional certificates
(not used for signing, but instead for path/chain building).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add /issuer/:ref/sign-intermediate endpoint

This endpoint allows existing issuers to be used to sign intermediate
CA certificates. In the process, we've updated the existing
/root/sign-intermediate endpoint to be equivalent to a call to
/issuer/default/sign-intermediate.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add /issuer/:ref/sign-self-issued endpoint

This endpoint allows existing issuers to be used to sign self-signed
certificates. In the process, we've updated the existing
/root/sign-self-issued endpoint to be equivalent to a call to
/issuer/default/sign-self-issued.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add /issuer/:ref/sign-verbatim endpoint

This endpoint allows existing issuers to be used to directly sign CSRs.
In the process, we've updated the existing /sign-verbatim endpoint to be
equivalent to a call to /issuer/:ref/sign-verbatim.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow configuration of default issuers

Using the new updateDefaultIssuerId(...) from the storage migration PR
allows for easy implementation of configuring the default issuer. We
restrict callers from setting blank defaults and setting default to
default.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix fetching default issuers

After setting a default issuer, one should be able to use the old /ca,
/ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer
(and its chain). Update the fetchCertBySerial helper to no longer
support fetching the ca and prefer fetchCAInfo for that instead (as
we've already updated that to support fetching the new issuer location).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add /issuer/:ref/{sign,issue}/:role

This updates the /sign and /issue endpoints, allowing them to take the
default issuer (if none is provided by a role) and adding
issuer-specific versions of them.

Note that at this point in time, the behavior isn't yet ideal (as
/sign/:role allows adding the ref=... parameter to override the default
issuer); a later change adding role-based issuer specification will fix
this incorrect behavior.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support root issuer generation

* Add support for issuer generate intermediate end-point

* Update issuer and key arguments to consistent values

 - Update all new API endpoints to use the new agreed upon argument names.
   - issuer_ref & key_ref to refer to existing
   - issuer_name & key_name for new definitions
 - Update returned values to always user issuer_id and key_id

* Add utility methods to fetch common ref and name arguments

 - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields.
 - Centralize the logic to clean up these inputs and apply various validations to all of them.

* Rename common PKI backend handlers

 - Use the buildPath convention for the function name instead of common...

* Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods

 - PR feedback, move setting up the default configuration references within
   the import methods instead of within the writeCaBundle method. This should
   now cover all use cases of us setting up the defaults properly.

* Introduce constants for issuer_ref, rename isKeyDefaultSet...

* Fix legacy PKI sign-verbatim api path

 - Addresses some test failures due to an incorrect refactoring of a legacy api
   path /sign-verbatim within PKI

* Use import code to handle intermediate, config/ca

The existing bundle import code will satisfy the intermediate import;
use it instead of the old ca_bundle import logic. Additionally, update
/config/ca to use the new import code as well.

While testing, a panic was discovered:

> reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId

This was caused by returning a map with type issuerId->keyId; instead
switch to returning string->string maps so the audit log can properly
HMAC them.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify error message on missing defaults

When the default issuer and key are missing (and haven't yet been
specified), we should clarify that error message.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update test semantics for new changes

This makes two minor changes to the existing test suite:

 1. Importing partial bundles should now succeed, where they'd
    previously error.
 2. fetchCertBySerial no longer handles CA certificates.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for deleting all keys, issuers

The old DELETE /root code must now delete all keys and issuers for
backwards compatibility. We strongly suggest calling individual delete
methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead,
for finer control.

In the process, we detect whether the deleted key/issuers was set as the
default. This will allow us to warn (from the single key/deletion issuer
code) whether or not the default was deleted (while allowing the
operation to succeed).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Introduce defaultRef constant within PKI

 - Replace hardcoded "default" references with a constant to easily identify various usages.
 - Use the addIssuerRefField function instead of redefining the field in various locations.

* Rework PKI test TestBackend_Root_Idempotency

 - Validate that generate/root calls are no longer idempotent, but the bundle importing
   does not generate new keys/issuers
 - As before make sure that the delete root api resets everything
 - Address a bug within the storage that we bombed when we had multiple different
   key types within storage.

* Assign Name=current to migrated key and issuer

 - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer.

* Build CRL upon PKI intermediary set-signed api called

 - Add a call to buildCRL if we created an issuer within pathImportIssuers
 - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call
 - Remove a code block writing out "ca" storage entry that is no longer used.

* Identify which certificate or key failed

When importing complex chains, we should identify in which certificate
or key the failure occurred.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* PKI migration writes out empty migration log entry

 - Since the elements of the struct were not exported we serialized an empty
   migration log to disk and would re-run the migration

* Add chain-building logic to PKI issuers path

With the one-entry-per-issuer approach, CA Chains become implicitly
constructed from the pool of issuers. This roughly matches the existing
expectations from /config/ca (wherein a chain could be provided) and
/intemediate/set-signed (where a chain may be provided). However, in
both of those cases, we simply accepted a chain. Here, we need to be
able to reconstruct the chain from parts on disk.

However, with potential rotation of roots, we need to be aware of
disparate chains. Simply concating together all issuers isn't
sufficient. Thus we need to be able to parse a certificate's Issuer and
Subject field and reconstruct valid (and potentially parallel)
parent<->child mappings.

This attempts to handle roots, intermediates, cross-signed
intermediates, cross-signed roots, and rotated keys (wherein one might
not have a valid signature due to changed key material with the same
subject).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Return CA Chain when fetching issuers

This returns the CA Chain attribute of an issuer, showing its computed
chain based on other issuers in the database, when fetching a specific
issuer.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add testing for chain building

Using the issuance infrastructure, we generate new certificates (either
roots or intermediates), positing that this is roughly equivalent to
importing an external bundle (minus error handling during partial
imports). This allows us to incrementally construct complex chains,
creating reissuance cliques and cross-signing cycles.

By using ECDSA certificates, we avoid high signature verification and
key generation times.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow manual construction of issuer chain

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix handling of duplicate names

With the new issuer field (manual_chain), we can no longer err when a
name already exists: we might be updating the existing issuer (with the
same name), but changing its manual_chain field. Detect this error and
correctly handle it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for manual chain building

We break the clique, instead building these chains manually, ensuring
that the remaining chains do not change and only the modified certs
change. We then reset them (back to implicit chain building) and ensure
we get the same results as earlier.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add stricter verification of issuers PEM format

This ensures each issuer is only a single certificate entry (as
validated by count and parsing) without any trailing data.

We further ensure that each certificate PEM has leading and trailing
spaces removed with only a single trailing new line remaining.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix full chain building

Don't set the legacy IssuingCA field on the certificate bundle, as we
prefer the CAChain field over it.

Additionally, building the full chain could result in duplicate
certificates when the CAChain included the leaf certificate itself. When
building the full chain, ensure we don't include the bundle's
certificate twice.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add stricter tests for full chain construction

We wish to ensure that each desired certificate in the chain is only
present once.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rename PKI types to avoid constant variable name collisions

 keyId -> keyID
 issuerId -> issuerID
 key -> keyEntry
 issuer -> issuerEntry
 keyConfig -> keyConfigEntry
 issuerConfig -> issuerConfigEntry

* Update CRL handling for multiple issuers

When building CRLs, we've gotta make sure certs issued by that issuer
land up on that issuer's CRL and not some other CRL. If no CRL is
found (matching a cert), we'll place it on the default CRL.
However, in the event of equivalent issuers (those with the same subject
AND the same key  material) -- perhaps due to reissuance -- we'll only
create a single (unified) CRL for them.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow fetching updated CRL locations

This updates fetchCertBySerial to support querying the default issuer's
CRL.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove legacy CRL storage location test case

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update to CRLv2 Format to copy RawIssuer

When using the older Certificate.CreateCRL(...) call, Go's x509 library
copies the parsed pkix.Name version of the CRL Issuer's Subject field.
For certain constructed CAs, this fails since pkix.Name is not suitable
for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL.

In updating to the newer x509.CreateRevocationList(...) call, we can
construct the CRL in the CRLv2 format and correctly copy the issuer's
name. However, this requires holding an additional field per-CRL, the
CRLNumber field, which is required in Go's implementation of CRLv2
(though OPTIONAL in the spec). We store this on the new
LocalCRLConfigEntry object, per-CRL.

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add comment regarding CRL non-assignment in GOTO

In previous versions of Vault, it was possible to sign an empty CRL
(when the CRL was disabled and a force-rebuild was requested). Add a
comment about this case.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow fetching the specified issuer's CRL

We add a new API endpoint to fetch the specified issuer's CRL directly
(rather than the default issuer's CRL at /crl and /certs/crl). We also
add a new test to validate the CRL in a multi-root scenario and ensure
it is signed with the correct keys.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add new PKI key prefix to seal wrapped storage (#15126)

* Refactor common backend initialization within backend_test

 - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage.

* Add ability to read legacy cert bundle if the migration has not occurred on secondaries.

 - Track the migration state forbidding an issuer/key writing api call if we have not migrated
 - For operations that just need to read the CA bundle, use the same tracking variable to
   switch between reading the legacy bundle or use the new key/issuer storage.
 - Add an invalidation function that will listen for updates to our log path to refresh the state
   on secondary clusters.

* Always write migration entry to trigger secondary clusters to wake up

 - Some PR feedback and handle a case in which the primary cluster does
   not have a CA bundle within storage but somehow a secondary does.

* Update CA Chain to report entire chain

This merges the ca_chain JSON field (of the /certs/ca_chain path) with
the regular certificate field, returning the root of trust always. This
also affects the non-JSON (raw) endpoints as well.

We return the default issuer's chain here, rather than all known issuers
(as that may not form a strict chain).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow explicit issuer override on roles

When a role is used to generate a certificate (such as with the sign/
and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer
that issuer to the one on the request. This allows operators to set an
issuer (other than default) for requests to be issued against,
effectively making the change no different from the users' perspective
as it is "just" a different role name.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for role-based issuer selection

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Expand NotAfter limit enforcement behavior

Vault previously strictly enforced NotAfter/ttl values on certificate
requests, erring if the requested TTL extended past the NotAfter date of
the issuer. In the event of issuing an intermediate, this behavior was
ignored, instead permitting the issuance.

Users generally do not think to check their issuer's NotAfter date when
requesting a certificate; thus this behavior was generally surprising.

Per RFC 5280 however, issuers need to maintain status information
throughout the life cycle of the issued cert. If this leaf cert were to
be issued for a longer duration than the parent issuer, the CA must
still maintain revocation information past its expiration.

Thus, we add an option to the issuer to change the desired behavior:

 - err, to err out,
 - permit, to permit the longer NotAfter date, or
 - truncate, to silently truncate the expiration to the issuer's
   NotAfter date.

Since expiration of certificates in the system's trust store are not
generally validated (when validating an arbitrary leaf, e.g., during TLS
validation), permit should generally only be used in that case. However,
browsers usually validate intermediate's validity periods, and thus
truncate should likely be used (as with permit, the leaf's chain will
not validate towards the end of the issuance period).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for expanded issuance behaviors

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning on keyless default issuer (#15178)

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update PKI to new Operations framework (#15180)

The backend Framework has updated Callbacks (used extensively in PKI) to
become deprecated; Operations takes their place and clarifies forwarding
of requests.

We switch to the new format everywhere, updating some bad assumptions
about forwarding along the way. Anywhere writes are handled (that should
be propagated to all nodes in all clusters), we choose to forward the
request all the way up to the performance primary cluster's primary
node. This holds for issuers/keys, roles, and configs (such as CRL
config, which is globally set for all clusters despite all clusters
having their own separate CRL).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Kitography/vault 5474 rebase (#15150)

* These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.)

* Import and Generate API calls w/ needed additions to SDK.

* make fmt

* Add Help/Sync Text, fix some of internal/exported/kms code.

* Fix PEM/DER Encoding issue.

* make fmt

* Standardize keyIdParam, keyNameParam, keyTypeParam

* Add error response if key to be deleted is in use.

* replaces all instances of "default" in code with defaultRef

* Updates from Callbacks to Operations Function with explicit forwarding.

* Fixes a panic with names not being updated everywhere.

* add a logged error in addition to warning on deleting default key.

* Normalize whitespace upon importing keys.

Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>

* Fix isKeyInUse functionality.

* Fixes tests associated with newline at end of key pem.

* Add alternative proposal PKI aliased paths (#15211)

* Add aliased path for root/rotate/:exported

This adds a user-friendly path name for generating a rotated root. We
automatically choose the name "next" for the newly generated root at
this path if it doesn't already exist.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add aliased path for intermediate/cross-sign

This allows cross-signatures to work.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add path for replacing the current root

This updates default to point to the value of the issuer with name
"next" rather than its current value.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove plural issuers/ in signing paths

These paths use a single issuer and thus shouldn't include the plural
issuers/ as a path prefix, instead using the singular issuer/ path
prefix.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Only warn if default issuer was imported

When the default issuer was not (re-)imported, we'd fail to find it,
causing an extraneous warning about missing keys, even though this
issuer indeed had a key.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing issuer sign/issue paths

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clean up various warnings within the PKI package (#15230)

* Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers

 - Hook into the backend invalidation function so that secondaries are notified of
   new/updated issuer or migrations occuring on the primary cluster. Upon notification
   schedule a CRL rebuild to take place upon the next process to read/update the CRL
   or within the periodic function if no request comes in.

* Schedule rebuilding PKI CRLs on active nodes only

 - Address an issue that we were scheduling the rebuilding of a CRL on standby
   nodes, which would not be able to write to storage.
 - Fix an issue with standby nodes not correctly determining that a migration previously
   occurred.

* Return legacy CRL storage path when no migration has occurred.

* Handle issuer, keys locking (#15227)

* Handle locking of issuers during writes

We need a write lock around writes to ensure serialization of
modifications. We use a single lock for both issuer and key
updates, in part because certain operations (like deletion) will
potentially affect both.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing b.useLegacyBundleCaStorage guards

Several locations needed to guard against early usage of the new issuers
endpoint pre-migration.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address PKI to properly support managed keys (#15256)

* Address codebase for managed key fixes
* Add proper public key comparison for better managed key support to importKeys
* Remove redundant public key fetching within PKI importKeys

* Correctly handle rebuilding remaining chains

When deleting a specific issuer, we might impact the chains. From a
consistency perspective, we need to ensure the remaining chains are
correct and don't refer to the since-deleted issuer, so trigger a full
rebuild here.

We don't need to call this in the delete-the-world (DELETE /root) code
path, as there shouldn't be any remaining issuers or chains to build.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove legacy CRL bundle on world deletion

When calling DELETE /root, we should remove the legacy CRL bundle, since
we're deleting the legacy CA issuer bundle as well.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove deleted issuers' CRL entries

Since CRLs are no longer resolvable after deletion (due to missing
issuer ID, which will cause resolution to fail regardless of if an ID or
a name/default reference was used), we should delete these CRLs from
storage to avoid leaking them.

In the event that this issuer comes back (with key material), we can
simply rebuild the CRL at that time (from the remaining revoked storage
entries).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add unauthed JSON fetching of CRLs, Issuers (#15253)

Default to fetching JSON CRL for consistency

This makes the bare issuer-specific CRL fetching endpoint return the
JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Add JSON-specific endpoint for fetching issuers

Unlike the unqualified /issuer/:ref endpoint (which also returns JSON),
we have a separate /issuer/:ref/json endpoint to return _only_ the
PEM-encoded certificate and the chain, mirroring the existing /cert/ca
endpoint but for a specific issuer. This allows us to make the endpoint
unauthenticated, whereas the bare endpoint would remain authenticated
and usually privileged.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Add tests for raw JSON endpoints

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add unauthenticated issuers endpoints to PKI table

This adds the unauthenticated issuers endpoints?

 - LIST /issuers,
 - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and
 - Fetching the CRL of this issuer (in JSON/DER/PEM form).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add issuer usage restrictions bitset

This allows issuers to have usage restrictions, limiting whether they
can be used to issue certificates or if they can generate CRLs. This
allows certain issuers to not generate a CRL (if the global config is
with the CRL enabled) or allows the issuer to not issue new certificates
(but potentially letting the CRL generation continue).

Setting both fields to false effectively forms a soft delete capability.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* PKI Pod rotation Add Base Changelog (#15283)

* PKI Pod rotation changelog.
* Use feature release-note formatting of changelog.

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Co-authored-by: Kit Haines <kit.haines@hashicorp.com>
Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 12:42:28 -04:00
Matt Schultz 5aabe4a5f8
Add Transit BYOK wrapping key endpoint (#15271)
* add wrapping key endpoint

* change how wrapping key is stored

* move wrapping key func to backend

* refactor wrapping key generation

* Initial unit tests for Transit wrapping key endpoint

* Wire up wrapping key unit tests to actual implementation.

* Clean up Transit BYOK wrapping key tests and imports.

* Fix Transit wrapping key endpoint formatting.

* Update transit wrapping key to use lock manager for safe concurrent use.

* Rename some Transit wrapping key variables. Ensure the Transit wrapping key is correctly typed and formatted in a unit test.

* Fix spacing issue in Transit wrapping key endpoint help string.

Co-authored-by: rculpepper <rculpepper@hashicorp.com>
2022-05-11 11:28:32 -05:00
claire bontempo 9d608a2e38
remove legend (#15359) 2022-05-11 09:06:39 -07:00
Chris Capurso ad4523ea34
add license termination time to API and CLI docs (#15349)
* add license termination time to API and CLI docs

* ensure consistent time format
2022-05-11 10:05:38 -04:00
claire bontempo 10a19b7c54
UI/ fix firefox not recognizing csv export (#15364)
* add extension to filenmae

* add changelog
2022-05-10 17:19:38 -07:00