Commit Graph

2520 Commits

Author SHA1 Message Date
Alexander Scheel 000df9559a
Start container-based PKI ACME tests (#20320)
* Start ACME containerized test suite

This starts a containerized ACME test suite using containers, running
both Vault and Certbot (in standalone mode) in the container to ensure
we successfully issue certificates.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to using hashicorp mirror

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-25 19:11:35 +00:00
Alexander Scheel 918305d450
Fix issuer patch of leaf_not_after_behavior (#20341)
* Fix patching issuer leaf_not_after_behavior

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for patching leaf_not_after_behavior

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-25 11:02:13 -04:00
Steven Clark 47605c0d48
Add support to load roles and issuers within ACME wrapper (#20333)
* Add support to load roles and issuers within ACME wrapper

* Add missing go doc to new test

* PR feedback

 - Move field definitions into fields.go
 - Update wording and associated errors to some role failures.
 - Add missing ':' to error messages
2023-04-25 13:29:07 +00:00
Nick Cabatoff 22b00eba12
Add support for docker testclusters (#20247) 2023-04-24 14:25:50 -04:00
Alexander Scheel d7f67b8856
Add additional fields to LIST issuers for Web UI (#20276)
* Add additional fields to LIST issuers for Web UI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-21 18:16:16 +00:00
Alexander Scheel dfb29f3b96
Add ACME DNS validator (#20293)
* Add DNS challenge validation to ACME

This allows us to validate against wildcard domain names, as the HTTP-01
challenge does not support wildcard validation.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Properly delay challenge retry attempts

Previously we'd essentially semi-busy wait for a challenge to become
retry-able, as the queue itself had no knowledge of the retry after
value of the validation attempt. Now, we plumb through this value into
the queue itself, to aide selection of validations to attempt.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-21 13:08:27 -04:00
Anton Averchenkov 3c76d90156
Small fixes for OpenAPI display attributes (#20285) 2023-04-21 17:04:12 +00:00
Alexander Scheel 3c8c46e172
Better ACME wildcard validation (#20289)
* Refactor wildcard validation checks

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add helper to determine if identifier is wildcard

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Better validate wildcard acceptance

This correctly validates wildcards to contain only a leading prefix
(*.) and must include another label, also ensuring that the remaining
domain components pass non-IP and IDNA validation, and removing them
from display on the authorization. Finally, we restrict the challenge
types available for various combinations of IP, DNS, and wildcard
addresses.

This then updates the tests to validate this as well.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-21 12:54:19 -04:00
Steven Clark 5a7173a154
Implement ACME CSR signing and certificate retrieval (#20266)
* WIP: Implement ACME CSR signing and certificate retrieval

* Add some validations within the ACME finalize API

 - Validate that the CSR we were given matches the DNS names
   and IP addresses within the order
 - Validate that the CSR does not share the same public as the
   account

* Gate ACME finalize order validating all authorizations are in valid state
2023-04-21 09:38:06 -04:00
Luis (LT) Carbonell d308c31cbf
Add Configurable LDAP Max Page Size (#19032)
* Add config flag for LDAP max page size

* Add changelog

* move changelog to correct file

* cleanup

* Default to non-paged searching for with -1

* Update website/content/api-docs/auth/ldap.mdx

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update tests

---------

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2023-04-20 20:39:27 +00:00
Alexander Scheel f0279b0d31
Move OCSP/PKI interop test to PKI engine (#20273)
This prevents SDK from having a circular import on the main Vault
package.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-20 14:57:20 +00:00
Alexander Scheel 189a776307
Add warnings to crl rebuilds, allowing notifying operator of empty issuer equivalency sets (#20253)
* Add infrastructure for warnings on CRL rebuilds

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning on issuer missing KU for CRL Signing

When an entire issuer equivalency class is missing CRL signing usage
(but otherwise has key material present), we should add a warning so
operators can either correct this issuer or create an equivalent version
with KU specified.

Resolves: https://github.com/hashicorp/vault/issues/20137

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for issuer warnings

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix return order of CRL builders

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-19 16:55:37 +00:00
Alexander Scheel 005d989230
Add acme challenge validation engine (#20221)
* Allow creating storageContext with timeout

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add challenge validation engine to ACME

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Initialize the ACME challenge validation engine

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Trigger challenge validation on endpoint submission

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix GetKeyThumbprint to use raw base64

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Point at localhost for testing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add cleanup of validation engine

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-19 12:31:19 -04:00
claire bontempo 5f64520dac
UI: remove references to comma separation for string array edit types (#20163)
* remove intercepting helpText

* add subtext directly to StringList input component

* update tests and add coverage for new openapi-attrs util

* update test

* add warning validation to input

* lol is this right i dont know go

* literally no idea what im doing

* add Description to display attrs struct

* update struct comment

* add descriptions to remaining go fields

* add missing comma

* remaining commas..."

* add description to display attrs

* update tests

* update tests

* add changelog;

* Update ui/app/utils/openapi-to-attrs.js

* update tests following backend changes

* clearly name variable

* format files

* no longer need to test for modified tooltip since coming from backend now
2023-04-19 16:16:30 +00:00
Alexander Scheel 45acac0e64
Return OCSP errors on cert auth login failures (#20234)
* Return OCSP errors on cert auth login failures

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to immediately returning the first match

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-19 08:54:45 -04:00
Max Bowsher 91abc177bb
Minor follow-ups to #16865 (#20220)
* Minor follow-ups to #16865

Fix PKI issuer upgrade logic when upgrading to 1.12 or later, to
actually turn off the issuer crl-signing usage when it intended to.

Fix minor typo in docs.

* changelog
2023-04-18 07:39:05 -04:00
Alexander Scheel 13dd4c0a99
Add ACME HTTP-01 Challenge (#20141)
* Add HTTP challenge validator

This will attempt to safely validate HTTP challenges, following a
limited number of redirects and timing out after too much time has
passed.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for ValidateKeyAuthorization

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test cases for ValidateHTTP01Challenge

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add token to HTTP challenge

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-17 15:23:04 -04:00
Steven Clark 7361ce1e57
Add tests for fetching ACME authorizations and challenges (#20205)
- Add tests to validate that we can load authorizations and
   challenges from the server
2023-04-17 17:52:54 +00:00
Alexander Scheel 4190212bbb
Remove extraneous certificate from OCSP response (#20201)
* Remove extraneous certificate from OCSP response

Since the issuer used to sign the certificate also signs the OCSP
response, no additional information is added by sending the issuer again
in the certs field of the BasicOCSPResponse structure. Removing it saves
bytes and avoids confusing Go-based OCSP verifiers which cannot handle
the cert issuer being duplicated in the certs field.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-17 16:40:26 +00:00
Steven Clark 6211595bef
Add a helper function to build ACME API patterns (#20180)
- Add a helper function that can accept the final API path along with
   the pattern function for an ACME api definition and generate the
   various flavors for the given API
2023-04-14 18:48:33 +00:00
Steven Clark 138f36eafe
Move all ACME wrappers into a dedicated go file (#20174)
* Move all ACME wrappers into a dedicated go file

 - Make it easier to figure out where the various wrappers for
   ACME exist by locating them inside a dedicated go file instead
   of spread out across the various path_acme_xxx files.

* Add missing copyright headers to PKI files
2023-04-14 14:12:31 -04:00
Steven Clark d324aa0d15
Implement ACME order API (#20127)
* Implement ACME new-order API
 - This is a very rough draft for the new order ACME API

* Add ACME order list API

* Implement ACME Get order API

* Misc order related fixes

 - Filter authorizations in GetOrders for valid
 - Validate notBefore and notAfter dates make sense
 - Add <order>/cert URL path to order response if set to valid

* Return account status within err authorized, if the account key verified
2023-04-14 14:54:48 +00:00
Kyle Schochenmaier 1b4ff1b1b4
Revert changes to STS leases but keep the ttl field (#20034)
* revert STS lease changes, now create a lease for STS credentials but keep the ttl
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2023-04-13 15:02:39 -05:00
Alexander Scheel 10e02aca02
Add missing cert auth ocsp read data (#20154)
* Add missing OCSP cert auth fields

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test to ensure OCSP values are persisted

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-13 18:59:09 +00:00
Jason O'Donnell ec9e08c931
sdk/ldaputil: add connection_timeout configurable (#20144)
* sdk/ldaputil: add connection_timeout configurable

* changelog

* Update doc

* Fix test

* Change default to 30s
2023-04-13 12:43:28 -04:00
Anton Averchenkov c025747e46
openapi: Add display attributes for Consul (#19413) 2023-04-13 11:31:37 -04:00
Anton Averchenkov c8803cb571
openapi: Add display attributes for userpass (#19411) 2023-04-13 11:30:54 -04:00
Anton Averchenkov 9a654ac3f1
openapi: Add display attributes for Okta auth (#19391) 2023-04-13 11:29:59 -04:00
Tom Proctor 9aa9686c81
Simplify tracking of external plugins (#20009) 2023-04-12 18:34:35 +01:00
Alexander Scheel b4edc81cd5
Add ACME authorizations & challenges (#20113)
* Distinguish POST-as-GET from POST-with-empty-body

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME authorization, identifier, and challenge types

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to load and save authorizations

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME authorizations path handling

This supports two methods: a fetch handler over the authorization, to
expose the underlying challenges, and a deactivate handler to revoke
the authorization and mark its challenges invalid.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME challenge path handling

These paths kick off processing and validation of the challenge by the
ACME client.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-12 16:55:25 +00:00
Steven Clark 8458154fc3
Use a UUID for ACME kid instead of a key fingerprint (#20110)
* Use a UUID for ACME kid instead of a key fingerprint

* PR feedback

 - Calculate thumbprint within CreateAccount instead of passing it in
 - Reorder writes within CreateAccount to now write out thumbprint entry
   first as we can easily recover/overwrite it if we fail mid-way
 - Change back LoadAccount in acme to return an error if it fails to
   lookup the entry

* Clearify comment within ACME CreateAccount
2023-04-12 15:29:54 +00:00
Steven Clark 4e6b88d58c
Rework ACME workflow test to leverage Golang's ACME client library (#19949)
* Rework ACME workflow test to leverage Golang's ACME client library

 - Instead of testing manually, leverage the Golang ACME library
   to test against our implementation from the unit tests.

* Add tests for new-account and misc fixes

 - Set and return the account status for registration
 - Add handlers for the account/ api/updates
 - Switch acme/ to cluster local storage
 - Disable terms of service checks for now as we don't set the url

* PR feedback

 - Implement account deactivation
 - Create separate account update handler, to not mix account creation
   logic
 - Add kid field to account update definition
 - Add support to update contact details on an existing account
2023-04-12 09:05:42 -04:00
Alexander Scheel 73a05ebbe5
Fix building unified delta WAL, unified delta CRLs (#20058)
* Correctly find certificates for unified delta CRL

When building the unified delta CRL, WAL entries from the non-primary
cluster were ignored. This resulted in an incomplete delta CRL,
preventing some entries from appearing.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly rebuild unified delta CRLs

When deciding if the Unified Delta CRL should be rebuilt, we need to
check the status of all clusters and their last revoked serial numbers.
If any new serial has been revoked on any cluster, we should rebuild the
unified delta CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly persist Unified Delta CRL build entries

When building the unified CRL, we need to read the last seen serial
number from all clusters, not just the present cluster, and write it
to the last built serial for that cluster's unified delta WAL entry.
This prevents us from continuously rebuilding unified CRLs now that we
have fixed our rebuild heuristic.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix getLastWALSerial for unified delta CRLs

getLastWALSerial ignored its path argument, preventing it from reading
the specified cluster-specific WAL entry. On the primary cluster, this
was mostly equivalent, but now that we're correctly reading WAL entries
and revocations for other clusters, we need to handle reading these
entries correctly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Copy delta WAL entries in event of failure

Any local delta WAL should be persisted to unified delta WAL space as
well. If such unified persistence fails, we need to ensure that they get
eventually moved up, otherwise they'll remain missing until the next
full CRL rebuild occurs, which might be significantly longer than when
the next delta CRL rebuild would otherwise occur. runUnifiedTransfer
already handles this for us, but it lacked logic for delta WAL serials.

The only interesting catch here is that we refuse to copy any entries
whose full unified revocation entry has not also been written.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make doUnifiedTransferMissingLocalSerials log an error

This message is mostly an error and would always be helpful information
to have when troubleshooting failures.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Warn on cross-cluster write failures during revoke

When revoking certificates, we log cross-cluster revocation failures,
but we should really expose this information to the caller, that their
local revocation was successful, but their cross-cluster revocation
failed.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure unified delta WAL entry has full entry

Delta WAL entries are empty files whose only information (a revoked
serial number) is contained in the file path. These depend implicitly on
a full revocation entry existing for this file (whether a cross-cluster
unified entry or a local entry).

We should not write unified delta WAL entries without the corresponding
full unified revocation entry existing. Add a warning in this case.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-11 18:02:58 +00:00
Alexander Scheel 5389477107
Make config load consistent with config write (#20059)
When reading the config, we attempt to detect if the running Vault
instance has been changed from its Enterprise status on write.
Similarly, we should detect if the mount is a local mount instead. While
this isn't changeable at runtime, using sys/raw to side-load an invalid
config could be possible.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-11 11:05:43 -04:00
Alexander Scheel 509f862494
Log, don't err, on unified delta WAL write failure (#20057)
* Log, don't err, on unified delta WAL write failure

When the PBPWF fails on the Active node of a PR Secondary cluster with a
read-only failure, there is no value in forwarding this request up to
the Active node of the PR Primary cluster: it does not have the local
revocation context necessary to write a Delta WAL entry for this
request, and would likely end up writing a cross-cluster revocation
entry (if it is enabled) or else erring completely.

Instead, log this error like we do when failing to write unified CRL
entries. Switch both to using Error instead of Debug for this type of
failure.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-11 09:36:10 -04:00
Anton Averchenkov 194ce13a95
openapi: Add display attributes for database plugin (#19414) 2023-04-10 14:22:02 -04:00
Anton Averchenkov 9d3ab193a5
openapi: Add display attributes for transit plugin (#19575) 2023-04-10 14:20:53 -04:00
Anton Averchenkov f257d68576
openapi: Add display attributes for RabbitMQ plugin (#19540) 2023-04-10 14:19:32 -04:00
Anton Averchenkov c61052ef29
openapi: Add display attributes for SSH plugin (#19543) 2023-04-10 14:18:00 -04:00
Anton Averchenkov 592d3464f6
openapi: Add display attributes for TOTP plugin (#19544) 2023-04-10 13:32:15 -04:00
Anton Averchenkov 33bc8f5d00
openapi: Add display attributes for GitHub auth (#19384)
Please see https://github.com/hashicorp/vault/pull/19319 for more details on how this will affect the generated OpenAPI schema.

____

### The following OperationID's will be generated for GitHub auth:

github-read-configuration
github-configure
github-log-in
github-read-teams
github-read-team-mapping
github-write-team-mapping
github-delete-team-mapping
github-read-users
github-read-user-mapping
github-write-user-mapping
github-delete-user-mapping
2023-04-07 13:30:26 -04:00
Anton Averchenkov 7ed4a429f7
openapi: Add display attributes for Nomad plugin (#19415)
Please see #19319 for more details on how this will affect the generated OpenAPI schema.
___

The following OperationID's will be generated for Nomad plugin:

nomad-read-access-configuration
nomad-configure-access
nomad-delete-access-configuration
nomad-read-lease-configuration
nomad-configure-lease
nomad-delete-lease-configuration
nomad-generate-credentials
nomad-list-roles
nomad-read-role
nomad-write-role
nomad-delete-role
2023-04-07 13:21:21 -04:00
Anton Averchenkov c915dea526
openapi: Add display attributes for LDAP auth (#19389) 2023-04-07 13:16:17 -04:00
Anton Averchenkov 2ec2d050c0
openapi: Add display attributes for Radius auth (#19392) 2023-04-07 13:14:44 -04:00
Anton Averchenkov 44d4f39e1b
openapi: Add display attributes for cert auth (#19383) 2023-04-07 13:12:56 -04:00
Anton Averchenkov 543efc83e4
openapi: Add display attributes for PKI plugin (#19422) 2023-04-06 11:10:01 -04:00
Anton Averchenkov 7cf86c6dae
openapi: Add display attributes for AWS (#19366) 2023-04-06 11:08:30 -04:00
Anton Averchenkov 5370d07b6c
openapi: Add display attributes for approle (#19363) 2023-04-06 11:03:11 -04:00
Alexander Scheel 3ed31ff262
Add acme account storage (#19953)
* Enable creation of accounts

 - Refactors many methods to take an acmeContext, which holds the
   storageContext on it.
 - Updates the core ACME Handlers to use *acmeContext, to avoid
   copying structs.
 - Makes JWK exported so the JSON parser can find it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Finish ACME account creation

 - This ensures a Kid is created when one doesn't exist
 - Expands the parsed handler capabilities, to format the response and
   set required headers.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-03 16:08:25 -04:00
Steven Clark 8ddead7a17
Rework the ACME test suite to use full Vault cluster to validate behavior (#19874)
- Instead of using tests that just test the plugin storage/interface
   layer, use a full Vault instance to validate that we can send/receive
   the proper headers and responses back to a client.
 - Found an issue with HEAD new-nounce api calls returning 500 errors.
 - Add the /acme/ suffix to the baseUrl in the acme context so we don't
   have to keep adding it a bit everywhere.
2023-04-03 09:38:20 -04:00
Alexander Scheel b4c3aca7a1
Merge ACME package back into the PKI package (#19826)
* Squash pki/acme package down to pki folder

Without refactoring most of PKI to export the storage layer, which we
were initially hesitant about, it would be nearly impossible to have the
ACME layer handle its own storage while being in the acme/ subpackage
under the pki package.

Thus, merge the two packages together again.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Properly format errors for missing parameters

When missing required ACME request parameters, don't return Vault-level
errors, but drop into the PKI package to return properly-formatted ACME
error messages.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Error type clarifications

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix GetOk with type conversion calls

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 21:08:31 +00:00
Alexander Scheel 73c468787b
Add ACME new account creation handlers (#19820)
* Identify whether JWKs existed or were created, set KIDs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Reclassify ErrAccountDoesNotExist as 400 per spec

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional stub methods for ACME accounts

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Start adding ACME newAccount handlers

This handler supports two pieces of functionality:

 1. Searching for whether an existing account already exists.
 2. Creating a new account.

One side effect of our JWS parsing logic is that we needed a way to
differentiate between whether a JWK existed on disk from an account or
if it was specified in the request. This technically means we're
potentially responding to certain requests with positive results (e.g.,
key search based on kid) versus erring earlier like other
implementations do.

No account storage has been done as part of this commit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Unify path fields handling, fix newAccount method

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 19:06:09 +00:00
Steven Clark 91d1628bb5
Initial ACME new-nonce API (#19822)
* Initial ACME new-nonce API implementation

* Return proper HTTP status codes for ACME new-nonce API handler
2023-03-29 18:22:48 +00:00
Steven Clark bc57865998
PKI: Initial ACME directory API support (#19803)
* PKI: Initial ACME directory API support along with basic tests for error handler and the directory itself across various paths.
2023-03-29 16:29:19 +00:00
Alexander Scheel 27f670abd5
Tighten JWS algorithm lists (#19797)
* Add new PKI ACME subpackage to test_packages list

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Restrict JWS keys to specified algorithms

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-28 20:49:07 +00:00
Alexander Scheel cb8be1d8be
Add initial ACME server library to PKI (#19778)
* Add ACME package to provide a nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add JWS parsing helper

Using go-jose v2, we start building a JWS parsing helper, ensuring that
fields are properly validated w.r.t. the ACME spec's intersection with
JWS.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add error context information

Start adding the ability to wrap errors returned by Vault to
ACME-specific errors.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make ACMEState exported

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-28 17:29:54 +00:00
Anton Averchenkov 833021759f
Fix response structure validation tests for non-2xx responses (#19726) 2023-03-23 16:33:44 -04:00
Mason Foster 09c6ff0623
aws: pass cancelable context with aws calls (#19365)
* auth/aws: use cancelable context with aws calls

* secrets/aws: use cancelable context with aws calls
2023-03-23 12:02:24 -05:00
Alexander Scheel 8c6b266b7b
Add test to test SSH endpoint authentication (#19705)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-23 16:00:21 +00:00
Alexander Scheel 3611eaa4af
Add tests for PKI endpoint authentication (#19704)
* Split (un,)authenticated issuer fetch endpoints

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow unauthed per-issuer unified-crl access

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests to validate endpoint authentication status

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Disable schema validation in TestBackend_IfModifiedSinceHeaders

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-23 15:54:26 +00:00
Steven Clark ae4b02e6c4
Fix for PKI.TestStandby_Operations test to work in ENT (#19647)
* Fix for PKI.TestStandby_Operations test to work in ENT

 - Remove wait call to testhelpers.WaitForActiveNodeAndStandbys and
   leverage testhelpers.WaitForStandbyNode instead.

* Use InmemBackendSetup for a proper HA backend in ENT
2023-03-20 21:11:56 +00:00
Alexander Scheel 411705d723
More wrapped errors - PKI, SSH, Transit (#19631)
* Add missing wrapped errors in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing wrapped errors in SSH

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing wrapped errors in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-20 16:00:49 +00:00
Steven Clark 2217731266
Forward PKI revocation requests received by standby nodes to active node (#19624)
* Forward PKI revocation requests received by standby nodes to active node

 - A refactoring that occurred in 1.13 timeframe removed what was
   considered a specific check for standby nodes that wasn't required
   as a writes should be returning ErrReadOnly.
 - That sadly exposed a long standing bug where the errors from the
   storage layer were not being properly wrapped, hiding the ErrReadOnly
   coming from a write and failing the request.

* Add cl

* Add test for basic PKI operations against standby nodes
2023-03-20 14:58:36 +00:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
AnPucel 65e5730c6c
PKI Responses Part 4 (#18612) 2023-03-14 15:00:37 -07:00
Anton Averchenkov e19dc98016
Ignore special HTTP fields in response validation tests (#19530) 2023-03-14 13:21:20 -04:00
Alexander Scheel ab3d6d61e0
Add support for importing RSA-PSS keys into Transit (#19519)
* Add support for importing RSA-PSS keys in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-13 17:03:01 +00:00
Jakob Beckmann 078a245939
Allow alias dereferencing in LDAP searches (#18230)
* impr(auth/ldap): allow to dereference aliases in searches

* docs: add documentation for LDAP alias dereferencing

* chore(auth/ldap): add changelog entry for PR 18230

* chore: run formatter

* fix: update default LDAP configuration with new default

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

* docs(ldap): add alias dereferencing to API docs for LDAP

---------

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
2023-02-24 13:49:17 -05:00
davidadeleon dd39b177f9
add nil check for secret id entry on delete via accessor (#19186)
* add nil check for secret id entry on delete via accessor

* add changelog

* add godoc to test

* improve feedback on nil entry

* fix error reporting on invalid secret id accessor

* fix test to expect implemented error
2023-02-24 13:18:08 -05:00
Jakob Beckmann 0bed33d84f
feat(auth/ldap): allow passing the LDAP password via an env var (#18225)
* feat(auth/ldap): allow passing the LDAP password via an environment variable when authenticating via the CLI

* chore(auth/ldap): add changelog entry for PR 18225
2023-02-23 11:16:17 -05:00
Raymond Ho 57ff9835f7
use github token env var if present when fetching org id (#19244) 2023-02-21 12:17:35 -08:00
AnPucel 4c0895188a
Adding PKI Responses 3 (#18596) 2023-02-16 17:31:45 -08:00
John-Michael Faircloth 678556f3df
plugin/secrets/auth: enable multiplexing (#19215)
* plugin/auth: enable multiplexing

- the plugin will be multiplexed when run as an external plugin
  by vault versions that support secrets/auth plugin multiplexing (> 1.12)
- we continue to set the TLSProviderFunc to maintain backwards
  compatibility with vault versions that don't support AutoMTLS (< 1.12)

* enable multiplexing for secrets engines

* add changelog

* revert call to ServeMultiplex for pki and transit

* Revert "revert call to ServeMultiplex for pki and transit"

This reverts commit 755be28d14b4c4c4d884d3cf4d2ec003dda579b9.
2023-02-16 22:25:15 +00:00
AnPucel 2d58144620
PKI Response Structures Part 2 (#18479)
Response structures from intermediate --> manage_keys
2023-02-15 15:09:57 -08:00
AnPucel e05c8931b9
Adding Response Structures to PKI Config (#18376) 2023-02-15 14:51:27 -08:00
Raymond Ho c6a455c92f
fix TestBackendAcc_LoginWithCallerIdentity (#19201) 2023-02-15 12:40:49 -08:00
Anton Averchenkov 14316ea44e
Refactor approle response validation tests (#19188) 2023-02-15 12:29:15 -05:00
Steven Clark 225dd869d4
Attempt at fixing the memory usage in CI (#19171)
- Do not keep many intervals of the in memory sink metrics
   collector. Otherwise we fill up the CI's memory
2023-02-14 08:40:25 -05:00
Ellie 08ef61cc00
add error message when trying to rotate mssql root without password in configuration (#19103)
* add error message when trying to rotate mssql root without password in configuration

* add changelog
2023-02-13 07:31:13 -05:00
Kit Haines 14adb3b825
Telemetry Metrics Configuration. (#18186)
* Telemetry Metrics Configuration.

* Err Shadowing Fix (woah, semgrep is cool).

* Fix TestBackend_RevokePlusTidy_Intermediate

* Add Changelog.

* Fix memory leak.  Code cleanup as suggested by Steve.

* Turn off metrics by default, breaking-change.

* Show on tidy-status before start-up.

* Fix tests

* make fmt

* Add emit metrics to periodicFunc

* Test not delivering unavailable metrics + fix.

* Better error message.

* Fixing the false-error bug.

* make fmt.

* Try to fix race issue, remove confusing comments.

* Switch metric counter variables to an atomic.Uint32

 - Switch the metric counter variables to an atomic variable type
   so that we are forced to properly load/store values to it

* Fix race-issue better by trying until the metric is sunk.

* make fmt.

* empty commit to retrigger non-race tests that all pass locally

---------

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2023-02-10 21:31:56 +00:00
Michael Dempsey 1582b743aa
Add default to allowed values for algorithm_signer (#17894)
* Add default to allowed values for algorithm_signer

* Add possible values for algorithm signer in ui
2023-02-09 13:03:53 -05:00
Steven Clark 720ab09feb
Add a comment around why we are grabbing a lock to update an atomic boolean (#19087) 2023-02-09 09:12:37 -05:00
Steven Clark a4aee2fc95
Revert URL encoding OCSP GET fix #18938 (#19037)
- This fix was incorrect as now the tests and program are double
   URL encoding the OCSP GET requests, so the base64 + characters
   when using Vault proper are becoming space characters.
2023-02-08 11:24:26 -05:00
miagilepner e873932bce
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
Nick Cabatoff 53afd2627b
Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Steven Clark 9e9d5d5645
Use the unified CRL on local CRL paths if UnifiedCRLOnExistingPaths is set (#18989)
* Use the unified CRL on legacy CRL paths if UnifiedCRLOnExistingPaths is set

 - If the crl configuration option unified_crl_on_existing_paths is set
   to true along with the unified_crl feature, provide the unified crl
   on the existing CRL paths.
 - Added some test helpers to help debugging, they are being used by
   the ENT test that validates this feature.

* Rename method to shouldLocalPathsUseUnified
2023-02-03 14:38:36 -05:00
Alexander Scheel b69055175a
Use UTC for leaf exceeding CA's notAfter (#18984)
* Use UTC for leaf exceeding CA's notAfter

When generating a leaf which exceeds the CA's validity period, Vault's
error message was confusing as the leaf would use the server's time
zone, but the CA's notAfter date would use UTC. This could cause
user confusion as the leaf's expiry might look before the latter, due
to using different time zones. E.g.:

> cannot satisfy request, as TTL would result in notAfter
> 2023-03-06T16:41:09.757694-08:00 that is beyond the expiration of
> the CA certificate at 2023-03-07T00:29:52Z

Consistently use UTC for this instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-03 17:00:42 +00:00
Steven Clark 449a0a68f5
Fix race accessing b.crls within cert auth (#18945)
* Fix race accessing b.crls within cert auth

 - Discovered by CircleCI the pathLogin, pathLoginRenew paths access
   and reloads the b.crls member variable without a lock.
 - Also discovered that pathLoginResolveRole never populated an empty
   b.crls before usage within b.verifyCredentials

* Add cl

* Misc cleanup

 - Introduce a login path wrapper instead of repeating in all the
   various login methods the crl reloading
 - Cleanup updatedConfig, never returned an error and nothing looked at
   the error returned
 - Make the test within TestCRLFetch a little less timing sensitive as
   I was able to trigger a failure due to my machine taking more than
   150ms to load the new CRL
2023-02-01 16:23:06 -05:00
Steven Clark baf66ff56e
Apply URL encoding/unencoding to OCSP Get requests (#18938)
* Apply URL encoding/unencoding to OCSP Get requests

 - Missed this during development and sadly the unit tests were written
   at a level that did not expose this issue originally, there are
   certain combinations of issuer cert + serial that lead to base64
   data containing a '/' which will lead to the OCSP handler not getting
   the full parameter.
 - Do as the spec says, this should be treated as url-encoded data.

* Add cl

* Add higher level PKI OCSP GET/POST tests

* Rename PKI ocsp files to path_ocsp to follow naming conventions

* make fmt
2023-02-01 11:03:43 -05:00
Alexander Scheel 5d17f9b142
Allow cleanup ssh dynamic keys host keys (#18939)
* Add ability to clean up host keys for dynamic keys

This adds a new endpoint, tidy/dynamic-keys that removes any stale host
keys still present on the mount. This does not clean up any pending
dynamic key leases and will not remove these keys from systems with
authorized hosts entries created by Vault.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-01 15:09:16 +00:00
Steven Clark 81689ae63d
Refactor an ENT test helper back to OSS to wait for updated CRLs within PKI tests (#18933)
- We had two separate helpers that were very close, one only in ENT, so unify them here in OSS.
2023-02-01 08:47:26 -05:00
Steven Clark 8d47ad792f
PKI Unified CRL/OCSP apis should be ent only (#18913)
* PKI Unified CRL/OCSP apis should be ent only

 - Do not enable any of the unified crl/ocsp related apis on OSS.

* Rollback refactoring of pathFetchCRLViaCertPath

 - As pointed out in the PR, this method isn't actually being used at
   the moment with the <serial> handler, pathFetchValid, matching
   everything under the cert/XXXX path.

* Fix schema for ent/oss diff

 - Define the OSS vs ENT urls we want to see within the schema
   definition even if they aren't really going to be used in the end.
2023-02-01 08:35:14 -05:00
Nick Cabatoff c2b222a11a
Vault test cluster helper refactorings, mostly audit related (#18928)
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package.  Consolidate on a single "noop audit" implementation.
2023-02-01 08:33:16 -05:00
Alexander Scheel 881ae5a303
Remove dynamic keys from SSH Secrets Engine (#18874)
* Remove dynamic keys from SSH Secrets Engine

This removes the functionality of Vault creating keys and adding them to
the authorized keys file on hosts.

This functionality has been deprecated since Vault version 0.7.2.

The preferred alternative is to use the SSH CA method, which also allows
key generation but places limits on TTL and doesn't require Vault reach
out to provision each key on the specified host, making it much more
secure.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove dynamic ssh references from documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove dynamic key secret type entirely

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify changelog language

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add removal notice to the website

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-31 16:02:22 -05:00
Nick Cabatoff db41c5319d
TestClusterCore's TLSConfig becomes a method and does a Clone. (#18914) 2023-01-31 11:05:16 -05:00
Steven Clark b737777f15
Address pki::TestAutoRebuild flakiness (#18903)
* Address pki::TestAutoRebuild flakiness

 - Wait for a CRL change before progressing to the next step after
   we change configuration. Prior to this we would be racing against
   the CRL reloading from the configuration change.
2023-01-30 21:38:38 +00:00
akshya96 16ce923ddb
Brute forcing unlock user bug (#18890)
* brute forcing unlock user bug

* add changelog

* fix changelog
2023-01-30 13:06:10 -08:00
Matt Schultz 20c1645329
Add some managed key build tags to support incoming enterprise feature work. (#18902) 2023-01-30 14:57:01 -06:00
Alexander Scheel 2b9a8c6c49
Fix race in tidy status with cert counting (#18899)
* Read total cert counts with atomic.LoadUint32(...)

When generating the tidy status, we read the values of two backend
atomics, b.certCount and b.revokedCertCount, without using the atomic
load operation. This resulted in a data race when the status was read
at the same time as an on-going tidy operation:

    WARNING: DATA RACE
    Write at 0x00c00c77680c by goroutine 90522:
      sync/atomic.AddInt32()
          /usr/local/go/src/runtime/race_amd64.s:281 +0xb
      sync/atomic.AddUint32()
          <autogenerated>:1 +0x1a
      github.com/hashicorp/vault/builtin/logical/pki.(*backend).tidyStatusIncRevokedCertCount()
          /home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:1236 +0x107
      github.com/hashicorp/vault/builtin/logical/pki.(*backend).doTidyRevocationStore()
          /home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:525 +0x1404
      github.com/hashicorp/vault/builtin/logical/pki.(*backend).startTidyOperation.func1.1()
          /home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:290 +0x1a4
      github.com/hashicorp/vault/builtin/logical/pki.(*backend).startTidyOperation.func1()
          /home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:342 +0x278

    Previous read at 0x00c00c77680c by goroutine 90528:
      reflect.Value.Uint()
          /usr/local/go/src/reflect/value.go:2584 +0x195
      encoding/json.uintEncoder()
          /usr/local/go/src/encoding/json/encode.go:562 +0x45
      encoding/json.ptrEncoder.encode()
          /usr/local/go/src/encoding/json/encode.go:944 +0x3c2
      encoding/json.ptrEncoder.encode-fm()
          <autogenerated>:1 +0x90
      encoding/json.(*encodeState).reflectValue()
          /usr/local/go/src/encoding/json/encode.go:359 +0x88
      encoding/json.interfaceEncoder()
          /usr/local/go/src/encoding/json/encode.go:715 +0x17b
      encoding/json.mapEncoder.encode()
          /usr/local/go/src/encoding/json/encode.go:813 +0x854
      ... more stack trace pointing into JSON encoding and http
      handler...

In particular, because the tidy status was directly reading the uint
value without resorting to the atomic side, the JSON serialization could
race with a later atomic update.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Also use atomic load in tests

Because no tidy operation is running here, it should be safe to read the
pointed value directly, but use the safer atomic.Load for consistency.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-30 14:13:40 -05:00
Rachel Culpepper c151d5fd58
Vault-11713: Add the ability for transit to use managed keys (#18844)
* add the ability for transit to use managed keys

* add fields for rotate

* reorganize managed keys for better error messaging

* add aad for managed keys

* change encrypting key interface

* fix import error

* fix fmt errors

* move managed key rotation to a separate function

* fix fmt issue

* fix import error

* fix go.mod issue

* fix go.mod

* remove extra rotate param

* fix locking issue in rotation

* fix errors in policy tests

* move managed key params to fit existing policy functions

* fix sign and verify

* fix policy tests

* fix test error
2023-01-27 13:39:58 -06:00