Commit graph

1495 commits

Author SHA1 Message Date
Alexander Scheel a18187c643
Correctly distinguish empty issuer names in PKI (#18466)
* Correctly distinguish empty issuer names

When using client.Logical().JSONMergePatch(...) with an empty issuer
name, patch incorrectly reports:

> issuer name contained invalid characters

In this case, both the error in getIssuerName(...) is incorrect and
patch should allow setting an empty issuer name explicitly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-10 10:04:30 -05:00
Alexander Scheel 38de21468e
Add cluster_aia_path templating variable (#18493)
* Add cluster_aia_path templating variable

Per discussion with maxb, allow using a non-Vault distribution point
which may use an insecure transport for RFC 5280 compliance.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address feedback from Max

Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
2023-01-10 09:51:37 -05:00
Mike Palmiotto 43a78c85f4
Mark deprecated builtins Removed (#18039)
* Remove logical database builtins

* Drop removed builtins from registry keys

* Update plugin prediction test

* Remove app-id builtin

* Add changelog
2023-01-09 09:16:35 -05:00
Steven Clark cfd5b8a933
Resolve unrecognized parameter warnings on batch_input parameter in transit (#18299)
* Resolve unused warnings on batch_input parameter in transit

* Add cl

* Fix text in hmac batch_input parameter description
2023-01-04 09:15:48 -05:00
Alexander Scheel 3ccbddab0e
Add issuer reference info on JSON endpoint (#18482)
* Add issuer reference info on JSON endpoint

This endpoint is unauthenticated and shouldn't contain sensitive
information. However, listing the issuers (LIST /issuers) already
returns both the issuer ID and the issuer name (if any) so this
information is safe to return here.

When fetching /pki/issuer/default/json, it would be nice to know exactly
which issuer ID and name it corresponds to, without having to fetch the
authenticated endpoint as well.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-19 21:39:01 +00:00
Alexander Scheel 3a5b48afe4
Correctly handle issuer tidying in auto-tidy config (#18347)
* Correctly handle issuer tidying in auto-tidy config

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing parameters to auto-tidy docs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-14 15:35:21 -05:00
Scott Miller c9531431a4
Add the batch reference field, as in Transform, to Transit operations (#18243)
* Add the batch reference field, as in Transform, to Transit operations

* changelog

* docs

* More mapstructure tags
2022-12-13 12:03:40 -06:00
Scott Miller c1cfc11a51
Return the partial success code override for all batch error types (#18310)
* Return the partial success code override for all batch error types

* changelog

* docs

* Lost the actual override logic. :)

* And don't hardcode 400

* gate on success
2022-12-12 17:08:22 -06:00
Alexander Scheel f3911cce66
Add transit key config to disable upserting (#18272)
* Rename path_config -> path_keys_config

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add config/keys to disable upserting

Transit would allow anyone with Create permissions on the encryption
endpoint to automatically create new encryption keys. This becomes hard
to reason about for operators, especially if typos are subtly
introduced (e.g., my-key vs my_key) -- there is no way to merge these
two keys afterwards.

Add the ability to globally disable upserting, so that if the
applications using Transit do not need the capability, it can be
globally disallowed even under permissive policies.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on disabling upsert

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update website/content/api-docs/secret/transit.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

* Update website/content/api-docs/secret/transit.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
2022-12-08 15:45:18 -05:00
Alexander Scheel a8764e0cf1
Refactor PKI to use shared storage context (#18266)
A lot of places took a (context, backend, request) tuple, ignoring the
request proper and only using it for its storage. This (modified) tuple
is exactly the set of elements in the shared storage context, so we
should be using that instead of manually passing all three elements
around.

This simplifies a few places where we'd generate a storage context at
the request level and then split it apart only to recreate it again
later (e.g., CRL building).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-08 09:27:02 -05:00
Steven Clark 735a8d4b31
Address race in PKI test case (#18267)
- Nick brought this to our attention, one of the PKI test suites
   is overwriting the production code's value leading to a data race
   issue.
 - Remove the setting of the variable with the same value from the test
   suite.
2022-12-08 09:11:05 -05:00
Alexander Scheel 2398634862
Respond with data to all writes in PKI engine (#18222)
* Respond with data to all writes in PKI engine

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-05 10:40:39 -05:00
Alexander Scheel f86fdf530f
Allow templating cluster-local AIA URIs (#18199)
* Allow templating of cluster-local AIA URIs

This adds a new configuration path, /config/cluster, which retains
cluster-local configuration. By extending /config/urls and its issuer
counterpart to include an enable_templating parameter, we can allow
operators to correctly identify the particular cluster a cert was
issued on, and tie its AIA information to this (cluster, issuer) pair
dynamically.

Notably, this does not solve all usage issues around AIA URIs: the CRL
and OCSP responder remain local, meaning that some merge capability is
required prior to passing it to other systems if they use CRL files and
must validate requests with certs from any arbitrary PR cluster.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation about templated AIAs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* AIA URIs -> AIA URLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* issuer.AIAURIs might be nil

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow non-nil response to config/urls

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Always validate URLs on config update

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure URLs lack templating parameters

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Review feedback

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-05 10:38:26 -05:00
Steven Clark 9bf3b4c582
Do not use possibly nil HttpRequest object in default OCSP handler (#18190) 2022-12-01 13:23:41 -05:00
Steven Clark 826e87884e
Address a nil panic when writing an empty POST request to the ocsp handler (#18184)
* Address a nil panic when writing an empty POST request to the ocsp handler

 - Seems when no JSON body is sent with a POST request Vault will not
   populate the HTTPRequest member variable which caused the nil panic
   - vault write -force pki/ocsp
 - Add a check for it and the Body member variable to be nil before use.

* Add cl
2022-12-01 15:10:12 +00:00
Alexander Scheel 347cdf811c
Disable nginx integration test in pki test suites (#18141)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-29 13:30:25 -05:00
Alexander Scheel a04855c98d
Add crl integraiton to tests (#17447)
* Add tests using client certificates

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor Go TLS client tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for CRLs

Note that Delta CRL support isn't present in nginx or apache, so we lack
a server-side test presently. Wget2 does appear to support it however,
if we wanted to add a client-side OpenSSL test.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add checks for delta CRL with wget2

This ensures the delta CRL is properly formatted and accepted by
OpenSSL.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Re-add missing test helpers

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rename clientFullChain->clientWireChain

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-28 10:32:22 -05:00
Alexander Scheel a8faa543e6
Add pki nginx/wget/curl/Go integration tests (#17320)
* Rename integation_test.go->integration_test.go

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to fetch container's network addresses

This lets us return the on-network container address, allowing us to
spawn client containers which contact server containers.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add integration tests with nginx, curl, wget, Go

We build new integration tests, spawning a test instance on nginx and
ensuring we can connect with a variety of clients against a variety of
CA and leaf certificate types. This will ultimately let us detect issues
with compatibility as we expand the matrix of supported servers and
clients.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make runner reference unique

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Attempt to fix CI with longer wait

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Finish moving nginx tests to pkiext package

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* make fmt

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add more debugging, work on CircleCI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-23 15:00:18 -05:00
Tom Proctor 853643d02b
Remove pinned builtin plugin versions from storage (#18051)
* Removes _builtin_ versions from mount storage where it already exists
* Stops new builtin versions being put into storage on mount creation/tuning
* Stops the plugin catalog from returning a builtin plugin that has been overridden, so it more accurately reflects the plugins that are available to actually run
2022-11-23 18:36:25 +00:00
Alexander Scheel 2a801895e3
Create CSR in Transit, sign with PKI (#17630)
Execute with:

$ go test -v -run=TestTransitPKICSR github.com/hashicorp/vault/builtin/logical/transit
...
    backend_test.go:1843: csr: -----BEGIN CERTIFICATE REQUEST-----
        MIICXjCCAUYCAQAwGTEXMBUGA1UEAxMOZGFkZ2FyY29ycC5jb20wggEiMA0GCSqG
        SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDD8GUy2Rut9ILPXH/Ef7lEaYijuBB9wogd
        hKD3uJyfK5PqBqM8166UsrP7Y+bWkwDrMke3aDxXRNybys33kIc8KfGwS3omNYd3
        17KN1D4ZgQ+oW6xISa3ISOl4D7XeFtHeTP0U1plVXBd9kqTlo4YPlUF/kTfqmxDu
        2a41BIS5HlORdLLG+jQ3shRgwHANONBhlaUnIqEeykdW8/iEBlqoYlMzty9W724R
        2mKk0FzrVAZ/X5ZO992dAMrQDvc3Nofl+ddzbElBJLumrcDSwALFVge+ag1N48kE
        CCfxjizEykGdCrR+VELb8b33IgFf6EOVRnS5Qy8whmw943v5Oru5AgMBAAGgADAN
        BgkqhkiG9w0BAQsFAAOCAQEAdg9SwbrWszMmz60JWQPPfwW+XhzR0MdY82adK8P6
        9xpWyJU+U649tAFQb+PCT1OcU7ETd59QcEV38VLndBPWhotTXl5oB5XAqg2bkKHV
        nLc0cGwyxBSs77LALc//m2f5v2otO9fLOmuM2RMfD02ZUliBmZUzeaUIJYEfT+cS
        M60uLKJvnNBu5xH1q0oG9P0uNkpEX+QGx6SwhR1/41pmygiUR+uwJxxuRGMvECoN
        dsHZtzi7ftEHBJ9tk94hd/RFnDsvWlHGyfRWhALNtbo6QjHxjBJIFKh+GHlI8Tnf
        6YWvD0VIodE609+RlCrhFlGd+3NUSt0b/f0bgkMJLzLqEw==
        -----END CERTIFICATE REQUEST-----
    backend_test.go:1878: root: -----BEGIN CERTIFICATE-----
        MIIDHTCCAgWgAwIBAgIUIwCzCdrsgkcNOi5liRNHeH+n+tUwDQYJKoZIhvcNAQEL
        BQAwFjEUMBIGA1UEAxMLUEtJIFJvb3QgWDEwHhcNMjIxMDIxMTQ1NjQyWhcNMjIx
        MTIyMTQ1NzExWjAWMRQwEgYDVQQDEwtQS0kgUm9vdCBYMTCCASIwDQYJKoZIhvcN
        AQEBBQADggEPADCCAQoCggEBAMdM65f5p3fLwQP1sezBRFqAxUZhOQwnnnp8mFXp
        3fIF9pqLMzNvyd+bCUXv+aFalX4KY1iOoKVHJWwtpXMoKn40U+DZkapR9CsVQt5Q
        9xzIcuPPuI+/oNwU4qB9mAuwG+U7KLosGnQOR3NI02A4dnl5I0z8Y/DJLz29GP/P
        1zPYMBRBpkMz4F2Xr0w6tTXWDsmqZ9j7ukBDoizmnB2xfKzSjCVmQvXa71UlqbG9
        td75LCgpiQh/50mHFHs6RKtqrlFUY5BtPPs+tHUf4nklieIzbAEwA8Fbq4d/Xpq1
        HRoRvWj3nelX/h/IRlj/VKJssd1ZL+1kdzxKB4N6AFC1nusCAwEAAaNjMGEwDgYD
        VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFCniHbjHy/UG
        ShROBOiikqxCe+OkMB8GA1UdIwQYMBaAFCniHbjHy/UGShROBOiikqxCe+OkMA0G
        CSqGSIb3DQEBCwUAA4IBAQBSKnl154oyc1Rncm4sr/1f1QM5rCaw/lqgISfvki8m
        t6yyxQL+9lDpebFjBDTL5teRzuMYyqN4pRkIhpITDGmFXRpEOv14mcbASX7nPBEN
        bYXhOh1UC8a0CLzT2ll0ERFNnUEPRi0s0ONRm3lIZAV3Mzf4sOdwfRwmP33hBe/1
        V9D7Lcx5N84EPrvGC/r8F/PsVKHyKFS46qB1MvhMppRG6fJ2cFmg5UGwdKdmxuvz
        FoT+RaTLkgcQgkDuYClNco5OVIM7Bd4JTNK3WbqvtGklOHslrz+ND0eMYM/LK+ZS
        zNM35nzK6QaN2M4IO4Wuy3y2yu8xllEfmssXwAtTi4wk
        -----END CERTIFICATE-----
    backend_test.go:1879: leaf: -----BEGIN CERTIFICATE-----
        MIIDDzCCAfegAwIBAgIUBGeUSi0p3ffndZqgvlBvMvn8qgMwDQYJKoZIhvcNAQEL
        BQAwFjEUMBIGA1UEAxMLUEtJIFJvb3QgWDEwHhcNMjIxMDIxMTQ1NjQyWhcNMjIx
        MDIxMTUwNzEyWjAZMRcwFQYDVQQDEw5kYWRnYXJjb3JwLmNvbTCCASIwDQYJKoZI
        hvcNAQEBBQADggEPADCCAQoCggEBAMPwZTLZG630gs9cf8R/uURpiKO4EH3CiB2E
        oPe4nJ8rk+oGozzXrpSys/tj5taTAOsyR7doPFdE3JvKzfeQhzwp8bBLeiY1h3fX
        so3UPhmBD6hbrEhJrchI6XgPtd4W0d5M/RTWmVVcF32SpOWjhg+VQX+RN+qbEO7Z
        rjUEhLkeU5F0ssb6NDeyFGDAcA040GGVpScioR7KR1bz+IQGWqhiUzO3L1bvbhHa
        YqTQXOtUBn9flk733Z0AytAO9zc2h+X513NsSUEku6atwNLAAsVWB75qDU3jyQQI
        J/GOLMTKQZ0KtH5UQtvxvfciAV/oQ5VGdLlDLzCGbD3je/k6u7kCAwEAAaNSMFAw
        DgYDVR0PAQH/BAQDAgOoMB0GA1UdDgQWBBSRCRR/62DjS1kjWHrVQ0Y58leUbDAf
        BgNVHSMEGDAWgBQp4h24x8v1BkoUTgToopKsQnvjpDANBgkqhkiG9w0BAQsFAAOC
        AQEAvWPLGqtC1SRy61Y17HtJ0giDUwpCZbOUkAwtdDAnKIhR1v4wrlY3sKUBLuhK
        xOJIWfVlCnPUt5uTnPaWyVyUfry6YNerish1k7ny/R1n58PjsPhUg8GJB9HHsME+
        gQQ22z6D/87n0bEE8PaTzIU6+cVHoIBJ0rqzjZVkBs0cEjf+l40RPP1h+ZiTw27u
        CR2iXmHJ9TQ8ZBWygIhxB9JOMbk5jpH6w6wJqq8XK9zuC1hlYbXH1K5KvZJxAPlh
        CJkoq2KxaIwByTHjRdGjDogSibsyY+CxQUnktefXb6tYKvFTpUFsh1fjQRCwUrlD
        SExMRHhFJBHfyPD1w26N3IjRlg==
        -----END CERTIFICATE-----

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-23 17:57:23 +00:00
Steven Clark 92c1a2bd0a
New PKI API to generate and sign a CRL based on input data (#18040)
* New PKI API to generate and sign a CRL based on input data

 - Add a new PKI API that allows an end-user to feed in all the
   information required to generate and sign a CRL by a given issuer.
 - This is pretty powerful API allowing an escape hatch for 3rd parties
   to craft customized CRLs with extensions based on their individual
   needs

* Add api-docs and error if reserved extension is provided as input

* Fix copy/paste error in Object Identifier constants

* Return nil on errors instead of partially filled slices

* Add cl
2022-11-22 11:41:04 -05:00
Scott Miller b51b2a7027
Add cached OCSP client support to Cert Auth (#17093)
* wip

* Add cached OCSP client support to Cert Auth

* ->pointer

* Code cleanup

* Fix unit tests

* Use an LRU cache, and only persist up to 1000 of the most recently used values to stay under the storage entry limit

* Fix caching, add fail open mode parameter to cert auth roles

* reduce logging

* Add the retry client and GET then POST logic

* Drop persisted cache, make cache size configurable, allow for parallel testing of multiple servers

* dead code

* Update builtin/credential/cert/path_certs.go

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Hook invalidate to reinit the ocsp cache size

* locking

* Conditionally init the ocsp client

* Remove cache size config from cert configs, it's a backend global

* Add field

* Remove strangely complex validity logic

* Address more feedback

* Rework error returning logic

* More edge cases

* MORE edge cases

* Add a test matrix with a builtin responder

* changelog

* Use an atomic for configUpdated

* Actually use ocsp_enabled, and bind to a random port for testing

* Update builtin/credential/cert/path_login.go

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor unit tests

* Add status to cache

* Make some functions private

* Rename for testing, and attribute

* Up to date gofumpt

* remove hash from key, and disable the vault dependent unit test

* Comment out TestMultiOCSP

* imports

* more imports

* Address semgrep results

* Attempt to pass some sort of logging to test_responder

* fix overzealous search&replace

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-21 10:39:24 -06:00
Steven Clark 01e87c481c
Add new PKI api to combine and sign different CRLs from the same issuer (#17813)
* Add new PKI api to combine and sign different CRLs from the same issuer

 - Add a new PKI api /issuer/<issuer ref>/resign-crls that will allow
   combining and signing different CRLs that were signed by the same
   issuer.
 - This allows external actors to combine CRLs into a single CRL across
   different Vault clusters that share the CA certificate and key material
   such as performance replica clusters and the primary cluster

* Update API docs

* PR Feedback - Delta CRL rename

* Update to latest version of main

* PR Feedback - Get rid of the new caEntry struct

* Address PR feedback in api-docs and PEM encoded response
2022-11-17 16:53:05 -05:00
Michael Anthony aa74bd7ed7
[QT-309] Resolve AWS config before testing ACC (#17949)
* Check to resolve AWS config before testing ACC

* Adjust wording of error to be more clear
2022-11-17 11:55:27 -07:00
Alexander Scheel c25b90831e
Move pki docker tests to pkiext (#17928)
* Export CreateBackendWithStorage for pkiext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move zlint_test.go to pkiext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix mount all test to ignore pkiext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-14 18:26:26 -05:00
Alexander Scheel a0136f5f19
Handle removed default issuers (#17930)
Credit to Steve for finding this one.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-14 15:13:39 -05:00
Michael Dempsey fd032831cb
Expose ssh algorithm_signer in web interface (#10114) (#10299)
* Expose ssh algorithm_signer in web interface (#10114)

* Adds allowed values for algorithm_signer to ssh plugin API
* Adds algorithm_signer as field in UI

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-10 14:24:53 -08:00
Alexander Scheel 5a2ee4ca7a
Add automatic tidy of expired issuers (#17823)
* Add automatic tidy of expired issuers

To aid PKI users like Consul, which periodically rotate intermediates,
and provided a little more consistency with older versions of Vault
which would silently (and dangerously!) replace the configured CA on
root/intermediate generation, we introduce an automatic tidy of expired
issuers.

This includes a longer safety buffer (1 year) and logging of the
relevant issuer information prior to deletion (certificate contents, key
ID, and issuer ID/name) to allow admins to recover this value if
desired, or perform further cleanup of keys.

From my PoV, removal of the issuer is thus a relatively safe operation
compared to keys (which I do not feel comfortable removing) as they can
always be re-imported if desired. Additionally, this is an opt-in tidy
operation, not enabled by default. Lastly, most major performance
penalties comes with lots of issuers within the mount, not as much
large numbers of keys (as only new issuer creation/import operations are
affected, unlike LIST /issuers which is a public, unauthenticated
endpoint).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add docs on tidy of issuers

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Restructure logging

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing fields to expected tidy output

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-10 10:53:26 -05:00
Alexander Scheel e7d57bfe90
Move from %v->%w for errs (#17860)
Also remove one duplicate error masked by return.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-09 15:40:26 -05:00
Alexander Scheel 06f30de35f
Optional automatic default issuer selection (#17824)
* Correctly preserve other issuer config params

When setting a new default issuer, our helper function would overwrite
other parameters in the issuer configuration entry. However, up until
now, there were none.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add new parameter to allow default to follow new

This parameter will allow operators to have the default issuer
automatically update when a new root is generated or a single issuer
with a key (potentially with others lacking key) is imported.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Storage migration tests fail on new members

These internal members shouldn't be tested by the storage migration
code, and so should be elided from the test results.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Follow new issuer on root generation, import

This updates the two places where issuers can be created (outside of
legacy CA bundle migration which already sets the default) to follow
newly created issuers when the config is set.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for new default-following behavior

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-08 14:40:29 -05:00
Steven Clark 419ba9159c
Add new API to PKI to list revoked certificates (#17779)
* Add new API to PKI to list revoked certificates

 - A new API that will return the list of serial numbers of
   revoked certificates on the local cluster.

* Add cl

* PR feedback
2022-11-03 14:17:17 -04:00
Alexander Scheel ffa4825693
PKI - Fix order of chain building writes (#17772)
* Ensure correct write ordering in rebuildIssuersChains

When troubleshooting a recent migration failure from 1.10->1.11, it was
noted that some PKI mounts had bad chain construction despite having
valid, chaining issuers. Due to the cluster's leadership trashing
between nodes, the migration logic was re-executed several times,
partially succeeding each time. While the legacy CA bundle migration
logic was written with this in mind, one shortcoming in the chain
building code lead us to truncate the ca_chain: by sorting the list of
issuers after including non-written issuers (with random IDs), these
issuers would occasionally be persisted prior to storage _prior_ to
existing CAs with modified chains.

The migration code carefully imported the active issuer prior to its
parents. However, due to this bug, there was a chance that, if write to
the pending parent succeeded but updating the active issuer didn't, the
active issuer's ca_chain field would only contain the self-reference and
not the parent's reference as well. Ultimately, a workaround of setting
and subsequently unsetting a manual chain would force a chain
regeneration.

In this patch, we simply fix the write ordering: because we need to
ensure a stable chain sorting, we leave the sort location in the same
place, but delay writing the provided referenceCert to the last
position. This is because the reference is meant to be the user-facing
action: without transactional write capabilities, other chains may
succeed, but if the last user-facing action fails, the user will
hopefully retry the action. This will also correct migration, by
ensuring the subsequent issuer import will be attempted again,
triggering another chain build and only persisting this issuer when
all other issuers have also been updated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remigrate ca_chains to fix any missing issuers

In the previous commit, we identified an issue that would occur on
legacy issuer migration to the new storage format. This is easy enough
to detect for any given mount (by an operator), but automating scanning
and remediating all PKI mounts in large deployments might be difficult.

Write a new storage migration version to regenerate all chains on
upgrade, once.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add issue to PKI considerations documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correct %v -> %w in chain building errs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-03 11:50:03 -04:00
Alexander Scheel 8e6e53cf63
Use hashicorp mirror for container pulls (#17778)
When running the test suite in CI (where requests are centralized from
relatively few IPs), we'd occasionally hit Dockerhub's rate limits.
Luckily Hashicorp runs a (limited) public mirror of the containers we
need, so we can switch to them here in the tests.

For consistency between developer and CI, we've opted to have the tests
always pull from the Hashicorp mirror, rather than updating the CI
runner to prefer the mirror.

We exclude nomad and influxdb as we don't presently mirror these repos.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-02 13:33:17 -04:00
Steven Clark 550fbdc41c
Return revocation info within existing certs/<serial> api (#17774)
* Return revocation info within existing certs/<serial> api

 - The api already returned both the certificate and a revocation_time
   field populated. Update the api to return revocation_time_rfc3339
   as we do elsewhere and also the issuer id if it was revoked.
 - This will allow callers to associate a revoked cert with an issuer

* Add cl

* PR feedback (docs update)
2022-11-02 13:06:04 -04:00
Alexander Scheel d5f6c36c1c
Clarify ssh/public_key response, recommend -format=raw (#17745)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-31 11:14:49 -04:00
Robert 5b34158edd
Update AWS secrets tests (#17736)
* Don't use a duplicate sync object for stepwise tests precheck

* Change STS test check to no longer look for a secret, add SetSourceIdentity policy to role
2022-10-31 09:52:35 -05:00
Alexander Scheel 5e0ce5ec3e
Bump validity period check to satisfy CircleCI (#17740)
* Bump validity period check to satisfy CircleCI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update builtin/logical/pki/backend_test.go

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-31 10:01:09 -04:00
Brian Howe cff0baf322
secrets/aws: don't create leases for AWS STS secrets (#15869)
* don't create leases for AWS STS secrets

* don't create leases for aws federation tokens
2022-10-28 16:28:25 -05:00
Alexander Scheel d67023c3b3
Add empty expiry crlConfig upgrade test (#17701)
* Add regression test for default CRL expiry

Also fixes a bug w.r.t. upgrading older entries and missing the Delta
Rebuild Interval field, setting it to the default.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog for earlier PR

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-27 11:20:12 -04:00
James Protzman a47848706e
Default crl expiry (#17693)
Ref: https://github.com/hashicorp/vault/issues/17642
2022-10-27 10:47:17 -04:00
Alexander Scheel 1733d2a3d6
Add support for PKCSv1_5_NoOID signatures (#17636)
* Add support for PKCSv1_5_NoOID signatures

This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.

These are specified in RFC 3447 Section 9.2.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Exclude new none type from PSS based tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for PKCS#1v1.5 signatures

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-27 08:26:20 -04:00
Alexander Scheel 09939f0ba9
Add AD mode to Transit's AEAD ciphers (#17638)
* Allow passing AssociatedData factories in keysutil

This allows the high-level, algorithm-agnostic Encrypt/Decrypt with
Factory to pass in AssociatedData, and potentially take multiple
factories (to allow KMS keys to work). On AEAD ciphers with a relevant
factory, an AssociatedData factory will be used to populate the
AdditionalData field of the SymmetricOpts struct, using it in the AEAD
Seal process.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add associated_data to Transit Encrypt/Decrypt API

This allows passing the associated_data (the last AD in AEAD) to
Transit's encrypt/decrypt when using an AEAD cipher (currently
aes128-gcm96, aes256-gcm96, and chacha20-poly1305). We err if this
parameter is passed on non-AEAD ciphers presently.

This associated data can be safely transited in plaintext, without risk
of modifications. In the event of tampering with either the ciphertext
or the associated data, decryption will fail.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add to documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-24 13:41:02 -04:00
Ben Roberts d710f8e8dc
Evaluate ssh validprincipals user template before splitting (#16622)
The SSH secrets engine previously split the `validPrincipals` field
on comma, then if user templating is enabled, evaluated the
templates on each substring. This meant the identity template was only
ever allowed to return a single principal. There are use cases
where it would be helpful for identity metadata to contain a list
of valid principals and for the identity template to be able to inject
all of those as valid principals.

This change inverts the order of processing. First the template
is evaluated, and then the resulting string is split on commas.
This allows the identity template to return a single comma-separated
string with multiple permitted principals.

There is a potential security implication here, that if a user is
allowed to update their own identity metadata, they may be able to
elevate privileges where previously this was not possible.

Fixes #11038
2022-10-13 17:34:36 -05:00
Nick Cabatoff 8e67651dcd
Fix a data race with rollbackPeriod. (#17387) 2022-10-13 09:59:07 -04:00
Alexander Scheel daf29de742
Add pki zlint tests (#17305)
* Add tests for zlint-clean CA building

This test ensures that we can consistently pass ZLint's CA linting
tests on a root certificate generated by Vault. In particular, nominal
requirements are placed on the structure on the issuer's Subject, which
we supply, and the remaining requirements pass.

The one exception is we include both RFC and CA/BF BR lints in the
default zlint checks; this means ECDSA P-521 (which isn't accepted by
Mozilla's root store policies) is rejected, so we ignore to lints
related to that.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add options to copy to/from container, fix stopping

Stopping the container takes a bit of time for some unknown reason so
I've instead opted to shorten the sleep in the zlint tests to avoid
consuming resources too long after the test finish.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make zlint tests execute in parallel

This improves the overall test time of the zlint tests, making the
container build up front once (provisioning zlint), and then copying the
cert into the new container image later.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* make fmt

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-12 13:29:06 -04:00
Peter Wilson cf961e2692
The 2 year temporary disabling of this test has come to an end (#17468) 2022-10-12 15:51:45 +01:00
Alexander Scheel d48e739b1d
Fix tidy-status, tidy-cancel on PR Secondaries (#17497)
* Fix tidy-status, tidy-cancel on PR Secondaries

PKI's tidy-status included a bug that prevented PR secondary nodes from
responding with the status of the running tidy operation: while the
operation constructor correctly forwarded the node on PR standby
instances, the handler itself forwarded also on PR secondary nodes.

This is incorrect as the PR secondary nodes are the active node in the
local PR cluster, and run tidy operations otherwise.

This meant that while auto-tidy and tidy operations would run, there was
no insight into the process.

When implementing tidy-cancel, tidy-status's handler logic was reused,
duplicating the bug there as well.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-12 09:15:06 -04:00
Alexander Scheel fdb13d7481
Don't include issuers on delta CRLs (#17463)
When revoking an issuer, we immediately force a full rebuild of all CRLs
(complete and delta). However, we had forgotten to guard the delta CRL's
inclusion of augmented issuers, resulting in double-listing the issuer's
serial number on both the complete and the delta CRL. This isn't
necessary as the delta's referenced complete CRL number has incremented
to the point where the issuer itself was included on the complete CRL.

Avoid this double reference and don't include issuers on delta CRLs;
they should always appear only on the complete CRL.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-07 12:36:22 -04:00
Steven Clark f0bf670b0c
Unit tests that validate OCSP signatures leverage revocation signature algo (#17452)
- Add some unit tests around the OCSP response validation that we
   are using the proper signature algorithms.
 - Add in test cases as well to validate SHA384 and SHA512 requested hash support
2022-10-07 12:33:17 -04:00
Ruben De Visscher 1f0cf558a9
Fix for duplicate SANs in signed certificates (#16700)
* Fix for duplicate SANs in signed certificates when othernames are present in the CSR SAN extension and UseCSRValues is true.

When UseCSRValues is true (as is the case on the sign-verbatim endpoint), all extensions including Subject Alternative Names are copied from the CSR to the final certificate.
If the Subject Alternative Name in question contains any othernames (such as a Microsoft UPN) the SAN extension is added again as a workaround for an encoding issue (in function HandleOtherSANs).
Having duplicate x509v3 extensions is invalid and is rejected by openssl on Ubuntu 20.04, and also by Go since https://github.com/golang/go/issues/50988 (including in Go 1.19).

In this fix I do not add the extension from the CSR if it will be added during HandleOtherSANs.

* Added unittest and changelog entry.
2022-10-07 12:19:08 -04:00