Commit graph

963 commits

Author SHA1 Message Date
Alexander Scheel df07170d26
Vault Raw Read Support (CLI & Client) (#14945)
* Expose raw request from client.Logical()

Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.

Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add raw formatter for direct []byte data

As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for reading raw API endpoints

Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:

> Error reading pki/ca/pem: invalid character '-' in numeric literal

Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove panic

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-28 09:45:32 -04:00
Alexander Scheel d67023c3b3
Add empty expiry crlConfig upgrade test (#17701)
* Add regression test for default CRL expiry

Also fixes a bug w.r.t. upgrading older entries and missing the Delta
Rebuild Interval field, setting it to the default.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog for earlier PR

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-27 11:20:12 -04:00
Violet Hynes 6d9ea2862e
VAULT-8519 fix spurious "unknown or unsupported fields" warnings for JSON config (#17660)
* VAULT-8519 add tests for HCL unknown field bug

* VAULT-8519 upversion hcl

* VAULT-8519 include correct comitts in tag

* VAULT-8519 Add changelog
2022-10-27 10:28:03 -04:00
Alexander Scheel 1733d2a3d6
Add support for PKCSv1_5_NoOID signatures (#17636)
* Add support for PKCSv1_5_NoOID signatures

This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.

These are specified in RFC 3447 Section 9.2.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Exclude new none type from PSS based tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for PKCS#1v1.5 signatures

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-27 08:26:20 -04:00
akshya96 1e189016e2
update protoc version to 3.21.7 oss (#17499)
* update protoc to 3.21.7

* adding changelog
2022-10-26 16:49:44 -07:00
Jordan Reimer 571851cee3
OIDC Alternate Path Bug (#17661)
* adds error handling to auth-jwt component for missing roles and fixes bug where role wasn't being retained when using alternate oidc mount path at login

* fixes jwt login bug from auth mount tabs and adds test

* updates okta-number-challenge success value to arg in template

* adds changelog entry

* fixes issues logging in manually with jwt

* reverts mistaken change
2022-10-26 15:34:43 -06:00
Mike Palmiotto cc96c6f470
Store login MFA secret with tokenhelper (#17040)
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
2022-10-26 17:02:26 -04:00
Luis (LT) Carbonell 3425f8b36c
Add Paging Interface for LDAP Connection (#17640) 2022-10-26 14:05:53 -05:00
Alexander Scheel 1721cc9f75
Add PATCH support to Vault CLI (#17650)
* Add patch support to CLI

This is based off the existing write command, using the
JSONMergePatch(...) API client method rather than Write(...), allowing
us to update specific fields.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on PATCH support

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-26 14:30:40 -04:00
Alexander Scheel 09939f0ba9
Add AD mode to Transit's AEAD ciphers (#17638)
* Allow passing AssociatedData factories in keysutil

This allows the high-level, algorithm-agnostic Encrypt/Decrypt with
Factory to pass in AssociatedData, and potentially take multiple
factories (to allow KMS keys to work). On AEAD ciphers with a relevant
factory, an AssociatedData factory will be used to populate the
AdditionalData field of the SymmetricOpts struct, using it in the AEAD
Seal process.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add associated_data to Transit Encrypt/Decrypt API

This allows passing the associated_data (the last AD in AEAD) to
Transit's encrypt/decrypt when using an AEAD cipher (currently
aes128-gcm96, aes256-gcm96, and chacha20-poly1305). We err if this
parameter is passed on non-AEAD ciphers presently.

This associated data can be safely transited in plaintext, without risk
of modifications. In the event of tampering with either the ciphertext
or the associated data, decryption will fail.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add to documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-24 13:41:02 -04:00
Austin Gebauer 5d0aab1099
auth/azure: documents auth support for VMSS flexible orchestration (#17540)
* auth/azure: documents auth support for VMSS flexible orchestration

* adds changelog
2022-10-20 12:36:29 -07:00
Scott Miller 473d3c26b3
Intercept key_id from generic configmap and turn it into a wrapperv2 option (#17612)
* Intercept key_id from generic configmap and turn it into a wrapperv2 option

* changelog

* Doubled append

* Only convert key_id to an opt in OCI
2022-10-19 15:42:56 -05:00
Calvin Leung Huang 9e6256178e
database/snowflake: update plugin to v0.6.1 (#17593)
* database/snowflake: update plugin to v0.6.1

* add changelog entry
2022-10-18 15:49:37 -07:00
Jordan Reimer be632db682
Ember Upgrade to 4.4 (#17086)
* runs ember-cli-update to 4.4.0

* updates yarn.lock

* updates dependencies causing runtime errors (#17135)

* Inject Store Service When Accessed Implicitly (#17345)

* adds codemod for injecting store service

* adds custom babylon parser with decorators-legacy plugin for jscodeshift transforms

* updates inject-store-service codemod to only look for .extend object expressions and adds recast options

* runs inject-store-service codemod on js files

* replace query-params helper with hash (#17404)

* Updates/removes dependencies throwing errors in Ember 4.4 (#17396)

* updates ember-responsive to latest

* updates ember-composable-helpers to latest and uses includes helper since contains was removed

* updates ember-concurrency to latest

* updates ember-cli-clipboard to latest

* temporary workaround for toolbar-link component throwing errors for using params arg with LinkTo

* adds missing store injection to auth configure route

* fixes issue with string-list component throwing error for accessing prop in same computation

* fixes non-iterable query params issue in mfa methods controller

* refactors field-to-attrs to handle belongsTo rather than fragments

* converts mount-config fragment to belongsTo on auth-method model

* removes ember-api-actions and adds tune method to auth-method adapter

* converts cluster replication attributes from fragment to relationship

* updates ember-data, removes ember-data-fragments and updates yarn to latest

* removes fragments from secret-engine model

* removes fragment from test-form-model

* removes commented out code

* minor change to inject-store-service codemod and runs again on js files

* Remove LinkTo positional params (#17421)

* updates ember-cli-page-object to latest version

* update toolbar-link to support link-to args and not positional params

* adds replace arg to toolbar-link component

* Clean up js lint errors (#17426)

* replaces assert.equal to assert.strictEqual

* update eslint no-console to error and disables invididual intended uses of console

* cleans up hbs lint warnings (#17432)

* Upgrade bug and test fixes (#17500)

* updates inject-service codemod to take arg for service name and runs for flashMessages service

* fixes hbs lint error after merging main

* fixes flash messages

* updates more deps

* bug fixes

* test fixes

* updates ember-cli-content-security-policy and prevents default form submission throwing errors

* more bug and test fixes

* removes commented out code

* fixes issue with code-mirror modifier sending change event on setup causing same computation error

* Upgrade Clean Up (#17543)

* updates deprecation workflow and filter

* cleans up build errors, removes unused ivy-codemirror and sass and updates ember-cli-sass and node-sass to latest

* fixes control groups test that was skipped after upgrade

* updates control group service tests

* addresses review feedback

* updates control group service handleError method to use router.currentURL rather that transition.intent.url

* adds changelog entry
2022-10-18 09:46:02 -06:00
Tom Proctor 455f1f6073
CLI: Fix secrets list -detailed headings (#17577) 2022-10-18 14:46:11 +01:00
Violet Hynes 5861c51e70
VAULT-8719 Support data array for alias clash error response so UI/machines can understand error (#17459)
* VAULT-8719 Support data array for alias clash error response so UI can understand error

* VAULT-8719 Changelog

* VAULT-8719 Update alias mount update logic

* VAULT-8719 Further restrict IsError()
2022-10-17 14:46:25 -04:00
Nick Cabatoff f94bd10540
Tolerate NamespaceByID returning (nil,nil) when looking up an mfa enforcement's ns (#17562) 2022-10-17 09:18:02 -04:00
Hamid Ghaf 8a624c1264
prevent memory leak when using control group factors in a policy (#17532)
* prevent a possible memory leak when using control group factors in a policy

* CL
2022-10-14 19:15:15 -04:00
Ben Roberts d710f8e8dc
Evaluate ssh validprincipals user template before splitting (#16622)
The SSH secrets engine previously split the `validPrincipals` field
on comma, then if user templating is enabled, evaluated the
templates on each substring. This meant the identity template was only
ever allowed to return a single principal. There are use cases
where it would be helpful for identity metadata to contain a list
of valid principals and for the identity template to be able to inject
all of those as valid principals.

This change inverts the order of processing. First the template
is evaluated, and then the resulting string is split on commas.
This allows the identity template to return a single comma-separated
string with multiple permitted principals.

There is a potential security implication here, that if a user is
allowed to update their own identity metadata, they may be able to
elevate privileges where previously this was not possible.

Fixes #11038
2022-10-13 17:34:36 -05:00
akshya96 2d6c6c01c4
vault operator init -output-curl-string bug (#17514)
* fixing -output-curl-string bug

* add changelog

* fix for -output-policy

* adding comment for -output-policy
2022-10-13 15:15:01 -07:00
Alexander Scheel d48e739b1d
Fix tidy-status, tidy-cancel on PR Secondaries (#17497)
* Fix tidy-status, tidy-cancel on PR Secondaries

PKI's tidy-status included a bug that prevented PR secondary nodes from
responding with the status of the running tidy operation: while the
operation constructor correctly forwarded the node on PR standby
instances, the handler itself forwarded also on PR secondary nodes.

This is incorrect as the PR secondary nodes are the active node in the
local PR cluster, and run tidy operations otherwise.

This meant that while auto-tidy and tidy operations would run, there was
no insight into the process.

When implementing tidy-cancel, tidy-status's handler logic was reused,
duplicating the bug there as well.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-12 09:15:06 -04:00
Mike Palmiotto 9dc8744885
Update main to go 1.19.2 (#17437)
* Update go version to 1.19.2

This commit updates the default version of go to 1.19.2. This update
includes minor security fixes for archive/tar, net/http/httputil, and
regexp packages.

For more information on the release, see: https://go.dev/doc/devel/release#go1.19.2

* Update Docker versions in CI to 20.10.17

After updating Vault to go version 1.19.2, there were several SIGABRTs
in the vault tests. These were related to a missing `pthread_create`
syscall in Docker. Since CI was using a much older version of Docker,
the fix was to bump it to latest-1 (20.10.17).

While we're at it, add a note in the developer docs encouraging the use
of the latest Docker version.
2022-10-07 14:24:14 -04:00
Ruben De Visscher 1f0cf558a9
Fix for duplicate SANs in signed certificates (#16700)
* Fix for duplicate SANs in signed certificates when othernames are present in the CSR SAN extension and UseCSRValues is true.

When UseCSRValues is true (as is the case on the sign-verbatim endpoint), all extensions including Subject Alternative Names are copied from the CSR to the final certificate.
If the Subject Alternative Name in question contains any othernames (such as a Microsoft UPN) the SAN extension is added again as a workaround for an encoding issue (in function HandleOtherSANs).
Having duplicate x509v3 extensions is invalid and is rejected by openssl on Ubuntu 20.04, and also by Go since https://github.com/golang/go/issues/50988 (including in Go 1.19).

In this fix I do not add the extension from the CSR if it will be added during HandleOtherSANs.

* Added unittest and changelog entry.
2022-10-07 12:19:08 -04:00
Nick Cabatoff 39c7e7c191
Add more raft metrics, emit more metrics on non-perf standbys (#12166)
Add some metrics helpful for monitoring raft cluster state.

Furthermore, we weren't emitting bolt metrics on regular (non-perf) standbys, and there were other metrics
in metricsLoop that would make sense to include in OSS but weren't.  We now have an active-node-only func,
emitMetricsActiveNode.  This runs metricsLoop on the active node.  Standbys and perf-standbys run metricsLoop
from a goroutine managed by the runStandby rungroup.
2022-10-07 09:09:08 -07:00
Tom Proctor 2cf29f93c7
Plugins: Add version info to CLI and server log output (#17430) 2022-10-06 12:54:27 +01:00
AnPucel d869496969
Fix for KV_V2 Custom Metadata Bug (#17395) 2022-10-05 16:43:54 -07:00
Nick Cabatoff d5e0353696
Fix unsafe access to perf standby status from systemview (#17186)
Ensure that we don't try to access Core.perfStandby or Core.PerfStandby() from dynamicSystemView, which might be accessed with or without stateLock held.
2022-10-05 08:56:36 -04:00
Peter Wilson 138c516498
Update plugin consul template v0.29.5 (#17419)
* Updated consul-template to v0.29.5
2022-10-04 22:34:28 +01:00
Angel Garbarino 345c65ff04
Bug Fix for Kymgmt keyType on default (#17407)
* the fix

* add test coverage

* changelog

* removes comment
2022-10-04 13:14:25 -06:00
Violet Hynes 6fddb953e9
VAULT-8631 Upgrade vault-plugin-secrets to v0.13.3, to enable synchronous KVV2 creation (#17406)
* VAULT-8631 Upgrade vault-plugin-secrets to v0.13.2

* VAULT-8631 Update version, go mod tidy

* VAULT-8631 Add changelog
2022-10-04 13:54:38 -04:00
Renato Costa ea1257be2e
fix incorrect use of loop variable (#16872)
This fixes a couple of references to loop variables in parallel tests
and deferred functions. When running a parallel test (calling
`t.Parallel()`) combined with the table-driven pattern, it's necessary
to copy the test case loop variable, otherwise only the last test case
is exercised. This is documented in the `testing` package:

https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks

`defer` statements that invoke a closure should also not reference a
loop variable directly as the referenced value will change in each
iteration of the loop.

Issues were automatically found with the `loopvarcapture` linter.
2022-10-04 09:23:37 -04:00
Steven Clark 10ecf10248
PKI: Add support for signature_bits param to the intermediate/generate api (#17388)
* PKI: Add support for signature_bits param to the intermediate/generate api

 - Mainly to work properly with GCP backed managed keys, we need to
   issue signatures that would match the GCP key algorithm.
 - At this time due to https://github.com/golang/go/issues/45990 we
   can't issue PSS signed CSRs, as the libraries in Go always request
   a PKCS1v15.
 - Add an extra check in intermediate/generate that validates the CSR's
   signature before providing it back to the client in case we generated
   a bad signature such as if an end-user used a GCP backed managed key
   with a RSA PSS algorithm.
   - GCP ignores the requested signature type and always signs with the
     key's algorithm which can lead to a CSR that says it is signed with
     a PKCS1v15 algorithm but is actually a RSA PSS signature

* Add cl

* PR feedback
2022-10-03 12:39:54 -04:00
Steven Clark bb1d36f401
PKI: Do not load revoked certificates if CRL has been disabled (#17385)
* PKI: Do not load revoked certificates if CRL has been disabled

 - Restore the prior behavior of not reading in all revoked certificates
   if the CRL has been disabled as there might be performance issues
   if a customer had or is still revoking a lot of certificates.

* Add cl
2022-10-03 10:04:32 -04:00
Angel Garbarino 4cfd47d5fb
Fix small TtlPIcker2 bug (#17376)
* fix

* add changelog

* wrong file name for changelog

* update from what core responded with re: type duration
2022-09-30 18:28:27 -04:00
Jordan Reimer 25f4cfac6b
removes hcp link ui changelog entry (#17373) 2022-09-30 12:10:13 -06:00
Tom Proctor 4bd5af87f4
Plugins: Fix file permissions check to always use the correct path (#17340)
* Add failing test for when command != plugin name
* wrapFactoryCheckPerms uses pluginCatalog.Get to fetch the correct command
* Use filepath.Rel for consistency with plugin read API handler
2022-09-30 10:33:31 +01:00
Peter Wilson 42ba1384ff
Added flag and env var which will disable client redirection (#17352)
* Added flag and env var which will disable client redirection

* Added changelog

* Docs fix for unsaved file, and test single request made

* Updated test for case when redirect is enabled, updated docs based on suggestions
2022-09-30 09:29:37 +01:00
Tom Proctor 12ca81bc9b
cli/api: Update plugin listing to always include version info in the response (#17347) 2022-09-29 18:22:33 +01:00
Peter Wilson e307450c63
fix: upgrade vault-plugin-secrets-kv to v0.13.1 (#17339)
* fix: upgrade vault-plugin-secrets-kv to v0.13.1

* Added changelog

* Update changelog/17339.txt

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-09-28 18:36:10 +01:00
Steven Clark 1f459a2df6
PKI: Fix managed key signatures when using specified signature_bits (#17328)
* PKI: Fix managed key signatures when using specified signature_bits

 - When calling sign-intermediate and other apis with signature_bits
   value overridden with a backing managed key we did not use that
   value as tests for the private key type were not working.

* Add cl
2022-09-28 09:08:23 -04:00
akshya96 542570c289
Return errInvalidCredentials when wrong credentials is provided for existent users (#17104)
* adding errInvalidCredentials

* fixing tests

* add changelog

* fixing fmt errors

* test if routeErr is seen externally and fixing error comment

* adding fmt changes

* adding comments
2022-09-27 16:49:14 -07:00
Tom Proctor 6bad04908a
Plugin versioning changelog entries (#17322) 2022-09-27 16:24:21 +01:00
Alexander Scheel 3eaa4b0d75
Write explicit -help output to stdout (#17308)
* Write explicit -help output to stdout

Per the consensus of most programs, and mirroring the GNU Coding
Standards for CLI design, when users request -help explicitly via the
CLI, this should be written to stdout to allow paging of output. stderr
is fine when an invalid usage triggers the help text however.

In our case, mitchellh/cli helpfully adds a HelpWriter that we
previously set to stderr explicitly. This writer is only called to print
user-requested help text; it is not called on error cases (e.g., bad
usage triggering additional help text to the user).

Thus it should safely be settable to stdout, enabling pagers without
additional redirects.

Notably, we do have to set ErrorWriter as it defaults to initializing to
the value of HelpWriter, which we want to set to stdout now, which is
less useful.

See also: https://www.gnu.org/prep/standards/html_node/_002d_002dhelp.html
Resolves: #17004

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-26 12:15:48 -04:00
Violet Hynes 5bcd0c31cb
VAULT-6938 Remove license from being cache exempt (#17265)
* VAULT-6938 Remove license from being cache exempt

* VAULT-6938 Add changelog

* VAULT-6938 Typo in changelog name

* Update changelog/17265.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-09-26 10:26:07 -04:00
Violet Hynes 2b8d8a3c6a
VAULT-8630 Fix goroutine leak from RLQ initialize (#17281)
* VAULT-8630 Fix goroutine leak from RLQ initialize

* VAULT-8630 Changelog

* VAULT-8630 additional nil check
2022-09-22 15:59:53 -04:00
Yoan Blanc fa8f7c793f
fix: PGP subkeys support (#16224)
* fix: PGP subkeys support

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2022-09-22 09:12:41 -04:00
Tom Proctor 4e51491f7a
Upgrade vault-plugin-auth-alicloud to v0.13.0 (#17251) 2022-09-21 21:05:18 +01:00
Kit Haines f2adbb3e47
Basics of Cert-Count Non-Locking Telemetry (#16676)
Basics of Cert-Count Telemetry, changelog,  "best attempt" slice to capture (and test for) duplicates, Move sorting of possibleDoubleCountedRevokedSerials to after compare of entries. Add values to counter when still initializing.
Set lists to nil after use, Fix atomic2 import, Delay reporting metrics until after deduplication has completed, 
The test works now, Move string slice to helper function; Add backendUUID to gauge name.
2022-09-20 10:32:20 -07:00
Nick Cabatoff 559754d580
Break grabLockOrStop into two pieces to facilitate investigating deadlocks (#17187)
Break grabLockOrStop into two pieces to facilitate investigating deadlocks.  Without this change, the "grab" goroutine looks the same regardless of who was calling grabLockOrStop, so there's no way to identify one of the deadlock parties.
2022-09-20 11:03:16 -04:00
Nick Cabatoff d8101f82ee
Handle when pluginCatalog.Get returns (nil,nil) during cred backend creation (#17204) 2022-09-20 08:57:08 -04:00