Commit graph

555 commits

Author SHA1 Message Date
Hamid Ghaf 9f12e0f892
Buffer body read up to MaxRequestSize (#24354) (#24367)
* Buffer body read up to MaxRequestSize (#24354)

* adding back a context
2023-12-04 17:45:23 -08:00
hc-github-team-secure-vault-core a73098c99e
backport of commit bcbd45b380d2cf776cb3cd920f03291301cee998 (#24198)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-11-20 16:15:28 +00:00
hc-github-team-secure-vault-core f3e2841fcd
backport of commit bc4be73a1c7998d6ee87bdba14981bc06c237d88 (#23679)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-10-16 21:09:36 +00:00
hc-github-team-secure-vault-core 28f2585da3
backport of commit 0fa36a36ae1b4842d96623eef0d20af5dea557c0 (#23443)
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
2023-10-02 09:49:05 -07:00
hc-github-team-secure-vault-core bdee24128a
backport of commit 7d4d8cb708de62167340fa84770f8237c7bfdd1e (#22900)
Co-authored-by: Scott Miller <smiller@hashicorp.com>
2023-09-08 14:48:26 +00:00
hc-github-team-secure-vault-core d2eb98e2cf
backport of commit f150a5259335117632d094bdf33ead0209172654 (#22756)
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-09-05 11:00:07 -04:00
hc-github-team-secure-vault-core a7f3af73ed
backport of commit 4e963c4c5bbd00d4150df1bc0d140bba43feb407 (#22704)
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2023-09-01 09:18:05 -04:00
hc-github-team-secure-vault-core d13671c155
backport of commit cccfdb088f218f5631195e8b653c07a77cfac2b5 (#22596)
Co-authored-by: Ellie <ellie.sterner@hashicorp.com>
2023-08-28 18:16:57 -05:00
hc-github-team-secure-vault-core 4a08ff8c48
backport of commit d50bd4eb05994781ba64974cc16fdf3409c82935 (#22486)
Co-authored-by: Josh Black <raskchanky@gmail.com>
2023-08-21 22:24:30 +00:00
hc-github-team-secure-vault-core e9f394c549
backport of commit 5374b3b046d13d3e7a9f6833efcaeff0e49fddc4 (#22366)
Co-authored-by: Hamid Ghaf <83242695+hghaf099@users.noreply.github.com>
2023-08-16 07:16:42 -07:00
hc-github-team-secure-vault-core 0ecf0f300e
backport of commit f8dd46acb830f4ef7baef759fb4d0e3752d03e9e (#22251) 2023-08-08 21:05:43 +00:00
hc-github-team-secure-vault-core 3fb1a15a4f
backport of commit c040f901e57d2d04772827b52f7b052757986897 (#22135)
Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2023-07-31 13:57:29 -04:00
hc-github-team-secure-vault-core 1d9e560f0f
backport of commit 2f677665b37fcced51737cdef7abbebb4c719529 (#21527)
Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
2023-07-06 20:01:33 +00:00
hc-github-team-secure-vault-core 7e8c0a1cae
backport of commit 4c1a7b53d362ee733707de2fa3280596e35d7f03 (#21609)
Co-authored-by: Bianca Moreira <48203644+biazmoreira@users.noreply.github.com>
2023-07-06 12:05:43 +02:00
hc-github-team-secure-vault-core 36365ed7f4
backport of commit 3a46ecc389e9096ccea6c6f847b68ada7f8068d7 (#21362)
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
2023-06-21 14:01:13 +00:00
hc-github-team-secure-vault-core a5e6a4614d
backport of commit 58f029d6082b5231ca49312c75b360300781d399 (#21240)
Co-authored-by: Victor Rodriguez <vrizo@hashicorp.com>
2023-06-14 17:59:08 -04:00
hc-github-team-secure-vault-core 1be0ebae8a
backport of commit 3b14cd2061b49c8c698205eef87ceb3d0e69983b (#21181)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-06-13 20:13:41 +00:00
hc-github-team-secure-vault-core a43e44812a
backport of commit 65157a6d3fc4412c01c26194468a38760e7e08ea (#21179)
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-06-13 18:52:09 +00:00
hc-github-team-secure-vault-core 6c26b96b31
backport of commit 0ff9059967aa3fc0f2be0fd46926f9a7f1de5573 (#21124)
Co-authored-by: Josh Black <raskchanky@gmail.com>
2023-06-09 22:04:11 +00:00
hc-github-team-secure-vault-core 96596f255b
backport of commit e65d295c8809f4cc61b63bbc03293586c71c5483 (#20906)
Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
2023-05-31 19:41:39 +00:00
hc-github-team-secure-vault-core 52835311b2
Backport of Fix transit import/export of hmac-only keys into release/1.14.x (#20902)
* backport of commit daf72aa42790144c3a0ca9c17bb19b1c5bce66c6

* Fix formatting

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-31 18:42:44 +00:00
hc-github-team-secure-vault-core a486b13957
backport of commit 63ccb60b9a6dadf717e6813f9789c7194a2375de (#20877)
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-30 23:49:24 +00:00
hc-github-team-secure-vault-core 3e6bc17036
backport of commit bd4d7269dd987395426275901afa06b17dc54220 (#20849)
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-30 21:30:35 +00:00
hc-github-team-secure-vault-core 127cef66a2
backport of commit 7490a2ebb7f5950321ec7930ee5f911c86febb04 (#20856)
Co-authored-by: Milena Zlaticanin <60530402+Zlaticanin@users.noreply.github.com>
2023-05-30 17:13:11 +00:00
Alexander Scheel 30488bc374
sdk/helper/nonce -> go-secure-stdlib/nonceutil (#20737)
Depends on https://github.com/hashicorp/go-secure-stdlib/pull/73

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-25 20:57:08 +00:00
Nick Cabatoff 46b0b33bda
Cluster test helper improvements (#20424) 2023-05-24 20:21:10 +00:00
Alexander Scheel 04bb7eef15
Update transit public keys for Ed25519 support (#20727)
* Refine documentation for public_key

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Support additional key types in importing version

This originally left off the custom support for Ed25519 and RSA-PSS
formatted keys that we've added manually.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for Ed25519 keys

Here, we prevent importing public-key only keys with derived Ed25519
keys. Notably, we still allow import of derived Ed25519 keys via private
key method, though this is a touch weird: this private key must have
been packaged in an Ed25519 format (and parseable through Go as such),
even though it is (strictly) an HKDF key and isn't ever used for Ed25519.

Outside of this, importing non-derived Ed25519 keys works as expected.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add public-key only export method to Transit

This allows the existing endpoints to retain private-key only, including
empty strings for versions which lack private keys. On the public-key
endpoint, all versions will have key material returned.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update tests for exporting via public-key interface

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add public-key export option to docs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-24 11:26:35 -04:00
Peter Wilson 8698650394
VAULT-11595: Augment forwarded requests with host:port info (from/to nodes) (Enterprise) (#20733)
* Allow audit entries to contain forwarded from host info
* adjust logical/request and audit format to use bool instead of string for 'to' host
2023-05-24 13:57:45 +01:00
Peter Wilson 21d6432ea6
Revert "Allow audit entries may contain forwarded to/from host info (#20689)" (#20732)
This reverts commit 732dda34e726859fbb24ea51a87f3a1772bf9b7a.
2023-05-24 09:44:57 +01:00
Alexander Scheel 83d32240c7
Add nonce service to sdk/helpers, use in PKI (#20688)
* Build a better nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add internal nonce service for testing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add benchmarks for nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add statistics around how long tidy took

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Replace ACME nonces with shared nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add an initialize method to nonce services

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use the new initialize helper on nonce service in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional tests for nonces

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Format sdk/helper/nonce

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use default 90s nonce expiry in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove parallel test case as covered by benchmark

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional commentary to encrypted nonce implementation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add nonce to test_packages

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-23 19:44:05 +00:00
Peter Wilson afb5610373
Allow audit entries may contain forwarded to/from host info (#20689) 2023-05-22 20:17:20 +01:00
John-Michael Faircloth f9541a1c96
pki: add subject key identifier to read key response (#20642)
* pki: add subject key identifier to read key response

This will be helpful for the Terraform Vault Provider to detect
migration of pre-1.11 exported keys (from CA generation) into post-1.11
Vault.

* add changelog

* Update builtin/logical/pki/path_fetch_keys.go

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* check for managed key first

* Validate the SKID matches on root CAs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Validate SKID matches on int CAs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix formatting of tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-18 16:49:22 +00:00
Hamid Ghaf 04d81e1c27
report intermediate error messages during request forwarding (#20643)
* report intermediate error messages during request forwarding

* CL
2023-05-18 05:07:54 -07:00
Luis (LT) Carbonell 95e6723aa9
Correct Default for MaximumPageSize (#20453)
* default max page size for config

* Add changelog

* update test int to *int

* add testing defaults

* update default to -1, i.e. dont paginate

* update test

* Add error message for invalid search

* Make 0 the default

* cleanup

* Add to known issues doc

* Update website/content/docs/upgrading/upgrade-to-1.13.x.mdx

* Update website/content/docs/upgrading/upgrade-to-1.11.x.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/upgrading/upgrade-to-1.13.x.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/upgrading/upgrade-to-1.12.x.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Add workaround to docs

* Update changelog/20453.txt

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

---------

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2023-05-17 20:56:53 +00:00
Alexander Scheel b204e51263
ACME tests for Intermediate CA issuance prevention (#20633)
* Do not set use_csr_values when issuing ACME certs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure CSRs with Basic Constraints are rejected

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test to ensure CA certificates cannot be issued

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update builtin/logical/pkiext/pkiext_binary/acme_test.go

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update builtin/logical/pkiext/pkiext_binary/acme_test.go

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update acme_test.go to include certutil

* Update acme_test.go - unused imports, reformat

* Update acme_test.go - hex really was used

This is why I can't use the GH web editor. :-)

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-05-17 19:54:37 +00:00
Alexander Scheel e58f3816a4
Start counting ACME certificate issuance as client activity (#20520)
* Add stub ACME billing interfaces

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add initial implementation of client count

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly attribute to mount, namespace

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor adding entities of custom types

This begins to add custom types of events; presently these are counted
as non-entity tokens, but prefixed with a custom ClientID prefix.

In the future, this will be the basis for counting these events
separately (into separate buckets and separate storage segments).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor creation of ACME mounts

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case for billing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Better support managed key system view casting

Without an additional parameter, SystemView could be of a different
internal implementation type that cannot be directly casted to in OSS.
Use a separate parameter for the managed key system view to use instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor creation of mounts for enterprise

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Validate mounts in ACME billing tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use a hopefully unique separator for encoded identifiers

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use mount accesor, not path

Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rename AddEventToFragment->AddActivityToFragment

Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
2023-05-17 16:12:04 +00:00
Violet Hynes b2468d3481
VAULT-15547 First pass at agent/proxy decoupling (#20548)
* VAULT-15547 First pass at agent/proxy decoupling

* VAULT-15547 Fix some imports

* VAULT-15547 cases instead of string.Title

* VAULT-15547 changelog

* VAULT-15547 Fix some imports

* VAULT-15547 some more dependency updates

* VAULT-15547 More dependency paths

* VAULT-15547 godocs for tests

* VAULT-15547 godocs for tests

* VAULT-15547 test package updates

* VAULT-15547 test packages

* VAULT-15547 add proxy to test packages

* VAULT-15547 gitignore

* VAULT-15547 address comments

* VAULT-15547 Some typos and small fixes
2023-05-17 09:38:34 -04:00
Gabriel Santos 05f3236c15
Provide public key encryption via transit engine (#17934)
* import rsa and ecdsa public keys

* allow import_version to update public keys - wip

* allow import_version to update public keys

* move check key fields into func

* put private/public keys in same switch cases

* fix method in UpdateKeyVersion

* move asymmetrics keys switch to its own method - WIP

* test import public and update it with private counterpart

* test import public keys

* use public_key to encrypt if RSAKey is not present and failed to decrypt
if key version does not have a private key

* move key to KeyEntry parsing from Policy to KeyEntry method

* move extracting of key from input fields into helper function

* change back policy Import signature to keep backwards compatibility and
add new method to import private or public keys

* test import with imported public rsa and ecdsa keys

* descriptions and error messages

* error messages, remove comments and unused code

* changelog

* documentation - wip

* suggested changes - error messages/typos and unwrap public key passed

* fix unwrap key error

* fail if both key fields have been set

* fix in extractKeyFromFields, passing a PolicyRequest wouldn't not work

* checks for read, sign and verify endpoints so they don't return errors when a private key was not imported and tests

* handle panic on "export key" endpoint if imported key is public

* fmt

* remove 'isPrivateKey' argument from 'UpdateKeyVersion' and
'parseFromKey' methods

also: rename 'UpdateKeyVersion' method to 'ImportPrivateKeyForVersion' and 'IsPublicKeyImported' to 'IsPrivateKeyMissing'

* delete 'RSAPublicKey' when private key is imported

* path_export: return public_key for ecdsa and rsa when there's no private key imported

* allow signed data validation with pss algorithm

* remove NOTE comment

* fix typo in EC public key export where empty derBytes was being used

* export rsa public key in pkcs8 format instead of pkcs1 and improve test

* change logic on how check for is private key missing is calculated

---------

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-11 11:56:46 +00:00
Pratyoy Mukhopadhyay 7b807a9bb0
add ns path to granting_policies (#20522) 2023-05-04 15:08:22 -07:00
Hamid Ghaf 148263084d
adding support for four cluster docker based test scenario (#20492) 2023-05-03 10:49:45 -07:00
Alexander Scheel d8c5456f8a
Add dns resolver to PKI Binary Cluster (#20485)
* Export DockerAPI for use by other consumers

As usage of DockerCluster gets more advanced, some users may want to
interact with the container nodes of the cluster. While, if you already
have a DockerAPI instance lying around you can reuse that safely, for
use cases where an existing e.g., docker/testhelpers's runner instance
is not available, reusing the existing cluster's DockerAPI is easiest.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to exec commands without runner

When modifying DockerTestCluster's containers manually, we might not
have a Runner instance; instead, expose the ability to run commands via
a DockerAPI instance directly, as they're awfully convenient.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add DNS resolver into ACME tests

This updates the pkiext_binary tests to use an adjacent DNS resolver,
allowing these tests to eventually be extended to solve DNS challenges,
as modifying the /etc/hosts file does not allow this.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix loading DNS resolver onto network

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix bug with DNS configuration validation

Both conditionals here were inverted: address being empty means a bad
specification was given, and the parse being nil means that it was not a
valid IP address.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix specifying TXT records, allow removing records

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-03 17:32:39 +00:00
Nick Cabatoff e0093a2791
Simple perf replication test using docker containers (#20393) 2023-04-28 19:43:30 +00:00
Nick Cabatoff f19f537a4f
Add support for restarting docker nodes, and for providing args (#20409) 2023-04-28 14:03:14 -04:00
Alexander Scheel e42fd09b47
Add custom DNS resolver to ACME configuration (#20400)
* Handle caching of ACME config

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add DNS resolvers to ACME configuration

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add custom DNS resolver to challenge verification

This required plumbing through the config, reloading it when necessary,
and creating a custom net.Resolver instance.

Not immediately clear is how we'd go about building a custom DNS
validation mechanism that supported multiple resolvers. Likely we'd need
to rely on meikg/dns and handle the resolution separately for each
container and use a custom Dialer that assumes the address is already
pre-resolved.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Improvements to Docker harness

 - Expose additional service information, allowing callers to figure out
   both the local address and the network-specific address of the
   service container, and
 - Allow modifying permissions on uploaded container files.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add infrastructure to run Bind9 in a container for tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Validate DNS-01 challenge works

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-27 15:30:29 -04:00
Nick Cabatoff f5b5662122
Don't require every test to specify a VaultLicense field (#20372) 2023-04-26 18:39:48 +00:00
Nick Cabatoff ad18fc6398
Docker testing: handle licensing, different images per node (#20347) 2023-04-25 17:11:46 -04:00
Nick Cabatoff 22b00eba12
Add support for docker testclusters (#20247) 2023-04-24 14:25:50 -04:00
Nick Cabatoff 313957b911
Add tests based on vault binary (#20224)
First steps towards docker-based tests: tests using vault binary in -dev or -dev-three-node modes.
2023-04-24 09:57:37 -04:00
Nick Cabatoff 75ad9a1de8
Bump the sdk go version to 1.20, document when go versions should be updated (#20278) 2023-04-24 08:44:39 -04:00
Luis (LT) Carbonell d308c31cbf
Add Configurable LDAP Max Page Size (#19032)
* Add config flag for LDAP max page size

* Add changelog

* move changelog to correct file

* cleanup

* Default to non-paged searching for with -1

* Update website/content/api-docs/auth/ldap.mdx

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update tests

---------

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2023-04-20 20:39:27 +00:00