Commit graph

1954 commits

Author SHA1 Message Date
Mark Gritter 8c67bed7ae
Send a test message before committing a new audit device. (#10520)
* Send a test message before committing a new audit device.
Also, lower timeout on connection attempts in socket device.
* added changelog
* go mod vendor (picked up some unrelated changes.)
* Skip audit device check in integration test.
Co-authored-by: swayne275 <swayne@hashicorp.com>
2020-12-16 16:00:32 -06:00
Clint 4d81e3be4d
Improve consistency in error messages (#10537)
* Improve consistency in error messages

* add changelog entry
2020-12-11 15:21:53 -06:00
Hridoy Roy 2c4e299391
allow null types in batch encryption [VAULT-849] (#10386)
* allow null types in batch encryption

* dont allow plaintext to be null
2020-11-23 11:55:08 -08:00
Tom Proctor b9b3796cfe
Sanitize private_key from returned db plugin config (#10416) 2020-11-19 10:58:55 +00:00
Michael Golowka bfd57bc2e8
Error on root rotation when username is empty (#10344)
* Error on root rotation when username is empty

* Don't panic if the field doesn't exist
2020-11-09 15:12:09 +00:00
Brian Kassouf 10668331e4
Update go version to 1.15.3 (#10279)
* Update go version to 1.15.3

* Fix OU ordering for go1.15.x testing

* Fix CI version

* Update docker image

* Fix test

* packagespec upgrade -version 0.1.8

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>
2020-10-30 16:44:06 -04:00
Michael Golowka bd79fbafb3
Add couchbase, elasticsearch, and mongodbatlas back (#10222)
Updated the `Serve` function so these can be added back into Vault
2020-10-22 17:20:17 -06:00
Michael Golowka e6c8ee24ea
DBPW - Enables AutoMTLS for DB plugins (#10220)
This also temporarily disables couchbase, elasticsearch, and
mongodbatlas because the `Serve` function needs to change signatures
and those plugins are vendored in from external repos, causing problems
when building.
2020-10-22 15:43:19 -06:00
Mike Grass e1541a4569
Improve errors for aws login with an unbound ARN (#10036)
* Improve errors for aws login with an unbound ARN

* Factor hasWildcardBind into its own function

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-10-22 11:24:47 -07:00
Danielle b821d35acb
approle: Include role_name in alias metadata (#9529)
This change allows people who are using templated policies to use the
role_name in their templates through {{
identity.entity.aliases.approle.metadata.role_name }}.

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-10-16 14:01:57 -07:00
Calvin Leung Huang af2c3a851d
database/test: use vault.TestWaitActive when we're starting up a test cluster (#10158) 2020-10-16 09:35:55 -07:00
Michael Golowka 1888323243
DBPW - Copy newdbplugin package to dbplugin/v5 (#10151)
This is part 1 of 4 for renaming the `newdbplugin` package. This copies the existing package to the new location but keeps the current one in place so we can migrate the existing references over more easily.
2020-10-15 13:20:12 -06:00
Scott Miller b513af3851
Expose generic versions of KDF and symmetric crypto (#10076)
* Support salt in DeriveKey

* Revert "Support salt in DeriveKey"

This reverts commit b295ae42673308a2d66d66b53527c6f9aba92ac9.

* Refactor out key derivation, symmetric encryption, and symmetric decryption into generic functions

* comments

* comments

* go mod vendor

* bump both go.mods

* This one too

* bump

* bump

* bump

* Make the lesser used params of symmetric ops a struct

* go fmt

* Call GetKey instead of DeriveKey

* Address feedback

* Wrong rv

* Rename calls

* Assign the nonce field

* trivial change

* Check nonce len instead

* go mod vendor
2020-10-01 21:04:36 -05:00
Michael Golowka fc0ed96066
DBPW - Revert AutoMTLS (#10065) 2020-09-30 17:08:37 -06:00
Theron Voran 52581cd472
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
Billy Keyes 26e8627cfc
Use us-gov-west-1 for global APIs in aws-us-gov (#9947)
* Use us-gov-west-1 for global APIs in aws-us-gov

Certain partition-global AWS services, like IAM, seem to require
specific regions. In the regular 'aws' partition, this is us-east-1. In
the 'aws-us-gov' partition, this is us-gov-west-1. Providing
us-gov-east-1 returns an error from AWS:

  SignatureDoesNotMatch: Credential should be scoped to a valid region, not 'us-gov-east-1'.

This resolves a problem where AWS authentication could randomly fail
depending on the value cached by Vault at startup.
2020-09-25 17:13:26 -07:00
Michael Golowka 41d8c89169
[DBPW 5/X] Use AutoMTLS with DB plugins (#10008) 2020-09-23 16:08:03 -06:00
Lauren Voswinkel 201fc8fd4d
Add content-sha256 as a default allowed STS header (#10009)
Also, alphabetize those headers... just because.
2020-09-22 10:02:37 -07:00
Marco Rieger b634e1964d
fix missing plaintext in bulk decrypt response (#9991)
Decrypting an ciphertext where its corresponding value equals empty, the payload property "plaintext" is missing in the response object. This fixes the problem by adding a new, distinct struct for decrypt batch response items where "omitempty" is not set.
2020-09-22 09:43:07 -04:00
Michael Golowka 60e0cbbc37
[DBPW 4/X] Update DB engine to support v4 and v5 interfaces with password policies (#9878) 2020-09-18 15:10:54 -06:00
ncabatoff 27c7a77624
When expiration attempts to revoke a cert that's not in storage (perhaps due to pki tidy), don't treat that as an error. Let the lease get expired. (#9880) 2020-09-17 16:15:03 -04:00
Lauren Voswinkel 5740e1ff9e
5844 AWS Root Credential Rotation (#9921)
* strip redundant field type declarations

* root credential rotation for aws creds plugin

* Change location of mocks awsutil and update methods that no longer exist

* Update website/pages/docs/auth/aws.mdx

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk version to get the awsutil mock file

* Re-vendor modules to pass CI

* Use write lock for the entirety of AWS root cred rotation

* Update docs for AWS root cred rotation for clarity

Co-authored-by: Becca Petrin <beccapetrin@gmail.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-09-15 15:26:56 -07:00
Mark Gritter 243d407dac
Disable flaky test: discard_role_newer_rotation_date (#9957)
* Temporarily disable discard_role_newer_rotation_date
2020-09-15 13:10:59 -05:00
ncabatoff b615da43d7
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Mark Gritter 0e6da5c7ef
Lower the interval for rotation during tests, to make it more likely that our five second grace period is sufficient. (#9895)
* Lower the interval for rotation during tests, to make it more likely
that our five second grace period is sufficient.
* Rewrite to make the rotateCredentials ticker a configurable value.
* 'go mod vendor' for SDK changes.
2020-09-08 17:53:15 -05:00
Jim Kalafut 2c737182e4
Import vault-plugin-mock (#9839)
Support testing of CI and GitHub actions by creating a real dependency
between Vault and a plugin. The plugin itself is a no-op.
2020-08-26 12:51:46 -07:00
Calvin Leung Huang 3a5c7a6946
secrets/ssh: allow algorithm_signer to use the key's default algo (#9824)
* secrets/ssh: allow algorithm_signer to use the key's default algo

* add test for ed25519 key signing

* test: add role upgrade test case

* test: rename and add more test cases

* test: clean up tests cases, fix broken test case on expected error

* test: fix broken test case on expected error
2020-08-26 12:31:56 -07:00
Scott Miller 4c4fb54806
Aws auth fixes (#9825)
* Bring over PSIRT-37 changes from ENT

* Add additional allowed headers

* Already had this one

* Change to string slice comma separated parsing

* Add allowed_sts_header_values to read output

* Only validate AWS related request headers

* one per line

* Import ordering

* Update test

* Add X-Amz-Credential

* Reorder imports
2020-08-25 17:37:59 -05:00
Artem Alexandrov 301ea4c0f0
pki: Allow to use not only one variable during templating in allowed_domains #8509 (#9498) 2020-08-17 11:37:00 -07:00
James Hodgkinson 8173ce777e
fixing a spelling error (#9693)
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-08-09 06:17:02 -07:00
Calvin Leung Huang fbe2a86693
pki: use revocationInfo.RevocationTimeUTC when revoking certs with ti… (#9609)
* pki: use revocationInfo.RevocationTimeUTC when revoking certs with tidy_revoked_certs set to true

* update comment

* tidy: use same time snapshot for OR comparison
2020-07-30 15:10:26 -07:00
Clint 5e80919d77
Forward requests for rotate-root on Performance secondary / standbys (#9606) 2020-07-28 09:22:32 -05:00
ncabatoff 0247a7533e
Upgrade to newer okta lib for pagination, fetch all groups using it (#9580) 2020-07-24 09:05:08 -04:00
Sorin Dumitru bb92e8840a
plugin: fix process leak when Setup() fails (#9557)
We've noticed some leftover processes from vault plugins on our boxes. Some
of them were even left over from previous instances of the service and reparented
to init.  This could cause issues if too many of them accumulate.

When running with TRACE logging the logs showed that there was an error return by
the call to Setup() the plugin. Looking through the code it looks like we do not
call Cleanup() in that case.

Co-authored-by: Sorin Dumitru <sorindumitru@users.noreply.github.com>
2020-07-22 10:29:50 -04:00
ncabatoff 36528cf9ec
Don't override MaxWait, its default is good for CI (#9478) 2020-07-14 14:21:37 -04:00
Andrej van der Zee 8f305b1531
Add option allowed_domains_template enabling identity templating for issuing PKI certs. (#8509) 2020-07-08 12:52:25 -04:00
ncabatoff 4a5d8fc212
Remove two unreliable tests from TestTransit_SignVerify_P256. (#7499)
* Remove two tests from TestTransit_SignVerify_P256.  These verify that
you can't verify something signed using JWS marshaling when claiming
it's ASN1, and vice versa.  The problem is that we rely on unmarshaling
failing, and it doesn't always.  Both encodings use base64, one with
padding, one without, so depending on the data sometimes unmarshaling
will work when we expect it to fail.  It would be nice to preserve
these tests if they could be made reliable, but I didn't see an easy way,
and I don't think they add enough value to warrant greater effort.

* Restore the tests I removed, and improve the verify function to broaden the notion of errExpected.

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-06-29 12:22:22 -04:00
Clint cbecc40e48
Stepwise docker env (#9292)
* add first stepwise test env, Docker, with example transit test

* update transit stepwise test

* add other tests that use stepwise

* cleanup test, make names different than just 'transit'

* return the stderr if compile fails with error

* minor cleanups

* minor cleanups

* go mod vendor

* cleanups

* remove some extra code, and un-export some fields/methods

* update vendor

* remove reference to vault.CoreConfig, which really wasn't used anyway

* update with go mod vendor

* restore Precheck method to test cases

* clean up some networking things; create networks with UUID, clean up during teardown

* vendor stepwise

* Update sdk/testing/stepwise/environments/docker/environment.go

haha thanks :D

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update sdk/testing/stepwise/environments/docker/environment.go

Great catch, thanks

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* fix redundant name

* update error message in test

* Update builtin/credential/userpass/stepwise_test.go

More explicit error checking and responding

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update builtin/logical/aws/stepwise_test.go

`test` -> `testFunc`

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update builtin/logical/transit/stepwise_test.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* fix typos

* update error messages to provide clarity

* Update sdk/testing/stepwise/environments/docker/environment.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* update error handling / collection in Teardown

* panic if GenerateUUID returns an error

* Update sdk/testing/stepwise/environments/docker/environment.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update builtin/credential/userpass/stepwise_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update builtin/logical/aws/stepwise_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update builtin/logical/transit/stepwise_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk/testing/stepwise/environments/docker/environment.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* import ordering

* standardize on dc from rc for cluster

* lowercase name

* CreateAPIClient -> NewAPIClient

* testWait -> ensure

* go mod cleanup

* cleanups

* move fields and method around

* make start and dockerclusternode private; use better random serial number

* use better random for SerialNumber

* add a timeout to the context used for terminating the docker container

* Use a constant for the Docker client version

* rearrange import statements

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-06-26 17:52:31 -05:00
Jason O'Donnell 0e294a7174
plugins/ssh: add diabled host key verification warning (#9304) 2020-06-24 08:40:19 -04:00
Clint 6b4bdb1882
VLT091 plugin testing framework stepwise (#9270)
* Resolve merge conflicts and updates from running a test

* move testing/_test.go over to legacy

* updates

* Add core of plugin test framework Stepwise  (#9166)

* adding stepwise testing, but there are protocol buff error :/

* move file and update sdk/go.mo

* update/sync modules

* update from other branch

* update sdk/go.mod

* some cleanups after feedback

* remove enviornments from this PR

* update vendor

* change from running go mod tidy

* change from go mod tidy

* Update sdk/testing/stepwise/helpers.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update sdk/testing/stepwise/helpers.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* change panic to error

* Update sdk/testing/stepwise/helpers.go

return `nil` and not `err` at the end

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Defer close() on successful Open of a file

* document the re-creation of steps

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* remove unused BarrierKeys()

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* updates from feedback

* fix return with bad arguments

* Rename things:

- StepOperation -> Operation
- StepwiseEnvironment -> Environment
- StepCheckFunc -> AssertionFunc
- step.Check -> step.Assert

* document the environment interface methods

* rename EnvironmentOptions to MountOptions

* rename Name to RegistryName

* remove ExpectError because it's redundant

* minor doc update

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* add checkShouldRun function

* remove redundant return

* remove vestigial PreCheck function

* add tt.Helper() to makeRequest

* minor code formatting and document 1-based index for log output of Steps

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* minor updates

* update sdk

* use local reference for api, vault dep

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* cleanup some defer functions

* call fatal if environment setup fails, and don't call teardown

* defer re-setting client token in makeRequest

* Move legacy logicaltest back to testhelpers

* update mods and test files with go mod tidy

* go mod vendor

* remove relative replace directives

* restore old logical test location

* move declaration to main stepwise file

* remove index var and use i+1

* add testing for write, delete paths of makeRequest

* update stepwise core testing to do request counting

* remove unused methods

* Update sdk/testing/stepwise/stepwise.go

remove dead line

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update sdk/testing/stepwise/stepwise.go

fix capitalization in code comment

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* update code comments for SkipTeardown to clarify its use

* update stepwise

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
2020-06-23 06:01:39 -05:00
Jason O'Donnell 3c8ceb4d1d
Fix database creds rotation panic for nil resp (#9258) 2020-06-18 13:49:42 -04:00
Michael Golowka f77bcc53c4
Move sdk/helper/random -> helper/random (#9226)
* This package is new for 1.5 so this is not a breaking change.
* This is being moved because this code was originally intended to be used
within plugins, however the design of password policies has changed such
that this is no longer needed. Thus, this code doesn't need to be in the
public SDK.
2020-06-17 14:24:38 -06:00
Michael Golowka a89f09802d
Integrate password policies into RabbitMQ secret engine (#9143)
* Add password policies to RabbitMQ & update docs
* Also updates some parts of the password policies to aid/fix testing
2020-06-11 16:08:20 -06:00
ncabatoff 2ec9049ef7
Add ssh signing algorithm as a role option. (#9096) 2020-06-11 08:10:13 -04:00
Austin Gebauer 821940f905
fix: invalidate cached clients after a config change in the aws secrets backend (#9186) 2020-06-10 20:53:48 -07:00
Félix Mattrat 40699d2b9e
Improving transit batch encrypt and decrypt latencies (#8775)
Optimized batch items decoder bypassing mapstructure
2020-06-10 13:31:46 -04:00
Theron Voran e1a432a167
AWS: Add iam_groups parameter to role create/update (#8811)
Allows vault roles to be associated with IAM groups in the AWS
secrets engine, since IAM groups are a recommended way to manage
IAM user policies. IAM users generated against a vault role will
be added to the IAM Groups. For a credential type of
`assumed_role` or `federation_token`, the policies sent to the
corresponding AWS call (sts:AssumeRole or sts:GetFederation) will
be the policies from each group in `iam_groups` combined with the
`policy_document` and `policy_arns` parameters.

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-06-09 16:56:12 -07:00
Peter J. Li 27cf73afa8
fix error message for when an invalid uri_sans is provided via the api (#8772) 2020-06-08 13:43:56 -04:00
Austin Gebauer cc16c6d08e
fix: remove mount prefix from config path used to invalidate connections (#9129) 2020-06-03 12:04:55 -07:00
Alexander Bezobchuk eb0b3ac286
Merge PR #9100: Add key_version to Transit Logical Response 2020-06-01 13:16:01 -04:00