Commit graph

744 commits

Author SHA1 Message Date
Anton Averchenkov 5a57db3f32
openapi: Remove pcf plugin (#20067) 2023-04-11 13:54:12 -04:00
Alexander Scheel 3e36a58cf7
Support namespaces in test helpers (#20048)
Sometimes the tests will modify the client to set a namespace; this
results in testhelpers sometimes trying to access sys/ endpoints with a
namespace, which usually don't work well.

Detect an unset namespaces, if present, before executing, and restore
afterwards so as not to affect the tests.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-11 14:09:16 +00:00
Peter Wilson 4fc4516b49
Moved 'WaitForNodesExcludingSelectedStandbys' to shared testhelpers file (#19976) 2023-04-04 15:32:01 +01:00
Violet Hynes a2f457e10c
VAULT-12940 Vault Agent uses Vault Agent specific User-Agent header when issuing requests (#19776)
* VAULT-12940 test for templating user agent

* VAULT-12940 User agent work so far

* VAULT-12940 Vault Agent uses Vault Agent specific User-Agent header when issuing requests

* VAULT-12940 Clean-up and godocs

* VAULT-12940 changelog

* VAULT-12940 Fix test checking headers

* VAULT-12940 Fix test checking headers

* VAULT-12940 Fix test checking headers

* VAULT-12940 Fix test checking headers

* VAULT-12940 copy/paste typos

* VAULT-12940 improve comments, use make(http.Header)

* VAULT-12940 small typos and clean-up
2023-04-03 14:14:47 -04:00
Peter Wilson a2bdf7250b
VAULT-14048: raft-autopilot appears to refuse to remove a node which has left and wouldn't impact stability (#19472)
* ensure we supply the node type when it's for a voter
* bumped autopilot version back to v0.2.0 and ran go mod tidy
* changed condition in knownservers and added some comments
* Export GetRaftBackend
* Updated tests for autopilot (related to dead server cleanup)
* Export Raft NewDelegate

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-04-03 11:58:57 -04:00
John-Michael Faircloth 72f5ed8fe1
fix race condition in string generator helper (#19875)
* fix race condition in string generator helper

* add changelog
2023-03-31 15:19:45 +00:00
Anton Averchenkov 1b8dd129ab
Remove 'oidc' from gen_openapi.sh (#19839) 2023-03-30 13:38:56 -04:00
vinay-gopalan f2a4b23b7f
Update pseudo-version for Secrets Terraform plugin (#19798) 2023-03-29 09:01:35 -07:00
Anton Averchenkov fe53ebd19c
openapi: Remove 'ad' from gen_openapi.sh 2023-03-27 19:50:36 +00:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Anton Averchenkov f19bcd79c5
Remove 'openldap' from gen_openapi.sh (#19401) 2023-02-28 13:24:11 -05:00
Austin Gebauer d8348490d5
secrets/ad: change deprecation status to deprecated (#19334)
* secrets/ad: change deprecation status to deprecated

* adds changelog
2023-02-24 00:13:32 +00:00
miagilepner 271e5b14d2
VAULT-12299 Use file.Stat when checking file permissions (#19311)
* use file.Stat for config files

* cleanup and add path

* include directory path

* revert changes to LoadConfigDir

* remove path, add additional test:

* add changelog
2023-02-23 18:05:00 +01:00
Anton Averchenkov e5770359b5
Simplify gen_openapi.sh script (#19245)
* Simplify gen_openapi.sh script

* Update scripts/gen_openapi.sh

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use correct import

---------

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-02-17 14:48:05 -05:00
John-Michael Faircloth eca810d06e
test/plugin: test external database plugin workflows (#19191)
* test/plugin: test external db plugin

* use test helper to get cluster and plugins

* create test helper to create a vault admin user

* add step to revoke lease

* make tests parallel and add reload test

* use more descriptive name for test group; check response
2023-02-16 15:52:24 -06:00
Christopher Swenson 98513eb784
Update namespace.FromContext comment (#18840)
It looks like namespace context caching was removed in
https://github.com/hashicorp/vault/pull/5200
but this comment was left referencing it, which I found confusing
at first glance.
2023-02-13 11:04:32 -08:00
Nick Cabatoff 53afd2627b
Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
John-Michael Faircloth 14e4d67026
test/plugin: refactor compilePlugin for reuse (#18952)
* test/plugin: refactor compilePlugin for reuse

- move compilePlugin to helper package
- make NewTestCluster use compilePlugin

* do not overwrite plugin directory in CoreConfig if set

* fix getting plugin directory path for go build
2023-02-03 16:27:11 -06:00
Nick Cabatoff c2b222a11a
Vault test cluster helper refactorings, mostly audit related (#18928)
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package.  Consolidate on a single "noop audit" implementation.
2023-02-01 08:33:16 -05:00
Nick Cabatoff db41c5319d
TestClusterCore's TLSConfig becomes a method and does a Clone. (#18914) 2023-01-31 11:05:16 -05:00
Hamid Ghaf edbf093290
Allow Token Create Requests To Be Replicated (#18689)
* Allow Token Create Requests To Be Replicated

* adding a test

* revert a test
2023-01-24 14:00:27 -05:00
Hamid Ghaf 65a41d4f08
named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Tom Proctor fc378c0908
Event system alpha experiment (#18795) 2023-01-23 19:26:49 +00:00
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Peter Wilson 5f5cad736a
VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Mike Palmiotto 43a78c85f4
Mark deprecated builtins Removed (#18039)
* Remove logical database builtins

* Drop removed builtins from registry keys

* Update plugin prediction test

* Remove app-id builtin

* Add changelog
2023-01-09 09:16:35 -05:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Peter Wilson f87b7f1737
Only attempt rotation if files already exist when a Write is requested. (#18262) 2022-12-07 15:47:43 +00:00
Peter Wilson 21a8bcaa7b
Updated go-hclog to v1.4.0 to allow access to GetLevel. Refactored TranslateLoggerLevel (#18260) 2022-12-07 14:25:54 +00:00
Peter Wilson 94a349c406
Ensure base logging uses IndependentLevels (#18249) 2022-12-06 22:10:44 +00:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Peter Wilson 33e6a3a87c
VAULT-9900: Log rotation for 'agent' and 'server' commands (#18031)
* Work to unify log-file for agent/server and add rotation
* Updates to rotation code, tried to centralise the log config setup
* logging + tests
* Move LogFile to ShareConfig in test
* Docs
2022-11-29 14:07:04 +00:00
Chris Capurso 2843cfcdc1
VAULT-9427: Add read support to sys/loggers endpoints (#17979)
* add logger->log-level str func

* ensure SetLogLevelByName accounts for duplicates

* add read handlers for sys/loggers endpoints

* add changelog entry

* update docs

* ignore base logger

* fix docs formatting issue

* add ReadOperation support to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_LoggersByName

* check for empty name in delete handler
2022-11-28 11:18:36 -05:00
Alexander Scheel a8faa543e6
Add pki nginx/wget/curl/Go integration tests (#17320)
* Rename integation_test.go->integration_test.go

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to fetch container's network addresses

This lets us return the on-network container address, allowing us to
spawn client containers which contact server containers.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add integration tests with nginx, curl, wget, Go

We build new integration tests, spawning a test instance on nginx and
ensuring we can connect with a variety of clients against a variety of
CA and leaf certificate types. This will ultimately let us detect issues
with compatibility as we expand the matrix of supported servers and
clients.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make runner reference unique

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Attempt to fix CI with longer wait

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Finish moving nginx tests to pkiext package

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* make fmt

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add more debugging, work on CircleCI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-23 15:00:18 -05:00
Tom Proctor 853643d02b
Remove pinned builtin plugin versions from storage (#18051)
* Removes _builtin_ versions from mount storage where it already exists
* Stops new builtin versions being put into storage on mount creation/tuning
* Stops the plugin catalog from returning a builtin plugin that has been overridden, so it more accurately reflects the plugins that are available to actually run
2022-11-23 18:36:25 +00:00
Michael Anthony 0624d8f36e
[QT-309] Ensure environment variables are populated before proceeding (#17915)
* Ensure environment variables are populated before proceeding

* DRY up credNames var
2022-11-17 11:55:17 -07:00
Peter Wilson 0fad0c3864
VAULT-8732: Add log-file to Vault Agent (#17841)
* Started work on adding log-file support to Agent
* Allow log file to be picked up and appended
* Use NewLogFile everywhere
* Tried to pull out the config aggregation from Agent.Run

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-11-11 10:59:16 +00:00
Mike Palmiotto 404422dba1
Bump up period and skew to prevent timeouts (#17804)
Give the default SetupLoginMFATOTP helper a more robust period/skew. 403 failures on test-go-race are likely due to TOTP code timeouts being too aggressive.
2022-11-03 11:57:25 -04:00
Alexander Scheel 8e6e53cf63
Use hashicorp mirror for container pulls (#17778)
When running the test suite in CI (where requests are centralized from
relatively few IPs), we'd occasionally hit Dockerhub's rate limits.
Luckily Hashicorp runs a (limited) public mirror of the containers we
need, so we can switch to them here in the tests.

For consistency between developer and CI, we've opted to have the tests
always pull from the Hashicorp mirror, rather than updating the CI
runner to prefer the mirror.

We exclude nomad and influxdb as we don't presently mirror these repos.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-02 13:33:17 -04:00
Milena Zlaticanin a7e6ec47c8
Fix tests - Update MongoDB driver (#17662)
* Fix tests - Update MongoDB driver

* increase timeout and disconnect client after ping

* increase timeout

* disconnect client after the ping
2022-10-27 15:48:33 -05:00
Mike Palmiotto a9dcc45f72
Tweak totp test to fix race failures (#17692) 2022-10-27 09:41:40 -04:00
akshya96 1e189016e2
update protoc version to 3.21.7 oss (#17499)
* update protoc to 3.21.7

* adding changelog
2022-10-26 16:49:44 -07:00
Mike Palmiotto cc96c6f470
Store login MFA secret with tokenhelper (#17040)
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
2022-10-26 17:02:26 -04:00
Violet Hynes 296297e53f
Fix format strings missing an argument (#17581) 2022-10-18 10:57:20 -04:00
Alexander Scheel daf29de742
Add pki zlint tests (#17305)
* Add tests for zlint-clean CA building

This test ensures that we can consistently pass ZLint's CA linting
tests on a root certificate generated by Vault. In particular, nominal
requirements are placed on the structure on the issuer's Subject, which
we supply, and the remaining requirements pass.

The one exception is we include both RFC and CA/BF BR lints in the
default zlint checks; this means ECDSA P-521 (which isn't accepted by
Mozilla's root store policies) is rejected, so we ignore to lints
related to that.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add options to copy to/from container, fix stopping

Stopping the container takes a bit of time for some unknown reason so
I've instead opted to shorten the sleep in the zlint tests to avoid
consuming resources too long after the test finish.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make zlint tests execute in parallel

This improves the overall test time of the zlint tests, making the
container build up front once (provisioning zlint), and then copying the
cert into the new container image later.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* make fmt

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-12 13:29:06 -04:00
Alexander Scheel 93a11a8678
Better docker support: image building + better command execution (#17231)
* Refactor Docker command execution

This refactor will allow others to interact with containers more easily,
providing two interfaces (RunCmdWithOutput and RunCmdInBackground) for
executing commands in running containers if they don't wish to do so
manually.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow building containerfiles in tests

By adding image building capabilities to testhelpers (and coupled with
the better command execution support), we can begin to build better,
more reliable integration tests on top of public base images without
needing to maintain separate forks of these images out-of-tree for any
shortcomings they might have.

In particular, rather than doing the rather messy echo hack for writing
clients.conf, it is far better to provision this via a slim
Containerfile overlay on top of the stock jumanjiman/radiusd:latest
image.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly parse stdout/stderr in RunCmdWithOutput

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* ctx -> bCtx for BuildContext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update errors to use %w instead of %v

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-12 10:29:39 -04:00
Robert 254608f579
Add test for multihost connection strings with Postgres (#16912)
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2022-09-22 14:00:56 -05:00
Yoan Blanc fa8f7c793f
fix: PGP subkeys support (#16224)
* fix: PGP subkeys support

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2022-09-22 09:12:41 -04:00