Commit Graph

354 Commits

Author SHA1 Message Date
hc-github-team-secure-vault-core c16d572ab8
backport of commit 3b5ca69b62a3c59468754278f579610c0902fa05 (#20839)
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-05-30 16:41:07 +00:00
Marc Boudreau 8928b30224
Refactor Code Focused on DevTLS Mode into New Function (#20376)
* refactor code focused on DevTLS mode into new function

* add tests for configureDevTLS function

* replace testcase comments with fields in testcase struct
2023-05-19 15:45:22 -04:00
miagilepner 7aa1bce6fb
VAULT-15703: Reload automated reporting (#20680)
* support config reloading for census

* changelog

* second changelog entry for license updates

* correct changelog PR
2023-05-19 14:42:50 +00:00
Nick Cabatoff 1a8d3e8948
Make -dev-three-node use perf standbys for ent binaries (#20629) 2023-05-17 18:37:44 +00:00
Jason O'Donnell 202d674682
command/server: add support to write pprof files to the filesystem via SIGUSR2 (#20609)
* core/server: add support to write pprof files to the filesystem via SIGUSR2

* changelog

* Fix filepath join

* Use core logger

* Simplify logic

* Break on error
2023-05-17 09:21:25 -04:00
Victor Rodriguez 2656c020ae
Convert seal.Access struct into a interface (OSS) (#20510)
* Move seal barrier type field from Access to autoSeal struct.

Remove method Access.SetType(), which was only being used by a single test, and
which can use the name option of NewTestSeal() to specify the type.

* Change method signatures of Access to match those of Wrapper.

* Turn seal.Access struct into an interface.

* Tweak Access implementation.

Change `access` struct to have a field of type wrapping.Wrapper, rather than
extending it.

* Add method Seal.GetShamirWrapper().

Add method Seal.GetShamirWrapper() for use by code that need to perform
Shamir-specific operations.
2023-05-04 14:22:30 -04:00
Peter Wilson a592e3a023
Fix panic when Vault enters recovery mode, added test (#20418)
* Fix panic when Vault enters recovery mode, added test

* Added changelog
2023-04-28 12:41:19 +00:00
Nick Cabatoff 313957b911
Add tests based on vault binary (#20224)
First steps towards docker-based tests: tests using vault binary in -dev or -dev-three-node modes.
2023-04-24 09:57:37 -04:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Peter Wilson 15302d9fe2
Restore 'server' and 'agent' base loggers to use their original names (#19304) 2023-02-23 14:56:21 +00:00
miagilepner e873932bce
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
Scott Miller 20551261bd
Revert #18683 (#18942)
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"

This reverts commit 990d3bacc203c229d0f6729929d7562e678a1ac2.

* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"

This reverts commit 2ffe49aab0fc1a527c5182637c8fa3ac39b08d45.
2023-02-01 13:34:53 -06:00
Scott Miller 25960fd034
Add the ability to unseal using recovery keys via an explicit seal option. (#18683)
* wip

* wip

* Got it 'working', but not happy about cleanliness yet

* Switch to a dedicated defaultSeal with recovery keys

This is simpler than trying to hijack SealAccess as before.  Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal.  Then everything proceeds
as if you had a shamir seal to begin with.

* Handle recovery rekeying

* changelog

* Revert go.mod redirect

* revert multi-blob info

* Dumb nil unmarshal target

* More comments

* Update vault/seal.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* pr feedback

* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split

* Better comment on recovery seal during adjustSealMigration

* Make it possible to migrate from an auto-seal in recovery mode to shamir

* Fix sealMigrated to account for a recovery seal

* comments

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Address PR feedback

* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate

* Don't shortcut the reast of seal migration

* get rid of redundant transit server cleanup

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-24 14:57:56 -06:00
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Peter Wilson 5f5cad736a
VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
divyaac f8ad8bc5a5
OSS PR for Config Changes PR (#18418)
* OSS PR for Config Changes PR

* Edited tests

* typo

* Added changelog

* Remove changelog
2022-12-15 12:19:19 -08:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Chris Capurso a545b1f22d
Add HCP connectivity info to `vault server` startup logs (#18315)
* add HCP org, proj, and resource ID to server startup logs

* add server config test for cloud stanza

* add changelog entry
2022-12-12 14:01:02 -05:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Ellie 695fe367c9
Log environment variable keys at startup (#18125)
* Log environment variable keys at startup

* run make fmt

* change name

* add changelog

* fix changelog nubmer

* fix title

* add test

* fix message

* Update changelog/18125.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* add trace test

* remove check for >= debug, trace

* Update changelog/18125.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2022-12-02 08:49:24 -06:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Peter Wilson 33e6a3a87c
VAULT-9900: Log rotation for 'agent' and 'server' commands (#18031)
* Work to unify log-file for agent/server and add rotation
* Updates to rotation code, tried to centralise the log config setup
* logging + tests
* Move LogFile to ShareConfig in test
* Docs
2022-11-29 14:07:04 +00:00
Nick Cabatoff 87f0939d11
I think there was a plan at some point to support a "vault server -diagnose" command line option, but that doesn't seem to have ever come to pass. Removing the dead code. (#17983) 2022-11-17 09:25:27 +00:00
Jason O'Donnell de70878e16
core/server: Add environment variable to write stacktrace to file (#17929)
* Add env to write stacktrace to file

* changelog

* Use os.MkdirTemp

* Properly close file

* Adding path override

* Use temp file
2022-11-15 12:03:17 -05:00
Chris Capurso b31dc45ced
add noop ent supported storage check (#17883) 2022-11-10 16:46:59 -05:00
Peter Wilson 469c102fd3
Removed the legacy env var: LOGXI_FORMAT (#17822)
* Removed the legacy env var: LOGXI_FORMAT

* Added changelog

* Actually filled in the CL

* Added the name of legacy env var
2022-11-04 22:16:18 +00:00
Renato Costa ea1257be2e
fix incorrect use of loop variable (#16872)
This fixes a couple of references to loop variables in parallel tests
and deferred functions. When running a parallel test (calling
`t.Parallel()`) combined with the table-driven pattern, it's necessary
to copy the test case loop variable, otherwise only the last test case
is exercised. This is documented in the `testing` package:

https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks

`defer` statements that invoke a closure should also not reference a
loop variable directly as the referenced value will change in each
iteration of the loop.

Issues were automatically found with the `loopvarcapture` linter.
2022-10-04 09:23:37 -04:00
Mike Palmiotto 0078822d1d
core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool (#17319)
* core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool

* docs: Update VAULT_ALLOW_PENDING_REMOVAL_MOUNTS doc
2022-09-29 14:35:52 -04:00
Hamid Ghaf a034ebfd27
HCP link integration (#16939)
* HCP link integration

* update configure-git.yml

* more OSS stuff

* removing internal repos

* adding a nil check

* removing config test to be included in ENT only

* updating hcp-sdk-go to v0.22.0

* remove Hostname and AuthURL link config params

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-09-06 14:11:04 -04:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
Jason O'Donnell 140406143e
command/server: add dev-tls flag (#16421)
* command/server: add dev-tls flag

* Add website documentation

* changelog

* Lower file permissions

* Update cert gen per review

* Add dev-tls-cert-dir flag and cert clean up

* fmt

* Update cert generation per review

* Remove unused function

* Add better error messages

* Log errors in cleanup, fix directory not existing bug

* Remove hidden flag from -dev-tls-cert-dir

* Add usage

* Update 16421.txt

* Update variable names for files

* Remove directory on cleanup
2022-07-22 14:04:03 -04:00
Chris Capurso 9501d44ed5
Add endpoints to provide ability to modify logging verbosity (#16111)
* add func to set level for specific logger

* add endpoints to modify log level

* initialize base logger with IndependentLevels

* test to ensure other loggers remain unchanged

* add DELETE loggers endpoints to revert back to config

* add API docs page

* add changelog entry

* remove extraneous line

* add log level field to Core struct

* add godoc for getLogLevel

* add some loggers to c.allLoggers
2022-06-27 11:39:53 -04:00
Chris Capurso 75aa55eb57
add OSS noop check for valid ent storage (#15894) 2022-06-08 17:15:28 -04:00
Alexander Scheel 8d8a95cbf6
Add missing nil check to FIPS EA verification (#15883)
This was causing failures when running `vault server -dev`:

> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x105c41c1c]
>
> goroutine 1 [running]:
> github.com/hashicorp/vault/command.(*ServerCommand).parseConfig(0x140005a2180)
> 	.../vault/command/server.go:429 +0x5c

Interestingly, we do not have a test case for running the dev
sever.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 15:45:46 -04:00
Alexander Scheel dd6c339440
Add warning about EA in FIPS mode (#15858)
* Add warning about EA in FIPS mode

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 08:57:48 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Alexander Tischenko b89fc5af96
When running under systemd, send ready when server completed reloading config #7028 (#15041) 2022-05-03 08:34:11 -04:00
akshya96 796003ddda
Vault 3992 ToB Config and Plugins Permissions (#14817)
* updating changes from ent PR

* adding changelog

* fixing err

* fixing semgrep error
2022-04-04 09:45:41 -07:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00
hghaf099 8f6a97b81b
Vault-4279 reporting redundant/unused keys in config (#14752)
* Vault-4279 reporting redundant/unused keys in config

* missing validate step

* CL

* Change the log level
2022-04-01 10:34:27 -04:00
Victor Rodriguez f7f6f937e4
Invalidate the ManagedKeyRegistry cache when Vault config is updated. (#14179)
Add the hook needed to implement this functionality in vault-enterprise.
2022-02-21 09:55:44 -05:00
Hridoy Roy ebf8e5308a
SSCT Tokens Feature [OSS] (#14109)
* port SSCT OSS

* port header hmac key to ent and generate token proto without make command

* remove extra nil check in request handling

* add changelog

* add comment to router.go

* change test var to use length constants

* remove local index is 0 check and extra defer which can be removed after use of ExternalID
2022-02-17 11:43:07 -08:00
Alexander Scheel ec4deb12ef
Add infrastructure for exposing FIPS status (#14127)
In future Vault Enterprise versions, we'll be building Vault with
FIPS-validated cryptography. To help operators understand their
environment, we'll want to expose information about their FIPS status
when they're running a FIPS version of Vault.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-17 13:03:57 -05:00
Nick Cabatoff 21be98ee7a
Support go-sockaddr templates in top-level cluster_addr config (#13678)
In doing some testing I found that the listener clusteraddr isn't really used, or at least isn't as important as the top-level clusteraddr setting.  As such, go-sockaddr templating needs to be implemented for the top-level `cluster_addr` setting or it's unusable for HA.

Also fix a nil pointer panic I discovered at the same time.
2022-01-19 10:56:04 -05:00
hghaf099 65845c7531
VAULT-1564 report in-flight requests (#13024)
* VAULT-1564 report in-flight requests

* adding a changelog

* Changing some variable names and fixing comments

* minor style change

* adding unauthenticated support for in-flight-req

* adding documentation for the listener.profiling stanza

* adding an atomic counter for the inflight requests
addressing comments

* addressing comments

* logging completed requests

* fixing a test

* providing log_requests_info as a config option to determine at which level requests should be logged

* removing a member and a method from the StatusHeaderResponseWriter struct

* adding api docks

* revert changes in NewHTTPResponseWriter

* Fix logging invalid log_requests_info value

* Addressing comments

* Fixing a test

* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core

* fixing go.sum

* minor refactoring

* protecting InFlightRequests from data race

* another try on fixing a data race

* another try to fix a data race

* addressing comments

* fixing couple of tests

* changing log_requests_info to log_requests_level

* minor style change

* fixing a test

* removing the lock in InFlightRequests

* use single-argument form for interface assertion

* adding doc for the new configuration paramter

* adding the new doc to the nav data file

* minor fix
2021-12-08 17:34:42 -05:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
Mayo 0bd0339c0b
cleanup unused code and fix t.Fatal usage in goroutine in testing (#11694) 2021-09-30 07:33:14 -04:00
Rachel Culpepper c0ada31a99
Vault-2840 Add no-op check for HTTP listener validity (#12421)
* fips tag and no-op code

* tls check

* change to error

* add tests

* add test file

* cleanup

* add changelog

* fix file difference

* remove changelog
2021-09-07 18:36:58 -04:00