Commit graph

341 commits

Author SHA1 Message Date
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Peter Wilson 5f5cad736a
VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
divyaac f8ad8bc5a5
OSS PR for Config Changes PR (#18418)
* OSS PR for Config Changes PR

* Edited tests

* typo

* Added changelog

* Remove changelog
2022-12-15 12:19:19 -08:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Chris Capurso a545b1f22d
Add HCP connectivity info to vault server startup logs (#18315)
* add HCP org, proj, and resource ID to server startup logs

* add server config test for cloud stanza

* add changelog entry
2022-12-12 14:01:02 -05:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Ellie 695fe367c9
Log environment variable keys at startup (#18125)
* Log environment variable keys at startup

* run make fmt

* change name

* add changelog

* fix changelog nubmer

* fix title

* add test

* fix message

* Update changelog/18125.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* add trace test

* remove check for >= debug, trace

* Update changelog/18125.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2022-12-02 08:49:24 -06:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Peter Wilson 33e6a3a87c
VAULT-9900: Log rotation for 'agent' and 'server' commands (#18031)
* Work to unify log-file for agent/server and add rotation
* Updates to rotation code, tried to centralise the log config setup
* logging + tests
* Move LogFile to ShareConfig in test
* Docs
2022-11-29 14:07:04 +00:00
Nick Cabatoff 87f0939d11
I think there was a plan at some point to support a "vault server -diagnose" command line option, but that doesn't seem to have ever come to pass. Removing the dead code. (#17983) 2022-11-17 09:25:27 +00:00
Jason O'Donnell de70878e16
core/server: Add environment variable to write stacktrace to file (#17929)
* Add env to write stacktrace to file

* changelog

* Use os.MkdirTemp

* Properly close file

* Adding path override

* Use temp file
2022-11-15 12:03:17 -05:00
Chris Capurso b31dc45ced
add noop ent supported storage check (#17883) 2022-11-10 16:46:59 -05:00
Peter Wilson 469c102fd3
Removed the legacy env var: LOGXI_FORMAT (#17822)
* Removed the legacy env var: LOGXI_FORMAT

* Added changelog

* Actually filled in the CL

* Added the name of legacy env var
2022-11-04 22:16:18 +00:00
Renato Costa ea1257be2e
fix incorrect use of loop variable (#16872)
This fixes a couple of references to loop variables in parallel tests
and deferred functions. When running a parallel test (calling
`t.Parallel()`) combined with the table-driven pattern, it's necessary
to copy the test case loop variable, otherwise only the last test case
is exercised. This is documented in the `testing` package:

https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks

`defer` statements that invoke a closure should also not reference a
loop variable directly as the referenced value will change in each
iteration of the loop.

Issues were automatically found with the `loopvarcapture` linter.
2022-10-04 09:23:37 -04:00
Mike Palmiotto 0078822d1d
core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool (#17319)
* core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool

* docs: Update VAULT_ALLOW_PENDING_REMOVAL_MOUNTS doc
2022-09-29 14:35:52 -04:00
Hamid Ghaf a034ebfd27
HCP link integration (#16939)
* HCP link integration

* update configure-git.yml

* more OSS stuff

* removing internal repos

* adding a nil check

* removing config test to be included in ENT only

* updating hcp-sdk-go to v0.22.0

* remove Hostname and AuthURL link config params

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-09-06 14:11:04 -04:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
Jason O'Donnell 140406143e
command/server: add dev-tls flag (#16421)
* command/server: add dev-tls flag

* Add website documentation

* changelog

* Lower file permissions

* Update cert gen per review

* Add dev-tls-cert-dir flag and cert clean up

* fmt

* Update cert generation per review

* Remove unused function

* Add better error messages

* Log errors in cleanup, fix directory not existing bug

* Remove hidden flag from -dev-tls-cert-dir

* Add usage

* Update 16421.txt

* Update variable names for files

* Remove directory on cleanup
2022-07-22 14:04:03 -04:00
Chris Capurso 9501d44ed5
Add endpoints to provide ability to modify logging verbosity (#16111)
* add func to set level for specific logger

* add endpoints to modify log level

* initialize base logger with IndependentLevels

* test to ensure other loggers remain unchanged

* add DELETE loggers endpoints to revert back to config

* add API docs page

* add changelog entry

* remove extraneous line

* add log level field to Core struct

* add godoc for getLogLevel

* add some loggers to c.allLoggers
2022-06-27 11:39:53 -04:00
Chris Capurso 75aa55eb57
add OSS noop check for valid ent storage (#15894) 2022-06-08 17:15:28 -04:00
Alexander Scheel 8d8a95cbf6
Add missing nil check to FIPS EA verification (#15883)
This was causing failures when running `vault server -dev`:

> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x105c41c1c]
>
> goroutine 1 [running]:
> github.com/hashicorp/vault/command.(*ServerCommand).parseConfig(0x140005a2180)
> 	.../vault/command/server.go:429 +0x5c

Interestingly, we do not have a test case for running the dev
sever.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 15:45:46 -04:00
Alexander Scheel dd6c339440
Add warning about EA in FIPS mode (#15858)
* Add warning about EA in FIPS mode

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 08:57:48 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Alexander Tischenko b89fc5af96
When running under systemd, send ready when server completed reloading config #7028 (#15041) 2022-05-03 08:34:11 -04:00
akshya96 796003ddda
Vault 3992 ToB Config and Plugins Permissions (#14817)
* updating changes from ent PR

* adding changelog

* fixing err

* fixing semgrep error
2022-04-04 09:45:41 -07:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00
hghaf099 8f6a97b81b
Vault-4279 reporting redundant/unused keys in config (#14752)
* Vault-4279 reporting redundant/unused keys in config

* missing validate step

* CL

* Change the log level
2022-04-01 10:34:27 -04:00
Victor Rodriguez f7f6f937e4
Invalidate the ManagedKeyRegistry cache when Vault config is updated. (#14179)
Add the hook needed to implement this functionality in vault-enterprise.
2022-02-21 09:55:44 -05:00
Hridoy Roy ebf8e5308a
SSCT Tokens Feature [OSS] (#14109)
* port SSCT OSS

* port header hmac key to ent and generate token proto without make command

* remove extra nil check in request handling

* add changelog

* add comment to router.go

* change test var to use length constants

* remove local index is 0 check and extra defer which can be removed after use of ExternalID
2022-02-17 11:43:07 -08:00
Alexander Scheel ec4deb12ef
Add infrastructure for exposing FIPS status (#14127)
In future Vault Enterprise versions, we'll be building Vault with
FIPS-validated cryptography. To help operators understand their
environment, we'll want to expose information about their FIPS status
when they're running a FIPS version of Vault.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-17 13:03:57 -05:00
Nick Cabatoff 21be98ee7a
Support go-sockaddr templates in top-level cluster_addr config (#13678)
In doing some testing I found that the listener clusteraddr isn't really used, or at least isn't as important as the top-level clusteraddr setting.  As such, go-sockaddr templating needs to be implemented for the top-level `cluster_addr` setting or it's unusable for HA.

Also fix a nil pointer panic I discovered at the same time.
2022-01-19 10:56:04 -05:00
hghaf099 65845c7531
VAULT-1564 report in-flight requests (#13024)
* VAULT-1564 report in-flight requests

* adding a changelog

* Changing some variable names and fixing comments

* minor style change

* adding unauthenticated support for in-flight-req

* adding documentation for the listener.profiling stanza

* adding an atomic counter for the inflight requests
addressing comments

* addressing comments

* logging completed requests

* fixing a test

* providing log_requests_info as a config option to determine at which level requests should be logged

* removing a member and a method from the StatusHeaderResponseWriter struct

* adding api docks

* revert changes in NewHTTPResponseWriter

* Fix logging invalid log_requests_info value

* Addressing comments

* Fixing a test

* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core

* fixing go.sum

* minor refactoring

* protecting InFlightRequests from data race

* another try on fixing a data race

* another try to fix a data race

* addressing comments

* fixing couple of tests

* changing log_requests_info to log_requests_level

* minor style change

* fixing a test

* removing the lock in InFlightRequests

* use single-argument form for interface assertion

* adding doc for the new configuration paramter

* adding the new doc to the nav data file

* minor fix
2021-12-08 17:34:42 -05:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
Mayo 0bd0339c0b
cleanup unused code and fix t.Fatal usage in goroutine in testing (#11694) 2021-09-30 07:33:14 -04:00
Rachel Culpepper c0ada31a99
Vault-2840 Add no-op check for HTTP listener validity (#12421)
* fips tag and no-op code

* tls check

* change to error

* add tests

* add test file

* cleanup

* add changelog

* fix file difference

* remove changelog
2021-09-07 18:36:58 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Scott Miller ee0d6603f3
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Josh Black 3189526cc4
OSS parts of sighup license reload test (#11816) 2021-06-11 09:08:19 -07:00
Hridoy Roy 65e3489c45
Diagnose resource creation checks (#11627)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* merge main branch

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* backup while I fix something else

* fix storage latency test errors

* runtime checks

* diagnose with timeout on seal
2021-06-10 12:29:32 -07:00
Nick Cabatoff bfae4e610b
Pull out license commands, and make the OSS changes needed for the license inspect PR in ent. (#11783) 2021-06-07 14:44:20 -04:00
Josh Black c8cfcd9514
OSS parts of sighup license reload (#11767) 2021-06-04 10:24:35 -07:00
Lars Lehtonen 42759b5a5f
command: deprecate errwrap.Wrapf() (#11744) 2021-06-02 09:22:31 -04:00
Nick Cabatoff 62ba5c39ca
Restore server license config lost in #11573 (#11730) 2021-06-01 08:42:30 -04:00
Hridoy Roy b6e1d62d33
Modularize Run Command (#11573)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* fix storage latency test errors

* review comments

* use random uuid for latency checks instead of static id
2021-05-25 15:23:20 -07:00
Nick Cabatoff 53c7d1de7d
config for autoloading license (oss parts) 2021-05-07 08:55:41 -04:00
Scott Miller 52930c5614
When running under systemd, send notifications about server startup, shutdown, and config reload (#11517) 2021-05-04 14:47:16 -05:00
Scott Miller 85fbd45e1c
Create helpers which integrate with OpenTelemetry for diagnose collection (#11454)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-04-29 13:32:41 -05:00
Hridoy Roy 36ae22c07a
Consul TLS Checks for Diagnose [draft] (#11467)
* consul tls checks

* fix some tests

* complete physical and service registration tls checks
2021-04-28 08:55:18 -07:00