Commit graph

93 commits

Author SHA1 Message Date
Jim Kalafut f17fc4e5c1
Run goimports (#8251) 2020-01-27 21:11:00 -08:00
Vishal Nayak 8891f2ba88 Raft retry join (#7856)
* Raft retry join

* update

* Make retry join work with shamir seal

* Return upon context completion

* Update vault/raft.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Address some review comments

* send leader information slice as a parameter

* Make retry join work properly with Shamir case. This commit has a blocking issue

* Fix join goroutine exiting before the job is done

* Polishing changes

* Don't return after a successful join during unseal

* Added config parsing test

* Add test and fix bugs

* minor changes

* Address review comments

* Fix build error

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-13 17:02:16 -08:00
Mike Jarmy e42bc0ffc0
Introduce optional service_registration stanza (#7887)
* move ServiceDiscovery into methods

* add ServiceDiscoveryFactory

* add serviceDiscovery field to vault.Core

* refactor ConsulServiceDiscovery into separate struct

* cleanup

* revert accidental change to go.mod

* cleanup

* get rid of un-needed struct tags in vault.CoreConfig

* add service_discovery parser

* add ServiceDiscovery to config

* cleanup

* cleanup

* add test for ConfigServiceDiscovery to Core

* unit testing for config service_discovery stanza

* cleanup

* get rid of un-needed redirect_addr stuff in service_discovery stanza

* improve test suite

* cleanup

* clean up test a bit

* create docs for service_discovery

* check if service_discovery is configured, but storage does not support HA

* tinker with test

* tinker with test

* tweak docs

* move ServiceDiscovery into its own package

* tweak a variable name

* fix comment

* rename service_discovery to service_registration

* tweak service_registration config

* Revert "tweak service_registration config"

This reverts commit 5509920a8ab4c5a216468f262fc07c98121dce35.

* simplify naming

* refactor into ./serviceregistration/consul
2019-12-06 09:46:39 -05:00
Michel Vocks beeec81df7
Add enable_hostname_label option to telementry stanza (#7902) 2019-12-02 09:26:46 +01:00
Lexman c86fe212c0
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Calvin Leung Huang d8875b1991
sys/config: config state endpoint (#7424)
* sys/config: initial work on adding config state endpoint

* server/config: add tests, fix Sanitized method

* thread config through NewTestCluster's config to avoid panic on dev modes

* properly guard endpoint against request forwarding

* add http tests, guard against panics on nil RawConfig

* ensure non-nil rawConfig on NewTestCluster cores

* update non-forwarding logic

* fix imports; use no-forward handler

* add missing config test fixture; update gitignore

* return sanitized config as a map

* fix test, use deep.Equal to check for equality

* fix http test

* minor comment fix

* config: change Sanitized to return snake-cased keys, update tests

* core: hold rlock when reading config; add docstring

* update docstring
2019-10-08 10:57:15 -07:00
Tommy Murphy fc3f1896ad telemetry: add stackdriver metrics sink (#6957)
* telemetry: add stackdriver metrics sink

* telemetry: stackdriver go mod tidy
2019-08-20 14:47:08 -07:00
Calvin Leung Huang 67170b378f command/server: fix TestLoadConfigFile_json2 test, fix hcl tags (#7300)
* command/server: fix TestLoadConfigFile_json2 test, fix hcl tags

Fixes test to call the equality check, and add missing values to the expected object. Fixes hcl tags in the Telemetry structs.

* fix PrometheusRetentionTime tag
2019-08-14 10:32:11 -04:00
Mike Jarmy 0d4ae949a8
Add 'log-format' CLI flag, along with associated config flag, for 'vault server' command. (#6840)
* Read config before creating logger when booting vault server

* Allow for specifying log output in JSON format in a config file, via a 'log_level' flag

* Create parser for log format flag

* Allow for specifying log format in a config file, via a 'log_format' flag. Also, get rid of 'log_json' flag.

* Add 'log-format' command line flag

* Update documentation to include description of log_format setting

* Tweak comment for VAULT_LOG_FORMAT environment variable

* add test for ParseEnvLogFormat()

* clarify how log format is set

* fix typos in documentation
2019-07-18 15:59:27 -04:00
Jeff Mitchell a4214a2161
Add the ability to use a dev Consul node for dev storage (#6965) 2019-06-24 13:29:47 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Jeff Mitchell a83ed04730 Add ability to migrate autoseal to autoseal (#5930)
* Add ability to migrate autoseal to autoseal

This adds the ability to migrate from shamir to autoseal, autoseal to
shamir, or autoseal to autoseal, by allowing multiple seal stanzas. A
disabled stanza will be used as the config being migrated from; this can
also be used to provide an unwrap seal on ent over multiple unseals.

A new test is added to ensure that autoseal to autoseal works as
expected.

* Fix test

* Provide default shamir info if not given in config

* Linting feedback

* Remove context var that isn't used

* Don't run auto unseal watcher when in migration, and move SetCores to SetSealsForMigration func

* Slight logic cleanup

* Fix test build and fix bug

* Updates

* remove GetRecoveryKey function
2019-03-04 14:11:56 -08:00
Martin 9044173d6e Prometheus support on v1/sys/metrics endpoint (#5308)
* initial commit for prometheus and sys/metrics support

* Throw an error if prometheusRetentionTime is 0,add prometheus in devmode

* return when format=prometheus is used and prom is disable

* parse prometheus_retention_time from string instead of int

* Initialize config.Telemetry if nil

* address PR issues

* add sys/metrics framework.Path in a factory

* Apply requiredMountTable entries's MountConfig to existing core table

* address pr comments

* enable prometheus sink by default

* Move Metric-related code in a separate metricsutil helper
2019-02-14 12:46:59 -08:00
Noel Cower 4f05192be3 Merge all configuration fields (#6028)
This changes (*Config).Merge to merge all fields of a Config.
Previously, when merging Configs, some configuration fields were
ignored and completely lost, including APIAddr, ClusterAddr, and
a couple boolean fields. This only occurs when using multiple config
files and does not affect single config files (even when loading from
a directory -- Merge is only called after a second file is loaded).

- Fix APIAddr not being merged.
- Fix ClusterAddr not being merged.
- Fix DisablePrintableCheck not being merged.
- Fix DisableClustering not being merged. The DisableClusteringRaw
  value is also preserved so that it can be used in overrides for
  storage fields.
- Use merged top-level config as storage field overrides.
- Update config dir test fixtures to set some fields missed by
  (*Config).Merge previously.
2019-01-23 11:27:21 -05:00
Jim Kalafut d0e2badbae Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Chris Hoffman 8c88eb3e2a
Add -dev-auto-seal option (#5629)
* adding a -dev-auto-seal option

* adding logger to TestSeal
2018-10-29 09:30:24 -04:00
Jeff Mitchell 91a970b81d Fix build 2018-10-23 15:09:35 -04:00
Jeff Mitchell a979f49cd7 Add disable-indexing 2018-10-23 15:03:17 -04:00
Jeff Mitchell 82992d6097
Seal migration (OSS) (#781) 2018-10-22 23:34:02 -07:00
Calvin Leung Huang a08ccbffa7
[Review Only] Autoseal OSS port (#757)
* Port awskms autoseal

* Rename files

* WIP autoseal

* Fix protobuf conflict

* Expose some structs to properly allow encrypting stored keys

* Update awskms with the latest changes

* Add KeyGuard implementation to abstract encryption/decryption of keys

* Fully decouple seal.Access implementations from sealwrap structs

* Add extra line to proto files, comment update

* Update seal_access_entry.go

* govendor sync

* Add endpoint info to configureAWSKMSSeal

* Update comment

* Refactor structs

* Update make proto

* Remove remove KeyGuard, move encrypt/decrypt to autoSeal

* Add rest of seals, update VerifyRecoveryKeys, add deps

* Fix some merge conflicts via govendor updates

* Rename SealWrapEntry to EncryptedBlobInfo

* Remove barrier type upgrade check in oss

* Add key to EncryptedBlobInfo proto

* Update barrierTypeUpgradeCheck signature
2018-10-19 14:43:57 -07:00
Jim Kalafut 4c80debe63
Add physical backend migrator command (#5143) 2018-09-25 16:18:22 -07:00
Jeff Mitchell 43aebacfa8 Fix default_max_request_duration HCL name and update docs (#5321)
* Fix default_max_request_duration HCL name and update docs

* Update tcp.html.md
2018-09-18 14:30:21 -07:00
Jeff Mitchell c28ed23972
Allow most parts of Vault's logging to have its level changed on-the-fly (#5280)
* Allow most parts of Vault's logging to have its level changed on-the-fly

* Use a const for not set
2018-09-05 15:52:54 -04:00
Jeff Mitchell aec9a689a0 Sync over some stuff 2018-08-24 12:09:03 -04:00
Jeff Mitchell 4261618d10 Add request timeouts in normal request path and to expirations (#4971)
* Add request timeouts in normal request path and to expirations

* Add ability to adjust default max request duration

* Some test fixes

* Ensure tests have defaults set for max request duration

* Add context cancel checking to inmem/file

* Fix tests

* Fix tests

* Set default max request duration to basically infinity for this release for BC

* Address feedback
2018-07-24 14:50:49 -07:00
Jeff Mitchell 8433bf26e9 Fix printable check key not being valid 2018-07-12 16:59:07 -04:00
Jeff Mitchell 954f6c4ece
Add config flag to disable non-printable character check (#4917) 2018-07-12 16:29:36 -04:00
Jeff Mitchell 4a3fe87a39
Allow max request size to be user-specified (#4824)
* Allow max request size to be user-specified

This turned out to be way more impactful than I'd expected because I
felt like the right granularity was per-listener, since an org may want
to treat external clients differently from internal clients. It's pretty
straightforward though.

This also introduces actually using request contexts for values, which
so far we have not done (using our own logical.Request struct instead),
but this allows non-logical methods to still get this benefit.

* Switch to ioutil.ReadAll()
2018-07-06 15:44:56 -04:00
Calvin Leung Huang c4abeb9ea5
Move checkHCLKeys into hclutil (#4749) 2018-06-12 12:38:08 -04:00
Jeff Mitchell 9e7f381fca Sync over changes to config.go 2018-05-30 08:34:46 -04:00
Jeff Mitchell 805b5e5160
X-Forwarded-For (#4380) 2018-04-17 18:52:09 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Calvin Leung Huang 882f85740b Move HA-related config values to top level (#3550)
* Move HA-related config values to top level

* Add config2.hcl test-fixture
2017-11-08 14:19:41 -05:00
Calvin Leung Huang 0762ac14cf aws_region->region on awskms config 2017-11-02 16:31:16 -04:00
Jeff Mitchell a25dae82dd Final sync 2017-10-23 17:39:21 -04:00
Jeff Mitchell 17a15cd594 Add option to disable client certificate requesting. (#3373)
Fixes #3372
2017-09-25 14:41:46 -04:00
Vishal Nayak e99640f462 Add 'pid_file' config option (#3321)
* add pid_file config option

* address review feedback

* address review comments
2017-09-16 17:09:37 -04:00
Chris Hoffman 9d73c81f38 Disable the sys/raw endpoint by default (#3329)
* disable raw endpoint by default

* adding docs

* config option raw -> raw_storage_endpoint

* docs updates

* adding listing on raw endpoint

* reworking tests for enabled raw endpoints

* root protecting base raw endpoint
2017-09-15 00:21:35 -04:00
Jeff Mitchell 3edb337a00 Add option to set cluster TLS cipher suites. (#3228)
* Add option to set cluster TLS cipher suites.

Fixes #3227
2017-08-30 16:28:23 -04:00
Gobin Sougrakpam 8e01c994bf tls_client_ca_file option for verifying client (#3034) 2017-08-03 07:33:06 -04:00
Jeff Mitchell 7e3ff5e56c Add PROXY protocol support (#3098) 2017-08-02 18:24:12 -04:00
Jeff Mitchell 1bfc6d4fe7 Add a -dev-three-node option for devs. (#3081) 2017-07-31 11:28:06 -04:00
Jeff Mitchell d55d75a79f Convert listener arguments to map[string]interface{} (#2905)
This allows people to use more natural constructs, e.g. for tls_disable
it can be a bool, int, or string.
2017-06-22 20:29:53 +01:00
Jeff Mitchell 33ca94773f Add DogStatsD metrics output. (#2883)
Fixes #2490
2017-06-16 23:51:46 -04:00
Brian Kassouf 8a3ef906d5 Update the plugin directory logic 2017-04-13 11:22:53 -07:00
Brian Kassouf 8ccf10641b Merge branch 'master' into database-refactor 2017-04-12 14:29:10 -07:00
Brian Kassouf e8781b6a2b Plugin catalog 2017-04-03 17:52:29 -07:00
Jeff Mitchell 5d760d4090 Add option to require valid client certificates (#2457) 2017-03-08 10:21:31 -05:00