Commit Graph

582 Commits

Author SHA1 Message Date
Pavlos Tzianos 0abc8f43fa
Add helper for encoding/decoding root tokens and OTP generation in SDK module (#10504) (#10505) 2021-12-01 08:05:49 -05:00
Nick Cabatoff a47a2c9fc4
Add "operator members" command to list nodes in the cluster. (#13292) 2021-11-30 14:49:58 -05:00
AnPucel eeb41dc76e
Allowing Unwrap w/ Newline files (#13044) 2021-11-24 10:13:45 -08:00
Anton Averchenkov 5af2b699fe
Respect WithWrappingToken for all secret ID's in approle auth (#13241) 2021-11-23 15:53:48 -08:00
divyapola5 5236fe93aa
Add a new parameter "allowed_managed_keys" to mount config (#13202)
* Add a new parameter "allowed_managed_keys" to mount config

* Adjust formatting in mount.go

* Add changelog entry
2021-11-21 19:08:38 -06:00
VAL e18f180609
GCP and Azure Login methods for Go client library (#13022)
* Add native Login method for GCP auth backend

* Add native Login method for Azure auth backend

* Add changelog entry

* Use official azure library Environment struct rather than passing string, add timeouts

* Use v1.3.0 which now has interface definition

* Don't throw away error and close resp body

* Back to WithResource so we can support non-Azure URLs for aud
2021-11-12 09:32:05 -08:00
Jonas-Taha El Sesiy 811c7a8133
Add PutAutoPilotRaftConfiguration to api (#12428) 2021-11-10 12:10:15 -05:00
VAL 558672797e
Remove reference to local api module, use v1.3.0 (#13105) 2021-11-09 14:49:46 -08:00
swayne275 849f4f8544
update sdk to 0.3.0 (#12946)
* update sdk to 0.3.0

* vault go mod update
2021-10-28 10:09:58 -06:00
VAL a44505dd06
Native Login method for Go client (#12796)
* Native Login method, userpass and approle interfaces to implement it

* Add AWS auth interface for Login, unexported struct fields for now

* Add Kubernetes client login

* Add changelog

* Add a test for approle client login

* Return errors from LoginOptions, use limited reader for secret ID

* Fix auth comment length

* Return actual type not interface, check for client token in tests

* Require specification of secret ID location using SecretID struct as AppRole arg

* Allow password from env, file, or plaintext

* Add flexibility in how to fetch k8s service token, but still with default

* Avoid passing strings that need to be validated by just having different login options

* Try a couple real tests with approle and userpass login

* Fix method name in comment

* Add context to Login methods, remove comments about certain sources being inherently insecure

* Perform read of secret ID at login time

* Read password from file at login time

* Pass context in integ tests

* Read env var values in at login time, add extra tests

* Update api version

* Revert "Update api version"

This reverts commit 1ef3949497dcf878c47e0e5ffcbc8cac1c3c1679.

* Update api version in all go.mod files
2021-10-26 16:48:48 -07:00
Ben Ash 0b095588c6
api.Client: support isolated read-after-write (#12814)
- add new configuration option, ReadYourWrites, which enables a Client
  to provide cluster replication states to every request. A curated set
  of cluster replication states are stored in the replicationStateStore,
  and is shared across clones.
2021-10-14 14:51:31 -04:00
Brian Kassouf 57c568e511
Update some SDK dependency versions (#12828)
* Update some SDK dependency versions

* Update API go.sum

* Update jsonpatch to v5
2021-10-14 09:47:32 -07:00
Chris Capurso bbb4ab4a41
Add HTTP PATCH support to KV (#12687)
* handle HTTP PATCH requests as logical.PatchOperation

* update go.mod, go.sum

* a nil response for logical.PatchOperation should result in 404

* respond with 415 for incorrect MIME type in PATCH Content-Type header

* add abstraction to handle PatchOperation requests

* add ACLs for patch

* Adding JSON Merge support to the API client

* add HTTP PATCH tests to check high level response logic

* add permission-based 'kv patch' tests in prep to add HTTP PATCH

* adding more 'kv patch' CLI command tests

* fix TestHandler_Patch_NotFound

* Fix TestKvPatchCommand_StdinValue

* add audit log test for HTTP PATCH

* patch CLI changes

* add patch CLI tests

* change JSONMergePatch func to accept a ctx

* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag

* go fmt

* add a test to verify patching works by default with the root token

* add changelog entry

* get vault-plugin-secrets-kv@add-patch-support

* PR feedback

* reorder some imports; go fmt

* add doc comment for HandlePatchOperation

* add json-patch@v5.5.0 to go.mod

* remove unnecessary cancelFunc for WriteBytes

* remove default for -method

* use stable version of json-patch; go mod tidy

* more PR feedback

* temp go get vault-plugin-secrets-kv@master until official release

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2021-10-13 15:24:31 -04:00
vinay-gopalan 458927c2ed
[VAULT-3157] Move `mergeStates` utils from Agent to api module (#12731)
* move merge and compare states to vault core

* move MergeState, CompareStates and ParseRequiredStates to api package

* fix merge state reference in API Proxy

* move mergeStates test to api package

* add changelog

* ghost commit to trigger CI

* rename CompareStates to CompareReplicationStates

* rename MergeStates and make compareStates and parseStates private methods

* improved error messaging in parseReplicationState

* export ParseReplicationState for enterprise files
2021-10-06 10:57:06 -07:00
Michael Boulding 79662d0842
Patch to support VAULT_HTTP_PROXY variable (#12582)
* patch to support VAULT_HTTP_PROXY variable

* simplify the proxy replacement

* internal code review

* rename to VAULT_HTTP_PROXY, apply within ReadEnvironment

* clean up some unintended whitespace changes

* add docs for the new env variable and a changelog entry

Co-authored-by: Dave Du Cros <davidducros@gmail.com>
2021-10-06 09:40:31 -07:00
VAL 1549af7e53
Add links to vault-examples repo (#12740) 2021-10-05 10:15:01 -07:00
Pratyoy Mukhopadhyay 92046f7d08
[VAULT-3248] Check api and sdk dirs in go_test (#12630)
* Check api and sdk dirs in go_test

* Update typo in script

* Append package names if non empty

* Don't fail command if no test packages found

* Add comments, clean up echoes

* Use pushd/popd, misc review fixes
2021-09-27 13:49:10 -07:00
Jinlong Chen 666b78911f
Fix client.go (#12608)
Modify one annotation.
2021-09-22 13:07:40 -07:00
Nick Cabatoff 45a83d8e0f
Add code to api.RaftSnapshot to detect incomplete snapshots (#12388) 2021-09-07 11:16:37 -04:00
Pratyoy Mukhopadhyay c379fd43a9
[MAR-3131] Set grace to 0 on non-positive lease duration (#12372)
* [MAR-3131] Set grace to 0 on non-positive lease duration

* [MAR-3131] Add changelog

* [VAULT-3131] Add test for negative lease duration
2021-08-24 19:06:40 -07:00
Nick Cabatoff 124bc87381
Upgrade snappy to fix panic with identity/packer on Go 1.16+arm64. (#12371) 2021-08-19 15:51:06 -04:00
Jason O'Donnell 5e86a34e3e
api: return parse errors if any for storage endpoints (#12338)
* logical/list: return parseErr if any

* changelog

* Add parseErr to other API endpoints

* Update 12338.txt
2021-08-17 13:19:39 -04:00
hghaf099 90c5b3c1c5
VAULT-1303 when a request to vault fails, show namespace if set (#12196)
* VAULT-1303 when a request to vault fails, show namespace if set

* Adding changelog

* Fix Changelog file name

* Set namespace in ResponseWriter headers if it is set

* Using consts.NamespaceHeaderName instead of the literal string
2021-07-30 12:32:05 -04:00
Jeff Mitchell 33ff878946
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Ben Ash e899e2adfa
Add ability to optionally clone an api.Client's headers (#12117) 2021-07-19 17:15:31 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Nick Cabatoff f027a1b1ff
Revert #12061 due to failures in TestLogical_RequestSizeLimit (#12093) 2021-07-15 12:55:09 -04:00
Jeff Mitchell fe18b6f9e0
Swap out sdk/helper libs with implementations in go-secure-stdlib (#12088)
* Swap out sdk/helper libs with implementations in go-secure-stdlib

* Fix transit batch test
2021-07-15 01:56:37 -04:00
hghaf099 f7635ec1b8
Add namespace in error (#12061)
* hghaf099-VAULT-1303-Adding namespace in error when it is set

* casting ResponseWriter in handleMonitor to logical.NamespaceResponseWriter

* Casting ResponseWriter conditionally for http.Flusher
Adding changelog

* Improving changlog message
2021-07-14 15:55:55 -04:00
Scott Miller ecb5474466
Update Vault main to new API/SDK Tags. (#12069)
* Update Vault main to new api/sdk tags

* go mod tidy

* Go mod tidy

* Go mod tidy on api

* go mod download on root
2021-07-13 18:54:31 -05:00
Jeff Mitchell c1f058f18b Bump API's SDK version 2021-07-13 14:50:49 -07:00
Sam Salisbury d4fa62e979
ci: cache go modules (#11935) 2021-06-25 17:17:05 +01:00
Richard Patel 864d52b9af
Support tls-skip-verify for output-curl-string (#11713) 2021-06-14 11:09:29 -04:00
Rajwinder Mahal 5870c3daa5
api/client: allow configurable values for RetryWaitMin and RetryWaitMax (#11773) 2021-06-11 15:15:21 -04:00
Calvin Leung Huang 8cb48018b7
api/client: provide the ability to set a logger on retryablehttp.Client (#11696)
* api/client: provide the ability to set a logger on retryablehttp.Client

* go mod tidy; fix import ordering

* go mod vendor
2021-05-27 10:25:25 -07:00
Nick Cabatoff 3b88a87b84
LifetimeWatcher should retry renew failures until end of lease (#11445)
Co-authored-by: Andrej van der Zee <andrejvanderzee@gmail.com>
2021-05-06 14:04:26 -04:00
Hridoy Roy fde9f2f71d
Add More TLS Tests and Verification of TLS Root Certificate (#11300)
* tls tests and root verification

* make the certificate verification check correct for non root CA case

* add expiry test

* addressed comments but struggling with the bug in parsing Cas and inters from single file:

* final checks on tls and listener

* cleanup
2021-04-12 08:39:40 -07:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Vishal Nayak a9232bf399
Add JSON marshaler to CLI output of autopilot config (#11166) 2021-03-22 10:23:12 -04:00
Clint 5353279e75
Add command to look up a lease by ID (#11129)
* snapshot

* basic test

* update command and add documentation

* update help text

* typo

* add changelog for lease lookup command

* run go mod vendor

* remove tabs from help output
2021-03-18 11:11:09 -05:00
Vishal Nayak 9839e76192
Remove unneeded fields from state output (#11073) 2021-03-10 12:08:12 -05:00
Vishal Nayak 3e55e79a3f
Autopilot: Server Stabilization, State and Dead Server Cleanup (#10856)
* k8s doc: update for 0.9.1 and 0.8.0 releases (#10825)

* k8s doc: update for 0.9.1 and 0.8.0 releases

* Update website/content/docs/platform/k8s/helm/configuration.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Autopilot initial commit

* Move autopilot related backend implementations to its own file

* Abstract promoter creation

* Add nil check for health

* Add server state oss no-ops

* Config ext stub for oss

* Make way for non-voters

* s/health/state

* s/ReadReplica/NonVoter

* Add synopsis and description

* Remove struct tags from AutopilotConfig

* Use var for config storage path

* Handle nin-config when reading

* Enable testing autopilot by using inmem cluster

* First passing test

* Only report the server as known if it is present in raft config

* Autopilot defaults to on for all existing and new clusters

* Add locking to some functions

* Persist initial config

* Clarify the command usage doc

* Add health metric for each node

* Fix audit logging issue

* Don't set DisablePerformanceStandby to true in test

* Use node id label for health metric

* Log updates to autopilot config

* Less aggressively consume config loading failures

* Return a mutable config

* Return early from known servers if raft config is unable to be pulled

* Update metrics name

* Reduce log level for potentially noisy log

* Add knob to disable autopilot

* Don't persist if default config is in use

* Autopilot: Dead server cleanup (#10857)

* Dead server cleanup

* Initialize channel in any case

* Fix a bunch of tests

* Fix panic

* Add follower locking in heartbeat tracker

* Add LastContactFailureThreshold to config

* Add log when marking node as dead

* Update follower state locking in heartbeat tracker

* Avoid follower states being nil

* Pull test to its own file

* Add execution status to state response

* Optionally enable autopilot in some tests

* Updates

* Added API function to fetch autopilot configuration

* Add test for default autopilot configuration

* Configuration tests

* Add State API test

* Update test

* Added TestClusterOptions.PhysicalFactoryConfig

* Update locking

* Adjust locking in heartbeat tracker

* s/last_contact_failure_threshold/left_server_last_contact_threshold

* Add disabling autopilot as a core config option

* Disable autopilot in some tests

* s/left_server_last_contact_threshold/dead_server_last_contact_threshold

* Set the lastheartbeat of followers to now when setting up active node

* Don't use config defaults from CLI command

* Remove config file support

* Remove HCL test as well

* Persist only supplied config; merge supplied config with default to operate

* Use pointer to structs for storing follower information

* Test update

* Retrieve non voter status from configbucket and set it up when a node comes up

* Manage desired suffrage

* Consider bucket being created already

* Move desired suffrage to its own entry

* s/DesiredSuffrageKey/LocalNodeConfigKey

* s/witnessSuffrage/recordSuffrage

* Fix test compilation

* Handle local node config post a snapshot install

* Commit to storage first; then record suffrage in fsm

* No need of local node config being nili case, post snapshot restore

* Reconcile autopilot config when a new leader takes over duty

* Grab fsm lock when recording suffrage

* s/Suffrage/DesiredSuffrage in FollowerState

* Instantiate autopilot only in leader

* Default to old ways in more scenarios

* Make API gracefully handle 404

* Address some feedback

* Make IsDead an atomic.Value

* Simplify follower hearbeat tracking

* Use uber.atomic

* Don't have multiple causes for having autopilot disabled

* Don't remove node from follower states if we fail to remove the dead server

* Autopilot server removals map (#11019)

* Don't remove node from follower states if we fail to remove the dead server

* Use map to track dead server removals

* Use lock and map

* Use delegate lock

* Adjust when to remove entry from map

* Only hold the lock while accessing map

* Fix race

* Don't set default min_quorum

* Fix test

* Ensure follower states is not nil before starting autopilot

* Fix race

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-03-03 13:59:50 -05:00
Andrej van der Zee df159d7622
Respect VAULT_MAX_RETRIES from environment in DefaultConfig() (#10883) 2021-03-02 10:39:20 -08:00
Scott Miller b13b27f37e
OSS side barrier encryption tracking and automatic rotation (#11007)
* Automatic barrier key rotation, OSS portion

* Fix build issues

* Vendored version

* Add missing encs field, not sure where this got lost.
2021-02-25 14:27:25 -06:00
Ace Eldeib 9584c989ca
don't panic on connection errors in DefaultRetryPolicy (#11002)
fixes #11001
2021-02-25 13:16:17 -05:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Vishal Nayak 53cb1deb38
Revert "Read-replica instead of non-voter (#10875)" (#10890)
This reverts commit fc745670cf34821f5834357d9caebc3351dbc1e7.
2021-02-10 16:41:58 -05:00
Vishal Nayak a2394e7353
Read-replica instead of non-voter (#10875) 2021-02-10 09:58:18 -05:00
Josh Black a7aac342bd
Only set the namespace if the env var isn't present (#1519) (#10556) 2020-12-14 11:40:48 -08:00
Brian Kassouf 275ca323e8
core: Record the time a node became active (#10489)
* core: Record the time a node became active

* Update vault/core.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Add omitempty field

* Update vendor

* Added CL entry and fixed test

* Fix test

* Fix command package tests

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2020-12-11 16:50:19 -08:00