Commit Graph

35 Commits

Author SHA1 Message Date
Sergey Lanzman 90b12f1386
Add AWS_DYNAMODB_REGION Environment variable (#15054)
Added AWS_DYNAMODB_REGION env
2022-04-28 12:29:51 -07:00
Brian Kassouf c705adc79c
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Jeff Mitchell 33ff878946
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Lars Lehtonen 5ac47a9265
physical: deprecate errwrap.Wrapf() (#11692) 2021-05-31 12:54:05 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Scott Miller 080c9ca6ba
Fix err shadowing (#11296) 2021-04-07 11:25:23 -05:00
Conrad Lara 3f51589be6
Potential data loss in DynamoDB backend (#10181)
fixes hashicorp/vault#5836

DynamoDB may when throttled return a 2xx response while not committing
all submitted items to the database.

Depending upon load all actions in a BatchWriteUpdate may be throttled
with ProvisionedThroughputExceededException in which case AWS SDK handles
the retry. If some messages were throttled but not all
ProvisionedThroughputExceededException is not returned to the SDK and it
is up to us to resubmit the request.

Using an exponential backoff as recommended in AWS SDK for times we possibly
get partially throttled repeatedly.
2021-04-07 09:44:42 -05:00
Theron Voran 52581cd472
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
ncabatoff b615da43d7
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Jeff Mitchell a0694943cc
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
ncabatoff cbde4d4357
Use a much longer delay before giving up and failing the test. (#7646)
On circleci we've seen the test run very slowly so it's often failing here.
2019-10-14 11:06:02 -04:00
EdwinRobbins de885660e8 DynamoDB: Make Unlock key delete conditional on being old leader's (#6637) 2019-04-30 17:44:47 -07:00
Jim Kalafut fa2accd000
Exit DynamoDB tryToLock when stop channel is closed (#6640)
* Exit DynamoDB tryToLock when stop channel is closed

If the stop channel is closed (e.g. an error is returned which triggers
close(stop) in Lock), this loop will spin and use 100% CPU.

* Ensure ticker is stopped
2019-04-29 22:51:48 -07:00
ncabatoff 06574da57a
Merge multiple functions for creating consul containers into one. (#6612)
Merge both functions for creating mongodb containers into one.
Add retries to docker container cleanups.
Require $VAULT_ACC be set to enable AWS tests.
2019-04-22 12:26:10 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Maarten van der Hoef d98b477a7e Spelling fix (#6523) 2019-04-03 06:57:32 -07:00
Jim Kalafut a939613f78 Reset retry count after a successful lock check (#6522) 2019-04-02 17:08:49 -07:00
Mahmoud Abdelsalam 6479a73f3a Fix dynamodb HA lock race (#6512)
* Fix DynamoDB HA race issue

* Add test for race condition (which fails on the released DynamoDB code)
2019-04-02 16:19:21 -07:00
Jeff Mitchell c8a029210b
Don't read AWS env vars (#5974)
* Don't read AWS env vars

Let AWS SDK env cred chain provider do it for us

Fixes #5965
2019-01-04 15:03:57 -05:00
Calvin Leung Huang 2824cb534d
Clean up HABackend tests (#5617) 2018-11-01 10:31:09 -07:00
Calvin Leung Huang b5c8b3d4c5 Clean up container on connection failure, switch to ory/dockertest on package postgresql (#5050) 2018-08-07 09:56:33 -07:00
Jeff Mitchell db9abad3c9 Remove unnecessary log scoping in some physical backends 2018-07-13 10:57:30 -04:00
Jeff Mitchell 4b354e1110
Re-add dockertest and fix up imports and update script (#4909) 2018-07-11 17:49:13 -04:00
Jeff Mitchell e52b554c0b
Add an idle timeout for the server (#4760)
* Add an idle timeout for the server

Because tidy operations can be long-running, this also changes all tidy
operations to behave the same operationally (kick off the process, get a
warning back, log errors to server log) and makes them all run in a
goroutine.

This could mean a sort of hard stop if Vault gets sealed because the
function won't have the read lock. This should generally be okay
(running tidy again should pick back up where it left off), but future
work could use cleanup funcs to trigger the functions to stop.

* Fix up tidy test

* Add deadline to cluster connections and an idle timeout to the cluster server, plus add readheader/read timeout to api server
2018-06-16 18:21:33 -04:00
Jim Kalafut 1f648271b6
Add DynamoDB marshaling update test (#4746)
This test fails before the d3604289be99 update.
2018-06-12 08:22:02 -07:00
Matthew Surabian 4e0bc43bf8 DynamoDB Nested Values Bug (#4570)
* Add tests to ExerciseBackend to expose nested-values bug

* Update DynamoDB physical backend Delete and hasChildren logic to prevent overzealous cleanup of folders and values
2018-05-16 13:30:56 -04:00
Matthew Surabian 01d63b8148 DynamoDB Deprecation Fixes (#4534)
* Use the AWS SDK's UnmarshalMap method for dynamodb backend, not the deprecated ConvertFromMap method

* Use the AWS SDK's MarshalMap method for dynamodb backend, not the deprecated ConvertToMap method

* Use the AWS SDK's session.NewSession method for dynamodb backend, not the deprecated session.New method

* Fix variable name awserr that colides with imported package in dynamodb backend
2018-05-10 08:25:51 -04:00
Becca Petrin abb621752f Clean up error string formatting (#4304) 2018-04-09 14:35:21 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Nicholas Watkins 475d5910e8 Allow configuration of dynamodb storage to specify the max retries of aws sdk (#4115) 2018-03-19 15:53:23 -04:00
Brian Kassouf aa387bb4c2
Add compile tests to verify physical stores satisfy the correct interfaces (#3820) 2018-01-19 17:44:24 -08:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Vlad Ungureanu 2ff547196f Remove unused recovery field in dynamodb backend (#3569) 2017-11-13 15:46:02 -05:00
Jeff Mitchell fdaaaadee2 Migrate physical backends into separate packages (#3106) 2017-08-03 13:24:27 -04:00