Commit Graph

363 Commits

Author SHA1 Message Date
Becca Petrin abb621752f Clean up error string formatting (#4304) 2018-04-09 14:35:21 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Jeff Mitchell 49ee1113aa Move close for s3 a bit earlier in case data is returned with 404 2018-03-30 13:03:33 -04:00
Jeff Mitchell 23e80d028f
Switch reading from S3 to io.Copy from io.ReadFull (#4225)
* Switch reading from S3 to io.Copy from io.ReadFull

If the Content-Length header wasn't being sent back, the current
behavior could panic. It's unclear when it will not be sent; it appears
to be CORS dependent. But this works around it by not trying to
preallocate a buffer of a specific size and instead just read until EOF.

In addition I noticed that Close wasn't being called.
https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#GetObjectOutput
specifies that Body is an io.ReadCloser so I added a call to Close.

Fixes #4222

* Add some extra efficiency
2018-03-30 12:42:48 -04:00
Seth Vargo b48a9878e7 Add HA support to the Google Cloud Storage backend (#4226) 2018-03-30 12:36:37 -04:00
Seth Vargo 56a48dc322 Fix broken build (#4212)
90e3ad2 broke the build. This fixes the syntax and ensures compilation.
I have no idea and make no claims that this is the correct behavior.
2018-03-28 15:47:40 -04:00
Nicholas Watkins 3b3af2f0a6 Add MySQL db max idle connections and connections lifetime assignment (#4211)
Allow the storage backend for MySQL to use a custom connection lifetime and max idle connection value if the parameter is specified in the config file of vault otherwise do not set in order to leave at default value.
2018-03-28 14:43:55 -04:00
Jim Kalafut d67ed1fdc2
Replace Batch operation in Cassandra Delete() (#4054)
This fixes failing Cassandra backend tests. It is also probably the
better route, since Batch operations aren't the preferred approach
unless necessary (https://docs.datastax.com/en/cql/3.3/cql/cql_using/useBatch.html).
2018-03-23 09:42:35 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Nicholas Watkins 475d5910e8 Allow configuration of dynamodb storage to specify the max retries of aws sdk (#4115) 2018-03-19 15:53:23 -04:00
Vishal Nayak a420d19bff
Remove limit on the couchdb listing (#4149) 2018-03-18 18:31:15 -04:00
Aleksandar a8304e5d4d Add the chunk_size optional parameter to gcs storage (#4060) 2018-03-05 08:32:48 -05:00
chris trott 78df6a630e Configurable Consul Service Address (#3971)
* Consul service address is blank

Setting an explicit service address eliminates the ability for Consul
to dynamically decide what it should be based on its translate_wan_addrs
setting.

translate_wan_addrs configures Consul to return its lan address to nodes
in its same datacenter but return its wan address to nodes in foreign
datacenters.

* service_address parameter for Consul storage backend

This parameter allows users to override the use of what Vault knows to
be its HA redirect address.

This option is particularly commpelling because if set to a blank
string, Consul will leverage the node configuration where the service is
registered which includes the `translate_wan_addrs` option. This option
conditionally associates nodes' lan or wan address based on where
requests originate.

* Add TestConsul_ServiceAddress

Ensures that the service_address configuration parameter is setting the
serviceAddress field of ConsulBackend instances properly.

If the "service_address" parameter is not set, the ConsulBackend
serviceAddress field must instantiate as nil to indicate that it can be
ignored.
2018-02-23 11:15:29 -05:00
Jeff Mitchell be53e38fe0 Fix test statement with formatting in fatal call 2018-02-20 00:26:41 -05:00
Vitya 66f4589195 Fix compatibility with some Postgres versions (#3986)
use server_version_num instead of parsing the text version. See: https://www.postgresql.org/docs/10/static/functions-info.html
2018-02-16 12:52:34 -05:00
Seth Vargo b5e4db975e Add useragent helper (#3991)
* Add useragent package

This helper provides a consistent user-agent header for Vault, taking into account different versions.

* Add user-agent headers to spanner and gcs
2018-02-15 18:30:31 -05:00
Seth Vargo cd930b1173 Add support for Google Cloud Spanner (#3977) 2018-02-14 20:31:20 -05:00
Jeff Mitchell a248a08102 Fix manta test 2018-02-14 10:39:52 -05:00
Jeff Mitchell 4f984569fa Plumb context through manta 2018-02-13 10:03:12 -05:00
Paul Stack 3c683dba92 Adding Manta Storage Backend (#3720)
This PR adds a new Storage Backend for Triton's Object Storage - Manta

```
make testacc TEST=./physical/manta
==> Checking that code complies with gofmt requirements...
==> Checking that build is using go version >= 1.9.1...
go generate
VAULT_ACC=1 go test -tags='vault' ./physical/manta -v  -timeout 45m
=== RUN   TestMantaBackend
--- PASS: TestMantaBackend (61.18s)
PASS
ok  	github.com/hashicorp/vault/physical/manta	61.210s
```

Manta behaves differently to how S3 works - it has no such concepts of Buckets - it is merely a filesystem style object store

Therefore, we have chosen the approach of when writing a secret `foo` it will actually map (on disk) as foo/.vault_value

The reason for this is because if we write the secret `foo/bar` and then try and Delete a key using the name `foo` then Manta
will complain that the folder is not empty because `foo/bar` exists. Therefore, `foo/bar` is written as `foo/bar/.vault_value`

The value of the key is *always* written to a directory tree of the name and put in a `.vault_value` file.
2018-02-12 18:22:41 -05:00
Jeff Mitchell ac382055d4
Validate Consul service name is RFC 1123 compliant (#3961) 2018-02-12 16:11:59 -05:00
Jeff Mitchell 844b7c395f Refactor fail logic in inmem 2018-02-12 11:25:48 -05:00
Jeff Mitchell 609648de4f Convert logical.InmemStorage to a wrapper around physical/inmem.
The original reason for the split was physical's dependencies, but those
haven't been onerous for a long time. Meanwhile it's a totally separate
implementation so we could be getting faulty results from tests. Get rid
of it and use the unified physical/inmem.
2018-02-12 11:16:16 -05:00
Jeff Mitchell 642b88c76a go vet fixes 2018-02-05 14:26:31 -05:00
Jeff Mitchell f33563f667 Some vet fixes 2018-02-04 20:37:57 -05:00
Xiang Li a120544b47 etcd: config etcd3 client's max response size (#3891) 2018-02-01 19:08:09 -05:00
Xiang Li 5fd85205cc etcd3: only create lock when lock is called (#3893) 2018-02-01 19:04:52 -05:00
Jeff Mitchell 60e2209532
Remove core restriction in cache and turn it into an active/standby restriction instead (#3849) 2018-01-25 22:21:51 -05:00
Brian Kassouf aa387bb4c2
Add compile tests to verify physical stores satisfy the correct interfaces (#3820) 2018-01-19 17:44:24 -08:00
Jeff Mitchell 395befc062 Update cache to satisfy Purge interface after context plumbing 2018-01-19 17:00:13 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jon Davies 66e2593ef9 s3.go: Added options to use paths with S3 and the ability to disable SSL (#3730) 2018-01-03 12:11:00 -05:00
Antergone 312db6cc02 fix consul tls settings (#3719) 2017-12-19 14:24:21 -05:00
Antergone d68cc66954 check schema and table before create it (#3716) 2017-12-19 14:23:58 -05:00
Chris Hoffman f966d20225
Adding ability to cache core values, cache transaction improvements (#3640) 2017-12-06 12:25:17 -05:00
Jeff Mitchell 548629e8ef Port over some changes 2017-11-30 09:43:07 -05:00
Vlad Ungureanu 2ff547196f Remove unused recovery field in dynamodb backend (#3569) 2017-11-13 15:46:02 -05:00
Ben Higgins f78ab356d4 vault: recover from standby losing etcd lease (#3031) (#3511)
This change makes these errors transient instead of permanent:

[ERROR] core: failed to acquire lock: error=etcdserver: requested lease not found

After this change, there can still be one of these errors when a
standby vault that lost its lease tries to become leader, but on the
next lock acquisition attempt a new session will be created. With this
new session, the standby will be able to become the leader.
2017-11-03 13:38:16 -04:00
Jeff Mitchell 8a610e1e78 Move underscore tests to file from physical testing 2017-10-26 15:29:10 -04:00
Jeff Mitchell 28b0db38cc Revert couchdb changes 2017-10-26 15:27:20 -04:00
Jeff Mitchell 85500b5c3a Change prefix to a string that can be specified, rather than a bool 2017-10-26 15:26:28 -04:00
Jeff Mitchell 7e32ac15ec Add prefixing to couch to fix the error that was exposed 2017-10-26 15:26:28 -04:00
Jeff Mitchell 425b781fc8 Fix more tests 2017-10-26 15:26:28 -04:00
Jeff Mitchell 40a6635cd6 Fix testing 2017-10-26 15:26:28 -04:00
Jeff Mitchell e122853746 Add some more tests 2017-10-26 15:26:28 -04:00
Jeff Mitchell 3af3cf2b73 Allow underscores at the start of directories in file backend.
Fixes #3476
2017-10-26 15:26:28 -04:00
Jeff Mitchell 8e9317792d Fix some merge/update bugs 2017-10-23 16:49:46 -04:00
Jeff Mitchell c144f95be0 Sync over 2017-10-23 16:43:07 -04:00
Jeff Mitchell 1d852a7243 Use 0700 for directory permissions in file physical backend. 2017-10-12 14:24:34 -04:00