Commit graph

432 commits

Author SHA1 Message Date
Jeff Mitchell 85a560abba
Refactor common token fields and operations into a helper (#5953) 2019-01-30 16:23:28 -05:00
Gordon Shankman cd2f7bbde8 Adding support for SSE in the S3 storage backend. (#5996) 2019-01-26 16:48:08 -05:00
Jeff Mitchell 94e56d964f Fix build 2019-01-23 16:52:51 -05:00
Jeff Mitchell 3f1a7d4fdd
Update to latest etcd and use the new repository packages (#6087)
This will be necessary for go mod work

Additionally, the srv api has changed. This adapts to it.
2019-01-23 14:35:03 -05:00
Becca Petrin aac271ed7f swap the forked aliyun sdk for the original (#6024) 2019-01-23 11:24:51 -05:00
Jim Kalafut f5b19f9ce8
Fix physical/postgresql to return the full entry key (#6044)
Fixes #6041
2019-01-15 12:06:01 -08:00
Jim Kalafut d0e2badbae Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Julien Blache 91d432fc85 FoundationDB backend TLS support and housekeeping (#5800)
* Fix typo in documentation

* Update fdb-go-install.sh for new release tags

* Exclude FoundationDB bindings from vendoring, delete vendored copy

FoundationDB bindings are tightly coupled to the server version and
client library version used in a specific deployment. Bindings need
to be installed using the fdb-go-install.sh script, as documented in
the foundationdb backend documentation.

* Add TLS support to FoundationDB backend

TLS support appeared in FoundationDB 5.2.4, raising the minimum API version
for TLS-aware FoundationDB code to 520.

* Update documentation for FoundationDB TLS support
2019-01-08 09:01:44 -08:00
Jeff Mitchell c8a029210b
Don't read AWS env vars (#5974)
* Don't read AWS env vars

Let AWS SDK env cred chain provider do it for us

Fixes #5965
2019-01-04 15:03:57 -05:00
bjorndolk 4dbe61568a Physical postgres docker test (#5926)
* Docker support for postgres backend testing

* Bug in handling of postgres connection url for non docker testing

* Test should fail if it cannot retrieve pg version

* internal helperfunctions pascalCasing
2019-01-03 09:25:32 -05:00
Brian Kassouf 9d74c933cf
Update encoding.go 2018-11-19 15:24:41 -08:00
Brian Kassouf 33776b89c2
Wrap storage calls with encoding checks (#5819)
* Add encoding backend

* More work on encoding checks

* Update error message

* Update physical/encoding.go

* Disable key checks if configured
2018-11-19 13:13:16 -08:00
Konstantinos Tsanaktsidis f75e3603ba Paper over GCS backend corruption issues (#5804)
We're having issues with leases in the GCS backend storage being
corrupted and failing MAC checking. When that happens, we need to know
the lease ID so we can address the corruption by hand and take
appropriate action.

This will hopefully prevent any instances of incomplete data being sent
to GSS
2018-11-16 08:07:06 -05:00
Calvin Leung Huang 2824cb534d
Clean up HABackend tests (#5617) 2018-11-01 10:31:09 -07:00
Calvin Leung Huang a08ccbffa7
[Review Only] Autoseal OSS port (#757)
* Port awskms autoseal

* Rename files

* WIP autoseal

* Fix protobuf conflict

* Expose some structs to properly allow encrypting stored keys

* Update awskms with the latest changes

* Add KeyGuard implementation to abstract encryption/decryption of keys

* Fully decouple seal.Access implementations from sealwrap structs

* Add extra line to proto files, comment update

* Update seal_access_entry.go

* govendor sync

* Add endpoint info to configureAWSKMSSeal

* Update comment

* Refactor structs

* Update make proto

* Remove remove KeyGuard, move encrypt/decrypt to autoSeal

* Add rest of seals, update VerifyRecoveryKeys, add deps

* Fix some merge conflicts via govendor updates

* Rename SealWrapEntry to EncryptedBlobInfo

* Remove barrier type upgrade check in oss

* Add key to EncryptedBlobInfo proto

* Update barrierTypeUpgradeCheck signature
2018-10-19 14:43:57 -07:00
dle-fr f86c930855 [5529] Mysql HA: table creation error using InnoDB and utf8 charset (#5543) 2018-10-18 13:35:04 -04:00
Brian Kassouf f52cd4950f
Port some changes (#5518) 2018-10-15 14:06:45 -07:00
Calvin Leung Huang b47e648ddf
Logger cleanup (#5480) 2018-10-09 09:43:17 -07:00
Brian Kassouf 2995c06a53
Fix build (#5457) 2018-10-03 14:53:08 -07:00
Jeff Mitchell 2d908d6962 Fix compile on 32-bit platforms 2018-10-02 14:42:50 -04:00
Saurabh Pal 77e635f7e1 Enable TLS based communication with Zookeeper Backend (#4856)
* The added method customTLSDial() creates a tls connection to the zookeeper backend when 'tls_enabled' is set to true in config

* Update to the document for TLS configuration that is  required to enable TLS connection to Zookeeper backend

* Minor formatting update

* Minor update to the description for example config

* As per review comments from @kenbreeman, additional property description indicating support for multiple Root CAs in a single file has been added

* minor formatting
2018-10-01 14:12:08 -07:00
Becca Petrin 3da8d38e7d point at a fork of aliyun-oss-go-sdk (#5358) 2018-10-01 10:05:08 -07:00
Jeff Mitchell 33065a60db Fix compilation/protobuf 2018-09-22 17:58:39 -04:00
pavel-usov 470131d100 Change in GET_LOCK call to make it MariaDB compatilble (#5343)
* Update mysql.go

Change in GET_LOCK call to make it MariaDB compatilble

* Use math.MaxUint32 for lock timeout, add comments
2018-09-19 12:05:04 -07:00
Jeff Mitchell 919b968c27
The big one (#5346) 2018-09-17 23:03:00 -04:00
Jeff Mitchell ea08052120
Add an option to log all ops to inmem (#5306) 2018-09-07 17:35:46 -04:00
Jeff Mitchell 09d97b1eca
Cache negative results in physical cache (#5303) 2018-09-07 14:56:33 -04:00
Jeff Mitchell d57dfc1875 Move things back 2018-08-29 19:13:10 -04:00
Jeff Mitchell c6f7312f6c Move physical types around 2018-08-29 19:05:33 -04:00
Jeff Mitchell 05a896703c Move sort from listresponse to file backend to solve 5141 2018-08-24 13:37:15 -04:00
Oleksiy Kovyrin 1fe159ccb8 A simple copy-paste fix (#5130)
The test for MySQL HA backend seems to have been based on the Zookeeper one and the error message in it did not get updated to be MySQL-specific.
2018-08-17 08:46:56 -04:00
brianvans f79385346f Add ha_enabled for mysql backend (#5122)
* Slight cleanup around mysql ha lock implementation

* Removes some duplication around lock table naming
* Escapes lock table name with backticks to handle weird characters
* Lock table defaults to regular table name + "_lock"
* Drop lock table after tests run

* Add `ha_enabled` option for mysql storage

It defaults to false, and we gate a few things like creating the lock
table and preparing lock related statements on it
2018-08-16 11:03:16 -07:00
Christoph Ludwig 24a368c1ba Add support for "sovereign" Azure cloud environments (#4997)
* Add support for "sovereign" Azure cloud environments

* Shorten variable names
2018-08-15 19:40:36 -04:00
Seth Vargo 6dcecbdfda Fix Google Cloud races (#5081)
* storage/gcs: fix race condition in releasing lock

Previously we were deleting a lock without first checking if the lock we were deleting was our own. There existed a small period of time where vault-0 would lose leadership and vault-1 would get leadership. vault-0 would delete the lock key while vault-1 would write it. If vault-0 won, there'd be another leader election, etc.

This fixes the race by using a CAS operation instead.

* storage/gcs: properly break out of loop during stop

* storage/spanner: properly break out of loop during stop
2018-08-14 09:53:36 -04:00
Nándor István Krácser b9fab6375b Alibaba Object Storage support (#4783) 2018-08-13 17:03:24 -04:00
Michael Schuett 63e7ac034f MySQL HA Backend Support (#4686) 2018-08-13 17:02:31 -04:00
bohr 79b571d90d add backtick to fix mysql database name contain special characters (#5054)
when use mysql storage, set` database = "dev-dassets-bc"` , create database and create table will throw exceptions as follows:

    Error initializing storage of type mysql: failed to create mysql database: Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-dassets-bc' at line 1
    Error initializing storage of type mysql: failed to create mysql table: Error 1046: No database selected

cause of `-` is  a MySQL  built-in symbol. so add backtick for create database sql\create table sql \dml sqls.
2018-08-10 16:38:20 -07:00
Calvin Leung Huang b5c8b3d4c5 Clean up container on connection failure, switch to ory/dockertest on package postgresql (#5050) 2018-08-07 09:56:33 -07:00
Jeff Mitchell 4261618d10 Add request timeouts in normal request path and to expirations (#4971)
* Add request timeouts in normal request path and to expirations

* Add ability to adjust default max request duration

* Some test fixes

* Ensure tests have defaults set for max request duration

* Add context cancel checking to inmem/file

* Fix tests

* Fix tests

* Set default max request duration to basically infinity for this release for BC

* Address feedback
2018-07-24 14:50:49 -07:00
Chris Hoffman 45be1ee3e1
Read all pages when list results are paged (#4983) 2018-07-24 14:24:32 -04:00
Julien Blache c8fb9ed6a8 FoundationDB physical backend (#4900) 2018-07-16 10:18:09 -04:00
Richie Yeung 8fb804ecce Fix empty string check for password (#4923) 2018-07-13 12:35:06 -07:00
Seth Vargo 76d72a5e86 Use context from stdlib in google physical backends (#4922)
* Use context from stdlib in google physical backends

* Do not prefix logs (Vault will do it)
2018-07-13 11:00:38 -04:00
Jeff Mitchell b85a5da767 Change x/net context package to go stdlib 2018-07-13 10:59:34 -04:00
Jeff Mitchell db9abad3c9 Remove unnecessary log scoping in some physical backends 2018-07-13 10:57:30 -04:00
Jeff Mitchell 4b354e1110
Re-add dockertest and fix up imports and update script (#4909) 2018-07-11 17:49:13 -04:00
Md. Nure Alam Nahid 7b9bedf94d Add additional config keys for swift (#4901)
* Add additional config keys for swift

* Add additional swift config keys in the doc page
2018-07-11 08:29:29 -07:00
Jeff Mitchell 28890ee198 Make proto
It appears the only thing that actually change is that the tag got
`proto3` values added.
2018-07-10 20:49:48 -04:00
Jeff Mitchell e52b554c0b
Add an idle timeout for the server (#4760)
* Add an idle timeout for the server

Because tidy operations can be long-running, this also changes all tidy
operations to behave the same operationally (kick off the process, get a
warning back, log errors to server log) and makes them all run in a
goroutine.

This could mean a sort of hard stop if Vault gets sealed because the
function won't have the read lock. This should generally be okay
(running tidy again should pick back up where it left off), but future
work could use cleanup funcs to trigger the functions to stop.

* Fix up tidy test

* Add deadline to cluster connections and an idle timeout to the cluster server, plus add readheader/read timeout to api server
2018-06-16 18:21:33 -04:00
Jeff Mitchell 75eb0f862e
Revert some of commit 050ab805a7565c5b0cadb0176023031ee5f0d17b. (#4768)
If we have a panic defer functions are run but unlocks aren't. Since we
can't really trust plugins and storage, this backs out the changes for
those parts of the request path.
2018-06-14 13:44:13 -04:00