luxolus/open-vault
2
0
Fork 0
Code Issues 1 Pull requests Releases Activity
10390 commits 1 branch 0 tags 214 MiB
Commit graph

465 commits

Author SHA1 Message Date
Jeff Mitchell 858deb9ca4 Don't allow parent references in file paths 2017-05-12 13:52:33 -04:00
Jeff Mitchell e98690d00c Ensure we aren't leaking any open FDs in the file backend if we hit certain error conditions 2017-05-09 09:24:43 -04:00
Chris Hoffman 847c86f788 Rename ParseDedupAndSortStrings to ParseDedupLowercaseAndSortStrings (#2614) 2017-04-19 10:39:07 -04:00
Jeff Mitchell 30af63c881 Fix azure test round 2 2017-04-17 14:52:52 -04:00
Jeff Mitchell 8cf0cd8cd2 Fix test for changed Azure 2017-04-17 13:18:34 -04:00
Jeff Mitchell e1e78b1409 Update to new Azure code after dep update (#2603) 2017-04-17 12:15:12 -04:00
Mevan Samaratunga 3b2c42f6dd Added "Domain" configuration parameter to Swift provider to enable V3 authentication (#2554) 2017-04-17 11:59:44 -04:00
Sebastian Haba 3322f637ac add mssql physical backend (#2546) 2017-04-06 09:33:49 -04:00
Jonathan Sokolowski a4ceaf0035 Etcd DNS discovery (#2521) 
* etcd: Add discovery_srv option
 2017-04-04 08:50:44 -07:00
VladV 1d4c901aeb Fix state change notification channels (#2548) 2017-03-31 09:01:55 -07:00
Vishal Nayak b9aa56c17e s3: use pooled transport for http client (#2481) 2017-03-29 10:27:27 -07:00
Marshall Brekka 1a73923a21 Etcd3: Write lock item with lease to ensure release on bad shutdown (#2526) 2017-03-28 11:08:41 -04:00
Dan Everton 4ef8ce1198 Add permitPool support to S3 (#2466) 2017-03-26 14:32:26 -04:00
Jeff Mitchell ff3c3db91b Have Consul's transaction handler use the permit pool 2017-03-09 12:59:42 -05:00
Jeff Mitchell 3d162b63cc Use locks in a slice rather than a map, which is faster and makes things cleaner (#2446) 2017-03-07 11:21:32 -05:00
Jeff Mitchell f5ffa229f4 Switch physical cache map index value to md5 from sha1 for all the performances 2017-03-06 13:11:14 -05:00
Jeff Mitchell 27399aeb7a Fix dynamo test that can double close a channel 2017-03-04 16:59:00 -05:00
Jeff Mitchell 111fbc5747 Make cache not actually cache values under core/ (#2439) 2017-03-03 16:04:31 -05:00
Marshall Brekka 184b47e20c Add a TTL to the dynamodb lock implementation. (#2141) 2017-02-27 14:30:34 -05:00
Jeff Mitchell 2cc0906b33 Fix breakage for HTTP2 support due to changes in wrapping introduced in 1.8 (#2412) 2017-02-27 12:49:35 -05:00
Jeff Mitchell 41ae5d14ce Add pseudo transactional test 2017-02-20 11:40:36 -05:00
Jeff Mitchell 4305900a64 Add faultPseudo for testing 2017-02-20 11:08:03 -05:00
Jeff Mitchell 3230f697bd Final rep porting (#2392) 2017-02-17 09:15:35 -05:00
Colin Arenz 99b01a3d82 Fix listing of deep paths in PostgreSQL backend (#2393) 
This change addresses an issue where deep paths would not be enumerated if parent paths did not contain a key.

Given the keys `shallow` and `deep` at the following paths...
```
secret/shallow
secret/path/deep
```

... a `LIST` request against `/v1/secret` would produce only one result, `shallow`.  With this change, the same list request will now list `shallow` and `path/`.
 2017-02-17 09:14:11 -05:00
Brian Kassouf 13ec9c5dbf Load leases into the expiration manager in parallel (#2370) 
* Add a benchmark for exiration.Restore

* Add benchmarks for consul Restore functions

* Add a parallel version of expiration.Restore

* remove debug code

* Up the MaxIdleConnsPerHost

* Add tests for etcd

* Return errors and ensure go routines are exited

* Refactor inmem benchmark

* Add s3 bench and refactor a bit

* Few tweaks

* Fix race with waitgroup.Add()

* Fix waitgroup race condition

* Move wait above the info log

* Add helper/consts package to store consts that are needed in cyclic packages

* Remove not used benchmarks
 2017-02-16 10:16:06 -08:00
Xiang Li 220930f539 etcdbackend: support version auto discovery (#2299) 2017-01-26 17:19:13 -05:00
Jeff Mitchell f856963706 Revert file backend base64ing, as we need to fix a pathological case for some keys 2017-01-25 12:27:18 -05:00
Brian Kassouf d6198b7e24 change consistency config value from a bool to a string (#2282) 2017-01-19 17:36:33 -05:00
Vishal Nayak bb1f28ce66 Merge pull request #2203 from hashicorp/file-backend-base64 
Base64 encode the file names in the 'file' physical backend
 2017-01-19 10:10:57 -05:00
Brian Kassouf 6aa097b727 Add require consistent flag to Consul Lock 2017-01-13 12:22:14 -08:00
Brian Kassouf ce6fa6b30e Add test for require_consistency option 2017-01-13 10:24:40 -08:00
Brian Kassouf fb19c81f62 add a option for strong consistancy for consul 2017-01-13 09:49:04 -08:00
vishalnayak 5aeb276018 correcting the error statement 2017-01-13 03:58:46 -05:00
vishalnayak 76a456cc97 file: correct the old entry check 2017-01-13 03:51:09 -05:00
vishalnayak 8b579d47a9 address review feedback 2017-01-13 03:39:33 -05:00
vishalnayak d2026364c7 physical/file: added test for base64 encoding the storage file names 2017-01-13 01:00:25 -05:00
vishalnayak cbccf9869d physical/file: Handle file duplication case while updating 2017-01-13 01:00:25 -05:00
vishalnayak 17652b486d physical/file: Fix the deletion flow 2017-01-13 01:00:25 -05:00
vishalnayak a952d324fe physical: file backend to have key base64 URL encoded 2017-01-13 01:00:25 -05:00
Adam Shannon 6fc53dc135 physical/zk: Ignore ErrNoNode when deleting znodes (#2256) 2017-01-11 09:42:30 -05:00
Jeff Mitchell 80dc5819d3 Use dockertest.v2 (#2247) 
New dockertest has a totally different API and will require some serious
refactoring. This will tide over until then by pinning the API version.
 2017-01-09 13:46:54 -05:00
Jeff Mitchell 9e5d1eaac9 Port some updates 2017-01-06 15:42:18 -05:00
Xiang Li 2faa3f5764 etcd3: remove wrong keys checking for prefix request (#2231) 2017-01-05 07:48:46 -05:00
Xiang Li 02070e0fc6 physical: add etcd3 backend (#2168) 2017-01-03 14:43:46 -05:00
Dan Everton e4a1f5a3bb Page results from S3. (#2224) 
S3 results require paging to ensure that all results are returned. This
PR changes the S3 physical backend to use the new ListObjectV2 method
and pages through all the results.

Fixes #2223.
 2017-01-03 11:15:48 -05:00
Jeff Mitchell f9c6fc2e6b Actually give the logger to inmem backend 2016-12-15 15:48:51 -05:00
Christopher Pauley f07a19c503 gcs physical backend (#2099) 2016-12-01 11:42:31 -08:00
Jeff Mitchell 736a4b111c Add some commenting to PermitPool 2016-11-28 18:34:58 -05:00
Kevin Goess 33bf26f320 check for failure on that mysql query (#2105) 2016-11-17 09:59:27 -05:00
Jeff Mitchell 9066f012a7 Fix cache default size and docs 2016-11-01 10:24:35 -04:00
Jeff Mitchell 9d5462ca04 Don't cache physical responses when thre was an error (#2040) 2016-10-28 12:55:56 -04:00
Paweł Rozlach 33b4683dfd Post-review fixes for file/zk recursive empty prefix delete 2016-10-05 08:08:00 -04:00
Pawel Rozlach 41ade15f73 Fix file backend so that it properly removes nested secrets. 
This patch makes file backend properly remove nested secrets, without leaving
empty directory artifacts, no matter how nested directories were.
 2016-10-04 21:56:12 +02:00
Pawel Rozlach 44b4704cfa Fix zookeeper backend so that properly deletes/lists secrets. 
This patch fixes two bugs in Zookeeper backends:
 * backend was determining if the node is a leaf or not basing on the number
   of the childer given node has. This is incorrect if you consider the fact
   that deleteing nested node can leave empty prefixes/dirs behind which have
   neither children nor data inside. The fix changes this situation by testing
   if the node has any data set - if not then it is not a leaf.
 * zookeeper does not delete nodes that do not have childern just like consul
   does and this leads to leaving empty nodes behind. In order to fix it, we
   scan the logical path of a secret being deleted for empty dirs/prefixes and
   remove them up until first non-empty one.
 2016-10-04 21:56:12 +02:00
Pawel Rozlach 68fc52958d Add tests for nested/prefixed secrets removal. 
Current tests were not checking if backends are properly removing
nested secrets. We follow here the behaviour of Consul backend, where
empty "directories/prefixes" are automatically removed by Consul itself.
 2016-10-04 21:55:33 +02:00
Jeff Mitchell 226ef5d78c Make HA in etcd off by default. (#1909) 
Fixes #1908

(Doesn't really "fix" it but someone from the community needs to step up
if they want to see this fixed.)
 2016-09-21 14:01:36 -04:00
James Pearson Hughes f598c78d98 DynamoDB: fix log typo (#1891) 2016-09-14 15:16:24 -04:00
Jeff Mitchell ffaaacd029 Have file backend remove empty dirs. (#1821) 
Add tests to check that prefixes are being properly removed (or at
least, not listed) from backends.
 2016-08-31 14:12:28 -04:00
Jeff Mitchell 2ce4397deb Plumb through the ability to set the storage read cache size. (#1784) 
Plumb through the ability to set the storage read cache size.

Fixes #1772
 2016-08-26 10:27:06 -04:00
Jeff Mitchell 58b32e5432 Convert to logxi 2016-08-21 18:13:37 -04:00
Jeff Mitchell 2860dcc60f gofmt 2016-08-19 16:48:32 -04:00
Jeff Mitchell 734e80ca56 Add permit pool to dynamodb 2016-08-15 19:45:06 -04:00
Jeff Mitchell dcba6129e3 Use dockertest for physical consul tests, and always run them 2016-08-15 16:20:32 -04:00
Jeff Mitchell 37320f8798 Request forwarding (#1721) 
Add request forwarding.
 2016-08-15 09:42:42 -04:00
Jeff Mitchell 9e204bd88c Add arbitrary string slice parsing. 
Like the KV function, this supports either separated strings or JSON
strings, base64-encoded or not.

Fixes #1619 in theory.
 2016-08-03 14:24:16 -04:00
Jeff Mitchell c025b292b5 Cleanup 2016-08-03 13:09:12 -04:00
vishalnayak 91e60a5824 Fixed the test after removing shutdown bool 2016-08-01 12:20:38 -04:00
Jeff Mitchell 6ffefb649d Close the shutdown channel instead of sending a value down 2016-08-01 11:58:45 -04:00
vishalnayak 05b8ce8348 Address review feedback 2016-08-01 11:15:25 -04:00
vishalnayak 21d155f4af Fix physical/consul test case 2016-08-01 10:55:47 -04:00
vishalnayak 5ed10f4074 Make the defer statement of waitgroup to execute last 2016-08-01 10:24:27 -04:00
vishalnayak ea2e677f02 Sharing shutdown message with physical consul backend 2016-07-31 10:09:16 -04:00
vishalnayak a8b4fc0d3c Add waitgroup wait to allow physical consul to deregister checks 2016-07-30 13:17:29 -04:00
vishalnayak 32a72e84dc Added test for service tags 2016-07-22 09:04:42 -04:00
vishalnayak 58bd985551 Address review feedback from @jefferai 2016-07-22 08:44:16 -04:00
vishalnayak 9bfb518e3f Make debug statements specify that the logs are for physical/consul 2016-07-22 07:03:14 -04:00
vishalnayak 248889dd1b Fix broken test 2016-07-22 06:42:56 -04:00
vishalnayak 765d131b47 Added service-tags config option to provide additional tags to registered service 2016-07-22 04:41:48 -04:00
vishalnayak bd8ff10462 Address review feedback from @sean 2016-07-21 19:04:43 -04:00
vishalnayak 6872df833a Uniquify the check ID 2016-07-19 14:17:50 -04:00
vishalnayak f54dc7a31e Set QueryOptions while fetching service information from Consul catalog 2016-07-19 14:07:06 -04:00
vishalnayak 11e6fe0fb4 Fix tests 2016-07-19 07:58:47 -04:00
vishalnayak ac7ecbce5c Fixed re-registrations and health-check flatenning issue 2016-07-19 07:06:41 -04:00
Jeff Mitchell c47fc73bd1 Use parsebool 2016-07-18 13:49:05 -04:00
Jeff Mitchell a3ce0dcb0c Turn off DynamoDB HA by default. 
The semantics are wonky and have caused issues from people not reading
docs. It can be enabled but by default is off.
 2016-07-18 13:19:58 -04:00
vishalnayak c14235b206 Merge branch 'master-oss' into json-use-number 
Conflicts:
	http/handler.go
	logical/framework/field_data.go
	logical/framework/wal.go
	vault/logical_passthrough.go
 2016-07-15 19:21:55 -04:00
vishalnayak 407722a9b4 Added tls_min_version to consul storage backend 2016-07-12 20:10:54 -04:00
vishalnayak f34f0ef503 Make 'tls_min_version' configurable 2016-07-12 19:32:47 -04:00
vishalnayak 46d34130ac Set minimum TLS version in all tls.Config objects 2016-07-12 17:06:28 -04:00
vishalnayak ad7cb2c8f1 Added JSON Decode and Encode helpers. 
Changed all the occurances of Unmarshal to use the helpers.
Fixed http/ package tests.
 2016-07-06 12:25:40 -04:00
Tim Schindler b00535bcf5 fixed typo: explitely -> explicitly 2016-06-30 19:10:15 +00:00
Tim Schindler 23f08a139c os.GetEnv -> os.Getenv 2016-06-30 18:53:55 +00:00
Tim Schindler 6a1c142924 added ETCD_ADDR env var to etcd backend 2016-06-30 18:46:00 +00:00
Jeff Mitchell e7a1e95037 Merge pull request #1548 from quixoten/nopreparepsql 
Remove prepared stmnts from pgsql physical backend
 2016-06-28 16:38:13 +02:00
Jeff Mitchell b8c30aea18 Merge pull request #1502 from hashicorp/pr-1425 
Staging area for me to fix up PR 1425
 2016-06-08 12:31:31 -04:00
Jeff Mitchell 72a25d018c Add permit pool and cleanhttp support to Swift 2016-06-08 12:20:21 -04:00
Jeff Mitchell da6371ffc3 Merge remote-tracking branch 'origin/master' into pr-1425 2016-06-08 12:10:29 -04:00
Jeff Mitchell 3bf06b47e4 Add permitPool support to Azure 2016-06-08 12:01:43 -04:00
Bill Monkman de8477244e #1486 : Fixed sealed and leader checks for consul backend 2016-06-03 16:00:31 -07:00
Devin Christensen b0f50ecb6c Remove prepared stmnts from pgsql physical backend 
Prepared statements prevent the use of connection multiplexing software
such as PGBouncer. Even when PGBouncer is configured for [session mode][1]
there's a possibility that a connection to PostgreSQL can be re-used by
different clients.  This leads to errors when clients use session based
features (like prepared statements).

This change removes prepared statements from the PostgreSQL physical
backend. This will allow vault to successfully work in infrastructures
that employ the use of PGBouncer or other connection multiplexing
software.

[1]: https://pgbouncer.github.io/config.html#poolmode
 2016-05-26 17:07:21 -06:00
Stuart Glenn 1fea2799a5 Add backend for OpenStack Swift 2016-05-16 17:29:23 -05:00
Sean Chittenden 7a4b31ce51
Speling police 2016-05-15 09:58:36 -07:00
Sean Chittenden 99a5213f0b Merge pull request #1355 from hashicorp/f-vault-service 
Vault/Consul Service refinement
 2016-05-12 11:48:29 -07:00
Joe Hillenbrand 3b14f5043f Fix default etcd address 
Should be `127.0.0.1`, not `128.0.0.1`
 2016-05-10 12:50:11 -07:00
Jeff Mitchell 1b0df1d46f Cleanups, add shared provider, ability to specify http client, and port S3 physical backend over 2016-05-03 17:01:02 -04:00
Sean Chittenden 7e5dbf409e Be idiomatic. Use a switch instead of if/else 2016-04-29 11:35:33 -07:00
Sean Chittenden 614104717c Remove useless return statement 2016-04-28 13:16:17 -07:00
Sean Chittenden 93ba3a0b8a Refactor the Consul Backend to match evented demuxer 2016-04-28 11:05:18 -07:00
Sean Chittenden e129273e4f Debug log consul configuration parameters when set 2016-04-28 11:05:18 -07:00
Sean Chittenden 0b72906fc3 Change the interface of ServiceDiscovery 
Instead of passing state, signal that the state has changed and provide a callback handler that can query Core.
 2016-04-28 11:05:18 -07:00
Sean Chittenden cc64778628 Fix logger output 
Pointed out by: ryanuber
 2016-04-28 11:05:18 -07:00
Jeff Mitchell e5c31d66a2 Better handle nil responses in S3 backend, also a case where error wasn't checked 2016-04-26 08:11:56 -04:00
Sean Chittenden 557d8b8a24 Make use of logger interface inside of the Consul BE 2016-04-25 20:10:55 -07:00
Sean Chittenden aeea7628d6 Add a *log.Logger argument to physical.Factory 
Logging in the backend is a good thing.  This is a noisy interface change but should be a functional noop.
 2016-04-25 20:10:32 -07:00
Sean Chittenden 5fd5869bc5 Rewriting history before it gets away from me 2016-04-25 18:05:50 -07:00
Sean Chittenden 9b8095d7ea Change to the pre-0.6.4 Consul Check API 
Consul is never going to pass in more than 1K of output.  This mitigates the pre-0.6.4 concern.
 2016-04-25 18:01:13 -07:00
Sean Chittenden f5183fa506 Collapse UpdateAdvertiseAddr() into RunServiceDiscovery() 2016-04-25 18:01:13 -07:00
Sean Chittenden 5104c58c54 Update tests to chase sealed -> unsealed transition 2016-04-25 18:01:13 -07:00
Sean Chittenden 7fe0b2c6a1 Persistently retry to update service registration 
If the local Consul agent is not available while attempting to step down from active or up to active, retry once a second.  Allow for concurrent changes to the state with a single registration updater.  Fix standby initialization.
 2016-04-25 18:01:13 -07:00
Sean Chittenden 3449fa1bc3 Consistently skip Consul checks 
Hide all Consul checks behind `CONSUL_HTTP_ADDR` env vs `CONSUL_ADDR` which is non-standard.
 2016-04-25 18:01:13 -07:00
Sean Chittenden 60006f550f Various refactoring to clean up code organization 
Brought to you by: Dept of 2nd thoughts before pushing enter on `git push`
 2016-04-25 18:01:13 -07:00
Sean Chittenden 53f9cea87c Compare the correct values when validating check_timeout 2016-04-25 18:01:13 -07:00
Sean Chittenden 70ae7f73b4 Detect type conversion failure 2016-04-25 18:01:13 -07:00
Sean Chittenden ae66e65bcf Don't export the builtin backends 2016-04-25 18:01:13 -07:00
Sean Chittenden bd3335c1bd go fmt the PostgreSQL backend 2016-04-25 18:01:13 -07:00
Sean Chittenden 6b2c83564e Teach Vault how to register with Consul 
Vault will now register itself with Consul.  The active node can be found using `active.vault.service.consul`.  All standby vaults are available via `standby.vault.service.consul`.  All unsealed vaults are considered healthy and available via `vault.service.consul`.  Change in status and registration is event driven and should happen at the speed of a write to Consul (~network RTT + ~1x fsync(2)).

Healthy/active:

```
curl -X GET 'http://127.0.0.1:8500/v1/health/service/vault?pretty' && echo;
[
    {
        "Node": {
            "Node": "vm1",
            "Address": "127.0.0.1",
            "TaggedAddresses": {
                "wan": "127.0.0.1"
            },
            "CreateIndex": 3,
            "ModifyIndex": 20
        },
        "Service": {
            "ID": "vault:127.0.0.1:8200",
            "Service": "vault",
            "Tags": [
                "active"
            ],
            "Address": "127.0.0.1",
            "Port": 8200,
            "EnableTagOverride": false,
            "CreateIndex": 17,
            "ModifyIndex": 20
        },
        "Checks": [
            {
                "Node": "vm1",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "CreateIndex": 3,
                "ModifyIndex": 3
            },
            {
                "Node": "vm1",
                "CheckID": "vault-sealed-check",
                "Name": "Vault Sealed Status",
                "Status": "passing",
                "Notes": "Vault service is healthy when Vault is in an unsealed status and can become an active Vault server",
                "Output": "",
                "ServiceID": "vault:127.0.0.1:8200",
                "ServiceName": "vault",
                "CreateIndex": 19,
                "ModifyIndex": 19
            }
        ]
    }
]
```

Healthy/standby:

```
[snip]
        "Service": {
            "ID": "vault:127.0.0.2:8200",
            "Service": "vault",
            "Tags": [
                "standby"
            ],
            "Address": "127.0.0.2",
            "Port": 8200,
            "EnableTagOverride": false,
            "CreateIndex": 17,
            "ModifyIndex": 20
        },
        "Checks": [
            {
                "Node": "vm2",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "CreateIndex": 3,
                "ModifyIndex": 3
            },
            {
                "Node": "vm2",
                "CheckID": "vault-sealed-check",
                "Name": "Vault Sealed Status",
                "Status": "passing",
                "Notes": "Vault service is healthy when Vault is in an unsealed status and can become an active Vault server",
                "Output": "",
                "ServiceID": "vault:127.0.0.2:8200",
                "ServiceName": "vault",
                "CreateIndex": 19,
                "ModifyIndex": 19
            }
        ]
    }
]
```

Sealed:

```
        "Checks": [
            {
                "Node": "vm2",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "CreateIndex": 3,
                "ModifyIndex": 3
            },
            {
                "Node": "vm2",
                "CheckID": "vault-sealed-check",
                "Name": "Vault Sealed Status",
                "Status": "critical",
                "Notes": "Vault service is healthy when Vault is in an unsealed status and can become an active Vault server",
                "Output": "Vault Sealed",
                "ServiceID": "vault:127.0.0.2:8200",
                "ServiceName": "vault",
                "CreateIndex": 19,
                "ModifyIndex": 38
            }
        ]
```
 2016-04-25 18:01:13 -07:00
Sean Chittenden 230b59f34c Stub out service discovery functionality 
Hook asynchronous notifications into Core to change the status of vault based on its active/standby, and sealed/unsealed status.
 2016-04-25 18:00:54 -07:00
Jeff Mitchell f00beb4e32 Update azure backend for newer sdk 2016-04-26 00:08:07 +00:00
Jeff Mitchell a481bff2b1 Fix commenting S3 -> Azure 2016-04-25 19:53:07 +00:00
Tobias Haag 175e3cc354 added Azure backend support 
updated Godeps
added website docs
updated vendor
 2016-03-30 19:49:38 -07:00
Jeff Mitchell deed5cc121 Output original error on etcd sync failure. 
Fixes #1141
 2016-02-26 15:15:23 -05:00
Jeff Mitchell 50d3b68c8d Merge pull request #1078 from eyal-lupu/master 
ZooKeeper Backend: Authnetication and Authorization support
 2016-02-19 15:13:09 -05:00
Eyal Lupu a6e9820e8d typo in comment 2016-02-19 13:28:02 +00:00
Eyal Lupu 23303429c0 'Eagerly' parse ZK authentication and authorization to fast-fail bad configuration 2016-02-19 13:24:57 +00:00
Eyal Lupu c7fe99b1e9 1. gofmt 
2. Change if expr syntax to be consist with the rest of Vault code
3. More details on error message
 2016-02-19 12:19:01 +00:00
Jeff Mitchell 5edaf522a8 Use a pooled transport for the Consul physical backend and give it 4 idle connections 2016-02-17 16:53:30 -05:00
Eyal Lupu e9c7a02850 https://github.com/hashicorp/vault/issues/1058 
Make sure locks are also using the same auth info as data
 2016-02-15 15:29:08 +00:00
Eyal Lupu d4db2ea79c fixes to https://github.com/hashicorp/vault/issues/1058 
Configuration now supports:
- auth_info
-znode_owner
 2016-02-15 15:03:12 +00:00
Devin Christensen 4112809fb5 Make the PostgreSQL backend more performant 2016-01-29 13:47:10 -07:00
Jeff Mitchell 68dc0e2dd3 Merge pull request #945 from quixoten/postgres_physical 
Add support for PostgreSQL as a physical backend
 2016-01-29 10:35:38 -05:00
Devin Christensen 737df30939 Improve naming 
Hopefully this naming scheme will be more straightforward.
 2016-01-27 17:15:48 -07:00
Jeff Mitchell b7a49922a9 Update etcd sync option to be a string. 
Ping #921
 2016-01-27 17:15:52 -05:00
Jeff Mitchell b0bd06f5a4 Merge pull request #921 from faradayio/hosted-etcd-support 
Load-balanced etcd support
 2016-01-27 17:09:43 -05:00
Devin Christensen 9d776351a3 Merge 'upstream/master' into postgres_physical 2016-01-22 20:56:07 -07:00
Devin Christensen c226b0be7d Update naming and pull DDL for upsert back out 2016-01-22 17:15:10 -07:00
Devin Christensen 32b712ddb1 Move the upsert definition back into the code 2016-01-22 09:47:02 -07:00
Devin Christensen bfbdc72e03 Remove options for column configuration 2016-01-22 08:41:31 -07:00
Jeff Mitchell be1b4c8a46 Only allow listing on folders and enforce this. Also remove string sorting from Consul backend as it's not a requirement and other backends don't do it. 2016-01-22 10:07:32 -05:00
Devin Christensen 06641570c7 Remove DDL statements from the code 2016-01-20 18:52:49 -07:00
Devin Christensen bcc720be11 Remove superfluous comparison 2016-01-20 17:05:21 -07:00
Devin Christensen 65bd200fae Ensure rows.Close() is called in List 2016-01-20 17:02:23 -07:00
Devin Christensen 614f7b7157 Prefer TEXT over VARCHAR 
From the PostgreSQL docs
(http://www.postgresql.org/docs/9.4/static/datatype-character.html):

 > Tip: There is no performance difference among these three types,
 > apart from increased storage space when using the blank-padded type,
 > and a few extra CPU cycles to check the length when storing into a
 > length-constrained column. While character(n) has performance
 > advantages in some other database systems, there is no such advantage
 > in PostgreSQL; in fact character(n) is usually the slowest of the
 > three because of its additional storage costs. In most situations
 > text or character varying should be used instead.
 2016-01-20 16:56:46 -07:00
Devin Christensen b4e9e204f7 Use native upsert when available 2016-01-20 10:47:54 -07:00
Devin Christensen fc94487f55 Add support for PostgreSQL as a physical backend 2016-01-19 17:00:09 -07:00
Eric Kidd 69434fd13e etcd: Allow disabling sync for load balanced etcd 
Some etcd configurations (such as that provided by compose.io) place the
etcd cluster behind multiple load balancers or proxies.  In this
configuration, calling Sync (or AutoSync) on the etcd client will
replace the load balancer addresses with the underlying etcd server
address.

This will cause the etcd client to bypass the load balancers, and may
cause the connection to fail completely if the etcd servers are
protected by a firewall.

This patch provides a "sync" option for the etcd backend, which defaults
to the current behavior, but which can be used to turn off of sync.
This corresponds to etcdctl's --no-sync option.
 2016-01-11 13:56:58 -05:00
Paul Seiffert 99f7659bb4 Add recovery option to DynamoDB backend 
When Vault is killed without the chance to clean up the lock
entry in DynamoDB, no further Vault nodes can become leaders after
that.

To recover from this situation, this commit adds an environment
variable and a configuration flag that when set to "1" causes Vault
to delete the lock entry from DynamoDB.
 2016-01-08 17:31:37 +01:00
Paul Seiffert 8853e50691 Explicitly read AWS credentials from environment 2016-01-08 17:31:37 +01:00
Paul Seiffert 277de77256 Add tests for DynamoDB backend 2016-01-08 17:31:37 +01:00
Paul Seiffert 870bc6c5b4 Implement DynamoDB physical HA backend 2016-01-08 17:31:37 +01:00
Jeff Mitchell 287954beef Replace physical cache with TwoQueue instead of LRU. 2016-01-07 09:21:33 -05:00
Jeff Mitchell bf2bf06997 Use cleanhttp.DefaultTransport rather than instantiating directly to avoid leaked FDs 2015-12-17 15:23:13 -05:00
Jeff Mitchell ade5bf0570 Make S3 act like other parts of vault by prioritizing environment 
variables over configuration values.
 2015-12-17 10:19:42 -05:00
Chi Vinh Le a090caf2c3 Basic Auth support for Etcd. 
Fixes #859
 2015-12-17 12:50:10 +01:00
Jeff Mitchell 5a1ea272ce Merge pull request #857 from hashicorp/issue-836 
Use an initialized client when using IAM roles with S3 physical backend
 2015-12-14 21:25:41 -05:00
Jeff Mitchell b2a0b48a2e Add test to ensure the right backend was used with separate HA 2015-12-14 20:48:22 -05:00
Jeff Mitchell 352bff96c8 Pass in an initialized client into EC2RoleProvider. 
Fixes #836
 2015-12-14 11:14:09 -05:00
Vicki Cheung 5c334293cd fixing etcd missing key error 2015-12-07 02:29:20 -05:00
Jeff Mitchell 3bdbd66f7d Remove datacenter from Consul configuration, as it cannot actually do 
anything

Fixes #816
 2015-12-03 15:16:37 -05:00
Jeff Mitchell 69b522f3ea Add new Consul API client MonitorRetries option 2015-12-01 00:08:14 -05:00
Jeff Mitchell 4a1a02a123 Merge pull request #780 from vicki-c/master 
Port to new etcd client with TLS support
 2015-11-18 10:33:09 -05:00
Vicki Cheung eb464ed79d rejecting etcd addresses without url scheme 2015-11-17 15:18:50 -08:00
Vicki Cheung 4a3bcc2adc adding check in etcd backend to validate machine urls 2015-11-16 14:35:04 -08:00
Vicki Cheung dfe284af43 adding PermitPool to etcd backend 2015-11-15 22:38:21 -08:00
Vicki Cheung a21c8fab26 porting to new etcd client 2015-11-15 22:12:06 -08:00
Yannick 8a594a7f61 Allow s3 bucket to come from config vars 2015-11-06 14:05:29 +01:00
Greg Brockman 141a71974a Correct typo in comment 2015-11-06 00:41:14 -08:00
Greg Brockman 171bd84330 Add support for etcd over TLS 2015-11-06 00:41:14 -08:00
Jeff Mitchell 08dbc70c9f Switch etcd default port to 2379, in line with 2.x. 
Fixes #753
 2015-11-05 09:47:50 -05:00
Jeff Mitchell 9fff3a350d Don't use the semaphore library as it's racy; instead use a simple 
buffered channel. Passes all tests, including inmem, which uses it.
 2015-11-04 12:27:13 -05:00
Sander van Harmelen 4ad533a5ba Add a line to the documentation to describe the new feature 2015-11-04 15:36:24 +01:00
Sander van Harmelen c65b63d152 Add an option to configure the S3 endpoint 
This enables the use of other (AWS S3 compatible) S3 endpoints.
 2015-11-04 15:04:36 +01:00
Jeff Mitchell 7f44a1b812 Add configuration parameter for max parallel connections to Consul 2015-11-03 15:26:07 -05:00
Jeff Mitchell 1b83eefd97 Address review feedback 2015-11-03 14:48:05 -05:00
Jeff Mitchell bf2e553785 Add a PermitPool to physical and consul/inmem 
The permit pool controls the number of outstanding operations that can
be queued for Consul (and inmem, for testing purposes). This prevents
possible situations where Vault launches thousands of concurrent
connections to Consul if e.g. a huge number of leases need to be
expired.

Fixes #677
 2015-11-03 11:49:20 -05:00
Seth Vargo 658bc0634a Fix breaking API changes 2015-10-30 18:22:48 -04:00
Jeff Mitchell cba4e82682 Don't use http.DefaultClient 
This strips out http.DefaultClient everywhere I could immediately find
it. Too many things use it and then modify it in incompatible ways.

Fixes #700, I believe.
 2015-10-15 17:54:00 -04:00
Tuomas Silen 5e8b3a28e4 Rename error return var 2015-09-15 11:18:43 +03:00
Tuomas Silen 42d3f90e37 Further cleanup, use named return vals 2015-09-14 13:30:15 +03:00
Tuomas Silen 7f384b2312 Cleanup defer func 2015-09-11 16:30:12 +03:00
Tuomas Silen 2652db825a Use defer to close the channel in case of error 2015-09-11 16:17:23 +03:00
Tuomas Silen f8ec771cec Renew the semaphore key periodically 
The semaphore key is used to determine whether we are the leader or not and is set to expire after TTL of 15 seconds. There was no logic implemented to renew the key before it expired, which caused the leader to step down and change every 15 seconds. A periodic timer is now added to update the key every 5 seconds to renew the TTL of the key.
 2015-09-09 19:33:07 +03:00
Jeff Mitchell 9f2f79cdf4 Fix tests with AWS changes. 2015-08-18 19:22:17 -07:00
Armon Dadgar 4c84080732 physical/s3: update for new AWS API 2015-08-17 12:19:55 -07:00
Daniel Rampelt 83ce6f2e70 Use varbinary instead of varchar for mysql, fixes #512 2015-08-11 15:03:10 -04:00
Paul Hinze fc9de56736 Update vault code to match latest aws-sdk-go APIs 2015-08-06 11:37:08 -05:00
Armon Dadgar f58f46c243 Merge pull request #439 from geckoboard/feature-tls-mysql 
Using SSL to encrypt connections to MYSQL
 2015-08-05 14:52:43 -07:00
Vivien Schilis 2a1dfdab4e Naming cleanup 2015-07-29 20:19:21 +00:00
Daniel Kaffee a5ad818d8e only use NewCertPool if there is a ca cert otherwise use host's certificates 2015-07-28 15:31:30 +03:00
Daniel Kaffee 280fec2913 fix potential insecure skip verification bug 2015-07-28 15:15:31 +03:00
Daniel Kaffee 7b743f12fe fix identification to go formatting 2015-07-28 15:06:56 +03:00
Daniel Kaffee 4146be770c refactor code 2015-07-28 14:55:33 +03:00
Lauro Balderas 9a51ca341b Granting S3 backend temporary access 2015-07-18 16:48:23 +10:00
Vivien Schilis f16a09dc48 Add tls.Config if sslca is provided 2015-07-17 22:33:06 +00:00
Armon Dadgar 26937498f6 physical/zk: Fixing node representation. Fixes #416 2015-07-13 19:33:23 +10:00
Armon Dadgar bfc0442750 physical/zk: remove recursive delete behavior, still broken 2015-07-13 19:05:17 +10:00
Armon Dadgar 29a5eb35f9 physical: ensure backend does NOT do recursive delete 2015-07-13 18:59:40 +10:00
Joe Beda 49b84db4a9 Fix zookeeper break. 
Fixes #393.
 2015-07-04 16:03:02 -07:00
Armon Dadgar 28ddff305c physical/mysql: cleanup and documentation 2015-06-18 14:31:00 -07:00
Pradeep Chhetri 53748c8c63 Fixed a failing test and drop table after running tests 2015-06-13 08:24:27 +05:45
Pradeep Chhetri 5fe59f4b8d Fixing List command behaviour 2015-06-12 23:16:46 +05:45
Pradeep Chhetri 0bf52546af Added the test as per suggestion 2015-06-12 15:32:45 +05:45
Pradeep Chhetri 30cef9fe77 Changes done as per feedback 2015-06-12 13:24:41 +05:45
Pradeep Chhetri ace36da4ce Physical MySQL backend implementation - First Cut 2015-06-09 01:37:25 +05:45
Seth Vargo a02f62ee77 AWS moved from labs to official 2015-06-03 15:02:49 -04:00
Eric Buth e2957ef463 etcd HA physical backend: added documention + style updates 2015-06-02 18:00:06 -04:00
Eric Buth 8c78cdddb1 etcd HA physical backend: stopchannel style, held state remote-only, lock value stored in semaphore key 2015-06-02 13:18:55 -04:00
Eric Buth baaa9bd10c etcd HA + tests 2015-06-01 18:29:54 -04:00
Armon Dadgar 9b79d43370 Merge pull request #252 from kenbreeman/physical_zookeeper_ha 
Physical zookeeper ha
 2015-06-01 13:03:27 +02:00
Ken Breeman c72dd5a38c Cleaned up error handling and HA lock monitoring for zookeeper physical backend based on PR feedback. 2015-05-28 00:39:12 -04:00
Eric Buth e4e4253d65 added etcd as a non-HA storage backend, updated documentation 2015-05-26 13:38:25 -04:00
Ken Breeman f6292eb441 Cleaned up zookeeper_ha locking, added tests and cleanup. 2015-05-26 00:12:16 -04:00
Ken Breeman efb455e5e8 Improvements based on PR feedback: removed empty detectAddress function, moved anonymous functions to named ones, added localLock mutex around i.held 2015-05-25 22:14:00 -04:00
Ken Breeman 13d47c11ab Merge branch 'master' into physical_zookeeper_ha 2015-05-25 21:01:59 -04:00
Jeff Mitchell bb866b0140 AWS changed their error interface; fix compile breakage. 2015-05-21 16:15:21 -04:00
Ken Breeman fa50ca026a Restore backwards compatibility for zookeeper physical backend. Vault already prevents locks and data from overlapping internally. 2015-05-20 23:15:31 -04:00
Ken Breeman a26882ebd4 Merge branch 'master' into physical_zookeeper_ha 
Conflicts:
	physical/zookeeper.go
 2015-05-20 22:59:37 -04:00
Ken Breeman ae74837e55 Implement HA lock loss detection for zookeeper physical backend 2015-05-20 22:54:35 -04:00
Armon Dadgar 1851434407 physical/s3: skip unit test if missing ENV vars 2015-05-20 17:42:41 -07:00
James Stremick 6726fcf7bc Removed erroneous mutex and tests. Delete operates on a single key now 2015-05-20 19:53:35 -04:00
James Stremick 53979d6f30 Physical S3 backend implementation 2015-05-20 10:59:03 -04:00
Spencer Herzberg 3a6a060b2e recursive zk delete 2015-05-12 11:50:32 -05:00
Spencer Herzberg f3f6466730 fixes #189; zk version conflict 
* multiple Puts to the same node causes zk errors
 2015-05-12 09:12:00 -05:00
Ken Breeman f6de41c31d Rough implementation of Zookeeper HA physical backend. Contains breaking changes to 'path' config. Has unresolved TODO's. 2015-05-12 00:37:08 -04:00
Armon Dadgar 47cfc85079 physical/consul: Fixing read of leader when standby. Fixes #178 2015-05-11 10:54:29 -07:00
Brandon Philips 3d3d725fc5 pysical: minor doc error in consul 
ot -> to
 2015-05-08 23:37:16 -07:00
Armon Dadgar ad3cfa206b physical/consul: Fixing path for locks 2015-05-08 15:34:29 -07:00
Armon Dadgar 0af92bdd2c physical/zk: Style changes and more error checking 2015-05-06 11:08:26 -07:00
Spencer Herzberg 985600a986 fixing default zookeeper port 2015-05-06 08:57:24 -05:00
Spencer Herzberg 2869efd6fb be optimistic on zk paths operations 
* zk requires paths to be set or the client returns an error
* catch these errors instead of creating the full path
 2015-05-05 21:23:24 -05:00
Spencer Herzberg 8a4c2eb691 cleanup zk HA leftover docs 2015-05-05 17:22:43 -05:00
Spencer Herzberg 9793986357 properly default zk address to localhost 2015-05-05 17:20:38 -05:00
Spencer Herzberg f10d993fb5 limit round trips on zk delete 2015-05-05 17:14:41 -05:00
Spencer Herzberg 7d16da4174 fixing comment; bad copy-paste-edit 2015-05-05 16:56:49 -05:00
Spencer Herzberg 966204d73f initial implementation of non-ha zookeeper 2015-05-05 16:49:18 -05:00
Armon Dadgar 5dad76d5a1 physical/consul: Support address detection using the agent 2015-05-02 15:34:39 -07:00
Armon Dadgar 06f3e498f0 physical: Adding optional interface for addr detection 2015-05-02 15:34:29 -07:00
Mitchell Hashimoto 1d7f78d3f3 physical/file: open for writing 2015-04-29 11:31:59 -07:00
jjshoe 3b53334d87 Sensible permissions on creating a file 
Open a file, create it if it doesn't exist, and for gods sake don't leave it 0666.
 2015-04-29 13:27:44 -05:00
Mitchell Hashimoto e9621cdfe3 physical: more sorting to make tests deterministic 2015-04-28 19:01:01 -07:00
Mitchell Hashimoto 68b3dd1a4b physical: sort strings in test 2015-04-28 18:51:21 -07:00