Commit Graph

154 Commits

Author SHA1 Message Date
Jeff Mitchell e5c31d66a2 Better handle nil responses in S3 backend, also a case where error wasn't checked 2016-04-26 08:11:56 -04:00
Sean Chittenden 557d8b8a24 Make use of logger interface inside of the Consul BE 2016-04-25 20:10:55 -07:00
Sean Chittenden aeea7628d6 Add a *log.Logger argument to physical.Factory
Logging in the backend is a good thing.  This is a noisy interface change but should be a functional noop.
2016-04-25 20:10:32 -07:00
Sean Chittenden 5fd5869bc5 Rewriting history before it gets away from me 2016-04-25 18:05:50 -07:00
Sean Chittenden 9b8095d7ea Change to the pre-0.6.4 Consul Check API
Consul is never going to pass in more than 1K of output.  This mitigates the pre-0.6.4 concern.
2016-04-25 18:01:13 -07:00
Sean Chittenden f5183fa506 Collapse UpdateAdvertiseAddr() into RunServiceDiscovery() 2016-04-25 18:01:13 -07:00
Sean Chittenden 5104c58c54 Update tests to chase sealed -> unsealed transition 2016-04-25 18:01:13 -07:00
Sean Chittenden 7fe0b2c6a1 Persistently retry to update service registration
If the local Consul agent is not available while attempting to step down from active or up to active, retry once a second.  Allow for concurrent changes to the state with a single registration updater.  Fix standby initialization.
2016-04-25 18:01:13 -07:00
Sean Chittenden 3449fa1bc3 Consistently skip Consul checks
Hide all Consul checks behind `CONSUL_HTTP_ADDR` env vs `CONSUL_ADDR` which is non-standard.
2016-04-25 18:01:13 -07:00
Sean Chittenden 60006f550f Various refactoring to clean up code organization
Brought to you by: Dept of 2nd thoughts before pushing enter on `git push`
2016-04-25 18:01:13 -07:00
Sean Chittenden 53f9cea87c Compare the correct values when validating check_timeout 2016-04-25 18:01:13 -07:00
Sean Chittenden 70ae7f73b4 Detect type conversion failure 2016-04-25 18:01:13 -07:00
Sean Chittenden ae66e65bcf Don't export the builtin backends 2016-04-25 18:01:13 -07:00
Sean Chittenden bd3335c1bd `go fmt` the PostgreSQL backend 2016-04-25 18:01:13 -07:00
Sean Chittenden 6b2c83564e Teach Vault how to register with Consul
Vault will now register itself with Consul.  The active node can be found using `active.vault.service.consul`.  All standby vaults are available via `standby.vault.service.consul`.  All unsealed vaults are considered healthy and available via `vault.service.consul`.  Change in status and registration is event driven and should happen at the speed of a write to Consul (~network RTT + ~1x fsync(2)).

Healthy/active:

```
curl -X GET 'http://127.0.0.1:8500/v1/health/service/vault?pretty' && echo;
[
    {
        "Node": {
            "Node": "vm1",
            "Address": "127.0.0.1",
            "TaggedAddresses": {
                "wan": "127.0.0.1"
            },
            "CreateIndex": 3,
            "ModifyIndex": 20
        },
        "Service": {
            "ID": "vault:127.0.0.1:8200",
            "Service": "vault",
            "Tags": [
                "active"
            ],
            "Address": "127.0.0.1",
            "Port": 8200,
            "EnableTagOverride": false,
            "CreateIndex": 17,
            "ModifyIndex": 20
        },
        "Checks": [
            {
                "Node": "vm1",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "CreateIndex": 3,
                "ModifyIndex": 3
            },
            {
                "Node": "vm1",
                "CheckID": "vault-sealed-check",
                "Name": "Vault Sealed Status",
                "Status": "passing",
                "Notes": "Vault service is healthy when Vault is in an unsealed status and can become an active Vault server",
                "Output": "",
                "ServiceID": "vault:127.0.0.1:8200",
                "ServiceName": "vault",
                "CreateIndex": 19,
                "ModifyIndex": 19
            }
        ]
    }
]
```

Healthy/standby:

```
[snip]
        "Service": {
            "ID": "vault:127.0.0.2:8200",
            "Service": "vault",
            "Tags": [
                "standby"
            ],
            "Address": "127.0.0.2",
            "Port": 8200,
            "EnableTagOverride": false,
            "CreateIndex": 17,
            "ModifyIndex": 20
        },
        "Checks": [
            {
                "Node": "vm2",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "CreateIndex": 3,
                "ModifyIndex": 3
            },
            {
                "Node": "vm2",
                "CheckID": "vault-sealed-check",
                "Name": "Vault Sealed Status",
                "Status": "passing",
                "Notes": "Vault service is healthy when Vault is in an unsealed status and can become an active Vault server",
                "Output": "",
                "ServiceID": "vault:127.0.0.2:8200",
                "ServiceName": "vault",
                "CreateIndex": 19,
                "ModifyIndex": 19
            }
        ]
    }
]
```

Sealed:

```
        "Checks": [
            {
                "Node": "vm2",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "CreateIndex": 3,
                "ModifyIndex": 3
            },
            {
                "Node": "vm2",
                "CheckID": "vault-sealed-check",
                "Name": "Vault Sealed Status",
                "Status": "critical",
                "Notes": "Vault service is healthy when Vault is in an unsealed status and can become an active Vault server",
                "Output": "Vault Sealed",
                "ServiceID": "vault:127.0.0.2:8200",
                "ServiceName": "vault",
                "CreateIndex": 19,
                "ModifyIndex": 38
            }
        ]
```
2016-04-25 18:01:13 -07:00
Sean Chittenden 230b59f34c Stub out service discovery functionality
Hook asynchronous notifications into Core to change the status of vault based on its active/standby, and sealed/unsealed status.
2016-04-25 18:00:54 -07:00
Jeff Mitchell f00beb4e32 Update azure backend for newer sdk 2016-04-26 00:08:07 +00:00
Jeff Mitchell a481bff2b1 Fix commenting S3 -> Azure 2016-04-25 19:53:07 +00:00
Tobias Haag 175e3cc354 added Azure backend support
updated Godeps
added website docs
updated vendor
2016-03-30 19:49:38 -07:00
Jeff Mitchell deed5cc121 Output original error on etcd sync failure.
Fixes #1141
2016-02-26 15:15:23 -05:00
Jeff Mitchell 50d3b68c8d Merge pull request #1078 from eyal-lupu/master
ZooKeeper Backend: Authnetication and Authorization support
2016-02-19 15:13:09 -05:00
Eyal Lupu a6e9820e8d typo in comment 2016-02-19 13:28:02 +00:00
Eyal Lupu 23303429c0 'Eagerly' parse ZK authentication and authorization to fast-fail bad configuration 2016-02-19 13:24:57 +00:00
Eyal Lupu c7fe99b1e9 1. gofmt
2. Change if expr syntax to be consist with the rest of Vault code
3. More details on error message
2016-02-19 12:19:01 +00:00
Jeff Mitchell 5edaf522a8 Use a pooled transport for the Consul physical backend and give it 4 idle connections 2016-02-17 16:53:30 -05:00
Eyal Lupu e9c7a02850 https://github.com/hashicorp/vault/issues/1058
Make sure locks are also using the same auth info as data
2016-02-15 15:29:08 +00:00
Eyal Lupu d4db2ea79c fixes to https://github.com/hashicorp/vault/issues/1058
Configuration now supports:
- auth_info
-znode_owner
2016-02-15 15:03:12 +00:00
Devin Christensen 4112809fb5 Make the PostgreSQL backend more performant 2016-01-29 13:47:10 -07:00
Jeff Mitchell 68dc0e2dd3 Merge pull request #945 from quixoten/postgres_physical
Add support for PostgreSQL as a physical backend
2016-01-29 10:35:38 -05:00
Devin Christensen 737df30939 Improve naming
Hopefully this naming scheme will be more straightforward.
2016-01-27 17:15:48 -07:00
Jeff Mitchell b7a49922a9 Update etcd sync option to be a string.
Ping #921
2016-01-27 17:15:52 -05:00
Jeff Mitchell b0bd06f5a4 Merge pull request #921 from faradayio/hosted-etcd-support
Load-balanced etcd support
2016-01-27 17:09:43 -05:00
Devin Christensen 9d776351a3 Merge 'upstream/master' into postgres_physical 2016-01-22 20:56:07 -07:00
Devin Christensen c226b0be7d Update naming and pull DDL for upsert back out 2016-01-22 17:15:10 -07:00
Devin Christensen 32b712ddb1 Move the upsert definition back into the code 2016-01-22 09:47:02 -07:00
Devin Christensen bfbdc72e03 Remove options for column configuration 2016-01-22 08:41:31 -07:00
Jeff Mitchell be1b4c8a46 Only allow listing on folders and enforce this. Also remove string sorting from Consul backend as it's not a requirement and other backends don't do it. 2016-01-22 10:07:32 -05:00
Devin Christensen 06641570c7 Remove DDL statements from the code 2016-01-20 18:52:49 -07:00
Devin Christensen bcc720be11 Remove superfluous comparison 2016-01-20 17:05:21 -07:00
Devin Christensen 65bd200fae Ensure rows.Close() is called in List 2016-01-20 17:02:23 -07:00
Devin Christensen 614f7b7157 Prefer TEXT over VARCHAR
From the PostgreSQL docs
(http://www.postgresql.org/docs/9.4/static/datatype-character.html):

 > Tip: There is no performance difference among these three types,
 > apart from increased storage space when using the blank-padded type,
 > and a few extra CPU cycles to check the length when storing into a
 > length-constrained column. While character(n) has performance
 > advantages in some other database systems, there is no such advantage
 > in PostgreSQL; in fact character(n) is usually the slowest of the
 > three because of its additional storage costs. In most situations
 > text or character varying should be used instead.
2016-01-20 16:56:46 -07:00
Devin Christensen b4e9e204f7 Use native upsert when available 2016-01-20 10:47:54 -07:00
Devin Christensen fc94487f55 Add support for PostgreSQL as a physical backend 2016-01-19 17:00:09 -07:00
Eric Kidd 69434fd13e etcd: Allow disabling sync for load balanced etcd
Some etcd configurations (such as that provided by compose.io) place the
etcd cluster behind multiple load balancers or proxies.  In this
configuration, calling Sync (or AutoSync) on the etcd client will
replace the load balancer addresses with the underlying etcd server
address.

This will cause the etcd client to bypass the load balancers, and may
cause the connection to fail completely if the etcd servers are
protected by a firewall.

This patch provides a "sync" option for the etcd backend, which defaults
to the current behavior, but which can be used to turn off of sync.
This corresponds to etcdctl's --no-sync option.
2016-01-11 13:56:58 -05:00
Paul Seiffert 99f7659bb4 Add recovery option to DynamoDB backend
When Vault is killed without the chance to clean up the lock
entry in DynamoDB, no further Vault nodes can become leaders after
that.

To recover from this situation, this commit adds an environment
variable and a configuration flag that when set to "1" causes Vault
to delete the lock entry from DynamoDB.
2016-01-08 17:31:37 +01:00
Paul Seiffert 8853e50691 Explicitly read AWS credentials from environment 2016-01-08 17:31:37 +01:00
Paul Seiffert 277de77256 Add tests for DynamoDB backend 2016-01-08 17:31:37 +01:00
Paul Seiffert 870bc6c5b4 Implement DynamoDB physical HA backend 2016-01-08 17:31:37 +01:00
Jeff Mitchell 287954beef Replace physical cache with TwoQueue instead of LRU. 2016-01-07 09:21:33 -05:00
Jeff Mitchell bf2bf06997 Use cleanhttp.DefaultTransport rather than instantiating directly to avoid leaked FDs 2015-12-17 15:23:13 -05:00