Commit Graph

181 Commits

Author SHA1 Message Date
James Phillips bc29610124 Adds support for snapshots and restores. (#2396)
* Updates Raft library to get new snapshot/restore API.

* Basic backup and restore working, but need some cleanup.

* Breaks out a snapshot module and adds a SHA256 integrity check.

* Adds snapshot ACL and fills in some missing comments.

* Require a consistent read for snapshots.

* Make sure snapshot works if ACLs aren't enabled.

* Adds a bit of package documentation.

* Returns an empty response from restore to avoid EOF errors.

* Adds API client support for snapshots.

* Makes internal file names match on-disk file snapshots.

* Adds DC and token coverage for snapshot API test.

* Adds missing documentation.

* Adds a unit test for the snapshot client endpoint.

* Moves the connection pool out of the client for easier testing.

* Fixes an incidental issue in the prepared query unit test.

I realized I had two servers in bootstrap mode so this wasn't a good setup.

* Adds a half close to the TCP stream and fixes panic on error.

* Adds client and endpoint tests for snapshots.

* Moves the pool back into the snapshot RPC client.

* Adds a TLS test and fixes half-closes for TLS connections.

* Tweaks some comments.

* Adds a low-level snapshot test.

This is independent of Consul so we can pull this out into a library
later if we want to.

* Cleans up snapshot and archive and completes archive tests.

* Sends a clear error for snapshot operations in dev mode.

Snapshots require the Raft snapshots to be readable, which isn't supported
in dev mode. Send a clear error instead of a deep-down Raft one.

* Adds docs for the snapshot endpoint.

* Adds a stale mode and index feedback for snapshot saves.

This gives folks a way to extract data even if the cluster has no
leader.

* Changes the internal format of a snapshot from zip to tgz.

* Pulls in Raft fix to cancel inflight before a restore.

* Pulls in new Raft restore interface.

* Adds metadata to snapshot saves and a verify function.

* Adds basic save and restore snapshot CLI commands.

* Gets rid of tarball extensions and adds restore message.

* Fixes an incidental bad link in the KV docs.

* Adds documentation for the snapshot CLI commands.

* Scuttle any request body when a snapshot is saved.

* Fixes archive unit test error message check.

* Allows for nil output writers in snapshot RPC handlers.

* Renames hash list Decode to DecodeAndVerify.

* Closes the client connection for snapshot ops.

* Lowers timeout for restore ops.

* Updates Raft vendor to get new Restore signature and integrates with Consul.

* Bounces the leader's internal state when we do a restore.
2016-10-25 19:20:24 -07:00
Adam Wolfe Gordon fa08665ad2 vendor: Remove go-reap
go-reap is no longer used since we no longer reap child processes.
2016-10-04 09:41:29 -06:00
R.B. Boyer 291a468c87
Validate gossip encryption key before made persistent in local.keyring 2016-08-12 11:42:45 -07:00
James Phillips 6a208e0797
Updates hashicorp/yamux. 2016-08-09 17:24:41 -07:00
James Phillips 1f46758761
Updates hashicorp/scada-client. 2016-08-09 17:24:40 -07:00
James Phillips 7caa29363d
Updates hashicorp/memberlist. 2016-08-09 17:24:40 -07:00
James Phillips 4412a7e9fa
Updates hashicorp/logutils. 2016-08-09 17:24:40 -07:00
James Phillips eceb58fff5
Updates hashicorp/go-uuid. 2016-08-09 17:24:40 -07:00
James Phillips 2e931733d0
Updates hashicorp/go-syslog. 2016-08-09 17:24:40 -07:00
James Phillips 4515a1d90b
Updates hashicorp/go-retryablehttp. 2016-08-09 17:24:39 -07:00
James Phillips e53f348a9c
Updates hashicorp/go-reap. 2016-08-09 17:24:39 -07:00
James Phillips d1d5c14c83
Updates hashicorp/go-cleanhttp. 2016-08-09 17:24:39 -07:00
James Phillips f431598fb6
Updates hashicorp/go-immutable-radix and hashicorp/go-memdb. 2016-08-09 17:24:39 -07:00
James Phillips 99ab3390c2
Updates hashicorp/hcl and hashicorp/hil.
This required a small mod to core Consul code to cope with an interface
change.
2016-08-09 17:24:13 -07:00
James Phillips 8d03a2c948
Removes some unused vendored dependencies. 2016-08-09 16:50:34 -07:00
James Phillips f1b47817f2
Updates hashicorp/golang-lru. 2016-08-09 16:26:59 -07:00
James Phillips 7f2346bb0f
Vendors first stage branch of the v2 Raft library. 2016-08-08 19:19:17 -07:00
James Phillips 954c32e6ee
Updates Serf to pick up intent queue fix.
This fixes #1062 by storing intents per-node instead of in a small, fixed-
size circular buffer.
2016-08-08 18:58:44 -07:00
James Phillips 044512c6de Updates armon/go-metrics and adds Circonus vendored libraries. 2016-07-19 16:54:21 -07:00
James Phillips bf9e334dde Updates go-immutable-radix to pull in caching fix. 2016-06-08 19:08:31 -07:00
James Phillips 7e75c894b0 Updates serf to get health score in stats. 2016-06-08 17:20:12 -07:00
James Phillips 1776316053 Updates to latest Serf/memberlist to get lifeguard and TCP joins over DNS. 2016-06-07 14:30:47 -07:00
James Phillips 6ee216eace Pulls in latest Serf to get flap metric. 2016-06-07 09:22:41 -07:00
Jeff Mitchell e061d89fe9 Use upstream high-level SCADA provider 2016-06-01 18:25:39 -04:00
Sean Chittenden 32c24b5447 Update Serf to include `serf.NumNodes()` 2016-03-23 22:10:50 -07:00
James Phillips 06087633f0 Adds in basic query template lookups and vendors newly-updated memdb as well as improved iradix tree. 2016-03-07 10:45:39 -08:00
James Phillips ee43212da4 Updates vendored dependencies.
This removes some unused dependencies that I know we plan to re-add later,
but this makes for a clean `godep save` for now.
2016-03-07 10:45:39 -08:00
James Phillips f4aab86081 Updates memberlist to get goroutine leak fix for TCP pings. 2016-03-07 10:41:54 -08:00
Mat Schaffer fa2ba64654 Metric name typo 2016-02-25 11:20:53 +09:00
Jeff Mitchell 1dfacd8a1e Update go-cleanhttp 2016-02-17 17:03:57 -05:00
Sean Chittenden 333ff22e9a Manage dependencies via Godep
Embrace the future and use Go 1.6's vendor support via Godep.

Go 1.5 users should `export GO15VENDOREXPERIMENT=1`
2016-02-12 16:50:37 -08:00