557df50be6
Adds fine-grained watches to internal endpoints.
2017-01-25 09:58:22 -08:00
e1dd427cf4
Adds fine-grained watches to coordinate endpoints.
2017-01-25 09:58:22 -08:00
2cbf45301e
Adds fine-grained watch support to ACL endpoints.
2017-01-25 09:58:22 -08:00
635afd1924
Adds fine-grained watches to health endpoints.
2017-01-25 09:58:22 -08:00
a604c438a2
Fixes a race condition when updating the state store during a snapshot restore.
2017-01-25 09:58:22 -08:00
0891b6c74d
Adds fine-grained watches to catalog endpoints.
2017-01-25 09:58:22 -08:00
5afa233d28
Adds diff check for node and service parts of register requests.
...
We always did an update before which caused excessive watch churn, even
with our new fine-grained queries. This does a diff any only updates the
node and service records if something actually changed.
2017-01-25 09:58:22 -08:00
40ef50c738
Don't do any watch tracking for non-blocking queries.
2017-01-25 09:58:22 -08:00
034d1960b6
Removes some incorrect comments.
...
We can't actually return a fine-grained index from these tables unless
support is added for tombstones. Otherwise, the index could slip backwards
as things are deleted.
2017-01-25 09:58:22 -08:00
085672a126
Adds new variant of blocking query wrapper with WatchSet support.
2017-01-25 09:58:22 -08:00
a09b6ea8b4
Fix test import
2017-01-23 21:34:01 -05:00
cc271aa436
Merge branch 'master' into f-prepared-query-nodemeta
2017-01-23 20:17:48 -05:00
3cd054e81f
Add tests for node meta in prepared queries and update docs
2017-01-23 19:17:30 -05:00
9e8952c126
Merge pull request #2661 from hashicorp/f-node-id
...
Adds basic support for node IDs.
2017-01-18 16:02:38 -08:00
60c135dd67
Uses clean replies each time so they are safe to receive map changes.
2017-01-18 15:40:19 -08:00
abe7b33986
Fixes a startup ordering issue between Raft and Serf.
...
This fixes #2663 and fixes #1899 . It's not super related to this PR,
but the startup time changes that this PR brings made this a lot worse
so I was able to track it down.
2017-01-18 15:06:15 -08:00
55cd1d975c
Adds catalog support for node IDs.
2017-01-18 14:26:42 -08:00
fdf2a10835
First pass at adding node meta filter to prepared queries
2017-01-18 16:23:33 -05:00
96bff003b7
Adds basic support for node IDs.
2017-01-17 22:47:59 -08:00
b3fdfb09d8
Minor formatting tweaks as a follow-up to #2654
2017-01-17 19:20:29 -05:00
9080006a37
Add support for multiple metadata filters to remaining endpoints
...
Enabled multiple meta filters for /v1/catalog/nodes and /v1/catalog/services
2017-01-13 20:49:13 -05:00
e37f25dc02
Add node metadata filtering to remaining health/catalog endpoints
2017-01-13 20:08:43 -05:00
c837d055cd
Breaks up the state store into several files.
2017-01-13 11:47:16 -08:00
3620e5e199
Fix inconsistency in TestStateStore_ServicesByNodeMeta
2017-01-12 19:46:58 -05:00
51a1834667
Add -node-meta to agent command line options
2017-01-11 16:09:04 -05:00
d77890a011
Validate metadata config earlier and handle multiple filters
2017-01-11 15:12:03 -05:00
6b5cf20b1c
Fix formatting
2017-01-09 13:49:33 -08:00
aee766baba
Add meta key validations and more tests
2017-01-09 11:21:49 -08:00
b25f4c7d35
Add tests for node metadata functionality
2017-01-05 17:21:56 -08:00
2a423c6e2c
Add support for setting node metadata fields
2017-01-05 14:10:26 -08:00
927aa3789a
Adds ACL management support to the agent.
2016-12-14 07:07:41 -08:00
d069c0805e
fix fmt.Errorf error
...
missing argument for Errorf("%q"): format reads arg 1, have only 0 args
2016-12-14 18:57:12 +08:00
71a775d986
Merge pull request #2592 from hashicorp/acl-complete-node-session
...
Adds complete ACL coverage for nodes and sessions.
2016-12-13 13:55:07 -08:00
535d6b21b4
Adds complete ACL coverage for /v1/session endpoints.
2016-12-12 21:59:22 -08:00
4aa070834b
Adds complete ACL coverage for /v1/internal/ui/node endpoints.
2016-12-12 18:22:10 -08:00
ac967176ee
Adds complete ACL support for /v1/query/<query id or name>/execute.
...
This was already supported by previous changes to the ACL filter, so
we just added a test to show it working.
2016-12-12 17:28:06 -08:00
7e7c1ebb13
Fixes implementation of node ACLs for /v1/catalog/node/<node>.
...
This would return a "permission denied" error, but this changes it to
return the same response as a node that doesn't exist (as was originally
intended and written in the code comments).
2016-12-12 16:53:31 -08:00
7a21a591b8
Adds full ACL coverage for /v1/health endpoints.
2016-12-12 16:28:52 -08:00
402f30d52e
Merge pull request #2591 from hashicorp/snapshot-interval
...
Change raft snapshot interval to 5 seconds
2016-12-12 19:15:10 -05:00
8db53b4ac2
Adds complete ACL coverage for /v1/coordinate/nodes and Coordinate.Update RPC.
2016-12-12 14:52:27 -08:00
67f1fe8d0b
Adds support for a new "acl_agent_token" which is used for internal
...
catalog operations.
2016-12-12 14:52:27 -08:00
99a5ae2737
Bans anonymous queries that aren't tied to a session.
...
This gets us coverage of PQ creation under the existing service
policy or the soon-to-be-added session policy.
2016-12-12 14:52:27 -08:00
3ae8bedd6d
Change raft snapshot interval to 5 seconds
2016-12-12 13:31:42 -05:00
1ec02e5af4
Adds complete ACL coverage for /v1/catalog/service/<service>.
2016-12-12 08:34:15 -08:00
4dc3b17078
Adds complete ACL coverage for /v1/catalog/nodes.
2016-12-10 16:49:19 -08:00
63a6abe03c
Adds complete ACL coverage for /v1/catalog/node/<node>.
2016-12-10 16:49:19 -08:00
8ed0b81adb
Adds complete ACL coverage for /v1/catalog/deregister.
...
This included some state store helpers to make this more efficient.
2016-12-09 21:04:44 -08:00
7fa693a338
Adds complete ACL coverage for /v1/catalog/register.
2016-12-09 21:04:37 -08:00
2b19a5340c
Removes the exception for the "consul" service in the catalog.
2016-12-07 17:58:23 -08:00
26a26a489d
Add retry with backoff to initial bootstrap checks ( #2561 )
2016-12-01 17:05:02 -05:00
1c55429a38
Add an API method for determining the best status
...
Given a list of HealthChecks, this determines the "best" status for the
collective group. This is useful for nodes and services, which may have
multiple checks associated with them.
2016-11-29 18:41:46 -05:00
2d37a07476
Add keyring http endpoints
2016-11-22 20:10:43 -05:00
9adc3854d1
Retry with backoff on session invalidation failure ( #2475 )
2016-11-04 21:53:22 -07:00
be4056789f
Moves the snapshot package up one level. ( #2472 )
2016-11-03 21:36:25 -07:00
169cae2203
Disallow -bootstrap-expect flag in dev mode ( #2464 )
2016-11-03 01:54:43 -04:00
440611f9f7
Add snapshot inspect subcommand ( #2451 )
2016-10-31 19:37:27 -04:00
c6f461aa25
Enable snapshots in dev mode ( #2453 )
2016-10-31 14:39:47 -04:00
bc29610124
Adds support for snapshots and restores. ( #2396 )
...
* Updates Raft library to get new snapshot/restore API.
* Basic backup and restore working, but need some cleanup.
* Breaks out a snapshot module and adds a SHA256 integrity check.
* Adds snapshot ACL and fills in some missing comments.
* Require a consistent read for snapshots.
* Make sure snapshot works if ACLs aren't enabled.
* Adds a bit of package documentation.
* Returns an empty response from restore to avoid EOF errors.
* Adds API client support for snapshots.
* Makes internal file names match on-disk file snapshots.
* Adds DC and token coverage for snapshot API test.
* Adds missing documentation.
* Adds a unit test for the snapshot client endpoint.
* Moves the connection pool out of the client for easier testing.
* Fixes an incidental issue in the prepared query unit test.
I realized I had two servers in bootstrap mode so this wasn't a good setup.
* Adds a half close to the TCP stream and fixes panic on error.
* Adds client and endpoint tests for snapshots.
* Moves the pool back into the snapshot RPC client.
* Adds a TLS test and fixes half-closes for TLS connections.
* Tweaks some comments.
* Adds a low-level snapshot test.
This is independent of Consul so we can pull this out into a library
later if we want to.
* Cleans up snapshot and archive and completes archive tests.
* Sends a clear error for snapshot operations in dev mode.
Snapshots require the Raft snapshots to be readable, which isn't supported
in dev mode. Send a clear error instead of a deep-down Raft one.
* Adds docs for the snapshot endpoint.
* Adds a stale mode and index feedback for snapshot saves.
This gives folks a way to extract data even if the cluster has no
leader.
* Changes the internal format of a snapshot from zip to tgz.
* Pulls in Raft fix to cancel inflight before a restore.
* Pulls in new Raft restore interface.
* Adds metadata to snapshot saves and a verify function.
* Adds basic save and restore snapshot CLI commands.
* Gets rid of tarball extensions and adds restore message.
* Fixes an incidental bad link in the KV docs.
* Adds documentation for the snapshot CLI commands.
* Scuttle any request body when a snapshot is saved.
* Fixes archive unit test error message check.
* Allows for nil output writers in snapshot RPC handlers.
* Renames hash list Decode to DecodeAndVerify.
* Closes the client connection for snapshot ops.
* Lowers timeout for restore ops.
* Updates Raft vendor to get new Restore signature and integrates with Consul.
* Bounces the leader's internal state when we do a restore.
2016-10-25 19:20:24 -07:00
f0aa65754b
Wait for agent joins to finish in TestClient_RPC
2016-10-25 17:48:11 -07:00
2379b822ad
Add wait logic to TestClient_RPC_Pool
2016-10-25 17:48:11 -07:00
b423dee303
Fixes port numbers in peers.info.
2016-10-05 18:09:15 -07:00
8e76af4311
Merge pull request #2319 from hashicorp/f-bootstrap-abort
...
Adds check that aborts bootstrap mode if there's an existing cluster.
2016-09-01 09:49:03 -07:00
94c0e961eb
Fixes error message in test.
2016-09-01 09:48:08 -07:00
ce93b82e1e
Makes port selection atomic in unit tests.
2016-09-01 01:01:28 -07:00
d04a706a7c
Tweaks comment to be more correct.
2016-08-31 23:54:53 -07:00
4dd9b4b08a
Adds check that aborts bootstrap mode if there's an existing cluster.
2016-08-31 21:25:56 -07:00
750e1751ac
Copies the member data instead of referencing by pointer.
2016-08-30 16:54:21 -07:00
6be1e07fec
Makes the Raft configuration API easier to consume.
2016-08-30 11:30:56 -07:00
5df4b6bef2
Adds a log warning when operator peer changes occur.
2016-08-30 10:23:32 -07:00
1b7a16b7d3
Adds new consul operator endpoint, CLI, and ACL and some basic Raft commands.
2016-08-30 00:02:50 -07:00
29e52307cb
Makes empty checkServiceNode return a nil.
...
The change in #2308 had an inadvertent interface change, so we fix that with
a special case in this fix.
2016-08-29 19:12:07 -07:00
327fe725d9
Preallocates result struct, which was a profiling hot spot.
2016-08-26 16:34:28 -07:00
c5b6ac3655
Removes leader_lease_timeout from stats.
2016-08-25 15:39:19 -07:00
2f4c237cff
Adds a max raft multiplier and tweaks documentation.
2016-08-25 15:36:05 -07:00
5df36fbd82
Stops scaling the commit timeout.
2016-08-25 15:05:40 -07:00
f65ef936cb
Increases RPC hold timeout for new default timing.
...
Rather than scale this we just bump it up a bit. It'll be on the edge in
the lower-performance default mode, and will have plenty of margin in the
high-performance mode. This seems like a reasonable compromise to keep the
logic here simple vs. scaling, and seems inline with the expectations of
the different modes of operation.
2016-08-24 23:35:28 -07:00
b339b0d2fc
Adds performance tuning capability for Raft, detuned defaults, and supplemental docs.
2016-08-24 21:58:37 -07:00
0bdbdf1ba8
Merge pull request #2226 from abhinavdahiya/rm-health-unknown
...
Fixes #1775 ; Removes 'unknown' state
2016-08-17 17:51:04 -07:00
1f539d9914
Makes the filled-in parts of ServiceNode more explicit.
2016-08-12 18:25:36 -07:00
360e196c93
Translate Address to tagged WAN address in HTTP API when appropriate.
2016-08-12 18:25:36 -07:00
d11a7a197c
Removes upper end of muxado handler.
2016-08-09 18:16:41 -07:00
97a25e8564
Closes the conn on bad protocol version.
2016-08-09 18:13:53 -07:00
359587f70e
Removes support for muxado and protocol version 1.
2016-08-09 18:10:04 -07:00
99ab3390c2
Updates hashicorp/hcl and hashicorp/hil.
...
This required a small mod to core Consul code to cope with an interface
change.
2016-08-09 17:24:13 -07:00
ff6d42389c
Merge pull request #2222 from hashicorp/f-raft-v2
...
Integrates Consul with "stage one" of HashiCorp Raft library v2.
2016-08-09 16:04:48 -07:00
cce38f9a4b
Moves the peers.info content down into a constant.
2016-08-09 11:56:39 -07:00
7aaa4bc913
Adds peers back into bootstrap log, makes initial case consistent.
2016-08-09 11:52:41 -07:00
7f58b05dfe
Tweaks select style.
2016-08-09 11:33:42 -07:00
544169999c
Adds I/O-sensitive metrics to ACL replication operations.
2016-08-09 11:32:12 -07:00
820509760d
Switches to a smooth rate limit vs. a bursty one.
2016-08-09 11:29:12 -07:00
129e327bc9
Clarifies replication index shown in the log message.
2016-08-09 11:10:32 -07:00
4203612bd7
Returns from the shutdown wait right away.
2016-08-09 11:09:48 -07:00
e03fbef6b3
Moves ACL ID sorting interface onto the iterator.
2016-08-09 11:08:26 -07:00
0fa059ec49
Switches all ACL caches to 2Q.
2016-08-09 11:00:22 -07:00
1e75fa0362
Moves ACL ID generation down into the endpoint.
...
We don't want ACL replication to have this behavior so it was a
little dangerous to have in the shared helper function.
2016-08-09 00:11:00 -07:00
06a510a808
Removes unsafe "recover to empty" code.
...
This isn't safe because it would implicitly commit all outstanding log
entries. The new Raft library already has logic to not start a vote if
the current node isn't in the configuration, so this shoudn't be needed.
2016-08-08 19:19:19 -07:00
dd3169b395
Tweaks recovery based on interface changes.
2016-08-08 19:19:18 -07:00
19004e7095
Moves to a safer design where we don't ingest the initial peers.json file.
2016-08-08 19:19:18 -07:00
44c468995f
Touches up Raft integration after latest changes.
2016-08-08 19:19:18 -07:00
fc25145e85
Formats log messages to be consistent.
2016-08-08 19:19:18 -07:00
6b157eada0
Adds more comments about the raftSafeFn.
2016-08-08 19:19:18 -07:00
fcd8bb157a
Clarifies a comment about no-op peer operations.
2016-08-08 19:19:18 -07:00
2bf633f206
Adds back "safing" the configuration when a server leaves.
2016-08-08 19:19:18 -07:00
6c8e8271e2
Integrates Consul with new version of Raft library.
2016-08-08 19:19:17 -07:00
4a931ae12e
Adds an ACL replication status endpoint.
2016-08-04 23:30:16 -07:00
c94f1e1b83
Increases the ACL cache size to 10k.
2016-08-04 18:03:07 -07:00
3906517f70
Adds a full integrated test for ACL replication.
2016-08-04 17:59:08 -07:00
f639f49cc0
Adds remaining core replication tests.
2016-08-04 16:33:40 -07:00
defb39f8d4
Removes a TODO comment.
...
Decided we don't need to log anything about the token here. If the
token is not valid then the client will get an error about that, so
anything that can happen here is related to talking to the server in
the ACL datacenter, so not specific to the token.
2016-08-04 07:46:59 -07:00
93a7fd0561
Adds tests for the ACL reconcile algorithm.
2016-08-03 21:24:09 -07:00
796933b45b
Activates fallback to replicated ACLs.
2016-08-03 21:24:09 -07:00
9cece515c0
Adds basic ACL replication plumbing.
2016-08-03 21:24:04 -07:00
9dc52449e3
Fixes #1775 ; Removes 'unknown' state
...
Signed-off-by: Abhinav Dahiya <abhinavdtu2012@gmail.com>
2016-07-30 19:33:14 +05:30
a1266e4164
Adds some supplemental tests for RPC "no leader" retries.
...
This adds some extra tests for #2175 .
2016-07-11 17:32:26 -06:00
2d8cf9ef4a
consul: change tests to not expect ErrNoLeader
2016-07-10 13:24:18 -04:00
5d0a977bdf
consul: Refactor forward to hold RPC when no leader is known
2016-07-10 13:24:06 -04:00
191876f87e
consul: Add RPCHoldTimeout as tunable hold period
2016-07-10 13:23:43 -04:00
d8fd470f4f
Merge pull request #1837 from cleung2010/obfuscate-acl-token
...
Obfuscate token for lookupACL error
2016-07-05 13:56:49 -07:00
38134f1b8c
Fix substring length on obfuscated token
2016-07-05 15:53:30 -04:00
577523fc73
consul: sort source node first if at position <= 10 in PQ's
2016-07-01 14:28:58 -07:00
e9960e6c85
Merge pull request #2137 from hashicorp/f-pq-near
...
Support "near" parameter in prepared query service block
2016-07-01 12:28:48 -07:00
ccbe86d7a8
consul: mention magic _agent token in struct comments
2016-07-01 11:50:30 -07:00
ebacaa2d67
consul: send agent source data as separate query source
2016-06-30 16:51:18 -07:00
782a081925
consul: use source parameter for near prepared queries
2016-06-30 12:11:20 -07:00
270270a33a
consul: send origin node + dc when executing prepared queries
2016-06-21 15:34:26 -07:00
925915c6ac
consul: test baked-in distance sort
2016-06-21 12:54:18 -07:00
114e57fff1
consul: use the Near field instead of PreferLocal
2016-06-21 12:39:40 -07:00
8358df599d
Merge pull request #2127 from hashicorp/b-remote-consuls-locking
...
Ensure locking of `Server`'s `remoteConsuls`.
2016-06-21 10:00:04 -07:00
f9e2900692
Merge pull request #2131 from hashicorp/b-misc-microoptimizations
...
Misc micro optimizations
2016-06-21 09:59:01 -07:00
ebdb72ce0a
Ensure locking of `Server`'s `remoteConsuls`.
2016-06-20 22:59:49 -07:00
72f7a4061c
Misc comment improvements
2016-06-20 15:29:38 -07:00
9bf6e61655
Initialize a non-empty number of Consul Datacenters. No functional change.
2016-06-20 15:26:59 -07:00
b78c95d37e
Prefer rand.Int31n() over rand.Int31().
2016-06-20 15:26:27 -07:00
e81bf2a505
Fix deadlock in Consul RTT.
...
- consul/rtt.go:388: s.getDatacentersByDistance(). Acquires RLock()
- consul/rtt.go:341: sortDatacentersByDistance() RLock still held.
- consul/rtt.go:282: getDatacenterDistance() RLock still held.
- consul/rtt.go:268: getNodesForDatacenter(). Attempts to reacquire RLock(), hangs indefinitely.
2016-06-20 14:59:54 -07:00
89fe991ab7
consul: test raw PreferLocal functionality
2016-06-20 14:53:13 -07:00
1fef85cd2e
consul: support PreferLocal in PQ's
2016-06-20 14:24:40 -07:00
7482a9207d
Chase casting types.CheckID to a string into the state_store.
...
It turns out the indexer can only use strings as arguments when
creating a query. Cast `types.CheckID` to a `string` before calling
into `memdb`.
Ideally the indexer would be smart enough to do this at compile-time,
but I need to look into how to do this without reflection and the
runtime package. For the time being statically cast `types.CheckID`
to a `string` at the call sites.
2016-06-07 16:59:02 -04:00
ff45f8c8ff
Revert "Move `structs.CheckID` to a new top-level package, `types`."
...
This reverts commit 2bbd52e3b44ff1b60939a8400264d534662d6d51.
2016-06-07 16:59:02 -04:00
a4554b945c
Move `structs.CheckID` to a new top-level package, `types`.
...
Per discussion w/ @slackpad, move this type to its own top-level package
2016-06-07 16:59:02 -04:00
cd68cd3868
Move `structs.CheckID` to a new top-level package, `types`.
...
Per discussion w/ @slackpad, move this type to its own top-level package
2016-06-07 16:59:02 -04:00
0857e93d0b
Float a type balloon. Some strings are square pegs in round holes.
...
This experiment was brought about because of variable naming
confusion where name and checkIDs were interchanged. Gave CheckID
an Qualified Type Name and chased downstream changes.
2016-06-07 16:59:02 -04:00
ffcba3df58
Merge pull request #2028 from hashicorp/f-atomic-kv
...
Adds support for atomic transactions spanning multiple KV entries.
2016-05-15 13:46:05 -07:00
3756fb23a6
Remove unused peers variable from setupRaft().
2016-05-15 06:40:46 -07:00
a11f32a1da
Adds a get-tree verb to KV transaction operations.
2016-05-13 16:57:39 -07:00
0f94a7a326
Switches GETs to a filtering model for ACLs.
2016-05-13 15:58:55 -07:00
5fd99b13ef
Removes null results for deletes, and preps for more than one result from an operation.
2016-05-13 01:47:55 -07:00
2649a6336e
Adds a read-only optimized path for transactions.
2016-05-13 00:34:05 -07:00
0c34ed078c
Adds a comment for the txnKVS() function.
2016-05-12 16:11:26 -07:00
88b1c7d054
Makes get fail a transaction if the key doesn't exist.
2016-05-11 14:18:31 -07:00
3d35acaa90
De-nests the KV output structure (removes DirEnt member).
2016-05-11 13:48:03 -07:00
James Phillips