Sean Chittenden
73497f7915
Move consul.serverConfig out of the consul package
...
Relocated to its own package, server_manager. This now greatly simplifies the RPC() call path and appropriately hides the locking behind the package boundary. More work is needed to be done here
2016-03-23 22:10:50 -07:00
Sean Chittenden
2a52d3eb80
Rename serverConfigMtx to serverConfigLock
...
Pointed out by: @slackpad
2016-03-23 22:10:32 -07:00
Sean Chittenden
49425c5371
Refactor out the management of Consul servers
...
Move the management of c.consulServers (fka c.consuls) into consul/server_manager.go.
This commit brings in a background task that proactively manages the server list and:
*) reshuffles the list
*) manages the timer out of the RPC() path
*) uses atomics to detect a server has failed
This is a WIP, more work in testing needs to be completed.
2016-03-23 22:09:46 -07:00
Sean Chittenden
ebdccf0f35
Move consul.serverConfig out of the consul package
...
Relocated to its own package, server_manager. This now greatly simplifies the RPC() call path and appropriately hides the locking behind the package boundary. More work is needed to be done here
2016-03-23 22:05:29 -07:00
Sean Chittenden
b7213d9daa
Rename serverConfigMtx to serverConfigLock
...
Pointed out by: @slackpad
2016-03-23 22:05:05 -07:00
Sean Chittenden
e29b8de0a6
Refactor out the management of Consul servers
...
Move the management of c.consulServers (fka c.consuls) into consul/server_manager.go.
This commit brings in a background task that proactively manages the server list and:
*) reshuffles the list
*) manages the timer out of the RPC() path
*) uses atomics to detect a server has failed
This is a WIP, more work in testing needs to be completed.
2016-03-23 22:03:20 -07:00
Sean Chittenden
3730eaf6df
Commit miss re: consuls variable rename
2016-03-23 16:24:29 -07:00
Sean Chittenden
b33648ca5c
Move consul.serverConfig out of the consul package
...
Relocated to its own package, server_manager. This now greatly simplifies the RPC() call path and appropriately hides the locking behind the package boundary. More work is needed to be done here
2016-03-23 16:16:22 -07:00
Sean Chittenden
f3a69c939d
Refactor consul.serverParts into server_details.ServerDetails
...
This may be short-lived, but it also seems like this is going to lead us down a path where ServerDetails is going to evolve into a more powerful package that will encapsulate more behavior behind a coherent API.
2016-03-23 16:15:47 -07:00
Sean Chittenden
b3192ca410
Rename serverConfigMtx to serverConfigLock
...
Pointed out by: @slackpad
2016-03-23 16:15:47 -07:00
Sean Chittenden
82458fa9e8
Handle the case where there are no healthy servers
...
Pointed out by: @slackpad
2016-03-23 16:15:47 -07:00
Sean Chittenden
09d4c6439c
Refactor out the management of Consul servers
...
Move the management of c.consulServers (fka c.consuls) into consul/server_manager.go.
This commit brings in a background task that proactively manages the server list and:
*) reshuffles the list
*) manages the timer out of the RPC() path
*) uses atomics to detect a server has failed
This is a WIP, more work in testing needs to be completed.
2016-03-23 16:15:47 -07:00
Sean Chittenden
6bda2c007c
Add a flag to denote that a server is disabled
...
A server is not normally disabled, but in the event of an RPC error, we want to mark a server as down to allow for fast failover to a different server. This value must be an int in order to support atomic operations.
Additionally, this is the preliminary work required to bring up a server in a disabled state. RPC health checks in the future could mark the server as alive, thereby creating an organic "slow start" feature for Consul.
2016-03-23 16:14:59 -07:00
Sean Chittenden
7de85906c1
Rename `lastServer` to `preferredServer`
...
Expanding the domain of lastServer beyond RPC() changes the meaning of this variable. Rename accordingly to match the intent coming in a subsequent commit: a background thread will be in charge of rotating preferredServer.
2016-03-23 16:14:59 -07:00
Sean Chittenden
2949980a64
Warn if serf events have queued up past 80% of the limit
...
It is theoretically possible that the number of queued serf events can back up. If this happens, emit a warning message if there are more than 200 events in queue.
Most notably, this can happen if `c.consulServerLock` is held for an "extended period of time". The probability of anyone ever seeing this log message is hopefully low to nonexistent, but if it happens, the warning message indicating a large number of serf events fired while a lock was held is likely to be helpful (vs serf mysteriously blocking when attempting to add an event to a channel).
2016-03-23 16:14:11 -07:00
Sean Chittenden
2a0c12460d
Commit miss re: consuls variable rename
2016-03-23 16:13:49 -07:00
Sean Chittenden
3ac1bcc799
Remove lastRPCTime
...
This mechanism isn't going to provide much value in the future. Preemptively reduce the complexity of future work.
2016-03-23 16:13:49 -07:00
Sean Chittenden
72b7856045
Rename c.consuls to c.consulServers
...
Prep for breaking out maintenance of consuls into a new goroutine.
2016-03-23 16:10:27 -07:00
Sean Chittenden
d1ef4ec7e2
Use `rand.Int31n()` to get power of two optimization
...
In cases where i+1 is a power of two, skip one modulo operation.
2016-03-23 16:00:39 -07:00
James Phillips
92e947dcc3
Gets rid of flaky sort check.
...
If we get a coordinate then this test will fail, so we only check the
first item in the list, which is deterministic.
2016-03-21 17:30:05 -07:00
James Phillips
265a8d4053
Increases timeouts for coordinate tests.
...
We take the interval and add the random stagger to it, so 2X is cutting it
too close and the unit tests are often flaky.
2016-03-21 16:44:35 -07:00
James Phillips
13b8ce0adc
Merge pull request #1851 from hashicorp/f-ipv6-bind
...
Allow [::] as a bind address (binds to first public IPv6 address)
2016-03-19 16:16:19 -07:00
James Phillips
18e12aa886
Adds more specific checks for ipv6 addresses.
2016-03-19 16:14:45 -07:00
James Phillips
e4ca18089f
Removes leader from members and changes name since it's an address.
2016-03-18 17:07:11 -07:00
Sergey Romanov
11b73bb1a5
#735 add information about leader to consul members
2016-03-18 17:05:40 -07:00
Wim
508bc796a8
Allow [::] as a bind address (binds to first public IPv6 address)
2016-03-18 23:59:44 +01:00
Calvin Leung Huang
7215d9bdef
Obfuscate token for lookupACL error
2016-03-15 17:16:25 -04:00
James Phillips
a9d640c024
Hardens the match interoplator against negative arguments.
2016-03-07 13:32:32 -08:00
James Phillips
63c826c2c0
Adds a comment about the embedded struct.
2016-03-07 10:45:39 -08:00
James Phillips
275c84a0cc
Renames "debug" endpoint and structures to "explain".
2016-03-07 10:45:39 -08:00
James Phillips
8493640b09
Adds a prepared query debug endpoint.
2016-03-07 10:45:39 -08:00
James Phillips
918b1ace47
Applies prefix ACL to a catch-all template as a special case.
2016-03-07 10:45:39 -08:00
James Phillips
3c512fc089
Adds a test for the custom prepared query template indexer.
2016-03-07 10:45:39 -08:00
James Phillips
39d3094d50
Adds core query template tests to the state store.
2016-03-07 10:45:39 -08:00
James Phillips
06087633f0
Adds in basic query template lookups and vendors newly-updated memdb as well as improved iradix tree.
2016-03-07 10:45:39 -08:00
James Phillips
142e69befe
Adds tests for the low-level template functions.
2016-03-07 10:45:39 -08:00
James Phillips
b578fbbfc4
Adds tests for the string visitor.
2016-03-07 10:45:39 -08:00
James Phillips
2a9a5f823e
Factors rendering down into the resolve function.
2016-03-07 10:45:39 -08:00
James Phillips
8e25451232
Splits walk functions out from the rest of the template code.
2016-03-07 10:45:39 -08:00
James Phillips
fa60d575bf
Integrates templates into state store and endpoint (sans tests).
2016-03-07 10:45:39 -08:00
James Phillips
62405110dc
Wraps the prepared query to also store the compiled template.
2016-03-07 10:45:39 -08:00
James Phillips
98281be7df
Adds basic query template compiler and renderer.
2016-03-07 10:45:39 -08:00
Mike Cowgill
25613895e3
one line schema change to not allow missing for sessions Table node index, Fixes #1774
2016-03-02 21:19:53 -08:00
James Phillips
f0150ff5ce
Adds missing token redact in the GET path.
2016-02-26 15:59:00 -08:00
James Phillips
48f2089d7f
Merge pull request #1757 from hashicorp/f-revert-1667
...
Reverts server connection rebalancing changes from #1667
2016-02-24 18:07:13 -08:00
James Phillips
c75256ac8b
Adds a check for users re-submitting the redacted token.
2016-02-24 17:35:26 -08:00
James Phillips
2f7eac8b86
Renames "prepared_query" ACL policy to "query".
2016-02-24 17:02:06 -08:00
James Phillips
3b91618d7d
Changes to more idiomatic "ok" pattern for prefix getter.
2016-02-24 16:26:43 -08:00
James Phillips
1c7ee582f9
Renames a unit test.
2016-02-24 16:17:20 -08:00
James Phillips
d660311fbb
Revert "Merge pull request #1667 from hashicorp/b-redistribute-clients"
...
This reverts commit 8f30dea4209491ebbe4ef9ab94dd8052d17bdbe9, reversing
changes made to eb27a02956e7e052c0bec6f96a0c0f7f6675f6a6.
2016-02-24 15:38:03 -08:00
James Phillips
54f0b7bbb6
Completes switch of prepared_query ACLs to govern query names.
2016-02-24 01:26:16 -08:00
James Phillips
633c231d67
Creates new "prepared-query" ACL type and new token capture behavior.
...
Prior to this change, prepared queries had the following behavior for
ACLs, which will need to change to support templates:
1. A management token, or a token with read access to the service being
queried needed to be provided in order to create a prepared query.
2. The token used to create the prepared query was stored with the query
in the state store and used to execute the query.
3. A management token, or the token used to create the query needed to be
supplied to perform and CRUD operations on an existing prepared query.
This was pretty subtle and complicated behavior, and won't work for
templates since the service name is computed at execution time. To solve
this, we introduce a new "prepared-query" ACL type, where the prefix
applies to the query name for static prepared query types and to the
prefix for template prepared query types.
With this change, the new behavior is:
1. A management token, or a token with "prepared-query" write access to
the query name or (soon) the given template prefix is required to do
any CRUD operations on a prepared query, or to list prepared queries
(the list is filtered by this ACL).
2. You will no longer need a management token to list prepared queries,
but you will only be able to see prepared queries that you have access
to (you get an empty list instead of permission denied).
3. When listing or getting a query, because it was easy to capture
management tokens given the past behavior, this will always blank out
the "Token" field (replacing the contents as <hidden>) for all tokens
unless a management token is supplied. Going forward, we should
discourage people from binding tokens for execution unless strictly
necessary.
4. No token will be captured by default when a prepared query is created.
If the user wishes to supply an execution token then can pass it in via
the "Token" field in the prepared query definition. Otherwise, this
field will default to empty.
5. At execution time, we will use the captured token if it exists with the
prepared query definition, otherwise we will use the token that's passed
in with the request, just like we do for other RPCs (or you can use the
agent's configured token for DNS).
6. Prepared queries with no name (accessible only by ID) will not require
ACLs to create or modify (execution time will depend on the service ACL
configuration). Our argument here is that these are designed to be
ephemeral and the IDs are as good as an ACL. Management tokens will be
able to list all of these.
These changes enable templates, but also enable delegation of authority to
manage the prepared query namespace.
2016-02-23 17:12:43 -08:00
James Phillips
f398e1880e
Adds a test for node registration and tagged addresses.
2016-02-07 13:15:22 -08:00
James Phillips
f163522f0c
Moves tagged wan address to be managed by anti-entropy, not serf.
2016-02-07 13:12:42 -08:00
James Phillips
3f50d2ae7e
Adds an FSM persist and restore test for tagged addresses.
2016-02-07 11:36:39 -08:00
James Phillips
79bd1fd4bb
Sets up config for more address tags down the road, renames struct members.
2016-02-07 10:37:34 -08:00
Evan Gilman
71cf39b5f8
Use a map for additional node addresses
2016-02-06 23:01:45 -08:00
Evan Gilman
e166d2a4c7
Use idiomatic name for wan_addr serf tag
2016-02-06 23:01:45 -08:00
James Phillips
1a828e3927
Store WanAddress during Service/Check sync
2016-02-06 23:01:45 -08:00
Evan Gilman
fc61143b46
Store WanAddress during node registration
2016-02-06 23:01:45 -08:00
Evan Gilman
369e501dd5
Store WanAddress on Node
2016-02-06 23:01:45 -08:00
Sean Chittenden
fc82b351b8
Use the server's address in debug logging, not the c.lastServer, which may be nil
2016-02-02 15:51:28 -08:00
Sean Chittenden
58225e0ee3
Remove unnecessary check, test was moved further up in scope
2016-02-02 11:13:58 -08:00
Sean Chittenden
6452b498e1
Use panic instead of returning a sentinel UUID values in unit tests
2016-02-01 23:15:19 -08:00
Sean Chittenden
ef8bbca48f
Continually rebalance client connections
...
Introduce a low-level background connection expiration mechanism wherein connections will be recycled periodically based on the size and health of the cluster.
For the vast majority of consul users, this will mean an average connection age of 150s. For 10K node clusters it will take ~3min for clusters to rebalance their connections. In the pathological case for a 100K cluster where 99K clients are in the minority talking to 1x server it will take ~26min to rebalance all connections.
It's possibe for clients recovering from a parititon to become fixated on a single server until the server or agent is restarted. This is of particular interest to long-running environments with stable agents, where `allow_stale` is true, and partitions occur periodically.
2016-01-30 17:13:50 -08:00
Sean Chittenden
8a37e76cb0
Use rand.Int31n() vs unconditionally using modulus
2016-01-30 15:47:58 -08:00
Sean Chittenden
727bb42b1d
Merge branch 'f-consul-lib' of ssh://github.com/hashicorp/consul into b-redistribute-clients
2016-01-30 15:40:54 -08:00
Sean Chittenden
b216d4c11f
Rename clientRPCCache to clientRPCConnMaxIdle, change value
...
Increase the max idle time for agents talking to servers from 30s to 127s in order to allow for the reuse of connections that are being initiated by cron.
127s was chosen as the first prime above 120s (arbitrarily chose to use a prime) with the intent of reusing connections who are used by once-a-minute cron(8) jobs *and* who use a 60s jitter window (e.g. in vixie cron job execution can drift by up to 59s per job, or 119s for a once-a-minute cron job).
2016-01-30 15:27:46 -08:00
Sean Chittenden
e83a5b7a70
Reuse the results from gettimeofday(2)...
...
Inside of a single RPC call, reuse time.Now().
2016-01-30 14:39:17 -08:00
Sean Chittenden
c470553b6b
Factor out duplicate functions into a lib package
...
Consolidate code duplication and tests into a single lib package. Most of these functions were from various **/util.go functions that couldn't be imported due to cyclic imports. The consul/lib package is intended to be a terminal node in an import DAG and a place to stash various consul-only helper functions. Pulled in hashicorp/go-uuid instead of consolidating UUID access.
2016-01-29 16:57:45 -08:00
James Phillips
0010b788ed
Prevents watches from being orphaned when KVS blocking queries loop.
2016-01-20 07:18:47 -08:00
James Phillips
e0ae4878f6
Merge pull request #948 from hashicorp/iface-down-fix
...
Don't try to bind on address from inactive interface
2016-01-14 17:00:54 -08:00
Ryan Uber
e3ac96679e
consul: address comments
2016-01-05 09:45:36 -08:00
Ryan Uber
689698eefa
consul: disable serf snapshots in dev mode
2015-12-26 20:39:49 -05:00
Ryan Uber
ce4cf8a542
consul: dev mode works
2015-12-26 20:19:36 -05:00
James Phillips
533c79bd2b
Cleans up some small `go vet` findings.
2015-12-18 22:14:48 -08:00
James Phillips
92531812b6
Adds source address logging on RPC errors.
2015-12-16 14:03:03 -08:00
James Phillips
38bdcc9907
Adds a check to make sure query names can't be registered twice.
2015-12-02 09:04:51 -08:00
Armon Dadgar
a033d7aed2
consul: shrink yamux recv buffer on idle streams
2015-11-27 17:20:57 -08:00
James Phillips
0e3f4af4ac
Makes all the query ops the correct type.
2015-11-17 09:27:10 -08:00
James Phillips
86bc3db7b3
Returns a zero index for a lookup error case.
2015-11-17 09:25:20 -08:00
James Phillips
bc60491890
Removes a useless empty import and fixes some stale comments.
2015-11-17 08:29:20 -08:00
James Phillips
f4943c1613
Makes UUID regex case-insensitive.
2015-11-16 22:57:47 -08:00
James Phillips
a1e02996e5
Moves conversion of nil slices up to HTTP layer for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
c032dee945
Adds a paranoia set of the nodes slice to nil.
2015-11-15 17:06:00 -08:00
James Phillips
a0211db3b9
Adds a test to ensure we don't return a nil slice.
2015-11-15 17:06:00 -08:00
James Phillips
375312f07a
Gets rid of some unused constants.
2015-11-15 17:06:00 -08:00
James Phillips
e6ccf5b9ed
Returns a 404 from a get or execute of a nonexistent query.
2015-11-15 17:06:00 -08:00
James Phillips
c9ef552385
Plumbs the service name back and uses agent-specific TTL settings as a fallback.
2015-11-15 17:06:00 -08:00
James Phillips
cad6938474
Adds unit tests for prepared queries and DNS, using existing tests for equivalence.
2015-11-15 17:06:00 -08:00
James Phillips
7762a828d3
Adds query metadata to prepared query execute response.
2015-11-15 17:06:00 -08:00
James Phillips
0c02365bf2
Makes an empty prepared query list an empty slice, not a nil one.
2015-11-15 17:06:00 -08:00
James Phillips
3029906254
Adds a unit test for the new RTT getDatacentersByDistance fn.
2015-11-15 17:06:00 -08:00
James Phillips
697e2f4e71
Completes FSM support for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
124410b66c
Adds an HTTP endpoint for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
b209c2afe2
Adds an RPC endpoint injection method for testing.
2015-11-15 17:06:00 -08:00
James Phillips
4051db4e88
Changes Lookup to Get since we don't need it (only Execute does).
2015-11-15 17:06:00 -08:00
James Phillips
095806584d
Always increments the failovers counter, even for error-ed DCs.
2015-11-15 17:06:00 -08:00
James Phillips
07361d3e96
Adds test for remote datacenter selection and query logic.
2015-11-15 17:06:00 -08:00
James Phillips
6a1c571a38
Adds a test for the server wrapper.
2015-11-15 17:06:00 -08:00
James Phillips
9428f025f9
Adds tag filter tests.
2015-11-15 17:06:00 -08:00
James Phillips
da5cf9cdf2
Adds execute tests for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
7c8404df4c
Adds status information about failovers to query results.
2015-11-15 17:06:00 -08:00
James Phillips
dc517aa288
Removes unused ACL filter.
2015-11-15 17:06:00 -08:00
James Phillips
00ea015770
Adds execute leader forward test for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
264e86e533
Adds a leader forwarding case for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
bb0f136412
Adds tests for query lookup and list endpoints.
2015-11-15 17:06:00 -08:00
James Phillips
495d00c0b8
Adds query parsing unit tests.
2015-11-15 17:06:00 -08:00
James Phillips
72f6c8a261
Adds ACL cases for apply.
2015-11-15 17:06:00 -08:00
James Phillips
76a55eed53
Completes non-ACL version of apply test.
2015-11-15 17:06:00 -08:00
James Phillips
bc0fba43d8
Adds lookup and list endpoints and basic end-to-end apply test.
2015-11-15 17:06:00 -08:00
James Phillips
3b3c7c02e4
Checks for valid UUIDs before calling in to index function.
2015-11-15 17:06:00 -08:00
James Phillips
ec5249357e
Clarifies comment about name vs. ID.
2015-11-15 17:06:00 -08:00
James Phillips
61e6cbc560
Skips unknown DCs during queries and chugs along in the face of errors.
2015-11-15 17:06:00 -08:00
James Phillips
0141438e6c
Moves sort to a query-time decision and adds back the limit.
2015-11-15 17:06:00 -08:00
James Phillips
25fac70924
Adds an explicit ACL check that will fail vs. trying other DCs.
2015-11-15 17:06:00 -08:00
James Phillips
347bb847c2
Changes "not" prefix from "~" to "!".
2015-11-15 17:06:00 -08:00
James Phillips
09034a84bd
Adds prefix "prepared" to everything prepared query-related.
2015-11-15 17:06:00 -08:00
James Phillips
2183565d83
Adds basic structure for prepared queries (needs tests).
2015-11-15 17:06:00 -08:00
James Phillips
6e7faa6239
Factors code for pulling the sorted list of DCs into a common place.
2015-11-15 17:06:00 -08:00
James Phillips
6a70cb9885
Adds a better shuffle test (similar to DNS).
2015-11-15 17:06:00 -08:00
James Phillips
781f9611e8
Changes structs and state store for prepared queries.
2015-11-15 17:06:00 -08:00
James Phillips
55f49d6e73
Merge pull request #1413 from hashicorp/b-coord-raft-errors
...
Adds a check for in-band error returns in the coordinate RaftApply.
2015-11-15 17:05:19 -08:00
James Phillips
26cf0a2974
Extends the session TTL max to 24 hours, and adds a warning to the docs.
2015-11-15 16:51:00 -08:00
James Phillips
04dffd7c46
Adds a check for in-band error returns in the coordinate RaftApply.
2015-11-15 16:50:46 -08:00
James Phillips
5b0697fc67
Changes batch update failure to a WARN since it's nbd.
2015-10-29 09:26:24 -07:00
James Phillips
eb4bfa3411
Prevents agents from considering Raft information when doing sync checks.
2015-10-28 14:32:00 -07:00
James Phillips
7c44a9b6c9
Fixes a bad error message.
2015-10-28 12:40:47 -07:00
James Phillips
5449096bc2
Adds a check for the minimum version as well.
2015-10-27 15:56:36 -07:00
James Phillips
1c678effde
Sets the ignore flag on coordinate update Raft log entries.
2015-10-27 14:44:34 -07:00
James Phillips
c1305a08ea
Makes the version upshift code look at the correct version field.
2015-10-27 14:44:34 -07:00
James Phillips
b91e2d3a97
Completes rebase of network coordinates to new memdb.
2015-10-23 15:23:01 -07:00
James Phillips
132e1d813b
Fixes configs now that Serf always caches coordinates.
2015-10-23 15:23:01 -07:00
James Phillips
aea2194ce3
Makes the default protocol 2 and lets 3 interoperate with 2.
2015-10-23 15:23:01 -07:00
James Phillips
7dd88f7de6
Zeroes out the height when testing exact distances.
2015-10-23 15:23:01 -07:00
James Phillips
7d5e29f6c6
Cleans up after latest rebase.
2015-10-23 15:23:01 -07:00
James Phillips
a74bdcba49
Moves sorting up into coordinate endpoint HTTP handlers.
2015-10-23 15:23:01 -07:00
James Phillips
a2bcef00a0
Adds endpoints for raw network coordinates.
2015-10-23 15:23:01 -07:00
James Phillips
e900fef95a
Fixes bad name for DC forwarding of Coordinate.Get.
2015-10-23 15:23:01 -07:00
James Phillips
ee1cf1e13b
Adds distance sorting to health endpoint. Cleans up unit tests.
2015-10-23 15:23:01 -07:00
James Phillips
019f656f39
Switches to the median over all DC nodes with known coordinates.
2015-10-23 15:23:01 -07:00
James Phillips
5609b2e889
Adds explicit check for empty node in source parameter.
2015-10-23 15:23:01 -07:00
James Phillips
5d75ce7852
Moves disable checks down into the sort routine.
2015-10-23 15:23:01 -07:00
James Phillips
59b710894d
Adds tests for HTTP interface. Removes a stray mark.
2015-10-23 15:23:01 -07:00
James Phillips
033e8e6625
Adds sort of DCs in catalog queries based on RTT. Cleans up.
...
* Makes the catalog endpoint respect disabling coordinates for all
RTT-sorting query types.
2015-10-23 15:23:01 -07:00
James Phillips
b63909cf67
Adds coordinate sorting support to catalog queries for nodes and service nodes.
2015-10-23 15:23:01 -07:00
James Phillips
9ba9a708f6
Scales coordinate sends to hit a fixed aggregate rate across the cluster.
2015-10-23 15:23:01 -07:00
James Phillips
d8b8a3719f
Simplifies the batching function and adds some comments.
2015-10-23 15:23:01 -07:00
James Phillips
a93a1a68b6
Adds snapshot save and restore of coordinates.
2015-10-23 15:23:01 -07:00
James Phillips
f71c79c53f
Does some small cleanups based on PR feedback.
...
* Holds coordinate updates in map and gets rid of the update channel.
* Cleans up config variables a bit.
2015-10-23 15:23:01 -07:00
James Phillips
1222772452
Hardens Consul from bad coordinates from other nodes.
2015-10-23 15:23:01 -07:00
James Phillips
e02ae7b6b4
Takes the node name out of the coordinate get call.
2015-10-23 15:23:01 -07:00
James Phillips
acb0dce829
Moves batching down into the state store and changes it to fail-fast.
...
* A batch of updates is done all in a single transaction.
* We no longer need to get an update to kick things, there's a periodic flush.
* If incoming updates overwhelm the configured flush rate they will be dumped with an error.
2015-10-23 15:23:01 -07:00
James Phillips
b6c31bdf2f
Flips the sense of the coordinate enable option.
2015-10-23 15:23:01 -07:00
James Phillips
9c069c5031
Removes one more WAN leftover.
2015-10-23 15:23:01 -07:00
James Phillips
edb9a119e2
Does a clean up pass on the Consul side.
2015-10-23 15:23:01 -07:00
James Phillips
ac4185b888
Merges config changes after rebase.
2015-10-23 15:23:01 -07:00
Derek Chiang
b805215237
Address comments
2015-10-23 15:23:01 -07:00
Derek Chiang
7d70d8b1d7
Add test for the GetWAN endpoint
2015-10-23 15:23:01 -07:00
Derek Chiang
e03a9d4c38
Add an endpoint for getting WAN coordinates
2015-10-23 15:23:01 -07:00
Derek Chiang
213f5a15e6
Fix tests
2015-10-23 15:23:01 -07:00
Derek Chiang
eb599a1745
Address comments
2015-10-23 15:23:01 -07:00
Derek Chiang
66d5a129bf
Add state store tests
2015-10-23 15:23:01 -07:00
Derek Chiang
88550714ca
Add more tests
2015-10-23 15:23:01 -07:00
Derek Chiang
139c9240ea
Address comments
2015-10-23 15:23:01 -07:00
Derek Chiang
23c08aeeb4
Use IndexedCoordinate instead
2015-10-23 15:23:01 -07:00
Derek Chiang
979c0c6c9e
Improve a test
2015-10-23 15:23:01 -07:00
Derek Chiang
b2cff43bb5
Complete logic for sending coordinates
2015-10-23 15:23:01 -07:00
Derek Chiang
019da1dae4
Fix tests
2015-10-23 15:23:01 -07:00
Derek Chiang
66b210afcb
Some fixes
2015-10-23 15:23:01 -07:00
Derek Chiang
def0a2f5ab
Change GET API a little bit
2015-10-23 15:23:01 -07:00
Derek Chiang
b5bbe2bcfa
Adding tests and stuff
2015-10-23 15:23:01 -07:00
Derek Chiang
e54c8f2ea0
Start adding stuff
2015-10-23 15:23:01 -07:00
James Phillips
aae298a179
Cleans up a little whitespace with go fmt.
2015-10-20 18:41:05 -07:00
James Phillips
54535d45d7
Gets rid of error prefixing in leader.go.
2015-10-20 13:37:11 -07:00
James Phillips
eb93d1d9ad
Puts all restore operations into a single transaction and optimizes watches.
2015-10-19 23:06:59 -07:00
James Phillips
96c5c9de2b
Gets rid of unique constraint on sessions index in session_checks.
2015-10-19 17:41:50 -07:00
James Phillips
3e93055cef
Switches sessions over to UUIDFieldIndex.
2015-10-19 17:09:02 -07:00
James Phillips
d905804514
Adds a special case for fill KVS listings to avoid a tombstone scan.
2015-10-19 16:22:27 -07:00
James Phillips
22a46f7bf5
Makes the iterator naming more consistent.
2015-10-19 15:51:11 -07:00
James Phillips
9a2fdff4c4
Converts sessions and ACLs over to iterators.
2015-10-19 14:56:22 -07:00
James Phillips
d459d94b3f
Converts KVS snapshot over to iterator.
2015-10-19 14:07:57 -07:00
James Phillips
498c4a989a
Converts nodes, services, checks to iterators duing dumps; fixes tag drift bug.
...
Realized that the conversions ServiceNode <-> NodeService were incomplete in a
few places so centralized those and added some tests.
2015-10-19 13:55:35 -07:00
James Phillips
8ee52e9850
Gets rid of non-idomatic "state_store" alias in FSM.
2015-10-16 14:07:48 -07:00
James Phillips
adef4894df
Ports a couple of new RPC calls to the updated codec mechanism.
2015-10-15 15:09:13 -07:00
James Phillips
268255dcbe
Runs go fmt after latest changes.
2015-10-15 14:59:09 -07:00
James Phillips
e982f976eb
Adds unit tests for new structs clone functions.
2015-10-15 14:59:09 -07:00
James Phillips
46be9fa2cf
Adds benchmarks back in to the state store.
2015-10-15 14:59:09 -07:00
James Phillips
6db1a8624d
Removes a todo that's no longer needed.
2015-10-15 14:59:09 -07:00
James Phillips
82f275d126
Adds unit tests for the graveyard.
2015-10-15 14:59:09 -07:00
James Phillips
e285af5d1c
Does some go fmt after latest round of changes.
2015-10-15 14:59:09 -07:00
James Phillips
409fd57e4a
Adds a delay test.
2015-10-15 14:59:09 -07:00
James Phillips
73ad5f0695
Adds watch unit tests and does some related watch cleanup.
2015-10-15 14:59:09 -07:00
James Phillips
9fce4aaf35
Adds tests for GC.
2015-10-15 14:59:09 -07:00
James Phillips
ca220c9717
Adds clone for service nodes so we don't twiddle the database's object.
2015-10-15 14:59:09 -07:00
James Phillips
25d7746f38
Fixes remaining non-KV index calclulations and adds a general getWatchTables thing.
2015-10-15 14:59:09 -07:00
James Phillips
32e2439f93
Fixes index management for KVS.
2015-10-15 14:59:09 -07:00
James Phillips
37da1faaf2
Makes session invalidate loops use a separate slice to protect the iterator.
2015-10-15 14:59:09 -07:00
James Phillips
8504646900
Makes all delete loops use a separate slice to protect the iterator.
2015-10-15 14:59:09 -07:00
James Phillips
768f6fd8db
Cleans up some go vet warnings.
2015-10-15 14:59:09 -07:00
James Phillips
495b276e44
Gets rid of todo that we discussed is ok.
2015-10-15 14:59:09 -07:00
James Phillips
3fedffbffb
Gets rid of the transitional "New" suffix on state store and RPC.
2015-10-15 14:59:09 -07:00
James Phillips
263c7e3fd3
Deletes the old state store and all its accoutrements.
2015-10-15 14:59:09 -07:00
James Phillips
3f07f0685f
Nukes old state store's connection to FSM and RPC.
2015-10-15 14:59:09 -07:00
James Phillips
450886246d
Integrates session TTL tests with new state store.
2015-10-15 14:59:09 -07:00
James Phillips
a15c24f771
Integrates new state store into internal endpoint.
2015-10-15 14:59:09 -07:00
James Phillips
76bdeadefb
Abstracts the table names away from the RPC call sites.
2015-10-15 14:59:09 -07:00
James Phillips
7a8e5b2866
Integrates new state store into session endpoint; returns table index always.
2015-10-15 14:59:09 -07:00
James Phillips
1463e6100a
Integrates new state store into leader and catalog/health endpoints.
2015-10-15 14:59:09 -07:00
James Phillips
6203c1e585
Integrates KVS endopint with new state store (changes KVSList to match old behavior).
2015-10-15 14:59:09 -07:00
James Phillips
837d8994b4
Cuts FSM unit tests over to new state store.
2015-10-15 14:59:09 -07:00
James Phillips
92e48b87bf
Adds twiddling of the real state store in snapshot tests.
2015-10-15 14:59:09 -07:00
James Phillips
87ff8d031c
Ports over session invalidation tests (and fixes some bugs).
2015-10-15 14:59:09 -07:00
James Phillips
8072138c9a
Adds a note about updating sessions after they are created.
2015-10-15 14:59:09 -07:00
James Phillips
a09571133c
Beefs up node and service watch tests for multi-table triggers.
2015-10-15 14:59:09 -07:00
James Phillips
ae21a33aca
Adds session snapshot/restore and basic watch tests (and fixes some bugs).
2015-10-15 14:59:09 -07:00
James Phillips
1725063067
Adds tombstone tests and gets rid of unused logger.
2015-10-15 14:59:09 -07:00
James Phillips
391c04de90
Makes sure we don't create a full table watch for tombstones.
2015-10-15 14:59:09 -07:00
James Phillips
4be951571e
Adds snapshot/restore and watch tests for KVS.
2015-10-15 14:59:09 -07:00
James Phillips
9bfe2c32f6
Allows lock holder to re-lock and set a KV, adds tests for corner cases around sessions.
2015-10-15 14:59:09 -07:00
James Phillips
1088145060
Adds kvs lock/unlock tests.
2015-10-15 14:59:09 -07:00
James Phillips
2c0ba7e44b
Adds snap and watch tests for nodes, services, and checks.
2015-10-15 14:59:09 -07:00
James Phillips
f7fca62dcd
Adds node snap and watch tests as well as a general watch test.
2015-10-15 14:59:09 -07:00
James Phillips
e25fb6c5ed
De-generalizes graveyard since that ended up as a YAGNI (only useful for KV).
2015-10-15 14:59:09 -07:00
James Phillips
e7de7a2b8c
Cleans up unit tests for consistency.
2015-10-15 14:59:09 -07:00
James Phillips
1bf432d5af
Adds ensure registration unit test (and fixes bugs).
2015-10-15 14:59:09 -07:00
James Phillips
e5773635e7
Adds reap tombstone test (and fixes bugs).
2015-10-15 14:59:09 -07:00
James Phillips
8a39581ae0
Adds ACL snapshot/restore test.
2015-10-15 14:59:09 -07:00
James Phillips
2dae65a6ec
Adds a watch tester helper that helps cut the cruft.
2015-10-15 14:59:09 -07:00
James Phillips
04b365495d
Completes state store for KV, sessions, tombstones, and nodes/services/checks (needs tests and integration).
2015-10-15 14:59:09 -07:00
James Phillips
edae626f36
Integrates new state store for ACLs.
2015-10-15 14:59:09 -07:00
James Phillips
cbcd977a39
Gets new structs changes to compile, adds some corner case handling and extra unit tests.
2015-10-15 14:59:09 -07:00
Ryan Uber
4cf5f6223e
consul/state: list acls
2015-10-15 14:59:09 -07:00
Ryan Uber
69b0ee0e5a
consul/state: implement acl delete
2015-10-15 14:59:09 -07:00
Ryan Uber
6e7c3e3579
consul/state: basic acl set/get/delete
2015-10-15 14:59:09 -07:00
Ryan Uber
e30c3cdea8
consul/state: basic session destroy works
2015-10-15 14:59:09 -07:00
Ryan Uber
747527fef5
consul/state: more tests
2015-10-15 14:59:09 -07:00
Ryan Uber
d0e349b190
consul/state: refactor some tests
2015-10-15 14:59:09 -07:00
Ryan Uber
d5f3648846
consul/state: session lookup by node id works
2015-10-15 14:59:09 -07:00
Ryan Uber
82fa9347e1
consul/state: add session list method
2015-10-15 14:59:09 -07:00
Ryan Uber
0d0b3c91ef
consul/state: adding session registration
2015-10-15 14:59:09 -07:00
Ryan Uber
a613f65e41
consul/state: adding tree delete for kvs store
2015-10-15 14:59:09 -07:00
Ryan Uber
66a211a70e
consul/state: list keys from the kv with a prefix/separator
2015-10-15 14:59:09 -07:00
Ryan Uber
0171c2ba3d
consul/state: fix for maxIndex and better tests
2015-10-15 14:59:09 -07:00
Ryan Uber
b0ae1c0967
consul/state: add CAS method for kv set
2015-10-15 14:59:09 -07:00
Ryan Uber
dc997beef3
consul/state: initial pass at CAS delete operation for kvs
2015-10-15 14:59:09 -07:00
Ryan Uber
57c36c274b
consul/state: adding KVSList for listing a given prefix
2015-10-15 14:59:09 -07:00
Ryan Uber
9b4f8cd800
consul/state: adding shallow delete for kvs store
2015-10-15 14:59:09 -07:00
Ryan Uber
b2dc11fed4
consul/state: basic k/v operations
2015-10-15 14:59:09 -07:00
Ryan Uber
a511e8a42d
consul/state: adding node dump methods
2015-10-15 14:59:09 -07:00