Commit Graph

183 Commits

Author SHA1 Message Date
James Phillips 1c7ee582f9 Renames a unit test. 2016-02-24 16:17:20 -08:00
James Phillips 54f0b7bbb6 Completes switch of prepared_query ACLs to govern query names. 2016-02-24 01:26:16 -08:00
James Phillips 633c231d67 Creates new "prepared-query" ACL type and new token capture behavior.
Prior to this change, prepared queries had the following behavior for
ACLs, which will need to change to support templates:

1. A management token, or a token with read access to the service being
   queried needed to be provided in order to create a prepared query.

2. The token used to create the prepared query was stored with the query
   in the state store and used to execute the query.

3. A management token, or the token used to create the query needed to be
   supplied to perform and CRUD operations on an existing prepared query.

This was pretty subtle and complicated behavior, and won't work for
templates since the service name is computed at execution time. To solve
this, we introduce a new "prepared-query" ACL type, where the prefix
applies to the query name for static prepared query types and to the
prefix for template prepared query types.

With this change, the new behavior is:

1. A management token, or a token with "prepared-query" write access to
   the query name or (soon) the given template prefix is required to do
   any CRUD operations on a prepared query, or to list prepared queries
   (the list is filtered by this ACL).

2. You will no longer need a management token to list prepared queries,
   but you will only be able to see prepared queries that you have access
   to (you get an empty list instead of permission denied).

3. When listing or getting a query, because it was easy to capture
   management tokens given the past behavior, this will always blank out
   the "Token" field (replacing the contents as <hidden>) for all tokens
   unless a management token is supplied. Going forward, we should
   discourage people from binding tokens for execution unless strictly
   necessary.

4. No token will be captured by default when a prepared query is created.
   If the user wishes to supply an execution token then can pass it in via
   the "Token" field in the prepared query definition. Otherwise, this
   field will default to empty.

5. At execution time, we will use the captured token if it exists with the
   prepared query definition, otherwise we will use the token that's passed
   in with the request, just like we do for other RPCs (or you can use the
   agent's configured token for DNS).

6. Prepared queries with no name (accessible only by ID) will not require
   ACLs to create or modify (execution time will depend on the service ACL
   configuration). Our argument here is that these are designed to be
   ephemeral and the IDs are as good as an ACL. Management tokens will be
   able to list all of these.

These changes enable templates, but also enable delegation of authority to
manage the prepared query namespace.
2016-02-23 17:12:43 -08:00
James Phillips 79bd1fd4bb Sets up config for more address tags down the road, renames struct members. 2016-02-07 10:37:34 -08:00
Evan Gilman 71cf39b5f8 Use a map for additional node addresses 2016-02-06 23:01:45 -08:00
Evan Gilman 369e501dd5 Store WanAddress on Node 2016-02-06 23:01:45 -08:00
James Phillips 0e3f4af4ac Makes all the query ops the correct type. 2015-11-17 09:27:10 -08:00
James Phillips bc60491890 Removes a useless empty import and fixes some stale comments. 2015-11-17 08:29:20 -08:00
James Phillips 375312f07a Gets rid of some unused constants. 2015-11-15 17:06:00 -08:00
James Phillips c9ef552385 Plumbs the service name back and uses agent-specific TTL settings as a fallback. 2015-11-15 17:06:00 -08:00
James Phillips 7762a828d3 Adds query metadata to prepared query execute response. 2015-11-15 17:06:00 -08:00
James Phillips 124410b66c Adds an HTTP endpoint for prepared queries. 2015-11-15 17:06:00 -08:00
James Phillips 4051db4e88 Changes Lookup to Get since we don't need it (only Execute does). 2015-11-15 17:06:00 -08:00
James Phillips 7c8404df4c Adds status information about failovers to query results. 2015-11-15 17:06:00 -08:00
James Phillips 76a55eed53 Completes non-ACL version of apply test. 2015-11-15 17:06:00 -08:00
James Phillips bc0fba43d8 Adds lookup and list endpoints and basic end-to-end apply test. 2015-11-15 17:06:00 -08:00
James Phillips 0141438e6c Moves sort to a query-time decision and adds back the limit. 2015-11-15 17:06:00 -08:00
James Phillips 09034a84bd Adds prefix "prepared" to everything prepared query-related. 2015-11-15 17:06:00 -08:00
James Phillips 2183565d83 Adds basic structure for prepared queries (needs tests). 2015-11-15 17:06:00 -08:00
James Phillips 6a70cb9885 Adds a better shuffle test (similar to DNS). 2015-11-15 17:06:00 -08:00
James Phillips 781f9611e8 Changes structs and state store for prepared queries. 2015-11-15 17:06:00 -08:00
James Phillips 26cf0a2974 Extends the session TTL max to 24 hours, and adds a warning to the docs. 2015-11-15 16:51:00 -08:00
James Phillips eb4bfa3411 Prevents agents from considering Raft information when doing sync checks. 2015-10-28 14:32:00 -07:00
James Phillips b91e2d3a97 Completes rebase of network coordinates to new memdb. 2015-10-23 15:23:01 -07:00
James Phillips a2bcef00a0 Adds endpoints for raw network coordinates. 2015-10-23 15:23:01 -07:00
James Phillips ee1cf1e13b Adds distance sorting to health endpoint. Cleans up unit tests. 2015-10-23 15:23:01 -07:00
James Phillips b63909cf67 Adds coordinate sorting support to catalog queries for nodes and service nodes. 2015-10-23 15:23:01 -07:00
James Phillips acb0dce829 Moves batching down into the state store and changes it to fail-fast.
* A batch of updates is done all in a single transaction.
* We no longer need to get an update to kick things, there's a periodic flush.
* If incoming updates overwhelm the configured flush rate they will be dumped with an error.
2015-10-23 15:23:01 -07:00
James Phillips 9c069c5031 Removes one more WAN leftover. 2015-10-23 15:23:01 -07:00
James Phillips edb9a119e2 Does a clean up pass on the Consul side. 2015-10-23 15:23:01 -07:00
Derek Chiang b805215237 Address comments 2015-10-23 15:23:01 -07:00
Derek Chiang eb599a1745 Address comments 2015-10-23 15:23:01 -07:00
Derek Chiang 23c08aeeb4 Use IndexedCoordinate instead 2015-10-23 15:23:01 -07:00
Derek Chiang def0a2f5ab Change GET API a little bit 2015-10-23 15:23:01 -07:00
Derek Chiang b5bbe2bcfa Adding tests and stuff 2015-10-23 15:23:01 -07:00
Derek Chiang e54c8f2ea0 Start adding stuff 2015-10-23 15:23:01 -07:00
James Phillips 498c4a989a Converts nodes, services, checks to iterators duing dumps; fixes tag drift bug.
Realized that the conversions ServiceNode <-> NodeService were incomplete in a
few places so centralized those and added some tests.
2015-10-19 13:55:35 -07:00
James Phillips 268255dcbe Runs go fmt after latest changes. 2015-10-15 14:59:09 -07:00
James Phillips e982f976eb Adds unit tests for new structs clone functions. 2015-10-15 14:59:09 -07:00
James Phillips e285af5d1c Does some go fmt after latest round of changes. 2015-10-15 14:59:09 -07:00
James Phillips ca220c9717 Adds clone for service nodes so we don't twiddle the database's object. 2015-10-15 14:59:09 -07:00
James Phillips 04b365495d Completes state store for KV, sessions, tombstones, and nodes/services/checks (needs tests and integration). 2015-10-15 14:59:09 -07:00
James Phillips cbcd977a39 Gets new structs changes to compile, adds some corner case handling and extra unit tests. 2015-10-15 14:59:09 -07:00
Ryan Uber 6e7c3e3579 consul/state: basic acl set/get/delete 2015-10-15 14:59:09 -07:00
Ryan Uber 0d0b3c91ef consul/state: adding session registration 2015-10-15 14:59:09 -07:00
Ryan Uber b2dc11fed4 consul/state: basic k/v operations 2015-10-15 14:59:09 -07:00
Ryan Uber c3cd5051ba consul/state: fetch node/check sets by service ID 2015-10-15 14:59:09 -07:00
Ryan Uber 9fe029abc3 consul/state: return highest index for queries with compound results 2015-10-15 14:59:09 -07:00
Ryan Uber ee8a1dc5d2 consul/state: set index if we have an existing health check 2015-10-15 14:59:09 -07:00
Ryan Uber 766c367ef4 consul/state: add function for returning all nodes 2015-10-15 14:59:09 -07:00
Ryan Uber 8fea5f8dc5 consul/state: track highest index when querying services 2015-10-15 14:59:09 -07:00
Ryan Uber 08553f0fef consul/state: querying node services works 2015-10-15 14:59:09 -07:00
Ryan Uber f0dd8b2923 consul/state: working on service registration storage 2015-10-15 14:59:09 -07:00
Ryan Uber 011e7584b4 consul/state: node registration and retrieval works 2015-10-15 14:59:09 -07:00
Shawn Cook 99be758411 Rename EnableTagOverride and update formatting 2015-09-11 08:35:29 -07:00
Shawn Cook f6814c89ed EnableTagDrift in NodeService struct 2015-08-18 10:34:55 -07:00
Armon Dadgar f797130228 Fixing merge conflict 2015-05-11 16:48:10 -07:00
Armon Dadgar 9b9c737a82 consul: Adding flag to support future incompatible commands. Future self will thank me. 2015-05-05 19:44:21 -07:00
Ryan Mills 2bd5bf04bd Remove 'unknown' as one of the valid states when setting the initial state of a check. 2015-04-13 20:46:01 +00:00
Ryan Mills 370853d7ff Allow specifying a status field in the agent/service/register and agent/check/register endpoints.
This status must be one of the valid check statuses: 'passing', 'warning', 'critical', 'unknown'.
If the status field is not present or the empty string, the default of 'critical' is used.
2015-04-12 02:00:31 +00:00
Michael Fraenkel c00c4ebaaa Support SesionTTLMin configuration
- Allow setting SessionTTLMin
- Validate on the Server
2015-03-27 05:13:57 -07:00
Ryan Uber 27de0adf3b agent: support multiple checks per service 2015-01-20 21:48:42 -08:00
Armon Dadgar fbc77b7501 consul: Adding KVSDeleteCAS operation type 2015-01-08 17:08:17 -08:00
Armon Dadgar 957fa1f5ec consul: Fixing merge conflict 2015-01-08 10:42:19 -08:00
Ryan Uber 5fe18b5093 Ignore vim swap files 2015-01-07 15:21:01 -08:00
Ryan Uber 7ecd015fc0 command: check LastIndex is not zero while waiting for leader 2015-01-07 15:18:00 -08:00
Armon Dadgar 0c9cbdb3d1 consul: TombstoneReapRequestType -> TombstoneRequestType 2015-01-05 14:43:55 -08:00
Armon Dadgar 02e984e4c4 consul: Adding new request to reap tombstones 2015-01-05 14:43:55 -08:00
Daniel Malon e56b3861dc advertise specific address for a service
Enable setting a specific address in a service definition for advertise. If no specific address is given it will fallback to the node address and reassemble the old behaviour.
2015-01-02 21:10:05 +00:00
Atin Malaviya 8369b77204 Clean up code based on feedback from armon 2014-12-10 20:49:06 -05:00
Atin Malaviya b623af776b Consul Session TTLs
The design of the session TTLs is based on the Google Chubby approach
(http://research.google.com/archive/chubby-osdi06.pdf). The Session
struct has an additional TTL field now. This attaches an implicit
heartbeat based failure detector. Tracking of heartbeats is done by
the current leader and not persisted via the Raft log. The implication
of this is during a leader failover, we do not retain the last
heartbeat times.

Similar to Chubby, the TTL represents a lower-bound. Consul promises
not to terminate a session before the TTL has expired, but is allowed
to extend the expiration past it. This enables us to reset the TTL on
a leader failover. The TTL is also extended when the client does a
heartbeat. Like Chubby, this means a TTL is extended on creation,
heartbeat or failover.

Additionally, because we must account for time requests are in transit
and the relative rates of clocks on the clients and servers, Consul
will take the conservative approach of internally multiplying the TTL
by 2x. This helps to compensate for network latency and clock skew
without violating the contract.

Reference: https://docs.google.com/document/d/1Y5-pahLkUaA7Kz4SBU_mehKiyt9yaaUGcBTMZR7lToY/edit?usp=sharing
2014-12-07 12:38:22 -05:00
Atin Malaviya aa0cecd04e Ephemeral Nodes for via Session behavior settings.
Added a "delete" behavior for session invalidation, in addition to
the default "release" behavior. On session invalidation, the sessions
Behavior field is checked and if it is set to "delete", all nodes owned
by the session are deleted. If it is "release", then just the locks
are released as default.
2014-11-20 11:34:45 -05:00
Ryan Uber 4a8249db00 consul: fix obscure bug when launching goroutines from for loop 2014-11-19 16:37:40 -08:00
Ryan Uber 2661bbfa27 consul: more tests, remove unused KeyManager() method 2014-11-19 16:37:40 -08:00
Ryan Uber db0084ccd0 consul: use keyring operation type to cut out duplicated logic 2014-11-19 16:34:18 -08:00
Ryan Uber 057c22db10 consul: generalize multi-DC RPC call broadcasts 2014-11-19 16:34:18 -08:00
Ryan Uber 001a579d47 command/keyring: cleanup 2014-11-19 16:34:18 -08:00
Ryan Uber 1ec111bbfc consul: kill unused struct fields 2014-11-19 16:34:17 -08:00
Ryan Uber f6b5fc8c08 consul: cross-dc key rotation works 2014-11-19 16:34:17 -08:00
Ryan Uber 71e9715c54 consul: restructuring 2014-11-19 16:34:17 -08:00
Ryan Uber a551a6e4a0 consul: refactor keyring, repeat RPC calls to all DC's 2014-11-19 16:34:17 -08:00
Armon Dadgar 3f36515544 Switching to the pinned version of msgpack 2014-10-17 18:26:19 -07:00
Armon Dadgar 34713fe970 Encode/Decode test 2014-10-17 18:23:13 -07:00
Armon Dadgar e51f9da84b consul: Deprecate ACLForceSet 2014-10-09 12:28:07 -07:00
Armon Dadgar 99d39db982 agent: First pass at multi-DC support 2014-08-28 15:00:49 -07:00
Armon Dadgar 8c5bb94c74 consul: Resolve parent ACLs 2014-08-18 15:46:23 -07:00
Armon Dadgar 5da5df716d consul: Create anonymous and master tokens 2014-08-18 15:46:22 -07:00
Armon Dadgar db8f896c58 consul: Support conditional policy fetch 2014-08-18 15:46:22 -07:00
Armon Dadgar 9a4778b7d3 consul: Enable ACL lookup 2014-08-18 15:46:22 -07:00
Armon Dadgar bd124a8da3 consul: Pulling in ACLs 2014-08-18 15:46:21 -07:00
Armon Dadgar 101d7da90a consul: Adding ACLs to the state store 2014-08-18 15:46:21 -07:00
Armon Dadgar da52fda65f consul: ACL structs 2014-08-18 15:46:21 -07:00
Armon Dadgar 9e925bf458 consul: Adding support for optional session name 2014-06-09 11:42:28 -07:00
Armon Dadgar c656bbfbcf Rename shared msgpack handle 2014-06-08 14:02:42 -07:00
Andrew M Bursavich d209517d50 reuse codec.MsgpackHandle 2014-06-07 01:13:38 -07:00
Armon Dadgar d42474a7cb Support wildcard for check lookup. Fixes #152 2014-05-21 12:45:12 -07:00
Armon Dadgar 3b0d3b76c2 consul: Adding support for lock-delay in sessions 2014-05-20 16:25:29 -07:00
Armon Dadgar d5b1330227 agent: First pass at session endpoints 2014-05-20 16:25:29 -07:00
Armon Dadgar e58e9bceb0 consul: First pass at Session RPC endpoints 2014-05-20 16:25:29 -07:00
Armon Dadgar 6ed9c4bdb6 consul: Adding support for sessions to FSM 2014-05-20 16:25:28 -07:00