luxolus/open-consul
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
6646 commits 1 branch 1 tag 358 MiB
Commit graph

267 commits

Author SHA1 Message Date
James Phillips 02a3f3f27b
Shows the segment name in the keyring API and command output. 2017-09-07 12:17:39 -07:00
James Phillips 34bae2487d
Populates the segment keyrings based on the LAN keyring. 2017-09-07 12:17:20 -07:00
James Phillips 7c616e3768
Moves reconcile loop into segment stub. 2017-09-06 18:01:53 -07:00
James Phillips 4e34c2af06
Takes the skip out of the client check. 
Without this the merge delegate won't check the segment for non-servers
a little below here.
 2017-09-06 17:05:40 -07:00
James Phillips 78ac144fff Merge pull request #3447 from hashicorp/issue-3070 
Skips unique node ID check for old versions of Consul.
 2017-09-06 13:24:15 -07:00
James Phillips 62d9299646
Fixes incorrect comment. 2017-09-06 13:23:19 -07:00
James Phillips 031f1874d0
Pulls down some code for the check loop. 2017-09-06 13:07:42 -07:00
James Phillips 2fd9328b21
Uses the Raft configuration for the self-add skip check. 2017-09-06 13:05:51 -07:00
Preetha Appan 1eae9f1e2f Change member join reconcile step to process joining itself, to handle node IP address changes correctly when number of servers < 3 2017-09-06 13:53:01 -05:00
James Phillips 353e037c9b
Skips unique node ID check for old versions of Consul. 
Fixes #3070.
 2017-09-05 22:57:29 -07:00
James Phillips aecf890054
Allow _all for WAN as a no-op. 2017-09-05 13:40:19 -07:00
James Phillips c629773b40
Makes the all segments query explict, and the default for consul members. 2017-09-05 12:22:20 -07:00
James Phillips bc9780baad Adds simple rate limiting for client agent RPC calls to Consul servers. (#3440) 
* Added rate limiting for agent RPC calls.
* Initializes the rate limiter based on the config.
* Adds the rate limiter into the snapshot RPC path.
* Adds unit tests for the RPC rate limiter.
* Groups the RPC limit parameters under "limits" in the config.
* Adds some documentation about the RPC limiter.
* Sends a 429 response when the rate limiter kicks in.
* Adds docs for new telemetry.
* Makes snapshot telemetry look like RPC telemetry and cleans up comments.
 2017-09-01 15:02:50 -07:00
Kyle Havlovitz 334e082848 Merge pull request #3431 from hashicorp/network-segments-oss 2017-09-01 10:24:58 -07:00
Kyle Havlovitz ff994e9ade
Pass listeners into setupSegments 2017-08-31 17:56:43 -07:00
Kyle Havlovitz 5cc4b32a5d
Organize segments for a cleaner split between enterprise and OSS 2017-08-31 17:39:46 -07:00
Kyle Havlovitz 5ebea7f049
Fill in the segment in the QuerySource for prepared query lookups 2017-08-31 03:35:59 -07:00
Kyle Havlovitz b77a0aa932
Fix some inconsistencies with segment logic and comments 2017-08-30 17:43:46 -07:00
Kyle Havlovitz 3b0df3350f
Default bind/advertise for segments to BindAddr/AdvertiseAddr 2017-08-30 12:51:10 -07:00
Preetha Appan 0728a04dbb Wire server provider for raft layer only on protocol version 3 and above, and update changelog 2017-08-30 14:36:47 -05:00
Kyle Havlovitz d9fc2b3d75
Update coord display in ui to account for segments 2017-08-30 11:58:29 -07:00
Kyle Havlovitz 6ded43131a
Add segment addr field to tags for LAN flood joiner 2017-08-30 11:58:29 -07:00
Kyle Havlovitz 1c04f1537a
Add agent.segment interpolation to prepared queries 2017-08-30 11:58:29 -07:00
Kyle Havlovitz 107d7f6c5a
Add rpc_listener option to segment config 2017-08-30 11:58:29 -07:00
Kyle Havlovitz e582d02079
Add segment config validation 2017-08-30 11:58:29 -07:00
James Phillips 6a6eadd8c7
Adds open source side of network segments (feature is Enterprise-only). 2017-08-30 11:58:29 -07:00
Preetha Appan e944370cde More cleanup from code review 2017-08-30 12:31:36 -05:00
Preetha Appan a215c764cd Remove copy pasted duplicate line, update documentation. 2017-08-30 10:02:10 -05:00
Preetha Appan 5a29eb7486 Consolidate server lookup into one place and replace usages of localConsuls. 2017-08-30 09:30:33 -05:00
Preetha Appan cac1c29ec5 Remove unused function 2017-08-30 09:30:33 -05:00
Preetha Appan d8fe01db4c Remove stray commented line 2017-08-30 09:30:33 -05:00
Preetha Appan ca48e7e4c2 Remove server address tracking logic from manager/router and maintain it as part of lan event listener instead. Used sync.Map to track this, and added unit tests 2017-08-30 09:30:33 -05:00
Preetha Appan b4a9d77d49 ServerAddressProvider interface also returns an error now 2017-08-30 09:30:33 -05:00
Preetha Appan edb408bc22 Use config struct to create NetworkTransport layer when setting up raft 2017-08-30 09:30:33 -05:00
Preetha Appan 01f8e469aa Implement AddressProvider and wire that up to raft transport layer to support server nodes changing their IP addresses in containerized environments 2017-08-30 09:30:33 -05:00
Frank Schroeder 62c77d70f0 build: make tests independent of build tags 
When the metadata server is scanning the agents for potential servers
it is parsing the version number which the agent provided when it
joined. This version number has to conform to a certain format, i.e.
'n.n.n'. Without this version number properly set some tests fail with
error messages that disguise the root cause.

The default version number is currently set to 'unknown' in
version/version.go which does not parse and triggers the tests to fail.
The work around is to use a build tag 'consul' which will use the
version number set in version_base.go instead which has the correct
format and is set to the current release version.

In addition, some parts of the code also require the version number to
be of a certain value. Setting it to '0.0.0' for example makes some
tests pass and others fail since they don't pass the semantic check.

When using go build/install/test one has to remember to use '-tags
consul' or tests will fail with non-obvious error messages.

Using build tags makes the build process more complex and error prone
since it prevents the use of the plain go toolchain and - at least in
its current form - introduces subtle build and test issues. We should
try to eliminate build tags for anything else but platform specific
code.

This patch removes all references to specific version numbers in the
code and tests and sets the default version to '9.9.9' which is
syntactically correct and passes the semantic check. This solves the
issue of running go build/install/test without tags for the OSS build.
 2017-08-30 13:40:18 +02:00
Frank Schroeder 84a1bf0a99 agent: drop status code comments 2017-08-23 22:36:23 +02:00
Frank Schroeder b06584f631 agent: use http.StatusRequestEntityTooLarge instead of 413 2017-08-23 22:36:23 +02:00
Frank Schroeder cc83590962 agent: use http.StatusInternalServerError instead of 500 2017-08-23 22:36:23 +02:00
Frank Schroeder bf426beb45 agent: use http.StatusMethodNotAllowed instead of 405 2017-08-23 22:36:23 +02:00
Frank Schroeder 0c3534cbf7 agent: use http.StatusNotFound instead of 404 2017-08-23 22:36:23 +02:00
Frank Schroeder 970a7f97ec agent: use http.StatusForbidden instead of 403 2017-08-23 22:36:23 +02:00
Frank Schroeder 2e586be5aa agent: use http.StatusUnauthorized instead of 401 2017-08-23 22:36:23 +02:00
Frank Schroeder 923f8e2364 agent: use http.StatusBadRequest instead of 400 2017-08-23 22:36:23 +02:00
Frank Schroeder a32eab5923 agent: support go-discover retry-join for wan 2017-08-23 21:23:34 +02:00
Frank Schröder 44e6b8122d acl: consolidate error handling (#3401) 
The error handling of the ACL code relies on the presence of certain
magic error messages. Since the error values are sent via RPC between
older and newer consul agents we cannot just replace the magic values
with typed errors and switch to type checks since this would break
compatibility with older clients.

Therefore, this patch moves all magic ACL error messages into the acl
package and provides default error values and helper functions which
determine the type of error.
 2017-08-23 16:52:48 +02:00
Frank Schroeder d9e2a51887 agent: drop unused code 
This code from http://github.com/hashicorp/consul/pull/3353 is no longer
required.
 2017-08-22 00:02:46 +02:00
Frank Schroeder 4bfcf7b613 dns: replace nameserver lookup with consistent rpc call 
This patch replaces the code which determines the list of servers in the
current cluster with an RPC call to get the list of active consul
service instances which only run on servers.

This replaces the previous implementation which was more complex and
relied on serf messages which can provide a different view than the
consistent response from the raft log.

As a side effect it makes the implementation independent of the server
and the agent which means it works consistently across both. Different
behavior for server and agent was the root cause for the bug in
http://github.com/hashicorp/consul/issue/3047.

Fixes #3407
 2017-08-22 00:02:46 +02:00
Frank Schroeder 8e1f9b9b68 dns: split node lookup from request handling 2017-08-22 00:02:46 +02:00
Frank Schroeder db8ad8922e dns: refactor label by unrolling loop 2017-08-22 00:02:46 +02:00
Frank Schroeder c35206db07 dns: move ttl closer to usage 2017-08-22 00:02:46 +02:00
James Phillips 738ac55d96
Switches to using a read lock for the agent's RPC dispatcher. 
This prevents RPC calls from getting serialized in this spot.

Fixes #3376
 2017-08-09 18:51:55 -07:00
Frank Schröder 32d4eecc1a agent: honor deprecated flags for retry-join-{ec2,azure,gce} (#3384) 2017-08-09 16:18:30 -07:00
James Phillips 3518e27a76 Revert "Return 403 rather than a 404 when acls cause all results to be filter…" 2017-08-09 15:06:57 -07:00
James Phillips 91205b2cd6 Revert "Ensure that we return a permission denied only if the list of keys/en…" 2017-08-09 15:06:20 -07:00
Preetha Appan 121326161e Added unit test case to kvs_endpointtest 2017-08-09 15:50:22 -05:00
Preetha Appan d06002dc62 Ensure that we return a permission denied only if the list of keys/entries prior to filtering by ACL is non empty 2017-08-09 15:32:18 -05:00
Frank Schroeder c38dcf2d17
agent: move agent/consul/agent to agent/metadata 2017-08-09 14:36:52 +02:00
Frank Schroeder 85bdb77d90
agent: move agent/consul/servers to agent/router 2017-08-09 14:36:37 +02:00
Frank Schroeder 1d0bbfed9c
agent: move agent/consul/structs to agent/structs 2017-08-09 14:32:12 +02:00
James Phillips a600f681d6
Cleans up some go fmt issues. 2017-08-08 21:52:50 -07:00
James Phillips 7b4d3d5576
Fixes a vet error. 2017-08-08 16:00:18 -07:00
Kyle Havlovitz 8c2e422074 Merge pull request #3369 from hashicorp/metrics-enhancements 
Add support for labels/filters from go-metrics
 2017-08-08 13:55:30 -07:00
Kyle Havlovitz 160395d3c7
Add doc links for metrics endpoint 2017-08-08 13:05:38 -07:00
Kyle Havlovitz 308d7b785d
Update docs for metrics endpoint 2017-08-08 12:33:30 -07:00
Frank Schroeder 0f4986dcc7
dns: minor cleanups 2017-08-08 13:55:58 +02:00
Kyle Havlovitz 975ded2714
Add support for labels/filters from go-metrics 2017-08-08 01:45:10 -07:00
Preetha Appan 2df084968c Go back to using <nodename>.node.dc.consul as the name of the ns record being returned. 2017-08-07 16:02:33 -05:00
Frank Schroeder b571cb8097
dns: keep NS names in consul domain 2017-08-07 11:11:55 +02:00
Frank Schroeder 7b39af2b2d
dns: postmaster -> hostmaster 2017-08-07 11:11:55 +02:00
Frank Schroeder 98de22e13e
dns: we do not support zone transfers 2017-08-07 11:11:55 +02:00
Frank Schroeder e1bcbc6832
dns: drop CNAME for primary name server 2017-08-07 11:11:55 +02:00
Preetha Appan 393a0eae93
Added test case with IPV6 bind address for NS records, rewrote tests to use verify library and other code review feedback 2017-08-07 11:11:55 +02:00
Preetha Appan 52075bda1c
Added back glue records in NS response, expanded unit test. Also reused same function used in node lookup for adding A/AAAA records in the extra section of the NS response 2017-08-07 11:11:55 +02:00
Preetha Appan c7c4100503
Don't add A records for NS requests, because the record being returned already resolves correctly. Also fixed all the unit tests, and ignored hostnames that don't meet valid dns hostname criteria 2017-08-07 11:11:55 +02:00
Frank Schroeder 450d8a69b5
dns: provide correct SOA and NS responses 
This patch changes the behavior of the DNS server as follows:

* The SOA response contains the SOA record in the Answer section instead
  of the Authority section. It also contains NS records in the Authority
  and the corresponding A glue records in the Extra section.
  In addition, CNAMEs are added to the Extra section to make the
  MNAME of the SOA record resolvable.

  AAAA glue records are not yet supported.

* The NS response returns up to three random servers from the
  consul cluster in the Answer section and the glue A
  records in the Extra section.

  AAAA glue records are not yet supported.
 2017-08-07 11:11:55 +02:00
Preetha Appan bff45ee1da
Unify regex used to identify invalid dns characters 2017-08-07 11:11:55 +02:00
Preetha Appan 6bac9355fd
Use sanitized version of node name of server in NS record, and start with "server" rather than "ns" 2017-08-07 11:11:55 +02:00
Preetha Appan 7e9d683ab1
Removed a copy pasted irrelevant comment, and other code review feedback 2017-08-07 11:11:54 +02:00
Preetha Appan c38906daad
Add NS records and A records for each server. Constructs ns host names using the advertise address of the server. 2017-08-07 11:11:54 +02:00
James Phillips 803ed9a245 Adds secure introduction for the ACL replication token. (#3357) 
Adds secure introduction for the ACL replication token, as well as a separate enable config for ACL replication.
 2017-08-03 15:39:31 -07:00
Frank Schroeder 1cb602e085
agent: fix code for updated go-discover signature 
Closes #3351
 2017-08-03 21:32:11 +02:00
James Phillips c31b56a03e Adds a new /v1/acl/bootstrap API (#3349) 2017-08-02 17:05:18 -07:00
Miguel Prokop ea6d610dee agent: Fix script quoting on windows (#1875) 
This patch fixes the quoting for executing scripts on windows
and splits the platform dependent code.

Fixes #1875
 2017-08-02 17:01:21 +02:00
Frank Schroeder 68e8f3d0f7 agent: use github.com/hashicorp/go-discover 
Replace the provider specific node discovery code
with go-discover to support AWS, Azure and GCE.

Fixes #3282
 2017-08-01 11:41:43 +02:00
Preetha Appan 307049e17f Return nil instead of empty list when returning a PermissionDenied error, updated unit test 2017-07-31 17:23:20 -05:00
Preetha Appan da29b74d03 Return 403 rather than a 404 when acls cause all results to be filtered out. This fixes #2637 2017-07-31 13:50:29 -05:00
preetapan 677949b14d Merge pull request #3332 from hashicorp/issue_3322 
This fixes #3322
 2017-07-28 17:54:30 -05:00
Preetha Appan 3b12545844 Tweaked parsing error message to quote properly 2017-07-28 17:52:35 -05:00
James Phillips 8f1f762ddd Adds missing autopilot snapshot test and avoids snapshotting nil. (#3333) 2017-07-28 15:48:42 -07:00
Preetha Appan 86b9e3c5f3 Validate unix sockets and ip addresses as needed, more test cases 2017-07-28 17:18:10 -05:00
Preetha Appan ac068de336 Modify ResolveTmplAddrs to parse advertise IPs, added test cases that fail to parse correctly 2017-07-28 15:01:32 -05:00
Preetha Appan 4b82d09df0 Removed extra newlines 2017-07-28 10:51:11 -05:00
Preetha Appan 7b99f7ca08 Fix comments, and remove redundant TestConfig init from a couple of unit tests 2017-07-28 10:40:43 -05:00
Frank Schroeder f27202b608
add tests for go-sockaddr template parsing 2017-07-28 15:40:22 +02:00
Frank Schroeder 5ee498cbc5
agent: unix sockets are not ip addrs 2017-07-28 14:53:21 +02:00
Frank Schroeder 0b13a38d90
config: refactor tmpl resolution fn 2017-07-28 12:20:49 +02:00
Preetha Appan 28016190e0 Moved handling advertise address to readConfig and out of the agent's constructor, plus unit test fixes 2017-07-27 22:06:31 -05:00
Preetha Appan 398c1e450c Move go-socketaddr template parsing into config package to make it happen before creating a new agent. Also removed redundant parsetemplate calls from agent.go. 2017-07-27 16:17:35 -05:00
James Phillips 6b51744ddf Adds option to prepared queries to remove empty tags. (#3330) 2017-07-26 22:46:43 -07:00