22c5951ec4
Merge pull request #4303 from pierresouchay/non_blocking_acl
...
Only send one single ACL cache refresh across network when TTL is over
2018-07-10 08:57:33 -04:00
0b50b84429
Agent/Proxy: Formatting and test cases fix
2018-07-09 12:46:10 -04:00
883b2a518a
Store the time CARoot is rotated out instead of when to prune
2018-07-06 16:05:25 -07:00
3c520019e9
connect/ca: add logic for pruning old stale RootCA entries
2018-07-02 10:35:05 -07:00
95a0ab9f99
Updated swith case to use same branch for async-cache and extend-cache
2018-07-02 17:39:34 +02:00
6dfbbf1350
Updated documentation and adding more test case for async-cache
2018-07-01 23:50:30 +02:00
382bec0897
Added async-cache with similar behaviour as extend-cache but asynchronously
2018-07-01 23:50:30 +02:00
da9c91fd3d
Only send one single ACL cache refresh across network when TTL is over
...
It will allow the following:
* when connectivity is limited (saturated linnks between DCs), only one
single request to refresh an ACL will be sent to ACL master DC instead
of statcking ACL refresh queries
* when extend-cache is used for ACL, do not wait for result, but refresh
the ACL asynchronously, so no delay is not impacting slave DC
* When extend-cache is not used, keep the existing blocking mechanism,
but only send a single refresh request.
This will fix https://github.com/hashicorp/consul/issues/3524
2018-07-01 23:50:30 +02:00
02719c52ff
Move starting enterprise functionality
2018-06-29 17:38:29 -04:00
66af873639
Move default uuid test into the consul package
2018-06-27 09:21:58 -04:00
dbc407cec9
go fmt changes
2018-06-27 09:07:22 -04:00
95291ec5ed
Make sure to generate UUIDs when services are registered without one
...
This makes the behavior line up with the docs and expected behavior
2018-06-26 17:04:08 -04:00
1da3c42867
Merge remote-tracking branch 'connect/f-connect'
2018-06-25 19:42:51 +00:00
859eaea5c4
connect/ca: pull the cluster ID from config during a rotation
2018-06-25 12:25:42 -07:00
fcc5dc6110
connect/ca: leave blank root key/cert out of the default config (unnecessary)
2018-06-25 12:25:42 -07:00
f3089a6647
connect/ca: undo the interface changes and use sign-self-issued in Vault
2018-06-25 12:25:42 -07:00
cea94d0bcf
connect/ca: update Consul provider to use new cross-sign CSR method
2018-06-25 12:25:41 -07:00
7b0845ccde
connect/ca: fix vault provider URI SANs and test
2018-06-25 12:25:41 -07:00
a98b85b25c
connect/ca: add the Vault CA provider
2018-06-25 12:25:41 -07:00
b4fbeb0453
Note leadership issues in comments
2018-06-25 12:25:41 -07:00
824a9b4943
Actually return Intermediate certificates bundled with a leaf!
2018-06-25 12:25:40 -07:00
81bd1b43a3
Fix hot loop in cache for RPC returning zero index.
2018-06-25 12:25:37 -07:00
d6b13463ed
Fix misc test failures (some from other PRs)
2018-06-25 12:25:13 -07:00
1283373a64
Only set precedence on write path
2018-06-25 12:25:13 -07:00
22b95283e9
Fix some tests failures caused by the sorting change and some cuased by previous UpdatePrecedence() change
2018-06-25 12:25:13 -07:00
e2938138f6
Sort intention list by precedence
2018-06-25 12:25:13 -07:00
a242e5b130
agent: update accepted CA config fields and defaults
2018-06-25 12:25:09 -07:00
4ebddd6adb
agent/consul: set precedence value on struct itself
2018-06-25 12:24:16 -07:00
52c10d2208
agent/consul: support a Connect option on prepared query request
2018-06-25 12:24:12 -07:00
e8c899b1b8
agent/consul: prepared query supports "Connect" field
2018-06-25 12:24:11 -07:00
ad382d7351
agent: switch ConnectNative to an embedded struct
2018-06-25 12:24:10 -07:00
a3e0ac1ee3
agent/consul/state: support querying by Connect native
2018-06-25 12:24:08 -07:00
8e02bbc897
agent/consul: support catalog registration with Connect native
2018-06-25 12:24:07 -07:00
6ccc4f39db
Merge pull request #4216 from hashicorp/rpc-limiting
...
Make RPC limits reloadable
2018-06-20 09:05:28 -04:00
787ce3b269
agent: address feedback
2018-06-14 09:42:20 -07:00
b5b29cd6af
agent: rename test to check
2018-06-14 09:42:18 -07:00
a48ff54318
agent/consul: forward request if necessary
2018-06-14 09:42:17 -07:00
b02502be73
agent: comments to point to differing logic
2018-06-14 09:42:17 -07:00
526cfc34bd
agent/consul: implement Intention.Test endpoint
2018-06-14 09:42:17 -07:00
bd5eb8b749
Add default CA config back - I didn't add it and causes nil panics
2018-06-14 09:42:17 -07:00
dbcf286d4c
Ooops remove the CA stuff from actual server defaults and make it test server only
2018-06-14 09:42:16 -07:00
834ed1d25f
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes.
2018-06-14 09:42:16 -07:00
30d90b3be4
Generate CSR using real trust-domain
2018-06-14 09:42:16 -07:00
5a1408f186
Add CSR signing verification of service ACL, trust domain and datacenter.
2018-06-14 09:42:16 -07:00
c808833a78
Return TrustDomain from CARoots RPC
2018-06-14 09:42:15 -07:00
d1265bc38b
Rename some of the CA structs/files
2018-06-14 09:42:15 -07:00
1660f9ebab
Add more metadata to structs.CARoot
2018-06-14 09:42:15 -07:00
baf4db1c72
Use provider state table for a global serial index
2018-06-14 09:42:15 -07:00
c90b353eea
Move connect CA provider to separate package
2018-06-14 09:42:15 -07:00
54a1662da8
agent/consul: change provider wait from goto to a loop
2018-06-14 09:42:14 -07:00
749f81373f
agent/consul: check nil on getCAProvider result
2018-06-14 09:42:14 -07:00
c57405b323
agent/consul: retry reading provider a few times
2018-06-14 09:42:14 -07:00
dcd277de8a
Wire up agent leaf endpoint to cache framework to support blocking.
2018-06-14 09:42:07 -07:00
b28e11fdd3
Fill out connect CA rpc endpoint tests
2018-06-14 09:42:06 -07:00
7c0976208d
Add tests for the built in CA's state store table
2018-06-14 09:42:06 -07:00
19b9399f2f
Add more tests for built-in provider
2018-06-14 09:42:06 -07:00
a29f3c6b96
Fix some inconsistencies around the CA provider code
2018-06-14 09:42:06 -07:00
2167713226
Add CA config to connect section of agent config
2018-06-14 09:42:05 -07:00
02fef5f9a2
Move ConsulCAProviderConfig into structs package
2018-06-14 09:42:04 -07:00
887cc98d7e
Simplify the CAProvider.Sign method
2018-06-14 09:42:04 -07:00
44b30476cb
Simplify the CA provider interface by moving some logic out
2018-06-14 09:42:04 -07:00
aa10fb2f48
Clarify some comments and names around CA bootstrapping
2018-06-14 09:42:04 -07:00
43f13d5a0b
Add cross-signing mechanism to root rotation
2018-06-14 09:42:00 -07:00
bbfcb278e1
Add the root rotation mechanism to the CA config endpoint
2018-06-14 09:41:59 -07:00
a585a0ba10
Have the built in CA store its state in raft
2018-06-14 09:41:59 -07:00
80eddb0bfb
Fix the testing endpoint's root set op
2018-06-14 09:41:59 -07:00
fc9ef9741b
Hook the CA RPC endpoint into the provider interface
2018-06-14 09:41:59 -07:00
a40db26ffe
Add CA bootstrapping on establishing leadership
2018-06-14 09:41:59 -07:00
e26819ed9c
Add the bootstrap config for the CA
2018-06-14 09:41:59 -07:00
ebdda17a30
Add CA config set to fsm operations
2018-06-14 09:41:58 -07:00
f7ff16669f
Add the Connect CA config to the state store
2018-06-14 09:41:58 -07:00
9d11cd9bf4
Fix various test failures and vet warnings.
...
Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.
2018-06-14 09:41:58 -07:00
280382c25f
Add tests all the way up through the endpoints to ensure duplicate src/destination is supported and so ultimately deny/allow nesting works.
...
Also adds a sanity check test for `api.Agent().ConnectAuthorize()` and a fix for a trivial bug in it.
2018-06-14 09:41:57 -07:00
adc5589329
Allow duplicate source or destination, but enforce uniqueness across all four.
2018-06-14 09:41:57 -07:00
1985655dff
agent/consul/state: ensure exactly one active CA exists when setting
2018-06-14 09:41:54 -07:00
da1bc48372
agent/connect: rename SpiffeID to CertURI
2018-06-14 09:41:53 -07:00
b0315811b9
agent/connect: use proper keyusage fields for CA and leaf
2018-06-14 09:41:53 -07:00
2026cf3753
agent/consul: encode issued cert serial number as hex encoded
2018-06-14 09:41:53 -07:00
746f80639a
agent: /v1/connect/ca/configuration PUT for setting configuration
2018-06-14 09:41:52 -07:00
2dfca5dbc2
agent/consul/fsm,state: snapshot/restore for CA roots
2018-06-14 09:41:52 -07:00
17d6b437d2
agent/consul/fsm,state: tests for CA root related changes
2018-06-14 09:41:52 -07:00
a8510f8224
agent/consul: set more fields on the issued cert
2018-06-14 09:41:52 -07:00
58b6f476e8
agent: /v1/connect/ca/leaf/:service_id
2018-06-14 09:41:52 -07:00
80a058a573
agent/consul: CAS operations for setting the CA root
2018-06-14 09:41:51 -07:00
712888258b
agent/consul: tests for CA endpoints
2018-06-14 09:41:51 -07:00
1928c07d0c
agent/consul: key the public key of the CSR, verify in test
2018-06-14 09:41:51 -07:00
9a8653f45e
agent/consul: test for ConnectCA.Sign
2018-06-14 09:41:51 -07:00
a360c5cca4
agent/consul: basic sign endpoint not tested yet
2018-06-14 09:41:51 -07:00
24830f4cfa
agent/consul: RPC endpoints to list roots
2018-06-14 09:41:50 -07:00
cfb62677c0
agent/consul/state: CARoot structs and initial state store
2018-06-14 09:41:49 -07:00
7e8d606717
agent: address PR feedback
2018-06-14 09:41:49 -07:00
f9a55aa7e0
agent: clarified a number of comments per PR feedback
2018-06-14 09:41:49 -07:00
62cbb892e3
agent/consul: Health.ServiceNodes ACL check for Connect
2018-06-14 09:41:49 -07:00
641c982480
agent/consul: Catalog endpoint ACL requirements for Connect proxies
2018-06-14 09:41:49 -07:00
566c98b2fc
agent/consul: require name for proxies
2018-06-14 09:41:48 -07:00
daaa6e2403
agent: clean up connect/non-connect duplication by using shared methods
2018-06-14 09:41:48 -07:00
119ffe3ed9
agent/consul: implement Health.ServiceNodes for Connect, DNS works
2018-06-14 09:41:47 -07:00
253256352c
agent/consul: Catalog.ServiceNodes supports Connect filtering
2018-06-14 09:41:47 -07:00
06957f6d7f
agent/consul/state: ConnectServiceNodes
2018-06-14 09:41:47 -07:00
200100d3f4
agent/consul: enforce ACL on ProxyDestination
2018-06-14 09:41:47 -07:00
8a72826483
agent/consul: proxy registration and tests
2018-06-14 09:41:46 -07:00
8777ff139c
agent: test /v1/catalog/node/:node to list connect proxies
2018-06-14 09:41:46 -07:00
761b561946
agent: /v1/catalog/service/:service works with proxies
2018-06-14 09:41:46 -07:00
58bff8dd05
agent/consul/state: convert proxy test to testify/assert
2018-06-14 09:41:46 -07:00
09568ce7b5
agent/consul/state: service registration with proxy works
2018-06-14 09:41:46 -07:00
23ee0888ec
agent/consul: convert intention ACLs to testify/assert
2018-06-14 09:41:46 -07:00
6a8bba7d48
agent/consul,structs: add tests for ACL filter and prefix for intentions
2018-06-14 09:41:45 -07:00
3e10a1ae7a
agent/consul: Intention.Match ACLs
2018-06-14 09:41:45 -07:00
db44a98a2d
agent/consul: Intention.Get ACLs
2018-06-14 09:41:45 -07:00
fd840da97a
agent/consul: Intention.Apply ACL on rename
2018-06-14 09:41:45 -07:00
14ca93e09c
agent/consul: tests for ACLs on Intention.Apply update/delete
2018-06-14 09:41:45 -07:00
c54be9bc09
agent/consul: Basic ACL on Intention.Apply
2018-06-14 09:41:44 -07:00
1d0b4ceedb
agent: convert all intention tests to testify/assert
2018-06-14 09:41:44 -07:00
f07340e94f
agent/consul/fsm,state: snapshot/restore for intentions
2018-06-14 09:41:44 -07:00
6f33b2d070
agent: use UTC time for intention times, move empty list check to
...
agent/consul
2018-06-14 09:41:43 -07:00
67b017c95c
agent/consul/fsm: switch tests to use structs.TestIntention
2018-06-14 09:41:43 -07:00
3a00564411
agent/consul/state: need to set Meta for intentions for tests
2018-06-14 09:41:43 -07:00
027dad8672
agent/consul/state: remove TODO
2018-06-14 09:41:43 -07:00
37f66e47ed
agent: use testing intention to get valid intentions
2018-06-14 09:41:43 -07:00
04bd4af99c
agent/consul: set default intention SourceType, validate it
2018-06-14 09:41:43 -07:00
8e2462e301
agent/structs: Intention validation
2018-06-14 09:41:42 -07:00
d34ee200de
agent/consul: support intention description, meta is non-nil
2018-06-14 09:41:42 -07:00
e81d1c88b7
agent/consul/fsm: add tests for intention requests
2018-06-14 09:41:42 -07:00
2b047fb09b
agent,agent/consul: set default namespaces
2018-06-14 09:41:42 -07:00
e630d65d9d
agent/consul: set CreatedAt, UpdatedAt on intentions
2018-06-14 09:41:42 -07:00
e9d208bcb6
agent/consul: RPC endpoint for Intention.Match
2018-06-14 09:41:42 -07:00
987b7ce0a2
agent/consul/state: IntentionMatch for performing match resolution
2018-06-14 09:41:41 -07:00
bebe6870ff
agent/consul: test that Apply works to delete an intention
2018-06-14 09:41:41 -07:00
95e1c92edf
agent/consul/state,fsm: support for deleting intentions
2018-06-14 09:41:41 -07:00
32ad54369c
agent/consul: creating intention must not have ID set
2018-06-14 09:41:40 -07:00
f219c766cb
agent/consul: support updating intentions
2018-06-14 09:41:40 -07:00
37572829ab
agent: GET /v1/connect/intentions/:id
2018-06-14 09:41:40 -07:00
2a8a2f8167
agent/consul: Intention.Get endpoint
2018-06-14 09:41:40 -07:00
48b9a43f1d
agent/consul: Intention.Apply, FSM methods, very little validation
2018-06-14 09:41:39 -07:00
b19a289596
agent/consul: start Intention RPC endpoints, starting with List
2018-06-14 09:41:39 -07:00
8b0ac7d9c5
agent/consul/state: list intentions
2018-06-14 09:41:39 -07:00
c05bed86e1
agent/consul/state: initial work on intentions memdb table
2018-06-14 09:41:39 -07:00
3ed73961b3
Attach server.Name label to client.rpc.failed
2018-06-13 14:56:14 +01:00
bda575074e
Attach server.ID label to client.rpc.failed
2018-06-13 14:53:44 +01:00
edd6a69541
Client: add metric for failed RPC calls to server
2018-06-13 12:35:45 +01:00
c41fa6c010
Add a Client ReloadConfig test
2018-06-11 16:23:51 -04:00
c5d9c2362f
Merge branch 'master' of github.com:hashicorp/consul into rpc-limiting
...
# Conflicts:
# agent/agent.go
# agent/consul/client.go
2018-06-11 16:11:36 -04:00
c589991452
Apply the limits to the clients rpcLimiter
2018-06-11 15:51:17 -04:00
14661a417b
Allow for easy enterprise/oss coexistence
...
Uses struct/interface embedding with the embedded structs/interfaces being empty for oss. Also methods on the server/client types are defaulted to do nothing for OSS
2018-05-24 10:36:42 -04:00
88514d6a82
Add support for reverse lookup of services
2018-05-19 19:39:02 +02:00
7400a78f8a
Change default raft threshold config values and add a section to upgrade notes
2018-05-11 10:45:41 -05:00
e28c5fbb4e
Also make snapshot interval configurable
2018-05-11 10:43:24 -05:00
eb4bc79118
Make raft snapshot commit threshold configurable
2018-05-11 10:43:24 -05:00
e611b1728a
Merge pull request #4097 from hashicorp/remove-deprecated
...
Remove deprecated check/service fields and metric names
2018-05-10 15:45:49 -07:00
60307ef328
Remove deprecated metric names
2018-05-08 16:23:15 -07:00
c55885efd8
Merge pull request #3970 from pierresouchay/node_health_should_change_service_index
...
[BUGFIX] When a node level check is removed, ensure all services of node are notified
2018-05-08 16:44:50 +01:00
ee47eb7d7d
Added Missing Service Meta synchronization and field
2018-04-21 17:34:29 +02:00
1b55e3559b
Allow renaming nodes when ID is unchanged
2018-04-18 15:39:38 +02:00
be10300d06
Update make static-assets goal and run format
2018-04-13 09:57:25 -07:00
ed94d356e0
Merge pull request #4023 from hashicorp/f-near-ip
...
Add near=_ip support for prepared queries
2018-04-12 12:10:48 -04:00
aa9151738a
GH-3798: A couple more PR updates
...
Test HTTP/DNS source IP without header/extra EDNS data.
Add WARN log for when prepared query with near=_ip is executed without specifying the source ip
2018-04-12 10:10:37 -04:00
3a0f7789ec
GH-3798: A few more PR updates
2018-04-11 20:32:35 -04:00
de3a9be3d0
GH-3798: Updates for PR
...
Allow DNS peer IP as the source IP.
Break early when the right node was found for executing the preapred query.
Update docs
2018-04-11 17:02:04 -04:00
89cd24aeca
GH-3798: Add near=_ip support for prepared queries
2018-04-10 14:50:50 -04:00
2ed0d2afcd
Allow ignoring checks by ID when defining a PreparedQuery. Fixes #3727 .
2018-04-10 14:04:16 +01:00
d9d9944179
Renames agent API layer for service metadata to "meta" for consistency
2018-03-28 09:04:50 -05:00
8dacb12c79
Merge pull request #3881 from pierresouchay/service_metadata
...
Feature Request: Support key-value attributes for services
2018-03-27 16:33:57 -05:00
b9ae4e647f
Added validation of ServiceMeta in Catalog
...
Fixed Error Message when ServiceMeta is not valid
Added Unit test for adding a Service with badly formatted ServiceMeta
2018-03-27 22:22:42 +02:00
17a011b9bd
fix typo and remove comment
2018-03-27 14:28:05 -05:00
6d16afc65c
Remove unnecessary nil checks
2018-03-27 10:59:42 -05:00
c21c2da690
Fix test and remove unused method
2018-03-27 09:44:41 -05:00
512f9a50fc
Allows disabling WAN federation by setting serf WAN port to -1
2018-03-26 14:21:06 -05:00
eccb56ade0
Added support for renaming nodes when their IP does not change
2018-03-26 16:44:13 +02:00
90d2f7bca1
Merge remote-tracking branch 'origin/master' into node_health_should_change_service_index
2018-03-22 13:07:11 +01:00
9cc9dce848
More test cases
2018-03-22 12:41:06 +01:00
7e8e4e014b
Added new test regarding checks index
2018-03-22 12:20:25 +01:00
a8b66fb7aa
Fixed minor typo in comments
...
Might fix unstable travis build
2018-03-22 10:30:10 +01:00
1dd8c378b9
Spelling ( #3958 )
...
* spelling: another
* spelling: autopilot
* spelling: beginning
* spelling: circonus
* spelling: default
* spelling: definition
* spelling: distance
* spelling: encountered
* spelling: enterprise
* spelling: expands
* spelling: exits
* spelling: formatting
* spelling: health
* spelling: hierarchy
* spelling: imposed
* spelling: independence
* spelling: inspect
* spelling: last
* spelling: latest
* spelling: client
* spelling: message
* spelling: minimum
* spelling: notify
* spelling: nonexistent
* spelling: operator
* spelling: payload
* spelling: preceded
* spelling: prepared
* spelling: programmatically
* spelling: required
* spelling: reconcile
* spelling: responses
* spelling: request
* spelling: response
* spelling: results
* spelling: retrieve
* spelling: service
* spelling: significantly
* spelling: specifies
* spelling: supported
* spelling: synchronization
* spelling: synchronous
* spelling: themselves
* spelling: unexpected
* spelling: validations
* spelling: value
2018-03-19 16:56:00 +00:00
3eb287f57d
Fixed typo in comments
2018-03-19 17:12:08 +01:00
eb2a4eaea3
Refactoring to have clearer code without weird bool
2018-03-19 16:12:54 +01:00
a5f6ac0df4
[BUGFIX] When a node level check is removed, ensure all services of node are notified
...
Bugfix for https://github.com/hashicorp/consul/pull/3899
When a node level check is removed (example: maintenance),
some watchers on services might have to recompute their state.
If those nodes are performing blocking queries, they have to be notified.
While their state was updated when node-level state did change or was added
this was not the case when the check was removed. This fixes it.
2018-03-19 14:14:03 +01:00
881d20c606
🐛 Formatting changes only; add missing trailing commas
2018-03-15 10:19:46 -07:00
fbac58280e
agent/consul/fsm: begin using testify/assert
2018-03-06 09:48:15 -08:00
628dcc9793
Merge pull request #3899 from pierresouchay/fix_blocking_queries_index
...
Services Indexes modified per service instead of using a global Index
2018-03-02 16:24:43 +00:00
85b73f8163
Simplified error handling for maxIndexForService
...
* added unit tests to ensure service index is properly garbage collected
* added Upgrade from Version 1.0.6 to higher section in documentation
2018-03-01 14:09:36 +01:00
77d35f1829
Remove extra newline
2018-02-21 13:21:47 -06:00
573500dc51
Unit test that calls revokeLeadership twice to make sure its idempotent
2018-02-21 12:48:53 -06:00
bd270b02ba
Make sure revokeLeadership is called if establishLeadership errors
2018-02-21 12:33:22 -06:00
535842004c
Test autopilots start/stop idempotency
2018-02-21 10:19:30 -08:00
4d99696f02
Improve autopilot shutdown to be idempotent
2018-02-20 15:51:59 -08:00
e6d85cb36a
Fixed comments for function maxIndexForService
2018-02-20 23:57:28 +01:00
b26ea3c230
[Revert] Only update services if tags are different
...
This patch did give some better results, but break watches on
the services of a node.
It is possible to apply the same optimization for nodes than
to services (one index per instance), but it would complicate
further the patch.
Let's do it in another PR.
2018-02-20 23:34:42 +01:00
903e866835
Only update services if tags are different
2018-02-20 23:08:04 +01:00
56d5c0bf22
Enable Raft index optimization per service name on health endpoint
...
Had to fix unit test in order to check properly indexes.
2018-02-20 01:35:50 +01:00
ec1b278595
Get only first service to test whether we have to cleanup index of a service
2018-02-19 22:44:49 +01:00
523feb0be4
Fixed comment about raftIndex + use test.Helper()
2018-02-19 19:30:25 +01:00
4c188c1d08
Services Indexes modified per service instead of using a global Index
...
This patch improves the watches for services on large cluster:
each service has now its own index, such watches on a specific service
are not modified by changes in the global catalog.
It should improve a lot the performance of tools such as consul-template
or libraries performing watches on very large clusters with many
services/watches.
2018-02-19 18:29:22 +01:00
05666113a4
remove golint warnings
2018-01-28 22:40:13 +04:00
0e76d62846
Reset clusterHealth when autopilot starts
2018-01-23 12:52:28 -08:00
6d1dbe6cc4
Move autopilot health loop into leader operations
2018-01-23 11:17:41 -08:00
62e97a6602
Fixes a `go fmt` cleanup.
2017-12-20 13:43:38 -08:00
74b0c58831
Fix vet error
2017-12-18 18:04:42 -08:00
dfc165a47b
Move autopilot initializing to oss file
2017-12-18 18:02:44 -08:00
044c38aa7b
Move autopilot setup to a separate file
2017-12-18 16:55:51 -08:00
9e1ba6fb4e
Make some final tweaks to autopilot package
2017-12-18 12:26:47 -08:00
6b58df5898
Merge pull request #3737 from hashicorp/autopilot-refactor
...
Move autopilot to a standalone package
2017-12-15 14:09:40 -08:00
262cbbd9ca
Merge pull request #3728 from weiwei04/fix_globalRPC_goroutine_leak
...
fix globalRPC goroutine leak
2017-12-14 17:54:19 -08:00
798aca92c5
Expose IsPotentialVoter for advanced autopilot logic
2017-12-13 17:53:51 -08:00
a4ac148077
Merge branch 'master' into autopilot-refactor
2017-12-13 11:54:32 -08:00
6c985132de
A few last autopilot adjustments
2017-12-13 11:19:17 -08:00
77d92bf15c
More autopilot reorganizing
2017-12-13 10:57:37 -08:00
984de6e2e0
Adds TODOs referencing #3744 .
2017-12-13 10:52:06 -08:00
f347c8a531
More refactoring to make autopilot consul-agnostic
2017-12-12 17:46:28 -08:00
8546a1d3c6
Move autopilot to a standalone package
2017-12-11 16:45:33 -08:00
32b64575d1
Moves Serf helper into lib to fix import cycle in consul-enterprise.
2017-12-07 16:57:58 -08:00
c16cce80bb
Turns of intent queue warnings and enables dynamic queue sizing.
2017-12-07 16:27:06 -08:00
04531ff0fb
fix globalRPC goroutine leak
...
Signed-off-by: Wei Wei <weiwei.inf@gmail.com>
2017-12-05 11:53:30 +08:00
c4bc89a187
Creates a registration mechanism for snapshot and restore.
2017-11-29 18:36:53 -08:00
8571555703
Begins split out of snapshots from the main FSM class.
2017-11-29 18:36:53 -08:00
4eaee8e0ba
Creates a registration mechanism for FSM commands.
2017-11-29 18:36:53 -08:00
3e7ea1931c
Moves the FSM into its own package.
...
This will help make it clearer what happens when we add some registration
plumbing for the different operations and snapshots.
2017-11-29 18:36:53 -08:00
7f3783f4be
Resolves an FSM snapshot TODO.
...
This adds checks for sink write calls before we continue the refactor, which
will resolve the other TODO comment we deleted as part of this change.
2017-11-29 18:36:53 -08:00
5a24d37ac0
Creates a registration mechanism for schemas.
...
This also splits out the registration into the table-specific source
files.
2017-11-29 18:36:52 -08:00
36bb30e67a
Creates a registration mechanism for RPC endpoints.
2017-11-29 18:36:52 -08:00
ba56669ea8
Renames stubs to be more consistent.
2017-11-29 18:36:52 -08:00
56552095c9
Sheds monotonic time info so tombstone GC bins work properly.
2017-11-29 10:34:24 -08:00
8656b7a3e9
Gives back the lock before writing to the expire channel.
...
The lock isn't needed after we clean up the expire bin, and as seen
in #3700 we can get into a deadlock waiting to place the expire index
into the channel while holding this lock.
Fixes #3700
2017-11-19 16:24:16 -08:00
8210523b1b
Moves the LAN event handler after the router is created.
...
Fixes #3680
2017-11-10 12:26:48 -08:00
bfbbfb62ca
Revert "Adds a small sleep to make sure we are in the next GC bucket."
2017-11-08 22:18:37 -08:00
d6328a5bf8
Adds a sleep to make sure we are in the next GC bucket, ups time.
...
Fixes #3670
2017-11-08 22:02:40 -08:00
91824375be
Skips the tombstone GC test in Travis for now.
...
Related to #3670
2017-11-08 20:14:20 -08:00
b94ba8aeb4
Removes bogus getPort() in favor of freeport.
2017-11-08 19:55:50 -08:00
444a345a3a
Tightens timing up and reorders GC test to be less flaky.
2017-11-08 15:09:29 -08:00
e00624425b
Doubles the GC timing.
2017-11-08 15:01:11 -08:00
8eb91777d9
Opens up test timing a little more.
2017-11-08 14:01:19 -08:00
d45c2a01f1
Shifts off a gran boundary to help make test less flaky.
2017-11-08 13:57:17 -08:00
757e353334
Opens up the tombstone GC test timing.
2017-11-08 13:43:39 -08:00
068ca11eb8
Move check definition to a sub-struct
2017-11-01 14:54:46 -07:00
bc3ba5f873
Merge branch 'master' into esm-changes
2017-11-01 11:37:48 -07:00
83524f44c4
Merge pull request #3622 from hashicorp/coordinate-node-endpoint
...
agent: add /v1/coordianate/node/:node endpoint
2017-11-01 11:35:50 -07:00
9909b661ac
Fill out the tests around coordinate/node functionality
2017-10-31 15:36:44 -07:00
fd4d9f1c16
Factor out registerNodes function
2017-10-31 13:34:49 -07:00
c6e0366c02
Relaxes Autopilot promotion logic. ( #3623 )
...
* Relaxes Autopilot promotion logic.
When we defaulted the Raft protocol version to 3 in #3477 we made
the numPeers() routine more strict to only count voters (this is
more conservative and more correct). This had the side effect of
breaking rolling updates because it's at odds with the Autopilot
non-voter promotion logic.
That logic used to wait to only promote to maintain an odd quorum
of servers. During a rolling update (add one new server, wait, and
then kill an old server) the dead server cleanup would still count
the old server as a peer, which is conservative and the right thing
to do, and no longer count the non-voter. This would wait to promote,
so you could get into a stalemate. It is safer to promote early than
remove early, so by promoting as soon as possible we have chosen
that as the solution here.
Fixes #3611
* Gets rid of unnecessary extra not-a-voter check.
2017-10-31 15:16:56 -05:00
496dd7ab5b
Merge branch 'coordinate-node-endpoint' of github.com:hashicorp/consul into esm-changes
2017-10-26 19:20:24 -07:00
f80e70271d
Added Coordinate.Node rpc endpoint and client api method
2017-10-26 19:16:40 -07:00
84a07ea113
Expose SkipNodeUpdate field and some health check info in the http api
2017-10-25 19:37:30 +02:00
74859ff3c0
test: replace porter tool with freeport lib
...
This patch removes the porter tool which hands out free ports from a
given range with a library which does the same thing. The challenge for
acquiring free ports in concurrent go test runs is that go packages are
tested concurrently and run in separate processes. There has to be some
inter-process synchronization in preventing processes allocating the
same ports.
freeport allocates blocks of ports from a range expected to be not in
heavy use and implements a system-wide mutex by binding to the first
port of that block for the lifetime of the application. Ports are then
provided sequentially from that block and are tested on localhost before
being returned as available.
2017-10-21 22:01:09 +02:00
6f05ea91a3
Replace time.Now().Sub(x) with time.Since(x)
2017-10-17 20:38:24 +02:00
e9670761f9
Cleans up some drift between the OSS and Enterprise trees.
2017-10-11 15:53:07 -07:00
d1ad538345
Makes RPC handling more robust when rolling servers. ( #3561 )
...
* Adds client-side retry for no leader errors.
This paves over the case where the client was connected to the leader
when it loses leadership.
* Adds a configurable server RPC drain time and a fail-fast path for RPCs.
When a server leaves it gets removed from the Raft configuration, so it will
never know who the new leader server ends up being. Without this we'd be
doomed to wait out the RPC hold timeout and then fail. This makes things fail
a little quicker while a sever is draining, and since we added a client retry
AND since the server doing this has already shut down and left the Serf LAN,
clients should retry against some other server.
* Makes the RPC hold timeout configurable.
* Reorders struct members.
* Sets the RPC hold timeout default for test servers.
* Bumps the leave drain time up to 5 seconds.
* Robustifies retries with a simpler client-side RPC hold.
* Reverts untended delete.
2017-10-10 15:19:50 -07:00
a1db119d02
Fixes handling of stop channel and failed barrier attempts. ( #3546 )
...
* Fixes handling of stop channel and failed barrier attempts.
There were two issues here. First, we needed to not exit when there
was a timeout trying to write the barrier, because Raft might not
step down, so we'd be left as the leader but having run all the step
down actions.
Second, we didn't close over the stopCh correctly, so it was possible
to nil that out and have the leaderLoop never exit. We close over it
properly AND sequence the nil-ing of it AFTER the leaderLoop exits for
good measure, so the code is more robust.
Fixes #3545
* Cleans up based on code review feedback.
* Tweaks comments.
* Renames variables and removes comments.
2017-10-06 07:54:49 -07:00
0063516e5e
Update metric names and add a legacy config flag
2017-10-04 16:43:27 -07:00
f38d20eb40
Remove extra newline
2017-10-03 15:19:31 -05:00
3c81e2db7c
Only allow 'list' policies within 'key' policy definitions. Consolidated two similar tests into one and fixed alignment.
2017-10-03 15:15:56 -05:00
d5acfc3982
Introduces new 'list' permission that applies to KV store recursive reads, and enforced only when opted in.
2017-10-02 17:10:21 -05:00
330ce87851
Gets rid of flaky clause in stats fetcher unit test.
...
Given how the rutine is coded we can still get data so this wasn't
a reliable thing to check.
2017-09-26 20:53:06 -07:00
783e24be64
Issue 3452 ( #3500 )
...
* Make sure that id and address are set in member created during reaping of catalog nodes that have been removed from serf
* Get address from node table in the state store rather than from service address
* Fix incorrect lookup by checkname instead of node name
* Make sure that serverlookup is called with the right address format, added unit test.
* Address code review comments
* Tweaks style stuff.
2017-09-26 20:49:41 -07:00
4b17c9618f
Cleans up some edge cases in TestSnapshot_Forward_Leader.
...
These could cause the tests to hang.
2017-09-26 14:07:28 -07:00
318d0232f7
Move Raft protocol version for list peers end point to server side, fix unit tests. This fixes #3449
2017-09-26 09:35:39 -05:00
fcaa889116
Bumps default Raft protocol to version 3. ( #3477 )
...
* Changes default Raft protocol to 3.
* Changes numPeers() to report only voters.
This should have been there before, but it's more obvious that this
is incorrect now that we default the Raft protocol to 3, which puts
new servers in a read-only state while Autopilot waits for them to
become healthy.
* Fixes TestLeader_RollRaftServer.
* Fixes TestOperator_RaftRemovePeerByAddress.
* Fixes TestServer_*.
Relaxed the check for a given number of voter peers and instead do
a thorough check that all servers see each other in their Raft
configurations.
* Fixes TestACL_*.
These now just check for Raft replication to be set up, and don't
care about the number of voter peers.
* Fixes TestOperator_Raft_ListPeers.
* Fixes TestAutopilot_CleanupDeadServerPeriodic.
* Fixes TestCatalog_ListNodes_ConsistentRead_Fail.
* Fixes TestLeader_ChangeServerID and adjusts the conn pool to throw away
sockets when it sees io.EOF.
* Changes version to 1.0.0 in the options doc.
* Makes metrics test more deterministic with autopilot metrics possible.
2017-09-25 15:27:04 -07:00
8394ad08db
Introduce Code Policy validation via sentinel, with a noop implementation
2017-09-25 13:44:55 -05:00
69a088ca85
New config parser, HCL support, multiple bind addrs ( #3480 )
...
* new config parser for agent
This patch implements a new config parser for the consul agent which
makes the following changes to the previous implementation:
* add HCL support
* all configuration fragments in tests and for default config are
expressed as HCL fragments
* HCL fragments can be provided on the command line so that they
can eventually replace the command line flags.
* HCL/JSON fragments are parsed into a temporary Config structure
which can be merged using reflection (all values are pointers).
The existing merge logic of overwrite for values and append
for slices has been preserved.
* A single builder process generates a typed runtime configuration
for the agent.
The new implementation is more strict and fails in the builder process
if no valid runtime configuration can be generated. Therefore,
additional validations in other parts of the code should be removed.
The builder also pre-computes all required network addresses so that no
address/port magic should be required where the configuration is used
and should therefore be removed.
* Upgrade github.com/hashicorp/hcl to support int64
* improve error messages
* fix directory permission test
* Fix rtt test
* Fix ForceLeave test
* Skip performance test for now until we know what to do
* Update github.com/hashicorp/memberlist to update log prefix
* Make memberlist use the default logger
* improve config error handling
* do not fail on non-existing data-dir
* experiment with non-uniform timeouts to get a handle on stalled leader elections
* Run tests for packages separately to eliminate the spurious port conflicts
* refactor private address detection and unify approach for ipv4 and ipv6.
Fixes #2825
* do not allow unix sockets for DNS
* improve bind and advertise addr error handling
* go through builder using test coverage
* minimal update to the docs
* more coverage tests fixed
* more tests
* fix makefile
* cleanup
* fix port conflicts with external port server 'porter'
* stop test server on error
* do not run api test that change global ENV concurrently with the other tests
* Run remaining api tests concurrently
* no need for retry with the port number service
* monkey patch race condition in go-sockaddr until we understand why that fails
* monkey patch hcl decoder race condidtion until we understand why that fails
* monkey patch spurious errors in strings.EqualFold from here
* add test for hcl decoder race condition. Run with go test -parallel 128
* Increase timeout again
* cleanup
* don't log port allocations by default
* use base command arg parsing to format help output properly
* handle -dc deprecation case in Build
* switch autopilot.max_trailing_logs to int
* remove duplicate test case
* remove unused methods
* remove comments about flag/config value inconsistencies
* switch got and want around since the error message was misleading.
* Removes a stray debug log.
* Removes a stray newline in imports.
* Fixes TestACL_Version8.
* Runs go fmt.
* Adds a default case for unknown address types.
* Reoders and reformats some imports.
* Adds some comments and fixes typos.
* Reorders imports.
* add unix socket support for dns later
* drop all deprecated flags and arguments
* fix wrong field name
* remove stray node-id file
* drop unnecessary patch section in test
* drop duplicate test
* add test for LeaveOnTerm and SkipLeaveOnInt in client mode
* drop "bla" and add clarifying comment for the test
* split up tests to support enterprise/non-enterprise tests
* drop raft multiplier and derive values during build phase
* sanitize runtime config reflectively and add test
* detect invalid config fields
* fix tests with invalid config fields
* use different values for wan sanitiziation test
* drop recursor in favor of recursors
* allow dns_config.udp_answer_limit to be zero
* make sure tests run on machines with multiple ips
* Fix failing tests in a few more places by providing a bind address in the test
* Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder.
* Add porter to server_test.go to make tests there less flaky
* go fmt
2017-09-25 11:40:42 -07:00
268018c558
Robustifies check in TestCatalog_ListNodes_ConsistentRead_Fail test.
...
Fixes #3469
2017-09-13 21:22:53 -07:00
8be4ee766a
Revert "Manages segments list via a pointer."
...
This reverts commit c277a4250461443cbd63de0259e5e32766f651ea.
2017-09-07 16:37:11 -07:00
5008aabb62
Manages segments list via a pointer.
2017-09-07 16:21:07 -07:00
908f7be97f
Cleans up formatting.
2017-09-07 12:26:58 -07:00
02a3f3f27b
Shows the segment name in the keyring API and command output.
2017-09-07 12:17:39 -07:00
7c616e3768
Moves reconcile loop into segment stub.
2017-09-06 18:01:53 -07:00
4e34c2af06
Takes the skip out of the client check.
...
Without this the merge delegate won't check the segment for non-servers
a little below here.
2017-09-06 17:05:40 -07:00
78ac144fff
Merge pull request #3447 from hashicorp/issue-3070
...
Skips unique node ID check for old versions of Consul.
2017-09-06 13:24:15 -07:00
62d9299646
Fixes incorrect comment.
2017-09-06 13:23:19 -07:00
031f1874d0
Pulls down some code for the check loop.
2017-09-06 13:07:42 -07:00
2fd9328b21
Uses the Raft configuration for the self-add skip check.
2017-09-06 13:05:51 -07:00
1eae9f1e2f
Change member join reconcile step to process joining itself, to handle node IP address changes correctly when number of servers < 3
2017-09-06 13:53:01 -05:00
353e037c9b
Skips unique node ID check for old versions of Consul.
...
Fixes #3070 .
2017-09-05 22:57:29 -07:00
c629773b40
Makes the all segments query explict, and the default for `consul members`.
2017-09-05 12:22:20 -07:00
bc9780baad
Adds simple rate limiting for client agent RPC calls to Consul servers. ( #3440 )
...
* Added rate limiting for agent RPC calls.
* Initializes the rate limiter based on the config.
* Adds the rate limiter into the snapshot RPC path.
* Adds unit tests for the RPC rate limiter.
* Groups the RPC limit parameters under "limits" in the config.
* Adds some documentation about the RPC limiter.
* Sends a 429 response when the rate limiter kicks in.
* Adds docs for new telemetry.
* Makes snapshot telemetry look like RPC telemetry and cleans up comments.
2017-09-01 15:02:50 -07:00
334e082848
Merge pull request #3431 from hashicorp/network-segments-oss
2017-09-01 10:24:58 -07:00
ff994e9ade
Pass listeners into setupSegments
2017-08-31 17:56:43 -07:00
5cc4b32a5d
Organize segments for a cleaner split between enterprise and OSS
2017-08-31 17:39:46 -07:00
b77a0aa932
Fix some inconsistencies with segment logic and comments
2017-08-30 17:43:46 -07:00
0728a04dbb
Wire server provider for raft layer only on protocol version 3 and above, and update changelog
2017-08-30 14:36:47 -05:00
6ded43131a
Add segment addr field to tags for LAN flood joiner
2017-08-30 11:58:29 -07:00
1c04f1537a
Add agent.segment interpolation to prepared queries
2017-08-30 11:58:29 -07:00
107d7f6c5a
Add rpc_listener option to segment config
2017-08-30 11:58:29 -07:00
6a6eadd8c7
Adds open source side of network segments (feature is Enterprise-only).
2017-08-30 11:58:29 -07:00
e944370cde
More cleanup from code review
2017-08-30 12:31:36 -05:00
a215c764cd
Remove copy pasted duplicate line, update documentation.
2017-08-30 10:02:10 -05:00
5a29eb7486
Consolidate server lookup into one place and replace usages of localConsuls.
2017-08-30 09:30:33 -05:00
d8fe01db4c
Remove stray commented line
2017-08-30 09:30:33 -05:00
ca48e7e4c2
Remove server address tracking logic from manager/router and maintain it as part of lan event listener instead. Used sync.Map to track this, and added unit tests
2017-08-30 09:30:33 -05:00
b4a9d77d49
ServerAddressProvider interface also returns an error now
2017-08-30 09:30:33 -05:00
edb408bc22
Use config struct to create NetworkTransport layer when setting up raft
2017-08-30 09:30:33 -05:00
01f8e469aa
Implement AddressProvider and wire that up to raft transport layer to support server nodes changing their IP addresses in containerized environments
2017-08-30 09:30:33 -05:00
62c77d70f0
build: make tests independent of build tags
...
When the metadata server is scanning the agents for potential servers
it is parsing the version number which the agent provided when it
joined. This version number has to conform to a certain format, i.e.
'n.n.n'. Without this version number properly set some tests fail with
error messages that disguise the root cause.
The default version number is currently set to 'unknown' in
version/version.go which does not parse and triggers the tests to fail.
The work around is to use a build tag 'consul' which will use the
version number set in version_base.go instead which has the correct
format and is set to the current release version.
In addition, some parts of the code also require the version number to
be of a certain value. Setting it to '0.0.0' for example makes some
tests pass and others fail since they don't pass the semantic check.
When using go build/install/test one has to remember to use '-tags
consul' or tests will fail with non-obvious error messages.
Using build tags makes the build process more complex and error prone
since it prevents the use of the plain go toolchain and - at least in
its current form - introduces subtle build and test issues. We should
try to eliminate build tags for anything else but platform specific
code.
This patch removes all references to specific version numbers in the
code and tests and sets the default version to '9.9.9' which is
syntactically correct and passes the semantic check. This solves the
issue of running go build/install/test without tags for the OSS build.
2017-08-30 13:40:18 +02:00
44e6b8122d
acl: consolidate error handling ( #3401 )
...
The error handling of the ACL code relies on the presence of certain
magic error messages. Since the error values are sent via RPC between
older and newer consul agents we cannot just replace the magic values
with typed errors and switch to type checks since this would break
compatibility with older clients.
Therefore, this patch moves all magic ACL error messages into the acl
package and provides default error values and helper functions which
determine the type of error.
2017-08-23 16:52:48 +02:00
d9e2a51887
agent: drop unused code
...
This code from http://github.com/hashicorp/consul/pull/3353 is no longer
required.
2017-08-22 00:02:46 +02:00
3518e27a76
Revert "Return 403 rather than a 404 when acls cause all results to be filter…"
2017-08-09 15:06:57 -07:00
91205b2cd6
Revert "Ensure that we return a permission denied only if the list of keys/en…"
2017-08-09 15:06:20 -07:00
121326161e
Added unit test case to kvs_endpointtest
2017-08-09 15:50:22 -05:00
d06002dc62
Ensure that we return a permission denied only if the list of keys/entries prior to filtering by ACL is non empty
2017-08-09 15:32:18 -05:00
c38dcf2d17
agent: move agent/consul/agent to agent/metadata
2017-08-09 14:36:52 +02:00
85bdb77d90
agent: move agent/consul/servers to agent/router
2017-08-09 14:36:37 +02:00
1d0bbfed9c
agent: move agent/consul/structs to agent/structs
2017-08-09 14:32:12 +02:00
8c2e422074
Merge pull request #3369 from hashicorp/metrics-enhancements
...
Add support for labels/filters from go-metrics
2017-08-08 13:55:30 -07:00
975ded2714
Add support for labels/filters from go-metrics
2017-08-08 01:45:10 -07:00
6bac9355fd
Use sanitized version of node name of server in NS record, and start with "server" rather than "ns"
2017-08-07 11:11:55 +02:00
7e9d683ab1
Removed a copy pasted irrelevant comment, and other code review feedback
2017-08-07 11:11:54 +02:00
c38906daad
Add NS records and A records for each server. Constructs ns host names using the advertise address of the server.
2017-08-07 11:11:54 +02:00
803ed9a245
Adds secure introduction for the ACL replication token. ( #3357 )
...
Adds secure introduction for the ACL replication token, as well as a separate enable config for ACL replication.
2017-08-03 15:39:31 -07:00
c31b56a03e
Adds a new /v1/acl/bootstrap API ( #3349 )
2017-08-02 17:05:18 -07:00
307049e17f
Return nil instead of empty list when returning a PermissionDenied error, updated unit test
2017-07-31 17:23:20 -05:00
da29b74d03
Return 403 rather than a 404 when acls cause all results to be filtered out. This fixes #2637
2017-07-31 13:50:29 -05:00
8f1f762ddd
Adds missing autopilot snapshot test and avoids snapshotting nil. ( #3333 )
2017-07-28 15:48:42 -07:00
6b51744ddf
Adds option to prepared queries to remove empty tags. ( #3330 )
2017-07-26 22:46:43 -07:00
6e794ea1b3
Adds support for agent-side ACL token management via API instead of config files. ( #3324 )
...
* Adds token store and removes all runtime use of config for ACL tokens.
* Adds a new API for changing agent tokens on the fly.
2017-07-26 11:03:43 -07:00
4692b1478e
Add extra test case for deleting entire tree with empty prefix
2017-07-26 09:42:07 -05:00
74ba4c3c6b
Don't insert tombstone for empty prefix delete. Other minor unit test fixes
2017-07-25 21:54:11 -05:00
a6b7e66e9a
Removed redundant comments and unit test
2017-07-25 20:39:33 -05:00
1503d63595
Removed redundant call to reap tombstone from unit test
2017-07-25 19:39:05 -05:00
996302c085
Improved unit test per code review
2017-07-25 19:17:40 -05:00
f4cccf44e3
Use new DeletePrefixMethod for implementing KVSDeleteTree operation. This makes deletes on sub trees larger than one million nodes about 100 times faster. Added unit tests.
2017-07-25 17:21:18 -05:00
e6e711a401
fix spelling in filenames
...
Fixes #3301
2017-07-19 13:16:38 +02:00
1ffd2ec05b
Add UpgradeVersionTag to autopilot config
2017-07-18 13:35:41 -07:00
788dd255a1
Adds new config to make script checks opt-in, updates documentation. ( #3284 )
2017-07-17 11:20:35 -07:00
d985dbc36b
Add TLS setting to router areas
2017-07-14 17:38:08 -07:00
8572931afe
Cleans up version 8 ACLs in the agent and the docs. ( #3248 )
...
* Moves magic check and service constants into shared structs package.
* Removes the "consul" service from local state.
Since this service is added by the leader, it doesn't really make sense to
also keep it in local state (which requires special ACLs to configure), and
requires a bunch of special cases in the local state logic. This requires
fewer special cases and makes ACL bootstrapping cleaner.
* Makes coordinate update ACL log message a warning, similar to other AE warnings.
* Adds much more detailed examples for bootstrapping ACLs.
This can hopefully replace https://gist.github.com/slackpad/d89ce0e1cc0802c3c4f2d84932fa3234 .
2017-07-13 22:33:47 -07:00
75cd1828b8
address review comments
2017-07-07 09:22:34 +02:00
33588d8fdd
agent: remove unused code
2017-07-07 09:22:34 +02:00
37202cc751
agent: make TestClient_RPC_ConsulServerPing more robust
2017-07-07 09:22:34 +02:00
7b54e325df
Adds a comment about flood joining.
2017-07-07 09:22:34 +02:00
247f4a7e41
Simplifies Serf dynamic port selection code.
...
This isn't racy, it's just a little dirty. The listen will happen and a port
will be selected and injected into the config once the Serf instance is
created, so we don't need the retry loop here.
2017-07-07 09:22:34 +02:00
e2935e7509
test: Changes WAN/LAN join confirmer to use port number vs. address.
...
This fixes TestServer_JoinSeparateLanAndWanAddresses which sets bogus
advertise addresses as part of the test. Port numbers uniquely identify
members since everything is running on localhost.
2017-07-07 09:22:34 +02:00
98dc634f17
test: make joinLAN/WAN reliable
...
only return if the members can see each other
2017-07-07 09:22:34 +02:00
74d3c4d896
rpc: make TestServer_JoinSeparateLanAndWanAddresses more robust
2017-07-07 09:22:34 +02:00
ae59198e38
rpc: make TestClient_SnapshotRPC_TLS more robust
2017-07-07 09:22:34 +02:00
96c03ce73b
rpc: try shutting down leader first to avoid hang in TestLeader_LeftServer
2017-07-07 09:22:34 +02:00
2eb2941e8c
rpc: fix logging and try quicker timing of TestServer_JoinSeparateLanAndWanAddresses
2017-07-07 09:22:34 +02:00
98510f898c
rpc: less agressive raft timeouts
...
Allowing more time for raft to consolidate should
drop the number of leader elections.
2017-07-07 09:22:34 +02:00
50c81a9397
rpc: run agent/consul tests in parallel
2017-07-07 09:22:34 +02:00
b3189a566a
rpc: refactor sessionTimers and fix racy tests
...
The sessionTimers map was secured by a lock which wasn't used
properly in the tests. This lead to data races and failing tests
when accessing the length or the members of the map.
This patch adds a separate SessionTimers struct which is safe
for concurrent use and which ecapsulates the behavior of the
sessionTimers map.
2017-07-07 09:22:34 +02:00
06ad8e96be
rpc: fix TestServer_Leave
...
wait for the leader election.
2017-07-07 09:22:34 +02:00
4a073aec1c
rpc: fix TestSession_Renew
...
make the timing less tight
2017-07-07 09:22:34 +02:00
77ff9f680f
rpc: fix TestReadyForConsistentRead
...
timing was too tight. Standardized name.
2017-07-07 09:22:34 +02:00
e3252f921a
rpc: fix for 'no leader' in TLS tests
...
Ensure both servers know about each other before looking
for a leader.
2017-07-07 09:22:34 +02:00
2497b8416b
rpc: fix TestServer_JoinWAN_Flood
...
The second server in the first data center should not be
in bootstrap mode.
2017-07-07 09:22:34 +02:00
7af30dd7d7
rpc: provide unique node names for server and client
2017-07-07 09:22:34 +02:00
457910b191
rpc: prefix log output with test name
2017-07-07 09:22:34 +02:00
c33f7ecbe2
rpc: discover serf wan port before starting serf lan
...
When using dynamic ports for the serf clusters then
the actual bind port of the serf WAN cluster needs to
be discovered before the serf LAN cluster is started
since the serf LAN cluster announces the port of the WAN
cluster.
2017-07-07 09:22:34 +02:00
84c90cbd07
rpc: bind rpc test server to port 0
2017-07-07 09:22:34 +02:00
7f5957ee93
rpc: refactor: unify test server setup
2017-07-07 09:22:34 +02:00
325637b6be
rpc: fix typos
2017-07-07 09:22:34 +02:00
85aa360843
agent: refactor: log to stderr during tests
2017-07-07 09:22:34 +02:00
ae656575ea
Rename to raftNotifyCh, fix typo
2017-07-06 09:10:36 -05:00
777504ff0e
Fixes deadlock between barrier write and leader notify channel read . Fixes #3230
2017-07-05 17:09:18 -05:00
728971afdb
Fixes broken HTTP header and method for health checks. ( #3178 )
...
* Fixes broken HTTP header and method for health checks.
* Adds a fuzz utility and test to make sure copy is complete.
2017-06-23 01:15:48 -07:00
d3ab99244b
agent: make the RPC endpoint overwrite mechanism more transparent
...
This patch hides the RPC handler overwrite mechanism from the
rest of the code so that it works in all cases and that there
is no cooperation required from the tested code, i.e. we can
drop a.getEndpoint().
2017-06-21 05:42:39 +02:00
3e20a2ba81
agent: move structs into consul/structs pkg
...
* CheckDefinition
* ServiceDefinition
* CheckType
2017-06-21 05:42:39 +02:00
db78252019
agent: move NotifyGroup into the agent pkg
2017-06-21 05:42:39 +02:00
2c47bc5d5b
agent: move conn pool for muxed connections into separate pkg
2017-06-21 05:42:39 +02:00
e930b55f71
agent: move the SnapshotReplyFn out of the way
...
When splitting up the consul package into server and client
the SnapshotReplyFn needs to be in a separate package to avoid
a circular dependency.
2017-06-21 05:42:39 +02:00
b805a79078
agent: use the delegate interface for local state
2017-06-21 05:42:39 +02:00
bb559d8e6e
Minor fixes per code review
2017-06-20 19:43:07 -05:00
f535a298f3
Added unit test to verify consistentRead method behavior
2017-06-16 11:58:12 -05:00
f616a8dd06
Code review feedback, fixed major logic bug
2017-06-16 10:49:54 -05:00
42d3a3f3db
Redo bug fix for stale reads on server startup, leveraging RPCHOldtimeout instead of maxQueryTime, plus tests
2017-06-15 22:41:30 -05:00
cd837b0b18
pkg refactor
...
command/agent/* -> agent/*
command/consul/* -> agent/consul/*
command/agent/command{,_test}.go -> command/agent{,_test}.go
command/base/command.go -> command/base.go
command/base/* -> command/*
commands.go -> command/commands.go
The script which did the refactor is:
(
cd $GOPATH/src/github.com/hashicorp/consul
git mv command/agent/command.go command/agent.go
git mv command/agent/command_test.go command/agent_test.go
git mv command/agent/flag_slice_value{,_test}.go command/
git mv command/agent .
git mv command/base/command.go command/base.go
git mv command/base/config_util{,_test}.go command/
git mv commands.go command/
git mv consul agent
rmdir command/base/
gsed -i -e 's|package agent|package command|' command/agent{,_test}.go
gsed -i -e 's|package agent|package command|' command/flag_slice_value{,_test}.go
gsed -i -e 's|package base|package command|' command/base.go command/config_util{,_test}.go
gsed -i -e 's|package main|package command|' command/commands.go
gsed -i -e 's|base.Command|BaseCommand|' command/commands.go
gsed -i -e 's|agent.Command|AgentCommand|' command/commands.go
gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/commands.go
gsed -i -e 's|base\.||' command/commands.go
gsed -i -e 's|command\.||' command/commands.go
gsed -i -e 's|command|c|' main.go
gsed -i -e 's|range Commands|range command.Commands|' main.go
gsed -i -e 's|Commands: Commands|Commands: command.Commands|' main.go
gsed -i -e 's|base\.BoolValue|BoolValue|' command/operator_autopilot_set.go
gsed -i -e 's|base\.DurationValue|DurationValue|' command/operator_autopilot_set.go
gsed -i -e 's|base\.StringValue|StringValue|' command/operator_autopilot_set.go
gsed -i -e 's|base\.UintValue|UintValue|' command/operator_autopilot_set.go
gsed -i -e 's|\bCommand\b|BaseCommand|' command/base.go
gsed -i -e 's|BaseCommand Options|Command Options|' command/base.go
gsed -i -e 's|base.Command|BaseCommand|' command/*.go
gsed -i -e 's|c\.Command|c.BaseCommand|g' command/*.go
gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/*_test.go
gsed -i -e 's|base\.||' command/*_test.go
gsed -i -e 's|\bCommand\b|AgentCommand|' command/agent{,_test}.go
gsed -i -e 's|cmd.AgentCommand|cmd.BaseCommand|' command/agent.go
gsed -i -e 's|cli.AgentCommand = new(Command)|cli.Command = new(AgentCommand)|' command/agent_test.go
gsed -i -e 's|exec.AgentCommand|exec.Command|' command/agent_test.go
gsed -i -e 's|exec.BaseCommand|exec.Command|' command/agent_test.go
gsed -i -e 's|NewTestAgent|agent.NewTestAgent|' command/agent_test.go
gsed -i -e 's|= TestConfig|= agent.TestConfig|' command/agent_test.go
gsed -i -e 's|: RetryJoin|: agent.RetryJoin|' command/agent_test.go
gsed -i -e 's|\.\./\.\./|../|' command/config_util_test.go
gsed -i -e 's|\bverifyUniqueListeners|VerifyUniqueListeners|' agent/config{,_test}.go command/agent.go
gsed -i -e 's|\bserfLANKeyring\b|SerfLANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
gsed -i -e 's|\bserfWANKeyring\b|SerfWANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
gsed -i -e 's|\bNewAgent\b|agent.New|g' command/agent{,_test}.go
gsed -i -e 's|\bNewAgent|New|' agent/{acl_test,agent,testagent}.go
gsed -i -e 's|\bAgent\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bBool\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bDefaultConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bDevConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bMergeConfig\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bReadConfigPaths\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bParseMetaPair\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bSerfLANKeyring\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|\bSerfWANKeyring\b|agent.&|g' command/agent{,_test}.go
gsed -i -e 's|circonus\.agent|circonus|g' command/agent{,_test}.go
gsed -i -e 's|logger\.agent|logger|g' command/agent{,_test}.go
gsed -i -e 's|metrics\.agent|metrics|g' command/agent{,_test}.go
gsed -i -e 's|// agent.Agent|// agent|' command/agent{,_test}.go
gsed -i -e 's|a\.agent\.Config|a.Config|' command/agent{,_test}.go
gsed -i -e 's|agent\.AppendSliceValue|AppendSliceValue|' command/{configtest,validate}.go
gsed -i -e 's|consul/consul|agent/consul|' GNUmakefile
gsed -i -e 's|\.\./test|../../test|' agent/consul/server_test.go
# fix imports
f=$(grep -rl 'github.com/hashicorp/consul/command/agent' * | grep '\.go')
gsed -i -e 's|github.com/hashicorp/consul/command/agent|github.com/hashicorp/consul/agent|' $f
goimports -w $f
f=$(grep -rl 'github.com/hashicorp/consul/consul' * | grep '\.go')
gsed -i -e 's|github.com/hashicorp/consul/consul|github.com/hashicorp/consul/agent/consul|' $f
goimports -w $f
goimports -w command/*.go main.go
)
2017-06-10 18:52:45 +02:00
Matt Keeler