Paul Banks
02ab461dae
TLS watching integrated into Service with some basic tests.
...
There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.
2018-06-14 09:42:07 -07:00
Paul Banks
dcd277de8a
Wire up agent leaf endpoint to cache framework to support blocking.
2018-06-14 09:42:07 -07:00
Kyle Havlovitz
b28e11fdd3
Fill out connect CA rpc endpoint tests
2018-06-14 09:42:06 -07:00
Kyle Havlovitz
0e184f3f5b
Fix config tests
2018-06-14 09:42:06 -07:00
Kyle Havlovitz
7c0976208d
Add tests for the built in CA's state store table
2018-06-14 09:42:06 -07:00
Kyle Havlovitz
19b9399f2f
Add more tests for built-in provider
2018-06-14 09:42:06 -07:00
Kyle Havlovitz
a29f3c6b96
Fix some inconsistencies around the CA provider code
2018-06-14 09:42:06 -07:00
Paul Banks
153808db7c
Don't allow connect watches in agent/cli yet
2018-06-14 09:42:06 -07:00
Paul Banks
072b2a79ca
Support legacy watch.HandlerFunc type for backward compat reduces impact of change
2018-06-14 09:42:05 -07:00
Paul Banks
6f566f750e
Basic `watch` support for connect proxy config and certificate endpoints.
...
- Includes some bug fixes for previous `api` work and `agent` that weren't tested
- Needed somewhat pervasive changes to support hash based blocking - some TODOs left in our watch toolchain that will explicitly fail on hash-based watches.
- Integration into `connect` is partially done here but still WIP
2018-06-14 09:42:05 -07:00
Kyle Havlovitz
2167713226
Add CA config to connect section of agent config
2018-06-14 09:42:05 -07:00
Kyle Havlovitz
02fef5f9a2
Move ConsulCAProviderConfig into structs package
2018-06-14 09:42:04 -07:00
Kyle Havlovitz
887cc98d7e
Simplify the CAProvider.Sign method
2018-06-14 09:42:04 -07:00
Kyle Havlovitz
44b30476cb
Simplify the CA provider interface by moving some logic out
2018-06-14 09:42:04 -07:00
Kyle Havlovitz
aa10fb2f48
Clarify some comments and names around CA bootstrapping
2018-06-14 09:42:04 -07:00
Mitchell Hashimoto
5abd43a567
agent: resolve flaky test by checking cache hits increase, rather than
...
exact
2018-06-14 09:42:04 -07:00
Mitchell Hashimoto
73838c9afa
agent: use helper/retry instead of timing related tests
2018-06-14 09:42:04 -07:00
Mitchell Hashimoto
dcb2671d10
agent/cache: address PR feedback, lots of typos
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
07d878a157
agent/cache: address feedback, clarify comments
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
ad3928b6bd
agent/cache: don't every block on NotifyCh
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
3f80a9f330
agent/cache: unit tests for ExpiryHeap, found a bug!
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
1c31e34e5b
agent/cache: send the total entries count on eviction to go-metrics
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
ec559d77bd
agent/cache: make edge case with prev/next idx == 0 handled better
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
b319d06276
agent/cache: rework how expiry data is stored to be more efficient
2018-06-14 09:42:03 -07:00
Mitchell Hashimoto
449bbd817d
agent/cache: initial TTL work
2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
3c6acbda5d
agent/cache: send the RefreshTimeout into the backend fetch
2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
257fc34e51
agent/cache: on error, return from Get immediately, don't block forever
2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
e9d58ca219
agent/cache: lots of comment/doc updates
2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
a1f8cb9570
agent: augment /v1/connect/authorize to cache intentions
2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
56774f24d0
agent/cache-types: support intention match queries
2018-06-14 09:42:02 -07:00
Mitchell Hashimoto
109bb946e9
agent/cache: return the error as part of Get
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
6ecc2da7ff
agent/cache: integrate go-metrics so the cache is debuggable
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
3b6c46b7d7
agent/structs: DCSpecificRequest sets all the proper fields for
...
CacheInfo
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
ccd7eeef1a
agent/cache-types/ca-leaf: proper result for timeout, race on setting CA
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
4509589427
agent/cache: support timeouts for cache reads and empty fetch results
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
b0f70f17db
agent/cache-types: rename to separate root and leaf cache types
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
e3b1c400e5
agent/cache-types: got basic CA leaf caching work, major problems still
2018-06-14 09:42:01 -07:00
Mitchell Hashimoto
9e44a319d3
agent: check cache hit count to verify CA root caching, background update
2018-06-14 09:42:00 -07:00
Mitchell Hashimoto
8bb4fd95a6
agent: initialize the cache and cache the CA roots
2018-06-14 09:42:00 -07:00
Mitchell Hashimoto
286217cbd8
agent/cache: partition by DC/ACL token
2018-06-14 09:42:00 -07:00
Mitchell Hashimoto
72c82a9b29
agent/cache: Reorganize some files, RequestInfo struct, prepare for partitioning
2018-06-14 09:42:00 -07:00
Mitchell Hashimoto
ecc789ddb5
agent/cache: ConnectCA roots caching type
2018-06-14 09:42:00 -07:00
Mitchell Hashimoto
c69df79e0c
agent/cache: blank cache key means to always fetch
2018-06-14 09:42:00 -07:00
Mitchell Hashimoto
8584e9262e
agent/cache: initial kind-of working cache
2018-06-14 09:42:00 -07:00
Kyle Havlovitz
43f13d5a0b
Add cross-signing mechanism to root rotation
2018-06-14 09:42:00 -07:00
Kyle Havlovitz
bbfcb278e1
Add the root rotation mechanism to the CA config endpoint
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
a585a0ba10
Have the built in CA store its state in raft
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
80eddb0bfb
Fix the testing endpoint's root set op
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
9fefac745e
Update the CA config endpoint to enable GETs
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
fc9ef9741b
Hook the CA RPC endpoint into the provider interface
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
a40db26ffe
Add CA bootstrapping on establishing leadership
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
e26819ed9c
Add the bootstrap config for the CA
2018-06-14 09:41:59 -07:00
Kyle Havlovitz
4d0713d5bb
Add the CA provider interface and built-in provider
2018-06-14 09:41:58 -07:00
Kyle Havlovitz
ebdda17a30
Add CA config set to fsm operations
2018-06-14 09:41:58 -07:00
Kyle Havlovitz
f7ff16669f
Add the Connect CA config to the state store
2018-06-14 09:41:58 -07:00
Paul Banks
a90f69faa4
Adds `api` client code and tests for new Proxy Config endpoint, registering with proxy and seeing proxy config in /agent/services list.
2018-06-14 09:41:58 -07:00
Paul Banks
9d11cd9bf4
Fix various test failures and vet warnings.
...
Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.
2018-06-14 09:41:58 -07:00
Paul Banks
8a4410b549
Refactor localBlockingQuery to use memdb.WatchSet. Much simpler and correct as a bonus!
2018-06-14 09:41:58 -07:00
Paul Banks
aed5e5b03e
Super ugly hack to get TeamCity build to work for this PR without adding a vendor that is being added elsewhere and will conflict...
2018-06-14 09:41:58 -07:00
Paul Banks
cbd8606651
Add X-Consul-ContentHash header; implement removing all proxies; add load/unload test.
2018-06-14 09:41:57 -07:00
Paul Banks
44afb5c699
Agent Connect Proxy config endpoint with hash-based blocking
2018-06-14 09:41:57 -07:00
Paul Banks
c2266b134a
HTTP agent registration allows proxy to be defined.
2018-06-14 09:41:57 -07:00
Paul Banks
78e48fd547
Added connect proxy config and local agent state setup on boot.
2018-06-14 09:41:57 -07:00
Paul Banks
280382c25f
Add tests all the way up through the endpoints to ensure duplicate src/destination is supported and so ultimately deny/allow nesting works.
...
Also adds a sanity check test for `api.Agent().ConnectAuthorize()` and a fix for a trivial bug in it.
2018-06-14 09:41:57 -07:00
Paul Banks
adc5589329
Allow duplicate source or destination, but enforce uniqueness across all four.
2018-06-14 09:41:57 -07:00
Paul Banks
51b1bc028d
Rework connect/proxy and command/connect/proxy. End to end demo working again
2018-06-14 09:41:57 -07:00
Paul Banks
2d6a2ce1e3
connect.Service based implementation after review feedback.
2018-06-14 09:41:56 -07:00
Mitchell Hashimoto
62b746c380
agent: rename authorize param ClientID to ClientCertURI
2018-06-14 09:41:56 -07:00
Mitchell Hashimoto
94e7a0a3c1
agent: add TODO for verification
2018-06-14 09:41:55 -07:00
Mitchell Hashimoto
f983978fb8
acl: IntentionDefault => IntentionDefaultAllow
2018-06-14 09:41:55 -07:00
Mitchell Hashimoto
b3584b6355
agent: ACL checks for authorize, default behavior
2018-06-14 09:41:55 -07:00
Mitchell Hashimoto
3e0e0a94a7
agent/structs: String format for Intention, used for logging
2018-06-14 09:41:55 -07:00
Mitchell Hashimoto
3f80808379
agent: bolster commenting for clearer understandability
2018-06-14 09:41:55 -07:00
Mitchell Hashimoto
c6269cda37
agent: default deny on connect authorize endpoint
2018-06-14 09:41:54 -07:00
Mitchell Hashimoto
5364a8cd90
agent: /v1/agent/connect/authorize is functional, with tests
2018-06-14 09:41:54 -07:00
Mitchell Hashimoto
7af99667b6
agent/connect: Authorize for CertURI
2018-06-14 09:41:54 -07:00
Mitchell Hashimoto
68fa4a83b1
agent: get rid of method checks since they're done in the http layer
2018-06-14 09:41:54 -07:00
Paul Banks
894ee3c5b0
Add Connect agent, catalog and health endpoints to api Client
2018-06-14 09:41:54 -07:00
Mitchell Hashimoto
1985655dff
agent/consul/state: ensure exactly one active CA exists when setting
2018-06-14 09:41:54 -07:00
Mitchell Hashimoto
9d93c52098
agent/connect: support any values in the URL
2018-06-14 09:41:54 -07:00
Mitchell Hashimoto
8934f00d03
agent/connect: support SpiffeIDSigning
2018-06-14 09:41:53 -07:00
Mitchell Hashimoto
da1bc48372
agent/connect: rename SpiffeID to CertURI
2018-06-14 09:41:53 -07:00
Mitchell Hashimoto
b0315811b9
agent/connect: use proper keyusage fields for CA and leaf
2018-06-14 09:41:53 -07:00
Mitchell Hashimoto
434d8750ae
agent/connect: address PR feedback for the CA.go file
2018-06-14 09:41:53 -07:00
Mitchell Hashimoto
e0562f1c21
agent: implement an always-200 authorize endpoint
2018-06-14 09:41:53 -07:00
Mitchell Hashimoto
2026cf3753
agent/consul: encode issued cert serial number as hex encoded
2018-06-14 09:41:53 -07:00
Mitchell Hashimoto
deb55c436d
agent/structs: hide some fields from JSON
2018-06-14 09:41:52 -07:00
Mitchell Hashimoto
746f80639a
agent: /v1/connect/ca/configuration PUT for setting configuration
2018-06-14 09:41:52 -07:00
Mitchell Hashimoto
2dfca5dbc2
agent/consul/fsm,state: snapshot/restore for CA roots
2018-06-14 09:41:52 -07:00
Mitchell Hashimoto
17d6b437d2
agent/consul/fsm,state: tests for CA root related changes
2018-06-14 09:41:52 -07:00
Mitchell Hashimoto
a8510f8224
agent/consul: set more fields on the issued cert
2018-06-14 09:41:52 -07:00
Mitchell Hashimoto
58b6f476e8
agent: /v1/connect/ca/leaf/:service_id
2018-06-14 09:41:52 -07:00
Mitchell Hashimoto
748a0bb824
agent: CA root HTTP endpoints
2018-06-14 09:41:51 -07:00
Mitchell Hashimoto
80a058a573
agent/consul: CAS operations for setting the CA root
2018-06-14 09:41:51 -07:00
Mitchell Hashimoto
712888258b
agent/consul: tests for CA endpoints
2018-06-14 09:41:51 -07:00
Mitchell Hashimoto
1928c07d0c
agent/consul: key the public key of the CSR, verify in test
2018-06-14 09:41:51 -07:00
Mitchell Hashimoto
9a8653f45e
agent/consul: test for ConnectCA.Sign
2018-06-14 09:41:51 -07:00
Mitchell Hashimoto
a360c5cca4
agent/consul: basic sign endpoint not tested yet
2018-06-14 09:41:51 -07:00
Mitchell Hashimoto
6550ff9492
agent/connect: package for agent-related Connect, parse SPIFFE IDs
2018-06-14 09:41:50 -07:00
Mitchell Hashimoto
f433f61fdf
agent/structs: json omit QueryMeta
2018-06-14 09:41:50 -07:00
Mitchell Hashimoto
9ad2a12441
agent: /v1/connect/ca/roots
2018-06-14 09:41:50 -07:00
Mitchell Hashimoto
24830f4cfa
agent/consul: RPC endpoints to list roots
2018-06-14 09:41:50 -07:00
Mitchell Hashimoto
cfb62677c0
agent/consul/state: CARoot structs and initial state store
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
7e8d606717
agent: address PR feedback
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
767d2eaef6
agent: commenting some tests
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
f9a55aa7e0
agent: clarified a number of comments per PR feedback
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
62cbb892e3
agent/consul: Health.ServiceNodes ACL check for Connect
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
641c982480
agent/consul: Catalog endpoint ACL requirements for Connect proxies
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
4cc4de1ff6
agent: remove ConnectProxyServiceName
2018-06-14 09:41:49 -07:00
Mitchell Hashimoto
566c98b2fc
agent/consul: require name for proxies
2018-06-14 09:41:48 -07:00
Mitchell Hashimoto
4207bb42c0
agent: validate service entry on register
2018-06-14 09:41:48 -07:00
Mitchell Hashimoto
b5fd3017bb
agent/structs: tests for PartialClone and IsSame for proxy fields
2018-06-14 09:41:48 -07:00
Mitchell Hashimoto
c43ccd024a
agent/local: anti-entropy for connect proxy services
2018-06-14 09:41:48 -07:00
Mitchell Hashimoto
daaa6e2403
agent: clean up connect/non-connect duplication by using shared methods
2018-06-14 09:41:48 -07:00
Mitchell Hashimoto
3d82d261bd
agent: /v1/health/connect/:service
2018-06-14 09:41:48 -07:00
Mitchell Hashimoto
119ffe3ed9
agent/consul: implement Health.ServiceNodes for Connect, DNS works
2018-06-14 09:41:47 -07:00
Mitchell Hashimoto
a5fe6204d5
agent: working DNS for Connect queries, I think, but have to
...
implement Health endpoints to be sure
2018-06-14 09:41:47 -07:00
Mitchell Hashimoto
fa4f0d353b
agent: /v1/catalog/connect/:service
2018-06-14 09:41:47 -07:00
Mitchell Hashimoto
253256352c
agent/consul: Catalog.ServiceNodes supports Connect filtering
2018-06-14 09:41:47 -07:00
Mitchell Hashimoto
06957f6d7f
agent/consul/state: ConnectServiceNodes
2018-06-14 09:41:47 -07:00
Mitchell Hashimoto
200100d3f4
agent/consul: enforce ACL on ProxyDestination
2018-06-14 09:41:47 -07:00
Mitchell Hashimoto
8a72826483
agent/consul: proxy registration and tests
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
6cd9e0e37c
agent: /v1/agent/services test with connect proxies (works w/ no change)
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
8777ff139c
agent: test /v1/catalog/node/:node to list connect proxies
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
761b561946
agent: /v1/catalog/service/:service works with proxies
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
58bff8dd05
agent/consul/state: convert proxy test to testify/assert
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
09568ce7b5
agent/consul/state: service registration with proxy works
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
23ee0888ec
agent/consul: convert intention ACLs to testify/assert
2018-06-14 09:41:46 -07:00
Mitchell Hashimoto
6a8bba7d48
agent/consul,structs: add tests for ACL filter and prefix for intentions
2018-06-14 09:41:45 -07:00
Mitchell Hashimoto
3e10a1ae7a
agent/consul: Intention.Match ACLs
2018-06-14 09:41:45 -07:00
Mitchell Hashimoto
db44a98a2d
agent/consul: Intention.Get ACLs
2018-06-14 09:41:45 -07:00
Mitchell Hashimoto
fd840da97a
agent/consul: Intention.Apply ACL on rename
2018-06-14 09:41:45 -07:00
Mitchell Hashimoto
14ca93e09c
agent/consul: tests for ACLs on Intention.Apply update/delete
2018-06-14 09:41:45 -07:00
Mitchell Hashimoto
c54be9bc09
agent/consul: Basic ACL on Intention.Apply
2018-06-14 09:41:44 -07:00
Mitchell Hashimoto
1d0b4ceedb
agent: convert all intention tests to testify/assert
2018-06-14 09:41:44 -07:00
Mitchell Hashimoto
f07340e94f
agent/consul/fsm,state: snapshot/restore for intentions
2018-06-14 09:41:44 -07:00
Mitchell Hashimoto
6f33b2d070
agent: use UTC time for intention times, move empty list check to
...
agent/consul
2018-06-14 09:41:43 -07:00
Mitchell Hashimoto
67b017c95c
agent/consul/fsm: switch tests to use structs.TestIntention
2018-06-14 09:41:43 -07:00
Mitchell Hashimoto
3a00564411
agent/consul/state: need to set Meta for intentions for tests
2018-06-14 09:41:43 -07:00
Mitchell Hashimoto
027dad8672
agent/consul/state: remove TODO
2018-06-14 09:41:43 -07:00
Mitchell Hashimoto
37f66e47ed
agent: use testing intention to get valid intentions
2018-06-14 09:41:43 -07:00
Mitchell Hashimoto
04bd4af99c
agent/consul: set default intention SourceType, validate it
2018-06-14 09:41:43 -07:00
Mitchell Hashimoto
8e2462e301
agent/structs: Intention validation
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
d34ee200de
agent/consul: support intention description, meta is non-nil
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
e81d1c88b7
agent/consul/fsm: add tests for intention requests
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
2b047fb09b
agent,agent/consul: set default namespaces
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
e630d65d9d
agent/consul: set CreatedAt, UpdatedAt on intentions
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
237da67da5
agent: GET /v1/connect/intentions/match
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
e9d208bcb6
agent/consul: RPC endpoint for Intention.Match
2018-06-14 09:41:42 -07:00
Mitchell Hashimoto
987b7ce0a2
agent/consul/state: IntentionMatch for performing match resolution
2018-06-14 09:41:41 -07:00