Commit Graph

1053 Commits

Author SHA1 Message Date
Mitchell Hashimoto be83efe61e agent/proxy: set the proper arguments so we only run the helper process 2018-06-25 12:25:11 -07:00
Mitchell Hashimoto a7690301f9 agent/config: add AllowManagedRoot 2018-06-25 12:25:11 -07:00
Kyle Havlovitz 549dc22944 connect: fix two CA tests that were broken in a previous PR (#60) 2018-06-25 12:25:10 -07:00
Paul Banks 3433020fa6 Fix roots race with CA setup hammering bug and defensive nil check hit during obscure upgrade scenario 2018-06-25 12:25:10 -07:00
Kyle Havlovitz 1ce8361aa2 agent: format all CA config fields 2018-06-25 12:25:09 -07:00
Kyle Havlovitz a242e5b130 agent: update accepted CA config fields and defaults 2018-06-25 12:25:09 -07:00
Mitchell Hashimoto 7846206753 agent/proxy: fix build on Windows 2018-06-25 12:24:18 -07:00
Paul Banks 6c77f7883e Misc comment cleanups 2018-06-25 12:24:16 -07:00
Paul Banks d0674cdd7a Warn about killing proxies in dev mode 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 4ebddd6adb agent/consul: set precedence value on struct itself 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 61c7e33a22 agent/config: move ports to `ports` structure, update docs 2018-06-25 12:24:15 -07:00
Paul Banks d140612350 Fixs a few issues that stopped this working in real life but not caught by tests:
- Dev mode assumed no persistence of services although proxy state is persisted which caused proxies to be killed on startup as their services were no longer registered. Fixed.
 - Didn't snapshot the ProxyID which meant that proxies were adopted OK from snapshot but failed to restart if they died since there was no proxyID in the ENV on restart
 - Dev mode with no persistence just kills all proxies on shutdown since it can't recover them later
 - Naming things
2018-06-25 12:24:14 -07:00
Paul Banks 3df45ac7f1 Don't kill proxies on agent shutdown; backport manager close fix 2018-06-25 12:24:13 -07:00
Paul Banks 877390cd28 Test for adopted process Stop race and fix 2018-06-25 12:24:13 -07:00
Mitchell Hashimoto e016f37ae7 agent: accept connect param for execute 2018-06-25 12:24:12 -07:00
Mitchell Hashimoto 52c10d2208 agent/consul: support a Connect option on prepared query request 2018-06-25 12:24:12 -07:00
Mitchell Hashimoto e8c899b1b8 agent/consul: prepared query supports "Connect" field 2018-06-25 12:24:11 -07:00
Mitchell Hashimoto e3562e39cc agent: intention create returns 500 for bad body 2018-06-25 12:24:10 -07:00
Mitchell Hashimoto ad382d7351 agent: switch ConnectNative to an embedded struct 2018-06-25 12:24:10 -07:00
Paul Banks 1e5a2561b6 Make tests pass and clean proxy persistence. No detached child changes yet.
This is a good state for persistence stuff to re-start the detached child work that got mixed up last time.
2018-06-25 12:24:10 -07:00
Paul Banks 3bac52480e Abandon daemonize for simpler solution (preserving history):
Reverts:
  - bdb274852ae469c89092d6050697c0ff97178465
  - 2c689179c4f61c11f0016214c0fc127a0b813bfe
  - d62e25c4a7ab753914b6baccd66f88ffd10949a3
  - c727ffbcc98e3e0bf41e1a7bdd40169bd2d22191
  - 31b4d18933fd0acbe157e28d03ad59c2abf9a1fb
  - 85c3f8df3eabc00f490cd392213c3b928a85aa44
2018-06-25 12:24:10 -07:00
Paul Banks 9ef748157a WIP 2018-06-25 12:24:09 -07:00
Paul Banks 9cea27c66e Sanity check that we are never trying to self-exec a test binary. Add daemonize bypass for TestAgent so that we don't have to jump through ridiculous self-execution hooks for every package that might possibly invoke a managed proxy 2018-06-25 12:24:09 -07:00
Mitchell Hashimoto 56f5924f3e agent/proxy: Manager.Close also has to stop all proxy watchers 2018-06-25 12:24:09 -07:00
Paul Banks 18e64dafbc Fix import tooling fail 2018-06-25 12:24:09 -07:00
Paul Banks e1aca748c4 Make daemoinze an option on test binary without hacks. Misc fixes for racey or broken tests. Still failing on several though. 2018-06-25 12:24:09 -07:00
Paul Banks c97db00903 Run daemon processes as a detached child.
This turns out to have a lot more subtelty than we accounted for. The test suite is especially prone to races now we can only poll the child and many extra levels of indirectoin are needed to correctly run daemon process without it becoming a Zombie.

I ran this test suite in a loop with parallel enabled to verify for races (-race doesn't find any as they are logical inter-process ones not actual data races). I made it through ~50 runs before hitting an error due to timing which is much better than before. I want to go back and see if we can do better though. Just getting this up.
2018-06-25 12:24:08 -07:00
Paul Banks 3a00574a13 Persist proxy state through agent restart 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto a3e0ac1ee3 agent/consul/state: support querying by Connect native 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto bb98686ec8 agent/cache: update comment from PR review to clarify 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto 418ed161dc agent: agent service registration supports Connect native services 2018-06-25 12:24:08 -07:00
Mitchell Hashimoto 8e02bbc897 agent/consul: support catalog registration with Connect native 2018-06-25 12:24:07 -07:00
Mitchell Hashimoto 55b3d5d6f4 agent/cache: update comments 2018-06-25 12:24:07 -07:00
Mitchell Hashimoto ea0270e6aa agent/cache: correct test name 2018-06-25 12:24:07 -07:00
Mitchell Hashimoto b5201276bc agent/cache: change behavior to return error rather than retry
The cache behavior should not be to mask errors and retry. Instead, it
should aim to return errors as quickly as possible. We do that here.
2018-06-25 12:24:07 -07:00
Mitchell Hashimoto 778e318a52 agent/cache: perform backoffs on error retries on blocking queries 2018-06-25 12:24:06 -07:00
Kyle Havlovitz 54bc937fed
Re-use uint8ToString 2018-06-14 09:42:23 -07:00
Kyle Havlovitz 4d46bba2c4
Support giving the duration as a string in CA config 2018-06-14 09:42:22 -07:00
Mitchell Hashimoto 771842255a
address comment feedback 2018-06-14 09:42:22 -07:00
Mitchell Hashimoto 9249662c6c
agent: leaf endpoint accepts name, not service ID
This change is important so that requests can made representing a
service that may not be registered with the same local agent.
2018-06-14 09:42:20 -07:00
Mitchell Hashimoto 787ce3b269
agent: address feedback 2018-06-14 09:42:20 -07:00
Mitchell Hashimoto b5b29cd6af
agent: rename test to check 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto b961bab08c
agent: implement HTTP endpoint 2018-06-14 09:42:18 -07:00
Mitchell Hashimoto a48ff54318
agent/consul: forward request if necessary 2018-06-14 09:42:17 -07:00
Mitchell Hashimoto b02502be73
agent: comments to point to differing logic 2018-06-14 09:42:17 -07:00
Mitchell Hashimoto 526cfc34bd
agent/consul: implement Intention.Test endpoint 2018-06-14 09:42:17 -07:00
Paul Banks bd5e569dc7
Make invalid clusterID be fatal 2018-06-14 09:42:17 -07:00
Paul Banks 919fd3e148
Fix logical conflicts with CA refactor 2018-06-14 09:42:17 -07:00
Paul Banks 73f2a49ef1
Fix broken api test for service Meta (logical conflict rom OSS). Add test that would make this much easier to catch in future. 2018-06-14 09:42:17 -07:00
Paul Banks bd5eb8b749
Add default CA config back - I didn't add it and causes nil panics 2018-06-14 09:42:17 -07:00
Paul Banks dbcf286d4c
Ooops remove the CA stuff from actual server defaults and make it test server only 2018-06-14 09:42:16 -07:00
Paul Banks 834ed1d25f
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes. 2018-06-14 09:42:16 -07:00
Paul Banks bdd30b191b
Comment cleanup 2018-06-14 09:42:16 -07:00
Paul Banks 5abf47472d
Verify trust domain on /authorize calls 2018-06-14 09:42:16 -07:00
Paul Banks 30d90b3be4
Generate CSR using real trust-domain 2018-06-14 09:42:16 -07:00
Paul Banks 5a1408f186
Add CSR signing verification of service ACL, trust domain and datacenter. 2018-06-14 09:42:16 -07:00
Paul Banks c808833a78
Return TrustDomain from CARoots RPC 2018-06-14 09:42:15 -07:00
Kyle Havlovitz d1265bc38b
Rename some of the CA structs/files 2018-06-14 09:42:15 -07:00
Kyle Havlovitz 1660f9ebab
Add more metadata to structs.CARoot 2018-06-14 09:42:15 -07:00
Kyle Havlovitz baf4db1c72
Use provider state table for a global serial index 2018-06-14 09:42:15 -07:00
Kyle Havlovitz 5998623c44
Add test for ca config http endpoint 2018-06-14 09:42:15 -07:00
Kyle Havlovitz c90b353eea
Move connect CA provider to separate package 2018-06-14 09:42:15 -07:00
Mitchell Hashimoto 4bb745a2d4
agent/cache: change uint8 to uint 2018-06-14 09:42:15 -07:00
Mitchell Hashimoto 6cf2e1ef1a
agent/cache: string through attempt rather than storing on the entry 2018-06-14 09:42:15 -07:00
Mitchell Hashimoto c42510e1ec
agent/cache: implement refresh backoff 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto 54a1662da8
agent/consul: change provider wait from goto to a loop 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto 749f81373f
agent/consul: check nil on getCAProvider result 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto c57405b323
agent/consul: retry reading provider a few times 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto b4f990bc6c
agent: verify local proxy tokens for CA leaf + tests 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto 8f7b5f93cd
agent: verify proxy token for ProxyConfig endpoint + tests 2018-06-14 09:42:14 -07:00
Mitchell Hashimoto 3a7aaa63bc
agent/proxy: pass proxy ID as an env var 2018-06-14 09:42:13 -07:00
Mitchell Hashimoto f69c8b85ef
agent/config: add managed proxy upstreams config to skip
agent/config will turn [{}] into {} (single element maps into a single
map) to work around HCL issues. These are resolved in HCL2 which I'm
sure Consul will switch to eventually.

This breaks the connect proxy configuration in service definition FILES
since we call this patch function. For now, let's just special-case skip
this. In the future we maybe Consul will adopt HCL2 and fix it, or we
can do something else if we want. This works and is tested.
2018-06-14 09:42:13 -07:00
Mitchell Hashimoto 662f38c625
agent/structs: validate service definitions, port required for proxy 2018-06-14 09:42:13 -07:00
Mitchell Hashimoto 498c63a6f1
agent/config: default connect enabled in dev mode
This enables `consul agent -dev` to begin using Connect features with
the built-in CA. I think this is expected behavior since you can imagine
that new users would want to try.

There is no real downside since we're just using the built-in CA.
2018-06-14 09:42:13 -07:00
Paul Banks 954d286d73
Make CSR work with jank domain 2018-06-14 09:42:13 -07:00
Mitchell Hashimoto 257d31e319
agent/proxy: delete pid file on Stop 2018-06-14 09:42:13 -07:00
Mitchell Hashimoto 1dfb4762f5
agent: increase timer for blocking cache endpoints 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto a89238a9d3
agent/proxy: address PR feedback 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto 7bb13246a8
agent: clarify why we Kill still 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto 147b066c67
agent: restore proxy snapshot but still Kill proxies 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto 1d24df3827
agent/proxy: check if process is alive in addition to Wait 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto 4301f7f1f5
agent: only set the proxy manager data dir if its set 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto e3be9f7a02
agent/proxy: improve comments on snapshotting 2018-06-14 09:42:12 -07:00
Mitchell Hashimoto eb31827fac
agent/proxy: implement periodic snapshotting in the manager 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto 64fc9e0218
agent/proxy: check if process is alive 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto a3a0bc7b13
agent/proxy: implement snapshotting for daemons 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto 9675ed626d
agent/proxy: manager configures the daemon pid path to write pids 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto 5e0f0ba178
agent/proxy: write pid file whenever the daemon process changes 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto 09093a1a1a
agent/proxy: change LogDir to DataDir to reuse for other things 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto e2133fd391
agent/proxy: make the logs test a bit more robust by waiting for file 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto d019d33bc6
agent/proxy: don't create the directory in newProxy 2018-06-14 09:42:11 -07:00
Mitchell Hashimoto 49bc7181a4
agent/proxy: send logs to the correct location for daemon proxies 2018-06-14 09:42:10 -07:00
Mitchell Hashimoto 515c47be7d
agent: add additional tests for defaulting in AddProxy 2018-06-14 09:42:10 -07:00
Mitchell Hashimoto 52665f7d23
agent: clean up defaulting of proxy configuration
This cleans up and unifies how proxy settings defaults are applied.
2018-06-14 09:42:10 -07:00
Mitchell Hashimoto ed14e9edf8
agent: resolve some conflicts and fix tests 2018-06-14 09:42:10 -07:00
Mitchell Hashimoto 657c09133a
agent/local: clarify the non-risk of a full buffer 2018-06-14 09:42:10 -07:00
Mitchell Hashimoto 31b09c0674
agent/local: remove outdated comment 2018-06-14 09:42:10 -07:00
Mitchell Hashimoto bae428326a
agent: use os.Executable 2018-06-14 09:42:09 -07:00
Mitchell Hashimoto 6a78ecea57
agent/proxy: local state event coalescing 2018-06-14 09:42:09 -07:00
Mitchell Hashimoto 16f529a13c
agent/proxy: implement force kill of unresponsive proxy process 2018-06-14 09:42:09 -07:00