Commit graph

105 commits

Author SHA1 Message Date
Paul Banks bc5333905a
connect: remove additional trust-domain validation (#4934)
* connct: Remove additional trust-domain validation

* Comment typos

* Update connect_ca.go
2018-11-12 20:20:12 +00:00
Paul Banks 0523efa2fe merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in 2018-10-10 16:55:34 +01:00
Paul Banks 979e1c9c94 Add -sidecar-for and new /agent/service/:service_id endpoint (#4691)
- A new endpoint `/v1/agent/service/:service_id` which is a generic way to look up the service for a single instance. The primary value here is that it:
   - **supports hash-based blocking** and so;
   - **replaces `/agent/connect/proxy/:proxy_id`** as the mechanism the built-in proxy uses to read its config.
   - It's not proxy specific and so works for any service.
   - It has a temporary shim to call through to the existing endpoint to preserve current managed proxy config defaulting behaviour until that is removed entirely (tested).
 - The built-in proxy now uses the new endpoint exclusively for it's config
 - The built-in proxy now has a `-sidecar-for` flag that allows the service ID of the _target_ service to be specified, on the condition that there is exactly one "sidecar" proxy (that is one that has `Proxy.DestinationServiceID` set) for the service registered.
 - Several fixes for edge cases for SidecarService
 - A fix for `Alias` checks - when running locally they didn't update their state until some external thing updated the target. If the target service has no checks registered as below, then the alias never made it past critical.
2018-10-10 16:55:34 +01:00
Paul Banks 92fe8c8e89 Add Proxy Upstreams to Service Definition (#4639)
* Refactor Service Definition ProxyDestination.

This includes:
 - Refactoring all internal structs used
 - Updated tests for both deprecated and new input for:
   - Agent Services endpoint response
   - Agent Service endpoint response
   - Agent Register endpoint
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Register
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Services endpoint response
   - Catalog Node endpoint response
   - Catalog Service endpoint response
 - Updated API tests for all of the above too (both deprecated and new forms of register)

TODO:
 - config package changes for on-disk service definitions
 - proxy config endpoint
 - built-in proxy support for new fields

* Agent proxy config endpoint updated with upstreams

* Config file changes for upstreams.

* Add upstream opaque config and update all tests to ensure it works everywhere.

* Built in proxy working with new Upstreams config

* Command fixes and deprecations

* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...

TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.

* Fix translated keys in API registration.
≈

* Fixes from docs
 - omit some empty undocumented fields in API
 - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.

* Documentation updates for Upstreams in service definition

* Fixes for tests broken by many refactors.

* Enable travis on f-connect branch in this branch too.

* Add consistent Deprecation comments to ProxyDestination uses

* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
2018-10-10 16:55:34 +01:00
Matt Keeler 62c631368d
Connect: Verify the leaf cert to determine its readiness. (#4540)
This improves the checking so that if a certificate were to expire or the roots changed then we will go into a non-ready state.

This parses the x509 certificates from the TLS certificate when the leaf is set. The readyCh will be closed whenever a parseable certificate is set and the ca roots are set. This does not mean that the certificate is valid but that it has been setup and is generally valid. The Ready function will now do x509 certificate verification which will in addition to verifying the signatures with the installed CA roots will also verify the certificate isn't expired or not set to become valid in the future. 

The correct way to use these functions is to wait for the ReadyWait chan to be closed and then periodically check the readiness to determine if the certificate is currently useable.
2018-09-07 10:58:06 -04:00
Pierre Souchay e974ebd62e Fixed flaky tests (#4626) 2018-09-04 12:31:51 +01:00
Pierre Souchay 0c2d9b0782 Fixed unstable test TestProxy_public (#4587) 2018-08-27 11:45:07 -04:00
Freddy ba90922b27
Improve flaky connect/proxy Listener tests (#4498)
Improve flaky connect/proxy Listener tests

- Add sleep to TestEchoConn to allow for Read/Write to finish before fetching data in reportStats

- Account for flakiness around interval for Gauge

- Improve debug output when dumping metrics
2018-08-08 14:56:03 -04:00
Paul Banks 018e4f12a1
Implement missing HTTP host to ConsulResolver func for Connect SDK. 2018-07-13 22:39:18 +01:00
Paul Banks 21fb98ad5a Fix test broken by final telemetry PR change! 2018-06-25 12:25:40 -07:00
Paul Banks ad4df3c3ef Fix merge error 2018-06-25 12:25:40 -07:00
Paul Banks d1810ba338 Make proxy only listen after initial certs are fetched 2018-06-25 12:25:40 -07:00
Paul Banks ca68136ac7 Refactor to use embedded struct. 2018-06-25 12:25:39 -07:00
Paul Banks 23be6ad1c8 StartupTelemetry => InitTelemetry 2018-06-25 12:25:39 -07:00
Paul Banks 530d4acc57 Misc rebase and test fixes 2018-06-25 12:25:38 -07:00
Paul Banks ca9640030e Basic proxy active conns and bandwidth telemetry 2018-06-25 12:25:38 -07:00
Paul Banks 8ed46d7701 Add accessor and helpers to SDK for fetching self-name and client service ID 2018-06-25 12:25:38 -07:00
Paul Banks ff162ffdde Basic proxy telemetry working; not sure if it's too ugly; need to instrument things we care about 2018-06-25 12:25:38 -07:00
Paul Banks ced9b2bee4 Expose telemetry config from RuntimeConfig to proxy config endpoint 2018-06-25 12:25:38 -07:00
Paul Banks a8d3131de9 Return defensive error if API response is jank 2018-06-25 12:25:10 -07:00
Paul Banks f6a804029f Refactor resolver logic to be clearer 2018-06-25 12:25:10 -07:00
Paul Banks 3433020fa6 Fix roots race with CA setup hammering bug and defensive nil check hit during obscure upgrade scenario 2018-06-25 12:25:10 -07:00
Paul Banks d0c2f88aba More misc comment cleanup 2018-06-25 12:24:17 -07:00
Paul Banks 6c77f7883e Misc comment cleanups 2018-06-25 12:24:16 -07:00
Mitchell Hashimoto 36adf98cc4 api: change Connect to a query option 2018-06-25 12:24:14 -07:00
Mitchell Hashimoto 83a06df778 connect: remove old unused code 2018-06-25 12:24:14 -07:00
Mitchell Hashimoto 4f8fbd53d3 connect: support prepared query resolution 2018-06-25 12:24:13 -07:00
Mitchell Hashimoto 489c84f953 connect: resolver works with native services 2018-06-25 12:24:12 -07:00
Mitchell Hashimoto 510a8a6a6c
connect/proxy: remove dev CA settings 2018-06-14 09:42:22 -07:00
Mitchell Hashimoto 99094d70b0
connect/proxy: add a full proxy test, parallel 2018-06-14 09:42:21 -07:00
Mitchell Hashimoto b88023c607
connect/proxy: don't start public listener if 0 port 2018-06-14 09:42:21 -07:00
Mitchell Hashimoto 27aa0743ec
connect/proxy: use the right variable for loading the new service 2018-06-14 09:42:20 -07:00
Mitchell Hashimoto b28e2b8622
connect/proxy: don't require proxy ID 2018-06-14 09:42:20 -07:00
Paul Banks 957aaf69ab
Make Service logger log to right place again 2018-06-14 09:42:17 -07:00
Paul Banks 69b668c951
Make connect client resolver resolve trust domain properly 2018-06-14 09:42:17 -07:00
Paul Banks 834ed1d25f
Fixed many tests after rebase. Some still failing and seem unrelated to any connect changes. 2018-06-14 09:42:16 -07:00
Mitchell Hashimoto 9435d8088c
command/connect/proxy: set proxy ID from env var if set 2018-06-14 09:42:14 -07:00
Paul Banks 02ab461dae
TLS watching integrated into Service with some basic tests.
There are also a lot of small bug fixes found when testing lots of things end-to-end for the first time and some cleanup now it's integrated with real CA code.
2018-06-14 09:42:07 -07:00
Paul Banks 554f367dad
Fix build error introduced in bad merge of TLS stuff 2018-06-14 09:42:07 -07:00
Paul Banks 8b38cdaba1
Add TODO for false-sharing 2018-06-14 09:42:07 -07:00
Paul Banks 4c1b82834b
Add support for measuring tx/rx packets through proxied connections. 2018-06-14 09:42:06 -07:00
Paul Banks 2b1660fdf7
Fix tests and listeners to work with Config changes (splitting host and port fields) 2018-06-14 09:42:05 -07:00
Paul Banks 072b2a79ca
Support legacy watch.HandlerFunc type for backward compat reduces impact of change 2018-06-14 09:42:05 -07:00
Paul Banks eca94dcc92
Working proxy config reload tests 2018-06-14 09:42:05 -07:00
Paul Banks 6f566f750e
Basic watch support for connect proxy config and certificate endpoints.
- Includes some bug fixes for previous `api` work and `agent` that weren't tested
 - Needed somewhat pervasive changes to support hash based blocking - some TODOs left in our watch toolchain that will explicitly fail on hash-based watches.
 - Integration into `connect` is partially done here but still WIP
2018-06-14 09:42:05 -07:00
Paul Banks 53dc914d21
Refactor reloadableTLSConfig and verifyier shenanigans into simpler dynamicTLSConfig 2018-06-14 09:42:05 -07:00
Paul Banks 216e74b4ad
Connect verification and AuthZ 2018-06-14 09:42:05 -07:00
Paul Banks 93ff59a132
Fix racy connect network tests that always fail in Docker due to listen races 2018-06-14 09:42:04 -07:00
Paul Banks 9d11cd9bf4
Fix various test failures and vet warnings.
Intention de-duplication in previously merged PR actualy failed some tests that were not caught be me or CI. I ran the test files for state changes but they happened not to trigger this case so I made sure they did first and then fixed. That fixed some upstream intention endpoint tests that I'd not run as part of testing the previous fix.
2018-06-14 09:41:58 -07:00
Paul Banks 51b1bc028d
Rework connect/proxy and command/connect/proxy. End to end demo working again 2018-06-14 09:41:57 -07:00
Paul Banks 67669abf82
Remove old connect client and proxy implementation 2018-06-14 09:41:56 -07:00
Paul Banks 2d6a2ce1e3
connect.Service based implementation after review feedback. 2018-06-14 09:41:56 -07:00
Paul Banks 800deb693c
Original proxy and connect.Client implementation. Working end to end. 2018-06-14 09:41:56 -07:00
Mitchell Hashimoto a360c5cca4
agent/consul: basic sign endpoint not tested yet 2018-06-14 09:41:51 -07:00
Mitchell Hashimoto d4e232f69b
connect: create connect package for helpers 2018-06-14 09:41:50 -07:00