Commit Graph

8980 Commits

Author SHA1 Message Date
Paul Banks 09e4c2995b
Fix CA pruning when CA config uses string durations. (#4669)
* Fix CA pruning when CA config uses string durations.

The tl;dr here is:

 - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported
 - Most of our tests managed to escape this by defining them as time.Duration directly
 - Out actual default value is a string
 - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC).
 - msgpack decode leaves the string as a `[]uint8`
 - Some of our parsers required string and failed
 - So after 1 hour, a default configured server would throw an error about pruning old CAs
 - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA.
 - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding.

tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D

We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them.

This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time!

* Typo fix

* Fix bad Uint8 usage
2018-09-13 15:43:00 +01:00
Mitchell Hashimoto 62ceacde67 website: document k8s go-discover (#4666)
This adds documentation for the `k8s` go-discover provider that will be part of 1.2.3.
2018-09-13 10:12:27 -04:00
Hans Hasselberg ef6ab91482
Update CHANGELOG.md 2018-09-13 16:09:08 +02:00
Hans Hasselberg 318bcb9bbb
Allow disabling the HTTP API again. (#4655)
If you provide an invalid HTTP configuration consul will still start again instead of failing. But if you do so the build-in proxy won't be able to start which you might need for connect.
2018-09-13 16:06:04 +02:00
John Cowen 34aced5f3c
Update CHANGELOG.md 2018-09-13 09:45:41 +01:00
John Cowen 7f8997a68d
ui: Test bugfix. Specifically set deny for intention creation (#4663)
Make sure we speficially set and test for deny on testing intention
creation
2018-09-13 09:10:18 +01:00
John Cowen ca2d993d64
ui: Tests Bugfix. Reflect extra json property `ExternalSources` in mocks (#4662)
The mocks where using randomly generated `ExternalSources` this change
makes sure they are fixed so we can reliably test the values. No change
to actual UI code
2018-09-13 09:09:59 +01:00
John Cowen dc03d5a4b9
UI: Set the CODE view as the default view for editing KV's (#4651)
Sets the code toggle on the KV edit/create page to be on by default, we figured most people probably prefer this view.

Also, previously we forced the KV toggle back to a default setting for every
time you visited a KV form page. We've now changed this so that the KV code
toggle button acts as a 'global' toggle. So whatever you set it as will
be the same for every KV for the lifetime of your 'ember session'

If we are to keep this, then consider saving this into localStorage
settings or similar, added some thoughts in comments re: this as it's very likely
to happen.
2018-09-13 09:09:30 +01:00
Rebecca Zanzig a98db43c41
Merge pull request #4665 from hashicorp/docs/gh-4616
Update required golang version in Readme
2018-09-12 15:19:58 -07:00
Rebecca Zanzig 468e7e8980 Update required golang version in Readme
Fixes #4616.
2018-09-12 14:44:07 -07:00
Kyle Havlovitz 70c43a27c3 connect/ca: hash the consul provider ID and include isRoot 2018-09-12 13:44:15 -07:00
John Cowen 008c08b69c
ui: [BUGFIX] Intentions were showing the wrong notification on creation (#4658)
The error notification was being shown on creation of an intention. This
was as a result of #4572 and/or #4572 and has not been included in a
release.

This includes a fix, plus tests to try to prevent any further regression.
2018-09-12 20:41:43 +01:00
John Cowen ab568f6b94
ui: Adds a `default` view helper for providing a default value (#4650)
If the first value passed to the helper is an empty string or undefined
then return the second value
2018-09-12 20:38:57 +01:00
John Cowen b279f23372
UI: External Source markers (#4640)
1. Addition of external source icons for services marked as such.
2. New %with-tooltip css component (wip)
3. New 'no healthcheck' icon as external sources might not have
healthchecks, also minus icon on node cards in the service detail view
4. If a service doesn't have healthchecks, we use the [Services] tabs as the
default instead of the [Health Checks] tab in the Service detail page. 
5. `css-var` helper. The idea here is that it will eventually be
replaced with pure css custom properties instead of having to use JS. It
would be nice to be able to build the css variables into the JS at build
time (you'd probably still want to specify in config which variables you
wanted available in JS), but that's possible future work.

Lastly there is probably a tiny bit more testing edits here than usual,
I noticed that there was an area where the dynamic mocking wasn't
happening, it was just using the mocks from consul-api-double, the mocks
I was 'dynamically' setting happened to be the same as the ones in
consul-api-double. I've fixed this here also but it wasn't effecting
anything until actually made certain values dynamic.
2018-09-12 20:23:39 +01:00
John Cowen b1d83f98b0
UI: Bugfix. Remove split view code editor (#4615)
When adding an auto resizing (heightwise) code editor, the
ivy-codemirror plugin seems to do this using more nested divs. This div
had a horizontal scroller but couldn't be seen on some platforms (with
hidden scrollbars). This commit makes the code editor slightly more
usable and more visually correct by removing the scroll bar in this div
to stop producing the 'split view look', yet keeping the horizontal
scroller at the bottom of the code editor for when you enter code that
is wider than the area. A max-width has also been added here to prevent
the text area from growing off the side of the page.

Another improvement to the code editor here is the addition of a nicer
color for hightlighting text selection so its at least visible.

Lastly, there was a way you could get the bottom horizontal scrollbar to overlay
the code in the editor. This makes sure there is always some space at
the bottom of the editor to make sure the code won't be obscured
2018-09-12 20:18:12 +01:00
John Cowen d9764ed04b
UI: Bugfix. Move to a different TextEncoder/Decoder (#4613)
1. The previously used TextEncoder/Decoder (used as a polyfill for
browsers that don't have a native version) didn't expose an encoder via
CommonJS. Use a different polyfill that exposes both a decoder and an
encoder.
2. The feature detection itself was flawed. This does a less error prone
detection that ensures native encoding/decoding where available and polyfilled
encoding/decoding where not available.
2018-09-12 20:15:58 +01:00
Jack Pearkes fcdea1ffbb
Update CHANGELOG.md 2018-09-12 09:57:03 -07:00
Benjamin Sago 9aa00d45b6 Exit with error code 1 when failing to list DCs (#4583)
Fixes #4582.
2018-09-12 09:55:02 -07:00
Mitchell Hashimoto b69342f0c1
Initial Helm Chart/K8S Docs (#4653)
* website: initial Kubernetes section with Helm information

* website: extraConfig for clients

* website: add more helm fields

* website: document extraVolumes

* website: document Consul DNS

* website: fix typos and show example of downward API
2018-09-12 08:44:30 -07:00
Pierre Souchay 5ecf9823d2 Fix more unstable tests in agent and command 2018-09-12 14:49:27 +01:00
Kyle Havlovitz 8fc2c77fdf
connect/ca: some cleanup and reorganizing of the new methods 2018-09-11 16:43:04 -07:00
Paul Banks 175c9e7250
Update CHANGELOG.md 2018-09-11 17:35:59 +01:00
Pierre Souchay 508b67c32a Ensure that Proxies ARE always cleaned up, event with DeregisterCriticalServiceAfter (#4649)
This fixes https://github.com/hashicorp/consul/issues/4648
2018-09-11 17:34:09 +01:00
Freddy 7a19f2a6da
Update snapshot agent docs to include s3-endpoint (#4652) 2018-09-11 16:32:31 +01:00
Matt Keeler 0eefa57ee8
Update CHANGELOG.md 2018-09-11 10:48:12 -04:00
Matt Keeler 2fb972948c
Update CHANGELOG.md 2018-09-11 10:42:55 -04:00
Paul Banks 759380877e
Update CHANGELOG.md 2018-09-11 15:34:24 +01:00
Matt Keeler cccfbbd71a
Update CHANGELOG.md 2018-09-11 09:40:17 -04:00
Matt Keeler 19d71c6eb4
Add ECS option to EDNS responses where appropriate (#4647)
This implements parts of RFC 7871 where Consul is acting as an authoritative name server (or forwarding resolver when recursors are configured)

If ECS opt is present in the request we will mirror it back and return a response with a scope of 0 (global) or with the same prefix length as the request (indicating its valid specifically for that subnet).

We only mirror the prefix-length (non-global) for prepared queries as those could potentially use nearness checks that could be affected by the subnet. In the future we could get more sophisticated with determining the scope bits and allow for better caching of prepared queries that don’t rely on nearness checks.

The other thing this does not do is implement the part of the ECS RFC related to originating ECS headers when acting as a intermediate DNS server (forwarding resolver). That would take a quite a bit more effort and in general provide very little value. Consul will currently forward the ECS headers between recursors and the clients transparently, we just don't originate them for non-ECS clients to get potentially more accurate "location aware" results.
2018-09-11 09:37:46 -04:00
Mitchell Hashimoto cc9aa2ab38
Merge pull request #4646 from hashicorp/b-fix-build
Detect correct GOOS/ARCH for copying binary with build_consul_local
2018-09-10 13:42:20 -07:00
Mitchell Hashimoto f01fc96161
Detect correct GOOS/ARCH for copying binary with build_consul_local
If GOOS/ARCH is set to something custom, we need to unset it before
testing the value so we can compare to the _original value_.
2018-09-10 10:25:15 -07:00
Pierre Souchay 7a42c31330 Fix unstable tests in agent, api, and command/watch 2018-09-10 16:58:53 +01:00
Freddy 93aaf00b6b
Add script and makefile goal to help debug flaky tests 2018-09-10 16:44:07 +01:00
Mitchell Hashimoto c491125ff5
Merge pull request #4642 from hashicorp/f-ui-meta
agent: aggregate service instance meta for UI purposes
2018-09-07 17:36:23 -07:00
Mitchell Hashimoto 553800ed58
agent: ExternalSources instead of Meta 2018-09-07 10:06:55 -07:00
Matt Keeler dcaf6916fd
Update CHANGELOG.md 2018-09-07 10:59:15 -04:00
Matt Keeler 62c631368d
Connect: Verify the leaf cert to determine its readiness. (#4540)
This improves the checking so that if a certificate were to expire or the roots changed then we will go into a non-ready state.

This parses the x509 certificates from the TLS certificate when the leaf is set. The readyCh will be closed whenever a parseable certificate is set and the ca roots are set. This does not mean that the certificate is valid but that it has been setup and is generally valid. The Ready function will now do x509 certificate verification which will in addition to verifying the signatures with the installed CA roots will also verify the certificate isn't expired or not set to become valid in the future. 

The correct way to use these functions is to wait for the ReadyWait chan to be closed and then periodically check the readiness to determine if the certificate is currently useable.
2018-09-07 10:58:06 -04:00
Matt Keeler 1f047b886f
Update CHANGELOG.md 2018-09-07 10:49:54 -04:00
Matt Keeler 61a5c965c9
Ensure that errors setting up the DNS servers get propagated back to the shell (#4598)
Fixes: #4578 

Prior to this fix if there was an error binding to ports for the DNS servers the error would be swallowed by the gated log writer and never output. This fix propagates the DNS server errors back to the shell with a multierror.
2018-09-07 10:48:29 -04:00
Paul Banks ba2e2d4af3
Update CHANGELOG.md 2018-09-07 15:32:42 +01:00
Pierre Souchay 473e589d86 Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Kyle Havlovitz e184a18e4b
connect/ca: add Configure/GenerateRoot to provider interface 2018-09-06 19:18:59 -07:00
Mitchell Hashimoto 568c0d105a
agent: aggregate service instance meta for UI purposes 2018-09-06 12:19:05 -07:00
Matt Keeler 7392157990
Update CHANGELOG.md 2018-09-06 15:08:51 -04:00
Matt Keeler ba4f912b25
Update Raft Vendoring (#4539)
Pulls in a fix for a potential memory leak regarding consistent reads that invoke VerifyLeader.
2018-09-06 15:07:42 -04:00
Mitchell Hashimoto 8a16fa95fd
Update CHANGELOG 2018-09-06 12:02:38 -07:00
Mitchell Hashimoto 5111a3e6c9
Merge pull request #4635 from hashicorp/f-discover-k8s
Update go-discover, add k8s auto-join
2018-09-06 12:01:14 -07:00
Paul Banks 18624befd6
Enable CI on f-envoy feature branch 2018-09-06 15:41:29 +01:00
Mitchell Hashimoto f7a95e1a28
vendor k8s client lib 2018-09-05 14:59:02 -07:00
Mitchell Hashimoto ca56e13d31
agent: configure k8s go-discover 2018-09-05 13:38:13 -07:00