Commit graph

9164 commits

Author SHA1 Message Date
Rebecca Zanzig f18a74e705 Update docs to include multiple tag support (#4797)
* Update docs to include multiple tag support

* Sort tags before using them in metrics

This addresses the potential proliferation of metrics if a query of
"?tag=foo&tag=bar" is treated differently than "?tag=bar&tag=foo".
Now, tags are always sorted before being recorded, making these two
emit the same metric.

* Add caveat about multiple tags returned by the metrics endpoint
2018-10-19 16:52:17 +01:00
John Cowen 52a62f2b8d UI: New ACLs (#4789)
UI to accompany the new ACLs APIs
2018-10-19 08:45:05 -07:00
Hans Hasselberg bd37633df8 Update CHANGELOG.md 2018-10-19 08:45:02 -07:00
Hans Hasselberg d9a530f9ae Update CHANGELOG.md 2018-10-19 08:44:18 -07:00
Hans Hasselberg 36bcea9f33 website: update sprockets and ffi to dodge CVEs (#4781)
* Update sprockets to dodge CVE-2018-3760 and CVE-2014-7819
* update ffi to dodge CVE-2018-1000201
2018-10-19 08:41:04 -07:00
Pierre Souchay a72f92cac6 dns: implements prefix lookups for DNS TTL (#4605)
This will fix https://github.com/hashicorp/consul/issues/4509 and allow forinstance lb-* to match services lb-001 or lb-service-007.
2018-10-19 08:41:04 -07:00
Aestek 2e1015b3ba website: add service weights documentation in API doc (#4776)
Adds weights documentation to the following API routes :
* Agent: List Services
* Agent Register Service
* Health: List Nodes for Service
2018-10-19 08:41:03 -07:00
Jack Pearkes e0e33aee26 website: note Envoy support in Nomad guide (#4787) 2018-10-19 08:41:03 -07:00
Jack Pearkes 197d62c6ca New command: consul debug (#4754)
* agent/debug: add package for debugging, host info

* api: add v1/agent/host endpoint

* agent: add v1/agent/host endpoint

* command/debug: implementation of static capture

* command/debug: tests and only configured targets

* agent/debug: add basic test for host metrics

* command/debug: add methods for dynamic data capture

* api: add debug/pprof endpoints

* command/debug: add pprof

* command/debug: timing, wg, logs to disk

* vendor: add gopsutil/disk

* command/debug: add a usage section

* website: add docs for consul debug

* agent/host: require operator:read

* api/host: improve docs and no retry timing

* command/debug: fail on extra arguments

* command/debug: fixup file permissions to 0644

* command/debug: remove server flags

* command/debug: improve clarity of usage section

* api/debug: add Trace for profiling, fix profile

* command/debug: capture profile and trace at the same time

* command/debug: add index document

* command/debug: use "clusters" in place of members

* command/debug: remove address in output

* command/debug: improve comment on metrics sleep

* command/debug: clarify usage

* agent: always register pprof handlers and protect

This will allow us to avoid a restart of a target agent
for profiling by always registering the pprof handlers.

Given this is a potentially sensitive path, it is protected
with an operator:read ACL and enable debug being
set to true on the target agent. enable_debug still requires
a restart.

If ACLs are disabled, enable_debug is sufficient.

* command/debug: use trace.out instead of .prof

More in line with golang docs.

* agent: fix comment wording

* agent: wrap table driven tests in t.run()
2018-10-19 08:41:03 -07:00
Kyle Havlovitz 96a35f8abc re-add Connect multi-dc config changes
This reverts commit 8bcfbaffb6588b024cd1a3cf0952e6bfa7d9e900.
2018-10-19 08:41:03 -07:00
John Cowen 3ec70b62a3
ui: Fix freetext searching within the nodes page (#4819)
Essentially this was missing a call to `super`. The error unfortuantely
didn't arise in the tests as it only errors when the node list has 4
items are more (the 4 columns), and the acceptence tests by change were
only filling the page with 3 nodes for test purposes.

I've bumped the amount of nodes up to 4 in the tests, which then causes
the tests to fail, made the fix by adding the `super` call, and the
tests now pass.

I also tested the UI/text searching on a 10,000 node system, and
everything now works as expected.
2018-10-19 14:31:22 +01:00
Rebecca Zanzig eb81b49c34
Merge pull request #4812 from hashicorp/docs/helm-16
Add additional formatting detail to the Helm `join` command docs
2018-10-18 13:43:13 -07:00
Rebecca Zanzig e81571d6e9 Add additional formatting detail to the Helm join command
In response to consul-helm issue 16.
2018-10-17 12:53:44 -07:00
R.B. Boyer f15bce9f20
fix some test hangs (#4785)
The default http.Client uses infinite timeouts, so if TestHTTPAPI_MethodNotAllowed_OSS experienced anything going wrong about setup it could hang forever.

Switching to hard coding various http.Client timeouts to non-infinite values at least bounds the failure time.
2018-10-16 16:04:51 -05:00
Jack Pearkes e3b4fafe36 Putting source back into Dev Mode 2018-10-11 13:31:27 -07:00
Jack Pearkes df2efb9658 website: bump website for 1.3.0 2018-10-11 11:27:54 -07:00
banks b4696ee554 Release v1.3.0 2018-10-11 18:21:46 +00:00
Assel Meher f2321679d6 typo: s/checks/services (#4783) 2018-10-11 18:48:51 +01:00
Jack Pearkes 847a0a5266 Revert "Connect multi-dc config" (#4784) 2018-10-11 17:32:45 +01:00
Paul Banks 29cf51812b
Update CHANGELOG.md 2018-10-11 13:35:02 +01:00
danielehc 6c12a35834
Update creating-certificates.html.md (#4780)
In case `verify_server_hostname` is set in the configuration, Consul checks the certificate against  `server.<datacenter>.<domain>`.

The name suggested by the guide generates errors like the following:
```
2018/10/10 12:42:20 [ERR] consul: Failed to confirm peer status for consul-3: rpc error getting client: failed to get conn: x509: certificate is valid for server.node.consul.labs, localhost, not server.consul.labs. Retrying in 16s...
```

Removing the `node` part from the certificate permits them to work also when that option is set.
2018-10-11 14:23:51 +02:00
Aestek 260a9880ae [Security] Add finer control over script checks (#4715)
* Add -enable-local-script-checks options

These options allow for a finer control over when script checks are enabled by
giving the option to only allow them when they are declared from the local
file system.

* Add documentation for the new option

* Nitpick doc wording
2018-10-11 13:22:11 +01:00
Paul Banks bf741cbc04
Quick fix for cache age flakiness in CI 2018-10-11 13:12:19 +01:00
Paul Banks 731e86a851
Update CHANGELOG.md 2018-10-11 12:56:05 +01:00
Rebecca Zanzig 0ec6d880f5 Support multiple tags for health and catalog http api endpoints (#4717)
* Support multiple tags for health and catalog api endpoints

Fixes #1781.

Adds a `ServiceTags` field to the ServiceSpecificRequest to support
multiple tags, updates the filter logic in the catalog store, and
propagates these change through to the health and catalog endpoints.

Note: Leaves `ServiceTag` in the struct, since it is being used as
part of the DNS lookup, which in turn uses the health check.

* Update the api package to support multiple tags

Includes additional tests.

* Update new tests to use the `require` library

* Update HealthConnect check after a bad merge
2018-10-11 12:50:05 +01:00
Pierre Souchay b0fc91a1d2 [Performance On Large clusters] Reduce updates on large services (#4720)
* [Performance On Large clusters] Checks do update services/nodes only when really modified to avoid too many updates on very large clusters

In a large cluster, when having a few thousands of nodes, the anti-entropy
mechanism performs lots of changes (several per seconds) while
there is no real change. This patch wants to improve this in order
to increase Consul scalability when using many blocking requests on
health for instance.

* [Performance for large clusters] Only updates index of service if service is really modified

* [Performance for large clusters] Only updates index of nodes if node is really modified

* Added comments / ensure IsSame() has clear semantics

* Avoid having modified boolean, return nil directly if stutures are Same

* Fixed unstable unit tests TestLeader_ChangeServerID

* Rewrite TestNode_IsSame() for better readability as suggested by @banks

* Rename ServiceNode.IsSame() into IsSameService() + added unit tests

* Do not duplicate TestStructs_ServiceNode_Conversions() and increase test coverage of IsSameService

* Clearer documentation in IsSameService

* Take into account ServiceProxy into ServiceNode.IsSameService()

* Fixed IsSameService() with all new structures
2018-10-11 12:42:39 +01:00
Paul Banks de9af1854b
Bump Go version for CI and build to 1.11.1 (#4782) 2018-10-11 12:21:53 +01:00
Paul Banks e95d78483d
Be more explicit about rolling upgrade process 2018-10-11 11:44:20 +01:00
Paul Banks bf634fafee
Move 1.0.6 upgrade docs that were added in the wrong place 2018-10-11 11:38:55 +01:00
Paul Banks 47dd508c80
f-envoy branch CHANGELOG entries 2018-10-11 11:21:42 +01:00
Paul Banks 94332edf2e
[WIP] Initial draft of Sidecar Service and Managed Proxy deprecation docs (#4752)
* Initial draft of Sidecar Service and Managed Proxy deprecation docs

* Service definition deprecation notices and sidecar service

* gRPC and sidecar service config options; Deprecate managed proxy options

* Envoy Docs: Basic envoy command; envoy getting started/intro

* Remove change that snuck in

* Envoy custom config example

* Add agent/service API docs; deprecate proxy config endpoint

* Misc grep cleanup for managed proxies; capitalize Envoy

* Updates to getting started guide

* Add missing link

* Refactor Envoy guide into a separate guide and add bootstrap reference notes.

* Add limitations to Envoy docs; Highlight no fixes for known managed proxy issues on deprecation page; clarify snake cae stuff; Sidecar Service lifecycle
2018-10-11 10:44:42 +01:00
John Cowen ecd0a06fb0
Update CHANGELOG.md 2018-10-11 10:32:33 +01:00
John Cowen ef891a23b5
ui: Improve layout of node cards on large and small screens (#4761)
1. The grid based unhealthy cards are now clamped to only four wide
maximum. This means that on larger screen the cards are much wider
meaning you can view more information. Grid gutters are also clamped at
a certain ideal width screen, remaining responsive for anything below
this.

2. The healthy node columns are finally responsive following the same
column rules as unhealthy nodes
2018-10-11 10:19:06 +01:00
John Cowen 0dface2010
ui: Add truncation to the .Address (#4760) 2018-10-11 10:17:56 +01:00
John Cowen 34f134caa0
ui: Cope with service names that contain slashes (#4756) 2018-10-11 10:14:04 +01:00
John Cowen 9fe12d2b10
ui: Reduce the breakpoint at which the smaller footer kicks in (#4723) 2018-10-11 10:13:17 +01:00
Matt Keeler 948e4af5b1
Update CHANGELOG.md 2018-10-10 15:52:13 -04:00
Pierre Souchay 42f250fa53 Added SOA configuration for DNS settings. (#4714)
This will allow to fine TUNE SOA settings sent by Consul in DNS responses,
for instance to be able to control negative ttl.

Will fix: https://github.com/hashicorp/consul/issues/4713

# Example

Override all settings:

* min_ttl: 0 => 60s
* retry: 600 (10m) => 300s (5 minutes),
* expire: 86400 (24h) => 43200 (12h)
* refresh: 3600 (1h) => 1800 (30 minutes)

```
consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}'
```

Result:
```
dig +multiline @localhost -p 8600 service.consul

; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service.consul.		IN A

;; AUTHORITY SECTION:
consul.			0 IN SOA ns.consul. hostmaster.consul. (
				1537959133 ; serial
				1800       ; refresh (30 minutes)
				300        ; retry (5 minutes)
				43200      ; expire (12 hours)
				60         ; minimum (1 minute)
				)

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed Sep 26 12:52:13 CEST 2018
;; MSG SIZE  rcvd: 93
```
2018-10-10 15:50:56 -04:00
Kyle Havlovitz c22ee028c1
Merge pull request #4687 from hashicorp/connect-multidc-config
Connect multi-dc config
2018-10-10 12:33:00 -07:00
Kyle Havlovitz 0cbd176a48 connect/ca: more OSS split for multi-dc 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 6d5160c139 connect/ca: split CA initialization logic between oss/enterprise 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 5b98a602af agent: add primary_datacenter and connect replication config options 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 304595f7a6 connect: add ExternalTrustDomain to CARoot fields 2018-10-10 12:16:47 -07:00
Kyle Havlovitz 475afd0300 docs: deprecate acl_datacenter and replace it with primary_datacenter 2018-10-10 12:16:47 -07:00
Paul Banks 0523efa2fe merge feedback: fix typos; actually use deliverLatest added previously but not plumbed in 2018-10-10 16:55:34 +01:00
Paul Banks bcf2cc2de3 cli: envoy command default gRPC port (#4768)
* Default gRPC port; Start on some basic tests for argument and ENV handling; Make Exec test less platform-dependent.

* Allow hot-restarts

* Remove debug
2018-10-10 16:55:34 +01:00
R.B. Boyer 4427417140 cli: avoid passing envoy bootstrap configuration as arguments (#4747)
Play a trick with CLOEXEC to pass the envoy bootstrap configuration as
an open file descriptor to the exec'd envoy process. The file only
briefly touches disk before being unlinked.

We convince envoy to read from this open file descriptor by using the
/dev/fd/$FDNUMBER mechanism to read the open file descriptor as a file.

Because the filename no longer has an extension envoy's sniffing logic
falls back on JSON instead of YAML, so the bootstrap configuration must
be generated as JSON instead.
2018-10-10 16:55:34 +01:00
Paul Banks 7a8023a57f Fix up tests broken by master merge; add proxy tests to services command (and fix it!); actually run the proxycfg.Manager 2018-10-10 16:55:34 +01:00
Paul Banks c5d5dbaf96 Fix bug in leaf-cert cache type where multiple client tokens collide (#4736)
* Fix bug in leaf-cert cache type where multiple clients with different tokens would share certs and block incorrectly

* Use hash for issued certs key to avoid ambiguity concatenating
2018-10-10 16:55:34 +01:00
Paul Banks 1e4c5a1811 Connect Envoy Command (#4735)
* Plumb xDS server and proxyxfg into the agent startup

* Add `consul connect envoy` command to allow running Envoy as a connect sidecar.

* Add test for help tabs; typos and style fixups from review
2018-10-10 16:55:34 +01:00