Commit Graph

12086 Commits

Author SHA1 Message Date
Blake Covarrubias a088d12a20 docs: Fix broken link to Nomad Consul Connect guide 2020-04-08 14:59:36 -07:00
Freddy c1f79c6b3c
Terminating gateway discovery (#7571)
* Enable discovering terminating gateways

* Add TerminatingGatewayServices to state store

* Use GatewayServices RPC endpoint for ingress/terminating
2020-04-08 12:37:24 -06:00
Freddy f5b9688336
Add decode rules for Expose cfg in service-defaults (#7611) 2020-04-07 19:37:47 -06:00
Daniel Nephin 575ad5c39f Fix CONSUL_HTTP_ADDR=https not enabling TLS
Use the config instead of attempting to reparse the env var.
2020-04-07 18:16:53 -04:00
Daniel Nephin 97c9f73261 Step 3: fix a bug in api.NewClient and fix the tests
The api client should never rever to HTTP if the user explicitly
requested TLS. This change broke some tests because the tests always use
an non-TLS http server, but some tests explicitly enable TLS.
2020-04-07 18:02:56 -04:00
Iryna Shustava 1b5cff80f2
docs: Add Helm docs for auto-encrypt and external servers (#7595)
* docs: Add Helm docs for auto-encrypt and external servers
2020-04-07 14:41:16 -07:00
Luke Kysow 812aedce48
Merge pull request #7586 from hashicorp/helm-docs
Document bootstrapACLs deprecation
2020-04-07 14:02:12 -07:00
Daniel Nephin ae42dea2d5 Step 2: extract the grpc address logic and a new type
The new grpcAddress function contains all of the logic to translate the
command line options into the values used in the template.

The new type has two advantages.

1. It introduces a logical grouping of values in the BootstrapTplArgs
   struct which is exceptionally large. This grouping makes the struct
   easier to understand because each set of nested values can be seen
   as a single entity.
2. It gives us a reasonable return value for this new function.
2020-04-07 16:36:51 -04:00
Daniel Nephin 5092aaf9b8 Step 1: move all the grpcAddr logic into the same spot
There is no reason a reader should have to jump around to find this value. It is only
used in 1 place
2020-04-07 15:53:12 -04:00
Alvin Huang c5f5cfddf6
filter out non go branches from the 'go-tests' workflow (#7606) 2020-04-07 15:39:23 -04:00
Matt Keeler 5851ea4f64
Update CHANGELOG.md 2020-04-07 11:49:36 -04:00
Matt Keeler 42f02e80c3
Enable filtering language support for the v1/connect/intentions… (#7593)
* Enable filtering language support for the v1/connect/intentions listing API

* Update website for filtering of Intentions

* Update website/source/api/connect/intentions.html.md
2020-04-07 11:48:44 -04:00
Daniel Nephin 72e2695986
Merge pull request #7598 from pierresouchay/preallocation_of_dns_meta
Pre-allocations of DNS meta to avoid several allocations
2020-04-06 14:00:32 -04:00
Luke Kysow 1cef40b6a4
Update website/source/docs/platform/k8s/helm.html.md
Co-Authored-By: Iryna Shustava <ishustava@users.noreply.github.com>
2020-04-06 09:16:49 -07:00
Pierre Souchay a7fbf003c1
[LINT] Close resp.Body to avoid linter complaining (#7600) 2020-04-06 09:11:04 -04:00
Pierre Souchay 91b3510821 Pre-allocations of DNS meta to avoid several allocations 2020-04-05 11:12:41 +02:00
Jono Sosulska ab14c969cd
Change style to match "join" singular (#7569)
* Change style to match "join" singular

- Replaced "(Consul) cluster" with  "Consul Datacenter"
- Removed "ing" so the feature fits "Consul Auto-join", and that the tense is correct.

Co-authored-by: danielehc <40759828+danielehc@users.noreply.github.com>
2020-04-03 16:04:07 -04:00
Daniel Nephin ab068325da agent/cache: move typeEntry lookup to the edge
This change moves all the typeEntry lookups to the first step in the exported methods,
and makes unexporter internals accept the typeEntry struct.

This change is primarily intended to make it easier to extract the container of caches
from the Cache type.

It may incidentally reduce locking in fetch, but that was not a goal.
2020-04-03 16:01:56 -04:00
David Yu 33b6bfd106
[docs] Built-in Proxies not meant for production (#7579)
* [docs] Built-in Proxies not meant for production

* Adding link to Envoy for Connect

* Update website/source/docs/connect/proxies/built-in.md

Co-Authored-By: Blake Covarrubias <blake@covarrubi.as>

* Revising note

* Update website/source/docs/connect/proxies/built-in.md

period

Co-Authored-By: Hans Hasselberg <me@hans.io>

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
Co-authored-by: Hans Hasselberg <me@hans.io>
2020-04-03 11:52:05 -07:00
Daniel Nephin 08e113ab20
Merge pull request #7589 from pierresouchay/fix_unstable_test_TestForwardSignals
Fixed unstable test TestForwardSignals()
2020-04-03 12:43:28 -04:00
Pierre Souchay 5f9f86a327 Fixed unstable test TestForwardSignals()
Sometimes, in the CI, it could receive a SIGURG, producing this line:

  FAIL: TestForwardSignals/signal-interrupt (0.06s)
        util_test.go:286: expected to read line "signal: interrupt" but got "signal: urgent I/O condition"

Only forward the signals we test to avoid this kind of false positive

Example of such unstable errors in CI:
https://circleci.com/gh/hashicorp/consul/153571
2020-04-03 14:23:03 +02:00
Pierre Souchay 984583d980
tests: more tolerance to latency for unstable test `TestCacheNotifyPolling()`. (#7574) 2020-04-03 10:29:38 +02:00
Luke Kysow b87d92147d
Document bootstrapACLs deprecation 2020-04-02 16:58:55 -07:00
Freddy f5eb6ab539
Fix regression with gateway registration and update docs (#7582) 2020-04-02 12:52:11 -06:00
Matt Keeler 5d0e661203
Ensure that token clone copies the roles (#7577) 2020-04-02 12:09:35 -04:00
Chris Piraino 24602931bc
Update CHANGELOG.md 2020-04-02 10:14:17 -05:00
Chris Piraino d7a870fd32
Fix flapping of mesh gateway connect-service watches (#7575) 2020-04-02 10:12:13 -05:00
Pierre Souchay 2b8da952a8
agent: show warning when enable_script_checks is enabled without safty net (#7437)
In order to enforce a bit security on Consul agents, add a new method in agent
to highlight possible security issues.

This does not return an error for now, but might in the future.

For now, it detects issues such as:

https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/

This would display this kind of messages:

```
2020-03-11T18:27:49.873+0100 [ERROR] agent: [SECURITY] issue: error="using enable-script-checks without ACLs and without allow_write_http_from is DANGEROUS, use enable-local-script-checks instead see https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/"
```
2020-04-02 09:59:23 +02:00
Andy Lindeman 0d1d5d0863
agent: rewrite checks with proxy address, not local service address (#7518)
Exposing checks is supposed to allow a Consul agent bound to a different
IP address (e.g., in a different Kubernetes pod) to access healthchecks
through the proxy while the underlying service binds to localhost. This
is an important security feature that makes sure no external traffic
reaches the service except through the proxy.

However, as far as I can tell, this is subtly broken in the case where
the Consul agent cannot reach the proxy over localhost.

If a proxy is configured with: `{ LocalServiceAddress: "127.0.0.1",
Checks: true }`, as is typical with a sidecar proxy, the Consul checks
are currently rewritten to `127.0.0.1:<random port>`. A Consul agent
that does not share the loopback address cannot reach this address. Just
to make sure I was not misunderstanding, I tried configuring the proxy
with `{ LocalServiceAddress: "<pod ip>", Checks: true }`. In this case,
while the checks are rewritten as expected and the agent can reach the
dynamic port, the proxy can no longer reach its backend because the
traffic is no longer on the loopback interface.

I think rewriting the checks to use `proxy.Address`, the proxy's own
address, is more correct in this case. That is the IP where the proxy
can be reached, both by other proxies and by a Consul agent running on
a different IP. The local service address should continue to use
`127.0.0.1` in most cases.
2020-04-02 09:35:43 +02:00
Andy Lindeman 42224fe45c
proxycfg: support path exposed with non-HTTP2 protocol (#7510)
If a proxied service is a gRPC or HTTP2 service, but a path is exposed
using the HTTP1 or TCP protocol, Envoy should not be configured with
`http2ProtocolOptions` for the cluster backing the path.

A situation where this comes up is a gRPC service whose healthcheck or
metrics route (e.g. for Prometheus) is an HTTP1 service running on
a different port. Previously, if these were exposed either using
`Expose: { Checks: true }` or `Expose: { Paths: ... }`, Envoy would
still be configured to communicate with the path over HTTP2, which would
not work properly.
2020-04-02 09:35:04 +02:00
Pierre Souchay 3b5e72913e
config: validate system limits against limits.http_max_conns_per_client (#7434)
I spent some time today on my local Mac to figure out why Consul 1.6.3+
was not accepting limits.http_max_conns_per_client.

This adds an explicit check on number of file descriptors to be sure
it might work (this is no guarantee as if many clients are reaching
the agent, it might consume even more file descriptors)

Anyway, many users are fighting with RLIMIT_NOFILE, having a clear
message would allow them to figure out what to fix.

Example of message (reload or start):

```
2020-03-11T16:38:37.062+0100 [ERROR] agent: Error starting agent: error="system allows a max of 512 file descriptors, but limits.http_max_conns_per_client: 8192 needs at least 8212"
```
2020-04-02 09:22:17 +02:00
Daniel Nephin d447a71a36
Merge pull request #7572 from hashicorp/dnephin/ci-fix-go-mod-download
ci: Fix working_directory for go mod download
2020-04-01 18:57:21 -04:00
Daniel Nephin 4abb00cd17 ci: Fix working_directory for go mod download
The previous PR which added these was accidentally performing the download
in the root directory. For the api, and sdk directories it should be in done
in the same directory that will be used to run tests. Otherwise the
wrong dependencies will be downloaded which may add unnecessary time to
the CI run.
2020-04-01 17:02:23 -04:00
Shaker Islam d8ac493395
docs: document exported functions in agent.go (closes #7101) (#7366)
and fix one linter error
2020-04-01 22:52:23 +02:00
R.B. Boyer 908099b868 update changelog 2020-04-01 13:16:01 -05:00
Pierre Souchay fa1f9eb144
[BUGFIX] Fix race condition in freeport (#7567)
This removes a race condition in reset since pendingPorts can be set to nil in reset()

If ticker is hit at wrong time, it would crash the unit test.

We ensure in reset to avoid this race condition by cancelling the goroutine using
killTicker chan.

We also properly clean up eveything, so garbage collector can work as expected.

To reproduce existing bug:
`while go test -timeout 30s github.com/hashicorp/consul/sdk/freeport -run '^(Test.*)$'; do go clean -testcache; done`

Will crash after a few 10s runs on my machine.

Error could be seen in unit tests sometimes:

[INFO] freeport: resetting the freeport package state
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1125536]

goroutine 25 [running]:
container/list.(*List).Len(...)
	/usr/local/Cellar/go/1.14/libexec/src/container/list/list.go:66
github.com/hashicorp/consul/sdk/freeport.checkFreedPortsOnce()
	/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:157 +0x86
github.com/hashicorp/consul/sdk/freeport.checkFreedPorts()
	/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:147 +0x71
created by github.com/hashicorp/consul/sdk/freeport.initialize
	/Users/p.souchay/go/src/github.com/hashicorp/consul/sdk/freeport/freeport.go:113 +0x2cf
FAIL	github.com/hashicorp/consul/sdk/freeport	1.607s
2020-04-01 13:14:33 -05:00
Daniel Nephin 660102bf7d
Merge pull request #7570 from pierresouchay/fix_lint_upstream_due_to_change_args_in_NewTestAgent
[FIX BUILD] fix build due to merge of #7562
2020-04-01 12:39:34 -04:00
Pierre Souchay 96d7229bd9 [FIX BUILD] fix build due to merge of #7562
Due to merge #7562, upstream does not compile anymore.

Error is:

ERRO Running error: gofmt: analysis skipped: errors in package: [/Users/p.souchay/go/src/github.com/hashicorp/consul/agent/config_endpoint_test.go:188:33: too many arguments]
2020-04-01 18:29:45 +02:00
Daniel Nephin bc02b8fbe6
Merge pull request #7562 from hashicorp/dnephin/remove-tname-from-name
testing: Remove old default value from NewTestAgent() calls
2020-04-01 11:48:45 -04:00
Daniel Nephin b490790f60
Merge pull request #7427 from hashicorp/dnephin/website-fix-errrors-in-upgrade-docs
docs: fix some errors in upgrade-specific
2020-04-01 11:36:53 -04:00
Daniel Nephin b099bb4b1a
Merge pull request #7465 from hashicorp/dnephin/correct-docs-log-file
docs: Update config reference for log-file
2020-04-01 11:29:20 -04:00
Daniel Nephin ab19e43b61
Merge pull request #7519 from hashicorp/dnephin/help-to-stdout
cli: send requested help text to stdout
2020-04-01 11:26:12 -04:00
Daniel Nephin 8d7c21b255
Merge pull request #7533 from hashicorp/dnephin/xds-server-1
agent/xds: small cleanup
2020-04-01 11:24:50 -04:00
Hans Hasselberg f6de4a0112
docs: revert "docs: fix param for register checks id->checkid (#7393)" (#7565)
This reverts commit ed5202943e43d054e31e958ff0178473974e4015.
2020-04-01 11:49:20 +02:00
Rasmus Mikkelsen 239b60b2ff
docs: fix param for register checks id->checkid (#7393) 2020-04-01 10:59:28 +02:00
John Cowen d598a98cbf
ui: Fix token duplication bug (#7552)
We need to detect whether an object is an ember-data snapshot or just a
plain object, and we where restricted from using `instanceof` due to
ember-data's `Snapshot` class being private.

We'd chosen to go with `constructor.name` instead, which seemed to work,
but when the javascript gets minifed the name of the contructor is also
minified and therefore is not what you are expecting.

This commit reverts to our original idea of checking for the existence
and type of the `.attributes` function, which is the function we require
within the conditional, and therefore is more reliable (if the function
doesn't exist it will error out during development aswell as production)
2020-04-01 09:55:20 +01:00
Emre Savcı 7a99f29adc
agent: add len, cap while initializing arrays 2020-04-01 10:54:51 +02:00
Pierre Souchay f28b18bc5b
tests: fixed unstable test TestAPI_AgentMonitor (#7561)
Similar fix as https://github.com/hashicorp/consul/pull/7541
2020-04-01 09:47:57 +02:00
Daniel Nephin 09c6ac8b92 Rename NewTestAgentWithFields to StartTestAgent
This function now only starts the agent.

Using:

git grep -l 'StartTestAgent(t, true,' | \
        xargs sed -i -e 's/StartTestAgent(t, true,/StartTestAgent(t,/g'
2020-03-31 17:14:55 -04:00
Daniel Nephin d623dcbd01 Convert the remaining calls to NewTestAgentWithFields
After removing the t.Name() parameter with sed, convert the last few tests which
use a custom name to call NewTestAgentWithFields instead.
2020-03-31 17:14:55 -04:00