Commit Graph

779 Commits

Author SHA1 Message Date
Dhia Ayachi 00f7e0772a
debug: remove the CLI check for debug_enabled (#10273)
* debug: remove the CLI check for debug_enabled

The API allows collecting profiles even debug_enabled=false as long as
ACLs are enabled. Remove this check from the CLI so that users do not
need to set debug_enabled=true for no reason.

Also:
- fix the API client to return errors on non-200 status codes for debug
  endpoints
- improve the failure messages when pprof data can not be collected

Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com>

* remove parallel test runs

parallel runs create a race condition that fail the debug tests

* Add changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-05-27 09:41:53 -04:00
Mike Morris da73899a10 changelog: add entry for network areas WAN config fix 2021-05-26 17:49:19 -04:00
Freddy c61e2bbda7
Ensure passthrough clusters can be created (#10301) 2021-05-26 15:05:14 -06:00
Freddy 7cfd7e9ec1
Avoid adding original_dst filter when not needed (#10302) 2021-05-26 15:04:45 -06:00
John Cowen f77615b872
ui: Unix Domain Socket support (#10287)
This commit adds UI support for Unix Domain Sockets for upstream and downstreams (see #9981 and #10252)
2021-05-26 17:52:25 +01:00
John Cowen 7083c39b96
ui: Support Route optional parameters/segments (#10212)
Moves our URLs with 'optional namespace segment' into a separately abstracted 'optional URL segment' feature
2021-05-26 17:43:46 +01:00
Rémi Lapeyre 5e33428b22
Always set the Content-Type header when a body is present (#10204)
* Always set the Content-Type header when a body is present

Closes https://github.com/hashicorp/consul/issues/10011

* Add Changelog entry

* Add more Content-Type exceptions

* Fix tests
2021-05-25 16:03:48 +01:00
Kenia e78e9de396
ui: Create and use collapsible notices component (#10270)
* Create and use collapsible notices

* Refactor collapsible-notices

* Split up the topology acceptance tests

* Add acceptance tests for tproxy notices

* Add component file

* Adds additional TProxy notices tests

* Adds conditional to only show collapsable if more than 2 notices are present

* Adds changelog

* Refactorting the conditonal for collapsing the notices

* Renaming undefinedIntention to be notDefinedIntention

* Refactor tests
2021-05-25 11:02:38 -04:00
Matt Keeler 7e4ea16149 Move some things around to allow for license updating via config reload
The bulk of this commit is moving the LeaderRoutineManager from the agent/consul package into its own package: lib/gort. It also got a renaming and its Start method now requires a context. Requiring that context required updating a whole bunch of other places in the code.
2021-05-25 09:57:50 -04:00
Matt Keeler 58b934133d hcs-1936: Prepare for adding license auto-retrieval to auto-config in enterprise 2021-05-24 13:20:30 -04:00
Daniel Nephin 21f35ab863
Merge pull request #10272 from hashicorp/dnephin/backport-namespace-license-fix
Backport some ent changes for serf tags
2021-05-21 12:31:34 -04:00
Matt Keeler 64359617b5 Deprecate API driven licensing.
The two methods in the API client to Put or Reset a license will now always return an error.
2021-05-21 11:08:50 -04:00
Matt Keeler 84c6c56578 Add OSS bits for supporting specifying the enterprise license via config 2021-05-20 16:11:33 -04:00
Daniel Nephin 4ac5cdd360 Add changelog 2021-05-20 12:57:15 -04:00
John Cowen f86218c39e
ui: Miscellaneous Lock Session fixes (#10225) 2021-05-19 11:05:54 +01:00
Daniel Nephin 57981acc51 add changelog 2021-05-18 15:04:12 -04:00
R.B. Boyer 7c9763d027
xds: emit a labeled gauge of connected xDS streams by version (#10243)
Fixes #10099
2021-05-14 13:59:13 -05:00
R.B. Boyer b90877b440
server: ensure that central service config flattening properly resets the state each time (#10239)
The prior solution to call reply.Reset() aged poorly since newer fields
were added to the reply, but not added to Reset() leading serial
blocking query loops on the server to blend replies.

This could manifest as a service-defaults protocol change from
default=>http not reverting back to default after the config entry
reponsible was deleted.
2021-05-14 10:21:44 -05:00
R.B. Boyer c42899eafa
agent: ensure we hash the non-deprecated upstream fields on ServiceConfigRequest (#10240) 2021-05-14 10:15:48 -05:00
Freddy 98819d9f49
Add changelog entry for network area timeout updates (#10241) 2021-05-13 15:05:38 -06:00
John Cowen 8b0314f26f
ui: Serf Health Check warning notice (#10194)
When the Consul serf health check is failing, this means that the health checks registered with the agent may no longer be correct. Therefore we show a notice to the user when we detect that the serf health check is failing both for the health check listing for nodes and for service instances.

There were a few little things we fixed up whilst we were here:

- We use our @replace decorator to replace an empty Type with serf in the model.
- We noticed that ServiceTags can be null, so we replace that with an empty array.
- We added docs for both our Notice component and the Consul::HealthCheck::List component. Notice now defaults to @type=info.
2021-05-13 11:36:51 +01:00
Iryna Shustava 7a41dbd9b6
Save exposed ports in agent's store and expose them via API (#10173)
* Save exposed HTTP or GRPC ports to the agent's store
* Add those the health checks API so we can retrieve them from the API
* Change redirect-traffic command to also exclude those ports from inbound traffic redirection when expose.checks is set to true.
2021-05-12 13:51:39 -07:00
R.B. Boyer 05b52a3d63
connect: update supported envoy versions to 1.18.3, 1.17.3, 1.16.4, and 1.15.5 (#10231) 2021-05-12 14:06:06 -05:00
Kenia 7103f9f44e
ui: Add conditionals to Lock Session list items (#10121)
* Add conditionals to Lock Session list items

* Add changelog

* Show ID in details if there is a name to go in title

* Add copy-button if ID is in the title

* Update TTL conditional

* Update .changelog/10121.txt

Co-authored-by: John Cowen <johncowen@users.noreply.github.com>

Co-authored-by: John Cowen <johncowen@users.noreply.github.com>
2021-05-11 11:35:15 -04:00
Daniel Nephin f63f1db498
Merge pull request #10188 from hashicorp/dnephin/dont-persist-agent-tokens
agent/local: do not persist the agent or user token
2021-05-10 15:58:20 -04:00
John Cowen 4fc8f93f92
ui: Fix nspace spacing issue (#10157)
This fixes the spacing bug in nspaces only by only showing Description if the namespace has one, and removing the extra 2 pixel margin of dds for when dts aren't rendered/don't exist.
2021-05-10 11:19:11 +01:00
John Cowen 0fd17a3e2b
ui: Adds ability to show a 'partial' list in list-collections (#10174)
* ui: Add support for showing partial lists in ListCollection

* Add CSS for partial 'View more' button, and move all CSS to /components

* Enable partial view for intention permissions
2021-05-07 16:54:45 +01:00
John Cowen b574093cf1
ui: Loader amends/improvements (#10181)
* ui: Loader amends/improvements

1. Create a JS compatible template only 'glimmer' component so we can
use it with or without glimmer.
2. Add a set of `rose` colors.
3. Animate the brand loader to keep it centered when the side
navigation appears.
4. Tweak the color of Consul::Loader to use a 'rose' color.
5. Move everything loader related to the `app/components/` folder and
add docs.
2021-05-07 12:23:29 +01:00
John Cowen 79f59f76e2
ui: [BUGFIX] De-duplicate Tag rendering (#10186)
* Add some tests for duplicated and non-duplicated tags

* Ensure tags get de-duped and add docs

* Update docs to include info on the recursive-ness
2021-05-07 12:07:11 +01:00
Daniel Nephin c9bd4ebf99 Add changelog 2021-05-06 18:35:52 -04:00
Daniel Nephin 72f52255ff http: set consistency header properly
A recent change in 1.9.x inverted the order of these two lines, which caused the
X-Consul-Effective-Consistency header to be missing for the servie health endpoints
2021-05-06 13:19:45 -04:00
Mark Anderson ecf416aaf6 Add changelog entry
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-05-04 12:41:43 -07:00
Daniel Nephin e7e5e8bc1e Add changelog 2021-05-04 14:09:14 -04:00
John Cowen 18a22c0c82
ui: Remove extra nspace value from service upstreams (#10152)
* ui: Remove extra nspace value from service upstreams
2021-05-04 17:42:13 +01:00
John Cowen 8a43d76c8c
ui: Fix text search for upstream instances (#10151)
* ui: Fix text search for upstream instances

* Clean up predicates for other model types

* Add some docs around DataCollection and searching

* Enable UI Engineering Docs for our preview sites

* Use debug CSS in dev and staging
2021-05-04 17:25:57 +01:00
John Cowen 07496c0180
ui: Adds CRD popover 'informed action' for intentions managed by CRDs (#10100)
* ui: Adds CRD popover 'informed action' for intentions add via CRDs
2021-05-04 17:21:54 +01:00
Paul Banks d47eea3a3f
Make Raft trailing logs and snapshot timing reloadable (#10129)
* WIP reloadable raft config

* Pre-define new raft gauges

* Update go-metrics to change gauge reset behaviour

* Update raft to pull in new metric and reloadable config

* Add snapshot persistance timing and installSnapshot to our 'protected' list as they can be infrequent but are important

* Update telemetry docs

* Update config and telemetry docs

* Add note to oldestLogAge on when it is visible

* Add changelog entry

* Update website/content/docs/agent/options.mdx

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2021-05-04 15:36:53 +01:00
Luke Kysow eb84a856c4
Give descriptive error if auth method not found (#10163)
* Give descriptive error if auth method not found

Previously during a `consul login -method=blah`, if the auth method was not found, the
error returned would be "ACL not found". This is potentially confusing
because there may be many different ACLs involved in a login: the ACL of
the Consul client, perhaps the binding rule or the auth method.

Now the error will be "auth method blah not found", which is much easier
to debug.
2021-05-03 13:39:13 -07:00
Freddy 5427a1465c
Only consider virtual IPs for transparent proxies (#10162)
Initially we were loading every potential upstream address into Envoy
and then routing traffic to the logical upstream service. The downside
of this behavior is that traffic meant to go to a specific instance
would be load balanced across ALL instances.

Traffic to specific instance IPs should be forwarded to the original
destination and if it's a destination in the mesh then we should ensure
the appropriate certificates are used.

This PR makes transparent proxying a Kubernetes-only feature for now
since support for other environments requires generating virtual IPs,
and Consul does not do that at the moment.
2021-05-03 14:15:22 -06:00
Hector Simpson 0e9d4ceb0f
Update brand assets (#10081)
* Update header logo and inline icon

* Update full logos + layout on loading screen

* Update favicon assets and strategy
- Switches to serve an ico file alongside an SVG file
- Introduces an apple-touch-icon

* Removes unused favicon/meta assets

* Changelog item for ui

* Create component for logo

* Simplify logo component, set brand color

* Fix docs loading state CSS issue
2021-05-03 16:19:09 +01:00
Daniel Nephin dcb5b924dc
Merge pull request #10149 from hashicorp/dnephin/config-use-streaming-backend-defualt-true
config: default UseStreamingBackend to true
2021-04-30 16:29:11 -04:00
R.B. Boyer 97e57aedfb
connect: update supported envoy versions to 1.18.2, 1.17.2, 1.16.3, and 1.15.4 (#10101)
The only thing that needed fixing up pertained to this section of the 1.18.x release notes:

> grpc_stats: the default value for stats_for_all_methods is switched from true to false, in order to avoid possible memory exhaustion due to an untrusted downstream sending a large number of unique method names. The previous default value was deprecated in version 1.14.0. This only changes the behavior when the value is not set. The previous behavior can be used by setting the value to true. This behavior change by be overridden by setting runtime feature envoy.deprecated_features.grpc_stats_filter_enable_stats_for_all_methods_by_default.

For now to maintain status-quo I'm explicitly setting `stats_for_all_methods=true` in all versions to avoid relying upon the default.

Additionally the naming of the emitted metrics for these gRPC requests changed slightly so the integration test assertions for `case-grpc` needed adjusting.
2021-04-29 15:22:03 -05:00
R.B. Boyer 06cd0aaa8d
xds: ensure that all envoyproxy/go-control-plane protobuf symbols are linked into the final binary (#10131)
This ensures that if someone does include some extension Consul does not currently make use of, that extension is actually usable. Without linking these envoy protobufs into the main binary it can't round trip the escape hatches to send them down to envoy.

Whenenver the go-control-plane library is upgraded next we just have to re-run 'make envoy-library'.
2021-04-29 14:58:26 -05:00
R.B. Boyer 91bee6246f
Support Incremental xDS mode (#9855)
This adds support for the Incremental xDS protocol when using xDS v3. This is best reviewed commit-by-commit and will not be squashed when merged.

Union of all commit messages follows to give an overarching summary:

xds: exclusively support incremental xDS when using xDS v3

Attempts to use SoTW via v3 will fail, much like attempts to use incremental via v2 will fail.
Work around a strange older envoy behavior involving empty CDS responses over incremental xDS.
xds: various cleanups and refactors that don't strictly concern the addition of incremental xDS support

Dissolve the connectionInfo struct in favor of per-connection ResourceGenerators instead.
Do a better job of ensuring the xds code uses a well configured logger that accurately describes the connected client.
xds: pull out checkStreamACLs method in advance of a later commit

xds: rewrite SoTW xDS protocol tests to use protobufs rather than hand-rolled json strings

In the test we very lightly reuse some of the more boring protobuf construction helper code that is also technically under test. The important thing of the protocol tests is testing the protocol. The actual inputs and outputs are largely already handled by the xds golden output tests now so these protocol tests don't have to do double-duty.

This also updates the SoTW protocol test to exclusively use xDS v2 which is the only variant of SoTW that will be supported in Consul 1.10.

xds: default xds.Server.AuthCheckFrequency at use-time instead of construction-time
2021-04-29 13:54:05 -05:00
Tara Tufano b9130f8e6a
make h2ping changelog entry conform with changelog template (#10111)
LGTM!
2021-04-29 14:48:16 -04:00
Iryna Shustava e7dcf9acd0
Implement traffic redirection exclusion based on proxy config and user-provided values (#10134)
* Use proxy outbound port from TransparentProxyConfig if provided
* If -proxy-id is provided to the redirect-traffic command, exclude any listener ports
  from inbound traffic redirection. This includes envoy_prometheus_bind_addr,
  envoy_stats_bind_addr, and the ListenerPort from the Expose configuration.
* Allow users to provide additional inbound and outbound ports, outbound CIDRs
  and additional user IDs to be excluded from traffic redirection.
  This affects both the traffic-redirect command and the iptables SDK package.
2021-04-29 09:21:15 -07:00
Daniel Nephin dd59a8ee70 Add changelog 2021-04-29 12:08:03 -04:00
Freddy 401f3010e0
Rename "cluster" config entry to "mesh" (#10127)
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.

Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
2021-04-28 16:13:29 -06:00
Daniel Nephin b4362552fb
Merge pull request #10112 from hashicorp/dnephin/remove-streaming-from-cache
streaming: replace agent/cache with submatview.Store
2021-04-28 17:31:42 -04:00
Kenia e5322b7376
ui: Add TProxy Mode notice banner to service instance Upstreams tab (#10136) 2021-04-28 14:06:18 -04:00
Kenia afa05b0276
ui: Update not-defined intention popover and banner (#10133) 2021-04-28 10:31:14 -04:00
Kenia d191b2c552
ui: Update conditional for topology empty state (#10124) 2021-04-28 09:23:02 -04:00
Kenia c9b7fe5051
ui: Fix empty SVG height to prevent service mesh from breaking when there are no upstreams (#10122) 2021-04-28 09:22:18 -04:00
John Cowen a389eab7b0
ui: Adds human formatting to nanosecond based session durations (#10062)
* ui: Adds human formatting to nanosecond based session durations
2021-04-28 12:12:56 +01:00
Daniel Nephin 6d444fd063 Add changelog 2021-04-27 19:03:17 -04:00
Kenia 61ee1f0ad1
Fix up changelog for 10002 (#10130) 2021-04-27 14:29:48 -04:00
Matt Keeler 8b20491a79
Update changelog and add telemetry docs (#10107) 2021-04-23 16:05:00 -04:00
Paul Banks bbe0d6aa30
Fix panic bug in snapshot inspect (#10091)
* Fix panic bug in snapshot inspect

* Add changelog entry

* Update .changelog/10091.txt

* Undo bad GitHub UI merge

* Undo bad GitHub UI merge
2021-04-23 20:48:10 +01:00
Paul Banks 5c409739c7
CLI: Allow snapshot inspect to work on internal raft snapshots directly. (#10089)
* CLI: Add support for reading internal raft snapshots to snapshot inspect

* Add snapshot inspect test for raw state files

* Add changelog entry

* Update .changelog/10089.txt
2021-04-23 16:17:08 +01:00
Freddy 55d67824d6
Disallow * as service-defaults name (#10069) 2021-04-19 14:23:01 -06:00
Freddy 7c9826040c
Merge pull request #10002 from hashicorp/ui/feature/banners-and-labels-for-tproxy-changes 2021-04-15 14:14:20 -06:00
Freddy 5a9b75a443
Merge pull request #10016 from hashicorp/topology-update 2021-04-15 14:11:23 -06:00
freddygv 6300b28e98 Add changelog entry 2021-04-15 13:58:28 -06:00
John Cowen 3d0632cf10
ui: remove old nspace argument from the proxy instance repository (#10039)
The extra argument meant that the blocking query configuration wasn't
being read properly, and therefore the correct ?index wasn't being sent
with the request.
2021-04-15 19:18:07 +01:00
kenia c514664ad1 Add changelog 2021-04-15 09:19:22 -04:00
Kent 'picat' Gruber 7e9bcb06ca
Merge pull request #10030 from hashicorp/fix-ent-audit-log-bypass
Add synthetic enterprise entry for CVE-2021-28156
2021-04-14 20:08:51 -04:00
Kent 'picat' Gruber 78ce699787
Add component name to entry
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-04-14 19:41:04 -04:00
Kent 'picat' Gruber 2998db9cda
Merge pull request #10023 from hashicorp/fix-raw-kv-xss
Add content type headers to raw KV responses
2021-04-14 18:49:14 -04:00
Kent 'picat' Gruber 8f897ee341 Add synthetic enterprise entry for CVE-2021-28156 2021-04-14 18:45:49 -04:00
Daniel Nephin 2a10f01bf5 snapshot: fix saving of auth methods
Previously only a single auth method would be saved to the snapshot. This commit fixes the typo
and adds to the test, to show that all auth methods are now saved.
2021-04-14 16:51:21 -04:00
Kent 'picat' Gruber 6aed60655f Add security release-note changelog entry 2021-04-14 16:40:47 -04:00
Daniel Nephin 5d493daf82
Merge pull request #10014 from hashicorp/dnephin/changelog
Add changelog for enterprise change
2021-04-14 14:09:35 -04:00
Matt Keeler aa0eb60f57
Move static token resolution into the ACLResolver (#10013) 2021-04-14 12:39:35 -04:00
Daniel Nephin ba4f2f853e Add changelog for enterprise change 2021-04-14 11:50:15 -04:00
Tara Tufano b8e7a90f77
add http2 ping health checks (#8431)
* add http2 ping checks

* fix test issue

* add h2ping check to config resources

* add new test and docs for h2ping

* fix grammatical inconsistency in H2PING documentation

* resolve rebase conflicts, add test for h2ping tls verification failure

* api documentation for h2ping

* update test config data with H2PING

* add H2PING to protocol buffers and update changelog

* fix typo in changelog entry
2021-04-09 15:12:10 -04:00
Iryna Shustava ff2e70f4ce
cli: Add new `consul connect redirect-traffic` command for applying traffic redirection rules when Transparent Proxy is enabled. (#9910)
* Add new consul connect redirect-traffic command for applying traffic redirection rules when Transparent Proxy is enabled.
* Add new iptables package for applying traffic redirection rules with iptables.
2021-04-09 11:48:10 -07:00
Freddy 38819dac1f
Merge pull request #9042 from lawliet89/tg-rewrite 2021-04-08 11:49:23 -06:00
Paul Banks ae5c0aad39
cache: Fix bug where connection errors can cause early cache expiry (#9979)
Fixes a cache bug where TTL is not updated while a value isn't changing or cache entry is returning fetch errors.
2021-04-08 11:11:15 +01:00
Paul Banks b61e00b772
cache: fix bug where TTLs were ignored leading to leaked memory in client agents (#9978)
* Fix bug in cache where TTLs are effectively ignored

This mostly affects streaming since streaming will immediately return from Fetch calls when the state is Closed on eviction which causes the race condition every time.

However this also affects all other cache types if the fetch call happens to return between the eviction and then next time around the Get loop by any client.

There is a separate bug that allows cache items to be evicted even when there are active clients which is the trigger here.

* Add changelog entry

* Update .changelog/9978.txt
2021-04-08 11:08:56 +01:00
R.B. Boyer 36c74bf865
command: when generating envoy bootstrap configs to stdout do not mix informational logs into the json (#9980)
Fixes #9921
2021-04-07 14:22:52 -05:00
R.B. Boyer 82245585c6
connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973)
This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.
2021-04-06 13:19:59 -05:00
Yong Wen Chua ea7762849c
Changelog 2021-04-06 17:12:10 +08:00
R.B. Boyer af78561018
api: ensure v1/health/ingress/:service endpoint works properly when streaming is enabled (#9967)
The streaming cache type for service health has no way to handle v1/health/ingress/:service queries as there is no equivalent topic that would return the appropriate data.

Ensure that attempts to use this endpoint will use the old cache-type for now so that they return appropriate data when streaming is enabled.
2021-04-05 13:23:00 -05:00
R.B. Boyer 0b6690fdec
changelog: fixup release-note formatting (#9966)
so these changelog entries will be parsed successfully
2021-04-02 14:59:47 -05:00
Kenia b8cf6e3e0b
changelog: Auth Method feature release (#9963) 2021-04-01 11:00:43 -04:00
Mike Morris df7c1d565f
changelog: fixup release-note formatting (#9960)
so these changelog entries will be parsed successfully
2021-04-01 10:31:48 -04:00
Florian Apolloner 0398833f54
Allow passing ALPN next protocols down to connect services. Fixes #4466. (#9920)
* Allow passing ALPN next protocols down to connect services. Fixes #4466.

* Update connect/proxy/proxy_test.go

Co-authored-by: Paul Banks <banks@banksco.de>

Co-authored-by: Paul Banks <banks@banksco.de>
2021-03-26 11:34:47 +00:00
Daniel Nephin c017fa1d5e Add changelog 2021-03-24 14:51:57 -04:00
Daniel Nephin c8cbff4c5b
Merge pull request #9903 from hashicorp/f-agent-qopts
api: enable query options on agent endpoints
2021-03-19 14:52:51 -04:00
Seth Hoenig dcbe30ef08 api: enable query options on agent endpoints
This PR adds support for setting QueryOptions on a few agent API
endpoints. Nomad needs to be able to set the Namespace field on
these endpoints to:
 - query for services / checks in a namespace
 - deregister services / checks in a namespace
 - update TTL status on checks in a namespace
2021-03-19 13:08:26 -05:00
John Cowen 49cf327ab7
ui: Ensure intention form cancel button works (#9901)
* ui: Ensure intention form cancel button works

By adding `@action` decorators to the actions called form within the
template
2021-03-19 15:14:46 +00:00
woz5999 e05877e633 update docs and add changelog 2021-03-18 19:02:34 -04:00
freddygv 39e6d41fdf Add changelog entry 2021-03-17 22:09:02 -06:00
Freddy fb252e87a4
Add per-upstream configuration to service-defaults 2021-03-17 16:59:51 -06:00
freddygv 15a145b9f6 Add changelog and cleanup todo for beta 2021-03-17 16:45:13 -06:00
John Cowen 5ff1897070
ui: Adds warning icon to side menu when ACLs are disabled (#9864)
* ui: Adds warning icon to side menu when ACLs are are disabled
2021-03-17 11:23:00 +00:00
John Cowen 62a9dffcae
ui: CSP Improvements (#9847)
* Configure ember-auto-import so we can use a stricter CSP

* Create a fake filesystem using JSON to avoid inline scripts in index

We used to have inline scripts in index.html in order to support embers
filepath fingerprinting and our configurable rootURL.

Instead of using inline scripts we use application/json plus a JSON blob
to create a fake filesystem JSON blob/hash/map to hold all of the
rootURL'ed fingerprinted file paths which we can then retrive later in
non-inline scripts.

We move our inlined polyfills script into the init.js external script,
and we move the CodeMirror syntax highlighting configuration inline
script into the main app itself - into the already existing CodeMirror
initializer (this has been moved so we can lookup a service located
document using ember's DI container)

* Set a strict-ish CSP policy during development
2021-03-17 10:46:21 +00:00
Daniel Nephin 96276fc7b8 Add changelog for 9475 2021-03-16 18:22:25 -04:00
Matt Keeler 8d09d610dd
AutopilotServerHealth now handles the 429 status code (#8599)
AutopilotServerHealthy now handles the 429 status code

Previously we would error out and not parse the response. Now either a 200 or 429 status code are considered expected statuses and will result in the method returning the reply allowing API consumers to not only see if the system is healthy or not but which server is unhealthy.
2021-03-12 09:40:49 -05:00
Kyle Havlovitz 237b41ac8f
Merge pull request #9672 from hashicorp/ca-force-skip-xc
connect/ca: Allow ForceWithoutCrossSigning for all providers
2021-03-11 11:49:15 -08:00
Kyle Havlovitz 5f946185d2 Add a changelog note 2021-03-11 11:47:35 -08:00
Kyle Havlovitz 7053fcdd0c
Merge pull request #9792 from dzeban/kv-import-prefix
command/kv: Add prefix option to kv import command
2021-03-11 09:47:53 -08:00
Kyle Havlovitz ae896b7016
Add a changelog entry 2021-03-11 09:46:53 -08:00
Silas Rech f71bba20e7
Fix advertise_addr_wan_ipv6 configuration key 2021-03-09 14:56:44 +01:00
John Cowen a2fa60681a
ui: a11y modals (#9819)
This PR uses the excellent a11y-dialog to implement our modal functionality across the UI.

This package covers all our a11y needs - overlay click and ESC to close, controlling aria-* attributes, focus trap and restore. It's also very small (1.6kb) and has good DOM and JS APIs and also seems to be widely used and well tested.

There is one downside to using this, and that is:

We made use of a very handy characteristic of the relationship between HTML labels and inputs in order to implement our modals previously. Adding a for="id" attribute to a label meant you can control an <input id="id" /> from anywhere else in the page without having to pass javascript objects around. It's just based on using the same string for the for attribute and the id attribute. This allowed us to easily open our login dialog with CSS from anywhere within the UI without having to manage passing around a javascript object/function/method in order to open the dialog.

We've PRed #9813 which includes an approach which would make passing around JS modal object easier to do. But in the meantime we've added a little 'hack' here using an additional <input /> element and a change listener which allows us to keep this label/input characteristic of our old modals. I'd originally thought this would be a temporary amend in order to wait on #9813 but the more I think about it, the more I think its quite a nice thing to keep - so longer term we may/may not keep this.
2021-03-09 09:30:01 +00:00
Nitya Dhanushkodi 9ff49034e7
Add flags to consul connect envoy for metrics merging. (#9768)
Allows setting -prometheus-backend-port to configure the cluster
envoy_prometheus_bind_addr points to.

Allows setting -prometheus-scrape-path to configure which path
envoy_prometheus_bind_addr exposes metrics on.

-prometheus-backend-port is used by the consul-k8s metrics merging feature, to
configure envoy_prometheus_bind_addr to point to the merged metrics
endpoint that combines Envoy and service metrics so that one set of
annotations on a Pod can scrape metrics from the service and it's Envoy
sidecar.

-prometheus-scrape-path is used to allow configurability of the path
where prometheus metrics are exposed on envoy_prometheus_bind_addr.
2021-03-04 16:15:47 -06:00
R.B. Boyer 3ab16c8402 fix typo in changelog 2021-02-26 17:08:33 -06:00
R.B. Boyer 503041f216
xds: default to speaking xDS v3, but allow for v2 to be spoken upon request (#9658)
- Also add support for envoy 1.17.0
2021-02-26 16:23:15 -06:00
Daniel Nephin 4ef9578a07
Merge pull request #9703 from pierresouchay/streaming_tags_and_case_insensitive
Streaming filter tags + case insensitive lookups for Service Names
2021-02-26 12:06:26 -05:00
John Cowen 2d500d24b8
ui: Remove any trailing fullstop/period DNS characters from Gateways UI API (#9752)
Previous to this commit, the API response would include Gateway
Addresses in the form `domain.name.:8080`, which due to the addition of
the port is probably not the expected response.

This commit rightTrims any `.` characters from the end of the domain
before formatting the address to include the port resulting in
`domain.name:8080`
2021-02-25 09:34:47 +00:00
R.B. Boyer cdc5e99184
xds: remove deprecated usages of xDS (#9602)
Note that this does NOT upgrade to xDS v3. That will come in a future PR.

Additionally:

- Ignored staticcheck warnings about how github.com/golang/protobuf is deprecated.
- Shuffled some agent/xds imports in advance of a later xDS v3 upgrade.
- Remove support for envoy 1.13.x but don't add in 1.17.x yet. We have to wait until the xDS v3 support is added in a follow-up PR.

Fixes #8425
2021-02-22 15:00:15 -06:00
R.B. Boyer b8f228781a
connect: if the token given to the vault provider returns no data avoid a panic (#9806)
Improves #9800
2021-02-22 14:08:49 -06:00
Daniel Nephin 8ec1b06f3a
Merge pull request #9683 from hashicorp/dnephin/fix-zombie-service-dereg
local: use agent token to deregister services
2021-02-19 18:56:40 -05:00
Daniel Nephin 76a365d410 local: default to the agent token instead of the user token
When de-registering in anti-entropy sync, when there is no service or
check token.

The agent token will fall back to the default (aka user) token if no agent
token is set, so the existing behaviour still works, but it will prefer
the agent token over the user token if both are set.

ref: https://www.consul.io/docs/agent/options#acl_tokens

The agent token seems more approrpiate in this case, since this is an
"internal operation", not something initiated by the user.
2021-02-19 18:35:08 -05:00
R.B. Boyer e87d2bb24f
xds: only try to create an ipv6 expose checks listener if ipv6 is supported by the kernel (#9765)
Fixes #9311

This only fails if the kernel has ipv6 hard-disabled. It is not sufficient to merely not provide an ipv6 address for a network interface.
2021-02-19 14:38:43 -06:00
John Cowen dc183b1786
ui: Restrict the viewing/editing of certain UI elements based on the users ACLs (#9687)
This commit use the internal authorize endpoint along wiht ember-can to further restrict user access to certain UI features and navigational elements depending on the users ACL token
2021-02-19 16:42:16 +00:00
Mark Anderson 1cf6a435dc
Add fields to the /acl/auth-methods endpoint. (#9741)
* A GET of the /acl/auth-method/:name endpoint returns the fields
MaxTokenTTL and TokenLocality, while a LIST (/acl/auth-methods) does
not.

The list command returns a filtered subset of the full set. This is
somewhat deliberate, so that secrets aren't shown, but the TTL and
Locality fields aren't (IMO) security critical, and it is useful for
the front end to be able to show them.

For consistency these changes mirror the 'omit empty' and string
representation choices made for the GET call.

This includes changes to the gRPC and API code in the client.

The new output looks similar to this
curl 'http://localhost:8500/v1/acl/auth-methods' | jq '.'

  {
    "MaxTokenTTL": "8m20s",
    "Name": "minikube-ttl-local2",
    "Type": "kubernetes",
    "Description": "minikube auth method",
    "TokenLocality": "local",
    "CreateIndex": 530,
    "ModifyIndex": 530,
    "Namespace": "default"
  }
]

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Add changelog

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-02-17 08:16:57 -08:00
Daniel Nephin a29b848e3b stream: fix a snapshot cache bug
Previously a snapshot created as part of a resumse-stream request could have incorrectly
cached the newSnapshotToFollow event. This would cause clients to error because they
received an unexpected framing event.
2021-02-16 12:52:23 -05:00
John Cowen 08cf0f18a6
ui: [BUGFIX] Ensure we show the correct count of instances for each node (#9749)
* Add MeshServiceInstances property to node model

* Use MeshServiceInstances property

* Make sure we show the 'No * checks' if Checks.length is zero
2021-02-11 11:36:36 +00:00
John Cowen 631ddff031
ui: Update browser targets to ~2016 browsers (#9729)
* ui: Reduce range of browsers in targets.js `'Chrome 55', 'Firefox 53', 'Safari 13', 'Edge 15'`

* ui: Remove auto-prefixer
2021-02-11 10:03:13 +00:00
John Cowen 551ac7b794
ui: [BUGFIX] Replace all replaceAll with split.join for older browsers without replaceAll (#9715)
* ui: replace all `replaceAll` with split.join

* Use a div instead of fieldset for flex-box reasons
2021-02-11 09:49:39 +00:00
R.B. Boyer 194fb0d144
connect: update supported envoy point releases to 1.16.2, 1.15.3, 1.14.6, 1.13.7 (#9737) 2021-02-10 13:11:15 -06:00
Matt Keeler 19c99dc104
Stop background refresh of cached data for requests that result in ACL not found errors (#9738) 2021-02-09 10:15:53 -05:00
Freddy 5a50b26767
Avoid potential proxycfg/xDS deadlock using non-blocking send 2021-02-08 16:14:06 -07:00
R.B. Boyer 91d9544803
connect: connect CA Roots in the primary datacenter should use a SigningKeyID derived from their local intermediate (#9428)
This fixes an issue where leaf certificates issued in primary
datacenters using Vault as a Connect CA would be reissued very
frequently (every ~20 seconds) because the logic meant to detect root
rotation was errantly triggering.

The hash of the rootCA was being compared against a hash of the
intermediateCA and always failing. This doesn't apply to the Consul
built-in CA provider because there is no intermediate in use in the
primary DC.

This is reminiscent of #6513
2021-02-08 13:18:51 -06:00
Pierre Souchay b999e5fe2c Updated changelog 2021-02-08 18:16:07 +01:00
freddygv 87d4b1911c Add changelog entry 2021-02-08 09:45:58 -07:00
R.B. Boyer 77424e179a
xds: prevent LDS flaps in mesh gateways due to unstable datacenter lists (#9651)
Also fix a similar issue in Terminating Gateways that was masked by an overzealous test.
2021-02-08 10:19:57 -06:00
R.B. Boyer 05d767b8d6
xds: deduplicate mesh gateway listeners in a stable way (#9650)
In a situation where the mesh gateway is configured to bind to multiple
network interfaces, we use a feature called 'tagged addresses'.
Sometimes an address is duplicated across multiple tags such as 'lan'
and 'lan_ipv4'.

There is code to deduplicate these things when creating envoy listeners,
but that code doesn't ensure that the same tag wins every time. If the
winning tag flaps between xDS discovery requests it will cause the
listener to be drained and replaced.
2021-02-05 16:28:07 -06:00
Pierre Souchay c466b08481 Streaming filter tags + case insensitive lookups for Service Names
Will fix:
 * https://github.com/hashicorp/consul/issues/9695
 * https://github.com/hashicorp/consul/issues/9702
2021-02-04 11:00:51 +01:00
John Cowen c7f8c9141a
chore: changelog for 9660 (#9668) 2021-01-29 16:08:39 +00:00
Matt Keeler 1379b5f7d6
Upgrade raft-autopilot and wait for autopilot it to stop when revoking leadership (#9644)
Fixes: 9626
2021-01-27 11:14:52 -05:00
Hans Hasselberg 623aab5880
Add flags to support CA generation for Connect (#9585) 2021-01-27 08:52:15 +01:00
Matt Keeler d0c9c8b271
Add changelog entry for change to the temporary client license duration (#9642) 2021-01-26 16:15:05 -05:00
John Cowen f38d6f7f13
ui: Sidebar navigation / redesign (#9553)
* CSS for moving from a horizontal main menu to a side/vertical one
* Add <App /> Component and rearrange <HashcorpConsul /> to use it

1. HashicorpConsul now uses <App />
2. <App /> is now translated and adds 'skip to main content' functionality
3. Adds ember-in-viewport addon in order to visibly hide main navigation
items in order to take them out of focus/tabbing
4. Slight amends to the dom service while I was there
2021-01-26 17:40:33 +00:00
R.B. Boyer 5777fa1f59
server: initialize mgw-wanfed to use local gateways more on startup (#9528)
Fixes #9342
2021-01-25 17:30:38 -06:00
Chris Boulton 448212060a
connect: add local_request_timeout_ms to configure local_app http timeouts (#9554) 2021-01-25 13:50:00 -06:00
R.B. Boyer 6622185d64
server: use the presense of stored federation state data as a sign that we already activated the federation state feature flag (#9519)
This way we only have to wait for the serf barrier to pass once before
we can make use of federation state APIs Without this patch every
restart needs to re-compute the change.
2021-01-25 13:24:32 -06:00
John Cowen 148b18b28c
ui: Search/filtering 'Filtered by:' search status (#9442)
Adds a 'status' for the filtering/searching in the UI, without this its not super clear that you are filtering a recordset due to the menu selections being hidden once closed. You can also use the pills in this status view to delete individual filters.
2021-01-25 18:13:54 +00:00
R.B. Boyer e3f5a77ade
server: add OSS stubs supporting validation of source namespaces in service-intentions config entries (#9527) 2021-01-25 11:27:38 -06:00
R.B. Boyer 0247f409a0
server: when wan federating via mesh gateways only do heuristic primary DC bypass on the leader (#9366)
Fixes #9341
2021-01-22 10:03:24 -06:00
John Cowen d3ecb6d7a0
Fix -ui-content-path without regex (#9569)
* Add templating to inject JSON into an application/json script tag

Plus an external script in order to pick it out and inject the values we
need injecting into ember's environment meta tag.

The UI still uses env style naming (CONSUL_*) but we uses the new style
JSON/golang props behind the scenes.

Co-authored-by: Paul Banks <banks@banksco.de>
2021-01-20 18:40:46 +00:00
John Cowen 921c2a2bd8
ui: Changelog for #9593 (#9598) 2021-01-20 18:18:46 +00:00
Kenia 7f380dde18
ui: Removing formatting to display LockDelay in nanoseconds (#9594)
* Removing formatting to display LockDelay in nanoseconds

* Update test

* Add changelog
2021-01-20 12:03:08 -05:00
John Cowen e0b5fb9a83
chore: Changelog entry for #9524 (#9595) 2021-01-20 15:44:41 +00:00
Freddy 5519051c84
Update topology mapping Refs on all proxy instance deletions (#9589)
* Insert new upstream/downstream mapping to persist new Refs

* Avoid upserting mapping copy if it's a no-op

* Add test with panic repro

* Avoid deleting up/downstreams from inside memdb iterator

* Avoid deleting gateway mappings from inside memdb iterator

* Add CHANGELOG entry

* Tweak changelog entry

Co-authored-by: Paul Banks <banks@banksco.de>
2021-01-20 15:17:26 +00:00
Matt Keeler 2d7a4073e3
Merge pull request #9570 from hashicorp/bugfix/9498 2021-01-19 16:30:04 -05:00
Matt Keeler 2d2ce1fb0c
Ensure that CA initialization does not block leader election.
After fixing that bug I uncovered a couple more:

Fix an issue where we might try to cross sign a cert when we never had a valid root.
Fix a potential issue where reconfiguring the CA could cause either the Vault or AWS PCA CA providers to delete resources that are still required by the new incarnation of the CA.
2021-01-19 15:27:48 -05:00
John Cowen 3372e6d14c
ui: Topology intention saving improvements (#9513)
* ui: Keep track of existing intentions and use those to save changes

Previously we risked overwriting existing data in an intention if we
tried to save an intention without having loaded it first, for example
Description and Metadata would have been overwritten.

This change loads in all the intentions for an origin service so we can
pick off the one we need to save and change to ensure that we don't
overwrite any existing data.
2021-01-19 15:40:39 +00:00
Daniel Nephin 45c0a71e7e
Merge pull request #8696 from hashicorp/dnephin/fix-load-limits
agent/consul: make Client/Server config reloading more obvious
2021-01-14 17:40:42 -05:00
Daniel Nephin e8427a48ab agent/consuk: Rename RPCRate -> RPCRateLimit
so that the field name is consistent across config structs.
2021-01-14 17:26:00 -05:00
Daniel Nephin ef9d44fdfe structs: fix caching of ServiceSpecificRequest when ingress=true
The field was not being included in the cache info key. This would result in a DNS request for
web.service.consul returning the same result as web.ingress.consul, when those results should
not be the same.
2021-01-14 17:01:40 -05:00
Chris Piraino baad708929
Fix bug in usage metrics when multiple service instances are changed in a single transaction (#9440)
* Fix bug in usage metrics that caused a negative count to occur

There were a couple of instances were usage metrics would do the wrong
thing and result in incorrect counts, causing the count to attempt to
decrement below zero and return an error. The usage metrics did not
account for various places where a single transaction could
delete/update/add multiple service instances at once.

We also remove the error when attempting to decrement below zero, and
instead just make sure we do not accidentally underflow the unsigned
integer. This is a more graceful failure than returning an error and not
allowing a transaction to commit.

* Add changelog
2021-01-12 15:31:47 -06:00
Michael Hofer acc843f04d
cli: Add consul intention list command (based on PR #6825) (#9468)
This PR is based on the previous work by @snuggie12 in PR #6825. It adds the command consul intention list to list all available intentions. The list functionality for intentions seems a bit overdue as it's just very handy. The web UI cannot list intentions outside of the default namespace, and using the API is sometimes not the friendliest option. ;)

I cherry picked snuggie12's commits who did most of the heavy lifting (thanks again @snuggie12 for your great work!). The changes in the original commit mostly still worked on the current HEAD. On top of that I added support for namespaces and fixed the docs as they are managed differently today. Also the requested changes related to the "Connect" references in the original PRs have been addressed.

Fixes #5652

Co-authored-by: Matt Hoey <mhoey05@jcu.edu>
2021-01-12 21:14:31 +01:00
Pierre Souchay 70d2da7582
Display a warning when rpc.enable_streaming = true is set on a client (#9530)
* Display a warning when rpc.enable_streaming = true is set on a client

This option has no effect when running as an agent

* Added warning when server starts with use_streaming_backend but without rpc.enable_streaming

* Added unit test
2021-01-08 15:23:23 -05:00
Chris Piraino 2eac571276
Log replication warnings when no error suppression is defined (#9320)
* Log replication warnings when no error suppression is defined

* Add changelog file
2021-01-08 14:03:06 -06:00
John Cowen 89e4e0bd97
chore: Adds changelog entry for accessibility improvements (#9509) 2021-01-08 18:17:01 +00:00
Daniel Nephin 0da01d7daf
Merge pull request #9512 from pierresouchay/streaming_fix_grpc_tls2
[Streaming][bugfix] handle TLS signalisation when TLS is disabled on client side (alternative to #9494)
2021-01-06 17:10:47 -05:00
Pierre Souchay 542852786c [Streaming][bugfix] handle TLS signalisation when TLS is disabled on client side
Tnis is an alternative to https://github.com/hashicorp/consul/pull/9494
2021-01-06 17:24:58 +01:00
Pierre Souchay 4f8b0b307c [bugfix] Prometheus metrics without warnings
go-metrics is updated to 0.3.6 to properly handle help in prometheus metrics

This fixes https://github.com/hashicorp/consul/issues/9303 and
https://github.com/hashicorp/consul/issues/9471
2021-01-06 13:54:05 +01:00
R.B. Boyer db62541676
acl: use the presence of a management policy in the state store as a sign that we already migrated to v2 acls (#9505)
This way we only have to wait for the serf barrier to pass once before
we can upgrade to v2 acls. Without this patch every restart needs to
re-compute the change, and potentially if a stray older node joins after
a migration it might regress back to v1 mode which would be problematic.
2021-01-05 17:04:27 -06:00
Matt Keeler a955eeba88
Add changelog for #9487 (#9491) 2021-01-05 13:05:42 -05:00
Daniel Nephin 3140c0a343
Merge pull request #9067 from naemono/6074-allow-config-MaxHeaderBytes
Adds option to configure HTTP Server's MaxHeaderBytes
2021-01-05 12:28:27 -05:00
R.B. Boyer 42dea6f01e
server: deletions of intentions by name using the intention API is now idempotent (#9278)
Restoring a behavior inadvertently changed while fixing #9254
2021-01-04 11:27:00 -06:00
John Cowen 7a21bd7720
ui: [BUGFIX] Request intention listing with ns parameter (#9432)
This PR adds the ns=* query parameter when namespaces are enabled to keep backwards compatibility with how the UI used to work (Intentions page always lists all intention across all namespace you have access to)

I found a tiny dev bug for printing out the current URL during acceptance testing and fixed that up while I was there.
2021-01-04 17:22:10 +00:00
John Cowen dc89b8d340
ui: [BUGFIX] Ensure namespace is used for node API requests (#9410)
Nodes themselves are not namespaced, so we'd originally assumed we did not need to pass through the ns query parameter when listing or viewing nodes.

As it turns out the API endpoints we use to list and view nodes (and related things) return things that are namespaced, therefore any API requests for nodes do require a the ns query parameter to be passed through to the request.

This PR adds the necessary ns query param to all things Node, apart from the querying for the leader which only returns node related information.

Additionally here we decided to show 0 Services text in the node listing if there are nodes with no service instances within the namespace you are viewing, as this is clearer than showing nothing at all. We also cleaned up/standardized the text we use to in the empty state for service instances.
2021-01-04 16:42:44 +00:00
Michael Montgomery a1748aa2cb Merge branch 'master' into 6074-allow-config-MaxHeaderBytes 2020-12-30 14:14:05 -06:00
Michael Montgomery 519f537b8b Fixed failing tests
Removed use of `NewTestAgent`, per review comment
Removed CLI flag, per review comment
Updated website documentation
Added changelog entry
2020-12-30 14:09:50 -06:00
Daniel Nephin 462a2b29c3
Merge pull request #9262 from hashicorp/dnephin/docs-deprecate-old-filters
docs: deprecate some old filter parameters
2020-12-15 17:11:41 -05:00
R.B. Boyer f9dcaf7f6b
acl: global tokens created by auth methods now correctly replicate to secondary datacenters (#9351)
Previously the tokens would fail to insert into the secondary's state
store because the AuthMethod field of the ACLToken did not point to a
known auth method from the primary.
2020-12-09 15:22:29 -06:00
Matt Keeler d313b4af77
Add changelog for fixing the namespace replication bug from #9271 (#9347) 2020-12-08 12:04:51 -05:00
Mike Morris 9e01633016
changelog: add entry for fixing active CA root unset (#9323) 2020-12-03 13:45:07 -05:00
Mike Morris 0650188f27
changelog: add entries for secondary datacenter CA fixes (#9322) 2020-12-03 13:33:29 -05:00
Mike Morris 24130f4f09
Merge pull request #9273 from hashicorp/merge/release-1.9.0
merge: release/1.9.0
2020-12-02 17:34:10 -05:00
John Cowen 08a866b239 Changelog 2020-11-30 17:27:39 +00:00
Daniel Nephin 4c5fab6e00 local: mark service and checks as InSync when added
If the existing service and checks are the same as the new registration.
2020-11-27 15:31:12 -05:00
Daniel Nephin 813f0d552d
Merge pull request #9247 from pierresouchay/streaming_predictible_order_for_health
[Streaming] Predictable order for results of /health/service/:serviceName to mimic memdb
2020-11-25 15:53:18 -05:00
Mike Morris 67a11e4d16 Merge pull request #9270 from hashicorp/release/1.9.0
merge: release/1.9.0 back into 1.9.x
2020-11-24 17:36:47 -05:00
R.B. Boyer 6d6b6c15c6
server: fix panic when deleting a non existent intention (#9254)
* server: fix panic when deleting a non existent intention

* add changelog

* Always return an error when deleting non-existent ixn

Co-authored-by: freddygv <gh@freddygv.xyz>
2020-11-24 13:44:20 -05:00
Daniel Nephin 685cb158cf docs: deprecate some old filter parameters
The filtering can be done with the general purpose `filter` query parameter.
2020-11-23 18:23:58 -05:00
Pierre Souchay c22f249a99 Added changelog entry for 9247 2020-11-20 18:23:01 +01:00
R.B. Boyer 7bcbc59dea
command: when generating envoy bootstrap configs use the datacenter returned from the agent services endpoint (#9229)
Fixes #9215
2020-11-19 15:27:31 -06:00
Freddy e4e306210a
Require operator:write to get Connect CA config (#9240)
A vulnerability was identified in Consul and Consul Enterprise (“Consul”) such that operators with `operator:read` ACL permissions are able to read the Consul Connect CA configuration when explicitly configured with the `/v1/connect/ca/configuration` endpoint, including the private key. This allows the user to effectively privilege escalate by enabling the ability to mint certificates for any Consul Connect services. This would potentially allow them to masquerade (receive/send traffic) as any service in the mesh.

--

This PR increases the permissions required to read the Connect CA's private key when it was configured via the `/connect/ca/configuration` endpoint. They are now `operator:write`.
2020-11-19 10:14:48 -07:00
Matt Keeler 4bca029be9
Refactor to call non-voting servers read replicas (#9191)
Co-authored-by: Kit Patella <kit@jepsen.io>
2020-11-17 10:53:57 -05:00
Kenia aa4b4c6cfa
ui: Changelog changes (#9209) 2020-11-17 10:35:56 -05:00
Kit Patella 4dfcdbab26
Merge pull request #9198 from hashicorp/mkcp/telemetry/add-all-metric-definitions
Add metric definitions for all metrics known at Consul start
2020-11-16 15:54:50 -08:00
Kit Patella 615a145e54 changelog component should mention agent not just server 2020-11-16 15:54:24 -08:00
Freddy 2763833d32
Add DC and NS support for Envoy metrics (#9207)
This PR updates the tags that we generate for Envoy stats.

Several of these come with breaking changes, since we can't keep two stats prefixes for a filter.
2020-11-16 16:37:19 -07:00
Kit Patella caba383427 add changelog entry 2020-11-16 15:32:18 -08:00
Matt Keeler 197a37a860
Prevent panic if autopilot health is requested prior to leader establishment finishing. (#9204) 2020-11-16 17:08:17 -05:00
Matt Keeler c01e0756d8
Add changelog entry for namespace licensing fix (#9203) 2020-11-16 15:45:55 -05:00
Kit Patella 179c6e22c0
Trim to one deprecation entry 2020-11-13 14:31:14 -08:00
Kit Patella 91a508995f add note about future metric fixes and deprecations under disable_compat_1.9 2020-11-13 14:16:53 -08:00
Mike Morris 4902e42ca4 Merge pull request #9155 from hashicorp/release/1.9.0-beta3
merge: 1.9.0-beta3
2020-11-13 16:45:50 -05:00
R.B. Boyer e323014faf
server: remove config entry CAS in legacy intention API bridge code (#9151)
Change so line-item intention edits via the API are handled via the state store instead of via CAS operations.

Fixes #9143
2020-11-13 14:42:21 -06:00
R.B. Boyer 6300abed18
server: skip deleted and deleting namespaces when migrating intentions to config entries (#9186) 2020-11-13 13:56:41 -06:00
R.B. Boyer 758384893d
server: break up Intention.Apply monolithic method (#9007)
The Intention.Apply RPC is quite large, so this PR attempts to break it down into smaller functions and dissolves the pre-config-entry approach to the breakdown as it only confused things.
2020-11-13 09:15:39 -06:00
R.B. Boyer a5bd1ba323
agent: return the default ACL policy to callers as a header (#9101)
Header is: X-Consul-Default-ACL-Policy=<allow|deny>

This is of particular utility when fetching matching intentions, as the
fallthrough for a request that doesn't match any intentions is to
enforce using the default acl policy.
2020-11-12 10:38:32 -06:00
Matt Keeler ed56d62ec2
Add changelog entry for autopilot state CLI (#9161) 2020-11-11 14:55:12 -05:00
Joel Watson 4b9034b976
Merge pull request #9098 from hashicorp/watsonian/kv-size-breakdown
Add detailed key size breakdown to snapshot inspect
2020-11-11 11:34:45 -06:00
Joel Watson ff0009bd91 Fix some minor wording issues 2020-11-11 11:33:38 -06:00
Matt Keeler 2badb01d30
Add a paramter in state store methods to indicate whether a resource insertion is from a snapshot restoration (#9156)
The Catalog, Config Entry, KV and Session resources potentially re-validate the input as its coming in. We need to prevent snapshot restoration failures due to missing namespaces or namespaces that are being deleted in enterprise.
2020-11-11 11:21:42 -05:00
Joel Watson 8af5a9aea9 Add changelog entry 2020-11-11 09:56:26 -06:00
Matt Keeler c998b74a68
Fixup the autopilot changelog (#9145) 2020-11-09 17:29:06 -05:00
Matt Keeler 114521af25
Add some autopilot docs and update the changelog (#9139) 2020-11-09 14:14:19 -05:00
Mike Morris 2be2be577c
connect: switch the default gateway port from 443 to 8443 (#9116)
* test: update ingress gateway golden file to port 8443

* test: update Envoy flags_test to port 8443

Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2020-11-06 20:47:29 -05:00
Mike Morris 2317a471b3
changelog: add entries for 1.9.0-beta2 (#9129) 2020-11-06 17:18:47 -05:00
R.B. Boyer 9b37ea7dcb
Revert "Add namespace support for metrics (OSS) (#9117)" (#9124)
This reverts commit 06b3b017d326853dbb53bc0ec08ce371265c5ce9.
2020-11-06 10:24:32 -06:00
Freddy 874efe705f
Add namespace support for metrics (OSS) (#9117) 2020-11-05 18:24:29 -07:00
s-christoff ee3eb03f50
cli: Add JSON and Pretty Print formatting for `consul snapshot inspect` (#9006) 2020-10-29 11:31:14 -05:00
Mike Morris 730426c6bd
changelog: update to hashicorp/sentinel@v0.16.0 in Enterprise (#8984) 2020-10-26 12:32:44 -04:00
Mike Morris d1a92026b7 Merge pull request #9027 from hashicorp/release/1.8.5
Merge back release/1.8.5
2020-10-26 10:59:48 -04:00
R.B. Boyer 0a80e82f21
server: config entry replication now correctly uses namespaces in comparisons (#9024)
Previously config entries sharing a kind & name but in different
namespaces could occasionally cause "stuck states" in replication
because the namespace fields were ignored during the differential
comparison phase.

Example:

Two config entries written to the primary:

    kind=A,name=web,namespace=bar
    kind=A,name=web,namespace=foo

Under the covers these both get saved to memdb, so they are sorted by
all 3 components (kind,name,namespace) during natural iteration. This
means that before the replication code does it's own incomplete sort,
the underlying data IS sorted by namespace ascending (bar comes before
foo).

After one pass of replication the primary and secondary datacenters have
the same set of config entries present. If
"kind=A,name=web,namespace=bar" were to be deleted, then things get
weird. Before replication the two sides look like:

primary: [
    kind=A,name=web,namespace=foo
]
secondary: [
    kind=A,name=web,namespace=bar
    kind=A,name=web,namespace=foo
]

The differential comparison phase walks these two lists in sorted order
and first compares "kind=A,name=web,namespace=foo" vs
"kind=A,name=web,namespace=bar" and falsely determines they are the SAME
and are thus cause an update of "kind=A,name=web,namespace=foo". Then it
compares "<nothing>" with "kind=A,name=web,namespace=foo" and falsely
determines that the latter should be DELETED.

During reconciliation the deletes are processed before updates, and so
for a brief moment in the secondary "kind=A,name=web,namespace=foo" is
erroneously deleted and then immediately restored.

Unfortunately after this replication phase the final state is identical
to the initial state, so when it loops around again (rate limited) it
repeats the same set of operations indefinitely.
2020-10-23 13:41:54 -05:00
Daniel Nephin 1d03a7a8a4
Merge pull request #8924 from ShimmerGlass/fix-sidecar-deregister-after-restart
Fix: service LocallyRegisteredAsSidecar property is not persisted
2020-10-22 13:26:55 -04:00
Daniel Nephin 1e1f963064
Merge pull request #8771 from amenzhinsky/fix-grpc-use-tls-mapping
Fix GRPCUseTLS flag HTTP API mapping
2020-10-21 18:37:11 -04:00
Daniel Nephin 73ebb31a1e Add changelog entry 2020-10-20 16:42:06 -04:00
Preetha 133ef84f0a
Merge pull request #8947 from hashicorp/dnephin/changelog-for-streaming
Add streaming changelog file
2020-10-14 09:42:10 -05:00
R.B. Boyer db56eea171
fix 1.9.0-beta1 changelog formatting (#8941) 2020-10-14 09:35:59 -05:00
Daniel Nephin cc37ed1c2d Add streaming changelog file 2020-10-13 18:16:33 -04:00
Mathilde Gilles a1ec792acc Fix: service LocallyRegisteredAsSidecar property is not persisted
When a service is deregistered, we check whever matching services were
registered as sidecar along with it and deregister them as well.
To determine if a service is indeed a sidecar we check the
structs.ServiceNode.LocallyRegisteredAsSidecar property. However, to
avoid interal API leakage, it is excluded from JSON serialization,
meaning it is not persisted to disk either.
When the agent is restarted, this property lost and sidecars are no
longer deregistered along with their parent service.
To fix this, we now specifically save this property in the persisted
service file.
2020-10-13 19:38:58 +02:00
Mike Morris 348060ad17 changelog: fixup note.tmpl syntax 2020-10-09 22:44:51 -04:00
Paul Banks c0e54b2374
changelog: add entries for ui_config and service metrics config (#8919)
* Create 8694.txt

* Apply suggestions from code review

Co-authored-by: Freddy <freddygv@users.noreply.github.com>

Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2020-10-09 17:31:00 -04:00
Paul Banks 5752552a14
changelog: add entries for UI topology viz (#8918)
* Create 8858.txt

* add separate changelog entries for original topology impl and intentions

Co-authored-by: Mike Morris <mikemorris@users.noreply.github.com>
2020-10-09 17:29:14 -04:00
Pierre Souchay 54f9f247f8
Consul Service meta wrongly computes and exposes non_voter meta (#8731)
* Consul Service meta wrongly computes and exposes non_voter meta

In Serf Tags, entreprise members being non-voters use the tag
`nonvoter=1`, not `non_voter = false`, so non-voters in members
were wrongly displayed as voter.

Demonstration:

```
consul members -detailed|grep voter
consul20-hk5 10.200.100.110:8301   alive   acls=1,build=1.8.4+ent,dc=hk5,expect=3,ft_fs=1,ft_ns=1,id=xxxxxxxx-5629-08f2-3a79-10a1ab3849d5,nonvoter=1,port=8300,raft_vsn=3,role=consul,segment=<all>,use_tls=1,vsn=2,vsn_max=3,vsn_min=2,wan_join_port=8302
```

* Added changelog

* Added changelog entry
2020-10-09 17:18:24 -04:00
s-christoff a62705101f
Enhance the output of consul snapshot inspect (#8787) 2020-10-09 14:57:29 -05:00
Kit Patella 1d42dd0cdb
Merge pull request #8914 from hashicorp/mkcp/changelog/8877
add changelog entries for 8877
2020-10-09 12:51:23 -07:00
Kit Patella fa4342524f add template generation for entries tagged deprecation 2020-10-09 12:40:41 -07:00
Kit Patella 52791506a0 add changelog entries for 8877 2020-10-09 12:38:57 -07:00
Kyle Havlovitz 926a393a5c
Merge pull request #8784 from hashicorp/renew-intermediate-primary
connect: Enable renewing the intermediate cert in the primary DC
2020-10-09 12:18:59 -07:00
Hans Hasselberg 26bb8ecb1b
note template with ent support (#8910) 2020-10-09 21:06:49 +02:00
Mike Morris 3692998f4a
changelog: update raft to v1.2.0 (#8901) 2020-10-09 11:28:13 -04:00
Kyle Havlovitz 01dbf27af3 Add changelog note 2020-10-09 08:01:55 -07:00
Matt Keeler 3c2b23cccb
Create _619.txt 2020-10-09 10:51:37 -04:00
Matt Keeler 891d05fada
Add capability for the v1/connect/ca/roots endpoint to return a PEM encoded certificate chain (#8774)
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2020-10-09 10:43:33 -04:00
R.B. Boyer 69af49441a
agent: allow the /v1/connect/intentions/match endpoint to use the agent cache (#8875)
This is the recommended proxy integration API for listing intentions
which should not require an active connection to the servers to resolve
after the initial cache filling.
2020-10-08 14:51:53 -05:00
Matt Keeler 141eb60f06
Add per-agent reconnect timeouts (#8781)
This allows for client agent to be run in a more stateless manner where they may be abruptly terminated and not expected to come back. If advertising a per-agent reconnect timeout using the advertise_reconnect_timeout configuration when that agent leaves, other agents will wait only that amount of time for the agent to come back before reaping it.

This has the advantageous side effect of causing servers to deregister the node/services/checks for that agent sooner than if the global reconnect_timeout was used.
2020-10-08 15:02:19 -04:00
R.B. Boyer 7d18407e6a
command: remove conditional envoy bootstrap generation for versions <=1.10.0 since those are not supported (#8855) 2020-10-07 10:53:23 -05:00
R.B. Boyer d257b49601 add missing changelog entry for #8839 2020-10-07 10:22:40 -05:00
Pierre Souchay a12056f57c Added changelog for merged PR #8221 2020-10-06 17:15:33 -04:00
R.B. Boyer d6dce2332a
connect: intentions are now managed as a new config entry kind "service-intentions" (#8834)
- Upgrade the ConfigEntry.ListAll RPC to be kind-aware so that older
copies of consul will not see new config entries it doesn't understand
replicate down.

- Add shim conversion code so that the old API/CLI method of interacting
with intentions will continue to work so long as none of these are
edited via config entry endpoints. Almost all of the read-only APIs will
continue to function indefinitely.

- Add new APIs that operate on individual intentions without IDs so that
the UI doesn't need to implement CAS operations.

- Add a new serf feature flag indicating support for
intentions-as-config-entries.

- The old line-item intentions way of interacting with the state store
will transparently flip between the legacy memdb table and the config
entry representations so that readers will never see a hiccup during
migration where the results are incomplete. It uses a piece of system
metadata to control the flip.

- The primary datacenter will begin migrating intentions into config
entries on startup once all servers in the datacenter are on a version
of Consul with the intentions-as-config-entries feature flag. When it is
complete the old state store representations will be cleared. We also
record a piece of system metadata indicating this has occurred. We use
this metadata to skip ALL of this code the next time the leader starts
up.

- The secondary datacenters continue to run the old intentions
replicator until all servers in the secondary DC and primary DC support
intentions-as-config-entries (via serf flag). Once this condition it met
the old intentions replicator ceases.

- The secondary datacenters replicate the new config entries as they are
migrated in the primary. When they detect that the primary has zeroed
it's old state store table it waits until all config entries up to that
point are replicated and then zeroes its own copy of the old state store
table. We also record a piece of system metadata indicating this has
occurred. We use this metadata to skip ALL of this code the next time
the leader starts up.
2020-10-06 13:24:05 -05:00
R.B. Boyer a77b518542
server: create new memdb table for storing system metadata (#8703)
This adds a new very tiny memdb table and corresponding raft operation
for updating a very small effective map[string]string collection of
"system metadata". This can persistently record a fact about the Consul
state machine itself.

The first use of this feature will come in a later PR.
2020-10-06 10:08:37 -05:00
R.B. Boyer 4b525e5d16
agent: enable enable_central_service_config by default (#8746) 2020-10-01 09:19:14 -05:00
R.B. Boyer ccd0200bd9
server: ensure that we also shutdown network segment serf instances on server shutdown (#8786)
This really only matters for unit tests, since typically if an agent shuts down its server, it follows that up by exiting the process, which would also clean up all of the networking anyway.
2020-09-30 16:23:43 -05:00
R.B. Boyer e84c032ea7
api: support GetMeta() and GetNamespace() on all config entry kinds (#8764)
Fixes #8755

Since I was updating the interface, i also added the missing `GetNamespace()`.

Depending upon how you look at it, this is a breaking change since it adds methods to the exported interface `api.ConfigEntry`. Given that you cannot define your own config entry kinds, and all of the machinery of the `api.Client` acts like a factory to construct the canned ones from the rest of the module, this feels like it's not a problematic change as it would only break someone who had reimplemented the `ConfigEntry` interface themselves for no apparent utility?
2020-09-29 09:11:57 -05:00
Daniel Nephin e1855afbe1
Merge pull request #8726 from amenzhinsky/grpc-hc-error
Return grpc serving status in health check errors
2020-09-25 13:24:32 -04:00
Daniel Nephin b611e074f9 Add changelog file 2020-09-25 12:03:49 -04:00
R.B. Boyer 657995cc93
agent: when enable_central_service_config is enabled ensure agent reload doesn't revert check state to critical (#8747)
Likely introduced when #7345 landed.
2020-09-24 16:24:04 -05:00
R.B. Boyer 45609fccdf
server: make sure that the various replication loggers use consistent logging (#8745) 2020-09-24 15:49:38 -05:00
R.B. Boyer 4af8c78f1f
agent: make the json/hcl decoding of ConnectProxyConfig fully work with CamelCase and snake_case (#8741)
Fixes #7418
2020-09-24 13:58:52 -05:00
Hans Hasselberg d48d2bf550
use service datacenter for dns name (#8704)
* Use args.Datacenter instead of configured datacenter
2020-09-22 20:34:09 +02:00
Kyle Havlovitz 8f83f7ac13
Merge pull request #8560 from hashicorp/vault-ca-renew-token
Automatically renew the token used by the Vault CA provider
2020-09-16 07:30:30 -07:00
Daniel Nephin 9f83eb3dc9
Merge pull request #8685 from pierresouchay/do_not_flood_logs_with_Non-server_in_server-only_area
[BUGFIX] Avoid GetDatacenter* methods to flood Consul servers logs
2020-09-15 17:57:05 -04:00
Daniel Nephin 7b81104357
Update .changelog/8685.txt 2020-09-15 17:56:06 -04:00
Kyle Havlovitz c8fd61abc7 Merge branch 'master' into vault-ca-renew-token 2020-09-15 14:39:04 -07:00
Kyle Havlovitz 316600a685 Update vault CA for latest api client 2020-09-15 13:33:55 -07:00
Kyle Havlovitz 6f1dd25139
Merge pull request #8646 from hashicorp/common-intermediate-ttl
Move IntermediateCertTTL to common CA config
2020-09-15 12:03:29 -07:00
Kyle Havlovitz b9704e3766
Create 8646.txt 2020-09-15 10:05:23 -07:00
Pierre Souchay 879885986e Added Changelog for PR #8685. 2020-09-15 12:30:12 +02:00
Freddy 4da691bdfd
Create 8585.txt 2020-09-14 14:16:47 -06:00
Hans Hasselberg e61f33d725
add entry for 8588 (#8650) 2020-09-10 18:53:36 +02:00
Tim Arenz 6dbb5f3234
Add support for -ca-path option in the connect envoy command (#8606)
* Add support for -ca-path option in the connect envoy command
* Adding changelog entry
2020-09-08 12:16:16 +02:00
Seth Hoenig 99b822c4b3
api: create fresh http client for unix sockets (#8602)
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2020-09-06 12:27:39 -04:00
Freddy 581f19b9d4
Adds changelog entry for snapshot agent improvement (#8622) 2020-09-04 14:07:57 -06:00
R.B. Boyer b0bde51e70
connect: all config entries pick up a meta field (#8596)
Fixes #8595
2020-09-02 14:10:25 -05:00
Chris Piraino df1381f77f
Merge pull request #8603 from hashicorp/feature/usage-metrics
Track node and service counts in the state store and emit them periodically as metrics
2020-09-02 13:23:39 -05:00
Chris Piraino 4bc16c2eb1 Changelog entry for usage metrics 2020-09-02 10:48:11 -05:00
R.B. Boyer 4197bed23b
connect: fix bug in preventing some namespaced config entry modifications (#8601)
Whenever an upsert/deletion of a config entry happens, within the open
state store transaction we speculatively test compile all discovery
chains that may be affected by the pending modification to verify that
the write would not create an erroneous scenario (such as splitting
traffic to a subset that did not exist).

If a single discovery chain evaluation references two config entries
with the same kind and name in different namespaces then sometimes the
upsert/deletion would be falsely rejected. It does not appear as though
this bug would've let invalid writes through to the state store so the
correction does not require a cleanup phase.
2020-09-02 10:47:19 -05:00
Daniel Nephin bd4e480a78
Merge pull request #8577 from hashicorp/dnephin/changelog-for-8537
Add missing changelog for PR 8537
2020-08-31 11:56:20 -04:00
Pierre Souchay 2a52f89b8d Added changelog for #8552 2020-08-28 23:01:04 +02:00
Jack 145bcdc2bb
Add http2 and grpc support to ingress gateways (#8458) 2020-08-27 15:34:08 -06:00
R.B. Boyer f2b8bf109c
xds: use envoy's rbac filter to handle intentions entirely within envoy (#8569) 2020-08-27 12:20:58 -05:00
R.B. Boyer a7a8b8d6d9
agent: ensure that we normalize bootstrapped config entries (#8547) 2020-08-27 11:37:25 -05:00
Daniel Nephin 85de324a86 Retroactively add changelog for PR 8537 2020-08-27 11:53:49 -04:00
Matt Keeler 5e2f0be305
Add helpers to the API client to help with getting information from `AgentMember` tags (#8575)
Lots of constants were added for various tags that would concern users and are not already parsed out.

Additionally two methods on the AgentMember type were added to ask a member what its ACL Mode is and whether its a server or not.
2020-08-27 11:00:48 -04:00
R.B. Boyer 6fad634512
agent: expose the list of supported envoy versions on /v1/agent/self (#8545) 2020-08-26 10:04:11 -05:00
Hans Hasselberg 02de4c8b76
add primary keys to list keyring (#8522)
During gossip encryption key rotation it would be nice to be able to see if all nodes are using the same key. This PR adds another field to the json response from `GET v1/operator/keyring` which lists the primary keys in use per dc. That way an operator can tell when a key was successfully setup as primary key.

Based on https://github.com/hashicorp/serf/pull/611 to add primary key to list keyring output:

```json
[
  {
    "WAN": true,
    "Datacenter": "dc2",
    "Segment": "",
    "Keys": {
      "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 6,
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6
    },
    "PrimaryKeys": {
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 6
    },
    "NumNodes": 6
  },
  {
    "WAN": false,
    "Datacenter": "dc2",
    "Segment": "",
    "Keys": {
      "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 8,
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "PrimaryKeys": {
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "NumNodes": 8
  },
  {
    "WAN": false,
    "Datacenter": "dc1",
    "Segment": "",
    "Keys": {
      "0OuM4oC3Os18OblWiBbZUaHA7Hk+tNs/6nhNYtaNduM=": 3,
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "PrimaryKeys": {
      "SINm887hKTzmMWeBNKTJReaTLX3mBEJKriDyt88Ad+g=": 8
    },
    "NumNodes": 8
  }
]
```

I intentionally did not change the CLI output because I didn't find a good way of displaying this information. There are a couple of options that we could implement later:
* add a flag to show the primary keys
* add a flag to show json output

Fixes #3393.
2020-08-18 09:50:24 +02:00
Hans Hasselberg 658c4cad0b
Link issue in note template (#8502)
Issue and PR numbers do not overlap, they are based of the same counter.
A PR can be also linked to via issues, if it is a PR, Github will
redirect to it.
This change has the benefit that one can link to both - issues and PRs.
2020-08-13 10:22:56 +02:00
R.B. Boyer 1593ce2948 update changelog snippet 2020-08-12 11:21:54 -05:00
Hans Hasselberg 1f9a941531
Introducing changelog-gen (#8387)
* add templates for changelog-gen
* add entry files for currently unreleased PRs on master
2020-08-06 23:15:29 +02:00