Commit graph

14513 commits

Author SHA1 Message Date
Daniel Nephin de1f766d03 api: properly close the response body
reads resp.Body until EOF, so that the http client is able to re-use the TCP connection.
2021-06-14 18:52:59 -04:00
R.B. Boyer 8d5f81b460
xds: ensure that dependent xDS resources are reconfigured during primary type warming (#10381)
Updates to a cluster will clear the associated endpoints, and updates to
a listener will clear the associated routes. Update the incremental xDS
logic to account for this implicit cleanup so that we can finish warming
the clusters and listeners.

Fixes #10379
2021-06-14 17:20:27 -05:00
Freddy f399fd2add
Rename CatalogDestinationsOnly (#10397)
CatalogDestinationsOnly is a passthrough that would enable dialing
addresses outside of Consul's catalog. However, when this flag is set to
true only _connect_ endpoints for services can be dialed.

This flag is being renamed to signal that non-Connect endpoints can't be
dialed by transparent proxies when the value is set to true.
2021-06-14 14:15:09 -06:00
R.B. Boyer b050232abb
grpc: move gRPC INFO logs to be emitted as TRACE logs from Consul (#10395)
Fixes #10183
2021-06-14 15:13:58 -05:00
Freddy f19b1f0058
Relax validation for expose.paths config (#10394)
Previously we would return an error if duplicate paths were specified.
This could lead to problems in cases where a user has the same path,
say /healthz, on two different ports.

This validation was added to signal a potential misconfiguration.
Instead we will only check for duplicate listener ports, since that is
what would lead to ambiguity issues when generating xDS config.

In the future we could look into using a single listener and creating
distinct filter chains for each path/port.
2021-06-14 14:04:11 -06:00
Dhia Ayachi c3eacac764
upgrade golang crypto from 0.0.0-20200930160638-afb6bcd081ae => v0.0.0-20210513164829-c07d793c2f9a (#10390) 2021-06-14 12:38:42 -04:00
Luke Kysow 168fbbbed1
Update k8s term gateway docs to make address clear (#10389)
Previously if you were to follow these docs and register two external
services, you would set the Address field on the node. The second
registered service would change the address of the node for the first
service.

Now the docs explain the address key and how to register more than one
external service.
2021-06-14 09:15:40 -07:00
Peter M cf8fc11f98
Homepage CTA button to point to Learn (#10380)
redirecting users to learn instead of to the download page.
2021-06-11 14:49:29 -07:00
Daniel Nephin 69262022da
Merge pull request #10378 from hashicorp/dnephin/agent-self-primary-dc
http: add PrimaryDatacenter to the /v1/agent/self  response
2021-06-11 13:44:07 -04:00
Peter M 652a782d57
Homepage Hero Upgrade w/ Alert (#10377)
* updating hero with ecs info

* updates to hero

* Include back the Basic Hero styles

The basic hero is still used on the use case pages

* Revert the tsconfig changes

Nothing in the scope of this PR requires these changes!

* Remove the old Carousel CSS file

This is no longer needed as we're using the @hashicorp/react-hero
which comes with all the styling required for this carousel to work.

* Rename ConsulHero -> HomepageHero imports/exports

This will help prevent any confusion for future devs here -- this is a
convention we have that helps us from having to trace every import,
which helps us find the source of the component without actually having
to look at the import.

* Pin the deps

These were previously pinned to the exact version; including ^ will
allow minor & patch updates to sneak in, which normally shouldn't cause
an issue but we tend to be more conservative on dep upgrades.

* Revert unneeded changes to the document file

* Revert changes to app.js file

Not needed in the scope of this PR!

* Hard pin react-alert

* Remove unneeded css

Co-authored-by: Brandon Romano <brandon@hashicorp.com>
2021-06-10 15:31:55 -07:00
Daniel Nephin 95315e9e06 http: add PrimaryDatacenter to the /v1/agent/self response
This field is available in DebugConfig, but that field is not stable and could change at any time.

The consul-k8s needs to be able to detect the primary DC for tests, so adding this field to the
stable part of the API response.
2021-06-10 17:19:16 -04:00
Nick Wales e01ea816c2
Aligns audit log code example (#10371) 2021-06-10 11:41:53 -07:00
R.B. Boyer 7ee812b22f
docs: update envoy docs for changes related to xDS v2->v3 and SoTW->Incremental (#10166)
Fixes #10098
2021-06-10 10:59:54 -05:00
freddygv b8b5dd2ecc Update CHANGELOG.md 2021-06-09 19:02:12 -06:00
Freddy 61ae2995b7
Add flag for transparent proxies to dial individual instances (#10329) 2021-06-09 14:34:17 -06:00
Brandon Romano 8e49142e65
Merge pull request #10366 from hashicorp/pcmccarron-ecs-additions
Adding ECS copy and other edits
2021-06-09 09:20:54 -07:00
Daniel Nephin 01c5ccc3fb
Merge pull request #10367 from hashicorp/dnephin/submatview-store-get-tests
submatview: add test cases for store.Get with timeout and no index
2021-06-09 11:53:46 -04:00
David Bariod 728165f31e
remove a race condition in sdk testutil server stop (#10342)
* remove a race condition in sdk testutil server stop

* ensure the process has actually stopped
2021-06-09 11:30:26 -04:00
Mike Wickett 99fc5354e7
Merge pull request #10369 from hashicorp/mw.post-hashiconf-alert-banner
Adjust alert banner for the end of HashiConf
2021-06-09 11:01:46 -04:00
Mike Wickett 408f38df0a Adjust alert banner for the end of HashiConf 2021-06-09 07:54:58 -04:00
Peter M 565b270783
removing alert component 2021-06-08 16:31:04 -06:00
Daniel Nephin b5503223ae submatview: add test cases for store.Get with timeout and no index
Also set a more unique name for the serviceRequest.Type to prevent potential name conflicts
in the future.
2021-06-08 18:04:38 -04:00
Daniel Nephin 450cce60a1
Merge pull request #10364 from hashicorp/dnephin/streaming-e2e-test
submatview: and Store integration test with stream backend
2021-06-08 16:13:45 -04:00
Daniel Nephin 175bd6d9aa
Merge pull request #10169 from hashicorp/dnephin/streaming-docs
docs: add streaming to api-docs
2021-06-08 16:13:34 -04:00
Peter M 7389f86dfc
Adding ECS copy and other edits
Updated the descriptions to match the right tiles and added a new alert for ECS integration
2021-06-08 13:48:09 -06:00
Freddy 62facc1a04
Revert "Avoid adding original_dst filter when not needed" (#10365) 2021-06-08 13:18:41 -06:00
Daniel Nephin 85ffcdb8db docs: move streaming docs to blocking query page 2021-06-08 14:17:53 -04:00
Daniel Nephin 6c3f1eaeac fixup! docs: Add streaming to api features 2021-06-08 13:13:16 -04:00
Daniel Nephin 0f6ad6fd5c docs: try to improve health api doc terminology 2021-06-08 13:10:32 -04:00
Daniel Nephin d4b077174c Document streaming on service health endpoint 2021-06-08 13:10:32 -04:00
Daniel Nephin 789478b542 docs: Add streaming to api features 2021-06-08 13:10:32 -04:00
Daniel Nephin b8717966f1 submatview: and Store integration test with stream backend 2021-06-08 12:15:35 -04:00
Brian Flad 979199696b Migrate from GitHub HashiBot to GitHub Actions
GitHub HashiBot is being decommissioned. This migrates the crash issue labeling and waiting-reply label removal to GitHub Actions. The stale issue handling is not migrated as neither behavior appears to be actively in use.
2021-06-08 12:14:46 -04:00
Mike Morris dd69252a18 website: bump latest binary version to 1.9.6 2021-06-08 08:35:21 -04:00
Mike Morris 05a859ed27 changelog: add entries for 1.8.11, 1.8.12, 1.9.6 and 1.10.0-beta3 2021-06-08 08:35:21 -04:00
Brandon Romano 12c813de38
Merge pull request #10333 from hashicorp/ks.website/updates-alert-banner
chore: updates alert banner
2021-06-08 04:20:44 -07:00
Daniel Nephin 8fbbe036e0
Merge pull request #10355 from hashicorp/dnephin/stream-fix-topic-head-bug
stream: fix a bug with subscriptions and the latest item in the topic buffers.
2021-06-07 17:27:57 -04:00
Daniel Nephin 20f7a72792 stream: remove bufferItem.NextLink
Both NextLink and NextNoBlock had the same logic, with slightly
different return values. By adding a bool return value (similar to map
lookups) we can remove the duplicate method.
2021-06-07 17:04:46 -04:00
Dhia Ayachi e3dd0f9a44
generate a single debug file for a long duration capture (#10279)
* debug: remove the CLI check for debug_enabled

The API allows collecting profiles even debug_enabled=false as long as
ACLs are enabled. Remove this check from the CLI so that users do not
need to set debug_enabled=true for no reason.

Also:
- fix the API client to return errors on non-200 status codes for debug
  endpoints
- improve the failure messages when pprof data can not be collected

Co-Authored-By: Dhia Ayachi <dhia@hashicorp.com>

* remove parallel test runs

parallel runs create a race condition that fail the debug tests

* snapshot the timestamp at the beginning of the capture

- timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures
- capture append to the file if it already exist

* Revert "snapshot the timestamp at the beginning of the capture"

This reverts commit c2d03346

* Refactor captureDynamic to extract capture logic for each item in a different func

* snapshot the timestamp at the beginning of the capture

- timestamp used to create the capture sub folder is snapshot only at the beginning of the capture and reused for subsequent captures
- capture append to the file if it already exist

* Revert "snapshot the timestamp at the beginning of the capture"

This reverts commit c2d03346

* Refactor captureDynamic to extract capture logic for each item in a different func

* extract wait group outside the go routine to avoid a race condition

* capture pprof in a separate go routine

* perform a single capture for pprof data for the whole duration

* add missing vendor dependency

* add a change log and fix documentation to reflect the change

* create function for timestamp dir creation and simplify error handling

* use error groups and ticker to simplify interval capture loop

* Logs, profile and traces are captured for the full duration. Metrics, Heap and Go routines are captured every interval

* refactor Logs capture routine and add log capture specific test

* improve error reporting when log test fail

* change test duration to 1s

* make time parsing in log line more robust

* refactor log time format in a const

* test on log line empty the earliest possible and return

Co-authored-by: Freddy <freddygv@users.noreply.github.com>

* rename function to captureShortLived

* more specific changelog

Co-authored-by: Paul Banks <banks@banksco.de>

* update documentation to reflect current implementation

* add test for behavior when invalid param is passed to the command

* fix argument line in test

* a more detailed description of the new behaviour

Co-authored-by: Paul Banks <banks@banksco.de>

* print success right after the capture is done

* remove an unnecessary error check

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* upgraded github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57 => v0.0.0-20210601050228-01bbb1931b22

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
Co-authored-by: Paul Banks <banks@banksco.de>
2021-06-07 13:00:51 -04:00
allisaurus f2c2809612
docs: Improve ECS routing example nesting (#10316) 2021-06-07 09:28:06 -07:00
Dhia Ayachi 60d51a50d1
fix monitor to only start the monitor in json format when requested (#10358)
* fix monitor to only start the monitor in json format when requested

* add release notes

* add test to validate json format when requested
2021-06-07 12:08:48 -04:00
Mark Anderson ce52d3502c
Docs for Unix Domain Sockets (#10252)
* Docs for Unix Domain Sockets

There are a number of cases where a user might wish to either 1)
expose a service through a Unix Domain Socket in the filesystem
('downstream') or 2) connect to an upstream service by a local unix
domain socket (upstream).
As of Consul (1.10-beta2) we've added new syntax and support to configure
the Envoy proxy to support this
To connect to a service via local Unix Domain Socket instead of a
port, add local_bind_socket_path and optionally local_bind_socket_mode
to the upstream config for a service:
    upstreams = [
      {
         destination_name = "service-1"
         local_bind_socket_path = "/tmp/socket_service_1"
         local_bind_socket_mode = "0700"
	 ...
      }
      ...
    ]
This will cause Envoy to create a socket with the path and mode
provided, and connect that to service-1
The mode field is optional, and if omitted will use the default mode
for Envoy. This is not applicable for abstract sockets. See
https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/address.proto#envoy-v3-api-msg-config-core-v3-pipe
for details
NOTE: These options conflict the local_bind_socket_port and
local_bind_socket_address options. We can bind to an port or we can
bind to a socket, but not both.
To expose a service listening on a Unix Domain socket to the service
mesh use either the 'socket_path' field in the service definition or the
'local_service_socket_path' field in the proxy definition. These
fields are analogous to the 'port' and 'service_port' fields in their
respective locations.
    services {
      name = "service-2"
      socket_path = "/tmp/socket_service_2"
      ...
    }
OR
    proxy {
      local_service_socket_path = "/tmp/socket_service_2"
      ...
    }
There is no mode field since the service is expected to create the
socket it is listening on, not the Envoy proxy.
Again, the socket_path and local_service_socket_path fields conflict
with address/port and local_service_address/local_service_port
configuration entries.
Set up a simple service mesh with dummy services:
socat -d UNIX-LISTEN:/tmp/downstream.sock,fork UNIX-CONNECT:/tmp/upstream.sock
socat -v tcp-l:4444,fork exec:/bin/cat
services {
  name = "sock_forwarder"
  id = "sock_forwarder.1"
  socket_path = "/tmp/downstream.sock"
  connect {
    sidecar_service {
      proxy {
	upstreams = [
	  {
	    destination_name = "echo-service"
	    local_bind_socket_path = "/tmp/upstream.sock"
	    config {
	      passive_health_check {
		interval = "10s"
		max_failures = 42
	      }
	    }
	  }
	]
      }
    }
  }
}
services {
  name = "echo-service"
  port = 4444
  connect = { sidecar_service {} }
Kind = "ingress-gateway"
Name = "ingress-service"
Listeners = [
 {
   Port = 8080
   Protocol = "tcp"
   Services = [
     {
       Name = "sock_forwarder"
     }
   ]
 }
]
consul agent -dev -enable-script-checks -config-dir=./consul.d
consul connect envoy -sidecar-for sock_forwarder.1
consul connect envoy -sidecar-for echo-service -admin-bind localhost:19001
consul config write ingress-gateway.hcl
consul connect envoy -gateway=ingress -register -service ingress-service -address '{{ GetInterfaceIP "eth0" }}:8888' -admin-bind localhost:19002
netcat 127.0.0.1 4444
netcat 127.0.0.1 8080

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* fixup Unix capitalization

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Update website/content/docs/connect/registration/service-registration.mdx

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

* Provide examples in hcl and json

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>

* One more fixup for docs

Signed-off-by: Mark Anderson <manderson@hashicorp.com>

Co-authored-by: Blake Covarrubias <blake@covarrubi.as>
2021-06-04 18:54:31 -07:00
Jeff Escalante dbd278c09c
rotate algolia api key (#10297) 2021-06-04 19:54:05 -04:00
Daniel Nephin 48f388f590 stream: fix a bug with creating a snapshot
The head of the topic buffer was being ignored when creating a snapshot. This commit fixes
the bug by ensuring that the head of the topic buffer is included in the snapshot
before handing it off to the subscription.
2021-06-04 18:33:04 -04:00
Matt Keeler 42007d4a94
Add license inspect command documentation and changelog (#10351)
Also reformatted another changelog entry.
2021-06-04 14:33:13 -04:00
Daniel Nephin 1766aa8a84
Merge pull request #10348 from hashicorp/dnephin/fix-submatview-store-bug
submatview: fix a bug with Store.Get
2021-06-04 12:04:29 -04:00
Daniel Nephin 4fb6c5a137 submatview: fix a bug with Store.Get
When info.Timeout is 0, it should have no timeout. Previously it was using a 0 duration timeout
which caused it to return without waiting.

This bug was masked by using a timeout in the tests. Removing the timeout caused the tests to fail.
2021-06-03 17:48:44 -04:00
Matt Keeler 65b8929acf
Follow on to PR 10336 (#10343)
There was some PR feedback that came in just after I merged that other PR. This addresses that feedback.
2021-06-03 12:29:41 -04:00
Daniel Nephin 58116be6ca
Merge pull request #10338 from hashicorp/dnephin/fix-logging-indent
agent: remove leading whitespace from agent log lines
2021-06-03 12:16:45 -04:00
Paul Ewing e454a9aae0
usagemetrics: add cluster members to metrics API (#10340)
This PR adds cluster members to the metrics API. The number of members per
segment are reported as well as the total number of members.

Tested by running a multi-node cluster locally and ensuring the numbers were
correct. Also added unit test coverage to add the new expected gauges to
existing test cases.
2021-06-03 08:25:53 -07:00