Commit Graph

15728 Commits

Author SHA1 Message Date
Daniel Upton caa5b5a5a6
xds: prefer fed state gateway definitions if they're fresher (#11522)
Fixes an issue described in #10132, where if two DCs are WAN federated
over mesh gateways, and the gateway in the non-primary DC is terminated
and receives a new IP address (as is commonly the case when running them
on ephemeral compute instances) the primary DC is unable to re-establish
its connection until the agent running on its own gateway is restarted.

This was happening because we always preferred gateways discovered by
the `Internal.ServiceDump` RPC (which would fail because there's no way
to dial the remote DC) over those discovered in the federation state,
which is replicated as long as the primary DC's gateway is reachable.
2021-11-09 16:45:36 +00:00
Freddy 0ad360fadf
Merge pull request #11514 from hashicorp/dnephin/ca-fix-secondary-init
ca: properly handle the case where the secondary initializes after the primary
2021-11-08 17:16:16 -07:00
freddygv 956da278ff Add changelog entry 2021-11-08 17:14:27 -07:00
freddygv e6622ab0ab Avoid returning empty roots with uninitialized CA
Currently getCARoots could return an empty object with an empty trust
domain before the CA is initialized. This commit returns an error while
there is no CA config or no trust domain.

There could be a CA config and no trust domain because the CA config can
be created in InitializeCA before initialization succeeds.
2021-11-08 16:51:49 -07:00
Dhia Ayachi f61892393f
refactor session state store tables to use the new index pattern (#11525)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* partition `tableSessions` table

* fix sessions to use UUID and fix prefix index

* fix oss build

* clean up unused functions

* fix oss compilation

* add a partition indexer for sessions

* Fix oss to not have partition index

* fix oss tests

* remove unused func `prefixIndexFromServiceNameAsString`

* fix test error check

* remove unused operations_ent.go and operations_oss.go func

* remove unused const

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-11-08 16:20:50 -05:00
Dhia Ayachi dfafd4e38c
KV refactoring, part 2 (#11512)
* add partition to the kv get pretty print

* fix failing test

* add test for kvs RPC endpoint
2021-11-08 11:43:21 -05:00
Dhia Ayachi 17190c0076
KV state store refactoring and partitioning (#11510)
* state: port KV and Tombstone tables to new pattern

* go fmt'ed

* handle wildcards for tombstones

* Fix graveyard ent vs oss

* fix oss compilation error

* add partition to tombstones and kv state store indexes

* refactor to use `indexWithEnterpriseIndexable`

* partition kvs indexID table

* add `partitionedIndexEntryName` in oss for test purpose

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add `singleValueID` implementation assertions

* remove entmeta reference from oss

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2021-11-08 09:35:56 -05:00
Freddy 0344f3579b
Fix caveat about resolvers operating at L4 (#11497)
Service resolvers can specify L4 rules such as redirects, or L7 rules such as
hash-based load balancing policies.
2021-11-08 07:11:36 -07:00
David Yu 0bd182feb7
docs: add `brew install hashicorp/tap/consul-k8s` and re-order install and uninstall workflows (#11489)
* docs: add `brew install hashicorp/tap/consul-k8s`

* add consul k8s cli brew install to reference

* Update k8s-cli.mdx

* split home-brew commands into two steps

* Update k8s-cli.mdx

* slight changes on recommended way of installing Consul K8s for CLI or multi-DC

* Update install.mdx

* reorder cli and helm uninstall

* Update website/content/docs/k8s/installation/install.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/k8s/installation/install.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/k8s/installation/install.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/k8s/k8s-cli.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/k8s/installation/install.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* Update website/content/docs/k8s/k8s-cli.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2021-11-05 16:33:11 -07:00
Daniel Nephin 69ad7c0544 ca: Only initialize clusterID in the primary
The secondary must get the clusterID from the primary
2021-11-05 18:08:44 -04:00
Daniel Nephin 3173582b75 ca: return an error when secondary fails to initialize
Previously secondaryInitialize would return nil in this case, which prevented the
deferred initialize from happening, and left the CA in an uninitialized state until a config
update or root rotation.

To fix this I extracted the common parts into the delegate implementation. However looking at this
again, it seems like the handling in secondaryUpdateRoots is impossible, because that function
should never be called before the secondary is initialzied. I beleive we can remove some of that
logic in a follow up.
2021-11-05 18:02:51 -04:00
R.B. Boyer 0eb175caee
update changelog (#11513) 2021-11-05 17:00:19 -05:00
Evan Culver b0b6866179
ci: Loosen apt source verification (#11493) 2021-11-05 16:38:23 -04:00
Daniel Nephin c475c4708f
Merge pull request #11415 from hashicorp/dnephin/acl-policy-id-and-revision
acl: remove Policy.ID and Policy.Revision
2021-11-05 16:01:53 -04:00
Daniel Nephin db29ad346b acl: remove id and revision from Policy constructors
The fields were removed in a previous commit.

Also remove an unused constructor for PolicyMerger
2021-11-05 15:45:08 -04:00
Daniel Nephin 617b11302f acl: remove Policy.ID and Policy.Revision
These two fields do not appear to be used anywhere. We use the structs.ACLPolicy ID in the
ACLResolver cache, but the acl.Policy ID and revision are not used.
2021-11-05 15:43:52 -04:00
R.B. Boyer 1d8e7bb565
rename helper method to reflect the non-deprecated terminology (#11509) 2021-11-05 13:51:50 -05:00
Luke Kysow c010fa076d
Document default storage class requirements (#11492) 2021-11-05 11:27:49 -07:00
Kenia 24d8236219
ui: Topology view with no dependencies (#11280) 2021-11-05 13:46:41 -04:00
Connor b3af482e09
Support Vault Namespaces explicitly in CA config (#11477)
* Support Vault Namespaces explicitly in CA config

If there is a Namespace entry included in the Vault CA configuration,
set it as the Vault Namespace on the Vault client

Currently the only way to support Vault namespaces in the Consul CA
config is by doing one of the following:
1) Set the VAULT_NAMESPACE environment variable which will be picked up
by the Vault API client
2) Prefix all Vault paths with the namespace

Neither of these are super pleasant. The first requires direct access
and modification to the Consul runtime environment. It's possible and
expected, not super pleasant.

The second requires more indepth knowledge of Vault and how it uses
Namespaces and could be confusing for anyone without that context. It
also infers that it is not supported

* Add changelog

* Remove fmt.Fprint calls

* Make comment clearer

* Add next consul version to website docs

* Add new test for default configuration

* go mod tidy

* Add skip if vault not present

* Tweak changelog text
2021-11-05 11:42:28 -05:00
FFMMM 573ea1a95d
change vault ca docs to mention root cert ttl config (#11488)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-11-04 15:44:22 -07:00
R.B. Boyer 7fbf749bc4
segments: ensure that the serf_lan_allowed_cidrs applies to network segments (#11495) 2021-11-04 17:17:19 -05:00
R.B. Boyer b7932cc39c
add changelog entry (#11494) 2021-11-04 16:52:54 -05:00
Mark Anderson e9a0fa7d36
Remove some usage of md5 from the system (#11491)
* Remove some usage of md5 from the system

OSS side of https://github.com/hashicorp/consul-enterprise/pull/1253

This is a potential security issue because an attacker could conceivably manipulate inputs to cause persistence files to collide, effectively deleting the persistence file for one of the colliding elements.

Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-11-04 13:07:54 -07:00
FFMMM 9afecfa10c
plumb thru root cert tll to the aws ca provider (#11449)
* plumb thru root cert ttl to the aws ca provider

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11449.txt

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>

Co-authored-by: Dhia Ayachi <dhia@hashicorp.com>
2021-11-04 12:19:08 -07:00
John Cowen e5e65b71ad
ui: Only show partition / partition selector if enabled (#11484)
I missed that partitions should never display if you can't use them, not just if you can't choose them.
2021-11-04 12:14:43 +00:00
John Cowen 599e8a05d3
ui: Ensure we check intention service prefix permissions for per service (#11409)
Port of: Ensure we check intention service prefix permissions for per service (#11270)

Previously, when showing some action buttons for 'per service intentions' we used a global 'can I do something with any intention' permission to decide whether to show a certain button or not. If a user has a token that does not have 'global' intention permissions, but does have intention permissions on one or more specific services (for example via service / service_prefix), this meant that we did not show them certain buttons required to create/edit the intentions for this specific service.

This PR adds that extra permissions check so we now check the intentions permissions per service instead of using the 'global' "can I edit intentions" question/request.

**Notes:**

- If a HTML button is `disabled` this means tippy.js doesn't adopt the
popover properly and subsequently hide it from the user, so aswell as
just disabling the button so you can't active the popover, we also don't
even put the popover on the page
- If `ability.item` or `ability.item.Resources` are empty then assume no access

**We don't try to disable service > right hand side intention actions here**

Whether you can create intentions for a service depends on the
_destination_ of the intention you would like to create. For the
topology view going from the LHS to the center, this is straightforwards
as we only need to know the permissions for the central service, as when
you are going from the LHS to the center, the center is the
_destination_.

When going from the center to the RHS the _destination[s]_ are on the
RHS. This means we need to know the permissions for potentially 1000s of
services all in one go in order to know when to show a button or not.

We can't realistically discover the permissions for service > RHS
services as we'd have either make a HTTP request per right hand service,
or potentially make an incredibly large POST request for all the
potentially 1000s of services on the right hand side (more preferable to
1000s of HTTP requests).

Therefore for the moment at least we keep the old functionality (thin client)
for the middle to RHS here. If you do go to click on the button and you
don't have permissions to update the intention you will still not be
able to update it, only you won't know this until you click the button
(at which point you'll get a UI visible 403 error)

Note: We reversed the conditional here between 1.10 and 1.11

So this make 100% sense that the port is different here to 1.11
2021-11-04 12:10:28 +00:00
David Yu a96a035973
docs: consul-k8s uninstall with namespace (#11478)
* docs:  consul-k8s uninstall with namespace

Uninstall with namespace

* change release name to consul in uninstall

* Update website/content/docs/k8s/operations/uninstall.mdx

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* add --create-namespace command to install for custom values file

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
2021-11-03 16:48:48 -07:00
Kevin Wang c5ecafdfde
chore: react-subnav (#11486) 2021-11-03 17:05:12 -04:00
Daniel Nephin 7a9d936a70
Merge pull request #10822 from hashicorp/krarey/patch-replication-policy
Update namespaced replication token example
2021-11-03 15:40:36 -04:00
Kyle Rarey 37e051ed36 Update namespaced replication token example 2021-11-03 15:33:30 -04:00
FFMMM e7ffef54ee
fix aws pca certs (#11470)
Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>
2021-11-03 12:21:24 -07:00
Thomas Eckert 4789e3a4d0
Update Helm Docs for v0.36.0 Consul K8s release (#11483)
Co-authored-by: David Yu <dyu@hashicorp.com>
2021-11-03 10:04:16 -07:00
Luke Kysow 0f8434a213
Add quick-link for users coming from UI (#11403)
The Consul UI topology view has an icon with the text
"Configure metrics dashboard" that links to this page. Add a notice at
the top of the page that links them directly to the relevant section.
2021-11-03 09:37:30 -07:00
Luke Kysow 56fd7e050a
Use "Open dashboard" everywhere (#11406)
Previously we had "Open metrics Dashboard" and "Configure metrics
dashboard" in the topology cards and then we had "Open Dashboard" in the
top nav when the dashboard was configured.

Now we use "Open dashboard" and "Configure dashboard".

This change was made for consistency in wording and casing. In addition,
the dashboard could be used for metrics but also other dashboards so
there's no need to scope it only to metrics. Also the config is:

```hcl
ui_config {
  dashboard_url_templates
}
```

Which does not mention metrics
2021-11-03 09:29:50 -07:00
Mathew Estafanous 508664440d
Convert (some) test endpoints to use ServeHTTP instead of direct calls to handlers. (#11445) 2021-11-03 11:12:36 -04:00
trujillo-adam 52086937b9
Merge pull request #11307 from hashicorp/docs/admin-partitions-cli-v1.11.0
admin partition docs part 1 - cli
2021-11-02 14:44:43 -07:00
Luke Kysow 6131a207cf
Remove Name/Namespace fields from upstream default (#11456)
The UpstreamConfig.Defaults field does not support setting Name or
Namespace because the purpose is to apply defaults to all upstreams.
I think this was just missed in the docs since those fields would
error if set under Defaults.

i.e. this is not supported:

```
UpstreamConfig {
  Defaults {
    Name = "foo"
    Namespace = "bar"
    # Defaults config here
  }
}
```
2021-11-02 14:21:15 -07:00
FFMMM 27227c0fd2
add root_cert_ttl option for consul connect, vault ca providers (#11428)
* add root_cert_ttl option for consul connect, vault ca providers

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>

* add changelog, pr feedback

Signed-off-by: FFMMM <FFMMM@users.noreply.github.com>

* Update .changelog/11428.txt, more docs

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* Update website/content/docs/agent/options.mdx

Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
Co-authored-by: Kyle Havlovitz <kylehav@gmail.com>
2021-11-02 11:02:10 -07:00
Daniel Nephin 0ec2a804df
Merge pull request #10690 from tarat44/h2c-support-in-ping-checks
add support for h2c in h2 ping health checks
2021-11-02 13:53:06 -04:00
Freddy a31d65b914
Prevent running build workflows on all PRs (#11469) 2021-11-01 17:29:45 -06:00
Daniel Nephin 00ed2b243f
Merge pull request #10771 from hashicorp/dnephin/emit-telemetry-metrics-immediately
telemetry: improve cert expiry metrics
2021-11-01 18:31:03 -04:00
Freddy 80e2aef31c
Merge pull request #11450 from hashicorp/ap/best-addr
Ensure calls to BestAddress consider partition
2021-11-01 15:44:41 -06:00
freddygv ecccf22fd7 Exclude default partition from GatewayKey string
This will behave the way we handle SNI and SPIFFE IDs, where the default
partition is excluded.

Excluding the default ensures that don't attempt to compare default.dc2
to dc2 in OSS.
2021-11-01 14:45:52 -06:00
freddygv d944e6ae3a Update GatewayKeys deduplication
Federation states data is only keyed on datacenter, so it cannot be
directly compared against keys for gateway groups.
2021-11-01 13:58:53 -06:00
freddygv ce43e8cf99 Store GatewayKey in proxycfg snapshot for re-use 2021-11-01 13:58:53 -06:00
freddygv 51c888a41a Update locality check in xds 2021-11-01 13:58:53 -06:00
freddygv 6657c88296 Update locality check in proxycfg 2021-11-01 13:58:53 -06:00
Peter M 632904ff41
adding K8s page to subnav (#11467)
* adding K8s page to subnav

per request from HLT, updating use case tab to lead to K8s page instead of service mesh.

* Update subnav.js
2021-11-01 12:41:55 -07:00
Melissa Kam 36ebca63ed
Merge pull request #11466 from hashicorp/cts-tls-typo
docs/nia: Fix typo in TLS configs for CTS
2021-11-01 14:23:15 -05:00