Commit Graph

1420 Commits

Author SHA1 Message Date
R.B. Boyer 0537922c6c
connect/ca: require new vault mount points when updating the key type/bits for the vault connect CA provider (#10331)
progress on #9572
2021-07-13 11:11:46 -05:00
Daniel Nephin 58cf5767a8
Merge pull request #10479 from hashicorp/dnephin/ca-provider-explore-2
ca: move Server.SignIntermediate to CAManager
2021-07-12 19:03:43 -04:00
Daniel Nephin a22bdb2ac9
Merge pull request #10445 from hashicorp/dnephin/ca-provider-explore
ca: isolate more of the CA logic in CAManager
2021-07-12 15:26:23 -04:00
Daniel Nephin fdb0ba8041 ca: use provider constructors to be more consistent
Adds a contructor for the one provider that did not have one.
2021-07-12 14:04:34 -04:00
Dhia Ayachi 3eac4ffda4 check error when `raftApplyMsgpack` 2021-07-12 13:42:51 -04:00
Daniel Nephin 34c8585b29 auto-config: move autoConfigBackend impl off of Server
Most of these methods are used exclusively for the AutoConfig RPC
endpoint. This PR uses a pattern that we've used in other places as an
incremental step to reducing the scope of Server.
2021-07-12 13:42:40 -04:00
Daniel Nephin 605275b4dc ca: move SignCertificate to the file where it is used 2021-07-12 13:42:39 -04:00
Daniel Nephin c2e85f25d4 ca: move SignCertificate to CAManager
To reduce the scope of Server, and keep all the CA logic together
2021-07-12 13:42:39 -04:00
Dhia Ayachi a0320169fe add missing state reset when stopping ca manager 2021-07-12 09:32:36 -04:00
Daniel Nephin 68d5f7769a ca: fix mockCAServerDelegate to work with the new interface
raftApply was removed so ApplyCARequest needs to handle all the possible operations

Also set the providerShim to use the mock provider.

other changes are small test improvements that were necessary to debug the failures.
2021-07-12 09:32:36 -04:00
Daniel Nephin 6d4b0ce194 ca: remove unused method
and small refactor to getCAProvider so that GoLand is less confused about what it is doing.
Previously it was reporting that the for condition was always true, which was not the case.
2021-07-12 09:32:35 -04:00
Daniel Nephin 4330122d9a ca: remove raftApply from delegate interface
After moving ca.ConsulProviderStateDelegate into the interface we now
have the ApplyCARequest method which does the same thing. Use this more
specific method instead of raftApply.
2021-07-12 09:32:35 -04:00
Daniel Nephin fae0a8f851 ca: move generateCASignRequest to the delegate
This method on Server was only used by the caDelegateWithState, so move it there
until we can move it entirely into CAManager.
2021-07-12 09:32:35 -04:00
Daniel Nephin d4bb9fd97a ca: move provider creation into CAManager
This further decouples the CAManager from Server. It reduces the interface between them and
removes the need for the SetLogger method on providers.
2021-07-12 09:32:33 -04:00
Daniel Nephin fc629d9eaa ca-manager: move provider shutdown into CAManager
Reducing the coupling between Server and CAManager
2021-07-12 09:27:28 -04:00
Daniel Nephin 1e23d181b5 config: remove misleading UseTLS field
This field was documented as enabling TLS for outgoing RPC, but that was not the case.
All this field did was set the use_tls serf tag.

Instead of setting this field in a place far from where it is used, move the logic to where
the serf tag is set, so that the code is much more obvious.
2021-07-09 19:01:45 -04:00
Daniel Nephin 3c60a46376 config: remove duplicate TLSConfig fields from agent/consul.Config
tlsutil.Config already presents an excellent structure for this
configuration. Copying the runtime config fields to agent/consul.Config
makes code harder to trace, and provides no advantage.

Instead of copying the fields around, use the tlsutil.Config struct
directly instead.

This is one small step in removing the many layers of duplicate
configuration.
2021-07-09 18:49:42 -04:00
Evan Culver 5ff191ad99
Add support for returning ACL secret IDs for accessors with acl:write (#10546) 2021-07-08 15:13:08 -07:00
Dhia Ayachi e5dbf5e55b
Add ca certificate metrics (#10504)
* add intermediate ca metric routine

* add Gauge config for intermediate cert

* Stop metrics routine when stopping leader

* add changelog entry

* updage changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* use variables instead of a map

* go imports sort

* Add metrics for primary and secondary ca

* start metrics routine in the right DC

* add telemetry documentation

* update docs

* extract expiry fetching in a func

* merge metrics for primary and secondary into signing ca metric

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-07-07 09:41:01 -04:00
Daniel Nephin 14527dd005
Merge pull request #10552 from hashicorp/dnephin/ca-remove-rotation-period
ca: remove unused RotationPeriod field
2021-07-06 18:49:33 -04:00
jkirschner-hashicorp 31bbab8ae7
Merge pull request #10560 from jkirschner-hashicorp/change-sane-to-reasonable
Replace use of 'sane' where appropriate
2021-07-06 11:46:04 -04:00
Daniel Nephin b4a10443d1 ca: remove unused RotationPeriod field
This field was never used. Since it is persisted as part of a map[string]interface{} it
is pretty easy to remove it.
2021-07-05 19:15:44 -04:00
Jared Kirschner 4c3b1b8b7b Replace use of 'sane' where appropriate
HashiCorp voice, style, and language guidelines recommend avoiding ableist
language unless its reference to ability is accurate in a particular use.
2021-07-02 12:18:46 -04:00
Dhia Ayachi b57cf27e8f
Format certificates properly (rfc7468) with a trailing new line (#10411)
* trim carriage return from certificates when inserting rootCA in the inMemDB

* format rootCA properly when returning the CA on the connect CA endpoint

* Fix linter warnings

* Fix providers to trim certs before returning it

* trim newlines on write when possible

* add changelog

* make sure all provider return a trailing newline after the root and intermediate certs

* Fix endpoint to return trailing new line

* Fix failing test with vault provider

* make test more robust

* make sure all provider return a trailing newline after the leaf certs

* Check for suffix before removing newline and use function

* Add comment to consul provider

* Update change log

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix typo

* simplify code callflow

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* extract requireNewLine as shared func

* remove dependency to testify in testing file

* remove extra newline in vault provider

* Add cert newline fix to envoy xds

* remove new line from mock provider

* Remove adding a new line from provider and fix it when the cert is read

* Add a comment to explain the fix

* Add missing for leaf certs

* fix missing new line

* fix missing new line in leaf certs

* remove extra new line in test

* updage changelog

Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>

* fix in vault provider and when reading cache (RPC call)

* fix AWS provider

* fix failing test in the provider

* remove comments and empty lines

* add check for empty cert in test

* fix linter warnings

* add new line for leaf and private key

* use string concat instead of Sprintf

* fix new lines for leaf signing

* preallocate slice and remove append

* Add new line to `SignIntermediate` and `CrossSignCA`

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-06-30 20:48:29 -04:00
Daniel Nephin 86244967c5 docs: correct some misleading telemetry docs
The query metrics are actually reported for all read queries, not only
ones that use a MinIndex to block for updates.

Also clarify the raft.apply metric is only on the leader.
2021-06-28 12:20:53 -04:00
R.B. Boyer c3d5a2a5ab
connect/ca: cease including the common name field in generated certs (#10424)
As part of this change, we ensure that the SAN extensions are marked as
critical when the subject is empty so that AWS PCA tolerates the loss of
common names well and continues to function as a Connect CA provider.

Parts of this currently hack around a bug in crypto/x509 and can be
removed after https://go-review.googlesource.com/c/go/+/329129 lands in
a Go release.

Note: the AWS PCA tests do not run automatically, but the following
passed locally for me:

    ENABLE_AWS_PCA_TESTS=1 go test ./agent/connect/ca -run TestAWS
2021-06-25 13:00:00 -05:00
Daniel Nephin 72b30174fa ca: replace ca.PrimaryIntermediateProviders
With an optional interface that providers can use to indicate if they
use an intermediate cert in the primary DC.

This removes the need to look up the provider config when renewing the
intermediate.
2021-06-23 15:47:30 -04:00
Daniel Nephin f4c1f982d1
Merge pull request #9924 from hashicorp/dnephin/cert-expiration-metric
connect: emit a metric for the seconds until root CA expiry
2021-06-18 14:18:55 -04:00
Daniel Nephin e36800cefa Update metric name
and handle the case where there is no active root CA.
2021-06-14 17:01:16 -04:00
Daniel Nephin 548796ae13 connect: emit a metric for the number of seconds until root CA expiration 2021-06-14 16:57:01 -04:00
Freddy f399fd2add
Rename CatalogDestinationsOnly (#10397)
CatalogDestinationsOnly is a passthrough that would enable dialing
addresses outside of Consul's catalog. However, when this flag is set to
true only _connect_ endpoints for services can be dialed.

This flag is being renamed to signal that non-Connect endpoints can't be
dialed by transparent proxies when the value is set to true.
2021-06-14 14:15:09 -06:00
Freddy 61ae2995b7
Add flag for transparent proxies to dial individual instances (#10329) 2021-06-09 14:34:17 -06:00
Daniel Nephin 20f7a72792 stream: remove bufferItem.NextLink
Both NextLink and NextNoBlock had the same logic, with slightly
different return values. By adding a bool return value (similar to map
lookups) we can remove the duplicate method.
2021-06-07 17:04:46 -04:00
Daniel Nephin 48f388f590 stream: fix a bug with creating a snapshot
The head of the topic buffer was being ignored when creating a snapshot. This commit fixes
the bug by ensuring that the head of the topic buffer is included in the snapshot
before handing it off to the subscription.
2021-06-04 18:33:04 -04:00
Paul Ewing e454a9aae0
usagemetrics: add cluster members to metrics API (#10340)
This PR adds cluster members to the metrics API. The number of members per
segment are reported as well as the total number of members.

Tested by running a multi-node cluster locally and ensuring the numbers were
correct. Also added unit test coverage to add the new expected gauges to
existing test cases.
2021-06-03 08:25:53 -07:00
Daniel Nephin 0dfb7da610 grpc: fix a data race by using a static resolver
We have seen test flakes caused by 'concurrent map read and map write', and the race detector
reports the problem as well (prevent us from running some tests with -race).

The root of the problem is the grpc expects resolvers to be registered at init time
before any requests are made, but we were using a separate resolver for each test.

This commit introduces a resolver registry. The registry is registered as the single
resolver for the consul scheme. Each test uses the Authority section of the target
(instead of the scheme) to identify the resolver that should be used for the test.
The scheme is used for lookup, which is why it can no longer be used as the unique
key.

This allows us to use a lock around the map of resolvers, preventing the data race.
2021-06-02 11:35:38 -04:00
Dhia Ayachi 0c13f80d5a
RPC Timeout/Retries account for blocking requests (#8978) 2021-05-27 17:29:43 -04:00
Matt Keeler 7e4ea16149 Move some things around to allow for license updating via config reload
The bulk of this commit is moving the LeaderRoutineManager from the agent/consul package into its own package: lib/gort. It also got a renaming and its Start method now requires a context. Requiring that context required updating a whole bunch of other places in the code.
2021-05-25 09:57:50 -04:00
Matt Keeler 58b934133d hcs-1936: Prepare for adding license auto-retrieval to auto-config in enterprise 2021-05-24 13:20:30 -04:00
Matt Keeler 82f5cb3f08 Preparation for changing where license management is done. 2021-05-24 10:19:31 -04:00
Daniel Nephin f2cf586414 Refactor of serf feature flag tags.
This refactor is to make it easier to see how serf feature flags are
encoded as serf tags, and where those feature flags are read.

- use constants for both the prefix and feature flag name. A constant
  makes it much easier for an IDE to locate the read and write location.
- isolate the feature-flag encoding logic in the metadata package, so
  that the feature flag prefix can be unexported. Only expose a function
  for encoding the flags into tags. This logic is now next to the logic
  which reads the tags.
- remove the duplicate `addEnterpriseSerfTags` functions. Both Client
  and Server structs had the same implementation. And neither
  implementation needed the method receiver.
2021-05-20 12:57:06 -04:00
R.B. Boyer b90877b440
server: ensure that central service config flattening properly resets the state each time (#10239)
The prior solution to call reply.Reset() aged poorly since newer fields
were added to the reply, but not added to Reset() leading serial
blocking query loops on the server to blend replies.

This could manifest as a service-defaults protocol change from
default=>http not reverting back to default after the config entry
reponsible was deleted.
2021-05-14 10:21:44 -05:00
Daniel Nephin 3dd951ab1e testing: don't run t.Parallel in a goroutine
TestACLEndpoint_Login_with_TokenLocality was reguardly being reported as failed even though
it was not failing. I took another look and I suspect it is because t.Parllel was being
called in a goroutine.

This would lead to strange behaviour which apparently confused the 'go test' runner.
2021-05-10 13:30:10 -04:00
Daniel Nephin c9ae72e72f
Merge pull request #10075 from hashicorp/dnephin/handle-raft-apply-errors
rpc: some cleanup of canRetry and ForwardRPC
2021-05-06 16:59:53 -04:00
Daniel Nephin 39d7d07922 state: reduce arguments to validateProposedConfigEntryInServiceGraph 2021-05-06 13:47:40 -04:00
Daniel Nephin 4905ac6f44 rpc: add tests for canRetry
Also accept an RPCInfo instead of interface{}. Accepting an interface
lead to a bug where the caller was expecting the arg to be the response
when in fact it was always passed the request. By accepting RPCInfo
it should indicate that this is actually the request value.

One caller of canRetry already passed an RPCInfo, the second handles
the type assertion before calling canRetry.
2021-05-06 13:30:07 -04:00
Daniel Nephin c38f4869ad rpc: remove unnecessary arg to ForwardRPC 2021-05-06 13:30:07 -04:00
Daniel Nephin 55f620d636
Merge pull request #10155 from hashicorp/dnephin/config-entry-remove-fields
config-entry: remove Kind and Name field from Mesh config entry
2021-05-04 17:27:56 -04:00
Daniel Nephin 0e5e1270b6 config-entries: add a test for the API client
Also fixes a bug with listing kind=mesh config entries. ValidateConfigEntryKind was only being used by
the List endpoint, and was yet another place where we have to enumerate all the kinds.

This commit removes ValidateConfigEntryKind and uses MakeConfigEntry instead. This change removes
the need to maintain two separate functions at the cost of creating an instance of the config entry which will be thrown away immediately.
2021-05-04 17:14:21 -04:00
Daniel Nephin df98027ad1 lint: fix warning by removing reference to deprecated interface 2021-05-04 14:09:14 -04:00
Paul Banks d47eea3a3f
Make Raft trailing logs and snapshot timing reloadable (#10129)
* WIP reloadable raft config

* Pre-define new raft gauges

* Update go-metrics to change gauge reset behaviour

* Update raft to pull in new metric and reloadable config

* Add snapshot persistance timing and installSnapshot to our 'protected' list as they can be infrequent but are important

* Update telemetry docs

* Update config and telemetry docs

* Add note to oldestLogAge on when it is visible

* Add changelog entry

* Update website/content/docs/agent/options.mdx

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>

Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2021-05-04 15:36:53 +01:00
Luke Kysow eb84a856c4
Give descriptive error if auth method not found (#10163)
* Give descriptive error if auth method not found

Previously during a `consul login -method=blah`, if the auth method was not found, the
error returned would be "ACL not found". This is potentially confusing
because there may be many different ACLs involved in a login: the ACL of
the Consul client, perhaps the binding rule or the auth method.

Now the error will be "auth method blah not found", which is much easier
to debug.
2021-05-03 13:39:13 -07:00
Daniel Nephin bf4c289804 config-entry: remove Kind and Name field from Mesh config entry
No config entry needs a Kind field. It is only used to determine the Go type to
target. As we introduce new config entries (like this one) we can remove the kind field
and have the GetKind method return the single supported value.

In this case (similar to proxy-defaults) the Name field is also unnecessary. We always
use the same value. So we can omit the name field entirely.
2021-04-29 17:11:21 -04:00
Freddy 401f3010e0
Rename "cluster" config entry to "mesh" (#10127)
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.

Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
2021-04-28 16:13:29 -06:00
Daniel Nephin 3cda0a7cc4 health: create health.Client in Agent.New 2021-04-27 19:03:16 -04:00
Matt Keeler 6c639be8ec
Add prometheus guage definitions for replication metrics. (#10109) 2021-04-23 17:05:33 -04:00
Matt Keeler 09bf05ec5d
Add replication metrics (#10073) 2021-04-22 11:20:53 -04:00
Daniel Nephin 281d7616fa
Merge pull request #10045 from hashicorp/dnephin/state-proxy-defaults
state: remove config-entries kind index
2021-04-20 17:12:50 -04:00
Daniel Nephin 6d1a5b3629 Handle ErrChunkingResubmit.Error properly
Previously canRetry was attempting to retrieve this error from args, however there was never
any callers that would pass an error to args.

With the change to raftApply to move this error to the error return value, it is now possible
to receive this error from the err argument.

This commit updates canRetry to check for ErrChunkingResubmit in err.
2021-04-20 13:29:31 -04:00
Daniel Nephin 8654adfc53 Handle FSM.Apply errors in raftApply
Previously we were inconsistently checking the response for errors. This
PR moves the response-is-error check into raftApply, so that all callers
can look at only the error response, instead of having to know that
errors could come from two places.

This should expose a few more errors that were previously hidden because
in some calls to raftApply we were ignoring the response return value.

Also handle errors more consistently. In some cases we would log the
error before returning it. This can be very confusing because it can
result in the same error being logged multiple times. Instead return
a wrapped error.
2021-04-20 13:29:29 -04:00
Daniel Nephin 95b361ecc8 state: remove unnecessary kind index
The query can be performed using a prefix query on the ID index.

Also backport some enterprise changes to prevent conflicts.
2021-04-15 17:37:28 -04:00
Daniel Nephin eb7f4b7ea4 state: use index constants for ConfigEntry indexes 2021-04-15 17:30:07 -04:00
Freddy 5a9b75a443
Merge pull request #10016 from hashicorp/topology-update 2021-04-15 14:11:23 -06:00
R.B. Boyer c88512fe14
connect: update centralized upstreams representation in service-defaults (#10015) 2021-04-15 14:21:44 -05:00
Daniel Nephin 2a10f01bf5 snapshot: fix saving of auth methods
Previously only a single auth method would be saved to the snapshot. This commit fixes the typo
and adds to the test, to show that all auth methods are now saved.
2021-04-14 16:51:21 -04:00
freddygv 2ff8b9f2f5 Avoid returning a nil slice 2021-04-14 10:52:05 -06:00
Matt Keeler aa0eb60f57
Move static token resolution into the ACLResolver (#10013) 2021-04-14 12:39:35 -04:00
freddygv 7fd4c569ce Update viz endpoint to include topology from intentions 2021-04-14 10:20:15 -06:00
freddygv 83501d5415 Augment intention decision summary with DefaultAllow mode 2021-04-12 19:32:09 -06:00
freddygv eeccba945d Replace TransparentProxy bool with ProxyMode
This PR replaces the original boolean used to configure transparent
proxy mode. It was replaced with a string mode that can be set to:

- "": Empty string is the default for when the setting should be
defaulted from other configuration like config entries.
- "direct": Direct mode is how applications originally opted into the
mesh. Proxy listeners need to be dialed directly.
- "transparent": Transparent mode enables configuring Envoy as a
transparent proxy. Traffic must be captured and redirected to the
inbound and outbound listeners.

This PR also adds a struct for transparent proxy specific configuration.
Initially this is not stored as a pointer. Will revisit that decision
before GA.
2021-04-12 09:35:14 -06:00
Freddy 920ba3db39
Merge pull request #9976 from hashicorp/centralized-upstream-fixups 2021-04-08 12:26:56 -06:00
Daniel Nephin 93818ebc5a
Merge pull request #9950 from hashicorp/dnephin/state-use-txn-everywhere
state: use Txn interface everywhere
2021-04-08 12:02:03 -04:00
Daniel Nephin 9db8ffb1c5
Merge pull request #9880 from hashicorp/dnephin/catalog-events-test-pattern
state: use runCase pattern for large test
2021-04-08 11:54:41 -04:00
freddygv a1fd3b0271 Pass down upstream defaults to client proxies
This is needed in case the client proxy is in TransparentProxy mode.
Typically they won't have explicit configuration for every upstream, so
this ensures the settings can be applied to all of them when generating
xDS config.
2021-04-07 09:32:47 -06:00
freddygv c2e74e21bc Prevent requests without UpstreamIDs from being flagged as legacy.
New clients in transparent proxy mode can send requests for service
config resolution without any upstream args because they do not have
explicitly defined upstreams.

Old clients on the other hand will never send requests without the
Upstreams args unless they don't have upstreams, in which case we do not
send back upstream config.
2021-04-07 09:32:47 -06:00
R.B. Boyer 82245585c6
connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled (#9973)
This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.
2021-04-06 13:19:59 -05:00
Daniel Nephin f0590e7c18 state: support additional test cases in indexer tests
And add a few additional cases.
2021-03-31 14:39:33 -04:00
Kyle Havlovitz 3cdd495600 Backport enterprise changes to prevent merge conflicts
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-31 14:05:26 -04:00
Daniel Nephin e4a60a2a8d state: use tableIndex constant 2021-03-29 18:52:20 -04:00
Daniel Nephin 7cb2255838 state: use ReadTxn and WriteTxn interface
Instead of *txn, so that we can replace the txn implementation with others, and so
that the function is easily documented as a read or write function.
2021-03-29 18:52:16 -04:00
Daniel Nephin d785c86db1 state: convert checks.service index to new pattern 2021-03-29 16:38:53 -04:00
Daniel Nephin f859ba6d4b state: convert checks.status indexer
As part of this change the indexer will now be case insensitive by using
the lower case value. This should be safe because previously we always
had lower case strings.

This change was made out of convenience. All the other indexers use
lowercase, so we can re-use the indexFromQuery function by using
lowercase here as well.
2021-03-29 16:38:50 -04:00
Daniel Nephin 9251ac881a state: add tests for checks indexers 2021-03-29 16:38:47 -04:00
Daniel Nephin 98d6dcbdf8 state: use constants for table checks 2021-03-29 16:38:43 -04:00
Daniel Nephin bcbdc9cab3 state: pass Query in from caller
To reduce the number of arguments
2021-03-29 15:42:30 -04:00
Daniel Nephin 372d274b34 state: convert services.kind to functional indexer pattern 2021-03-29 15:42:30 -04:00
Daniel Nephin bcde8d2fad state: add tests for services.kind indexer 2021-03-29 15:42:27 -04:00
Daniel Nephin 9f9eadd569 state: convert services table service and connect indexer
To the new functional indexer pattern
2021-03-29 15:42:24 -04:00
Daniel Nephin 11311c1fcc state: add tests for services table service and connect indexers 2021-03-29 15:42:22 -04:00
Daniel Nephin 9a3daf3100 state: use constant for tableServices 2021-03-29 15:42:18 -04:00
Daniel Nephin ec04df66bd state: remove duplication of Query indexer 2021-03-29 14:35:11 -04:00
Daniel Nephin 28866e48ad state: remove duplication in acl tables schema 2021-03-29 14:21:27 -04:00
Daniel Nephin c6a1ca701d state: reduce duplication in catalog table schema 2021-03-29 14:21:23 -04:00
Daniel Nephin d9dacb8388 state: share more indexer functions for config_entries 2021-03-29 14:21:20 -04:00
Daniel Nephin f303120f2d state: remove old schema test
This test has been replaced by TestNewDBSchema_Indexers
2021-03-29 14:21:13 -04:00
Daniel Nephin 150decff2a state: use addNamespaceIndex again 2021-03-29 14:21:02 -04:00
Daniel Nephin 4a3b462c28
Merge pull request #9911 from hashicorp/dnephin/state-index-acl-roles
state: convert ACLRoles policies index to new functional indexer pattern
2021-03-24 18:28:19 -04:00
Daniel Nephin 25b791ba47 state: add tests for checks.ID indexer 2021-03-22 18:06:43 -04:00
Daniel Nephin abbe5c3701 state: use tx.First instead of tx.FirstWatch
Where appropriate. After removing the helper function a bunch of  these calls can
be changed to tx.First.
2021-03-22 18:06:33 -04:00
Daniel Nephin 49938bc472 state: convert checks.ID index to new pattern 2021-03-22 18:06:08 -04:00
Hans Hasselberg 052662bcf9
introduce certopts (#9606)
* introduce cert opts

* it should be using the same signer

* lint and omit serial
2021-03-22 10:16:41 +01:00
Daniel Nephin 1d3fe64bba state: use uuid for acl-roles.policies index
Previously we were encoding the UUID as a string, but the index it references uses a UUID
so this index can also use an encoded UUID to save a bit of memory.
2021-03-19 19:45:37 -04:00
Daniel Nephin 3c01bb1156 state: convert acl-roles.policies index to new pattern 2021-03-19 19:45:37 -04:00
Daniel Nephin 474e95b9f5 state: convert acl-roles.name index to the functional indexer pattern 2021-03-19 19:45:37 -04:00
Daniel Nephin f836ed256b state: add indexer tests for acl-roles table 2021-03-19 19:45:37 -04:00
Daniel Nephin 6bc2c0e1ce state: use constants for acl-roles table and indexes 2021-03-19 19:45:37 -04:00
Daniel Nephin d4e02024fe state: convert acl-policies table to new pattern 2021-03-19 15:24:00 -04:00
Daniel Nephin 845a10354e state: use constants and add tests for acl-policies table 2021-03-19 15:19:57 -04:00
Daniel Nephin f6533a08f8 state: add indexer test for services.ID index 2021-03-19 14:13:14 -04:00
Daniel Nephin 1d1c03d0cd state: handle wildcard for services.ID index
When listing services, use the id_prefix directly if wildcards are allowed.

Error if a wildcard is used for a query that does not index the wildcard
2021-03-19 14:12:19 -04:00
Daniel Nephin bae69b2352 state: fix prefix index with the new pattern
Prefix queries are generally being used to match part of a partial
index. We can support these indexes by using a function that accept
different types for each subset of the index.

What I found interesting is that in the generic StringFieldIndexer the
implementation for PrefixFromArgs would remove the trailing null, but
at least in these 2 cases we actually want a null terminated string.
We simply want fewer components in the string.
2021-03-19 14:12:17 -04:00
Daniel Nephin ec50454fb3 state: move services.ID to new pattern 2021-03-19 14:11:59 -04:00
Daniel Nephin f5a52a4501 state: add tests for gateway-service table indexers 2021-03-18 12:09:42 -04:00
Daniel Nephin 66632538d8 state: use constants and remove wrapping
for GatewayServices table
2021-03-18 12:08:59 -04:00
Daniel Nephin d77bdd26c5 state: Move UpstreamDownstream to state package 2021-03-18 12:08:59 -04:00
Daniel Nephin ca3686f4aa state: add tests for mesh-topology table indexers 2021-03-18 12:08:57 -04:00
Daniel Nephin 8a1a11814d state: use constants for mesh-topology table operations 2021-03-18 12:08:03 -04:00
Freddy 8ac9f2521b
Merge pull request #9900 from hashicorp/ent-fixes
Fixup enterprise tests from tproxy changes
2021-03-18 08:33:30 -06:00
Freddy 28c29e6ab4
Merge pull request #9899 from hashicorp/wildcard-ixn-oss
Add methods to check intention has wildcard src or dst
2021-03-18 08:33:07 -06:00
freddygv b56bd690aa Fixup enterprise tests from tproxy changes 2021-03-17 23:05:00 -06:00
freddygv 1c46470a29 Add methods to check intention has wildcard src or dst 2021-03-17 22:15:48 -06:00
freddygv 6c43195e2a Merge master and fix upstream config protocol defaulting 2021-03-17 21:13:40 -06:00
freddygv 0c8b618ca0 Temporarily silence spurious wakeup. Addressing false positive in beta. 2021-03-17 17:25:29 -06:00
freddygv 60690cf5c9 Merge remote-tracking branch 'origin/master' into intention-topology-endpoint 2021-03-17 17:14:38 -06:00
Freddy 63dcb7fa76
Add TransparentProxy option to proxy definitions 2021-03-17 17:01:45 -06:00
Freddy fb252e87a4
Add per-upstream configuration to service-defaults 2021-03-17 16:59:51 -06:00
freddygv 15a145b9f6 Add changelog and cleanup todo for beta 2021-03-17 16:45:13 -06:00
freddygv d19a5830dd Do not include consul as upstream or downstream 2021-03-17 13:40:04 -06:00
Daniel Nephin d2591312f8 state: add tests for config-entry indexers 2021-03-17 14:41:46 -04:00
Daniel Nephin 1b8f8b135e state: convert config-entries kind index to new pattern 2021-03-17 14:40:57 -04:00
Daniel Nephin bfcf463c3a state: remove config-entries namespace index
Use a prefix of the ID index instead.
2021-03-17 14:40:57 -04:00
Daniel Nephin dcbb1ba5dd state: remove unnecessary method receiver 2021-03-17 14:40:57 -04:00
Daniel Nephin b43977423f state: convert config-entries table to new indexer pattern
Using functional indexes to isolate enterprise differentiation and
remove reflection.
2021-03-17 14:40:57 -04:00
Daniel Nephin 98c32599e4
Merge pull request #9881 from hashicorp/dnephin/state-index-service-check-nodes
state: convert services.node and checks.node indexes
2021-03-17 14:12:02 -04:00
Daniel Nephin b771baa1f5
Merge pull request #9863 from hashicorp/dnephin/config-entry-kind-name
state: move ConfigEntryKindName
2021-03-17 14:09:39 -04:00
Daniel Nephin 0b3930272d state: convert services.node and checks.node indexes
Using NodeIdentity to share the indexes with both.
2021-03-16 13:00:31 -04:00
freddygv b79039c21c Prefix match type vars to match use 2021-03-16 09:49:24 -06:00
freddygv fed983fe9a Pass txn into service list queries 2021-03-16 09:33:08 -06:00
freddygv 26ba0c0fc8 Pass txn into intention match queries 2021-03-16 08:03:52 -06:00
freddygv d7f3bcc8bb Replace CertURI.Authorize() calls.
AuthorizeIntentionTarget is a generalized version of the old function,
and can be evaluated against sources or destinations.
2021-03-15 18:06:04 -06:00
freddygv eb6c0cbea0 Fixup typo, comments, and regression 2021-03-15 17:50:47 -06:00
freddygv 940b7a98d1 Finish cleanup from ServiceConfigRequest changes 2021-03-15 16:38:01 -06:00
Daniel Nephin 0b5dfee00a state: use runCase pattern for large test
The TestServiceHealthEventsFromChanges function was over 1400 lines.
Attempting to debug test failures in test functions this large is
difficult. It requires scrolling to the line which defines the testcase
because the failure message only includes the line number of the
assertion, not the line number of the test case.

This is an excellent example of where test tables stop working well, and
start being a problem. To mitigate this problem, the runCase pattern can
be used. When one of these tests fails, a failure message will print the
line number of both the test case and the assertion. This allows a
developer to quickly jump to both of the relevant lines, signficanting
reducing the time it takes to debug test failures.

For example, one such failure could look like this:

    catalog_events_test.go:1610: case: service reg, new node
    catalog_events_test.go:1605: assertion failed: values are not equal
2021-03-15 17:53:16 -04:00
freddygv 04fbc104cd Pass MeshGateway config in service config request
ResolveServiceConfig is called by service manager before the proxy
registration is in the catalog. Therefore we should pass proxy
registration flags in the request rather than trying to fetch
them from the state store (where they may not exist yet).
2021-03-15 14:32:13 -06:00
freddygv d90240d367 Restore old Envoy prefix on escape hatches
This is done because after removing ID and NodeName from
ServiceConfigRequest we will no longer know whether a request coming in
is for a Consul client earlier than v1.10.
2021-03-15 14:12:57 -06:00
freddygv 3b2169b36d Add RPC endpoint for intention upstreams 2021-03-15 08:50:35 -06:00
freddygv e4e14639b2 Add state store function for intention upstreams 2021-03-15 08:50:35 -06:00
freddygv 4976c000b7 Refactor IntentionDecision
This enables it to be called for many upstreams or downstreams of a
service while only querying intentions once.

Additionally, decisions are now optionally denied due to L7 permissions
being present. This enables the function to be used to filter for
potential upstreams/downstreams of a service.
2021-03-15 08:50:35 -06:00
Daniel Nephin 2a53b8293a proxycfg: use rpcclient/health.Client instead of passing around cache name
This should allow us to swap out the implementation with something other
than `agent/cache` without making further code changes.
2021-03-12 11:46:04 -05:00
Daniel Nephin c33570be34 catalog_events: set the right key for connect snapshots 2021-03-12 11:35:43 -05:00