3f79aaf509
Cleanup unnecessary normalizing method ( #11169 )
2021-09-28 15:31:12 -04:00
4505cb2920
Refactor table index acl phase 2 ( #11133 )
...
* extract common methods from oss and ent
* remove unreachable code
* add missing normalize for binding rules
* fix oss to use Query
2021-09-23 15:26:09 -04:00
ebe333b947
Refactor table index ( #11131 )
...
* convert tableIndex to use the new pattern
* make `indexFromString` available for oss as well
* refactor `indexUpdateMaxTxn`
2021-09-23 11:06:23 -04:00
c87d57bfeb
partitions/authmethod-index work from enterprise ( #11056 )
...
* partitions/authmethod-index work from enterprise
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-22 13:19:20 -07:00
bc04a155fb
Merge pull request #11090 from hashicorp/clly/kv-usage-metrics
...
Add KVUsage to consul state usage metrics
2021-09-22 11:26:56 -05:00
bfe6b64ca7
Strip out go 1.17 bits
2021-09-22 11:04:48 -05:00
5493ff06cc
Merge pull request #10985 from hashicorp/dnephin/acl-legacy-remove-replication
...
acl: remove legacy ACL replication
2021-09-21 17:56:54 -04:00
64852cd3e5
Apply suggestions from code review
...
Co-authored-by: Matt Keeler <mkeeler@users.noreply.github.com>
2021-09-21 10:52:46 -05:00
698fc291a9
Add KVUsage to consul state usage metrics
...
This change will add the number of entries in the consul KV store to the
already existing usage metrics.
2021-09-20 12:41:54 -05:00
caafc1905e
Fixup test
2021-09-16 14:39:01 -06:00
8a9bf3748c
Account for partitions in ixn match/decision
2021-09-16 14:39:01 -06:00
dc549eca30
Default partition in match endpoint
2021-09-15 17:23:52 -06:00
08b222cfc3
ACL Binding Rules table partitioning ( #11044 )
...
* ACL Binding Rules table partitioning
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-15 13:26:08 -07:00
25ea1a9276
use const instead of literals for `tableIndex` ( #11039 )
2021-09-15 10:24:04 -04:00
ffe3806aaf
Refactor `indexAuthMethod` in `tableACLBindingRules` ( #11029 )
...
* Port consul-enterprise #1123 to OSS
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
* Fixup missing query field
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
* change to re-trigger ci system
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
2021-09-15 09:34:19 -04:00
4992218676
convert expiration indexed in ACLToken table to use `indexerSingle` ( #11018 )
...
* move intFromBool to be available for oss
* add expiry indexes
* remove dead code: `TokenExpirationIndex`
* fix remove indexer `TokenExpirationIndex`
* fix rebase issue
2021-09-13 14:37:16 -04:00
1f23bdf388
add locality indexer partitioning ( #11016 )
...
* convert `Roles` index to use `indexerSingle`
* split authmethod write indexer to oss and ent
* add index locality
* add locality unit tests
* move intFromBool to be available for oss
* use Bool func
* refactor `aclTokenList` to merge func
2021-09-13 11:53:00 -04:00
3638825db8
convert `indexAuthMethod` index to use `indexerSingle` ( #11014 )
...
* convert `Roles` index to use `indexerSingle`
* fix oss build
* split authmethod write indexer to oss and ent
* add auth method unit tests
2021-09-10 16:56:56 -04:00
82b30f8020
convert `Roles` index to use `indexerMulti` ( #11013 )
...
* convert `Roles` index to use `indexerMulti`
* add role test in oss
* fix oss to use the right index func
* preallocate slice
2021-09-10 16:04:33 -04:00
569e18d002
convert indexPolicies in ACLTokens table to the new index ( #11011 )
2021-09-10 14:57:37 -04:00
0d0edeec27
convert indexSecret to the new index ( #11007 )
2021-09-10 09:10:11 -04:00
f0cbe25ca6
convert indexAccessor to the new index ( #11002 )
2021-09-09 16:28:04 -04:00
3fb797382b
Sync enterprise changes to oss ( #10994 )
...
This commit updates OSS with files for enterprise-specific admin partitions feature work
2021-09-08 11:59:30 -04:00
96d7842118
partition dicovery chains ( #10983 )
...
* partition dicovery chains
* fix default partition for OSS
2021-09-07 16:29:32 -04:00
4dd5bb8e3b
acl: remove legacy ACL replication
2021-09-03 12:42:06 -04:00
efbdf7e117
api: expose upstream routing configurations in topology view ( #10811 )
...
Some users are defining routing configurations that do not have associated services. This commit surfaces these configs in the topology visualization. Also fixes a minor internal bug with non-transparent proxy upstream/downstream references.
2021-08-25 15:20:32 -04:00
b6be94e7fa
fixing various bits of enterprise meta plumbing to be more correct ( #10889 )
2021-08-20 14:34:23 -05:00
d730298f59
state: partition the nodes.uuid and nodes.meta indexes as well ( #10882 )
2021-08-19 16:17:59 -05:00
e565409c6a
state: partition the usage metrics subsystem ( #10867 )
2021-08-18 09:27:15 -05:00
1cef3c99c2
state: adjust streaming event generation to account for partitioned nodes ( #10860 )
...
Also re-enabled some tests that had to be disabled in the prior PR.
2021-08-17 16:49:26 -05:00
e50e13d2ab
state: partition nodes and coordinates in the state store ( #10859 )
...
Additionally:
- partitioned the catalog indexes appropriately for partitioning
- removed a stray reference to a non-existent index named "node.checks"
2021-08-17 13:29:39 -05:00
4f54d9708c
acl: add some notes about removing legacy ACL system
2021-08-17 13:08:29 -04:00
c271976445
state: refactor some node/coordinate state store functions to take an EnterpriseMeta ( #10687 )
...
Note the field is not used yet.
2021-07-23 13:42:23 -05:00
254557a1f6
sync changes to oss files made in enterprise ( #10670 )
2021-07-22 13:58:08 -05:00
62ac98b564
agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning ( #10669 )
2021-07-22 13:20:45 -05:00
b725605fe4
config raft apply silent error ( #10657 )
...
* return an error when the index is not valid
* check response as bool when applying `CAOpSetConfig`
* remove check for bool response
* fix error message and add check to test
* fix comment
* add changelog
2021-07-22 10:32:27 -04:00
b4a10443d1
ca: remove unused RotationPeriod field
...
This field was never used. Since it is persisted as part of a map[string]interface{} it
is pretty easy to remove it.
2021-07-05 19:15:44 -04:00
b57cf27e8f
Format certificates properly (rfc7468) with a trailing new line ( #10411 )
...
* trim carriage return from certificates when inserting rootCA in the inMemDB
* format rootCA properly when returning the CA on the connect CA endpoint
* Fix linter warnings
* Fix providers to trim certs before returning it
* trim newlines on write when possible
* add changelog
* make sure all provider return a trailing newline after the root and intermediate certs
* Fix endpoint to return trailing new line
* Fix failing test with vault provider
* make test more robust
* make sure all provider return a trailing newline after the leaf certs
* Check for suffix before removing newline and use function
* Add comment to consul provider
* Update change log
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* fix typo
* simplify code callflow
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* extract requireNewLine as shared func
* remove dependency to testify in testing file
* remove extra newline in vault provider
* Add cert newline fix to envoy xds
* remove new line from mock provider
* Remove adding a new line from provider and fix it when the cert is read
* Add a comment to explain the fix
* Add missing for leaf certs
* fix missing new line
* fix missing new line in leaf certs
* remove extra new line in test
* updage changelog
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
* fix in vault provider and when reading cache (RPC call)
* fix AWS provider
* fix failing test in the provider
* remove comments and empty lines
* add check for empty cert in test
* fix linter warnings
* add new line for leaf and private key
* use string concat instead of Sprintf
* fix new lines for leaf signing
* preallocate slice and remove append
* Add new line to `SignIntermediate` and `CrossSignCA`
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: Daniel Nephin <dnephin@hashicorp.com>
2021-06-30 20:48:29 -04:00
39d7d07922
state: reduce arguments to validateProposedConfigEntryInServiceGraph
2021-05-06 13:47:40 -04:00
401f3010e0
Rename "cluster" config entry to "mesh" ( #10127 )
...
This config entry is being renamed primarily because in k8s the name
cluster could be confusing given that the config entry applies across
federated datacenters.
Additionally, this config entry will only apply to Consul as a service
mesh, so the more generic "cluster" name is not needed.
2021-04-28 16:13:29 -06:00
95b361ecc8
state: remove unnecessary kind index
...
The query can be performed using a prefix query on the ID index.
Also backport some enterprise changes to prevent conflicts.
2021-04-15 17:37:28 -04:00
eb7f4b7ea4
state: use index constants for ConfigEntry indexes
2021-04-15 17:30:07 -04:00
2ff8b9f2f5
Avoid returning a nil slice
2021-04-14 10:52:05 -06:00
7fd4c569ce
Update viz endpoint to include topology from intentions
2021-04-14 10:20:15 -06:00
83501d5415
Augment intention decision summary with DefaultAllow mode
2021-04-12 19:32:09 -06:00
93818ebc5a
Merge pull request #9950 from hashicorp/dnephin/state-use-txn-everywhere
...
state: use Txn interface everywhere
2021-04-08 12:02:03 -04:00
9db8ffb1c5
Merge pull request #9880 from hashicorp/dnephin/catalog-events-test-pattern
...
state: use runCase pattern for large test
2021-04-08 11:54:41 -04:00
82245585c6
connect: add toggle to globally disable wildcard outbound network access when transparent proxy is enabled ( #9973 )
...
This adds a new config entry kind "cluster" with a single special name "cluster" where this can be controlled.
2021-04-06 13:19:59 -05:00
f0590e7c18
state: support additional test cases in indexer tests
...
And add a few additional cases.
2021-03-31 14:39:33 -04:00
3cdd495600
Backport enterprise changes to prevent merge conflicts
...
Co-Authored-By: Kyle Havlovitz <kylehav@gmail.com>
2021-03-31 14:05:26 -04:00
e4a60a2a8d
state: use tableIndex constant
2021-03-29 18:52:20 -04:00
7cb2255838
state: use ReadTxn and WriteTxn interface
...
Instead of *txn, so that we can replace the txn implementation with others, and so
that the function is easily documented as a read or write function.
2021-03-29 18:52:16 -04:00
d785c86db1
state: convert checks.service index to new pattern
2021-03-29 16:38:53 -04:00
f859ba6d4b
state: convert checks.status indexer
...
As part of this change the indexer will now be case insensitive by using
the lower case value. This should be safe because previously we always
had lower case strings.
This change was made out of convenience. All the other indexers use
lowercase, so we can re-use the indexFromQuery function by using
lowercase here as well.
2021-03-29 16:38:50 -04:00
9251ac881a
state: add tests for checks indexers
2021-03-29 16:38:47 -04:00
98d6dcbdf8
state: use constants for table checks
2021-03-29 16:38:43 -04:00
bcbdc9cab3
state: pass Query in from caller
...
To reduce the number of arguments
2021-03-29 15:42:30 -04:00
372d274b34
state: convert services.kind to functional indexer pattern
2021-03-29 15:42:30 -04:00
bcde8d2fad
state: add tests for services.kind indexer
2021-03-29 15:42:27 -04:00
9f9eadd569
state: convert services table service and connect indexer
...
To the new functional indexer pattern
2021-03-29 15:42:24 -04:00
11311c1fcc
state: add tests for services table service and connect indexers
2021-03-29 15:42:22 -04:00
9a3daf3100
state: use constant for tableServices
2021-03-29 15:42:18 -04:00
ec04df66bd
state: remove duplication of Query indexer
2021-03-29 14:35:11 -04:00
28866e48ad
state: remove duplication in acl tables schema
2021-03-29 14:21:27 -04:00
c6a1ca701d
state: reduce duplication in catalog table schema
2021-03-29 14:21:23 -04:00
d9dacb8388
state: share more indexer functions for config_entries
2021-03-29 14:21:20 -04:00
f303120f2d
state: remove old schema test
...
This test has been replaced by TestNewDBSchema_Indexers
2021-03-29 14:21:13 -04:00
150decff2a
state: use addNamespaceIndex again
2021-03-29 14:21:02 -04:00
4a3b462c28
Merge pull request #9911 from hashicorp/dnephin/state-index-acl-roles
...
state: convert ACLRoles policies index to new functional indexer pattern
2021-03-24 18:28:19 -04:00
25b791ba47
state: add tests for checks.ID indexer
2021-03-22 18:06:43 -04:00
abbe5c3701
state: use tx.First instead of tx.FirstWatch
...
Where appropriate. After removing the helper function a bunch of these calls can
be changed to tx.First.
2021-03-22 18:06:33 -04:00
49938bc472
state: convert checks.ID index to new pattern
2021-03-22 18:06:08 -04:00
1d3fe64bba
state: use uuid for acl-roles.policies index
...
Previously we were encoding the UUID as a string, but the index it references uses a UUID
so this index can also use an encoded UUID to save a bit of memory.
2021-03-19 19:45:37 -04:00
3c01bb1156
state: convert acl-roles.policies index to new pattern
2021-03-19 19:45:37 -04:00
474e95b9f5
state: convert acl-roles.name index to the functional indexer pattern
2021-03-19 19:45:37 -04:00
f836ed256b
state: add indexer tests for acl-roles table
2021-03-19 19:45:37 -04:00
6bc2c0e1ce
state: use constants for acl-roles table and indexes
2021-03-19 19:45:37 -04:00
d4e02024fe
state: convert acl-policies table to new pattern
2021-03-19 15:24:00 -04:00
845a10354e
state: use constants and add tests for acl-policies table
2021-03-19 15:19:57 -04:00
f6533a08f8
state: add indexer test for services.ID index
2021-03-19 14:13:14 -04:00
1d1c03d0cd
state: handle wildcard for services.ID index
...
When listing services, use the id_prefix directly if wildcards are allowed.
Error if a wildcard is used for a query that does not index the wildcard
2021-03-19 14:12:19 -04:00
bae69b2352
state: fix prefix index with the new pattern
...
Prefix queries are generally being used to match part of a partial
index. We can support these indexes by using a function that accept
different types for each subset of the index.
What I found interesting is that in the generic StringFieldIndexer the
implementation for PrefixFromArgs would remove the trailing null, but
at least in these 2 cases we actually want a null terminated string.
We simply want fewer components in the string.
2021-03-19 14:12:17 -04:00
ec50454fb3
state: move services.ID to new pattern
2021-03-19 14:11:59 -04:00
f5a52a4501
state: add tests for gateway-service table indexers
2021-03-18 12:09:42 -04:00
66632538d8
state: use constants and remove wrapping
...
for GatewayServices table
2021-03-18 12:08:59 -04:00
d77bdd26c5
state: Move UpstreamDownstream to state package
2021-03-18 12:08:59 -04:00
ca3686f4aa
state: add tests for mesh-topology table indexers
2021-03-18 12:08:57 -04:00
8a1a11814d
state: use constants for mesh-topology table operations
2021-03-18 12:08:03 -04:00
28c29e6ab4
Merge pull request #9899 from hashicorp/wildcard-ixn-oss
...
Add methods to check intention has wildcard src or dst
2021-03-18 08:33:07 -06:00
1c46470a29
Add methods to check intention has wildcard src or dst
2021-03-17 22:15:48 -06:00
6c43195e2a
Merge master and fix upstream config protocol defaulting
2021-03-17 21:13:40 -06:00
0c8b618ca0
Temporarily silence spurious wakeup. Addressing false positive in beta.
2021-03-17 17:25:29 -06:00
60690cf5c9
Merge remote-tracking branch 'origin/master' into intention-topology-endpoint
2021-03-17 17:14:38 -06:00
d19a5830dd
Do not include consul as upstream or downstream
2021-03-17 13:40:04 -06:00
d2591312f8
state: add tests for config-entry indexers
2021-03-17 14:41:46 -04:00
1b8f8b135e
state: convert config-entries kind index to new pattern
2021-03-17 14:40:57 -04:00
bfcf463c3a
state: remove config-entries namespace index
...
Use a prefix of the ID index instead.
2021-03-17 14:40:57 -04:00
dcbb1ba5dd
state: remove unnecessary method receiver
2021-03-17 14:40:57 -04:00
b43977423f
state: convert config-entries table to new indexer pattern
...
Using functional indexes to isolate enterprise differentiation and
remove reflection.
2021-03-17 14:40:57 -04:00
98c32599e4
Merge pull request #9881 from hashicorp/dnephin/state-index-service-check-nodes
...
state: convert services.node and checks.node indexes
2021-03-17 14:12:02 -04:00
Chris S. Kim