Commit Graph

540 Commits

Author SHA1 Message Date
Aestek 24c29e195b kv: do not trigger watches when setting the same value (#5885)
If a KVSet is performed but does not update the entry, do not trigger
watches for this key.
This avoids releasing blocking queries for KV values that did not
actually changed.
2019-06-18 15:06:29 +02:00
Paul Banks e90fab0aec
Add rate limiting to RPCs sent within a server instance too (#5927) 2019-06-13 04:26:27 -05:00
Freddy 8f5fe058ea
Increase reliability of TestResetSessionTimerLocked_Renew 2019-05-24 13:54:51 -04:00
Freddy f7f0207f78
Run TestServer_Expect on its own (#5890) 2019-05-23 19:52:33 -04:00
Freddy e9bdb3a4f9
Flaky test: ACLReplication_Tokens (#5891)
* Exclude non-go workflows while testing

* Wait for s2 global-management policy

* Revert "Exclude non-go workflows while testing"

This reverts commit 47a83cbe9f19d0e1e475eabaa223d61fb4c56019.
2019-05-23 19:52:02 -04:00
Freddy c9e6640337
Add retries to StatsFetcherTest (#5892) 2019-05-23 19:51:31 -04:00
freddygv d133d565a5 Wait for s2 global-management policy 2019-05-21 17:58:37 -06:00
Freddy 7ce28bbfee
Stop running TestLeader_ChangeServerID in parallel 2019-05-21 15:28:08 -06:00
Kyle Havlovitz ad24456f49
Set the dead node reclaim timer at 30s 2019-05-15 11:59:33 -07:00
Kyle Havlovitz dcbffdb956
Merge branch 'master' into change-node-id 2019-05-15 10:51:04 -07:00
Matt Keeler 46956ed769
Copy the proxy config instead of direct assignment (#5786)
This prevents modifying the data in the state store which is supposed to be immutable.
2019-05-06 12:09:59 -04:00
R.B. Boyer 372bb06c83
acl: a role binding rule for a role that does not exist should be ignored (#5778)
I wrote the docs under this assumption but completely forgot to actually
enforce it.
2019-05-03 14:22:44 -05:00
R.B. Boyer 7d0f729f77
acl: enforce that you cannot persist tokens and roles with missing links except during replication (#5779) 2019-05-02 15:02:21 -05:00
Matt Keeler 26708570c5
Fix ConfigEntryResponse binary marshaller and ensure we watch the chan in ConfigEntry.Get even when no entry exists. (#5773) 2019-05-02 15:25:29 -04:00
Paul Banks df0c61fd31
Fix previous accidental master push 🤦 (#5771)
* Fix previous accidental master push 🤦

* Fix ACL test
2019-05-02 15:49:37 +01:00
Paul Banks 95bb1e368f
Fix panic in Resolving service config when proxy-defaults isn't defined yet (#5769) 2019-05-02 14:12:21 +01:00
Paul Banks cf24e7d1ed
Fix uint8 conversion issues for service config response maps. 2019-05-02 14:11:33 +01:00
Paul Banks 078f4cf5bb Add integration test for central config; fix central config WIP (#5752)
* Add integration test for central config; fix central config WIP

* Add integration test for central config; fix central config WIP

* Set proxy protocol correctly and begin adding upstream support

* Add upstreams to service config cache key and start new notify watcher if they change.

This doesn't update the tests to pass though.

* Fix some merging logic get things working manually with a hack (TODO fix properly)

* Simplification to not allow enabling sidecars centrally - it makes no sense without upstreams anyway

* Test compile again and obvious ones pass. Lots of failures locally not debugged yet but may be flakes. Pushing up to see what CI does

* Fix up service manageer and API test failures

* Remove the enable command since it no longer makes much sense without being able to turn on sidecar proxies centrally

* Remove version.go hack - will make integration test fail until release

* Remove unused code from commands and upstream merge

* Re-bump version to 1.5.0
2019-05-01 16:39:31 -07:00
Matt Keeler 9c77f2c52a
Update to use a consulent build tag instead of just ent (#5759) 2019-05-01 11:11:27 -04:00
Matt Keeler ea6cbf01a5 Centralized Config CLI (#5731)
* Add HTTP endpoints for config entry management

* Finish implementing decoding in the HTTP Config entry apply endpoint

* Add CAS operation to the config entry apply endpoint

Also use this for the bootstrapping and move the config entry decoding function into the structs package.

* First pass at the API client for the config entries

* Fixup some of the ConfigEntry APIs

Return a singular response object instead of a list for the ConfigEntry.Get RPC. This gets plumbed through the HTTP API as well.

Dont return QueryMeta in the JSON response for the config entry listing HTTP API. Instead just return a list of config entries.

* Minor API client fixes

* Attempt at some ConfigEntry api client tests

These don’t currently work due to weak typing in JSON

* Get some of the api client tests passing

* Implement reflectwalk magic to correct JSON encoding a ProxyConfigEntry

Also added a test for the HTTP endpoint that exposes the problem. However, since the test doesn’t actually do the JSON encode/decode its still failing.

* Move MapWalk magic into a binary marshaller instead of JSON.

* Add a MapWalk test

* Get rid of unused func

* Get rid of unused imports

* Fixup some tests now that the decoding from msgpack coerces things into json compat types

* Stub out most of the central config cli

Fully implement the config read command.

* Basic config delete command implementation

* Implement config write command

* Implement config list subcommand

Not entirely sure about the output here. Its basically the read output indented with a line specifying the kind/name of each type which is also duplicated in the indented output.

* Update command usage

* Update some help usage formatting

* Add the connect enable helper cli command

* Update list command output

* Rename the config entry API client methods.

* Use renamed apis

* Implement config write tests

Stub the others with the noTabs tests.

* Change list output format

Now just simply output 1 line per named config

* Add config read tests

* Add invalid args write test.

* Add config delete tests

* Add config list tests

* Add connect enable tests

* Update some CLI commands to use CAS ops

This also modifies the HTTP API for a write op to return a boolean indicating whether the value was written or not.

* Fix up the HTTP API CAS tests as I realized they weren’t testing what they should.

* Update config entry rpc tests to properly test CAS

* Fix up a few more tests

* Fix some tests that using ConfigEntries.Apply

* Update config_write_test.go

* Get rid of unused import
2019-04-30 16:27:16 -07:00
Matt Keeler 697efb588c
Make a few config entry endpoints return 404s and allow for snake_case and lowercase key names. (#5748) 2019-04-30 18:19:19 -04:00
Matt Keeler 8beb5c6082
ACL Token ID Initialization (#5307) 2019-04-30 11:45:36 -04:00
Kyle Havlovitz 64174f13d6 Add HTTP endpoints for config entry management (#5718) 2019-04-29 18:08:09 -04:00
R.B. Boyer ea2740fd32
Merge pull request #5617 from hashicorp/f-acl-ux
Secure ACL Introduction for Kubernetes
2019-04-26 15:34:26 -05:00
Aestek 9813abcb09 Fix: fail messages after a node rename replace the new node definition (#5520)
When receiving a serf faild message for a node which is not in the
catalog, do not perform a register request to set is serf heath to
critical as it could overwrite the node information and services if it
was renamed.

Fixes : #5518
2019-04-26 21:33:41 +01:00
R.B. Boyer 5a505c5b3a acl: adding support for kubernetes auth provider login (#5600)
* auth providers
* binding rules
* auth provider for kubernetes
* login/logout
2019-04-26 14:49:25 -05:00
R.B. Boyer 9542fdc9bc acl: adding Roles to Tokens (#5514)
Roles are named and can express the same bundle of permissions that can
currently be assigned to a Token (lists of Policies and Service
Identities). The difference with a Role is that it not itself a bearer
token, but just another entity that can be tied to a Token.

This lets an operator potentially curate a set of smaller reusable
Policies and compose them together into reusable Roles, rather than
always exploding that same list of Policies on any Token that needs
similar permissions.

This also refactors the acl replication code to be semi-generic to avoid
3x copypasta.
2019-04-26 14:49:12 -05:00
R.B. Boyer f43bc981e9 making ACLToken.ExpirationTime a *time.Time value instead of time.Time (#5663)
This is mainly to avoid having the API return "0001-01-01T00:00:00Z" as
a value for the ExpirationTime field when it is not set. Unfortunately
time.Time doesn't respect the json marshalling "omitempty" directive.
2019-04-26 14:48:16 -05:00
R.B. Boyer b3956e511c acl: ACL Tokens can now be assigned an optional set of service identities (#5390)
These act like a special cased version of a Policy Template for granting
a token the privileges necessary to register a service and its connect
proxy, and read upstreams from the catalog.
2019-04-26 14:48:04 -05:00
R.B. Boyer 76321aa952 acl: tokens can be created with an optional expiration time (#5353) 2019-04-26 14:47:51 -05:00
Matt Keeler 3ea9fe3bff
Implement bootstrapping proxy defaults from the config file (#5714) 2019-04-26 14:25:03 -04:00
Matt Keeler 3b5d38fb49
Implement config entry replication (#5706) 2019-04-26 13:38:39 -04:00
Alvin Huang 96c2c79908
Add fmt and vet (#5671)
* add go fmt and vet

* go fmt fixes
2019-04-25 12:26:33 -04:00
Kyle Havlovitz 6faa8ba451 Fill out the service manager functionality and fix tests 2019-04-23 00:17:28 -07:00
Kyle Havlovitz d51fd740bf
Merge pull request #5615 from hashicorp/config-entry-rpc
Add RPC endpoints for config entry operations
2019-04-23 00:16:54 -07:00
Kyle Havlovitz e64d1b8016 Rename config entry ACL methods 2019-04-22 23:55:11 -07:00
kaitlincarter-hc 7859d8c409
[docs] Server Performance (#5627)
* Moving server performance guide to docs.

* fixing broken links

* updating broken link

* fixing broken links
2019-04-17 13:17:12 -05:00
Matt Keeler ac78c23021
Implement data filtering of some endpoints (#5579)
Fixes: #4222 

# Data Filtering

This PR will implement filtering for the following endpoints:

## Supported HTTP Endpoints

- `/agent/checks`
- `/agent/services`
- `/catalog/nodes`
- `/catalog/service/:service`
- `/catalog/connect/:service`
- `/catalog/node/:node`
- `/health/node/:node`
- `/health/checks/:service`
- `/health/service/:service`
- `/health/connect/:service`
- `/health/state/:state`
- `/internal/ui/nodes`
- `/internal/ui/services`

More can be added going forward and any endpoint which is used to list some data is a good candidate.

## Usage

When using the HTTP API a `filter` query parameter can be used to pass a filter expression to Consul. Filter Expressions take the general form of:

```
<selector> == <value>
<selector> != <value>
<value> in <selector>
<value> not in <selector>
<selector> contains <value>
<selector> not contains <value>
<selector> is empty
<selector> is not empty
not <other expression>
<expression 1> and <expression 2>
<expression 1> or <expression 2>
```

Normal boolean logic and precedence is supported. All of the actual filtering and evaluation logic is coming from the [go-bexpr](https://github.com/hashicorp/go-bexpr) library

## Other changes

Adding the `Internal.ServiceDump` RPC endpoint. This will allow the UI to filter services better.
2019-04-16 12:00:15 -04:00
Kyle Havlovitz 2cffe4894f Move the ACL logic into the ConfigEntry interface 2019-04-10 14:27:28 -07:00
Kyle Havlovitz 81254deb59 Add RPC endpoints for config entry operations 2019-04-06 23:38:08 -07:00
Alvin Huang aacb81a566
Merge pull request #5376 from hashicorp/fix-tests
Fix tests in prep for CircleCI Migration
2019-04-04 17:09:32 -04:00
Kyle Havlovitz d6c25a13a5
Merge pull request #5539 from hashicorp/service-config
Service config state model
2019-04-02 16:34:58 -07:00
Kyle Havlovitz 63c9434779 Cleaned up some error handling/comments around config entries 2019-04-02 15:42:12 -07:00
Kyle Havlovitz ace5c7a1cb Encode config entry FSM messages in a generic type 2019-03-28 00:06:56 -07:00
Kyle Havlovitz 96a460c0cf Clean up service config state store methods 2019-03-27 16:52:38 -07:00
R.B. Boyer ab57b02ff8
acl: memdb filter of tokens-by-policy was inverted (#5575)
The inversion wasn't noticed because the parallel execution of TokenList
tests was operating incorrectly due to variable shadowing.
2019-03-27 15:24:44 -05:00
Jeff Mitchell d3c7d57209
Move internal/ to sdk/ (#5568)
* Move internal/ to sdk/

* Add a readme to the SDK folder
2019-03-27 08:54:56 -04:00
Jeff Mitchell a41c865059
Convert to Go Modules (#5517)
* First conversion

* Use serf 0.8.2 tag and associated updated deps

* * Move freeport and testutil into internal/

* Make internal/ its own module

* Update imports

* Add replace statements so API and normal Consul code are
self-referencing for ease of development

* Adapt to newer goe/values

* Bump to new cleanhttp

* Fix ban nonprintable chars test

* Update lock bad args test

The error message when the duration cannot be parsed changed in Go 1.12
(ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test.

* Update another test as well

* Bump travis

* Bump circleci

* Bump go-discover and godo to get rid of launchpad dep

* Bump dockerfile go version

* fix tar command

* Bump go-cleanhttp
2019-03-26 17:04:58 -04:00
Paul Banks 68e8933ba5
Connect: Make Connect health queries unblock correctly (#5508)
* Make Connect health queryies unblock correctly in all cases and use optimal number of watch chans. Fixes #5506.

* Node check test cases and clearer bug test doc

* Comment update
2019-03-21 16:01:56 +00:00
Kyle Havlovitz c2cba68042 Fix fsm serialization and add snapshot/restore 2019-03-20 16:13:13 -07:00