Commit graph

50 commits

Author SHA1 Message Date
Tim Gross 881a4cfaff
metrics: Add remaining server RPC rate metrics (#15901) 2023-01-27 08:29:53 -05:00
Piotr Kazmierczak b63944b5c1
cleanup: replace TypeToPtr helper methods with pointer.Of (#14151)
Bumping compile time requirement to go 1.18 allows us to simplify our pointer helper methods.
2022-08-17 18:26:34 +02:00
Seth Hoenig 2631659551 ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
Charlie Voiselle 98a240cd99
Make number of scheduler workers reloadable (#11593)
## Development Environment Changes
* Added stringer to build deps

## New HTTP APIs
* Added scheduler worker config API
* Added scheduler worker info API

## New Internals
* (Scheduler)Worker API refactor—Start(), Stop(), Pause(), Resume()
* Update shutdown to use context
* Add mutex for contended server data
    - `workerLock` for the `workers` slice
    - `workerConfigLock` for the `Server.Config.NumSchedulers` and
      `Server.Config.EnabledSchedulers` values

## Other
* Adding docs for scheduler worker api
* Add changelog message

Co-authored-by: Derek Strickland <1111455+DerekStrickland@users.noreply.github.com>
2022-01-06 11:56:13 -05:00
Mahmood Ali 14d6fec05a tests: deflake some SetServer related tests
Some tests assert on numbers on numbers of servers, e.g.
TestHTTP_AgentSetServers and TestHTTP_AgentListServers_ACL . Though, in dev and
test modes, the agent starts with servers having duplicate entries for
advertised and normalized RPC values, then settles with one unique value after
Raft/Serf re-sets servers with one single unique value.

This leads to flakiness, as the test will fail if assertion runs before Serf
update takes effect.

Here, we update the inital dev handling so it only adds a unique value if the
advertised and normalized values are the same.

Sample log lines illustrating the problem:

```
=== CONT  TestHTTP_AgentSetServers
    TestHTTP_AgentSetServers: testlog.go:34: 2020-04-06T21:47:51.016Z [INFO]  nomad.raft: initial configuration: index=1 servers="[{Suffrage:Voter ID:127.0.0.1:9008 Address:127.0.0.1:9008}]"
    TestHTTP_AgentSetServers: testlog.go:34: 2020-04-06T21:47:51.016Z [INFO]  nomad: serf: EventMemberJoin: TestHTTP_AgentSetServers.global 127.0.0.1
    TestHTTP_AgentSetServers: testlog.go:34: 2020-04-06T21:47:51.035Z [DEBUG] client.server_mgr: new server list: new_servers=[127.0.0.1:9008, 127.0.0.1:9008] old_servers=[]
...
    TestHTTP_AgentSetServers: agent_endpoint_test.go:759:
                Error Trace:    agent_endpoint_test.go:759
                                                        http_test.go:1089
                                                        agent_endpoint_test.go:705
                Error:          "[127.0.0.1:9008 127.0.0.1:9008]" should have 1 item(s), but has 2
                Test:           TestHTTP_AgentSetServers
```
2020-04-07 09:27:48 -04:00
Michael Schurter 1a27b8a07d test: assert monitor endpoint sets proper headers 2020-03-25 09:45:43 -04:00
Michael Schurter fd81208db7 test: fix flaky health test
Test set Agent.client=nil which prevented the client from being
shutdown. This leaked goroutines and could cause panics due to the
leaked client goroutines logging after their parent test had finished.

Removed ACLs from the server test because I couldn't get it to work with
the test agent, and it tested very little.
2020-02-07 15:50:53 -08:00
Drew Bailey d830998572
agent Profile req nil check s.agent.Server()
clean up logic and tests
2020-02-03 13:20:05 -05:00
Drew Bailey c4f45f9bde
Fix panic when monitoring a local client node
Fixes a panic when accessing a.agent.Server() when agent is a client
instead. This pr removes a redundant ACL check since ACLs are validated
at the RPC layer. It also nil checks the agent server and uses Client()
when appropriate.
2020-02-03 13:20:04 -05:00
Drew Bailey 085659f6ff
condense table test 2020-01-09 15:15:10 -05:00
Drew Bailey 1b8af920f3
address pr feedback 2020-01-09 15:15:09 -05:00
Drew Bailey 4ced73875b
leave acl checking to rpc endpoints
fix test expectation

test wrapNonJSON
2020-01-09 15:15:08 -05:00
Drew Bailey 7bbba613a5
prevent doubly wrapping with rpc error 2020-01-09 15:15:07 -05:00
Drew Bailey 9a80938fb1
region forwarding; prevent recursive forwards for impossible requests
prevent region forwarding loop, backfill tests

fix failing test
2020-01-09 15:15:06 -05:00
Drew Bailey fc37448683
warn when enabled debug is on when registering
m -> a receiver name

return codederrors, fix query
2020-01-09 15:15:04 -05:00
Drew Bailey 62eb2d76a6
acl and debug test table
rename implementation method
2020-01-09 15:15:03 -05:00
Drew Bailey 901f362858
test for known pprof endpoints 2020-01-09 15:15:02 -05:00
Drew Bailey 49ad5fbc85
agent pprof endpoints
wip, agent endpoint and client endpoint for pprof profiles

agent endpoint test
2020-01-09 15:15:02 -05:00
Drew Bailey df57f70a68
wireup plain=true|false query param 2019-11-05 11:44:28 -05:00
Drew Bailey 0e759c401c
moving endpoints over to frames 2019-11-05 09:51:54 -05:00
Drew Bailey 32f62edbb0
return 400 if invalid log_json param is given
Addresses feedback around monitor implementation

subselect on stopCh to prevent blocking forever.

Set up a separate goroutine to check every 3 seconds for dropped
messages.

rename returned ch to avoid confusion
2019-11-05 09:51:53 -05:00
Drew Bailey 17d876d5ef
rename function, initialize log level better
underscores instead of dashes for query params
2019-11-05 09:51:53 -05:00
Drew Bailey f46fd5b3e1
only look up rpchandler for node if we have nodeid
fix some comments and nomad monitor -h output
2019-11-05 09:51:51 -05:00
Drew Bailey a45ae1cd58
enable json formatting, use queryoptions 2019-11-05 09:51:49 -05:00
Drew Bailey e076204820
get local rpc endpoint working 2019-11-05 09:51:48 -05:00
Drew Bailey 976c43157c
remove log_writer
prefix output with proper spacing

update gzip handler, adjust first byte flow to allow gzip handler bypass

wip, first stab at wiring up rpc endpoint
2019-11-05 09:51:48 -05:00
Drew Bailey 0de94466b2
Display error when remote side ended monitor
multisink logger

remove usage of logwriter
2019-11-05 09:51:48 -05:00
Drew Bailey f60e44afc7
Adds nomad monitor command
Adds nomad monitor command. Like consul monitor, this command allows you
to stream logs from a nomad agent in real time with a a specified log
level

add endpoint tests

Upgrade go-hclog to latest version

The current version of go-hclog pads log prefixes to equal lengths
so info becomes [INFO ] and debug becomes [DEBUG]. This breaks
hashicorp/logutils/level.go Check function. Upgrading to the latest
version removes this padding and fixes log filtering that uses logutils
Check
2019-11-05 09:51:47 -05:00
Danielle Lancashire 39fe07f66b
api: Redact tokens in /agent/self 2019-09-23 19:07:27 +02:00
Danielle Lancashire 8b44369073
api: Redact ACL Replication Token
Currently when hitting the /v1/agent/self API with ACL Replication
enabled results in the token being returned in the API. This commit
redacts that information, as it should be treated as a shared secret.
2019-09-22 14:35:53 +02:00
Alex Dadgar 993727c28f Use in-mem rpc 2018-02-15 13:59:01 -08:00
Alex Dadgar 9bc75f0ad4 Fix manager tests and make testagent recover from port conflicts 2018-02-15 13:59:01 -08:00
Michael Schurter 9d3f5a043e Better client health check and error handling 2017-10-13 16:25:25 -07:00
Michael Schurter c53aac9eea Agent Health Endpoint 2017-10-13 15:37:44 -07:00
Michael Schurter 492c861419 /v1/client/agent/* ACL enforcement 2017-10-09 12:18:54 -07:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Alex Dadgar 01180fec58 use assert 2017-09-14 14:20:22 -07:00
Alex Dadgar c55b7ce4d6 Sort /v1/agent/servers output
This PR sorts the output of the endpoint since its results are used as
part of Consul checks to avoid the value changing unnecessarily.

Fixes https://github.com/hashicorp/nomad/issues/3211
2017-09-14 14:20:22 -07:00
Jeremy Voorhis 554f6efe02 Redact Vault.Token from AgentSelf response.
If Config.Vault.Token is defined, /v1/agent/self will return the string
`<redacted>`. If the token is not set, This endpoint will continue to
return the empty string.
2017-08-07 21:34:43 -07:00
Alex Dadgar 4e90d56098 More parallel 2017-07-20 09:36:34 -07:00
Alex Dadgar 9037693436 New test agent 2017-07-19 22:14:36 -07:00
Diptanu Choudhury a70fbd7cf1 Fixed a test 2016-11-08 13:01:56 -08:00
Diptanu Choudhury 0f6e0d10b6 Enable serf encryption (#1791)
* Added the keygen command

* Added support for gossip encryption

* Changed the URL for keyring management

* Fixed the cli

* Added some tests

* Added tests for keyring operations

* Added a test for removal of keys

* Added some docs

* Fixed some docs

* Added general options
2016-10-17 10:48:04 -07:00
Michael Schurter a45921c402 Fix agent server set test
Test asserted endpoint appendend servers, but the new/desired behavior
is for the endpoint to set/overwrite servers.
2016-09-26 11:57:06 -07:00
Sean Chittenden 39fb0f2469
Change the endpoint for /v1/agent/servers and fix tests.
When an agent is running a server, the list of servers includes the
Raft peers.  When the agent is running a client (which is always the
case?), include a list of the servers found in the Client's RpcProxy.
Dedupe and provide a unique list back to the caller.
2016-06-10 15:50:11 -04:00
Jake Champlin e2f0244776 Update agent_endpoint_test 2016-01-28 23:33:01 -05:00
Ryan Uber 114b38b8a8 agent: add HTTP endpoint for querying/modifying the server list 2015-09-24 20:39:03 -07:00
Armon Dadgar 53834b39ba SCADA support 2015-09-14 15:33:08 -07:00
Ryan Uber 92c2c58a37 command: convert members so we get a descriptive status 2015-09-14 13:24:03 -07:00
Armon Dadgar 8a389414ff http: adding the agent endpoints 2015-09-06 19:08:05 -07:00