Commit graph

149 commits

Author SHA1 Message Date
Drew Bailey f60e44afc7
Adds nomad monitor command
Adds nomad monitor command. Like consul monitor, this command allows you
to stream logs from a nomad agent in real time with a a specified log
level

add endpoint tests

Upgrade go-hclog to latest version

The current version of go-hclog pads log prefixes to equal lengths
so info becomes [INFO ] and debug becomes [DEBUG]. This breaks
hashicorp/logutils/level.go Check function. Upgrading to the latest
version removes this padding and fixes log filtering that uses logutils
Check
2019-11-05 09:51:47 -05:00
Drew Bailey b386119d15
Add Agent Monitor to receive streaming logs
Queries /v1/agent/monitor and receives streaming logs from client
2019-11-05 09:51:47 -05:00
Mahmood Ali 66982a1660 agent: add websocket handler for nomad exec
This adds a websocket endpoint for handling `nomad exec`.

The endpoint is a websocket interface, as we require a bi-directional
streaming (to handle both input and output), which is not very appropriate for
plain HTTP 1.0. Using websocket makes implementing the web ui a bit simpler. I
considered using golang http hijack capability to treat http request as a plain
connection, but the web interface would be too complicated potentially.

Furthermore, the API endpoint operates against the raw core nomad exec streaming
datastructures, defined in protobuf, with json serializer.  Our APIs use json
interfaces in general, and protobuf generates json friendly golang structs.
Reusing the structs here simplify interface and reduce conversion overhead.
2019-05-09 16:49:08 -04:00
Preetha Appan fd0ba320da
change path to v1/scheduler/configuration 2018-11-12 15:57:45 -06:00
Alex Dadgar 98398a8a44
Merge pull request #4842 from hashicorp/b-deployment-progress-deadline
Fix multiple bugs with progress deadline handling
2018-11-08 13:31:54 -08:00
Alex Dadgar 1c31970464 Fix multiple tgs with progress deadline handling
Fix an issue in which the deployment watcher would fail the deployment
based on the earliest progress deadline of the deployment regardless of
if the task group has finished.

Further fix an issue where the blocked eval optimization would make it
so no evals were created to progress the deployment. To reproduce this
issue, prior to this commit, you can create a job with two task groups.
The first group has count 1 and resources such that it can not be
placed. The second group has count 3, max_parallel=1, and can be placed.
Run this first and then update the second group to do a deployment. It
will place the first of three, but never progress since there exists a
blocked eval. However, that doesn't capture the fact that there are two
groups being deployed.
2018-11-05 16:06:17 -08:00
Preetha Appan bd34cbb1f7
Support for new scheduler config API, first use case is to disable preemption 2018-10-30 11:06:32 -05:00
Alex Dadgar 7739ef51ce agent + consul 2018-09-13 10:43:40 -07:00
Chelsea Holland Komlo 38f611a7f2 refactor NewTLSConfiguration to pass in verifyIncoming/verifyOutgoing
add missing fields to TLS merge method
2018-05-23 18:35:30 -04:00
Nick Ethier f2db03e56c
command/agent: add /v1/jobs/parse endpoint
The parse endpoint accepts a hcl jobspec body within a json object
and returns the parsed json object for the job. This allows users to
register jobs with the nomad json api without specifically needing
a nomad binary to parse their hcl encoded jobspec file.
2018-04-16 19:21:06 -04:00
Josh Soref 6f48e31c00 spelling: convenience 2018-03-11 17:50:48 +00:00
Alex Dadgar 8b86e64fd8 Show HTTP request method 2018-02-16 15:55:26 -08:00
Kyle Havlovitz 0eb0acacdc Fix remaining issues with autopilot change 2018-01-30 15:21:28 -08:00
Chelsea Komlo d09cc2a69f
Merge pull request #3492 from hashicorp/f-client-tls-reload
Client/Server TLS dynamic reload
2018-01-23 05:51:32 -05:00
Kyle Havlovitz 12ff22ea70 Merge branch 'master' into autopilot 2018-01-18 13:29:25 -08:00
Michael Schurter 9f179e9fab Fix HTTP code for permission denied errors
Fixes #3697

The existing code and test case only covered the leader behavior. When
querying against non-leaders the error has an "rpc error: " prefix.

To provide consistency in HTTP error response I also strip the "rpc
error: " prefix for 403 responses as they offer no beneficial additional
information (and in theory disclose a tiny bit of data to unauthorized
users, but it would be a pretty weird bit of data to use in a malicious
way).
2018-01-09 15:25:53 -08:00
Chelsea Holland Komlo 6a2432659a code review fixups 2018-01-08 09:21:06 -05:00
Chelsea Holland Komlo c0ad9a4627 add ability to upgrade/downgrade nomad agents tls configurations via sighup 2018-01-08 09:21:06 -05:00
Kyle Havlovitz 1c07066064 Add autopilot functionality based on Consul's autopilot 2017-12-18 14:29:41 -08:00
Michael Lange 96403746b1 Add CORS headers to client fs endpoints 2017-11-21 11:22:42 -08:00
Chelsea Komlo 2dfda33703 Nomad agent reload TLS configuration on SIGHUP (#3479)
* Allow server TLS configuration to be reloaded via SIGHUP

* dynamic tls reloading for nomad agents

* code cleanup and refactoring

* ensure keyloader is initialized, add comments

* allow downgrading from TLS

* initalize keyloader if necessary

* integration test for tls reload

* fix up test to assert success on reloaded TLS configuration

* failure in loading a new TLS config should remain at current

Reload only the config if agent is already using TLS

* reload agent configuration before specific server/client

lock keyloader before loading/caching a new certificate

* introduce a get-or-set method for keyloader

* fixups from code review

* fix up linting errors

* fixups from code review

* add lock for config updates; improve copy of tls config

* GetCertificate only reloads certificates dynamically for the server

* config updates/copies should be on agent

* improve http integration test

* simplify agent reloading storing a local copy of config

* reuse the same keyloader when reloading

* Test that server and client get reloaded but keep keyloader

* Keyloader exposes GetClientCertificate as well for outgoing connections

* Fix spelling

* correct changelog style
2017-11-14 17:53:23 -08:00
Michael Lange 157abf2c76 Remove the connect-src self restriction for the UI 2017-11-10 13:28:11 -08:00
Alex Dadgar 701f462d33 remove atlas 2017-11-02 11:27:21 -07:00
Michael Schurter c9a73ac76e Support CORS for client endpoints
Added to /v1/client/stats and /v1/client/allocation/
2017-10-18 17:32:36 -07:00
Michael Schurter c53aac9eea Agent Health Endpoint 2017-10-13 15:37:44 -07:00
Michael Schurter 15b3df0b80 Merge pull request #3374 from hashicorp/f-auth-token
SecretID -> AuthToken
2017-10-12 16:57:49 -07:00
Alex Dadgar d6b970eec9 Handle invalid token as well 2017-10-12 15:39:05 -07:00
Michael Schurter 84d8a51be1 SecretID -> AuthToken 2017-10-12 15:16:33 -07:00
Alex Dadgar 0b538ded83 403 instead of 500 for permission denied 2017-10-12 14:10:20 -07:00
Alex Dadgar f16167b5e1 always gzip 2017-09-19 10:37:49 -05:00
Alex Dadgar e5ec915ac3 sync 2017-09-19 10:08:23 -05:00
Chelsea Holland Komlo f5975dceb7 refactoring prometheus endpoint 2017-09-13 19:21:21 +00:00
Chelsea Holland Komlo d8e9f2fef7 add endpoint for prometheus 2017-09-13 19:21:21 +00:00
Alex Dadgar 84d06f6abe Sync namespace changes 2017-09-07 17:04:21 -07:00
Chelsea Holland Komlo 751fc5324e add http endpoint for in memory metrics
prevent against flaky test due to timing/initialization issues
2017-09-06 13:51:19 +00:00
Armon Dadgar 0dabcb8659 agent: fix routing for token-specific request 2017-09-04 13:07:44 -07:00
Armon Dadgar 5c94e7e99f agent: thread through token for ACL endpoint tests 2017-09-04 13:05:53 -07:00
Armon Dadgar 4107335cb2 agent: Adding X-Nomad-Token header parsing 2017-09-04 13:05:53 -07:00
Armon Dadgar 866fe5e216 nomad: adding ACL bootstrapping endpoint 2017-09-04 13:05:53 -07:00
Armon Dadgar bd2db18c80 agent: Adding HTTP endpoints for ACL tokens 2017-09-04 13:04:45 -07:00
Armon Dadgar 18e6053b58 agent: Adding ACL Policy endpoints 2017-09-04 13:03:15 -07:00
Chelsea Holland Komlo ffe95a1a62 adds any resource autocomplete
defaults to listing jobs if no id is provided
2017-08-25 16:42:11 +00:00
Chelsea Holland Komlo 465c4d7082 change endpoint to /v1/search 2017-08-14 17:38:10 +00:00
Chelsea Holland Komlo 5ee58a391b rename to cluster search
comment updates
2017-08-14 17:36:14 +00:00
Chelsea Holland Komlo 1b77f9a216 further refactoring 2017-08-04 22:50:41 +00:00
Chelsea Holland Komlo 4dd6b46198 Retrieve job information for resources endpoint
requires further refactoring and logic for more contexts
2017-08-04 14:34:25 +00:00
Alex Dadgar 9037693436 New test agent 2017-07-19 22:14:36 -07:00
Alex Dadgar 580eed5c88 HTTP Endpoints 2017-07-07 12:03:11 -07:00
Alex Dadgar ba70cc4f01 Merge branch 'master' into f-bolt-db 2017-05-09 11:11:55 -07:00
Alex Dadgar d779defe65 Use batching 2017-05-01 14:50:34 -07:00
Alex Dadgar bddedd7aba Don't deepcopy job when retrieving copy of Alloc
This PR removes deepcopying of the job attached to the allocation in the
alloc runner. This operation is called very often so removing reflect
from the code path and the potentially large number of mallocs need to
create a job reduced memory and cpu pressure.
2017-05-01 14:50:34 -07:00
Pete Wildsmith 642fbd2f56 address feedback 2017-04-29 08:26:12 +01:00
Pete Wildsmith 1b8a1614ca reduce to one configuration option
There should be just one option, verify_https_client, which
controls incoming and outgoing validation for the HTTPS wrapper
2017-04-28 10:45:09 +01:00
Pete Wildsmith 3070d5ab9d Copy TLSConfig verification flags in server create 2017-04-25 23:33:12 +01:00
Adam Stankiewicz 4daf4cb8c9
Remove unnecessary parameter from NewHTTPServer 2017-04-10 16:24:49 +02:00
Alex Dadgar b67c59f03c Merge branch 'master' into refactor-parser 2017-02-20 15:13:21 -08:00
Diptanu Choudhury 7567209857 Making the job spec return api.Job 2017-02-16 13:52:39 -08:00
Alex Dadgar 627ac3fc45 Fix escaping 2017-02-15 15:14:47 -08:00
Alex Dadgar 2d4d9b79d8 Operator command/endpoint/documentation 2017-02-09 18:04:46 -08:00
Diptanu Choudhury afdaa979f7 Added a garbage collector for allocations 2016-12-14 15:01:12 -08:00
Michael Schurter a75e5b5803 Addresses are just addresses - no ports
Store address+port in an unexported field for ease-of-use
2016-11-09 11:49:55 -08:00
Michael Schurter c735589f41 Choose safer default advertise address
* -dev mode defaults bind & advertise to localhost
* Normal mode defaults bind to 0.0.0.0 & advertise to the resolved
  hostname. If the hostname resolves to localhost it will refuse to
  start and advertise must be manually set.
2016-11-08 11:17:16 -08:00
Michael Schurter 536c2921e9 Remove ServerName because we verify based on region 2016-11-01 14:17:31 -07:00
Evan Gilman de33949df8 Add address selector methods to the agent 2016-10-27 10:51:11 -07:00
Diptanu Choudhury 7c61e115bd Moved tlsutil into helpers 2016-10-25 16:05:37 -07:00
Diptanu Choudhury eefc8db3b3 Enabling TLS on cli 2016-10-25 10:39:17 -07:00
Diptanu Choudhury e03927bb5c Changed the way TLS config is parsed 2016-10-24 13:56:19 -07:00
Diptanu Choudhury 2e3118e69c Implemented TLS support for http and rpc 2016-10-23 22:22:00 -07:00
Diptanu Choudhury 0f6e0d10b6 Enable serf encryption (#1791)
* Added the keygen command

* Added support for gossip encryption

* Changed the URL for keyring management

* Fixed the cli

* Added some tests

* Added tests for keyring operations

* Added a test for removal of keys

* Added some docs

* Fixed some docs

* Added general options
2016-10-17 10:48:04 -07:00
Diptanu Choudhury 88d383c47f Updated tests and comments 2016-08-04 11:29:36 -07:00
Diptanu Choudhury 74caed0c7a Added an endpoint for users to reconcile job summaries 2016-08-03 16:12:47 -07:00
Alex Dadgar c84964bf6e Fix buffer reuse 2016-07-25 11:16:01 -07:00
Alex Dadgar 5797534d16 Don't require slash for node stats 2016-06-11 13:15:57 -07:00
Diptanu Choudhury 464812ea3b Changed the stats endpoints 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 0fb0e0237f Added a client API to display resource usage of an allocation 2016-05-28 19:42:34 -07:00
Alex Dadgar 2a4d5f0ef9 Compress HTTP API responses and have the api client utilize this 2016-05-24 10:26:53 -07:00
Alex Dadgar 636f886e30 Use codegen for json marshalling: 20% faster, 12% less bytes allocated, 85% less allocations 2016-05-18 09:45:46 -07:00
Sean Chittenden 7b8a5dcc15
Fix the unit tests for TestPrettyPrintOff
A different failing test masked that this wasn't even being run.

Pointy Hat: sean- for testing first on Travis in a branch vs verifying locally
2016-05-11 15:48:43 -07:00
Sean Chittenden 2fe9b6e97a
Append a newline when pretty-printing the response
Instead of typing `curl http://foo/v1/nodes?pretty; echo` to inject a
newline into your terminal, now pretty will provide its own newline.
2016-05-10 15:46:20 -07:00
Alex Dadgar da984cb4d2 Guard client/ api to ensure the client is running 2016-03-07 11:26:54 -08:00
Alex Dadgar 143972b6d9 Job GC endpoint 2016-02-20 15:50:41 -08:00
Diptanu Choudhury b86e60b9de Allow users to configure arbitrary headers in config 2016-01-22 11:00:43 -08:00
Diptanu Choudhury e3d7e693dc Added methods for listing directories inside an alloc 2016-01-12 15:03:53 -08:00
Ivo Verberk 7838b2a57c Refactoring
* Reverted changes to get methods
* Added prefix query parameter
* Updated node status to use prefix based searching
* Fixed tests
* Removed truncation logic
2015-12-22 23:44:33 +01:00
Ryan Uber a230a70cc7 nomad: testing region list 2015-11-23 22:27:07 -08:00
Ryan Uber ad6b55a37a nomad: support listing regions 2015-11-23 22:27:03 -08:00
Ryan Uber 114b38b8a8 agent: add HTTP endpoint for querying/modifying the server list 2015-09-24 20:39:03 -07:00
Ryan Uber f0c21b79f1 agent: json decode directly to structs in api 2015-09-22 20:02:10 -07:00
Armon Dadgar 53834b39ba SCADA support 2015-09-14 15:33:08 -07:00
Ryan Uber 35580ef9ec agent: use ports/addresses/advertise_addrs for network config 2015-09-10 18:37:42 -07:00
Ryan Uber 936bca5194 agent: addresses and ports are configurable 2015-09-10 17:48:02 -07:00
Armon Dadgar 8a389414ff http: adding the agent endpoints 2015-09-06 19:08:05 -07:00
Armon Dadgar 30b34777ff http: adding status APIs 2015-09-06 18:39:03 -07:00
Armon Dadgar 7c162542aa http: adding the eval endpoints 2015-09-06 16:02:53 -07:00
Armon Dadgar 05a8ae06c7 http: adding allocs list endpoint 2015-09-06 15:37:21 -07:00
Armon Dadgar bb0ba026bc http/node: adding a few node endpoints 2015-09-06 14:18:11 -07:00
Armon Dadgar f858c31426 http: framework jobs endpoints 2015-09-05 18:00:30 -07:00
Armon Dadgar 4bf257e951 http: basic framework 2015-09-05 17:06:05 -07:00
Armon Dadgar 0c3064bada agent: adding HTTP server 2015-08-23 17:50:33 -07:00