Commit graph

1405 commits

Author SHA1 Message Date
Jeff Mitchell a41c865059
Convert to Go Modules (#5517)
* First conversion

* Use serf 0.8.2 tag and associated updated deps

* * Move freeport and testutil into internal/

* Make internal/ its own module

* Update imports

* Add replace statements so API and normal Consul code are
self-referencing for ease of development

* Adapt to newer goe/values

* Bump to new cleanhttp

* Fix ban nonprintable chars test

* Update lock bad args test

The error message when the duration cannot be parsed changed in Go 1.12
(ae0c435877d3aacb9af5e706c40f9dddde5d3e67). This updates that test.

* Update another test as well

* Bump travis

* Bump circleci

* Bump go-discover and godo to get rid of launchpad dep

* Bump dockerfile go version

* fix tar command

* Bump go-cleanhttp
2019-03-26 17:04:58 -04:00
Matt Keeler a34f8c751e
Pass a testing.T into NewTestAgent and TestAgent.Start (#5342)
This way we can avoid unnecessary panics which cause other tests not to run.

This doesn't remove all the possibilities for panics causing other tests not to run, it just fixes the TestAgent
2019-02-14 10:59:14 -05:00
Paul Banks 1e4c5a1811 Connect Envoy Command (#4735)
* Plumb xDS server and proxyxfg into the agent startup

* Add `consul connect envoy` command to allow running Envoy as a connect sidecar.

* Add test for help tabs; typos and style fixups from review
2018-10-10 16:55:34 +01:00
Paul Banks cba42d6790 XDS Server Config (#4730)
* Config for the coming XDS server

* Default gRPC to 8502 for -dev mode; Re-merge the command Info output that shows gRPC.
2018-10-10 16:55:34 +01:00
Pierre Souchay 473e589d86 Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Siva Prasad 59dea9a31f
Adds a new command line flag -log-file for file based logging. (#4581)
* Added log-file flag to capture Consul logs in a user specified file

* Refactored code.

* Refactored code. Added flags to rotate logs based on bytes and duration

* Added the flags for log file and log rotation on the webpage

* Fixed TestSantize from failing due to the addition of 3 flags

* Introduced changes : mutex, data-dir log writes, rotation logic

* Added test for logfile and updated the default log destination for docs

* Log name now uses UnixNano

* TestLogFile is now uses t.Parallel()

* Removed unnecessary int64Val function

* Updated docs to reflect default log name for log-file

* No longer writes to data-dir and adds .log if the filename has no extension
2018-08-29 16:56:58 -04:00
Siva 2182e289a3 Merge branch 'master' of github.com:hashicorp/consul into WinService 2018-06-26 16:49:50 -04:00
Paul Banks ca68136ac7 Refactor to use embedded struct. 2018-06-25 12:25:39 -07:00
Paul Banks 23be6ad1c8 StartupTelemetry => InitTelemetry 2018-06-25 12:25:39 -07:00
Paul Banks 93f346431b WIP 2018-06-25 12:25:38 -07:00
Siva e54fbbba51 Graceful exits added 2018-06-20 14:42:08 -04:00
Kyle Havlovitz 1e47c757c2
command/agent: don't re-parse the flags on reload 2018-05-31 16:59:51 -07:00
Paul Banks 06e1a62653
Merge pull request #4016 from pierresouchay/support_for_prometheus
Support for prometheus for metrics endpoint
2018-04-24 16:14:43 +01:00
James Nugent e96b2b8d09 Only call signal.Notify once during agent startup
Calling twice appears to have no adverse effects, however serves to
confuse as to what the semantics of such code may be! This seems like it
was probably introduced while resolving conflicts during the merge of
the fix for #2404.
2018-04-10 20:44:50 -05:00
Pierre Souchay 2e495ec8a6 Now use prometheus_retention_time > 0 to enable prometheus support 2018-04-06 14:21:05 +02:00
Pierre Souchay 583744d8c5 Added support exposing metrics in Prometheus format 2018-04-06 09:18:06 +02:00
Guido Iaquinti 244fc72b05 Add package name to log output 2018-03-21 15:56:14 +00:00
Frank Schroeder a57e5acbf2
agent: fix TestRetryJoin 2017-10-24 20:35:37 +02:00
Frank Schroeder f398fe83b7
agent: fix TestRetryJoinFail 2017-10-24 20:35:37 +02:00
Frank Schroeder 8defdddf31
agent: fix TestRetryJoinWanFail 2017-10-24 20:35:36 +02:00
Frank Schroeder c624c72d5c
config: return error on extra command line arguments (#3397)
The `consul agent` command was ignoring extra command line arguments
which can lead to confusion when the user has for example forgotten to
add a dash in front of an argument or is not using an `=` when setting
boolean flags to `true`. `-bootstrap true` is not the same as
`-bootstrap=true`, for example.

Since all command line flags are known and we don't expect unparsed
arguments we can return an error. However, this may make it slightly
more difficult in the future if we ever wanted to have these kinds of
arguments.

Fixes #3397
2017-10-23 08:07:48 +02:00
Frank Schroeder 8f58a603ea commands: get HTTP API flags for usage automatically 2017-10-18 00:08:45 +02:00
Frank Schroeder c10885f828 commands: run all tests in parallel (again) 2017-10-18 00:08:45 +02:00
Frank Schroeder efab66e616 commands: cleanup help and synopsis.
* move Help and Synopsis to bottom
* make help and synopsis constants
* make sure help output is formatted
2017-10-18 00:08:45 +02:00
Frank Schroeder a3a805d7b8 commands: do not run cmd tests in parallel
Package level parallelization is sufficient.
2017-10-18 00:08:45 +02:00
Frank Schroeder a0b017d976 commands: move agent command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder cd837b0b18 pkg refactor
command/agent/*                  -> agent/*
    command/consul/*                 -> agent/consul/*
    command/agent/command{,_test}.go -> command/agent{,_test}.go
    command/base/command.go          -> command/base.go
    command/base/*                   -> command/*
    commands.go                      -> command/commands.go

The script which did the refactor is:

(
	cd $GOPATH/src/github.com/hashicorp/consul
	git mv command/agent/command.go command/agent.go
	git mv command/agent/command_test.go command/agent_test.go
	git mv command/agent/flag_slice_value{,_test}.go command/
	git mv command/agent .
	git mv command/base/command.go command/base.go
	git mv command/base/config_util{,_test}.go command/
	git mv commands.go command/
	git mv consul agent
	rmdir command/base/

	gsed -i -e 's|package agent|package command|' command/agent{,_test}.go
	gsed -i -e 's|package agent|package command|' command/flag_slice_value{,_test}.go
	gsed -i -e 's|package base|package command|' command/base.go command/config_util{,_test}.go
	gsed -i -e 's|package main|package command|' command/commands.go

	gsed -i -e 's|base.Command|BaseCommand|' command/commands.go
	gsed -i -e 's|agent.Command|AgentCommand|' command/commands.go
	gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/commands.go
	gsed -i -e 's|base\.||' command/commands.go
	gsed -i -e 's|command\.||' command/commands.go

	gsed -i -e 's|command|c|' main.go
	gsed -i -e 's|range Commands|range command.Commands|' main.go
	gsed -i -e 's|Commands: Commands|Commands: command.Commands|' main.go

	gsed -i -e 's|base\.BoolValue|BoolValue|' command/operator_autopilot_set.go
	gsed -i -e 's|base\.DurationValue|DurationValue|' command/operator_autopilot_set.go
	gsed -i -e 's|base\.StringValue|StringValue|' command/operator_autopilot_set.go
	gsed -i -e 's|base\.UintValue|UintValue|' command/operator_autopilot_set.go

	gsed -i -e 's|\bCommand\b|BaseCommand|' command/base.go
	gsed -i -e 's|BaseCommand Options|Command Options|' command/base.go
	gsed -i -e 's|base.Command|BaseCommand|' command/*.go
	gsed -i -e 's|c\.Command|c.BaseCommand|g' command/*.go
	gsed -i -e 's|\tCommand:|\tBaseCommand:|' command/*_test.go
	gsed -i -e 's|base\.||' command/*_test.go

	gsed -i -e 's|\bCommand\b|AgentCommand|' command/agent{,_test}.go
	gsed -i -e 's|cmd.AgentCommand|cmd.BaseCommand|' command/agent.go

	gsed -i -e 's|cli.AgentCommand = new(Command)|cli.Command = new(AgentCommand)|' command/agent_test.go
	gsed -i -e 's|exec.AgentCommand|exec.Command|' command/agent_test.go
	gsed -i -e 's|exec.BaseCommand|exec.Command|' command/agent_test.go
	gsed -i -e 's|NewTestAgent|agent.NewTestAgent|' command/agent_test.go
	gsed -i -e 's|= TestConfig|= agent.TestConfig|' command/agent_test.go
	gsed -i -e 's|: RetryJoin|: agent.RetryJoin|' command/agent_test.go

	gsed -i -e 's|\.\./\.\./|../|' command/config_util_test.go

	gsed -i -e 's|\bverifyUniqueListeners|VerifyUniqueListeners|' agent/config{,_test}.go command/agent.go
	gsed -i -e 's|\bserfLANKeyring\b|SerfLANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
	gsed -i -e 's|\bserfWANKeyring\b|SerfWANKeyring|g' agent/{agent,keyring,testagent}.go command/agent.go
	gsed -i -e 's|\bNewAgent\b|agent.New|g' command/agent{,_test}.go
	gsed -i -e 's|\bNewAgent|New|' agent/{acl_test,agent,testagent}.go

	gsed -i -e 's|\bAgent\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bBool\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bDefaultConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bDevConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bMergeConfig\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bReadConfigPaths\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bParseMetaPair\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bSerfLANKeyring\b|agent.&|g' command/agent{,_test}.go
	gsed -i -e 's|\bSerfWANKeyring\b|agent.&|g' command/agent{,_test}.go

	gsed -i -e 's|circonus\.agent|circonus|g' command/agent{,_test}.go
	gsed -i -e 's|logger\.agent|logger|g' command/agent{,_test}.go
	gsed -i -e 's|metrics\.agent|metrics|g' command/agent{,_test}.go
	gsed -i -e 's|// agent.Agent|// agent|' command/agent{,_test}.go
	gsed -i -e 's|a\.agent\.Config|a.Config|' command/agent{,_test}.go

	gsed -i -e 's|agent\.AppendSliceValue|AppendSliceValue|' command/{configtest,validate}.go

	gsed -i -e 's|consul/consul|agent/consul|' GNUmakefile

	gsed -i -e 's|\.\./test|../../test|' agent/consul/server_test.go

	# fix imports
	f=$(grep -rl 'github.com/hashicorp/consul/command/agent' * | grep '\.go')
	gsed -i -e 's|github.com/hashicorp/consul/command/agent|github.com/hashicorp/consul/agent|' $f
	goimports -w $f

	f=$(grep -rl 'github.com/hashicorp/consul/consul' * | grep '\.go')
	gsed -i -e 's|github.com/hashicorp/consul/consul|github.com/hashicorp/consul/agent/consul|' $f
	goimports -w $f

	goimports -w command/*.go main.go
)
2017-06-10 18:52:45 +02:00
Frank Schroeder c21d0f2284 test: move discover tests 2017-06-10 18:52:45 +02:00
Frank Schroeder a56d0cc95f agent: move watch plans into agent 2017-06-10 18:52:45 +02:00
preetapan f34730e3ee Fixed regression with dns server start condition (#3137) 2017-06-09 15:50:06 -07:00
Seth Vargo f4b36e8d90
Simplify 2017-06-09 14:55:04 -04:00
Seth Vargo 55161869a7
Update comment 2017-06-09 14:51:34 -04:00
Seth Vargo b20902b7e2
Parse values given to ?passing in the API
This PR fixes GH-2212 in the most backwards-compatible way I can think
of. If the user does not pass a value for `?passing`, it's assumed to be
true, which mirrors the current behavior. However, if the user passes
any value for passing, that value is parsed as a bool using strconv.

It's important to note that this is technically a breaking change.
Previously using `?passing=false` would return only passing nodes. While
this behavior is obviously incorrect, it was the previous behavior. We
should call this out very clearly in the CHANGELOG.
2017-06-09 14:36:00 -04:00
James Phillips a5cc8434ce
Opens up timing bound on flaky test. 2017-06-08 21:37:52 -07:00
James Phillips a7c910be7c
Fixes HTTP header compare by including standard headers. 2017-06-08 14:10:46 -07:00
Frank Schroeder 17051c6870 agent: fix time.Duration parsing
The duration can be passed as a string, a float64 or a time.Duration
and this patch handles this properly.
2017-06-08 22:42:49 +02:00
Frank Schroeder d284d061c9
agent: log events and exit code to the log file
This patch logs the signals, events, errors and the exit
code to the log file instead of printing it on the console.
This should provide a more complete picture for debugging.
2017-06-08 10:07:55 +02:00
Frank Schroeder d98f0f7bab
log: use prefix [ERR] instead of [ERROR] 2017-06-08 09:50:47 +02:00
James Phillips e32b42a222
Updates static assets to latest. 2017-06-07 21:16:59 -07:00
James Phillips fd7c3c5cfa Merge pull request #3121 from hashicorp/increase-graceful-timeout
agent: increase graceful shutdown timeout
2017-06-07 10:42:19 -07:00
Frank Schroeder 247e44e787
agent: increase graceful shutdown timeout
When triggering a leave through an INT/TERM signal the hard-coded
timeout of 5 seconds is too short to complete the leave successfully.
Therefore, the agent always times out.

This value should probably configurable.
2017-06-07 13:12:34 +02:00
Frank Schroeder fc2ed18423
agent: log differnt shutdown stages
Log the different code paths during shutdown.
2017-06-07 13:11:04 +02:00
Frank Schröder e206ab5cb6 agent: support custom header and method for http checks (#3106)
This patch adds support for custom headers and
method for HTTP checks.

Fixes #2474
Fixes #2657
Fixes #3106
2017-06-07 01:11:56 +02:00
Frank Schroeder 0133ab135a test: refactor TestDecodeConfig test
This patch refactors the multiple tests for
DecodeConfig into a single table-driven test
that tests most fields individually.
2017-06-04 23:55:12 +02:00
Frank Schroeder 35f2377702 test: Fix TestAgent_Reload
This test was still referring to cmd.agent which has been removed.
2017-06-04 22:45:40 +02:00
James Phillips 037f72813b Merge pull request #3105 from hashicorp/test-stability
Tweaks things for better test stability.
2017-06-03 10:09:12 -07:00
Frank Schroeder 92b2192b75 agent: fix graceful http server shutdown 2017-06-03 19:08:50 +02:00
Frank Schroeder 9779f7ef02 agent: cleanup run method 2017-06-03 19:08:50 +02:00
Frank Schroeder b3e94082b0 agent: move registerWatches out of the run method 2017-06-03 19:08:50 +02:00
Frank Schroeder 5c08953449 agent: move telemetry out of the run method 2017-06-03 19:08:50 +02:00
Frank Schroeder 7c8b2be86d agent: move update check out of the run method 2017-06-03 19:08:50 +02:00
Frank Schroeder 055347f7cd agent: simplify signal handling 2017-06-03 19:08:50 +02:00
Frank Schroeder 6262d85dab agent: move config reloading into the agent 2017-06-03 19:08:50 +02:00
Frank Schroeder 02ce45e5bb agent: move parseMetaPair to config and export 2017-06-03 19:08:50 +02:00
Frank Schroeder 34c981ba55 agent: remove agent from command
Decouple command and agent a bit more.
2017-06-03 19:08:50 +02:00
Frank Schroeder 63d336e11c agent: move retry join into agent 2017-06-03 19:08:50 +02:00
Frank Schroeder 488beeb208 agent: add GossipEncrypted function
Remove dead code and expose a GossipEncrypted() method
on the agent instead of accessing the internal delegate.
2017-06-03 19:08:50 +02:00
James Phillips 26677b4117
Cleans up an unrelated vet error. 2017-06-02 21:07:15 -07:00
James Phillips 3795f17b3b
Adds back the port shuffle on agent start retries. 2017-06-02 20:53:20 -07:00
Frank Schroeder d314a539da Do not recurse when the service address is a local cname
Fixes #2795
2017-06-01 18:20:31 +02:00
Frank Schroeder 7f18fe1a5b agent: LastContact cannot be negative (#3067)
The X-Consul-LastContact header cannot be negative.

Fixes #3067
2017-06-01 18:20:12 +02:00
Frank Schroeder 00117ed09b
test: we will not use dev mode for testing for now 2017-05-31 11:34:00 +02:00
Frank Schroeder d14182c2de
test: stagger retry attempts of TestAgent.Start 2017-05-31 11:33:25 +02:00
Frank Schroeder 5224699341
test: make TestAgent setup reusable
Pick the random ports only once and try starting with them
a number of times so that the configuration can be re-used.
This is because the ports are written into the data files
and a subsequent agent reading the files needs to have the
same ports.

For the same reason we do not remove the data directory on
every attempt since this makes it impossible to re-read the
data files.
2017-05-31 11:32:31 +02:00
Frank Schroeder b06c969deb
agent: use agent logger for consul client and deps 2017-05-31 11:05:02 +02:00
Frank Schroeder 2588f22be7
test: start secondary agents also via TestAgent 2017-05-31 10:56:19 +02:00
Frank Schroeder 4e6e9cb05a
agent: set tcpKeepAliveListener when there is no error 2017-05-31 10:24:32 +02:00
Frank Schroeder cfe101ce49
test: use test name as log prefix 2017-05-31 09:54:09 +02:00
Frank Schroeder c587561572
agent: simplify shutdown timeout logging
Note that we are currently timing out every time. This still
needs to be investigated.
2017-05-31 09:41:58 +02:00
Frank Schroeder b220133785
agent: set proto to http since we are logging it now 2017-05-31 09:41:16 +02:00
Frank Schroeder 361b69bb30
test: Use Bool() instead of &BoolTrue/False since 2017-05-31 09:21:01 +02:00
Frank Schroeder bb8a21c385
Address review comments 2017-05-31 01:05:21 +02:00
Frank Schroeder 78cd7a9087
agent: start HTTPS only if there is a cert and key 2017-05-31 00:29:29 +02:00
Frank Schroeder 45a9c2be8b
agent: refactor DNS and HTTP server
* refactor DNS server to be ready for multiple bind addresses
* drop tcpKeepAliveListener since it is default for the HTTP servers
* add startup timeout watcher for HTTP servers identical to DNS server
2017-05-31 00:29:29 +02:00
Frank Schroeder 35757e6c09
test: increase sleep time for test 2017-05-31 00:29:28 +02:00
Frank Schroeder 42ade30551
test: allow tests to control anti-entropy syncs 2017-05-31 00:29:28 +02:00
Frank Schroeder 5735b31ebc
test: run mock dns recursors on random ports 2017-05-31 00:29:28 +02:00
Frank Schroeder d07f2a2542
test: skip tests that use cmd.Run until signal handling is fixed 2017-05-31 00:29:28 +02:00
Frank Schroeder 83d9fb77b7
test: add test for starting/stopping lots of agents 2017-05-31 00:29:28 +02:00
Frank Schroeder dc1548af06
test: use UniqueID prefix for check tests to unify log output 2017-05-31 00:29:28 +02:00
Frank Schroeder 828f40054a
test: improve TestAgent
* don't use retry to try restarting the agent
  this caused some issues when the startup would fail in
  a separate go routine

* clear out the data directory on every retry since the ports
  are stored in the raft data files

* set a unique id for every agent to allow for tracking of
  concurrent output
2017-05-31 00:29:28 +02:00
Frank Schroeder 2b627b95dd
agent: fix logging
* use agent logger for consul/serf/raft/dns/agent/...
* support optional id for concurrent tests
2017-05-31 00:29:27 +02:00
Frank Schroeder a433d13b20
test: call t.Fatal() from main go routine 2017-05-31 00:29:27 +02:00
Frank Schroeder 9d4529972e
dns: do not log socket close during shutdown 2017-05-31 00:29:27 +02:00
Frank Schroeder ae5270a1e1
agent: print more useful shutdown message 2017-05-31 00:29:27 +02:00
Frank Schroeder f53b90c16b
test: start sync after agent start 2017-05-31 00:29:27 +02:00
Frank Schroeder bb3b099d60
test: add test name to output 2017-05-31 00:29:27 +02:00
Frank Schroeder 9fbac59f78
test: perform blocking query on agent 2017-05-31 00:29:27 +02:00
Frank Schroeder 36a2312639
test: parallelize dns test 2017-05-31 00:29:26 +02:00
Frank Schroeder d0cb508382
test: remove output to stdout 2017-05-31 00:29:26 +02:00
Frank Schroeder 846fe5d630
test: use cli.NewMockUi instead of new(cli.MockUi)
Use the constructor function to prevent a nil panic
if there was no write to the output buffers.
2017-05-31 00:29:26 +02:00
Frank Schroeder ad153f0e76
test: fix data race with endpoints 2017-05-31 00:29:26 +02:00
Frank Schroeder 449eceec11
agent: shutdown delegate if created
When the TestAgent shuts down a half-started agent
the delegate may not have been created at this point.
2017-05-31 00:29:26 +02:00
Frank Schroeder 6a40fe026e
agent: shutdown dns servers on error 2017-05-31 00:29:26 +02:00
Frank Schroeder 60a47addc7
test: Shutdown half-started agent before retrying 2017-05-31 00:29:25 +02:00
Frank Schroeder d1ee368da9
test: fix data race in MockNotify
42 -> 32 data races
2017-05-31 00:29:25 +02:00
Frank Schroeder 48bff52614
test: address pull request comments 2017-05-31 00:29:25 +02:00
Frank Schroeder b9ae289e34
test: remove ACL options from default test config 2017-05-31 00:29:25 +02:00
Frank Schroeder e8651261f9
test: allocate ports in blocks of 10 2017-05-31 00:29:24 +02:00
Frank Schroeder 26474ce9c2
test: refactor command tests to use TestAgent 2017-05-31 00:29:24 +02:00