Commit graph

1700 commits

Author SHA1 Message Date
Frank Schroeder 0a9478b1f1 commands: move catalog subcommands to subdirs 2017-10-18 00:08:45 +02:00
Frank Schroeder abd7c73627 commands: move snapshot subcommands to subdirs 2017-10-18 00:08:45 +02:00
Frank Schroeder 0fb4ea3a30 commands: simplify commands.go 2017-10-18 00:08:45 +02:00
Frank Schroeder 639bf6d8ec commands: drop base command and utils 2017-10-18 00:08:45 +02:00
Frank Schroeder a0b017d976 commands: move agent command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 127bd3d295 commands: move watch command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 3f7cfca203 add token and addr to http flags 2017-10-18 00:08:45 +02:00
Frank Schroeder 6998d82be5 commands: move snapshot save command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 3a16c93eab commands: move snapshot restore command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 758199813b commands: move version command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 776cffa33d commands: move snapshot inspect command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 8724aece07 commands: move snapshot command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 602e896fb9 commands: move rtt command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 1ba816b0ae commands: move reload command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder eb0640efd0 commands: move operator autopilot set command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 3dd4841b36 commands: move operator autopilot get command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 393bf50e71 commands: move operator autopilot command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder c024fcaaa5 commands: move operator list remove-peer command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 6850c8723f commands: move operator raft list-peers command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 78aa062990 commands: move operator raft command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder fe84af448b commands: move operator command to separate pkg 2017-10-18 00:08:45 +02:00
Preetha Appan 684362b1d0 Move monitor command to its own package 2017-10-18 00:08:45 +02:00
Preetha Appan 28490d6dcb Fix import order 2017-10-18 00:08:45 +02:00
Preetha Appan 4fa3987bdc Move members command to its own package 2017-10-18 00:08:45 +02:00
Preetha Appan 8d2e6f10c7 Move maint command to its own package 2017-10-18 00:08:45 +02:00
Preetha Appan 4b06c971d2 Move lock command to its own package 2017-10-18 00:08:45 +02:00
Preetha Appan e1935590b1 Fix leave and validate commands to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 3c4363389e Fix KV CLI subcommands to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 75ec8a29a3 Fix Keyring and keygen commands to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan b8dd539037 Fix join command to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 0e1ee70a5e Fix up info and forceleave to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 95053297a9 s/initFlags/init/g 2017-10-18 00:08:45 +02:00
Preetha Appan 0cb1c92977 Fix exec and event commands to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 3bd4b738c0 Fix up catalog list services to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 8761232b2a Fix up list datacenters to build help string in constructor 2017-10-18 00:08:45 +02:00
Preetha Appan 5473dbc5a4 Fix tests by calling initFlags for each test case to reset state. 2017-10-18 00:08:45 +02:00
Preetha Appan 75ecc6a86e Better name for usage string and moving constant definition down 2017-10-18 00:08:45 +02:00
Preetha Appan 7898780106 Construct the help string in constructor using helper function. 2017-10-18 00:08:45 +02:00
Frank Schroeder 9825b6709a commands: move catalog list services to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 33a4e997b4 commands: move catalog list nodes command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder cfd110395b commands: move catalog list datacenters command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder bcf53b98d1 commands: move catalog command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 5d75449419 commands: move leave command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder ffb747a744 commands: move force-leave command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 41f13de7f5 commands: move keyring command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder b1dac89b95 commands: move kv put command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder e50c8d8f73 commands: move kv get command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 19f2dab416 commands: move kv import command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 3f0c61666d commands: move export entry struct to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 215cda4775 commands: move kv export command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 926aa72246 commands: move kv delete command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 50f039191d commands: move kv command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder bd825687dd commands: move keygen command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 74cf5c1c2c commands: move info command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder c1c883957f commands: move rExec helper to bottom 2017-10-18 00:08:45 +02:00
Frank Schroeder 8c7f013ae1 commands: move exec command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder ad50e2a410 commands: move event command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder dc3c722b2c commands: move join command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 8880ae714b commands: move validate command to separate pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder e65bd2a268 commands: move flag handling into flags pkg 2017-10-18 00:08:45 +02:00
Frank Schroeder 7568a19433 config: refactor commands to print help for flags (#3536)
This patch refactors the commands that use the mitchellh/cli library to
populate the command line flag set in both the Run() and the Help()
method. Earlier versions of the mitchellh/cli library relied on the
Run() method to populuate the flagset for generating the usage screen.
This has changed in later versions and was previously solved with a
small monkey patch to the library to restore the old behavior.

However, this makes upgrading the library difficult since the patch has
to be restored every time.

This patch addresses this by moving the command line flags into an
initFlags() method where appropriate and also moving all variables for
the flags from the Run() method into the command itself.

Fixes #3536
2017-10-18 00:08:45 +02:00
Ryan Slade 6f05ea91a3 Replace time.Now().Sub(x) with time.Since(x) 2017-10-17 20:38:24 +02:00
James Phillips e9670761f9
Cleans up some drift between the OSS and Enterprise trees. 2017-10-11 15:53:07 -07:00
Frank Schroeder db016a0c8d
command: show full help text on usage 2017-10-05 18:17:54 +02:00
Frank Schroeder 2320836d7f command: do not merge flags in-place 2017-10-05 18:08:35 +02:00
Frank Schroeder 56ebd5b30f command: simplify duplicate code 2017-10-05 18:08:35 +02:00
Frank Schroeder 3028d52158 command: drop hidden flags from base command 2017-10-05 18:08:35 +02:00
Frank Schroeder 13266a2e65 command: drop unused hidden flags for 'validate' 2017-10-05 18:08:35 +02:00
Frank Schroeder 7182829f15
command: drop legacy 'operator raft' tests 2017-10-05 17:56:16 +02:00
Frank Schroeder a4be9e2e67 command: simplify 'operator raft'
The cli library can handle subcommands. Therefore, most of the code is
no longer necessary.
2017-10-05 17:40:57 +02:00
Frank Schroeder cac55a1e80 command: don't show confusing error on usage output 2017-10-05 17:29:45 +02:00
Frank Schroeder 7bab8d3eb7 command: drop deprecated 'configtest' command
'configtest' has been replaced with 'validate'
2017-10-05 17:22:34 +02:00
Kyle Havlovitz dde743700f Merge pull request #3535 from hashicorp/metric-docs
Update metric names and add a legacy config flag
2017-10-04 17:39:16 -07:00
Kyle Havlovitz be04bfed34 Clean up subprocess handling and make shell use optional (#3509)
* Clean up handling of subprocesses and make using a shell optional

* Update docs for subprocess changes

* Fix tests for new subprocess behavior

* More cleanup of subprocesses

* Minor adjustments and cleanup for subprocess logic

* Makes the watch handler reload test use the new path.

* Adds check tests for new args path, and updates existing tests to use new path.

* Adds support for script args in Docker checks.

* Fixes the sanitize unit test.

* Adds panic for unknown watch type, and reverts back to Run().

* Adds shell option back to consul lock command.

* Adds shell option back to consul exec command.

* Adds shell back into consul watch command.

* Refactors signal forwarding and makes Windows-friendly.

* Adds a clarifying comment.

* Changes error wording to a warning.

* Scopes signals to interrupt and kill.

This avoids us trying to send SIGCHILD to the dead process.

* Adds an error for shell=false for consul exec.

* Adds notes about the deprecated script and handler fields.

* De-nests an if statement.
2017-10-04 16:48:00 -07:00
Kyle Havlovitz 0063516e5e
Update metric names and add a legacy config flag 2017-10-04 16:43:27 -07:00
Kyle Havlovitz 624be65d35
Split BaseCommand http config into a separate function 2017-09-29 12:26:14 -07:00
Kyle Havlovitz 8403a98ea7
Add base command option for hiding generated help for normal args 2017-09-28 18:43:28 -07:00
James Phillips 7f02f9dad2
Returns errors properly from the consul operator raft list-peers command. 2017-09-28 15:37:59 -07:00
Frank Schröder 707f8e329a Metrics service prefix (#3498)
* metrics: replace statsite_prefix with service_prefix

The metrics prefix isn't statsite specific and is in fact used
for all metrics providers. Since we are deprecating fields
anyway we should fix this one as well.

Fixes #3293

* Updates docs and sorts telemetry section.

* Renames to "metrics_prefix" to disambiguate with Consul services.

* Updates the change log.
2017-09-26 17:49:55 -07:00
Preetha Appan 318d0232f7 Move Raft protocol version for list peers end point to server side, fix unit tests. This fixes #3449 2017-09-26 09:35:39 -05:00
preetapan 4ced57c1f8 Merge pull request #3494 from hashicorp/enforce_json_extension
Enforce json or hcl extension to Consul config files, updated unit tests
2017-09-25 17:30:33 -05:00
James Phillips fcaa889116 Bumps default Raft protocol to version 3. (#3477)
* Changes default Raft protocol to 3.

* Changes numPeers() to report only voters.

This should have been there before, but it's more obvious that this
is incorrect now that we default the Raft protocol to 3, which puts
new servers in a read-only state while Autopilot waits for them to
become healthy.

* Fixes TestLeader_RollRaftServer.

* Fixes TestOperator_RaftRemovePeerByAddress.

* Fixes TestServer_*.

Relaxed the check for a given number of voter peers and instead do
a thorough check that all servers see each other in their Raft
configurations.

* Fixes TestACL_*.

These now just check for Raft replication to be set up, and don't
care about the number of voter peers.

* Fixes TestOperator_Raft_ListPeers.

* Fixes TestAutopilot_CleanupDeadServerPeriodic.

* Fixes TestCatalog_ListNodes_ConsistentRead_Fail.

* Fixes TestLeader_ChangeServerID and adjusts the conn pool to throw away
sockets when it sees io.EOF.

* Changes version to 1.0.0 in the options doc.

* Makes metrics test more deterministic with autopilot metrics possible.
2017-09-25 15:27:04 -07:00
Preetha Appan 1e8385df2c Enforce json or hcl extension to Consul config files, updated unit tests 2017-09-25 17:17:12 -05:00
Frank Schröder 69a088ca85 New config parser, HCL support, multiple bind addrs (#3480)
* new config parser for agent

This patch implements a new config parser for the consul agent which
makes the following changes to the previous implementation:

 * add HCL support
 * all configuration fragments in tests and for default config are
   expressed as HCL fragments
 * HCL fragments can be provided on the command line so that they
   can eventually replace the command line flags.
 * HCL/JSON fragments are parsed into a temporary Config structure
   which can be merged using reflection (all values are pointers).
   The existing merge logic of overwrite for values and append
   for slices has been preserved.
 * A single builder process generates a typed runtime configuration
   for the agent.

The new implementation is more strict and fails in the builder process
if no valid runtime configuration can be generated. Therefore,
additional validations in other parts of the code should be removed.

The builder also pre-computes all required network addresses so that no
address/port magic should be required where the configuration is used
and should therefore be removed.

* Upgrade github.com/hashicorp/hcl to support int64

* improve error messages

* fix directory permission test

* Fix rtt test

* Fix ForceLeave test

* Skip performance test for now until we know what to do

* Update github.com/hashicorp/memberlist to update log prefix

* Make memberlist use the default logger

* improve config error handling

* do not fail on non-existing data-dir

* experiment with non-uniform timeouts to get a handle on stalled leader elections

* Run tests for packages separately to eliminate the spurious port conflicts

* refactor private address detection and unify approach for ipv4 and ipv6.

Fixes #2825

* do not allow unix sockets for DNS

* improve bind and advertise addr error handling

* go through builder using test coverage

* minimal update to the docs

* more coverage tests fixed

* more tests

* fix makefile

* cleanup

* fix port conflicts with external port server 'porter'

* stop test server on error

* do not run api test that change global ENV concurrently with the other tests

* Run remaining api tests concurrently

* no need for retry with the port number service

* monkey patch race condition in go-sockaddr until we understand why that fails

* monkey patch hcl decoder race condidtion until we understand why that fails

* monkey patch spurious errors in strings.EqualFold from here

* add test for hcl decoder race condition. Run with go test -parallel 128

* Increase timeout again

* cleanup

* don't log port allocations by default

* use base command arg parsing to format help output properly

* handle -dc deprecation case in Build

* switch autopilot.max_trailing_logs to int

* remove duplicate test case

* remove unused methods

* remove comments about flag/config value inconsistencies

* switch got and want around since the error message was misleading.

* Removes a stray debug log.

* Removes a stray newline in imports.

* Fixes TestACL_Version8.

* Runs go fmt.

* Adds a default case for unknown address types.

* Reoders and reformats some imports.

* Adds some comments and fixes typos.

* Reorders imports.

* add unix socket support for dns later

* drop all deprecated flags and arguments

* fix wrong field name

* remove stray node-id file

* drop unnecessary patch section in test

* drop duplicate test

* add test for LeaveOnTerm and SkipLeaveOnInt in client mode

* drop "bla" and add clarifying comment for the test

* split up tests to support enterprise/non-enterprise tests

* drop raft multiplier and derive values during build phase

* sanitize runtime config reflectively and add test

* detect invalid config fields

* fix tests with invalid config fields

* use different values for wan sanitiziation test

* drop recursor in favor of recursors

* allow dns_config.udp_answer_limit to be zero

* make sure tests run on machines with multiple ips

* Fix failing tests in a few more places by providing a bind address in the test

* Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder.

* Add porter to server_test.go to make tests there less flaky

* go fmt
2017-09-25 11:40:42 -07:00
James Phillips 02a3f3f27b
Shows the segment name in the keyring API and command output. 2017-09-07 12:17:39 -07:00
James Phillips cc6797dd75
Remaps servers to "<all>" only when listing multiple segments. 2017-09-05 13:50:38 -07:00
James Phillips c629773b40
Makes the all segments query explict, and the default for consul members. 2017-09-05 12:22:20 -07:00
James Phillips 0b44abd9db
Simplifies members command to not fetch segments. 2017-09-05 11:46:44 -07:00
Kyle Havlovitz 1a36ca524f
Remove duplicate error message 2017-09-01 10:18:43 -07:00
Kyle Havlovitz 5cc4b32a5d
Organize segments for a cleaner split between enterprise and OSS 2017-08-31 17:39:46 -07:00
Kyle Havlovitz b77a0aa932
Fix some inconsistencies with segment logic and comments 2017-08-30 17:43:46 -07:00
Kyle Havlovitz d9fc2b3d75
Update coord display in ui to account for segments 2017-08-30 11:58:29 -07:00
Kyle Havlovitz 1c04f1537a
Add agent.segment interpolation to prepared queries 2017-08-30 11:58:29 -07:00
Kyle Havlovitz 107d7f6c5a
Add rpc_listener option to segment config 2017-08-30 11:58:29 -07:00
James Phillips 6a6eadd8c7
Adds open source side of network segments (feature is Enterprise-only). 2017-08-30 11:58:29 -07:00
Frank Schroeder 62c77d70f0 build: make tests independent of build tags
When the metadata server is scanning the agents for potential servers
it is parsing the version number which the agent provided when it
joined. This version number has to conform to a certain format, i.e.
'n.n.n'. Without this version number properly set some tests fail with
error messages that disguise the root cause.

The default version number is currently set to 'unknown' in
version/version.go which does not parse and triggers the tests to fail.
The work around is to use a build tag 'consul' which will use the
version number set in version_base.go instead which has the correct
format and is set to the current release version.

In addition, some parts of the code also require the version number to
be of a certain value. Setting it to '0.0.0' for example makes some
tests pass and others fail since they don't pass the semantic check.

When using go build/install/test one has to remember to use '-tags
consul' or tests will fail with non-obvious error messages.

Using build tags makes the build process more complex and error prone
since it prevents the use of the plain go toolchain and - at least in
its current form - introduces subtle build and test issues. We should
try to eliminate build tags for anything else but platform specific
code.

This patch removes all references to specific version numbers in the
code and tests and sets the default version to '9.9.9' which is
syntactically correct and passes the semantic check. This solves the
issue of running go build/install/test without tags for the OSS build.
2017-08-30 13:40:18 +02:00
wuxin 792a535c87 fix command/kv_import.go help text (#3387) 2017-08-10 09:17:37 +02:00
Frank Schröder 32d4eecc1a agent: honor deprecated flags for retry-join-{ec2,azure,gce} (#3384) 2017-08-09 16:18:30 -07:00
Frank Schroeder 1d0bbfed9c
agent: move agent/consul/structs to agent/structs 2017-08-09 14:32:12 +02:00
Kyle Havlovitz 975ded2714
Add support for labels/filters from go-metrics 2017-08-08 01:45:10 -07:00