From 69a088ca85e710f41c176441f7ff9a37c86908c2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Frank=20Schr=C3=B6der?= Date: Mon, 25 Sep 2017 20:40:42 +0200 Subject: [PATCH] New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt --- GNUmakefile | 26 +- agent/acl.go | 5 +- agent/acl_endpoint_test.go | 8 +- agent/acl_test.go | 222 +- agent/agent.go | 416 +- agent/agent_endpoint.go | 7 +- agent/agent_endpoint_test.go | 193 +- agent/agent_test.go | 413 +- agent/catalog_endpoint_test.go | 96 +- agent/config.go | 2259 ----------- agent/config/builder.go | 1174 ++++++ agent/config/config.go | 423 ++ agent/config/default.go | 243 ++ agent/config/doc.go | 81 + agent/config/flags.go | 104 + agent/config/flags_test.go | 75 + agent/config/flagset.go | 200 + agent/config/merge.go | 59 + agent/config/merge_test.go | 57 + agent/config/patch_hcl.go | 73 + agent/config/patch_hcl_test.go | 78 + agent/config/runtime.go | 258 ++ agent/config/runtime_test.go | 3560 +++++++++++++++++ agent/config/segments_oss.go | 17 + agent/config/segments_oss_test.go | 45 + agent/config_test.go | 1648 -------- agent/consul/client_serf.go | 7 +- agent/consul/config.go | 14 - agent/consul/server_serf.go | 7 +- agent/consul/server_test.go | 13 +- agent/coordinate_endpoint_test.go | 4 +- agent/dns.go | 60 +- agent/dns_test.go | 542 ++- agent/event_endpoint_test.go | 16 +- agent/health_endpoint_test.go | 46 +- agent/http.go | 14 +- agent/http_test.go | 72 +- agent/keyring_test.go | 66 +- agent/kvs_endpoint_test.go | 18 +- agent/local.go | 5 +- agent/local_test.go | 129 +- agent/operator_endpoint_test.go | 60 +- agent/prepared_query_endpoint.go | 4 +- agent/prepared_query_endpoint_test.go | 48 +- agent/remote_exec_test.go | 83 +- agent/retry_join.go | 12 +- agent/segment_stub.go | 26 +- agent/session_endpoint_test.go | 34 +- agent/snapshot_endpoint_test.go | 8 +- agent/status_endpoint_test.go | 8 +- agent/structs/check_definition_test.go | 43 + agent/structs/operator.go | 18 + agent/testagent.go | 175 +- agent/testagent_test.go | 28 + agent/translate_addr.go | 4 +- agent/txn_endpoint_test.go | 238 +- agent/ui_endpoint_test.go | 12 +- agent/user_event.go | 2 +- agent/user_event_test.go | 12 +- agent/util.go | 49 +- agent/util_test.go | 12 +- api/agent_test.go | 5 +- api/api.go | 6 +- api/api_test.go | 10 +- api/catalog_test.go | 3 + api/health_test.go | 2 + api/operator_keyring_test.go | 2 +- command/agent.go | 601 +-- command/agent_test.go | 248 +- command/catalog_list_datacenters_test.go | 2 +- command/catalog_list_nodes_test.go | 2 +- command/catalog_list_services_test.go | 2 +- command/configtest.go | 8 +- command/configtest_test.go | 41 +- command/event_test.go | 2 +- command/exec_test.go | 50 +- command/force_leave_test.go | 8 +- command/info_test.go | 2 +- command/join_test.go | 13 +- command/keyring_test.go | 6 +- command/kv_delete_test.go | 6 +- command/kv_export_test.go | 2 +- command/kv_get_test.go | 16 +- command/kv_import_test.go | 2 +- command/kv_put_test.go | 16 +- command/leave_test.go | 4 +- command/lock_test.go | 16 +- command/maint_test.go | 12 +- command/members_test.go | 10 +- command/operator_autopilot_get_test.go | 2 +- command/operator_autopilot_set_test.go | 2 +- command/operator_raft_list_test.go | 4 +- command/operator_raft_remove_test.go | 2 +- command/reload_test.go | 2 +- command/rtt_test.go | 14 +- command/snapshot_inspect_test.go | 2 +- command/snapshot_restore_test.go | 2 +- command/snapshot_save_test.go | 2 +- command/validate.go | 9 +- command/validate_test.go | 45 +- command/version.go | 14 +- command/watch_test.go | 2 +- ipaddr/detect.go | 138 + ipaddr/detect_test.go | 48 + ipaddr/ipaddr.go | 14 +- test/porter/client.go | 36 + test/porter/cmd/porter/main.go | 133 + testutil/server.go | 43 +- .../hashicorp/go-sockaddr/ipv4addr.go | 3 +- vendor/github.com/hashicorp/hcl/Makefile | 1 + vendor/github.com/hashicorp/hcl/README.md | 10 + vendor/github.com/hashicorp/hcl/appveyor.yml | 5 +- vendor/github.com/hashicorp/hcl/decoder.go | 99 +- .../github.com/hashicorp/hcl/hcl/ast/ast.go | 7 +- .../hashicorp/memberlist/memberlist.go | 4 +- vendor/vendor.json | 4 +- website/source/docs/agent/options.html.md | 22 +- 117 files changed, 8849 insertions(+), 6546 deletions(-) create mode 100644 agent/config/builder.go create mode 100644 agent/config/config.go create mode 100644 agent/config/default.go create mode 100644 agent/config/doc.go create mode 100644 agent/config/flags.go create mode 100644 agent/config/flags_test.go create mode 100644 agent/config/flagset.go create mode 100644 agent/config/merge.go create mode 100644 agent/config/merge_test.go create mode 100644 agent/config/patch_hcl.go create mode 100644 agent/config/patch_hcl_test.go create mode 100644 agent/config/runtime.go create mode 100644 agent/config/runtime_test.go create mode 100644 agent/config/segments_oss.go create mode 100644 agent/config/segments_oss_test.go delete mode 100644 agent/config_test.go create mode 100644 agent/testagent_test.go create mode 100644 ipaddr/detect.go create mode 100644 ipaddr/detect_test.go create mode 100644 test/porter/client.go create mode 100644 test/porter/cmd/porter/main.go diff --git a/GNUmakefile b/GNUmakefile index 0ad26d643..4a6dc3a3f 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -34,12 +34,14 @@ bin: tools dev: vendorfmt dev-build dev-build: + @echo "--> Building consul" mkdir -p pkg/$(GOOS)_$(GOARCH)/ bin/ go install -ldflags '$(GOLDFLAGS)' -tags '$(GOTAGS)' cp $(GOPATH)/bin/consul bin/ cp $(GOPATH)/bin/consul pkg/$(GOOS)_$(GOARCH) vendorfmt: + @echo "--> Formatting vendor/vendor.json" test -x $(GOPATH)/bin/vendorfmt || go get -u github.com/magiconair/vendorfmt/cmd/vendorfmt vendorfmt @@ -56,22 +58,34 @@ cov: gocov test $(GOFILES) | gocov-html > /tmp/coverage.html open /tmp/coverage.html -test: dev-build vet +test: other-consul porter dev-build vet + @echo "--> Running go test" + @rm -f test.log exit-code go test -tags '$(GOTAGS)' -i ./... - go test $(GOTEST_FLAGS) -tags '$(GOTAGS)' -timeout 7m -v ./... 2>&1 >test.log ; echo $$? > exit-code - @echo "Exit code: `cat exit-code`" >> test.log - @echo "----" + porter go test $(GOTEST_FLAGS) -tags '$(GOTAGS)' -timeout 5m -v ./... &>test.log ; echo $$? > exit-code + @echo "Exit code: $$(cat exit-code)" >> test.log @grep -A5 'DATA RACE' test.log || true @grep -A10 'panic: test timed out' test.log || true - @grep '^PASS' test.log | uniq || true + @grep -A1 -- '--- SKIP:' test.log || true @grep -A1 -- '--- FAIL:' test.log || true @grep '^FAIL' test.log || true @test "$$TRAVIS" == "true" && cat test.log || true - @exit $$(cat exit-code) + @if [ "$$(cat exit-code)" == "0" ] ; then echo "PASS" ; exit 0 ; else exit 1 ; fi test-race: $(MAKE) GOTEST_FLAGS=-race +other-consul: + @echo "--> Checking for other consul instances" + @if ps -ef | grep 'consul agent' | grep -v grep ; then \ + echo "Found other running consul agents. This may affect your tests." ; \ + exit 1 ; \ + fi + +porter: + @echo "--> Building port number service..." + go install github.com/hashicorp/consul/test/porter/cmd/porter + cover: go test $(GOFILES) --cover diff --git a/agent/acl.go b/agent/acl.go index 7316d0159..fd057ff61 100644 --- a/agent/acl.go +++ b/agent/acl.go @@ -7,6 +7,7 @@ import ( "github.com/armon/go-metrics" "github.com/hashicorp/consul/acl" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/types" "github.com/hashicorp/golang-lru" @@ -67,7 +68,7 @@ type aclManager struct { } // newACLManager returns an ACL manager based on the given config. -func newACLManager(config *Config) (*aclManager, error) { +func newACLManager(config *config.RuntimeConfig) (*aclManager, error) { // Set up the cache from ID to ACL (we don't cache policies like the // servers; only one level). acls, err := lru.New2Q(aclCacheSize) @@ -218,7 +219,7 @@ func (m *aclManager) lookupACL(a *Agent, id string) (acl.ACL, error) { // and some is informative (e.g. catalog and health). func (a *Agent) resolveToken(id string) (acl.ACL, error) { // Disable ACLs if version 8 enforcement isn't enabled. - if !(*a.config.ACLEnforceVersion8) { + if !a.config.ACLEnforceVersion8 { return nil, nil } diff --git a/agent/acl_endpoint_test.go b/agent/acl_endpoint_test.go index 6a5874166..4e5ad6300 100644 --- a/agent/acl_endpoint_test.go +++ b/agent/acl_endpoint_test.go @@ -33,9 +33,9 @@ func makeTestACL(t *testing.T, srv *HTTPServer) string { func TestACL_Bootstrap(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLMasterToken = "" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_master_token = "" + `) defer a.Shutdown() tests := []struct { @@ -278,7 +278,7 @@ func TestACL_List(t *testing.T) { func TestACLReplicationStatus(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), TestACLConfig()) defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/acl/replication", nil) diff --git a/agent/acl_test.go b/agent/acl_test.go index 90d84b504..ff45cec00 100644 --- a/agent/acl_test.go +++ b/agent/acl_test.go @@ -2,11 +2,13 @@ package agent import ( "fmt" + "os" "strings" "testing" "time" rawacl "github.com/hashicorp/consul/acl" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/testutil" "github.com/hashicorp/consul/types" @@ -15,9 +17,18 @@ import ( func TestACL_Bad_Config(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDownPolicy = "nope" - cfg.DataDir = testutil.TempDir(t, "agent") + + dataDir := testutil.TempDir(t, "agent") + defer os.Remove(dataDir) + + cfg := TestConfig(config.Source{ + Name: "acl", + Format: "hcl", + Data: ` + acl_down_policy = "nope" + data_dir = "` + dataDir + `" + `, + }) // do not use TestAgent here since we want // the agent to fail during startup. @@ -40,33 +51,61 @@ func (m *MockServer) GetPolicy(args *structs.ACLPolicyRequest, reply *structs.AC func TestACL_Version8(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLEnforceVersion8 = Bool(false) - a := NewTestAgent(t.Name(), cfg) - defer a.Shutdown() - m := MockServer{ - // With version 8 enforcement off, this should not get called. - getPolicyFn: func(*structs.ACLPolicyRequest, *structs.ACLPolicy) error { - t.Fatalf("should not have called to server") - return nil - }, - } - if err := a.registerEndpoint("ACL", &m); err != nil { - t.Fatalf("err: %v", err) - } + t.Run("version 8 disabled", func(t *testing.T) { + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = false + `) + defer a.Shutdown() - if token, err := a.resolveToken("nope"); token != nil || err != nil { - t.Fatalf("bad: %v err: %v", token, err) - } + m := MockServer{ + getPolicyFn: func(*structs.ACLPolicyRequest, *structs.ACLPolicy) error { + t.Fatalf("should not have called to server") + return nil + }, + } + if err := a.registerEndpoint("ACL", &m); err != nil { + t.Fatalf("err: %v", err) + } + + if token, err := a.resolveToken("nope"); token != nil || err != nil { + t.Fatalf("bad: %v err: %v", token, err) + } + }) + + t.Run("version 8 enabled", func(t *testing.T) { + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) + defer a.Shutdown() + + var called bool + m := MockServer{ + getPolicyFn: func(*structs.ACLPolicyRequest, *structs.ACLPolicy) error { + called = true + return fmt.Errorf("token not found") + }, + } + if err := a.registerEndpoint("ACL", &m); err != nil { + t.Fatalf("err: %v", err) + } + + if _, err := a.resolveToken("nope"); err != nil { + t.Fatalf("err: %v", err) + } + + if !called { + t.Fatalf("bad") + } + }) } func TestACL_Disabled(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLDisabledTTL = 10 * time.Millisecond - cfg.ACLEnforceVersion8 = Bool(true) - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_disabled_ttl = "10ms" + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{ @@ -103,7 +142,7 @@ func TestACL_Disabled(t *testing.T) { // Wait the waiting period and make sure it checks again. Do a few tries // to make sure we don't think it's disabled. - time.Sleep(2 * cfg.ACLDisabledTTL) + time.Sleep(2 * 10 * time.Millisecond) for i := 0; i < 10; i++ { _, err := a.resolveToken("nope") if !rawacl.IsErrNotFound(err) { @@ -117,10 +156,10 @@ func TestACL_Disabled(t *testing.T) { func TestACL_Special_IDs(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - cfg.ACLAgentMasterToken = "towel" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + acl_agent_master_token = "towel" + `) defer a.Shutdown() m := MockServer{ @@ -159,10 +198,10 @@ func TestACL_Special_IDs(t *testing.T) { if acl == nil { t.Fatalf("should not be nil") } - if !acl.AgentRead(cfg.NodeName) { + if !acl.AgentRead(a.config.NodeName) { t.Fatalf("should be able to read agent") } - if !acl.AgentWrite(cfg.NodeName) { + if !acl.AgentWrite(a.config.NodeName) { t.Fatalf("should be able to write agent") } if !acl.NodeRead("hello") { @@ -175,11 +214,10 @@ func TestACL_Special_IDs(t *testing.T) { func TestACL_Down_Deny(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLDownPolicy = "deny" - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_down_policy = "deny" + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{ @@ -199,18 +237,17 @@ func TestACL_Down_Deny(t *testing.T) { if acl == nil { t.Fatalf("should not be nil") } - if acl.AgentRead(cfg.NodeName) { + if acl.AgentRead(a.config.NodeName) { t.Fatalf("should deny") } } func TestACL_Down_Allow(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLDownPolicy = "allow" - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_down_policy = "allow" + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{ @@ -230,18 +267,17 @@ func TestACL_Down_Allow(t *testing.T) { if acl == nil { t.Fatalf("should not be nil") } - if !acl.AgentRead(cfg.NodeName) { + if !acl.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } } func TestACL_Down_Extend(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLDownPolicy = "extend-cache" - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_down_policy = "extend-cache" + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{ @@ -252,7 +288,7 @@ func TestACL_Down_Extend(t *testing.T) { Policy: &rawacl.Policy{ Agents: []*rawacl.AgentPolicy{ &rawacl.AgentPolicy{ - Node: cfg.NodeName, + Node: a.config.NodeName, Policy: "read", }, }, @@ -272,10 +308,10 @@ func TestACL_Down_Extend(t *testing.T) { if acl == nil { t.Fatalf("should not be nil") } - if !acl.AgentRead(cfg.NodeName) { + if !acl.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } - if acl.AgentWrite(cfg.NodeName) { + if acl.AgentWrite(a.config.NodeName) { t.Fatalf("should deny") } @@ -290,10 +326,10 @@ func TestACL_Down_Extend(t *testing.T) { if acl == nil { t.Fatalf("should not be nil") } - if acl.AgentRead(cfg.NodeName) { + if acl.AgentRead(a.config.NodeName) { t.Fatalf("should deny") } - if acl.AgentWrite(cfg.NodeName) { + if acl.AgentWrite(a.config.NodeName) { t.Fatalf("should deny") } @@ -306,20 +342,19 @@ func TestACL_Down_Extend(t *testing.T) { if acl == nil { t.Fatalf("should not be nil") } - if !acl.AgentRead(cfg.NodeName) { + if !acl.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } - if acl.AgentWrite(cfg.NodeName) { + if acl.AgentWrite(a.config.NodeName) { t.Fatalf("should deny") } } func TestACL_Cache(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{ @@ -331,7 +366,7 @@ func TestACL_Cache(t *testing.T) { Policy: &rawacl.Policy{ Agents: []*rawacl.AgentPolicy{ &rawacl.AgentPolicy{ - Node: cfg.NodeName, + Node: a.config.NodeName, Policy: "read", }, }, @@ -352,10 +387,10 @@ func TestACL_Cache(t *testing.T) { if rule == nil { t.Fatalf("should not be nil") } - if !rule.AgentRead(cfg.NodeName) { + if !rule.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } - if rule.AgentWrite(cfg.NodeName) { + if rule.AgentWrite(a.config.NodeName) { t.Fatalf("should deny") } if rule.NodeRead("nope") { @@ -374,10 +409,10 @@ func TestACL_Cache(t *testing.T) { if rule == nil { t.Fatalf("should not be nil") } - if !rule.AgentRead(cfg.NodeName) { + if !rule.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } - if rule.AgentWrite(cfg.NodeName) { + if rule.AgentWrite(a.config.NodeName) { t.Fatalf("should deny") } if rule.NodeRead("nope") { @@ -403,7 +438,7 @@ func TestACL_Cache(t *testing.T) { Policy: &rawacl.Policy{ Agents: []*rawacl.AgentPolicy{ &rawacl.AgentPolicy{ - Node: cfg.NodeName, + Node: a.config.NodeName, Policy: "write", }, }, @@ -419,10 +454,10 @@ func TestACL_Cache(t *testing.T) { if rule == nil { t.Fatalf("should not be nil") } - if !rule.AgentRead(cfg.NodeName) { + if !rule.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } - if !rule.AgentWrite(cfg.NodeName) { + if !rule.AgentWrite(a.config.NodeName) { t.Fatalf("should allow") } if rule.NodeRead("nope") { @@ -449,10 +484,10 @@ func TestACL_Cache(t *testing.T) { if rule == nil { t.Fatalf("should not be nil") } - if !rule.AgentRead(cfg.NodeName) { + if !rule.AgentRead(a.config.NodeName) { t.Fatalf("should allow") } - if !rule.AgentWrite(cfg.NodeName) { + if !rule.AgentWrite(a.config.NodeName) { t.Fatalf("should allow") } if rule.NodeRead("nope") { @@ -499,10 +534,9 @@ func catalogPolicy(req *structs.ACLPolicyRequest, reply *structs.ACLPolicy) erro func TestACL_vetServiceRegister(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} @@ -545,10 +579,9 @@ func TestACL_vetServiceRegister(t *testing.T) { func TestACL_vetServiceUpdate(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} @@ -581,10 +614,9 @@ func TestACL_vetServiceUpdate(t *testing.T) { func TestACL_vetCheckRegister(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} @@ -664,10 +696,9 @@ func TestACL_vetCheckRegister(t *testing.T) { func TestACL_vetCheckUpdate(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} @@ -720,10 +751,9 @@ func TestACL_vetCheckUpdate(t *testing.T) { func TestACL_filterMembers(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} @@ -756,10 +786,9 @@ func TestACL_filterMembers(t *testing.T) { func TestACL_filterServices(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} @@ -787,10 +816,9 @@ func TestACL_filterServices(t *testing.T) { func TestACL_filterChecks(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLEnforceVersion8 = Bool(true) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_enforce_version_8 = true + `) defer a.Shutdown() m := MockServer{catalogPolicy} diff --git a/agent/agent.go b/agent/agent.go index 0c1483b95..c1a37e357 100644 --- a/agent/agent.go +++ b/agent/agent.go @@ -20,6 +20,7 @@ import ( "github.com/armon/go-metrics" "github.com/hashicorp/consul/acl" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/consul" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/agent/systemd" @@ -83,7 +84,7 @@ type notifier interface { // requests to other Consul servers. type Agent struct { // config is the agent configuration. - config *Config + config *config.RuntimeConfig // Used for writing our logs logger *log.Logger @@ -164,15 +165,9 @@ type Agent struct { endpoints map[string]string endpointsLock sync.RWMutex - // dnsAddr is the address the DNS server binds to - dnsAddrs []ProtoAddr - // dnsServer provides the DNS API dnsServers []*DNSServer - // httpAddrs are the addresses per protocol the HTTP server binds to - httpAddrs []ProtoAddr - // httpServers provides the HTTP API on various endpoints httpServers []*HTTPServer @@ -189,21 +184,13 @@ type Agent struct { tokens *token.Store } -func New(c *Config) (*Agent, error) { +func New(c *config.RuntimeConfig) (*Agent, error) { if c.Datacenter == "" { return nil, fmt.Errorf("Must configure a Datacenter") } if c.DataDir == "" && !c.DevMode { return nil, fmt.Errorf("Must configure a DataDir") } - dnsAddrs, err := c.DNSAddrs() - if err != nil { - return nil, fmt.Errorf("Invalid DNS bind address: %s", err) - } - httpAddrs, err := c.HTTPAddrs() - if err != nil { - return nil, fmt.Errorf("Invalid HTTP bind address: %s", err) - } acls, err := newACLManager(c) if err != nil { return nil, err @@ -225,8 +212,6 @@ func New(c *Config) (*Agent, error) { retryJoinCh: make(chan error), shutdownCh: make(chan struct{}), endpoints: make(map[string]string), - dnsAddrs: dnsAddrs, - httpAddrs: httpAddrs, tokens: new(token.Store), } @@ -269,7 +254,7 @@ func (a *Agent) Start() error { consulCfg.ServerUp = a.state.ConsulServerUp // Setup either the client or the server. - if c.Server { + if c.ServerMode { server, err := consul.NewServerLogger(consulCfg, a.logger, a.tokens) if err != nil { return fmt.Errorf("Failed to start Consul server: %v", err) @@ -322,14 +307,14 @@ func (a *Agent) Start() error { // create listeners and unstarted servers // see comment on listenHTTP why we are doing this - httpln, err := a.listenHTTP(a.httpAddrs) + httpln, err := a.listenHTTP() if err != nil { return err } - // start HTTP servers + // start HTTP and HTTPS servers for _, l := range httpln { - srv := NewHTTPServer(l.Addr().String(), a) + srv := NewHTTPServer(l.Addr(), a) if err := a.serveHTTP(l, srv); err != nil { return err } @@ -349,10 +334,8 @@ func (a *Agent) Start() error { } func (a *Agent) listenAndServeDNS() error { - notif := make(chan ProtoAddr, len(a.dnsAddrs)) - for _, p := range a.dnsAddrs { - p := p // capture loop var - + notif := make(chan net.Addr, len(a.config.DNSAddrs)) + for _, addr := range a.config.DNSAddrs { // create server s, err := NewDNSServer(a) if err != nil { @@ -362,22 +345,21 @@ func (a *Agent) listenAndServeDNS() error { // start server a.wgServers.Add(1) - go func() { + go func(addr net.Addr) { defer a.wgServers.Done() - - err := s.ListenAndServe(p.Net, p.Addr, func() { notif <- p }) + err := s.ListenAndServe(addr.Network(), addr.String(), func() { notif <- addr }) if err != nil && !strings.Contains(err.Error(), "accept") { - a.logger.Printf("[ERR] agent: Error starting DNS server %s (%s): %v", p.Addr, p.Net, err) + a.logger.Printf("[ERR] agent: Error starting DNS server %s (%s): %v", addr.String(), addr.Network(), err) } - }() + }(addr) } // wait for servers to be up timeout := time.After(time.Second) - for range a.dnsAddrs { + for range a.config.DNSAddrs { select { - case p := <-notif: - a.logger.Printf("[INFO] agent: Started DNS server %s (%s)", p.Addr, p.Net) + case addr := <-notif: + a.logger.Printf("[INFO] agent: Started DNS server %s (%s)", addr.String(), addr.Network()) continue case <-timeout: return fmt.Errorf("agent: timeout starting DNS servers") @@ -401,43 +383,56 @@ func (a *Agent) listenAndServeDNS() error { // // This approach should ultimately be refactored to the point where we just // start the server and any error should trigger a proper shutdown of the agent. -func (a *Agent) listenHTTP(addrs []ProtoAddr) ([]net.Listener, error) { +func (a *Agent) listenHTTP() ([]net.Listener, error) { var ln []net.Listener - for _, p := range addrs { - var l net.Listener - var err error - switch { - case p.Net == "unix": - l, err = a.listenSocket(p.Addr, a.config.UnixSockets) + start := func(proto string, addrs []net.Addr) error { + for _, addr := range addrs { + var l net.Listener + var err error - case p.Net == "tcp" && p.Proto == "http": - l, err = net.Listen("tcp", p.Addr) + switch x := addr.(type) { + case *net.UnixAddr: + l, err = a.listenSocket(x.Name) + if err != nil { + return err + } - case p.Net == "tcp" && p.Proto == "https": - var tlscfg *tls.Config - tlscfg, err = a.config.IncomingHTTPSConfig() - if err != nil { - break + case *net.TCPAddr: + l, err = net.Listen("tcp", x.String()) + if err != nil { + return err + } + + l = &tcpKeepAliveListener{l.(*net.TCPListener)} + + if proto == "https" { + tlscfg, err := a.config.IncomingHTTPSConfig() + if err != nil { + return err + } + l = tls.NewListener(l, tlscfg) + } + + default: + return fmt.Errorf("unsupported address type %T", addr) } - l, err = tls.Listen("tcp", p.Addr, tlscfg) - - default: - return nil, fmt.Errorf("%s:%s listener not supported", p.Net, p.Proto) + ln = append(ln, l) } + return nil + } - if err != nil { - for _, l := range ln { - l.Close() - } - return nil, err + if err := start("http", a.config.HTTPAddrs); err != nil { + for _, l := range ln { + l.Close() } - - if tcpl, ok := l.(*net.TCPListener); ok { - l = &tcpKeepAliveListener{tcpl} + return nil, err + } + if err := start("https", a.config.HTTPSAddrs); err != nil { + for _, l := range ln { + l.Close() } - - ln = append(ln, l) + return nil, err } return ln, nil } @@ -459,7 +454,7 @@ func (ln tcpKeepAliveListener) Accept() (c net.Conn, err error) { return tc, nil } -func (a *Agent) listenSocket(path string, perm FilePermissions) (net.Listener, error) { +func (a *Agent) listenSocket(path string) (net.Listener, error) { if _, err := os.Stat(path); !os.IsNotExist(err) { a.logger.Printf("[WARN] agent: Replacing socket %q", path) } @@ -470,8 +465,9 @@ func (a *Agent) listenSocket(path string, perm FilePermissions) (net.Listener, e if err != nil { return nil, err } - if err := setFilePermissions(path, perm); err != nil { - return nil, fmt.Errorf("Failed setting up HTTP socket: %s", err) + user, group, mode := a.config.UnixSocketUser, a.config.UnixSocketGroup, a.config.UnixSocketMode + if err := setFilePermissions(path, user, group, mode); err != nil { + return nil, fmt.Errorf("Failed setting up socket: %s", err) } return l, nil } @@ -492,11 +488,11 @@ func (a *Agent) serveHTTP(l net.Listener, srv *HTTPServer) error { if strings.Contains("*tls.listener", fmt.Sprintf("%T", l)) { srv.proto = "https" } - notif := make(chan string) + notif := make(chan net.Addr) a.wgServers.Add(1) go func() { defer a.wgServers.Done() - notif <- srv.Addr + notif <- l.Addr() err := srv.Serve(l) if err != nil && err != http.ErrServerClosed { a.logger.Print(err) @@ -506,9 +502,9 @@ func (a *Agent) serveHTTP(l net.Listener, srv *HTTPServer) error { select { case addr := <-notif: if srv.proto == "https" { - a.logger.Printf("[INFO] agent: Started HTTPS server on %s", addr) + a.logger.Printf("[INFO] agent: Started HTTPS server on %s (%s)", addr.String(), addr.Network()) } else { - a.logger.Printf("[INFO] agent: Started HTTP server on %s", addr) + a.logger.Printf("[INFO] agent: Started HTTP server on %s (%s)", addr.String(), addr.Network()) } return nil case <-time.After(time.Second): @@ -518,32 +514,49 @@ func (a *Agent) serveHTTP(l net.Listener, srv *HTTPServer) error { // reloadWatches stops any existing watch plans and attempts to load the given // set of watches. -func (a *Agent) reloadWatches(cfg *Config) error { +func (a *Agent) reloadWatches(cfg *config.RuntimeConfig) error { // Watches use the API to talk to this agent, so that must be enabled. - addrs, err := cfg.HTTPAddrs() - if err != nil { - return err - } - if len(addrs) == 0 { + if len(cfg.HTTPAddrs) == 0 { return fmt.Errorf("watch plans require an HTTP or HTTPS endpoint") } + // Compile the watches + var watchPlans []*watch.Plan + for _, params := range cfg.Watches { + // Parse the watches, excluding the handler + wp, err := watch.ParseExempt(params, []string{"handler"}) + if err != nil { + return fmt.Errorf("Failed to parse watch (%#v): %v", params, err) + } + + // Get the handler + h := wp.Exempt["handler"] + if _, ok := h.(string); h == nil || !ok { + return fmt.Errorf("Watch handler must be a string") + } + + // Store the watch plan + watchPlans = append(watchPlans, wp) + } + // Stop the current watches. for _, wp := range a.watchPlans { wp.Stop() } a.watchPlans = nil + // deterine the primary http endpoint + addr := cfg.HTTPAddrs[0].String() + if cfg.HTTPAddrs[0].Network() == "unix" { + addr = "unix://" + addr + } + // Fire off a goroutine for each new watch plan. - for _, wp := range cfg.WatchPlans { + for _, wp := range watchPlans { a.watchPlans = append(a.watchPlans, wp) go func(wp *watch.Plan) { wp.Handler = makeWatchHandler(a.LogOutput, wp.Exempt["handler"]) wp.LogOutput = a.LogOutput - addr := addrs[0].String() - if addrs[0].Net == "unix" { - addr = "unix://" + addr - } if err := wp.Run(addr); err != nil { a.logger.Printf("[ERR] Failed to run watch: %v", err) } @@ -557,99 +570,60 @@ func (a *Agent) consulConfig() (*consul.Config, error) { // Start with the provided config or default config base := consul.DefaultConfig() - // a.config.ConsulConfig, if set, is a partial configuration for the - // consul server or client. Therefore, clone and augment it but - // don't use it as base directly. - if a.config.ConsulConfig != nil { - base = new(consul.Config) - *base = *a.config.ConsulConfig - } - // This is set when the agent starts up base.NodeID = a.config.NodeID // Apply dev mode base.DevMode = a.config.DevMode - // Apply performance factors - if a.config.Performance.RaftMultiplier > 0 { - base.ScaleRaft(a.config.Performance.RaftMultiplier) - } - // Override with our config - if a.config.Datacenter != "" { - base.Datacenter = a.config.Datacenter - } - if a.config.DataDir != "" { - base.DataDir = a.config.DataDir - } - if a.config.NodeName != "" { - base.NodeName = a.config.NodeName - } - if a.config.Ports.SerfLan != 0 { - base.SerfLANConfig.MemberlistConfig.BindPort = a.config.Ports.SerfLan - base.SerfLANConfig.MemberlistConfig.AdvertisePort = a.config.Ports.SerfLan - } - if a.config.Ports.SerfWan != 0 { - base.SerfWANConfig.MemberlistConfig.BindPort = a.config.Ports.SerfWan - base.SerfWANConfig.MemberlistConfig.AdvertisePort = a.config.Ports.SerfWan - } - if a.config.BindAddr != "" { - bindAddr := &net.TCPAddr{ - IP: net.ParseIP(a.config.BindAddr), - Port: a.config.Ports.Server, - } - base.RPCAddr = bindAddr + // todo(fs): these are now always set in the runtime config so we can simplify this + // todo(fs): or is there a reason to keep it like that? + base.Datacenter = a.config.Datacenter + base.DataDir = a.config.DataDir + base.NodeName = a.config.NodeName - // Set the Serf configs using the old default behavior, we may - // override these in the code right below. - base.SerfLANConfig.MemberlistConfig.BindAddr = a.config.BindAddr - base.SerfWANConfig.MemberlistConfig.BindAddr = a.config.BindAddr + base.CoordinateUpdateBatchSize = a.config.ConsulCoordinateUpdateBatchSize + base.CoordinateUpdateMaxBatches = a.config.ConsulCoordinateUpdateMaxBatches + base.CoordinateUpdatePeriod = a.config.ConsulCoordinateUpdatePeriod + + base.RaftConfig.HeartbeatTimeout = a.config.ConsulRaftHeartbeatTimeout + base.RaftConfig.LeaderLeaseTimeout = a.config.ConsulRaftLeaderLeaseTimeout + base.RaftConfig.ElectionTimeout = a.config.ConsulRaftElectionTimeout + + base.SerfLANConfig.MemberlistConfig.BindAddr = a.config.SerfBindAddrLAN.IP.String() + base.SerfLANConfig.MemberlistConfig.BindPort = a.config.SerfPortLAN + base.SerfLANConfig.MemberlistConfig.AdvertiseAddr = a.config.SerfAdvertiseAddrLAN.IP.String() + base.SerfLANConfig.MemberlistConfig.AdvertisePort = a.config.SerfPortLAN + base.SerfLANConfig.MemberlistConfig.GossipVerifyIncoming = a.config.EncryptVerifyIncoming + base.SerfLANConfig.MemberlistConfig.GossipVerifyOutgoing = a.config.EncryptVerifyOutgoing + base.SerfLANConfig.MemberlistConfig.GossipInterval = a.config.ConsulSerfLANGossipInterval + base.SerfLANConfig.MemberlistConfig.ProbeInterval = a.config.ConsulSerfLANProbeInterval + base.SerfLANConfig.MemberlistConfig.ProbeTimeout = a.config.ConsulSerfLANProbeTimeout + base.SerfLANConfig.MemberlistConfig.SuspicionMult = a.config.ConsulSerfLANSuspicionMult + + base.SerfWANConfig.MemberlistConfig.BindAddr = a.config.SerfBindAddrWAN.IP.String() + base.SerfWANConfig.MemberlistConfig.BindPort = a.config.SerfPortWAN + base.SerfWANConfig.MemberlistConfig.AdvertiseAddr = a.config.SerfAdvertiseAddrWAN.IP.String() + base.SerfWANConfig.MemberlistConfig.AdvertisePort = a.config.SerfPortWAN + base.SerfWANConfig.MemberlistConfig.GossipVerifyIncoming = a.config.EncryptVerifyIncoming + base.SerfWANConfig.MemberlistConfig.GossipVerifyOutgoing = a.config.EncryptVerifyOutgoing + base.SerfWANConfig.MemberlistConfig.GossipInterval = a.config.ConsulSerfWANGossipInterval + base.SerfWANConfig.MemberlistConfig.ProbeInterval = a.config.ConsulSerfWANProbeInterval + base.SerfWANConfig.MemberlistConfig.ProbeTimeout = a.config.ConsulSerfWANProbeTimeout + base.SerfWANConfig.MemberlistConfig.SuspicionMult = a.config.ConsulSerfWANSuspicionMult + + base.RPCAddr = a.config.RPCBindAddr + base.RPCAdvertise = a.config.RPCAdvertiseAddr + + if a.config.ReconnectTimeoutLAN != 0 { + base.SerfLANConfig.ReconnectTimeout = a.config.ReconnectTimeoutLAN } - if a.config.SerfLanBindAddr != "" { - base.SerfLANConfig.MemberlistConfig.BindAddr = a.config.SerfLanBindAddr - } - if a.config.SerfWanBindAddr != "" { - base.SerfWANConfig.MemberlistConfig.BindAddr = a.config.SerfWanBindAddr + if a.config.ReconnectTimeoutWAN != 0 { + base.SerfWANConfig.ReconnectTimeout = a.config.ReconnectTimeoutWAN } - if a.config.AdvertiseAddr != "" { - base.SerfLANConfig.MemberlistConfig.AdvertiseAddr = a.config.AdvertiseAddr - base.SerfWANConfig.MemberlistConfig.AdvertiseAddr = a.config.AdvertiseAddr - if a.config.AdvertiseAddrWan != "" { - base.SerfWANConfig.MemberlistConfig.AdvertiseAddr = a.config.AdvertiseAddrWan - } - base.RPCAdvertise = &net.TCPAddr{ - IP: net.ParseIP(a.config.AdvertiseAddr), - Port: a.config.Ports.Server, - } - } - if a.config.AdvertiseAddrs.SerfLan != nil { - base.SerfLANConfig.MemberlistConfig.AdvertiseAddr = a.config.AdvertiseAddrs.SerfLan.IP.String() - base.SerfLANConfig.MemberlistConfig.AdvertisePort = a.config.AdvertiseAddrs.SerfLan.Port - } - if a.config.AdvertiseAddrs.SerfWan != nil { - base.SerfWANConfig.MemberlistConfig.AdvertiseAddr = a.config.AdvertiseAddrs.SerfWan.IP.String() - base.SerfWANConfig.MemberlistConfig.AdvertisePort = a.config.AdvertiseAddrs.SerfWan.Port - } - if a.config.ReconnectTimeoutLan != 0 { - base.SerfLANConfig.ReconnectTimeout = a.config.ReconnectTimeoutLan - } - if a.config.ReconnectTimeoutWan != 0 { - base.SerfWANConfig.ReconnectTimeout = a.config.ReconnectTimeoutWan - } - if a.config.EncryptVerifyIncoming != nil { - base.SerfWANConfig.MemberlistConfig.GossipVerifyIncoming = *a.config.EncryptVerifyIncoming - base.SerfLANConfig.MemberlistConfig.GossipVerifyIncoming = *a.config.EncryptVerifyIncoming - } - if a.config.EncryptVerifyOutgoing != nil { - base.SerfWANConfig.MemberlistConfig.GossipVerifyOutgoing = *a.config.EncryptVerifyOutgoing - base.SerfLANConfig.MemberlistConfig.GossipVerifyOutgoing = *a.config.EncryptVerifyOutgoing - } - if a.config.AdvertiseAddrs.RPC != nil { - base.RPCAdvertise = a.config.AdvertiseAddrs.RPC - } - base.Segment = a.config.Segment + base.Segment = a.config.SegmentName if len(a.config.Segments) > 0 { segments, err := a.segmentConfig() if err != nil { @@ -666,8 +640,8 @@ func (a *Agent) consulConfig() (*consul.Config, error) { if a.config.BootstrapExpect != 0 { base.BootstrapExpect = a.config.BootstrapExpect } - if a.config.Protocol > 0 { - base.ProtocolVersion = uint8(a.config.Protocol) + if a.config.RPCProtocol > 0 { + base.ProtocolVersion = uint8(a.config.RPCProtocol) } if a.config.RaftProtocol != 0 { base.RaftConfig.ProtocolVersion = raft.ProtocolVersion(a.config.RaftProtocol) @@ -678,7 +652,7 @@ func (a *Agent) consulConfig() (*consul.Config, error) { if a.config.ACLDatacenter != "" { base.ACLDatacenter = a.config.ACLDatacenter } - if a.config.ACLTTLRaw != "" { + if a.config.ACLTTL != 0 { base.ACLTTL = a.config.ACLTTL } if a.config.ACLDefaultPolicy != "" { @@ -688,35 +662,35 @@ func (a *Agent) consulConfig() (*consul.Config, error) { base.ACLDownPolicy = a.config.ACLDownPolicy } base.EnableACLReplication = a.config.EnableACLReplication - if a.config.ACLEnforceVersion8 != nil { - base.ACLEnforceVersion8 = *a.config.ACLEnforceVersion8 + if a.config.ACLEnforceVersion8 { + base.ACLEnforceVersion8 = a.config.ACLEnforceVersion8 } - if a.config.SessionTTLMinRaw != "" { + if a.config.SessionTTLMin != 0 { base.SessionTTLMin = a.config.SessionTTLMin } - if a.config.Autopilot.CleanupDeadServers != nil { - base.AutopilotConfig.CleanupDeadServers = *a.config.Autopilot.CleanupDeadServers + if a.config.AutopilotCleanupDeadServers { + base.AutopilotConfig.CleanupDeadServers = a.config.AutopilotCleanupDeadServers } - if a.config.Autopilot.LastContactThreshold != nil { - base.AutopilotConfig.LastContactThreshold = *a.config.Autopilot.LastContactThreshold + if a.config.AutopilotLastContactThreshold != 0 { + base.AutopilotConfig.LastContactThreshold = a.config.AutopilotLastContactThreshold } - if a.config.Autopilot.MaxTrailingLogs != nil { - base.AutopilotConfig.MaxTrailingLogs = *a.config.Autopilot.MaxTrailingLogs + if a.config.AutopilotMaxTrailingLogs != 0 { + base.AutopilotConfig.MaxTrailingLogs = uint64(a.config.AutopilotMaxTrailingLogs) } - if a.config.Autopilot.ServerStabilizationTime != nil { - base.AutopilotConfig.ServerStabilizationTime = *a.config.Autopilot.ServerStabilizationTime + if a.config.AutopilotServerStabilizationTime != 0 { + base.AutopilotConfig.ServerStabilizationTime = a.config.AutopilotServerStabilizationTime } if a.config.NonVotingServer { base.NonVoter = a.config.NonVotingServer } - if a.config.Autopilot.RedundancyZoneTag != "" { - base.AutopilotConfig.RedundancyZoneTag = a.config.Autopilot.RedundancyZoneTag + if a.config.AutopilotRedundancyZoneTag != "" { + base.AutopilotConfig.RedundancyZoneTag = a.config.AutopilotRedundancyZoneTag } - if a.config.Autopilot.DisableUpgradeMigration != nil { - base.AutopilotConfig.DisableUpgradeMigration = *a.config.Autopilot.DisableUpgradeMigration + if a.config.AutopilotDisableUpgradeMigration { + base.AutopilotConfig.DisableUpgradeMigration = a.config.AutopilotDisableUpgradeMigration } - if a.config.Autopilot.UpgradeVersionTag != "" { - base.AutopilotConfig.UpgradeVersionTag = a.config.Autopilot.UpgradeVersionTag + if a.config.AutopilotUpgradeVersionTag != "" { + base.AutopilotConfig.UpgradeVersionTag = a.config.AutopilotUpgradeVersionTag } // make sure the advertise address is always set @@ -725,11 +699,11 @@ func (a *Agent) consulConfig() (*consul.Config, error) { } // Rate limiting for RPC calls. - if a.config.Limits.RPCRate > 0 { - base.RPCRate = a.config.Limits.RPCRate + if a.config.RPCRateLimit > 0 { + base.RPCRate = a.config.RPCRateLimit } - if a.config.Limits.RPCMaxBurst > 0 { - base.RPCMaxBurst = a.config.Limits.RPCMaxBurst + if a.config.RPCMaxBurst > 0 { + base.RPCMaxBurst = a.config.RPCMaxBurst } // set the src address for outgoing rpc connections @@ -757,7 +731,7 @@ func (a *Agent) consulConfig() (*consul.Config, error) { base.CertFile = a.config.CertFile base.KeyFile = a.config.KeyFile base.ServerName = a.config.ServerName - base.Domain = a.config.Domain + base.Domain = a.config.DNSDomain base.TLSMinVersion = a.config.TLSMinVersion base.TLSCipherSuites = a.config.TLSCipherSuites base.TLSPreferServerCipherSuites = a.config.TLSPreferServerCipherSuites @@ -787,45 +761,37 @@ func (a *Agent) segmentConfig() ([]consul.NetworkSegment, error) { var segments []consul.NetworkSegment config := a.config - for _, segment := range config.Segments { + for _, s := range config.Segments { serfConf := consul.DefaultConfig().SerfLANConfig - if segment.Advertise != "" { - serfConf.MemberlistConfig.AdvertiseAddr = segment.Advertise - } else { - serfConf.MemberlistConfig.AdvertiseAddr = a.config.AdvertiseAddr - } - if segment.Bind != "" { - serfConf.MemberlistConfig.BindAddr = segment.Bind - } else { - serfConf.MemberlistConfig.BindAddr = a.config.BindAddr - } - serfConf.MemberlistConfig.AdvertisePort = segment.Port - serfConf.MemberlistConfig.BindPort = segment.Port + serfConf.MemberlistConfig.BindAddr = s.Bind.IP.String() + serfConf.MemberlistConfig.BindPort = s.Bind.Port + serfConf.MemberlistConfig.AdvertiseAddr = s.Advertise.IP.String() + serfConf.MemberlistConfig.AdvertisePort = s.Advertise.Port - if config.ReconnectTimeoutLan != 0 { - serfConf.ReconnectTimeout = config.ReconnectTimeoutLan + if config.ReconnectTimeoutLAN != 0 { + serfConf.ReconnectTimeout = config.ReconnectTimeoutLAN } - if config.EncryptVerifyIncoming != nil { - serfConf.MemberlistConfig.GossipVerifyIncoming = *config.EncryptVerifyIncoming + if config.EncryptVerifyIncoming { + serfConf.MemberlistConfig.GossipVerifyIncoming = config.EncryptVerifyIncoming } - if config.EncryptVerifyOutgoing != nil { - serfConf.MemberlistConfig.GossipVerifyOutgoing = *config.EncryptVerifyOutgoing + if config.EncryptVerifyOutgoing { + serfConf.MemberlistConfig.GossipVerifyOutgoing = config.EncryptVerifyOutgoing } var rpcAddr *net.TCPAddr - if segment.RPCListener { + if s.RPCListener { rpcAddr = &net.TCPAddr{ - IP: net.ParseIP(segment.Bind), - Port: a.config.Ports.Server, + IP: s.Bind.IP, + Port: a.config.ServerPort, } } segments = append(segments, consul.NetworkSegment{ - Name: segment.Name, + Name: s.Name, Bind: serfConf.MemberlistConfig.BindAddr, - Port: segment.Port, Advertise: serfConf.MemberlistConfig.AdvertiseAddr, + Port: s.Bind.Port, RPCAddr: rpcAddr, SerfConfig: serfConf, }) @@ -852,7 +818,7 @@ func (a *Agent) makeRandomID() (string, error) { // gopsutil change implementations without affecting in-place upgrades of nodes. func (a *Agent) makeNodeID() (string, error) { // If they've disabled host-based IDs then just make a random one. - if *a.config.DisableHostNodeID { + if a.config.DisableHostNodeID { return a.makeRandomID() } @@ -889,7 +855,7 @@ func (a *Agent) makeNodeID() (string, error) { // setupNodeID will pull the persisted node ID, if any, or create a random one // and persist it. -func (a *Agent) setupNodeID(config *Config) error { +func (a *Agent) setupNodeID(config *config.RuntimeConfig) error { // If they've configured a node ID manually then just use that, as // long as it's valid. if config.NodeID != "" { @@ -961,7 +927,7 @@ func (a *Agent) setupBaseKeyrings(config *consul.Config) error { if err := loadKeyring(config.SerfLANConfig, keys); err != nil { return err } - if a.config.Server { + if a.config.ServerMode { if err := loadKeyring(config.SerfWANConfig, keys); err != nil { return err } @@ -981,7 +947,7 @@ func (a *Agent) setupBaseKeyrings(config *consul.Config) error { return err } } - if a.config.Server { + if a.config.ServerMode { if _, err := os.Stat(fileWAN); err != nil { if err := initKeyring(fileWAN, a.config.EncryptKey); err != nil { return err @@ -996,7 +962,7 @@ LOAD: if err := loadKeyringFile(config.SerfLANConfig); err != nil { return err } - if a.config.Server { + if a.config.ServerMode { if _, err := os.Stat(fileWAN); err == nil { config.SerfWANConfig.KeyringFile = fileWAN } @@ -1150,12 +1116,12 @@ func (a *Agent) ShutdownEndpoints() { a.dnsServers = nil for _, srv := range a.httpServers { - a.logger.Printf("[INFO] agent: Stopping %s server %s", strings.ToUpper(srv.proto), srv.Addr) + a.logger.Printf("[INFO] agent: Stopping %s server %s (%s)", strings.ToUpper(srv.proto), srv.addr.String(), srv.addr.Network()) ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() srv.Shutdown(ctx) if ctx.Err() == context.DeadlineExceeded { - a.logger.Printf("[WARN] agent: Timeout stopping %s server %s", strings.ToUpper(srv.proto), srv.Addr) + a.logger.Printf("[WARN] agent: Timeout stopping %s server %s (%s)", strings.ToUpper(srv.proto), srv.addr.String(), srv.addr.Network()) } } a.httpServers = nil @@ -2007,7 +1973,7 @@ func (a *Agent) deletePid() error { // loadServices will load service definitions from configuration and persisted // definitions on disk, and load them into the local agent. -func (a *Agent) loadServices(conf *Config) error { +func (a *Agent) loadServices(conf *config.RuntimeConfig) error { // Register the services from config for _, service := range conf.Services { ns := service.NodeService() @@ -2096,7 +2062,7 @@ func (a *Agent) unloadServices() error { // loadChecks loads check definitions and/or persisted check definitions from // disk and re-registers them with the local agent. -func (a *Agent) loadChecks(conf *Config) error { +func (a *Agent) loadChecks(conf *config.RuntimeConfig) error { // Register the checks from config for _, check := range conf.Checks { health := check.HealthCheck(conf.NodeName) @@ -2200,15 +2166,15 @@ func (a *Agent) restoreCheckState(snap map[types.CheckID]*structs.HealthCheck) { // loadMetadata loads node metadata fields from the agent config and // updates them on the local agent. -func (a *Agent) loadMetadata(conf *Config) error { +func (a *Agent) loadMetadata(conf *config.RuntimeConfig) error { a.state.Lock() defer a.state.Unlock() - for key, value := range conf.Meta { + for key, value := range conf.NodeMeta { a.state.metadata[key] = value } - a.state.metadata[structs.MetaSegmentKey] = conf.Segment + a.state.metadata[structs.MetaSegmentKey] = conf.SegmentName a.state.changeMade() @@ -2316,7 +2282,7 @@ func (a *Agent) DisableNodeMaintenance() { a.logger.Printf("[INFO] agent: Node left maintenance mode") } -func (a *Agent) ReloadConfig(newCfg *Config) error { +func (a *Agent) ReloadConfig(newCfg *config.RuntimeConfig) error { // Bulk update the services and checks a.PauseSync() defer a.ResumeSync() @@ -2351,7 +2317,7 @@ func (a *Agent) ReloadConfig(newCfg *Config) error { } // Update filtered metrics - metrics.UpdateFilter(newCfg.Telemetry.AllowedPrefixes, newCfg.Telemetry.BlockedPrefixes) + metrics.UpdateFilter(newCfg.TelemetryAllowedPrefixes, newCfg.TelemetryBlockedPrefixes) return nil } diff --git a/agent/agent_endpoint.go b/agent/agent_endpoint.go index 7c2276452..02528c7bf 100644 --- a/agent/agent_endpoint.go +++ b/agent/agent_endpoint.go @@ -8,6 +8,7 @@ import ( "strings" "github.com/hashicorp/consul/acl" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/ipaddr" @@ -20,7 +21,7 @@ import ( ) type Self struct { - Config *Config + Config config.RuntimeConfig Coord *coordinate.Coordinate Member serf.Member Stats map[string]map[string]string @@ -48,8 +49,8 @@ func (s *HTTPServer) AgentSelf(resp http.ResponseWriter, req *http.Request) (int } return Self{ - Config: s.agent.config, - Coord: cs[s.agent.config.Segment], + Config: s.agent.config.Sanitized(), + Coord: cs[s.agent.config.SegmentName], Member: s.agent.LocalMember(), Stats: s.agent.Stats(), Meta: s.agent.state.Metadata(), diff --git a/agent/agent_endpoint_test.go b/agent/agent_endpoint_test.go index ff77b5597..6504dcd10 100644 --- a/agent/agent_endpoint_test.go +++ b/agent/agent_endpoint_test.go @@ -14,12 +14,12 @@ import ( "time" "github.com/hashicorp/consul/acl" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/logger" "github.com/hashicorp/consul/testutil/retry" "github.com/hashicorp/consul/types" - "github.com/hashicorp/consul/watch" "github.com/hashicorp/serf/serf" ) @@ -41,7 +41,7 @@ func makeReadOnlyAgentACL(t *testing.T, srv *HTTPServer) string { func TestAgent_Services(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() srv1 := &structs.NodeService{ @@ -106,7 +106,7 @@ func TestAgent_Services_ACLFilter(t *testing.T) { func TestAgent_Checks(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() chk1 := &structs.HealthCheck{ @@ -171,9 +171,11 @@ func TestAgent_Checks_ACLFilter(t *testing.T) { func TestAgent_Self(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Meta = map[string]string{"somekey": "somevalue"} - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_meta { + somekey = "somevalue" + } + `) defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/agent/self", nil) @@ -183,11 +185,11 @@ func TestAgent_Self(t *testing.T) { } val := obj.(Self) - if int(val.Member.Port) != a.Config.Ports.SerfLan { + if int(val.Member.Port) != a.Config.SerfPortLAN { t.Fatalf("incorrect port: %v", obj) } - if int(val.Config.Ports.SerfLan) != a.Config.Ports.SerfLan { + if int(val.Config.SerfPortLAN) != a.Config.SerfPortLAN { t.Fatalf("incorrect port: %v", obj) } @@ -195,21 +197,12 @@ func TestAgent_Self(t *testing.T) { if err != nil { t.Fatalf("err: %v", err) } - if c := cs[cfg.Segment]; !reflect.DeepEqual(c, val.Coord) { + if c := cs[a.config.SegmentName]; !reflect.DeepEqual(c, val.Coord) { t.Fatalf("coordinates are not equal: %v != %v", c, val.Coord) } delete(val.Meta, structs.MetaSegmentKey) // Added later, not in config. - if !reflect.DeepEqual(cfg.Meta, val.Meta) { - t.Fatalf("meta fields are not equal: %v != %v", cfg.Meta, val.Meta) - } - - // Make sure there's nothing called "token" that's leaked. - raw, err := a.srv.marshalJSON(req, obj) - if err != nil { - t.Fatalf("err: %v", err) - } - if bytes.Contains(bytes.ToLower(raw), []byte("token")) { - t.Fatalf("bad: %s", raw) + if !reflect.DeepEqual(a.config.NodeMeta, val.Meta) { + t.Fatalf("meta fields are not equal: %v != %v", a.config.NodeMeta, val.Meta) } } @@ -271,36 +264,44 @@ func TestAgent_Metrics_ACLDeny(t *testing.T) { func TestAgent_Reload(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLEnforceVersion8 = Bool(false) - cfg.Services = []*structs.ServiceDefinition{ - &structs.ServiceDefinition{Name: "redis"}, - } - - params := map[string]interface{}{ - "datacenter": "dc1", - "type": "key", - "key": "test", - "handler": "true", - } - wp, err := watch.ParseExempt(params, []string{"handler"}) - if err != nil { - t.Fatalf("Expected watch.Parse to succeed %v", err) - } - cfg.WatchPlans = append(cfg.WatchPlans, wp) - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + acl_enforce_version_8 = false + services = [ + { + name = "redis" + } + ] + watches = [ + { + datacenter = "dc1" + type = "key" + key = "test" + handler = "true" + } + ] + `) defer a.Shutdown() if _, ok := a.state.services["redis"]; !ok { t.Fatalf("missing redis service") } - cfg2 := TestConfig() - cfg2.ACLEnforceVersion8 = Bool(false) - cfg2.Services = []*structs.ServiceDefinition{ - &structs.ServiceDefinition{Name: "redis-reloaded"}, - } + cfg2 := TestConfig(config.Source{ + Name: "reload", + Format: "hcl", + Data: ` + data_dir = "` + a.Config.DataDir + `" + node_id = "` + string(a.Config.NodeID) + `" + node_name = "` + a.Config.NodeName + `" + + acl_enforce_version_8 = false + services = [ + { + name = "redis-reloaded" + } + ] + `, + }) if err := a.ReloadConfig(cfg2); err != nil { t.Fatalf("got error %v want nil", err) @@ -309,7 +310,7 @@ func TestAgent_Reload(t *testing.T) { t.Fatalf("missing redis-reloaded service") } - for _, wp := range cfg.WatchPlans { + for _, wp := range a.watchPlans { if !wp.IsStopped() { t.Fatalf("Reloading configs should stop watch plans of the previous configuration") } @@ -344,7 +345,7 @@ func TestAgent_Reload_ACLDeny(t *testing.T) { func TestAgent_Members(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/agent/members", nil) @@ -357,14 +358,14 @@ func TestAgent_Members(t *testing.T) { t.Fatalf("bad members: %v", obj) } - if int(val[0].Port) != a.Config.Ports.SerfLan { + if int(val[0].Port) != a.Config.SerfPortLAN { t.Fatalf("not lan: %v", obj) } } func TestAgent_Members_WAN(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/agent/members?wan=true", nil) @@ -377,7 +378,7 @@ func TestAgent_Members_WAN(t *testing.T) { t.Fatalf("bad members: %v", obj) } - if int(val[0].Port) != a.Config.Ports.SerfWan { + if int(val[0].Port) != a.Config.SerfPortWAN { t.Fatalf("not wan: %v", obj) } } @@ -414,12 +415,12 @@ func TestAgent_Members_ACLFilter(t *testing.T) { func TestAgent_Join(t *testing.T) { t.Parallel() - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - a2 := NewTestAgent(t.Name(), nil) + a2 := NewTestAgent(t.Name(), "") defer a2.Shutdown() - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan) + addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.SerfPortLAN) req, _ := http.NewRequest("GET", fmt.Sprintf("/v1/agent/join/%s", addr), nil) obj, err := a1.srv.AgentJoin(nil, req) if err != nil { @@ -442,12 +443,12 @@ func TestAgent_Join(t *testing.T) { func TestAgent_Join_WAN(t *testing.T) { t.Parallel() - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - a2 := NewTestAgent(t.Name(), nil) + a2 := NewTestAgent(t.Name(), "") defer a2.Shutdown() - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.SerfPortWAN) req, _ := http.NewRequest("GET", fmt.Sprintf("/v1/agent/join/%s?wan=true", addr), nil) obj, err := a1.srv.AgentJoin(nil, req) if err != nil { @@ -472,10 +473,10 @@ func TestAgent_Join_ACLDeny(t *testing.T) { t.Parallel() a1 := NewTestAgent(t.Name(), TestACLConfig()) defer a1.Shutdown() - a2 := NewTestAgent(t.Name(), nil) + a2 := NewTestAgent(t.Name(), "") defer a2.Shutdown() - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan) + addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.SerfPortLAN) t.Run("no token", func(t *testing.T) { req, _ := http.NewRequest("GET", fmt.Sprintf("/v1/agent/join/%s", addr), nil) @@ -510,19 +511,19 @@ func (n *mockNotifier) Notify(state string) error { func TestAgent_JoinLANNotify(t *testing.T) { t.Parallel() - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Server = false - cfg2.Bootstrap = false - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + server = false + bootstrap = false + `) defer a2.Shutdown() notif := &mockNotifier{} a1.joinLANNotifier = notif - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan) + addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.SerfPortLAN) _, err := a1.JoinLAN([]string{addr}) if err != nil { t.Fatalf("err: %v", err) @@ -535,17 +536,17 @@ func TestAgent_JoinLANNotify(t *testing.T) { func TestAgent_Leave(t *testing.T) { t.Parallel() - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Server = false - cfg2.Bootstrap = false - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + server = false + bootstrap = false + `) defer a2.Shutdown() // Join first - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan) + addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.SerfPortLAN) _, err := a1.JoinLAN([]string{addr}) if err != nil { t.Fatalf("err: %v", err) @@ -600,18 +601,18 @@ func TestAgent_Leave_ACLDeny(t *testing.T) { func TestAgent_ForceLeave(t *testing.T) { t.Parallel() - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - a2 := NewTestAgent(t.Name(), nil) + a2 := NewTestAgent(t.Name(), "") // Join first - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan) + addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.SerfPortLAN) _, err := a1.JoinLAN([]string{addr}) if err != nil { t.Fatalf("err: %v", err) } - // todo(fs): this test probably needs work + // this test probably needs work a2.Shutdown() // Force leave now @@ -662,7 +663,7 @@ func TestAgent_ForceLeave_ACLDeny(t *testing.T) { func TestAgent_RegisterCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -703,7 +704,7 @@ func TestAgent_RegisterCheck(t *testing.T) { func TestAgent_RegisterCheck_Passing(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -739,7 +740,7 @@ func TestAgent_RegisterCheck_Passing(t *testing.T) { func TestAgent_RegisterCheck_BadStatus(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -785,7 +786,7 @@ func TestAgent_RegisterCheck_ACLDeny(t *testing.T) { func TestAgent_DeregisterCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() chk := &structs.HealthCheck{Name: "test", CheckID: "test"} @@ -836,7 +837,7 @@ func TestAgent_DeregisterCheckACLDeny(t *testing.T) { func TestAgent_PassCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() chk := &structs.HealthCheck{Name: "test", CheckID: "test"} @@ -889,7 +890,7 @@ func TestAgent_PassCheck_ACLDeny(t *testing.T) { func TestAgent_WarnCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() chk := &structs.HealthCheck{Name: "test", CheckID: "test"} @@ -942,7 +943,7 @@ func TestAgent_WarnCheck_ACLDeny(t *testing.T) { func TestAgent_FailCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() chk := &structs.HealthCheck{Name: "test", CheckID: "test"} @@ -995,7 +996,7 @@ func TestAgent_FailCheck_ACLDeny(t *testing.T) { func TestAgent_UpdateCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() chk := &structs.HealthCheck{Name: "test", CheckID: "test"} @@ -1122,7 +1123,7 @@ func TestAgent_UpdateCheck_ACLDeny(t *testing.T) { func TestAgent_RegisterService(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() args := &structs.ServiceDefinition{ @@ -1211,7 +1212,7 @@ func TestAgent_RegisterService_ACLDeny(t *testing.T) { func TestAgent_RegisterService_InvalidAddress(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() for _, addr := range []string{"0.0.0.0", "::", "[::]"} { @@ -1239,7 +1240,7 @@ func TestAgent_RegisterService_InvalidAddress(t *testing.T) { func TestAgent_DeregisterService(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() service := &structs.NodeService{ @@ -1299,7 +1300,7 @@ func TestAgent_DeregisterService_ACLDeny(t *testing.T) { func TestAgent_ServiceMaintenance_BadRequest(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() t.Run("not PUT", func(t *testing.T) { @@ -1349,7 +1350,7 @@ func TestAgent_ServiceMaintenance_BadRequest(t *testing.T) { func TestAgent_ServiceMaintenance_Enable(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register the service @@ -1391,7 +1392,7 @@ func TestAgent_ServiceMaintenance_Enable(t *testing.T) { func TestAgent_ServiceMaintenance_Disable(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register the service @@ -1456,7 +1457,7 @@ func TestAgent_ServiceMaintenance_ACLDeny(t *testing.T) { func TestAgent_NodeMaintenance_BadRequest(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Fails on non-PUT @@ -1482,7 +1483,7 @@ func TestAgent_NodeMaintenance_BadRequest(t *testing.T) { func TestAgent_NodeMaintenance_Enable(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Force the node into maintenance mode @@ -1514,7 +1515,7 @@ func TestAgent_NodeMaintenance_Enable(t *testing.T) { func TestAgent_NodeMaintenance_Disable(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Force the node into maintenance mode @@ -1558,7 +1559,7 @@ func TestAgent_NodeMaintenance_ACLDeny(t *testing.T) { func TestAgent_RegisterCheck_Service(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() args := &structs.ServiceDefinition{ @@ -1687,11 +1688,11 @@ func TestAgent_Monitor_ACLDeny(t *testing.T) { func TestAgent_Token(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLToken = "" - cfg.ACLAgentToken = "" - cfg.ACLAgentMasterToken = "" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_token = "" + acl_agent_token = "" + acl_agent_master_token = "" + `) defer a.Shutdown() type tokens struct { diff --git a/agent/agent_test.go b/agent/agent_test.go index 348b3b48c..114b468c1 100644 --- a/agent/agent_test.go +++ b/agent/agent_test.go @@ -9,7 +9,6 @@ import ( "os" "path/filepath" "reflect" - "runtime" "strings" "testing" "time" @@ -19,7 +18,7 @@ import ( "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/testutil" "github.com/hashicorp/consul/types" - "github.com/hashicorp/go-uuid" + uuid "github.com/hashicorp/go-uuid" "github.com/hashicorp/raft" "github.com/pascaldekloe/goe/verify" ) @@ -40,10 +39,10 @@ func externalIP() (string, error) { } func TestAgent_MultiStartStop(t *testing.T) { - for i := 0; i < 100; i++ { + for i := 0; i < 10; i++ { t.Run("", func(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") time.Sleep(250 * time.Millisecond) a.Shutdown() }) @@ -52,7 +51,7 @@ func TestAgent_MultiStartStop(t *testing.T) { func TestAgent_StartStop(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") // defer a.Shutdown() if err := a.Leave(); err != nil { @@ -71,7 +70,7 @@ func TestAgent_StartStop(t *testing.T) { func TestAgent_RPCPing(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() var out struct{} @@ -80,79 +79,14 @@ func TestAgent_RPCPing(t *testing.T) { } } -func TestAgent_CheckSerfBindAddrsSettings(t *testing.T) { - t.Parallel() - if runtime.GOOS == "darwin" { - t.Skip("skip test on macOS to avoid firewall warning dialog") - } - - cfg := TestConfig() - ip, err := externalIP() - if err != nil { - t.Fatalf("Unable to get a non-loopback IP: %v", err) - } - cfg.SerfLanBindAddr = ip - cfg.SerfWanBindAddr = ip - a := NewTestAgent(t.Name(), cfg) - defer a.Shutdown() - - serfWanBind := a.consulConfig().SerfWANConfig.MemberlistConfig.BindAddr - if serfWanBind != ip { - t.Fatalf("SerfWanBindAddr is should be a non-loopback IP not %s", serfWanBind) - } - - serfLanBind := a.consulConfig().SerfLANConfig.MemberlistConfig.BindAddr - if serfLanBind != ip { - t.Fatalf("SerfLanBindAddr is should be a non-loopback IP not %s", serfWanBind) - } -} -func TestAgent_CheckAdvertiseAddrsSettings(t *testing.T) { - t.Parallel() - cfg := TestConfig() - cfg.AdvertiseAddrs.SerfLan, _ = net.ResolveTCPAddr("tcp", "127.0.0.42:1233") - cfg.AdvertiseAddrs.SerfWan, _ = net.ResolveTCPAddr("tcp", "127.0.0.43:1234") - cfg.AdvertiseAddrs.RPC, _ = net.ResolveTCPAddr("tcp", "127.0.0.44:1235") - cfg.SetupTaggedAndAdvertiseAddrs() - a := NewTestAgent(t.Name(), cfg) - defer a.Shutdown() - - serfLanAddr := a.consulConfig().SerfLANConfig.MemberlistConfig.AdvertiseAddr - if serfLanAddr != "127.0.0.42" { - t.Fatalf("SerfLan is not properly set to '127.0.0.42': %s", serfLanAddr) - } - serfLanPort := a.consulConfig().SerfLANConfig.MemberlistConfig.AdvertisePort - if serfLanPort != 1233 { - t.Fatalf("SerfLan is not properly set to '1233': %d", serfLanPort) - } - serfWanAddr := a.consulConfig().SerfWANConfig.MemberlistConfig.AdvertiseAddr - if serfWanAddr != "127.0.0.43" { - t.Fatalf("SerfWan is not properly set to '127.0.0.43': %s", serfWanAddr) - } - serfWanPort := a.consulConfig().SerfWANConfig.MemberlistConfig.AdvertisePort - if serfWanPort != 1234 { - t.Fatalf("SerfWan is not properly set to '1234': %d", serfWanPort) - } - rpc := a.consulConfig().RPCAdvertise - if rpc != cfg.AdvertiseAddrs.RPC { - t.Fatalf("RPC is not properly set to %v: %s", cfg.AdvertiseAddrs.RPC, rpc) - } - expected := map[string]string{ - "lan": a.Config.AdvertiseAddr, - "wan": a.Config.AdvertiseAddrWan, - } - if !reflect.DeepEqual(a.Config.TaggedAddresses, expected) { - t.Fatalf("Tagged addresses not set up properly: %v", a.Config.TaggedAddresses) - } -} - func TestAgent_TokenStore(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLToken = "user" - cfg.ACLAgentToken = "agent" - cfg.ACLAgentMasterToken = "master" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + acl_token = "user" + acl_agent_token = "agent" + acl_agent_master_token = "master"`, + ) defer a.Shutdown() if got, want := a.tokens.UserToken(), "user"; got != want { @@ -166,49 +100,10 @@ func TestAgent_TokenStore(t *testing.T) { } } -func TestAgent_CheckPerformanceSettings(t *testing.T) { - t.Parallel() - // Try a default config. - { - cfg := TestConfig() - cfg.Bootstrap = false - cfg.ConsulConfig = nil - a := NewTestAgent(t.Name(), cfg) - defer a.Shutdown() - - raftMult := time.Duration(consul.DefaultRaftMultiplier) - r := a.consulConfig().RaftConfig - def := raft.DefaultConfig() - if r.HeartbeatTimeout != raftMult*def.HeartbeatTimeout || - r.ElectionTimeout != raftMult*def.ElectionTimeout || - r.LeaderLeaseTimeout != raftMult*def.LeaderLeaseTimeout { - t.Fatalf("bad: %#v", *r) - } - } - - // Try a multiplier. - { - cfg := TestConfig() - cfg.Bootstrap = false - cfg.Performance.RaftMultiplier = 99 - a := NewTestAgent(t.Name(), cfg) - defer a.Shutdown() - - const raftMult time.Duration = 99 - r := a.consulConfig().RaftConfig - def := raft.DefaultConfig() - if r.HeartbeatTimeout != raftMult*def.HeartbeatTimeout || - r.ElectionTimeout != raftMult*def.ElectionTimeout || - r.LeaderLeaseTimeout != raftMult*def.LeaderLeaseTimeout { - t.Fatalf("bad: %#v", *r) - } - } -} - func TestAgent_ReconnectConfigSettings(t *testing.T) { t.Parallel() func() { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() lan := a.consulConfig().SerfLANConfig.ReconnectTimeout @@ -223,10 +118,10 @@ func TestAgent_ReconnectConfigSettings(t *testing.T) { }() func() { - cfg := TestConfig() - cfg.ReconnectTimeoutLan = 24 * time.Hour - cfg.ReconnectTimeoutWan = 36 * time.Hour - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + reconnect_timeout = "24h" + reconnect_timeout_wan = "36h" + `) defer a.Shutdown() lan := a.consulConfig().SerfLANConfig.ReconnectTimeout @@ -243,11 +138,13 @@ func TestAgent_ReconnectConfigSettings(t *testing.T) { func TestAgent_setupNodeID(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeID = "" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_id = "" + `) defer a.Shutdown() + cfg := a.config + // The auto-assigned ID should be valid. id := a.consulConfig().NodeID if _, err := uuid.ParseUUID(string(id)); err != nil { @@ -309,9 +206,9 @@ func TestAgent_setupNodeID(t *testing.T) { func TestAgent_makeNodeID(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeID = "" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_id = "" + `) defer a.Shutdown() // We should get a valid host-based ID initially. @@ -334,7 +231,7 @@ func TestAgent_makeNodeID(t *testing.T) { // Turn on host-based IDs and try again. We should get the same ID // each time (and a different one from the random one above). - a.Config.DisableHostNodeID = Bool(false) + a.Config.DisableHostNodeID = false id, err = a.makeNodeID() if err != nil { t.Fatalf("err: %v", err) @@ -355,9 +252,9 @@ func TestAgent_makeNodeID(t *testing.T) { func TestAgent_AddService(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeName = "node1" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_name = "node1" + `) defer a.Shutdown() tests := []struct { @@ -501,7 +398,7 @@ func TestAgent_AddService(t *testing.T) { func TestAgent_RemoveService(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Remove a service that doesn't exist @@ -595,9 +492,9 @@ func TestAgent_RemoveService(t *testing.T) { func TestAgent_RemoveServiceRemovesAllChecks(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeName = "node1" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_name = "node1" + `) defer a.Shutdown() svc := &structs.NodeService{ID: "redis", Service: "redis", Port: 8000} @@ -645,9 +542,9 @@ func TestAgent_RemoveServiceRemovesAllChecks(t *testing.T) { func TestAgent_AddCheck(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + enable_script_checks = true + `) defer a.Shutdown() health := &structs.HealthCheck{ @@ -684,9 +581,9 @@ func TestAgent_AddCheck(t *testing.T) { func TestAgent_AddCheck_StartPassing(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + enable_script_checks = true + `) defer a.Shutdown() health := &structs.HealthCheck{ @@ -723,9 +620,9 @@ func TestAgent_AddCheck_StartPassing(t *testing.T) { func TestAgent_AddCheck_MinInterval(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + enable_script_checks = true + `) defer a.Shutdown() health := &structs.HealthCheck{ @@ -758,9 +655,9 @@ func TestAgent_AddCheck_MinInterval(t *testing.T) { func TestAgent_AddCheck_MissingService(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + enable_script_checks = true + `) defer a.Shutdown() health := &structs.HealthCheck{ @@ -781,7 +678,7 @@ func TestAgent_AddCheck_MissingService(t *testing.T) { func TestAgent_AddCheck_RestoreState(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create some state and persist it @@ -825,7 +722,7 @@ func TestAgent_AddCheck_RestoreState(t *testing.T) { func TestAgent_AddCheck_ExecDisable(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() health := &structs.HealthCheck{ @@ -851,9 +748,9 @@ func TestAgent_AddCheck_ExecDisable(t *testing.T) { func TestAgent_RemoveCheck(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + enable_script_checks = true + `) defer a.Shutdown() // Remove check that doesn't exist @@ -899,7 +796,7 @@ func TestAgent_RemoveCheck(t *testing.T) { func TestAgent_updateTTLCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() health := &structs.HealthCheck{ @@ -933,11 +830,15 @@ func TestAgent_updateTTLCheck(t *testing.T) { func TestAgent_PersistService(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Server = false - cfg.DataDir = testutil.TempDir(t, "agent") // we manage the data dir - a := NewTestAgent(t.Name(), cfg) - defer os.RemoveAll(cfg.DataDir) + dataDir := testutil.TempDir(t, "agent") // we manage the data dir + cfg := ` + server = false + bootstrap = false + data_dir = "` + dataDir + `" + ` + a := &TestAgent{Name: t.Name(), HCL: cfg, DataDir: dataDir} + a.Start() + defer os.RemoveAll(dataDir) defer a.Shutdown() svc := &structs.NodeService{ @@ -1001,7 +902,8 @@ func TestAgent_PersistService(t *testing.T) { a.Shutdown() // Should load it back during later start - a2 := NewTestAgent(t.Name()+"-a2", cfg) + a2 := &TestAgent{Name: t.Name(), HCL: cfg, DataDir: dataDir} + a2.Start() defer a2.Shutdown() restored, ok := a2.state.services[svc.ID] @@ -1019,7 +921,7 @@ func TestAgent_PersistService(t *testing.T) { func TestAgent_persistedService_compat(t *testing.T) { t.Parallel() // Tests backwards compatibility of persisted services from pre-0.5.1 - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() svc := &structs.NodeService{ @@ -1063,7 +965,7 @@ func TestAgent_persistedService_compat(t *testing.T) { func TestAgent_PurgeService(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() svc := &structs.NodeService{ @@ -1102,10 +1004,16 @@ func TestAgent_PurgeService(t *testing.T) { func TestAgent_PurgeServiceOnDuplicate(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Server = false - a := NewTestAgent(t.Name(), cfg) + dataDir := testutil.TempDir(t, "agent") // we manage the data dir + cfg := ` + data_dir = "` + dataDir + `" + server = false + bootstrap = false + ` + a := &TestAgent{Name: t.Name(), HCL: cfg, DataDir: dataDir} + a.Start() defer a.Shutdown() + defer os.RemoveAll(dataDir) svc1 := &structs.NodeService{ ID: "redis", @@ -1122,42 +1030,46 @@ func TestAgent_PurgeServiceOnDuplicate(t *testing.T) { // Try bringing the agent back up with the service already // existing in the config - svc2 := &structs.ServiceDefinition{ - ID: "redis", - Name: "redis", - Tags: []string{"bar"}, - Port: 9000, - } - - cfg.Services = []*structs.ServiceDefinition{svc2} - a2 := NewTestAgent(t.Name()+"-a2", cfg) + a2 := &TestAgent{Name: t.Name() + "-a2", HCL: cfg + ` + service = { + id = "redis" + name = "redis" + tags = ["bar"] + port = 9000 + } + `, DataDir: dataDir} + a2.Start() defer a2.Shutdown() file := filepath.Join(a.Config.DataDir, servicesDir, stringHash(svc1.ID)) if _, err := os.Stat(file); err == nil { t.Fatalf("should have removed persisted service") } - result, ok := a2.state.services[svc2.ID] + result, ok := a2.state.services["redis"] if !ok { t.Fatalf("missing service registration") } - if !reflect.DeepEqual(result.Tags, svc2.Tags) || result.Port != svc2.Port { + if !reflect.DeepEqual(result.Tags, []string{"bar"}) || result.Port != 9000 { t.Fatalf("bad: %#v", result) } } func TestAgent_PersistCheck(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Server = false - cfg.DataDir = testutil.TempDir(t, "agent") // we manage the data dir - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) - defer os.RemoveAll(cfg.DataDir) + dataDir := testutil.TempDir(t, "agent") // we manage the data dir + cfg := ` + data_dir = "` + dataDir + `" + server = false + bootstrap = false + enable_script_checks = true + ` + a := &TestAgent{Name: t.Name(), HCL: cfg, DataDir: dataDir} + a.Start() + defer os.RemoveAll(dataDir) defer a.Shutdown() check := &structs.HealthCheck{ - Node: cfg.NodeName, + Node: a.config.NodeName, CheckID: "mem", Name: "memory check", Status: api.HealthPassing, @@ -1223,7 +1135,8 @@ func TestAgent_PersistCheck(t *testing.T) { a.Shutdown() // Should load it back during later start - a2 := NewTestAgent(t.Name()+"-a2", cfg) + a2 := &TestAgent{Name: t.Name() + "-a2", HCL: cfg, DataDir: dataDir} + a2.Start() defer a2.Shutdown() result, ok := a2.state.checks[check.CheckID] @@ -1248,7 +1161,7 @@ func TestAgent_PersistCheck(t *testing.T) { func TestAgent_PurgeCheck(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() check := &structs.HealthCheck{ @@ -1282,16 +1195,21 @@ func TestAgent_PurgeCheck(t *testing.T) { func TestAgent_PurgeCheckOnDuplicate(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Server = false - cfg.DataDir = testutil.TempDir(t, "agent") // we manage the data dir - cfg.EnableScriptChecks = true - a := NewTestAgent(t.Name(), cfg) - defer os.RemoveAll(cfg.DataDir) + nodeID := NodeID() + dataDir := testutil.TempDir(t, "agent") + a := NewTestAgent(t.Name(), ` + node_id = "`+nodeID+`" + node_name = "Node `+nodeID+`" + data_dir = "`+dataDir+`" + server = false + bootstrap = false + enable_script_checks = true + `) + defer os.RemoveAll(dataDir) defer a.Shutdown() check1 := &structs.HealthCheck{ - Node: cfg.NodeName, + Node: a.Config.NodeName, CheckID: "mem", Name: "memory check", Status: api.HealthPassing, @@ -1304,42 +1222,53 @@ func TestAgent_PurgeCheckOnDuplicate(t *testing.T) { a.Shutdown() // Start again with the check registered in config - check2 := &structs.CheckDefinition{ - ID: "mem", - Name: "memory check", - Notes: "my cool notes", - Script: "/bin/check-redis.py", - Interval: 30 * time.Second, - } - - cfg.Checks = []*structs.CheckDefinition{check2} - a2 := NewTestAgent(t.Name()+"-a2", cfg) + a2 := NewTestAgent(t.Name()+"-a2", ` + node_id = "`+nodeID+`" + node_name = "Node `+nodeID+`" + data_dir = "`+dataDir+`" + server = false + bootstrap = false + enable_script_checks = true + check = { + id = "mem" + name = "memory check" + notes = "my cool notes" + script = "/bin/check-redis.py" + interval = "30s" + } + `) defer a2.Shutdown() - file := filepath.Join(a.Config.DataDir, checksDir, checkIDHash(check1.CheckID)) + file := filepath.Join(dataDir, checksDir, checkIDHash(check1.CheckID)) if _, err := os.Stat(file); err == nil { t.Fatalf("should have removed persisted check") } - result, ok := a2.state.checks[check2.ID] + result, ok := a2.state.checks["mem"] if !ok { t.Fatalf("missing check registration") } - expected := check2.HealthCheck(cfg.NodeName) - if !reflect.DeepEqual(expected, result) { - t.Fatalf("bad: %#v", result) + expected := &structs.HealthCheck{ + Node: a2.Config.NodeName, + CheckID: "mem", + Name: "memory check", + Status: api.HealthCritical, + Notes: "my cool notes", + } + if got, want := result, expected; !verify.Values(t, "", got, want) { + t.FailNow() } } func TestAgent_loadChecks_token(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Checks = append(cfg.Checks, &structs.CheckDefinition{ - ID: "rabbitmq", - Name: "rabbitmq", - Token: "abc123", - TTL: 10 * time.Second, - }) - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + check = { + id = "rabbitmq" + name = "rabbitmq" + token = "abc123" + ttl = "10s" + } + `) defer a.Shutdown() checks := a.state.Checks() @@ -1353,7 +1282,7 @@ func TestAgent_loadChecks_token(t *testing.T) { func TestAgent_unloadChecks(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // First register a service @@ -1405,14 +1334,14 @@ func TestAgent_unloadChecks(t *testing.T) { func TestAgent_loadServices_token(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Services = append(cfg.Services, &structs.ServiceDefinition{ - ID: "rabbitmq", - Name: "rabbitmq", - Port: 5672, - Token: "abc123", - }) - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + service = { + id = "rabbitmq" + name = "rabbitmq" + port = 5672 + token = "abc123" + } + `) defer a.Shutdown() services := a.state.Services() @@ -1426,7 +1355,7 @@ func TestAgent_loadServices_token(t *testing.T) { func TestAgent_unloadServices(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() svc := &structs.NodeService{ @@ -1462,7 +1391,7 @@ func TestAgent_unloadServices(t *testing.T) { func TestAgent_Service_MaintenanceMode(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() svc := &structs.NodeService{ @@ -1526,10 +1455,10 @@ func TestAgent_Service_MaintenanceMode(t *testing.T) { func TestAgent_Service_Reap(t *testing.T) { // t.Parallel() // timing test. no parallel - cfg := TestConfig() - cfg.CheckReapInterval = 50 * time.Millisecond - cfg.CheckDeregisterIntervalMin = 0 - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + check_reap_interval = "50ms" + check_deregister_interval_min = "0s" + `) defer a.Shutdown() svc := &structs.NodeService{ @@ -1600,10 +1529,10 @@ func TestAgent_Service_Reap(t *testing.T) { func TestAgent_Service_NoReap(t *testing.T) { // t.Parallel() // timing test. no parallel - cfg := TestConfig() - cfg.CheckReapInterval = 50 * time.Millisecond - cfg.CheckDeregisterIntervalMin = 0 - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + check_reap_interval = "50ms" + check_deregister_interval_min = "0s" + `) defer a.Shutdown() svc := &structs.NodeService{ @@ -1653,7 +1582,7 @@ func TestAgent_Service_NoReap(t *testing.T) { func TestAgent_addCheck_restoresSnapshot(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // First register a service @@ -1696,7 +1625,7 @@ func TestAgent_addCheck_restoresSnapshot(t *testing.T) { func TestAgent_NodeMaintenanceMode(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Enter maintenance mode for the node @@ -1741,7 +1670,7 @@ func TestAgent_NodeMaintenanceMode(t *testing.T) { func TestAgent_checkStateSnapshot(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // First register a service @@ -1798,7 +1727,7 @@ func TestAgent_checkStateSnapshot(t *testing.T) { func TestAgent_loadChecks_checkFails(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Persist a health check with an invalid service ID @@ -1833,7 +1762,7 @@ func TestAgent_loadChecks_checkFails(t *testing.T) { func TestAgent_persistCheckState(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create the TTL check to persist @@ -1880,7 +1809,7 @@ func TestAgent_persistCheckState(t *testing.T) { func TestAgent_loadCheckState(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create a check whose state will expire immediately @@ -1941,7 +1870,7 @@ func TestAgent_loadCheckState(t *testing.T) { func TestAgent_purgeCheckState(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // No error if the state does not exist @@ -1974,9 +1903,9 @@ func TestAgent_purgeCheckState(t *testing.T) { func TestAgent_GetCoordinate(t *testing.T) { t.Parallel() check := func(server bool) { - cfg := TestConfig() - cfg.Server = server - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + server = true + `) defer a.Shutdown() // This doesn't verify the returned coordinate, but it makes diff --git a/agent/catalog_endpoint_test.go b/agent/catalog_endpoint_test.go index b4ebd2144..3ac20afa0 100644 --- a/agent/catalog_endpoint_test.go +++ b/agent/catalog_endpoint_test.go @@ -14,7 +14,7 @@ import ( func TestCatalogRegister(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -33,7 +33,7 @@ func TestCatalogRegister(t *testing.T) { t.Fatalf("bad: %v", res) } - // todo(fs): data race + // data race func() { a.state.Lock() defer a.state.Unlock() @@ -53,7 +53,7 @@ func TestCatalogRegister(t *testing.T) { func TestCatalogRegister_Service_InvalidAddress(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() for _, addr := range []string{"0.0.0.0", "::", "[::]"} { @@ -78,7 +78,7 @@ func TestCatalogRegister_Service_InvalidAddress(t *testing.T) { func TestCatalogDeregister(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -97,7 +97,7 @@ func TestCatalogDeregister(t *testing.T) { func TestCatalogDatacenters(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() retry.Run(t, func(r *retry.R) { @@ -115,7 +115,7 @@ func TestCatalogDatacenters(t *testing.T) { func TestCatalogNodes(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -148,7 +148,7 @@ func TestCatalogNodes(t *testing.T) { func TestCatalogNodes_MetaFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a meta field @@ -188,22 +188,22 @@ func TestCatalogNodes_MetaFilter(t *testing.T) { func TestCatalogNodes_WanTranslation(t *testing.T) { t.Parallel() - cfg1 := TestConfig() - cfg1.Datacenter = "dc1" - cfg1.TranslateWanAddrs = true - cfg1.ACLDatacenter = "" - a1 := NewTestAgent(t.Name(), cfg1) + a1 := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Datacenter = "dc2" - cfg2.TranslateWanAddrs = true - cfg2.ACLDatacenter = "" - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + datacenter = "dc2" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a2.Shutdown() // Wait for the WAN join. - addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.SerfPortWAN) if _, err := a2.JoinWAN([]string{addr}); err != nil { t.Fatalf("err: %v", err) } @@ -282,7 +282,7 @@ func TestCatalogNodes_WanTranslation(t *testing.T) { func TestCatalogNodes_Blocking(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -350,7 +350,7 @@ func TestCatalogNodes_Blocking(t *testing.T) { func TestCatalogNodes_DistanceSort(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register nodes. @@ -434,7 +434,7 @@ func TestCatalogNodes_DistanceSort(t *testing.T) { func TestCatalogServices(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -469,7 +469,7 @@ func TestCatalogServices(t *testing.T) { func TestCatalogServices_NodeMetaFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -510,7 +510,7 @@ func TestCatalogServices_NodeMetaFilter(t *testing.T) { func TestCatalogServiceNodes(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Make sure an empty list is returned, not a nil @@ -563,7 +563,7 @@ func TestCatalogServiceNodes(t *testing.T) { func TestCatalogServiceNodes_NodeMetaFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Make sure an empty list is returned, not a nil @@ -618,22 +618,22 @@ func TestCatalogServiceNodes_NodeMetaFilter(t *testing.T) { func TestCatalogServiceNodes_WanTranslation(t *testing.T) { t.Parallel() - cfg1 := TestConfig() - cfg1.Datacenter = "dc1" - cfg1.TranslateWanAddrs = true - cfg1.ACLDatacenter = "" - a1 := NewTestAgent(t.Name(), cfg1) + a1 := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Datacenter = "dc2" - cfg2.TranslateWanAddrs = true - cfg2.ACLDatacenter = "" - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + datacenter = "dc2" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a2.Shutdown() // Wait for the WAN join. - addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.SerfPortWAN) if _, err := a2.srv.agent.JoinWAN([]string{addr}); err != nil { t.Fatalf("err: %v", err) } @@ -703,7 +703,7 @@ func TestCatalogServiceNodes_WanTranslation(t *testing.T) { func TestCatalogServiceNodes_DistanceSort(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register nodes. @@ -790,7 +790,7 @@ func TestCatalogServiceNodes_DistanceSort(t *testing.T) { func TestCatalogNodeServices(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -825,22 +825,22 @@ func TestCatalogNodeServices(t *testing.T) { func TestCatalogNodeServices_WanTranslation(t *testing.T) { t.Parallel() - cfg1 := TestConfig() - cfg1.Datacenter = "dc1" - cfg1.TranslateWanAddrs = true - cfg1.ACLDatacenter = "" - a1 := NewTestAgent(t.Name(), cfg1) + a1 := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Datacenter = "dc2" - cfg2.TranslateWanAddrs = true - cfg2.ACLDatacenter = "" - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + datacenter = "dc2" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a2.Shutdown() // Wait for the WAN join. - addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.SerfPortWAN) if _, err := a2.srv.agent.JoinWAN([]string{addr}); err != nil { t.Fatalf("err: %v", err) } diff --git a/agent/config.go b/agent/config.go index e7c6716e3..3ba1437fe 100644 --- a/agent/config.go +++ b/agent/config.go @@ -1,1549 +1,12 @@ package agent import ( - "crypto/tls" - "encoding/base64" - "encoding/json" "errors" "fmt" - "io" - "net" - "os" - "path/filepath" - "sort" "strings" "time" - - "github.com/hashicorp/consul/agent/consul" - "github.com/hashicorp/consul/agent/structs" - "github.com/hashicorp/consul/ipaddr" - "github.com/hashicorp/consul/lib" - "github.com/hashicorp/consul/tlsutil" - "github.com/hashicorp/consul/types" - "github.com/hashicorp/consul/watch" - "github.com/hashicorp/go-sockaddr/template" - "github.com/mitchellh/mapstructure" - "golang.org/x/time/rate" ) -const ( - // SegmentLimit is the maximum number of network segments that may be declared. - SegmentLimit = 64 - - // SegmentNameLimit is the maximum segment name length. - SegmentNameLimit = 64 -) - -// Ports is used to simplify the configuration by -// providing default ports, and allowing the addresses -// to only be specified once -type PortConfig struct { - DNS int // DNS Query interface - HTTP int // HTTP API - HTTPS int // HTTPS API - SerfLan int `mapstructure:"serf_lan"` // LAN gossip (Client + Server) - SerfWan int `mapstructure:"serf_wan"` // WAN gossip (Server only) - Server int // Server internal RPC - - // RPC is deprecated and is no longer used. It will be removed in a future - // version. - RPC int // CLI RPC -} - -// AddressConfig is used to provide address overrides -// for specific services. By default, either ClientAddress -// or ServerAddress is used. -type AddressConfig struct { - DNS string // DNS Query interface - HTTP string // HTTP API - HTTPS string // HTTPS API - - // RPC is deprecated and is no longer used. It will be removed in a future - // version. - RPC string // CLI RPC -} - -type AdvertiseAddrsConfig struct { - SerfLan *net.TCPAddr `mapstructure:"-"` - SerfLanRaw string `mapstructure:"serf_lan"` - SerfWan *net.TCPAddr `mapstructure:"-"` - SerfWanRaw string `mapstructure:"serf_wan"` - RPC *net.TCPAddr `mapstructure:"-"` - RPCRaw string `mapstructure:"rpc"` -} - -// DNSConfig is used to fine tune the DNS sub-system. -// It can be used to control cache values, and stale -// reads -type DNSConfig struct { - // NodeTTL provides the TTL value for a node query - NodeTTL time.Duration `mapstructure:"-"` - NodeTTLRaw string `mapstructure:"node_ttl" json:"-"` - - // ServiceTTL provides the TTL value for a service - // query for given service. The "*" wildcard can be used - // to set a default for all services. - ServiceTTL map[string]time.Duration `mapstructure:"-"` - ServiceTTLRaw map[string]string `mapstructure:"service_ttl" json:"-"` - - // AllowStale is used to enable lookups with stale - // data. This gives horizontal read scalability since - // any Consul server can service the query instead of - // only the leader. - AllowStale *bool `mapstructure:"allow_stale"` - - // EnableTruncate is used to enable setting the truncate - // flag for UDP DNS queries. This allows unmodified - // clients to re-query the consul server using TCP - // when the total number of records exceeds the number - // returned by default for UDP. - EnableTruncate bool `mapstructure:"enable_truncate"` - - // UDPAnswerLimit is used to limit the maximum number of DNS Resource - // Records returned in the ANSWER section of a DNS response. This is - // not normally useful and will be limited based on the querying - // protocol, however systems that implemented §6 Rule 9 in RFC3484 - // may want to set this to `1` in order to subvert §6 Rule 9 and - // re-obtain the effect of randomized resource records (i.e. each - // answer contains only one IP, but the IP changes every request). - // RFC3484 sorts answers in a deterministic order, which defeats the - // purpose of randomized DNS responses. This RFC has been obsoleted - // by RFC6724 and restores the desired behavior of randomized - // responses, however a large number of Linux hosts using glibc(3) - // implemented §6 Rule 9 and may need this option (e.g. CentOS 5-6, - // Debian Squeeze, etc). - UDPAnswerLimit int `mapstructure:"udp_answer_limit"` - - // MaxStale is used to bound how stale of a result is - // accepted for a DNS lookup. This can be used with - // AllowStale to limit how old of a value is served up. - // If the stale result exceeds this, another non-stale - // stale read is performed. - MaxStale time.Duration `mapstructure:"-"` - MaxStaleRaw string `mapstructure:"max_stale" json:"-"` - - // OnlyPassing is used to determine whether to filter nodes - // whose health checks are in any non-passing state. By - // default, only nodes in a critical state are excluded. - OnlyPassing bool `mapstructure:"only_passing"` - - // DisableCompression is used to control whether DNS responses are - // compressed. In Consul 0.7 this was turned on by default and this - // config was added as an opt-out. - DisableCompression bool `mapstructure:"disable_compression"` - - // RecursorTimeout specifies the timeout in seconds - // for Consul's internal dns client used for recursion. - // This value is used for the connection, read and write timeout. - // Default: 2s - RecursorTimeout time.Duration `mapstructure:"-"` - RecursorTimeoutRaw string `mapstructure:"recursor_timeout" json:"-"` -} - -// HTTPConfig is used to fine tune the Http sub-system. -type HTTPConfig struct { - // BlockEndpoints is a list of endpoint prefixes to block in the - // HTTP API. Any requests to these will get a 403 response. - BlockEndpoints []string `mapstructure:"block_endpoints"` - - // ResponseHeaders are used to add HTTP header response fields to the HTTP API responses. - ResponseHeaders map[string]string `mapstructure:"response_headers"` -} - -// RetryJoinEC2 is used to configure discovery of instances via Amazon's EC2 api -type RetryJoinEC2 struct { - // The AWS region to look for instances in - Region string `mapstructure:"region"` - - // The tag key and value to use when filtering instances - TagKey string `mapstructure:"tag_key"` - TagValue string `mapstructure:"tag_value"` - - // The AWS credentials to use for making requests to EC2 - AccessKeyID string `mapstructure:"access_key_id" json:"-"` - SecretAccessKey string `mapstructure:"secret_access_key" json:"-"` -} - -// RetryJoinGCE is used to configure discovery of instances via Google Compute -// Engine's API. -type RetryJoinGCE struct { - // The name of the project the instances reside in. - ProjectName string `mapstructure:"project_name"` - - // A regular expression (RE2) pattern for the zones you want to discover the instances in. - // Example: us-west1-.*, or us-(?west|east).*. - ZonePattern string `mapstructure:"zone_pattern"` - - // The tag value to search for when filtering instances. - TagValue string `mapstructure:"tag_value"` - - // A path to a JSON file with the service account credentials necessary to - // connect to GCE. If this is not defined, the following chain is respected: - // 1. A JSON file whose path is specified by the - // GOOGLE_APPLICATION_CREDENTIALS environment variable. - // 2. A JSON file in a location known to the gcloud command-line tool. - // On Windows, this is %APPDATA%/gcloud/application_default_credentials.json. - // On other systems, $HOME/.config/gcloud/application_default_credentials.json. - // 3. On Google Compute Engine, it fetches credentials from the metadata - // server. (In this final case any provided scopes are ignored.) - CredentialsFile string `mapstructure:"credentials_file"` -} - -// RetryJoinAzure is used to configure discovery of instances via AzureRM API -type RetryJoinAzure struct { - // The tag name and value to use when filtering instances - TagName string `mapstructure:"tag_name"` - TagValue string `mapstructure:"tag_value"` - - // The Azure credentials to use for making requests to AzureRM - SubscriptionID string `mapstructure:"subscription_id" json:"-"` - TenantID string `mapstructure:"tenant_id" json:"-"` - ClientID string `mapstructure:"client_id" json:"-"` - SecretAccessKey string `mapstructure:"secret_access_key" json:"-"` -} - -// Limits is used to configure limits enforced by the agent. -type Limits struct { - // RPCRate and RPCMaxBurst control how frequently RPC calls are allowed - // to happen. In any large enough time interval, rate limiter limits the - // rate to RPCRate tokens per second, with a maximum burst size of - // RPCMaxBurst events. As a special case, if RPCRate == Inf (the infinite - // rate), RPCMaxBurst is ignored. - // - // See https://en.wikipedia.org/wiki/Token_bucket for more about token - // buckets. - RPCRate rate.Limit `mapstructure:"rpc_rate"` - RPCMaxBurst int `mapstructure:"rpc_max_burst"` -} - -// Performance is used to tune the performance of Consul's subsystems. -type Performance struct { - // RaftMultiplier is an integer multiplier used to scale Raft timing - // parameters: HeartbeatTimeout, ElectionTimeout, and LeaderLeaseTimeout. - RaftMultiplier uint `mapstructure:"raft_multiplier"` -} - -// Telemetry is the telemetry configuration for the server -type Telemetry struct { - // StatsiteAddr is the address of a statsite instance. If provided, - // metrics will be streamed to that instance. - StatsiteAddr string `mapstructure:"statsite_address"` - - // StatsdAddr is the address of a statsd instance. If provided, - // metrics will be sent to that instance. - StatsdAddr string `mapstructure:"statsd_address"` - - // StatsitePrefix is the prefix used to write stats values to. By - // default this is set to 'consul'. - StatsitePrefix string `mapstructure:"statsite_prefix"` - - // DisableHostname will disable hostname prefixing for all metrics - DisableHostname bool `mapstructure:"disable_hostname"` - - // PrefixFilter is a list of filter rules to apply for allowing/blocking metrics - // by prefix. - PrefixFilter []string `mapstructure:"prefix_filter"` - AllowedPrefixes []string `mapstructure:"-" json:"-"` - BlockedPrefixes []string `mapstructure:"-" json:"-"` - - // FilterDefault is the default for whether to allow a metric that's not - // covered by the filter. - FilterDefault *bool `mapstructure:"filter_default"` - - // DogStatsdAddr is the address of a dogstatsd instance. If provided, - // metrics will be sent to that instance - DogStatsdAddr string `mapstructure:"dogstatsd_addr"` - - // DogStatsdTags are the global tags that should be sent with each packet to dogstatsd - // It is a list of strings, where each string looks like "my_tag_name:my_tag_value" - DogStatsdTags []string `mapstructure:"dogstatsd_tags"` - - // Circonus: see https://github.com/circonus-labs/circonus-gometrics - // for more details on the various configuration options. - // Valid configuration combinations: - // - CirconusAPIToken - // metric management enabled (search for existing check or create a new one) - // - CirconusSubmissionUrl - // metric management disabled (use check with specified submission_url, - // broker must be using a public SSL certificate) - // - CirconusAPIToken + CirconusCheckSubmissionURL - // metric management enabled (use check with specified submission_url) - // - CirconusAPIToken + CirconusCheckID - // metric management enabled (use check with specified id) - - // CirconusAPIToken is a valid API Token used to create/manage check. If provided, - // metric management is enabled. - // Default: none - CirconusAPIToken string `mapstructure:"circonus_api_token" json:"-"` - // CirconusAPIApp is an app name associated with API token. - // Default: "consul" - CirconusAPIApp string `mapstructure:"circonus_api_app"` - // CirconusAPIURL is the base URL to use for contacting the Circonus API. - // Default: "https://api.circonus.com/v2" - CirconusAPIURL string `mapstructure:"circonus_api_url"` - // CirconusSubmissionInterval is the interval at which metrics are submitted to Circonus. - // Default: 10s - CirconusSubmissionInterval string `mapstructure:"circonus_submission_interval"` - // CirconusCheckSubmissionURL is the check.config.submission_url field from a - // previously created HTTPTRAP check. - // Default: none - CirconusCheckSubmissionURL string `mapstructure:"circonus_submission_url"` - // CirconusCheckID is the check id (not check bundle id) from a previously created - // HTTPTRAP check. The numeric portion of the check._cid field. - // Default: none - CirconusCheckID string `mapstructure:"circonus_check_id"` - // CirconusCheckForceMetricActivation will force enabling metrics, as they are encountered, - // if the metric already exists and is NOT active. If check management is enabled, the default - // behavior is to add new metrics as they are encoutered. If the metric already exists in the - // check, it will *NOT* be activated. This setting overrides that behavior. - // Default: "false" - CirconusCheckForceMetricActivation string `mapstructure:"circonus_check_force_metric_activation"` - // CirconusCheckInstanceID serves to uniquely identify the metrics coming from this "instance". - // It can be used to maintain metric continuity with transient or ephemeral instances as - // they move around within an infrastructure. - // Default: hostname:app - CirconusCheckInstanceID string `mapstructure:"circonus_check_instance_id"` - // CirconusCheckSearchTag is a special tag which, when coupled with the instance id, helps to - // narrow down the search results when neither a Submission URL or Check ID is provided. - // Default: service:app (e.g. service:consul) - CirconusCheckSearchTag string `mapstructure:"circonus_check_search_tag"` - // CirconusCheckTags is a comma separated list of tags to apply to the check. Note that - // the value of CirconusCheckSearchTag will always be added to the check. - // Default: none - CirconusCheckTags string `mapstructure:"circonus_check_tags"` - // CirconusCheckDisplayName is the name for the check which will be displayed in the Circonus UI. - // Default: value of CirconusCheckInstanceID - CirconusCheckDisplayName string `mapstructure:"circonus_check_display_name"` - // CirconusBrokerID is an explicit broker to use when creating a new check. The numeric portion - // of broker._cid. If metric management is enabled and neither a Submission URL nor Check ID - // is provided, an attempt will be made to search for an existing check using Instance ID and - // Search Tag. If one is not found, a new HTTPTRAP check will be created. - // Default: use Select Tag if provided, otherwise, a random Enterprise Broker associated - // with the specified API token or the default Circonus Broker. - // Default: none - CirconusBrokerID string `mapstructure:"circonus_broker_id"` - // CirconusBrokerSelectTag is a special tag which will be used to select a broker when - // a Broker ID is not provided. The best use of this is to as a hint for which broker - // should be used based on *where* this particular instance is running. - // (e.g. a specific geo location or datacenter, dc:sfo) - // Default: none - CirconusBrokerSelectTag string `mapstructure:"circonus_broker_select_tag"` -} - -// Autopilot is used to configure helpful features for operating Consul servers. -type Autopilot struct { - // CleanupDeadServers enables the automatic cleanup of dead servers when new ones - // are added to the peer list. Defaults to true. - CleanupDeadServers *bool `mapstructure:"cleanup_dead_servers"` - - // LastContactThreshold is the limit on the amount of time a server can go - // without leader contact before being considered unhealthy. - LastContactThreshold *time.Duration `mapstructure:"-" json:"-"` - LastContactThresholdRaw string `mapstructure:"last_contact_threshold"` - - // MaxTrailingLogs is the amount of entries in the Raft Log that a server can - // be behind before being considered unhealthy. - MaxTrailingLogs *uint64 `mapstructure:"max_trailing_logs"` - - // ServerStabilizationTime is the minimum amount of time a server must be - // in a stable, healthy state before it can be added to the cluster. Only - // applicable with Raft protocol version 3 or higher. - ServerStabilizationTime *time.Duration `mapstructure:"-" json:"-"` - ServerStabilizationTimeRaw string `mapstructure:"server_stabilization_time"` - - // (Enterprise-only) RedundancyZoneTag is the Meta tag to use for separating servers - // into zones for redundancy. If left blank, this feature will be disabled. - RedundancyZoneTag string `mapstructure:"redundancy_zone_tag"` - - // (Enterprise-only) DisableUpgradeMigration will disable Autopilot's upgrade migration - // strategy of waiting until enough newer-versioned servers have been added to the - // cluster before promoting them to voters. - DisableUpgradeMigration *bool `mapstructure:"disable_upgrade_migration"` - - // (Enterprise-only) UpgradeVersionTag is the node tag to use for version info when - // performing upgrade migrations. If left blank, the Consul version will be used. - UpgradeVersionTag string `mapstructure:"upgrade_version_tag"` -} - -// (Enterprise-only) NetworkSegment is the configuration for a network segment, which is an -// isolated serf group on the LAN. -type NetworkSegment struct { - // Name is the name of the segment. - Name string `mapstructure:"name"` - - // Bind is the bind address for this segment. - Bind string `mapstructure:"bind"` - - // Port is the port for this segment. - Port int `mapstructure:"port"` - - // RPCListener is whether to bind a separate RPC listener on the bind address - // for this segment. - RPCListener bool `mapstructure:"rpc_listener"` - - // Advertise is the advertise address of this segment. - Advertise string `mapstructure:"advertise"` -} - -// Config is the configuration that can be set for an Agent. -// Some of this is configurable as CLI flags, but most must -// be set using a configuration file. -type Config struct { - // DevMode enables a fast-path mode of operation to bring up an in-memory - // server with minimal configuration. Useful for developing Consul. - DevMode bool `mapstructure:"-"` - - // Limits is used to configure limits enforced by the agent. - Limits Limits `mapstructure:"limits"` - - // Performance is used to tune the performance of Consul's subsystems. - Performance Performance `mapstructure:"performance"` - - // Bootstrap is used to bring up the first Consul server, and - // permits that node to elect itself leader - Bootstrap bool `mapstructure:"bootstrap"` - - // BootstrapExpect tries to automatically bootstrap the Consul cluster, - // by withholding peers until enough servers join. - BootstrapExpect int `mapstructure:"bootstrap_expect"` - - // Server controls if this agent acts like a Consul server, - // or merely as a client. Servers have more state, take part - // in leader election, etc. - Server bool `mapstructure:"server"` - - // (Enterprise-only) NonVotingServer is whether this server will act as a non-voting member - // of the cluster to help provide read scalability. - NonVotingServer bool `mapstructure:"non_voting_server"` - - // Datacenter is the datacenter this node is in. Defaults to dc1 - Datacenter string `mapstructure:"datacenter"` - - // DataDir is the directory to store our state in - DataDir string `mapstructure:"data_dir"` - - // DNSRecursors can be set to allow the DNS servers to recursively - // resolve non-consul domains. It is deprecated, and merges into the - // recursors array. - DNSRecursor string `mapstructure:"recursor"` - - // DNSRecursors can be set to allow the DNS servers to recursively - // resolve non-consul domains - DNSRecursors []string `mapstructure:"recursors"` - - // DNS configuration - DNSConfig DNSConfig `mapstructure:"dns_config"` - - // Domain is the DNS domain for the records. Defaults to "consul." - Domain string `mapstructure:"domain"` - - // HTTP configuration - HTTPConfig HTTPConfig `mapstructure:"http_config"` - - // Encryption key to use for the Serf communication - EncryptKey string `mapstructure:"encrypt" json:"-"` - - // Disables writing the keyring to a file. - DisableKeyringFile bool `mapstructure:"disable_keyring_file"` - - // EncryptVerifyIncoming and EncryptVerifyOutgoing are used to enforce - // incoming/outgoing gossip encryption and can be used to upshift to - // encrypted gossip on a running cluster. - EncryptVerifyIncoming *bool `mapstructure:"encrypt_verify_incoming"` - EncryptVerifyOutgoing *bool `mapstructure:"encrypt_verify_outgoing"` - - // LogLevel is the level of the logs to putout - LogLevel string `mapstructure:"log_level"` - - // Node ID is a unique ID for this node across space and time. Defaults - // to a randomly-generated ID that persists in the data-dir. - NodeID types.NodeID `mapstructure:"node_id"` - - // DisableHostNodeID will prevent Consul from using information from the - // host to generate a node ID, and will cause Consul to generate a - // random ID instead. - DisableHostNodeID *bool `mapstructure:"disable_host_node_id"` - - // Node name is the name we use to advertise. Defaults to hostname. - NodeName string `mapstructure:"node_name"` - - // ClientAddr is used to control the address we bind to for - // client services (DNS, HTTP, HTTPS, RPC) - ClientAddr string `mapstructure:"client_addr"` - - // BindAddr is used to control the address we bind to. - // If not specified, the first private IP we find is used. - // This controls the address we use for cluster facing - // services (Gossip, Server RPC) - BindAddr string `mapstructure:"bind_addr"` - - // SerfWanBindAddr is used to control the address we bind to. - // If not specified, the first private IP we find is used. - // This controls the address we use for cluster facing - // services (Gossip) Serf - SerfWanBindAddr string `mapstructure:"serf_wan_bind"` - - // SerfLanBindAddr is used to control the address we bind to. - // If not specified, the first private IP we find is used. - // This controls the address we use for cluster facing - // services (Gossip) Serf - SerfLanBindAddr string `mapstructure:"serf_lan_bind"` - - // AdvertiseAddr is the address we use for advertising our Serf, - // and Consul RPC IP. If not specified, bind address is used. - AdvertiseAddr string `mapstructure:"advertise_addr"` - - // AdvertiseAddrs configuration - AdvertiseAddrs AdvertiseAddrsConfig `mapstructure:"advertise_addrs"` - - // AdvertiseAddrWan is the address we use for advertising our - // Serf WAN IP. If not specified, the general advertise address is used. - AdvertiseAddrWan string `mapstructure:"advertise_addr_wan"` - - // TranslateWanAddrs controls whether or not Consul should prefer - // the "wan" tagged address when doing lookups in remote datacenters. - // See TaggedAddresses below for more details. - TranslateWanAddrs bool `mapstructure:"translate_wan_addrs"` - - // Port configurations - Ports PortConfig - - // Address configurations - Addresses AddressConfig - - // (Enterprise-only) NetworkSegment is the network segment for this client to join. - Segment string `mapstructure:"segment"` - - // (Enterprise-only) Segments is the list of network segments for this server to - // initialize. - Segments []NetworkSegment `mapstructure:"segments"` - - // Tagged addresses. These are used to publish a set of addresses for - // for a node, which can be used by the remote agent. We currently - // populate only the "wan" tag based on the SerfWan advertise address, - // but this structure is here for possible future features with other - // user-defined tags. The "wan" tag will be used by remote agents if - // they are configured with TranslateWanAddrs set to true. - TaggedAddresses map[string]string - - // Node metadata key/value pairs. These are excluded from JSON output - // because they can be reloaded and might be stale when shown from the - // config instead of the local state. - Meta map[string]string `mapstructure:"node_meta" json:"-"` - - // LeaveOnTerm controls if Serf does a graceful leave when receiving - // the TERM signal. Defaults true on clients, false on servers. This can - // be changed on reload. - LeaveOnTerm *bool `mapstructure:"leave_on_terminate"` - - // SkipLeaveOnInt controls if Serf skips a graceful leave when - // receiving the INT signal. Defaults false on clients, true on - // servers. This can be changed on reload. - SkipLeaveOnInt *bool `mapstructure:"skip_leave_on_interrupt"` - - // Autopilot is used to configure helpful features for operating Consul servers. - Autopilot Autopilot `mapstructure:"autopilot"` - - Telemetry Telemetry `mapstructure:"telemetry"` - - // Protocol is the Consul protocol version to use. - Protocol int `mapstructure:"protocol"` - - // RaftProtocol sets the Raft protocol version to use on this server. - RaftProtocol int `mapstructure:"raft_protocol"` - - // EnableDebug is used to enable various debugging features - EnableDebug bool `mapstructure:"enable_debug"` - - // VerifyIncoming is used to verify the authenticity of incoming connections. - // This means that TCP requests are forbidden, only allowing for TLS. TLS connections - // must match a provided certificate authority. This can be used to force client auth. - VerifyIncoming bool `mapstructure:"verify_incoming"` - - // VerifyIncomingRPC is used to verify the authenticity of incoming RPC connections. - // This means that TCP requests are forbidden, only allowing for TLS. TLS connections - // must match a provided certificate authority. This can be used to force client auth. - VerifyIncomingRPC bool `mapstructure:"verify_incoming_rpc"` - - // VerifyIncomingHTTPS is used to verify the authenticity of incoming HTTPS connections. - // This means that TCP requests are forbidden, only allowing for TLS. TLS connections - // must match a provided certificate authority. This can be used to force client auth. - VerifyIncomingHTTPS bool `mapstructure:"verify_incoming_https"` - - // VerifyOutgoing is used to verify the authenticity of outgoing connections. - // This means that TLS requests are used. TLS connections must match a provided - // certificate authority. This is used to verify authenticity of server nodes. - VerifyOutgoing bool `mapstructure:"verify_outgoing"` - - // VerifyServerHostname is used to enable hostname verification of servers. This - // ensures that the certificate presented is valid for server... - // This prevents a compromised client from being restarted as a server, and then - // intercepting request traffic as well as being added as a raft peer. This should be - // enabled by default with VerifyOutgoing, but for legacy reasons we cannot break - // existing clients. - VerifyServerHostname bool `mapstructure:"verify_server_hostname"` - - // CAFile is a path to a certificate authority file. This is used with VerifyIncoming - // or VerifyOutgoing to verify the TLS connection. - CAFile string `mapstructure:"ca_file"` - - // CAPath is a path to a directory of certificate authority files. This is used with - // VerifyIncoming or VerifyOutgoing to verify the TLS connection. - CAPath string `mapstructure:"ca_path"` - - // CertFile is used to provide a TLS certificate that is used for serving TLS connections. - // Must be provided to serve TLS connections. - CertFile string `mapstructure:"cert_file"` - - // KeyFile is used to provide a TLS key that is used for serving TLS connections. - // Must be provided to serve TLS connections. - KeyFile string `mapstructure:"key_file"` - - // ServerName is used with the TLS certificates to ensure the name we - // provide matches the certificate - ServerName string `mapstructure:"server_name"` - - // TLSMinVersion is used to set the minimum TLS version used for TLS connections. - TLSMinVersion string `mapstructure:"tls_min_version"` - - // TLSCipherSuites is used to specify the list of supported ciphersuites. - TLSCipherSuites []uint16 `mapstructure:"-" json:"-"` - TLSCipherSuitesRaw string `mapstructure:"tls_cipher_suites"` - - // TLSPreferServerCipherSuites specifies whether to prefer the server's ciphersuite - // over the client ciphersuites. - TLSPreferServerCipherSuites bool `mapstructure:"tls_prefer_server_cipher_suites"` - - // StartJoin is a list of addresses to attempt to join when the - // agent starts. If Serf is unable to communicate with any of these - // addresses, then the agent will error and exit. - StartJoin []string `mapstructure:"start_join"` - - // StartJoinWan is a list of addresses to attempt to join -wan when the - // agent starts. If Serf is unable to communicate with any of these - // addresses, then the agent will error and exit. - StartJoinWan []string `mapstructure:"start_join_wan"` - - // RetryJoin is a list of addresses to join with retry enabled. - RetryJoin []string `mapstructure:"retry_join" json:"-"` - - // RetryMaxAttempts specifies the maximum number of times to retry joining a - // host on startup. This is useful for cases where we know the node will be - // online eventually. - RetryMaxAttempts int `mapstructure:"retry_max"` - - // RetryInterval specifies the amount of time to wait in between join - // attempts on agent start. The minimum allowed value is 1 second and - // the default is 30s. - RetryInterval time.Duration `mapstructure:"-" json:"-"` - RetryIntervalRaw string `mapstructure:"retry_interval"` - - // RetryJoinWan is a list of addresses to join -wan with retry enabled. - RetryJoinWan []string `mapstructure:"retry_join_wan"` - - // RetryMaxAttemptsWan specifies the maximum number of times to retry joining a - // -wan host on startup. This is useful for cases where we know the node will be - // online eventually. - RetryMaxAttemptsWan int `mapstructure:"retry_max_wan"` - - // RetryIntervalWan specifies the amount of time to wait in between join - // -wan attempts on agent start. The minimum allowed value is 1 second and - // the default is 30s. - RetryIntervalWan time.Duration `mapstructure:"-" json:"-"` - RetryIntervalWanRaw string `mapstructure:"retry_interval_wan"` - - // ReconnectTimeout* specify the amount of time to wait to reconnect with - // another agent before deciding it's permanently gone. This can be used to - // control the time it takes to reap failed nodes from the cluster. - ReconnectTimeoutLan time.Duration `mapstructure:"-"` - ReconnectTimeoutLanRaw string `mapstructure:"reconnect_timeout"` - ReconnectTimeoutWan time.Duration `mapstructure:"-"` - ReconnectTimeoutWanRaw string `mapstructure:"reconnect_timeout_wan"` - - // EnableUI enables the statically-compiled assets for the Consul web UI and - // serves them at the default /ui/ endpoint automatically. - EnableUI bool `mapstructure:"ui"` - - // UIDir is the directory containing the Web UI resources. - // If provided, the UI endpoints will be enabled. - UIDir string `mapstructure:"ui_dir"` - - // PidFile is the file to store our PID in - PidFile string `mapstructure:"pid_file"` - - // EnableSyslog is used to also tee all the logs over to syslog. Only supported - // on linux and OSX. Other platforms will generate an error. - EnableSyslog bool `mapstructure:"enable_syslog"` - - // SyslogFacility is used to control where the syslog messages go - // By default, goes to LOCAL0 - SyslogFacility string `mapstructure:"syslog_facility"` - - // RejoinAfterLeave controls our interaction with the cluster after leave. - // When set to false (default), a leave causes Consul to not rejoin - // the cluster until an explicit join is received. If this is set to - // true, we ignore the leave, and rejoin the cluster on start. - RejoinAfterLeave bool `mapstructure:"rejoin_after_leave"` - - // EnableScriptChecks controls whether health checks which execute - // scripts are enabled. This includes regular script checks and Docker - // checks. - EnableScriptChecks bool `mapstructure:"enable_script_checks"` - - // CheckUpdateInterval controls the interval on which the output of a health check - // is updated if there is no change to the state. For example, a check in a steady - // state may run every 5 second generating a unique output (timestamp, etc), forcing - // constant writes. This allows Consul to defer the write for some period of time, - // reducing the write pressure when the state is steady. - CheckUpdateInterval time.Duration `mapstructure:"-"` - CheckUpdateIntervalRaw string `mapstructure:"check_update_interval" json:"-"` - - // CheckReapInterval controls the interval on which we will look for - // failed checks and reap their associated services, if so configured. - CheckReapInterval time.Duration `mapstructure:"-"` - - // CheckDeregisterIntervalMin is the smallest allowed interval to set - // a check's DeregisterCriticalServiceAfter value to. - CheckDeregisterIntervalMin time.Duration `mapstructure:"-"` - - // ACLToken is the default token used to make requests if a per-request - // token is not provided. If not configured the 'anonymous' token is used. - ACLToken string `mapstructure:"acl_token" json:"-"` - - // ACLAgentMasterToken is a special token that has full read and write - // privileges for this agent, and can be used to call agent endpoints - // when no servers are available. - ACLAgentMasterToken string `mapstructure:"acl_agent_master_token" json:"-"` - - // ACLAgentToken is the default token used to make requests for the agent - // itself, such as for registering itself with the catalog. If not - // configured, the 'acl_token' will be used. - ACLAgentToken string `mapstructure:"acl_agent_token" json:"-"` - - // ACLMasterToken is used to bootstrap the ACL system. It should be specified - // on the servers in the ACLDatacenter. When the leader comes online, it ensures - // that the Master token is available. This provides the initial token. - ACLMasterToken string `mapstructure:"acl_master_token" json:"-"` - - // ACLDatacenter is the central datacenter that holds authoritative - // ACL records. This must be the same for the entire cluster. - // If this is not set, ACLs are not enabled. Off by default. - ACLDatacenter string `mapstructure:"acl_datacenter"` - - // ACLTTL is used to control the time-to-live of cached ACLs . This has - // a major impact on performance. By default, it is set to 30 seconds. - ACLTTL time.Duration `mapstructure:"-"` - ACLTTLRaw string `mapstructure:"acl_ttl"` - - // ACLDefaultPolicy is used to control the ACL interaction when - // there is no defined policy. This can be "allow" which means - // ACLs are used to black-list, or "deny" which means ACLs are - // white-lists. - ACLDefaultPolicy string `mapstructure:"acl_default_policy"` - - // ACLDisabledTTL is used by clients to determine how long they will - // wait to check again with the servers if they discover ACLs are not - // enabled. - ACLDisabledTTL time.Duration `mapstructure:"-"` - - // ACLDownPolicy is used to control the ACL interaction when we cannot - // reach the ACLDatacenter and the token is not in the cache. - // There are two modes: - // * allow - Allow all requests - // * deny - Deny all requests - // * extend-cache - Ignore the cache expiration, and allow cached - // ACL's to be used to service requests. This - // is the default. If the ACL is not in the cache, - // this acts like deny. - ACLDownPolicy string `mapstructure:"acl_down_policy"` - - // EnableACLReplication is used to turn on ACL replication when using - // /v1/agent/token/acl_replication_token to introduce the token, instead - // of setting acl_replication_token in the config. Setting the token via - // config will also set this to true for backward compatibility. - EnableACLReplication bool `mapstructure:"enable_acl_replication"` - - // ACLReplicationToken is used to fetch ACLs from the ACLDatacenter in - // order to replicate them locally. Setting this to a non-empty value - // also enables replication. Replication is only available in datacenters - // other than the ACLDatacenter. - ACLReplicationToken string `mapstructure:"acl_replication_token" json:"-"` - - // ACLEnforceVersion8 is used to gate a set of ACL policy features that - // are opt-in prior to Consul 0.8 and opt-out in Consul 0.8 and later. - ACLEnforceVersion8 *bool `mapstructure:"acl_enforce_version_8"` - - // Watches are used to monitor various endpoints and to invoke a - // handler to act appropriately. These are managed entirely in the - // agent layer using the standard APIs. - Watches []map[string]interface{} `mapstructure:"watches"` - - // DisableRemoteExec is used to turn off the remote execution - // feature. This is for security to prevent unknown scripts from running. - DisableRemoteExec *bool `mapstructure:"disable_remote_exec"` - - // DisableUpdateCheck is used to turn off the automatic update and - // security bulletin checking. - DisableUpdateCheck bool `mapstructure:"disable_update_check"` - - // DisableAnonymousSignature is used to turn off the anonymous signature - // send with the update check. This is used to deduplicate messages. - DisableAnonymousSignature bool `mapstructure:"disable_anonymous_signature"` - - // AEInterval controls the anti-entropy interval. This is how often - // the agent attempts to reconcile its local state with the server's - // representation of our state. Defaults to every 60s. - AEInterval time.Duration `mapstructure:"-" json:"-"` - - // DisableCoordinates controls features related to network coordinates. - DisableCoordinates bool `mapstructure:"disable_coordinates"` - - // SyncCoordinateRateTarget controls the rate for sending network - // coordinates to the server, in updates per second. This is the max rate - // that the server supports, so we scale our interval based on the size - // of the cluster to try to achieve this in aggregate at the server. - SyncCoordinateRateTarget float64 `mapstructure:"-" json:"-"` - - // SyncCoordinateIntervalMin sets the minimum interval that coordinates - // will be sent to the server. We scale the interval based on the cluster - // size, but below a certain interval it doesn't make sense send them any - // faster. - SyncCoordinateIntervalMin time.Duration `mapstructure:"-" json:"-"` - - // Checks holds the provided check definitions - Checks []*structs.CheckDefinition `mapstructure:"-" json:"-"` - - // Services holds the provided service definitions - Services []*structs.ServiceDefinition `mapstructure:"-" json:"-"` - - // ConsulConfig can either be provided or a default one created - ConsulConfig *consul.Config `mapstructure:"-" json:"-"` - - // Revision is the GitCommit this maps to - Revision string `mapstructure:"-"` - - // Version is the release version number - Version string `mapstructure:"-"` - - // VersionPrerelease is a label for pre-release builds - VersionPrerelease string `mapstructure:"-"` - - // WatchPlans contains the compiled watches - WatchPlans []*watch.Plan `mapstructure:"-" json:"-"` - - // UnixSockets is a map of socket configuration data - UnixSockets UnixSocketConfig `mapstructure:"unix_sockets"` - - // Minimum Session TTL - SessionTTLMin time.Duration `mapstructure:"-"` - SessionTTLMinRaw string `mapstructure:"session_ttl_min"` - - // deprecated fields - // keep them exported since otherwise the error messages don't show up - DeprecatedAtlasInfrastructure string `mapstructure:"atlas_infrastructure" json:"-"` - DeprecatedAtlasToken string `mapstructure:"atlas_token" json:"-"` - DeprecatedAtlasACLToken string `mapstructure:"atlas_acl_token" json:"-"` - DeprecatedAtlasJoin bool `mapstructure:"atlas_join" json:"-"` - DeprecatedAtlasEndpoint string `mapstructure:"atlas_endpoint" json:"-"` - DeprecatedHTTPAPIResponseHeaders map[string]string `mapstructure:"http_api_response_headers"` - DeprecatedRetryJoinEC2 RetryJoinEC2 `mapstructure:"retry_join_ec2"` - DeprecatedRetryJoinGCE RetryJoinGCE `mapstructure:"retry_join_gce"` - DeprecatedRetryJoinAzure RetryJoinAzure `mapstructure:"retry_join_azure"` -} - -// IncomingHTTPSConfig returns the TLS configuration for HTTPS -// connections to consul. -func (c *Config) IncomingHTTPSConfig() (*tls.Config, error) { - tc := &tlsutil.Config{ - VerifyIncoming: c.VerifyIncoming || c.VerifyIncomingHTTPS, - VerifyOutgoing: c.VerifyOutgoing, - CAFile: c.CAFile, - CAPath: c.CAPath, - CertFile: c.CertFile, - KeyFile: c.KeyFile, - NodeName: c.NodeName, - ServerName: c.ServerName, - TLSMinVersion: c.TLSMinVersion, - CipherSuites: c.TLSCipherSuites, - PreferServerCipherSuites: c.TLSPreferServerCipherSuites, - } - return tc.IncomingTLSConfig() -} - -type ProtoAddr struct { - Proto, Net, Addr string -} - -func (p ProtoAddr) String() string { - return p.Proto + "://" + p.Addr -} - -func (c *Config) DNSAddrs() ([]ProtoAddr, error) { - if c.Ports.DNS <= 0 { - return nil, nil - } - a, err := c.ClientListener(c.Addresses.DNS, c.Ports.DNS) - if err != nil { - return nil, err - } - addrs := []ProtoAddr{ - {"dns", "tcp", a.String()}, - {"dns", "udp", a.String()}, - } - return addrs, nil -} - -// HTTPAddrs returns the bind addresses for the HTTP server and -// the application protocol which should be served, e.g. 'http' -// or 'https'. -func (c *Config) HTTPAddrs() ([]ProtoAddr, error) { - var addrs []ProtoAddr - if c.Ports.HTTP > 0 { - a, err := c.ClientListener(c.Addresses.HTTP, c.Ports.HTTP) - if err != nil { - return nil, err - } - addrs = append(addrs, ProtoAddr{"http", a.Network(), a.String()}) - } - if c.Ports.HTTPS > 0 && c.CertFile != "" && c.KeyFile != "" { - a, err := c.ClientListener(c.Addresses.HTTPS, c.Ports.HTTPS) - if err != nil { - return nil, err - } - addrs = append(addrs, ProtoAddr{"https", a.Network(), a.String()}) - } - return addrs, nil -} - -// Bool is used to initialize bool pointers in struct literals. -func Bool(b bool) *bool { - return &b -} - -// Uint64 is used to initialize uint64 pointers in struct literals. -func Uint64(i uint64) *uint64 { - return &i -} - -// Duration is used to initialize time.Duration pointers in struct literals. -func Duration(d time.Duration) *time.Duration { - return &d -} - -// UnixSocketPermissions contains information about a unix socket, and -// implements the FilePermissions interface. -type UnixSocketPermissions struct { - Usr string `mapstructure:"user"` - Grp string `mapstructure:"group"` - Perms string `mapstructure:"mode"` -} - -func (u UnixSocketPermissions) User() string { - return u.Usr -} - -func (u UnixSocketPermissions) Group() string { - return u.Grp -} - -func (u UnixSocketPermissions) Mode() string { - return u.Perms -} - -func (s *Telemetry) GoString() string { - return fmt.Sprintf("*%#v", *s) -} - -// UnixSocketConfig stores information about various unix sockets which -// Consul creates and uses for communication. -type UnixSocketConfig struct { - UnixSocketPermissions `mapstructure:",squash"` -} - -// socketPath tests if a given address describes a domain socket, -// and returns the relevant path part of the string if it is. -func socketPath(addr string) string { - if !strings.HasPrefix(addr, "unix://") { - return "" - } - return strings.TrimPrefix(addr, "unix://") -} - -type dirEnts []os.FileInfo - -// DefaultConfig is used to return a sane default configuration -func DefaultConfig() *Config { - return &Config{ - Limits: Limits{ - RPCRate: rate.Inf, - RPCMaxBurst: 1000, - }, - Bootstrap: false, - BootstrapExpect: 0, - Server: false, - Datacenter: consul.DefaultDC, - Domain: "consul.", - LogLevel: "INFO", - ClientAddr: "127.0.0.1", - BindAddr: "0.0.0.0", - Ports: PortConfig{ - DNS: 8600, - HTTP: 8500, - HTTPS: -1, - SerfLan: consul.DefaultLANSerfPort, - SerfWan: consul.DefaultWANSerfPort, - Server: 8300, - }, - DNSConfig: DNSConfig{ - AllowStale: Bool(true), - UDPAnswerLimit: 3, - MaxStale: 10 * 365 * 24 * time.Hour, - RecursorTimeout: 2 * time.Second, - }, - Telemetry: Telemetry{ - StatsitePrefix: "consul", - FilterDefault: Bool(true), - }, - Meta: make(map[string]string), - SyslogFacility: "LOCAL0", - Protocol: consul.ProtocolVersion2Compatible, - CheckUpdateInterval: 5 * time.Minute, - CheckDeregisterIntervalMin: time.Minute, - CheckReapInterval: 30 * time.Second, - AEInterval: time.Minute, - DisableCoordinates: false, - - // SyncCoordinateRateTarget is set based on the rate that we want - // the server to handle as an aggregate across the entire cluster. - // If you update this, you'll need to adjust CoordinateUpdate* in - // the server-side config accordingly. - SyncCoordinateRateTarget: 64.0, // updates / second - SyncCoordinateIntervalMin: 15 * time.Second, - - ACLTTL: 30 * time.Second, - ACLDownPolicy: "extend-cache", - ACLDefaultPolicy: "allow", - ACLDisabledTTL: 120 * time.Second, - ACLEnforceVersion8: Bool(true), - DisableRemoteExec: Bool(true), - RetryInterval: 30 * time.Second, - RetryIntervalWan: 30 * time.Second, - - TLSMinVersion: "tls10", - - EncryptVerifyIncoming: Bool(true), - EncryptVerifyOutgoing: Bool(true), - - DisableHostNodeID: Bool(true), - } -} - -// DevConfig is used to return a set of configuration to use for dev mode. -func DevConfig() *Config { - conf := DefaultConfig() - conf.DevMode = true - conf.LogLevel = "DEBUG" - conf.Server = true - conf.EnableDebug = true - conf.DisableAnonymousSignature = true - conf.EnableUI = true - conf.BindAddr = "127.0.0.1" - conf.DisableKeyringFile = true - - conf.ConsulConfig = consul.DefaultConfig() - conf.ConsulConfig.SerfLANConfig.MemberlistConfig.ProbeTimeout = 100 * time.Millisecond - conf.ConsulConfig.SerfLANConfig.MemberlistConfig.ProbeInterval = 100 * time.Millisecond - conf.ConsulConfig.SerfLANConfig.MemberlistConfig.GossipInterval = 100 * time.Millisecond - - conf.ConsulConfig.SerfWANConfig.MemberlistConfig.SuspicionMult = 3 - conf.ConsulConfig.SerfWANConfig.MemberlistConfig.ProbeTimeout = 100 * time.Millisecond - conf.ConsulConfig.SerfWANConfig.MemberlistConfig.ProbeInterval = 100 * time.Millisecond - conf.ConsulConfig.SerfWANConfig.MemberlistConfig.GossipInterval = 100 * time.Millisecond - - conf.ConsulConfig.RaftConfig.LeaderLeaseTimeout = 20 * time.Millisecond - conf.ConsulConfig.RaftConfig.HeartbeatTimeout = 40 * time.Millisecond - conf.ConsulConfig.RaftConfig.ElectionTimeout = 40 * time.Millisecond - - conf.ConsulConfig.CoordinateUpdatePeriod = 100 * time.Millisecond - - return conf -} - -// EncryptBytes returns the encryption key configured. -func (c *Config) EncryptBytes() ([]byte, error) { - return base64.StdEncoding.DecodeString(c.EncryptKey) -} - -// ClientListener is used to format a listener for a -// port on a ClientAddr -func (c *Config) ClientListener(override string, port int) (net.Addr, error) { - addr := c.ClientAddr - if override != "" { - addr = override - } - if path := socketPath(addr); path != "" { - return &net.UnixAddr{Name: path, Net: "unix"}, nil - } - ip := net.ParseIP(addr) - if ip == nil { - return nil, fmt.Errorf("Failed to parse IP: %v", addr) - } - return &net.TCPAddr{IP: ip, Port: port}, nil -} - -// VerifyUniqueListeners checks to see if an address was used more than once in -// the config -func (c *Config) VerifyUniqueListeners() error { - listeners := []struct { - host string - port int - descr string - }{ - {c.Addresses.DNS, c.Ports.DNS, "DNS"}, - {c.Addresses.HTTP, c.Ports.HTTP, "HTTP"}, - {c.Addresses.HTTPS, c.Ports.HTTPS, "HTTPS"}, - {c.AdvertiseAddr, c.Ports.Server, "Server RPC"}, - {c.AdvertiseAddr, c.Ports.SerfLan, "Serf LAN"}, - {c.AdvertiseAddr, c.Ports.SerfWan, "Serf WAN"}, - } - - type key struct { - host string - port int - } - m := make(map[key]string, len(listeners)) - - for _, l := range listeners { - if l.host == "" { - l.host = "0.0.0.0" - } else if strings.HasPrefix(l.host, "unix") { - // Don't compare ports on unix sockets - l.port = 0 - } - if l.host == "0.0.0.0" && l.port <= 0 { - continue - } - - k := key{l.host, l.port} - v, ok := m[k] - if ok { - return fmt.Errorf("%s address already configured for %s", l.descr, v) - } - m[k] = l.descr - } - return nil -} - -// DecodeConfig reads the configuration from the given reader in JSON -// format and decodes it into a proper Config structure. -func DecodeConfig(r io.Reader) (*Config, error) { - var raw interface{} - if err := json.NewDecoder(r).Decode(&raw); err != nil { - return nil, err - } - - // Check the result type - var result Config - if obj, ok := raw.(map[string]interface{}); ok { - // Check for a "services", "service" or "check" key, meaning - // this is actually a definition entry - if sub, ok := obj["services"]; ok { - if list, ok := sub.([]interface{}); ok { - for _, srv := range list { - service, err := DecodeServiceDefinition(srv) - if err != nil { - return nil, err - } - result.Services = append(result.Services, service) - } - } - } - if sub, ok := obj["service"]; ok { - service, err := DecodeServiceDefinition(sub) - if err != nil { - return nil, err - } - result.Services = append(result.Services, service) - } - if sub, ok := obj["checks"]; ok { - if list, ok := sub.([]interface{}); ok { - for _, chk := range list { - check, err := DecodeCheckDefinition(chk) - if err != nil { - return nil, err - } - result.Checks = append(result.Checks, check) - } - } - } - if sub, ok := obj["check"]; ok { - check, err := DecodeCheckDefinition(sub) - if err != nil { - return nil, err - } - result.Checks = append(result.Checks, check) - } - - // A little hacky but upgrades the old stats config directives to the new way - if sub, ok := obj["statsd_addr"]; ok && result.Telemetry.StatsdAddr == "" { - result.Telemetry.StatsdAddr = sub.(string) - } - - if sub, ok := obj["statsite_addr"]; ok && result.Telemetry.StatsiteAddr == "" { - result.Telemetry.StatsiteAddr = sub.(string) - } - - if sub, ok := obj["statsite_prefix"]; ok && result.Telemetry.StatsitePrefix == "" { - result.Telemetry.StatsitePrefix = sub.(string) - } - - if sub, ok := obj["dogstatsd_addr"]; ok && result.Telemetry.DogStatsdAddr == "" { - result.Telemetry.DogStatsdAddr = sub.(string) - } - - if sub, ok := obj["dogstatsd_tags"].([]interface{}); ok && len(result.Telemetry.DogStatsdTags) == 0 { - result.Telemetry.DogStatsdTags = make([]string, len(sub)) - for i := range sub { - result.Telemetry.DogStatsdTags[i] = sub[i].(string) - } - } - } - - // Decode - var md mapstructure.Metadata - msdec, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{ - Metadata: &md, - Result: &result, - }) - if err != nil { - return nil, err - } - - if err := msdec.Decode(raw); err != nil { - return nil, err - } - - // Check for deprecations - if result.Ports.RPC != 0 { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: ports.rpc is deprecated and is "+ - "no longer used. Please remove it from your configuration.") - } - if result.Addresses.RPC != "" { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: addresses.rpc is deprecated and "+ - "is no longer used. Please remove it from your configuration.") - } - if result.DeprecatedAtlasInfrastructure != "" { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: atlas_infrastructure is deprecated and "+ - "is no longer used. Please remove it from your configuration.") - } - if result.DeprecatedAtlasToken != "" { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: atlas_token is deprecated and "+ - "is no longer used. Please remove it from your configuration.") - } - if result.DeprecatedAtlasACLToken != "" { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: atlas_acl_token is deprecated and "+ - "is no longer used. Please remove it from your configuration.") - } - if result.DeprecatedAtlasJoin != false { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: atlas_join is deprecated and "+ - "is no longer used. Please remove it from your configuration.") - } - if result.DeprecatedAtlasEndpoint != "" { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: atlas_endpoint is deprecated and "+ - "is no longer used. Please remove it from your configuration.") - } - - // Check unused fields and verify that no bad configuration options were - // passed to Consul. There are a few additional fields which don't directly - // use mapstructure decoding, so we need to account for those as well. These - // telemetry-related fields used to be available as top-level keys, so they - // are here for backward compatibility with the old format. - allowedKeys := []string{ - "service", "services", "check", "checks", "statsd_addr", "statsite_addr", "statsite_prefix", - "dogstatsd_addr", "dogstatsd_tags", - } - - var unused []string - for _, field := range md.Unused { - if !lib.StrContains(allowedKeys, field) { - unused = append(unused, field) - } - } - if len(unused) > 0 { - return nil, fmt.Errorf("Config has invalid keys: %s", strings.Join(unused, ",")) - } - - // Handle time conversions - if raw := result.DNSConfig.NodeTTLRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("NodeTTL invalid: %v", err) - } - result.DNSConfig.NodeTTL = dur - } - - if raw := result.DNSConfig.MaxStaleRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("MaxStale invalid: %v", err) - } - result.DNSConfig.MaxStale = dur - } - - if raw := result.DNSConfig.RecursorTimeoutRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("RecursorTimeout invalid: %v", err) - } - result.DNSConfig.RecursorTimeout = dur - } - - if len(result.DNSConfig.ServiceTTLRaw) != 0 { - if result.DNSConfig.ServiceTTL == nil { - result.DNSConfig.ServiceTTL = make(map[string]time.Duration) - } - for service, raw := range result.DNSConfig.ServiceTTLRaw { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("ServiceTTL %s invalid: %v", service, err) - } - result.DNSConfig.ServiceTTL[service] = dur - } - } - - if raw := result.CheckUpdateIntervalRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("CheckUpdateInterval invalid: %v", err) - } - result.CheckUpdateInterval = dur - } - - if raw := result.ACLTTLRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("ACL TTL invalid: %v", err) - } - result.ACLTTL = dur - } - - if raw := result.RetryIntervalRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("RetryInterval invalid: %v", err) - } - result.RetryInterval = dur - } - - if raw := result.RetryIntervalWanRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("RetryIntervalWan invalid: %v", err) - } - result.RetryIntervalWan = dur - } - - const reconnectTimeoutMin = 8 * time.Hour - if raw := result.ReconnectTimeoutLanRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("ReconnectTimeoutLan invalid: %v", err) - } - if dur < reconnectTimeoutMin { - return nil, fmt.Errorf("ReconnectTimeoutLan must be >= %s", reconnectTimeoutMin.String()) - } - result.ReconnectTimeoutLan = dur - } - if raw := result.ReconnectTimeoutWanRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("ReconnectTimeoutWan invalid: %v", err) - } - if dur < reconnectTimeoutMin { - return nil, fmt.Errorf("ReconnectTimeoutWan must be >= %s", reconnectTimeoutMin.String()) - } - result.ReconnectTimeoutWan = dur - } - - if raw := result.Autopilot.LastContactThresholdRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("LastContactThreshold invalid: %v", err) - } - result.Autopilot.LastContactThreshold = &dur - } - if raw := result.Autopilot.ServerStabilizationTimeRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("ServerStabilizationTime invalid: %v", err) - } - result.Autopilot.ServerStabilizationTime = &dur - } - - // Merge the single recursor - if result.DNSRecursor != "" { - result.DNSRecursors = append(result.DNSRecursors, result.DNSRecursor) - } - - if raw := result.SessionTTLMinRaw; raw != "" { - dur, err := time.ParseDuration(raw) - if err != nil { - return nil, fmt.Errorf("Session TTL Min invalid: %v", err) - } - result.SessionTTLMin = dur - } - - if result.AdvertiseAddrs.SerfLanRaw != "" { - ipStr, err := parseSingleIPTemplate(result.AdvertiseAddrs.SerfLanRaw) - if err != nil { - return nil, fmt.Errorf("Serf Advertise LAN address resolution failed: %v", err) - } - result.AdvertiseAddrs.SerfLanRaw = ipStr - - addr, err := net.ResolveTCPAddr("tcp", result.AdvertiseAddrs.SerfLanRaw) - if err != nil { - return nil, fmt.Errorf("AdvertiseAddrs.SerfLan is invalid: %v", err) - } - result.AdvertiseAddrs.SerfLan = addr - } - - if result.AdvertiseAddrs.SerfWanRaw != "" { - ipStr, err := parseSingleIPTemplate(result.AdvertiseAddrs.SerfWanRaw) - if err != nil { - return nil, fmt.Errorf("Serf Advertise WAN address resolution failed: %v", err) - } - result.AdvertiseAddrs.SerfWanRaw = ipStr - - addr, err := net.ResolveTCPAddr("tcp", result.AdvertiseAddrs.SerfWanRaw) - if err != nil { - return nil, fmt.Errorf("AdvertiseAddrs.SerfWan is invalid: %v", err) - } - result.AdvertiseAddrs.SerfWan = addr - } - - if result.AdvertiseAddrs.RPCRaw != "" { - ipStr, err := parseSingleIPTemplate(result.AdvertiseAddrs.RPCRaw) - if err != nil { - return nil, fmt.Errorf("RPC Advertise address resolution failed: %v", err) - } - result.AdvertiseAddrs.RPCRaw = ipStr - - addr, err := net.ResolveTCPAddr("tcp", result.AdvertiseAddrs.RPCRaw) - if err != nil { - return nil, fmt.Errorf("AdvertiseAddrs.RPC is invalid: %v", err) - } - result.AdvertiseAddrs.RPC = addr - } - - // Validate segment config. - if err := ValidateSegments(&result); err != nil { - return nil, err - } - - // Enforce the max Raft multiplier. - if result.Performance.RaftMultiplier > consul.MaxRaftMultiplier { - return nil, fmt.Errorf("Performance.RaftMultiplier must be <= %d", consul.MaxRaftMultiplier) - } - - if raw := result.TLSCipherSuitesRaw; raw != "" { - ciphers, err := tlsutil.ParseCiphers(raw) - if err != nil { - return nil, fmt.Errorf("TLSCipherSuites invalid: %v", err) - } - result.TLSCipherSuites = ciphers - } - - // This is for backwards compatibility. - // HTTPAPIResponseHeaders has been replaced with HTTPConfig.ResponseHeaders - if len(result.DeprecatedHTTPAPIResponseHeaders) > 0 { - fmt.Fprintln(os.Stderr, "==> DEPRECATION: http_api_response_headers is deprecated and "+ - "is no longer used. Please use http_config.response_headers instead.") - if result.HTTPConfig.ResponseHeaders == nil { - result.HTTPConfig.ResponseHeaders = make(map[string]string) - } - for field, value := range result.DeprecatedHTTPAPIResponseHeaders { - result.HTTPConfig.ResponseHeaders[field] = value - } - result.DeprecatedHTTPAPIResponseHeaders = nil - } - - // Set the ACL replication enable if they set a token, for backwards - // compatibility. - if result.ACLReplicationToken != "" { - result.EnableACLReplication = true - } - - // Parse the metric filters - for _, rule := range result.Telemetry.PrefixFilter { - if rule == "" { - return nil, fmt.Errorf("Cannot have empty filter rule in prefix_filter") - } - switch rule[0] { - case '+': - result.Telemetry.AllowedPrefixes = append(result.Telemetry.AllowedPrefixes, rule[1:]) - case '-': - result.Telemetry.BlockedPrefixes = append(result.Telemetry.BlockedPrefixes, rule[1:]) - default: - return nil, fmt.Errorf("Filter rule must begin with either '+' or '-': %q", rule) - } - } - - // Validate node meta fields - if err := structs.ValidateMetadata(result.Meta, false); err != nil { - return nil, fmt.Errorf("Failed to parse node metadata: %v", err) - } - - return &result, nil -} - -// DecodeServiceDefinition is used to decode a service definition -func DecodeServiceDefinition(raw interface{}) (*structs.ServiceDefinition, error) { - rawMap, ok := raw.(map[string]interface{}) - if !ok { - goto AFTER_FIX - } - - // If no 'tags', handle the deprecated 'tag' value. - if _, ok := rawMap["tags"]; !ok { - if tag, ok := rawMap["tag"]; ok { - rawMap["tags"] = []interface{}{tag} - } - } - - for k, v := range rawMap { - switch strings.ToLower(k) { - case "check": - if err := FixupCheckType(v); err != nil { - return nil, err - } - case "checks": - chkTypes, ok := v.([]interface{}) - if !ok { - goto AFTER_FIX - } - for _, chkType := range chkTypes { - if err := FixupCheckType(chkType); err != nil { - return nil, err - } - } - } - } -AFTER_FIX: - var md mapstructure.Metadata - var result structs.ServiceDefinition - msdec, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{ - Metadata: &md, - Result: &result, - }) - if err != nil { - return nil, err - } - if err := msdec.Decode(raw); err != nil { - return nil, err - } - return &result, nil -} - var errInvalidHeaderFormat = errors.New("agent: invalid format of 'header' field") func FixupCheckType(raw interface{}) error { @@ -1636,728 +99,6 @@ func FixupCheckType(raw interface{}) error { return nil } -// DecodeCheckDefinition is used to decode a check definition -func DecodeCheckDefinition(raw interface{}) (*structs.CheckDefinition, error) { - if err := FixupCheckType(raw); err != nil { - return nil, err - } - var md mapstructure.Metadata - var result structs.CheckDefinition - msdec, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{ - Metadata: &md, - Result: &result, - }) - if err != nil { - return nil, err - } - if err := msdec.Decode(raw); err != nil { - return nil, err - } - return &result, nil -} - -// MergeConfig merges two configurations together to make a single new -// configuration. -func MergeConfig(a, b *Config) *Config { - var result Config = *a - - if b.Limits.RPCRate > 0 { - result.Limits.RPCRate = b.Limits.RPCRate - } - if b.Limits.RPCMaxBurst > 0 { - result.Limits.RPCMaxBurst = b.Limits.RPCMaxBurst - } - - // Propagate non-default performance settings - if b.Performance.RaftMultiplier > 0 { - result.Performance.RaftMultiplier = b.Performance.RaftMultiplier - } - - // Copy the strings if they're set - if b.Bootstrap { - result.Bootstrap = true - } - if b.BootstrapExpect != 0 { - result.BootstrapExpect = b.BootstrapExpect - } - if b.Datacenter != "" { - result.Datacenter = b.Datacenter - } - if b.DataDir != "" { - result.DataDir = b.DataDir - } - - // Copy the dns recursors - result.DNSRecursors = make([]string, 0, len(a.DNSRecursors)+len(b.DNSRecursors)) - result.DNSRecursors = append(result.DNSRecursors, a.DNSRecursors...) - result.DNSRecursors = append(result.DNSRecursors, b.DNSRecursors...) - - if b.Domain != "" { - result.Domain = b.Domain - } - if b.EncryptKey != "" { - result.EncryptKey = b.EncryptKey - } - if b.DisableKeyringFile { - result.DisableKeyringFile = true - } - if b.EncryptVerifyIncoming != nil { - result.EncryptVerifyIncoming = b.EncryptVerifyIncoming - } - if b.EncryptVerifyOutgoing != nil { - result.EncryptVerifyOutgoing = b.EncryptVerifyOutgoing - } - if b.LogLevel != "" { - result.LogLevel = b.LogLevel - } - if b.Protocol > 0 { - result.Protocol = b.Protocol - } - if b.RaftProtocol > 0 { - result.RaftProtocol = b.RaftProtocol - } - if b.NodeID != "" { - result.NodeID = b.NodeID - } - if b.DisableHostNodeID != nil { - result.DisableHostNodeID = b.DisableHostNodeID - } - if b.NodeName != "" { - result.NodeName = b.NodeName - } - if b.ClientAddr != "" { - result.ClientAddr = b.ClientAddr - } - if b.BindAddr != "" { - result.BindAddr = b.BindAddr - } - if b.AdvertiseAddr != "" { - result.AdvertiseAddr = b.AdvertiseAddr - } - if b.AdvertiseAddrWan != "" { - result.AdvertiseAddrWan = b.AdvertiseAddrWan - } - if b.SerfWanBindAddr != "" { - result.SerfWanBindAddr = b.SerfWanBindAddr - } - if b.SerfLanBindAddr != "" { - result.SerfLanBindAddr = b.SerfLanBindAddr - } - if b.TranslateWanAddrs == true { - result.TranslateWanAddrs = true - } - if b.AdvertiseAddrs.SerfLan != nil { - result.AdvertiseAddrs.SerfLan = b.AdvertiseAddrs.SerfLan - result.AdvertiseAddrs.SerfLanRaw = b.AdvertiseAddrs.SerfLanRaw - } - if b.AdvertiseAddrs.SerfWan != nil { - result.AdvertiseAddrs.SerfWan = b.AdvertiseAddrs.SerfWan - result.AdvertiseAddrs.SerfWanRaw = b.AdvertiseAddrs.SerfWanRaw - } - if b.AdvertiseAddrs.RPC != nil { - result.AdvertiseAddrs.RPC = b.AdvertiseAddrs.RPC - result.AdvertiseAddrs.RPCRaw = b.AdvertiseAddrs.RPCRaw - } - if b.Server == true { - result.Server = b.Server - } - if b.NonVotingServer == true { - result.NonVotingServer = b.NonVotingServer - } - if b.LeaveOnTerm != nil { - result.LeaveOnTerm = b.LeaveOnTerm - } - if b.SkipLeaveOnInt != nil { - result.SkipLeaveOnInt = b.SkipLeaveOnInt - } - if b.Autopilot.CleanupDeadServers != nil { - result.Autopilot.CleanupDeadServers = b.Autopilot.CleanupDeadServers - } - if b.Autopilot.LastContactThreshold != nil { - result.Autopilot.LastContactThreshold = b.Autopilot.LastContactThreshold - } - if b.Autopilot.MaxTrailingLogs != nil { - result.Autopilot.MaxTrailingLogs = b.Autopilot.MaxTrailingLogs - } - if b.Autopilot.ServerStabilizationTime != nil { - result.Autopilot.ServerStabilizationTime = b.Autopilot.ServerStabilizationTime - } - if b.Autopilot.RedundancyZoneTag != "" { - result.Autopilot.RedundancyZoneTag = b.Autopilot.RedundancyZoneTag - } - if b.Autopilot.DisableUpgradeMigration != nil { - result.Autopilot.DisableUpgradeMigration = b.Autopilot.DisableUpgradeMigration - } - if b.Autopilot.UpgradeVersionTag != "" { - result.Autopilot.UpgradeVersionTag = b.Autopilot.UpgradeVersionTag - } - if b.Telemetry.DisableHostname == true { - result.Telemetry.DisableHostname = true - } - if len(b.Telemetry.PrefixFilter) != 0 { - result.Telemetry.PrefixFilter = append(result.Telemetry.PrefixFilter, b.Telemetry.PrefixFilter...) - } - if b.Telemetry.FilterDefault != nil { - result.Telemetry.FilterDefault = b.Telemetry.FilterDefault - } - if b.Telemetry.StatsdAddr != "" { - result.Telemetry.StatsdAddr = b.Telemetry.StatsdAddr - } - if b.Telemetry.StatsiteAddr != "" { - result.Telemetry.StatsiteAddr = b.Telemetry.StatsiteAddr - } - if b.Telemetry.StatsitePrefix != "" { - result.Telemetry.StatsitePrefix = b.Telemetry.StatsitePrefix - } - if b.Telemetry.DogStatsdAddr != "" { - result.Telemetry.DogStatsdAddr = b.Telemetry.DogStatsdAddr - } - if b.Telemetry.DogStatsdTags != nil { - result.Telemetry.DogStatsdTags = b.Telemetry.DogStatsdTags - } - if b.Telemetry.CirconusAPIToken != "" { - result.Telemetry.CirconusAPIToken = b.Telemetry.CirconusAPIToken - } - if b.Telemetry.CirconusAPIApp != "" { - result.Telemetry.CirconusAPIApp = b.Telemetry.CirconusAPIApp - } - if b.Telemetry.CirconusAPIURL != "" { - result.Telemetry.CirconusAPIURL = b.Telemetry.CirconusAPIURL - } - if b.Telemetry.CirconusCheckSubmissionURL != "" { - result.Telemetry.CirconusCheckSubmissionURL = b.Telemetry.CirconusCheckSubmissionURL - } - if b.Telemetry.CirconusSubmissionInterval != "" { - result.Telemetry.CirconusSubmissionInterval = b.Telemetry.CirconusSubmissionInterval - } - if b.Telemetry.CirconusCheckID != "" { - result.Telemetry.CirconusCheckID = b.Telemetry.CirconusCheckID - } - if b.Telemetry.CirconusCheckForceMetricActivation != "" { - result.Telemetry.CirconusCheckForceMetricActivation = b.Telemetry.CirconusCheckForceMetricActivation - } - if b.Telemetry.CirconusCheckInstanceID != "" { - result.Telemetry.CirconusCheckInstanceID = b.Telemetry.CirconusCheckInstanceID - } - if b.Telemetry.CirconusCheckSearchTag != "" { - result.Telemetry.CirconusCheckSearchTag = b.Telemetry.CirconusCheckSearchTag - } - if b.Telemetry.CirconusCheckDisplayName != "" { - result.Telemetry.CirconusCheckDisplayName = b.Telemetry.CirconusCheckDisplayName - } - if b.Telemetry.CirconusCheckTags != "" { - result.Telemetry.CirconusCheckTags = b.Telemetry.CirconusCheckTags - } - if b.Telemetry.CirconusBrokerID != "" { - result.Telemetry.CirconusBrokerID = b.Telemetry.CirconusBrokerID - } - if b.Telemetry.CirconusBrokerSelectTag != "" { - result.Telemetry.CirconusBrokerSelectTag = b.Telemetry.CirconusBrokerSelectTag - } - if b.EnableDebug { - result.EnableDebug = true - } - if b.VerifyIncoming { - result.VerifyIncoming = true - } - if b.VerifyIncomingRPC { - result.VerifyIncomingRPC = true - } - if b.VerifyIncomingHTTPS { - result.VerifyIncomingHTTPS = true - } - if b.VerifyOutgoing { - result.VerifyOutgoing = true - } - if b.VerifyServerHostname { - result.VerifyServerHostname = true - } - if b.CAFile != "" { - result.CAFile = b.CAFile - } - if b.CAPath != "" { - result.CAPath = b.CAPath - } - if b.CertFile != "" { - result.CertFile = b.CertFile - } - if b.KeyFile != "" { - result.KeyFile = b.KeyFile - } - if b.ServerName != "" { - result.ServerName = b.ServerName - } - if b.TLSMinVersion != "" { - result.TLSMinVersion = b.TLSMinVersion - } - if len(b.TLSCipherSuites) != 0 { - result.TLSCipherSuites = append(result.TLSCipherSuites, b.TLSCipherSuites...) - } - if b.TLSPreferServerCipherSuites { - result.TLSPreferServerCipherSuites = true - } - if b.Checks != nil { - result.Checks = append(result.Checks, b.Checks...) - } - if b.Services != nil { - result.Services = append(result.Services, b.Services...) - } - if b.Ports.DNS != 0 { - result.Ports.DNS = b.Ports.DNS - } - if b.Ports.HTTP != 0 { - result.Ports.HTTP = b.Ports.HTTP - } - if b.Ports.HTTPS != 0 { - result.Ports.HTTPS = b.Ports.HTTPS - } - if b.Ports.RPC != 0 { - result.Ports.RPC = b.Ports.RPC - } - if b.Ports.SerfLan != 0 { - result.Ports.SerfLan = b.Ports.SerfLan - } - if b.Ports.SerfWan != 0 { - result.Ports.SerfWan = b.Ports.SerfWan - } - if b.Ports.Server != 0 { - result.Ports.Server = b.Ports.Server - } - if b.Addresses.DNS != "" { - result.Addresses.DNS = b.Addresses.DNS - } - if b.Addresses.HTTP != "" { - result.Addresses.HTTP = b.Addresses.HTTP - } - if b.Addresses.HTTPS != "" { - result.Addresses.HTTPS = b.Addresses.HTTPS - } - if b.Addresses.RPC != "" { - result.Addresses.RPC = b.Addresses.RPC - } - if b.Segment != "" { - result.Segment = b.Segment - } - if len(b.Segments) > 0 { - result.Segments = append(result.Segments, b.Segments...) - } - if b.EnableUI { - result.EnableUI = true - } - if b.UIDir != "" { - result.UIDir = b.UIDir - } - if b.PidFile != "" { - result.PidFile = b.PidFile - } - if b.EnableSyslog { - result.EnableSyslog = true - } - if b.RejoinAfterLeave { - result.RejoinAfterLeave = true - } - if b.RetryMaxAttempts != 0 { - result.RetryMaxAttempts = b.RetryMaxAttempts - } - if b.RetryInterval != 0 { - result.RetryInterval = b.RetryInterval - } - if b.DeprecatedRetryJoinEC2.AccessKeyID != "" { - result.DeprecatedRetryJoinEC2.AccessKeyID = b.DeprecatedRetryJoinEC2.AccessKeyID - } - if b.DeprecatedRetryJoinEC2.SecretAccessKey != "" { - result.DeprecatedRetryJoinEC2.SecretAccessKey = b.DeprecatedRetryJoinEC2.SecretAccessKey - } - if b.DeprecatedRetryJoinEC2.Region != "" { - result.DeprecatedRetryJoinEC2.Region = b.DeprecatedRetryJoinEC2.Region - } - if b.DeprecatedRetryJoinEC2.TagKey != "" { - result.DeprecatedRetryJoinEC2.TagKey = b.DeprecatedRetryJoinEC2.TagKey - } - if b.DeprecatedRetryJoinEC2.TagValue != "" { - result.DeprecatedRetryJoinEC2.TagValue = b.DeprecatedRetryJoinEC2.TagValue - } - if b.DeprecatedRetryJoinGCE.ProjectName != "" { - result.DeprecatedRetryJoinGCE.ProjectName = b.DeprecatedRetryJoinGCE.ProjectName - } - if b.DeprecatedRetryJoinGCE.ZonePattern != "" { - result.DeprecatedRetryJoinGCE.ZonePattern = b.DeprecatedRetryJoinGCE.ZonePattern - } - if b.DeprecatedRetryJoinGCE.TagValue != "" { - result.DeprecatedRetryJoinGCE.TagValue = b.DeprecatedRetryJoinGCE.TagValue - } - if b.DeprecatedRetryJoinGCE.CredentialsFile != "" { - result.DeprecatedRetryJoinGCE.CredentialsFile = b.DeprecatedRetryJoinGCE.CredentialsFile - } - if b.DeprecatedRetryJoinAzure.TagName != "" { - result.DeprecatedRetryJoinAzure.TagName = b.DeprecatedRetryJoinAzure.TagName - } - if b.DeprecatedRetryJoinAzure.TagValue != "" { - result.DeprecatedRetryJoinAzure.TagValue = b.DeprecatedRetryJoinAzure.TagValue - } - if b.DeprecatedRetryJoinAzure.SubscriptionID != "" { - result.DeprecatedRetryJoinAzure.SubscriptionID = b.DeprecatedRetryJoinAzure.SubscriptionID - } - if b.DeprecatedRetryJoinAzure.TenantID != "" { - result.DeprecatedRetryJoinAzure.TenantID = b.DeprecatedRetryJoinAzure.TenantID - } - if b.DeprecatedRetryJoinAzure.ClientID != "" { - result.DeprecatedRetryJoinAzure.ClientID = b.DeprecatedRetryJoinAzure.ClientID - } - if b.DeprecatedRetryJoinAzure.SecretAccessKey != "" { - result.DeprecatedRetryJoinAzure.SecretAccessKey = b.DeprecatedRetryJoinAzure.SecretAccessKey - } - if b.RetryMaxAttemptsWan != 0 { - result.RetryMaxAttemptsWan = b.RetryMaxAttemptsWan - } - if b.RetryIntervalWan != 0 { - result.RetryIntervalWan = b.RetryIntervalWan - } - if b.ReconnectTimeoutLan != 0 { - result.ReconnectTimeoutLan = b.ReconnectTimeoutLan - result.ReconnectTimeoutLanRaw = b.ReconnectTimeoutLanRaw - } - if b.ReconnectTimeoutWan != 0 { - result.ReconnectTimeoutWan = b.ReconnectTimeoutWan - result.ReconnectTimeoutWanRaw = b.ReconnectTimeoutWanRaw - } - if b.DNSConfig.NodeTTL != 0 { - result.DNSConfig.NodeTTL = b.DNSConfig.NodeTTL - } - if len(b.DNSConfig.ServiceTTL) != 0 { - if result.DNSConfig.ServiceTTL == nil { - result.DNSConfig.ServiceTTL = make(map[string]time.Duration) - } - for service, dur := range b.DNSConfig.ServiceTTL { - result.DNSConfig.ServiceTTL[service] = dur - } - } - if b.DNSConfig.AllowStale != nil { - result.DNSConfig.AllowStale = b.DNSConfig.AllowStale - } - if b.DNSConfig.UDPAnswerLimit != 0 { - result.DNSConfig.UDPAnswerLimit = b.DNSConfig.UDPAnswerLimit - } - if b.DNSConfig.EnableTruncate { - result.DNSConfig.EnableTruncate = true - } - if b.DNSConfig.MaxStale != 0 { - result.DNSConfig.MaxStale = b.DNSConfig.MaxStale - } - if b.DNSConfig.OnlyPassing { - result.DNSConfig.OnlyPassing = true - } - if b.DNSConfig.DisableCompression { - result.DNSConfig.DisableCompression = true - } - if b.DNSConfig.RecursorTimeout != 0 { - result.DNSConfig.RecursorTimeout = b.DNSConfig.RecursorTimeout - } - if b.EnableScriptChecks { - result.EnableScriptChecks = true - } - if b.CheckUpdateIntervalRaw != "" || b.CheckUpdateInterval != 0 { - result.CheckUpdateInterval = b.CheckUpdateInterval - } - if b.SyslogFacility != "" { - result.SyslogFacility = b.SyslogFacility - } - if b.ACLToken != "" { - result.ACLToken = b.ACLToken - } - if b.ACLAgentMasterToken != "" { - result.ACLAgentMasterToken = b.ACLAgentMasterToken - } - if b.ACLAgentToken != "" { - result.ACLAgentToken = b.ACLAgentToken - } - if b.ACLMasterToken != "" { - result.ACLMasterToken = b.ACLMasterToken - } - if b.ACLDatacenter != "" { - result.ACLDatacenter = b.ACLDatacenter - } - if b.ACLTTLRaw != "" { - result.ACLTTL = b.ACLTTL - result.ACLTTLRaw = b.ACLTTLRaw - } - if b.ACLDownPolicy != "" { - result.ACLDownPolicy = b.ACLDownPolicy - } - if b.ACLDefaultPolicy != "" { - result.ACLDefaultPolicy = b.ACLDefaultPolicy - } - if b.EnableACLReplication { - result.EnableACLReplication = true - } - if b.ACLReplicationToken != "" { - result.ACLReplicationToken = b.ACLReplicationToken - } - if b.ACLEnforceVersion8 != nil { - result.ACLEnforceVersion8 = b.ACLEnforceVersion8 - } - if len(b.Watches) != 0 { - result.Watches = append(result.Watches, b.Watches...) - } - if len(b.WatchPlans) != 0 { - result.WatchPlans = append(result.WatchPlans, b.WatchPlans...) - } - if b.DisableRemoteExec != nil { - result.DisableRemoteExec = b.DisableRemoteExec - } - if b.DisableUpdateCheck { - result.DisableUpdateCheck = true - } - if b.DisableAnonymousSignature { - result.DisableAnonymousSignature = true - } - if b.UnixSockets.Usr != "" { - result.UnixSockets.Usr = b.UnixSockets.Usr - } - if b.UnixSockets.Grp != "" { - result.UnixSockets.Grp = b.UnixSockets.Grp - } - if b.UnixSockets.Perms != "" { - result.UnixSockets.Perms = b.UnixSockets.Perms - } - if b.DisableCoordinates { - result.DisableCoordinates = true - } - if b.SessionTTLMinRaw != "" { - result.SessionTTLMin = b.SessionTTLMin - result.SessionTTLMinRaw = b.SessionTTLMinRaw - } - - result.HTTPConfig.BlockEndpoints = append(a.HTTPConfig.BlockEndpoints, - b.HTTPConfig.BlockEndpoints...) - if len(b.HTTPConfig.ResponseHeaders) > 0 { - if result.HTTPConfig.ResponseHeaders == nil { - result.HTTPConfig.ResponseHeaders = make(map[string]string) - } - for field, value := range b.HTTPConfig.ResponseHeaders { - result.HTTPConfig.ResponseHeaders[field] = value - } - } - - if len(b.Meta) != 0 { - if result.Meta == nil { - result.Meta = make(map[string]string) - } - for field, value := range b.Meta { - result.Meta[field] = value - } - } - - // Copy the start join addresses - result.StartJoin = make([]string, 0, len(a.StartJoin)+len(b.StartJoin)) - result.StartJoin = append(result.StartJoin, a.StartJoin...) - result.StartJoin = append(result.StartJoin, b.StartJoin...) - - // Copy the start join addresses - result.StartJoinWan = make([]string, 0, len(a.StartJoinWan)+len(b.StartJoinWan)) - result.StartJoinWan = append(result.StartJoinWan, a.StartJoinWan...) - result.StartJoinWan = append(result.StartJoinWan, b.StartJoinWan...) - - // Copy the retry join addresses - result.RetryJoin = make([]string, 0, len(a.RetryJoin)+len(b.RetryJoin)) - result.RetryJoin = append(result.RetryJoin, a.RetryJoin...) - result.RetryJoin = append(result.RetryJoin, b.RetryJoin...) - - // Copy the retry join -wan addresses - result.RetryJoinWan = make([]string, 0, len(a.RetryJoinWan)+len(b.RetryJoinWan)) - result.RetryJoinWan = append(result.RetryJoinWan, a.RetryJoinWan...) - result.RetryJoinWan = append(result.RetryJoinWan, b.RetryJoinWan...) - - return &result -} - -// ReadConfigPaths reads the paths in the given order to load configurations. -// The paths can be to files or directories. If the path is a directory, -// we read one directory deep and read any files ending in ".json" as -// configuration files. -func ReadConfigPaths(paths []string) (*Config, error) { - result := new(Config) - for _, path := range paths { - f, err := os.Open(path) - if err != nil { - return nil, fmt.Errorf("Error reading '%s': %s", path, err) - } - - fi, err := f.Stat() - if err != nil { - f.Close() - return nil, fmt.Errorf("Error reading '%s': %s", path, err) - } - - if !fi.IsDir() { - config, err := DecodeConfig(f) - f.Close() - - if err != nil { - return nil, fmt.Errorf("Error decoding '%s': %s", path, err) - } - - result = MergeConfig(result, config) - continue - } - - contents, err := f.Readdir(-1) - f.Close() - if err != nil { - return nil, fmt.Errorf("Error reading '%s': %s", path, err) - } - - // Sort the contents, ensures lexical order - sort.Sort(dirEnts(contents)) - - for _, fi := range contents { - // Don't recursively read contents - if fi.IsDir() { - continue - } - - // If it isn't a JSON file, ignore it - if !strings.HasSuffix(fi.Name(), ".json") { - continue - } - // If the config file is empty, ignore it - if fi.Size() == 0 { - continue - } - - subpath := filepath.Join(path, fi.Name()) - f, err := os.Open(subpath) - if err != nil { - return nil, fmt.Errorf("Error reading '%s': %s", subpath, err) - } - - config, err := DecodeConfig(f) - f.Close() - - if err != nil { - return nil, fmt.Errorf("Error decoding '%s': %s", subpath, err) - } - - result = MergeConfig(result, config) - } - } - - return result, nil -} - -// ResolveTmplAddrs iterates over the myriad of addresses in the agent's config -// and performs go-sockaddr/template Parse on each known address in case the -// user specified a template config for any of their values. -func (c *Config) ResolveTmplAddrs() (err error) { - parse := func(addr *string, socketAllowed bool, name string) { - if *addr == "" || err != nil { - return - } - var ip string - ip, err = parseSingleIPTemplate(*addr) - if err != nil { - err = fmt.Errorf("Resolution of %s failed: %v", name, err) - return - } - ipAddr := net.ParseIP(ip) - if !socketAllowed && ipAddr == nil { - err = fmt.Errorf("Failed to parse %s: %v", name, ip) - return - } - if socketAllowed && socketPath(ip) == "" && ipAddr == nil { - err = fmt.Errorf("Failed to parse %s, %q is not a valid IP address or socket", name, ip) - return - } - - *addr = ip - } - - if c == nil { - return - } - parse(&c.Addresses.DNS, true, "DNS address") - parse(&c.Addresses.HTTP, true, "HTTP address") - parse(&c.Addresses.HTTPS, true, "HTTPS address") - parse(&c.AdvertiseAddr, false, "Advertise address") - parse(&c.AdvertiseAddrWan, false, "Advertise WAN address") - parse(&c.BindAddr, true, "Bind address") - parse(&c.ClientAddr, true, "Client address") - parse(&c.SerfLanBindAddr, false, "Serf LAN address") - parse(&c.SerfWanBindAddr, false, "Serf WAN address") - for i, segment := range c.Segments { - parse(&c.Segments[i].Bind, false, fmt.Sprintf("Segment %q bind address", segment.Name)) - parse(&c.Segments[i].Advertise, false, fmt.Sprintf("Segment %q advertise address", segment.Name)) - } - - return -} - -// SetupTaggedAndAdvertiseAddrs configures advertise addresses and sets up a map of tagged addresses -func (cfg *Config) SetupTaggedAndAdvertiseAddrs() error { - if cfg.AdvertiseAddr == "" { - switch { - - case cfg.BindAddr != "" && !ipaddr.IsAny(cfg.BindAddr): - cfg.AdvertiseAddr = cfg.BindAddr - - default: - ip, err := consul.GetPrivateIP() - if ipaddr.IsAnyV6(cfg.BindAddr) { - ip, err = consul.GetPublicIPv6() - } - if err != nil { - return fmt.Errorf("Failed to get advertise address: %v", err) - } - cfg.AdvertiseAddr = ip.String() - } - } - - // Try to get an advertise address for the wan - if cfg.AdvertiseAddrWan == "" { - cfg.AdvertiseAddrWan = cfg.AdvertiseAddr - } - - // Create the default set of tagged addresses. - cfg.TaggedAddresses = map[string]string{ - "lan": cfg.AdvertiseAddr, - "wan": cfg.AdvertiseAddrWan, - } - return nil -} - -// parseSingleIPTemplate is used as a helper function to parse out a single IP -// address from a config parameter. -func parseSingleIPTemplate(ipTmpl string) (string, error) { - out, err := template.Parse(ipTmpl) - if err != nil { - return "", fmt.Errorf("Unable to parse address template %q: %v", ipTmpl, err) - } - - ips := strings.Split(out, " ") - switch len(ips) { - case 0: - return "", errors.New("No addresses found, please configure one.") - case 1: - return ips[0], nil - default: - return "", fmt.Errorf("Multiple addresses found (%q), please configure one.", out) - } -} - -// Implement the sort interface for dirEnts -func (d dirEnts) Len() int { - return len(d) -} - -func (d dirEnts) Less(i, j int) bool { - return d[i].Name() < d[j].Name() -} - -func (d dirEnts) Swap(i, j int) { - d[i], d[j] = d[j], d[i] -} - -// ParseMetaPair parses a key/value pair of the form key:value func ParseMetaPair(raw string) (string, string) { pair := strings.SplitN(raw, ":", 2) if len(pair) == 2 { diff --git a/agent/config/builder.go b/agent/config/builder.go new file mode 100644 index 000000000..9f7c0e005 --- /dev/null +++ b/agent/config/builder.go @@ -0,0 +1,1174 @@ +package config + +import ( + "encoding/base64" + "encoding/json" + "fmt" + "io/ioutil" + "net" + "os" + "path/filepath" + "reflect" + "regexp" + "sort" + "strings" + "time" + + "github.com/hashicorp/consul/agent/consul" + "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/ipaddr" + "github.com/hashicorp/consul/tlsutil" + "github.com/hashicorp/consul/types" + multierror "github.com/hashicorp/go-multierror" + "github.com/hashicorp/go-sockaddr/template" + "golang.org/x/time/rate" +) + +// Builder constructs a valid runtime configuration from multiple +// configuration sources. +// +// To build the runtime configuration first call Build() which merges +// the sources in a pre-defined order, converts the data types and +// structures into their final form and performs the syntactic +// validation. +// +// The sources are merged in the following order: +// +// * default configuration +// * config files in alphabetical order +// * command line arguments +// +// The config sources are merged sequentially and later values +// overwrite previously set values. Slice values are merged by +// concatenating the two slices. +// +// Then call Validate() to perform the semantic validation to ensure +// that the configuration is ready to be used. +// +// Splitting the construction into two phases greatly simplifies testing +// since not all pre-conditions have to be satisfied when performing +// syntactical tests. +type Builder struct { + // Flags contains the parsed command line arguments. + Flags Flags + + // Head, Sources, and Tail are used to manage the order of the + // config sources, as described in the comments above. + Head []Source + Sources []Source + Tail []Source + + // Warnings contains the warnings encountered when + // parsing the configuration. + Warnings []string + + // Hostname returns the hostname of the machine. If nil, os.Hostname + // is called. + Hostname func() (string, error) + + // GetPrivateIPv4 and GetPublicIPv6 return suitable default addresses + // for cases when the user doesn't supply them. + GetPrivateIPv4 func() ([]*net.IPAddr, error) + GetPublicIPv6 func() ([]*net.IPAddr, error) + + // err contains the first error that occurred during + // building the runtime configuration. + err error +} + +// NewBuilder returns a new configuration builder based on the given command +// line flags. +func NewBuilder(flags Flags) (*Builder, error) { + newSource := func(name string, v interface{}) Source { + b, err := json.MarshalIndent(v, "", " ") + if err != nil { + panic(err) + } + return Source{Name: name, Format: "json", Data: string(b)} + } + + b := &Builder{ + Flags: flags, + Head: []Source{DefaultSource()}, + } + + if b.boolVal(b.Flags.DevMode) { + b.Head = append(b.Head, DevSource()) + } + + // Since the merge logic is to overwrite all fields with later + // values except slices which are merged by appending later values + // we need to merge all slice values defined in flags before we + // merge the config files since the flag values for slices are + // otherwise appended instead of prepended. + slices, values := b.splitSlicesAndValues(b.Flags.Config) + b.Head = append(b.Head, newSource("flags.slices", slices)) + for _, path := range b.Flags.ConfigFiles { + if err := b.ReadPath(path); err != nil { + return nil, err + } + } + b.Tail = append(b.Tail, newSource("flags.values", values)) + for i, s := range b.Flags.HCL { + b.Tail = append(b.Tail, Source{ + Name: fmt.Sprintf("flags.hcl.%d", i), + Format: "hcl", + Data: s, + }) + } + b.Tail = append(b.Tail, NonUserSource(), DefaultConsulSource(), DefaultVersionSource()) + if b.boolVal(b.Flags.DevMode) { + b.Tail = append(b.Tail, DevConsulSource()) + } + return b, nil +} + +// ReadPath reads a single config file or all files in a directory (but +// not its sub-directories) and appends them to the list of config +// sources. If path refers to a file then the format is assumed to be +// JSON unless the file has a '.hcl' suffix. If path refers to a +// directory then the format is determined by the suffix and only files +// with a '.json' or '.hcl' suffix are processed. +func (b *Builder) ReadPath(path string) error { + f, err := os.Open(path) + if err != nil { + return fmt.Errorf("config: Open failed on %s. %s", path, err) + } + defer f.Close() + + fi, err := f.Stat() + if err != nil { + return fmt.Errorf("config: Stat failed on %s. %s", path, err) + } + + if !fi.IsDir() { + return b.ReadFile(path) + } + + fis, err := f.Readdir(-1) + if err != nil { + return fmt.Errorf("config: Readdir failed on %s. %s", path, err) + } + + // sort files by name + sort.Sort(byName(fis)) + + for _, fi := range fis { + // do not recurse into sub dirs + if fi.IsDir() { + continue + } + + // skip files without json or hcl extension + if !strings.HasSuffix(fi.Name(), ".json") && !strings.HasSuffix(fi.Name(), ".hcl") { + continue + } + + if err := b.ReadFile(filepath.Join(path, fi.Name())); err != nil { + return err + } + } + return nil +} + +// ReadFile parses a JSON or HCL config file and appends it to the list of +// config sources. +func (b *Builder) ReadFile(path string) error { + data, err := ioutil.ReadFile(path) + if err != nil { + return fmt.Errorf("config: ReadFile failed on %s: %s", path, err) + } + b.Sources = append(b.Sources, NewSource(path, string(data))) + return nil +} + +type byName []os.FileInfo + +func (a byName) Len() int { return len(a) } +func (a byName) Swap(i, j int) { a[i], a[j] = a[j], a[i] } +func (a byName) Less(i, j int) bool { return a[i].Name() < a[j].Name() } + +func (b *Builder) BuildAndValidate() (RuntimeConfig, error) { + rt, err := b.Build() + if err != nil { + return RuntimeConfig{}, err + } + if err := b.Validate(rt); err != nil { + return RuntimeConfig{}, err + } + return rt, nil +} + +// Build constructs the runtime configuration from the config sources +// and the command line flags. The config sources are processed in the +// order they were added with the flags being processed last to give +// precedence over the other sources. If the error is nil then +// warnings can still contain deprecation or format warnings that should +// be presented to the user. +func (b *Builder) Build() (rt RuntimeConfig, err error) { + b.err = nil + b.Warnings = nil + + // ---------------------------------------------------------------- + // merge config sources as follows + // + + // build the list of config sources + var srcs []Source + srcs = append(srcs, b.Head...) + srcs = append(srcs, b.Sources...) + srcs = append(srcs, b.Tail...) + + // parse the config sources into a configuration + var c Config + for _, s := range srcs { + if s.Name == "" || s.Data == "" { + continue + } + c2, err := Parse(s.Data, s.Format) + if err != nil { + return RuntimeConfig{}, fmt.Errorf("Error parsing %s: %s", s.Name, err) + } + c = Merge(c, c2) + } + + // ---------------------------------------------------------------- + // process/merge some complex values + // + + var dnsServiceTTL = map[string]time.Duration{} + for k, v := range c.DNS.ServiceTTL { + dnsServiceTTL[k] = b.durationVal(fmt.Sprintf("dns_config.service_ttl[%q]", k), &v) + } + + leaveOnTerm := !b.boolVal(c.ServerMode) + if c.LeaveOnTerm != nil { + leaveOnTerm = b.boolVal(c.LeaveOnTerm) + } + + skipLeaveOnInt := b.boolVal(c.ServerMode) + if c.SkipLeaveOnInt != nil { + skipLeaveOnInt = b.boolVal(c.SkipLeaveOnInt) + } + + // ---------------------------------------------------------------- + // checks and services + // + + var checks []*structs.CheckDefinition + if c.Check != nil { + checks = append(checks, b.checkVal(c.Check)) + } + for _, check := range c.Checks { + checks = append(checks, b.checkVal(&check)) + } + + var services []*structs.ServiceDefinition + for _, service := range c.Services { + services = append(services, b.serviceVal(&service)) + } + if c.Service != nil { + services = append(services, b.serviceVal(c.Service)) + } + + // ---------------------------------------------------------------- + // addresses + // + + // determine port values and replace values <= 0 and > 65535 with -1 + dnsPort := b.portVal("ports.dns", c.Ports.DNS) + httpPort := b.portVal("ports.http", c.Ports.HTTP) + httpsPort := b.portVal("ports.https", c.Ports.HTTPS) + serverPort := b.portVal("ports.server", c.Ports.Server) + serfPortLAN := b.portVal("ports.serf_lan", c.Ports.SerfLAN) + serfPortWAN := b.portVal("ports.serf_wan", c.Ports.SerfWAN) + + // determine the default bind and advertise address + // + // First check whether the user provided an ANY address or whether + // the expanded template results in an ANY address. In that case we + // derive an advertise address from the current network + // configuration since we can listen on an ANY address for incoming + // traffic but cannot advertise it as the address on which the + // server can be reached. + + bindAddrs := b.expandAddrs("bind_addr", c.BindAddr) + if len(bindAddrs) == 0 { + return RuntimeConfig{}, fmt.Errorf("bind_addr cannot be empty") + } + if len(bindAddrs) > 1 { + return RuntimeConfig{}, fmt.Errorf("bind_addr cannot contain multiple addresses. Use 'addresses.{dns,http,https}' instead.") + } + if isUnixAddr(bindAddrs[0]) { + return RuntimeConfig{}, fmt.Errorf("bind_addr cannot be a unix socket") + } + if !isIPAddr(bindAddrs[0]) { + return RuntimeConfig{}, fmt.Errorf("bind_addr must be an ip address") + } + + bindAddr := bindAddrs[0].(*net.IPAddr) + + var addrtyp string + var detect func() ([]*net.IPAddr, error) + switch { + case ipaddr.IsAnyV4(c.BindAddr) || ipaddr.IsAnyV4(bindAddr): + addrtyp = "private IPv4" + detect = b.GetPrivateIPv4 + if detect == nil { + detect = ipaddr.GetPrivateIPv4 + } + + case ipaddr.IsAnyV6(c.BindAddr) || ipaddr.IsAnyV6(bindAddr): + addrtyp = "public IPv6" + detect = b.GetPublicIPv6 + if detect == nil { + detect = ipaddr.GetPublicIPv6 + } + } + + advertiseAddr := bindAddr + if detect != nil { + advertiseAddrs, err := detect() + if err != nil { + return RuntimeConfig{}, fmt.Errorf("Error detecting %s address: %s", addrtyp, err) + } + if len(advertiseAddrs) == 0 { + return RuntimeConfig{}, fmt.Errorf("No %s address found", addrtyp) + } + if len(advertiseAddrs) > 1 { + return RuntimeConfig{}, fmt.Errorf("Multiple %s addresses found. Please configure one", addrtyp) + } + advertiseAddr = advertiseAddrs[0] + } + + // derive other bind addresses from the bindAddr + rpcBindAddr := b.makeTCPAddr(bindAddr, nil, serverPort) + serfBindAddrLAN := b.makeTCPAddr(b.expandFirstIP("serf_lan", c.SerfBindAddrLAN), bindAddr, serfPortLAN) + serfBindAddrWAN := b.makeTCPAddr(b.expandFirstIP("serf_wan", c.SerfBindAddrWAN), bindAddr, serfPortWAN) + + // derive other advertise addresses from the advertise address + advertiseAddrLAN := b.makeIPAddr(b.expandFirstIP("advertise_addr", c.AdvertiseAddrLAN), advertiseAddr) + advertiseAddrWAN := b.makeIPAddr(b.expandFirstIP("advertise_addr_wan", c.AdvertiseAddrWAN), advertiseAddrLAN) + rpcAdvertiseAddr := b.makeTCPAddr(b.expandFirstIP("advertise_addresses.rpc", c.AdvertiseAddrs.RPC), advertiseAddrLAN, serverPort) + serfAdvertiseAddrLAN := b.makeTCPAddr(b.expandFirstIP("advertise_addresses.serf_lan", c.AdvertiseAddrs.SerfLAN), advertiseAddrLAN, serfPortLAN) + serfAdvertiseAddrWAN := b.makeTCPAddr(b.expandFirstIP("advertise_addresses.serf_wan", c.AdvertiseAddrs.SerfWAN), advertiseAddrWAN, serfPortWAN) + + // determine client addresses + clientAddrs := b.expandIPs("client_addr", c.ClientAddr) + dnsAddrs := b.makeAddrs(b.expandAddrs("addresses.dns", c.Addresses.DNS), clientAddrs, dnsPort) + httpAddrs := b.makeAddrs(b.expandAddrs("addresses.http", c.Addresses.HTTP), clientAddrs, httpPort) + httpsAddrs := b.makeAddrs(b.expandAddrs("addresses.https", c.Addresses.HTTPS), clientAddrs, httpsPort) + + for _, a := range dnsAddrs { + if x, ok := a.(*net.TCPAddr); ok { + dnsAddrs = append(dnsAddrs, &net.UDPAddr{IP: x.IP, Port: x.Port}) + } + } + + // Create the default set of tagged addresses. + if c.TaggedAddresses == nil { + c.TaggedAddresses = make(map[string]string) + } + c.TaggedAddresses["lan"] = advertiseAddrLAN.IP.String() + c.TaggedAddresses["wan"] = advertiseAddrWAN.IP.String() + + // segments + var segments []structs.NetworkSegment + for _, s := range c.Segments { + name := b.stringVal(s.Name) + port := b.portVal(fmt.Sprintf("segments[%s].port", name), s.Port) + + bind := b.makeTCPAddr( + b.expandFirstIP(fmt.Sprintf("segments[%s].bind", name), s.Bind), + bindAddr, + port, + ) + + advertise := b.makeTCPAddr( + b.expandFirstIP(fmt.Sprintf("segments[%s].advertise", name), s.Advertise), + advertiseAddrLAN, + port, + ) + + segments = append(segments, structs.NetworkSegment{ + Name: name, + Bind: bind, + Advertise: advertise, + RPCListener: b.boolVal(s.RPCListener), + }) + } + + // Parse the metric filters + var telemetryAllowedPrefixes, telemetryBlockedPrefixes []string + for _, rule := range c.Telemetry.PrefixFilter { + if rule == "" { + b.warn("Cannot have empty filter rule in prefix_filter") + continue + } + switch rule[0] { + case '+': + telemetryAllowedPrefixes = append(telemetryAllowedPrefixes, rule[1:]) + case '-': + telemetryBlockedPrefixes = append(telemetryBlockedPrefixes, rule[1:]) + default: + b.warn("Filter rule must begin with either '+' or '-': %q", rule) + } + } + + // raft performance scaling + performanceRaftMultiplier := b.intVal(c.Performance.RaftMultiplier) + if performanceRaftMultiplier < 1 || uint(performanceRaftMultiplier) > consul.MaxRaftMultiplier { + return RuntimeConfig{}, fmt.Errorf("performance.raft_multiplier cannot be %d. Must be between 1 and %d", performanceRaftMultiplier, consul.MaxRaftMultiplier) + } + consulRaftElectionTimeout := b.durationVal("consul.raft.election_timeout", c.Consul.Raft.ElectionTimeout) * time.Duration(performanceRaftMultiplier) + consulRaftHeartbeatTimeout := b.durationVal("consul.raft.heartbeat_timeout", c.Consul.Raft.HeartbeatTimeout) * time.Duration(performanceRaftMultiplier) + consulRaftLeaderLeaseTimeout := b.durationVal("consul.raft.leader_lease_timeout", c.Consul.Raft.LeaderLeaseTimeout) * time.Duration(performanceRaftMultiplier) + + // ---------------------------------------------------------------- + // build runtime config + // + rt = RuntimeConfig{ + // non-user configurable values + ACLDisabledTTL: b.durationVal("acl_disabled_ttl", c.ACLDisabledTTL), + AEInterval: b.durationVal("ae_interval", c.AEInterval), + CheckDeregisterIntervalMin: b.durationVal("check_deregister_interval_min", c.CheckDeregisterIntervalMin), + CheckReapInterval: b.durationVal("check_reap_interval", c.CheckReapInterval), + Revision: b.stringVal(c.Revision), + SegmentLimit: b.intVal(c.SegmentLimit), + SegmentNameLimit: b.intVal(c.SegmentNameLimit), + SyncCoordinateIntervalMin: b.durationVal("sync_coordinate_interval_min", c.SyncCoordinateIntervalMin), + SyncCoordinateRateTarget: b.float64Val(c.SyncCoordinateRateTarget), + Version: b.stringVal(c.Version), + VersionPrerelease: b.stringVal(c.VersionPrerelease), + + // consul configuration + ConsulCoordinateUpdateBatchSize: b.intVal(c.Consul.Coordinate.UpdateBatchSize), + ConsulCoordinateUpdateMaxBatches: b.intVal(c.Consul.Coordinate.UpdateMaxBatches), + ConsulCoordinateUpdatePeriod: b.durationVal("consul.coordinate.update_period", c.Consul.Coordinate.UpdatePeriod), + ConsulRaftElectionTimeout: consulRaftElectionTimeout, + ConsulRaftHeartbeatTimeout: consulRaftHeartbeatTimeout, + ConsulRaftLeaderLeaseTimeout: consulRaftLeaderLeaseTimeout, + ConsulSerfLANGossipInterval: b.durationVal("consul.serf_lan.gossip_interval", c.Consul.SerfLAN.Memberlist.GossipInterval), + ConsulSerfLANProbeInterval: b.durationVal("consul.serf_lan.probe_interval", c.Consul.SerfLAN.Memberlist.ProbeInterval), + ConsulSerfLANProbeTimeout: b.durationVal("consul.serf_lan.probe_timeout", c.Consul.SerfLAN.Memberlist.ProbeTimeout), + ConsulSerfLANSuspicionMult: b.intVal(c.Consul.SerfLAN.Memberlist.SuspicionMult), + ConsulSerfWANGossipInterval: b.durationVal("consul.serf_wan.gossip_interval", c.Consul.SerfWAN.Memberlist.GossipInterval), + ConsulSerfWANProbeInterval: b.durationVal("consul.serf_wan.probe_interval", c.Consul.SerfWAN.Memberlist.ProbeInterval), + ConsulSerfWANProbeTimeout: b.durationVal("consul.serf_wan.probe_timeout", c.Consul.SerfWAN.Memberlist.ProbeTimeout), + ConsulSerfWANSuspicionMult: b.intVal(c.Consul.SerfWAN.Memberlist.SuspicionMult), + ConsulServerHealthInterval: b.durationVal("consul.server.health_interval", c.Consul.Server.HealthInterval), + + // ACL + ACLAgentMasterToken: b.stringVal(c.ACLAgentMasterToken), + ACLAgentToken: b.stringVal(c.ACLAgentToken), + ACLDatacenter: strings.ToLower(b.stringVal(c.ACLDatacenter)), + ACLDefaultPolicy: b.stringVal(c.ACLDefaultPolicy), + ACLDownPolicy: b.stringVal(c.ACLDownPolicy), + ACLEnforceVersion8: b.boolVal(c.ACLEnforceVersion8), + ACLMasterToken: b.stringVal(c.ACLMasterToken), + ACLReplicationToken: b.stringVal(c.ACLReplicationToken), + ACLTTL: b.durationVal("acl_ttl", c.ACLTTL), + ACLToken: b.stringVal(c.ACLToken), + EnableACLReplication: b.boolVal(c.EnableACLReplication), + + // Autopilot + AutopilotCleanupDeadServers: b.boolVal(c.Autopilot.CleanupDeadServers), + AutopilotDisableUpgradeMigration: b.boolVal(c.Autopilot.DisableUpgradeMigration), + AutopilotLastContactThreshold: b.durationVal("autopilot.last_contact_threshold", c.Autopilot.LastContactThreshold), + AutopilotMaxTrailingLogs: b.intVal(c.Autopilot.MaxTrailingLogs), + AutopilotRedundancyZoneTag: b.stringVal(c.Autopilot.RedundancyZoneTag), + AutopilotServerStabilizationTime: b.durationVal("autopilot.server_stabilization_time", c.Autopilot.ServerStabilizationTime), + AutopilotUpgradeVersionTag: b.stringVal(c.Autopilot.UpgradeVersionTag), + + // DNS + DNSAddrs: dnsAddrs, + DNSAllowStale: b.boolVal(c.DNS.AllowStale), + DNSDisableCompression: b.boolVal(c.DNS.DisableCompression), + DNSDomain: b.stringVal(c.DNSDomain), + DNSEnableTruncate: b.boolVal(c.DNS.EnableTruncate), + DNSMaxStale: b.durationVal("dns_config.max_stale", c.DNS.MaxStale), + DNSNodeTTL: b.durationVal("dns_config.node_ttl", c.DNS.NodeTTL), + DNSOnlyPassing: b.boolVal(c.DNS.OnlyPassing), + DNSPort: dnsPort, + DNSRecursorTimeout: b.durationVal("recursor_timeout", c.DNS.RecursorTimeout), + DNSRecursors: c.DNSRecursors, + DNSServiceTTL: dnsServiceTTL, + DNSUDPAnswerLimit: b.intVal(c.DNS.UDPAnswerLimit), + + // HTTP + HTTPPort: httpPort, + HTTPSPort: httpsPort, + HTTPAddrs: httpAddrs, + HTTPSAddrs: httpsAddrs, + HTTPBlockEndpoints: c.HTTPConfig.BlockEndpoints, + HTTPResponseHeaders: c.HTTPConfig.ResponseHeaders, + + // Telemetry + TelemetryCirconusAPIApp: b.stringVal(c.Telemetry.CirconusAPIApp), + TelemetryCirconusAPIToken: b.stringVal(c.Telemetry.CirconusAPIToken), + TelemetryCirconusAPIURL: b.stringVal(c.Telemetry.CirconusAPIURL), + TelemetryCirconusBrokerID: b.stringVal(c.Telemetry.CirconusBrokerID), + TelemetryCirconusBrokerSelectTag: b.stringVal(c.Telemetry.CirconusBrokerSelectTag), + TelemetryCirconusCheckDisplayName: b.stringVal(c.Telemetry.CirconusCheckDisplayName), + TelemetryCirconusCheckForceMetricActivation: b.stringVal(c.Telemetry.CirconusCheckForceMetricActivation), + TelemetryCirconusCheckID: b.stringVal(c.Telemetry.CirconusCheckID), + TelemetryCirconusCheckInstanceID: b.stringVal(c.Telemetry.CirconusCheckInstanceID), + TelemetryCirconusCheckSearchTag: b.stringVal(c.Telemetry.CirconusCheckSearchTag), + TelemetryCirconusCheckTags: b.stringVal(c.Telemetry.CirconusCheckTags), + TelemetryCirconusSubmissionInterval: b.stringVal(c.Telemetry.CirconusSubmissionInterval), + TelemetryCirconusSubmissionURL: b.stringVal(c.Telemetry.CirconusSubmissionURL), + TelemetryDisableHostname: b.boolVal(c.Telemetry.DisableHostname), + TelemetryDogstatsdAddr: b.stringVal(c.Telemetry.DogstatsdAddr), + TelemetryDogstatsdTags: c.Telemetry.DogstatsdTags, + TelemetryFilterDefault: b.boolVal(c.Telemetry.FilterDefault), + TelemetryAllowedPrefixes: telemetryAllowedPrefixes, + TelemetryBlockedPrefixes: telemetryBlockedPrefixes, + TelemetryStatsdAddr: b.stringVal(c.Telemetry.StatsdAddr), + TelemetryStatsiteAddr: b.stringVal(c.Telemetry.StatsiteAddr), + TelemetryStatsitePrefix: b.stringVal(c.Telemetry.StatsitePrefix), + + // Agent + AdvertiseAddrLAN: advertiseAddrLAN, + AdvertiseAddrWAN: advertiseAddrWAN, + BindAddr: bindAddr, + Bootstrap: b.boolVal(c.Bootstrap), + BootstrapExpect: b.intVal(c.BootstrapExpect), + CAFile: b.stringVal(c.CAFile), + CAPath: b.stringVal(c.CAPath), + CertFile: b.stringVal(c.CertFile), + CheckUpdateInterval: b.durationVal("check_update_interval", c.CheckUpdateInterval), + Checks: checks, + ClientAddrs: clientAddrs, + DataDir: b.stringVal(c.DataDir), + Datacenter: strings.ToLower(b.stringVal(c.Datacenter)), + DevMode: b.boolVal(b.Flags.DevMode), + DisableAnonymousSignature: b.boolVal(c.DisableAnonymousSignature), + DisableCoordinates: b.boolVal(c.DisableCoordinates), + DisableHostNodeID: b.boolVal(c.DisableHostNodeID), + DisableKeyringFile: b.boolVal(c.DisableKeyringFile), + DisableRemoteExec: b.boolVal(c.DisableRemoteExec), + DisableUpdateCheck: b.boolVal(c.DisableUpdateCheck), + EnableDebug: b.boolVal(c.EnableDebug), + EnableScriptChecks: b.boolVal(c.EnableScriptChecks), + EnableSyslog: b.boolVal(c.EnableSyslog), + EnableUI: b.boolVal(c.EnableUI), + EncryptKey: b.stringVal(c.EncryptKey), + EncryptVerifyIncoming: b.boolVal(c.EncryptVerifyIncoming), + EncryptVerifyOutgoing: b.boolVal(c.EncryptVerifyOutgoing), + KeyFile: b.stringVal(c.KeyFile), + LeaveOnTerm: leaveOnTerm, + LogLevel: b.stringVal(c.LogLevel), + NodeID: types.NodeID(b.stringVal(c.NodeID)), + NodeMeta: c.NodeMeta, + NodeName: b.nodeName(c.NodeName), + NonVotingServer: b.boolVal(c.NonVotingServer), + PidFile: b.stringVal(c.PidFile), + RPCAdvertiseAddr: rpcAdvertiseAddr, + RPCBindAddr: rpcBindAddr, + RPCMaxBurst: b.intVal(c.Limits.RPCMaxBurst), + RPCProtocol: b.intVal(c.RPCProtocol), + RPCRateLimit: rate.Limit(b.float64Val(c.Limits.RPCRate)), + RaftProtocol: b.intVal(c.RaftProtocol), + ReconnectTimeoutLAN: b.durationVal("reconnect_timeout", c.ReconnectTimeoutLAN), + ReconnectTimeoutWAN: b.durationVal("reconnect_timeout_wan", c.ReconnectTimeoutWAN), + RejoinAfterLeave: b.boolVal(c.RejoinAfterLeave), + RetryJoinIntervalLAN: b.durationVal("retry_interval", c.RetryJoinIntervalLAN), + RetryJoinIntervalWAN: b.durationVal("retry_interval_wan", c.RetryJoinIntervalWAN), + RetryJoinLAN: c.RetryJoinLAN, + RetryJoinMaxAttemptsLAN: b.intVal(c.RetryJoinMaxAttemptsLAN), + RetryJoinMaxAttemptsWAN: b.intVal(c.RetryJoinMaxAttemptsWAN), + RetryJoinWAN: c.RetryJoinWAN, + SegmentName: b.stringVal(c.SegmentName), + Segments: segments, + SerfAdvertiseAddrLAN: serfAdvertiseAddrLAN, + SerfAdvertiseAddrWAN: serfAdvertiseAddrWAN, + SerfBindAddrLAN: serfBindAddrLAN, + SerfBindAddrWAN: serfBindAddrWAN, + SerfPortLAN: serfPortLAN, + SerfPortWAN: serfPortWAN, + ServerMode: b.boolVal(c.ServerMode), + ServerName: b.stringVal(c.ServerName), + ServerPort: serverPort, + Services: services, + SessionTTLMin: b.durationVal("session_ttl_min", c.SessionTTLMin), + SkipLeaveOnInt: skipLeaveOnInt, + StartJoinAddrsLAN: c.StartJoinAddrsLAN, + StartJoinAddrsWAN: c.StartJoinAddrsWAN, + SyslogFacility: b.stringVal(c.SyslogFacility), + TLSCipherSuites: b.tlsCipherSuites("tls_cipher_suites", c.TLSCipherSuites), + TLSMinVersion: b.stringVal(c.TLSMinVersion), + TLSPreferServerCipherSuites: b.boolVal(c.TLSPreferServerCipherSuites), + TaggedAddresses: c.TaggedAddresses, + TranslateWANAddrs: b.boolVal(c.TranslateWANAddrs), + UIDir: b.stringVal(c.UIDir), + UnixSocketGroup: b.stringVal(c.UnixSocket.Group), + UnixSocketMode: b.stringVal(c.UnixSocket.Mode), + UnixSocketUser: b.stringVal(c.UnixSocket.User), + VerifyIncoming: b.boolVal(c.VerifyIncoming), + VerifyIncomingHTTPS: b.boolVal(c.VerifyIncomingHTTPS), + VerifyIncomingRPC: b.boolVal(c.VerifyIncomingRPC), + VerifyOutgoing: b.boolVal(c.VerifyOutgoing), + VerifyServerHostname: b.boolVal(c.VerifyServerHostname), + Watches: c.Watches, + } + + if rt.BootstrapExpect == 1 { + rt.Bootstrap = true + rt.BootstrapExpect = 0 + b.warn(`BootstrapExpect is set to 1; this is the same as Bootstrap mode.`) + } + + if rt.ACLReplicationToken != "" { + rt.EnableACLReplication = true + } + + return rt, nil +} + +// Validate performs semantical validation of the runtime configuration. +func (b *Builder) Validate(rt RuntimeConfig) error { + // reDatacenter defines a regexp for a valid datacenter name + var reDatacenter = regexp.MustCompile("^[a-z0-9_-]+$") + + // ---------------------------------------------------------------- + // check required params we cannot recover from first + // + + if rt.Datacenter == "" { + return fmt.Errorf("datacenter cannot be empty") + } + if !reDatacenter.MatchString(rt.Datacenter) { + return fmt.Errorf("datacenter cannot be %q. Please use only [a-z0-9-_].", rt.Datacenter) + } + if rt.DataDir == "" && !rt.DevMode { + return fmt.Errorf("data_dir cannot be empty") + } + if !rt.DevMode { + fi, err := os.Stat(rt.DataDir) + switch { + case err != nil && !os.IsNotExist(err): + return fmt.Errorf("Error getting info on data_dir: %s", err) + case err == nil && !fi.IsDir(): + return fmt.Errorf("data_dir %q is not a directory", rt.DataDir) + } + } + if rt.NodeName == "" { + return fmt.Errorf("node_name cannot be empty") + } + if ipaddr.IsAny(rt.AdvertiseAddrLAN.IP) { + return fmt.Errorf("Advertise address cannot be 0.0.0.0, :: or [::]") + } + if ipaddr.IsAny(rt.AdvertiseAddrWAN.IP) { + return fmt.Errorf("Advertise WAN address cannot be 0.0.0.0, :: or [::]") + } + if ipaddr.IsAny(rt.RPCAdvertiseAddr) { + return fmt.Errorf("advertise_addrs.rpc cannot be 0.0.0.0, :: or [::]") + } + if ipaddr.IsAny(rt.SerfAdvertiseAddrLAN) { + return fmt.Errorf("advertise_addrs.serf_lan cannot be 0.0.0.0, :: or [::]") + } + if ipaddr.IsAny(rt.SerfAdvertiseAddrWAN) { + return fmt.Errorf("advertise_addrs.serf_wan cannot be 0.0.0.0, :: or [::]") + } + if err := b.validateSegments(rt); err != nil { + return err + } + for _, a := range rt.DNSAddrs { + if _, ok := a.(*net.UnixAddr); ok { + return fmt.Errorf("DNS address cannot be a unix socket") + } + } + if rt.Bootstrap && !rt.ServerMode { + return fmt.Errorf("'bootstrap = true' requires 'server = true'") + } + if rt.BootstrapExpect < 0 { + return fmt.Errorf("bootstrap_expect cannot be %d. Must be greater than or equal to zero", rt.BootstrapExpect) + } + if rt.BootstrapExpect > 0 && !rt.ServerMode { + return fmt.Errorf("'bootstrap_expect > 0' requires 'server = true'") + } + if rt.BootstrapExpect > 0 && rt.DevMode { + return fmt.Errorf("'bootstrap_expect > 0' not allowed in dev mode") + } + if rt.BootstrapExpect > 0 && rt.Bootstrap { + return fmt.Errorf("'bootstrap_expect > 0' and 'bootstrap = true' are mutually exclusive") + } + if rt.AEInterval <= 0 { + return fmt.Errorf("ae_interval cannot be %s. Must be positive", rt.AEInterval) + } + if rt.AutopilotMaxTrailingLogs < 0 { + return fmt.Errorf("autopilot.max_trailing_logs cannot be %d. Must be greater than or equal to zero", rt.AutopilotMaxTrailingLogs) + } + if rt.ACLDatacenter != "" && !reDatacenter.MatchString(rt.ACLDatacenter) { + return fmt.Errorf("acl_datacenter cannot be %q. Please use only [a-z0-9-_].", rt.ACLDatacenter) + } + if rt.EnableUI && rt.UIDir != "" { + return fmt.Errorf( + "Both the ui and ui-dir flags were specified, please provide only one.\n" + + "If trying to use your own web UI resources, use the ui-dir flag.\n" + + "If using Consul version 0.7.0 or later, the web UI is included in the binary so use ui to enable it") + } + if rt.DNSUDPAnswerLimit < 0 { + return fmt.Errorf("dns_config.udp_answer_limit cannot be %d. Must be greater than or equal to zero", rt.DNSUDPAnswerLimit) + } + if err := structs.ValidateMetadata(rt.NodeMeta, false); err != nil { + return fmt.Errorf("node_meta invalid: %v", err) + } + if rt.EncryptKey != "" { + if _, err := decodeBytes(rt.EncryptKey); err != nil { + return fmt.Errorf("encrypt has invalid key: %s", err) + } + keyfileLAN := filepath.Join(rt.DataDir, SerfLANKeyring) + if _, err := os.Stat(keyfileLAN); err == nil { + b.warn("WARNING: LAN keyring exists but -encrypt given, using keyring") + } + if rt.ServerMode { + keyfileWAN := filepath.Join(rt.DataDir, SerfWANKeyring) + if _, err := os.Stat(keyfileWAN); err == nil { + b.warn("WARNING: WAN keyring exists but -encrypt given, using keyring") + } + } + } + + // Check the data dir for signs of an un-migrated Consul 0.5.x or older + // server. Consul refuses to start if this is present to protect a server + // with existing data from starting on a fresh data set. + if rt.ServerMode { + mdbPath := filepath.Join(rt.DataDir, "mdb") + if _, err := os.Stat(mdbPath); !os.IsNotExist(err) { + if os.IsPermission(err) { + return fmt.Errorf( + "CRITICAL: Permission denied for data folder at %q!\n"+ + "Consul will refuse to boot without access to this directory.\n"+ + "Please correct permissions and try starting again.", mdbPath) + } + return fmt.Errorf("CRITICAL: Deprecated data folder found at %q!\n"+ + "Consul will refuse to boot with this directory present.\n"+ + "See https://www.consul.io/docs/upgrade-specific.html for more information.", mdbPath) + } + } + + inuse := map[string]string{} + if err := addrsUnique(inuse, "DNS", rt.DNSAddrs); err != nil { + // cannot happen since this is the first address + // we leave this for consistency + return err + } + if err := addrsUnique(inuse, "HTTP", rt.HTTPAddrs); err != nil { + return err + } + if err := addrsUnique(inuse, "HTTPS", rt.HTTPSAddrs); err != nil { + return err + } + if err := addrUnique(inuse, "RPC Advertise", rt.RPCAdvertiseAddr); err != nil { + return err + } + if err := addrUnique(inuse, "Serf Advertise LAN", rt.SerfAdvertiseAddrLAN); err != nil { + return err + } + if err := addrUnique(inuse, "Serf Advertise WAN", rt.SerfAdvertiseAddrWAN); err != nil { + return err + } + if b.err != nil { + return b.err + } + + // ---------------------------------------------------------------- + // warnings + // + + if rt.ServerMode && !rt.DevMode && !rt.Bootstrap && rt.BootstrapExpect == 2 { + b.warn(`bootstrap_expect = 2: A cluster with 2 servers will provide no failure tolerance. See https://www.consul.io/docs/internals/consensus.html#deployment-table`) + } + + if rt.ServerMode && !rt.Bootstrap && rt.BootstrapExpect > 2 && rt.BootstrapExpect%2 == 0 { + b.warn(`bootstrap_expect is even number: A cluster with an even number of servers does not achieve optimum fault tolerance. See https://www.consul.io/docs/internals/consensus.html#deployment-table`) + } + + if rt.ServerMode && rt.Bootstrap && rt.BootstrapExpect == 0 { + b.warn(`bootstrap = true: do not enable unless necessary`) + } + + if rt.ServerMode && !rt.DevMode && !rt.Bootstrap && rt.BootstrapExpect > 1 { + b.warn("bootstrap_expect > 0: expecting %d servers", rt.BootstrapExpect) + } + + return nil +} + +// addrUnique checks if the given address is already in use for another +// protocol. +func addrUnique(inuse map[string]string, name string, addr net.Addr) error { + key := addr.Network() + ":" + addr.String() + if other, ok := inuse[key]; ok { + return fmt.Errorf("%s address %s already configured for %s", name, addr.String(), other) + } + inuse[key] = name + return nil +} + +// addrsUnique checks if any of the give addresses is already in use for +// another protocol. +func addrsUnique(inuse map[string]string, name string, addrs []net.Addr) error { + for _, a := range addrs { + if err := addrUnique(inuse, name, a); err != nil { + return err + } + } + return nil +} + +// splitSlicesAndValues moves all slice values defined in c to 'slices' +// and all other values to 'values'. +func (b *Builder) splitSlicesAndValues(c Config) (slices, values Config) { + v, t := reflect.ValueOf(c), reflect.TypeOf(c) + rs, rv := reflect.New(t), reflect.New(t) + + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if f.Type.Kind() == reflect.Slice { + rs.Elem().Field(i).Set(v.Field(i)) + } else { + rv.Elem().Field(i).Set(v.Field(i)) + } + } + return rs.Elem().Interface().(Config), rv.Elem().Interface().(Config) +} + +func (b *Builder) warn(msg string, args ...interface{}) { + b.Warnings = append(b.Warnings, fmt.Sprintf(msg, args...)) +} + +func (b *Builder) checkVal(v *CheckDefinition) *structs.CheckDefinition { + if v == nil { + return nil + } + + id := types.CheckID(b.stringVal(v.ID)) + if v.CheckID != nil { + id = types.CheckID(b.stringVal(v.CheckID)) + } + + return &structs.CheckDefinition{ + ID: id, + Name: b.stringVal(v.Name), + Notes: b.stringVal(v.Notes), + ServiceID: b.stringVal(v.ServiceID), + Token: b.stringVal(v.Token), + Status: b.stringVal(v.Status), + Script: b.stringVal(v.Script), + HTTP: b.stringVal(v.HTTP), + Header: v.Header, + Method: b.stringVal(v.Method), + TCP: b.stringVal(v.TCP), + Interval: b.durationVal(fmt.Sprintf("check[%s].interval", id), v.Interval), + DockerContainerID: b.stringVal(v.DockerContainerID), + Shell: b.stringVal(v.Shell), + TLSSkipVerify: b.boolVal(v.TLSSkipVerify), + Timeout: b.durationVal(fmt.Sprintf("check[%s].timeout", id), v.Timeout), + TTL: b.durationVal(fmt.Sprintf("check[%s].ttl", id), v.TTL), + DeregisterCriticalServiceAfter: b.durationVal(fmt.Sprintf("check[%s].deregister_critical_service_after", id), v.DeregisterCriticalServiceAfter), + } +} + +func (b *Builder) serviceVal(v *ServiceDefinition) *structs.ServiceDefinition { + if v == nil { + return nil + } + + var checks structs.CheckTypes + for _, check := range v.Checks { + checks = append(checks, b.checkVal(&check).CheckType()) + } + if v.Check != nil { + checks = append(checks, b.checkVal(v.Check).CheckType()) + } + + return &structs.ServiceDefinition{ + ID: b.stringVal(v.ID), + Name: b.stringVal(v.Name), + Tags: v.Tags, + Address: b.stringVal(v.Address), + Port: b.intVal(v.Port), + Token: b.stringVal(v.Token), + EnableTagOverride: b.boolVal(v.EnableTagOverride), + Checks: checks, + } +} + +func (b *Builder) boolVal(v *bool) bool { + if v == nil { + return false + } + return *v +} + +func (b *Builder) durationVal(name string, v *string) (d time.Duration) { + if v == nil { + return 0 + } + d, err := time.ParseDuration(*v) + if err != nil { + b.err = multierror.Append(fmt.Errorf("%s: invalid duration: %q: %s", name, *v, err)) + } + return d +} + +func (b *Builder) intVal(v *int) int { + if v == nil { + return 0 + } + return *v +} + +func (b *Builder) portVal(name string, v *int) int { + if v == nil || *v <= 0 { + return -1 + } + if *v > 65535 { + b.err = multierror.Append(b.err, fmt.Errorf("%s: invalid port: %d", name, *v)) + } + return *v +} + +func (b *Builder) stringVal(v *string) string { + if v == nil { + return "" + } + return *v +} + +func (b *Builder) float64Val(v *float64) float64 { + if v == nil { + return 0 + } + + return *v +} + +func (b *Builder) tlsCipherSuites(name string, v *string) []uint16 { + if v == nil { + return nil + } + + var a []uint16 + a, err := tlsutil.ParseCiphers(*v) + if err != nil { + b.err = multierror.Append(b.err, fmt.Errorf("%s: invalid tls cipher suites: %s", name, err)) + } + return a +} + +func (b *Builder) nodeName(v *string) string { + nodeName := b.stringVal(v) + if nodeName == "" { + fn := b.Hostname + if fn == nil { + fn = os.Hostname + } + name, err := fn() + if err != nil { + b.err = multierror.Append(b.err, fmt.Errorf("node_name: %s", err)) + return "" + } + nodeName = name + } + return strings.TrimSpace(nodeName) +} + +// expandAddrs expands the go-sockaddr template in s and returns the +// result as a list of *net.IPAddr and *net.UnixAddr. +func (b *Builder) expandAddrs(name string, s *string) []net.Addr { + if s == nil || *s == "" { + return nil + } + + x, err := template.Parse(*s) + if err != nil { + b.err = multierror.Append(b.err, fmt.Errorf("%s: error parsing %q: %s", name, s, err)) + return nil + } + + var addrs []net.Addr + for _, a := range strings.Fields(x) { + switch { + case strings.HasPrefix(a, "unix://"): + addrs = append(addrs, &net.UnixAddr{Name: a[len("unix://"):], Net: "unix"}) + default: + // net.ParseIP does not like '[::]' + ip := net.ParseIP(a) + if a == "[::]" { + ip = net.ParseIP("::") + } + if ip == nil { + b.err = multierror.Append(b.err, fmt.Errorf("%s: invalid ip address: %s", name, a)) + return nil + } + addrs = append(addrs, &net.IPAddr{IP: ip}) + } + } + + return addrs +} + +// expandIPs expands the go-sockaddr template in s and returns a list of +// *net.IPAddr. If one of the expanded addresses is a unix socket +// address an error is set and nil is returned. +func (b *Builder) expandIPs(name string, s *string) []*net.IPAddr { + if s == nil || *s == "" { + return nil + } + + addrs := b.expandAddrs(name, s) + var x []*net.IPAddr + for _, addr := range addrs { + switch a := addr.(type) { + case *net.IPAddr: + x = append(x, a) + case *net.UnixAddr: + b.err = multierror.Append(b.err, fmt.Errorf("%s cannot be a unix socket", name)) + return nil + default: + b.err = multierror.Append(b.err, fmt.Errorf("%s has invalid address type %T", name, a)) + return nil + } + } + return x +} + +// expandFirstAddr expands the go-sockaddr template in s and returns the +// first address which is either a *net.IPAddr or a *net.UnixAddr. If +// the template expands to multiple addresses an error is set and nil +// is returned. +func (b *Builder) expandFirstAddr(name string, s *string) net.Addr { + if s == nil || *s == "" { + return nil + } + + addrs := b.expandAddrs(name, s) + if len(addrs) == 0 { + return nil + } + if len(addrs) > 1 { + var x []string + for _, a := range addrs { + x = append(x, a.String()) + } + b.err = multierror.Append(b.err, fmt.Errorf("%s: multiple addresses found: %s", name, strings.Join(x, " "))) + return nil + } + return addrs[0] +} + +// expandFirstIP exapnds the go-sockaddr template in s and returns the +// first address if it is not a unix socket address. If the template +// expands to multiple addresses an error is set and nil is returned. +func (b *Builder) expandFirstIP(name string, s *string) *net.IPAddr { + if s == nil || *s == "" { + return nil + } + + addr := b.expandFirstAddr(name, s) + if addr == nil { + return nil + } + switch a := addr.(type) { + case *net.IPAddr: + return a + case *net.UnixAddr: + b.err = multierror.Append(b.err, fmt.Errorf("%s cannot be a unix socket", name)) + return nil + default: + b.err = multierror.Append(b.err, fmt.Errorf("%s has invalid address type %T", name, a)) + return nil + } +} + +func (b *Builder) makeIPAddr(pri *net.IPAddr, sec *net.IPAddr) *net.IPAddr { + if pri != nil { + return pri + } + return sec +} + +func (b *Builder) makeTCPAddr(pri *net.IPAddr, sec net.Addr, port int) *net.TCPAddr { + if pri == nil && reflect.ValueOf(sec).IsNil() || port <= 0 { + return nil + } + addr := pri + if addr == nil { + switch a := sec.(type) { + case *net.IPAddr: + addr = a + case *net.TCPAddr: + addr = &net.IPAddr{IP: a.IP} + default: + panic(fmt.Sprintf("makeTCPAddr requires a net.IPAddr or a net.TCPAddr. Got %T", a)) + } + } + return &net.TCPAddr{IP: addr.IP, Port: port} +} + +// makeAddr creates an *net.TCPAddr or a *net.UnixAddr from either the +// primary or secondary address and the given port. If the port is <= 0 +// then the address is considered to be disabled and nil is returned. +func (b *Builder) makeAddr(pri, sec net.Addr, port int) net.Addr { + if reflect.ValueOf(pri).IsNil() && reflect.ValueOf(sec).IsNil() || port <= 0 { + return nil + } + addr := pri + if addr == nil { + addr = sec + } + switch a := addr.(type) { + case *net.IPAddr: + return &net.TCPAddr{IP: a.IP, Port: port} + case *net.UnixAddr: + return a + default: + panic(fmt.Sprintf("invalid address type %T", a)) + } +} + +// makeAddrs creates a list of *net.TCPAddr or *net.UnixAddr entries +// from either the primary or secondary addresses and the given port. +// If the port is <= 0 then the address is considered to be disabled +// and nil is returned. +func (b *Builder) makeAddrs(pri []net.Addr, sec []*net.IPAddr, port int) []net.Addr { + if len(pri) == 0 && len(sec) == 0 || port <= 0 { + return nil + } + addrs := pri + if len(addrs) == 0 { + addrs = []net.Addr{} + for _, a := range sec { + addrs = append(addrs, a) + } + } + var x []net.Addr + for _, a := range addrs { + x = append(x, b.makeAddr(a, nil, port)) + } + return x +} + +// isUnixAddr returns true when the given address is a unix socket address type. +func (b *Builder) isUnixAddr(a net.Addr) bool { + _, ok := a.(*net.UnixAddr) + return a != nil && ok +} + +// decodeBytes returns the encryption key decoded. +func decodeBytes(key string) ([]byte, error) { + return base64.StdEncoding.DecodeString(key) +} + +func isIPAddr(a net.Addr) bool { + _, ok := a.(*net.IPAddr) + return ok +} + +func isUnixAddr(a net.Addr) bool { + _, ok := a.(*net.UnixAddr) + return ok +} diff --git a/agent/config/config.go b/agent/config/config.go new file mode 100644 index 000000000..238f9164f --- /dev/null +++ b/agent/config/config.go @@ -0,0 +1,423 @@ +package config + +import ( + "encoding/json" + "fmt" + "strings" + + multierror "github.com/hashicorp/go-multierror" + "github.com/hashicorp/hcl" + "github.com/mitchellh/mapstructure" +) + +const ( + SerfLANKeyring = "serf/local.keyring" + SerfWANKeyring = "serf/remote.keyring" +) + +type Source struct { + Name string + Format string + Data string +} + +func NewSource(name, data string) Source { + return Source{Name: name, Format: FormatFrom(name), Data: data} +} + +func FormatFrom(name string) string { + if strings.HasSuffix(name, ".hcl") { + return "hcl" + } + return "json" +} + +// Parse parses a config fragment in either JSON or HCL format. +func Parse(data string, format string) (c Config, err error) { + var raw map[string]interface{} + switch format { + case "json": + err = json.Unmarshal([]byte(data), &raw) + case "hcl": + err = hcl.Decode(&raw, data) + default: + err = fmt.Errorf("invalid format: %s", format) + } + if err != nil { + return Config{}, err + } + + // We want to be able to report fields which we cannot map as an + // error so that users find typos in their configuration quickly. To + // achieve this we use the mapstructure library which maps a a raw + // map[string]interface{} to a nested structure and reports unused + // fields. The input for a mapstructure.Decode expects a + // map[string]interface{} as produced by encoding/json. + // + // The HCL language allows to repeat map keys which forces it to + // store nested structs as []map[string]interface{} instead of + // map[string]interface{}. This is an ambiguity which makes the + // generated structures incompatible with a corresponding JSON + // struct. It also does not work well with the mapstructure library. + // + // In order to still use the mapstructure library to find unused + // fields we patch instances of []map[string]interface{} to a + // map[string]interface{} before we decode that into a Config + // struct. + // + // However, Config has some fields which are either + // []map[string]interface{} or are arrays of structs which + // encoding/json will decode to []map[string]interface{}. Therefore, + // we need to be able to specify exceptions for this mapping. The + // patchSliceOfMaps() implements that mapping. All fields of type + // []map[string]interface{} are mapped to map[string]interface{} if + // it contains at most one value. If there is more than one value it + // panics. To define exceptions one can specify the nested field + // names in dot notation. + // + // todo(fs): There might be an easier way to achieve the same thing + // todo(fs): but this approach works for now. + m := patchSliceOfMaps(raw, []string{ + "checks", + "segments", + "service.checks", + "services", + "services.checks", + "watches", + }) + + // toJSON := func(v interface{}) string { + // b, err := json.MarshalIndent(v, "", " ") + // if err != nil { + // panic(err) + // } + // return string(b) + // } + // fmt.Println("raw:", toJSON(raw)) + // fmt.Println("patched:", toJSON(m)) + + var md mapstructure.Metadata + d, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{ + Metadata: &md, + Result: &c, + }) + if err != nil { + return Config{}, err + } + if err := d.Decode(m); err != nil { + return Config{}, err + } + for _, k := range md.Unused { + err = multierror.Append(err, fmt.Errorf("invalid config key %s", k)) + } + return +} + +// Config defines the format of a configuration file in either JSON or +// HCL format. +// +// It must contain only pointer values, slices and maps to support +// standardized merging of multiple Config structs into one. +// +// Since this is the format which users use to specify their +// configuration it should be treated as an external API which cannot be +// changed and refactored at will since this will break existing setups. +type Config struct { + ACLAgentMasterToken *string `json:"acl_agent_master_token,omitempty" hcl:"acl_agent_master_token" mapstructure:"acl_agent_master_token"` + ACLAgentToken *string `json:"acl_agent_token,omitempty" hcl:"acl_agent_token" mapstructure:"acl_agent_token"` + ACLDatacenter *string `json:"acl_datacenter,omitempty" hcl:"acl_datacenter" mapstructure:"acl_datacenter"` + ACLDefaultPolicy *string `json:"acl_default_policy,omitempty" hcl:"acl_default_policy" mapstructure:"acl_default_policy"` + ACLDownPolicy *string `json:"acl_down_policy,omitempty" hcl:"acl_down_policy" mapstructure:"acl_down_policy"` + ACLEnforceVersion8 *bool `json:"acl_enforce_version_8,omitempty" hcl:"acl_enforce_version_8" mapstructure:"acl_enforce_version_8"` + ACLMasterToken *string `json:"acl_master_token,omitempty" hcl:"acl_master_token" mapstructure:"acl_master_token"` + ACLReplicationToken *string `json:"acl_replication_token,omitempty" hcl:"acl_replication_token" mapstructure:"acl_replication_token"` + ACLTTL *string `json:"acl_ttl,omitempty" hcl:"acl_ttl" mapstructure:"acl_ttl"` + ACLToken *string `json:"acl_token,omitempty" hcl:"acl_token" mapstructure:"acl_token"` + Addresses Addresses `json:"addresses,omitempty" hcl:"addresses" mapstructure:"addresses"` + AdvertiseAddrLAN *string `json:"advertise_addr,omitempty" hcl:"advertise_addr" mapstructure:"advertise_addr"` + AdvertiseAddrWAN *string `json:"advertise_addr_wan,omitempty" hcl:"advertise_addr_wan" mapstructure:"advertise_addr_wan"` + AdvertiseAddrs AdvertiseAddrsConfig `json:"advertise_addrs,omitempty" hcl:"advertise_addrs" mapstructure:"advertise_addrs"` + Autopilot Autopilot `json:"autopilot,omitempty" hcl:"autopilot" mapstructure:"autopilot"` + BindAddr *string `json:"bind_addr,omitempty" hcl:"bind_addr" mapstructure:"bind_addr"` + Bootstrap *bool `json:"bootstrap,omitempty" hcl:"bootstrap" mapstructure:"bootstrap"` + BootstrapExpect *int `json:"bootstrap_expect,omitempty" hcl:"bootstrap_expect" mapstructure:"bootstrap_expect"` + CAFile *string `json:"ca_file,omitempty" hcl:"ca_file" mapstructure:"ca_file"` + CAPath *string `json:"ca_path,omitempty" hcl:"ca_path" mapstructure:"ca_path"` + CertFile *string `json:"cert_file,omitempty" hcl:"cert_file" mapstructure:"cert_file"` + Check *CheckDefinition `json:"check,omitempty" hcl:"check" mapstructure:"check"` // needs to be a pointer to avoid partial merges + CheckUpdateInterval *string `json:"check_update_interval,omitempty" hcl:"check_update_interval" mapstructure:"check_update_interval"` + Checks []CheckDefinition `json:"checks,omitempty" hcl:"checks" mapstructure:"checks"` + ClientAddr *string `json:"client_addr,omitempty" hcl:"client_addr" mapstructure:"client_addr"` + DNS DNS `json:"dns_config,omitempty" hcl:"dns_config" mapstructure:"dns_config"` + DNSDomain *string `json:"domain,omitempty" hcl:"domain" mapstructure:"domain"` + DNSRecursors []string `json:"recursors,omitempty" hcl:"recursors" mapstructure:"recursors"` + DataDir *string `json:"data_dir,omitempty" hcl:"data_dir" mapstructure:"data_dir"` + Datacenter *string `json:"datacenter,omitempty" hcl:"datacenter" mapstructure:"datacenter"` + DisableAnonymousSignature *bool `json:"disable_anonymous_signature,omitempty" hcl:"disable_anonymous_signature" mapstructure:"disable_anonymous_signature"` + DisableCoordinates *bool `json:"disable_coordinates,omitempty" hcl:"disable_coordinates" mapstructure:"disable_coordinates"` + DisableHostNodeID *bool `json:"disable_host_node_id,omitempty" hcl:"disable_host_node_id" mapstructure:"disable_host_node_id"` + DisableKeyringFile *bool `json:"disable_keyring_file,omitempty" hcl:"disable_keyring_file" mapstructure:"disable_keyring_file"` + DisableRemoteExec *bool `json:"disable_remote_exec,omitempty" hcl:"disable_remote_exec" mapstructure:"disable_remote_exec"` + DisableUpdateCheck *bool `json:"disable_update_check,omitempty" hcl:"disable_update_check" mapstructure:"disable_update_check"` + EnableACLReplication *bool `json:"enable_acl_replication,omitempty" hcl:"enable_acl_replication" mapstructure:"enable_acl_replication"` + EnableDebug *bool `json:"enable_debug,omitempty" hcl:"enable_debug" mapstructure:"enable_debug"` + EnableScriptChecks *bool `json:"enable_script_checks,omitempty" hcl:"enable_script_checks" mapstructure:"enable_script_checks"` + EnableSyslog *bool `json:"enable_syslog,omitempty" hcl:"enable_syslog" mapstructure:"enable_syslog"` + EnableUI *bool `json:"enable_ui,omitempty" hcl:"enable_ui" mapstructure:"enable_ui"` + EncryptKey *string `json:"encrypt,omitempty" hcl:"encrypt" mapstructure:"encrypt"` + EncryptVerifyIncoming *bool `json:"encrypt_verify_incoming,omitempty" hcl:"encrypt_verify_incoming" mapstructure:"encrypt_verify_incoming"` + EncryptVerifyOutgoing *bool `json:"encrypt_verify_outgoing,omitempty" hcl:"encrypt_verify_outgoing" mapstructure:"encrypt_verify_outgoing"` + HTTPConfig HTTPConfig `json:"http_config,omitempty" hcl:"http_config" mapstructure:"http_config"` + KeyFile *string `json:"key_file,omitempty" hcl:"key_file" mapstructure:"key_file"` + LeaveOnTerm *bool `json:"leave_on_terminate,omitempty" hcl:"leave_on_terminate" mapstructure:"leave_on_terminate"` + Limits Limits `json:"limits,omitempty" hcl:"limits" mapstructure:"limits"` + LogLevel *string `json:"log_level,omitempty" hcl:"log_level" mapstructure:"log_level"` + NodeID *string `json:"node_id,omitempty" hcl:"node_id" mapstructure:"node_id"` + NodeMeta map[string]string `json:"node_meta,omitempty" hcl:"node_meta" mapstructure:"node_meta"` + NodeName *string `json:"node_name,omitempty" hcl:"node_name" mapstructure:"node_name"` + NonVotingServer *bool `json:"non_voting_server,omitempty" hcl:"non_voting_server" mapstructure:"non_voting_server"` + Performance Performance `json:"performance,omitempty" hcl:"performance" mapstructure:"performance"` + PidFile *string `json:"pid_file,omitempty" hcl:"pid_file" mapstructure:"pid_file"` + Ports Ports `json:"ports,omitempty" hcl:"ports" mapstructure:"ports"` + RPCProtocol *int `json:"protocol,omitempty" hcl:"protocol" mapstructure:"protocol"` + RaftProtocol *int `json:"raft_protocol,omitempty" hcl:"raft_protocol" mapstructure:"raft_protocol"` + ReconnectTimeoutLAN *string `json:"reconnect_timeout,omitempty" hcl:"reconnect_timeout" mapstructure:"reconnect_timeout"` + ReconnectTimeoutWAN *string `json:"reconnect_timeout_wan,omitempty" hcl:"reconnect_timeout_wan" mapstructure:"reconnect_timeout_wan"` + RejoinAfterLeave *bool `json:"rejoin_after_leave,omitempty" hcl:"rejoin_after_leave" mapstructure:"rejoin_after_leave"` + RetryJoinIntervalLAN *string `json:"retry_interval,omitempty" hcl:"retry_interval" mapstructure:"retry_interval"` + RetryJoinIntervalWAN *string `json:"retry_interval_wan,omitempty" hcl:"retry_interval_wan" mapstructure:"retry_interval_wan"` + RetryJoinLAN []string `json:"retry_join,omitempty" hcl:"retry_join" mapstructure:"retry_join"` + RetryJoinMaxAttemptsLAN *int `json:"retry_max,omitempty" hcl:"retry_max" mapstructure:"retry_max"` + RetryJoinMaxAttemptsWAN *int `json:"retry_max_wan,omitempty" hcl:"retry_max_wan" mapstructure:"retry_max_wan"` + RetryJoinWAN []string `json:"retry_join_wan,omitempty" hcl:"retry_join_wan" mapstructure:"retry_join_wan"` + SegmentName *string `json:"segment,omitempty" hcl:"segment" mapstructure:"segment"` + Segments []Segment `json:"segments,omitempty" hcl:"segments" mapstructure:"segments"` + SerfBindAddrLAN *string `json:"serf_lan,omitempty" hcl:"serf_lan" mapstructure:"serf_lan"` + SerfBindAddrWAN *string `json:"serf_wan,omitempty" hcl:"serf_wan" mapstructure:"serf_wan"` + ServerMode *bool `json:"server,omitempty" hcl:"server" mapstructure:"server"` + ServerName *string `json:"server_name,omitempty" hcl:"server_name" mapstructure:"server_name"` + Service *ServiceDefinition `json:"service,omitempty" hcl:"service" mapstructure:"service"` + Services []ServiceDefinition `json:"services,omitempty" hcl:"services" mapstructure:"services"` + SessionTTLMin *string `json:"session_ttl_min,omitempty" hcl:"session_ttl_min" mapstructure:"session_ttl_min"` + SkipLeaveOnInt *bool `json:"skip_leave_on_interrupt,omitempty" hcl:"skip_leave_on_interrupt" mapstructure:"skip_leave_on_interrupt"` + StartJoinAddrsLAN []string `json:"start_join,omitempty" hcl:"start_join" mapstructure:"start_join"` + StartJoinAddrsWAN []string `json:"start_join_wan,omitempty" hcl:"start_join_wan" mapstructure:"start_join_wan"` + SyslogFacility *string `json:"syslog_facility,omitempty" hcl:"syslog_facility" mapstructure:"syslog_facility"` + TLSCipherSuites *string `json:"tls_cipher_suites,omitempty" hcl:"tls_cipher_suites" mapstructure:"tls_cipher_suites"` + TLSMinVersion *string `json:"tls_min_version,omitempty" hcl:"tls_min_version" mapstructure:"tls_min_version"` + TLSPreferServerCipherSuites *bool `json:"tls_prefer_server_cipher_suites,omitempty" hcl:"tls_prefer_server_cipher_suites" mapstructure:"tls_prefer_server_cipher_suites"` + TaggedAddresses map[string]string `json:"tagged_addresses,omitempty" hcl:"tagged_addresses" mapstructure:"tagged_addresses"` + Telemetry Telemetry `json:"telemetry,omitempty" hcl:"telemetry" mapstructure:"telemetry"` + TranslateWANAddrs *bool `json:"translate_wan_addrs,omitempty" hcl:"translate_wan_addrs" mapstructure:"translate_wan_addrs"` + UIDir *string `json:"ui_dir,omitempty" hcl:"ui_dir" mapstructure:"ui_dir"` + UnixSocket UnixSocket `json:"unix_sockets,omitempty" hcl:"unix_sockets" mapstructure:"unix_sockets"` + VerifyIncoming *bool `json:"verify_incoming,omitempty" hcl:"verify_incoming" mapstructure:"verify_incoming"` + VerifyIncomingHTTPS *bool `json:"verify_incoming_https,omitempty" hcl:"verify_incoming_https" mapstructure:"verify_incoming_https"` + VerifyIncomingRPC *bool `json:"verify_incoming_rpc,omitempty" hcl:"verify_incoming_rpc" mapstructure:"verify_incoming_rpc"` + VerifyOutgoing *bool `json:"verify_outgoing,omitempty" hcl:"verify_outgoing" mapstructure:"verify_outgoing"` + VerifyServerHostname *bool `json:"verify_server_hostname,omitempty" hcl:"verify_server_hostname" mapstructure:"verify_server_hostname"` + Watches []map[string]interface{} `json:"watches,omitempty" hcl:"watches" mapstructure:"watches"` + + // non-user configurable values + ACLDisabledTTL *string `json:"acl_disabled_ttl,omitempty" hcl:"acl_disabled_ttl" mapstructure:"acl_disabled_ttl"` + AEInterval *string `json:"ae_interval,omitempty" hcl:"ae_interval" mapstructure:"ae_interval"` + CheckDeregisterIntervalMin *string `json:"check_deregister_interval_min,omitempty" hcl:"check_deregister_interval_min" mapstructure:"check_deregister_interval_min"` + CheckReapInterval *string `json:"check_reap_interval,omitempty" hcl:"check_reap_interval" mapstructure:"check_reap_interval"` + Consul Consul `json:"consul,omitempty" hcl:"consul" mapstructure:"consul"` + Revision *string `json:"revision,omitempty" hcl:"revision" mapstructure:"revision"` + SegmentLimit *int `json:"segment_limit,omitempty" hcl:"segment_limit" mapstructure:"segment_limit"` + SegmentNameLimit *int `json:"segment_name_limit,omitempty" hcl:"segment_name_limit" mapstructure:"segment_name_limit"` + SyncCoordinateIntervalMin *string `json:"sync_coordinate_interval_min,omitempty" hcl:"sync_coordinate_interval_min" mapstructure:"sync_coordinate_interval_min"` + SyncCoordinateRateTarget *float64 `json:"sync_coordinate_rate_target,omitempty" hcl:"sync_coordinate_rate_target" mapstructure:"sync_coordinate_rate_target"` + Version *string `json:"version,omitempty" hcl:"version" mapstructure:"version"` + VersionPrerelease *string `json:"version_prerelease,omitempty" hcl:"version_prerelease" mapstructure:"version_prerelease"` +} + +type Consul struct { + Coordinate struct { + UpdateBatchSize *int `json:"update_batch_size,omitempty" hcl:"update_batch_size" mapstructure:"update_batch_size"` + UpdateMaxBatches *int `json:"update_max_batches,omitempty" hcl:"update_max_batches" mapstructure:"update_max_batches"` + UpdatePeriod *string `json:"update_period,omitempty" hcl:"update_period" mapstructure:"update_period"` + } `json:"coordinate,omitempty" hcl:"coordinate" mapstructure:"coordinate"` + + Raft struct { + ElectionTimeout *string `json:"election_timeout,omitempty" hcl:"election_timeout" mapstructure:"election_timeout"` + HeartbeatTimeout *string `json:"heartbeat_timeout,omitempty" hcl:"heartbeat_timeout" mapstructure:"heartbeat_timeout"` + LeaderLeaseTimeout *string `json:"leader_lease_timeout,omitempty" hcl:"leader_lease_timeout" mapstructure:"leader_lease_timeout"` + } `json:"raft,omitempty" hcl:"raft" mapstructure:"raft"` + + SerfLAN struct { + Memberlist struct { + GossipInterval *string `json:"gossip_interval,omitempty" hcl:"gossip_interval" mapstructure:"gossip_interval"` + ProbeInterval *string `json:"probe_interval,omitempty" hcl:"probe_interval" mapstructure:"probe_interval"` + ProbeTimeout *string `json:"probe_timeout,omitempty" hcl:"probe_timeout" mapstructure:"probe_timeout"` + SuspicionMult *int `json:"suspicion_mult,omitempty" hcl:"suspicion_mult" mapstructure:"suspicion_mult"` + } `json:"memberlist,omitempty" hcl:"memberlist" mapstructure:"memberlist"` + } `json:"serf_lan,omitempty" hcl:"serf_lan" mapstructure:"serf_lan"` + + SerfWAN struct { + Memberlist struct { + GossipInterval *string `json:"gossip_interval,omitempty" hcl:"gossip_interval" mapstructure:"gossip_interval"` + ProbeInterval *string `json:"probe_interval,omitempty" hcl:"probe_interval" mapstructure:"probe_interval"` + ProbeTimeout *string `json:"probe_timeout,omitempty" hcl:"probe_timeout" mapstructure:"probe_timeout"` + SuspicionMult *int `json:"suspicion_mult,omitempty" hcl:"suspicion_mult" mapstructure:"suspicion_mult"` + } `json:"memberlist,omitempty" hcl:"memberlist" mapstructure:"memberlist"` + } `json:"serf_wan,omitempty" hcl:"serf_wan" mapstructure:"serf_wan"` + + Server struct { + HealthInterval *string `json:"health_interval,omitempty" hcl:"health_interval" mapstructure:"health_interval"` + } `json:"server,omitempty" hcl:"server" mapstructure:"server"` +} + +type Addresses struct { + DNS *string `json:"dns,omitempty" hcl:"dns" mapstructure:"dns"` + HTTP *string `json:"http,omitempty" hcl:"http" mapstructure:"http"` + HTTPS *string `json:"https,omitempty" hcl:"https" mapstructure:"https"` +} + +type AdvertiseAddrsConfig struct { + RPC *string `json:"rpc,omitempty" hcl:"rpc" mapstructure:"rpc"` + SerfLAN *string `json:"serf_lan,omitempty" hcl:"serf_lan" mapstructure:"serf_lan"` + SerfWAN *string `json:"serf_wan,omitempty" hcl:"serf_wan" mapstructure:"serf_wan"` +} + +type Autopilot struct { + CleanupDeadServers *bool `json:"cleanup_dead_servers,omitempty" hcl:"cleanup_dead_servers" mapstructure:"cleanup_dead_servers"` + DisableUpgradeMigration *bool `json:"disable_upgrade_migration,omitempty" hcl:"disable_upgrade_migration" mapstructure:"disable_upgrade_migration"` + LastContactThreshold *string `json:"last_contact_threshold,omitempty" hcl:"last_contact_threshold" mapstructure:"last_contact_threshold"` + MaxTrailingLogs *int `json:"max_trailing_logs,omitempty" hcl:"max_trailing_logs" mapstructure:"max_trailing_logs"` + RedundancyZoneTag *string `json:"redundancy_zone_tag,omitempty" hcl:"redundancy_zone_tag" mapstructure:"redundancy_zone_tag"` + ServerStabilizationTime *string `json:"server_stabilization_time,omitempty" hcl:"server_stabilization_time" mapstructure:"server_stabilization_time"` + UpgradeVersionTag *string `json:"upgrade_version_tag,omitempty" hcl:"upgrade_version_tag" mapstructure:"upgrade_version_tag"` +} + +type ServiceDefinition struct { + ID *string `json:"id,omitempty" hcl:"id" mapstructure:"id"` + Name *string `json:"name,omitempty" hcl:"name" mapstructure:"name"` + Tags []string `json:"tags,omitempty" hcl:"tags" mapstructure:"tags"` + Address *string `json:"address,omitempty" hcl:"address" mapstructure:"address"` + Port *int `json:"port,omitempty" hcl:"port" mapstructure:"port"` + Check *CheckDefinition `json:"check,omitempty" hcl:"check" mapstructure:"check"` + Checks []CheckDefinition `json:"checks,omitempty" hcl:"checks" mapstructure:"checks"` + Token *string `json:"token,omitempty" hcl:"token" mapstructure:"token"` + EnableTagOverride *bool `json:"enable_tag_override,omitempty" hcl:"enable_tag_override" mapstructure:"enable_tag_override"` +} + +type CheckDefinition struct { + ID *string `json:"id,omitempty" hcl:"id" mapstructure:"id"` + CheckID *string `json:"check_id,omitempty" hcl:"check_id" mapstructure:"check_id"` + Name *string `json:"name,omitempty" hcl:"name" mapstructure:"name"` + Notes *string `json:"notes,omitempty" hcl:"notes" mapstructure:"notes"` + ServiceID *string `json:"service_id,omitempty" hcl:"service_id" mapstructure:"service_id"` + Token *string `json:"token,omitempty" hcl:"token" mapstructure:"token"` + Status *string `json:"status,omitempty" hcl:"status" mapstructure:"status"` + Script *string `json:"script,omitempty" hcl:"script" mapstructure:"script"` + HTTP *string `json:"http,omitempty" hcl:"http" mapstructure:"http"` + Header map[string][]string `json:"header,omitempty" hcl:"header" mapstructure:"header"` + Method *string `json:"method,omitempty" hcl:"method" mapstructure:"method"` + TCP *string `json:"tcp,omitempty" hcl:"tcp" mapstructure:"tcp"` + Interval *string `json:"interval,omitempty" hcl:"interval" mapstructure:"interval"` + DockerContainerID *string `json:"docker_container_id,omitempty" hcl:"docker_container_id" mapstructure:"docker_container_id"` + Shell *string `json:"shell,omitempty" hcl:"shell" mapstructure:"shell"` + TLSSkipVerify *bool `json:"tls_skip_verify,omitempty" hcl:"tls_skip_verify" mapstructure:"tls_skip_verify"` + Timeout *string `json:"timeout,omitempty" hcl:"timeout" mapstructure:"timeout"` + TTL *string `json:"ttl,omitempty" hcl:"ttl" mapstructure:"ttl"` + DeregisterCriticalServiceAfter *string `json:"deregister_critical_service_after,omitempty" hcl:"deregister_critical_service_after" mapstructure:"deregister_critical_service_after"` +} + +type DNS struct { + AllowStale *bool `json:"allow_stale,omitempty" hcl:"allow_stale" mapstructure:"allow_stale"` + DisableCompression *bool `json:"disable_compression,omitempty" hcl:"disable_compression" mapstructure:"disable_compression"` + EnableTruncate *bool `json:"enable_truncate,omitempty" hcl:"enable_truncate" mapstructure:"enable_truncate"` + MaxStale *string `json:"max_stale,omitempty" hcl:"max_stale" mapstructure:"max_stale"` + NodeTTL *string `json:"node_ttl,omitempty" hcl:"node_ttl" mapstructure:"node_ttl"` + OnlyPassing *bool `json:"only_passing,omitempty" hcl:"only_passing" mapstructure:"only_passing"` + RecursorTimeout *string `json:"recursor_timeout,omitempty" hcl:"recursor_timeout" mapstructure:"recursor_timeout"` + ServiceTTL map[string]string `json:"service_ttl,omitempty" hcl:"service_ttl" mapstructure:"service_ttl"` + UDPAnswerLimit *int `json:"udp_answer_limit,omitempty" hcl:"udp_answer_limit" mapstructure:"udp_answer_limit"` +} + +type HTTPConfig struct { + BlockEndpoints []string `json:"block_endpoints,omitempty" hcl:"block_endpoints" mapstructure:"block_endpoints"` + ResponseHeaders map[string]string `json:"response_headers,omitempty" hcl:"response_headers" mapstructure:"response_headers"` +} + +type Performance struct { + RaftMultiplier *int `json:"raft_multiplier,omitempty" hcl:"raft_multiplier" mapstructure:"raft_multiplier"` // todo(fs): validate as uint +} + +type Telemetry struct { + CirconusAPIApp *string `json:"circonus_api_app,omitempty" hcl:"circonus_api_app" mapstructure:"circonus_api_app"` + CirconusAPIToken *string `json:"circonus_api_token,omitempty" json:"-" hcl:"circonus_api_token" mapstructure:"circonus_api_token" json:"-"` + CirconusAPIURL *string `json:"circonus_api_url,omitempty" hcl:"circonus_api_url" mapstructure:"circonus_api_url"` + CirconusBrokerID *string `json:"circonus_broker_id,omitempty" hcl:"circonus_broker_id" mapstructure:"circonus_broker_id"` + CirconusBrokerSelectTag *string `json:"circonus_broker_select_tag,omitempty" hcl:"circonus_broker_select_tag" mapstructure:"circonus_broker_select_tag"` + CirconusCheckDisplayName *string `json:"circonus_check_display_name,omitempty" hcl:"circonus_check_display_name" mapstructure:"circonus_check_display_name"` + CirconusCheckForceMetricActivation *string `json:"circonus_check_force_metric_activation,omitempty" hcl:"circonus_check_force_metric_activation" mapstructure:"circonus_check_force_metric_activation"` + CirconusCheckID *string `json:"circonus_check_id,omitempty" hcl:"circonus_check_id" mapstructure:"circonus_check_id"` + CirconusCheckInstanceID *string `json:"circonus_check_instance_id,omitempty" hcl:"circonus_check_instance_id" mapstructure:"circonus_check_instance_id"` + CirconusCheckSearchTag *string `json:"circonus_check_search_tag,omitempty" hcl:"circonus_check_search_tag" mapstructure:"circonus_check_search_tag"` + CirconusCheckTags *string `json:"circonus_check_tags,omitempty" hcl:"circonus_check_tags" mapstructure:"circonus_check_tags"` + CirconusSubmissionInterval *string `json:"circonus_submission_interval,omitempty" hcl:"circonus_submission_interval" mapstructure:"circonus_submission_interval"` + CirconusSubmissionURL *string `json:"circonus_submission_url,omitempty" hcl:"circonus_submission_url" mapstructure:"circonus_submission_url"` + DisableHostname *bool `json:"disable_hostname,omitempty" hcl:"disable_hostname" mapstructure:"disable_hostname"` + DogstatsdAddr *string `json:"dogstatsd_addr,omitempty" hcl:"dogstatsd_addr" mapstructure:"dogstatsd_addr"` + DogstatsdTags []string `json:"dogstatsd_tags,omitempty" hcl:"dogstatsd_tags" mapstructure:"dogstatsd_tags"` + FilterDefault *bool `json:"filter_default,omitempty" hcl:"filter_default" mapstructure:"filter_default"` + PrefixFilter []string `json:"prefix_filter,omitempty" hcl:"prefix_filter" mapstructure:"prefix_filter"` + StatsdAddr *string `json:"statsd_address,omitempty" hcl:"statsd_address" mapstructure:"statsd_address"` + StatsiteAddr *string `json:"statsite_address,omitempty" hcl:"statsite_address" mapstructure:"statsite_address"` + StatsitePrefix *string `json:"statsite_prefix,omitempty" hcl:"statsite_prefix" mapstructure:"statsite_prefix"` +} + +type Ports struct { + DNS *int `json:"dns,omitempty" hcl:"dns" mapstructure:"dns"` + HTTP *int `json:"http,omitempty" hcl:"http" mapstructure:"http"` + HTTPS *int `json:"https,omitempty" hcl:"https" mapstructure:"https"` + SerfLAN *int `json:"serf_lan,omitempty" hcl:"serf_lan" mapstructure:"serf_lan"` + SerfWAN *int `json:"serf_wan,omitempty" hcl:"serf_wan" mapstructure:"serf_wan"` + Server *int `json:"server,omitempty" hcl:"server" mapstructure:"server"` +} + +type RetryJoinAzure struct { + ClientID *string `json:"client_id,omitempty" hcl:"client_id" mapstructure:"client_id"` + SecretAccessKey *string `json:"secret_access_key,omitempty" hcl:"secret_access_key" mapstructure:"secret_access_key"` + SubscriptionID *string `json:"subscription_id,omitempty" hcl:"subscription_id" mapstructure:"subscription_id"` + TagName *string `json:"tag_name,omitempty" hcl:"tag_name" mapstructure:"tag_name"` + TagValue *string `json:"tag_value,omitempty" hcl:"tag_value" mapstructure:"tag_value"` + TenantID *string `json:"tenant_id,omitempty" hcl:"tenant_id" mapstructure:"tenant_id"` +} + +type RetryJoinEC2 struct { + AccessKeyID *string `json:"access_key_id,omitempty" hcl:"access_key_id" mapstructure:"access_key_id"` + Region *string `json:"region,omitempty" hcl:"region" mapstructure:"region"` + SecretAccessKey *string `json:"secret_access_key,omitempty" hcl:"secret_access_key" mapstructure:"secret_access_key"` + TagKey *string `json:"tag_key,omitempty" hcl:"tag_key" mapstructure:"tag_key"` + TagValue *string `json:"tag_value,omitempty" hcl:"tag_value" mapstructure:"tag_value"` +} + +type RetryJoinGCE struct { + CredentialsFile *string `json:"credentials_file,omitempty" hcl:"credentials_file" mapstructure:"credentials_file"` + ProjectName *string `json:"project_name,omitempty" hcl:"project_name" mapstructure:"project_name"` + TagValue *string `json:"tag_value,omitempty" hcl:"tag_value" mapstructure:"tag_value"` + ZonePattern *string `json:"zone_pattern,omitempty" hcl:"zone_pattern" mapstructure:"zone_pattern"` +} + +type UnixSocket struct { + Group *string `json:"group,omitempty" hcl:"group" mapstructure:"group"` + Mode *string `json:"mode,omitempty" hcl:"mode" mapstructure:"mode"` + User *string `json:"user,omitempty" hcl:"user" mapstructure:"user"` +} + +type Limits struct { + RPCMaxBurst *int `json:"rpc_max_burst,omitempty" hcl:"rpc_max_burst" mapstructure:"rpc_max_burst"` + RPCRate *float64 `json:"rpc_rate,omitempty" hcl:"rpc_rate" mapstructure:"rpc_rate"` +} + +type Segment struct { + Advertise *string `json:"advertise,omitempty" hcl:"advertise" mapstructure:"advertise"` + Bind *string `json:"bind,omitempty" hcl:"bind" mapstructure:"bind"` + Name *string `json:"name,omitempty" hcl:"name" mapstructure:"name"` + Port *int `json:"port,omitempty" hcl:"port" mapstructure:"port"` + RPCListener *bool `json:"rpc_listener,omitempty" hcl:"rpc_listener" mapstructure:"rpc_listener"` +} diff --git a/agent/config/default.go b/agent/config/default.go new file mode 100644 index 000000000..e9d8f2360 --- /dev/null +++ b/agent/config/default.go @@ -0,0 +1,243 @@ +package config + +import ( + "fmt" + "strconv" + + "github.com/hashicorp/consul/agent/consul" + "github.com/hashicorp/consul/version" +) + +func DefaultRPCProtocol() (int, error) { + src := DefaultSource() + c, err := Parse(src.Data, src.Format) + if err != nil { + return 0, fmt.Errorf("Error parsing default config: %s", err) + } + if c.RPCProtocol == nil { + return 0, fmt.Errorf("No default RPC protocol set") + } + return *c.RPCProtocol, nil +} + +// DefaultSource is the default agent configuration. +// This needs to be merged first in the head. +// todo(fs): The values are sourced from multiple sources. +// todo(fs): IMO, this should be the definitive default for all configurable values +// todo(fs): and whatever is in here should clobber every default value. Hence, no sourcing. +func DefaultSource() Source { + return Source{ + Name: "default", + Format: "hcl", + Data: ` + acl_default_policy = "allow" + acl_down_policy = "extend-cache" + acl_enforce_version_8 = true + acl_ttl = "30s" + bind_addr = "0.0.0.0" + bootstrap = false + bootstrap_expect = 0 + check_update_interval = "5m" + client_addr = "127.0.0.1" + datacenter = "` + consul.DefaultDC + `" + disable_coordinates = false + disable_host_node_id = true + disable_remote_exec = true + domain = "consul." + encrypt_verify_incoming = true + encrypt_verify_outgoing = true + log_level = "INFO" + protocol = 2 + retry_interval = "30s" + retry_interval_wan = "30s" + server = false + syslog_facility = "LOCAL0" + tls_min_version = "tls10" + + dns_config = { + allow_stale = true + udp_answer_limit = 3 + max_stale = "87600h" + recursor_timeout = "2s" + } + limits = { + rpc_rate = -1 + rpc_max_burst = 1000 + } + performance = { + raft_multiplier = ` + strconv.Itoa(int(consul.DefaultRaftMultiplier)) + ` + } + ports = { + dns = 8600 + http = 8500 + https = -1 + serf_lan = ` + strconv.Itoa(consul.DefaultLANSerfPort) + ` + serf_wan = ` + strconv.Itoa(consul.DefaultWANSerfPort) + ` + server = ` + strconv.Itoa(consul.DefaultRPCPort) + ` + } + telemetry = { + statsite_prefix = "consul" + filter_default = true + } + `, + } +} + +// DevSource is the additional default configuration for dev mode. +// This should be merged in the head after the default configuration. +func DevSource() Source { + return Source{ + Name: "dev", + Format: "hcl", + Data: ` + bind_addr = "127.0.0.1" + disable_anonymous_signature = true + disable_keyring_file = true + enable_debug = true + enable_ui = true + log_level = "DEBUG" + server = true + performance = { + raft_multiplier = 1 + } + `, + } +} + +// NonUserSource contains the values the user cannot configure. +// This needs to be merged in the tail. +func NonUserSource() Source { + return Source{ + Name: "non-user", + Format: "hcl", + Data: ` + acl_disabled_ttl = "120s" + check_deregister_interval_min = "1m" + check_reap_interval = "30s" + ae_interval = "1m" + sync_coordinate_rate_target = 64 + sync_coordinate_interval_min = "15s" + + # segment_limit is the maximum number of network segments that may be declared. + segment_limit = 64 + + # SegmentNameLimit is the maximum segment name length. + segment_name_limit = 64 + `, + } +} + +// VersionSource creates a config source for the version parameters. +// This should be merged in the tail since these values are not +// user configurable. +func VersionSource(rev, ver, verPre string) Source { + return Source{ + Name: "version", + Format: "hcl", + Data: fmt.Sprintf(`revision = %q version = %q version_prerelease = %q`, rev, ver, verPre), + } +} + +// DefaultVersionSource returns the version config source for the embedded +// version numbers. +func DefaultVersionSource() Source { + return VersionSource(version.GitCommit, version.Version, version.VersionPrerelease) +} + +// DefaultConsulSource returns the default configuration for the consul agent. +// This should be merged in the tail since these values are not user configurable. +func DefaultConsulSource() Source { + cfg := consul.DefaultConfig() + raft := cfg.RaftConfig + serfLAN := cfg.SerfLANConfig.MemberlistConfig + serfWAN := cfg.SerfWANConfig.MemberlistConfig + return Source{ + Name: "consul", + Format: "hcl", + Data: ` + consul = { + coordinate = { + update_batch_size = ` + strconv.Itoa(cfg.CoordinateUpdateBatchSize) + ` + update_max_batches = ` + strconv.Itoa(cfg.CoordinateUpdateMaxBatches) + ` + update_period = "` + cfg.CoordinateUpdatePeriod.String() + `" + } + raft = { + election_timeout = "` + raft.ElectionTimeout.String() + `" + heartbeat_timeout = "` + raft.HeartbeatTimeout.String() + `" + leader_lease_timeout = "` + raft.LeaderLeaseTimeout.String() + `" + } + serf_lan = { + memberlist = { + gossip_interval = "` + serfLAN.GossipInterval.String() + `" + probe_interval = "` + serfLAN.ProbeInterval.String() + `" + probe_timeout = "` + serfLAN.ProbeTimeout.String() + `" + suspicion_mult = ` + strconv.Itoa(serfLAN.SuspicionMult) + ` + } + } + serf_wan = { + memberlist = { + gossip_interval = "` + serfWAN.GossipInterval.String() + `" + probe_interval = "` + serfWAN.ProbeInterval.String() + `" + probe_timeout = "` + serfWAN.ProbeTimeout.String() + `" + suspicion_mult = ` + strconv.Itoa(serfWAN.SuspicionMult) + ` + } + } + server = { + health_interval = "` + cfg.ServerHealthInterval.String() + `" + } + } + `, + } +} + +// DevConsulSource returns the consul agent configuration for the dev mode. +// This should be merged in the tail after the DefaultConsulSource. +func DevConsulSource() Source { + return Source{ + Name: "consul-dev", + Format: "hcl", + Data: ` + consul = { + coordinate = { + update_period = "100ms" + } + raft = { + election_timeout = "52ms" + heartbeat_timeout = "35ms" + leader_lease_timeout = "20ms" + } + serf_lan = { + memberlist = { + gossip_interval = "100ms" + probe_interval = "100ms" + probe_timeout = "100ms" + suspicion_mult = 3 + } + } + serf_wan = { + memberlist = { + gossip_interval = "100ms" + probe_interval = "100ms" + probe_timeout = "100ms" + suspicion_mult = 3 + } + } + server = { + health_interval = "10ms" + } + } + `, + } +} + +func DefaultRuntimeConfig(hcl string) *RuntimeConfig { + b, err := NewBuilder(Flags{HCL: []string{hcl}}) + if err != nil { + panic(err) + } + rt, err := b.BuildAndValidate() + if err != nil { + panic(err) + } + return &rt +} diff --git a/agent/config/doc.go b/agent/config/doc.go new file mode 100644 index 000000000..bea218c92 --- /dev/null +++ b/agent/config/doc.go @@ -0,0 +1,81 @@ +// Package config contains the command line and config file code for the +// consul agent. +// +// The consul agent configuration is generated from multiple sources: +// +// * config files +// * environment variables (which?) +// * cmd line args +// +// Each of these argument sets needs to be parsed, validated and then +// merged with the other sources to build the final configuration. +// +// This patch introduces a distinction between the user and the runtime +// configuration. The user configuration defines the external interface for +// the user, i.e. the command line flags, the environment variables and the +// config file format which cannot be changed without breaking the users' +// setup. +// +// The runtime configuration is the merged, validated and mangled +// configuration structure suitable for the consul agent. Both structures +// are similar but different and the runtime configuration can be +// refactored at will without affecting the user configuration format. +// +// For this, the user configuration consists of several structures for +// config files and command line arguments. Again, the config file and +// command line structs are similar but not identical for historical +// reasons and to allow evolving them differently. +// +// All of the user configuration structs have pointer values to +// unambiguously merge values from several sources into the final value. +// +// The runtime configuration has no pointer values and should be passed by +// value to avoid accidental or malicious runtime configuration changes. +// Runtime updates need to be handled through a new configuration +// instances. + +// # Removed command line flags +// +// * "-atlas" is deprecated and is no longer used. Please remove it from your configuration. +// * "-atlas-token" is deprecated and is no longer used. Please remove it from your configuration. +// * "-atlas-join" is deprecated and is no longer used. Please remove it from your configuration. +// * "-atlas-endpoint" is deprecated and is no longer used. Please remove it from your configuration. +// * "-dc" is deprecated. Please use "-datacenter" instead +// * "-retry-join-azure-tag-name" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-azure-tag-value" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-ec2-region" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-ec2-tag-key" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-ec2-tag-value" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-gce-credentials-file" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-gce-project-name" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-gce-tag-name" is deprecated. Please use "-retry-join" instead. +// * "-retry-join-gce-zone-pattern" is deprecated. Please use "-retry-join" instead. +// +// # Removed configuration fields +// +// * "addresses.rpc" is deprecated and is no longer used. Please remove it from your configuration. +// * "ports.rpc" is deprecated and is no longer used. Please remove it from your configuration. +// * "atlas_infrastructure" is deprecated and is no longer used. Please remove it from your configuration. +// * "atlas_token" is deprecated and is no longer used. Please remove it from your configuration. +// * "atlas_acl_token" is deprecated and is no longer used. Please remove it from your configuration. +// * "atlas_join" is deprecated and is no longer used. Please remove it from your configuration. +// * "atlas_endpoint" is deprecated and is no longer used. Please remove it from your configuration. +// * "http_api_response_headers" is deprecated. Please use "http_config.response_headers" instead. +// * "dogstatsd_addr" is deprecated. Please use "telemetry.dogstatsd_addr" instead. +// * "dogstatsd_tags" is deprecated. Please use "telemetry.dogstatsd_tags" instead. +// * "recursor" is deprecated. Please use "recursors" instead. +// * "statsd_addr" is deprecated. Please use "telemetry.statsd_addr" instead. +// * "statsite_addr" is deprecated. Please use "telemetry.statsite_addr" instead. +// * "statsite_prefix" is deprecated. Please use "telemetry.statsite_prefix" instead. +// * "retry_join_azure" is deprecated. Please use "retry_join" instead. +// * "retry_join_ec2" is deprecated. Please use "retry_join" instead. +// * "retry_join_gce" is deprecated. Please use "retry_join" instead. +// +// # Removed service config alias fields +// +// * "serviceid" is deprecated in service definitions. Please use "service_id" instead. +// * "dockercontainerid" is deprecated in service definitions. Please use "docker_container_id" instead. +// * "tlsskipverify" is deprecated in service definitions. Please use "tls_skip_verify" instead. +// * "deregistercriticalserviceafter" is deprecated in service definitions. Please use "deregister_critical_service_after" instead. + +package config diff --git a/agent/config/flags.go b/agent/config/flags.go new file mode 100644 index 000000000..6c73beeb7 --- /dev/null +++ b/agent/config/flags.go @@ -0,0 +1,104 @@ +package config + +import ( + "flag" + "fmt" + "time" +) + +// Flags defines the command line flags. +type Flags struct { + // Config contains the command line arguments that can also be set + // in a config file. + Config Config + + // ConfigFiles contains the list of config files and directories + // that should be read. + ConfigFiles []string + + // DevMode indicates whether the agent should be started in development + // mode. This cannot be configured in a config file. + DevMode *bool + + // HCL contains an arbitrary config in hcl format. + HCL []string +} + +// ParseFlag parses the arguments into a Flags struct. +func ParseFlags(args []string) (Flags, error) { + var f Flags + fs := flag.NewFlagSet("agent", flag.ContinueOnError) + AddFlags(fs, &f) + if err := fs.Parse(args); err != nil { + return Flags{}, err + } + return f, nil +} + +// AddFlags adds the command line flags for the agent. +func AddFlags(fs *flag.FlagSet, f *Flags) { + add := func(p interface{}, name, help string) { + switch x := p.(type) { + case **bool: + fs.Var(newBoolPtrValue(x), name, help) + case **time.Duration: + fs.Var(newDurationPtrValue(x), name, help) + case **int: + fs.Var(newIntPtrValue(x), name, help) + case **string: + fs.Var(newStringPtrValue(x), name, help) + case *[]string: + fs.Var(newStringSliceValue(x), name, help) + case *map[string]string: + fs.Var(newStringMapValue(x), name, help) + default: + panic(fmt.Sprintf("invalid type: %T", p)) + } + } + + // command line flags ordered by flag name + add(&f.Config.AdvertiseAddrLAN, "advertise", "Sets the advertise address to use.") + add(&f.Config.AdvertiseAddrWAN, "advertise-wan", "Sets address to advertise on WAN instead of -advertise address.") + add(&f.Config.BindAddr, "bind", "Sets the bind address for cluster communication.") + add(&f.Config.Bootstrap, "bootstrap", "Sets server to bootstrap mode.") + add(&f.Config.BootstrapExpect, "bootstrap-expect", "Sets server to expect bootstrap mode.") + add(&f.Config.ClientAddr, "client", "Sets the address to bind for client access. This includes RPC, DNS, HTTP and HTTPS (if configured).") + add(&f.ConfigFiles, "config-dir", "Path to a directory to read configuration files from. This will read every file ending in '.json' as configuration in this directory in alphabetical order. Can be specified multiple times.") + add(&f.ConfigFiles, "config-file", "Path to a JSON file to read configuration from. Can be specified multiple times.") + add(&f.Config.DataDir, "data-dir", "Path to a data directory to store agent state.") + add(&f.Config.Datacenter, "datacenter", "Datacenter of the agent.") + add(&f.DevMode, "dev", "Starts the agent in development mode.") + add(&f.Config.DisableHostNodeID, "disable-host-node-id", "Setting this to true will prevent Consul from using information from the host to generate a node ID, and will cause Consul to generate a random node ID instead.") + add(&f.Config.DisableKeyringFile, "disable-keyring-file", "Disables the backing up of the keyring to a file.") + add(&f.Config.Ports.DNS, "dns-port", "DNS port to use.") + add(&f.Config.DNSDomain, "domain", "Domain to use for DNS interface.") + add(&f.Config.EnableScriptChecks, "enable-script-checks", "Enables health check scripts.") + add(&f.Config.EncryptKey, "encrypt", "Provides the gossip encryption key.") + add(&f.Config.Ports.HTTP, "http-port", "Sets the HTTP API port to listen on.") + add(&f.Config.StartJoinAddrsLAN, "join", "Address of an agent to join at start time. Can be specified multiple times.") + add(&f.Config.StartJoinAddrsWAN, "join-wan", "Address of an agent to join -wan at start time. Can be specified multiple times.") + add(&f.Config.LogLevel, "log-level", "Log level of the agent.") + add(&f.Config.NodeName, "node", "Name of this node. Must be unique in the cluster.") + add(&f.Config.NodeID, "node-id", "A unique ID for this node across space and time. Defaults to a randomly-generated ID that persists in the data-dir.") + add(&f.Config.NodeMeta, "node-meta", "An arbitrary metadata key/value pair for this node, of the format `key:value`. Can be specified multiple times.") + add(&f.Config.NonVotingServer, "non-voting-server", "(Enterprise-only) This flag is used to make the server not participate in the Raft quorum, and have it only receive the data replication stream. This can be used to add read scalability to a cluster in cases where a high volume of reads to servers are needed.") + add(&f.Config.PidFile, "pid-file", "Path to file to store agent PID.") + add(&f.Config.RPCProtocol, "protocol", "Sets the protocol version. Defaults to latest.") + add(&f.Config.RaftProtocol, "raft-protocol", "Sets the Raft protocol version. Defaults to latest.") + add(&f.Config.DNSRecursors, "recursor", "Address of an upstream DNS server. Can be specified multiple times.") + add(&f.Config.RejoinAfterLeave, "rejoin", "Ignores a previous leave and attempts to rejoin the cluster.") + add(&f.Config.RetryJoinIntervalLAN, "retry-interval", "Time to wait between join attempts.") + add(&f.Config.RetryJoinIntervalWAN, "retry-interval-wan", "Time to wait between join -wan attempts.") + add(&f.Config.RetryJoinLAN, "retry-join", "Address of an agent to join at start time with retries enabled. Can be specified multiple times.") + add(&f.Config.RetryJoinWAN, "retry-join-wan", "Address of an agent to join -wan at start time with retries enabled. Can be specified multiple times.") + add(&f.Config.RetryJoinMaxAttemptsLAN, "retry-max", "Maximum number of join attempts. Defaults to 0, which will retry indefinitely.") + add(&f.Config.RetryJoinMaxAttemptsWAN, "retry-max-wan", "Maximum number of join -wan attempts. Defaults to 0, which will retry indefinitely.") + add(&f.Config.SerfBindAddrLAN, "serf-lan-bind", "Address to bind Serf LAN listeners to.") + add(&f.Config.SegmentName, "segment", "(Enterprise-only) Sets the network segment to join.") + add(&f.Config.SerfBindAddrWAN, "serf-wan-bind", "Address to bind Serf WAN listeners to.") + add(&f.Config.ServerMode, "server", "Switches agent to server mode.") + add(&f.Config.EnableSyslog, "syslog", "Enables logging to syslog.") + add(&f.Config.EnableUI, "ui", "Enables the built-in static web UI server.") + add(&f.Config.UIDir, "ui-dir", "Path to directory containing the web UI resources.") + add(&f.HCL, "hcl", "hcl config fragment. Can be specified multiple times.") +} diff --git a/agent/config/flags_test.go b/agent/config/flags_test.go new file mode 100644 index 000000000..87aca32cc --- /dev/null +++ b/agent/config/flags_test.go @@ -0,0 +1,75 @@ +package config + +import ( + "reflect" + "strings" + "testing" + + "github.com/pascaldekloe/goe/verify" +) + +// TestParseFlags tests whether command line flags are properly parsed +// into the Flags/File structure. It contains an example for every type +// that is parsed. It does not test the conversion into the final +// runtime configuration. See TestConfig for that. +func TestParseFlags(t *testing.T) { + tests := []struct { + args []string + flags Flags + err error + }{ + {}, + { + args: []string{`-bind`, `a`}, + flags: Flags{Config: Config{BindAddr: pString("a")}}, + }, + { + args: []string{`-bootstrap`}, + flags: Flags{Config: Config{Bootstrap: pBool(true)}}, + }, + { + args: []string{`-bootstrap=true`}, + flags: Flags{Config: Config{Bootstrap: pBool(true)}}, + }, + { + args: []string{`-bootstrap=false`}, + flags: Flags{Config: Config{Bootstrap: pBool(false)}}, + }, + { + args: []string{`-bootstrap`, `true`}, + flags: Flags{Config: Config{Bootstrap: pBool(true)}}, + }, + { + args: []string{`-config-file`, `a`, `-config-dir`, `b`, `-config-file`, `c`, `-config-dir`, `d`}, + flags: Flags{ConfigFiles: []string{"a", "b", "c", "d"}}, + }, + { + args: []string{`-datacenter`, `a`}, + flags: Flags{Config: Config{Datacenter: pString("a")}}, + }, + { + args: []string{`-dns-port`, `1`}, + flags: Flags{Config: Config{Ports: Ports{DNS: pInt(1)}}}, + }, + { + args: []string{`-join`, `a`, `-join`, `b`}, + flags: Flags{Config: Config{StartJoinAddrsLAN: []string{"a", "b"}}}, + }, + { + args: []string{`-node-meta`, `a:b`, `-node-meta`, `c:d`}, + flags: Flags{Config: Config{NodeMeta: map[string]string{"a": "b", "c": "d"}}}, + }, + } + + for _, tt := range tests { + t.Run(strings.Join(tt.args, " "), func(t *testing.T) { + flags, err := ParseFlags(tt.args) + if got, want := err, tt.err; !reflect.DeepEqual(got, want) { + t.Fatalf("got error %v want %v", got, want) + } + if !verify.Values(t, "flag", flags, tt.flags) { + t.FailNow() + } + }) + } +} diff --git a/agent/config/flagset.go b/agent/config/flagset.go new file mode 100644 index 000000000..2b56dfd42 --- /dev/null +++ b/agent/config/flagset.go @@ -0,0 +1,200 @@ +package config + +import ( + "strconv" + "strings" + "time" +) + +// boolPtrValue is a flag.Value which stores the value in a *bool if it +// can be parsed with strconv.ParseBool. If the value was not set the +// pointer is nil. +type boolPtrValue struct { + v **bool + b bool +} + +func newBoolPtrValue(p **bool) *boolPtrValue { + return &boolPtrValue{p, false} +} + +func (s *boolPtrValue) IsBoolFlag() bool { return true } + +func (s *boolPtrValue) Set(val string) error { + b, err := strconv.ParseBool(val) + if err != nil { + return err + } + *s.v, s.b = &b, true + return nil +} + +func (s *boolPtrValue) Get() interface{} { + if s.b { + return *s.v + } + return (*bool)(nil) +} + +func (s *boolPtrValue) String() string { + if s.b { + return strconv.FormatBool(**s.v) + } + return "" +} + +// durationPtrValue is a flag.Value which stores the value in a +// *time.Duration if it can be parsed with time.ParseDuration. If the +// value was not set the pointer is nil. +type durationPtrValue struct { + v **time.Duration + b bool +} + +func newDurationPtrValue(p **time.Duration) *durationPtrValue { + return &durationPtrValue{p, false} +} + +func (s *durationPtrValue) Set(val string) error { + d, err := time.ParseDuration(val) + if err != nil { + return err + } + *s.v, s.b = &d, true + return nil +} + +func (s *durationPtrValue) Get() interface{} { + if s.b { + return *s.v + } + return (*time.Duration)(nil) +} + +func (s *durationPtrValue) String() string { + if s.b { + return (*(*s).v).String() + } + return "" +} + +// intPtrValue is a flag.Value which stores the value in a *int if it +// can be parsed with strconv.Atoi. If the value was not set the pointer +// is nil. +type intPtrValue struct { + v **int + b bool +} + +func newIntPtrValue(p **int) *intPtrValue { + return &intPtrValue{p, false} +} + +func (s *intPtrValue) Set(val string) error { + n, err := strconv.Atoi(val) + if err != nil { + return err + } + *s.v, s.b = &n, true + return nil +} + +func (s *intPtrValue) Get() interface{} { + if s.b { + return *s.v + } + return (*int)(nil) +} + +func (s *intPtrValue) String() string { + if s.b { + return strconv.Itoa(**s.v) + } + return "" +} + +// stringMapValue is a flag.Value which stores the value in a map[string]string if the +// value is in "key:value" format. This can be specified multiple times. +type stringMapValue map[string]string + +func newStringMapValue(p *map[string]string) *stringMapValue { + *p = map[string]string{} + return (*stringMapValue)(p) +} + +func (s *stringMapValue) Set(val string) error { + p := strings.SplitN(val, ":", 2) + k, v := p[0], "" + if len(p) == 2 { + v = p[1] + } + (*s)[k] = v + return nil +} + +func (s *stringMapValue) Get() interface{} { + return s +} + +func (s *stringMapValue) String() string { + var x []string + for k, v := range *s { + if v == "" { + x = append(x, k) + } else { + x = append(x, k+":"+v) + } + } + return strings.Join(x, " ") +} + +// stringPtrValue is a flag.Value which stores the value in a *string. +// If the value was not set the pointer is nil. +type stringPtrValue struct { + v **string + b bool +} + +func newStringPtrValue(p **string) *stringPtrValue { + return &stringPtrValue{p, false} +} + +func (s *stringPtrValue) Set(val string) error { + *s.v, s.b = &val, true + return nil +} + +func (s *stringPtrValue) Get() interface{} { + if s.b { + return *s.v + } + return (*string)(nil) +} + +func (s *stringPtrValue) String() string { + if s.b { + return **s.v + } + return "" +} + +// stringSliceValue is a flag.Value which appends the value to a []string. +// This can be specified multiple times. +type stringSliceValue []string + +func newStringSliceValue(p *[]string) *stringSliceValue { + return (*stringSliceValue)(p) +} + +func (s *stringSliceValue) Set(val string) error { + *s = append(*s, val) + return nil +} + +func (s *stringSliceValue) Get() interface{} { + return s +} + +func (s *stringSliceValue) String() string { + return strings.Join(*s, " ") +} diff --git a/agent/config/merge.go b/agent/config/merge.go new file mode 100644 index 000000000..68bb22e6d --- /dev/null +++ b/agent/config/merge.go @@ -0,0 +1,59 @@ +package config + +import ( + "fmt" + "reflect" +) + +// Merge recursively combines a set of config file structures into a single structure +// according to the following rules: +// +// * only values of type struct, slice, map and pointer to simple types are allowed. Other types panic. +// * when merging two structs the result is the recursive merge of all fields according to the rules below +// * when merging two slices the result is the second slice appended to the first +// * when merging two maps the result is the second map if it is not empty, otherwise the first +// * when merging two pointer values the result is the second value if it is not nil, otherwise the first +func Merge(files ...Config) Config { + var a Config + for _, b := range files { + a = merge(a, b).(Config) + } + return a +} + +func merge(a, b interface{}) interface{} { + return mergeValue(reflect.ValueOf(a), reflect.ValueOf(b)).Interface() +} + +func mergeValue(a, b reflect.Value) reflect.Value { + switch a.Kind() { + case reflect.Map: + if b.Len() > 0 { + return b + } + return a + + case reflect.Ptr: + if !b.IsNil() { + return b + } + return a + + case reflect.Slice: + if !a.IsValid() { + a = reflect.Zero(a.Type()) + } + return reflect.AppendSlice(a, b) + + case reflect.Struct: + r := reflect.New(a.Type()) // &struct{} + for i := 0; i < a.NumField(); i++ { + v := mergeValue(a.Field(i), b.Field(i)) + r.Elem().Field(i).Set(v) + } + return r.Elem() // *struct + + default: + panic(fmt.Sprintf("unsupported element type: %v", a.Type())) + } +} diff --git a/agent/config/merge_test.go b/agent/config/merge_test.go new file mode 100644 index 000000000..94a1a22a3 --- /dev/null +++ b/agent/config/merge_test.go @@ -0,0 +1,57 @@ +package config + +import ( + "testing" + "time" + + "github.com/pascaldekloe/goe/verify" +) + +func TestMerge(t *testing.T) { + tests := []struct { + desc string + cfgs []Config + want Config + }{ + { + "top level fields", + []Config{ + {AdvertiseAddrLAN: pString("a")}, + {AdvertiseAddrLAN: pString("b")}, + {RaftProtocol: pInt(1)}, + {RaftProtocol: pInt(2)}, + {ServerMode: pBool(false)}, + {ServerMode: pBool(true)}, + {StartJoinAddrsLAN: []string{"a"}}, + {StartJoinAddrsLAN: []string{"b"}}, + {NodeMeta: map[string]string{"a": "b"}}, + {NodeMeta: map[string]string{"c": "d"}}, + {Ports: Ports{DNS: pInt(1)}}, + {Ports: Ports{DNS: pInt(2), HTTP: pInt(3)}}, + }, + Config{ + AdvertiseAddrLAN: pString("b"), + RaftProtocol: pInt(2), + ServerMode: pBool(true), + StartJoinAddrsLAN: []string{"a", "b"}, + NodeMeta: map[string]string{"c": "d"}, + Ports: Ports{DNS: pInt(2), HTTP: pInt(3)}, + }, + }, + } + + for _, tt := range tests { + t.Run(tt.desc, func(t *testing.T) { + got, want := Merge(tt.cfgs...), tt.want + if !verify.Values(t, "", got, want) { + t.FailNow() + } + }) + } +} + +func pBool(v bool) *bool { return &v } +func pInt(v int) *int { return &v } +func pString(v string) *string { return &v } +func pDuration(v time.Duration) *string { s := v.String(); return &s } +func pFloat64(v float64) *float64 { return &v } diff --git a/agent/config/patch_hcl.go b/agent/config/patch_hcl.go new file mode 100644 index 000000000..53000a977 --- /dev/null +++ b/agent/config/patch_hcl.go @@ -0,0 +1,73 @@ +package config + +import ( + "fmt" +) + +func patchSliceOfMaps(m map[string]interface{}, skip []string) map[string]interface{} { + return patchValue("", m, skip).(map[string]interface{}) +} + +func patchValue(name string, v interface{}, skip []string) interface{} { + // fmt.Printf("%q: %T\n", name, v) + switch x := v.(type) { + case map[string]interface{}: + if len(x) == 0 { + return x + } + mm := make(map[string]interface{}) + for k, v := range x { + key := k + if name != "" { + key = name + "." + k + } + mm[k] = patchValue(key, v, skip) + } + return mm + + case []interface{}: + if len(x) == 0 { + return nil + } + if strSliceContains(name, skip) { + for i, y := range x { + x[i] = patchValue(name, y, skip) + } + return x + } + if _, ok := x[0].(map[string]interface{}); !ok { + return x + } + if len(x) > 1 { + panic(fmt.Sprintf("%s: []map[string]interface{} with more than one element not supported: %s", name, v)) + } + return patchValue(name, x[0], skip) + + case []map[string]interface{}: + if len(x) == 0 { + return nil + } + if strSliceContains(name, skip) { + for i, y := range x { + x[i] = patchValue(name, y, skip).(map[string]interface{}) + } + return x + } + if len(x) > 1 { + panic(fmt.Sprintf("%s: []map[string]interface{} with more than one element not supported: %s", name, v)) + } + return patchValue(name, x[0], skip) + + default: + return v + } +} + +func strSliceContains(s string, v []string) bool { + for _, vv := range v { + if s == vv { + return true + } + } + return false +} diff --git a/agent/config/patch_hcl_test.go b/agent/config/patch_hcl_test.go new file mode 100644 index 000000000..976fb89dd --- /dev/null +++ b/agent/config/patch_hcl_test.go @@ -0,0 +1,78 @@ +package config + +import ( + "encoding/json" + "fmt" + "reflect" + "testing" +) + +func parse(s string) map[string]interface{} { + var m map[string]interface{} + if err := json.Unmarshal([]byte(s), &m); err != nil { + panic(s + ":" + err.Error()) + } + return m +} + +func TestPatchSliceOfMaps(t *testing.T) { + tests := []struct { + in, out string + skip []string + }{ + { + in: `{"a":{"b":"c"}}`, + out: `{"a":{"b":"c"}}`, + }, + { + in: `{"a":[{"b":"c"}]}`, + out: `{"a":{"b":"c"}}`, + }, + { + in: `{"a":[{"b":[{"c":"d"}]}]}`, + out: `{"a":{"b":{"c":"d"}}}`, + }, + { + in: `{"a":[{"b":"c"}]}`, + out: `{"a":[{"b":"c"}]}`, + skip: []string{"a"}, + }, + { + in: `{ + "services": [ + { + "checks": [ + { + "header": [ + {"a":"b"} + ] + } + ] + } + ] + }`, + out: `{ + "services": [ + { + "checks": [ + { + "header": {"a":"b"} + } + ] + } + ] + }`, + skip: []string{"services", "services.checks"}, + }, + } + + for i, tt := range tests { + desc := fmt.Sprintf("%02d: %s -> %s skip: %v", i, tt.in, tt.out, tt.skip) + t.Run(desc, func(t *testing.T) { + out := patchSliceOfMaps(parse(tt.in), tt.skip) + if got, want := out, parse(tt.out); !reflect.DeepEqual(got, want) { + t.Fatalf("\ngot %#v\nwant %#v", got, want) + } + }) + } +} diff --git a/agent/config/runtime.go b/agent/config/runtime.go new file mode 100644 index 000000000..6248bc910 --- /dev/null +++ b/agent/config/runtime.go @@ -0,0 +1,258 @@ +package config + +import ( + "crypto/tls" + "net" + "reflect" + "strings" + "time" + + "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/tlsutil" + "github.com/hashicorp/consul/types" + "golang.org/x/time/rate" +) + +// RuntimeConfig specifies the configuration the consul agent actually +// uses. Is is derived from one or more Config structures which can come +// from files, flags and/or environment variables. +type RuntimeConfig struct { + // non-user configurable values + AEInterval time.Duration + ACLDisabledTTL time.Duration + CheckDeregisterIntervalMin time.Duration + CheckReapInterval time.Duration + SegmentLimit int + SegmentNameLimit int + SyncCoordinateRateTarget float64 + SyncCoordinateIntervalMin time.Duration + Revision string + Version string + VersionPrerelease string + + // consul config + ConsulCoordinateUpdateMaxBatches int + ConsulCoordinateUpdateBatchSize int + ConsulCoordinateUpdatePeriod time.Duration + ConsulRaftElectionTimeout time.Duration + ConsulRaftHeartbeatTimeout time.Duration + ConsulRaftLeaderLeaseTimeout time.Duration + ConsulSerfLANGossipInterval time.Duration + ConsulSerfLANProbeInterval time.Duration + ConsulSerfLANProbeTimeout time.Duration + ConsulSerfLANSuspicionMult int + ConsulSerfWANGossipInterval time.Duration + ConsulSerfWANProbeInterval time.Duration + ConsulSerfWANProbeTimeout time.Duration + ConsulSerfWANSuspicionMult int + ConsulServerHealthInterval time.Duration + + ACLAgentMasterToken string + ACLAgentToken string + ACLDatacenter string + ACLDefaultPolicy string + ACLDownPolicy string + ACLEnforceVersion8 bool + ACLMasterToken string + ACLReplicationToken string + ACLTTL time.Duration + ACLToken string + + AutopilotCleanupDeadServers bool + AutopilotDisableUpgradeMigration bool + AutopilotLastContactThreshold time.Duration + AutopilotMaxTrailingLogs int + AutopilotRedundancyZoneTag string + AutopilotServerStabilizationTime time.Duration + AutopilotUpgradeVersionTag string + + DNSAllowStale bool + DNSDisableCompression bool + DNSDomain string + DNSEnableTruncate bool + DNSMaxStale time.Duration + DNSNodeTTL time.Duration + DNSOnlyPassing bool + DNSRecursorTimeout time.Duration + DNSServiceTTL map[string]time.Duration + DNSUDPAnswerLimit int + DNSRecursors []string + + HTTPBlockEndpoints []string + HTTPResponseHeaders map[string]string + + TelemetryCirconusAPIApp string + TelemetryCirconusAPIToken string + TelemetryCirconusAPIURL string + TelemetryCirconusBrokerID string + TelemetryCirconusBrokerSelectTag string + TelemetryCirconusCheckDisplayName string + TelemetryCirconusCheckForceMetricActivation string + TelemetryCirconusCheckID string + TelemetryCirconusCheckInstanceID string + TelemetryCirconusCheckSearchTag string + TelemetryCirconusCheckTags string + TelemetryCirconusSubmissionInterval string + TelemetryCirconusSubmissionURL string + TelemetryDisableHostname bool + TelemetryDogstatsdAddr string + TelemetryDogstatsdTags []string + TelemetryFilterDefault bool + TelemetryAllowedPrefixes []string + TelemetryBlockedPrefixes []string + TelemetryStatsdAddr string + TelemetryStatsiteAddr string + TelemetryStatsitePrefix string + + AdvertiseAddrLAN *net.IPAddr + AdvertiseAddrWAN *net.IPAddr + BindAddr *net.IPAddr + Bootstrap bool + BootstrapExpect int + CAFile string + CAPath string + CertFile string + CheckUpdateInterval time.Duration + Checks []*structs.CheckDefinition + ClientAddrs []*net.IPAddr + DNSAddrs []net.Addr + DNSPort int + DataDir string + Datacenter string + DevMode bool + DisableAnonymousSignature bool + DisableCoordinates bool + DisableHostNodeID bool + DisableKeyringFile bool + DisableRemoteExec bool + DisableUpdateCheck bool + EnableACLReplication bool + EnableDebug bool + EnableScriptChecks bool + EnableSyslog bool + EnableUI bool + EncryptKey string + EncryptVerifyIncoming bool + EncryptVerifyOutgoing bool + HTTPAddrs []net.Addr + HTTPPort int + HTTPSAddrs []net.Addr + HTTPSPort int + KeyFile string + LeaveOnTerm bool + LogLevel string + NodeID types.NodeID + NodeMeta map[string]string + NodeName string + NonVotingServer bool + PidFile string + RPCAdvertiseAddr *net.TCPAddr + RPCBindAddr *net.TCPAddr + RPCMaxBurst int + RPCProtocol int + RPCRateLimit rate.Limit + RaftProtocol int + ReconnectTimeoutLAN time.Duration + ReconnectTimeoutWAN time.Duration + RejoinAfterLeave bool + RetryJoinIntervalLAN time.Duration + RetryJoinIntervalWAN time.Duration + RetryJoinLAN []string + RetryJoinMaxAttemptsLAN int + RetryJoinMaxAttemptsWAN int + RetryJoinWAN []string + SegmentName string + Segments []structs.NetworkSegment + SerfAdvertiseAddrLAN *net.TCPAddr + SerfAdvertiseAddrWAN *net.TCPAddr + SerfBindAddrLAN *net.TCPAddr + SerfBindAddrWAN *net.TCPAddr + SerfPortLAN int + SerfPortWAN int + ServerMode bool + ServerName string + ServerPort int + Services []*structs.ServiceDefinition + SessionTTLMin time.Duration + SkipLeaveOnInt bool + StartJoinAddrsLAN []string + StartJoinAddrsWAN []string + SyslogFacility string + TLSCipherSuites []uint16 + TLSMinVersion string + TLSPreferServerCipherSuites bool + TaggedAddresses map[string]string + TranslateWANAddrs bool + UIDir string + UnixSocketGroup string + UnixSocketMode string + UnixSocketUser string + VerifyIncoming bool + VerifyIncomingHTTPS bool + VerifyIncomingRPC bool + VerifyOutgoing bool + VerifyServerHostname bool + Watches []map[string]interface{} +} + +// IncomingHTTPSConfig returns the TLS configuration for HTTPS +// connections to consul. +func (c *RuntimeConfig) IncomingHTTPSConfig() (*tls.Config, error) { + tc := &tlsutil.Config{ + VerifyIncoming: c.VerifyIncoming || c.VerifyIncomingHTTPS, + VerifyOutgoing: c.VerifyOutgoing, + CAFile: c.CAFile, + CAPath: c.CAPath, + CertFile: c.CertFile, + KeyFile: c.KeyFile, + NodeName: c.NodeName, + ServerName: c.ServerName, + TLSMinVersion: c.TLSMinVersion, + CipherSuites: c.TLSCipherSuites, + PreferServerCipherSuites: c.TLSPreferServerCipherSuites, + } + return tc.IncomingTLSConfig() +} + +func (c *RuntimeConfig) Sanitized() RuntimeConfig { + isSecret := func(name string) bool { + name = strings.ToLower(name) + return strings.Contains(name, "key") || strings.Contains(name, "token") || strings.Contains(name, "secret") + } + + cleanRetryJoin := func(a []string) (b []string) { + for _, line := range a { + var fields []string + for _, f := range strings.Fields(line) { + if isSecret(f) { + kv := strings.SplitN(f, "=", 2) + fields = append(fields, kv[0]+"=hidden") + } else { + fields = append(fields, f) + } + } + b = append(b, strings.Join(fields, " ")) + } + return b + } + + // sanitize all fields with secrets + typ := reflect.TypeOf(RuntimeConfig{}) + rawval := reflect.ValueOf(*c) + sanval := reflect.New(typ) // *RuntimeConfig + for i := 0; i < typ.NumField(); i++ { + f := typ.Field(i) + if f.Type.Kind() == reflect.String && isSecret(f.Name) { + sanval.Elem().Field(i).Set(reflect.ValueOf("hidden")) + } else { + sanval.Elem().Field(i).Set(rawval.Field(i)) + } + } + san := sanval.Elem().Interface().(RuntimeConfig) + + // sanitize retry-join config strings + san.RetryJoinLAN = cleanRetryJoin(san.RetryJoinLAN) + san.RetryJoinWAN = cleanRetryJoin(san.RetryJoinWAN) + + return san +} diff --git a/agent/config/runtime_test.go b/agent/config/runtime_test.go new file mode 100644 index 000000000..7d2759fd9 --- /dev/null +++ b/agent/config/runtime_test.go @@ -0,0 +1,3560 @@ +package config + +import ( + "bytes" + "crypto/tls" + "encoding/base64" + "errors" + "flag" + "fmt" + "io/ioutil" + "net" + "os" + "path/filepath" + "reflect" + "strconv" + "strings" + "testing" + "time" + + "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/testutil" + "github.com/hashicorp/consul/types" + "github.com/pascaldekloe/goe/verify" +) + +type configTest struct { + desc string + flags []string + pre, post func() + json, jsontail []string + hcl, hcltail []string + privatev4 func() ([]*net.IPAddr, error) + publicv6 func() ([]*net.IPAddr, error) + patch func(rt *RuntimeConfig) + err string + warns []string + hostname func() (string, error) +} + +// TestConfigFlagsAndEdgecases tests the command line flags and +// edgecases for the config parsing. It provides a test structure which +// checks for warnings on deprecated fields and flags. These tests +// should check one option at a time if possible and should use generic +// values, e.g. 'a' or 1 instead of 'servicex' or 3306. + +func TestConfigFlagsAndEdgecases(t *testing.T) { + dataDir := testutil.TempDir(t, "consul") + defer os.RemoveAll(dataDir) + + tests := []configTest{ + // ------------------------------------------------------------ + // cmd line flags + // + + { + desc: "-advertise", + flags: []string{ + `-advertise=1.2.3.4`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("1.2.3.4") + rt.AdvertiseAddrWAN = ipAddr("1.2.3.4") + rt.RPCAdvertiseAddr = tcpAddr("1.2.3.4:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("1.2.3.4:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("1.2.3.4:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "1.2.3.4", + "wan": "1.2.3.4", + } + rt.DataDir = dataDir + }, + }, + { + desc: "-advertise-wan", + flags: []string{ + `-advertise-wan=1.2.3.4`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrWAN = ipAddr("1.2.3.4") + rt.SerfAdvertiseAddrWAN = tcpAddr("1.2.3.4:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "10.0.0.1", + "wan": "1.2.3.4", + } + rt.DataDir = dataDir + }, + }, + { + desc: "-advertise and -advertise-wan", + flags: []string{ + `-advertise=1.2.3.4`, + `-advertise-wan=5.6.7.8`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("1.2.3.4") + rt.AdvertiseAddrWAN = ipAddr("5.6.7.8") + rt.RPCAdvertiseAddr = tcpAddr("1.2.3.4:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("1.2.3.4:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("5.6.7.8:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "1.2.3.4", + "wan": "5.6.7.8", + } + rt.DataDir = dataDir + }, + }, + { + desc: "-bind", + flags: []string{ + `-bind=1.2.3.4`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.BindAddr = ipAddr("1.2.3.4") + rt.AdvertiseAddrLAN = ipAddr("1.2.3.4") + rt.AdvertiseAddrWAN = ipAddr("1.2.3.4") + rt.RPCAdvertiseAddr = tcpAddr("1.2.3.4:8300") + rt.RPCBindAddr = tcpAddr("1.2.3.4:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("1.2.3.4:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("1.2.3.4:8302") + rt.SerfBindAddrLAN = tcpAddr("1.2.3.4:8301") + rt.SerfBindAddrWAN = tcpAddr("1.2.3.4:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "1.2.3.4", + "wan": "1.2.3.4", + } + rt.DataDir = dataDir + }, + }, + { + desc: "-bootstrap", + flags: []string{ + `-bootstrap`, + `-server`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.Bootstrap = true + rt.ServerMode = true + rt.LeaveOnTerm = false + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + warns: []string{"bootstrap = true: do not enable unless necessary"}, + }, + { + desc: "-bootstrap-expect", + flags: []string{ + `-bootstrap-expect=3`, + `-server`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.BootstrapExpect = 3 + rt.ServerMode = true + rt.LeaveOnTerm = false + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + warns: []string{"bootstrap_expect > 0: expecting 3 servers"}, + }, + { + desc: "-client", + flags: []string{ + `-client=1.2.3.4`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("1.2.3.4")} + rt.DNSAddrs = []net.Addr{tcpAddr("1.2.3.4:8600"), udpAddr("1.2.3.4:8600")} + rt.HTTPAddrs = []net.Addr{tcpAddr("1.2.3.4:8500")} + rt.DataDir = dataDir + }, + }, + { + desc: "-data-dir empty", + flags: []string{ + `-data-dir=`, + }, + err: "data_dir cannot be empty", + }, + { + desc: "-data-dir non-directory", + flags: []string{ + `-data-dir=runtime_test.go`, + }, + err: `data_dir "runtime_test.go" is not a directory`, + }, + { + desc: "-datacenter", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.Datacenter = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "-datacenter empty", + flags: []string{ + `-datacenter=`, + `-data-dir=` + dataDir, + }, + err: "datacenter cannot be empty", + }, + { + desc: "-dev", + flags: []string{ + `-dev`, + }, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("127.0.0.1") + rt.AdvertiseAddrWAN = ipAddr("127.0.0.1") + rt.BindAddr = ipAddr("127.0.0.1") + rt.DevMode = true + rt.DisableAnonymousSignature = true + rt.DisableKeyringFile = true + rt.EnableDebug = true + rt.EnableUI = true + rt.LeaveOnTerm = false + rt.LogLevel = "DEBUG" + rt.RPCAdvertiseAddr = tcpAddr("127.0.0.1:8300") + rt.RPCBindAddr = tcpAddr("127.0.0.1:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("127.0.0.1:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("127.0.0.1:8302") + rt.SerfBindAddrLAN = tcpAddr("127.0.0.1:8301") + rt.SerfBindAddrWAN = tcpAddr("127.0.0.1:8302") + rt.ServerMode = true + rt.SkipLeaveOnInt = true + rt.TaggedAddresses = map[string]string{"lan": "127.0.0.1", "wan": "127.0.0.1"} + rt.ConsulCoordinateUpdatePeriod = 100 * time.Millisecond + rt.ConsulRaftElectionTimeout = 52 * time.Millisecond + rt.ConsulRaftHeartbeatTimeout = 35 * time.Millisecond + rt.ConsulRaftLeaderLeaseTimeout = 20 * time.Millisecond + rt.ConsulSerfLANGossipInterval = 100 * time.Millisecond + rt.ConsulSerfLANProbeInterval = 100 * time.Millisecond + rt.ConsulSerfLANProbeTimeout = 100 * time.Millisecond + rt.ConsulSerfLANSuspicionMult = 3 + rt.ConsulSerfWANGossipInterval = 100 * time.Millisecond + rt.ConsulSerfWANProbeInterval = 100 * time.Millisecond + rt.ConsulSerfWANProbeTimeout = 100 * time.Millisecond + rt.ConsulSerfWANSuspicionMult = 3 + rt.ConsulServerHealthInterval = 10 * time.Millisecond + }, + }, + { + desc: "-disable-host-node-id", + flags: []string{ + `-disable-host-node-id`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.DisableHostNodeID = true + rt.DataDir = dataDir + }, + }, + { + desc: "-disable-keyring-file", + flags: []string{ + `-disable-keyring-file`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.DisableKeyringFile = true + rt.DataDir = dataDir + }, + }, + { + desc: "-dns-port", + flags: []string{ + `-dns-port=123`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.DNSPort = 123 + rt.DNSAddrs = []net.Addr{tcpAddr("127.0.0.1:123"), udpAddr("127.0.0.1:123")} + rt.DataDir = dataDir + }, + }, + { + desc: "-domain", + flags: []string{ + `-domain=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.DNSDomain = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "-enable-script-checks", + flags: []string{ + `-enable-script-checks`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.EnableScriptChecks = true + rt.DataDir = dataDir + }, + }, + { + desc: "-encrypt", + flags: []string{ + `-encrypt=i0P+gFTkLPg0h53eNYjydg==`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.EncryptKey = "i0P+gFTkLPg0h53eNYjydg==" + rt.DataDir = dataDir + }, + }, + { + desc: "-http-port", + flags: []string{ + `-http-port=123`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.HTTPPort = 123 + rt.HTTPAddrs = []net.Addr{tcpAddr("127.0.0.1:123")} + rt.DataDir = dataDir + }, + }, + { + desc: "-join", + flags: []string{ + `-join=a`, + `-join=b`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.StartJoinAddrsLAN = []string{"a", "b"} + rt.DataDir = dataDir + }, + }, + { + desc: "-join-wan", + flags: []string{ + `-join-wan=a`, + `-join-wan=b`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.StartJoinAddrsWAN = []string{"a", "b"} + rt.DataDir = dataDir + }, + }, + { + desc: "-log-level", + flags: []string{ + `-log-level=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.LogLevel = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "-node", + flags: []string{ + `-node=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.NodeName = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "-node-id", + flags: []string{ + `-node-id=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.NodeID = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "-node-meta", + flags: []string{ + `-node-meta=a:b`, + `-node-meta=c:d`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.NodeMeta = map[string]string{"a": "b", "c": "d"} + rt.DataDir = dataDir + }, + }, + { + desc: "-non-voting-server", + flags: []string{ + `-non-voting-server`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.NonVotingServer = true + rt.DataDir = dataDir + }, + }, + { + desc: "-pid-file", + flags: []string{ + `-pid-file=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.PidFile = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "-protocol", + flags: []string{ + `-protocol=1`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RPCProtocol = 1 + rt.DataDir = dataDir + }, + }, + { + desc: "-raft-protocol", + flags: []string{ + `-raft-protocol=1`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RaftProtocol = 1 + rt.DataDir = dataDir + }, + }, + { + desc: "-recursor", + flags: []string{ + `-recursor=a`, + `-recursor=b`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.DNSRecursors = []string{"a", "b"} + rt.DataDir = dataDir + }, + }, + { + desc: "-rejoin", + flags: []string{ + `-rejoin`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RejoinAfterLeave = true + rt.DataDir = dataDir + }, + }, + { + desc: "-retry-interval", + flags: []string{ + `-retry-interval=5s`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RetryJoinIntervalLAN = 5 * time.Second + rt.DataDir = dataDir + }, + }, + { + desc: "-retry-interval-wan", + flags: []string{ + `-retry-interval-wan=5s`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RetryJoinIntervalWAN = 5 * time.Second + rt.DataDir = dataDir + }, + }, + { + desc: "-retry-join", + flags: []string{ + `-retry-join=a`, + `-retry-join=b`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RetryJoinLAN = []string{"a", "b"} + rt.DataDir = dataDir + }, + }, + { + desc: "-retry-join-wan", + flags: []string{ + `-retry-join-wan=a`, + `-retry-join-wan=b`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RetryJoinWAN = []string{"a", "b"} + rt.DataDir = dataDir + }, + }, + { + desc: "-retry-max", + flags: []string{ + `-retry-max=1`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RetryJoinMaxAttemptsLAN = 1 + rt.DataDir = dataDir + }, + }, + { + desc: "-retry-max-wan", + flags: []string{ + `-retry-max-wan=1`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.RetryJoinMaxAttemptsWAN = 1 + rt.DataDir = dataDir + }, + }, + { + desc: "-serf-lan-bind", + flags: []string{ + `-serf-lan-bind=1.2.3.4`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.SerfBindAddrLAN = tcpAddr("1.2.3.4:8301") + rt.DataDir = dataDir + }, + }, + { + desc: "-serf-wan-bind", + flags: []string{ + `-serf-wan-bind=1.2.3.4`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.SerfBindAddrWAN = tcpAddr("1.2.3.4:8302") + rt.DataDir = dataDir + }, + }, + { + desc: "-server", + flags: []string{ + `-server`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.ServerMode = true + rt.LeaveOnTerm = false + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + }, + { + desc: "-syslog", + flags: []string{ + `-syslog`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.EnableSyslog = true + rt.DataDir = dataDir + }, + }, + { + desc: "-ui", + flags: []string{ + `-ui`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.EnableUI = true + rt.DataDir = dataDir + }, + }, + { + desc: "-ui-dir", + flags: []string{ + `-ui-dir=a`, + `-data-dir=` + dataDir, + }, + patch: func(rt *RuntimeConfig) { + rt.UIDir = "a" + rt.DataDir = dataDir + }, + }, + + // ------------------------------------------------------------ + // ports and addresses + // + + { + desc: "bind addr any v4", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr":"0.0.0.0" }`}, + hcl: []string{`bind_addr = "0.0.0.0"`}, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("10.0.0.1") + rt.AdvertiseAddrWAN = ipAddr("10.0.0.1") + rt.BindAddr = ipAddr("0.0.0.0") + rt.RPCAdvertiseAddr = tcpAddr("10.0.0.1:8300") + rt.RPCBindAddr = tcpAddr("0.0.0.0:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("10.0.0.1:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("10.0.0.1:8302") + rt.SerfBindAddrLAN = tcpAddr("0.0.0.0:8301") + rt.SerfBindAddrWAN = tcpAddr("0.0.0.0:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "10.0.0.1", + "wan": "10.0.0.1", + } + rt.DataDir = dataDir + }, + }, + { + desc: "bind addr any v6", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr":"::" }`}, + hcl: []string{`bind_addr = "::"`}, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("dead:beef::1") + rt.AdvertiseAddrWAN = ipAddr("dead:beef::1") + rt.BindAddr = ipAddr("::") + rt.RPCAdvertiseAddr = tcpAddr("[dead:beef::1]:8300") + rt.RPCBindAddr = tcpAddr("[::]:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("[dead:beef::1]:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("[dead:beef::1]:8302") + rt.SerfBindAddrLAN = tcpAddr("[::]:8301") + rt.SerfBindAddrWAN = tcpAddr("[::]:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "dead:beef::1", + "wan": "dead:beef::1", + } + rt.DataDir = dataDir + }, + publicv6: func() ([]*net.IPAddr, error) { + return []*net.IPAddr{ipAddr("dead:beef::1")}, nil + }, + }, + { + desc: "client addr and ports == 0", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr":"0.0.0.0", + "ports":{} + }`}, + hcl: []string{` + client_addr = "0.0.0.0" + ports {} + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("0.0.0.0")} + rt.DNSAddrs = []net.Addr{tcpAddr("0.0.0.0:8600"), udpAddr("0.0.0.0:8600")} + rt.HTTPAddrs = []net.Addr{tcpAddr("0.0.0.0:8500")} + rt.DataDir = dataDir + }, + }, + { + desc: "client addr and ports < 0", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr":"0.0.0.0", + "ports": { "dns":-1, "http":-2, "https":-3 } + }`}, + hcl: []string{` + client_addr = "0.0.0.0" + ports { dns = -1 http = -2 https = -3 } + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("0.0.0.0")} + rt.DNSPort = -1 + rt.DNSAddrs = nil + rt.HTTPPort = -1 + rt.HTTPAddrs = nil + rt.DataDir = dataDir + }, + }, + { + desc: "client addr and ports > 0", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr":"0.0.0.0", + "ports":{ "dns": 1, "http": 2, "https": 3 } + }`}, + hcl: []string{` + client_addr = "0.0.0.0" + ports { dns = 1 http = 2 https = 3 } + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("0.0.0.0")} + rt.DNSPort = 1 + rt.DNSAddrs = []net.Addr{tcpAddr("0.0.0.0:1"), udpAddr("0.0.0.0:1")} + rt.HTTPPort = 2 + rt.HTTPAddrs = []net.Addr{tcpAddr("0.0.0.0:2")} + rt.HTTPSPort = 3 + rt.HTTPSAddrs = []net.Addr{tcpAddr("0.0.0.0:3")} + rt.DataDir = dataDir + }, + }, + + { + desc: "client addr, addresses and ports == 0", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr":"0.0.0.0", + "addresses": { "dns": "1.1.1.1", "http": "2.2.2.2", "https": "3.3.3.3" }, + "ports":{} + }`}, + hcl: []string{` + client_addr = "0.0.0.0" + addresses = { dns = "1.1.1.1" http = "2.2.2.2" https = "3.3.3.3" } + ports {} + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("0.0.0.0")} + rt.DNSAddrs = []net.Addr{tcpAddr("1.1.1.1:8600"), udpAddr("1.1.1.1:8600")} + rt.HTTPAddrs = []net.Addr{tcpAddr("2.2.2.2:8500")} + rt.DataDir = dataDir + }, + }, + { + desc: "client addr, addresses and ports < 0", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr":"0.0.0.0", + "addresses": { "dns": "1.1.1.1", "http": "2.2.2.2", "https": "3.3.3.3" }, + "ports": { "dns":-1, "http":-2, "https":-3 } + }`}, + hcl: []string{` + client_addr = "0.0.0.0" + addresses = { dns = "1.1.1.1" http = "2.2.2.2" https = "3.3.3.3" } + ports { dns = -1 http = -2 https = -3 } + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("0.0.0.0")} + rt.DNSPort = -1 + rt.DNSAddrs = nil + rt.HTTPPort = -1 + rt.HTTPAddrs = nil + rt.DataDir = dataDir + }, + }, + { + desc: "client addr, addresses and ports", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr": "0.0.0.0", + "addresses": { "dns": "1.1.1.1", "http": "2.2.2.2", "https": "3.3.3.3" }, + "ports":{ "dns":1, "http":2, "https":3 } + }`}, + hcl: []string{` + client_addr = "0.0.0.0" + addresses = { dns = "1.1.1.1" http = "2.2.2.2" https = "3.3.3.3" } + ports { dns = 1 http = 2 https = 3 } + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("0.0.0.0")} + rt.DNSPort = 1 + rt.DNSAddrs = []net.Addr{tcpAddr("1.1.1.1:1"), udpAddr("1.1.1.1:1")} + rt.HTTPPort = 2 + rt.HTTPAddrs = []net.Addr{tcpAddr("2.2.2.2:2")} + rt.HTTPSPort = 3 + rt.HTTPSAddrs = []net.Addr{tcpAddr("3.3.3.3:3")} + rt.DataDir = dataDir + }, + }, + { + desc: "client template and ports", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr": "{{ printf \"1.2.3.4 2001:db8::1\" }}", + "ports":{ "dns":1, "http":2, "https":3 } + }`}, + hcl: []string{` + client_addr = "{{ printf \"1.2.3.4 2001:db8::1\" }}" + ports { dns = 1 http = 2 https = 3 } + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("1.2.3.4"), ipAddr("2001:db8::1")} + rt.DNSPort = 1 + rt.DNSAddrs = []net.Addr{tcpAddr("1.2.3.4:1"), tcpAddr("[2001:db8::1]:1"), udpAddr("1.2.3.4:1"), udpAddr("[2001:db8::1]:1")} + rt.HTTPPort = 2 + rt.HTTPAddrs = []net.Addr{tcpAddr("1.2.3.4:2"), tcpAddr("[2001:db8::1]:2")} + rt.HTTPSPort = 3 + rt.HTTPSAddrs = []net.Addr{tcpAddr("1.2.3.4:3"), tcpAddr("[2001:db8::1]:3")} + rt.DataDir = dataDir + }, + }, + { + desc: "client, address template and ports", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ + "client_addr": "{{ printf \"1.2.3.4 2001:db8::1\" }}", + "addresses": { + "dns": "{{ printf \"1.1.1.1 2001:db8::10 \" }}", + "http": "{{ printf \"2.2.2.2 unix://http 2001:db8::20 \" }}", + "https": "{{ printf \"3.3.3.3 unix://https 2001:db8::30 \" }}" + }, + "ports":{ "dns":1, "http":2, "https":3 } + }`}, + hcl: []string{` + client_addr = "{{ printf \"1.2.3.4 2001:db8::1\" }}" + addresses = { + dns = "{{ printf \"1.1.1.1 2001:db8::10 \" }}" + http = "{{ printf \"2.2.2.2 unix://http 2001:db8::20 \" }}" + https = "{{ printf \"3.3.3.3 unix://https 2001:db8::30 \" }}" + } + ports { dns = 1 http = 2 https = 3 } + `}, + patch: func(rt *RuntimeConfig) { + rt.ClientAddrs = []*net.IPAddr{ipAddr("1.2.3.4"), ipAddr("2001:db8::1")} + rt.DNSPort = 1 + rt.DNSAddrs = []net.Addr{tcpAddr("1.1.1.1:1"), tcpAddr("[2001:db8::10]:1"), udpAddr("1.1.1.1:1"), udpAddr("[2001:db8::10]:1")} + rt.HTTPPort = 2 + rt.HTTPAddrs = []net.Addr{tcpAddr("2.2.2.2:2"), unixAddr("unix://http"), tcpAddr("[2001:db8::20]:2")} + rt.HTTPSPort = 3 + rt.HTTPSAddrs = []net.Addr{tcpAddr("3.3.3.3:3"), unixAddr("unix://https"), tcpAddr("[2001:db8::30]:3")} + rt.DataDir = dataDir + }, + }, + { + desc: "advertise address lan template", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "advertise_addr": "{{ printf \"1.2.3.4\" }}" }`}, + hcl: []string{`advertise_addr = "{{ printf \"1.2.3.4\" }}"`}, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("1.2.3.4") + rt.AdvertiseAddrWAN = ipAddr("1.2.3.4") + rt.RPCAdvertiseAddr = tcpAddr("1.2.3.4:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("1.2.3.4:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("1.2.3.4:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "1.2.3.4", + "wan": "1.2.3.4", + } + rt.DataDir = dataDir + }, + }, + { + desc: "advertise address wan template", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "advertise_addr_wan": "{{ printf \"1.2.3.4\" }}" }`}, + hcl: []string{`advertise_addr_wan = "{{ printf \"1.2.3.4\" }}"`}, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrWAN = ipAddr("1.2.3.4") + rt.SerfAdvertiseAddrWAN = tcpAddr("1.2.3.4:8302") + rt.TaggedAddresses = map[string]string{ + "lan": "10.0.0.1", + "wan": "1.2.3.4", + } + rt.DataDir = dataDir + }, + }, + { + desc: "serf advertise address lan template", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "advertise_addrs": { "serf_lan": "{{ printf \"1.2.3.4\" }}" } }`}, + hcl: []string{`advertise_addrs = { serf_lan = "{{ printf \"1.2.3.4\" }}" }`}, + patch: func(rt *RuntimeConfig) { + rt.SerfAdvertiseAddrLAN = tcpAddr("1.2.3.4:8301") + rt.DataDir = dataDir + }, + }, + { + desc: "serf advertise address wan template", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "advertise_addrs": { "serf_wan": "{{ printf \"1.2.3.4\" }}" } }`}, + hcl: []string{`advertise_addrs = { serf_wan = "{{ printf \"1.2.3.4\" }}" }`}, + patch: func(rt *RuntimeConfig) { + rt.SerfAdvertiseAddrWAN = tcpAddr("1.2.3.4:8302") + rt.DataDir = dataDir + }, + }, + { + desc: "serf bind address lan template", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "serf_lan": "{{ printf \"1.2.3.4\" }}" }`}, + hcl: []string{`serf_lan = "{{ printf \"1.2.3.4\" }}"`}, + patch: func(rt *RuntimeConfig) { + rt.SerfBindAddrLAN = tcpAddr("1.2.3.4:8301") + rt.DataDir = dataDir + }, + }, + { + desc: "serf bind address wan template", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "serf_wan": "{{ printf \"1.2.3.4\" }}" }`}, + hcl: []string{`serf_wan = "{{ printf \"1.2.3.4\" }}"`}, + patch: func(rt *RuntimeConfig) { + rt.SerfBindAddrWAN = tcpAddr("1.2.3.4:8302") + rt.DataDir = dataDir + }, + }, + + // ------------------------------------------------------------ + // precedence rules + // + + { + desc: "precedence: merge order", + flags: []string{`-data-dir=` + dataDir}, + json: []string{ + `{ + "bootstrap": true, + "bootstrap_expect": 1, + "datacenter": "a", + "start_join": ["a", "b"], + "node_meta": {"a":"b"} + }`, + `{ + "bootstrap": false, + "bootstrap_expect": 0, + "datacenter":"b", + "start_join": ["c", "d"], + "node_meta": {"c":"d"} + }`, + }, + hcl: []string{ + ` + bootstrap = true + bootstrap_expect = 1 + datacenter = "a" + start_join = ["a", "b"] + node_meta = { "a" = "b" } + `, + ` + bootstrap = false + bootstrap_expect = 0 + datacenter = "b" + start_join = ["c", "d"] + node_meta = { "c" = "d" } + `, + }, + patch: func(rt *RuntimeConfig) { + rt.Bootstrap = false + rt.BootstrapExpect = 0 + rt.Datacenter = "b" + rt.StartJoinAddrsLAN = []string{"a", "b", "c", "d"} + rt.NodeMeta = map[string]string{"c": "d"} + rt.DataDir = dataDir + }, + }, + { + desc: "precedence: flag before file", + json: []string{ + `{ + "advertise_addr": "1.2.3.4", + "advertise_addr_wan": "5.6.7.8", + "bootstrap":true, + "bootstrap_expect": 3, + "datacenter":"a", + "node_meta": {"a":"b"}, + "recursors":["a", "b"], + "serf_lan": "a", + "serf_wan": "a", + "start_join":["a", "b"] + }`, + }, + hcl: []string{ + ` + advertise_addr = "1.2.3.4" + advertise_addr_wan = "5.6.7.8" + bootstrap = true + bootstrap_expect = 3 + datacenter = "a" + node_meta = { "a" = "b" } + recursors = ["a", "b"] + serf_lan = "a" + serf_wan = "a" + start_join = ["a", "b"] + `, + }, + flags: []string{ + `-advertise=1.1.1.1`, + `-advertise-wan=2.2.2.2`, + `-bootstrap=false`, + `-bootstrap-expect=0`, + `-datacenter=b`, + `-data-dir=` + dataDir, + `-join`, `c`, `-join=d`, + `-node-meta=c:d`, + `-recursor`, `c`, `-recursor=d`, + `-serf-lan-bind=3.3.3.3`, + `-serf-wan-bind=4.4.4.4`, + }, + patch: func(rt *RuntimeConfig) { + rt.AdvertiseAddrLAN = ipAddr("1.1.1.1") + rt.AdvertiseAddrWAN = ipAddr("2.2.2.2") + rt.RPCAdvertiseAddr = tcpAddr("1.1.1.1:8300") + rt.SerfAdvertiseAddrLAN = tcpAddr("1.1.1.1:8301") + rt.SerfAdvertiseAddrWAN = tcpAddr("2.2.2.2:8302") + rt.Datacenter = "b" + rt.DNSRecursors = []string{"c", "d", "a", "b"} + rt.NodeMeta = map[string]string{"c": "d"} + rt.SerfBindAddrLAN = tcpAddr("3.3.3.3:8301") + rt.SerfBindAddrWAN = tcpAddr("4.4.4.4:8302") + rt.StartJoinAddrsLAN = []string{"c", "d", "a", "b"} + rt.TaggedAddresses = map[string]string{ + "lan": "1.1.1.1", + "wan": "2.2.2.2", + } + rt.DataDir = dataDir + }, + }, + + // ------------------------------------------------------------ + // transformations + // + { + desc: "raft performance scaling", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "performance": { "raft_multiplier": 9} }`}, + hcl: []string{`performance = { raft_multiplier=9 }`}, + patch: func(rt *RuntimeConfig) { + rt.ConsulRaftElectionTimeout = 9 * 1000 * time.Millisecond + rt.ConsulRaftHeartbeatTimeout = 9 * 1000 * time.Millisecond + rt.ConsulRaftLeaderLeaseTimeout = 9 * 500 * time.Millisecond + rt.DataDir = dataDir + }, + }, + + // ------------------------------------------------------------ + // validations + // + + { + desc: "invalid input", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`this is not JSON`}, + hcl: []string{`*** 0123 this is not HCL`}, + err: "Error parsing", + }, + { + desc: "datacenter is lower-cased", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "datacenter": "A" }`}, + hcl: []string{`datacenter = "A"`}, + patch: func(rt *RuntimeConfig) { + rt.Datacenter = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "acl_datacenter is lower-cased", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "acl_datacenter": "A" }`}, + hcl: []string{`acl_datacenter = "A"`}, + patch: func(rt *RuntimeConfig) { + rt.ACLDatacenter = "a" + rt.DataDir = dataDir + }, + }, + { + desc: "acl_replication_token enables acl replication", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "acl_replication_token": "a" }`}, + hcl: []string{`acl_replication_token = "a"`}, + patch: func(rt *RuntimeConfig) { + rt.ACLReplicationToken = "a" + rt.EnableACLReplication = true + rt.DataDir = dataDir + }, + }, + { + desc: "advertise address detect fails v4", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "0.0.0.0"}`}, + hcl: []string{`bind_addr = "0.0.0.0"`}, + privatev4: func() ([]*net.IPAddr, error) { + return nil, errors.New("some error") + }, + err: "Error detecting private IPv4 address: some error", + }, + { + desc: "advertise address detect none v4", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "0.0.0.0"}`}, + hcl: []string{`bind_addr = "0.0.0.0"`}, + privatev4: func() ([]*net.IPAddr, error) { + return nil, nil + }, + err: "No private IPv4 address found", + }, + { + desc: "advertise address detect multiple v4", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "0.0.0.0"}`}, + hcl: []string{`bind_addr = "0.0.0.0"`}, + privatev4: func() ([]*net.IPAddr, error) { + return []*net.IPAddr{ipAddr("1.1.1.1"), ipAddr("2.2.2.2")}, nil + }, + err: "Multiple private IPv4 addresses found. Please configure one", + }, + { + desc: "advertise address detect fails v6", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "::"}`}, + hcl: []string{`bind_addr = "::"`}, + publicv6: func() ([]*net.IPAddr, error) { + return nil, errors.New("some error") + }, + err: "Error detecting public IPv6 address: some error", + }, + { + desc: "advertise address detect none v6", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "::"}`}, + hcl: []string{`bind_addr = "::"`}, + publicv6: func() ([]*net.IPAddr, error) { + return nil, nil + }, + err: "No public IPv6 address found", + }, + { + desc: "advertise address detect multiple v6", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "::"}`}, + hcl: []string{`bind_addr = "::"`}, + publicv6: func() ([]*net.IPAddr, error) { + return []*net.IPAddr{ipAddr("dead:beef::1"), ipAddr("dead:beef::2")}, nil + }, + err: "Multiple public IPv6 addresses found. Please configure one", + }, + { + desc: "ae_interval invalid == 0", + flags: []string{`-data-dir=` + dataDir}, + jsontail: []string{`{ "ae_interval": "0s" }`}, + hcltail: []string{`ae_interval = "0s"`}, + err: `ae_interval cannot be 0s. Must be positive`, + }, + { + desc: "ae_interval invalid < 0", + flags: []string{`-data-dir=` + dataDir}, + jsontail: []string{`{ "ae_interval": "-1s" }`}, + hcltail: []string{`ae_interval = "-1s"`}, + err: `ae_interval cannot be -1s. Must be positive`, + }, + { + desc: "acl_datacenter invalid", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "acl_datacenter": "%" }`}, + hcl: []string{`acl_datacenter = "%"`}, + err: `acl_datacenter cannot be "%". Please use only [a-z0-9-_]`, + }, + { + desc: "autopilot.max_trailing_logs invalid", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "autopilot": { "max_trailing_logs": -1 } }`}, + hcl: []string{`autopilot = { max_trailing_logs = -1 }`}, + err: "autopilot.max_trailing_logs cannot be -1. Must be greater than or equal to zero", + }, + { + desc: "bind_addr cannot be empty", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "" }`}, + hcl: []string{`bind_addr = ""`}, + err: "bind_addr cannot be empty", + }, + { + desc: "bind_addr does not allow multiple addresses", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "1.1.1.1 2.2.2.2" }`}, + hcl: []string{`bind_addr = "1.1.1.1 2.2.2.2"`}, + err: "bind_addr cannot contain multiple addresses", + }, + { + desc: "bind_addr cannot be a unix socket", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "bind_addr": "unix:///foo" }`}, + hcl: []string{`bind_addr = "unix:///foo"`}, + err: "bind_addr cannot be a unix socket", + }, + { + desc: "bootstrap without server", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap": true }`}, + hcl: []string{`bootstrap = true`}, + err: "'bootstrap = true' requires 'server = true'", + }, + { + desc: "bootstrap-expect without server", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap_expect": 3 }`}, + hcl: []string{`bootstrap_expect = 3`}, + err: "'bootstrap_expect > 0' requires 'server = true'", + }, + { + desc: "bootstrap-expect invalid", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap_expect": -1 }`}, + hcl: []string{`bootstrap_expect = -1`}, + err: "bootstrap_expect cannot be -1. Must be greater than or equal to zero", + }, + { + desc: "bootstrap-expect and dev mode", + flags: []string{ + `-dev`, + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap_expect": 3, "server": true }`}, + hcl: []string{`bootstrap_expect = 3 server = true`}, + err: "'bootstrap_expect > 0' not allowed in dev mode", + }, + { + desc: "bootstrap-expect and boostrap", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap": true, "bootstrap_expect": 3, "server": true }`}, + hcl: []string{`bootstrap = true bootstrap_expect = 3 server = true`}, + err: "'bootstrap_expect > 0' and 'bootstrap = true' are mutually exclusive", + }, + { + desc: "bootstrap-expect=1 equals boostrap", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap_expect": 1, "server": true }`}, + hcl: []string{`bootstrap_expect = 1 server = true`}, + patch: func(rt *RuntimeConfig) { + rt.Bootstrap = true + rt.BootstrapExpect = 0 + rt.LeaveOnTerm = false + rt.ServerMode = true + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + warns: []string{"BootstrapExpect is set to 1; this is the same as Bootstrap mode.", "bootstrap = true: do not enable unless necessary"}, + }, + { + desc: "bootstrap-expect=2 warning", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap_expect": 2, "server": true }`}, + hcl: []string{`bootstrap_expect = 2 server = true`}, + patch: func(rt *RuntimeConfig) { + rt.BootstrapExpect = 2 + rt.LeaveOnTerm = false + rt.ServerMode = true + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + warns: []string{ + `bootstrap_expect = 2: A cluster with 2 servers will provide no failure tolerance. See https://www.consul.io/docs/internals/consensus.html#deployment-table`, + `bootstrap_expect > 0: expecting 2 servers`, + }, + }, + { + desc: "bootstrap-expect > 2 but even warning", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "bootstrap_expect": 4, "server": true }`}, + hcl: []string{`bootstrap_expect = 4 server = true`}, + patch: func(rt *RuntimeConfig) { + rt.BootstrapExpect = 4 + rt.LeaveOnTerm = false + rt.ServerMode = true + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + warns: []string{ + `bootstrap_expect is even number: A cluster with an even number of servers does not achieve optimum fault tolerance. See https://www.consul.io/docs/internals/consensus.html#deployment-table`, + `bootstrap_expect > 0: expecting 4 servers`, + }, + }, + { + desc: "client mode sets LeaveOnTerm and SkipLeaveOnInt correctly", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "server": false }`}, + hcl: []string{` server = false`}, + patch: func(rt *RuntimeConfig) { + rt.LeaveOnTerm = true + rt.ServerMode = false + rt.SkipLeaveOnInt = false + rt.DataDir = dataDir + }, + }, + { + desc: "client does not allow socket", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "client_addr": "unix:///foo" }`}, + hcl: []string{`client_addr = "unix:///foo"`}, + err: "client_addr cannot be a unix socket", + }, + { + desc: "datacenter invalid", + flags: []string{`-data-dir=` + dataDir}, + json: []string{`{ "datacenter": "%" }`}, + hcl: []string{`datacenter = "%"`}, + err: `datacenter cannot be "%". Please use only [a-z0-9-_]`, + }, + { + desc: "dns does not allow socket", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "addresses": {"dns": "unix:///foo" } }`}, + hcl: []string{`addresses = { dns = "unix:///foo" }`}, + err: "DNS address cannot be a unix socket", + }, + { + desc: "enable_ui and ui_dir", + flags: []string{ + `-datacenter=a`, + `-data-dir=` + dataDir, + }, + json: []string{`{ "enable_ui": true, "ui_dir": "a" }`}, + hcl: []string{`enable_ui = true ui_dir = "a"`}, + err: "Both the ui and ui-dir flags were specified, please provide only one.\n" + + "If trying to use your own web UI resources, use the ui-dir flag.\n" + + "If using Consul version 0.7.0 or later, the web UI is included in the binary so use ui to enable it", + }, + + // test ANY address failures + // to avoid combinatory explosion for tests use 0.0.0.0, :: or [::] but not all of them + { + desc: "advertise_addr any", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "advertise_addr": "0.0.0.0" }`}, + hcl: []string{`advertise_addr = "0.0.0.0"`}, + err: "Advertise address cannot be 0.0.0.0, :: or [::]", + }, + { + desc: "advertise_addr_wan any", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "advertise_addr_wan": "::" }`}, + hcl: []string{`advertise_addr_wan = "::"`}, + err: "Advertise WAN address cannot be 0.0.0.0, :: or [::]", + }, + { + desc: "advertise_addrs.rpc any", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "advertise_addrs":{ "rpc": "[::]" } }`}, + hcl: []string{`advertise_addrs = { rpc = "[::]" }`}, + err: "advertise_addrs.rpc cannot be 0.0.0.0, :: or [::]", + }, + { + desc: "advertise_addrs.serf_lan any", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "advertise_addrs":{ "serf_lan": "[::]" } }`}, + hcl: []string{`advertise_addrs = { serf_lan = "[::]" }`}, + err: "advertise_addrs.serf_lan cannot be 0.0.0.0, :: or [::]", + }, + { + desc: "advertise_addrs.serf_wan any", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "advertise_addrs":{ "serf_wan": "0.0.0.0" } }`}, + hcl: []string{`advertise_addrs = { serf_wan = "0.0.0.0" }`}, + err: "advertise_addrs.serf_wan cannot be 0.0.0.0, :: or [::]", + }, + { + desc: "dns_config.udp_answer_limit invalid", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "dns_config": { "udp_answer_limit": -1 } }`}, + hcl: []string{`dns_config = { udp_answer_limit = -1 }`}, + err: "dns_config.udp_answer_limit cannot be -1. Must be greater than or equal to zero", + }, + { + desc: "performance.raft_multiplier < 0", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "performance": { "raft_multiplier": -1 } }`}, + hcl: []string{`performance = { raft_multiplier = -1 }`}, + err: `performance.raft_multiplier cannot be -1. Must be between 1 and 10`, + }, + { + desc: "performance.raft_multiplier == 0", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "performance": { "raft_multiplier": 0 } }`}, + hcl: []string{`performance = { raft_multiplier = 0 }`}, + err: `performance.raft_multiplier cannot be 0. Must be between 1 and 10`, + }, + { + desc: "performance.raft_multiplier > 10", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "performance": { "raft_multiplier": 20 } }`}, + hcl: []string{`performance = { raft_multiplier = 20 }`}, + err: `performance.raft_multiplier cannot be 20. Must be between 1 and 10`, + }, + { + desc: "node_name invalid", + flags: []string{ + `-data-dir=` + dataDir, + `-node=`, + }, + hostname: func() (string, error) { return "", nil }, + err: "node_name cannot be empty", + }, + { + desc: "node_meta key too long", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{ + `{ "dns_config": { "udp_answer_limit": 1 } }`, + `{ "node_meta": { "` + randomString(130) + `": "a" } }`, + }, + hcl: []string{ + `dns_config = { udp_answer_limit = 1 }`, + `node_meta = { "` + randomString(130) + `" = "a" }`, + }, + err: "Key is too long (limit: 128 characters)", + }, + { + desc: "node_meta value too long", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{ + `{ "dns_config": { "udp_answer_limit": 1 } }`, + `{ "node_meta": { "a": "` + randomString(520) + `" } }`, + }, + hcl: []string{ + `dns_config = { udp_answer_limit = 1 }`, + `node_meta = { "a" = "` + randomString(520) + `" }`, + }, + err: "Value is too long (limit: 512 characters)", + }, + { + desc: "node_meta too many keys", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{ + `{ "dns_config": { "udp_answer_limit": 1 } }`, + `{ "node_meta": {` + metaPairs(70, "json") + `} }`, + }, + hcl: []string{ + `dns_config = { udp_answer_limit = 1 }`, + `node_meta = {` + metaPairs(70, "hcl") + ` }`, + }, + err: "Node metadata cannot contain more than 64 key/value pairs", + }, + { + desc: "unique listeners dns vs http", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "client_addr": "1.2.3.4", + "ports": { "dns": 1000, "http": 1000 } + }`}, + hcl: []string{` + client_addr = "1.2.3.4" + ports = { dns = 1000 http = 1000 } + `}, + err: "HTTP address 1.2.3.4:1000 already configured for DNS", + }, + { + desc: "unique listeners dns vs https", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "client_addr": "1.2.3.4", + "ports": { "dns": 1000, "https": 1000 } + }`}, + hcl: []string{` + client_addr = "1.2.3.4" + ports = { dns = 1000 https = 1000 } + `}, + err: "HTTPS address 1.2.3.4:1000 already configured for DNS", + }, + { + desc: "unique listeners http vs https", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "client_addr": "1.2.3.4", + "ports": { "http": 1000, "https": 1000 } + }`}, + hcl: []string{` + client_addr = "1.2.3.4" + ports = { http = 1000 https = 1000 } + `}, + err: "HTTPS address 1.2.3.4:1000 already configured for HTTP", + }, + { + desc: "unique advertise addresses HTTP vs RPC", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "addresses": { "http": "10.0.0.1" }, + "ports": { "http": 1000, "server": 1000 } + }`}, + hcl: []string{` + addresses = { http = "10.0.0.1" } + ports = { http = 1000 server = 1000 } + `}, + err: "RPC Advertise address 10.0.0.1:1000 already configured for HTTP", + }, + { + desc: "unique advertise addresses RPC vs Serf LAN", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "ports": { "server": 1000, "serf_lan": 1000 } + }`}, + hcl: []string{` + ports = { server = 1000 serf_lan = 1000 } + `}, + err: "Serf Advertise LAN address 10.0.0.1:1000 already configured for RPC Advertise", + }, + { + desc: "unique advertise addresses RPC vs Serf WAN", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "ports": { "server": 1000, "serf_wan": 1000 } + }`}, + hcl: []string{` + ports = { server = 1000 serf_wan = 1000 } + `}, + err: "Serf Advertise WAN address 10.0.0.1:1000 already configured for RPC Advertise", + }, + { + desc: "telemetry.prefix_filter cannot be empty", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "telemetry": { "prefix_filter": [""] } + }`}, + hcl: []string{` + telemetry = { prefix_filter = [""] } + `}, + patch: func(rt *RuntimeConfig) { + rt.DataDir = dataDir + }, + warns: []string{"Cannot have empty filter rule in prefix_filter"}, + }, + { + desc: "telemetry.prefix_filter must start with + or -", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ + "telemetry": { "prefix_filter": ["+foo", "-bar", "nix"] } + }`}, + hcl: []string{` + telemetry = { prefix_filter = ["+foo", "-bar", "nix"] } + `}, + patch: func(rt *RuntimeConfig) { + rt.DataDir = dataDir + rt.TelemetryAllowedPrefixes = []string{"foo"} + rt.TelemetryBlockedPrefixes = []string{"bar"} + }, + warns: []string{`Filter rule must begin with either '+' or '-': "nix"`}, + }, + { + desc: "encrypt has invalid key", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "encrypt": "this is not a valid key" }`}, + hcl: []string{` encrypt = "this is not a valid key" `}, + err: "encrypt has invalid key: illegal base64 data at input byte 4", + }, + { + desc: "encrypt given but LAN keyring exists", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "encrypt": "i0P+gFTkLPg0h53eNYjydg==" }`}, + hcl: []string{` encrypt = "i0P+gFTkLPg0h53eNYjydg==" `}, + patch: func(rt *RuntimeConfig) { + rt.EncryptKey = "i0P+gFTkLPg0h53eNYjydg==" + rt.DataDir = dataDir + }, + pre: func() { + writeFile(filepath.Join(dataDir, SerfLANKeyring), []byte("i0P+gFTkLPg0h53eNYjydg==")) + }, + post: func() { + os.Remove(filepath.Join(filepath.Join(dataDir, SerfLANKeyring))) + }, + warns: []string{`WARNING: LAN keyring exists but -encrypt given, using keyring`}, + }, + { + desc: "encrypt given but WAN keyring exists", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "encrypt": "i0P+gFTkLPg0h53eNYjydg==", "server": true }`}, + hcl: []string{` encrypt = "i0P+gFTkLPg0h53eNYjydg==" server = true `}, + patch: func(rt *RuntimeConfig) { + rt.EncryptKey = "i0P+gFTkLPg0h53eNYjydg==" + rt.ServerMode = true + rt.LeaveOnTerm = false + rt.SkipLeaveOnInt = true + rt.DataDir = dataDir + }, + pre: func() { + writeFile(filepath.Join(dataDir, SerfWANKeyring), []byte("i0P+gFTkLPg0h53eNYjydg==")) + }, + post: func() { + os.Remove(filepath.Join(filepath.Join(dataDir, SerfWANKeyring))) + }, + warns: []string{`WARNING: WAN keyring exists but -encrypt given, using keyring`}, + }, + } + + testConfig(t, tests, dataDir) +} + +func testConfig(t *testing.T, tests []configTest, dataDir string) { + for _, tt := range tests { + for pass, format := range []string{"json", "hcl"} { + // when we test only flags then there are no JSON or HCL + // sources and we need to make only one pass over the + // tests. + flagsOnly := len(tt.json) == 0 && len(tt.hcl) == 0 + if flagsOnly && pass > 0 { + continue + } + + // json and hcl sources need to be in sync + // to make sure we're generating the same config + if len(tt.json) != len(tt.hcl) { + t.Fatal(tt.desc, ": JSON and HCL test case out of sync") + } + + // select the source + srcs, tails := tt.json, tt.jsontail + if format == "hcl" { + srcs, tails = tt.hcl, tt.hcltail + } + + // build the description + var desc []string + if !flagsOnly { + desc = append(desc, format) + } + if tt.desc != "" { + desc = append(desc, tt.desc) + } + + t.Run(strings.Join(desc, ":"), func(t *testing.T) { + // first parse the flags + flags, err := ParseFlags(tt.flags) + if err != nil { + t.Fatalf("ParseFlags failed: %s", err) + } + + // Then create a builder with the flags. + b, err := NewBuilder(flags) + if err != nil { + t.Fatal("NewBuilder", err) + } + + // mock the hostname function unless a mock is provided + b.Hostname = tt.hostname + if b.Hostname == nil { + b.Hostname = func() (string, error) { return "nodex", nil } + } + + // mock the ip address detection + privatev4 := tt.privatev4 + if privatev4 == nil { + privatev4 = func() ([]*net.IPAddr, error) { + return []*net.IPAddr{ipAddr("10.0.0.1")}, nil + } + } + b.GetPrivateIPv4 = privatev4 + b.GetPublicIPv6 = tt.publicv6 + + // read the source fragements + for i, data := range srcs { + b.Sources = append(b.Sources, Source{ + Name: fmt.Sprintf("%s-%d", format, i), + Format: format, + Data: data, + }) + } + for i, data := range tails { + b.Tail = append(b.Tail, Source{ + Name: fmt.Sprintf("%s-%d", format, i), + Format: format, + Data: data, + }) + } + + // build/merge the config fragments + if tt.pre != nil { + tt.pre() + } + defer func() { + if tt.post != nil { + tt.post() + } + }() + rt, err := b.BuildAndValidate() + if err == nil && tt.err != "" { + t.Fatalf("got no error want %q", tt.err) + } + if err != nil && tt.err == "" { + t.Fatalf("got error %s want nil", err) + } + if err == nil && tt.err != "" { + t.Fatalf("got nil want error to contain %q", tt.err) + } + if err != nil && tt.err != "" && !strings.Contains(err.Error(), tt.err) { + t.Fatalf("error %q does not contain %q", err.Error(), tt.err) + } + + // check the warnings + if !verify.Values(t, "warnings", b.Warnings, tt.warns) { + t.FailNow() + } + + // stop if we expected an error + if tt.err != "" { + return + } + + // build a default configuration, then patch the fields we expect to change + // and compare it with the generated configuration. Since the expected + // runtime config has been validated we do not need to validate it again. + x, err := NewBuilder(Flags{}) + if err != nil { + t.Fatal(err) + } + x.Hostname = b.Hostname + x.GetPrivateIPv4 = b.GetPrivateIPv4 + x.GetPublicIPv6 = b.GetPublicIPv6 + patchedRT, err := x.Build() + if err != nil { + t.Fatalf("build default failed: %s", err) + } + if tt.patch != nil { + tt.patch(&patchedRT) + } + // if err := x.Validate(wantRT); err != nil { + // t.Fatalf("validate default failed: %s", err) + // } + if got, want := rt, patchedRT; !verify.Values(t, "", got, want) { + t.FailNow() + } + }) + } + } +} + +// TestFullConfig tests the conversion from a fully populated JSON or +// HCL config file to a RuntimeConfig structure. All fields must be set +// to a unique non-zero value. +// +// To aid populating the fields the following bash functions can be used +// to generate random strings and ints: +// +// random-int() { echo $RANDOM } +// random-string() { base64 /dev/urandom | tr -d '/+' | fold -w ${1:-32} | head -n 1 } +// +// To generate a random string of length 8 run the following command in +// a terminal: +// +// random-string 8 +// +func TestFullConfig(t *testing.T) { + dataDir := testutil.TempDir(t, "consul") + defer os.RemoveAll(dataDir) + + flagSrc := []string{`-dev`} + src := map[string]string{ + "json": `{ + "acl_agent_master_token": "furuQD0b", + "acl_agent_token": "cOshLOQ2", + "acl_datacenter": "m3urck3z", + "acl_default_policy": "ArK3WIfE", + "acl_down_policy": "vZXMfMP0", + "acl_enforce_version_8": true, + "acl_master_token": "C1Q1oIwh", + "acl_replication_token": "LMmgy5dO", + "acl_token": "O1El0wan", + "acl_ttl": "18060s", + "addresses": { + "dns": "93.95.95.81", + "http": "83.39.91.39", + "https": "95.17.17.19" + }, + "advertise_addr": "17.99.29.16", + "advertise_addr_wan": "78.63.37.19", + "advertise_addrs": { + "rpc": "28.27.94.38", + "serf_lan": "49.38.36.95", + "serf_wan": "63.38.52.13" + }, + "autopilot": { + "cleanup_dead_servers": true, + "disable_upgrade_migration": true, + "last_contact_threshold": "12705s", + "max_trailing_logs": 17849, + "redundancy_zone_tag": "3IsufDJf", + "server_stabilization_time": "23057s", + "upgrade_version_tag": "W9pDwFAL" + }, + "bind_addr": "16.99.34.17", + "bootstrap": true, + "bootstrap_expect": 53, + "ca_file": "erA7T0PM", + "ca_path": "mQEN1Mfp", + "cert_file": "7s4QAzDk", + "check": { + "id": "fZaCAXww", + "name": "OOM2eo0f", + "notes": "zXzXI9Gt", + "service_id": "L8G0QNmR", + "token": "oo4BCTgJ", + "status": "qLykAl5u", + "script": "dhGfIF8n", + "http": "29B93haH", + "header": { + "hBq0zn1q": [ "2a9o9ZKP", "vKwA5lR6" ], + "f3r6xFtM": [ "RyuIdDWv", "QbxEcIUM" ] + }, + "method": "Dou0nGT5", + "tcp": "JY6fTTcw", + "interval": "18714s", + "docker_container_id": "qF66POS9", + "shell": "sOnDy228", + "tls_skip_verify": true, + "timeout": "5954s", + "ttl": "30044s", + "deregister_critical_service_after": "13209s" + }, + "checks": [ + { + "id": "uAjE6m9Z", + "name": "QsZRGpYr", + "notes": "VJ7Sk4BY", + "service_id": "lSulPcyz", + "token": "toO59sh8", + "status": "9RlWsXMV", + "script": "8qbd8tWw", + "http": "dohLcyQ2", + "header": { + "ZBfTin3L": [ "1sDbEqYG", "lJGASsWK" ], + "Ui0nU99X": [ "LMccm3Qe", "k5H5RggQ" ] + }, + "method": "aldrIQ4l", + "tcp": "RJQND605", + "interval": "22164s", + "docker_container_id": "ipgdFtjd", + "shell": "qAeOYy0M", + "tls_skip_verify": true, + "timeout": "1813s", + "ttl": "21743s", + "deregister_critical_service_after": "14232s" + }, + { + "id": "Cqq95BhP", + "name": "3qXpkS0i", + "notes": "sb5qLTex", + "service_id": "CmUUcRna", + "token": "a3nQzHuy", + "status": "irj26nf3", + "script": "FJsI1oXt", + "http": "yzhgsQ7Y", + "header": { + "zcqwA8dO": [ "qb1zx0DL", "sXCxPFsD" ], + "qxvdnSE9": [ "6wBPUYdF", "YYh8wtSZ" ] + }, + "method": "gLrztrNw", + "tcp": "4jG5casb", + "interval": "28767s", + "docker_container_id": "THW6u7rL", + "shell": "C1Zt3Zwh", + "tls_skip_verify": true, + "timeout": "18506s", + "ttl": "31006s", + "deregister_critical_service_after": "2366s" + } + ], + "check_update_interval": "16507s", + "client_addr": "93.83.18.19", + "data_dir": "` + dataDir + `", + "datacenter": "rzo029wg", + "disable_anonymous_signature": true, + "disable_coordinates": true, + "disable_host_node_id": true, + "disable_keyring_file": true, + "disable_remote_exec": true, + "disable_update_check": true, + "domain": "7W1xXSqd", + "dns_config": { + "allow_stale": true, + "disable_compression": true, + "enable_truncate": true, + "max_stale": "29685s", + "node_ttl": "7084s", + "only_passing": true, + "recursor_timeout": "4427s", + "service_ttl": { + "*": "32030s" + }, + "udp_answer_limit": 29909 + }, + "enable_acl_replication": true, + "enable_debug": true, + "enable_script_checks": true, + "enable_syslog": true, + "enable_ui": true, + "encrypt": "A4wELWqH", + "encrypt_verify_incoming": true, + "encrypt_verify_outgoing": true, + "http_config": { + "block_endpoints": [ "RBvAFcGD", "fWOWFznh" ], + "response_headers": { + "M6TKa9NP": "xjuxjOzQ", + "JRCrHZed": "rl0mTx81" + } + }, + "key_file": "IEkkwgIA", + "leave_on_terminate": true, + "limits": { + "rpc_rate": 12029.43, + "rpc_max_burst": 44848 + }, + "log_level": "k1zo9Spt", + "node_id": "AsUIlw99", + "node_meta": { + "5mgGQMBk": "mJLtVMSG", + "A7ynFMJB": "0Nx6RGab" + }, + "node_name": "otlLxGaI", + "non_voting_server": true, + "performance": { + "raft_multiplier": 5 + }, + "pid_file": "43xN80Km", + "ports": { + "dns": 7001, + "http": 7999, + "https": 15127, + "server": 3757 + }, + "protocol": 30793, + "raft_protocol": 19016, + "reconnect_timeout": "23739s", + "reconnect_timeout_wan": "26694s", + "recursors": [ "FtFhoUHl", "UYkwck1k" ], + "rejoin_after_leave": true, + "retry_interval": "8067s", + "retry_interval_wan": "28866s", + "retry_join": [ "pbsSFY7U", "l0qLtWij" ], + "retry_join_wan": [ "PFsR02Ye", "rJdQIhER" ], + "retry_max": 913, + "retry_max_wan": 23160, + "segment": "BC2NhTDi", + "segments": [ + { + "name": "PExYMe2E", + "bind": "36.73.36.19", + "port": 38295, + "rpc_listener": true, + "advertise": "63.39.19.18" + }, + { + "name": "UzCvJgup", + "bind": "37.58.38.19", + "port": 39292, + "rpc_listener": true, + "advertise": "83.58.26.27" + } + ], + "serf_lan": "99.43.63.15", + "serf_wan": "67.88.33.19", + "server": true, + "server_name": "Oerr9n1G", + "service": { + "id": "dLOXpSCI", + "name": "o1ynPkp0", + "tags": ["nkwshvM5", "NTDWn3ek"], + "address": "cOlSOhbp", + "token": "msy7iWER", + "port": 24237, + "enable_tag_override": true, + "check": { + "check_id": "RMi85Dv8", + "name": "iehanzuq", + "status": "rCvn53TH", + "notes": "fti5lfF3", + "script": "rtj34nfd", + "http": "dl3Fgme3", + "header": { + "rjm4DEd3": ["2m3m2Fls"], + "l4HwQ112": ["fk56MNlo", "dhLK56aZ"] + }, + "method": "9afLm3Mj", + "tcp": "fjiLFqVd", + "interval": "23926s", + "docker_container_id": "dO5TtRHk", + "shell": "e6q2ttES", + "tls_skip_verify": true, + "timeout": "38483s", + "ttl": "10943s", + "deregister_critical_service_after": "68787s" + }, + "checks": [ + { + "id": "Zv99e9Ka", + "name": "sgV4F7Pk", + "notes": "yP5nKbW0", + "status": "7oLMEyfu", + "script": "NlUQ3nTE", + "http": "KyDjGY9H", + "header": { + "gv5qefTz": [ "5Olo2pMG", "PvvKWQU5" ], + "SHOVq1Vv": [ "jntFhyym", "GYJh32pp" ] + }, + "method": "T66MFBfR", + "tcp": "bNnNfx2A", + "interval": "22224s", + "docker_container_id": "ipgdFtjd", + "shell": "omVZq7Sz", + "tls_skip_verify": true, + "timeout": "18913s", + "ttl": "44743s", + "deregister_critical_service_after": "8482s" + }, + { + "id": "G79O6Mpr", + "name": "IEqrzrsd", + "notes": "SVqApqeM", + "status": "XXkVoZXt", + "script": "IXLZTM6E", + "http": "kyICZsn8", + "header": { + "4ebP5vL4": [ "G20SrL5Q", "DwPKlMbo" ], + "p2UI34Qz": [ "UsG1D0Qh", "NHhRiB6s" ] + }, + "method": "ciYHWors", + "tcp": "FfvCwlqH", + "interval": "12356s", + "docker_container_id": "HBndBU6R", + "shell": "hVI33JjA", + "tls_skip_verify": true, + "timeout": "38282s", + "ttl": "1181s", + "deregister_critical_service_after": "4992s" + } + ] + }, + "services": [ + { + "id": "wI1dzxS4", + "name": "7IszXMQ1", + "tags": ["0Zwg8l6v", "zebELdN5"], + "address": "9RhqPSPB", + "token": "myjKJkWH", + "port": 72219, + "enable_tag_override": true, + "check": { + "check_id": "qmfeO5if", + "name": "atDGP7n5", + "status": "pDQKEhWL", + "notes": "Yt8EDLev", + "script": "MDu7wjlD", + "http": "qzHYvmJO", + "header": { + "UkpmZ3a3": ["2dfzXuxZ"], + "cVFpko4u": ["gGqdEB6k", "9LsRo22u"] + }, + "method": "X5DrovFc", + "tcp": "ICbxkpSF", + "interval": "24392s", + "docker_container_id": "ZKXr68Yb", + "shell": "CEfzx0Fo", + "tls_skip_verify": true, + "timeout": "38333s", + "ttl": "57201s", + "deregister_critical_service_after": "44214s" + } + }, + { + "id": "MRHVMZuD", + "name": "6L6BVfgH", + "tags": ["7Ale4y6o", "PMBW08hy"], + "address": "R6H6g8h0", + "token": "ZgY8gjMI", + "port": 38292, + "enable_tag_override": true, + "checks": [ + { + "id": "GTti9hCo", + "name": "9OOS93ne", + "notes": "CQy86DH0", + "status": "P0SWDvrk", + "script": "6BhLJ7R9", + "http": "u97ByEiW", + "header": { + "MUlReo8L": [ "AUZG7wHG", "gsN0Dc2N" ], + "1UJXjVrT": [ "OJgxzTfk", "xZZrFsq7" ] + }, + "method": "5wkAxCUE", + "tcp": "MN3oA9D2", + "interval": "32718s", + "docker_container_id": "cU15LMet", + "shell": "nEz9qz2l", + "tls_skip_verify": true, + "timeout": "34738s", + "ttl": "22773s", + "deregister_critical_service_after": "84282s" + }, + { + "id": "UHsDeLxG", + "name": "PQSaPWlT", + "notes": "jKChDOdl", + "status": "5qFz6OZn", + "script": "PbdxFZ3K", + "http": "1LBDJhw4", + "header": { + "cXPmnv1M": [ "imDqfaBx", "NFxZ1bQe" ], + "vr7wY7CS": [ "EtCoNPPL", "9vAarJ5s" ] + }, + "method": "wzByP903", + "tcp": "2exjZIGE", + "interval": "5656s", + "docker_container_id": "5tDBWpfA", + "shell": "rlTpLM8s", + "tls_skip_verify": true, + "timeout": "4868s", + "ttl": "11222s", + "deregister_critical_service_after": "68482s" + } + ] + } + ], + "session_ttl_min": "26627s", + "skip_leave_on_interrupt": true, + "start_join": [ "LR3hGDoG", "MwVpZ4Up" ], + "start_join_wan": [ "EbFSc3nA", "kwXTh623" ], + "syslog_facility": "hHv79Uia", + "tagged_addresses": { + "7MYgHrYH": "dALJAhLD", + "h6DdBy6K": "ebrr9zZ8" + }, + "telemetry": { + "circonus_api_app": "p4QOTe9j", + "circonus_api_token": "E3j35V23", + "circonus_api_url": "mEMjHpGg", + "circonus_broker_id": "BHlxUhed", + "circonus_broker_select_tag": "13xy1gHm", + "circonus_check_display_name": "DRSlQR6n", + "circonus_check_force_metric_activation": "Ua5FGVYf", + "circonus_check_id": "kGorutad", + "circonus_check_instance_id": "rwoOL6R4", + "circonus_check_search_tag": "ovT4hT4f", + "circonus_check_tags": "prvO4uBl", + "circonus_submission_interval": "DolzaflP", + "circonus_submission_url": "gTcbS93G", + "disable_hostname": true, + "dogstatsd_addr": "0wSndumK", + "dogstatsd_tags": [ "3N81zSUB","Xtj8AnXZ" ], + "filter_default": true, + "prefix_filter": [ "+oJotS8XJ","-cazlEhGn" ], + "statsd_address": "drce87cy", + "statsite_address": "HpFwKB8R", + "statsite_prefix": "ftO6DySn" + }, + "tls_cipher_suites": "TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA", + "tls_min_version": "pAOWafkR", + "tls_prefer_server_cipher_suites": true, + "translate_wan_addrs": true, + "ui_dir": "11IFzAUn", + "unix_sockets": { + "group": "8pFodrV8", + "mode": "E8sAwOv4", + "user": "E0nB1DwA" + }, + "verify_incoming": true, + "verify_incoming_https": true, + "verify_incoming_rpc": true, + "verify_outgoing": true, + "verify_server_hostname": true, + "watches": [ + { + "type": "key", + "datacenter": "GyE6jpeW", + "key": "j9lF1Tve", + "handler": "90N7S4LN" + } + ] + }`, + "hcl": ` + acl_agent_master_token = "furuQD0b" + acl_agent_token = "cOshLOQ2" + acl_datacenter = "m3urck3z" + acl_default_policy = "ArK3WIfE" + acl_down_policy = "vZXMfMP0" + acl_enforce_version_8 = true + acl_master_token = "C1Q1oIwh" + acl_replication_token = "LMmgy5dO" + acl_token = "O1El0wan" + acl_ttl = "18060s" + addresses = { + dns = "93.95.95.81" + http = "83.39.91.39" + https = "95.17.17.19" + } + advertise_addr = "17.99.29.16" + advertise_addr_wan = "78.63.37.19" + advertise_addrs = { + rpc = "28.27.94.38" + serf_lan = "49.38.36.95" + serf_wan = "63.38.52.13" + } + autopilot = { + cleanup_dead_servers = true + disable_upgrade_migration = true + last_contact_threshold = "12705s" + max_trailing_logs = 17849 + redundancy_zone_tag = "3IsufDJf" + server_stabilization_time = "23057s" + upgrade_version_tag = "W9pDwFAL" + } + bind_addr = "16.99.34.17" + bootstrap = true + bootstrap_expect = 53 + ca_file = "erA7T0PM" + ca_path = "mQEN1Mfp" + cert_file = "7s4QAzDk" + check = { + id = "fZaCAXww" + name = "OOM2eo0f" + notes = "zXzXI9Gt" + service_id = "L8G0QNmR" + token = "oo4BCTgJ" + status = "qLykAl5u" + script = "dhGfIF8n" + http = "29B93haH" + header = { + hBq0zn1q = [ "2a9o9ZKP", "vKwA5lR6" ] + f3r6xFtM = [ "RyuIdDWv", "QbxEcIUM" ] + } + method = "Dou0nGT5" + tcp = "JY6fTTcw" + interval = "18714s" + docker_container_id = "qF66POS9" + shell = "sOnDy228" + tls_skip_verify = true + timeout = "5954s" + ttl = "30044s" + deregister_critical_service_after = "13209s" + }, + checks = [ + { + id = "uAjE6m9Z" + name = "QsZRGpYr" + notes = "VJ7Sk4BY" + service_id = "lSulPcyz" + token = "toO59sh8" + status = "9RlWsXMV" + script = "8qbd8tWw" + http = "dohLcyQ2" + header = { + "ZBfTin3L" = [ "1sDbEqYG", "lJGASsWK" ] + "Ui0nU99X" = [ "LMccm3Qe", "k5H5RggQ" ] + } + method = "aldrIQ4l" + tcp = "RJQND605" + interval = "22164s" + docker_container_id = "ipgdFtjd" + shell = "qAeOYy0M" + tls_skip_verify = true + timeout = "1813s" + ttl = "21743s" + deregister_critical_service_after = "14232s" + }, + { + id = "Cqq95BhP" + name = "3qXpkS0i" + notes = "sb5qLTex" + service_id = "CmUUcRna" + token = "a3nQzHuy" + status = "irj26nf3" + script = "FJsI1oXt" + http = "yzhgsQ7Y" + header = { + "zcqwA8dO" = [ "qb1zx0DL", "sXCxPFsD" ] + "qxvdnSE9" = [ "6wBPUYdF", "YYh8wtSZ" ] + } + method = "gLrztrNw" + tcp = "4jG5casb" + interval = "28767s" + docker_container_id = "THW6u7rL" + shell = "C1Zt3Zwh" + tls_skip_verify = true + timeout = "18506s" + ttl = "31006s" + deregister_critical_service_after = "2366s" + } + ] + check_update_interval = "16507s" + client_addr = "93.83.18.19" + data_dir = "` + dataDir + `" + datacenter = "rzo029wg" + disable_anonymous_signature = true + disable_coordinates = true + disable_host_node_id = true + disable_keyring_file = true + disable_remote_exec = true + disable_update_check = true + domain = "7W1xXSqd" + dns_config { + allow_stale = true + disable_compression = true + enable_truncate = true + max_stale = "29685s" + node_ttl = "7084s" + only_passing = true + recursor_timeout = "4427s" + service_ttl = { + "*" = "32030s" + } + udp_answer_limit = 29909 + } + enable_acl_replication = true + enable_debug = true + enable_script_checks = true + enable_syslog = true + enable_ui = true + encrypt = "A4wELWqH" + encrypt_verify_incoming = true + encrypt_verify_outgoing = true + http_config { + block_endpoints = [ "RBvAFcGD", "fWOWFznh" ] + response_headers = { + "M6TKa9NP" = "xjuxjOzQ" + "JRCrHZed" = "rl0mTx81" + } + } + key_file = "IEkkwgIA" + leave_on_terminate = true + limits { + rpc_rate = 12029.43 + rpc_max_burst = 44848 + } + log_level = "k1zo9Spt" + node_id = "AsUIlw99" + node_meta { + "5mgGQMBk" = "mJLtVMSG" + "A7ynFMJB" = "0Nx6RGab" + } + node_name = "otlLxGaI" + non_voting_server = true + performance { + raft_multiplier = 5 + } + pid_file = "43xN80Km" + ports { + dns = 7001, + http = 7999, + https = 15127 + server = 3757 + } + protocol = 30793 + raft_protocol = 19016 + reconnect_timeout = "23739s" + reconnect_timeout_wan = "26694s" + recursors = [ "FtFhoUHl", "UYkwck1k" ] + rejoin_after_leave = true + retry_interval = "8067s" + retry_interval_wan = "28866s" + retry_join = [ "pbsSFY7U", "l0qLtWij" ] + retry_join_wan = [ "PFsR02Ye", "rJdQIhER" ] + retry_max = 913 + retry_max_wan = 23160 + segment = "BC2NhTDi" + segments = [ + { + name = "PExYMe2E" + bind = "36.73.36.19" + port = 38295 + rpc_listener = true + advertise = "63.39.19.18" + }, + { + name = "UzCvJgup" + bind = "37.58.38.19" + port = 39292 + rpc_listener = true + advertise = "83.58.26.27" + } + ] + serf_lan = "99.43.63.15" + serf_wan = "67.88.33.19" + server = true + server_name = "Oerr9n1G" + service = { + id = "dLOXpSCI" + name = "o1ynPkp0" + tags = ["nkwshvM5", "NTDWn3ek"] + address = "cOlSOhbp" + token = "msy7iWER" + port = 24237 + enable_tag_override = true + check = { + check_id = "RMi85Dv8" + name = "iehanzuq" + status = "rCvn53TH" + notes = "fti5lfF3" + script = "rtj34nfd" + http = "dl3Fgme3" + header = { + rjm4DEd3 = [ "2m3m2Fls" ] + l4HwQ112 = [ "fk56MNlo", "dhLK56aZ" ] + } + method = "9afLm3Mj" + tcp = "fjiLFqVd" + interval = "23926s" + docker_container_id = "dO5TtRHk" + shell = "e6q2ttES" + tls_skip_verify = true + timeout = "38483s" + ttl = "10943s" + deregister_critical_service_after = "68787s" + } + checks = [ + { + id = "Zv99e9Ka" + name = "sgV4F7Pk" + notes = "yP5nKbW0" + status = "7oLMEyfu" + script = "NlUQ3nTE" + http = "KyDjGY9H" + header = { + "gv5qefTz" = [ "5Olo2pMG", "PvvKWQU5" ] + "SHOVq1Vv" = [ "jntFhyym", "GYJh32pp" ] + } + method = "T66MFBfR" + tcp = "bNnNfx2A" + interval = "22224s" + docker_container_id = "ipgdFtjd" + shell = "omVZq7Sz" + tls_skip_verify = true + timeout = "18913s" + ttl = "44743s" + deregister_critical_service_after = "8482s" + }, + { + id = "G79O6Mpr" + name = "IEqrzrsd" + notes = "SVqApqeM" + status = "XXkVoZXt" + script = "IXLZTM6E" + http = "kyICZsn8" + header = { + "4ebP5vL4" = [ "G20SrL5Q", "DwPKlMbo" ] + "p2UI34Qz" = [ "UsG1D0Qh", "NHhRiB6s" ] + } + method = "ciYHWors" + tcp = "FfvCwlqH" + interval = "12356s" + docker_container_id = "HBndBU6R" + shell = "hVI33JjA" + tls_skip_verify = true + timeout = "38282s" + ttl = "1181s" + deregister_critical_service_after = "4992s" + } + ] + } + services = [ + { + id = "wI1dzxS4" + name = "7IszXMQ1" + tags = ["0Zwg8l6v", "zebELdN5"] + address = "9RhqPSPB" + token = "myjKJkWH" + port = 72219 + enable_tag_override = true + check = { + check_id = "qmfeO5if" + name = "atDGP7n5" + status = "pDQKEhWL" + notes = "Yt8EDLev" + script = "MDu7wjlD" + http = "qzHYvmJO" + header = { + UkpmZ3a3 = [ "2dfzXuxZ" ] + cVFpko4u = [ "gGqdEB6k", "9LsRo22u" ] + } + method = "X5DrovFc" + tcp = "ICbxkpSF" + interval = "24392s" + docker_container_id = "ZKXr68Yb" + shell = "CEfzx0Fo" + tls_skip_verify = true + timeout = "38333s" + ttl = "57201s" + deregister_critical_service_after = "44214s" + } + }, + { + id = "MRHVMZuD" + name = "6L6BVfgH" + tags = ["7Ale4y6o", "PMBW08hy"] + address = "R6H6g8h0" + token = "ZgY8gjMI" + port = 38292 + enable_tag_override = true + checks = [ + { + id = "GTti9hCo" + name = "9OOS93ne" + notes = "CQy86DH0" + status = "P0SWDvrk" + script = "6BhLJ7R9" + http = "u97ByEiW" + header = { + "MUlReo8L" = [ "AUZG7wHG", "gsN0Dc2N" ] + "1UJXjVrT" = [ "OJgxzTfk", "xZZrFsq7" ] + } + method = "5wkAxCUE" + tcp = "MN3oA9D2" + interval = "32718s" + docker_container_id = "cU15LMet" + shell = "nEz9qz2l" + tls_skip_verify = true + timeout = "34738s" + ttl = "22773s" + deregister_critical_service_after = "84282s" + }, + { + id = "UHsDeLxG" + name = "PQSaPWlT" + notes = "jKChDOdl" + status = "5qFz6OZn" + script = "PbdxFZ3K" + http = "1LBDJhw4" + header = { + "cXPmnv1M" = [ "imDqfaBx", "NFxZ1bQe" ], + "vr7wY7CS" = [ "EtCoNPPL", "9vAarJ5s" ] + } + method = "wzByP903" + tcp = "2exjZIGE" + interval = "5656s" + docker_container_id = "5tDBWpfA" + shell = "rlTpLM8s" + tls_skip_verify = true + timeout = "4868s" + ttl = "11222s" + deregister_critical_service_after = "68482s" + } + ] + } + ] + session_ttl_min = "26627s" + skip_leave_on_interrupt = true + start_join = [ "LR3hGDoG", "MwVpZ4Up" ] + start_join_wan = [ "EbFSc3nA", "kwXTh623" ] + syslog_facility = "hHv79Uia" + tagged_addresses = { + "7MYgHrYH" = "dALJAhLD" + "h6DdBy6K" = "ebrr9zZ8" + } + telemetry { + circonus_api_app = "p4QOTe9j" + circonus_api_token = "E3j35V23" + circonus_api_url = "mEMjHpGg" + circonus_broker_id = "BHlxUhed" + circonus_broker_select_tag = "13xy1gHm" + circonus_check_display_name = "DRSlQR6n" + circonus_check_force_metric_activation = "Ua5FGVYf" + circonus_check_id = "kGorutad" + circonus_check_instance_id = "rwoOL6R4" + circonus_check_search_tag = "ovT4hT4f" + circonus_check_tags = "prvO4uBl" + circonus_submission_interval = "DolzaflP" + circonus_submission_url = "gTcbS93G" + disable_hostname = true + dogstatsd_addr = "0wSndumK" + dogstatsd_tags = [ "3N81zSUB","Xtj8AnXZ" ] + filter_default = true + prefix_filter = [ "+oJotS8XJ","-cazlEhGn" ] + statsd_address = "drce87cy" + statsite_address = "HpFwKB8R" + statsite_prefix = "ftO6DySn" + } + tls_cipher_suites = "TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA" + tls_min_version = "pAOWafkR" + tls_prefer_server_cipher_suites = true + translate_wan_addrs = true + ui_dir = "11IFzAUn" + unix_sockets = { + group = "8pFodrV8" + mode = "E8sAwOv4" + user = "E0nB1DwA" + } + verify_incoming = true + verify_incoming_https = true + verify_incoming_rpc = true + verify_outgoing = true + verify_server_hostname = true + watches = [{ + type = "key" + datacenter = "GyE6jpeW" + key = "j9lF1Tve" + handler = "90N7S4LN" + }] + `} + + tail := map[string][]Source{ + "json": []Source{ + { + Name: "tail.non-user.json", + Format: "json", + Data: ` + { + "acl_disabled_ttl": "957s", + "ae_interval": "10003s", + "check_deregister_interval_min": "27870s", + "check_reap_interval": "10662s", + "segment_limit": 24705, + "segment_name_limit": 27046, + "sync_coordinate_interval_min": "27983s", + "sync_coordinate_rate_target": 137.81 + }`, + }, + { + Name: "tail.consul.json", + Format: "json", + Data: ` + { + "consul": { + "coordinate": { + "update_batch_size": 9244, + "update_max_batches": 15164, + "update_period": "25093s" + }, + "raft": { + "election_timeout": "31947s", + "heartbeat_timeout": "25699s", + "leader_lease_timeout": "15351s" + }, + "serf_lan": { + "memberlist": { + "gossip_interval": "25252s", + "probe_interval": "5105s", + "probe_timeout": "29179s", + "suspicion_mult": 8263 + } + }, + "serf_wan": { + "memberlist": { + "gossip_interval": "6966s", + "probe_interval": "20148s", + "probe_timeout": "3007s", + "suspicion_mult": 32096 + } + }, + "server": { + "health_interval": "17455s" + } + } + }`, + }, + }, + "hcl": []Source{ + { + Name: "tail.non-user.hcl", + Format: "hcl", + Data: ` + acl_disabled_ttl = "957s" + ae_interval = "10003s" + check_deregister_interval_min = "27870s" + check_reap_interval = "10662s" + segment_limit = 24705 + segment_name_limit = 27046 + sync_coordinate_interval_min = "27983s" + sync_coordinate_rate_target = 137.81 + `, + }, + { + Name: "tail.consul.hcl", + Format: "hcl", + Data: ` + consul = { + coordinate = { + update_batch_size = 9244 + update_max_batches = 15164 + update_period = "25093s" + } + raft = { + election_timeout = "31947s" + heartbeat_timeout = "25699s" + leader_lease_timeout = "15351s" + } + serf_lan = { + memberlist = { + gossip_interval = "25252s" + probe_interval = "5105s" + probe_timeout = "29179s" + suspicion_mult = 8263 + } + } + serf_wan = { + memberlist = { + gossip_interval = "6966s" + probe_interval = "20148s" + probe_timeout = "3007s" + suspicion_mult = 32096 + } + } + server = { + health_interval = "17455s" + } + } + `, + }, + }, + } + + want := RuntimeConfig{ + // non-user configurable values + ACLDisabledTTL: 957 * time.Second, + AEInterval: 10003 * time.Second, + CheckDeregisterIntervalMin: 27870 * time.Second, + CheckReapInterval: 10662 * time.Second, + SegmentLimit: 24705, + SegmentNameLimit: 27046, + SyncCoordinateIntervalMin: 27983 * time.Second, + SyncCoordinateRateTarget: 137.81, + + Revision: "JNtPSav3", + Version: "R909Hblt", + VersionPrerelease: "ZT1JOQLn", + + // consul configuration + ConsulCoordinateUpdateBatchSize: 9244, + ConsulCoordinateUpdateMaxBatches: 15164, + ConsulCoordinateUpdatePeriod: 25093 * time.Second, + ConsulRaftElectionTimeout: 5 * 31947 * time.Second, + ConsulRaftHeartbeatTimeout: 5 * 25699 * time.Second, + ConsulRaftLeaderLeaseTimeout: 5 * 15351 * time.Second, + ConsulSerfLANGossipInterval: 25252 * time.Second, + ConsulSerfLANProbeInterval: 5105 * time.Second, + ConsulSerfLANProbeTimeout: 29179 * time.Second, + ConsulSerfLANSuspicionMult: 8263, + ConsulSerfWANGossipInterval: 6966 * time.Second, + ConsulSerfWANProbeInterval: 20148 * time.Second, + ConsulSerfWANProbeTimeout: 3007 * time.Second, + ConsulSerfWANSuspicionMult: 32096, + ConsulServerHealthInterval: 17455 * time.Second, + + // user configurable values + + ACLAgentMasterToken: "furuQD0b", + ACLAgentToken: "cOshLOQ2", + ACLDatacenter: "m3urck3z", + ACLDefaultPolicy: "ArK3WIfE", + ACLDownPolicy: "vZXMfMP0", + ACLEnforceVersion8: true, + ACLMasterToken: "C1Q1oIwh", + ACLReplicationToken: "LMmgy5dO", + ACLTTL: 18060 * time.Second, + ACLToken: "O1El0wan", + AdvertiseAddrLAN: ipAddr("17.99.29.16"), + AdvertiseAddrWAN: ipAddr("78.63.37.19"), + AutopilotCleanupDeadServers: true, + AutopilotDisableUpgradeMigration: true, + AutopilotLastContactThreshold: 12705 * time.Second, + AutopilotMaxTrailingLogs: 17849, + AutopilotRedundancyZoneTag: "3IsufDJf", + AutopilotServerStabilizationTime: 23057 * time.Second, + AutopilotUpgradeVersionTag: "W9pDwFAL", + BindAddr: ipAddr("16.99.34.17"), + Bootstrap: true, + BootstrapExpect: 53, + CAFile: "erA7T0PM", + CAPath: "mQEN1Mfp", + CertFile: "7s4QAzDk", + Checks: []*structs.CheckDefinition{ + &structs.CheckDefinition{ + ID: "fZaCAXww", + Name: "OOM2eo0f", + Notes: "zXzXI9Gt", + ServiceID: "L8G0QNmR", + Token: "oo4BCTgJ", + Status: "qLykAl5u", + Script: "dhGfIF8n", + HTTP: "29B93haH", + Header: map[string][]string{ + "hBq0zn1q": {"2a9o9ZKP", "vKwA5lR6"}, + "f3r6xFtM": {"RyuIdDWv", "QbxEcIUM"}, + }, + Method: "Dou0nGT5", + TCP: "JY6fTTcw", + Interval: 18714 * time.Second, + DockerContainerID: "qF66POS9", + Shell: "sOnDy228", + TLSSkipVerify: true, + Timeout: 5954 * time.Second, + TTL: 30044 * time.Second, + DeregisterCriticalServiceAfter: 13209 * time.Second, + }, + &structs.CheckDefinition{ + ID: "uAjE6m9Z", + Name: "QsZRGpYr", + Notes: "VJ7Sk4BY", + ServiceID: "lSulPcyz", + Token: "toO59sh8", + Status: "9RlWsXMV", + Script: "8qbd8tWw", + HTTP: "dohLcyQ2", + Header: map[string][]string{ + "ZBfTin3L": []string{"1sDbEqYG", "lJGASsWK"}, + "Ui0nU99X": []string{"LMccm3Qe", "k5H5RggQ"}, + }, + Method: "aldrIQ4l", + TCP: "RJQND605", + Interval: 22164 * time.Second, + DockerContainerID: "ipgdFtjd", + Shell: "qAeOYy0M", + TLSSkipVerify: true, + Timeout: 1813 * time.Second, + TTL: 21743 * time.Second, + DeregisterCriticalServiceAfter: 14232 * time.Second, + }, + &structs.CheckDefinition{ + ID: "Cqq95BhP", + Name: "3qXpkS0i", + Notes: "sb5qLTex", + ServiceID: "CmUUcRna", + Token: "a3nQzHuy", + Status: "irj26nf3", + Script: "FJsI1oXt", + HTTP: "yzhgsQ7Y", + Header: map[string][]string{ + "zcqwA8dO": []string{"qb1zx0DL", "sXCxPFsD"}, + "qxvdnSE9": []string{"6wBPUYdF", "YYh8wtSZ"}, + }, + Method: "gLrztrNw", + TCP: "4jG5casb", + Interval: 28767 * time.Second, + DockerContainerID: "THW6u7rL", + Shell: "C1Zt3Zwh", + TLSSkipVerify: true, + Timeout: 18506 * time.Second, + TTL: 31006 * time.Second, + DeregisterCriticalServiceAfter: 2366 * time.Second, + }, + }, + CheckUpdateInterval: 16507 * time.Second, + ClientAddrs: []*net.IPAddr{ipAddr("93.83.18.19")}, + DNSAddrs: []net.Addr{tcpAddr("93.95.95.81:7001"), udpAddr("93.95.95.81:7001")}, + DNSAllowStale: true, + DNSDisableCompression: true, + DNSDomain: "7W1xXSqd", + DNSEnableTruncate: true, + DNSMaxStale: 29685 * time.Second, + DNSNodeTTL: 7084 * time.Second, + DNSOnlyPassing: true, + DNSPort: 7001, + DNSRecursorTimeout: 4427 * time.Second, + DNSRecursors: []string{"FtFhoUHl", "UYkwck1k"}, + DNSServiceTTL: map[string]time.Duration{"*": 32030 * time.Second}, + DNSUDPAnswerLimit: 29909, + DataDir: dataDir, + Datacenter: "rzo029wg", + DevMode: true, + DisableAnonymousSignature: true, + DisableCoordinates: true, + DisableHostNodeID: true, + DisableKeyringFile: true, + DisableRemoteExec: true, + DisableUpdateCheck: true, + EnableACLReplication: true, + EnableDebug: true, + EnableScriptChecks: true, + EnableSyslog: true, + EnableUI: true, + EncryptKey: "A4wELWqH", + EncryptVerifyIncoming: true, + EncryptVerifyOutgoing: true, + HTTPAddrs: []net.Addr{tcpAddr("83.39.91.39:7999")}, + HTTPBlockEndpoints: []string{"RBvAFcGD", "fWOWFznh"}, + HTTPPort: 7999, + HTTPResponseHeaders: map[string]string{"M6TKa9NP": "xjuxjOzQ", "JRCrHZed": "rl0mTx81"}, + HTTPSAddrs: []net.Addr{tcpAddr("95.17.17.19:15127")}, + HTTPSPort: 15127, + KeyFile: "IEkkwgIA", + LeaveOnTerm: true, + LogLevel: "k1zo9Spt", + NodeID: types.NodeID("AsUIlw99"), + NodeMeta: map[string]string{"5mgGQMBk": "mJLtVMSG", "A7ynFMJB": "0Nx6RGab"}, + NodeName: "otlLxGaI", + NonVotingServer: true, + PidFile: "43xN80Km", + RPCAdvertiseAddr: tcpAddr("28.27.94.38:3757"), + RPCBindAddr: tcpAddr("16.99.34.17:3757"), + RPCProtocol: 30793, + RPCRateLimit: 12029.43, + RPCMaxBurst: 44848, + RaftProtocol: 19016, + ReconnectTimeoutLAN: 23739 * time.Second, + ReconnectTimeoutWAN: 26694 * time.Second, + RejoinAfterLeave: true, + RetryJoinIntervalLAN: 8067 * time.Second, + RetryJoinIntervalWAN: 28866 * time.Second, + RetryJoinLAN: []string{"pbsSFY7U", "l0qLtWij"}, + RetryJoinMaxAttemptsLAN: 913, + RetryJoinMaxAttemptsWAN: 23160, + RetryJoinWAN: []string{"PFsR02Ye", "rJdQIhER"}, + SegmentName: "BC2NhTDi", + Segments: []structs.NetworkSegment{ + { + Name: "PExYMe2E", + Bind: tcpAddr("36.73.36.19:38295"), + Advertise: tcpAddr("63.39.19.18:38295"), + RPCListener: true, + }, + { + Name: "UzCvJgup", + Bind: tcpAddr("37.58.38.19:39292"), + Advertise: tcpAddr("83.58.26.27:39292"), + RPCListener: true, + }, + }, + SerfPortLAN: 8301, + SerfPortWAN: 8302, + ServerMode: true, + ServerName: "Oerr9n1G", + ServerPort: 3757, + Services: []*structs.ServiceDefinition{ + { + ID: "wI1dzxS4", + Name: "7IszXMQ1", + Tags: []string{"0Zwg8l6v", "zebELdN5"}, + Address: "9RhqPSPB", + Token: "myjKJkWH", + Port: 72219, + EnableTagOverride: true, + Checks: []*structs.CheckType{ + &structs.CheckType{ + CheckID: "qmfeO5if", + Name: "atDGP7n5", + Status: "pDQKEhWL", + Notes: "Yt8EDLev", + Script: "MDu7wjlD", + HTTP: "qzHYvmJO", + Header: map[string][]string{ + "UkpmZ3a3": {"2dfzXuxZ"}, + "cVFpko4u": {"gGqdEB6k", "9LsRo22u"}, + }, + Method: "X5DrovFc", + TCP: "ICbxkpSF", + Interval: 24392 * time.Second, + DockerContainerID: "ZKXr68Yb", + Shell: "CEfzx0Fo", + TLSSkipVerify: true, + Timeout: 38333 * time.Second, + TTL: 57201 * time.Second, + DeregisterCriticalServiceAfter: 44214 * time.Second, + }, + }, + }, + { + ID: "MRHVMZuD", + Name: "6L6BVfgH", + Tags: []string{"7Ale4y6o", "PMBW08hy"}, + Address: "R6H6g8h0", + Token: "ZgY8gjMI", + Port: 38292, + EnableTagOverride: true, + Checks: structs.CheckTypes{ + &structs.CheckType{ + CheckID: "GTti9hCo", + Name: "9OOS93ne", + Notes: "CQy86DH0", + Status: "P0SWDvrk", + Script: "6BhLJ7R9", + HTTP: "u97ByEiW", + Header: map[string][]string{ + "MUlReo8L": {"AUZG7wHG", "gsN0Dc2N"}, + "1UJXjVrT": {"OJgxzTfk", "xZZrFsq7"}, + }, + Method: "5wkAxCUE", + TCP: "MN3oA9D2", + Interval: 32718 * time.Second, + DockerContainerID: "cU15LMet", + Shell: "nEz9qz2l", + TLSSkipVerify: true, + Timeout: 34738 * time.Second, + TTL: 22773 * time.Second, + DeregisterCriticalServiceAfter: 84282 * time.Second, + }, + &structs.CheckType{ + CheckID: "UHsDeLxG", + Name: "PQSaPWlT", + Notes: "jKChDOdl", + Status: "5qFz6OZn", + Script: "PbdxFZ3K", + HTTP: "1LBDJhw4", + Header: map[string][]string{ + "cXPmnv1M": {"imDqfaBx", "NFxZ1bQe"}, + "vr7wY7CS": {"EtCoNPPL", "9vAarJ5s"}, + }, + Method: "wzByP903", + TCP: "2exjZIGE", + Interval: 5656 * time.Second, + DockerContainerID: "5tDBWpfA", + Shell: "rlTpLM8s", + TLSSkipVerify: true, + Timeout: 4868 * time.Second, + TTL: 11222 * time.Second, + DeregisterCriticalServiceAfter: 68482 * time.Second, + }, + }, + }, + { + ID: "dLOXpSCI", + Name: "o1ynPkp0", + Tags: []string{"nkwshvM5", "NTDWn3ek"}, + Address: "cOlSOhbp", + Token: "msy7iWER", + Port: 24237, + EnableTagOverride: true, + Checks: structs.CheckTypes{ + &structs.CheckType{ + CheckID: "Zv99e9Ka", + Name: "sgV4F7Pk", + Notes: "yP5nKbW0", + Status: "7oLMEyfu", + Script: "NlUQ3nTE", + HTTP: "KyDjGY9H", + Header: map[string][]string{ + "gv5qefTz": {"5Olo2pMG", "PvvKWQU5"}, + "SHOVq1Vv": {"jntFhyym", "GYJh32pp"}, + }, + Method: "T66MFBfR", + TCP: "bNnNfx2A", + Interval: 22224 * time.Second, + DockerContainerID: "ipgdFtjd", + Shell: "omVZq7Sz", + TLSSkipVerify: true, + Timeout: 18913 * time.Second, + TTL: 44743 * time.Second, + DeregisterCriticalServiceAfter: 8482 * time.Second, + }, + &structs.CheckType{ + CheckID: "G79O6Mpr", + Name: "IEqrzrsd", + Notes: "SVqApqeM", + Status: "XXkVoZXt", + Script: "IXLZTM6E", + HTTP: "kyICZsn8", + Header: map[string][]string{ + "4ebP5vL4": {"G20SrL5Q", "DwPKlMbo"}, + "p2UI34Qz": {"UsG1D0Qh", "NHhRiB6s"}, + }, + Method: "ciYHWors", + TCP: "FfvCwlqH", + Interval: 12356 * time.Second, + DockerContainerID: "HBndBU6R", + Shell: "hVI33JjA", + TLSSkipVerify: true, + Timeout: 38282 * time.Second, + TTL: 1181 * time.Second, + DeregisterCriticalServiceAfter: 4992 * time.Second, + }, + &structs.CheckType{ + CheckID: "RMi85Dv8", + Name: "iehanzuq", + Status: "rCvn53TH", + Notes: "fti5lfF3", + Script: "rtj34nfd", + HTTP: "dl3Fgme3", + Header: map[string][]string{ + "rjm4DEd3": {"2m3m2Fls"}, + "l4HwQ112": {"fk56MNlo", "dhLK56aZ"}, + }, + Method: "9afLm3Mj", + TCP: "fjiLFqVd", + Interval: 23926 * time.Second, + DockerContainerID: "dO5TtRHk", + Shell: "e6q2ttES", + TLSSkipVerify: true, + Timeout: 38483 * time.Second, + TTL: 10943 * time.Second, + DeregisterCriticalServiceAfter: 68787 * time.Second, + }, + }, + }, + }, + SerfAdvertiseAddrLAN: tcpAddr("49.38.36.95:8301"), + SerfAdvertiseAddrWAN: tcpAddr("63.38.52.13:8302"), + SerfBindAddrLAN: tcpAddr("99.43.63.15:8301"), + SerfBindAddrWAN: tcpAddr("67.88.33.19:8302"), + SessionTTLMin: 26627 * time.Second, + SkipLeaveOnInt: true, + StartJoinAddrsLAN: []string{"LR3hGDoG", "MwVpZ4Up"}, + StartJoinAddrsWAN: []string{"EbFSc3nA", "kwXTh623"}, + SyslogFacility: "hHv79Uia", + TelemetryCirconusAPIApp: "p4QOTe9j", + TelemetryCirconusAPIToken: "E3j35V23", + TelemetryCirconusAPIURL: "mEMjHpGg", + TelemetryCirconusBrokerID: "BHlxUhed", + TelemetryCirconusBrokerSelectTag: "13xy1gHm", + TelemetryCirconusCheckDisplayName: "DRSlQR6n", + TelemetryCirconusCheckForceMetricActivation: "Ua5FGVYf", + TelemetryCirconusCheckID: "kGorutad", + TelemetryCirconusCheckInstanceID: "rwoOL6R4", + TelemetryCirconusCheckSearchTag: "ovT4hT4f", + TelemetryCirconusCheckTags: "prvO4uBl", + TelemetryCirconusSubmissionInterval: "DolzaflP", + TelemetryCirconusSubmissionURL: "gTcbS93G", + TelemetryDisableHostname: true, + TelemetryDogstatsdAddr: "0wSndumK", + TelemetryDogstatsdTags: []string{"3N81zSUB", "Xtj8AnXZ"}, + TelemetryFilterDefault: true, + TelemetryAllowedPrefixes: []string{"oJotS8XJ"}, + TelemetryBlockedPrefixes: []string{"cazlEhGn"}, + TelemetryStatsdAddr: "drce87cy", + TelemetryStatsiteAddr: "HpFwKB8R", + TelemetryStatsitePrefix: "ftO6DySn", + TLSCipherSuites: []uint16{tls.TLS_RSA_WITH_RC4_128_SHA, tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA}, + TLSMinVersion: "pAOWafkR", + TLSPreferServerCipherSuites: true, + TaggedAddresses: map[string]string{ + "7MYgHrYH": "dALJAhLD", + "h6DdBy6K": "ebrr9zZ8", + "lan": "17.99.29.16", + "wan": "78.63.37.19", + }, + TranslateWANAddrs: true, + UIDir: "11IFzAUn", + UnixSocketUser: "E0nB1DwA", + UnixSocketGroup: "8pFodrV8", + UnixSocketMode: "E8sAwOv4", + VerifyIncoming: true, + VerifyIncomingHTTPS: true, + VerifyIncomingRPC: true, + VerifyOutgoing: true, + VerifyServerHostname: true, + Watches: []map[string]interface{}{ + map[string]interface{}{ + "type": "key", + "datacenter": "GyE6jpeW", + "key": "j9lF1Tve", + "handler": "90N7S4LN", + }, + }, + } + + warns := []string{ + `bootstrap_expect > 0: expecting 53 servers`, + } + + // ensure that all fields are set to unique non-zero values + // todo(fs): This currently fails since ServiceDefinition.Check is not used + // todo(fs): not sure on how to work around this. Possible options are: + // todo(fs): * move first check into the Check field + // todo(fs): * ignore the Check field + // todo(fs): both feel like a hack + if err := nonZero("RuntimeConfig", nil, want); err != nil { + t.Log(err) + } + + for format, data := range src { + t.Run(format, func(t *testing.T) { + // parse the flags since this is the only way we can set the + // DevMode flag + var flags Flags + fs := flag.NewFlagSet("", flag.ContinueOnError) + AddFlags(fs, &flags) + if err := fs.Parse(flagSrc); err != nil { + t.Fatalf("ParseFlags: %s", err) + } + + // ensure that all fields are set to unique non-zero values + // if err := nonZero("Config", nil, c); err != nil { + // t.Fatal(err) + // } + + b, err := NewBuilder(flags) + if err != nil { + t.Fatalf("NewBuilder: %s", err) + } + b.Sources = append(b.Sources, Source{Name: "full", Format: format, Data: data}) + b.Tail = append(b.Tail, tail[format]...) + b.Tail = append(b.Tail, VersionSource("JNtPSav3", "R909Hblt", "ZT1JOQLn")) + + // construct the runtime config + rt, err := b.Build() + if err != nil { + t.Fatalf("Build: %s", err) + } + + // verify that all fields are set + if !verify.Values(t, "runtime_config", rt, want) { + t.FailNow() + } + + // at this point we have confirmed that the parsing worked + // for all fields but the validation will fail since certain + // combinations are not allowed. Since it is not possible to have + // all fields with non-zero values and to have a valid configuration + // we are patching a handful of safe fields to make validation pass. + rt.Bootstrap = false + rt.DevMode = false + rt.EnableUI = false + rt.SegmentName = "" + rt.Segments = nil + + // validate the runtime config + if err := b.Validate(rt); err != nil { + t.Fatalf("Validate: %s", err) + } + + // check the warnings + if got, want := b.Warnings, warns; !verify.Values(t, "warnings", got, want) { + t.FailNow() + } + }) + } +} + +// nonZero verifies recursively that all fields are set to unique, +// non-zero and non-nil values. +// +// struct: check all fields recursively +// slice: check len > 0 and all values recursively +// ptr: check not nil +// bool: check not zero (cannot check uniqueness) +// string, int, uint: check not zero and unique +// other: error +func nonZero(name string, uniq map[interface{}]string, v interface{}) error { + if v == nil { + return fmt.Errorf("%q is nil", name) + } + + if uniq == nil { + uniq = map[interface{}]string{} + } + + isUnique := func(v interface{}) error { + if other := uniq[v]; other != "" { + return fmt.Errorf("%q and %q both use vaule %q", name, other, v) + } + uniq[v] = name + return nil + } + + val, typ := reflect.ValueOf(v), reflect.TypeOf(v) + // fmt.Printf("%s: %T\n", name, v) + switch typ.Kind() { + case reflect.Struct: + for i := 0; i < typ.NumField(); i++ { + f := typ.Field(i) + fieldname := fmt.Sprintf("%s.%s", name, f.Name) + err := nonZero(fieldname, uniq, val.Field(i).Interface()) + if err != nil { + return err + } + } + + case reflect.Slice: + if val.Len() == 0 { + return fmt.Errorf("%q is empty slice", name) + } + for i := 0; i < val.Len(); i++ { + elemname := fmt.Sprintf("%s[%d]", name, i) + err := nonZero(elemname, uniq, val.Index(i).Interface()) + if err != nil { + return err + } + } + + case reflect.Map: + if val.Len() == 0 { + return fmt.Errorf("%q is empty map", name) + } + for _, key := range val.MapKeys() { + keyname := fmt.Sprintf("%s[%s]", name, key.String()) + if err := nonZero(keyname, uniq, key.Interface()); err != nil { + if strings.Contains(err.Error(), "is zero value") { + return fmt.Errorf("%q has zero value map key", name) + } + return err + } + if err := nonZero(keyname, uniq, val.MapIndex(key).Interface()); err != nil { + return err + } + } + + case reflect.Bool: + if val.Bool() != true { + return fmt.Errorf("%q is zero value", name) + } + // do not test bool for uniqueness since there are only two values + + case reflect.String: + if val.Len() == 0 { + return fmt.Errorf("%q is zero value", name) + } + return isUnique(v) + + case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: + if val.Int() == 0 { + return fmt.Errorf("%q is zero value", name) + } + return isUnique(v) + + case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64: + if val.Uint() == 0 { + return fmt.Errorf("%q is zero value", name) + } + return isUnique(v) + + case reflect.Float32, reflect.Float64: + if val.Float() == 0 { + return fmt.Errorf("%q is zero value", name) + } + return isUnique(v) + + case reflect.Ptr: + if val.IsNil() { + return fmt.Errorf("%q is nil", name) + } + return nonZero("*"+name, uniq, val.Elem().Interface()) + + default: + return fmt.Errorf("%T is not supported", v) + } + return nil +} + +func TestNonZero(t *testing.T) { + var empty string + + tests := []struct { + desc string + v interface{} + err error + }{ + {"nil", nil, errors.New(`"x" is nil`)}, + {"zero bool", false, errors.New(`"x" is zero value`)}, + {"zero string", "", errors.New(`"x" is zero value`)}, + {"zero int", int(0), errors.New(`"x" is zero value`)}, + {"zero int8", int8(0), errors.New(`"x" is zero value`)}, + {"zero int16", int16(0), errors.New(`"x" is zero value`)}, + {"zero int32", int32(0), errors.New(`"x" is zero value`)}, + {"zero int64", int64(0), errors.New(`"x" is zero value`)}, + {"zero uint", uint(0), errors.New(`"x" is zero value`)}, + {"zero uint8", uint8(0), errors.New(`"x" is zero value`)}, + {"zero uint16", uint16(0), errors.New(`"x" is zero value`)}, + {"zero uint32", uint32(0), errors.New(`"x" is zero value`)}, + {"zero uint64", uint64(0), errors.New(`"x" is zero value`)}, + {"zero float32", float32(0), errors.New(`"x" is zero value`)}, + {"zero float64", float64(0), errors.New(`"x" is zero value`)}, + {"ptr to zero value", &empty, errors.New(`"*x" is zero value`)}, + {"empty slice", []string{}, errors.New(`"x" is empty slice`)}, + {"slice with zero value", []string{""}, errors.New(`"x[0]" is zero value`)}, + {"empty map", map[string]string{}, errors.New(`"x" is empty map`)}, + {"map with zero value key", map[string]string{"": "y"}, errors.New(`"x" has zero value map key`)}, + {"map with zero value elem", map[string]string{"y": ""}, errors.New(`"x[y]" is zero value`)}, + {"struct with nil field", struct{ Y *int }{}, errors.New(`"x.Y" is nil`)}, + {"struct with zero value field", struct{ Y string }{}, errors.New(`"x.Y" is zero value`)}, + {"struct with empty array", struct{ Y []string }{}, errors.New(`"x.Y" is empty slice`)}, + } + + for _, tt := range tests { + t.Run(tt.desc, func(t *testing.T) { + if got, want := nonZero("x", nil, tt.v), tt.err; !reflect.DeepEqual(got, want) { + t.Fatalf("got error %v want %v", got, want) + } + }) + } +} + +func TestConfigDecodeBytes(t *testing.T) { + t.Parallel() + // Test with some input + src := []byte("abc") + key := base64.StdEncoding.EncodeToString(src) + + result, err := decodeBytes(key) + if err != nil { + t.Fatalf("err: %s", err) + } + + if !bytes.Equal(src, result) { + t.Fatalf("bad: %#v", result) + } + + // Test with no input + result, err = decodeBytes("") + if err != nil { + t.Fatalf("err: %s", err) + } + + if len(result) > 0 { + t.Fatalf("bad: %#v", result) + } +} + +func TestSanitize(t *testing.T) { + rt := RuntimeConfig{ + RetryJoinLAN: []string{ + "foo=bar key=baz secret=boom bang=bar", + }, + RetryJoinWAN: []string{ + "wan_foo=bar wan_key=baz wan_secret=boom wan_bang=bar", + }, + } + + want := RuntimeConfig{ + ACLAgentMasterToken: "hidden", + ACLAgentToken: "hidden", + ACLMasterToken: "hidden", + ACLReplicationToken: "hidden", + ACLToken: "hidden", + EncryptKey: "hidden", + KeyFile: "hidden", + TelemetryCirconusAPIToken: "hidden", + RetryJoinLAN: []string{ + "foo=bar key=hidden secret=hidden bang=bar", + }, + RetryJoinWAN: []string{ + "wan_foo=bar wan_key=hidden wan_secret=hidden wan_bang=bar", + }, + } + + if got := rt.Sanitized(); !verify.Values(t, "", got, want) { + t.Fail() + } +} + +func splitIPPort(hostport string) (net.IP, int) { + h, p, err := net.SplitHostPort(hostport) + if err != nil { + panic(err) + } + port, err := strconv.Atoi(p) + if err != nil { + panic(err) + } + return net.ParseIP(h), port +} + +func ipAddr(addr string) *net.IPAddr { + return &net.IPAddr{IP: net.ParseIP(addr)} +} + +func tcpAddr(addr string) *net.TCPAddr { + ip, port := splitIPPort(addr) + return &net.TCPAddr{IP: ip, Port: port} +} + +func udpAddr(addr string) *net.UDPAddr { + ip, port := splitIPPort(addr) + return &net.UDPAddr{IP: ip, Port: port} +} + +func unixAddr(addr string) *net.UnixAddr { + if !strings.HasPrefix(addr, "unix://") { + panic("not a unix socket addr: " + addr) + } + return &net.UnixAddr{Net: "unix", Name: addr[len("unix://"):]} +} + +func writeFile(path string, data []byte) { + if err := os.MkdirAll(filepath.Dir(path), 0750); err != nil { + panic(err) + } + if err := ioutil.WriteFile(path, data, 0640); err != nil { + panic(err) + } +} + +func randomString(n int) string { + s := "" + for ; n > 0; n-- { + s += "x" + } + return s +} + +func metaPairs(n int, format string) string { + var s []string + for i := 0; i < n; i++ { + switch format { + case "json": + s = append(s, fmt.Sprintf(`"%d":"%d"`, i, i)) + case "hcl": + s = append(s, fmt.Sprintf(`"%d"="%d"`, i, i)) + default: + panic("invalid format: " + format) + } + } + switch format { + case "json": + return strings.Join(s, ",") + case "hcl": + return strings.Join(s, " ") + default: + panic("invalid format: " + format) + } +} diff --git a/agent/config/segments_oss.go b/agent/config/segments_oss.go new file mode 100644 index 000000000..dd8c36539 --- /dev/null +++ b/agent/config/segments_oss.go @@ -0,0 +1,17 @@ +// +build !ent + +package config + +import ( + "github.com/hashicorp/consul/agent/structs" +) + +func (b *Builder) validateSegments(rt RuntimeConfig) error { + if rt.SegmentName != "" { + return structs.ErrSegmentsNotSupported + } + if len(rt.Segments) > 0 { + return structs.ErrSegmentsNotSupported + } + return nil +} diff --git a/agent/config/segments_oss_test.go b/agent/config/segments_oss_test.go new file mode 100644 index 000000000..35b6db008 --- /dev/null +++ b/agent/config/segments_oss_test.go @@ -0,0 +1,45 @@ +// +build !ent + +package config + +import ( + "net" + "os" + "testing" + + "github.com/hashicorp/consul/testutil" +) + +func TestSegments(t *testing.T) { + dataDir := testutil.TempDir(t, "consul") + defer os.RemoveAll(dataDir) + + tests := []configTest{ + { + desc: "segment name not in OSS", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "server": true, "segment": "a" }`}, + hcl: []string{` server = true segment = "a" `}, + err: `Network segments are not supported in this version of Consul`, + privatev4: func() ([]*net.IPAddr, error) { + return []*net.IPAddr{ipAddr("10.0.0.1")}, nil + }, + }, + { + desc: "segments not in OSS", + flags: []string{ + `-data-dir=` + dataDir, + }, + json: []string{`{ "segments":[{ "name":"x", "advertise": "unix:///foo" }] }`}, + hcl: []string{`segments = [{ name = "x" advertise = "unix:///foo" }]`}, + err: `Network segments are not supported in this version of Consul`, + privatev4: func() ([]*net.IPAddr, error) { + return []*net.IPAddr{ipAddr("10.0.0.1")}, nil + }, + }, + } + + testConfig(t, tests, dataDir) +} diff --git a/agent/config_test.go b/agent/config_test.go deleted file mode 100644 index 35a17dc1b..000000000 --- a/agent/config_test.go +++ /dev/null @@ -1,1648 +0,0 @@ -package agent - -import ( - "bytes" - "crypto/tls" - "encoding/base64" - "errors" - "io/ioutil" - "net" - "os" - "path/filepath" - "reflect" - "strings" - "testing" - "time" - - "github.com/hashicorp/consul/agent/structs" - "github.com/hashicorp/consul/testutil" - "github.com/pascaldekloe/goe/verify" -) - -func TestConfigEncryptBytes(t *testing.T) { - t.Parallel() - // Test with some input - src := []byte("abc") - c := &Config{ - EncryptKey: base64.StdEncoding.EncodeToString(src), - } - - result, err := c.EncryptBytes() - if err != nil { - t.Fatalf("err: %s", err) - } - - if !bytes.Equal(src, result) { - t.Fatalf("bad: %#v", result) - } - - // Test with no input - c = &Config{} - result, err = c.EncryptBytes() - if err != nil { - t.Fatalf("err: %s", err) - } - - if len(result) > 0 { - t.Fatalf("bad: %#v", result) - } -} - -func TestDecodeConfig(t *testing.T) { - tests := []struct { - desc string - in string - c *Config - err error - parseTemplateErr error - }{ - // special flows - { - in: `{"bad": "no way jose"}`, - err: errors.New("Config has invalid keys: bad"), - }, - { - in: `{"advertise_addr":"unix:///path/to/file"}`, - parseTemplateErr: errors.New("Failed to parse Advertise address: unix:///path/to/file"), - c: &Config{AdvertiseAddr: "unix:///path/to/file"}, - }, - { - in: `{"advertise_addr_wan":"unix:///path/to/file"}`, - parseTemplateErr: errors.New("Failed to parse Advertise WAN address: unix:///path/to/file"), - c: &Config{AdvertiseAddrWan: "unix:///path/to/file"}, - }, - { - in: `{"addresses":{"http":"notunix://blah"}}`, - parseTemplateErr: errors.New("Failed to parse HTTP address, \"notunix://blah\" is not a valid IP address or socket"), - c: &Config{Addresses: AddressConfig{HTTP: "notunix://blah"}}, - }, - - // happy flows in alphabetical order - { - in: `{"acl_agent_master_token":"a"}`, - c: &Config{ACLAgentMasterToken: "a"}, - }, - { - in: `{"acl_agent_token":"a"}`, - c: &Config{ACLAgentToken: "a"}, - }, - { - in: `{"acl_datacenter":"a"}`, - c: &Config{ACLDatacenter: "a"}, - }, - { - in: `{"acl_default_policy":"a"}`, - c: &Config{ACLDefaultPolicy: "a"}, - }, - { - in: `{"acl_down_policy":"a"}`, - c: &Config{ACLDownPolicy: "a"}, - }, - { - in: `{"acl_enforce_version_8":true}`, - c: &Config{ACLEnforceVersion8: Bool(true)}, - }, - { - in: `{"acl_master_token":"a"}`, - c: &Config{ACLMasterToken: "a"}, - }, - { - in: `{"acl_replication_token":"a"}`, - c: &Config{ - EnableACLReplication: true, - ACLReplicationToken: "a", - }, - }, - { - in: `{"acl_token":"a"}`, - c: &Config{ACLToken: "a"}, - }, - { - in: `{"acl_ttl":"2s"}`, - c: &Config{ACLTTL: 2 * time.Second, ACLTTLRaw: "2s"}, - }, - { - in: `{"addresses":{"dns":"1.2.3.4"}}`, - c: &Config{Addresses: AddressConfig{DNS: "1.2.3.4"}}, - }, - { - in: `{"addresses":{"dns":"{{\"1.2.3.4\"}}"}}`, - c: &Config{Addresses: AddressConfig{DNS: "1.2.3.4"}}, - }, - { - in: `{"addresses":{"http":"1.2.3.4"}}`, - c: &Config{Addresses: AddressConfig{HTTP: "1.2.3.4"}}, - }, - { - in: `{"addresses":{"http":"unix:///var/foo/bar"}}`, - c: &Config{Addresses: AddressConfig{HTTP: "unix:///var/foo/bar"}}, - }, - { - in: `{"addresses":{"http":"{{\"1.2.3.4\"}}"}}`, - c: &Config{Addresses: AddressConfig{HTTP: "1.2.3.4"}}, - }, - { - in: `{"addresses":{"https":"1.2.3.4"}}`, - c: &Config{Addresses: AddressConfig{HTTPS: "1.2.3.4"}}, - }, - { - in: `{"addresses":{"https":"unix:///var/foo/bar"}}`, - c: &Config{Addresses: AddressConfig{HTTPS: "unix:///var/foo/bar"}}, - }, - { - in: `{"addresses":{"https":"{{\"1.2.3.4\"}}"}}`, - c: &Config{Addresses: AddressConfig{HTTPS: "1.2.3.4"}}, - }, - { - in: `{"addresses":{"rpc":"a"}}`, - c: &Config{Addresses: AddressConfig{RPC: "a"}}, - }, - { - in: `{"advertise_addr":"1.2.3.4"}`, - c: &Config{AdvertiseAddr: "1.2.3.4"}, - }, - { - in: `{"advertise_addr":"{{\"1.2.3.4\"}}"}`, - c: &Config{AdvertiseAddr: "1.2.3.4"}, - }, - { - in: `{"advertise_addr_wan":"1.2.3.4"}`, - c: &Config{AdvertiseAddrWan: "1.2.3.4"}, - }, - { - in: `{"advertise_addr_wan":"{{\"1.2.3.4\"}}"}`, - c: &Config{AdvertiseAddrWan: "1.2.3.4"}, - }, - { - in: `{"advertise_addrs":{"rpc":"1.2.3.4:5678"}}`, - c: &Config{ - AdvertiseAddrs: AdvertiseAddrsConfig{ - RPC: &net.TCPAddr{IP: net.ParseIP("1.2.3.4"), Port: 5678}, - RPCRaw: "1.2.3.4:5678", - }, - }, - }, - { - in: `{"advertise_addrs":{"serf_lan":"1.2.3.4:5678"}}`, - c: &Config{ - AdvertiseAddrs: AdvertiseAddrsConfig{ - SerfLan: &net.TCPAddr{IP: net.ParseIP("1.2.3.4"), Port: 5678}, - SerfLanRaw: "1.2.3.4:5678", - }, - }, - }, - { - in: `{"advertise_addrs":{"serf_wan":"1.2.3.4:5678"}}`, - c: &Config{ - AdvertiseAddrs: AdvertiseAddrsConfig{ - SerfWan: &net.TCPAddr{IP: net.ParseIP("1.2.3.4"), Port: 5678}, - SerfWanRaw: "1.2.3.4:5678", - }, - }, - }, - { - in: `{"atlas_acl_token":"a"}`, - c: &Config{DeprecatedAtlasACLToken: "a"}, - }, - { - in: `{"atlas_endpoint":"a"}`, - c: &Config{DeprecatedAtlasEndpoint: "a"}, - }, - { - in: `{"atlas_infrastructure":"a"}`, - c: &Config{DeprecatedAtlasInfrastructure: "a"}, - }, - { - in: `{"atlas_join":true}`, - c: &Config{DeprecatedAtlasJoin: true}, - }, - { - in: `{"atlas_token":"a"}`, - c: &Config{DeprecatedAtlasToken: "a"}, - }, - { - in: `{"autopilot":{"cleanup_dead_servers":true}}`, - c: &Config{Autopilot: Autopilot{CleanupDeadServers: Bool(true)}}, - }, - { - in: `{"autopilot":{"disable_upgrade_migration":true}}`, - c: &Config{Autopilot: Autopilot{DisableUpgradeMigration: Bool(true)}}, - }, - { - in: `{"autopilot":{"upgrade_version_tag":"rev"}}`, - c: &Config{Autopilot: Autopilot{UpgradeVersionTag: "rev"}}, - }, - { - in: `{"autopilot":{"last_contact_threshold":"2s"}}`, - c: &Config{Autopilot: Autopilot{LastContactThreshold: Duration(2 * time.Second), LastContactThresholdRaw: "2s"}}, - }, - { - in: `{"autopilot":{"max_trailing_logs":10}}`, - c: &Config{Autopilot: Autopilot{MaxTrailingLogs: Uint64(10)}}, - }, - { - in: `{"autopilot":{"server_stabilization_time":"2s"}}`, - c: &Config{Autopilot: Autopilot{ServerStabilizationTime: Duration(2 * time.Second), ServerStabilizationTimeRaw: "2s"}}, - }, - { - in: `{"autopilot":{"cleanup_dead_servers":true}}`, - c: &Config{Autopilot: Autopilot{CleanupDeadServers: Bool(true)}}, - }, - { - in: `{"bind_addr":"1.2.3.4"}`, - c: &Config{BindAddr: "1.2.3.4"}, - }, - { - in: `{"bind_addr":"{{\"1.2.3.4\"}}"}`, - c: &Config{BindAddr: "1.2.3.4"}, - }, - { - in: `{"bootstrap":true}`, - c: &Config{Bootstrap: true}, - }, - { - in: `{"bootstrap_expect":3}`, - c: &Config{BootstrapExpect: 3}, - }, - { - in: `{"ca_file":"a"}`, - c: &Config{CAFile: "a"}, - }, - { - in: `{"ca_path":"a"}`, - c: &Config{CAPath: "a"}, - }, - { - in: `{"check_update_interval":"2s"}`, - c: &Config{CheckUpdateInterval: 2 * time.Second, CheckUpdateIntervalRaw: "2s"}, - }, - { - in: `{"cert_file":"a"}`, - c: &Config{CertFile: "a"}, - }, - { - in: `{"client_addr":"1.2.3.4"}`, - c: &Config{ClientAddr: "1.2.3.4"}, - }, - { - in: `{"client_addr":"{{\"1.2.3.4\"}}"}`, - c: &Config{ClientAddr: "1.2.3.4"}, - }, - { - in: `{"data_dir":"a"}`, - c: &Config{DataDir: "a"}, - }, - { - in: `{"datacenter":"a"}`, - c: &Config{Datacenter: "a"}, - }, - { - in: `{"disable_coordinates":true}`, - c: &Config{DisableCoordinates: true}, - }, - { - in: `{"disable_host_node_id":false}`, - c: &Config{DisableHostNodeID: Bool(false)}, - }, - { - in: `{"dns_config":{"allow_stale":true}}`, - c: &Config{DNSConfig: DNSConfig{AllowStale: Bool(true)}}, - }, - { - in: `{"dns_config":{"disable_compression":true}}`, - c: &Config{DNSConfig: DNSConfig{DisableCompression: true}}, - }, - { - in: `{"dns_config":{"enable_truncate":true}}`, - c: &Config{DNSConfig: DNSConfig{EnableTruncate: true}}, - }, - { - in: `{"dns_config":{"max_stale":"2s"}}`, - c: &Config{DNSConfig: DNSConfig{MaxStale: 2 * time.Second, MaxStaleRaw: "2s"}}, - }, - { - in: `{"dns_config":{"node_ttl":"2s"}}`, - c: &Config{DNSConfig: DNSConfig{NodeTTL: 2 * time.Second, NodeTTLRaw: "2s"}}, - }, - { - in: `{"dns_config":{"only_passing":true}}`, - c: &Config{DNSConfig: DNSConfig{OnlyPassing: true}}, - }, - { - in: `{"dns_config":{"recursor_timeout":"2s"}}`, - c: &Config{DNSConfig: DNSConfig{RecursorTimeout: 2 * time.Second, RecursorTimeoutRaw: "2s"}}, - }, - { - in: `{"dns_config":{"service_ttl":{"*":"2s","a":"456s"}}}`, - c: &Config{ - DNSConfig: DNSConfig{ - ServiceTTL: map[string]time.Duration{"*": 2 * time.Second, "a": 456 * time.Second}, - ServiceTTLRaw: map[string]string{"*": "2s", "a": "456s"}, - }, - }, - }, - { - in: `{"dns_config":{"udp_answer_limit":123}}`, - c: &Config{DNSConfig: DNSConfig{UDPAnswerLimit: 123}}, - }, - { - in: `{"disable_anonymous_signature":true}`, - c: &Config{DisableAnonymousSignature: true}, - }, - { - in: `{"disable_remote_exec":false}`, - c: &Config{DisableRemoteExec: Bool(false)}, - }, - { - in: `{"disable_update_check":true}`, - c: &Config{DisableUpdateCheck: true}, - }, - { - in: `{"dogstatsd_addr":"a"}`, - c: &Config{Telemetry: Telemetry{DogStatsdAddr: "a"}}, - }, - { - in: `{"dogstatsd_tags":["a:b","c:d"]}`, - c: &Config{Telemetry: Telemetry{DogStatsdTags: []string{"a:b", "c:d"}}}, - }, - { - in: `{"domain":"a"}`, - c: &Config{Domain: "a"}, - }, - { - in: `{"enable_acl_replication":true}`, - c: &Config{EnableACLReplication: true}, - }, - { - in: `{"enable_debug":true}`, - c: &Config{EnableDebug: true}, - }, - { - in: `{"enable_syslog":true}`, - c: &Config{EnableSyslog: true}, - }, - { - in: `{"disable_keyring_file":true}`, - c: &Config{DisableKeyringFile: true}, - }, - { - in: `{"enable_script_checks":true}`, - c: &Config{EnableScriptChecks: true}, - }, - { - in: `{"encrypt_verify_incoming":true}`, - c: &Config{EncryptVerifyIncoming: Bool(true)}, - }, - { - in: `{"encrypt_verify_outgoing":true}`, - c: &Config{EncryptVerifyOutgoing: Bool(true)}, - }, - { - in: `{"http_config":{"block_endpoints":["a","b","c","d"]}}`, - c: &Config{HTTPConfig: HTTPConfig{BlockEndpoints: []string{"a", "b", "c", "d"}}}, - }, - { - in: `{"http_api_response_headers":{"a":"b","c":"d"}}`, - c: &Config{HTTPConfig: HTTPConfig{ResponseHeaders: map[string]string{"a": "b", "c": "d"}}}, - }, - { - in: `{"http_config":{"response_headers":{"a":"b","c":"d"}}}`, - c: &Config{HTTPConfig: HTTPConfig{ResponseHeaders: map[string]string{"a": "b", "c": "d"}}}, - }, - { - in: `{"key_file":"a"}`, - c: &Config{KeyFile: "a"}, - }, - { - in: `{"leave_on_terminate":true}`, - c: &Config{LeaveOnTerm: Bool(true)}, - }, - { - in: `{"limits": {"rpc_rate": 100, "rpc_max_burst": 50}}}`, - c: &Config{Limits: Limits{RPCRate: 100, RPCMaxBurst: 50}}, - }, - { - in: `{"log_level":"a"}`, - c: &Config{LogLevel: "a"}, - }, - { - in: `{"node_id":"a"}`, - c: &Config{NodeID: "a"}, - }, - { - in: `{"node_meta":{"a":"b","c":"d"}}`, - c: &Config{Meta: map[string]string{"a": "b", "c": "d"}}, - }, - { - in: `{"node_name":"a"}`, - c: &Config{NodeName: "a"}, - }, - { - in: `{"performance": { "raft_multiplier": 3 }}`, - c: &Config{Performance: Performance{RaftMultiplier: 3}}, - }, - { - in: `{"performance": { "raft_multiplier": 11 }}`, - err: errors.New("Performance.RaftMultiplier must be <= 10"), - }, - { - in: `{"pid_file":"a"}`, - c: &Config{PidFile: "a"}, - }, - { - in: `{"ports":{"dns":1234}}`, - c: &Config{Ports: PortConfig{DNS: 1234}}, - }, - { - in: `{"ports":{"http":1234}}`, - c: &Config{Ports: PortConfig{HTTP: 1234}}, - }, - { - in: `{"ports":{"https":1234}}`, - c: &Config{Ports: PortConfig{HTTPS: 1234}}, - }, - { - in: `{"ports":{"serf_lan":1234}}`, - c: &Config{Ports: PortConfig{SerfLan: 1234}}, - }, - { - in: `{"ports":{"serf_wan":1234}}`, - c: &Config{Ports: PortConfig{SerfWan: 1234}}, - }, - { - in: `{"ports":{"server":1234}}`, - c: &Config{Ports: PortConfig{Server: 1234}}, - }, - { - in: `{"ports":{"rpc":1234}}`, - c: &Config{Ports: PortConfig{RPC: 1234}}, - }, - { - in: `{"raft_protocol":3}`, - c: &Config{RaftProtocol: 3}, - }, - { - in: `{"reconnect_timeout":"4h"}`, - err: errors.New("ReconnectTimeoutLan must be >= 8h0m0s"), - }, - { - in: `{"reconnect_timeout":"8h"}`, - c: &Config{ReconnectTimeoutLan: 8 * time.Hour, ReconnectTimeoutLanRaw: "8h"}, - }, - { - in: `{"reconnect_timeout_wan":"4h"}`, - err: errors.New("ReconnectTimeoutWan must be >= 8h0m0s"), - }, - { - in: `{"reconnect_timeout_wan":"8h"}`, - c: &Config{ReconnectTimeoutWan: 8 * time.Hour, ReconnectTimeoutWanRaw: "8h"}, - }, - { - in: `{"recursor":"a"}`, - c: &Config{DNSRecursor: "a", DNSRecursors: []string{"a"}}, - }, - { - in: `{"recursors":["a","b"]}`, - c: &Config{DNSRecursors: []string{"a", "b"}}, - }, - { - in: `{"rejoin_after_leave":true}`, - c: &Config{RejoinAfterLeave: true}, - }, - { - in: `{"retry_interval":"2s"}`, - c: &Config{RetryInterval: 2 * time.Second, RetryIntervalRaw: "2s"}, - }, - { - in: `{"retry_interval_wan":"2s"}`, - c: &Config{RetryIntervalWan: 2 * time.Second, RetryIntervalWanRaw: "2s"}, - }, - { - in: `{"retry_join":["a","b"]}`, - c: &Config{RetryJoin: []string{"a", "b"}}, - }, - // todo(fs): temporarily disabling tests after moving the code - // todo(fs): to patch the deprecated retry-join flags to command/agent.go - // todo(fs): where it cannot be tested. - // { - // in: `{"retry_join_azure":{"client_id":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=azure client_id=a"}}, - // }, - // { - // in: `{"retry_join_azure":{"tag_name":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=azure tag_name=a"}}, - // }, - // { - // in: `{"retry_join_azure":{"tag_value":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=azure tag_value=a"}}, - // }, - // { - // in: `{"retry_join_azure":{"secret_access_key":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=azure secret_access_key=a"}}, - // }, - // { - // in: `{"retry_join_azure":{"subscription_id":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=azure subscription_id=a"}}, - // }, - // { - // in: `{"retry_join_azure":{"tenant_id":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=azure tenant_id=a"}}, - // }, - // { - // in: `{"retry_join_ec2":{"access_key_id":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=aws access_key_id=a"}}, - // }, - // { - // in: `{"retry_join_ec2":{"region":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=aws region=a"}}, - // }, - // { - // in: `{"retry_join_ec2":{"tag_key":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=aws tag_key=a"}}, - // }, - // { - // in: `{"retry_join_ec2":{"tag_value":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=aws tag_value=a"}}, - // }, - // { - // in: `{"retry_join_ec2":{"secret_access_key":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=aws secret_access_key=a"}}, - // }, - // { - // in: `{"retry_join_gce":{"credentials_file":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=gce credentials_file=a"}}, - // }, - // { - // in: `{"retry_join_gce":{"project_name":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=gce project_name=a"}}, - // }, - // { - // in: `{"retry_join_gce":{"tag_value":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=gce tag_value=a"}}, - // }, - // { - // in: `{"retry_join_gce":{"zone_pattern":"a"}}`, - // c: &Config{RetryJoin: []string{"provider=gce zone_pattern=a"}}, - // }, - { - in: `{"retry_join_wan":["a","b"]}`, - c: &Config{RetryJoinWan: []string{"a", "b"}}, - }, - { - in: `{"retry_max":123}`, - c: &Config{RetryMaxAttempts: 123}, - }, - { - in: `{"retry_max_wan":123}`, - c: &Config{RetryMaxAttemptsWan: 123}, - }, - { - in: `{"serf_lan_bind":"1.2.3.4"}`, - c: &Config{SerfLanBindAddr: "1.2.3.4"}, - }, - { - in: `{"serf_lan_bind":"unix:///var/foo/bar"}`, - c: &Config{SerfLanBindAddr: "unix:///var/foo/bar"}, - parseTemplateErr: errors.New("Failed to parse Serf LAN address: unix:///var/foo/bar"), - }, - { - in: `{"serf_lan_bind":"{{\"1.2.3.4\"}}"}`, - c: &Config{SerfLanBindAddr: "1.2.3.4"}, - }, - { - in: `{"serf_wan_bind":"1.2.3.4"}`, - c: &Config{SerfWanBindAddr: "1.2.3.4"}, - }, - { - in: `{"serf_wan_bind":"unix:///var/foo/bar"}`, - c: &Config{SerfWanBindAddr: "unix:///var/foo/bar"}, - parseTemplateErr: errors.New("Failed to parse Serf WAN address: unix:///var/foo/bar"), - }, - { - in: `{"serf_wan_bind":"{{\"1.2.3.4\"}}"}`, - c: &Config{SerfWanBindAddr: "1.2.3.4"}, - }, - { - in: `{"server":true}`, - c: &Config{Server: true}, - }, - { - in: `{"server_name":"a"}`, - c: &Config{ServerName: "a"}, - }, - { - in: `{"session_ttl_min":"2s"}`, - c: &Config{SessionTTLMin: 2 * time.Second, SessionTTLMinRaw: "2s"}, - }, - { - in: `{"skip_leave_on_interrupt":true}`, - c: &Config{SkipLeaveOnInt: Bool(true)}, - }, - { - in: `{"start_join":["a","b"]}`, - c: &Config{StartJoin: []string{"a", "b"}}, - }, - { - in: `{"start_join_wan":["a","b"]}`, - c: &Config{StartJoinWan: []string{"a", "b"}}, - }, - { - in: `{"statsd_addr":"a"}`, - c: &Config{Telemetry: Telemetry{StatsdAddr: "a"}}, - }, - { - in: `{"statsite_addr":"a"}`, - c: &Config{Telemetry: Telemetry{StatsiteAddr: "a"}}, - }, - { - in: `{"statsite_prefix":"a"}`, - c: &Config{Telemetry: Telemetry{StatsitePrefix: "a"}}, - }, - { - in: `{"syslog_facility":"a"}`, - c: &Config{SyslogFacility: "a"}, - }, - { - in: `{"telemetry":{"circonus_api_app":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusAPIApp: "a"}}, - }, - { - in: `{"telemetry":{"circonus_api_token":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusAPIToken: "a"}}, - }, - { - in: `{"telemetry":{"circonus_api_url":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusAPIURL: "a"}}, - }, - { - in: `{"telemetry":{"circonus_broker_id":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusBrokerID: "a"}}, - }, - { - in: `{"telemetry":{"circonus_broker_select_tag":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusBrokerSelectTag: "a"}}, - }, - { - in: `{"telemetry":{"circonus_check_display_name":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckDisplayName: "a"}}, - }, - { - in: `{"telemetry":{"circonus_check_force_metric_activation":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckForceMetricActivation: "a"}}, - }, - { - in: `{"telemetry":{"circonus_check_id":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckID: "a"}}, - }, - { - in: `{"telemetry":{"circonus_check_instance_id":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckInstanceID: "a"}}, - }, - { - in: `{"telemetry":{"circonus_check_search_tag":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckSearchTag: "a"}}, - }, - { - in: `{"telemetry":{"circonus_check_tags":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckTags: "a"}}, - }, - { - in: `{"telemetry":{"circonus_submission_interval":"2s"}}`, - c: &Config{Telemetry: Telemetry{CirconusSubmissionInterval: "2s"}}, - }, - { - in: `{"telemetry":{"circonus_submission_url":"a"}}`, - c: &Config{Telemetry: Telemetry{CirconusCheckSubmissionURL: "a"}}, - }, - { - in: `{"telemetry":{"disable_hostname":true}}`, - c: &Config{Telemetry: Telemetry{DisableHostname: true}}, - }, - { - in: `{"telemetry":{"dogstatsd_addr":"a"}}`, - c: &Config{Telemetry: Telemetry{DogStatsdAddr: "a"}}, - }, - { - in: `{"telemetry":{"dogstatsd_tags":["a","b"]}}`, - c: &Config{Telemetry: Telemetry{DogStatsdTags: []string{"a", "b"}}}, - }, - { - in: `{"telemetry":{"filter_default":true}}`, - c: &Config{Telemetry: Telemetry{FilterDefault: Bool(true)}}, - }, - { - in: `{"telemetry":{"prefix_filter":["+consul.metric","-consul.othermetric"]}}`, - c: &Config{Telemetry: Telemetry{ - PrefixFilter: []string{"+consul.metric", "-consul.othermetric"}, - AllowedPrefixes: []string{"consul.metric"}, - BlockedPrefixes: []string{"consul.othermetric"}, - }}, - }, - { - in: `{"telemetry":{"statsd_address":"a"}}`, - c: &Config{Telemetry: Telemetry{StatsdAddr: "a"}}, - }, - { - in: `{"telemetry":{"statsite_address":"a"}}`, - c: &Config{Telemetry: Telemetry{StatsiteAddr: "a"}}, - }, - { - in: `{"telemetry":{"statsite_prefix":"a"}}`, - c: &Config{Telemetry: Telemetry{StatsitePrefix: "a"}}, - }, - { - in: `{"tls_cipher_suites":"TLS_RSA_WITH_AES_256_CBC_SHA"}`, - c: &Config{ - TLSCipherSuites: []uint16{tls.TLS_RSA_WITH_AES_256_CBC_SHA}, - TLSCipherSuitesRaw: "TLS_RSA_WITH_AES_256_CBC_SHA", - }, - }, - { - in: `{"tls_min_version":"a"}`, - c: &Config{TLSMinVersion: "a"}, - }, - { - in: `{"tls_prefer_server_cipher_suites":true}`, - c: &Config{TLSPreferServerCipherSuites: true}, - }, - { - in: `{"translate_wan_addrs":true}`, - c: &Config{TranslateWanAddrs: true}, - }, - { - in: `{"ui":true}`, - c: &Config{EnableUI: true}, - }, - { - in: `{"ui_dir":"a"}`, - c: &Config{UIDir: "a"}, - }, - { - in: `{"unix_sockets":{"user":"a"}}`, - c: &Config{UnixSockets: UnixSocketConfig{UnixSocketPermissions{Usr: "a"}}}, - }, - { - in: `{"unix_sockets":{"group":"a"}}`, - c: &Config{UnixSockets: UnixSocketConfig{UnixSocketPermissions{Grp: "a"}}}, - }, - { - in: `{"unix_sockets":{"mode":"a"}}`, - c: &Config{UnixSockets: UnixSocketConfig{UnixSocketPermissions{Perms: "a"}}}, - }, - { - in: `{"verify_incoming":true}`, - c: &Config{VerifyIncoming: true}, - }, - { - in: `{"verify_incoming_https":true}`, - c: &Config{VerifyIncomingHTTPS: true}, - }, - { - in: `{"verify_incoming_rpc":true}`, - c: &Config{VerifyIncomingRPC: true}, - }, - { - in: `{"verify_outgoing":true}`, - c: &Config{VerifyOutgoing: true}, - }, - { - in: `{"verify_server_hostname":true}`, - c: &Config{VerifyServerHostname: true}, - }, - { - in: `{"watches":[{"type":"a","prefix":"b","handler":"c"}]}`, - c: &Config{ - Watches: []map[string]interface{}{ - map[string]interface{}{ - "type": "a", - "prefix": "b", - "handler": "c", - }, - }, - }, - }, - - // complex flows - { - desc: "single service with check", - in: `{ - "service": { - "ID": "a", - "Name": "b", - "Tags": ["c", "d"], - "Address": "e", - "Token": "f", - "Port": 123, - "EnableTagOverride": true, - "Check": { - "CheckID": "g", - "Name": "h", - "Status": "i", - "Notes": "j", - "Script": "k", - "HTTP": "l", - "Header": {"a":["b"], "c":["d", "e"]}, - "Method": "x", - "TCP": "m", - "DockerContainerID": "n", - "Shell": "o", - "TLSSkipVerify": true, - "Interval": "2s", - "Timeout": "3s", - "TTL": "4s", - "DeregisterCriticalServiceAfter": "5s" - } - } - }`, - c: &Config{ - Services: []*structs.ServiceDefinition{ - &structs.ServiceDefinition{ - ID: "a", - Name: "b", - Tags: []string{"c", "d"}, - Address: "e", - Port: 123, - Token: "f", - EnableTagOverride: true, - Check: structs.CheckType{ - CheckID: "g", - Name: "h", - Status: "i", - Notes: "j", - Script: "k", - HTTP: "l", - Header: map[string][]string{"a": []string{"b"}, "c": []string{"d", "e"}}, - Method: "x", - TCP: "m", - DockerContainerID: "n", - Shell: "o", - TLSSkipVerify: true, - Interval: 2 * time.Second, - Timeout: 3 * time.Second, - TTL: 4 * time.Second, - DeregisterCriticalServiceAfter: 5 * time.Second, - }, - }, - }, - }, - }, - { - desc: "single service with multiple checks", - in: `{ - "service": { - "ID": "a", - "Name": "b", - "Tags": ["c", "d"], - "Address": "e", - "Token": "f", - "Port": 123, - "EnableTagOverride": true, - "Checks": [ - { - "CheckID": "g", - "Name": "h", - "Status": "i", - "Notes": "j", - "Script": "k", - "HTTP": "l", - "Header": {"a":["b"], "c":["d", "e"]}, - "Method": "x", - "TCP": "m", - "DockerContainerID": "n", - "Shell": "o", - "TLSSkipVerify": true, - "Interval": "2s", - "Timeout": "3s", - "TTL": "4s", - "DeregisterCriticalServiceAfter": "5s" - }, - { - "CheckID": "gg", - "Name": "hh", - "Status": "ii", - "Notes": "jj", - "Script": "kk", - "HTTP": "ll", - "Header": {"aa":["bb"], "cc":["dd", "ee"]}, - "Method": "xx", - "TCP": "mm", - "DockerContainerID": "nn", - "Shell": "oo", - "TLSSkipVerify": false, - "Interval": "22s", - "Timeout": "33s", - "TTL": "44s", - "DeregisterCriticalServiceAfter": "55s" - } - ] - } - }`, - c: &Config{ - Services: []*structs.ServiceDefinition{ - &structs.ServiceDefinition{ - ID: "a", - Name: "b", - Tags: []string{"c", "d"}, - Address: "e", - Port: 123, - Token: "f", - EnableTagOverride: true, - Checks: []*structs.CheckType{ - { - CheckID: "g", - Name: "h", - Status: "i", - Notes: "j", - Script: "k", - HTTP: "l", - Header: map[string][]string{"a": []string{"b"}, "c": []string{"d", "e"}}, - Method: "x", - TCP: "m", - DockerContainerID: "n", - Shell: "o", - TLSSkipVerify: true, - Interval: 2 * time.Second, - Timeout: 3 * time.Second, - TTL: 4 * time.Second, - DeregisterCriticalServiceAfter: 5 * time.Second, - }, - { - CheckID: "gg", - Name: "hh", - Status: "ii", - Notes: "jj", - Script: "kk", - HTTP: "ll", - Header: map[string][]string{"aa": []string{"bb"}, "cc": []string{"dd", "ee"}}, - Method: "xx", - TCP: "mm", - DockerContainerID: "nn", - Shell: "oo", - TLSSkipVerify: false, - Interval: 22 * time.Second, - Timeout: 33 * time.Second, - TTL: 44 * time.Second, - DeregisterCriticalServiceAfter: 55 * time.Second, - }, - }, - }, - }, - }, - }, - { - desc: "multiple services with check", - in: `{ - "services": [ - { - "ID": "a", - "Name": "b", - "Tags": ["c", "d"], - "Address": "e", - "Token": "f", - "Port": 123, - "EnableTagOverride": true, - "Check": { - "CheckID": "g", - "Name": "h", - "Status": "i", - "Notes": "j", - "Script": "k", - "HTTP": "l", - "Header": {"a":["b"], "c":["d", "e"]}, - "Method": "x", - "TCP": "m", - "DockerContainerID": "n", - "Shell": "o", - "TLSSkipVerify": true, - "Interval": "2s", - "Timeout": "3s", - "TTL": "4s", - "DeregisterCriticalServiceAfter": "5s" - } - }, - { - "ID": "aa", - "Name": "bb", - "Tags": ["cc", "dd"], - "Address": "ee", - "Token": "ff", - "Port": 246, - "EnableTagOverride": false, - "Check": { - "CheckID": "gg", - "Name": "hh", - "Status": "ii", - "Notes": "jj", - "Script": "kk", - "HTTP": "ll", - "Header": {"aa":["bb"], "cc":["dd", "ee"]}, - "Method": "xx", - "TCP": "mm", - "DockerContainerID": "nn", - "Shell": "oo", - "TLSSkipVerify": false, - "Interval": "22s", - "Timeout": "33s", - "TTL": "44s", - "DeregisterCriticalServiceAfter": "55s" - } - } - ] - }`, - c: &Config{ - Services: []*structs.ServiceDefinition{ - &structs.ServiceDefinition{ - ID: "a", - Name: "b", - Tags: []string{"c", "d"}, - Address: "e", - Port: 123, - Token: "f", - EnableTagOverride: true, - Check: structs.CheckType{ - CheckID: "g", - Name: "h", - Status: "i", - Notes: "j", - Script: "k", - HTTP: "l", - Header: map[string][]string{"a": []string{"b"}, "c": []string{"d", "e"}}, - Method: "x", - TCP: "m", - DockerContainerID: "n", - Shell: "o", - TLSSkipVerify: true, - Interval: 2 * time.Second, - Timeout: 3 * time.Second, - TTL: 4 * time.Second, - DeregisterCriticalServiceAfter: 5 * time.Second, - }, - }, - &structs.ServiceDefinition{ - ID: "aa", - Name: "bb", - Tags: []string{"cc", "dd"}, - Address: "ee", - Port: 246, - Token: "ff", - EnableTagOverride: false, - Check: structs.CheckType{ - CheckID: "gg", - Name: "hh", - Status: "ii", - Notes: "jj", - Script: "kk", - HTTP: "ll", - Header: map[string][]string{"aa": []string{"bb"}, "cc": []string{"dd", "ee"}}, - Method: "xx", - TCP: "mm", - DockerContainerID: "nn", - Shell: "oo", - TLSSkipVerify: false, - Interval: 22 * time.Second, - Timeout: 33 * time.Second, - TTL: 44 * time.Second, - DeregisterCriticalServiceAfter: 55 * time.Second, - }, - }, - }, - }, - }, - - { - desc: "single check", - in: `{ - "check": { - "id": "a", - "name": "b", - "notes": "c", - "service_id": "x", - "token": "y", - "status": "z", - "script": "d", - "shell": "e", - "http": "f", - "Header": {"a":["b"], "c":["d", "e"]}, - "Method": "x", - "tcp": "g", - "docker_container_id": "h", - "tls_skip_verify": true, - "interval": "2s", - "timeout": "3s", - "ttl": "4s", - "deregister_critical_service_after": "5s" - } - }`, - c: &Config{ - Checks: []*structs.CheckDefinition{ - &structs.CheckDefinition{ - ID: "a", - Name: "b", - Notes: "c", - ServiceID: "x", - Token: "y", - Status: "z", - Script: "d", - Shell: "e", - HTTP: "f", - Header: map[string][]string{"a": []string{"b"}, "c": []string{"d", "e"}}, - Method: "x", - TCP: "g", - DockerContainerID: "h", - TLSSkipVerify: true, - Interval: 2 * time.Second, - Timeout: 3 * time.Second, - TTL: 4 * time.Second, - DeregisterCriticalServiceAfter: 5 * time.Second, - }, - }, - }, - }, - { - desc: "multiple checks", - in: `{ - "checks": [ - { - "id": "a", - "name": "b", - "notes": "c", - "service_id": "d", - "token": "e", - "status": "f", - "script": "g", - "shell": "h", - "http": "i", - "Header": {"a":["b"], "c":["d", "e"]}, - "Method": "x", - "tcp": "j", - "docker_container_id": "k", - "tls_skip_verify": true, - "interval": "2s", - "timeout": "3s", - "ttl": "4s", - "deregister_critical_service_after": "5s" - }, - { - "id": "aa", - "name": "bb", - "notes": "cc", - "service_id": "dd", - "token": "ee", - "status": "ff", - "script": "gg", - "shell": "hh", - "http": "ii", - "Header": {"aa":["bb"], "cc":["dd", "ee"]}, - "Method": "xx", - "tcp": "jj", - "docker_container_id": "kk", - "tls_skip_verify": false, - "interval": "22s", - "timeout": "33s", - "ttl": "44s", - "deregister_critical_service_after": "55s" - } - ] - }`, - c: &Config{ - Checks: []*structs.CheckDefinition{ - &structs.CheckDefinition{ - ID: "a", - Name: "b", - Notes: "c", - ServiceID: "d", - Token: "e", - Status: "f", - Script: "g", - Shell: "h", - HTTP: "i", - Header: map[string][]string{"a": []string{"b"}, "c": []string{"d", "e"}}, - Method: "x", - TCP: "j", - DockerContainerID: "k", - TLSSkipVerify: true, - Interval: 2 * time.Second, - Timeout: 3 * time.Second, - TTL: 4 * time.Second, - DeregisterCriticalServiceAfter: 5 * time.Second, - }, - &structs.CheckDefinition{ - ID: "aa", - Name: "bb", - Notes: "cc", - ServiceID: "dd", - Token: "ee", - Status: "ff", - Script: "gg", - Shell: "hh", - HTTP: "ii", - Header: map[string][]string{"aa": []string{"bb"}, "cc": []string{"dd", "ee"}}, - Method: "xx", - TCP: "jj", - DockerContainerID: "kk", - TLSSkipVerify: false, - Interval: 22 * time.Second, - Timeout: 33 * time.Second, - TTL: 44 * time.Second, - DeregisterCriticalServiceAfter: 55 * time.Second, - }, - }, - }, - }, - } - - for _, tt := range tests { - desc := tt.desc - if desc == "" { - desc = tt.in - } - t.Run(desc, func(t *testing.T) { - c, err := DecodeConfig(strings.NewReader(tt.in)) - if got, want := err, tt.err; !reflect.DeepEqual(got, want) { - t.Fatalf("got error %v want %v", got, want) - } - err = c.ResolveTmplAddrs() - if got, want := err, tt.parseTemplateErr; !reflect.DeepEqual(got, want) { - t.Fatalf("got error %v on ResolveTmplAddrs, expected %v", err, want) - } - got, want := c, tt.c - verify.Values(t, "", got, want) - }) - } -} - -func TestDecodeConfig_VerifyUniqueListeners(t *testing.T) { - t.Parallel() - tests := []struct { - desc string - in string - err error - }{ - { - "http_dns1", - `{"addresses": {"http": "0.0.0.0", "dns": "127.0.0.1"}, "ports": {"dns": 8000}}`, - nil, - }, - { - "http_dns IP identical", - `{"addresses": {"http": "0.0.0.0", "dns": "0.0.0.0"}, "ports": {"http": 8000, "dns": 8000}}`, - errors.New("HTTP address already configured for DNS"), - }, - } - - for _, tt := range tests { - t.Run(tt.desc, func(t *testing.T) { - c, err := DecodeConfig(strings.NewReader(tt.in)) - if err != nil { - t.Fatalf("got error %v want nil", err) - } - - err = c.VerifyUniqueListeners() - if got, want := err, tt.err; !reflect.DeepEqual(got, want) { - t.Fatalf("got error %v want %v", got, want) - } - }) - } -} - -func TestDefaultConfig(t *testing.T) { - t.Parallel() - - // ACL flag for Consul version 0.8 features (broken out since we will - // eventually remove this). - config := DefaultConfig() - if *config.ACLEnforceVersion8 != true { - t.Fatalf("bad: %#v", config) - } - - // Remote exec is disabled by default. - if *config.DisableRemoteExec != true { - t.Fatalf("bad: %#v", config) - } -} - -func TestMergeConfig(t *testing.T) { - t.Parallel() - a := &Config{ - Bootstrap: false, - BootstrapExpect: 0, - Datacenter: "dc1", - DataDir: "/tmp/foo", - Domain: "basic", - LogLevel: "debug", - NodeID: "bar", - NodeName: "foo", - ClientAddr: "127.0.0.1", - BindAddr: "127.0.0.1", - AdvertiseAddr: "127.0.0.1", - Server: false, - LeaveOnTerm: new(bool), - SkipLeaveOnInt: new(bool), - EnableDebug: false, - CheckUpdateIntervalRaw: "8m", - RetryIntervalRaw: "10s", - RetryIntervalWanRaw: "10s", - DeprecatedRetryJoinEC2: RetryJoinEC2{ - Region: "us-east-1", - TagKey: "Key1", - TagValue: "Value1", - AccessKeyID: "nope", - SecretAccessKey: "nope", - }, - Telemetry: Telemetry{ - DisableHostname: false, - StatsdAddr: "nope", - StatsiteAddr: "nope", - StatsitePrefix: "nope", - DogStatsdAddr: "nope", - DogStatsdTags: []string{"nope"}, - }, - Meta: map[string]string{ - "key": "value1", - }, - } - - b := &Config{ - Limits: Limits{ - RPCRate: 100, - RPCMaxBurst: 50, - }, - Performance: Performance{ - RaftMultiplier: 99, - }, - Bootstrap: true, - BootstrapExpect: 3, - Datacenter: "dc2", - DataDir: "/tmp/bar", - DNSRecursors: []string{"127.0.0.2:1001"}, - DNSConfig: DNSConfig{ - AllowStale: Bool(false), - EnableTruncate: true, - DisableCompression: true, - MaxStale: 30 * time.Second, - NodeTTL: 10 * time.Second, - ServiceTTL: map[string]time.Duration{ - "api": 10 * time.Second, - }, - UDPAnswerLimit: 4, - RecursorTimeout: 30 * time.Second, - }, - Domain: "other", - LogLevel: "info", - NodeID: "bar", - DisableHostNodeID: Bool(false), - NodeName: "baz", - ClientAddr: "127.0.0.2", - BindAddr: "127.0.0.2", - AdvertiseAddr: "127.0.0.2", - AdvertiseAddrWan: "127.0.0.2", - Ports: PortConfig{ - DNS: 1, - HTTP: 2, - SerfLan: 4, - SerfWan: 5, - Server: 6, - HTTPS: 7, - }, - Addresses: AddressConfig{ - DNS: "127.0.0.1", - HTTP: "127.0.0.2", - HTTPS: "127.0.0.4", - }, - Segment: "alpha", - Segments: []NetworkSegment{ - { - Name: "alpha", - Bind: "127.0.0.1", - Port: 1234, - Advertise: "127.0.0.2", - }, - }, - Server: true, - LeaveOnTerm: Bool(true), - SkipLeaveOnInt: Bool(true), - RaftProtocol: 3, - Autopilot: Autopilot{ - CleanupDeadServers: Bool(true), - LastContactThreshold: Duration(time.Duration(10)), - MaxTrailingLogs: Uint64(10), - ServerStabilizationTime: Duration(time.Duration(100)), - }, - EnableDebug: true, - VerifyIncoming: true, - VerifyOutgoing: true, - CAFile: "test/ca.pem", - CertFile: "test/cert.pem", - KeyFile: "test/key.pem", - TLSMinVersion: "tls12", - Checks: []*structs.CheckDefinition{nil}, - Services: []*structs.ServiceDefinition{nil}, - StartJoin: []string{"1.1.1.1"}, - StartJoinWan: []string{"1.1.1.1"}, - EnableUI: true, - UIDir: "/opt/consul-ui", - EnableSyslog: true, - RejoinAfterLeave: true, - RetryJoin: []string{"1.1.1.1"}, - RetryIntervalRaw: "10s", - RetryInterval: 10 * time.Second, - RetryJoinWan: []string{"1.1.1.1"}, - RetryIntervalWanRaw: "10s", - RetryIntervalWan: 10 * time.Second, - ReconnectTimeoutLanRaw: "24h", - ReconnectTimeoutLan: 24 * time.Hour, - ReconnectTimeoutWanRaw: "36h", - ReconnectTimeoutWan: 36 * time.Hour, - EnableScriptChecks: true, - CheckUpdateInterval: 8 * time.Minute, - CheckUpdateIntervalRaw: "8m", - ACLToken: "1111", - ACLAgentMasterToken: "2222", - ACLAgentToken: "3333", - ACLMasterToken: "4444", - ACLDatacenter: "dc2", - ACLTTL: 15 * time.Second, - ACLTTLRaw: "15s", - ACLDownPolicy: "deny", - ACLDefaultPolicy: "deny", - ACLReplicationToken: "8765309", - ACLEnforceVersion8: Bool(true), - Watches: []map[string]interface{}{ - map[string]interface{}{ - "type": "keyprefix", - "prefix": "foo/", - "handler": "foobar", - }, - }, - DisableRemoteExec: Bool(true), - Telemetry: Telemetry{ - StatsiteAddr: "127.0.0.1:7250", - StatsitePrefix: "stats_prefix", - StatsdAddr: "127.0.0.1:7251", - DisableHostname: true, - DogStatsdAddr: "127.0.0.1:7254", - DogStatsdTags: []string{"tag_1:val_1", "tag_2:val_2"}, - }, - Meta: map[string]string{ - "key": "value2", - }, - DisableUpdateCheck: true, - DisableAnonymousSignature: true, - HTTPConfig: HTTPConfig{ - BlockEndpoints: []string{ - "/v1/agent/self", - "/v1/acl", - }, - ResponseHeaders: map[string]string{ - "Access-Control-Allow-Origin": "*", - }, - }, - UnixSockets: UnixSocketConfig{ - UnixSocketPermissions{ - Usr: "500", - Grp: "500", - Perms: "0700", - }, - }, - DeprecatedRetryJoinEC2: RetryJoinEC2{ - Region: "us-east-2", - TagKey: "Key2", - TagValue: "Value2", - AccessKeyID: "foo", - SecretAccessKey: "bar", - }, - SessionTTLMinRaw: "1000s", - SessionTTLMin: 1000 * time.Second, - AdvertiseAddrs: AdvertiseAddrsConfig{ - SerfLan: &net.TCPAddr{}, - SerfLanRaw: "127.0.0.5:1231", - SerfWan: &net.TCPAddr{}, - SerfWanRaw: "127.0.0.5:1232", - RPC: &net.TCPAddr{}, - RPCRaw: "127.0.0.5:1233", - }, - } - - c := MergeConfig(a, b) - - if !reflect.DeepEqual(c, b) { - t.Fatalf("should be equal %#v %#v", c, b) - } -} - -func TestReadConfigPaths_badPath(t *testing.T) { - t.Parallel() - _, err := ReadConfigPaths([]string{"/i/shouldnt/exist/ever/rainbows"}) - if err == nil { - t.Fatal("should have err") - } -} - -func TestReadConfigPaths_file(t *testing.T) { - t.Parallel() - tf := testutil.TempFile(t, "consul") - tf.Write([]byte(`{"node_name":"bar"}`)) - tf.Close() - defer os.Remove(tf.Name()) - - config, err := ReadConfigPaths([]string{tf.Name()}) - if err != nil { - t.Fatalf("err: %s", err) - } - - if config.NodeName != "bar" { - t.Fatalf("bad: %#v", config) - } -} - -func TestReadConfigPaths_dir(t *testing.T) { - t.Parallel() - td := testutil.TempDir(t, "consul") - defer os.RemoveAll(td) - - err := ioutil.WriteFile(filepath.Join(td, "a.json"), - []byte(`{"node_name": "bar"}`), 0644) - if err != nil { - t.Fatalf("err: %s", err) - } - - err = ioutil.WriteFile(filepath.Join(td, "b.json"), - []byte(`{"node_name": "baz"}`), 0644) - if err != nil { - t.Fatalf("err: %s", err) - } - - // A non-json file, shouldn't be read - err = ioutil.WriteFile(filepath.Join(td, "c"), - []byte(`{"node_name": "bad"}`), 0644) - if err != nil { - t.Fatalf("err: %s", err) - } - - // An empty file shouldn't be read - err = ioutil.WriteFile(filepath.Join(td, "d.json"), - []byte{}, 0664) - if err != nil { - t.Fatalf("err: %s", err) - } - - config, err := ReadConfigPaths([]string{td}) - if err != nil { - t.Fatalf("err: %s", err) - } - - if config.NodeName != "baz" { - t.Fatalf("bad: %#v", config) - } -} - -func TestUnixSockets(t *testing.T) { - t.Parallel() - if p := socketPath("unix:///path/to/socket"); p != "/path/to/socket" { - t.Fatalf("bad: %q", p) - } - if p := socketPath("notunix://blah"); p != "" { - t.Fatalf("bad: %q", p) - } -} - -func TestCheckDefinitionToCheckType(t *testing.T) { - t.Parallel() - got := &structs.CheckDefinition{ - ID: "id", - Name: "name", - Status: "green", - Notes: "notes", - - ServiceID: "svcid", - Token: "tok", - Script: "/bin/foo", - HTTP: "someurl", - TCP: "host:port", - Interval: 1 * time.Second, - DockerContainerID: "abc123", - Shell: "/bin/ksh", - TLSSkipVerify: true, - Timeout: 2 * time.Second, - TTL: 3 * time.Second, - DeregisterCriticalServiceAfter: 4 * time.Second, - } - want := &structs.CheckType{ - CheckID: "id", - Name: "name", - Status: "green", - Notes: "notes", - - Script: "/bin/foo", - HTTP: "someurl", - TCP: "host:port", - Interval: 1 * time.Second, - DockerContainerID: "abc123", - Shell: "/bin/ksh", - TLSSkipVerify: true, - Timeout: 2 * time.Second, - TTL: 3 * time.Second, - DeregisterCriticalServiceAfter: 4 * time.Second, - } - verify.Values(t, "", got.CheckType(), want) -} diff --git a/agent/consul/client_serf.go b/agent/consul/client_serf.go index 0b64e4114..c47811bb5 100644 --- a/agent/consul/client_serf.go +++ b/agent/consul/client_serf.go @@ -23,8 +23,11 @@ func (c *Client) setupSerf(conf *serf.Config, ch chan serf.Event, path string) ( conf.Tags["vsn_min"] = fmt.Sprintf("%d", ProtocolVersionMin) conf.Tags["vsn_max"] = fmt.Sprintf("%d", ProtocolVersionMax) conf.Tags["build"] = c.config.Build - conf.MemberlistConfig.LogOutput = c.config.LogOutput - conf.LogOutput = c.config.LogOutput + if c.logger == nil { + conf.MemberlistConfig.LogOutput = c.config.LogOutput + conf.LogOutput = c.config.LogOutput + } + conf.MemberlistConfig.Logger = c.logger conf.Logger = c.logger conf.EventCh = ch conf.ProtocolVersion = protocolVersionMap[c.config.ProtocolVersion] diff --git a/agent/consul/config.go b/agent/consul/config.go index f01847310..a2cbbbb7c 100644 --- a/agent/consul/config.go +++ b/agent/consul/config.go @@ -438,7 +438,6 @@ func DefaultConfig() *Config { // Use a transitional version of the raft protocol to interoperate with // versions 1 and 3 conf.RaftConfig.ProtocolVersion = 2 - conf.ScaleRaft(DefaultRaftMultiplier) // Disable shutdown on removal conf.RaftConfig.ShutdownOnRemove = false @@ -449,19 +448,6 @@ func DefaultConfig() *Config { return conf } -// ScaleRaft sets the config to have Raft timing parameters scaled by the given -// performance multiplier. This is done in an idempotent way so it's not tricky -// to call this when composing configurations and potentially calling this -// multiple times on the same structure. -func (c *Config) ScaleRaft(raftMultRaw uint) { - raftMult := time.Duration(raftMultRaw) - - def := raft.DefaultConfig() - c.RaftConfig.HeartbeatTimeout = raftMult * def.HeartbeatTimeout - c.RaftConfig.ElectionTimeout = raftMult * def.ElectionTimeout - c.RaftConfig.LeaderLeaseTimeout = raftMult * def.LeaderLeaseTimeout -} - // tlsConfig maps this config into a tlsutil config. func (c *Config) tlsConfig() *tlsutil.Config { tlsConf := &tlsutil.Config{ diff --git a/agent/consul/server_serf.go b/agent/consul/server_serf.go index 7c150f752..42fce4c8c 100644 --- a/agent/consul/server_serf.go +++ b/agent/consul/server_serf.go @@ -67,8 +67,11 @@ func (s *Server) setupSerf(conf *serf.Config, ch chan serf.Event, path string, w if s.config.UseTLS { conf.Tags["use_tls"] = "1" } - conf.MemberlistConfig.LogOutput = s.config.LogOutput - conf.LogOutput = s.config.LogOutput + if s.logger == nil { + conf.MemberlistConfig.LogOutput = s.config.LogOutput + conf.LogOutput = s.config.LogOutput + } + conf.MemberlistConfig.Logger = s.logger conf.Logger = s.logger conf.EventCh = ch conf.ProtocolVersion = protocolVersionMap[s.config.ProtocolVersion] diff --git a/agent/consul/server_test.go b/agent/consul/server_test.go index c646f66d1..d1ba235ec 100644 --- a/agent/consul/server_test.go +++ b/agent/consul/server_test.go @@ -13,6 +13,7 @@ import ( "github.com/hashicorp/consul/agent/metadata" "github.com/hashicorp/consul/agent/token" + "github.com/hashicorp/consul/test/porter" "github.com/hashicorp/consul/testrpc" "github.com/hashicorp/consul/testutil" "github.com/hashicorp/consul/testutil/retry" @@ -40,6 +41,10 @@ func testServerConfig(t *testing.T) (string, *Config) { dir := testutil.TempDir(t, "consul") config := DefaultConfig() + ports, err := porter.RandomPorts(3) + if err != nil { + t.Fatal("RandomPorts:", err) + } config.NodeName = uniqueNodeName(t.Name()) config.Bootstrap = true config.Datacenter = "dc1" @@ -48,7 +53,7 @@ func testServerConfig(t *testing.T) (string, *Config) { // bind the rpc server to a random port. config.RPCAdvertise will be // set to the listen address unless it was set in the configuration. // In that case get the address from srv.Listener.Addr(). - config.RPCAddr = &net.TCPAddr{IP: []byte{127, 0, 0, 1}} + config.RPCAddr = &net.TCPAddr{IP: []byte{127, 0, 0, 1}, Port: ports[0]} nodeID, err := uuid.GenerateUUID() if err != nil { @@ -60,14 +65,16 @@ func testServerConfig(t *testing.T) (string, *Config) { // memberlist will update the value of BindPort after bind // to the actual value. config.SerfLANConfig.MemberlistConfig.BindAddr = "127.0.0.1" - config.SerfLANConfig.MemberlistConfig.BindPort = 0 + config.SerfLANConfig.MemberlistConfig.BindPort = ports[1] + config.SerfLANConfig.MemberlistConfig.AdvertisePort = ports[1] config.SerfLANConfig.MemberlistConfig.SuspicionMult = 2 config.SerfLANConfig.MemberlistConfig.ProbeTimeout = 50 * time.Millisecond config.SerfLANConfig.MemberlistConfig.ProbeInterval = 100 * time.Millisecond config.SerfLANConfig.MemberlistConfig.GossipInterval = 100 * time.Millisecond config.SerfWANConfig.MemberlistConfig.BindAddr = "127.0.0.1" - config.SerfWANConfig.MemberlistConfig.BindPort = 0 + config.SerfWANConfig.MemberlistConfig.BindPort = ports[2] + config.SerfWANConfig.MemberlistConfig.AdvertisePort = ports[2] config.SerfWANConfig.MemberlistConfig.SuspicionMult = 2 config.SerfWANConfig.MemberlistConfig.ProbeTimeout = 50 * time.Millisecond config.SerfWANConfig.MemberlistConfig.ProbeInterval = 100 * time.Millisecond diff --git a/agent/coordinate_endpoint_test.go b/agent/coordinate_endpoint_test.go index 414d9e6fd..455611ea6 100644 --- a/agent/coordinate_endpoint_test.go +++ b/agent/coordinate_endpoint_test.go @@ -12,7 +12,7 @@ import ( func TestCoordinate_Datacenters(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/coordinate/datacenters", nil) @@ -33,7 +33,7 @@ func TestCoordinate_Datacenters(t *testing.T) { func TestCoordinate_Nodes(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Make sure an empty list is non-nil. diff --git a/agent/dns.go b/agent/dns.go index d9fcf4dab..c2ccd5249 100644 --- a/agent/dns.go +++ b/agent/dns.go @@ -12,6 +12,7 @@ import ( "regexp" "github.com/armon/go-metrics" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/consul" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/lib" @@ -34,12 +35,26 @@ const ( var InvalidDnsRe = regexp.MustCompile(`[^A-Za-z0-9\\-]+`) +type dnsConfig struct { + AllowStale bool + Datacenter string + EnableTruncate bool + MaxStale time.Duration + NodeName string + NodeTTL time.Duration + OnlyPassing bool + RecursorTimeout time.Duration + SegmentName string + ServiceTTL map[string]time.Duration + UDPAnswerLimit int +} + // DNSServer is used to wrap an Agent and expose various // service discovery endpoints using a DNS interface. type DNSServer struct { *dns.Server agent *Agent - config *DNSConfig + config *dnsConfig domain string recursors []string logger *log.Logger @@ -61,20 +76,37 @@ func NewDNSServer(a *Agent) (*DNSServer, error) { } // Make sure domain is FQDN, make it case insensitive for ServeMux - domain := dns.Fqdn(strings.ToLower(a.config.Domain)) + domain := dns.Fqdn(strings.ToLower(a.config.DNSDomain)) + dnscfg := GetDNSConfig(a.config) srv := &DNSServer{ agent: a, - config: &a.config.DNSConfig, + config: dnscfg, domain: domain, logger: a.logger, recursors: recursors, } - srv.disableCompression.Store(a.config.DNSConfig.DisableCompression) + srv.disableCompression.Store(a.config.DNSDisableCompression) return srv, nil } +func GetDNSConfig(conf *config.RuntimeConfig) *dnsConfig { + return &dnsConfig{ + AllowStale: conf.DNSAllowStale, + Datacenter: conf.Datacenter, + EnableTruncate: conf.DNSEnableTruncate, + MaxStale: conf.DNSMaxStale, + NodeName: conf.NodeName, + NodeTTL: conf.DNSNodeTTL, + OnlyPassing: conf.DNSOnlyPassing, + RecursorTimeout: conf.DNSRecursorTimeout, + SegmentName: conf.SegmentName, + ServiceTTL: conf.DNSServiceTTL, + UDPAnswerLimit: conf.DNSUDPAnswerLimit, + } +} + func (s *DNSServer) ListenAndServe(network, addr string, notif func()) error { mux := dns.NewServeMux() mux.HandleFunc("arpa.", s.handlePtr) @@ -150,7 +182,7 @@ func (d *DNSServer) handlePtr(resp dns.ResponseWriter, req *dns.Msg) { Datacenter: datacenter, QueryOptions: structs.QueryOptions{ Token: d.agent.tokens.UserToken(), - AllowStale: *d.config.AllowStale, + AllowStale: d.config.AllowStale, }, } var out structs.IndexedNodes @@ -309,7 +341,6 @@ func (d *DNSServer) nameservers(edns bool) (ns []dns.RR, extra []dns.RR) { } ns = append(ns, nsrr) - // A or AAAA glue record glue := d.formatNodeRecord(addr, fqdn, dns.TypeANY, d.config.NodeTTL, edns) extra = append(extra, glue...) @@ -466,7 +497,7 @@ func (d *DNSServer) nodeLookup(network, datacenter, node string, req, resp *dns. Node: node, QueryOptions: structs.QueryOptions{ Token: d.agent.tokens.UserToken(), - AllowStale: *d.config.AllowStale, + AllowStale: d.config.AllowStale, }, } var out structs.IndexedNodeServices @@ -621,7 +652,7 @@ func syncExtra(index map[string]dns.RR, resp *dns.Msg) { // 1035. Enforce an arbitrary limit that can be further ratcheted down by // config, and then make sure the response doesn't exceed 512 bytes. Any extra // records will be trimmed along with answers. -func trimUDPResponse(config *DNSConfig, req, resp *dns.Msg) (trimmed bool) { +func trimUDPResponse(req, resp *dns.Msg, udpAnswerLimit int) (trimmed bool) { numAnswers := len(resp.Answer) hasExtra := len(resp.Extra) > 0 maxSize := defaultMaxUDPSize @@ -642,7 +673,7 @@ func trimUDPResponse(config *DNSConfig, req, resp *dns.Msg) (trimmed bool) { } // This cuts UDP responses to a useful but limited number of responses. - maxAnswers := lib.MinInt(maxUDPAnswerLimit, config.UDPAnswerLimit) + maxAnswers := lib.MinInt(maxUDPAnswerLimit, udpAnswerLimit) if maxSize == defaultMaxUDPSize && numAnswers > maxAnswers { resp.Answer = resp.Answer[:maxAnswers] if hasExtra { @@ -678,7 +709,7 @@ func (d *DNSServer) lookupServiceNodes(datacenter, service, tag string) (structs TagFilter: tag != "", QueryOptions: structs.QueryOptions{ Token: d.agent.tokens.UserToken(), - AllowStale: *d.config.AllowStale, + AllowStale: d.config.AllowStale, }, } @@ -703,7 +734,6 @@ func (d *DNSServer) lookupServiceNodes(datacenter, service, tag string) (structs // Filter out any service nodes due to health checks out.Nodes = out.Nodes.Filter(d.config.OnlyPassing) - return out, nil } @@ -746,7 +776,7 @@ func (d *DNSServer) serviceLookup(network, datacenter, service, tag string, req, // If the network is not TCP, restrict the number of responses if network != "tcp" { - wasTrimmed := trimUDPResponse(d.config, req, resp) + wasTrimmed := trimUDPResponse(req, resp, d.config.UDPAnswerLimit) // Flag that there are more records to return in the UDP response if wasTrimmed && d.config.EnableTruncate { @@ -769,7 +799,7 @@ func (d *DNSServer) preparedQueryLookup(network, datacenter, query string, req, QueryIDOrName: query, QueryOptions: structs.QueryOptions{ Token: d.agent.tokens.UserToken(), - AllowStale: *d.config.AllowStale, + AllowStale: d.config.AllowStale, }, // Always pass the local agent through. In the DNS interface, there @@ -778,7 +808,7 @@ func (d *DNSServer) preparedQueryLookup(network, datacenter, query string, req, // relative to ourself on the server side. Agent: structs.QuerySource{ Datacenter: d.agent.config.Datacenter, - Segment: d.agent.config.Segment, + Segment: d.agent.config.SegmentName, Node: d.agent.config.NodeName, }, } @@ -855,7 +885,7 @@ RPC: // If the network is not TCP, restrict the number of responses. if network != "tcp" { - wasTrimmed := trimUDPResponse(d.config, req, resp) + wasTrimmed := trimUDPResponse(req, resp, d.config.UDPAnswerLimit) // Flag that there are more records to return in the UDP response if wasTrimmed && d.config.EnableTruncate { diff --git a/agent/dns_test.go b/agent/dns_test.go index f6a6e73d9..02247db2a 100644 --- a/agent/dns_test.go +++ b/agent/dns_test.go @@ -9,6 +9,7 @@ import ( "testing" "time" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/lib" @@ -90,7 +91,7 @@ func TestRecursorAddr(t *testing.T) { func TestDNS_NodeLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -112,8 +113,7 @@ func TestDNS_NodeLookup(t *testing.T) { m.SetQuestion("foo.node.consul.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -138,7 +138,7 @@ func TestDNS_NodeLookup(t *testing.T) { m.SetQuestion("foo.node.dc1.consul.", dns.TypeANY) c = new(dns.Client) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -163,7 +163,7 @@ func TestDNS_NodeLookup(t *testing.T) { m.SetQuestion("nofoo.node.dc1.consul.", dns.TypeANY) c = new(dns.Client) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -184,7 +184,7 @@ func TestDNS_NodeLookup(t *testing.T) { func TestDNS_CaseInsensitiveNodeLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -203,8 +203,7 @@ func TestDNS_CaseInsensitiveNodeLookup(t *testing.T) { m.SetQuestion("fOO.node.dc1.consul.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -216,7 +215,7 @@ func TestDNS_CaseInsensitiveNodeLookup(t *testing.T) { func TestDNS_NodeLookup_PeriodName(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node with period in name @@ -235,8 +234,7 @@ func TestDNS_NodeLookup_PeriodName(t *testing.T) { m.SetQuestion("foo.bar.node.consul.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -256,7 +254,7 @@ func TestDNS_NodeLookup_PeriodName(t *testing.T) { func TestDNS_NodeLookup_AAAA(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -275,8 +273,7 @@ func TestDNS_NodeLookup_AAAA(t *testing.T) { m.SetQuestion("bar.node.consul.", dns.TypeAAAA) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -307,9 +304,9 @@ func TestDNS_NodeLookup_CNAME(t *testing.T) { }) defer recursor.Shutdown() - cfg := TestConfig() - cfg.DNSRecursor = recursor.Addr - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+recursor.Addr+`"] + `) defer a.Shutdown() // Register node @@ -328,8 +325,7 @@ func TestDNS_NodeLookup_CNAME(t *testing.T) { m.SetQuestion("google.node.consul.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -353,7 +349,7 @@ func TestDNS_NodeLookup_CNAME(t *testing.T) { func TestDNS_EDNS0(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -373,8 +369,7 @@ func TestDNS_EDNS0(t *testing.T) { m.SetQuestion("foo.node.dc1.consul.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -393,7 +388,7 @@ func TestDNS_EDNS0(t *testing.T) { func TestDNS_ReverseLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -412,8 +407,7 @@ func TestDNS_ReverseLookup(t *testing.T) { m.SetQuestion("2.0.0.127.in-addr.arpa.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -433,9 +427,9 @@ func TestDNS_ReverseLookup(t *testing.T) { func TestDNS_ReverseLookup_CustomDomain(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Domain = "custom" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + domain = "custom" + `) defer a.Shutdown() // Register node @@ -454,8 +448,7 @@ func TestDNS_ReverseLookup_CustomDomain(t *testing.T) { m.SetQuestion("2.0.0.127.in-addr.arpa.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -475,7 +468,7 @@ func TestDNS_ReverseLookup_CustomDomain(t *testing.T) { func TestDNS_ReverseLookup_IPV6(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -494,8 +487,7 @@ func TestDNS_ReverseLookup_IPV6(t *testing.T) { m.SetQuestion("2.4.2.4.2.4.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -515,7 +507,7 @@ func TestDNS_ReverseLookup_IPV6(t *testing.T) { func TestDNS_ServiceLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service. @@ -565,8 +557,7 @@ func TestDNS_ServiceLookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -614,8 +605,7 @@ func TestDNS_ServiceLookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -637,9 +627,9 @@ func TestDNS_ServiceLookup(t *testing.T) { func TestDNS_ServiceLookupWithInternalServiceAddress(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeName = "my.test-node" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_name = "my.test-node" + `) defer a.Shutdown() // Register a node with a service. @@ -667,8 +657,7 @@ func TestDNS_ServiceLookupWithInternalServiceAddress(t *testing.T) { m.SetQuestion("db.service.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -698,7 +687,7 @@ func TestDNS_ServiceLookupWithInternalServiceAddress(t *testing.T) { func TestDNS_ExternalServiceLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with an external service. @@ -728,8 +717,7 @@ func TestDNS_ExternalServiceLookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -770,10 +758,10 @@ func TestDNS_ExternalServiceLookup(t *testing.T) { func TestDNS_ExternalServiceToConsulCNAMELookup(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Domain = "CONSUL." - cfg.NodeName = "test node" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + domain = "CONSUL." + node_name = "test node" + `) defer a.Shutdown() // Register the initial node with a service @@ -822,8 +810,7 @@ func TestDNS_ExternalServiceToConsulCNAMELookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -883,33 +870,17 @@ func TestDNS_ExternalServiceToConsulCNAMELookup(t *testing.T) { func TestDNS_NSRecords(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Domain = "CONSUL." - cfg.NodeName = "server1" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + domain = "CONSUL." + node_name = "server1" + `) defer a.Shutdown() - // Register node - args := &structs.RegisterRequest{ - Datacenter: "dc1", - Node: "foo", - Address: "127.0.0.1", - TaggedAddresses: map[string]string{ - "wan": "127.0.0.2", - }, - } - - var out struct{} - if err := a.RPC("Catalog.Register", args, &out); err != nil { - t.Fatalf("err: %v", err) - } - m := new(dns.Msg) m.SetQuestion("something.node.consul.", dns.TypeNS) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -929,40 +900,22 @@ func TestDNS_NSRecords(t *testing.T) { } verify.Values(t, "extra", in.Extra, wantExtra) - } func TestDNS_NSRecords_IPV6(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.Domain = "CONSUL." - cfg.NodeName = "server1" - cfg.AdvertiseAddr = "::1" - cfg.AdvertiseAddrWan = "::1" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + domain = "CONSUL." + node_name = "server1" + advertise_addr = "::1" + `) defer a.Shutdown() - // Register node - args := &structs.RegisterRequest{ - Datacenter: "dc1", - Node: "foo", - Address: "127.0.0.1", - TaggedAddresses: map[string]string{ - "wan": "127.0.0.2", - }, - } - - var out struct{} - if err := a.RPC("Catalog.Register", args, &out); err != nil { - t.Fatalf("err: %v", err) - } - m := new(dns.Msg) m.SetQuestion("server1.node.dc1.consul.", dns.TypeNS) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -987,9 +940,9 @@ func TestDNS_NSRecords_IPV6(t *testing.T) { func TestDNS_ExternalServiceToConsulCNAMENestedLookup(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeName = "test-node" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_name = "test-node" + `) defer a.Shutdown() // Register the initial node with a service @@ -1055,8 +1008,7 @@ func TestDNS_ExternalServiceToConsulCNAMENestedLookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1129,7 +1081,7 @@ func TestDNS_ExternalServiceToConsulCNAMENestedLookup(t *testing.T) { func TestDNS_ServiceLookup_ServiceAddress_A(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service. @@ -1180,8 +1132,7 @@ func TestDNS_ServiceLookup_ServiceAddress_A(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1222,7 +1173,7 @@ func TestDNS_ServiceLookup_ServiceAddress_A(t *testing.T) { func TestDNS_ServiceLookup_ServiceAddress_CNAME(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service whose address isn't an IP. @@ -1273,8 +1224,7 @@ func TestDNS_ServiceLookup_ServiceAddress_CNAME(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1315,7 +1265,7 @@ func TestDNS_ServiceLookup_ServiceAddress_CNAME(t *testing.T) { func TestDNS_ServiceLookup_ServiceAddressIPV6(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service. @@ -1366,8 +1316,7 @@ func TestDNS_ServiceLookup_ServiceAddressIPV6(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1408,22 +1357,22 @@ func TestDNS_ServiceLookup_ServiceAddressIPV6(t *testing.T) { func TestDNS_ServiceLookup_WanAddress(t *testing.T) { t.Parallel() - cfg1 := TestConfig() - cfg1.Datacenter = "dc1" - cfg1.TranslateWanAddrs = true - cfg1.ACLDatacenter = "" - a1 := NewTestAgent(t.Name(), cfg1) + a1 := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Datacenter = "dc2" - cfg2.TranslateWanAddrs = true - cfg2.ACLDatacenter = "" - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + datacenter = "dc2" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a2.Shutdown() // Join WAN cluster - addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.SerfPortWAN) if _, err := a2.JoinWAN([]string{addr}); err != nil { t.Fatalf("err: %v", err) } @@ -1486,7 +1435,8 @@ func TestDNS_ServiceLookup_WanAddress(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a1.Config.ClientListener("", a1.Config.Ports.DNS) + + addr := a1.config.DNSAddrs[0] in, _, err := c.Exchange(m, addr.String()) if err != nil { t.Fatalf("err: %v", err) @@ -1514,7 +1464,7 @@ func TestDNS_ServiceLookup_WanAddress(t *testing.T) { m.SetQuestion(question, dns.TypeA) c := new(dns.Client) - addr, _ := a1.Config.ClientListener("", a1.Config.Ports.DNS) + addr := a1.config.DNSAddrs[0] in, _, err := c.Exchange(m, addr.String()) if err != nil { t.Fatalf("err: %v", err) @@ -1542,7 +1492,7 @@ func TestDNS_ServiceLookup_WanAddress(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a2.Config.ClientListener("", a2.Config.Ports.DNS) + addr := a2.Config.DNSAddrs[0] in, _, err := c.Exchange(m, addr.String()) if err != nil { t.Fatalf("err: %v", err) @@ -1570,7 +1520,7 @@ func TestDNS_ServiceLookup_WanAddress(t *testing.T) { m.SetQuestion(question, dns.TypeA) c := new(dns.Client) - addr, _ := a2.Config.ClientListener("", a2.Config.Ports.DNS) + addr := a2.Config.DNSAddrs[0] in, _, err := c.Exchange(m, addr.String()) if err != nil { t.Fatalf("err: %v", err) @@ -1595,7 +1545,7 @@ func TestDNS_ServiceLookup_WanAddress(t *testing.T) { func TestDNS_CaseInsensitiveServiceLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service. @@ -1652,8 +1602,7 @@ func TestDNS_CaseInsensitiveServiceLookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1666,7 +1615,7 @@ func TestDNS_CaseInsensitiveServiceLookup(t *testing.T) { func TestDNS_ServiceLookup_TagPeriod(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -1690,8 +1639,7 @@ func TestDNS_ServiceLookup_TagPeriod(t *testing.T) { m.SetQuestion("v1.master.db.service.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1725,7 +1673,7 @@ func TestDNS_ServiceLookup_TagPeriod(t *testing.T) { func TestDNS_ServiceLookup_PreparedQueryNamePeriod(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service. @@ -1769,8 +1717,7 @@ func TestDNS_ServiceLookup_PreparedQueryNamePeriod(t *testing.T) { m.SetQuestion("some.query.we.like.query.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1804,7 +1751,7 @@ func TestDNS_ServiceLookup_PreparedQueryNamePeriod(t *testing.T) { func TestDNS_ServiceLookup_Dedup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a single node with multiple instances of a service. @@ -1885,8 +1832,7 @@ func TestDNS_ServiceLookup_Dedup(t *testing.T) { m.SetQuestion(question, dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -1907,7 +1853,7 @@ func TestDNS_ServiceLookup_Dedup(t *testing.T) { func TestDNS_ServiceLookup_Dedup_SRV(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a single node with multiple instances of a service. @@ -1988,8 +1934,7 @@ func TestDNS_ServiceLookup_Dedup_SRV(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -2043,17 +1988,16 @@ func TestDNS_Recurse(t *testing.T) { }) defer recursor.Shutdown() - cfg := TestConfig() - cfg.DNSRecursor = recursor.Addr - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+recursor.Addr+`"] + `) defer a.Shutdown() m := new(dns.Msg) m.SetQuestion("apple.com.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -2075,17 +2019,16 @@ func TestDNS_Recurse_Truncation(t *testing.T) { }) defer recursor.Shutdown() - cfg := TestConfig() - cfg.DNSRecursor = recursor.Addr - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+recursor.Addr+`"] + `) defer a.Shutdown() m := new(dns.Msg) m.SetQuestion("apple.com.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != dns.ErrTruncated { t.Fatalf("err: %v", err) } @@ -2116,10 +2059,12 @@ func TestDNS_RecursorTimeout(t *testing.T) { } defer resolver.Close() - cfg := TestConfig() - cfg.DNSRecursor = resolver.LocalAddr().String() // host must cause a connection|read|write timeout - cfg.DNSConfig.RecursorTimeout = serverClientTimeout - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+resolver.LocalAddr().String()+`"] // host must cause a connection|read|write timeout + dns_config { + recursor_timeout = "`+serverClientTimeout.String()+`" + } + `) defer a.Shutdown() m := new(dns.Msg) @@ -2127,10 +2072,9 @@ func TestDNS_RecursorTimeout(t *testing.T) { // This client calling the server under test must have a longer timeout than the one we set internally c := &dns.Client{Timeout: testClientTimeout} - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) start := time.Now() - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) duration := time.Now().Sub(start) @@ -2153,7 +2097,7 @@ func TestDNS_RecursorTimeout(t *testing.T) { func TestDNS_ServiceLookup_FilterCritical(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register nodes with health checks in various states. @@ -2281,8 +2225,7 @@ func TestDNS_ServiceLookup_FilterCritical(t *testing.T) { m.SetQuestion(question, dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -2309,7 +2252,7 @@ func TestDNS_ServiceLookup_FilterCritical(t *testing.T) { func TestDNS_ServiceLookup_OnlyFailing(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register nodes with all health checks in a critical state. @@ -2403,8 +2346,7 @@ func TestDNS_ServiceLookup_OnlyFailing(t *testing.T) { m.SetQuestion(question, dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -2422,9 +2364,11 @@ func TestDNS_ServiceLookup_OnlyFailing(t *testing.T) { func TestDNS_ServiceLookup_OnlyPassing(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.DNSConfig.OnlyPassing = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + dns_config { + only_passing = true + } + `) defer a.Shutdown() // Register nodes with health checks in various states. @@ -2523,8 +2467,7 @@ func TestDNS_ServiceLookup_OnlyPassing(t *testing.T) { m.SetQuestion(question, dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -2545,7 +2488,7 @@ func TestDNS_ServiceLookup_OnlyPassing(t *testing.T) { func TestDNS_ServiceLookup_Randomize(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a large number of nodes. @@ -2592,13 +2535,12 @@ func TestDNS_ServiceLookup_Randomize(t *testing.T) { } for _, question := range questions { uniques := map[string]struct{}{} - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) for i := 0; i < 10; i++ { m := new(dns.Msg) m.SetQuestion(question, dns.TypeANY) c := &dns.Client{Net: "udp"} - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -2636,9 +2578,11 @@ func TestDNS_ServiceLookup_Randomize(t *testing.T) { func TestDNS_ServiceLookup_Truncate(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.DNSConfig.EnableTruncate = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + dns_config { + enable_truncate = true + } + `) defer a.Shutdown() // Register a large number of nodes. @@ -2687,9 +2631,8 @@ func TestDNS_ServiceLookup_Truncate(t *testing.T) { m := new(dns.Msg) m.SetQuestion(question, dns.TypeANY) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) c := new(dns.Client) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil && err != dns.ErrTruncated { t.Fatalf("err: %v", err) } @@ -2703,9 +2646,11 @@ func TestDNS_ServiceLookup_Truncate(t *testing.T) { func TestDNS_ServiceLookup_LargeResponses(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.DNSConfig.EnableTruncate = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + dns_config { + enable_truncate = true + } + `) defer a.Shutdown() longServiceName := "this-is-a-very-very-very-very-very-long-name-for-a-service" @@ -2758,8 +2703,7 @@ func TestDNS_ServiceLookup_LargeResponses(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil && err != dns.ErrTruncated { t.Fatalf("err: %v", err) } @@ -2803,10 +2747,12 @@ func TestDNS_ServiceLookup_LargeResponses(t *testing.T) { func testDNS_ServiceLookup_responseLimits(t *testing.T, answerLimit int, qType uint16, expectedService, expectedQuery, expectedQueryID int) (bool, error) { - cfg := TestConfig() - cfg.DNSConfig.UDPAnswerLimit = answerLimit - cfg.NodeName = "test-node" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + node_name = "test-node" + dns_config { + udp_answer_limit = `+fmt.Sprintf("%d", answerLimit)+` + } + `) defer a.Shutdown() for i := 0; i < generateNumNodes; i++ { @@ -2857,9 +2803,8 @@ func testDNS_ServiceLookup_responseLimits(t *testing.T, answerLimit int, qType u m := new(dns.Msg) m.SetQuestion(question, qType) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) c := &dns.Client{Net: "udp"} - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { return false, fmt.Errorf("err: %v", err) } @@ -2964,9 +2909,9 @@ func TestDNS_ServiceLookup_CNAME(t *testing.T) { }) defer recursor.Shutdown() - cfg := TestConfig() - cfg.DNSRecursor = recursor.Addr - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+recursor.Addr+`"] + `) defer a.Shutdown() // Register a node with a name for an address. @@ -3015,8 +2960,7 @@ func TestDNS_ServiceLookup_CNAME(t *testing.T) { m.SetQuestion(question, dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3061,12 +3005,14 @@ func TestDNS_NodeLookup_TTL(t *testing.T) { }) defer recursor.Shutdown() - cfg := TestConfig() - cfg.DNSRecursor = recursor.Addr - cfg.DNSConfig.NodeTTL = 10 * time.Second - cfg.DNSConfig.AllowStale = Bool(true) - cfg.DNSConfig.MaxStale = time.Second - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+recursor.Addr+`"] + dns_config { + node_ttl = "10s" + allow_stale = true + max_stale = "1s" + } + `) defer a.Shutdown() // Register node @@ -3085,8 +3031,7 @@ func TestDNS_NodeLookup_TTL(t *testing.T) { m.SetQuestion("foo.node.consul.", dns.TypeANY) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3120,7 +3065,7 @@ func TestDNS_NodeLookup_TTL(t *testing.T) { m = new(dns.Msg) m.SetQuestion("bar.node.consul.", dns.TypeANY) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3153,7 +3098,7 @@ func TestDNS_NodeLookup_TTL(t *testing.T) { m = new(dns.Msg) m.SetQuestion("google.node.consul.", dns.TypeANY) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3177,14 +3122,16 @@ func TestDNS_NodeLookup_TTL(t *testing.T) { func TestDNS_ServiceLookup_TTL(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.DNSConfig.ServiceTTL = map[string]time.Duration{ - "db": 10 * time.Second, - "*": 5 * time.Second, - } - cfg.DNSConfig.AllowStale = Bool(true) - cfg.DNSConfig.MaxStale = time.Second - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + dns_config { + service_ttl = { + "db" = "10s" + "*" = "5s" + } + allow_stale = true + max_stale = "1s" + } + `) defer a.Shutdown() // Register node with 2 services @@ -3221,8 +3168,7 @@ func TestDNS_ServiceLookup_TTL(t *testing.T) { m.SetQuestion("db.service.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3249,7 +3195,7 @@ func TestDNS_ServiceLookup_TTL(t *testing.T) { m = new(dns.Msg) m.SetQuestion("api.service.consul.", dns.TypeSRV) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3277,14 +3223,16 @@ func TestDNS_ServiceLookup_TTL(t *testing.T) { func TestDNS_PreparedQuery_TTL(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.DNSConfig.ServiceTTL = map[string]time.Duration{ - "db": 10 * time.Second, - "*": 5 * time.Second, - } - cfg.DNSConfig.AllowStale = Bool(true) - cfg.DNSConfig.MaxStale = time.Second - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + dns_config { + service_ttl = { + "db" = "10s" + "*" = "5s" + } + allow_stale = true + max_stale = "1s" + } + `) defer a.Shutdown() // Register a node and a service. @@ -3378,8 +3326,7 @@ func TestDNS_PreparedQuery_TTL(t *testing.T) { m.SetQuestion("db-ttl.query.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3408,7 +3355,7 @@ func TestDNS_PreparedQuery_TTL(t *testing.T) { // config otherwise. m = new(dns.Msg) m.SetQuestion("db-nottl.query.consul.", dns.TypeSRV) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3437,7 +3384,7 @@ func TestDNS_PreparedQuery_TTL(t *testing.T) { // card value should be used. m = new(dns.Msg) m.SetQuestion("api-nottl.query.consul.", dns.TypeSRV) - in, _, err = c.Exchange(m, addr.String()) + in, _, err = c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3465,22 +3412,22 @@ func TestDNS_PreparedQuery_TTL(t *testing.T) { func TestDNS_PreparedQuery_Failover(t *testing.T) { t.Parallel() - cfg1 := TestConfig() - cfg1.Datacenter = "dc1" - cfg1.TranslateWanAddrs = true - cfg1.ACLDatacenter = "" - a1 := NewTestAgent(t.Name(), cfg1) + a1 := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Datacenter = "dc2" - cfg2.TranslateWanAddrs = true - cfg2.ACLDatacenter = "" - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + datacenter = "dc2" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a2.Shutdown() // Join WAN cluster. - addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.SerfPortWAN) if _, err := a2.JoinWAN([]string{addr}); err != nil { t.Fatalf("err: %v", err) } @@ -3541,7 +3488,7 @@ func TestDNS_PreparedQuery_Failover(t *testing.T) { m.SetQuestion("my-query.query.consul.", dns.TypeSRV) c := new(dns.Client) - cl_addr, _ := a1.Config.ClientListener("", a1.Config.Ports.DNS) + cl_addr := a1.config.DNSAddrs[0] in, _, err := c.Exchange(m, cl_addr.String()) if err != nil { t.Fatalf("err: %v", err) @@ -3577,7 +3524,7 @@ func TestDNS_PreparedQuery_Failover(t *testing.T) { func TestDNS_ServiceLookup_SRV_RFC(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -3609,8 +3556,7 @@ func TestDNS_ServiceLookup_SRV_RFC(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3652,7 +3598,7 @@ func TestDNS_ServiceLookup_SRV_RFC(t *testing.T) { func TestDNS_ServiceLookup_SRV_RFC_TCP_Default(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node @@ -3684,8 +3630,7 @@ func TestDNS_ServiceLookup_SRV_RFC_TCP_Default(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3736,13 +3681,13 @@ func TestDNS_ServiceLookup_FilterACL(t *testing.T) { } for _, tt := range tests { t.Run("ACLToken == "+tt.token, func(t *testing.T) { - cfg := TestConfig() - cfg.ACLToken = tt.token - cfg.ACLMasterToken = "root" - cfg.ACLDatacenter = "dc1" - cfg.ACLDownPolicy = "deny" - cfg.ACLDefaultPolicy = "deny" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + acl_token = "`+tt.token+`" + acl_master_token = "root" + acl_datacenter = "dc1" + acl_down_policy = "deny" + acl_default_policy = "deny" + `) defer a.Shutdown() // Register a service @@ -3763,11 +3708,10 @@ func TestDNS_ServiceLookup_FilterACL(t *testing.T) { // Set up the DNS query c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) m := new(dns.Msg) m.SetQuestion("foo.service.consul.", dns.TypeA) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3780,7 +3724,7 @@ func TestDNS_ServiceLookup_FilterACL(t *testing.T) { func TestDNS_AddressLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Look up the addresses @@ -3792,8 +3736,7 @@ func TestDNS_AddressLookup(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3817,7 +3760,7 @@ func TestDNS_AddressLookup(t *testing.T) { func TestDNS_AddressLookupIPV6(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Look up the addresses @@ -3830,8 +3773,7 @@ func TestDNS_AddressLookupIPV6(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3855,17 +3797,15 @@ func TestDNS_AddressLookupIPV6(t *testing.T) { func TestDNS_NonExistingLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - // lookup a non-existing node, we should receive a SOA m := new(dns.Msg) m.SetQuestion("nonexisting.consul.", dns.TypeANY) c := new(dns.Client) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3885,7 +3825,7 @@ func TestDNS_NonExistingLookup(t *testing.T) { func TestDNS_NonExistingLookupEmptyAorAAAA(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a v6-only service and a v4-only service. @@ -3964,9 +3904,8 @@ func TestDNS_NonExistingLookupEmptyAorAAAA(t *testing.T) { m := new(dns.Msg) m.SetQuestion(question, dns.TypeAAAA) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) c := new(dns.Client) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -3998,9 +3937,8 @@ func TestDNS_NonExistingLookupEmptyAorAAAA(t *testing.T) { m := new(dns.Msg) m.SetQuestion(question, dns.TypeA) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) c := new(dns.Client) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -4025,10 +3963,12 @@ func TestDNS_NonExistingLookupEmptyAorAAAA(t *testing.T) { func TestDNS_PreparedQuery_AllowStale(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.DNSConfig.AllowStale = Bool(true) - cfg.DNSConfig.MaxStale = time.Second - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + dns_config { + allow_stale = true + max_stale = "1s" + } + `) defer a.Shutdown() m := MockPreparedQuery{ @@ -4050,8 +3990,7 @@ func TestDNS_PreparedQuery_AllowStale(t *testing.T) { m.SetQuestion("nope.query.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -4073,7 +4012,7 @@ func TestDNS_PreparedQuery_AllowStale(t *testing.T) { func TestDNS_InvalidQueries(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Try invalid forms of queries that should hit the special invalid case @@ -4089,8 +4028,7 @@ func TestDNS_InvalidQueries(t *testing.T) { m.SetQuestion(question, dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - in, _, err := c.Exchange(m, addr.String()) + in, _, err := c.Exchange(m, a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -4112,7 +4050,7 @@ func TestDNS_InvalidQueries(t *testing.T) { func TestDNS_PreparedQuery_AgentSource(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -4136,8 +4074,7 @@ func TestDNS_PreparedQuery_AgentSource(t *testing.T) { m.SetQuestion("foo.query.consul.", dns.TypeSRV) c := new(dns.Client) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - if _, _, err := c.Exchange(m, addr.String()); err != nil { + if _, _, err := c.Exchange(m, a.DNSAddr()); err != nil { t.Fatalf("err: %v", err) } } @@ -4169,8 +4106,8 @@ func TestDNS_trimUDPResponse_NoTrim(t *testing.T) { }, } - config := &DefaultConfig().DNSConfig - if trimmed := trimUDPResponse(config, req, resp); trimmed { + cfg := config.DefaultRuntimeConfig(`data_dir = "a" bind_addr = "127.0.0.1"`) + if trimmed := trimUDPResponse(req, resp, cfg.DNSUDPAnswerLimit); trimmed { t.Fatalf("Bad %#v", *resp) } @@ -4203,10 +4140,10 @@ func TestDNS_trimUDPResponse_NoTrim(t *testing.T) { func TestDNS_trimUDPResponse_TrimLimit(t *testing.T) { t.Parallel() - config := &DefaultConfig().DNSConfig + cfg := config.DefaultRuntimeConfig(`data_dir = "a" bind_addr = "127.0.0.1"`) req, resp, expected := &dns.Msg{}, &dns.Msg{}, &dns.Msg{} - for i := 0; i < config.UDPAnswerLimit+1; i++ { + for i := 0; i < cfg.DNSUDPAnswerLimit+1; i++ { target := fmt.Sprintf("ip-10-0-1-%d.node.dc1.consul.", 185+i) srv := &dns.SRV{ Hdr: dns.RR_Header{ @@ -4227,13 +4164,13 @@ func TestDNS_trimUDPResponse_TrimLimit(t *testing.T) { resp.Answer = append(resp.Answer, srv) resp.Extra = append(resp.Extra, a) - if i < config.UDPAnswerLimit { + if i < cfg.DNSUDPAnswerLimit { expected.Answer = append(expected.Answer, srv) expected.Extra = append(expected.Extra, a) } } - if trimmed := trimUDPResponse(config, req, resp); !trimmed { + if trimmed := trimUDPResponse(req, resp, cfg.DNSUDPAnswerLimit); !trimmed { t.Fatalf("Bad %#v", *resp) } if !reflect.DeepEqual(resp, expected) { @@ -4243,7 +4180,7 @@ func TestDNS_trimUDPResponse_TrimLimit(t *testing.T) { func TestDNS_trimUDPResponse_TrimSize(t *testing.T) { t.Parallel() - config := &DefaultConfig().DNSConfig + cfg := config.DefaultRuntimeConfig(`data_dir = "a" bind_addr = "127.0.0.1"`) req, resp := &dns.Msg{}, &dns.Msg{} for i := 0; i < 100; i++ { @@ -4271,7 +4208,7 @@ func TestDNS_trimUDPResponse_TrimSize(t *testing.T) { // We don't know the exact trim, but we know the resulting answer // data should match its extra data. - if trimmed := trimUDPResponse(config, req, resp); !trimmed { + if trimmed := trimUDPResponse(req, resp, cfg.DNSUDPAnswerLimit); !trimmed { t.Fatalf("Bad %#v", *resp) } if len(resp.Answer) == 0 || len(resp.Answer) != len(resp.Extra) { @@ -4296,7 +4233,7 @@ func TestDNS_trimUDPResponse_TrimSize(t *testing.T) { func TestDNS_trimUDPResponse_TrimSizeEDNS(t *testing.T) { t.Parallel() - config := &DefaultConfig().DNSConfig + cfg := config.DefaultRuntimeConfig(`data_dir = "a" bind_addr = "127.0.0.1"`) req, resp := &dns.Msg{}, &dns.Msg{} @@ -4330,10 +4267,10 @@ func TestDNS_trimUDPResponse_TrimSizeEDNS(t *testing.T) { respEDNS.Extra = append(respEDNS.Extra, resp.Extra...) // Trim each response - if trimmed := trimUDPResponse(config, req, resp); !trimmed { + if trimmed := trimUDPResponse(req, resp, cfg.DNSUDPAnswerLimit); !trimmed { t.Errorf("expected response to be trimmed: %#v", resp) } - if trimmed := trimUDPResponse(config, reqEDNS, respEDNS); !trimmed { + if trimmed := trimUDPResponse(reqEDNS, respEDNS, cfg.DNSUDPAnswerLimit); !trimmed { t.Errorf("expected edns to be trimmed: %#v", resp) } @@ -4599,10 +4536,10 @@ func TestDNS_syncExtra(t *testing.T) { func TestDNS_Compression_trimUDPResponse(t *testing.T) { t.Parallel() - config := &DefaultConfig().DNSConfig + cfg := config.DefaultRuntimeConfig(`data_dir = "a" bind_addr = "127.0.0.1"`) req, m := dns.Msg{}, dns.Msg{} - trimUDPResponse(config, &req, &m) + trimUDPResponse(&req, &m, cfg.DNSUDPAnswerLimit) if m.Compress { t.Fatalf("compression should be off") } @@ -4610,7 +4547,7 @@ func TestDNS_Compression_trimUDPResponse(t *testing.T) { // The trim function temporarily turns off compression, so we need to // make sure the setting gets restored properly. m.Compress = true - trimUDPResponse(config, &req, &m) + trimUDPResponse(&req, &m, cfg.DNSUDPAnswerLimit) if !m.Compress { t.Fatalf("compression should be on") } @@ -4618,7 +4555,7 @@ func TestDNS_Compression_trimUDPResponse(t *testing.T) { func TestDNS_Compression_Query(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node with a service. @@ -4667,8 +4604,7 @@ func TestDNS_Compression_Query(t *testing.T) { m := new(dns.Msg) m.SetQuestion(question, dns.TypeSRV) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - conn, err := dns.Dial("udp", addr.String()) + conn, err := dns.Dial("udp", a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -4705,7 +4641,7 @@ func TestDNS_Compression_Query(t *testing.T) { func TestDNS_Compression_ReverseLookup(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register node. @@ -4722,8 +4658,7 @@ func TestDNS_Compression_ReverseLookup(t *testing.T) { m := new(dns.Msg) m.SetQuestion("2.0.0.127.in-addr.arpa.", dns.TypeANY) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - conn, err := dns.Dial("udp", addr.String()) + conn, err := dns.Dial("udp", a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } @@ -4762,16 +4697,15 @@ func TestDNS_Compression_Recurse(t *testing.T) { }) defer recursor.Shutdown() - cfg := TestConfig() - cfg.DNSRecursor = recursor.Addr - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + recursors = ["`+recursor.Addr+`"] + `) defer a.Shutdown() m := new(dns.Msg) m.SetQuestion("apple.com.", dns.TypeANY) - addr, _ := a.Config.ClientListener("", a.Config.Ports.DNS) - conn, err := dns.Dial("udp", addr.String()) + conn, err := dns.Dial("udp", a.DNSAddr()) if err != nil { t.Fatalf("err: %v", err) } diff --git a/agent/event_endpoint_test.go b/agent/event_endpoint_test.go index 87388374d..90ba6fca1 100644 --- a/agent/event_endpoint_test.go +++ b/agent/event_endpoint_test.go @@ -16,7 +16,7 @@ import ( func TestEventFire(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer([]byte("test")) @@ -55,9 +55,9 @@ func TestEventFire(t *testing.T) { func TestEventFire_token(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLDefaultPolicy = "deny" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_default_policy = "deny" + `) defer a.Shutdown() // Create an ACL token @@ -116,7 +116,7 @@ func TestEventFire_token(t *testing.T) { func TestEventList(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() p := &UserEvent{Name: "test"} @@ -148,7 +148,7 @@ func TestEventList(t *testing.T) { func TestEventList_Filter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() p := &UserEvent{Name: "test"} @@ -235,7 +235,7 @@ func TestEventList_ACLFilter(t *testing.T) { func TestEventList_Blocking(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() p := &UserEvent{Name: "test"} @@ -286,7 +286,7 @@ func TestEventList_Blocking(t *testing.T) { func TestEventList_EventBufOrder(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Fire some events in a non-sequential order diff --git a/agent/health_endpoint_test.go b/agent/health_endpoint_test.go index 33011fcbb..5d2ae1445 100644 --- a/agent/health_endpoint_test.go +++ b/agent/health_endpoint_test.go @@ -18,7 +18,7 @@ import ( func TestHealthChecksInState(t *testing.T) { t.Parallel() t.Run("warning", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/state/warning?dc=dc1", nil) @@ -41,7 +41,7 @@ func TestHealthChecksInState(t *testing.T) { }) t.Run("passing", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/state/passing?dc=dc1", nil) @@ -66,7 +66,7 @@ func TestHealthChecksInState(t *testing.T) { func TestHealthChecksInState_NodeMetaFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() args := &structs.RegisterRequest{ @@ -106,7 +106,7 @@ func TestHealthChecksInState_NodeMetaFilter(t *testing.T) { func TestHealthChecksInState_DistanceSort(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() args := &structs.RegisterRequest{ @@ -180,7 +180,7 @@ func TestHealthChecksInState_DistanceSort(t *testing.T) { func TestHealthNodeChecks(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/node/nope?dc=dc1", nil) @@ -214,7 +214,7 @@ func TestHealthNodeChecks(t *testing.T) { func TestHealthServiceChecks(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/checks/consul?dc=dc1", nil) @@ -265,7 +265,7 @@ func TestHealthServiceChecks(t *testing.T) { func TestHealthServiceChecks_NodeMetaFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/checks/consul?dc=dc1&node-meta=somekey:somevalue", nil) @@ -317,7 +317,7 @@ func TestHealthServiceChecks_NodeMetaFilter(t *testing.T) { func TestHealthServiceChecks_DistanceSort(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create a service check @@ -396,7 +396,7 @@ func TestHealthServiceChecks_DistanceSort(t *testing.T) { func TestHealthServiceNodes(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/service/consul?dc=dc1", nil) @@ -462,7 +462,7 @@ func TestHealthServiceNodes(t *testing.T) { func TestHealthServiceNodes_NodeMetaFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/health/service/consul?dc=dc1&node-meta=somekey:somevalue", nil) @@ -514,7 +514,7 @@ func TestHealthServiceNodes_NodeMetaFilter(t *testing.T) { func TestHealthServiceNodes_DistanceSort(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create a service check @@ -593,7 +593,7 @@ func TestHealthServiceNodes_DistanceSort(t *testing.T) { func TestHealthServiceNodes_PassingFilter(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create a failing service check @@ -687,22 +687,22 @@ func TestHealthServiceNodes_PassingFilter(t *testing.T) { func TestHealthServiceNodes_WanTranslation(t *testing.T) { t.Parallel() - cfg1 := TestConfig() - cfg1.Datacenter = "dc1" - cfg1.TranslateWanAddrs = true - cfg1.ACLDatacenter = "" - a1 := NewTestAgent(t.Name(), cfg1) + a1 := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a1.Shutdown() - cfg2 := TestConfig() - cfg2.Datacenter = "dc2" - cfg2.TranslateWanAddrs = true - cfg2.ACLDatacenter = "" - a2 := NewTestAgent(t.Name(), cfg2) + a2 := NewTestAgent(t.Name(), ` + datacenter = "dc2" + translate_wan_addrs = true + acl_datacenter = "" + `) defer a2.Shutdown() // Wait for the WAN join. - addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.Ports.SerfWan) + addr := fmt.Sprintf("127.0.0.1:%d", a1.Config.SerfPortWAN) if _, err := a2.JoinWAN([]string{addr}); err != nil { t.Fatalf("err: %v", err) } diff --git a/agent/http.go b/agent/http.go index 10527ea86..229b81ee9 100644 --- a/agent/http.go +++ b/agent/http.go @@ -3,6 +3,7 @@ package agent import ( "encoding/json" "fmt" + "net" "net/http" "net/http/pprof" "net/url" @@ -25,14 +26,17 @@ type HTTPServer struct { // proto is filled by the agent to "http" or "https". proto string + addr net.Addr } -func NewHTTPServer(addr string, a *Agent) *HTTPServer { +func NewHTTPServer(addr net.Addr, a *Agent) *HTTPServer { s := &HTTPServer{ - Server: &http.Server{Addr: addr}, + Server: &http.Server{Addr: addr.String()}, agent: a, - blacklist: NewBlacklist(a.config.HTTPConfig.BlockEndpoints), + blacklist: NewBlacklist(a.config.HTTPBlockEndpoints), + addr: addr, } + s.Server.Handler = s.handler(a.config.EnableDebug) return s } @@ -200,8 +204,8 @@ var ( // wrap is used to wrap functions to make them more convenient func (s *HTTPServer) wrap(handler func(resp http.ResponseWriter, req *http.Request) (interface{}, error)) http.HandlerFunc { return func(resp http.ResponseWriter, req *http.Request) { - setHeaders(resp, s.agent.config.HTTPConfig.ResponseHeaders) - setTranslateAddr(resp, s.agent.config.TranslateWanAddrs) + setHeaders(resp, s.agent.config.HTTPResponseHeaders) + setTranslateAddr(resp, s.agent.config.TranslateWANAddrs) // Obfuscate any tokens from appearing in the logs formVals, err := url.ParseQuery(req.URL.RawQuery) diff --git a/agent/http_test.go b/agent/http_test.go index ac342d6b1..d607d4625 100644 --- a/agent/http_test.go +++ b/agent/http_test.go @@ -33,14 +33,16 @@ func TestHTTPServer_UnixSocket(t *testing.T) { defer os.RemoveAll(tempDir) socket := filepath.Join(tempDir, "test.sock") - cfg := TestConfig() - cfg.Addresses.HTTP = "unix://" + socket - // Only testing mode, since uid/gid might not be settable // from test environment. - cfg.UnixSockets = UnixSocketConfig{} - cfg.UnixSockets.Perms = "0777" - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + addresses { + http = "unix://`+socket+`" + } + unix_sockets { + mode = "0777" + } + `) defer a.Shutdown() // Ensure the socket was created @@ -58,10 +60,9 @@ func TestHTTPServer_UnixSocket(t *testing.T) { } // Ensure we can get a response from the socket. - path := socketPath(a.Config.Addresses.HTTP) trans := cleanhttp.DefaultTransport() trans.DialContext = func(_ context.Context, _, _ string) (net.Conn, error) { - return net.Dial("unix", path) + return net.Dial("unix", socket) } client := &http.Client{ Transport: trans, @@ -103,9 +104,11 @@ func TestHTTPServer_UnixSocket_FileExists(t *testing.T) { t.Fatalf("not a regular file: %s", socket) } - cfg := TestConfig() - cfg.Addresses.HTTP = "unix://" + socket - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + addresses { + http = "unix://`+socket+`" + } + `) defer a.Shutdown() // Ensure the file was replaced by the socket @@ -198,12 +201,11 @@ func TestSetMeta(t *testing.T) { func TestHTTPAPI_BlockEndpoints(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.HTTPConfig.BlockEndpoints = []string{ - "/v1/agent/self", - } - - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + http_config { + block_endpoints = ["/v1/agent/self"] + } + `) defer a.Shutdown() handler := func(resp http.ResponseWriter, req *http.Request) (interface{}, error) { @@ -235,7 +237,7 @@ func TestHTTPAPI_TranslateAddrHeader(t *testing.T) { t.Parallel() // Header should not be present if address translation is off. { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() resp := httptest.NewRecorder() @@ -254,9 +256,9 @@ func TestHTTPAPI_TranslateAddrHeader(t *testing.T) { // Header should be set to true if it's turned on. { - cfg := TestConfig() - cfg.TranslateWanAddrs = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + translate_wan_addrs = true + `) defer a.Shutdown() resp := httptest.NewRecorder() @@ -276,12 +278,14 @@ func TestHTTPAPI_TranslateAddrHeader(t *testing.T) { func TestHTTPAPIResponseHeaders(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.HTTPConfig.ResponseHeaders = map[string]string{ - "Access-Control-Allow-Origin": "*", - "X-XSS-Protection": "1; mode=block", - } - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + http_config { + response_headers = { + "Access-Control-Allow-Origin" = "*" + "X-XSS-Protection" = "1; mode=block" + } + } + `) defer a.Shutdown() resp := httptest.NewRecorder() @@ -305,7 +309,7 @@ func TestHTTPAPIResponseHeaders(t *testing.T) { func TestContentTypeIsJSON(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() resp := httptest.NewRecorder() @@ -389,7 +393,7 @@ func TestPrettyPrintBare(t *testing.T) { } func testPrettyPrint(pretty string, t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() r := &structs.DirEntry{Key: "key"} @@ -417,7 +421,7 @@ func testPrettyPrint(pretty string, t *testing.T) { func TestParseSource(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Default is agent's DC and no node (since the user didn't care, then @@ -565,7 +569,7 @@ func TestACLResolution(t *testing.T) { reqBothTokens, _ := http.NewRequest("GET", "/v1/catalog/nodes?token=baz", nil) reqBothTokens.Header.Add("X-Consul-Token", "zap") - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Check when no token is set @@ -603,9 +607,9 @@ func TestACLResolution(t *testing.T) { func TestEnableWebUI(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.EnableUI = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + enable_ui = true + `) defer a.Shutdown() req, _ := http.NewRequest("GET", "/ui/", nil) diff --git a/agent/keyring_test.go b/agent/keyring_test.go index aae7cf12d..c74b7ef25 100644 --- a/agent/keyring_test.go +++ b/agent/keyring_test.go @@ -33,10 +33,10 @@ func TestAgent_LoadKeyrings(t *testing.T) { // Should be no configured keyring file by default t.Run("no keys", func(t *testing.T) { - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - c1 := a1.Config.ConsulConfig + c1 := a1.consulConfig() if c1.SerfLANConfig.KeyringFile != "" { t.Fatalf("bad: %#v", c1.SerfLANConfig.KeyringFile) } @@ -57,7 +57,7 @@ func TestAgent_LoadKeyrings(t *testing.T) { a2.Start() defer a2.Shutdown() - c2 := a2.Config.ConsulConfig + c2 := a2.consulConfig() if c2.SerfLANConfig.KeyringFile == "" { t.Fatalf("should have keyring file") } @@ -80,13 +80,14 @@ func TestAgent_LoadKeyrings(t *testing.T) { // Client should auto-load only the LAN keyring file t.Run("client with keys", func(t *testing.T) { - cfg3 := TestConfig() - cfg3.Server = false - a3 := &TestAgent{Name: t.Name(), Config: cfg3, Key: key} + a3 := &TestAgent{Name: t.Name(), HCL: ` + server = false + bootstrap = false + `, Key: key} a3.Start() defer a3.Shutdown() - c3 := a3.Config.ConsulConfig + c3 := a3.consulConfig() if c3.SerfLANConfig.KeyringFile == "" { t.Fatalf("should have keyring file") } @@ -111,10 +112,10 @@ func TestAgent_InmemKeyrings(t *testing.T) { // Should be no configured keyring file by default t.Run("no keys", func(t *testing.T) { - a1 := NewTestAgent(t.Name(), nil) + a1 := NewTestAgent(t.Name(), "") defer a1.Shutdown() - c1 := a1.Config.ConsulConfig + c1 := a1.consulConfig() if c1.SerfLANConfig.KeyringFile != "" { t.Fatalf("bad: %#v", c1.SerfLANConfig.KeyringFile) } @@ -131,15 +132,14 @@ func TestAgent_InmemKeyrings(t *testing.T) { // Server should auto-load LAN and WAN keyring t.Run("server with keys", func(t *testing.T) { - cfg2 := TestConfig() - cfg2.EncryptKey = key - cfg2.DisableKeyringFile = true - - a2 := &TestAgent{Name: t.Name(), Config: cfg2} + a2 := &TestAgent{Name: t.Name(), HCL: ` + encrypt = "` + key + `" + disable_keyring_file = true + `} a2.Start() defer a2.Shutdown() - c2 := a2.Config.ConsulConfig + c2 := a2.consulConfig() if c2.SerfLANConfig.KeyringFile != "" { t.Fatalf("should not have keyring file") } @@ -162,15 +162,16 @@ func TestAgent_InmemKeyrings(t *testing.T) { // Client should auto-load only the LAN keyring t.Run("client with keys", func(t *testing.T) { - cfg3 := TestConfig() - cfg3.EncryptKey = key - cfg3.DisableKeyringFile = true - cfg3.Server = false - a3 := &TestAgent{Name: t.Name(), Config: cfg3} + a3 := &TestAgent{Name: t.Name(), HCL: ` + encrypt = "` + key + `" + server = false + bootstrap = false + disable_keyring_file = true + `} a3.Start() defer a3.Shutdown() - c3 := a3.Config.ConsulConfig + c3 := a3.consulConfig() if c3.SerfLANConfig.KeyringFile != "" { t.Fatalf("should not have keyring file") } @@ -201,16 +202,15 @@ func TestAgent_InmemKeyrings(t *testing.T) { t.Fatalf("err: %v", err) } - cfg4 := TestConfig() - cfg4.EncryptKey = key - cfg4.DisableKeyringFile = true - cfg4.DataDir = dir - - a4 := &TestAgent{Name: t.Name(), Config: cfg4} + a4 := &TestAgent{Name: t.Name(), HCL: ` + encrypt = "` + key + `" + disable_keyring_file = true + data_dir = "` + dir + `" + `} a4.Start() defer a4.Shutdown() - c4 := a4.Config.ConsulConfig + c4 := a4.consulConfig() if c4.SerfLANConfig.KeyringFile != "" { t.Fatalf("should not have keyring file") } @@ -276,11 +276,11 @@ func TestAgentKeyring_ACL(t *testing.T) { key1 := "tbLJg26ZJyJ9pK3qhc9jig==" key2 := "4leC33rgtXKIVUr9Nr0snQ==" - cfg := TestACLConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLDefaultPolicy = "deny" - a := &TestAgent{Name: t.Name(), Config: cfg, Key: key1} + a := &TestAgent{Name: t.Name(), HCL: TestACLConfig() + ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_default_policy = "deny" + `, Key: key1} a.Start() defer a.Shutdown() diff --git a/agent/kvs_endpoint_test.go b/agent/kvs_endpoint_test.go index 6977679a0..023dff9eb 100644 --- a/agent/kvs_endpoint_test.go +++ b/agent/kvs_endpoint_test.go @@ -13,7 +13,7 @@ import ( func TestKVSEndpoint_PUT_GET_DELETE(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() keys := []string{ @@ -72,7 +72,7 @@ func TestKVSEndpoint_PUT_GET_DELETE(t *testing.T) { func TestKVSEndpoint_Recurse(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() keys := []string{ @@ -148,7 +148,7 @@ func TestKVSEndpoint_Recurse(t *testing.T) { func TestKVSEndpoint_DELETE_CAS(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() { @@ -214,7 +214,7 @@ func TestKVSEndpoint_DELETE_CAS(t *testing.T) { func TestKVSEndpoint_CAS(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() { @@ -290,7 +290,7 @@ func TestKVSEndpoint_CAS(t *testing.T) { func TestKVSEndpoint_ListKeys(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() keys := []string{ @@ -339,7 +339,7 @@ func TestKVSEndpoint_ListKeys(t *testing.T) { func TestKVSEndpoint_AcquireRelease(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Acquire the lock @@ -396,7 +396,7 @@ func TestKVSEndpoint_AcquireRelease(t *testing.T) { func TestKVSEndpoint_GET_Raw(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() buf := bytes.NewBuffer([]byte("test")) @@ -426,7 +426,7 @@ func TestKVSEndpoint_GET_Raw(t *testing.T) { func TestKVSEndpoint_PUT_ConflictingFlags(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("PUT", "/v1/kv/test?cas=0&acquire=xxx", nil) @@ -445,7 +445,7 @@ func TestKVSEndpoint_PUT_ConflictingFlags(t *testing.T) { func TestKVSEndpoint_DELETE_ConflictingFlags(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("DELETE", "/v1/kv/test?recurse&cas=0", nil) diff --git a/agent/local.go b/agent/local.go index c46aaed03..fdc86142d 100644 --- a/agent/local.go +++ b/agent/local.go @@ -10,6 +10,7 @@ import ( "time" "github.com/hashicorp/consul/acl" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/agent/token" "github.com/hashicorp/consul/api" @@ -90,10 +91,10 @@ type localState struct { } // NewLocalState creates a is used to initialize the local state -func NewLocalState(c *Config, lg *log.Logger, tokens *token.Store) *localState { +func NewLocalState(c *config.RuntimeConfig, lg *log.Logger, tokens *token.Store) *localState { lc := localStateConfig{ AEInterval: c.AEInterval, - AdvertiseAddr: c.AdvertiseAddr, + AdvertiseAddr: c.AdvertiseAddrLAN.String(), CheckUpdateInterval: c.CheckUpdateInterval, Datacenter: c.Datacenter, NodeID: c.NodeID, diff --git a/agent/local_test.go b/agent/local_test.go index 993224395..2b2d78d3d 100644 --- a/agent/local_test.go +++ b/agent/local_test.go @@ -5,11 +5,13 @@ import ( "testing" "time" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/agent/token" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/testutil/retry" "github.com/hashicorp/consul/types" + "github.com/pascaldekloe/goe/verify" ) func TestAgentAntiEntropy_Services(t *testing.T) { @@ -128,11 +130,9 @@ func TestAgentAntiEntropy_Services(t *testing.T) { addrs := services.NodeServices.Node.TaggedAddresses meta := services.NodeServices.Node.Meta delete(meta, structs.MetaSegmentKey) // Added later, not in config. - if id != a.Config.NodeID || - !reflect.DeepEqual(addrs, a.Config.TaggedAddresses) || - !reflect.DeepEqual(meta, a.Config.Meta) { - r.Fatalf("bad: %v", services.NodeServices.Node) - } + verify.Values(r, "node id", id, a.config.NodeID) + verify.Values(r, "tagged addrs", addrs, a.config.TaggedAddresses) + verify.Values(r, "node meta", meta, a.config.NodeMeta) // We should have 6 services (consul included) if len(services.NodeServices.Services) != 6 { @@ -356,7 +356,7 @@ func TestAgentAntiEntropy_EnableTagOverride(t *testing.T) { func TestAgentAntiEntropy_Services_WithChecks(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() { @@ -484,27 +484,27 @@ func TestAgentAntiEntropy_Services_WithChecks(t *testing.T) { } var testRegisterRules = ` -node "" { - policy = "write" -} + node "" { + policy = "write" + } -service "api" { - policy = "write" -} + service "api" { + policy = "write" + } -service "consul" { - policy = "write" -} -` + service "consul" { + policy = "write" + } + ` func TestAgentAntiEntropy_Services_ACLDeny(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLDefaultPolicy = "deny" - cfg.ACLEnforceVersion8 = Bool(true) - a := &TestAgent{Name: t.Name(), Config: cfg, NoInitialSync: true} + a := &TestAgent{Name: t.Name(), HCL: ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_default_policy = "deny" + acl_enforce_version_8 = true + `, NoInitialSync: true} a.Start() defer a.Shutdown() @@ -830,11 +830,9 @@ func TestAgentAntiEntropy_Checks(t *testing.T) { addrs := services.NodeServices.Node.TaggedAddresses meta := services.NodeServices.Node.Meta delete(meta, structs.MetaSegmentKey) // Added later, not in config. - if id != a.Config.NodeID || - !reflect.DeepEqual(addrs, a.Config.TaggedAddresses) || - !reflect.DeepEqual(meta, a.Config.Meta) { - t.Fatalf("bad: %v", services.NodeServices.Node) - } + verify.Values(t, "node id", id, a.config.NodeID) + verify.Values(t, "tagged addrs", addrs, a.config.TaggedAddresses) + verify.Values(t, "node meta", meta, a.config.NodeMeta) } // Remove one of the checks @@ -900,12 +898,12 @@ func TestAgentAntiEntropy_Checks(t *testing.T) { func TestAgentAntiEntropy_Checks_ACLDeny(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLDefaultPolicy = "deny" - cfg.ACLEnforceVersion8 = Bool(true) - a := &TestAgent{Name: t.Name(), Config: cfg, NoInitialSync: true} + a := &TestAgent{Name: t.Name(), HCL: ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_default_policy = "deny" + acl_enforce_version_8 = true + `, NoInitialSync: true} a.Start() defer a.Shutdown() @@ -1155,9 +1153,9 @@ func TestAgentAntiEntropy_Checks_ACLDeny(t *testing.T) { func TestAgentAntiEntropy_Check_DeferSync(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.CheckUpdateInterval = 500 * time.Millisecond - a := &TestAgent{Name: t.Name(), Config: cfg, NoInitialSync: true} + a := &TestAgent{Name: t.Name(), HCL: ` + check_update_interval = "500ms" + `, NoInitialSync: true} a.Start() defer a.Shutdown() @@ -1230,7 +1228,7 @@ func TestAgentAntiEntropy_Check_DeferSync(t *testing.T) { reg := structs.RegisterRequest{ Datacenter: a.Config.Datacenter, Node: a.Config.NodeName, - Address: a.Config.AdvertiseAddr, + Address: a.Config.AdvertiseAddrLAN.IP.String(), TaggedAddresses: a.Config.TaggedAddresses, Check: eCopy, WriteRequest: structs.WriteRequest{}, @@ -1329,10 +1327,16 @@ func TestAgentAntiEntropy_Check_DeferSync(t *testing.T) { func TestAgentAntiEntropy_NodeInfo(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.NodeID = types.NodeID("40e4a748-2192-161a-0510-9bf59fe950b5") - cfg.Meta["somekey"] = "somevalue" - a := &TestAgent{Name: t.Name(), Config: cfg, NoInitialSync: true} + nodeID := types.NodeID("40e4a748-2192-161a-0510-9bf59fe950b5") + nodeMeta := map[string]string{ + "somekey": "somevalue", + } + a := &TestAgent{Name: t.Name(), HCL: ` + node_id = "40e4a748-2192-161a-0510-9bf59fe950b5" + node_meta { + somekey = "somevalue" + } + `, NoInitialSync: true} a.Start() defer a.Shutdown() @@ -1367,9 +1371,9 @@ func TestAgentAntiEntropy_NodeInfo(t *testing.T) { addrs := services.NodeServices.Node.TaggedAddresses meta := services.NodeServices.Node.Meta delete(meta, structs.MetaSegmentKey) // Added later, not in config. - if id != cfg.NodeID || - !reflect.DeepEqual(addrs, cfg.TaggedAddresses) || - !reflect.DeepEqual(meta, cfg.Meta) { + if id != nodeID || + !reflect.DeepEqual(addrs, a.config.TaggedAddresses) || + !reflect.DeepEqual(meta, nodeMeta) { r.Fatalf("bad: %v", services.NodeServices.Node) } }) @@ -1391,9 +1395,9 @@ func TestAgentAntiEntropy_NodeInfo(t *testing.T) { addrs := services.NodeServices.Node.TaggedAddresses meta := services.NodeServices.Node.Meta delete(meta, structs.MetaSegmentKey) // Added later, not in config. - if id != cfg.NodeID || - !reflect.DeepEqual(addrs, cfg.TaggedAddresses) || - !reflect.DeepEqual(meta, cfg.Meta) { + if id != nodeID || + !reflect.DeepEqual(addrs, a.config.TaggedAddresses) || + !reflect.DeepEqual(meta, nodeMeta) { r.Fatalf("bad: %v", services.NodeServices.Node) } }) @@ -1428,7 +1432,7 @@ func TestAgent_serviceTokens(t *testing.T) { tokens := new(token.Store) tokens.UpdateUserToken("default") - l := NewLocalState(TestConfig(), nil, tokens) + l := NewLocalState(config.DefaultRuntimeConfig(`bind_addr = "127.0.0.1" data_dir = "dummy"`), nil, tokens) l.AddService(&structs.NodeService{ ID: "redis", @@ -1457,7 +1461,7 @@ func TestAgent_checkTokens(t *testing.T) { tokens := new(token.Store) tokens.UpdateUserToken("default") - l := NewLocalState(TestConfig(), nil, tokens) + l := NewLocalState(config.DefaultRuntimeConfig(`bind_addr = "127.0.0.1" data_dir = "dummy"`), nil, tokens) // Returns default when no token is set if token := l.CheckToken("mem"); token != "default" { @@ -1479,8 +1483,7 @@ func TestAgent_checkTokens(t *testing.T) { func TestAgent_checkCriticalTime(t *testing.T) { t.Parallel() - cfg := TestConfig() - l := NewLocalState(cfg, nil, new(token.Store)) + l := NewLocalState(config.DefaultRuntimeConfig(`bind_addr = "127.0.0.1" data_dir = "dummy"`), nil, new(token.Store)) svc := &structs.NodeService{ID: "redis", Service: "redis", Port: 8000} l.AddService(svc, "") @@ -1542,8 +1545,7 @@ func TestAgent_checkCriticalTime(t *testing.T) { func TestAgent_AddCheckFailure(t *testing.T) { t.Parallel() - cfg := TestConfig() - l := NewLocalState(cfg, nil, new(token.Store)) + l := NewLocalState(config.DefaultRuntimeConfig(`bind_addr = "127.0.0.1" data_dir = "dummy"`), nil, new(token.Store)) // Add a check for a service that does not exist and verify that it fails checkID := types.CheckID("redis:1") @@ -1595,15 +1597,22 @@ func TestAgent_nestedPauseResume(t *testing.T) { func TestAgent_sendCoordinate(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.SyncCoordinateRateTarget = 10.0 // updates/sec - cfg.SyncCoordinateIntervalMin = 1 * time.Millisecond - cfg.ConsulConfig.CoordinateUpdatePeriod = 100 * time.Millisecond - cfg.ConsulConfig.CoordinateUpdateBatchSize = 10 - cfg.ConsulConfig.CoordinateUpdateMaxBatches = 1 - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + sync_coordinate_interval_min = "1ms" + sync_coordinate_rate_target = 10.0 + consul = { + coordinate = { + update_period = "100ms" + update_batch_size = 10 + update_max_batches = 1 + } + } + `) defer a.Shutdown() + t.Logf("%d %d %s", a.consulConfig().CoordinateUpdateBatchSize, a.consulConfig().CoordinateUpdateMaxBatches, + a.consulConfig().CoordinateUpdatePeriod.String()) + // Make sure the coordinate is present. req := structs.DCSpecificRequest{ Datacenter: a.Config.Datacenter, diff --git a/agent/operator_endpoint_test.go b/agent/operator_endpoint_test.go index c0da418d7..e069ae48e 100644 --- a/agent/operator_endpoint_test.go +++ b/agent/operator_endpoint_test.go @@ -7,7 +7,6 @@ import ( "net/http/httptest" "strings" "testing" - "time" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" @@ -16,7 +15,7 @@ import ( func TestOperator_RaftConfiguration(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -43,7 +42,7 @@ func TestOperator_RaftConfiguration(t *testing.T) { func TestOperator_RaftPeer(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -59,7 +58,7 @@ func TestOperator_RaftPeer(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -79,9 +78,9 @@ func TestOperator_KeyringInstall(t *testing.T) { t.Parallel() oldKey := "H3/9gBxcKKRf45CaI2DlRg==" newKey := "z90lFx3sZZLtTOkutXcwYg==" - cfg := TestConfig() - cfg.EncryptKey = oldKey - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + encrypt = "`+oldKey+`" + `) defer a.Shutdown() body := bytes.NewBufferString(fmt.Sprintf("{\"Key\":\"%s\"}", newKey)) @@ -114,9 +113,9 @@ func TestOperator_KeyringInstall(t *testing.T) { func TestOperator_KeyringList(t *testing.T) { t.Parallel() key := "H3/9gBxcKKRf45CaI2DlRg==" - cfg := TestConfig() - cfg.EncryptKey = key - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + encrypt = "`+key+`" + `) defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/operator/keyring", nil) @@ -163,9 +162,9 @@ func TestOperator_KeyringRemove(t *testing.T) { t.Parallel() key := "H3/9gBxcKKRf45CaI2DlRg==" tempKey := "z90lFx3sZZLtTOkutXcwYg==" - cfg := TestConfig() - cfg.EncryptKey = key - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + encrypt = "`+key+`" + `) defer a.Shutdown() _, err := a.InstallKey(tempKey, "", 0) @@ -221,9 +220,9 @@ func TestOperator_KeyringUse(t *testing.T) { t.Parallel() oldKey := "H3/9gBxcKKRf45CaI2DlRg==" newKey := "z90lFx3sZZLtTOkutXcwYg==" - cfg := TestConfig() - cfg.EncryptKey = oldKey - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + encrypt = "`+oldKey+`" + `) defer a.Shutdown() if _, err := a.InstallKey(newKey, "", 0); err != nil { @@ -264,9 +263,9 @@ func TestOperator_KeyringUse(t *testing.T) { func TestOperator_Keyring_InvalidRelayFactor(t *testing.T) { t.Parallel() key := "H3/9gBxcKKRf45CaI2DlRg==" - cfg := TestConfig() - cfg.EncryptKey = key - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + encrypt = "`+key+`" + `) defer a.Shutdown() cases := map[string]string{ @@ -289,7 +288,7 @@ func TestOperator_Keyring_InvalidRelayFactor(t *testing.T) { func TestOperator_AutopilotGetConfiguration(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -313,7 +312,7 @@ func TestOperator_AutopilotGetConfiguration(t *testing.T) { func TestOperator_AutopilotSetConfiguration(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer([]byte(`{"CleanupDeadServers": false}`)) @@ -341,7 +340,7 @@ func TestOperator_AutopilotSetConfiguration(t *testing.T) { func TestOperator_AutopilotCASConfiguration(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer([]byte(`{"CleanupDeadServers": false}`)) @@ -408,9 +407,9 @@ func TestOperator_AutopilotCASConfiguration(t *testing.T) { func TestOperator_ServerHealth(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.RaftProtocol = 3 - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + raft_protocol = 3 + `) defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -440,11 +439,12 @@ func TestOperator_ServerHealth(t *testing.T) { func TestOperator_ServerHealth_Unhealthy(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.RaftProtocol = 3 - threshold := time.Duration(-1) - cfg.Autopilot.LastContactThreshold = &threshold - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + raft_protocol = 3 + autopilot { + last_contact_threshold = "-1s" + } + `) defer a.Shutdown() body := bytes.NewBuffer(nil) diff --git a/agent/prepared_query_endpoint.go b/agent/prepared_query_endpoint.go index 70c7bb107..1f12729f9 100644 --- a/agent/prepared_query_endpoint.go +++ b/agent/prepared_query_endpoint.go @@ -96,7 +96,7 @@ func (s *HTTPServer) preparedQueryExecute(id string, resp http.ResponseWriter, r Agent: structs.QuerySource{ Node: s.agent.config.NodeName, Datacenter: s.agent.config.Datacenter, - Segment: s.agent.config.Segment, + Segment: s.agent.config.SegmentName, }, } s.parseSource(req, &args.Source) @@ -141,7 +141,7 @@ func (s *HTTPServer) preparedQueryExplain(id string, resp http.ResponseWriter, r Agent: structs.QuerySource{ Node: s.agent.config.NodeName, Datacenter: s.agent.config.Datacenter, - Segment: s.agent.config.Segment, + Segment: s.agent.config.SegmentName, }, } s.parseSource(req, &args.Source) diff --git a/agent/prepared_query_endpoint_test.go b/agent/prepared_query_endpoint_test.go index 3f41878ee..5443195b1 100644 --- a/agent/prepared_query_endpoint_test.go +++ b/agent/prepared_query_endpoint_test.go @@ -70,7 +70,7 @@ func (m *MockPreparedQuery) Explain(args *structs.PreparedQueryExecuteRequest, func TestPreparedQuery_Create(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -155,7 +155,7 @@ func TestPreparedQuery_Create(t *testing.T) { func TestPreparedQuery_List(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -188,7 +188,7 @@ func TestPreparedQuery_List(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -238,7 +238,7 @@ func TestPreparedQuery_List(t *testing.T) { func TestPreparedQuery_Execute(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -271,7 +271,7 @@ func TestPreparedQuery_Execute(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -327,7 +327,7 @@ func TestPreparedQuery_Execute(t *testing.T) { // Ensure the proper params are set when no special args are passed t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -358,10 +358,10 @@ func TestPreparedQuery_Execute(t *testing.T) { // Ensure WAN translation occurs for a response outside of the local DC. t.Run("", func(t *testing.T) { - cfg := TestConfig() - cfg.Datacenter = "dc1" - cfg.TranslateWanAddrs = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + `) defer a.Shutdown() m := MockPreparedQuery{ @@ -408,10 +408,10 @@ func TestPreparedQuery_Execute(t *testing.T) { // Ensure WAN translation doesn't occur for the local DC. t.Run("", func(t *testing.T) { - cfg := TestConfig() - cfg.Datacenter = "dc1" - cfg.TranslateWanAddrs = true - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + datacenter = "dc1" + translate_wan_addrs = true + `) defer a.Shutdown() m := MockPreparedQuery{ @@ -457,7 +457,7 @@ func TestPreparedQuery_Execute(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -475,7 +475,7 @@ func TestPreparedQuery_Execute(t *testing.T) { func TestPreparedQuery_Explain(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -530,7 +530,7 @@ func TestPreparedQuery_Explain(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -548,7 +548,7 @@ func TestPreparedQuery_Explain(t *testing.T) { func TestPreparedQuery_Get(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -596,7 +596,7 @@ func TestPreparedQuery_Get(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -613,7 +613,7 @@ func TestPreparedQuery_Get(t *testing.T) { func TestPreparedQuery_Update(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -691,7 +691,7 @@ func TestPreparedQuery_Update(t *testing.T) { func TestPreparedQuery_Delete(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() m := MockPreparedQuery{ @@ -740,7 +740,7 @@ func TestPreparedQuery_Delete(t *testing.T) { func TestPreparedQuery_BadMethods(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -755,7 +755,7 @@ func TestPreparedQuery_BadMethods(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -801,7 +801,7 @@ func TestPreparedQuery_parseLimit(t *testing.T) { // correctly when calling through to the real endpoints. func TestPreparedQuery_Integration(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Register a node and a service. diff --git a/agent/remote_exec_test.go b/agent/remote_exec_test.go index e36bb8dc9..fd586fa41 100644 --- a/agent/remote_exec_test.go +++ b/agent/remote_exec_test.go @@ -6,6 +6,7 @@ import ( "fmt" "reflect" "testing" + "time" "github.com/hashicorp/consul/agent/structs" @@ -95,40 +96,40 @@ func TestRexecWriter(t *testing.T) { func TestRemoteExecGetSpec(t *testing.T) { t.Parallel() - testRemoteExecGetSpec(t, nil, "", true) + testRemoteExecGetSpec(t, "", "", true) } func TestRemoteExecGetSpec_ACLToken(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLToken = "root" - cfg.ACLDefaultPolicy = "deny" - testRemoteExecGetSpec(t, cfg, "root", true) + testRemoteExecGetSpec(t, ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_token = "root" + acl_default_policy = "deny" + `, "root", true) } func TestRemoteExecGetSpec_ACLAgentToken(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLAgentToken = "root" - cfg.ACLDefaultPolicy = "deny" - testRemoteExecGetSpec(t, cfg, "root", true) + testRemoteExecGetSpec(t, ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_agent_token = "root" + acl_default_policy = "deny" + `, "root", true) } func TestRemoteExecGetSpec_ACLDeny(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLDefaultPolicy = "deny" - testRemoteExecGetSpec(t, cfg, "root", false) + testRemoteExecGetSpec(t, ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_default_policy = "deny" + `, "root", false) } -func testRemoteExecGetSpec(t *testing.T, c *Config, token string, shouldSucceed bool) { - a := NewTestAgent(t.Name(), c) +func testRemoteExecGetSpec(t *testing.T, hcl string, token string, shouldSucceed bool) { + a := NewTestAgent(t.Name(), hcl) defer a.Shutdown() event := &remoteExecEvent{ @@ -160,40 +161,40 @@ func testRemoteExecGetSpec(t *testing.T, c *Config, token string, shouldSucceed func TestRemoteExecWrites(t *testing.T) { t.Parallel() - testRemoteExecWrites(t, nil, "", true) + testRemoteExecWrites(t, "", "", true) } func TestRemoteExecWrites_ACLToken(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLToken = "root" - cfg.ACLDefaultPolicy = "deny" - testRemoteExecWrites(t, cfg, "root", true) + testRemoteExecWrites(t, ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_token = "root" + acl_default_policy = "deny" + `, "root", true) } func TestRemoteExecWrites_ACLAgentToken(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLAgentToken = "root" - cfg.ACLDefaultPolicy = "deny" - testRemoteExecWrites(t, cfg, "root", true) + testRemoteExecWrites(t, ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_agent_token = "root" + acl_default_policy = "deny" + `, "root", true) } func TestRemoteExecWrites_ACLDeny(t *testing.T) { t.Parallel() - cfg := TestConfig() - cfg.ACLDatacenter = "dc1" - cfg.ACLMasterToken = "root" - cfg.ACLDefaultPolicy = "deny" - testRemoteExecWrites(t, cfg, "root", false) + testRemoteExecWrites(t, ` + acl_datacenter = "dc1" + acl_master_token = "root" + acl_default_policy = "deny" + `, "root", false) } -func testRemoteExecWrites(t *testing.T, c *Config, token string, shouldSucceed bool) { - a := NewTestAgent(t.Name(), c) +func testRemoteExecWrites(t *testing.T, hcl string, token string, shouldSucceed bool) { + a := NewTestAgent(t.Name(), hcl) defer a.Shutdown() event := &remoteExecEvent{ @@ -250,7 +251,7 @@ func testRemoteExecWrites(t *testing.T, c *Config, token string, shouldSucceed b } func testHandleRemoteExec(t *testing.T, command string, expectedSubstring string, expectedReturnCode string) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() event := &remoteExecEvent{ diff --git a/agent/retry_join.go b/agent/retry_join.go index 272e7445f..8a5c418a3 100644 --- a/agent/retry_join.go +++ b/agent/retry_join.go @@ -12,9 +12,9 @@ import ( func (a *Agent) retryJoinLAN() { r := &retryJoiner{ cluster: "LAN", - addrs: a.config.RetryJoin, - maxAttempts: a.config.RetryMaxAttempts, - interval: a.config.RetryInterval, + addrs: a.config.RetryJoinLAN, + maxAttempts: a.config.RetryJoinMaxAttemptsLAN, + interval: a.config.RetryJoinIntervalLAN, join: a.JoinLAN, logger: a.logger, } @@ -26,9 +26,9 @@ func (a *Agent) retryJoinLAN() { func (a *Agent) retryJoinWAN() { r := &retryJoiner{ cluster: "WAN", - addrs: a.config.RetryJoinWan, - maxAttempts: a.config.RetryMaxAttemptsWan, - interval: a.config.RetryIntervalWan, + addrs: a.config.RetryJoinWAN, + maxAttempts: a.config.RetryJoinMaxAttemptsWAN, + interval: a.config.RetryJoinIntervalWAN, join: a.JoinWAN, logger: a.logger, } diff --git a/agent/segment_stub.go b/agent/segment_stub.go index 0771d7394..881acc5cd 100644 --- a/agent/segment_stub.go +++ b/agent/segment_stub.go @@ -2,18 +2,14 @@ package agent -import ( - "github.com/hashicorp/consul/agent/structs" -) - -func ValidateSegments(conf *Config) error { - if conf.Segment != "" { - return structs.ErrSegmentsNotSupported - } - - if len(conf.Segments) > 0 { - return structs.ErrSegmentsNotSupported - } - - return nil -} +// done(fs): func ValidateSegments(conf *Config) error { +// done(fs): if conf.Segment != "" { +// done(fs): return structs.ErrSegmentsNotSupported +// done(fs): } +// done(fs): +// done(fs): if len(conf.Segments) > 0 { +// done(fs): return structs.ErrSegmentsNotSupported +// done(fs): } +// done(fs): +// done(fs): return nil +// done(fs): } diff --git a/agent/session_endpoint_test.go b/agent/session_endpoint_test.go index 1088a2c92..b8cbe2354 100644 --- a/agent/session_endpoint_test.go +++ b/agent/session_endpoint_test.go @@ -15,7 +15,7 @@ import ( func TestSessionCreate(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create a health check @@ -61,7 +61,7 @@ func TestSessionCreate(t *testing.T) { func TestSessionCreateDelete(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Create a health check @@ -190,7 +190,7 @@ func makeTestSessionTTL(t *testing.T, srv *HTTPServer, ttl string) string { func TestSessionDestroy(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() id := makeTestSession(t, a.srv) @@ -209,10 +209,9 @@ func TestSessionDestroy(t *testing.T) { func TestSessionCustomTTL(t *testing.T) { t.Parallel() ttl := 250 * time.Millisecond - cfg := TestConfig() - cfg.SessionTTLMin = ttl - cfg.SessionTTLMinRaw = ttl.String() - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + session_ttl_min = "250ms" + `) defer a.Shutdown() id := makeTestSessionTTL(t, a.srv, ttl.String()) @@ -251,10 +250,9 @@ func TestSessionCustomTTL(t *testing.T) { func TestSessionTTLRenew(t *testing.T) { // t.Parallel() // timing test. no parallel ttl := 250 * time.Millisecond - cfg := TestConfig() - cfg.SessionTTLMin = ttl - cfg.SessionTTLMinRaw = ttl.String() - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + session_ttl_min = "250ms" + `) defer a.Shutdown() id := makeTestSessionTTL(t, a.srv, ttl.String()) @@ -331,7 +329,7 @@ func TestSessionTTLRenew(t *testing.T) { func TestSessionGet(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/session/info/adf4238a-882b-9ddc-4a9d-5b6758e4159e", nil) @@ -350,7 +348,7 @@ func TestSessionGet(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() id := makeTestSession(t, a.srv) @@ -374,7 +372,7 @@ func TestSessionGet(t *testing.T) { func TestSessionList(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/session/list", nil) @@ -393,7 +391,7 @@ func TestSessionList(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() var ids []string @@ -420,7 +418,7 @@ func TestSessionList(t *testing.T) { func TestSessionsForNode(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", "/v1/session/node/"+a.Config.NodeName, nil) @@ -439,7 +437,7 @@ func TestSessionsForNode(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() var ids []string @@ -465,7 +463,7 @@ func TestSessionsForNode(t *testing.T) { func TestSessionDeleteDestroy(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() id := makeTestSessionDelete(t, a.srv) diff --git a/agent/snapshot_endpoint_test.go b/agent/snapshot_endpoint_test.go index ec971c3cd..81df2cfd0 100644 --- a/agent/snapshot_endpoint_test.go +++ b/agent/snapshot_endpoint_test.go @@ -15,7 +15,7 @@ func TestSnapshot(t *testing.T) { t.Parallel() var snap io.Reader t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -41,7 +41,7 @@ func TestSnapshot(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("PUT", "/v1/snapshot?token=root", snap) @@ -105,7 +105,7 @@ func TestSnapshot_Options(t *testing.T) { func TestSnapshot_BadMethods(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) @@ -121,7 +121,7 @@ func TestSnapshot_BadMethods(t *testing.T) { }) t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() body := bytes.NewBuffer(nil) diff --git a/agent/status_endpoint_test.go b/agent/status_endpoint_test.go index f8314192a..ffe2f1be1 100644 --- a/agent/status_endpoint_test.go +++ b/agent/status_endpoint_test.go @@ -1,12 +1,10 @@ package agent -import ( - "testing" -) +import "testing" func TestStatusLeader(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() obj, err := a.srv.StatusLeader(nil, nil) @@ -21,7 +19,7 @@ func TestStatusLeader(t *testing.T) { func TestStatusPeers(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() obj, err := a.srv.StatusPeers(nil, nil) diff --git a/agent/structs/check_definition_test.go b/agent/structs/check_definition_test.go index b415f5124..c7fdae81c 100644 --- a/agent/structs/check_definition_test.go +++ b/agent/structs/check_definition_test.go @@ -3,10 +3,12 @@ package structs import ( "reflect" "testing" + "time" "github.com/google/gofuzz" "github.com/hashicorp/consul/api" "github.com/mitchellh/reflectwalk" + "github.com/pascaldekloe/goe/verify" ) func TestCheckDefinition_Defaults(t *testing.T) { @@ -70,3 +72,44 @@ func TestCheckDefinition_CheckType(t *testing.T) { } } } + +func TestCheckDefinitionToCheckType(t *testing.T) { + t.Parallel() + got := &CheckDefinition{ + ID: "id", + Name: "name", + Status: "green", + Notes: "notes", + + ServiceID: "svcid", + Token: "tok", + Script: "/bin/foo", + HTTP: "someurl", + TCP: "host:port", + Interval: 1 * time.Second, + DockerContainerID: "abc123", + Shell: "/bin/ksh", + TLSSkipVerify: true, + Timeout: 2 * time.Second, + TTL: 3 * time.Second, + DeregisterCriticalServiceAfter: 4 * time.Second, + } + want := &CheckType{ + CheckID: "id", + Name: "name", + Status: "green", + Notes: "notes", + + Script: "/bin/foo", + HTTP: "someurl", + TCP: "host:port", + Interval: 1 * time.Second, + DockerContainerID: "abc123", + Shell: "/bin/ksh", + TLSSkipVerify: true, + Timeout: 2 * time.Second, + TTL: 3 * time.Second, + DeregisterCriticalServiceAfter: 4 * time.Second, + } + verify.Values(t, "", got.CheckType(), want) +} diff --git a/agent/structs/operator.go b/agent/structs/operator.go index 3bafa9f56..a3eadb060 100644 --- a/agent/structs/operator.go +++ b/agent/structs/operator.go @@ -1,6 +1,7 @@ package structs import ( + "net" "time" "github.com/hashicorp/raft" @@ -223,3 +224,20 @@ type OperatorHealthReply struct { // Servers holds the health of each server. Servers []ServerHealth } + +// (Enterprise-only) NetworkSegment is the configuration for a network segment, which is an +// isolated serf group on the LAN. +type NetworkSegment struct { + // Name is the name of the segment. + Name string + + // Bind is the bind address for this segment. + Bind *net.TCPAddr + + // Advertise is the advertise address of this segment. + Advertise *net.TCPAddr + + // RPCListener is whether to bind a separate RPC listener on the bind address + // for this segment. + RPCListener bool +} diff --git a/agent/testagent.go b/agent/testagent.go index e02efb339..93ae33447 100644 --- a/agent/testagent.go +++ b/agent/testagent.go @@ -15,13 +15,13 @@ import ( "strings" "time" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/consul" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/logger" + "github.com/hashicorp/consul/test/porter" "github.com/hashicorp/consul/testutil/retry" - "github.com/hashicorp/consul/types" - "github.com/hashicorp/consul/version" uuid "github.com/hashicorp/go-uuid" ) @@ -40,12 +40,14 @@ type TestAgent struct { // Name is an optional name of the agent. Name string + HCL string + // Config is the agent configuration. If Config is nil then // TestConfig() is used. If Config.DataDir is set then it is // the callers responsibility to clean up the data directory. // Otherwise, a temporary data directory is created and removed // when Shutdown() is called. - Config *Config + Config *config.RuntimeConfig // LogOutput is the sink for the logs. If nil, logs are written // to os.Stderr. @@ -83,8 +85,8 @@ type TestAgent struct { // configuration. It panics if the agent could not be started. The // caller should call Shutdown() to stop the agent and remove temporary // directories. -func NewTestAgent(name string, c *Config) *TestAgent { - a := &TestAgent{Name: name, Config: c} +func NewTestAgent(name string, hcl string) *TestAgent { + a := &TestAgent{Name: name, HCL: hcl} a.Start() return a } @@ -99,13 +101,8 @@ func (a *TestAgent) Start() *TestAgent { if a.Agent != nil { panic("TestAgent already started") } - if a.Config == nil { - a.Config = TestConfig() - } - if a.Config.DNSRecursor != "" { - a.Config.DNSRecursors = append(a.Config.DNSRecursors, a.Config.DNSRecursor) - } - if a.Config.DataDir == "" { + var hclDataDir string + if a.DataDir == "" { name := "agent" if a.Name != "" { name = a.Name + "-agent" @@ -115,13 +112,16 @@ func (a *TestAgent) Start() *TestAgent { if err != nil { panic(fmt.Sprintf("Error creating data dir %s: %s", filepath.Join(TempDir, name), err)) } - a.DataDir = d - a.Config.DataDir = d + hclDataDir = `data_dir = "` + d + `"` } id := UniqueID() for i := 10; i >= 0; i-- { - pickRandomPorts(a.Config) + a.Config = TestConfig( + config.Source{Name: a.Name, Format: "hcl", Data: a.HCL}, + config.Source{Name: a.Name + ".data_dir", Format: "hcl", Data: hclDataDir}, + randomPortsSource(), + ) // write the keyring if a.Key != "" { @@ -146,7 +146,7 @@ func (a *TestAgent) Start() *TestAgent { } agent.LogOutput = logOutput agent.LogWriter = a.LogWriter - agent.logger = log.New(logOutput, a.Name+" - ", log.LstdFlags) + agent.logger = log.New(logOutput, a.Name+" - ", log.LstdFlags|log.Lmicroseconds) // we need the err var in the next exit condition if err := agent.Start(); err == nil { @@ -182,7 +182,7 @@ func (a *TestAgent) Start() *TestAgent { if len(a.httpServers) == 0 { r.Fatal(a.Name, "waiting for server") } - if a.Config.Bootstrap && a.Config.Server { + if a.Config.Bootstrap && a.Config.ServerMode { // Ensure we have a leader and a node registration. args := &structs.DCSpecificRequest{ Datacenter: a.Config.Datacenter, @@ -217,17 +217,28 @@ func (a *TestAgent) Start() *TestAgent { // Shutdown stops the agent and removes the data directory if it is // managed by the test agent. func (a *TestAgent) Shutdown() error { + /* Removed this because it was breaking persistence tests where we would + persist a service and load it through a new agent with the same data-dir. + Not sure if we still need this for other things, everywhere we manually make + a data dir we already do 'defer os.RemoveAll()' defer func() { if a.DataDir != "" { os.RemoveAll(a.DataDir) } - }() + }()*/ // shutdown agent before endpoints defer a.Agent.ShutdownEndpoints() return a.Agent.ShutdownAgent() } +func (a *TestAgent) DNSAddr() string { + if a.dns == nil { + return "" + } + return a.dns.Addr +} + func (a *TestAgent) HTTPAddr() string { if a.srv == nil { return "" @@ -275,12 +286,6 @@ func UniqueID() string { return id } -// TenPorts returns the first port number of a block of -// ten random ports. -func TenPorts() int { - return 1030 + int(rand.Int31n(6440))*10 -} - // pickRandomPorts selects random ports from fixed size random blocks of // ports. This does not eliminate the chance for port conflict but // reduces it significanltly with little overhead. Furthermore, asking @@ -289,71 +294,85 @@ func TenPorts() int { // chance of port conflicts for concurrently executed test binaries. // Instead of relying on one set of ports to be sufficient we retry // starting the agent with different ports on port conflict. -func pickRandomPorts(c *Config) { - port := TenPorts() - c.Ports.DNS = port + 1 - c.Ports.HTTP = port + 2 - // when we enable HTTPS then we need to fix finding the - // "first" HTTP server since that might be HTTPS server - // c.Ports.HTTPS = port + 3 - c.Ports.SerfLan = port + 4 - c.Ports.SerfWan = port + 5 - c.Ports.Server = port + 6 +func randomPortsSource() config.Source { + ports, err := porter.RandomPorts(5) + if err != nil { + panic(err) + } + return config.Source{ + Name: "ports", + Format: "hcl", + Data: ` + ports = { + dns = ` + strconv.Itoa(ports[0]) + ` + http = ` + strconv.Itoa(ports[1]) + ` + https = -1 + serf_lan = ` + strconv.Itoa(ports[2]) + ` + serf_wan = ` + strconv.Itoa(ports[3]) + ` + server = ` + strconv.Itoa(ports[4]) + ` + } + `, + } +} + +func NodeID() string { + id, err := uuid.GenerateUUID() + if err != nil { + panic(err) + } + return id } // TestConfig returns a unique default configuration for testing an // agent. -func TestConfig() *Config { - nodeID, err := uuid.GenerateUUID() - if err != nil { - panic(err) +func TestConfig(sources ...config.Source) *config.RuntimeConfig { + nodeID := NodeID() + testsrc := config.Source{ + Name: "test", + Format: "hcl", + Data: ` + bind_addr = "127.0.0.1" + advertise_addr = "127.0.0.1" + datacenter = "dc1" + bootstrap = true + server = true + node_id = "` + nodeID + `" + node_name = "Node ` + nodeID + `" + performance { + raft_multiplier = 1 + } + `, } - cfg := DefaultConfig() + b, err := config.NewBuilder(config.Flags{}) + if err != nil { + panic("NewBuilder failed: " + err.Error()) + } + b.Head = append(b.Head, testsrc) + b.Tail = append(b.Tail, config.DefaultConsulSource(), config.DevConsulSource()) + b.Tail = append(b.Tail, sources...) - cfg.Version = version.Version - cfg.VersionPrerelease = "c.d" + cfg, err := b.BuildAndValidate() + if err != nil { + panic("Error building config: " + err.Error()) + } - cfg.NodeID = types.NodeID(nodeID) - cfg.NodeName = "Node " + nodeID - cfg.BindAddr = "127.0.0.1" - cfg.AdvertiseAddr = "127.0.0.1" - cfg.Datacenter = "dc1" - cfg.Bootstrap = true - cfg.Server = true + for _, w := range b.Warnings { + fmt.Println("WARNING:", w) + } - ccfg := consul.DefaultConfig() - cfg.ConsulConfig = ccfg - - ccfg.SerfLANConfig.MemberlistConfig.SuspicionMult = 3 - ccfg.SerfLANConfig.MemberlistConfig.ProbeTimeout = 100 * time.Millisecond - ccfg.SerfLANConfig.MemberlistConfig.ProbeInterval = 100 * time.Millisecond - ccfg.SerfLANConfig.MemberlistConfig.GossipInterval = 100 * time.Millisecond - - ccfg.SerfWANConfig.MemberlistConfig.SuspicionMult = 3 - ccfg.SerfWANConfig.MemberlistConfig.ProbeTimeout = 100 * time.Millisecond - ccfg.SerfWANConfig.MemberlistConfig.ProbeInterval = 100 * time.Millisecond - ccfg.SerfWANConfig.MemberlistConfig.GossipInterval = 100 * time.Millisecond - - ccfg.RaftConfig.LeaderLeaseTimeout = 20 * time.Millisecond - ccfg.RaftConfig.HeartbeatTimeout = 40 * time.Millisecond - ccfg.RaftConfig.ElectionTimeout = 40 * time.Millisecond - - ccfg.CoordinateUpdatePeriod = 100 * time.Millisecond - ccfg.ServerHealthInterval = 10 * time.Millisecond - cfg.SetupTaggedAndAdvertiseAddrs() - return cfg + return &cfg } // TestACLConfig returns a default configuration for testing an agent // with ACLs. -func TestACLConfig() *Config { - cfg := TestConfig() - cfg.ACLDatacenter = cfg.Datacenter - cfg.ACLDefaultPolicy = "deny" - cfg.ACLMasterToken = "root" - cfg.ACLAgentToken = "root" - cfg.ACLAgentMasterToken = "towel" - cfg.ACLEnforceVersion8 = Bool(true) - return cfg +func TestACLConfig() string { + return ` + acl_datacenter = "dc1" + acl_default_policy = "deny" + acl_master_token = "root" + acl_agent_token = "root" + acl_agent_master_token = "towel" + acl_enforce_version_8 = true + ` } diff --git a/agent/testagent_test.go b/agent/testagent_test.go new file mode 100644 index 000000000..12be557a3 --- /dev/null +++ b/agent/testagent_test.go @@ -0,0 +1,28 @@ +package agent + +import ( + "testing" + + "github.com/hashicorp/consul/agent/config" + "github.com/hashicorp/hcl" +) + +// TestDefaultConfig triggers a data race in the HCL parser. +func TestDefaultConfig(t *testing.T) { + for i := 0; i < 500; i++ { + t.Run("", func(t *testing.T) { + t.Parallel() + var c config.Config + data := config.DefaultSource().Data + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + hcl.Decode(&c, data) + }) + } +} diff --git a/agent/translate_addr.go b/agent/translate_addr.go index f93a685c0..138f7aadc 100644 --- a/agent/translate_addr.go +++ b/agent/translate_addr.go @@ -10,7 +10,7 @@ import ( // depending on how the agent and the other node are configured. The dc // parameter is the dc the datacenter this node is from. func (a *Agent) TranslateAddress(dc string, addr string, taggedAddresses map[string]string) string { - if a.config.TranslateWanAddrs && (a.config.Datacenter != dc) { + if a.config.TranslateWANAddrs && (a.config.Datacenter != dc) { wanAddr := taggedAddresses["wan"] if wanAddr != "" { addr = wanAddr @@ -34,7 +34,7 @@ func (a *Agent) TranslateAddresses(dc string, subj interface{}) { // done. This also happens to skip looking at any of the incoming // structure for the common case of not needing to translate, so it will // skip a lot of work if no translation needs to be done. - if !a.config.TranslateWanAddrs || (a.config.Datacenter == dc) { + if !a.config.TranslateWANAddrs || (a.config.Datacenter == dc) { return } diff --git a/agent/txn_endpoint_test.go b/agent/txn_endpoint_test.go index a8241abd8..faf653ff9 100644 --- a/agent/txn_endpoint_test.go +++ b/agent/txn_endpoint_test.go @@ -14,7 +14,7 @@ import ( func TestTxnEndpoint_Bad_JSON(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() buf := bytes.NewBuffer([]byte("{")) @@ -33,7 +33,7 @@ func TestTxnEndpoint_Bad_JSON(t *testing.T) { func TestTxnEndpoint_Bad_Method(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() buf := bytes.NewBuffer([]byte("{}")) @@ -49,20 +49,20 @@ func TestTxnEndpoint_Bad_Method(t *testing.T) { func TestTxnEndpoint_Bad_Size_Item(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() buf := bytes.NewBuffer([]byte(fmt.Sprintf(` -[ - { - "KV": { - "Verb": "set", - "Key": "key", - "Value": %q - } - } -] -`, strings.Repeat("bad", 2*maxKVSize)))) + [ + { + "KV": { + "Verb": "set", + "Key": "key", + "Value": %q + } + } + ] + `, strings.Repeat("bad", 2*maxKVSize)))) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() if _, err := a.srv.Txn(resp, req); err != nil { @@ -75,35 +75,35 @@ func TestTxnEndpoint_Bad_Size_Item(t *testing.T) { func TestTxnEndpoint_Bad_Size_Net(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() value := strings.Repeat("X", maxKVSize/2) buf := bytes.NewBuffer([]byte(fmt.Sprintf(` -[ - { - "KV": { - "Verb": "set", - "Key": "key1", - "Value": %q - } - }, - { - "KV": { - "Verb": "set", - "Key": "key1", - "Value": %q - } - }, - { - "KV": { - "Verb": "set", - "Key": "key1", - "Value": %q - } - } -] -`, value, value, value))) + [ + { + "KV": { + "Verb": "set", + "Key": "key1", + "Value": %q + } + }, + { + "KV": { + "Verb": "set", + "Key": "key1", + "Value": %q + } + }, + { + "KV": { + "Verb": "set", + "Key": "key1", + "Value": %q + } + } + ] + `, value, value, value))) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() if _, err := a.srv.Txn(resp, req); err != nil { @@ -116,21 +116,21 @@ func TestTxnEndpoint_Bad_Size_Net(t *testing.T) { func TestTxnEndpoint_Bad_Size_Ops(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() buf := bytes.NewBuffer([]byte(fmt.Sprintf(` -[ - %s - { - "KV": { - "Verb": "set", - "Key": "key", - "Value": "" - } - } -] -`, strings.Repeat(`{ "KV": { "Verb": "get", "Key": "key" } },`, 2*maxTxnOps)))) + [ + %s + { + "KV": { + "Verb": "set", + "Key": "key", + "Value": "" + } + } + ] + `, strings.Repeat(`{ "KV": { "Verb": "get", "Key": "key" } },`, 2*maxTxnOps)))) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() if _, err := a.srv.Txn(resp, req); err != nil { @@ -144,7 +144,7 @@ func TestTxnEndpoint_Bad_Size_Ops(t *testing.T) { func TestTxnEndpoint_KV_Actions(t *testing.T) { t.Parallel() t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() // Make sure all incoming fields get converted properly to the internal @@ -153,24 +153,24 @@ func TestTxnEndpoint_KV_Actions(t *testing.T) { id := makeTestSession(t, a.srv) { buf := bytes.NewBuffer([]byte(fmt.Sprintf(` -[ - { - "KV": { - "Verb": "lock", - "Key": "key", - "Value": "aGVsbG8gd29ybGQ=", - "Flags": 23, - "Session": %q - } - }, - { - "KV": { - "Verb": "get", - "Key": "key" - } - } -] -`, id))) + [ + { + "KV": { + "Verb": "lock", + "Key": "key", + "Value": "aGVsbG8gd29ybGQ=", + "Flags": 23, + "Session": %q + } + }, + { + "KV": { + "Verb": "get", + "Key": "key" + } + } + ] + `, id))) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() obj, err := a.srv.Txn(resp, req) @@ -228,21 +228,21 @@ func TestTxnEndpoint_KV_Actions(t *testing.T) { // fast-path endpoint. { buf := bytes.NewBuffer([]byte(` -[ - { - "KV": { - "Verb": "get", - "Key": "key" - } - }, - { - "KV": { - "Verb": "get-tree", - "Key": "key" - } - } -] -`)) + [ + { + "KV": { + "Verb": "get", + "Key": "key" + } + }, + { + "KV": { + "Verb": "get-tree", + "Key": "key" + } + } + ] + `)) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() obj, err := a.srv.Txn(resp, req) @@ -310,23 +310,23 @@ func TestTxnEndpoint_KV_Actions(t *testing.T) { // index field gets translated to the RPC format. { buf := bytes.NewBuffer([]byte(fmt.Sprintf(` -[ - { - "KV": { - "Verb": "cas", - "Key": "key", - "Value": "Z29vZGJ5ZSB3b3JsZA==", - "Index": %d - } - }, - { - "KV": { - "Verb": "get", - "Key": "key" - } - } -] -`, index))) + [ + { + "KV": { + "Verb": "cas", + "Key": "key", + "Value": "Z29vZGJ5ZSB3b3JsZA==", + "Index": %d + } + }, + { + "KV": { + "Verb": "get", + "Key": "key" + } + } + ] + `, index))) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() obj, err := a.srv.Txn(resp, req) @@ -379,27 +379,27 @@ func TestTxnEndpoint_KV_Actions(t *testing.T) { // Verify an error inside a transaction. t.Run("", func(t *testing.T) { - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() buf := bytes.NewBuffer([]byte(` -[ - { - "KV": { - "Verb": "lock", - "Key": "key", - "Value": "aGVsbG8gd29ybGQ=", - "Session": "nope" - } - }, - { - "KV": { - "Verb": "get", - "Key": "key" - } - } -] -`)) + [ + { + "KV": { + "Verb": "lock", + "Key": "key", + "Value": "aGVsbG8gd29ybGQ=", + "Session": "nope" + } + }, + { + "KV": { + "Verb": "get", + "Key": "key" + } + } + ] + `)) req, _ := http.NewRequest("PUT", "/v1/txn", buf) resp := httptest.NewRecorder() if _, err := a.srv.Txn(resp, req); err != nil { diff --git a/agent/ui_endpoint_test.go b/agent/ui_endpoint_test.go index d780d9147..df1a22d01 100644 --- a/agent/ui_endpoint_test.go +++ b/agent/ui_endpoint_test.go @@ -15,7 +15,7 @@ import ( "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/testutil" - "github.com/hashicorp/go-cleanhttp" + cleanhttp "github.com/hashicorp/go-cleanhttp" ) func TestUiIndex(t *testing.T) { @@ -25,9 +25,9 @@ func TestUiIndex(t *testing.T) { defer os.RemoveAll(uiDir) // Make the server - cfg := TestConfig() - cfg.UIDir = uiDir - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), ` + ui_dir = "`+uiDir+`" + `) defer a.Shutdown() // Create file @@ -63,7 +63,7 @@ func TestUiIndex(t *testing.T) { func TestUiNodes(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() args := &structs.RegisterRequest{ @@ -100,7 +100,7 @@ func TestUiNodes(t *testing.T) { func TestUiNodeInfo(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() req, _ := http.NewRequest("GET", fmt.Sprintf("/v1/internal/ui/node/%s", a.Config.NodeName), nil) diff --git a/agent/user_event.go b/agent/user_event.go index 9506c6c14..a83cb7fdc 100644 --- a/agent/user_event.go +++ b/agent/user_event.go @@ -209,7 +209,7 @@ func (a *Agent) ingestUserEvent(msg *UserEvent) { // Special handling for internal events switch msg.Name { case remoteExecName: - if *a.config.DisableRemoteExec { + if a.config.DisableRemoteExec { a.logger.Printf("[INFO] agent: ignoring remote exec event (%s), disabled.", msg.ID) } else { go a.handleRemoteExec(msg) diff --git a/agent/user_event_test.go b/agent/user_event_test.go index 17c08b568..0f1e59d00 100644 --- a/agent/user_event_test.go +++ b/agent/user_event_test.go @@ -48,7 +48,7 @@ func TestValidateUserEventParams(t *testing.T) { func TestShouldProcessUserEvent(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() srv1 := &structs.NodeService{ @@ -117,7 +117,7 @@ func TestShouldProcessUserEvent(t *testing.T) { func TestIngestUserEvent(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() for i := 0; i < 512; i++ { @@ -148,7 +148,7 @@ func TestIngestUserEvent(t *testing.T) { func TestFireReceiveEvent(t *testing.T) { t.Parallel() - a := NewTestAgent(t.Name(), nil) + a := NewTestAgent(t.Name(), "") defer a.Shutdown() srv1 := &structs.NodeService{ @@ -184,9 +184,9 @@ func TestFireReceiveEvent(t *testing.T) { func TestUserEventToken(t *testing.T) { t.Parallel() - cfg := TestACLConfig() - cfg.ACLDefaultPolicy = "deny" // Set the default policies to deny - a := NewTestAgent(t.Name(), cfg) + a := NewTestAgent(t.Name(), TestACLConfig()+` + acl_default_policy = "deny" + `) defer a.Shutdown() // Create an ACL token diff --git a/agent/util.go b/agent/util.go index 3739b0000..9811aac04 100644 --- a/agent/util.go +++ b/agent/util.go @@ -6,7 +6,7 @@ import ( "fmt" "math" "os" - "os/user" + osuser "os/user" "strconv" "time" @@ -65,47 +65,34 @@ func checkIDHash(checkID types.CheckID) string { return stringHash(string(checkID)) } -// FilePermissions is an interface which allows a struct to set -// ownership and permissions easily on a file it describes. -type FilePermissions interface { - // User returns a user ID or user name - User() string - - // Group returns a group ID. Group names are not supported. - Group() string - - // Mode returns a string of file mode bits e.g. "0644" - Mode() string -} - -// setFilePermissions handles configuring ownership and permissions settings -// on a given file. It takes a path and any struct implementing the -// FilePermissions interface. All permission/ownership settings are optional. -// If no user or group is specified, the current user/group will be used. Mode -// is optional, and has no default (the operation is not performed if absent). -// User may be specified by name or ID, but group may only be specified by ID. -func setFilePermissions(path string, p FilePermissions) error { +// setFilePermissions handles configuring ownership and permissions +// settings on a given file. All permission/ownership settings are +// optional. If no user or group is specified, the current user/group +// will be used. Mode is optional, and has no default (the operation is +// not performed if absent). User may be specified by name or ID, but +// group may only be specified by ID. +func setFilePermissions(path string, user, group, mode string) error { var err error uid, gid := os.Getuid(), os.Getgid() - if p.User() != "" { - if uid, err = strconv.Atoi(p.User()); err == nil { + if user != "" { + if uid, err = strconv.Atoi(user); err == nil { goto GROUP } // Try looking up the user by name - if u, err := user.Lookup(p.User()); err == nil { + if u, err := osuser.Lookup(user); err == nil { uid, _ = strconv.Atoi(u.Uid) goto GROUP } - return fmt.Errorf("invalid user specified: %v", p.User()) + return fmt.Errorf("invalid user specified: %v", user) } GROUP: - if p.Group() != "" { - if gid, err = strconv.Atoi(p.Group()); err != nil { - return fmt.Errorf("invalid group specified: %v", p.Group()) + if group != "" { + if gid, err = strconv.Atoi(group); err != nil { + return fmt.Errorf("invalid group specified: %v", group) } } if err := os.Chown(path, uid, gid); err != nil { @@ -113,10 +100,10 @@ GROUP: uid, gid, path, err) } - if p.Mode() != "" { - mode, err := strconv.ParseUint(p.Mode(), 8, 32) + if mode != "" { + mode, err := strconv.ParseUint(mode, 8, 32) if err != nil { - return fmt.Errorf("invalid mode specified: %v", p.Mode()) + return fmt.Errorf("invalid mode specified: %v", mode) } if err := os.Chmod(path, os.FileMode(mode)); err != nil { return fmt.Errorf("failed setting permissions to %d on %q: %s", diff --git a/agent/util_test.go b/agent/util_test.go index f2e2e6509..ea12c708d 100644 --- a/agent/util_test.go +++ b/agent/util_test.go @@ -46,22 +46,22 @@ func TestSetFilePermissions(t *testing.T) { defer os.Remove(path) // Bad UID fails - if err := setFilePermissions(path, UnixSocketPermissions{Usr: "%"}); err == nil { + if err := setFilePermissions(path, "%", "", ""); err == nil { t.Fatalf("should fail") } // Bad GID fails - if err := setFilePermissions(path, UnixSocketPermissions{Grp: "%"}); err == nil { + if err := setFilePermissions(path, "", "%", ""); err == nil { t.Fatalf("should fail") } // Bad mode fails - if err := setFilePermissions(path, UnixSocketPermissions{Perms: "%"}); err == nil { + if err := setFilePermissions(path, "", "", "%"); err == nil { t.Fatalf("should fail") } // Allows omitting user/group/mode - if err := setFilePermissions(path, UnixSocketPermissions{}); err != nil { + if err := setFilePermissions(path, "", "", ""); err != nil { t.Fatalf("err: %s", err) } @@ -69,7 +69,7 @@ func TestSetFilePermissions(t *testing.T) { if err := os.Chmod(path, 0700); err != nil { t.Fatalf("err: %s", err) } - if err := setFilePermissions(path, UnixSocketPermissions{}); err != nil { + if err := setFilePermissions(path, "", "", ""); err != nil { t.Fatalf("err: %s", err) } fi, err := os.Stat(path) @@ -81,7 +81,7 @@ func TestSetFilePermissions(t *testing.T) { } // Changes mode if given - if err := setFilePermissions(path, UnixSocketPermissions{Perms: "0777"}); err != nil { + if err := setFilePermissions(path, "", "", "0777"); err != nil { t.Fatalf("err: %s", err) } fi, err = os.Stat(path) diff --git a/api/agent_test.go b/api/agent_test.go index fda397132..a51f467a0 100644 --- a/api/agent_test.go +++ b/api/agent_test.go @@ -1,6 +1,7 @@ package api import ( + "fmt" "io/ioutil" "strings" "testing" @@ -630,7 +631,9 @@ func TestAPI_AgentJoin(t *testing.T) { } // Join ourself - addr := info["Config"]["AdvertiseAddr"].(string) + ip := info["Config"]["SerfAdvertiseAddrLAN"].(map[string]interface{})["IP"].(string) + port := info["Config"]["SerfAdvertiseAddrLAN"].(map[string]interface{})["Port"].(float64) + addr := fmt.Sprintf("%s:%d", ip, int(port)) err = agent.Join(addr, false) if err != nil { t.Fatalf("err: %v", err) diff --git a/api/api.go b/api/api.go index 31efecca0..3c28afe28 100644 --- a/api/api.go +++ b/api/api.go @@ -446,6 +446,7 @@ func NewClient(config *Config) (*Client, error) { if len(parts) == 2 { switch parts[0] { case "http": + config.Scheme = "http" case "https": config.Scheme = "https" case "unix": @@ -466,10 +467,7 @@ func NewClient(config *Config) (*Client, error) { config.Token = defConfig.Token } - client := &Client{ - config: *config, - } - return client, nil + return &Client{config: *config}, nil } // NewHttpClient returns an http client configured with the given Transport and TLS diff --git a/api/api_test.go b/api/api_test.go index 341956c4f..8ba266a5a 100644 --- a/api/api_test.go +++ b/api/api_test.go @@ -52,6 +52,7 @@ func makeClientWithConfig( // Create client client, err := NewClient(conf) if err != nil { + server.Stop() t.Fatalf("err: %v", err) } @@ -73,7 +74,11 @@ func testKey() string { } func TestAPI_DefaultConfig_env(t *testing.T) { - t.Parallel() + // t.Parallel() // DO NOT ENABLE !!! + // do not enable t.Parallel for this test since it modifies global state + // (environment) which has non-deterministic effects on the other tests + // which derive their default configuration from the environment + addr := "1.2.3.4:5678" token := "abcd1234" auth := "username:password" @@ -151,6 +156,7 @@ func TestAPI_DefaultConfig_env(t *testing.T) { } func TestAPI_SetupTLSConfig(t *testing.T) { + t.Parallel() // A default config should result in a clean default client config. tlsConfig := &TLSConfig{} cc, err := SetupTLSConfig(tlsConfig) @@ -505,6 +511,7 @@ func TestAPI_UnixSocket(t *testing.T) { } func TestAPI_durToMsec(t *testing.T) { + t.Parallel() if ms := durToMsec(0); ms != "0ms" { t.Fatalf("bad: %s", ms) } @@ -523,6 +530,7 @@ func TestAPI_durToMsec(t *testing.T) { } func TestAPI_IsServerError(t *testing.T) { + t.Parallel() if IsServerError(nil) { t.Fatalf("should not be a server error") } diff --git a/api/catalog_test.go b/api/catalog_test.go index 57cf5acdc..11f50a919 100644 --- a/api/catalog_test.go +++ b/api/catalog_test.go @@ -26,6 +26,7 @@ func TestAPI_CatalogDatacenters(t *testing.T) { } func TestAPI_CatalogNodes(t *testing.T) { + t.Parallel() c, s := makeClient(t) defer s.Stop() @@ -62,6 +63,7 @@ func TestAPI_CatalogNodes(t *testing.T) { } func TestAPI_CatalogNodes_MetaFilter(t *testing.T) { + t.Parallel() meta := map[string]string{"somekey": "somevalue"} c, s := makeClientWithConfig(t, nil, func(conf *testutil.TestServerConfig) { conf.NodeMeta = meta @@ -137,6 +139,7 @@ func TestAPI_CatalogServices(t *testing.T) { } func TestAPI_CatalogServices_NodeMetaFilter(t *testing.T) { + t.Parallel() meta := map[string]string{"somekey": "somevalue"} c, s := makeClientWithConfig(t, nil, func(conf *testutil.TestServerConfig) { conf.NodeMeta = meta diff --git a/api/health_test.go b/api/health_test.go index 9cbb55e21..c4ef11651 100644 --- a/api/health_test.go +++ b/api/health_test.go @@ -256,6 +256,7 @@ func TestAPI_HealthChecks_NodeMetaFilter(t *testing.T) { } func TestAPI_HealthService(t *testing.T) { + t.Parallel() c, s := makeClient(t) defer s.Stop() @@ -282,6 +283,7 @@ func TestAPI_HealthService(t *testing.T) { } func TestAPI_HealthService_NodeMetaFilter(t *testing.T) { + t.Parallel() meta := map[string]string{"somekey": "somevalue"} c, s := makeClientWithConfig(t, nil, func(conf *testutil.TestServerConfig) { conf.NodeMeta = meta diff --git a/api/operator_keyring_test.go b/api/operator_keyring_test.go index e877170fe..354eea391 100644 --- a/api/operator_keyring_test.go +++ b/api/operator_keyring_test.go @@ -7,9 +7,9 @@ import ( ) func TestAPI_OperatorKeyringInstallListPutRemove(t *testing.T) { + t.Parallel() oldKey := "d8wu8CSUrqgtjVsvcBPmhQ==" newKey := "qxycTi/SsePj/TZzCBmNXw==" - t.Parallel() c, s := makeClientWithConfig(t, nil, func(c *testutil.TestServerConfig) { c.Encrypt = oldKey }) diff --git a/command/agent.go b/command/agent.go index 2713b5002..a01de8f7e 100644 --- a/command/agent.go +++ b/command/agent.go @@ -8,7 +8,6 @@ import ( "os" "os/signal" "path/filepath" - "reflect" "regexp" "strings" "syscall" @@ -18,14 +17,10 @@ import ( "github.com/armon/go-metrics/circonus" "github.com/armon/go-metrics/datadog" "github.com/hashicorp/consul/agent" - "github.com/hashicorp/consul/agent/structs" - "github.com/hashicorp/consul/configutil" - "github.com/hashicorp/consul/ipaddr" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/lib" "github.com/hashicorp/consul/logger" - "github.com/hashicorp/consul/watch" "github.com/hashicorp/go-checkpoint" - discover "github.com/hashicorp/go-discover" multierror "github.com/hashicorp/go-multierror" "github.com/hashicorp/logutils" "github.com/mitchellh/cli" @@ -53,500 +48,30 @@ type AgentCommand struct { // readConfig is responsible for setup of our configuration using // the command line and any file configs -func (cmd *AgentCommand) readConfig() *agent.Config { - var cmdCfg agent.Config - var cfgFiles []string - var retryInterval string - var retryIntervalWan string - var dnsRecursors []string - var dev bool - var nodeMeta []string - - f := cmd.BaseCommand.NewFlagSet(cmd) - - f.Var((*configutil.AppendSliceValue)(&cfgFiles), "config-file", - "Path to a JSON file to read configuration from. This can be specified multiple times.") - f.Var((*configutil.AppendSliceValue)(&cfgFiles), "config-dir", - "Path to a directory to read configuration files from. This will read every file ending "+ - "in '.json' as configuration in this directory in alphabetical order. This can be "+ - "specified multiple times.") - f.Var((*configutil.AppendSliceValue)(&dnsRecursors), "recursor", - "Address of an upstream DNS server. Can be specified multiple times.") - f.Var((*configutil.AppendSliceValue)(&nodeMeta), "node-meta", - "An arbitrary metadata key/value pair for this node, of the format `key:value`. Can be specified multiple times.") - f.BoolVar(&dev, "dev", false, "Starts the agent in development mode.") - - f.StringVar(&cmdCfg.LogLevel, "log-level", "", "Log level of the agent.") - f.StringVar(&cmdCfg.NodeName, "node", "", "Name of this node. Must be unique in the cluster.") - f.StringVar((*string)(&cmdCfg.NodeID), "node-id", "", - "A unique ID for this node across space and time. Defaults to a randomly-generated ID"+ - " that persists in the data-dir.") - - f.BoolVar(&cmdCfg.EnableScriptChecks, "enable-script-checks", false, "Enables health check scripts.") - var disableHostNodeID configutil.BoolValue - f.Var(&disableHostNodeID, "disable-host-node-id", - "Setting this to true will prevent Consul from using information from the"+ - " host to generate a node ID, and will cause Consul to generate a"+ - " random node ID instead.") - - f.StringVar(&cmdCfg.Datacenter, "datacenter", "", "Datacenter of the agent.") - f.StringVar(&cmdCfg.DataDir, "data-dir", "", "Path to a data directory to store agent state.") - f.BoolVar(&cmdCfg.EnableUI, "ui", false, "Enables the built-in static web UI server.") - f.StringVar(&cmdCfg.UIDir, "ui-dir", "", "Path to directory containing the web UI resources.") - f.StringVar(&cmdCfg.PidFile, "pid-file", "", "Path to file to store agent PID.") - f.StringVar(&cmdCfg.EncryptKey, "encrypt", "", "Provides the gossip encryption key.") - f.BoolVar(&cmdCfg.DisableKeyringFile, "disable-keyring-file", false, "Disables the backing up "+ - "of the keyring to a file.") - - f.BoolVar(&cmdCfg.Server, "server", false, "Switches agent to server mode.") - f.BoolVar(&cmdCfg.NonVotingServer, "non-voting-server", false, - "(Enterprise-only) This flag is used to make the server not participate in the Raft quorum, "+ - "and have it only receive the data replication stream. This can be used to add read scalability "+ - "to a cluster in cases where a high volume of reads to servers are needed.") - f.BoolVar(&cmdCfg.Bootstrap, "bootstrap", false, "Sets server to bootstrap mode.") - f.IntVar(&cmdCfg.BootstrapExpect, "bootstrap-expect", 0, "Sets server to expect bootstrap mode.") - f.StringVar(&cmdCfg.Domain, "domain", "", "Domain to use for DNS interface.") - - f.StringVar(&cmdCfg.ClientAddr, "client", "", - "Sets the address to bind for client access. This includes RPC, DNS, HTTP and HTTPS (if configured).") - f.StringVar(&cmdCfg.BindAddr, "bind", "", "Sets the bind address for cluster communication.") - f.StringVar(&cmdCfg.SerfWanBindAddr, "serf-wan-bind", "", "Address to bind Serf WAN listeners to.") - f.StringVar(&cmdCfg.SerfLanBindAddr, "serf-lan-bind", "", "Address to bind Serf LAN listeners to.") - f.IntVar(&cmdCfg.Ports.HTTP, "http-port", 0, "Sets the HTTP API port to listen on.") - f.IntVar(&cmdCfg.Ports.DNS, "dns-port", 0, "DNS port to use.") - f.StringVar(&cmdCfg.AdvertiseAddr, "advertise", "", "Sets the advertise address to use.") - f.StringVar(&cmdCfg.AdvertiseAddrWan, "advertise-wan", "", - "Sets address to advertise on WAN instead of -advertise address.") - f.StringVar(&cmdCfg.Segment, "segment", "", "(Enterprise-only) Sets the network segment to join.") - - f.IntVar(&cmdCfg.Protocol, "protocol", -1, - "Sets the protocol version. Defaults to latest.") - f.IntVar(&cmdCfg.RaftProtocol, "raft-protocol", -1, - "Sets the Raft protocol version. Defaults to latest.") - - f.BoolVar(&cmdCfg.EnableSyslog, "syslog", false, - "Enables logging to syslog.") - f.BoolVar(&cmdCfg.RejoinAfterLeave, "rejoin", false, - "Ignores a previous leave and attempts to rejoin the cluster.") - f.Var((*configutil.AppendSliceValue)(&cmdCfg.StartJoin), "join", - "Address of an agent to join at start time. Can be specified multiple times.") - f.Var((*configutil.AppendSliceValue)(&cmdCfg.StartJoinWan), "join-wan", - "Address of an agent to join -wan at start time. Can be specified multiple times.") - f.Var((*configutil.AppendSliceValue)(&cmdCfg.RetryJoin), "retry-join", - "Address of an agent to join at start time with retries enabled. Can be specified multiple times.") - f.IntVar(&cmdCfg.RetryMaxAttempts, "retry-max", 0, - "Maximum number of join attempts. Defaults to 0, which will retry indefinitely.") - f.StringVar(&retryInterval, "retry-interval", "", - "Time to wait between join attempts.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinEC2.Region, "retry-join-ec2-region", "", - "EC2 Region to discover servers in.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinEC2.TagKey, "retry-join-ec2-tag-key", "", - "EC2 tag key to filter on for server discovery.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinEC2.TagValue, "retry-join-ec2-tag-value", "", - "EC2 tag value to filter on for server discovery.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinGCE.ProjectName, "retry-join-gce-project-name", "", - "Google Compute Engine project to discover servers in.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinGCE.ZonePattern, "retry-join-gce-zone-pattern", "", - "Google Compute Engine region or zone to discover servers in (regex pattern).") - f.StringVar(&cmdCfg.DeprecatedRetryJoinGCE.TagValue, "retry-join-gce-tag-value", "", - "Google Compute Engine tag value to filter on for server discovery.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinGCE.CredentialsFile, "retry-join-gce-credentials-file", "", - "Path to credentials JSON file to use with Google Compute Engine.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinAzure.TagName, "retry-join-azure-tag-name", "", - "Azure tag name to filter on for server discovery.") - f.StringVar(&cmdCfg.DeprecatedRetryJoinAzure.TagValue, "retry-join-azure-tag-value", "", - "Azure tag value to filter on for server discovery.") - f.Var((*configutil.AppendSliceValue)(&cmdCfg.RetryJoinWan), "retry-join-wan", - "Address of an agent to join -wan at start time with retries enabled. "+ - "Can be specified multiple times.") - f.IntVar(&cmdCfg.RetryMaxAttemptsWan, "retry-max-wan", 0, - "Maximum number of join -wan attempts. Defaults to 0, which will retry indefinitely.") - f.StringVar(&retryIntervalWan, "retry-interval-wan", "", - "Time to wait between join -wan attempts.") - - // deprecated flags - var dcDeprecated string - var atlasJoin bool - var atlasInfrastructure, atlasToken, atlasEndpoint string - f.StringVar(&dcDeprecated, "dc", "", - "(deprecated) Datacenter of the agent (use 'datacenter' instead).") - f.StringVar(&atlasInfrastructure, "atlas", "", - "(deprecated) Sets the Atlas infrastructure name, enables SCADA.") - f.StringVar(&atlasToken, "atlas-token", "", - "(deprecated) Provides the Atlas API token.") - f.BoolVar(&atlasJoin, "atlas-join", false, - "(deprecated) Enables auto-joining the Atlas cluster.") - f.StringVar(&atlasEndpoint, "atlas-endpoint", "", - "(deprecated) The address of the endpoint for Atlas integration.") +func (cmd *AgentCommand) readConfig() *config.RuntimeConfig { + var flags config.Flags + fs := cmd.BaseCommand.NewFlagSet(cmd) + config.AddFlags(fs, &flags) if err := cmd.BaseCommand.Parse(cmd.args); err != nil { + cmd.UI.Error(fmt.Sprintf("error parsing flags: %v", err)) return nil } - // check deprecated flags - if atlasInfrastructure != "" { - cmd.UI.Warn("WARNING: 'atlas' is deprecated") - } - if atlasToken != "" { - cmd.UI.Warn("WARNING: 'atlas-token' is deprecated") - } - if atlasJoin { - cmd.UI.Warn("WARNING: 'atlas-join' is deprecated") - } - if atlasEndpoint != "" { - cmd.UI.Warn("WARNING: 'atlas-endpoint' is deprecated") - } - if dcDeprecated != "" && cmdCfg.Datacenter == "" { - cmd.UI.Warn("WARNING: 'dc' is deprecated. Use 'datacenter' instead") - cmdCfg.Datacenter = dcDeprecated - } - - if retryInterval != "" { - dur, err := time.ParseDuration(retryInterval) - if err != nil { - cmd.UI.Error(fmt.Sprintf("Error: %s", err)) - return nil - } - cmdCfg.RetryInterval = dur - } - - if retryIntervalWan != "" { - dur, err := time.ParseDuration(retryIntervalWan) - if err != nil { - cmd.UI.Error(fmt.Sprintf("Error: %s", err)) - return nil - } - cmdCfg.RetryIntervalWan = dur - } - - if len(nodeMeta) > 0 { - cmdCfg.Meta = make(map[string]string) - for _, entry := range nodeMeta { - key, value := agent.ParseMetaPair(entry) - cmdCfg.Meta[key] = value - } - if err := structs.ValidateMetadata(cmdCfg.Meta, false); err != nil { - cmd.UI.Error(fmt.Sprintf("Failed to parse node metadata: %v", err)) - return nil - } - } - - cfg := agent.DefaultConfig() - if dev { - cfg = agent.DevConfig() - } - - if len(cfgFiles) > 0 { - fileConfig, err := agent.ReadConfigPaths(cfgFiles) - if err != nil { - cmd.UI.Error(err.Error()) - return nil - } - - cfg = agent.MergeConfig(cfg, fileConfig) - } - - cmdCfg.DNSRecursors = append(cmdCfg.DNSRecursors, dnsRecursors...) - - cfg = agent.MergeConfig(cfg, &cmdCfg) - disableHostNodeID.Merge(cfg.DisableHostNodeID) - - if cfg.NodeName == "" { - hostname, err := os.Hostname() - if err != nil { - cmd.UI.Error(fmt.Sprintf("Error determining node name: %s", err)) - return nil - } - cfg.NodeName = hostname - } - cfg.NodeName = strings.TrimSpace(cfg.NodeName) - if cfg.NodeName == "" { - cmd.UI.Error("Node name can not be empty") + b, err := config.NewBuilder(flags) + if err != nil { + cmd.UI.Error(err.Error()) return nil } - - // Make sure LeaveOnTerm and SkipLeaveOnInt are set to the right - // defaults based on the agent's mode (client or server). - if cfg.LeaveOnTerm == nil { - cfg.LeaveOnTerm = agent.Bool(!cfg.Server) - } - if cfg.SkipLeaveOnInt == nil { - cfg.SkipLeaveOnInt = agent.Bool(cfg.Server) - } - - // Ensure we have a data directory if we are not in dev mode. - if !dev { - if cfg.DataDir == "" { - cmd.UI.Error("Must specify data directory using -data-dir") - return nil - } - - if finfo, err := os.Stat(cfg.DataDir); err != nil { - if !os.IsNotExist(err) { - cmd.UI.Error(fmt.Sprintf("Error getting data-dir: %s", err)) - return nil - } - } else if !finfo.IsDir() { - cmd.UI.Error(fmt.Sprintf("The data-dir specified at %q is not a directory", cfg.DataDir)) - return nil - } - } - - // Ensure all endpoints are unique - if err := cfg.VerifyUniqueListeners(); err != nil { - cmd.UI.Error(fmt.Sprintf("All listening endpoints must be unique: %s", err)) + cfg, err := b.BuildAndValidate() + if err != nil { + cmd.UI.Error(err.Error()) return nil } - - // Check the data dir for signs of an un-migrated Consul 0.5.x or older - // server. Consul refuses to start if this is present to protect a server - // with existing data from starting on a fresh data set. - if cfg.Server { - mdbPath := filepath.Join(cfg.DataDir, "mdb") - if _, err := os.Stat(mdbPath); !os.IsNotExist(err) { - if os.IsPermission(err) { - cmd.UI.Error(fmt.Sprintf("CRITICAL: Permission denied for data folder at %q!", mdbPath)) - cmd.UI.Error("Consul will refuse to boot without access to this directory.") - cmd.UI.Error("Please correct permissions and try starting again.") - return nil - } - cmd.UI.Error(fmt.Sprintf("CRITICAL: Deprecated data folder found at %q!", mdbPath)) - cmd.UI.Error("Consul will refuse to boot with this directory present.") - cmd.UI.Error("See https://www.consul.io/docs/upgrade-specific.html for more information.") - return nil - } + for _, w := range b.Warnings { + cmd.UI.Warn(w) } - - // Verify DNS settings - if cfg.DNSConfig.UDPAnswerLimit < 1 { - cmd.UI.Error(fmt.Sprintf("dns_config.udp_answer_limit %d too low, must always be greater than zero", cfg.DNSConfig.UDPAnswerLimit)) - } - - if cfg.EncryptKey != "" { - if _, err := cfg.EncryptBytes(); err != nil { - cmd.UI.Error(fmt.Sprintf("Invalid encryption key: %s", err)) - return nil - } - keyfileLAN := filepath.Join(cfg.DataDir, agent.SerfLANKeyring) - if _, err := os.Stat(keyfileLAN); err == nil { - cmd.UI.Error("WARNING: LAN keyring exists but -encrypt given, using keyring") - } - if cfg.Server { - keyfileWAN := filepath.Join(cfg.DataDir, agent.SerfWANKeyring) - if _, err := os.Stat(keyfileWAN); err == nil { - cmd.UI.Error("WARNING: WAN keyring exists but -encrypt given, using keyring") - } - } - } - - // Ensure the datacenter is always lowercased. The DNS endpoints automatically - // lowercase all queries, and internally we expect DC1 and dc1 to be the same. - cfg.Datacenter = strings.ToLower(cfg.Datacenter) - - // Verify datacenter is valid - if !validDatacenter.MatchString(cfg.Datacenter) { - cmd.UI.Error("Datacenter must be alpha-numeric with underscores and hypens only") - return nil - } - - // If 'acl_datacenter' is set, ensure it is lowercased. - if cfg.ACLDatacenter != "" { - cfg.ACLDatacenter = strings.ToLower(cfg.ACLDatacenter) - - // Verify 'acl_datacenter' is valid - if !validDatacenter.MatchString(cfg.ACLDatacenter) { - cmd.UI.Error("ACL datacenter must be alpha-numeric with underscores and hypens only") - return nil - } - } - - // Only allow bootstrap mode when acting as a server - if cfg.Bootstrap && !cfg.Server { - cmd.UI.Error("Bootstrap mode cannot be enabled when server mode is not enabled") - return nil - } - - // Expect can only work when acting as a server - if cfg.BootstrapExpect != 0 && !cfg.Server { - cmd.UI.Error("Expect mode cannot be enabled when server mode is not enabled") - return nil - } - - // Expect can only work when dev mode is off - if cfg.BootstrapExpect > 0 && cfg.DevMode { - cmd.UI.Error("Expect mode cannot be enabled when dev mode is enabled") - return nil - } - - // Expect & Bootstrap are mutually exclusive - if cfg.BootstrapExpect != 0 && cfg.Bootstrap { - cmd.UI.Error("Bootstrap cannot be provided with an expected server count") - return nil - } - - if ipaddr.IsAny(cfg.AdvertiseAddr) { - cmd.UI.Error("Advertise address cannot be " + cfg.AdvertiseAddr) - return nil - } - - if ipaddr.IsAny(cfg.AdvertiseAddrWan) { - cmd.UI.Error("Advertise WAN address cannot be " + cfg.AdvertiseAddrWan) - return nil - } - - if cfg.Server && cfg.Segment != "" { - cmd.UI.Error("Segment option can only be set on clients") - return nil - } - - if !cfg.Server && len(cfg.Segments) > 0 { - cmd.UI.Error("Segments can only be configured on servers") - return nil - } - - // patch deprecated retry-join-{gce,azure,ec2)-* parameters - // into -retry-join and issue warning. - // todo(fs): this should really be in DecodeConfig where it can be tested - if !reflect.DeepEqual(cfg.DeprecatedRetryJoinEC2, agent.RetryJoinEC2{}) { - m := discover.Config{ - "provider": "aws", - "region": cfg.DeprecatedRetryJoinEC2.Region, - "tag_key": cfg.DeprecatedRetryJoinEC2.TagKey, - "tag_value": cfg.DeprecatedRetryJoinEC2.TagValue, - "access_key_id": cfg.DeprecatedRetryJoinEC2.AccessKeyID, - "secret_access_key": cfg.DeprecatedRetryJoinEC2.SecretAccessKey, - } - cfg.RetryJoin = append(cfg.RetryJoin, m.String()) - cfg.DeprecatedRetryJoinEC2 = agent.RetryJoinEC2{} - - // redact m before output - if m["access_key_id"] != "" { - m["access_key_id"] = "hidden" - } - if m["secret_access_key"] != "" { - m["secret_access_key"] = "hidden" - } - - cmd.UI.Warn(fmt.Sprintf("==> DEPRECATION: retry_join_ec2 is deprecated. "+ - "Please add %q to retry_join\n", m)) - } - if !reflect.DeepEqual(cfg.DeprecatedRetryJoinAzure, agent.RetryJoinAzure{}) { - m := discover.Config{ - "provider": "azure", - "tag_name": cfg.DeprecatedRetryJoinAzure.TagName, - "tag_value": cfg.DeprecatedRetryJoinAzure.TagValue, - "subscription_id": cfg.DeprecatedRetryJoinAzure.SubscriptionID, - "tenant_id": cfg.DeprecatedRetryJoinAzure.TenantID, - "client_id": cfg.DeprecatedRetryJoinAzure.ClientID, - "secret_access_key": cfg.DeprecatedRetryJoinAzure.SecretAccessKey, - } - cfg.RetryJoin = append(cfg.RetryJoin, m.String()) - cfg.DeprecatedRetryJoinAzure = agent.RetryJoinAzure{} - - // redact m before output - if m["subscription_id"] != "" { - m["subscription_id"] = "hidden" - } - if m["tenant_id"] != "" { - m["tenant_id"] = "hidden" - } - if m["client_id"] != "" { - m["client_id"] = "hidden" - } - if m["secret_access_key"] != "" { - m["secret_access_key"] = "hidden" - } - - cmd.UI.Warn(fmt.Sprintf("==> DEPRECATION: retry_join_azure is deprecated. "+ - "Please add %q to retry_join\n", m)) - } - if !reflect.DeepEqual(cfg.DeprecatedRetryJoinGCE, agent.RetryJoinGCE{}) { - m := discover.Config{ - "provider": "gce", - "project_name": cfg.DeprecatedRetryJoinGCE.ProjectName, - "zone_pattern": cfg.DeprecatedRetryJoinGCE.ZonePattern, - "tag_value": cfg.DeprecatedRetryJoinGCE.TagValue, - "credentials_file": cfg.DeprecatedRetryJoinGCE.CredentialsFile, - } - cfg.RetryJoin = append(cfg.RetryJoin, m.String()) - cfg.DeprecatedRetryJoinGCE = agent.RetryJoinGCE{} - - // redact m before output - if m["credentials_file"] != "" { - m["credentials_file"] = "hidden" - } - - cmd.UI.Warn(fmt.Sprintf("==> DEPRECATION: retry_join_gce is deprecated. "+ - "Please add %q to retry_join\n", m)) - } - - // Compile all the watches - for _, params := range cfg.Watches { - // Parse the watches, excluding the handler - wp, err := watch.ParseExempt(params, []string{"handler"}) - if err != nil { - cmd.UI.Error(fmt.Sprintf("Failed to parse watch (%#v): %v", params, err)) - return nil - } - - // Get the handler - h := wp.Exempt["handler"] - if _, ok := h.(string); h == nil || !ok { - cmd.UI.Error("Watch handler must be a string") - return nil - } - - // Store the watch plan - cfg.WatchPlans = append(cfg.WatchPlans, wp) - } - - // Warn if we are in expect mode - if cfg.BootstrapExpect == 1 { - cmd.UI.Error("WARNING: BootstrapExpect Mode is specified as 1; this is the same as Bootstrap mode.") - cfg.BootstrapExpect = 0 - cfg.Bootstrap = true - } else if cfg.BootstrapExpect > 0 { - cmd.UI.Error(fmt.Sprintf("WARNING: Expect Mode enabled, expecting %d servers", cfg.BootstrapExpect)) - } - - // Warn if we are expecting an even number of servers - if cfg.BootstrapExpect != 0 && cfg.BootstrapExpect%2 == 0 { - if cfg.BootstrapExpect == 2 { - cmd.UI.Error("WARNING: A cluster with 2 servers will provide no failure tolerance. See https://www.consul.io/docs/internals/consensus.html#deployment-table") - } else { - cmd.UI.Error("WARNING: A cluster with an even number of servers does not achieve optimum fault tolerance. See https://www.consul.io/docs/internals/consensus.html#deployment-table") - } - } - - // Warn if we are in bootstrap mode - if cfg.Bootstrap { - cmd.UI.Error("WARNING: Bootstrap mode enabled! Do not enable unless necessary") - } - - // It doesn't make sense to include both UI options. - if cfg.EnableUI == true && cfg.UIDir != "" { - cmd.UI.Error("Both the ui and ui-dir flags were specified, please provide only one") - cmd.UI.Error("If trying to use your own web UI resources, use the ui-dir flag") - cmd.UI.Error("If using Consul version 0.7.0 or later, the web UI is included in the binary so use ui to enable it") - return nil - } - - // Set the version info - cfg.Revision = cmd.Revision - cfg.Version = cmd.Version - cfg.VersionPrerelease = cmd.VersionPrerelease - - if err := cfg.ResolveTmplAddrs(); err != nil { - cmd.UI.Error(fmt.Sprintf("Failed to parse config: %v", err)) - return nil - } - - if err := cfg.SetupTaggedAndAdvertiseAddrs(); err != nil { - cmd.UI.Error(fmt.Sprintf("Failed to set up tagged and advertise addresses: %v", err)) - return nil - } - - return cfg + return &cfg } // checkpointResults is used to handler periodic results from our update checker @@ -568,7 +93,7 @@ func (cmd *AgentCommand) checkpointResults(results *checkpoint.CheckResponse, er } } -func (cmd *AgentCommand) startupUpdateCheck(config *agent.Config) { +func (cmd *AgentCommand) startupUpdateCheck(config *config.RuntimeConfig) { version := config.Version if config.VersionPrerelease != "" { version += fmt.Sprintf("-%s", config.VersionPrerelease) @@ -592,13 +117,13 @@ func (cmd *AgentCommand) startupUpdateCheck(config *agent.Config) { } // startupJoin is invoked to handle any joins specified to take place at start time -func (cmd *AgentCommand) startupJoin(agent *agent.Agent, cfg *agent.Config) error { - if len(cfg.StartJoin) == 0 { +func (cmd *AgentCommand) startupJoin(agent *agent.Agent, cfg *config.RuntimeConfig) error { + if len(cfg.StartJoinAddrsLAN) == 0 { return nil } cmd.UI.Output("Joining cluster...") - n, err := agent.JoinLAN(cfg.StartJoin) + n, err := agent.JoinLAN(cfg.StartJoinAddrsLAN) if err != nil { return err } @@ -608,13 +133,13 @@ func (cmd *AgentCommand) startupJoin(agent *agent.Agent, cfg *agent.Config) erro } // startupJoinWan is invoked to handle any joins -wan specified to take place at start time -func (cmd *AgentCommand) startupJoinWan(agent *agent.Agent, cfg *agent.Config) error { - if len(cfg.StartJoinWan) == 0 { +func (cmd *AgentCommand) startupJoinWan(agent *agent.Agent, cfg *config.RuntimeConfig) error { + if len(cfg.StartJoinAddrsWAN) == 0 { return nil } cmd.UI.Output("Joining -wan cluster...") - n, err := agent.JoinWAN(cfg.StartJoinWan) + n, err := agent.JoinWAN(cfg.StartJoinAddrsWAN) if err != nil { return err } @@ -623,51 +148,51 @@ func (cmd *AgentCommand) startupJoinWan(agent *agent.Agent, cfg *agent.Config) e return nil } -func statsiteSink(config *agent.Config, hostname string) (metrics.MetricSink, error) { - if config.Telemetry.StatsiteAddr == "" { +func statsiteSink(config *config.RuntimeConfig, hostname string) (metrics.MetricSink, error) { + if config.TelemetryStatsiteAddr == "" { return nil, nil } - return metrics.NewStatsiteSink(config.Telemetry.StatsiteAddr) + return metrics.NewStatsiteSink(config.TelemetryStatsiteAddr) } -func statsdSink(config *agent.Config, hostname string) (metrics.MetricSink, error) { - if config.Telemetry.StatsdAddr == "" { +func statsdSink(config *config.RuntimeConfig, hostname string) (metrics.MetricSink, error) { + if config.TelemetryStatsdAddr == "" { return nil, nil } - return metrics.NewStatsdSink(config.Telemetry.StatsdAddr) + return metrics.NewStatsdSink(config.TelemetryStatsdAddr) } -func dogstatdSink(config *agent.Config, hostname string) (metrics.MetricSink, error) { - if config.Telemetry.DogStatsdAddr == "" { +func dogstatdSink(config *config.RuntimeConfig, hostname string) (metrics.MetricSink, error) { + if config.TelemetryDogstatsdAddr == "" { return nil, nil } - sink, err := datadog.NewDogStatsdSink(config.Telemetry.DogStatsdAddr, hostname) + sink, err := datadog.NewDogStatsdSink(config.TelemetryDogstatsdAddr, hostname) if err != nil { return nil, err } - sink.SetTags(config.Telemetry.DogStatsdTags) + sink.SetTags(config.TelemetryDogstatsdTags) return sink, nil } -func circonusSink(config *agent.Config, hostname string) (metrics.MetricSink, error) { - if config.Telemetry.CirconusAPIToken == "" && config.Telemetry.CirconusCheckSubmissionURL == "" { +func circonusSink(config *config.RuntimeConfig, hostname string) (metrics.MetricSink, error) { + if config.TelemetryCirconusAPIToken == "" && config.TelemetryCirconusSubmissionURL == "" { return nil, nil } cfg := &circonus.Config{} - cfg.Interval = config.Telemetry.CirconusSubmissionInterval - cfg.CheckManager.API.TokenKey = config.Telemetry.CirconusAPIToken - cfg.CheckManager.API.TokenApp = config.Telemetry.CirconusAPIApp - cfg.CheckManager.API.URL = config.Telemetry.CirconusAPIURL - cfg.CheckManager.Check.SubmissionURL = config.Telemetry.CirconusCheckSubmissionURL - cfg.CheckManager.Check.ID = config.Telemetry.CirconusCheckID - cfg.CheckManager.Check.ForceMetricActivation = config.Telemetry.CirconusCheckForceMetricActivation - cfg.CheckManager.Check.InstanceID = config.Telemetry.CirconusCheckInstanceID - cfg.CheckManager.Check.SearchTag = config.Telemetry.CirconusCheckSearchTag - cfg.CheckManager.Check.DisplayName = config.Telemetry.CirconusCheckDisplayName - cfg.CheckManager.Check.Tags = config.Telemetry.CirconusCheckTags - cfg.CheckManager.Broker.ID = config.Telemetry.CirconusBrokerID - cfg.CheckManager.Broker.SelectTag = config.Telemetry.CirconusBrokerSelectTag + cfg.Interval = config.TelemetryCirconusSubmissionInterval + cfg.CheckManager.API.TokenKey = config.TelemetryCirconusAPIToken + cfg.CheckManager.API.TokenApp = config.TelemetryCirconusAPIApp + cfg.CheckManager.API.URL = config.TelemetryCirconusAPIURL + cfg.CheckManager.Check.SubmissionURL = config.TelemetryCirconusSubmissionURL + cfg.CheckManager.Check.ID = config.TelemetryCirconusCheckID + cfg.CheckManager.Check.ForceMetricActivation = config.TelemetryCirconusCheckForceMetricActivation + cfg.CheckManager.Check.InstanceID = config.TelemetryCirconusCheckInstanceID + cfg.CheckManager.Check.SearchTag = config.TelemetryCirconusCheckSearchTag + cfg.CheckManager.Check.DisplayName = config.TelemetryCirconusCheckDisplayName + cfg.CheckManager.Check.Tags = config.TelemetryCirconusCheckTags + cfg.CheckManager.Broker.ID = config.TelemetryCirconusBrokerID + cfg.CheckManager.Broker.SelectTag = config.TelemetryCirconusBrokerSelectTag if cfg.CheckManager.Check.DisplayName == "" { cfg.CheckManager.Check.DisplayName = "Consul" @@ -689,19 +214,19 @@ func circonusSink(config *agent.Config, hostname string) (metrics.MetricSink, er return sink, nil } -func startupTelemetry(config *agent.Config) (*metrics.InmemSink, error) { +func startupTelemetry(conf *config.RuntimeConfig) (*metrics.InmemSink, error) { // Setup telemetry // Aggregate on 10 second intervals for 1 minute. Expose the // metrics over stderr when there is a SIGUSR1 received. memSink := metrics.NewInmemSink(10*time.Second, time.Minute) metrics.DefaultInmemSignal(memSink) - metricsConf := metrics.DefaultConfig(config.Telemetry.StatsitePrefix) - metricsConf.EnableHostname = !config.Telemetry.DisableHostname - metricsConf.FilterDefault = *config.Telemetry.FilterDefault + metricsConf := metrics.DefaultConfig(conf.TelemetryStatsitePrefix) + metricsConf.EnableHostname = !conf.TelemetryDisableHostname + metricsConf.FilterDefault = conf.TelemetryFilterDefault var sinks metrics.FanoutSink - addSink := func(name string, fn func(*agent.Config, string) (metrics.MetricSink, error)) error { - s, err := fn(config, metricsConf.HostName) + addSink := func(name string, fn func(*config.RuntimeConfig, string) (metrics.MetricSink, error)) error { + s, err := fn(conf, metricsConf.HostName) if err != nil { return err } @@ -737,7 +262,7 @@ func startupTelemetry(config *agent.Config) (*metrics.InmemSink, error) { func (cmd *AgentCommand) Run(args []string) int { code := cmd.run(args) if cmd.logger != nil { - cmd.logger.Println("[INFO] Exit code: ", code) + cmd.logger.Println("[INFO] Exit code:", code) } return code } @@ -814,8 +339,8 @@ func (cmd *AgentCommand) run(args []string) int { // Let the agent know we've finished registration agent.StartSync() - segment := config.Segment - if config.Server { + segment := config.SegmentName + if config.ServerMode { segment = "" } @@ -824,11 +349,11 @@ func (cmd *AgentCommand) run(args []string) int { cmd.UI.Info(fmt.Sprintf(" Node ID: '%s'", config.NodeID)) cmd.UI.Info(fmt.Sprintf(" Node name: '%s'", config.NodeName)) cmd.UI.Info(fmt.Sprintf(" Datacenter: '%s' (Segment: '%s')", config.Datacenter, segment)) - cmd.UI.Info(fmt.Sprintf(" Server: %v (Bootstrap: %v)", config.Server, config.Bootstrap)) - cmd.UI.Info(fmt.Sprintf(" Client Addr: %v (HTTP: %d, HTTPS: %d, DNS: %d)", config.ClientAddr, - config.Ports.HTTP, config.Ports.HTTPS, config.Ports.DNS)) - cmd.UI.Info(fmt.Sprintf(" Cluster Addr: %v (LAN: %d, WAN: %d)", config.AdvertiseAddr, - config.Ports.SerfLan, config.Ports.SerfWan)) + cmd.UI.Info(fmt.Sprintf(" Server: %v (Bootstrap: %v)", config.ServerMode, config.Bootstrap)) + cmd.UI.Info(fmt.Sprintf(" Client Addr: %v (HTTP: %d, HTTPS: %d, DNS: %d)", config.ClientAddrs, + config.HTTPPort, config.HTTPSPort, config.DNSPort)) + cmd.UI.Info(fmt.Sprintf(" Cluster Addr: %v (LAN: %d, WAN: %d)", config.AdvertiseAddrLAN, + config.SerfPortLAN, config.SerfPortWAN)) cmd.UI.Info(fmt.Sprintf(" Encrypt: Gossip: %v, TLS-Outgoing: %v, TLS-Incoming: %v", agent.GossipEncrypted(), config.VerifyOutgoing, config.VerifyIncoming)) @@ -883,7 +408,7 @@ func (cmd *AgentCommand) run(args []string) int { default: cmd.logger.Println("[INFO] Caught signal: ", sig) - graceful := (sig == os.Interrupt && !(*config.SkipLeaveOnInt)) || (sig == syscall.SIGTERM && (*config.LeaveOnTerm)) + graceful := (sig == os.Interrupt && !(config.SkipLeaveOnInt)) || (sig == syscall.SIGTERM && (config.LeaveOnTerm)) if !graceful { cmd.logger.Println("[INFO] Graceful shutdown disabled. Exiting") return 1 @@ -916,7 +441,7 @@ func (cmd *AgentCommand) run(args []string) int { } // handleReload is invoked when we should reload our configs, e.g. SIGHUP -func (cmd *AgentCommand) handleReload(agent *agent.Agent, cfg *agent.Config) (*agent.Config, error) { +func (cmd *AgentCommand) handleReload(agent *agent.Agent, cfg *config.RuntimeConfig) (*config.RuntimeConfig, error) { cmd.logger.Println("[INFO] Reloading configuration...") var errs error newCfg := cmd.readConfig() diff --git a/command/agent_test.go b/command/agent_test.go index 414fef968..431fff7e6 100644 --- a/command/agent_test.go +++ b/command/agent_test.go @@ -6,7 +6,6 @@ import ( "os" "os/exec" "path/filepath" - "reflect" "strings" "testing" @@ -56,33 +55,45 @@ func TestValidDatacenter(t *testing.T) { // TestConfigFail should test command line flags that lead to an immediate error. func TestConfigFail(t *testing.T) { t.Parallel() + + dataDir := testutil.TempDir(t, "consul") + defer os.RemoveAll(dataDir) + tests := []struct { args []string out string }{ { - args: []string{"agent", "-server", "-data-dir", "foo", "-advertise", "0.0.0.0"}, - out: "==> Advertise address cannot be 0.0.0.0\n", + args: []string{"agent", "-server", "-bind=10.0.0.1", "-datacenter="}, + out: "==> datacenter cannot be empty\n", }, { - args: []string{"agent", "-server", "-data-dir", "foo", "-advertise", "::"}, - out: "==> Advertise address cannot be ::\n", + args: []string{"agent", "-server", "-bind=10.0.0.1"}, + out: "==> data_dir cannot be empty\n", }, { - args: []string{"agent", "-server", "-data-dir", "foo", "-advertise", "[::]"}, - out: "==> Advertise address cannot be [::]\n", + args: []string{"agent", "-server", "-data-dir", dataDir, "-advertise", "0.0.0.0", "-bind", "10.0.0.1"}, + out: "==> Advertise address cannot be 0.0.0.0, :: or [::]\n", }, { - args: []string{"agent", "-server", "-data-dir", "foo", "-advertise-wan", "0.0.0.0"}, - out: "==> Advertise WAN address cannot be 0.0.0.0\n", + args: []string{"agent", "-server", "-data-dir", dataDir, "-advertise", "::", "-bind", "10.0.0.1"}, + out: "==> Advertise address cannot be 0.0.0.0, :: or [::]\n", }, { - args: []string{"agent", "-server", "-data-dir", "foo", "-advertise-wan", "::"}, - out: "==> Advertise WAN address cannot be ::\n", + args: []string{"agent", "-server", "-data-dir", dataDir, "-advertise", "[::]", "-bind", "10.0.0.1"}, + out: "==> Advertise address cannot be 0.0.0.0, :: or [::]\n", }, { - args: []string{"agent", "-server", "-data-dir", "foo", "-advertise-wan", "[::]"}, - out: "==> Advertise WAN address cannot be [::]\n", + args: []string{"agent", "-server", "-data-dir", dataDir, "-advertise-wan", "0.0.0.0", "-bind", "10.0.0.1"}, + out: "==> Advertise WAN address cannot be 0.0.0.0, :: or [::]\n", + }, + { + args: []string{"agent", "-server", "-data-dir", dataDir, "-advertise-wan", "::", "-bind", "10.0.0.1"}, + out: "==> Advertise WAN address cannot be 0.0.0.0, :: or [::]\n", + }, + { + args: []string{"agent", "-server", "-data-dir", dataDir, "-advertise-wan", "[::]", "-bind", "10.0.0.1"}, + out: "==> Advertise WAN address cannot be 0.0.0.0, :: or [::]\n", }, } @@ -103,7 +114,7 @@ func TestConfigFail(t *testing.T) { func TestRetryJoin(t *testing.T) { t.Skip("fs: skipping tests that use cmd.Run until signal handling is fixed") t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), "") defer a.Shutdown() cfg2 := agent.TestConfig() @@ -124,25 +135,15 @@ func TestRetryJoin(t *testing.T) { BaseCommand: baseCommand(cli.NewMockUi()), } - serfAddr := fmt.Sprintf( - "%s:%d", - a.Config.BindAddr, - a.Config.Ports.SerfLan) - - serfWanAddr := fmt.Sprintf( - "%s:%d", - a.Config.BindAddr, - a.Config.Ports.SerfWan) - args := []string{ "-server", - "-bind", a.Config.BindAddr, + "-bind", a.Config.BindAddr.String(), "-data-dir", tmpDir, "-node", fmt.Sprintf(`"%s"`, cfg2.NodeName), - "-advertise", a.Config.BindAddr, - "-retry-join", serfAddr, + "-advertise", a.Config.BindAddr.String(), + "-retry-join", a.Config.SerfBindAddrLAN.String(), "-retry-interval", "1s", - "-retry-join-wan", serfWanAddr, + "-retry-join-wan", a.Config.SerfBindAddrWAN.String(), "-retry-interval-wan", "1s", } @@ -162,181 +163,6 @@ func TestRetryJoin(t *testing.T) { }) } -func TestReadCliConfig(t *testing.T) { - t.Parallel() - tmpDir := testutil.TempDir(t, "consul") - defer os.RemoveAll(tmpDir) - - shutdownCh := make(chan struct{}) - defer close(shutdownCh) - - // Test config parse - { - cmd := &AgentCommand{ - args: []string{ - "-data-dir", tmpDir, - "-node", `"a"`, - "-bind", "1.2.3.4", - "-advertise-wan", "1.2.3.4", - "-serf-wan-bind", "4.3.2.1", - "-serf-lan-bind", "4.3.2.2", - "-node-meta", "somekey:somevalue", - }, - ShutdownCh: shutdownCh, - BaseCommand: baseCommand(cli.NewMockUi()), - } - - config := cmd.readConfig() - if config.AdvertiseAddrWan != "1.2.3.4" { - t.Fatalf("expected -advertise-addr-wan 1.2.3.4 got %s", config.AdvertiseAddrWan) - } - if config.SerfWanBindAddr != "4.3.2.1" { - t.Fatalf("expected -serf-wan-bind 4.3.2.1 got %s", config.SerfWanBindAddr) - } - if config.SerfLanBindAddr != "4.3.2.2" { - t.Fatalf("expected -serf-lan-bind 4.3.2.2 got %s", config.SerfLanBindAddr) - } - expected := map[string]string{ - "somekey": "somevalue", - } - if !reflect.DeepEqual(config.Meta, expected) { - t.Fatalf("bad: %v %v", config.Meta, expected) - } - } - - // Test multiple node meta flags - { - cmd := &AgentCommand{ - args: []string{ - "-data-dir", tmpDir, - "-node-meta", "somekey:somevalue", - "-node-meta", "otherkey:othervalue", - "-bind", "1.2.3.4", - }, - ShutdownCh: shutdownCh, - BaseCommand: baseCommand(cli.NewMockUi()), - } - config := cmd.readConfig() - expected := map[string]string{ - "somekey": "somevalue", - "otherkey": "othervalue", - } - if !reflect.DeepEqual(config.Meta, expected) { - t.Fatalf("bad: %v %v", config.Meta, expected) - } - } - - // Test LeaveOnTerm and SkipLeaveOnInt defaults for server mode - { - ui := cli.NewMockUi() - cmd := &AgentCommand{ - args: []string{ - "-node", `"server1"`, - "-server", - "-data-dir", tmpDir, - "-bind", "1.2.3.4", - }, - ShutdownCh: shutdownCh, - BaseCommand: baseCommand(ui), - } - - config := cmd.readConfig() - if config == nil { - t.Fatalf(`Expected non-nil config object: %s`, ui.ErrorWriter.String()) - } - if config.Server != true { - t.Errorf(`Expected -server to be true`) - } - if (*config.LeaveOnTerm) != false { - t.Errorf(`Expected LeaveOnTerm to be false in server mode`) - } - if (*config.SkipLeaveOnInt) != true { - t.Errorf(`Expected SkipLeaveOnInt to be true in server mode`) - } - } - - // Test LeaveOnTerm and SkipLeaveOnInt defaults for client mode - { - ui := cli.NewMockUi() - cmd := &AgentCommand{ - args: []string{ - "-data-dir", tmpDir, - "-node", `"client"`, - "-bind", "1.2.3.4", - }, - ShutdownCh: shutdownCh, - BaseCommand: baseCommand(ui), - } - - config := cmd.readConfig() - if config == nil { - t.Fatalf(`Expected non-nil config object: %s`, ui.ErrorWriter.String()) - } - if config.Server != false { - t.Errorf(`Expected server to be false`) - } - if (*config.LeaveOnTerm) != true { - t.Errorf(`Expected LeaveOnTerm to be true in client mode`) - } - if *config.SkipLeaveOnInt != false { - t.Errorf(`Expected SkipLeaveOnInt to be false in client mode`) - } - } - - // Test empty node name - { - cmd := &AgentCommand{ - args: []string{"-node", `""`}, - ShutdownCh: shutdownCh, - BaseCommand: baseCommand(cli.NewMockUi()), - } - - config := cmd.readConfig() - if config != nil { - t.Errorf(`Expected -node="" to fail`) - } - } -} - -func TestAgent_HostBasedIDs(t *testing.T) { - t.Parallel() - tmpDir := testutil.TempDir(t, "consul") - defer os.RemoveAll(tmpDir) - - // Host-based IDs are disabled by default. - { - cmd := &AgentCommand{ - args: []string{ - "-data-dir", tmpDir, - "-bind", "127.0.0.1", - }, - BaseCommand: baseCommand(cli.NewMockUi()), - } - - config := cmd.readConfig() - if *config.DisableHostNodeID != true { - t.Fatalf("expected host-based node IDs to be disabled") - } - } - - // Try enabling host-based IDs. - { - cmd := &AgentCommand{ - args: []string{ - "-data-dir", tmpDir, - "-disable-host-node-id=false", - "-bind", "127.0.0.1", - }, - BaseCommand: baseCommand(cli.NewMockUi()), - } - - config := cmd.readConfig() - if *config.DisableHostNodeID != false { - t.Fatalf("expected host-based node IDs to be enabled") - } - } -} - func TestRetryJoinFail(t *testing.T) { t.Skip("fs: skipping tests that use cmd.Run until signal handling is fixed") t.Parallel() @@ -352,12 +178,10 @@ func TestRetryJoinFail(t *testing.T) { BaseCommand: baseCommand(cli.NewMockUi()), } - serfAddr := fmt.Sprintf("%s:%d", cfg.BindAddr, cfg.Ports.SerfLan) - args := []string{ - "-bind", cfg.BindAddr, + "-bind", cfg.BindAddr.String(), "-data-dir", tmpDir, - "-retry-join", serfAddr, + "-retry-join", cfg.SerfBindAddrLAN.String(), "-retry-max", "1", "-retry-interval", "10ms", } @@ -382,13 +206,11 @@ func TestRetryJoinWanFail(t *testing.T) { BaseCommand: baseCommand(cli.NewMockUi()), } - serfAddr := fmt.Sprintf("%s:%d", cfg.BindAddr, cfg.Ports.SerfWan) - args := []string{ "-server", - "-bind", cfg.BindAddr, + "-bind", cfg.BindAddr.String(), "-data-dir", tmpDir, - "-retry-join-wan", serfAddr, + "-retry-join-wan", cfg.SerfBindAddrWAN.String(), "-retry-max-wan", "1", "-retry-interval-wan", "10ms", } @@ -410,7 +232,7 @@ func TestProtectDataDir(t *testing.T) { cfgFile := testutil.TempFile(t, "consul") defer os.Remove(cfgFile.Name()) - content := fmt.Sprintf(`{"server": true, "data_dir": "%s"}`, dir) + content := fmt.Sprintf(`{"server": true, "bind_addr" : "10.0.0.1", "data_dir": "%s"}`, dir) _, err := cfgFile.Write([]byte(content)) if err != nil { t.Fatalf("err: %v", err) @@ -443,7 +265,7 @@ func TestBadDataDirPermissions(t *testing.T) { ui := cli.NewMockUi() cmd := &AgentCommand{ BaseCommand: baseCommand(ui), - args: []string{"-data-dir=" + dataDir, "-server=true"}, + args: []string{"-data-dir=" + dataDir, "-server=true", "-bind=10.0.0.1"}, } if conf := cmd.readConfig(); conf != nil { t.Fatalf("Should fail with bad data directory permissions") diff --git a/command/catalog_list_datacenters_test.go b/command/catalog_list_datacenters_test.go index da28e76e5..821f6449e 100644 --- a/command/catalog_list_datacenters_test.go +++ b/command/catalog_list_datacenters_test.go @@ -60,7 +60,7 @@ func TestCatalogListDatacentersCommand_Validation(t *testing.T) { func TestCatalogListDatacentersCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testCatalogListDatacentersCommand(t) diff --git a/command/catalog_list_nodes_test.go b/command/catalog_list_nodes_test.go index 43e1068c4..ae3469219 100644 --- a/command/catalog_list_nodes_test.go +++ b/command/catalog_list_nodes_test.go @@ -60,7 +60,7 @@ func TestCatalogListNodesCommand_Validation(t *testing.T) { func TestCatalogListNodesCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() t.Run("simple", func(t *testing.T) { diff --git a/command/catalog_list_services_test.go b/command/catalog_list_services_test.go index 23eca94b2..9c4e90972 100644 --- a/command/catalog_list_services_test.go +++ b/command/catalog_list_services_test.go @@ -61,7 +61,7 @@ func TestCatalogListServicesCommand_Validation(t *testing.T) { func TestCatalogListServicesCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() // Add another service with tags for testing diff --git a/command/configtest.go b/command/configtest.go index 6471d4fd1..3bde95846 100644 --- a/command/configtest.go +++ b/command/configtest.go @@ -4,7 +4,7 @@ import ( "fmt" "strings" - "github.com/hashicorp/consul/agent" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/configutil" ) @@ -52,11 +52,15 @@ func (c *ConfigTestCommand) Run(args []string) int { return 1 } - _, err := agent.ReadConfigPaths(configFiles) + b, err := config.NewBuilder(config.Flags{ConfigFiles: configFiles}) if err != nil { c.UI.Error(fmt.Sprintf("Config validation failed: %v", err.Error())) return 1 } + if _, err := b.BuildAndValidate(); err != nil { + c.UI.Error(fmt.Sprintf("Config validation failed: %v", err.Error())) + return 1 + } return 0 } diff --git a/command/configtest_test.go b/command/configtest_test.go index a1dbf857d..335489d7e 100644 --- a/command/configtest_test.go +++ b/command/configtest_test.go @@ -41,29 +41,13 @@ func TestConfigTestCommandFailOnEmptyFile(t *testing.T) { } } -func TestConfigTestCommandSucceedOnEmptyDir(t *testing.T) { - t.Parallel() - td := testutil.TempDir(t, "consul") - defer os.RemoveAll(td) - - ui, cmd := testConfigTestCommand(t) - - args := []string{ - "-config-dir", td, - } - - if code := cmd.Run(args); code != 0 { - t.Fatalf("bad: %d, %s", code, ui.ErrorWriter.String()) - } -} - func TestConfigTestCommandSucceedOnMinimalConfigFile(t *testing.T) { t.Parallel() td := testutil.TempDir(t, "consul") defer os.RemoveAll(td) fp := filepath.Join(td, "config.json") - err := ioutil.WriteFile(fp, []byte(`{}`), 0644) + err := ioutil.WriteFile(fp, []byte(`{"bind_addr":"10.0.0.1", "data_dir": "`+td+`"}`), 0644) if err != nil { t.Fatalf("err: %s", err) } @@ -84,28 +68,7 @@ func TestConfigTestCommandSucceedOnMinimalConfigDir(t *testing.T) { td := testutil.TempDir(t, "consul") defer os.RemoveAll(td) - err := ioutil.WriteFile(filepath.Join(td, "config.json"), []byte(`{}`), 0644) - if err != nil { - t.Fatalf("err: %s", err) - } - - _, cmd := testConfigTestCommand(t) - - args := []string{ - "-config-dir", td, - } - - if code := cmd.Run(args); code != 0 { - t.Fatalf("bad: %d", code) - } -} - -func TestConfigTestCommandSucceedOnConfigDirWithEmptyFile(t *testing.T) { - t.Parallel() - td := testutil.TempDir(t, "consul") - defer os.RemoveAll(td) - - err := ioutil.WriteFile(filepath.Join(td, "config.json"), []byte{}, 0644) + err := ioutil.WriteFile(filepath.Join(td, "config.json"), []byte(`{"bind_addr":"10.0.0.1", "data_dir": "`+td+`"}`), 0644) if err != nil { t.Fatalf("err: %s", err) } diff --git a/command/event_test.go b/command/event_test.go index 478a88fda..892bf5457 100644 --- a/command/event_test.go +++ b/command/event_test.go @@ -15,7 +15,7 @@ func TestEventCommand_implements(t *testing.T) { func TestEventCommandRun(t *testing.T) { t.Parallel() - a1 := agent.NewTestAgent(t.Name(), nil) + a1 := agent.NewTestAgent(t.Name(), ``) defer a1.Shutdown() ui := cli.NewMockUi() diff --git a/command/exec_test.go b/command/exec_test.go index e4d71ac47..c7c807c26 100644 --- a/command/exec_test.go +++ b/command/exec_test.go @@ -1,7 +1,6 @@ package command import ( - "fmt" "strings" "testing" "time" @@ -29,9 +28,9 @@ func TestExecCommand_implements(t *testing.T) { func TestExecCommandRun(t *testing.T) { t.Parallel() - cfg := agent.TestConfig() - cfg.DisableRemoteExec = agent.Bool(false) - a := agent.NewTestAgent(t.Name(), cfg) + a := agent.NewTestAgent(t.Name(), ` + disable_remote_exec = false + `) defer a.Shutdown() ui, c := testExecCommand(t) @@ -49,20 +48,19 @@ func TestExecCommandRun(t *testing.T) { func TestExecCommandRun_CrossDC(t *testing.T) { t.Parallel() - cfg1 := agent.TestConfig() - cfg1.DisableRemoteExec = agent.Bool(false) - a1 := agent.NewTestAgent(t.Name(), cfg1) + a1 := agent.NewTestAgent(t.Name(), ` + disable_remote_exec = false + `) defer a1.Shutdown() - cfg2 := agent.TestConfig() - cfg2.Datacenter = "dc2" - cfg2.DisableRemoteExec = agent.Bool(false) - a2 := agent.NewTestAgent(t.Name(), cfg2) - defer a1.Shutdown() + a2 := agent.NewTestAgent(t.Name(), ` + datacenter = "dc2" + disable_remote_exec = false + `) + defer a2.Shutdown() // Join over the WAN - wanAddr := fmt.Sprintf("%s:%d", a1.Config.BindAddr, a1.Config.Ports.SerfWan) - _, err := a2.JoinWAN([]string{wanAddr}) + _, err := a2.JoinWAN([]string{a1.Config.SerfBindAddrWAN.String()}) if err != nil { t.Fatalf("err: %v", err) } @@ -127,9 +125,9 @@ func TestExecCommand_Validate(t *testing.T) { func TestExecCommand_Sessions(t *testing.T) { t.Parallel() - cfg := agent.TestConfig() - cfg.DisableRemoteExec = agent.Bool(false) - a := agent.NewTestAgent(t.Name(), cfg) + a := agent.NewTestAgent(t.Name(), ` + disable_remote_exec = false + `) defer a.Shutdown() client := a.Client() @@ -166,9 +164,9 @@ func TestExecCommand_Sessions(t *testing.T) { func TestExecCommand_Sessions_Foreign(t *testing.T) { t.Parallel() - cfg := agent.TestConfig() - cfg.DisableRemoteExec = agent.Bool(false) - a := agent.NewTestAgent(t.Name(), cfg) + a := agent.NewTestAgent(t.Name(), ` + disable_remote_exec = false + `) defer a.Shutdown() client := a.Client() @@ -216,9 +214,9 @@ func TestExecCommand_Sessions_Foreign(t *testing.T) { func TestExecCommand_UploadDestroy(t *testing.T) { t.Parallel() - cfg := agent.TestConfig() - cfg.DisableRemoteExec = agent.Bool(false) - a := agent.NewTestAgent(t.Name(), cfg) + a := agent.NewTestAgent(t.Name(), ` + disable_remote_exec = false + `) defer a.Shutdown() client := a.Client() @@ -271,9 +269,9 @@ func TestExecCommand_UploadDestroy(t *testing.T) { func TestExecCommand_StreamResults(t *testing.T) { t.Parallel() - cfg := agent.TestConfig() - cfg.DisableRemoteExec = agent.Bool(false) - a := agent.NewTestAgent(t.Name(), cfg) + a := agent.NewTestAgent(t.Name(), ` + disable_remote_exec = false + `) defer a.Shutdown() client := a.Client() diff --git a/command/force_leave_test.go b/command/force_leave_test.go index 0f1750dbf..14438becb 100644 --- a/command/force_leave_test.go +++ b/command/force_leave_test.go @@ -1,7 +1,6 @@ package command import ( - "fmt" "strings" "testing" @@ -28,13 +27,12 @@ func TestForceLeaveCommand_implements(t *testing.T) { func TestForceLeaveCommandRun(t *testing.T) { t.Parallel() - a1 := agent.NewTestAgent(t.Name(), nil) - a2 := agent.NewTestAgent(t.Name(), nil) + a1 := agent.NewTestAgent(t.Name(), ``) + a2 := agent.NewTestAgent(t.Name(), ``) defer a1.Shutdown() defer a2.Shutdown() - addr := fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan) - _, err := a1.JoinLAN([]string{addr}) + _, err := a2.JoinLAN([]string{a1.Config.SerfBindAddrLAN.String()}) if err != nil { t.Fatalf("err: %s", err) } diff --git a/command/info_test.go b/command/info_test.go index 13918250d..483d000b6 100644 --- a/command/info_test.go +++ b/command/info_test.go @@ -15,7 +15,7 @@ func TestInfoCommand_implements(t *testing.T) { func TestInfoCommandRun(t *testing.T) { t.Parallel() - a1 := agent.NewTestAgent(t.Name(), nil) + a1 := agent.NewTestAgent(t.Name(), ``) defer a1.Shutdown() ui := cli.NewMockUi() diff --git a/command/join_test.go b/command/join_test.go index 3db03ac03..2fa77b642 100644 --- a/command/join_test.go +++ b/command/join_test.go @@ -1,7 +1,6 @@ package command import ( - "fmt" "strings" "testing" @@ -26,15 +25,15 @@ func TestJoinCommand_implements(t *testing.T) { func TestJoinCommandRun(t *testing.T) { t.Parallel() - a1 := agent.NewTestAgent(t.Name(), nil) - a2 := agent.NewTestAgent(t.Name(), nil) + a1 := agent.NewTestAgent(t.Name(), ``) + a2 := agent.NewTestAgent(t.Name(), ``) defer a1.Shutdown() defer a2.Shutdown() ui, c := testJoinCommand(t) args := []string{ "-http-addr=" + a1.HTTPAddr(), - fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfLan), + a2.Config.SerfBindAddrLAN.String(), } code := c.Run(args) @@ -49,8 +48,8 @@ func TestJoinCommandRun(t *testing.T) { func TestJoinCommandRun_wan(t *testing.T) { t.Parallel() - a1 := agent.NewTestAgent(t.Name(), nil) - a2 := agent.NewTestAgent(t.Name(), nil) + a1 := agent.NewTestAgent(t.Name(), ``) + a2 := agent.NewTestAgent(t.Name(), ``) defer a1.Shutdown() defer a2.Shutdown() @@ -58,7 +57,7 @@ func TestJoinCommandRun_wan(t *testing.T) { args := []string{ "-http-addr=" + a1.HTTPAddr(), "-wan", - fmt.Sprintf("127.0.0.1:%d", a2.Config.Ports.SerfWan), + a2.Config.SerfBindAddrWAN.String(), } code := c.Run(args) diff --git a/command/keyring_test.go b/command/keyring_test.go index 73c6f5c48..6746a719f 100644 --- a/command/keyring_test.go +++ b/command/keyring_test.go @@ -29,9 +29,9 @@ func TestKeyringCommandRun(t *testing.T) { key2 := "kZyFABeAmc64UMTrm9XuKA==" // Begin with a single key - cfg := agent.TestConfig() - cfg.EncryptKey = key1 - a1 := agent.NewTestAgent(t.Name(), cfg) + a1 := agent.NewTestAgent(t.Name(), ` + encrypt = "`+key1+`" + `) defer a1.Shutdown() // The LAN and WAN keyrings were initialized with key1 diff --git a/command/kv_delete_test.go b/command/kv_delete_test.go index dbe1991bf..aa6a21fd3 100644 --- a/command/kv_delete_test.go +++ b/command/kv_delete_test.go @@ -83,7 +83,7 @@ func TestKVDeleteCommand_Validation(t *testing.T) { func TestKVDeleteCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -119,7 +119,7 @@ func TestKVDeleteCommand_Run(t *testing.T) { func TestKVDeleteCommand_Recurse(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -162,7 +162,7 @@ func TestKVDeleteCommand_Recurse(t *testing.T) { func TestKVDeleteCommand_CAS(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/kv_export_test.go b/command/kv_export_test.go index fd9a6cb28..0bc2e2aa6 100644 --- a/command/kv_export_test.go +++ b/command/kv_export_test.go @@ -12,7 +12,7 @@ import ( func TestKVExportCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/kv_get_test.go b/command/kv_get_test.go index f3078c67a..d0de623c4 100644 --- a/command/kv_get_test.go +++ b/command/kv_get_test.go @@ -71,7 +71,7 @@ func TestKVGetCommand_Validation(t *testing.T) { func TestKVGetCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -104,7 +104,7 @@ func TestKVGetCommand_Run(t *testing.T) { func TestKVGetCommand_Missing(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() _, c := testKVGetCommand(t) @@ -122,7 +122,7 @@ func TestKVGetCommand_Missing(t *testing.T) { func TestKVGetCommand_Empty(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -150,7 +150,7 @@ func TestKVGetCommand_Empty(t *testing.T) { func TestKVGetCommand_Detailed(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -193,7 +193,7 @@ func TestKVGetCommand_Detailed(t *testing.T) { func TestKVGetCommand_Keys(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -227,7 +227,7 @@ func TestKVGetCommand_Keys(t *testing.T) { func TestKVGetCommand_Recurse(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -266,7 +266,7 @@ func TestKVGetCommand_Recurse(t *testing.T) { func TestKVGetCommand_RecurseBase64(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -306,7 +306,7 @@ func TestKVGetCommand_RecurseBase64(t *testing.T) { func TestKVGetCommand_DetailedBase64(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/kv_import_test.go b/command/kv_import_test.go index 235639523..86acb79a7 100644 --- a/command/kv_import_test.go +++ b/command/kv_import_test.go @@ -10,7 +10,7 @@ import ( func TestKVImportCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/kv_put_test.go b/command/kv_put_test.go index 3f105adf2..e9640caf1 100644 --- a/command/kv_put_test.go +++ b/command/kv_put_test.go @@ -88,7 +88,7 @@ func TestKVPutCommand_Validation(t *testing.T) { func TestKVPutCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -116,7 +116,7 @@ func TestKVPutCommand_Run(t *testing.T) { func TestKVPutCommand_RunEmptyDataQuoted(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -144,7 +144,7 @@ func TestKVPutCommand_RunEmptyDataQuoted(t *testing.T) { func TestKVPutCommand_RunBase64(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -180,7 +180,7 @@ func TestKVPutCommand_RunBase64(t *testing.T) { func TestKVPutCommand_File(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -233,7 +233,7 @@ func TestKVPutCommand_FileNoExist(t *testing.T) { func TestKVPutCommand_Stdin(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -269,7 +269,7 @@ func TestKVPutCommand_Stdin(t *testing.T) { func TestKVPutCommand_NegativeVal(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -297,7 +297,7 @@ func TestKVPutCommand_NegativeVal(t *testing.T) { func TestKVPutCommand_Flags(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() @@ -326,7 +326,7 @@ func TestKVPutCommand_Flags(t *testing.T) { func TestKVPutCommand_CAS(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/leave_test.go b/command/leave_test.go index 61e0915f6..92bc0bbed 100644 --- a/command/leave_test.go +++ b/command/leave_test.go @@ -25,7 +25,7 @@ func TestLeaveCommand_implements(t *testing.T) { func TestLeaveCommandRun(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLeaveCommand(t) @@ -43,7 +43,7 @@ func TestLeaveCommandRun(t *testing.T) { func TestLeaveCommandFailOnNonFlagArgs(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() _, c := testLeaveCommand(t) diff --git a/command/lock_test.go b/command/lock_test.go index d6f028e84..0dd768b72 100644 --- a/command/lock_test.go +++ b/command/lock_test.go @@ -48,7 +48,7 @@ func TestLockCommand_BadArgs(t *testing.T) { func TestLockCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -70,7 +70,7 @@ func TestLockCommand_Run(t *testing.T) { func TestLockCommand_Try_Lock(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -101,7 +101,7 @@ func TestLockCommand_Try_Lock(t *testing.T) { func TestLockCommand_Try_Semaphore(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -132,7 +132,7 @@ func TestLockCommand_Try_Semaphore(t *testing.T) { func TestLockCommand_MonitorRetry_Lock_Default(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -164,7 +164,7 @@ func TestLockCommand_MonitorRetry_Lock_Default(t *testing.T) { func TestLockCommand_MonitorRetry_Semaphore_Default(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -196,7 +196,7 @@ func TestLockCommand_MonitorRetry_Semaphore_Default(t *testing.T) { func TestLockCommand_MonitorRetry_Lock_Arg(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -228,7 +228,7 @@ func TestLockCommand_MonitorRetry_Lock_Arg(t *testing.T) { func TestLockCommand_MonitorRetry_Semaphore_Arg(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testLockCommand(t) @@ -260,7 +260,7 @@ func TestLockCommand_MonitorRetry_Semaphore_Arg(t *testing.T) { func TestLockCommand_ChildExitCode(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() t.Run("clean exit", func(t *testing.T) { diff --git a/command/maint_test.go b/command/maint_test.go index 11b799507..495b8aee6 100644 --- a/command/maint_test.go +++ b/command/maint_test.go @@ -47,7 +47,7 @@ func TestMaintCommandRun_ConflictingArgs(t *testing.T) { func TestMaintCommandRun_NoArgs(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() // Register the service and put it into maintenance mode @@ -94,7 +94,7 @@ func TestMaintCommandRun_NoArgs(t *testing.T) { func TestMaintCommandRun_EnableNodeMaintenance(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMaintCommand(t) @@ -116,7 +116,7 @@ func TestMaintCommandRun_EnableNodeMaintenance(t *testing.T) { func TestMaintCommandRun_DisableNodeMaintenance(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMaintCommand(t) @@ -137,7 +137,7 @@ func TestMaintCommandRun_DisableNodeMaintenance(t *testing.T) { func TestMaintCommandRun_EnableServiceMaintenance(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() // Register the service @@ -169,7 +169,7 @@ func TestMaintCommandRun_EnableServiceMaintenance(t *testing.T) { func TestMaintCommandRun_DisableServiceMaintenance(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() // Register the service @@ -200,7 +200,7 @@ func TestMaintCommandRun_DisableServiceMaintenance(t *testing.T) { func TestMaintCommandRun_ServiceMaintenance_NoService(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMaintCommand(t) diff --git a/command/members_test.go b/command/members_test.go index e48bd124e..ae838f495 100644 --- a/command/members_test.go +++ b/command/members_test.go @@ -26,7 +26,7 @@ func TestMembersCommand_implements(t *testing.T) { func TestMembersCommandRun(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMembersCommand(t) @@ -55,7 +55,7 @@ func TestMembersCommandRun(t *testing.T) { func TestMembersCommandRun_WAN(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMembersCommand(t) @@ -66,14 +66,14 @@ func TestMembersCommandRun_WAN(t *testing.T) { t.Fatalf("bad: %d. %#v", code, ui.ErrorWriter.String()) } - if !strings.Contains(ui.OutputWriter.String(), fmt.Sprintf("%d", a.Config.Ports.SerfWan)) { + if !strings.Contains(ui.OutputWriter.String(), fmt.Sprintf("%d", a.Config.SerfPortWAN)) { t.Fatalf("bad: %#v", ui.OutputWriter.String()) } } func TestMembersCommandRun_statusFilter(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMembersCommand(t) @@ -94,7 +94,7 @@ func TestMembersCommandRun_statusFilter(t *testing.T) { func TestMembersCommandRun_statusFilter_failed(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui, c := testMembersCommand(t) diff --git a/command/operator_autopilot_get_test.go b/command/operator_autopilot_get_test.go index e2e4fba1d..e89f232c6 100644 --- a/command/operator_autopilot_get_test.go +++ b/command/operator_autopilot_get_test.go @@ -15,7 +15,7 @@ func TestOperator_Autopilot_Get_Implements(t *testing.T) { func TestOperator_Autopilot_Get(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui := cli.NewMockUi() diff --git a/command/operator_autopilot_set_test.go b/command/operator_autopilot_set_test.go index cc51e81fd..e73fae09d 100644 --- a/command/operator_autopilot_set_test.go +++ b/command/operator_autopilot_set_test.go @@ -17,7 +17,7 @@ func TestOperator_Autopilot_Set_Implements(t *testing.T) { func TestOperator_Autopilot_Set(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui := cli.NewMockUi() diff --git a/command/operator_raft_list_test.go b/command/operator_raft_list_test.go index c3d6d7b82..e43b820c0 100644 --- a/command/operator_raft_list_test.go +++ b/command/operator_raft_list_test.go @@ -16,11 +16,11 @@ func TestOperator_Raft_ListPeers_Implements(t *testing.T) { func TestOperator_Raft_ListPeers(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() expected := fmt.Sprintf("%s 127.0.0.1:%d 127.0.0.1:%d leader true 2", - a.Config.NodeName, a.Config.Ports.Server, a.Config.Ports.Server) + a.Config.NodeName, a.Config.ServerPort, a.Config.ServerPort) // Test the legacy mode with 'consul operator raft -list-peers' { diff --git a/command/operator_raft_remove_test.go b/command/operator_raft_remove_test.go index cc8a8d53d..c30cfa0f9 100644 --- a/command/operator_raft_remove_test.go +++ b/command/operator_raft_remove_test.go @@ -15,7 +15,7 @@ func TestOperator_Raft_RemovePeer_Implements(t *testing.T) { func TestOperator_Raft_RemovePeer(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() // Test the legacy mode with 'consul operator raft -remove-peer' diff --git a/command/reload_test.go b/command/reload_test.go index 942244df2..23e19125a 100644 --- a/command/reload_test.go +++ b/command/reload_test.go @@ -15,7 +15,7 @@ func TestReloadCommand_implements(t *testing.T) { func TestReloadCommandRun(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() // Setup a dummy response to errCh to simulate a successful reload diff --git a/command/rtt_test.go b/command/rtt_test.go index 76e3876f0..b2d978199 100644 --- a/command/rtt_test.go +++ b/command/rtt_test.go @@ -4,7 +4,6 @@ import ( "fmt" "strings" "testing" - "time" "github.com/hashicorp/consul/agent" "github.com/hashicorp/consul/agent/structs" @@ -55,10 +54,13 @@ func TestRTTCommand_Run_BadArgs(t *testing.T) { func TestRTTCommand_Run_LAN(t *testing.T) { t.Parallel() - updatePeriod := 10 * time.Millisecond - cfg := agent.TestConfig() - cfg.ConsulConfig.CoordinateUpdatePeriod = updatePeriod - a := agent.NewTestAgent(t.Name(), cfg) + a := agent.NewTestAgent(t.Name(), ` + consul = { + coordinate = { + update_period = "10ms" + } + } + `) defer a.Shutdown() // Inject some known coordinates. @@ -157,7 +159,7 @@ func TestRTTCommand_Run_LAN(t *testing.T) { func TestRTTCommand_Run_WAN(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() node := fmt.Sprintf("%s.%s", a.Config.NodeName, a.Config.Datacenter) diff --git a/command/snapshot_inspect_test.go b/command/snapshot_inspect_test.go index 1e501ec5e..d6dd4cc6c 100644 --- a/command/snapshot_inspect_test.go +++ b/command/snapshot_inspect_test.go @@ -73,7 +73,7 @@ func TestSnapshotInspectCommand_Validation(t *testing.T) { func TestSnapshotInspectCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/snapshot_restore_test.go b/command/snapshot_restore_test.go index d52dc9f89..f886cf441 100644 --- a/command/snapshot_restore_test.go +++ b/command/snapshot_restore_test.go @@ -73,7 +73,7 @@ func TestSnapshotRestoreCommand_Validation(t *testing.T) { func TestSnapshotRestoreCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/snapshot_save_test.go b/command/snapshot_save_test.go index 40d02ee4e..addc83c16 100644 --- a/command/snapshot_save_test.go +++ b/command/snapshot_save_test.go @@ -72,7 +72,7 @@ func TestSnapshotSaveCommand_Validation(t *testing.T) { func TestSnapshotSaveCommand_Run(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() client := a.Client() diff --git a/command/validate.go b/command/validate.go index 232d602be..1eec360d1 100644 --- a/command/validate.go +++ b/command/validate.go @@ -4,7 +4,7 @@ import ( "fmt" "strings" - "github.com/hashicorp/consul/agent" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/configutil" ) @@ -46,6 +46,7 @@ func (c *ValidateCommand) Run(args []string) int { c.BaseCommand.HideFlags("config-file", "config-dir") if err := c.BaseCommand.Parse(args); err != nil { + c.UI.Error(err.Error()) return 1 } @@ -58,11 +59,15 @@ func (c *ValidateCommand) Run(args []string) int { return 1 } - _, err := agent.ReadConfigPaths(configFiles) + b, err := config.NewBuilder(config.Flags{ConfigFiles: configFiles}) if err != nil { c.UI.Error(fmt.Sprintf("Config validation failed: %v", err.Error())) return 1 } + if _, err := b.BuildAndValidate(); err != nil { + c.UI.Error(fmt.Sprintf("Config validation failed: %v", err.Error())) + return 1 + } if !quiet { c.UI.Output("Configuration is valid!") diff --git a/command/validate_test.go b/command/validate_test.go index 8d428c91e..d49914b95 100644 --- a/command/validate_test.go +++ b/command/validate_test.go @@ -39,27 +39,12 @@ func TestValidateCommandFailOnEmptyFile(t *testing.T) { } } -func TestValidateCommandSucceedOnEmptyDir(t *testing.T) { - t.Parallel() - td := testutil.TempDir(t, "consul") - defer os.RemoveAll(td) - - ui, cmd := testValidateCommand(t) - - args := []string{td} - - if code := cmd.Run(args); code != 0 { - t.Fatalf("bad: %d, %s", code, ui.ErrorWriter.String()) - } -} - func TestValidateCommandSucceedOnMinimalConfigFile(t *testing.T) { - t.Parallel() td := testutil.TempDir(t, "consul") defer os.RemoveAll(td) fp := filepath.Join(td, "config.json") - err := ioutil.WriteFile(fp, []byte(`{}`), 0644) + err := ioutil.WriteFile(fp, []byte(`{"bind_addr":"10.0.0.1", "data_dir":"`+td+`"}`), 0644) if err != nil { t.Fatalf("err: %s", err) } @@ -74,30 +59,10 @@ func TestValidateCommandSucceedOnMinimalConfigFile(t *testing.T) { } func TestValidateCommandSucceedOnMinimalConfigDir(t *testing.T) { - t.Parallel() td := testutil.TempDir(t, "consul") defer os.RemoveAll(td) - err := ioutil.WriteFile(filepath.Join(td, "config.json"), []byte(`{}`), 0644) - if err != nil { - t.Fatalf("err: %s", err) - } - - _, cmd := testValidateCommand(t) - - args := []string{td} - - if code := cmd.Run(args); code != 0 { - t.Fatalf("bad: %d", code) - } -} - -func TestValidateCommandSucceedOnConfigDirWithEmptyFile(t *testing.T) { - t.Parallel() - td := testutil.TempDir(t, "consul") - defer os.RemoveAll(td) - - err := ioutil.WriteFile(filepath.Join(td, "config.json"), []byte{}, 0644) + err := ioutil.WriteFile(filepath.Join(td, "config.json"), []byte(`{"bind_addr":"10.0.0.1", "data_dir":"`+td+`"}`), 0644) if err != nil { t.Fatalf("err: %s", err) } @@ -116,6 +81,12 @@ func TestValidateCommandQuiet(t *testing.T) { td := testutil.TempDir(t, "consul") defer os.RemoveAll(td) + fp := filepath.Join(td, "config.json") + err := ioutil.WriteFile(fp, []byte(`{"bind_addr":"10.0.0.1", "data_dir":"`+td+`"}`), 0644) + if err != nil { + t.Fatalf("err: %s", err) + } + ui, cmd := testValidateCommand(t) args := []string{"-quiet", td} diff --git a/command/version.go b/command/version.go index 581cb5d43..4fcda5693 100644 --- a/command/version.go +++ b/command/version.go @@ -3,7 +3,7 @@ package command import ( "fmt" - "github.com/hashicorp/consul/agent" + "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/consul" "github.com/mitchellh/cli" ) @@ -21,14 +21,18 @@ func (c *VersionCommand) Help() string { func (c *VersionCommand) Run(_ []string) int { c.UI.Output(fmt.Sprintf("Consul %s", c.HumanVersion)) - config := agent.DefaultConfig() + rpcProtocol, err := config.DefaultRPCProtocol() + if err != nil { + c.UI.Error(err.Error()) + return 2 + } var supplement string - if config.Protocol < consul.ProtocolVersionMax { + if rpcProtocol < consul.ProtocolVersionMax { supplement = fmt.Sprintf(" (agent will automatically use protocol >%d when speaking to compatible agents)", - config.Protocol) + rpcProtocol) } c.UI.Output(fmt.Sprintf("Protocol %d spoken by default, understands %d to %d%s", - config.Protocol, consul.ProtocolVersionMin, consul.ProtocolVersionMax, supplement)) + rpcProtocol, consul.ProtocolVersionMin, consul.ProtocolVersionMax, supplement)) return 0 } diff --git a/command/watch_test.go b/command/watch_test.go index 4fe32c9aa..ef10087c5 100644 --- a/command/watch_test.go +++ b/command/watch_test.go @@ -15,7 +15,7 @@ func TestWatchCommand_implements(t *testing.T) { func TestWatchCommandRun(t *testing.T) { t.Parallel() - a := agent.NewTestAgent(t.Name(), nil) + a := agent.NewTestAgent(t.Name(), ``) defer a.Shutdown() ui := cli.NewMockUi() diff --git a/ipaddr/detect.go b/ipaddr/detect.go new file mode 100644 index 000000000..1512a0049 --- /dev/null +++ b/ipaddr/detect.go @@ -0,0 +1,138 @@ +package ipaddr + +import ( + "fmt" + "net" +) + +// GetPrivateIPv4 returns the list of private network IPv4 addresses on +// all active interfaces. +func GetPrivateIPv4() ([]*net.IPAddr, error) { + addresses, err := activeInterfaceAddresses() + if err != nil { + return nil, fmt.Errorf("Failed to get interface addresses: %v", err) + } + + var addrs []*net.IPAddr + for _, rawAddr := range addresses { + var ip net.IP + switch addr := rawAddr.(type) { + case *net.IPAddr: + ip = addr.IP + case *net.IPNet: + ip = addr.IP + default: + continue + } + if ip.To4() == nil { + continue + } + if !isPrivate(ip) { + continue + } + addrs = append(addrs, &net.IPAddr{IP: ip}) + } + return addrs, nil +} + +// GetPublicIPv6 returns the list of all public IPv6 addresses +// on all active interfaces. +func GetPublicIPv6() ([]*net.IPAddr, error) { + addresses, err := net.InterfaceAddrs() + if err != nil { + return nil, fmt.Errorf("Failed to get interface addresses: %v", err) + } + + var addrs []*net.IPAddr + for _, rawAddr := range addresses { + var ip net.IP + switch addr := rawAddr.(type) { + case *net.IPAddr: + ip = addr.IP + case *net.IPNet: + ip = addr.IP + default: + continue + } + if ip.To4() != nil { + continue + } + if isPrivate(ip) { + continue + } + addrs = append(addrs, &net.IPAddr{IP: ip}) + } + return addrs, nil +} + +// privateBlocks contains non-forwardable address blocks which are used +// for private networks. RFC 6890 provides an overview of special +// address blocks. +var privateBlocks = []*net.IPNet{ + parseCIDR("10.0.0.0/8"), // RFC 1918 IPv4 private network address + parseCIDR("100.64.0.0/10"), // RFC 6598 IPv4 shared address space + parseCIDR("127.0.0.0/8"), // RFC 1122 IPv4 loopback address + parseCIDR("169.254.0.0/16"), // RFC 3927 IPv4 link local address + parseCIDR("172.16.0.0/12"), // RFC 1918 IPv4 private network address + parseCIDR("192.0.0.0/24"), // RFC 6890 IPv4 IANA address + parseCIDR("192.0.2.0/24"), // RFC 5737 IPv4 documentation address + parseCIDR("192.168.0.0/16"), // RFC 1918 IPv4 private network address + parseCIDR("::1/128"), // RFC 1884 IPv6 loopback address + parseCIDR("fe80::/10"), // RFC 4291 IPv6 link local addresses + parseCIDR("fc00::/7"), // RFC 4193 IPv6 unique local addresses + parseCIDR("fec0::/10"), // RFC 1884 IPv6 site-local addresses + parseCIDR("2001:db8::/32"), // RFC 3849 IPv6 documentation address +} + +func parseCIDR(s string) *net.IPNet { + _, block, err := net.ParseCIDR(s) + if err != nil { + panic(fmt.Sprintf("Bad CIDR %s: %s", s, err)) + } + return block +} + +func isPrivate(ip net.IP) bool { + for _, priv := range privateBlocks { + if priv.Contains(ip) { + return true + } + } + return false +} + +// Returns addresses from interfaces that is up +func activeInterfaceAddresses() ([]net.Addr, error) { + var upAddrs []net.Addr + var loAddrs []net.Addr + + interfaces, err := net.Interfaces() + if err != nil { + return nil, fmt.Errorf("Failed to get interfaces: %v", err) + } + + for _, iface := range interfaces { + // Require interface to be up + if iface.Flags&net.FlagUp == 0 { + continue + } + + addresses, err := iface.Addrs() + if err != nil { + return nil, fmt.Errorf("Failed to get interface addresses: %v", err) + } + + if iface.Flags&net.FlagLoopback != 0 { + loAddrs = append(loAddrs, addresses...) + continue + } + + upAddrs = append(upAddrs, addresses...) + } + + if len(upAddrs) == 0 { + return loAddrs, nil + } + + return upAddrs, nil +} diff --git a/ipaddr/detect_test.go b/ipaddr/detect_test.go new file mode 100644 index 000000000..91eea1b9f --- /dev/null +++ b/ipaddr/detect_test.go @@ -0,0 +1,48 @@ +package ipaddr + +import ( + "net" + "testing" +) + +func TestIsPrivateIP(t *testing.T) { + tests := []struct { + ip string + private bool + }{ + // IPv4 private addresses + {"10.0.0.1", true}, // private network address + {"100.64.0.1", true}, // shared address space + {"172.16.0.1", true}, // private network address + {"192.168.0.1", true}, // private network address + {"192.0.0.1", true}, // IANA address + {"192.0.2.1", true}, // documentation address + {"127.0.0.1", true}, // loopback address + {"169.254.0.1", true}, // link local address + + // IPv4 public addresses + {"1.2.3.4", false}, + + // IPv6 private addresses + {"::1", true}, // loopback address + {"fe80::1", true}, // link local address + {"fc00::1", true}, // unique local address + {"fec0::1", true}, // site local address + {"2001:db8::1", true}, // documentation address + + // IPv6 public addresses + {"2004:db6::1", false}, + } + + for _, tt := range tests { + t.Run(tt.ip, func(t *testing.T) { + ip := net.ParseIP(tt.ip) + if ip == nil { + t.Fatalf("%s is not a valid ip address", tt.ip) + } + if got, want := isPrivate(ip), tt.private; got != want { + t.Fatalf("got %v for %v want %v", got, ip, want) + } + }) + } +} diff --git a/ipaddr/ipaddr.go b/ipaddr/ipaddr.go index b75eade4a..6755682cd 100644 --- a/ipaddr/ipaddr.go +++ b/ipaddr/ipaddr.go @@ -3,6 +3,7 @@ package ipaddr import ( "fmt" "net" + "reflect" ) // IsAny checks if the given ip address is an IPv4 or IPv6 ANY address. ip @@ -25,16 +26,27 @@ func IsAnyV6(ip interface{}) bool { } func iptos(ip interface{}) string { - if ip == nil { + if ip == nil || reflect.TypeOf(ip).Kind() == reflect.Ptr && reflect.ValueOf(ip).IsNil() { return "" } switch x := ip.(type) { case string: return x + case *string: + if x == nil { + return "" + } + return *x case net.IP: return x.String() case *net.IP: return x.String() + case *net.IPAddr: + return x.IP.String() + case *net.TCPAddr: + return x.IP.String() + case *net.UDPAddr: + return x.IP.String() default: panic(fmt.Sprintf("invalid type: %T", ip)) } diff --git a/test/porter/client.go b/test/porter/client.go new file mode 100644 index 000000000..34d3579e9 --- /dev/null +++ b/test/porter/client.go @@ -0,0 +1,36 @@ +package porter + +import ( + "encoding/json" + "fmt" + "io/ioutil" + "net/http" + "os" +) + +var DefaultAddr = "127.0.0.1:7965" + +func RandomPorts(n int) ([]int, error) { + addr := os.Getenv("PORTER_ADDR") + if addr == "" { + b, err := ioutil.ReadFile("/tmp/porter.addr") + if err == nil { + addr = string(b) + } + } + if addr == "" { + addr = DefaultAddr + } + resp, err := http.Get(fmt.Sprintf("http://%s/%d", addr, n)) + if err != nil { + return nil, err + } + defer resp.Body.Close() + data, err := ioutil.ReadAll(resp.Body) + if err != nil { + return nil, err + } + var p []int + err = json.Unmarshal(data, &p) + return p, err +} diff --git a/test/porter/cmd/porter/main.go b/test/porter/cmd/porter/main.go new file mode 100644 index 000000000..f7d2069d5 --- /dev/null +++ b/test/porter/cmd/porter/main.go @@ -0,0 +1,133 @@ +package main + +import ( + "encoding/json" + "flag" + "fmt" + "io/ioutil" + "log" + "net/http" + "os" + "os/exec" + "os/signal" + "strconv" + "sync" + + "github.com/hashicorp/consul/test/porter" +) + +var addrFile = "/tmp/porter.addr" + +var ( + addr string + firstPort int + lastPort int + verbose bool + + mu sync.Mutex + port int +) + +func main() { + log.SetFlags(0) + + flag.StringVar(&addr, "addr", porter.DefaultAddr, "host:port") + flag.IntVar(&firstPort, "first-port", 10000, "first port to allocate") + flag.IntVar(&lastPort, "last-port", 20000, "last port to allocate") + flag.BoolVar(&verbose, "verbose", false, "log port allocations") + flag.Parse() + + // check if there is an instance running + b, err := ioutil.ReadFile(addrFile) + switch { + // existing instance but no command to run + case err == nil && len(flag.Args()) == 0: + log.Println("porter already running on", string(b)) + os.Exit(0) + + // existing instance with command to run + case err == nil: + addr = string(b) + log.Println("re-using porter instance on", addr) + + // new instance + case os.IsNotExist(err): + if err := ioutil.WriteFile(addrFile, []byte(addr), 0644); err != nil { + log.Fatalf("Cannot write %s: %s", addrFile, err) + } + defer os.Remove(addrFile) + go func() { + http.HandleFunc("/", servePort) + if err := http.ListenAndServe(addr, nil); err != nil { + log.Fatal(err) + } + }() + } + + args := flag.Args() + + // no command to run: wait for CTRL-C + if len(args) == 0 { + log.Print("PORTER_ADDR=" + addr) + c := make(chan os.Signal, 1) + signal.Notify(c, os.Interrupt) + s := <-c + log.Println("Got signal:", s) + return + } + + // run command and exit with 1 in case of error + if err := run(args); err != nil { + log.Fatal(err) + } +} + +func run(args []string) error { + path, err := exec.LookPath(args[0]) + if err != nil { + return fmt.Errorf("Cannot find %q in path", args[0]) + } + + cmd := exec.Command(path, args[1:]...) + if os.Getenv("PORTER_ADDR") == "" { + cmd.Env = append(os.Environ(), "PORTER_ADDR="+addr) + } + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + return cmd.Run() +} + +// todo(fs): check which ports are currently bound and exclude them +func servePort(w http.ResponseWriter, r *http.Request) { + var count int + n, err := strconv.Atoi(r.RequestURI[1:]) + if err == nil { + count = n + } + if count <= 0 { + count = 1 + } + + mu.Lock() + if port < firstPort { + port = firstPort + } + if port+count >= lastPort { + port = firstPort + } + from, to := port, port+count + port = to + mu.Unlock() + + p := make([]int, count) + for i := 0; i < count; i++ { + p[i] = from + i + } + if err := json.NewEncoder(w).Encode(p); err != nil { + // this shouldn't happen so we panic since we can't recover + panic(err) + } + if verbose { + log.Printf("porter: allocated ports %d-%d (%d)", from, to, count) + } +} diff --git a/testutil/server.go b/testutil/server.go index 9852c055e..c01471271 100644 --- a/testutil/server.go +++ b/testutil/server.go @@ -27,6 +27,7 @@ import ( "testing" "time" + "github.com/hashicorp/consul/test/porter" "github.com/hashicorp/consul/testutil/retry" "github.com/hashicorp/go-cleanhttp" "github.com/hashicorp/go-uuid" @@ -47,9 +48,6 @@ type TestPortConfig struct { SerfLan int `json:"serf_lan,omitempty"` SerfWan int `json:"serf_wan,omitempty"` Server int `json:"server,omitempty"` - - // Deprecated - RPC int `json:"rpc,omitempty"` } // TestAddressConfig contains the bind addresses for various @@ -113,8 +111,12 @@ func defaultServerConfig() *TestServerConfig { panic(err) } + ports, err := porter.RandomPorts(6) + if err != nil { + panic(err) + } return &TestServerConfig{ - NodeName: fmt.Sprintf("node%d", randomPort()), + NodeName: "node-" + nodeID, NodeID: nodeID, DisableCheckpoint: true, Performance: &TestPerformanceConfig{ @@ -126,28 +128,17 @@ func defaultServerConfig() *TestServerConfig { Bind: "127.0.0.1", Addresses: &TestAddressConfig{}, Ports: &TestPortConfig{ - DNS: randomPort(), - HTTP: randomPort(), - HTTPS: randomPort(), - SerfLan: randomPort(), - SerfWan: randomPort(), - Server: randomPort(), - RPC: randomPort(), + DNS: ports[0], + HTTP: ports[1], + HTTPS: ports[2], + SerfLan: ports[3], + SerfWan: ports[4], + Server: ports[5], }, ReadyTimeout: 10 * time.Second, } } -// randomPort asks the kernel for a random port to use. -func randomPort() int { - l, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - panic(err) - } - defer l.Close() - return l.Addr().(*net.TCPAddr).Port -} - // TestService is used to serialize a service definition. type TestService struct { ID string `json:",omitempty"` @@ -200,15 +191,7 @@ func NewTestServerConfig(cb ServerConfigCallback) (*TestServer, error) { // configuring or starting the server, the server will NOT be running when the // function returns (thus you do not need to stop it). func NewTestServerConfigT(t *testing.T, cb ServerConfigCallback) (*TestServer, error) { - var server *TestServer - retry.Run(t, func(r *retry.R) { - var err error - server, err = newTestServerConfigT(t, cb) - if err != nil { - r.Fatalf("failed starting test server: %v", err) - } - }) - return server, nil + return newTestServerConfigT(t, cb) } // newTestServerConfigT is the internal helper for NewTestServerConfigT. diff --git a/vendor/github.com/hashicorp/go-sockaddr/ipv4addr.go b/vendor/github.com/hashicorp/go-sockaddr/ipv4addr.go index 4d395dc95..89b56a24d 100644 --- a/vendor/github.com/hashicorp/go-sockaddr/ipv4addr.go +++ b/vendor/github.com/hashicorp/go-sockaddr/ipv4addr.go @@ -58,7 +58,8 @@ func NewIPv4Addr(ipv4Str string) (IPv4Addr, error) { // Strip off any bogus hex-encoded netmasks that will be mis-parsed by Go. In // particular, clients with the Barracuda VPN client will see something like: // `192.168.3.51/00ffffff` as their IP address. - trailingHexNetmaskRe := trailingHexNetmaskRE.Copy() + // trailingHexNetmaskRe := trailingHexNetmaskRE.Copy() + trailingHexNetmaskRe := regexp.MustCompile(`/([0f]{8})$`) if match := trailingHexNetmaskRe.FindStringIndex(ipv4Str); match != nil { ipv4Str = ipv4Str[:match[0]] } diff --git a/vendor/github.com/hashicorp/hcl/Makefile b/vendor/github.com/hashicorp/hcl/Makefile index ad404a811..84fd743f5 100644 --- a/vendor/github.com/hashicorp/hcl/Makefile +++ b/vendor/github.com/hashicorp/hcl/Makefile @@ -6,6 +6,7 @@ fmt: generate go fmt ./... test: generate + go get -t ./... go test $(TEST) $(TESTARGS) generate: diff --git a/vendor/github.com/hashicorp/hcl/README.md b/vendor/github.com/hashicorp/hcl/README.md index e292d5999..c8223326d 100644 --- a/vendor/github.com/hashicorp/hcl/README.md +++ b/vendor/github.com/hashicorp/hcl/README.md @@ -103,6 +103,16 @@ variable "ami" { description = "the AMI to use" } ``` +This would be equivalent to the following json: +``` json +{ + "variable": { + "ami": { + "description": "the AMI to use" + } + } +} +``` ## Thanks diff --git a/vendor/github.com/hashicorp/hcl/appveyor.yml b/vendor/github.com/hashicorp/hcl/appveyor.yml index e70f03b96..4db0b7112 100644 --- a/vendor/github.com/hashicorp/hcl/appveyor.yml +++ b/vendor/github.com/hashicorp/hcl/appveyor.yml @@ -4,7 +4,7 @@ clone_folder: c:\gopath\src\github.com\hashicorp\hcl environment: GOPATH: c:\gopath init: - - git config --global core.autocrlf true + - git config --global core.autocrlf false install: - cmd: >- echo %Path% @@ -12,5 +12,8 @@ install: go version go env + + go get -t ./... + build_script: - cmd: go test -v ./... diff --git a/vendor/github.com/hashicorp/hcl/decoder.go b/vendor/github.com/hashicorp/hcl/decoder.go index cfddbf360..125d3d50f 100644 --- a/vendor/github.com/hashicorp/hcl/decoder.go +++ b/vendor/github.com/hashicorp/hcl/decoder.go @@ -56,7 +56,7 @@ func DecodeObject(out interface{}, n ast.Node) error { n = f.Node } - var d decoder + d := &decoder{} return d.decode("root", n, val.Elem()) } @@ -89,9 +89,9 @@ func (d *decoder) decode(name string, node ast.Node, result reflect.Value) error switch k.Kind() { case reflect.Bool: return d.decodeBool(name, node, result) - case reflect.Float64: + case reflect.Float32, reflect.Float64: return d.decodeFloat(name, node, result) - case reflect.Int: + case reflect.Int, reflect.Int32, reflect.Int64: return d.decodeInt(name, node, result) case reflect.Interface: // When we see an interface, we make our own thing @@ -137,13 +137,13 @@ func (d *decoder) decodeBool(name string, node ast.Node, result reflect.Value) e func (d *decoder) decodeFloat(name string, node ast.Node, result reflect.Value) error { switch n := node.(type) { case *ast.LiteralType: - if n.Token.Type == token.FLOAT { + if n.Token.Type == token.FLOAT || n.Token.Type == token.NUMBER { v, err := strconv.ParseFloat(n.Token.Text, 64) if err != nil { return err } - result.Set(reflect.ValueOf(v)) + result.Set(reflect.ValueOf(v).Convert(result.Type())) return nil } } @@ -164,7 +164,11 @@ func (d *decoder) decodeInt(name string, node ast.Node, result reflect.Value) er return err } - result.Set(reflect.ValueOf(int(v))) + if result.Kind() == reflect.Interface { + result.Set(reflect.ValueOf(int(v))) + } else { + result.SetInt(v) + } return nil case token.STRING: v, err := strconv.ParseInt(n.Token.Value().(string), 0, 0) @@ -172,7 +176,11 @@ func (d *decoder) decodeInt(name string, node ast.Node, result reflect.Value) er return err } - result.Set(reflect.ValueOf(int(v))) + if result.Kind() == reflect.Interface { + result.Set(reflect.ValueOf(int(v))) + } else { + result.SetInt(v) + } return nil } } @@ -409,7 +417,6 @@ func (d *decoder) decodeSlice(name string, node ast.Node, result reflect.Value) if result.Kind() == reflect.Interface { result = result.Elem() } - // Create the slice if it isn't nil resultType := result.Type() resultElemType := resultType.Elem() @@ -443,6 +450,12 @@ func (d *decoder) decodeSlice(name string, node ast.Node, result reflect.Value) // Decode val := reflect.Indirect(reflect.New(resultElemType)) + + // if item is an object that was decoded from ambiguous JSON and + // flattened, make sure it's expanded if it needs to decode into a + // defined structure. + item := expandObject(item, val) + if err := d.decode(fieldName, item, val); err != nil { return err } @@ -455,6 +468,57 @@ func (d *decoder) decodeSlice(name string, node ast.Node, result reflect.Value) return nil } +// expandObject detects if an ambiguous JSON object was flattened to a List which +// should be decoded into a struct, and expands the ast to properly deocode. +func expandObject(node ast.Node, result reflect.Value) ast.Node { + item, ok := node.(*ast.ObjectItem) + if !ok { + return node + } + + elemType := result.Type() + + // our target type must be a struct + switch elemType.Kind() { + case reflect.Ptr: + switch elemType.Elem().Kind() { + case reflect.Struct: + //OK + default: + return node + } + case reflect.Struct: + //OK + default: + return node + } + + // A list value will have a key and field name. If it had more fields, + // it wouldn't have been flattened. + if len(item.Keys) != 2 { + return node + } + + keyToken := item.Keys[0].Token + item.Keys = item.Keys[1:] + + // we need to un-flatten the ast enough to decode + newNode := &ast.ObjectItem{ + Keys: []*ast.ObjectKey{ + &ast.ObjectKey{ + Token: keyToken, + }, + }, + Val: &ast.ObjectType{ + List: &ast.ObjectList{ + Items: []*ast.ObjectItem{item}, + }, + }, + } + + return newNode +} + func (d *decoder) decodeString(name string, node ast.Node, result reflect.Value) error { switch n := node.(type) { case *ast.LiteralType: @@ -489,7 +553,7 @@ func (d *decoder) decodeStruct(name string, node ast.Node, result reflect.Value) // the yacc parser would always ensure top-level elements were arrays. The new // parser does not make the same guarantees, thus we need to convert any // top-level literal elements into a list. - if _, ok := node.(*ast.LiteralType); ok { + if _, ok := node.(*ast.LiteralType); ok && item != nil { node = &ast.ObjectList{Items: []*ast.ObjectItem{item}} } @@ -509,7 +573,12 @@ func (d *decoder) decodeStruct(name string, node ast.Node, result reflect.Value) // Compile the list of all the fields that we're going to be decoding // from all the structs. - fields := make(map[*reflect.StructField]reflect.Value) + // fields := make(map[*reflect.StructField]reflect.Value) + type x struct { + t reflect.StructField + v reflect.Value + } + fields := []x{} for len(structs) > 0 { structVal := structs[0] structs = structs[1:] @@ -552,7 +621,8 @@ func (d *decoder) decodeStruct(name string, node ast.Node, result reflect.Value) } // Normal struct field, store it away - fields[&fieldType] = structVal.Field(i) + fields = append(fields, x{fieldType, structVal.Field(i)}) + // fields[&fieldType] = structVal.Field(i) } } @@ -560,7 +630,9 @@ func (d *decoder) decodeStruct(name string, node ast.Node, result reflect.Value) decodedFields := make([]string, 0, len(fields)) decodedFieldsVal := make([]reflect.Value, 0) unusedKeysVal := make([]reflect.Value, 0) - for fieldType, field := range fields { + // for fieldType, field := range fields { + for _, x := range fields { + fieldType, field := x.t, x.v if !field.IsValid() { // This should never happen panic("field is not valid") @@ -605,7 +677,10 @@ func (d *decoder) decodeStruct(name string, node ast.Node, result reflect.Value) // Determine the element we'll use to decode. If it is a single // match (only object with the field), then we decode it exactly. // If it is a prefix match, then we decode the matches. + // d.mu.Lock() filter := list.Filter(fieldName) + // d.mu.Unlock() + prefixMatches := filter.Children() matches := filter.Elem() if len(matches.Items) == 0 && len(prefixMatches.Items) == 0 { diff --git a/vendor/github.com/hashicorp/hcl/hcl/ast/ast.go b/vendor/github.com/hashicorp/hcl/hcl/ast/ast.go index 692ac24e2..5d4c56739 100644 --- a/vendor/github.com/hashicorp/hcl/hcl/ast/ast.go +++ b/vendor/github.com/hashicorp/hcl/hcl/ast/ast.go @@ -61,9 +61,10 @@ func (o *ObjectList) Filter(keys ...string) *ObjectList { } match := true - for i, key := range item.Keys[:len(keys)] { - key := key.Token.Value().(string) - if key != keys[i] && !strings.EqualFold(key, keys[i]) { + for i, k := range item.Keys[:len(keys)] { + keyi := keys[i] + key := k.Token.Value().(string) + if key != keyi && !strings.EqualFold(key, keyi) { match = false break } diff --git a/vendor/github.com/hashicorp/memberlist/memberlist.go b/vendor/github.com/hashicorp/memberlist/memberlist.go index d5c175e5b..e1e38fc94 100644 --- a/vendor/github.com/hashicorp/memberlist/memberlist.go +++ b/vendor/github.com/hashicorp/memberlist/memberlist.go @@ -127,7 +127,7 @@ func newMemberlist(conf *Config) (*Memberlist, error) { return nt, nil } if strings.Contains(err.Error(), "address already in use") { - logger.Printf("[DEBUG] Got bind error: %v", err) + logger.Printf("[DEBUG] memberlist: Got bind error: %v", err) continue } } @@ -154,7 +154,7 @@ func newMemberlist(conf *Config) (*Memberlist, error) { port := nt.GetAutoBindPort() conf.BindPort = port conf.AdvertisePort = port - logger.Printf("[DEBUG] Using dynamic bind port %d", port) + logger.Printf("[DEBUG] memberlist: Using dynamic bind port %d", port) } transport = nt } diff --git a/vendor/vendor.json b/vendor/vendor.json index 856aee35b..0b0c46735 100644 --- a/vendor/vendor.json +++ b/vendor/vendor.json @@ -43,7 +43,7 @@ {"path":"github.com/hashicorp/go-version","checksumSHA1":"tUGxc7rfX0cmhOOUDhMuAZ9rWsA=","revision":"03c5bf6be031b6dd45afec16b1cf94fc8938bc77","revisionTime":"2017-02-02T08:07:59Z"}, {"path":"github.com/hashicorp/golang-lru","checksumSHA1":"d9PxF1XQGLMJZRct2R8qVM/eYlE=","revision":"a0d98a5f288019575c6d1f4bb1573fef2d1fcdc4","revisionTime":"2016-02-07T21:47:19Z"}, {"path":"github.com/hashicorp/golang-lru/simplelru","checksumSHA1":"2nOpYjx8Sn57bqlZq17yM4YJuM4=","revision":"a0d98a5f288019575c6d1f4bb1573fef2d1fcdc4","revisionTime":"2016-02-07T21:47:19Z"}, - {"path":"github.com/hashicorp/hcl","checksumSHA1":"ydHBPi04mEh+Tir+2JkpSIMckcw=","revision":"d8c773c4cba11b11539e3d45f93daeaa5dcf1fa1","revisionTime":"2016-07-11T23:17:52Z"}, + {"path":"github.com/hashicorp/hcl","checksumSHA1":"7JBkp3EZoc0MSbiyWfzVhO4RYoY=","revision":"8f6b1344a92ff8877cf24a5de9177bf7d0a2a187","revisionTime":"2017-04-17T18:02:14Z"}, {"path":"github.com/hashicorp/hcl/hcl/ast","checksumSHA1":"IxyvRpCFeoJBGl2obLKJV7RCGjg=","revision":"d8c773c4cba11b11539e3d45f93daeaa5dcf1fa1","revisionTime":"2016-07-11T23:17:52Z"}, {"path":"github.com/hashicorp/hcl/hcl/parser","checksumSHA1":"l2oQxBsZRwn6eZjf+whXr8c9+8c=","revision":"d8c773c4cba11b11539e3d45f93daeaa5dcf1fa1","revisionTime":"2016-07-11T23:17:52Z"}, {"path":"github.com/hashicorp/hcl/hcl/scanner","checksumSHA1":"vjhDQVlgHhdxml1V8/cj0vOe+j8=","revision":"d8c773c4cba11b11539e3d45f93daeaa5dcf1fa1","revisionTime":"2016-07-11T23:17:52Z"}, @@ -55,7 +55,7 @@ {"path":"github.com/hashicorp/hil","checksumSHA1":"kqCMCHy2b+RBMKC+ER+OPqp8C3E=","revision":"1e86c6b523c55d1fa6c6e930ce80b548664c95c2","revisionTime":"2016-07-11T23:18:37Z"}, {"path":"github.com/hashicorp/hil/ast","checksumSHA1":"UICubs001+Q4MsUf9zl2vcMzWQQ=","revision":"1e86c6b523c55d1fa6c6e930ce80b548664c95c2","revisionTime":"2016-07-11T23:18:37Z"}, {"path":"github.com/hashicorp/logutils","checksumSHA1":"vt+P9D2yWDO3gdvdgCzwqunlhxU=","revision":"0dc08b1671f34c4250ce212759ebd880f743d883","revisionTime":"2015-06-09T07:04:31Z"}, - {"path":"github.com/hashicorp/memberlist","checksumSHA1":"ml0MTqOsKTrsqv/mZhy78Vz4SfA=","revision":"d6c1fb0b99c33d0a8e22acea9da9709b369b5d39","revisionTime":"2017-08-15T22:46:17Z"}, + {"path":"github.com/hashicorp/memberlist","checksumSHA1":"vwj2yOi577Mmn+IfJwV8YXYeALk=","revision":"687988a0b5daaf7ed5051e5e374aef27f8254822","revisionTime":"2017-09-19T17:31:51Z"}, {"path":"github.com/hashicorp/net-rpc-msgpackrpc","checksumSHA1":"qnlqWJYV81ENr61SZk9c65R1mDo=","revision":"a14192a58a694c123d8fe5481d4a4727d6ae82f3","revisionTime":"2015-11-16T02:03:38Z"}, {"path":"github.com/hashicorp/raft","checksumSHA1":"f2QYddVWZ2eWxdlCEhearTH4XOs=","revision":"c837e57a6077e74a4a3749959fb6cfefc26d7705","revisionTime":"2017-08-30T14:31:53Z","version":"library-v2-stage-one","versionExact":"library-v2-stage-one"}, {"path":"github.com/hashicorp/raft-boltdb","checksumSHA1":"QAxukkv54/iIvLfsUP6IK4R0m/A=","revision":"d1e82c1ec3f15ee991f7cc7ffd5b67ff6f5bbaee","revisionTime":"2015-02-01T20:08:39Z"}, diff --git a/website/source/docs/agent/options.html.md b/website/source/docs/agent/options.html.md index 624ec1bd1..df9172e40 100644 --- a/website/source/docs/agent/options.html.md +++ b/website/source/docs/agent/options.html.md @@ -19,13 +19,16 @@ Configuration precedence is evaluated in the following order: 2. Environment Variables 3. Configuration files -When loading configuration, Consul loads the configuration from files -and directories in lexical order. For example, configuration file `basic_config.json` -will be processed before `extra_config.json`. Configuration specified later -will be merged into configuration specified earlier. In most cases, -"merge" means that the later version will override the earlier. In -some cases, such as event handlers, merging appends the handlers to the -existing configuration. The exact merging behavior is specified for each +When loading configuration, Consul loads the configuration from files and +directories in lexical order. For example, configuration file +`basic_config.json` will be processed before `extra_config.json`. Configuration +can be in either HCL or JSON format. Files with an `.hcl` extension are assumed +to be in HCL format. All other files are considered to be in JSON format +however you should append a `.json` suffix to JSON config files for clarity. +Configuration specified later will be merged into configuration specified +earlier. In most cases, "merge" means that the later version will override the +earlier. In some cases, such as event handlers, merging appends the handlers to +the existing configuration. The exact merging behavior is specified for each option below. Consul also supports reloading configuration when it receives the @@ -177,6 +180,11 @@ will exit with an error at startup. initialized with an encryption key, then the provided key is ignored and a warning will be displayed. +* `-hcl` - A HCL configuration fragment. + This HCL configuration fragment is appended to the configuration and allows + to specify the full range of options of a config file on the command line. + This option can be specified multiple times. This was added in Consul 1.0. + * `-http-port` - the HTTP API port to listen on. This overrides the default port 8500. This option is very useful when deploying Consul to an environment which communicates the HTTP port through the environment e.g. PaaS like CloudFoundry, allowing