luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity
open-consul/agent/consul/usagemetrics/usagemetrics_oss_test.go

317 lines
10 KiB
Go
Raw Normal View History

Report node/service usage metrics from every server Using the newly provided state store methods, we periodically emit usage metrics from the servers. We decided to emit these metrics from all servers, not just the leader, because that means we do not have to care about leader election flapping causing metrics turbulence, and it seems reasonable for each server to emit its own view of the state, even if they should always converge rapidly.
2020-09-02 15:24:17 +00:00
// +build !consulent
package usagemetrics
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
import (
"testing"
"time"
"github.com/armon/go-metrics"
stream: Use a no-op event publisher if streaming is disabled
2020-10-23 19:21:37 +00:00
"github.com/stretchr/testify/require"
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
"github.com/hashicorp/serf/serf"
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
"github.com/hashicorp/consul/agent/consul/state"
"github.com/hashicorp/consul/agent/structs"
"github.com/hashicorp/consul/sdk/testutil"
)
Report node/service usage metrics from every server Using the newly provided state store methods, we periodically emit usage metrics from the servers. We decided to emit these metrics from all servers, not just the leader, because that means we do not have to care about leader election flapping causing metrics turbulence, and it seems reasonable for each server to emit its own view of the state, even if they should always converge rapidly.
2020-09-02 15:24:17 +00:00
restore prior signature of test helper so enterprise compiles
2020-10-29 18:52:15 +00:00
func newStateStore() (*state.Store, error) {
return state.NewStateStore(nil), nil
Report node/service usage metrics from every server Using the newly provided state store methods, we periodically emit usage metrics from the servers. We decided to emit these metrics from all servers, not just the leader, because that means we do not have to care about leader election flapping causing metrics turbulence, and it seems reasonable for each server to emit its own view of the state, even if they should always converge rapidly.
2020-09-02 15:24:17 +00:00
}
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
func TestUsageReporter_emitNodeUsage_OSS(t *testing.T) {
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
type testCase struct {
modfiyStateStore func(t *testing.T, s *state.Store)
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
getMembersFunc getMembersFunc
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
expectedGauges map[string]metrics.GaugeValue
}
cases := map[string]testCase{
"empty-state": {
expectedGauges: map[string]metrics.GaugeValue{
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
// --- node ---
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
"consul.usage.test.consul.state.nodes;datacenter=dc1": {
Name: "consul.usage.test.consul.state.nodes",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
// --- member ---
"consul.usage.test.consul.members.clients;datacenter=dc1": {
Name: "consul.usage.test.consul.members.clients",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
"consul.usage.test.consul.members.servers;datacenter=dc1": {
Name: "consul.usage.test.consul.members.servers",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
// --- service ---
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
"consul.usage.test.consul.state.services;datacenter=dc1": {
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
Name: "consul.usage.test.consul.state.services",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
"consul.usage.test.consul.state.service_instances;datacenter=dc1": {
Name: "consul.usage.test.consul.state.service_instances",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
},
getMembersFunc: func() []serf.Member { return []serf.Member{} },
},
"nodes": {
modfiyStateStore: func(t *testing.T, s *state.Store) {
require.NoError(t, s.EnsureNode(1, &structs.Node{Node: "foo", Address: "127.0.0.1"}))
require.NoError(t, s.EnsureNode(2, &structs.Node{Node: "bar", Address: "127.0.0.2"}))
require.NoError(t, s.EnsureNode(3, &structs.Node{Node: "baz", Address: "127.0.0.2"}))
},
getMembersFunc: func() []serf.Member {
return []serf.Member{
{
Name: "foo",
Tags: map[string]string{"role": "consul"},
Status: serf.StatusAlive,
},
{
Name: "bar",
Tags: map[string]string{"role": "consul"},
Status: serf.StatusAlive,
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
},
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
{
Name: "baz",
Tags: map[string]string{"role": "node"},
Status: serf.StatusAlive,
},
}
},
expectedGauges: map[string]metrics.GaugeValue{
// --- node ---
"consul.usage.test.consul.state.nodes;datacenter=dc1": {
Name: "consul.usage.test.consul.state.nodes",
Value: 3,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
// --- member ---
"consul.usage.test.consul.members.servers;datacenter=dc1": {
Name: "consul.usage.test.consul.members.servers",
Value: 2,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
"consul.usage.test.consul.members.clients;datacenter=dc1": {
Name: "consul.usage.test.consul.members.clients",
Value: 1,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
// --- service ---
"consul.usage.test.consul.state.services;datacenter=dc1": {
Name: "consul.usage.test.consul.state.services",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
},
"consul.usage.test.consul.state.service_instances;datacenter=dc1": {
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
Name: "consul.usage.test.consul.state.service_instances",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
},
},
}
for name, tcase := range cases {
t.Run(name, func(t *testing.T) {
// Only have a single interval for the test
sink := metrics.NewInmemSink(1*time.Minute, 1*time.Minute)
cfg := metrics.DefaultConfig("consul.usage.test")
cfg.EnableHostname = false
metrics.NewGlobal(cfg, sink)
mockStateProvider := &mockStateProvider{}
s, err := newStateStore()
require.NoError(t, err)
if tcase.modfiyStateStore != nil {
tcase.modfiyStateStore(t, s)
}
mockStateProvider.On("State").Return(s)
reporter, err := NewUsageMetricsReporter(
new(Config).
WithStateProvider(mockStateProvider).
WithLogger(testutil.Logger(t)).
WithDatacenter("dc1").
WithGetMembersFunc(tcase.getMembersFunc),
)
require.NoError(t, err)
reporter.runOnce()
intervals := sink.Data()
require.Len(t, intervals, 1)
intv := intervals[0]
assertEqualGaugeMaps(t, tcase.expectedGauges, intv.Gauges)
})
}
}
func TestUsageReporter_emitServiceUsage_OSS(t *testing.T) {
type testCase struct {
modfiyStateStore func(t *testing.T, s *state.Store)
getMembersFunc getMembersFunc
expectedGauges map[string]metrics.GaugeValue
}
cases := map[string]testCase{
"empty-state": {
expectedGauges: map[string]metrics.GaugeValue{
// --- node ---
"consul.usage.test.consul.state.nodes;datacenter=dc1": {
Name: "consul.usage.test.consul.state.nodes",
Value: 0,
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
// --- member ---
"consul.usage.test.consul.members.servers;datacenter=dc1": {
Name: "consul.usage.test.consul.members.servers",
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
Value: 0,
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
"consul.usage.test.consul.members.clients;datacenter=dc1": {
Name: "consul.usage.test.consul.members.clients",
Value: 0,
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
// --- service ---
"consul.usage.test.consul.state.services;datacenter=dc1": {
Name: "consul.usage.test.consul.state.services",
Value: 0,
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
"consul.usage.test.consul.state.service_instances;datacenter=dc1": {
Name: "consul.usage.test.consul.state.service_instances",
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
Value: 0,
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
},
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
getMembersFunc: func() []serf.Member { return []serf.Member{} },
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
},
"nodes-and-services": {
modfiyStateStore: func(t *testing.T, s *state.Store) {
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
require.NoError(t, s.EnsureNode(1, &structs.Node{Node: "foo", Address: "127.0.0.1"}))
require.NoError(t, s.EnsureNode(2, &structs.Node{Node: "bar", Address: "127.0.0.2"}))
require.NoError(t, s.EnsureNode(3, &structs.Node{Node: "baz", Address: "127.0.0.2"}))
require.NoError(t, s.EnsureNode(4, &structs.Node{Node: "qux", Address: "127.0.0.3"}))
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
// Typical services and some consul services spread across two nodes
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
require.NoError(t, s.EnsureService(5, "foo", &structs.NodeService{ID: "db", Service: "db", Tags: nil, Address: "", Port: 5000}))
require.NoError(t, s.EnsureService(6, "bar", &structs.NodeService{ID: "api", Service: "api", Tags: nil, Address: "", Port: 5000}))
require.NoError(t, s.EnsureService(7, "foo", &structs.NodeService{ID: "consul", Service: "consul", Tags: nil}))
require.NoError(t, s.EnsureService(8, "bar", &structs.NodeService{ID: "consul", Service: "consul", Tags: nil}))
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
},
getMembersFunc: func() []serf.Member {
return []serf.Member{
{
Name: "foo",
Tags: map[string]string{"role": "consul"},
Status: serf.StatusAlive,
},
{
Name: "bar",
Tags: map[string]string{"role": "consul"},
Status: serf.StatusAlive,
},
{
Name: "baz",
Tags: map[string]string{"role": "node", "segment": "a"},
Status: serf.StatusAlive,
},
{
Name: "qux",
Tags: map[string]string{"role": "node", "segment": "b"},
Status: serf.StatusAlive,
},
}
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
},
expectedGauges: map[string]metrics.GaugeValue{
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
// --- node ---
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
"consul.usage.test.consul.state.nodes;datacenter=dc1": {
Name: "consul.usage.test.consul.state.nodes",
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
Value: 4,
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
Labels: []metrics.Label{{Name: "datacenter", Value: "dc1"}},
},
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
// --- member ---
"consul.usage.test.consul.members.servers;datacenter=dc1": {
Name: "consul.usage.test.consul.members.servers",
Value: 2,
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
"consul.usage.test.consul.members.clients;datacenter=dc1": {
Name: "consul.usage.test.consul.members.clients",
Value: 2,
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
// --- service ---
"consul.usage.test.consul.state.services;datacenter=dc1": {
Name: "consul.usage.test.consul.state.services",
Value: 3,
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
"consul.usage.test.consul.state.service_instances;datacenter=dc1": {
Name: "consul.usage.test.consul.state.service_instances",
Value: 4,
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
Labels: []metrics.Label{
{Name: "datacenter", Value: "dc1"},
},
},
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
},
},
}
for name, tcase := range cases {
t.Run(name, func(t *testing.T) {
// Only have a single interval for the test
sink := metrics.NewInmemSink(1*time.Minute, 1*time.Minute)
cfg := metrics.DefaultConfig("consul.usage.test")
cfg.EnableHostname = false
metrics.NewGlobal(cfg, sink)
mockStateProvider := &mockStateProvider{}
stream: Use a no-op event publisher if streaming is disabled
2020-10-23 19:21:37 +00:00
s := state.NewStateStore(nil)
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
if tcase.modfiyStateStore != nil {
tcase.modfiyStateStore(t, s)
}
mockStateProvider.On("State").Return(s)
reporter, err := NewUsageMetricsReporter(
new(Config).
WithStateProvider(mockStateProvider).
WithLogger(testutil.Logger(t)).
usagemetrics: add cluster members to metrics API (#10340) This PR adds cluster members to the metrics API. The number of members per segment are reported as well as the total number of members. Tested by running a multi-node cluster locally and ensuring the numbers were correct. Also added unit test coverage to add the new expected gauges to existing test cases.
2021-06-03 15:25:53 +00:00
WithDatacenter("dc1").
WithGetMembersFunc(tcase.getMembersFunc),
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
)
require.NoError(t, err)
reporter.runOnce()
intervals := sink.Data()
require.Len(t, intervals, 1)
intv := intervals[0]
state: partition the usage metrics subsystem (#10867)
2021-08-18 14:27:15 +00:00
assertEqualGaugeMaps(t, tcase.expectedGauges, intv.Gauges)
Emit service usage metrics with correct labeling strategy (#8856) Previously, we would emit service usage metrics both with and without a namespace label attached. This is problematic in the case when you want to aggregate metrics together, i.e. "sum(consul.state.services)". This would cause services to be counted twice in that aggregate, once via the metric emitted with a namespace label, and once in the metric emited without any namespace label.
2020-10-09 16:01:45 +00:00
})
}
}