open-vault/vendor/github.com/armon/go-metrics
Brian Kassouf ed14061578
Raft Storage Backend (#6888)
* Work on raft backend

* Add logstore locally

* Add encryptor and unsealable interfaces

* Add clustering support to raft

* Remove client and handler

* Bootstrap raft on init

* Cleanup raft logic a bit

* More raft work

* Work on TLS config

* More work on bootstrapping

* Fix build

* More work on bootstrapping

* More bootstrapping work

* fix build

* Remove consul dep

* Fix build

* merged oss/master into raft-storage

* Work on bootstrapping

* Get bootstrapping to work

* Clean up FMS and node-id

* Update local node ID logic

* Cleanup node-id change

* Work on snapshotting

* Raft: Add remove peer API (#906)

* Add remove peer API

* Add some comments

* Fix existing snapshotting (#909)

* Raft get peers API (#912)

* Read raft configuration

* address review feedback

* Use the Leadership Transfer API to step-down the active node (#918)

* Raft join and unseal using Shamir keys (#917)

* Raft join using shamir

* Store AEAD instead of master key

* Split the raft join process to answer the challenge after a successful unseal

* get the follower to standby state

* Make unseal work

* minor changes

* Some input checks

* reuse the shamir seal access instead of new default seal access

* refactor joinRaftSendAnswer function

* Synchronously send answer in auto-unseal case

* Address review feedback

* Raft snapshots (#910)

* Fix existing snapshotting

* implement the noop snapshotting

* Add comments and switch log libraries

* add some snapshot tests

* add snapshot test file

* add TODO

* More work on raft snapshotting

* progress on the ConfigStore strategy

* Don't use two buckets

* Update the snapshot store logic to hide the file logic

* Add more backend tests

* Cleanup code a bit

* [WIP] Raft recovery (#938)

* Add recovery functionality

* remove fmt.Printfs

* Fix a few fsm bugs

* Add max size value for raft backend (#942)

* Add max size value for raft backend

* Include physical.ErrValueTooLarge in the message

* Raft snapshot Take/Restore API  (#926)

* Inital work on raft snapshot APIs

* Always redirect snapshot install/download requests

* More work on the snapshot APIs

* Cleanup code a bit

* On restore handle special cases

* Use the seal to encrypt the sha sum file

* Add sealer mechanism and fix some bugs

* Call restore while state lock is held

* Send restore cb trigger through raft log

* Make error messages nicer

* Add test helpers

* Add snapshot test

* Add shamir unseal test

* Add more raft snapshot API tests

* Fix locking

* Change working to initalize

* Add underlying raw object to test cluster core

* Move leaderUUID to core

* Add raft TLS rotation logic (#950)

* Add TLS rotation logic

* Cleanup logic a bit

* Add/Remove from follower state on add/remove peer

* add comments

* Update more comments

* Update request_forwarding_service.proto

* Make sure we populate all nodes in the followerstate obj

* Update times

* Apply review feedback

* Add more raft config setting (#947)

* Add performance config setting

* Add more config options and fix tests

* Test Raft Recovery (#944)

* Test raft recovery

* Leave out a node during recovery

* remove unused struct

* Update physical/raft/snapshot_test.go

* Update physical/raft/snapshot_test.go

* fix vendoring

* Switch to new raft interface

* Remove unused files

* Switch a gogo -> proto instance

* Remove unneeded vault dep in go.sum

* Update helper/testhelpers/testhelpers.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* Update vault/cluster/cluster.go

* track active key within the keyring itself (#6915)

* track active key within the keyring itself

* lookup and store using the active key ID

* update docstring

* minor refactor

* Small text fixes (#6912)

* Update physical/raft/raft.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* review feedback

* Move raft logical system into separate file

* Update help text a bit

* Enforce cluster addr is set and use it for raft bootstrapping

* Fix tests

* fix http test panic

* Pull in latest raft-snapshot library

* Add comment
2019-06-20 12:14:58 -07:00
..
circonus Bump deps 2017-09-05 18:06:47 -04:00
datadog Bump deps 2017-09-05 18:06:47 -04:00
prometheus Update Deps (#5454) 2018-10-03 09:55:26 -07:00
.gitignore Switch to go modules (#6585) 2019-04-13 03:44:06 -04:00
.travis.yml Raft Storage Backend (#6888) 2019-06-20 12:14:58 -07:00
const_unix.go Migrate to built-in Go vendoring. 2016-02-18 15:06:02 -05:00
const_windows.go Migrate to built-in Go vendoring. 2016-02-18 15:06:02 -05:00
go.mod Raft Storage Backend (#6888) 2019-06-20 12:14:58 -07:00
go.sum Raft Storage Backend (#6888) 2019-06-20 12:14:58 -07:00
inmem.go Raft Storage Backend (#6888) 2019-06-20 12:14:58 -07:00
inmem_endpoint.go Raft Storage Backend (#6888) 2019-06-20 12:14:58 -07:00
inmem_signal.go Bump deps 2017-09-05 18:06:47 -04:00
LICENSE Migrate to built-in Go vendoring. 2016-02-18 15:06:02 -05:00
metrics.go Raft Storage Backend (#6888) 2019-06-20 12:14:58 -07:00
README.md Bump Deps (#4868) 2018-07-06 12:09:34 -04:00
sink.go Bump deps 2017-09-05 18:06:47 -04:00
start.go Bump Deps (#4868) 2018-07-06 12:09:34 -04:00
statsd.go Bump deps 2017-09-05 18:06:47 -04:00
statsite.go Bump deps 2017-09-05 18:06:47 -04:00

go-metrics

This library provides a metrics package which can be used to instrument code, expose application metrics, and profile runtime performance in a flexible manner.

Current API: GoDoc

Sinks

The metrics package makes use of a MetricSink interface to support delivery to any type of backend. Currently the following sinks are provided:

  • StatsiteSink : Sinks to a statsite instance (TCP)
  • StatsdSink: Sinks to a StatsD / statsite instance (UDP)
  • PrometheusSink: Sinks to a Prometheus metrics endpoint (exposed via HTTP for scrapes)
  • InmemSink : Provides in-memory aggregation, can be used to export stats
  • FanoutSink : Sinks to multiple sinks. Enables writing to multiple statsite instances for example.
  • BlackholeSink : Sinks to nowhere

In addition to the sinks, the InmemSignal can be used to catch a signal, and dump a formatted output of recent metrics. For example, when a process gets a SIGUSR1, it can dump to stderr recent performance metrics for debugging.

Labels

Most metrics do have an equivalent ending with WithLabels, such methods allow to push metrics with labels and use some features of underlying Sinks (ex: translated into Prometheus labels).

Since some of these labels may increase greatly cardinality of metrics, the library allow to filter labels using a blacklist/whitelist filtering system which is global to all metrics.

  • If Config.AllowedLabels is not nil, then only labels specified in this value will be sent to underlying Sink, otherwise, all labels are sent by default.
  • If Config.BlockedLabels is not nil, any label specified in this value will not be sent to underlying Sinks.

By default, both Config.AllowedLabels and Config.BlockedLabels are nil, meaning that no tags are filetered at all, but it allow to a user to globally block some tags with high cardinality at application level.

Examples

Here is an example of using the package:

func SlowMethod() {
    // Profiling the runtime of a method
    defer metrics.MeasureSince([]string{"SlowMethod"}, time.Now())
}

// Configure a statsite sink as the global metrics sink
sink, _ := metrics.NewStatsiteSink("statsite:8125")
metrics.NewGlobal(metrics.DefaultConfig("service-name"), sink)

// Emit a Key/Value pair
metrics.EmitKey([]string{"questions", "meaning of life"}, 42)

Here is an example of setting up a signal handler:

// Setup the inmem sink and signal handler
inm := metrics.NewInmemSink(10*time.Second, time.Minute)
sig := metrics.DefaultInmemSignal(inm)
metrics.NewGlobal(metrics.DefaultConfig("service-name"), inm)

// Run some code
inm.SetGauge([]string{"foo"}, 42)
inm.EmitKey([]string{"bar"}, 30)

inm.IncrCounter([]string{"baz"}, 42)
inm.IncrCounter([]string{"baz"}, 1)
inm.IncrCounter([]string{"baz"}, 80)

inm.AddSample([]string{"method", "wow"}, 42)
inm.AddSample([]string{"method", "wow"}, 100)
inm.AddSample([]string{"method", "wow"}, 22)

....

When a signal comes in, output like the following will be dumped to stderr:

[2014-01-28 14:57:33.04 -0800 PST][G] 'foo': 42.000
[2014-01-28 14:57:33.04 -0800 PST][P] 'bar': 30.000
[2014-01-28 14:57:33.04 -0800 PST][C] 'baz': Count: 3 Min: 1.000 Mean: 41.000 Max: 80.000 Stddev: 39.509
[2014-01-28 14:57:33.04 -0800 PST][S] 'method.wow': Count: 3 Min: 22.000 Mean: 54.667 Max: 100.000 Stddev: 40.513