open-consul/agent/structs/connect_ca.go

340 lines
9.7 KiB
Go
Raw Normal View History

package structs
import (
"fmt"
Fix CA pruning when CA config uses string durations. (#4669) * Fix CA pruning when CA config uses string durations. The tl;dr here is: - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported - Most of our tests managed to escape this by defining them as time.Duration directly - Out actual default value is a string - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC). - msgpack decode leaves the string as a `[]uint8` - Some of our parsers required string and failed - So after 1 hour, a default configured server would throw an error about pruning old CAs - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA. - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding. tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them. This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time! * Typo fix * Fix bad Uint8 usage
2018-09-13 14:43:00 +00:00
"reflect"
2018-03-21 17:55:39 +00:00
"time"
"github.com/mitchellh/mapstructure"
)
// IndexedCARoots is the list of currently trusted CA Roots.
type IndexedCARoots struct {
// ActiveRootID is the ID of a root in Roots that is the active CA root.
// Other roots are still valid if they're in the Roots list but are in
// the process of being rotated out.
ActiveRootID string
2018-05-08 13:23:44 +00:00
// TrustDomain is the identification root for this Consul cluster. All
// certificates signed by the cluster's CA must have their identifying URI in
// this domain.
//
// This does not include the protocol (currently spiffe://) since we may
// implement other protocols in future with equivalent semantics. It should be
// compared against the "authority" section of a URI (i.e. host:port).
//
// We need to support migrating a cluster between trust domains to support
// Multi-DC migration in Enterprise. In this case the current trust domain is
// here but entries in Roots may also have ExternalTrustDomain set to a
// non-empty value implying they were previous roots that are still trusted
// but under a different trust domain.
//
// Note that we DON'T validate trust domain during AuthZ since it causes
// issues of loss of connectivity during migration between trust domains. The
// only time the additional validation adds value is where the cluster shares
// an external root (e.g. organization-wide root) with another distinct Consul
// cluster or PKI system. In this case, x509 Name Constraints can be added to
// enforce that Consul's CA can only validly sign or trust certs within the
// same trust-domain. Name constraints as enforced by TLS handshake also allow
// seamless rotation between trust domains thanks to cross-signing.
2018-05-08 13:23:44 +00:00
TrustDomain string
// Roots is a list of root CA certs to trust.
Roots []*CARoot
2018-03-19 05:07:52 +00:00
// QueryMeta contains the meta sent via a header. We ignore for JSON
// so this whole structure can be returned.
QueryMeta `json:"-"`
}
// CARoot represents a root CA certificate that is trusted.
type CARoot struct {
// ID is a globally unique ID (UUID) representing this CA root.
ID string
// Name is a human-friendly name for this CA root. This value is
// opaque to Consul and is not used for anything internally.
Name string
2018-05-04 23:01:54 +00:00
// SerialNumber is the x509 serial number of the certificate.
SerialNumber uint64
// SigningKeyID is the ID of the public key that corresponds to the private
// key used to sign the certificate. Is is the HexString format of the raw
// AuthorityKeyID bytes.
2018-05-04 23:01:54 +00:00
SigningKeyID string
// ExternalTrustDomain is the trust domain this root was generated under. It
// is usually empty implying "the current cluster trust-domain". It is set
// only in the case that a cluster changes trust domain and then all old roots
// that are still trusted have the old trust domain set here.
//
// We currently DON'T validate these trust domains explicitly anywhere, see
// IndexedRoots.TrustDomain doc. We retain this information for debugging and
// future flexibility.
ExternalTrustDomain string
2018-05-04 23:01:54 +00:00
// Time validity bounds.
NotBefore time.Time
NotAfter time.Time
// RootCert is the PEM-encoded public certificate.
RootCert string
// IntermediateCerts is a list of PEM-encoded intermediate certs to
// attach to any leaf certs signed by this CA.
IntermediateCerts []string
// SigningCert is the PEM-encoded signing certificate and SigningKey
// is the PEM-encoded private key for the signing certificate. These
// may actually be empty if the CA plugin in use manages these for us.
SigningCert string `json:",omitempty"`
SigningKey string `json:",omitempty"`
2018-03-20 03:29:14 +00:00
// Active is true if this is the current active CA. This must only
// be true for exactly one CA. For any method that modifies roots in the
// state store, tests should be written to verify that multiple roots
// cannot be active.
Active bool
// RotatedOutAt is the time at which this CA was removed from the state.
// This will only be set on roots that have been rotated out from being the
// active root.
RotatedOutAt time.Time `json:"-"`
RaftIndex
}
// CARoots is a list of CARoot structures.
type CARoots []*CARoot
// CASignRequest is the request for signing a service certificate.
type CASignRequest struct {
// Datacenter is the target for this request.
Datacenter string
// CSR is the PEM-encoded CSR.
CSR string
// WriteRequest is a common struct containing ACL tokens and other
// write-related common elements for requests.
WriteRequest
}
// RequestDatacenter returns the datacenter for a given request.
func (q *CASignRequest) RequestDatacenter() string {
return q.Datacenter
}
// IssuedCert is a certificate that has been issued by a Connect CA.
type IssuedCert struct {
// SerialNumber is the unique serial number for this certificate.
// This is encoded in standard hex separated by :.
SerialNumber string
2018-03-21 17:55:39 +00:00
// CertPEM and PrivateKeyPEM are the PEM-encoded certificate and private
// key for that cert, respectively. This should not be stored in the
// state store, but is present in the sign API response.
2018-03-21 17:55:39 +00:00
CertPEM string `json:",omitempty"`
PrivateKeyPEM string `json:",omitempty"`
2018-03-21 17:55:39 +00:00
// Service is the name of the service for which the cert was issued.
// ServiceURI is the cert URI value.
Service string
ServiceURI string
// ValidAfter and ValidBefore are the validity periods for the
// certificate.
ValidAfter time.Time
ValidBefore time.Time
RaftIndex
}
// CAOp is the operation for a request related to intentions.
type CAOp string
const (
CAOpSetRoots CAOp = "set-roots"
CAOpSetConfig CAOp = "set-config"
CAOpSetProviderState CAOp = "set-provider-state"
CAOpDeleteProviderState CAOp = "delete-provider-state"
CAOpSetRootsAndConfig CAOp = "set-roots-config"
)
// CARequest is used to modify connect CA data. This is used by the
// FSM (agent/consul/fsm) to apply changes.
type CARequest struct {
// Op is the type of operation being requested. This determines what
// other fields are required.
Op CAOp
// Datacenter is the target for this request.
Datacenter string
// Index is used by CAOpSetRoots and CAOpSetConfig for a CAS operation.
Index uint64
// Roots is a list of roots. This is used for CAOpSet. One root must
// always be active.
Roots []*CARoot
// Config is the configuration for the current CA plugin.
Config *CAConfiguration
// ProviderState is the state for the builtin CA provider.
ProviderState *CAConsulProviderState
// WriteRequest is a common struct containing ACL tokens and other
// write-related common elements for requests.
WriteRequest
}
// RequestDatacenter returns the datacenter for a given request.
func (q *CARequest) RequestDatacenter() string {
return q.Datacenter
}
const (
ConsulCAProvider = "consul"
2018-06-13 08:40:03 +00:00
VaultCAProvider = "vault"
)
// CAConfiguration is the configuration for the current CA plugin.
type CAConfiguration struct {
// ClusterID is a unique identifier for the cluster
ClusterID string `json:"-"`
// Provider is the CA provider implementation to use.
Provider string
// Configuration is arbitrary configuration for the provider. This
// should only contain primitive values and containers (such as lists
// and maps).
Config map[string]interface{}
RaftIndex
}
func (c *CAConfiguration) GetCommonConfig() (*CommonCAProviderConfig, error) {
if c == nil {
return nil, fmt.Errorf("config map was nil")
}
var config CommonCAProviderConfig
decodeConf := &mapstructure.DecoderConfig{
Fix CA pruning when CA config uses string durations. (#4669) * Fix CA pruning when CA config uses string durations. The tl;dr here is: - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported - Most of our tests managed to escape this by defining them as time.Duration directly - Out actual default value is a string - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC). - msgpack decode leaves the string as a `[]uint8` - Some of our parsers required string and failed - So after 1 hour, a default configured server would throw an error about pruning old CAs - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA. - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding. tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them. This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time! * Typo fix * Fix bad Uint8 usage
2018-09-13 14:43:00 +00:00
DecodeHook: ParseDurationFunc(),
Result: &config,
WeaklyTypedInput: true,
}
decoder, err := mapstructure.NewDecoder(decodeConf)
if err != nil {
return nil, err
}
if err := decoder.Decode(c.Config); err != nil {
return nil, fmt.Errorf("error decoding config: %s", err)
}
return &config, nil
}
type CommonCAProviderConfig struct {
LeafCertTTL time.Duration
SkipValidate bool
}
func (c CommonCAProviderConfig) Validate() error {
if c.SkipValidate {
return nil
}
if c.LeafCertTTL < time.Hour {
return fmt.Errorf("leaf cert TTL must be greater than 1h")
}
if c.LeafCertTTL > 365*24*time.Hour {
return fmt.Errorf("leaf cert TTL must be less than 1 year")
}
return nil
}
type ConsulCAProviderConfig struct {
CommonCAProviderConfig `mapstructure:",squash"`
PrivateKey string
RootCert string
RotationPeriod time.Duration
}
// CAConsulProviderState is used to track the built-in Consul CA provider's state.
type CAConsulProviderState struct {
ID string
PrivateKey string
RootCert string
IntermediateCert string
RaftIndex
}
2018-06-13 08:40:03 +00:00
type VaultCAProviderConfig struct {
CommonCAProviderConfig `mapstructure:",squash"`
2018-06-13 08:40:03 +00:00
Address string
Token string
RootPKIPath string
IntermediatePKIPath string
CAFile string
CAPath string
CertFile string
KeyFile string
TLSServerName string
TLSSkipVerify bool
2018-06-13 08:40:03 +00:00
}
Fix CA pruning when CA config uses string durations. (#4669) * Fix CA pruning when CA config uses string durations. The tl;dr here is: - Configuring LeafCertTTL with a string like "72h" is how we do it by default and should be supported - Most of our tests managed to escape this by defining them as time.Duration directly - Out actual default value is a string - Since this is stored in a map[string]interface{} config, when it is written to Raft it goes through a msgpack encode/decode cycle (even though it's written from server not over RPC). - msgpack decode leaves the string as a `[]uint8` - Some of our parsers required string and failed - So after 1 hour, a default configured server would throw an error about pruning old CAs - If a new CA was configured that set LeafCertTTL as a time.Duration, things might be OK after that, but if a new CA was just configured from config file, intialization would cause same issue but always fail still so would never prune the old CA. - Mostly this is just a janky error that got passed tests due to many levels of complicated encoding/decoding. tl;dr of the tl;dr: Yay for type safety. Map[string]interface{} combined with msgpack always goes wrong but we somehow get bitten every time in a new way :D We already fixed this once! The main CA config had the same problem so @kyhavlov already wrote the mapstructure DecodeHook that fixes it. It wasn't used in several places it needed to be and one of those is notw in `structs` which caused a dependency cycle so I've moved them. This adds a whole new test thta explicitly tests the case that broke here. It also adds tests that would have failed in other places before (Consul and Vaul provider parsing functions). I'm not sure if they would ever be affected as it is now as we've not seen things broken with them but it seems better to explicitly test that and support it to not be bitten a third time! * Typo fix * Fix bad Uint8 usage
2018-09-13 14:43:00 +00:00
// ParseDurationFunc is a mapstructure hook for decoding a string or
// []uint8 into a time.Duration value.
func ParseDurationFunc() mapstructure.DecodeHookFunc {
return func(
f reflect.Type,
t reflect.Type,
data interface{}) (interface{}, error) {
var v time.Duration
if t != reflect.TypeOf(v) {
return data, nil
}
switch {
case f.Kind() == reflect.String:
if dur, err := time.ParseDuration(data.(string)); err != nil {
return nil, err
} else {
v = dur
}
return v, nil
case f == reflect.SliceOf(reflect.TypeOf(uint8(0))):
s := Uint8ToString(data.([]uint8))
if dur, err := time.ParseDuration(s); err != nil {
return nil, err
} else {
v = dur
}
return v, nil
default:
return data, nil
}
}
}
func Uint8ToString(bs []uint8) string {
b := make([]byte, len(bs))
for i, v := range bs {
b[i] = byte(v)
}
return string(b)
}