Commit Graph

1881 Commits

Author SHA1 Message Date
Jeff Mitchell 4e1470f483
Handpick cluster cipher suites when they're not user-set (#7487)
* Handpick cluster cipher suites when they're not user-set

There is an undocumented way for users to choose cluster cipher suites
but for the most part this is to paper over the fact that there are
undesirable suites in TLS 1.2.

If not explicitly set, have the set of cipher suites for the cluster
port come from a hand-picked list; either the allowed TLS 1.3 set (for
forwards compatibility) or the three identical ones for TLS 1.2.

The 1.2 suites have been supported in Go until at least as far back as
Go 1.9 from two years ago. As a result in cases where no specific suites
have been chosen this _ought_ to have no compatibility issues.

Also includes a useful test script.
2019-10-28 12:51:45 -04:00
Brian Kassouf d05b401cd8
Update token_store.go 2019-10-28 09:31:58 -07:00
Jack Kleeman 65c67dd6f3 Add a counter for root token creation (#7172)
It would be useful to be able to page on root token creation. This PR
adds a counter which increments on this event.
2019-10-28 09:30:11 -07:00
Jeff Mitchell 0c88218dd4 Port some changes that got out of sync 2019-10-28 11:38:14 -04:00
Brian Kassouf a20e73c2da
Port filtered paths changes back to OSS (#7741)
* Port filtered paths changes back to OSS

* Fix build
2019-10-27 13:30:38 -07:00
Matt Morrison 1e7acd0800 path-help missing or incorrect for raft paths (#7326) 2019-10-25 12:37:48 -04:00
Mike Jarmy 56725e694f
fix token counter test so the token won't time out (#7737) 2019-10-25 10:55:38 -04:00
ncabatoff 7c6cc95a24
Fix a regression introduced in #7698 that breaks root token generation. (#7727) 2019-10-24 10:23:31 -04:00
Vishal Nayak 23b0fb62de Abstract generate-root authentication into the strategy interface (#7698)
* Abstract generate-root authentication into the strategy interface

* Generate root strategy ncabatoff (#7700)

* Adapt to new shamir-as-kek reality.

* Don't try to verify the master key when we might still be sealed (in
recovery mode).  Instead, verify it in the authenticate methods.
2019-10-23 09:52:28 -07:00
ncabatoff 13c00dfa38
Use docker instead of an external LDAP server that sometimes goes down (#7522) 2019-10-22 13:37:41 -04:00
ncabatoff 68656a86cb
Fix a nil map pointer in mergeEntity. (#7711) 2019-10-22 09:57:24 -04:00
ncabatoff 8543da27bd
TestSysRekey_Verification would fail sometimes when recovery=true (#7710)
because when unsealing it wouldn't wait for core 0 to come up and become
the active node. Much of our testing code assumes that core0 is the
active node.
2019-10-22 09:35:48 -04:00
ncabatoff 1c98152fa0
Shamir seals now come in two varieties: legacy and new-style. (#7694)
Shamir seals now come in two varieties: legacy and new-style. Legacy
Shamir is automatically converted to new-style when a rekey operation
is performed. All new Vault initializations using Shamir are new-style.

New-style Shamir writes an encrypted master key to storage, just like
AutoUnseal. The stored master key is encrypted using the shared key that
is split via Shamir's algorithm. Thus when unsealing, we take the key
fragments given, combine them into a Key-Encryption-Key, and use that
to decrypt the master key on disk. Then the master key is used to read
the keyring that decrypts the barrier.
2019-10-18 14:46:00 -04:00
Lexman c86fe212c0
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Michael Gaffney 4f7bd872ed
core: Log if an error is returned from postSealMigration (#7675) 2019-10-16 14:00:00 -04:00
Michael Gaffney 24f663403e
core: add postSealMigration method (#7579)
* core: add postSealMigration method

The postSealMigration method is called at the end of the postUnseal
method if a seal migration has occurred. This starts a seal rewrap
process in the enterprise version of. It is a no-op in the OSS version.
2019-10-16 12:52:37 -04:00
Michael Gaffney 6ae92ede11
core: add hook for initializing seals for migration (#7666)
* core: add hook for initializing seals for migration

Needed in enterprise version.
2019-10-15 15:48:23 -04:00
Vishal Nayak 0d077d7945
Recovery Mode (#7559)
* Initial work

* rework

* s/dr/recovery

* Add sys/raw support to recovery mode (#7577)

* Factor the raw paths out so they can be run with a SystemBackend.

# Conflicts:
#	vault/logical_system.go

* Add handleLogicalRecovery which is like handleLogical but is only
sufficient for use with the sys-raw endpoint in recovery mode.  No
authentication is done yet.

* Integrate with recovery-mode.  We now handle unauthenticated sys/raw
requests, albeit on path v1/raw instead v1/sys/raw.

* Use sys/raw instead raw during recovery.

* Don't bother persisting the recovery token.  Authenticate sys/raw
requests with it.

* RecoveryMode: Support generate-root for autounseals (#7591)

* Recovery: Abstract config creation and log settings

* Recovery mode integration test. (#7600)

* Recovery: Touch up (#7607)

* Recovery: Touch up

* revert the raw backend creation changes

* Added recovery operation token prefix

* Move RawBackend to its own file

* Update API path and hit it using CLI flag on generate-root

* Fix a panic triggered when handling a request that yields a nil response. (#7618)

* Improve integ test to actually make changes while in recovery mode and
verify they're still there after coming back in regular mode.

* Refuse to allow a second recovery token to be generated.

* Resize raft cluster to size 1 and start as leader (#7626)

* RecoveryMode: Setup raft cluster post unseal (#7635)

* Setup raft cluster post unseal in recovery mode

* Remove marking as unsealed as its not needed

* Address review comments

* Accept only one seal config in recovery mode as there is no scope for migration
2019-10-15 00:55:31 -04:00
Matthew Irish 87d4e6e068
UI - raft config and snapshotting (#7410)
* add storage route

* template out the routes and new raft storage overview

* fetch raft config and add new server model

* pngcrush the favicon

* add view components and binary-file component

* add form-save-buttons component

* adjust rawRequest so that it can send a request body and returns the response on errors

* hook up restore

* rename binary-file to file-to-array-buffer

* add ember-service-worker

* use forked version of ember-service-worker for now

* scope the service worker to a single endpoint

* show both download buttons for now

* add service worker download with a fallback to JS in-mem download

* add remove peer functionality

* lint go file

* add storage-type to the cluster and node models

* update edit for to take a cancel action

* separate out a css table styles to be used by http-requests-table and on the raft-overview component

* add raft-join adapter, model, component and use on the init page

* fix styling and gate the menu item on the cluster using raft storage

* style tweaks to the raft-join component

* fix linting

* add form-save-buttons component to storybook

* add cancel functionality for backup uploads, and add a success message for successful uploads

* add component tests

* add filesize.js

* add filesize and modified date to file-to-array-buffer

* fix linting

* fix server section showing in the cluster nav

* don't use babel transforms in service worker lib because we don't want 2 copies of babel polyfill

* add file-to-array-buffer to storybook

* add comments and use removeObjectURL to raft-storage-overview

* update alert-banner markdown

* messaging change for upload alert banner

* Update ui/app/templates/components/raft-storage-restore.hbs

Co-Authored-By: Joshua Ogle <joshua@joshuaogle.com>

* more comments

* actually render the label if passed and update stories with knobs
2019-10-14 13:23:29 -05:00
Brian Kassouf 1167fad704
Improve raft write performance by utilizing FSM Batching (#7527)
* Start benchmark work

* Add batching FSM function

* dedupe some code

* Update dependency on chunking FSM

* fix raft external tests

* fix go.mod

* Add batching test

* uncomment test

* update raft deps

* update vendor

* Update physical/raft/fsm.go

Co-Authored-By: Michel Vocks <michelvocks@gmail.com>

* Update physical/raft/fsm.go
2019-10-14 09:25:07 -06:00
Brian Kassouf 024c29c36a
OSS portions of raft non-voters (#7634)
* OSS portions of raft non-voters

* add file

* Update vault/raft.go

Co-Authored-By: Vishal Nayak <vishalnayak@users.noreply.github.com>
2019-10-11 11:56:59 -07:00
isbric e6e20e9eb3 Correct spelling of error message (#7630) 2019-10-11 11:14:41 -04:00
ncabatoff c16e3bbceb
Cache whether we've been initialized to reduce load on storage (#7549) 2019-10-08 17:52:38 -04:00
Mike Jarmy 5986ce922d
add counters for active service tokens, and identity entities (#7541) 2019-10-08 13:58:19 -04:00
Calvin Leung Huang d8875b1991
sys/config: config state endpoint (#7424)
* sys/config: initial work on adding config state endpoint

* server/config: add tests, fix Sanitized method

* thread config through NewTestCluster's config to avoid panic on dev modes

* properly guard endpoint against request forwarding

* add http tests, guard against panics on nil RawConfig

* ensure non-nil rawConfig on NewTestCluster cores

* update non-forwarding logic

* fix imports; use no-forward handler

* add missing config test fixture; update gitignore

* return sanitized config as a map

* fix test, use deep.Equal to check for equality

* fix http test

* minor comment fix

* config: change Sanitized to return snake-cased keys, update tests

* core: hold rlock when reading config; add docstring

* update docstring
2019-10-08 10:57:15 -07:00
Michel Vocks a0c122926a
Unauth metrics: Fix missing parse form and error response (#7569)
* Unauth metrics: Fix missing parse form and error response

* Change metrics error response to text/plain content type always
2019-10-08 14:55:25 +02:00
Brian Shumate 66a3218331 Typo (#7586) 2019-10-07 08:08:18 -07:00
Michel Vocks a7a6dd55a5
Add config parameter to allow unauthenticated metrics access (#7550)
* Implement config parameter to allow unathenticated metricss access

* Add unit test for unauthenticated metrics access parameter

* go mod tidy
2019-10-04 09:29:51 +02:00
Calvin Leung Huang 8239612352
sys/pprof: fix pprof index description (#7564) 2019-10-03 17:02:41 -07:00
Calvin Leung Huang 920fa1452b
sys/host-info: fix host-info description typo (#7565) 2019-10-03 17:02:19 -07:00
Michael Gaffney 6b1e1909e9
core: re-encrypt barrier and recovery keys if the unseal key is updated (#7493)
Seal keys can be rotated. When this happens, the barrier and recovery
keys should be re-encrypted with the new seal key. This change
automatically re-encrypts the barrier and recovery keys with the latest
seal key on the active node during the 'postUnseal' phase.
2019-10-03 16:40:18 -04:00
Calvin Leung Huang 3f1c7c86a0
sys: add host-info endpoint (#7330)
* sys: add host-info endpoint, add client API method

* remove old commented handler

* add http tests, fix bugs

* query all partitions for disk usage

* fix Timestamp decoding

* add comments for clarification

* dont append a nil entry on disk usage query error

* remove HostInfo from the sdk api

We can use Logical().Read(...) to query this endpoint since the payload is contained with the data object. All warnings are preserved under Secret.Warnings.

* ensure that we're testing failure case against a standby node

* add and use TestWaitStandby to ensure core is on standby

* remove TestWaitStandby

* respond with local-only error

* move HostInfo into its own helper package

* fix imports; use new no-forward handler

* add cpu times to collection

* emit clearer multierrors/warnings by collection type

* add comments on HostInfo fields
2019-10-03 09:43:52 -07:00
David Adams 98a6a428c0 Add response_types_supported to OIDC configuration (#7533)
The OIDC Discovery standard requires the response_types_supported field
to be returned in the .well-known/openid-configuration response.

Also, the AWS IAM OIDC consumer won't accept Vault as an identity
provider without this field.

Based on examples in the OIDC Core documentation, it appears Vault
supports only the `id_token` flow, and thus that is the only value that
makes sense to be set in this field. See:

https://openid.net/specs/openid-connect-core-1_0.html#AuthorizationExamples
2019-10-02 08:59:57 -07:00
ncabatoff e7fe4b6d92
Return a useful error on attempts to renew a token via sys/leases/renew (#7298) 2019-10-02 10:55:20 -04:00
Vishal Nayak c84f804f48
Fix identity case sensitivity loading in secondary cluster (#7327)
* Fix identity case sensitivity loading in secondary cluster

* Add nil check
2019-09-30 10:27:25 -04:00
Connor Zapfel f09b88b71e Added sys/health path-help content (#7360) 2019-09-26 13:16:21 -07:00
Calvin Leung Huang f4fd84d52b
sys: add pprof endpoint (#7473)
* sys/pprof: add pprof routes to the system backend

* sys/pprof: add pprof paths to handler with local-only check

* fix trailing slash on pprof index endpoint

* use new no-forward handler on pprof

* go mod tidy

* add pprof external tests

* disallow streaming requests to exceed DefaultMaxRequestDuration

* add max request duration test
2019-09-19 13:44:37 -07:00
Lars Lehtonen 404cb5f6dd Fix token_store_test.go (#7490)
* vault: fix dropped error in test goroutine

* vault: fix dropped test errors
2019-09-18 14:18:08 -07:00
Matthew Irish 6e4cc02f4d
expose 'storage_type' on the sys/seal-status endpoint (#7486)
* expose 'storage_type' on the sys/seal-status endpoint

* add comments

* Update vault/core.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
2019-09-18 14:07:18 -05:00
Calvin Leung Huang 0caff07201
testing: set MetricsHelper if it not provided through base config (#7480) 2019-09-16 17:50:51 -07:00
Calvin Leung Huang 801e15ffe3
http: add handleLogicalNoForward handler for local-only requests (#7482)
* http: add handleLogicalNoForward handler for local-only requests

* fix missing no-forward case; add docstring

* add docstring for handleLogicalInternal

* fix docstring reference

* fix comment on not-ok case
2019-09-16 17:50:28 -07:00
Jeff Mitchell a965378b17
Support batch tokens in renewer and caching proxy for transit auto-unseal (#7441)
Currently this uses a fork of the api.Renewer code, which we should consolidate in 1.3.
2019-09-09 12:58:28 -07:00
Calvin Leung Huang ec64b7c672
logical/request: store the entire http.Request object instead (#7412)
This allows logical operations (along with a non-nil response writer) to
process http handler funcs within the operation function while keeping
auth and audit checks that the logical request flow provides.
2019-09-06 12:40:15 -07:00
Vu Pham 3e06f71ea8 OCI KMS auto-unseal plugin (#6950) 2019-09-04 12:40:54 -07:00
ncabatoff ed147b7ae7
Make clusterListener an atomic.Value to avoid races with getGRPCDialer. (#7408) 2019-09-03 11:59:56 -04:00
Jeff Mitchell 9816963355
Move SudoPrivilege out of SystemView (#7266)
* Move SudoPrivilege out of SystemView

We only use this in token store and it literally doesn't work anything
that isn't the token store or system mount, so we should stop exposing
something that doesn't work.

* Reconcile extended system view with sdk/logical a bit and put an explanation for why SudoPrivilege isn't moved over
2019-08-26 10:23:46 -04:00
ncabatoff 366add2979
Generalization of the PhysicalFactory notion introduced by Raft (#7217)
Generalization of the PhysicalFactory notion introduced by Raft, so it can be used by other storage backends in tests.  These are the OSS changes needed for my rework of the ent integ tests and cluster helpers.
2019-08-23 15:51:25 -04:00
Najib Ben 64936d5038 Remove 512 entity limit for groups (#7317)
* Consul 1.5.3 has configurable value limit for KV storage
* Integrated Raft
2019-08-14 13:47:11 -04:00
Jim Kalafut 72a15422d5
Fix identity store 'key not found' response (#7267)
The existing custom response results in a 400 instead of the typical
404 which confuses the Terraform provider (and is inconsistent).
2019-08-07 09:46:45 -07:00
ncabatoff e1f8a82d81
Create test cores with an error injector. (#7243)
It's created with a 0% error rate, which means it's a no-op, but tests can opt-in to errors when
needed via core.underlyingPhysical.
2019-08-06 15:21:23 -04:00