Commit Graph

675 Commits

Author SHA1 Message Date
hc-github-team-secure-vault-core 69053efc00
backport of commit 6157b5ffcb6e0fcd1d0823e4929bf50ef1b00bc8 (#20837)
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-06-21 22:56:07 +00:00
hc-github-team-secure-vault-core 36365ed7f4
backport of commit 3a46ecc389e9096ccea6c6f847b68ada7f8068d7 (#21362)
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
2023-06-21 14:01:13 +00:00
hc-github-team-secure-vault-core 433197bf2b
backport of commit 6b87a087afe24e8bd52a2b9ada4b506b20d4fc7e (#21254)
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-06-15 11:53:16 -04:00
Victor Rodriguez 2656c020ae
Convert seal.Access struct into a interface (OSS) (#20510)
* Move seal barrier type field from Access to autoSeal struct.

Remove method Access.SetType(), which was only being used by a single test, and
which can use the name option of NewTestSeal() to specify the type.

* Change method signatures of Access to match those of Wrapper.

* Turn seal.Access struct into an interface.

* Tweak Access implementation.

Change `access` struct to have a field of type wrapping.Wrapper, rather than
extending it.

* Add method Seal.GetShamirWrapper().

Add method Seal.GetShamirWrapper() for use by code that need to perform
Shamir-specific operations.
2023-05-04 14:22:30 -04:00
Nick Cabatoff 22b00eba12
Add support for docker testclusters (#20247) 2023-04-24 14:25:50 -04:00
Peter Wilson a2bdf7250b
VAULT-14048: raft-autopilot appears to refuse to remove a node which has left and wouldn't impact stability (#19472)
* ensure we supply the node type when it's for a voter
* bumped autopilot version back to v0.2.0 and ran go mod tidy
* changed condition in knownservers and added some comments
* Export GetRaftBackend
* Updated tests for autopilot (related to dead server cleanup)
* Export Raft NewDelegate

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-04-03 11:58:57 -04:00
Hamid Ghaf 85ead99d64
vault-12244 (#19591)
* vault-12244

* CL
2023-03-17 07:52:54 -07:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Hamid Ghaf 4822d4ab6d
replace existing zookeeper import with an actively maintained one (#19086)
* replace existing zookeeper import with an actively maintained one

* remove empty lines
2023-02-10 11:56:27 -05:00
miagilepner e873932bce
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
Hamid Ghaf edbf093290
Allow Token Create Requests To Be Replicated (#18689)
* Allow Token Create Requests To Be Replicated

* adding a test

* revert a test
2023-01-24 14:00:27 -05:00
Josh Black c9763996d4
Enable undo logs by default (#18692)
* Enable undo logs by default

* add consul test

* update go.mod/sum

* add a better non-existent key
2023-01-17 13:38:18 -08:00
Mike Palmiotto 41f8d2edc6
[QT-309] Ensure creds are available for OCI and S3 (#18602)
* Ensure OCI creds are set for acc test

* Ensure AWS creds are resolvable before testing

Co-authored-by: Michael Anthony <5498095+manthonygfp@users.noreply.github.com>
2023-01-17 14:15:40 -05:00
Nick Cabatoff b5f19fffe9
Speculative fix for a panic that might arise during raft teardown (#18704) 2023-01-16 13:49:11 -05:00
Mike Palmiotto 28d99481d3
Fix race in fsm.db (#18386)
We need to take a read lock when reading any of the FSM fields. Expose a
new fsm.Stats to handle a racy read and make sure we're consistently using
the f.db read lock wrappers.
2022-12-15 10:04:27 -05:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Josh Black e75633eddc
Don't panic on unknown raft ops (#17732)
* Don't panic on unknown raft ops

* avoid excessive logging

* track at the struct level, not the function level

* add changelog
2022-11-30 15:37:58 -08:00
Tom Proctor dc85e37cf4
storage/raft: Add retry_join_as_non_voter config option (#18030) 2022-11-18 17:58:16 +00:00
Alexander Scheel 8e6e53cf63
Use hashicorp mirror for container pulls (#17778)
When running the test suite in CI (where requests are centralized from
relatively few IPs), we'd occasionally hit Dockerhub's rate limits.
Luckily Hashicorp runs a (limited) public mirror of the containers we
need, so we can switch to them here in the tests.

For consistency between developer and CI, we've opted to have the tests
always pull from the Hashicorp mirror, rather than updating the CI
runner to prefer the mirror.

We exclude nomad and influxdb as we don't presently mirror these repos.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-02 13:33:17 -04:00
akshya96 1e189016e2
update protoc version to 3.21.7 oss (#17499)
* update protoc to 3.21.7

* adding changelog
2022-10-26 16:49:44 -07:00
Nick Cabatoff 39c7e7c191
Add more raft metrics, emit more metrics on non-perf standbys (#12166)
Add some metrics helpful for monitoring raft cluster state.

Furthermore, we weren't emitting bolt metrics on regular (non-perf) standbys, and there were other metrics
in metricsLoop that would make sense to include in OSS but weren't.  We now have an active-node-only func,
emitMetricsActiveNode.  This runs metricsLoop on the active node.  Standbys and perf-standbys run metricsLoop
from a goroutine managed by the runStandby rungroup.
2022-10-07 09:09:08 -07:00
Josh Black c45c6e51c0
only enable undo logs if all cluster members support it (#17378) 2022-10-06 11:24:16 -07:00
Steven Clark dae2ef535b
Update protos to match update of protobuf go library (#17215) 2022-09-19 16:45:44 -04:00
Josh Black 04a2396573
Adjust raft transactions to be safer with get operations (#17151) 2022-09-16 09:35:48 -07:00
Josh Black 1e6401a8eb
make proto (#17120) 2022-09-13 16:06:11 -04:00
Josh Black 6d94dd991d
merkle sync undo logs (#17103) 2022-09-13 10:03:19 -07:00
Josh Black d8e0a13aae
update gofumpt to 0.3.1 and reformat the repo (#17055)
* update gofumpt to 0.3.1 and reformat the repo

* output the version of the formatter we're using
2022-09-07 17:31:20 -07:00
Ivan Buymov 99213c2616
Add retry policy and fix documentation for Cassandra storage backend (#10467)
* add simple_retry policy and initial_connection_timeout options,
fix docs for connection_timeout

* Cassandra: policy fix - added changelog.

Co-authored-by: Mehdi Ahmadi <aphorise@gmail.com>
2022-08-30 11:00:48 -07:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
Eng Zer Jun 61262ad98e
refactor: replace strings.Replace with strings.ReplaceAll (#15392)
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-08-03 15:22:48 -04:00
swayne275 4632a26a09
Use %q for quoted strings where appropriate (#15216)
* change '%s' to %q where single vs double quotes shouldn't matter

* replace double quotes with %q in logs and errors
2022-08-03 12:32:45 -06:00
Austin Gebauer 7df39640e0
Update gopsutil to v3 to fix MacOS deprecation warnings (#16321)
* Update gopsutil to v3

* Adds v2 field names in host-info response to allow eventual deprecation in favor of v3 field names

* Map v3 to v2 field names to keep host-info api compat

* copy gopsutil license into source
2022-07-20 16:37:10 -07:00
Violet Hynes 0c80ee5cf5
VAULT-6614 Enable role based quotas for lease-count quotas (OSS) (#16157)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* VAULT-6614 start of changes for roles in LCQs

* Expiration changes for leases

* Add role information to RequestAuth

* VAULT-6614 Test updates

* VAULT-6614 Add expiration test with roles

* VAULT-6614 fix comment

* VAULT-6614 Protobuf on OSS

* VAULT-6614 Add rlock to determine role code

* VAULT-6614 Try lock instead of rlock

* VAULT-6614 back to rlock while I think about this more

* VAULT-6614 Additional safety for nil dereference

* VAULT-6614 Use %q over %s

* VAULT-6614 Add overloading to plugin backends

* VAULT-6614 RLocks instead

* VAULT-6614 Fix return for backend factory
2022-07-05 13:02:00 -04:00
bhowe34 763f9ad732
pass context to postgres queries (#15866)
* pass context to postgres queries

* add changelog

* Update changelog/15866.txt

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
2022-06-08 17:54:19 -04:00
Josh Black 9c48c62d6e
Use the incoming request version to populate follower state (#15806) 2022-06-06 08:44:24 -07:00
Josh Black a3d44a46c8
autopilot_upgrade_version should be parseable (#15590) 2022-05-25 09:09:45 -07:00
Jim Kalafut a3b0b60a73
postgres: replace the package lib/pq with pgx (#15343)
* WIP replacing lib/pq

* change timezome param to be URI format

* add changelog

* add changelog for redshift

* update changelog

* add test for DSN style connection string

* more parseurl and quoteidentify to sdk; include copyright and license

* call dbutil.ParseURL instead, fix import ordering

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2022-05-23 12:49:18 -07:00
Josh Black 416504d8c3
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
Nick Cabatoff bc9f69af2e
Forward autopilot state reqs, avoid self-dialing (#15493)
Make sure that autopilot is disabled when we step down from active node state.  Forward autopilot state requests to the active node.  Avoid self-dialing due to stale advertisement.
2022-05-18 14:50:18 -04:00
Nick Cabatoff c5928c1d15
Raft: use a larger initial heartbeat/election timeout (#15042) 2022-04-29 08:32:16 -04:00
Sergey Lanzman 90b12f1386
Add AWS_DYNAMODB_REGION Environment variable (#15054)
Added AWS_DYNAMODB_REGION env
2022-04-28 12:29:51 -07:00
Rémi Lapeyre bf4c4595f3
secrets/consul: Add support to auto-bootstrap Consul ACL system (#10751)
* Automatically bootstraps the Consul ACL system if no management token is given on the access config
2022-04-20 17:16:15 -05:00
Josh Black 41a4b7a170
Ensure initialMmapSize is 0 on Windows (#14977)
* ensure initialMmapSize is 0 on windows

* add changelog
2022-04-08 12:07:21 -07:00
Austin Gebauer f9ce8d7615
Adds Vault version prerelease and metadata to logical.PluginEnvironment (#14851) 2022-04-04 22:31:01 -07:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00
hghaf099 aafb5d6427
VAULT-4240 time.After() in a select statement can lead to memory leak (#14814)
* VAULT-4240 time.After() in a select statement can lead to memory leak

* CL
2022-04-01 10:17:11 -04:00
Joel Kenny 825ffd130a
cockroachdb: add high-availability support (#12965)
This commit adds high-availability support to the CockroachDB backend. The
locking strategy implemented is heavily influenced from the very similar
Postgres backend.
2022-03-29 13:12:06 -04:00
Dave Rawks 35ec91f1ca
Increase column width of vault_key on mysql (#14231)
* resolves The default schema used in the mysql backend is insufficient for KVv2 storage #14114
* increases column width of vault_key from 512 to 3072 in mysql physical backend
* updates changelog
2022-02-24 09:21:57 -05:00
Nick Cabatoff 5fe1c16201
Remove support for etcd v2 storage backend. (#14193) 2022-02-22 16:48:04 -05:00