Commit Graph

651 Commits

Author SHA1 Message Date
Josh Black 04a2396573
Adjust raft transactions to be safer with get operations (#17151) 2022-09-16 09:35:48 -07:00
Josh Black 1e6401a8eb
make proto (#17120) 2022-09-13 16:06:11 -04:00
Josh Black 6d94dd991d
merkle sync undo logs (#17103) 2022-09-13 10:03:19 -07:00
Josh Black d8e0a13aae
update gofumpt to 0.3.1 and reformat the repo (#17055)
* update gofumpt to 0.3.1 and reformat the repo

* output the version of the formatter we're using
2022-09-07 17:31:20 -07:00
Ivan Buymov 99213c2616
Add retry policy and fix documentation for Cassandra storage backend (#10467)
* add simple_retry policy and initial_connection_timeout options,
fix docs for connection_timeout

* Cassandra: policy fix - added changelog.

Co-authored-by: Mehdi Ahmadi <aphorise@gmail.com>
2022-08-30 11:00:48 -07:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
Eng Zer Jun 61262ad98e
refactor: replace strings.Replace with strings.ReplaceAll (#15392)
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-08-03 15:22:48 -04:00
swayne275 4632a26a09
Use %q for quoted strings where appropriate (#15216)
* change '%s' to %q where single vs double quotes shouldn't matter

* replace double quotes with %q in logs and errors
2022-08-03 12:32:45 -06:00
Austin Gebauer 7df39640e0
Update gopsutil to v3 to fix MacOS deprecation warnings (#16321)
* Update gopsutil to v3

* Adds v2 field names in host-info response to allow eventual deprecation in favor of v3 field names

* Map v3 to v2 field names to keep host-info api compat

* copy gopsutil license into source
2022-07-20 16:37:10 -07:00
Violet Hynes 0c80ee5cf5
VAULT-6614 Enable role based quotas for lease-count quotas (OSS) (#16157)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* VAULT-6614 start of changes for roles in LCQs

* Expiration changes for leases

* Add role information to RequestAuth

* VAULT-6614 Test updates

* VAULT-6614 Add expiration test with roles

* VAULT-6614 fix comment

* VAULT-6614 Protobuf on OSS

* VAULT-6614 Add rlock to determine role code

* VAULT-6614 Try lock instead of rlock

* VAULT-6614 back to rlock while I think about this more

* VAULT-6614 Additional safety for nil dereference

* VAULT-6614 Use %q over %s

* VAULT-6614 Add overloading to plugin backends

* VAULT-6614 RLocks instead

* VAULT-6614 Fix return for backend factory
2022-07-05 13:02:00 -04:00
bhowe34 763f9ad732
pass context to postgres queries (#15866)
* pass context to postgres queries

* add changelog

* Update changelog/15866.txt

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
2022-06-08 17:54:19 -04:00
Josh Black 9c48c62d6e
Use the incoming request version to populate follower state (#15806) 2022-06-06 08:44:24 -07:00
Josh Black a3d44a46c8
autopilot_upgrade_version should be parseable (#15590) 2022-05-25 09:09:45 -07:00
Jim Kalafut a3b0b60a73
postgres: replace the package lib/pq with pgx (#15343)
* WIP replacing lib/pq

* change timezome param to be URI format

* add changelog

* add changelog for redshift

* update changelog

* add test for DSN style connection string

* more parseurl and quoteidentify to sdk; include copyright and license

* call dbutil.ParseURL instead, fix import ordering

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2022-05-23 12:49:18 -07:00
Josh Black 416504d8c3
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
Nick Cabatoff bc9f69af2e
Forward autopilot state reqs, avoid self-dialing (#15493)
Make sure that autopilot is disabled when we step down from active node state.  Forward autopilot state requests to the active node.  Avoid self-dialing due to stale advertisement.
2022-05-18 14:50:18 -04:00
Nick Cabatoff c5928c1d15
Raft: use a larger initial heartbeat/election timeout (#15042) 2022-04-29 08:32:16 -04:00
Sergey Lanzman 90b12f1386
Add AWS_DYNAMODB_REGION Environment variable (#15054)
Added AWS_DYNAMODB_REGION env
2022-04-28 12:29:51 -07:00
Rémi Lapeyre bf4c4595f3
secrets/consul: Add support to auto-bootstrap Consul ACL system (#10751)
* Automatically bootstraps the Consul ACL system if no management token is given on the access config
2022-04-20 17:16:15 -05:00
Josh Black 41a4b7a170
Ensure initialMmapSize is 0 on Windows (#14977)
* ensure initialMmapSize is 0 on windows

* add changelog
2022-04-08 12:07:21 -07:00
Austin Gebauer f9ce8d7615
Adds Vault version prerelease and metadata to logical.PluginEnvironment (#14851) 2022-04-04 22:31:01 -07:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00
hghaf099 aafb5d6427
VAULT-4240 time.After() in a select statement can lead to memory leak (#14814)
* VAULT-4240 time.After() in a select statement can lead to memory leak

* CL
2022-04-01 10:17:11 -04:00
Joel Kenny 825ffd130a
cockroachdb: add high-availability support (#12965)
This commit adds high-availability support to the CockroachDB backend. The
locking strategy implemented is heavily influenced from the very similar
Postgres backend.
2022-03-29 13:12:06 -04:00
Dave Rawks 35ec91f1ca
Increase column width of vault_key on mysql (#14231)
* resolves The default schema used in the mysql backend is insufficient for KVv2 storage #14114
* increases column width of vault_key from 512 to 3072 in mysql physical backend
* updates changelog
2022-02-24 09:21:57 -05:00
Nick Cabatoff 5fe1c16201
Remove support for etcd v2 storage backend. (#14193) 2022-02-22 16:48:04 -05:00
Josh Black e83471d7de
Login MFA (#14025)
* Login MFA

* ENT OSS segragation (#14088)

* Delete method id if not used in an MFA enforcement config (#14063)

* Delete an MFA methodID only if it is not used by an MFA enforcement config

* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path

* adding use_passcode field to DUO config (#14059)

* add changelog

* preventing replay attack on MFA passcodes (#14056)

* preventing replay attack on MFA passcodes

* using %w instead of %s for error

* Improve CLI command for login mfa (#14106)

CLI prints a warning message indicating the login request needs to get validated

* adding the validity period of a passcode to error messages (#14115)

* PR feedback

* duo to handle preventing passcode reuse

Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
2022-02-17 13:08:51 -08:00
John-Michael Faircloth 1cf74e1179
feature: multiplexing support for database plugins (#14033)
* feat: DB plugin multiplexing (#13734)

* WIP: start from main and get a plugin runner from core

* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode

* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient

* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient

* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists

* add ID to dbplugin.DatabasePluginClient struct

* only create one plugin process per plugin type

* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server

* add v6 multiplexing  protocol version

* WIP: backwards compat for db plugins

* Ensure locking on plugin catalog access

- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin

* use the New constructor for db plugins

* grpc server: use write lock for Close and rlock for CRUD

* cleanup MultiplexedClients on Close

* remove TODO

* fix multiplexing regression with grpc server connection

* cleanup grpc server instances on close

* embed ClientProtocol in Multiplexer interface

* use PluginClientConfig arg to make NewPluginClient plugin type agnostic

* create a new plugin process for non-muxed plugins

* feat: plugin multiplexing: handle plugin client cleanup (#13896)

* use closure for plugin client cleanup

* log and return errors; add comments

* move rpcClient wrapping to core for ID injection

* refactor core plugin client and sdk

* remove unused ID method

* refactor and only wrap clientConn on multiplexed plugins

* rename structs and do not export types

* Slight refactor of system view interface

* Revert "Slight refactor of system view interface"

This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.

* Revert "Revert "Slight refactor of system view interface""

This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.

* only provide pluginRunner arg to the internal newPluginClient method

* embed ClientProtocol in pluginClient and name logger

* Add back MLock support

* remove enableMlock arg from setupPluginCatalog

* rename plugin util interface to PluginClient

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>

* feature: multiplexing: fix unit tests (#14007)

* fix grpc_server tests and add coverage

* update run_config tests

* add happy path test case for grpc_server ID from context

* update test helpers

* feat: multiplexing: handle v5 plugin compiled with new sdk

* add mux supported flag and increase test coverage

* set multiplexingSupport field in plugin server

* remove multiplexingSupport field in sdk

* revert postgres to non-multiplexed

* add comments on grpc server fields

* use pointer receiver on grpc server methods

* add changelog

* use pointer for grpcserver instance

* Use a gRPC server to determine if a plugin should be multiplexed

* Apply suggestions from code review

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* add lock to removePluginClient

* add multiplexingSupport field to externalPlugin struct

* do not send nil to grpc MultiplexingSupport

* check err before logging

* handle locking scenario for cleanupFunc

* allow ServeConfigMultiplex to dispense v5 plugin

* reposition structs, add err check and comments

* add comment on locking for cleanupExternalPlugin

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2022-02-17 08:50:33 -06:00
Robert d0832a1993
secret/consul: Add support for consul namespaces and admin partitions (#13850)
* Add support for consul namespaces and admin partitions
2022-02-09 15:44:00 -06:00
Josh Black d249fad2df
reformat using 'make fmt' (#13794) 2022-01-27 10:06:34 -08:00
davidadeleon 96dfbfbd02
Raft/fix raft telemetry metric unit (#13749)
Converting raft time metrics to Milliseconds over Default Nanoseconds to maintain consistency
2022-01-24 10:51:35 -05:00
Nick Cabatoff 07555c8bfc
Depend explicitly on go-msgpack v1.1.5 (#13693) 2022-01-19 10:32:19 -05:00
Nick Cabatoff 4ee4374b3e
Use MAP_POPULATE for our bbolt mmaps (#13573)
* Use MAP_POPULATE for our bbolt mmaps, assuming the files fit in memory.  This should improve startup times when freelist sync is disabled.
2022-01-11 08:16:53 -05:00
Scott Miller 89f617a97c
Convert to Go 1.17 go:build directive (#13579) 2022-01-05 12:02:03 -06:00
Matt Mukerjee f574e75f11
fix consul tls error checking (#13135)
We don't check for errors in the consul storage TLS setup. We might fail here
because of a missing certificate, bad permissions, etc. If anything is wrong,
vault just ignores the issues and continues, resulting in a lot of confusion.

Instead, lets return an error to the caller if this fails.
2021-12-21 09:38:58 -08:00
Jim Kalafut 22c4ae5933
Rename master key to root key (#13324)
* See what it looks like to replace "master key" with "root key".  There are two places that would require more challenging code changes: the storage path `core/master`, and its contents (the JSON-serialized EncodedKeyringtructure.)

* Restore accidentally deleted line

* Add changelog

* Update root->recovery

* Fix test

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-12-06 17:12:20 -08:00
Eugene R f39f1ce8de
Aerospike backend update (#12165)
* upgrade aerospike-client-go to v5.2.0

* use strings.Contains to check an error

* add changelog file

* go mod tidy

* go mod tidy

* update the changelog

* revert .gitignore update

* go mod tidy
2021-11-29 11:09:12 -08:00
Nick Cabatoff 997a5ace91
Prevent raft transactions from containing overlarge keys. (#13286) 2021-11-26 08:38:39 -05:00
Nick Cabatoff f85908e1df
Return an error when trying to store a too-large key with Raft (#13282) 2021-11-25 14:07:03 -05:00
Josh Black d7c54b50e7
fix bolt 32 bit test (#13249) 2021-11-23 10:50:15 -08:00
Josh Black fe0dd6f867
Add InitialMmapSize to bolt options (#13178) 2021-11-22 20:16:57 -08:00
Brian Kassouf c705adc79c
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Mayo 0bd0339c0b
cleanup unused code and fix t.Fatal usage in goroutine in testing (#11694) 2021-09-30 07:33:14 -04:00
Tero Saarni 944332d12d
Update Go client libraries for etcd (#11980)
* Update Go client libraries for etcd

* Added etcd server container to run etcd3 tests automatically.

* Removed etcd2 test case: it fails the backend tests but the failure is
  unrelated to the uplift.  The etcd2 backend implementation does not
  remove empty nested nodes when removing leaf (see comments in #11980).
2021-09-29 14:28:13 -04:00
akshya96 c643dc1d53
Add Custom metadata field to alias (#12502)
* adding changes

* removing q.Q

* removing empty lines

* testing

* checking tests

* fixing tests

* adding changes

* added requested changes

* added requested changes

* added policy templating changes and fixed tests

* adding proto changes

* making changes

* adding unit tests

* using suggested function
2021-09-17 11:03:47 -07:00
Jason O'Donnell 1cf3ff046e
plugin/snowflake: update gosnowflake to v1.6.1 (#12378)
* plugin/snowflake: update gosnowflake to v1.6.1

* changelog

* go mod tidy
2021-08-20 11:52:31 -04:00
Pratyoy Mukhopadhyay 5fda05adee
[VAULT-3226] Use os.rename on windows os (#12377)
* [VAULT-3226] Use os.rename on windows os

* [VAULT-3226] Add changelog
2021-08-19 16:05:53 -07:00
Nick Cabatoff 72499c3215
Check to make sure context isn't expired before doing a raft operation. (#12162) 2021-08-19 12:03:56 -04:00
Pratyoy Mukhopadhyay fa29e780f0
[NO-TICKET] Upgrade protoc-gen-go to 1.26, upgrade protoc to 3.17.3 (#12171)
* [NO-TICKET] Set protoc-gen-go to 1.23, upgrade protoc to 3.17.3

* [NO-TICKET] Upgrade version of protoc-gen-go to 1.26
2021-07-28 14:51:36 -07:00
Nick Cabatoff f7ecb978a6
Use a mode when opening the db file that won't result in excessive perms. (#12160) 2021-07-23 13:43:50 -04:00