Commit Graph

1472 Commits

Author SHA1 Message Date
Lars Lehtonen 7d9ba86145
builtin/logical/pki: fix dropped test error (#14140) 2022-02-17 17:44:42 -05:00
Alexander Scheel 45c028a2fb
Allow specifying multiple allowed SSH key lengths (#13991)
* Allow specifying multiple allowed SSH key lengths

In the ssh secrets engine, only a single allowed key length was allowed
for each algorithm type. However, many algorithms have multiple safe
values (such as RSA and ECDSA); allowing a single role to have multiple
values for a single algorithm is thus helpful.

On creation or update, roles can now specify multiple types using a list
or comma separated string of allowed values:

    allowed_user_key_lengths: map[string][]int{"rsa": []int{2048, 4096}}

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Break out ssh upgrade logic into separate function

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update parseutil for optional lists of integers

    go get -u github.com/hashicorp/go-secure-stdlib/parseutil
    go mod tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Simplify parse logic using new parseutil

The newly introduced parseutil.ParseIntSlice handles the more
complicated optional int-like slice logic for us.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-17 15:36:56 -05:00
Jordan Reimer b936db8332
Revert "MFA (#14049)" (#14135)
This reverts commit 5f17953b5980e6438215d5cb62c8575d16c63193.
2022-02-17 13:17:59 -07:00
Jordan Reimer 36ccfaa3aa
MFA (#14049)
* adds development workflow to mirage config

* adds mirage handler and factory for mfa workflow

* adds mfa handling to auth service and cluster adapter

* moves auth success logic from form to controller

* adds mfa form component

* shows delayed auth message for all methods

* adds new code delay to mfa form

* adds error views

* fixes merge conflict

* adds integration tests for mfa-form component

* fixes auth tests

* updates mfa response handling to align with backend

* updates mfa-form to handle multiple methods and constraints

* adds noDefault arg to Select component

* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios

* adds tests

* flaky test fix attempt

* reverts test fix attempt

* adds changelog entry

* updates comments for todo items

* removes faker from mfa mirage factory and handler

* adds number to word helper

* fixes tests

* Revert "Merge branch 'main' into ui/mfa"

This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.

* format-ttl helper fix from main
2022-02-17 09:10:56 -07:00
John-Michael Faircloth 1cf74e1179
feature: multiplexing support for database plugins (#14033)
* feat: DB plugin multiplexing (#13734)

* WIP: start from main and get a plugin runner from core

* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode

* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient

* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient

* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists

* add ID to dbplugin.DatabasePluginClient struct

* only create one plugin process per plugin type

* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server

* add v6 multiplexing  protocol version

* WIP: backwards compat for db plugins

* Ensure locking on plugin catalog access

- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin

* use the New constructor for db plugins

* grpc server: use write lock for Close and rlock for CRUD

* cleanup MultiplexedClients on Close

* remove TODO

* fix multiplexing regression with grpc server connection

* cleanup grpc server instances on close

* embed ClientProtocol in Multiplexer interface

* use PluginClientConfig arg to make NewPluginClient plugin type agnostic

* create a new plugin process for non-muxed plugins

* feat: plugin multiplexing: handle plugin client cleanup (#13896)

* use closure for plugin client cleanup

* log and return errors; add comments

* move rpcClient wrapping to core for ID injection

* refactor core plugin client and sdk

* remove unused ID method

* refactor and only wrap clientConn on multiplexed plugins

* rename structs and do not export types

* Slight refactor of system view interface

* Revert "Slight refactor of system view interface"

This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.

* Revert "Revert "Slight refactor of system view interface""

This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.

* only provide pluginRunner arg to the internal newPluginClient method

* embed ClientProtocol in pluginClient and name logger

* Add back MLock support

* remove enableMlock arg from setupPluginCatalog

* rename plugin util interface to PluginClient

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>

* feature: multiplexing: fix unit tests (#14007)

* fix grpc_server tests and add coverage

* update run_config tests

* add happy path test case for grpc_server ID from context

* update test helpers

* feat: multiplexing: handle v5 plugin compiled with new sdk

* add mux supported flag and increase test coverage

* set multiplexingSupport field in plugin server

* remove multiplexingSupport field in sdk

* revert postgres to non-multiplexed

* add comments on grpc server fields

* use pointer receiver on grpc server methods

* add changelog

* use pointer for grpcserver instance

* Use a gRPC server to determine if a plugin should be multiplexed

* Apply suggestions from code review

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* add lock to removePluginClient

* add multiplexingSupport field to externalPlugin struct

* do not send nil to grpc MultiplexingSupport

* check err before logging

* handle locking scenario for cleanupFunc

* allow ServeConfigMultiplex to dispense v5 plugin

* reposition structs, add err check and comments

* add comment on locking for cleanupExternalPlugin

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2022-02-17 08:50:33 -06:00
Robert 91f5069c03
secret/consul: Add Consul ACL roles support (#14014)
Co-authored-by: Brandon Ingalls <brandon@ingalls.io>
2022-02-16 19:31:08 -06:00
Matt Schultz c379e41c4c
Rename transit's auto_rotate_interval to auto_rotate_period for consistency and to achieve formatting benefits in CLI output. Update UI handling of the renamed field to account for recent data type change from time string to integral seconds. (#14103) 2022-02-16 14:33:13 -06:00
Alexander Scheel f45ad6e284
Fix ed25519 generated SSH key marshalling (#14101)
* Ensure we can issue against generated SSH CA keys

This adds a test to ensure that we can issue leaf SSH certificates using
the newly generated SSH CA keys. Presently this fails because the
ed25519 key private is stored using PKIX's PKCS8 PrivateKey object
format rather than using OpenSSH's desired private key format:

> path_config_ca_test.go:211: bad case 12: err: failed to parse stored CA private key: ssh: invalid openssh private key format, resp: <nil>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add dependency on edkey for OpenSSH ed25519 keys

As mentioned in various terraform-provider-tls discussions, OpenSSH
doesn't understand the standard OpenSSL/PKIX ed25519 key structure (as
generated by PKCS8 marshalling). Instead, we need to place it into the
OpenSSH RFC 8709 format. As mentioned in this dependency's README,
support in golang.org/x/crypto/ssh is presently lacking for this.
When the associated CL is merged, we should be able to remove this dep
and rely on the (extended) standard library, however, no review progress
appears to have been made since the CL was opened by the author.

See also: https://go-review.googlesource.com/c/crypto/+/218620/

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-16 14:28:57 -05:00
Matt Schultz 50b86240e8
Return integral seconds for transit auto_rotate_interval instead of a time string. (#14078)
* Return integral seconds for transit auto_rotate_interval instead of a time string.

* Fix transit unit tests around auto_rotate_interval.
2022-02-15 14:05:58 -06:00
Alexander Scheel 3da261518b
Allow generation of other types of SSH CA keys (#14008)
* Add generation support for other SSH CA key types

This adds two new arguments to config/ca, mirroring the values of PKI
secrets engine but tailored towards SSH mounts. Key types are specified
as x/crypto/ssh KeyAlgo identifiers (e.g., ssh-rsa or ssh-ed25519)
and respect current defaults (ssh-rsa/4096). Key bits defaults to 0,
which for ssh-rsa then takes a value of 4096.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on key_type, key_bits for ssh/config/ca

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-15 14:14:05 -05:00
Robert d0832a1993
secret/consul: Add support for consul namespaces and admin partitions (#13850)
* Add support for consul namespaces and admin partitions
2022-02-09 15:44:00 -06:00
Alexander Scheel 26c993107d
Allow all other_sans in sign-intermediate and sign-verbatim (#13958)
* Allow all other_sans in sign-intermediate and sign-verbatim

/sign-verbatim and /sign-intermediate are more dangerous endpoints in
that they (usually) do not have an associated role. In this case, a
permissive role is constructed during execution of these tests. However,
the AllowedOtherSANs field was missing from this, prohibiting its use
when issuing certificates.

Resolves: #13157

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-09 10:09:19 -05:00
Scott Miller f226d0103f
Add duration/count metrics to PKI issue and revoke flows (#13889)
* Add duration/count metrics to PKI issue and revoke flows

* docs, changelog

* tidy

* last tidy

* remove err

* Update callsites

* Simple returns

* Handle the fact that test cases don't have namespaces

* Add mount point to the request

* fmt

* Handle empty mount point, and add it to unit tests

* improvement

* Turns out sign-verbatim is tricky, it can take a role but doesn't have to

* Get around the field schema problem
2022-02-08 10:37:40 -06:00
Alexander Scheel a0feefb2fa
Use application/pem-certificate-chain for PEMs (#13927)
* Use application/pem-certificate-chain for PEMs

As mentioned in #10948, it appears we're incorrectly using the
`application/pkix-cert` media type for PEM blobs, when
`application/x-pem-file` is more appropriate. Per RFC 5280 Section
4.2.1.13, `application/pkix-crl` is only appropriate when the CRL is in
DER form. Likewise, Section 4.2.2.1 states that `application/pkix-cert`
is only applicable when a single DER certificate is used.

Per recommendation in RFC 8555 ("ACME"), Section 7.4.2 and 9.1, we use
the newer `application/pem-certificate-chain` media type for
certificates. However, this is not applicable for CRLs, so we use fall
back to `application/x-pem-file` for these. Notably, no official IETF
source is present for the latter. On the OpenSSL PKI tutorial
(https://pki-tutorial.readthedocs.io/en/latest/mime.html), this type is
cited as coming from S/MIME's predecessor, PEM, but neither of the main
PEM RFCs (RFC 934, 1421, 1422, 1423, or 1424) mention this type.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-08 08:12:33 -05:00
Alexander Scheel 33a9218115
Add full CA Chain to /pki/cert/ca_chain response (#13935)
* Include full chain in /cert/ca_chain response

This allows callers to get the full chain (including issuing
certificates) from a call to /cert/ca_chain. Previously, most endpoints
(including during issuance) do not include the root authority, requiring
an explicit call to /cert/ca to fetch. This allows full chains to be
constructed without without needing multiple calls to the API.

Resolves: #13489

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case for full CA issuance

We test three main scenarios:

 1. A root-only CA's `/cert/ca_chain`'s `.data.ca_chain` field should
    contain only the root,
 2. An intermediate CA (with root provide) should contain both the root
    and the intermediate.
 3. An external (e.g., `/config/ca`-provided) CA with both root and
    intermediate should contain both certs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation for new ca_chain field

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add note about where to find the entire chain

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-02-07 14:37:01 -05:00
Andrew Briening ed457aeae7
Adds "raw(/pem)" format to individual cert routes (#10947) (#10948)
Similar to "/pki/ca(/pem)" routes to retrieve
certificates in raw or pem formats, this adds
"pki/cert/{serial}/raw(/pem)" routes for any
certificate.
2022-02-07 09:47:13 -05:00
Tom Proctor 5032cfaf47
Add make fmt CI check (#13803)
* Add make fmt CI check

* Don't suppress patch output
2022-01-31 23:24:16 +00:00
Gregory Harris c260d35ab1
Support Y10K value in notAfter field when signing non-CA certificates (#13736)
* Support Y10K value in notAfter field when signing non-CA certificates

* Add changelog entry for 13736

* Add test for using not_after parameter for non-CA certificates that are being signed

* Fix CA value for test for not_after value when signing non-CA certs

* Address formatting

* Add changelog file

* Revert changelog entry commit f28b54e7b5ad21144c8a2da942d766e64a332caf
2022-01-31 15:37:50 -06:00
Victor Rodriguez e58c80801e
Add new parameter managed_key_id for PKI CA key generation. (#13825) 2022-01-28 11:14:20 -05:00
Josh Black d249fad2df
reformat using 'make fmt' (#13794) 2022-01-27 10:06:34 -08:00
Matt Schultz b8f46407dd
Fix a transit deadlock (#13795)
* Fix a transit deadlock caused by indefinite lock holding in key autorotation.

* Move down manual policy locking in transit autorotation to avoid NPE.

* Wrap conditional transit key autorotation in a function to allow for cleaner policy lock management.

* Remove a dnagling continue statement from transit key autorotation.
2022-01-27 06:57:11 -06:00
Steven Clark 43087c96b2
OSS integration of the PKI plugin with managed key infrastructure (#13793)
- The OSS side of things to leverage managed keys from the PKI secrets engine
2022-01-26 23:06:25 -05:00
Scott Miller 4ee5a8b1cd
PKI - Allow performance secondaries to generate and store certificates locally to them (#13759)
* PKI - Allow performance secondaries to generate and store certificates locally to them

* changelog

Co-authored-by: divyapola5 <divya@hashicorp.com>
2022-01-24 10:03:04 -06:00
Matt Schultz fc7deabfd7
Time-based transit key autorotation (#13691)
* Add auto_rotate_interval field to transit key creation path.

* Add auto_rotate_interval field to transit key config update path.

* Implement transit automatic key rotation on an hourly interval.

* Fixes transit key autorotation key listing typo.

* Add unit tests for transit key autorotation.

* Add unit tests for transit key creation with autorotation interval.

* Add unit tests for transit key config update with autorotation interval.

* Document new auto_rotate_interval fields in key creation and key config update endpoints.

* Add changelog for transit key autorotation.

* Wrap individual transit key autorotation in a policy lock.

* Add a safeguard to transit key autorotation to ensure only one execution happens simultaneously.
2022-01-20 09:10:15 -06:00
divyapola5 d9c9d06710
Add validation for nonce size when we aren't in convergent encryption mode within transit backend (#13690)
* Add validation for nonce size when we aren't in convergent encryption mode within transit backend

* Add changelog entry
2022-01-19 13:02:49 +05:30
divyapola5 e8a8853fcc
Replace '-' with ':' when listing certificate serial numbers (#13468)
* CLI changes for new mount tune config parameter allowed_managed_keys

* Correct allowed_managed_keys description in auth and secrets

* Documentation update for secrets and removed changes for auth

* Add changelog and remove documentation changes for auth

* removed changelog

* Correct the field description

* Replace - with : when listing certificate serials
2021-12-17 13:33:05 -06:00
Pete Bohman ccc1098ea3
Add allowed_uri_sans_template (#10249)
* Add allowed_uri_sans_template

Enables identity templating for the allowed_uri_sans field in PKI cert roles.

Implemented as suggested in #8509

* changelog++

* Update docs with URI SAN templating
2021-12-15 09:18:28 -06:00
Alexander Scheel 31ff2be589
Add universal default key_bits value for PKI endpoints (#13080)
* Allow universal default for key_bits

This allows the key_bits field to take a universal default value, 0,
which, depending on key_type, gets adjusted appropriately into a
specific default value (rsa->2048, ec->256, ignored under ed25519).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Handle universal default key size in certutil

Also move RSA < 2048 error message into certutil directly, instead of in
ca_util/path_roles.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing RSA key sizes to pki/backend_test.go

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to returning updated values

When determining the default, don't pass in pointer types, but instead
return the newly updated value.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Re-add fix for ed25519 from #13254

Ed25519 internally specifies a hash length; by changing the default from
256 to 0, we fail validation in ValidateSignatureLength(...) unless we
specify the key algorithm.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2021-12-13 15:26:42 -05:00
Steven Clark 834f0bd8ee
Fix test validating convergent encryption behaviour across key types (#13371)
- The test was attempting to test the convergent encryption behaviour
  with several key types but the common function never used the passed
  in key type. So we ran the test with the default aes256-gcm96 only.
2021-12-10 12:55:50 -05:00
Steven Clark f158382f56
Warn user supplying nonce values in FIPS mode for transit encryption requests (#13366)
* Warn user supplying nonce values in FIPS mode for transit encryption requests

 - Send back a warning within the response if an end-user supplies nonce
   values that we use within the various transit encrypt apis.
 - We do not send a warning if an end-user supplies a nonce value but we
   don't use it.
 - Affected api methods are encrypt, rewrap and datakey
 - The warning is only sent when we are operating in FIPS mode.
2021-12-08 14:37:25 -05:00
Matt Schultz 85f5cfc356
Adds support for SHA-3 to transit (#13367)
* Adding support for SHA3 in the transit backend.

* Adds SHA-3 tests for transit sign/verify path. Adds SHA-3 tests for logical system tools path hash functionality. Updates documentation to include SHA-3 algorithms in system tools path hashing.

* Adds changelog entry.

Co-authored-by: robison jacka <robison@packetized.io>
2021-12-08 12:29:33 -06:00
Theron Voran 81e8ad438e
aws/logical: fix backend test (#13323)
Remove the `[:64]` from
`TestAcceptanceBackend_AssumedRoleWithPolicyDoc()` since max length is
handled in `generateUniqueRoleName()`
2021-12-01 10:44:22 -08:00
Theron Voran 8353c27c24
tests: Add `iam_tags` to expected responses (#13150)
Update the AWS auth backend acceptance tests to account for the new
`iam_tags` field that comes back on responses.

* marked only tests requiring creds as acceptance

Renamed TestBackend_* to TestAcceptanceBackend_* if the test requires
AWS credentials. Otherwise left the name as TestBackend_* and set
`AcceptanceTest: false`.

* ensure generated names aren't too long

IAM roles and users have a 64 character limit, and adding Acceptance
to the test names was putting some over the length limit, so modified
generateUniqueName() to take a max length parameter and added
functions for each type of name generation (user, role, group).
2021-11-29 16:00:42 -08:00
Navaneeth Rameshan 4e05632881
recognize ed25519 key type and return PKCS8 format (#13257)
* return pkcs8 format for ed25519 curve

convertRespToPKCS8 does not recognize the ed25519 key. Changes
to recognize ed25519 key and return its PKCS8 format
2021-11-24 14:24:06 -05:00
Matt Schultz 0abd248c9f
Return non-retryable errors on transit encrypt and decrypt failures (#13111)
* Return HTTP 400s on transit decrypt requests where decryption fails. (#10842)

* Don't abort transit batch encryption when a single batch item fails.

* Add unit tests for updated transit batch decryption behavior.

* Add changelog entry for transit encrypt/decrypt batch abort fix.

* Simplify transit batch error message generation when ciphertext is empty.

* Return error HTTP status codes in transit on partial batch decrypt failure.

* Return error HTTP status codes in transit on partial batch encrypt failure.

* Properly account for non-batch transit decryption failure return. Simplify transit batch decryption test data. Ensure HTTP status codes are expected values on batch transit batch decryption partial failure.

* Properly account for non-batch transit encryption failure return. Actually return error HTTP status code on transit batch encryption failure (partial or full).
2021-11-15 15:53:22 -06:00
Alexander Scheel cd213f5fca
Restrict ECDSA/NIST P-Curve hash function sizes for cert signing (#12872)
* Restrict ECDSA signatures with NIST P-Curve hashes

When using an ECDSA signature with a NIST P-Curve, we should follow
recommendations from BIS (Section 4.2) and Mozilla's root store policy
(section 5.1.2) to ensure that arbitrary selection of signature_bits
does not exceed what the curve is capable of signing.

Related: #11245

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to certutil.ValidateKeyTypeSignatureLength(...)

Replaces previous calls to certutil.ValidateKeyTypeLength(...) and
certutil.ValidateSignatureLength(...) with a single call, allowing for
curve<->hash validation.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to autodetection of signature_bits

This enables detection of whether the caller manually specified a value
for signature_bits or not; when not manually specified, we can provision
a value that complies with new NIST P-Curve policy.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Select hash function length automatically

Due to our change in behavior (to default to -1 as the value to
signature_bits to allow for automatic hash selection), switch
ValidateKeyTypeSignatureLength(...) to accept a pointer to hashBits and
provision it with valid default values.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prevent invalid Curve size lookups

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch from -1 to 0 as default SignatureBits

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2021-11-12 12:18:38 -05:00
skhilar 7adacfaf03
Added notAfter and support Y10K expiry for IEEE 802.1AR-2018 (#12795) 2021-11-10 19:09:06 -05:00
Victor Rodriguez 991ab5aeed
VAULT-444: Use sync.RWMutex rather than DeadlockRWMutex in PKI backend. (#13018) 2021-11-02 17:02:54 -04:00
Victor Rodriguez f6e35369f0
VAULT-444: Add PKI tidy-status endpoint. (#12885)
VAULT-444: Add PKI tidy-status endpoint.

Add metrics so that the PKI tidy status can be monitored using telemetry as well.

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2021-11-02 11:12:49 -04:00
Scott Miller 33edf1f556
Flip the semantics of the matching key algorithms in sign-self-issued. (#12988)
* Document allow_different_signature_algorithm param

* Flip the semantics of different key types for sign self issued

* More language tweaks

* Fix the field definition description

* Rework differenttype test for the new flag

* typo
2021-11-01 14:42:25 -05:00
VAL a44505dd06
Native Login method for Go client (#12796)
* Native Login method, userpass and approle interfaces to implement it

* Add AWS auth interface for Login, unexported struct fields for now

* Add Kubernetes client login

* Add changelog

* Add a test for approle client login

* Return errors from LoginOptions, use limited reader for secret ID

* Fix auth comment length

* Return actual type not interface, check for client token in tests

* Require specification of secret ID location using SecretID struct as AppRole arg

* Allow password from env, file, or plaintext

* Add flexibility in how to fetch k8s service token, but still with default

* Avoid passing strings that need to be validated by just having different login options

* Try a couple real tests with approle and userpass login

* Fix method name in comment

* Add context to Login methods, remove comments about certain sources being inherently insecure

* Perform read of secret ID at login time

* Read password from file at login time

* Pass context in integ tests

* Read env var values in at login time, add extra tests

* Update api version

* Revert "Update api version"

This reverts commit 1ef3949497dcf878c47e0e5ffcbc8cac1c3c1679.

* Update api version in all go.mod files
2021-10-26 16:48:48 -07:00
Alexander Scheel 6e069e94ca
Fix PKI Weak Cryptographic Key Lenghths Warning (#12886)
* Modernize SSH key lengths

No default change was made in this commit; note that the code already
enforced a default of 2048 bits. ssh-keygen and Go's RSA key generation
allows for key sizes including 3072, 4096, 8192; update the values of
SSH key generation to match PKI's allowed RSA key sizes (from
certutil.ValidateKeyTypeLength(...)). We still allow the legacy SSH key
size of 1024; in the near future we should likely remove it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure minimum of 2048-bit PKI RSA keys

While the stated path is a false-positive, verifying all paths is
non-trivial. We largely validate API call lengths using
certutil.ValidateKeyTypeLength(...), but ensuring no other path calls
certutil.generatePrivateKey(...) --- directly or indirectly --- is
non-trivial. Thus enforcing a minimum in this method sounds like a sane
compromise.

Resolves: https://github.com/hashicorp/vault/security/code-scanning/55

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2021-10-26 09:30:09 -04:00
Nick Cabatoff ff74f49047
Move to go 1.17 (#12868)
Also ensure that the go 1.17 breaking changes to net.ParseCIDR don't make us choke on stored CIDRs that were acceptable to older Go versions.
2021-10-21 09:32:03 -04:00
vinay-gopalan 4834bb854c
[VAULT-3008] Update RabbitMQ dependency and fix regression in UserInfo.Tags in v3.9 (#12877) 2021-10-20 09:46:37 -07:00
Philipp Hossner 824f097a7d
Let allowed_users template mix templated and non-templated parts (#10886)
* Let allowed_users template mix templated and non-templated parts (#10388)

* Add documentation

* Change test function names

* Add documentation

* Add changelog entry
2021-10-19 15:00:15 -07:00
Brian Kassouf c705adc79c
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Steven Clark 3428de017a
Forbid ssh key signing with specified extensions when role allowed_extensions is not set (#12847)
* Forbid ssh key signing with specified extensions when role allowed_extensions is not set

 - This is a behaviour change on how we process the allowed_extensions role
   parameter when it does not contain a value. The previous handling allowed
   a client to override and specify any extension they requested.
 - We now require a role to explicitly set this behaviour by setting the parameter
   to a '*' value which matches the behaviour of other keys such as allowed_users
   within the role.
 - No migration of existing roles is provided either, so operators if they truly
   want this behaviour will need to update existing roles appropriately.
2021-10-15 17:55:18 -04:00
Anner J. Bonilla 8c29f49e1a
Add support for ed25519 (#11780)
* update azure instructions

Update instructions in regards to azure AD Authentication and OIDC

* Initial pass of ed25519

* Fix typos on marshal function

* test wip

* typo

* fix tests

* missef changelog

* fix mismatch between signature and algo

* added test coverage for ed25519

* remove pkcs1 since does not exist for ed25519

* add ed25519 support to getsigner

* pull request feedback

Signed-off-by: Anner J. Bonilla <abonilla@hoyosintegrity.com>

* typo on key

Signed-off-by: Anner J. Bonilla <abonilla@hoyosintegrity.com>

* cast mistake

Signed-off-by: Anner J. Bonilla <abonilla@hoyosintegrity.com>

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2021-10-05 11:28:49 -04:00
Lars Lehtonen 838e20778d
builtin/logical/consul: fix dropped test error (#12733) 2021-10-05 12:09:13 +01:00
Steven Clark fa57ba0ccf
Fix 1.8 regression preventing email addresses being used as common name within pki certificates (#12336) (#12716)
* Fix 1.8 regression preventing email addresses being used as common name within pki certs (#12336)

* Add changelog
2021-10-04 14:02:47 -04:00
divyapola5 2a194a0804
Add missing read unlock calls in transit backend code (#12652)
* Add missing read unlock calls in transit backend code

* Correct formatting in changelog entry
2021-09-28 11:59:30 -05:00
Calvin Leung Huang 7ad62f5be4
core: set namespace within GeneratePasswordFromPolicy (#12635)
* core: set namespace from the sysview's mount entry on GeneratePasswordFromPolicy

* test: update TestDynamicSystemView to be ns-aware, update tests

* add changelog entry
2021-09-27 09:08:07 -07:00
Calvin Leung Huang 3826042daf
test: pin docker image on postgres to 13.4-buster (#12636)
* test: pin docker image on postgres to 13.4-buster

* test: update all tests that uses postgres image to use 13.4-buster
2021-09-27 08:38:39 -07:00
Hridoy Roy dbd178250e
Port: Premature Rotation For autorotate (#12563)
* port of ldap fix for early cred rotation

* some more porting

* another couple lines to port

* final commits before report

* remove deadlock

* needs testing

* updates

* Sync with OpenLDAP PR

* Update the update error handling for items not found in the queue

* WIP unit tests
* Need to configure DB mount correctly, with db type mockv5
* Need to find a way to inject errors into that mock db

* throw error on role creation failure

* do not swallow error on role creation

* comment out wip tests and add in a test for disallowed role

* Use newly generated password in WAL

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* return err on popFromRotationQueueByKey error; cleanup on setStaticAccount

* test: fix TestPlugin_lifecycle

* Uncomment and fix unit tests
* Use mock database plugin to inject errors
* Tidy test code to rely less on code internals where possible
* Some stronger test assertions

* Undo logging updates

* Add changelog

* Remove ticker and background threads from WAL tests

* Keep pre-existing API behaviour of allowing update static role to act as a create

* Switch test back to update operation

* Revert my revert, and fix some test bugs

* Fix TestBackend_StaticRole_LockRegression

* clean up defer on TestPlugin_lifecycle

* unwrap reqs on cleanup

* setStaticAccount: don't hold a write lock

* TestStoredWALsCorrectlyProcessed: set replication state to unknown

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-21 17:45:04 -07:00
Scott Miller 241a78a2f2
Use the system rand reader for SSH keypair generation (#12560)
* Use the system rand reader for SSH keypair generation

* changelog
2021-09-15 11:59:28 -05:00
Scott Miller 33d7dc5fb4
Use the system rand reader for CA root and intermediate generation (#12559)
* Use the system rand reader for CA root and intermediate generation

* changelog
2021-09-15 11:59:12 -05:00
Scott Miller 6f18a9b6be
Allow signing self issued certs with a different public key algorithm. (#12514)
* WIP: Unset the certificate's SignatureAlgorithm to allown cross-signing of different key types

* Allow signing self issued certs with a different public key algorithm

* Remove cruft

* Remove stale import

* changelog

* eliminate errwrap

* Add a test to cover the lack of opt-in flag

* Better comment

Co-authored-by: catsby <clint@ctshryock.com>
2021-09-14 10:07:27 -05:00
divyapola5 30563097ea
Enforce minimum cache size for transit backend (#12418)
* Enforce Minimum cache size for transit backend

* enfore minimum cache size and log a warning during backend construction

* Update documentation for transit backend cache configuration

* Added changelog

* Addressed review feedback and added unit test

* Modify code in pathCacheConfigWrite to make use of the updated cache size

* Updated code to refresh cache size on transit backend without restart

* Update code to acquire read and write locks appropriately
2021-09-13 16:44:56 -05:00
jhart-cpi fa1611f427
improvement: add signature_bits field to CA and signers (#11245)
This change adds the ability to set the signature algorithm of the
CAs that Vault generates and any certificates it signs.  This is a
potentially useful stepping stone for a SHA3 transition down the line.

Summary:
* Adds the field "signature_bits" to CA and Sign endpoints
* Adds support for SHA256, SHA384 and SHA512 signatures on EC and RSA
keytypes.
2021-09-10 14:39:05 -07:00
Lars Lehtonen 5af6d5f07f
builtin/logical/aws: fix dropped test error (#12417) 2021-08-26 15:55:39 +01:00
vinay-gopalan 23770cc2a7
Update genUsername to cap STS usernames at 32 chars (#12185)
* update genUsername to cap STS usernames at 64 chars

* add changelog

* refactor tests into t.Run block

* patch: remove warningExpected bool and include expected string

* patch: revert sts to cap at 32 chars and add assume_role case in genUsername

* update changelog

* update genUsername to return error if username generated exceeds length limits

* update changelog

* add conditional default username template to provide custom STS usernames

* update changelog

* include test for failing STS length case

* update comments for more clarity
2021-08-09 09:40:47 -07:00
Scott Miller 653bfef52e
Forward cert signing requests to the primary on perf secondaries as well as perf standbys (#12180) 2021-07-28 10:21:01 -05:00
Nick Cabatoff 03b2208f04
Pin RabbitMQ and Cassandra docker image versions (#12174)
* Work around rabbitmq regression with UserInfo.Tags in rabbitmq 3.9: use v3.8 docker image in tests.

* Also pin cassandra docker image version to 3.11 (4.00 was making tests fail)
2021-07-27 08:45:32 -04:00
Jeff Mitchell 33ff878946
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Lars Lehtonen a9153d7348
builtin/logical/database: fix dropped test errors (#12123) 2021-07-20 11:13:50 -07:00
Jeff Mitchell fb473a8b9b
Swap out stepwise for external repo version (#12089) 2021-07-20 13:20:23 -04:00
vinay-gopalan 859b60cafc
[VAULT-1969] Add support for custom IAM usernames based on templates (#12066)
* add ability to customize IAM usernames based on templates

* add changelog

* remove unnecessary logs

* patch: add test for readConfig

* patch: add default STS Template

* patch: remove unnecessary if cases

* patch: add regex checks in username test

* patch: update genUsername to return an error instead of warnings

* patch: separate tests for default and custom templates

* patch: return truncate warning from genUsername and trigger a 400 response on errors

* patch: truncate midString to 42 chars in default template

* docs: add new username_template field to aws docs
2021-07-20 09:48:29 -07:00
swayne275 ed361ee8da
Fix minor typo in Internals/Plugins documentation (#12113)
* fix minor plugin doc typo

* fix limits of of typo and related

* forgot to save on this typo fix
2021-07-20 07:21:24 -06:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Austin Gebauer d1c090fe63
secrets/database: fixes external plugin reconnect after shutdown for v4 and v5 interface (#12087)
* secrets/database: fixes external plugin shutdown reconnect for v5 interface

* adds changelog entry

* fixes handling of plugin shutdown for password generation on v4 interface
2021-07-15 13:41:04 -07:00
Jeff Mitchell fe18b6f9e0
Swap out sdk/helper libs with implementations in go-secure-stdlib (#12088)
* Swap out sdk/helper libs with implementations in go-secure-stdlib

* Fix transit batch test
2021-07-15 01:56:37 -04:00
Martin Lee 10f29e0503
base32.DecodeString expects length 8 for the buffer (#11887)
Add padding to the input key to ensure it reaches that length.
2021-07-14 07:38:10 -04:00
Lars Lehtonen b6f990d1b6
builtin/logical/nomad: fix dropped test errors (#12052) 2021-07-13 07:28:46 -04:00
Lars Lehtonen 0196f43cbe
builtin/logical/pki: fix dropped test errors (#12013) 2021-07-08 10:14:38 -04:00
MilenaHC 5483eba5fc
RabbitMQ - Add username customization (#11899)
* add username customization for rabbitmq

* add changelog for rabbitmq

* Update builtin/logical/rabbitmq/path_config_connection.go

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* updating API docs

* moved to changelog folder

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2021-06-22 14:50:46 -05:00
rerorero 9ebb14bab3
Fix: Transit encrypt batch does not honor key_version (#11628)
* fix(secret/transit): #10232 Transit encrypt batch does not honor key_version

* add changelog for 11628
2021-05-27 14:05:20 -05:00
Ricardo Cardenas d02a20bd2b
feat(aws): add ability to provide a role session name when generating STS credentials (#11345)
* feat(aws): add ability to provide a sessionName to sts credentials

Co-authored-by: Brad Vernon <bvernon@nvidia.com>
Co-authored-by: Jim Kalafut <jim@kalafut.net>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2021-05-17 11:03:09 -07:00
Michael Golowka 056a59859f
Add ability to customize some timeouts in MongoDB database plugin (#11600) 2021-05-17 11:40:35 -06:00
Robison Jacka 491f71faf0
Add support for templated values in SSH CA DefaultExtensions. (#11495)
* Add support for templated values in SSH CA DefaultExtensions.

* Reworking the logic per feedback, adding basic test.

* Adding test, so we cover both default extension templating & ignoring default when user-provided extensions are present.

* Fixed up an unintentional extension handling defect, added test to cover the case.

* Refactor Default Extension tests into `enabled` and `disabled`.
2021-05-13 14:37:22 -07:00
Michael Golowka b27a3e9f70
DB engine: Check ErrPluginStaticUnsupported in rollback code (#11601) 2021-05-12 17:09:56 -06:00
Michael Golowka a18bd5ab94
Check ErrPluginStaticUnsupported for fallback to RotateRootCredentials (#11585) 2021-05-12 15:22:41 -06:00
Austin Gebauer 57b6e786fc
Update container image tag used in SSH secrets tests (#11548) 2021-05-06 11:57:54 -04:00
Lars Lehtonen d8f7dd364a
builtin: deprecate errwrap.Wrapf() throughout (#11430)
* audit: deprecate errwrap.Wrapf()

* builtin/audit/file: deprecate errwrap.Wrapf()

* builtin/crediential/app-id: deprecate errwrap.Wrapf()

* builtin/credential/approle: deprecate errwrap.Wrapf()

* builtin/credential/aws: deprecate errwrap.Wrapf()

* builtin/credentials/token: deprecate errwrap.Wrapf()

* builtin/credential/github: deprecate errwrap.Wrapf()

* builtin/credential/cert: deprecate errwrap.Wrapf()

* builtin/logical/transit: deprecate errwrap.Wrapf()

* builtin/logical/totp: deprecate errwrap.Wrapf()

* builtin/logical/ssh: deprecate errwrap.Wrapf()

* builtin/logical/rabbitmq: deprecate errwrap.Wrapf()

* builtin/logical/postgresql: deprecate errwrap.Wrapf()

* builtin/logical/pki: deprecate errwrap.Wrapf()

* builtin/logical/nomad: deprecate errwrap.Wrapf()

* builtin/logical/mssql: deprecate errwrap.Wrapf()

* builtin/logical/database: deprecate errwrap.Wrapf()

* builtin/logical/consul: deprecate errwrap.Wrapf()

* builtin/logical/cassandra: deprecate errwrap.Wrapf()

* builtin/logical/aws: deprecate errwrap.Wrapf()
2021-04-22 11:20:59 -04:00
Calvin Leung Huang a8cafab083
pki: fix tidy removal on revoked entries (#11367)
* pki: fix tidy removal on revoked entries

* add CL entry
2021-04-19 09:40:40 -07:00
Michael Golowka 4279bc8b34
Validate hostnames when using TLS in Cassandra (#11365) 2021-04-16 15:52:35 -06:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Scott Miller 7ecbbcd5b9
Make all duplicate removals stable in PKI (#11259) 2021-04-02 10:33:24 -05:00
Clint dcf3344887
Extract replication state before go routine for initQueue (#11161)
Querying the state before launching the go routine avoids a possible
race condition with replication.
2021-03-30 15:19:00 -05:00
Evgeniy Kosov cf39c9e161
Wrap sign error from an external lib (#10301) 2021-03-15 16:26:06 -06:00
Liwei Fu 170a0800e6
Make cert domain name validation case insensitive (#10959)
* make cert domain name validation case insensitive

* reafctor TestPki_PermitFQDNs mutliple cases

* TestPki_PermitFQDNS: fail uppercase alt_name

* add change log

* fix tests

* use EqualFold for potential utf-8 string comparison

Co-authored-by: Freyert <Freyert@users.noreply.github.com>
2021-03-09 21:28:27 -08:00
Lauren Voswinkel 075898cf73
Add IAM tagging support for iam_user roles in AWS secret engine (#10953)
* Added support for iam_tags for AWS secret roles

This change allows iam_users generated by the secrets engine
to add custom tags in the form of key-value pairs to users
that are created.
2021-02-25 16:03:24 -08:00
Hridoy Roy 4a96126d5a
Revert "Vault Dependency Upgrades [VAULT-871] (#10903)" (#10939)
This reverts commit eb74ca61fc4dcb7038f39defb127d5d639ba0ca1.
2021-02-18 15:40:18 -05:00
Hridoy Roy a26d1300e8
Vault Dependency Upgrades [VAULT-871] (#10903)
* upgrade vault dependency set

* etcd and grpc issues:

* better for tests

* testing

* all upgrades for hashicorp deps

* kubernetes plugin upgrade seems to work

* kubernetes plugin upgrade seems to work

* etcd and a bunch of other stuff

* all vulnerable packages upgraded

* k8s is broken in linux env but not locally

* test fixes

* fix testing

* fix etcd and grpc

* fix etcd and grpc

* use master branch of go-testing-interface

* roll back etcd upgrade

* have to fix grpc since other vendors pull in grpc 1.35.0 but we cant due to etcd

* rolling back in the replace directives

* a few more testing dependencies to clean up

* fix go mod vendor
2021-02-18 12:31:57 -08:00
Jim Kalafut 42bae71806
Improve error messages (#10843)
- Fix: "bytes" should be less than %!s(int=131072) message
- Also add a missing openapi type that was throwing warnings
2021-02-11 19:51:12 -08:00
Hridoy Roy 0e3bddf295
Revert "allow create to create transit keys (#10706)" (#10724)
This reverts commit 4144ee0d3da10fbfef4d081aa72529f2e513f8e2.
2021-01-19 11:49:57 -08:00
Hridoy Roy e8164ad09a
allow create to create transit keys (#10706)
* allow create to create transit keys

* changelog
2021-01-15 12:20:32 -08:00
Hridoy Roy 2c4e299391
allow null types in batch encryption [VAULT-849] (#10386)
* allow null types in batch encryption

* dont allow plaintext to be null
2020-11-23 11:55:08 -08:00
Tom Proctor b9b3796cfe
Sanitize private_key from returned db plugin config (#10416) 2020-11-19 10:58:55 +00:00
Michael Golowka bfd57bc2e8
Error on root rotation when username is empty (#10344)
* Error on root rotation when username is empty

* Don't panic if the field doesn't exist
2020-11-09 15:12:09 +00:00
Brian Kassouf 10668331e4
Update go version to 1.15.3 (#10279)
* Update go version to 1.15.3

* Fix OU ordering for go1.15.x testing

* Fix CI version

* Update docker image

* Fix test

* packagespec upgrade -version 0.1.8

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>
2020-10-30 16:44:06 -04:00
Michael Golowka bd79fbafb3
Add couchbase, elasticsearch, and mongodbatlas back (#10222)
Updated the `Serve` function so these can be added back into Vault
2020-10-22 17:20:17 -06:00
Michael Golowka e6c8ee24ea
DBPW - Enables AutoMTLS for DB plugins (#10220)
This also temporarily disables couchbase, elasticsearch, and
mongodbatlas because the `Serve` function needs to change signatures
and those plugins are vendored in from external repos, causing problems
when building.
2020-10-22 15:43:19 -06:00
Calvin Leung Huang af2c3a851d
database/test: use vault.TestWaitActive when we're starting up a test cluster (#10158) 2020-10-16 09:35:55 -07:00
Michael Golowka 1888323243
DBPW - Copy `newdbplugin` package to `dbplugin/v5` (#10151)
This is part 1 of 4 for renaming the `newdbplugin` package. This copies the existing package to the new location but keeps the current one in place so we can migrate the existing references over more easily.
2020-10-15 13:20:12 -06:00
Scott Miller b513af3851
Expose generic versions of KDF and symmetric crypto (#10076)
* Support salt in DeriveKey

* Revert "Support salt in DeriveKey"

This reverts commit b295ae42673308a2d66d66b53527c6f9aba92ac9.

* Refactor out key derivation, symmetric encryption, and symmetric decryption into generic functions

* comments

* comments

* go mod vendor

* bump both go.mods

* This one too

* bump

* bump

* bump

* Make the lesser used params of symmetric ops a struct

* go fmt

* Call GetKey instead of DeriveKey

* Address feedback

* Wrong rv

* Rename calls

* Assign the nonce field

* trivial change

* Check nonce len instead

* go mod vendor
2020-10-01 21:04:36 -05:00
Michael Golowka fc0ed96066
DBPW - Revert AutoMTLS (#10065) 2020-09-30 17:08:37 -06:00
Theron Voran 52581cd472
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
Michael Golowka 41d8c89169
[DBPW 5/X] Use AutoMTLS with DB plugins (#10008) 2020-09-23 16:08:03 -06:00
Marco Rieger b634e1964d
fix missing plaintext in bulk decrypt response (#9991)
Decrypting an ciphertext where its corresponding value equals empty, the payload property "plaintext" is missing in the response object. This fixes the problem by adding a new, distinct struct for decrypt batch response items where "omitempty" is not set.
2020-09-22 09:43:07 -04:00
Michael Golowka 60e0cbbc37
[DBPW 4/X] Update DB engine to support v4 and v5 interfaces with password policies (#9878) 2020-09-18 15:10:54 -06:00
ncabatoff 27c7a77624
When expiration attempts to revoke a cert that's not in storage (perhaps due to pki tidy), don't treat that as an error. Let the lease get expired. (#9880) 2020-09-17 16:15:03 -04:00
Lauren Voswinkel 5740e1ff9e
5844 AWS Root Credential Rotation (#9921)
* strip redundant field type declarations

* root credential rotation for aws creds plugin

* Change location of mocks awsutil and update methods that no longer exist

* Update website/pages/docs/auth/aws.mdx

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk version to get the awsutil mock file

* Re-vendor modules to pass CI

* Use write lock for the entirety of AWS root cred rotation

* Update docs for AWS root cred rotation for clarity

Co-authored-by: Becca Petrin <beccapetrin@gmail.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-09-15 15:26:56 -07:00
Mark Gritter 243d407dac
Disable flaky test: discard_role_newer_rotation_date (#9957)
* Temporarily disable discard_role_newer_rotation_date
2020-09-15 13:10:59 -05:00
ncabatoff b615da43d7
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Mark Gritter 0e6da5c7ef
Lower the interval for rotation during tests, to make it more likely that our five second grace period is sufficient. (#9895)
* Lower the interval for rotation during tests, to make it more likely
that our five second grace period is sufficient.
* Rewrite to make the rotateCredentials ticker a configurable value.
* 'go mod vendor' for SDK changes.
2020-09-08 17:53:15 -05:00
Calvin Leung Huang 3a5c7a6946
secrets/ssh: allow algorithm_signer to use the key's default algo (#9824)
* secrets/ssh: allow algorithm_signer to use the key's default algo

* add test for ed25519 key signing

* test: add role upgrade test case

* test: rename and add more test cases

* test: clean up tests cases, fix broken test case on expected error

* test: fix broken test case on expected error
2020-08-26 12:31:56 -07:00
Artem Alexandrov 301ea4c0f0
pki: Allow to use not only one variable during templating in allowed_domains #8509 (#9498) 2020-08-17 11:37:00 -07:00
James Hodgkinson 8173ce777e
fixing a spelling error (#9693)
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-08-09 06:17:02 -07:00
Calvin Leung Huang fbe2a86693
pki: use revocationInfo.RevocationTimeUTC when revoking certs with ti… (#9609)
* pki: use revocationInfo.RevocationTimeUTC when revoking certs with tidy_revoked_certs set to true

* update comment

* tidy: use same time snapshot for OR comparison
2020-07-30 15:10:26 -07:00
Clint 5e80919d77
Forward requests for rotate-root on Performance secondary / standbys (#9606) 2020-07-28 09:22:32 -05:00
ncabatoff 36528cf9ec
Don't override MaxWait, its default is good for CI (#9478) 2020-07-14 14:21:37 -04:00
Andrej van der Zee 8f305b1531
Add option allowed_domains_template enabling identity templating for issuing PKI certs. (#8509) 2020-07-08 12:52:25 -04:00
ncabatoff 4a5d8fc212
Remove two unreliable tests from TestTransit_SignVerify_P256. (#7499)
* Remove two tests from TestTransit_SignVerify_P256.  These verify that
you can't verify something signed using JWS marshaling when claiming
it's ASN1, and vice versa.  The problem is that we rely on unmarshaling
failing, and it doesn't always.  Both encodings use base64, one with
padding, one without, so depending on the data sometimes unmarshaling
will work when we expect it to fail.  It would be nice to preserve
these tests if they could be made reliable, but I didn't see an easy way,
and I don't think they add enough value to warrant greater effort.

* Restore the tests I removed, and improve the verify function to broaden the notion of errExpected.

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-06-29 12:22:22 -04:00
Clint cbecc40e48
Stepwise docker env (#9292)
* add first stepwise test env, Docker, with example transit test

* update transit stepwise test

* add other tests that use stepwise

* cleanup test, make names different than just 'transit'

* return the stderr if compile fails with error

* minor cleanups

* minor cleanups

* go mod vendor

* cleanups

* remove some extra code, and un-export some fields/methods

* update vendor

* remove reference to vault.CoreConfig, which really wasn't used anyway

* update with go mod vendor

* restore Precheck method to test cases

* clean up some networking things; create networks with UUID, clean up during teardown

* vendor stepwise

* Update sdk/testing/stepwise/environments/docker/environment.go

haha thanks :D

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update sdk/testing/stepwise/environments/docker/environment.go

Great catch, thanks

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* fix redundant name

* update error message in test

* Update builtin/credential/userpass/stepwise_test.go

More explicit error checking and responding

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update builtin/logical/aws/stepwise_test.go

`test` -> `testFunc`

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update builtin/logical/transit/stepwise_test.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* fix typos

* update error messages to provide clarity

* Update sdk/testing/stepwise/environments/docker/environment.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* update error handling / collection in Teardown

* panic if GenerateUUID returns an error

* Update sdk/testing/stepwise/environments/docker/environment.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update builtin/credential/userpass/stepwise_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update builtin/logical/aws/stepwise_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update builtin/logical/transit/stepwise_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk/testing/stepwise/environments/docker/environment.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* import ordering

* standardize on dc from rc for cluster

* lowercase name

* CreateAPIClient -> NewAPIClient

* testWait -> ensure

* go mod cleanup

* cleanups

* move fields and method around

* make start and dockerclusternode private; use better random serial number

* use better random for SerialNumber

* add a timeout to the context used for terminating the docker container

* Use a constant for the Docker client version

* rearrange import statements

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-06-26 17:52:31 -05:00
Jason O'Donnell 0e294a7174
plugins/ssh: add diabled host key verification warning (#9304) 2020-06-24 08:40:19 -04:00
Clint 6b4bdb1882
VLT091 plugin testing framework stepwise (#9270)
* Resolve merge conflicts and updates from running a test

* move testing/_test.go over to legacy

* updates

* Add core of plugin test framework Stepwise  (#9166)

* adding stepwise testing, but there are protocol buff error :/

* move file and update sdk/go.mo

* update/sync modules

* update from other branch

* update sdk/go.mod

* some cleanups after feedback

* remove enviornments from this PR

* update vendor

* change from running go mod tidy

* change from go mod tidy

* Update sdk/testing/stepwise/helpers.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update sdk/testing/stepwise/helpers.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* change panic to error

* Update sdk/testing/stepwise/helpers.go

return `nil` and not `err` at the end

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Defer close() on successful Open of a file

* document the re-creation of steps

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* remove unused BarrierKeys()

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* updates from feedback

* fix return with bad arguments

* Rename things:

- StepOperation -> Operation
- StepwiseEnvironment -> Environment
- StepCheckFunc -> AssertionFunc
- step.Check -> step.Assert

* document the environment interface methods

* rename EnvironmentOptions to MountOptions

* rename Name to RegistryName

* remove ExpectError because it's redundant

* minor doc update

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* add checkShouldRun function

* remove redundant return

* remove vestigial PreCheck function

* add tt.Helper() to makeRequest

* minor code formatting and document 1-based index for log output of Steps

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* minor updates

* update sdk

* use local reference for api, vault dep

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* cleanup some defer functions

* call fatal if environment setup fails, and don't call teardown

* defer re-setting client token in makeRequest

* Move legacy logicaltest back to testhelpers

* update mods and test files with go mod tidy

* go mod vendor

* remove relative replace directives

* restore old logical test location

* move declaration to main stepwise file

* remove index var and use i+1

* add testing for write, delete paths of makeRequest

* update stepwise core testing to do request counting

* remove unused methods

* Update sdk/testing/stepwise/stepwise.go

remove dead line

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update sdk/testing/stepwise/stepwise.go

fix capitalization in code comment

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* update code comments for SkipTeardown to clarify its use

* update stepwise

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
2020-06-23 06:01:39 -05:00
Jason O'Donnell 3c8ceb4d1d
Fix database creds rotation panic for nil resp (#9258) 2020-06-18 13:49:42 -04:00
Michael Golowka f77bcc53c4
Move sdk/helper/random -> helper/random (#9226)
* This package is new for 1.5 so this is not a breaking change.
* This is being moved because this code was originally intended to be used
within plugins, however the design of password policies has changed such
that this is no longer needed. Thus, this code doesn't need to be in the
public SDK.
2020-06-17 14:24:38 -06:00
Michael Golowka a89f09802d
Integrate password policies into RabbitMQ secret engine (#9143)
* Add password policies to RabbitMQ & update docs
* Also updates some parts of the password policies to aid/fix testing
2020-06-11 16:08:20 -06:00
ncabatoff 2ec9049ef7
Add ssh signing algorithm as a role option. (#9096) 2020-06-11 08:10:13 -04:00
Austin Gebauer 821940f905
fix: invalidate cached clients after a config change in the aws secrets backend (#9186) 2020-06-10 20:53:48 -07:00
Félix Mattrat 40699d2b9e
Improving transit batch encrypt and decrypt latencies (#8775)
Optimized batch items decoder bypassing mapstructure
2020-06-10 13:31:46 -04:00
Theron Voran e1a432a167
AWS: Add iam_groups parameter to role create/update (#8811)
Allows vault roles to be associated with IAM groups in the AWS
secrets engine, since IAM groups are a recommended way to manage
IAM user policies. IAM users generated against a vault role will
be added to the IAM Groups. For a credential type of
`assumed_role` or `federation_token`, the policies sent to the
corresponding AWS call (sts:AssumeRole or sts:GetFederation) will
be the policies from each group in `iam_groups` combined with the
`policy_document` and `policy_arns` parameters.

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-06-09 16:56:12 -07:00
Peter J. Li 27cf73afa8
fix error message for when an invalid uri_sans is provided via the api (#8772) 2020-06-08 13:43:56 -04:00
Austin Gebauer cc16c6d08e
fix: remove mount prefix from config path used to invalidate connections (#9129) 2020-06-03 12:04:55 -07:00
Alexander Bezobchuk eb0b3ac286
Merge PR #9100: Add key_version to Transit Logical Response 2020-06-01 13:16:01 -04:00
ncabatoff 8870b2e51c
Add mongodbatlas static roles support (#8987)
* Refactor PG container creation.
* Rework rotation tests to use shorter sleeps.
* Refactor rotation tests.
* Add a static role rotation test for MongoDB Atlas.
2020-05-29 14:21:23 -04:00
Lauren Voswinkel 4d98430964
Use parameters when executing prepared statements rather than fmt.Sprintf (#9013)
* Don't use string formatting to prepare queries.

We should, when possible, use the built-in params and ? format when
preparing and executing a query. This is done to prevent SQL Injection
attacks.

* Revert some changes due to failing tests, update mssql go driver

* Add docker container startup for some MSSQL tests

* Remove acceptance test flagging, add more SQL injection protection

* Refactor MSSQL prepareTestContainer to a test helper

Also, remove all ? references and convert them to @p*
2020-05-21 16:07:18 -07:00
Clint 8bd8d7dba6
secrets/database: return any error from rotations (#8997) 2020-05-19 12:05:09 -05:00
Jeff Mitchell 1d3d89e2aa
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Andrew N Golovkov 753b2c135a
More helpful errors when import bundled certificates (#8951)
* helpful errors: print not only CN but also exactly what we are comparing
* helpful errors: return different errors for non-existent and unknown keys
* helpful errors: print error about encrypted key instead of "private key not found"
2020-05-11 17:01:10 -06:00
ncabatoff d0754790ab
Define Consul version to test against in one place, and let it be overriden by environment. Exception: some tests that verify interoperability with older Consul versions still specify a version explicitly. (#8901) 2020-04-30 13:06:24 -04:00
Austin Gebauer 01e701f008
Fix: rotate root credentials for database plugins using WAL (#8782)
* fix: rotate root credentials for database plugins using WAL

* test: adds a test for WAL rollback logic

* fix: progress on wal rollback

* docs: updates some comments

* docs: updates some comments

* test: adds additional test coverage for WAL rollback

* chore: remove unneeded log

* style: error handling, imports, signature line wraps

* fix: always close db plugin connection
2020-04-22 16:21:28 -07:00
Becca Petrin ae28ebb94a
Rabbitmq: surface errors from responses (#8619)
* surface errs from responses

* add test

* Update builtin/logical/rabbitmq/path_role_create.go

Co-Authored-By: Vishal Nayak <vishalnayak@users.noreply.github.com>

* improve error message

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-03-27 14:14:59 -07:00
Lars Lehtonen 85301166fe
builtin/logical/pki: fix JSON tag (#8324) 2020-03-06 18:41:26 -08:00
Denis Subbotin a9e605cc43
fix minor potential nil-pointer panic on line 89 (#8488) 2020-03-06 13:32:36 -08:00
JulesRenz c54c8c92bd
RSA3072 implementation in transit secrets engine (#8151)
* RSA3072 implementation in transit secrets engine

* moved new KeyType at the end of the list
So already stored keys still work properly

Co-authored-by: Jim Kalafut <jim@kalafut.net>
2020-02-15 14:40:50 -08:00
Gerardo Di Giacomo 8573eefe90
enabling TLS 1.3 support for TCP listeners (#8305)
* adding support for TLS 1.3 for TCP listeners

* removed test as CI uses go 1.12

* removed Cassandra support, added deprecation notice

* re-added TestTCPListener_tls13
2020-02-15 11:40:18 -08:00
Fredrik Hoem Grelland 13e68015aa
identity propagation in ssh secrets engine #7547 (#7548)
* identity propagation in ssh secrets engine #7547

* flag to enable templating allowed_users ssh (ca) secrets backend.
2020-02-15 11:04:33 -08:00
Daniel Spangenberg 415303cc02
Allow FQDNs in DNS Name for PKI Secrets Engine (#8288)
Fixes #4837
2020-02-04 23:46:38 +01:00
Michel Vocks f695eb737b
Add Consul TLS options to access API endpoint (#8253) 2020-01-29 09:44:35 +01:00
Michel Vocks 027ada452e
Mongodb driver switch to mongo-driver (#8140)
* Switch mongodb driver to mongo-driver

* Tidy mod

* Make writeConcern private

* Implement review feedback

* Add retry functionality

* Added backoff time

* go mod vendor

* Fix failing test

* goimport
2020-01-24 09:32:47 +01:00
Anthony Dong d4267b4250 ssh backend: support at character in role name (#8038) 2020-01-21 11:46:29 +01:00
Michel Vocks 13ebf5460c
Add TLS options per Nomad backend (#8083) 2020-01-15 11:03:38 +01:00
Jeff Mitchell a0694943cc
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
Michel Vocks 80bc527726 Fix DB static role credential rotation replication issue (#8105)
* Fix DB static role credential rotation replication issue

* Rebased and switched to new path forward options

* Removed unnecesary write to storage
2020-01-09 16:45:07 -08:00
Becca Petrin a94f2d3e6f
Replace deprecated AWS client instantiations (#8060)
* replace deprecated aws client instantiation

* fix imports
2020-01-09 14:58:33 -08:00
Becca Petrin c2894b8d05
Add Kerberos auth agent (#7999)
* add kerberos auth agent

* strip old comment

* changes from feedback

* strip appengine indirect dependency
2020-01-09 14:56:34 -08:00
Michel Vocks 02cdd8a6da
Fix DB root rotation replication issue (#8106)
* Fix DB root rotation replication issue

* Rebase and switch to new path forward options
2020-01-09 15:59:58 +01:00
Michel Vocks 1198fe848b
Fix potential panic in database credential role rotation (#8098) 2020-01-07 16:52:51 +01:00
catsby a15c6e3c72
remove redundant check and clarify code comment 2019-12-11 10:16:09 -06:00
Clint 80382d52d4
Transit: error when restoring to a name that looks like a path (#7998)
* Add test to verify #7663

* Validate name in transit key restore to not be a path
2019-12-11 09:32:22 -06:00
ncabatoff fde5e55ce9
Handle otherName SANs in CSRs (#6163)
If a CSR contains a SAN of type otherName, encoded in UTF-8, and the signing role specifies use_csr_sans, the otherName SAN will be included in the signed cert's SAN extension.

Allow single star in allowed_other_sans to match any OtherName.  Update documentation to clarify globbing behaviour.
2019-12-11 10:16:44 -05:00
Chris Hoffman ea0974b578
if storing the certificate, always generate/sign the certificate on the primary (#7904) 2019-12-05 13:50:28 -05:00
Brian Kassouf d7332c33d8
Fix issue deleting DB connections on Secondaries (#7853) 2019-11-11 09:04:24 -08:00
Pascal Enz 33c1b7150f Rabbitmq topic permissions (#7751)
* Upgraded rabbit hole library to 2.0

* Added RabbitMQ topic permission support.

* Updated docs to cover RabbitMQ topic permissions.

* Improved comments and docs as suggested.
2019-10-30 14:19:49 -07:00
Denis Subbotin e9cdd451d1 Don't allow duplicate SAN names in PKI-issued certs (#7605)
* fix https://github.com/hashicorp/vault/issues/6571

* fix test TestBackend_OID_SANs because now SANs are alphabetic sorted
2019-10-28 12:31:56 -04:00
will-quan-bird 6456fd6222 allows emails@sign to be within the aws secrets engine path (#7553) 2019-10-25 09:01:01 -07:00
Lexman c86fe212c0
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Jeff Mitchell 4b5572bf35 Don't continue in a few places in pki tidy if value is nil (#7589)
Fixes #7588
2019-10-15 09:55:08 -04:00
Jeff Mitchell 4252f5c9e4
Add AES128-GCM96 support to transit (#7555) 2019-10-03 16:11:43 -04:00
Jeff Mitchell 6d1e804a22
Add P384 and P521 support to Transit (#7551) 2019-10-03 12:32:43 -04:00
ncabatoff 71cb7cbf18
Fix issue with rotateCredentials deadlocking with itself (#7518) 2019-10-03 12:28:29 -04:00
Joel Thompson 551b7a5e5c secret/aws: Support permissions boundaries on iam_user creds (#6786)
* secrets/aws: Support permissions boundaries on iam_user creds

This allows configuring Vault to attach a permissions boundary policy to
IAM users that it creates, configured on a per-Vault-role basis.

* Fix indentation of policy in docs

Use spaces instead of tabs
2019-09-19 16:35:12 -07:00
Joel Thompson e8e59cd87c logical/aws: Clean up test user (#7459)
I probably left this cleanup commented out as part of debugging test
errors in #6789 and forgot to uncomment it, so actually cleaning up the
test user.
2019-09-13 11:05:43 -07:00
Joel Thompson 8a981004ec Add reading AWS root/config endpoint (#7245) 2019-09-13 10:07:04 -07:00
Joel Thompson ac18a44fae secret/aws: Pass policy ARNs to AssumedRole and FederationToken roles (#6789)
* secret/aws: Pass policy ARNs to AssumedRole and FederationToken roles

AWS now allows you to pass policy ARNs as well as, and in addition to,
policy documents for AssumeRole and GetFederationToken (see
https://aws.amazon.com/about-aws/whats-new/2019/05/session-permissions/).
Vault already collects policy ARNs for iam_user credential types; now it
will allow policy ARNs for assumed_role and federation_token credential
types and plumb them through to the appropriate AWS calls.

This brings along a minor breaking change. Vault roles of the
federation_token credential type are now required to have either a
policy_document or a policy_arns specified. This was implicit
previously; a missing policy_document would result in a validation error
from the AWS SDK when retrieving credentials. However, it would still
allow creating a role that didn't have a policy_document specified and
then later specifying it, after which retrieving the AWS credentials
would work. Similar workflows in which the Vault role didn't have a
policy_document specified for some period of time, such as deleting the
policy_document and then later adding it back, would also have worked
previously but will now be broken.

The reason for this breaking change is because a credential_type of
federation_token without either a policy_document or policy_arns
specified will return credentials that have equivalent permissions to
the credentials the Vault server itself is using. This is quite
dangerous (e.g., it could allow Vault clients access to retrieve
credentials that could modify Vault's underlying storage) and so should
be discouraged. This scenario is still possible when passing in an
appropriate policy_document or policy_arns parameter, but clients should
be explicitly aware of what they are doing and opt in to it by passing
in the appropriate role parameters.

* Error out on dangerous federation token retrieval

The AWS secrets role code now disallows creation of a dangerous role
configuration; however, pre-existing roles could have existed that would
trigger this now-dangerous code path, so also adding a check for this
configuration at credential retrieval time.

* Run makefmt

* Fix tests

* Fix comments/docs
2019-08-20 12:34:41 -07:00
Vishal Nayak 9b878b0717 go fmt on aws path role files 2019-08-16 11:25:33 -04:00
Joel Thompson e4b9efd37f logical/aws: Refactor role validation (#7276)
This refactors role validation for the AWS secrets engine to be in a
separate method. Previously, all validation was interspersed with the
parsing of parameters when creating/updating a role, which led to a high
degree of complexity. Now, all validation is centralized which makes it
easier to understand and also easier to test (and so a number of test
cases have been added).
2019-08-08 11:53:06 -07:00
Nicolas Corrarello 446b564a2c Add maximum amount of random entropy requested (#7144)
* Add maximum amount of random characters requested at any given time

* Readability changes

* Removing sys/tools/random from the default policy

* Setting the maxBytes value as const

* Declaring maxBytes in the package to use it everywhere

* Using maxBytes in the error message
2019-07-24 18:22:23 -07:00
Jeff Mitchell 3c03f8d7e0
Don't try to revoke certs from PKI on perf standby (#7173)
It needs to do a write for the CRL and to move to the revoked prefix
2019-07-23 08:40:08 -04:00
Jeff Mitchell 7b8c0b58f1
Call goimports as well as gofmt when doing a `make fmt` (#7148)
Closes #7147
2019-07-18 21:04:56 -04:00
Clint f27dc7d5f8 Combined Database backend: Add Static Account support to MongoDB (#7003)
* Implement SetCredentials for MongoDB, adding support for static accounts

* rework SetCredentials to split from CreateUser, and to parse the url for database

* Add integration test for mongodb static account rotation

* check the length of the password results to avoid out-of-bounds

* remove unused method

* use the pre-existing test helper for this. Add parse method to helper

* remove unused command
2019-07-05 14:57:01 -04:00
Jeff Mitchell 45c0afa582 Fix up some displayattr text 2019-07-01 08:47:13 -04:00
Jeff Mitchell fe7bb0b630
Standardize how we format deprecated values in traditional path-help (#7007) 2019-06-27 14:52:52 -04:00
Clint 4b00597609
Combined Database backend: remove create/delete support (#6951)
* remove create/update database user for static accounts

* update tests after create/delete removed

* small cleanups

* update postgresql setcredentials test
2019-06-23 15:58:07 -05:00
Matthew Irish f0d7dc9a6d
UI - add kmip engine (#6936)
* add kmip engine

* adjust where kmip engine is mounted and sketch out routes

* add secret mount path service to share params to engines

* move list-controller and list-route mixins to core addon and adjust imports

* properly link kmip secrets from the secrets list page

* tweak routes and add list controllers

* stub out some models and adapters

* fix mixin exports

* move a bunch of components into the core addon

* use new empty yield in list-view in the namespace template

* scopes list using list-view and list-item components

* simplify and flatten routes, templates for all of the list pages

* role show route and template and scope create template

* add ember-router-helpers

* add more packages to the dependencies of the core addon

* add field-group-show component for listing fields from a model

* move more components to the shared addon

* make configure and configuration routes work and save a generated model

* save and list scopes

* role create, list, read

* list credentials properly

* move allowed attributes to field group

* show allowed operations on role details page

* add kmip logo to mount secrets engine list page

* add role edit page

* show all model attributes on role show page

* enable role edit

* fix newFields error by creating open api role model on the role list route

* only show selected fields on role edit page

* do not send scope and backend attrs to api

* move path-or-array to core addon

* move string-list component to core addon

* remove extra top border when there is only one field group

* add icons for all of the list pages

* update kmip config model so defaultValue doesn't error

* generate credentials

* credential create and show

* only show kmip when feature is enabled

* fix saving of TTL fields generated from Open API

* move masked-input and list-pagination components to core addon

* add param on edit form to allow for calling onSave after render happens

* polish credential show page and redirect there after generating credentials

* add externalLink for kmip engine

* add kmip-breadcrumb component

* use kmip-breadcrumb component

* add linkPrefix param to linked-block component to allow for routing programmatically inside an engine

* redirect to the right place when enabling kmip

* fix linting

* review feedback

* update signature for path-help usage

* fix ttl field expansion test

* remove role filed from role form, fix generate redirect

* remove field-group-show because it's in the core addon

* remove bottom rule from show pages

* fix Max TTL displayAttrs for ssh role

* update edit-form to take fields or attrs

* fix linting

* remove listenAddrs and set default val on ttl if a val is passed in
2019-06-21 16:05:45 -05:00
Madalyn a2606ddccf
update OpenAPI output to use DisplayAttributes struct (#6928) 2019-06-21 11:08:08 -04:00
Clint b55303eddb
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk

* fix issue of storing pointers and now copy

* update to use copy structure

* Remove file, put Item struct def. into other file

* add link

* clean up docs

* refactor internal data structure to hide heap method implementations. Other cleanup after feedback

* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods

* updates after feedback

* refactoring/renaming

* guard against pushing a nil item

* minor updates after feedback

* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC

* Initial Combined database backend implementation of static accounts and automatic rotation

* vendor updates

* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation

* add lock and setup of rotation queue

* vendor the queue

* rebase on new method signature of queue

* remove mongo tests for now

* update default role sql

* gofmt after rebase

* cleanup after rebasing to remove checks for ErrNotFound error

* rebase cdcr-priority-queue

* vendor dependencies with 'go mod vendor'

* website database docs for Static Role support

* document the rotate-role API endpoint

* postgres specific static role docs

* use constants for paths

* updates from review

* remove dead code

* combine and clarify error message for older plugins

* Update builtin/logical/database/backend.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups from feedback

* code and comment cleanups

* move db.RLock higher to protect db.GenerateCredentials call

* Return output with WALID if we failed to delete the WAL

* Update builtin/logical/database/path_creds_create.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* updates after running 'make fmt'

* update after running 'make proto'

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update comment and remove and rearrange some dead code

* Update website/source/api/secret/databases/index.html.md

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups after review

* Update sdk/database/dbplugin/grpc_transport.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* code cleanup after feedback

* remove PasswordLastSet; it's not used

* document GenerateCredentials and SetCredentials

* Update builtin/logical/database/path_rotate_credentials.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* wrap pop and popbykey in backend methods to protect against nil cred rotation queue

* use strings.HasPrefix instead of direct equality check for path

* Forgot to commit this

* updates after feedback

* re-purpose an outdated test to now check that static and dynamic roles cannot share a name

* check for unique name across dynamic and static roles

* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials

* remove commented out code

* refactor to have loadstaticwals filter out wals for roles that no longer exist

* return error if nil input given

* add nil check for input into setStaticAccount

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* add constant for queue tick time in seconds, used for comparrison in updates

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* code cleanup after review

* remove misplaced code comment

* remove commented out code

* create a queue in the Factory method, even if it's never used

* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type

* document new method

* move rotation things into a specific file

* rename test file and consolidate some static account tests

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update code comments, method names, and move more methods into rotation.go

* update comments to be capitalized

* remove the item from the queue before we try to destroy it

* findStaticWAL returns an error

* use lowercase keys when encoding WAL entries

* small cleanups

* remove vestigial static account check

* remove redundant DeleteWAL call in populate queue

* if we error on loading role, push back to queue with 10 second backoff

* poll in initqueue to make sure the backend is setup and can write/delete data

* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false

* add code comments on read-only loop

* code comment updates

* re-push if error returned from find static wal

* add locksutil and acquire locks when pop'ing from the queue

* grab exclusive locks for updating static roles

* Add SetCredentials and GenerateCredentials stubs to mockPlugin

* add a switch in initQueue to listen for cancelation

* remove guard on zero time, it should have no affect

* create a new context in Factory to pass on and use for closing the backend queue

* restore master copy of vendor dir
2019-06-19 14:45:39 -05:00
Lexman 9aa4662cec transit cache is an Interface implemented by wrapped versions of sync… (#6225)
* transit cache is an Interface implemented by wrapped versions of syncmap and golang-lru

* transit cache is an Interface implemented by wrapped versions of syncmap and golang-lru

* changed some import paths to point to sdk

* Apply suggestions from code review

Co-Authored-By: Lexman42 <Lexman42@users.noreply.github.com>

* updates docs with information on transit/cache-config endpoint

* updates vendored files

* fixes policy tests to actually use a cache where expected and renames the struct and storage path used for cache configurations to be more generic

* updates document links

* fixed a typo in a documentation link

* changes cache_size to just size for the cache-config endpoint
2019-06-04 15:40:56 -07:00
Nick Cabatoff 7380c2fd9d Fix a test bug I introduced in 1d13290b361314466f76e251826f60c92aa67bb7 by failing to update my PR to the latest master before merging. 2019-05-09 11:59:22 -04:00
ncabatoff c48936c4fd
Refactor cert util (#6676)
Break dataBundle into two pieces: inputBundle, which contains data that
is specific to the pki backend, and creationBundle, which is a more
generic bundle of validated inputs given to certificate creation/signing routines.

Move functions that only take creationBundle to certutil and make them public.
2019-05-09 11:43:11 -04:00
Mark Gritter 4cab0047a1
Fix test to use stable order to generate expected result. (#6692) 2019-05-07 14:01:49 -05:00
mgritter 2d3d6a856b gofmt fixes. 2019-05-02 16:29:41 -07:00
Jim Kalafut 2835131117
Apply suggestions from code review
Co-Authored-By: mgritter <mgritter@gmail.com>
2019-05-02 18:02:15 -05:00
mgritter 4e22fb6704 Ensure OU entries are not reordered. 2019-05-02 14:31:29 -07:00
Becca Petrin 22a6e54957
Merge pull request #6380 from povils/aws_user_path
AWS add user_path option for role.
2019-04-23 09:05:35 -07:00
ncabatoff 06574da57a
Merge multiple functions for creating consul containers into one. (#6612)
Merge both functions for creating mongodb containers into one.
Add retries to docker container cleanups.
Require $VAULT_ACC be set to enable AWS tests.
2019-04-22 12:26:10 -04:00
Povilas Susinskas 67f5bbe88f AWS backend: Add user_path option for role. 2019-04-22 18:07:21 +02:00
Jeff Mitchell 213b9fd1cf Update to api 1.0.1 and sdk 0.1.8 2019-04-15 14:10:07 -04:00
Becca Petrin d3b7c3ccaf Add explanation to dbplugin.Database (#6581)
* add explanation to the database interface

* wordsmithing

* wrap comments, add comments for Type and Close methods

* will be stored, not with be stored

* update comment on the Type method
2019-04-15 08:39:44 -07:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 1b5155080b Update protobufs, sdk changes 2019-04-12 23:13:14 -04:00
Jeff Mitchell 33faef3840 Fix some test breakages 2019-04-12 22:05:01 -04:00
Jeff Mitchell 28e2ce8577 Fix build breakages 2019-04-12 22:01:13 -04:00
Jeff Mitchell 80c303ac83 Move ldaputil and tlsutil over to sdk 2019-04-12 18:26:54 -04:00
Jeff Mitchell 7ca424e8d2 Move cidrutil to sdk 2019-04-12 18:03:59 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Brian Kassouf f53b728d38
Update plugin.go 2019-04-01 16:45:59 -07:00
Matt Greenfield 080d4652f0 Fix uri_sans param being ignored when use_csr_values=false (#6505) 2019-04-01 16:08:22 -04:00
T.K 453f1ac109 changed misspelled english words (#6432) 2019-03-19 09:32:45 -04:00
Matthew Bamber 4283e6a408 Fix SSH zero address OTP delete (#6390)
* Fix SSH zero address OTP delete

Fixed bug where SSH OTP roles could not be deleted if a zero-address role
previously existed, and there currently exist no zero-address roles.

Fixes #6382

* Eliminate zeroAddressRoles remove function
2019-03-14 08:56:40 -07:00
Jeff Mitchell 3b0d07a440
ToUpper base32 values for TOTP key ingress (#6400)
Fixes #6396
2019-03-12 11:07:03 -04:00
Jim Kalafut a34099b9bb
Use HashTypeMap and remove structs in batch HMAC (#6334) 2019-03-04 14:49:29 -08:00
martinwaite 04c174214c Batch hmac - (#5850) (#5875) 2019-03-04 12:26:20 -08:00
Brian Kassouf efe5671f36 make fmt 2019-02-20 12:12:21 -08:00
madalynrose 625f0c7546
Update OpenAPI responses to include information the UI can use (#6204) 2019-02-14 12:42:44 -05:00
Brian Kassouf 524b65cb9b
Remove netRPC based plugins (#6173)
* Remove netRPC backend plugins

* Remove netRPC database plugins

* Fix tests and comments
2019-02-12 09:31:03 -08:00
Clint 0db43e697b Add signed key constraints to SSH CA [continued] (#6030)
* Adds the ability to enforce particular ssh key types and minimum key
lengths when using Signed SSH Certificates via the SSH Secret Engine.
2019-02-11 13:03:26 -05:00
Becca Petrin ba3ed879f8
Use null strings in MSSQL to prevent errs (#6099) 2019-02-08 10:04:54 -08:00
Brian Nuszkowski 707c6d1813 Add SHA1 signing/verification support to transit engine (#6037)
* Add SHA1 signing/verification support to transit engine

* Update signing/verification endpoint documentation to include sha1 hash algorithm
2019-02-07 15:31:31 -08:00
Becca Petrin 3225a66d34 Return a more helpful error message for unknown db roles (#6157)
* return a more helpful err msg

* update test, print fmt

* fix other test failure
2019-02-07 11:16:23 -08:00
Becca Petrin 421a526e8f dont automatically accept mssql eula (#6169) 2019-02-05 14:11:06 -08:00
Jeff Mitchell 5f249d4005
Add allowed_response_headers (#6115) 2019-02-05 16:02:15 -05:00
Jeff Mitchell 2f9a7c6203
Add more perf standby guards (#6149) 2019-02-01 16:56:57 -05:00
Jeff Mitchell bbc1d53a5d Revert "Refactor common token fields and operations into a helper (#5953)"
This reverts commit 66c226c593bb1cd48cfd8364ac8510cb42b7d67a.
2019-02-01 11:23:40 -05:00
Jeff Mitchell 85a560abba
Refactor common token fields and operations into a helper (#5953) 2019-01-30 16:23:28 -05:00
Jeff Mitchell 5e126f6de8
Implement JWS-compatible signature marshaling (#6077)
This currently only applies to ECDSA signatures, and is a toggleable
option.
2019-01-23 12:31:34 -05:00
ncabatoff 3e3498073e Fix #5973 on windows by disregarding errors when querying legacy cert path. (#6013) 2019-01-08 18:08:21 -08:00
Jim Kalafut d0e2badbae Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Jeff Mitchell e11c7966fa
Change credential_types output to credential_type (#5975)
Fixes #5972
2019-01-04 14:49:53 -05:00
Jeff Mitchell cb1a686e3b
Strip empty strings from database revocation stmts (#5955)
* Strip empty strings from database revocation stmts

It's technically valid to give empty strings as statements to run on
most databases. However, in the case of revocation statements, it's not
only generally inadvisable but can lead to lack of revocations when you
expect them. This strips empty strings from the array of revocation
statements.

It also makes two other changes:

* Return statements on read as empty but valid arrays rather than nulls,
so that typing information is inferred (this is more in line with the
rest of Vault these days)

* Changes field data for TypeStringSlice and TypeCommaStringSlice such
that a client-supplied value of `""` doesn't turn into `[]string{""}`
but rather `[]string{}`.

The latter and the explicit revocation statement changes are related,
and defense in depth.
2018-12-14 09:12:26 -05:00
Lukasz Jagiello 76008b2e1e Remove an empty line for /pki/ca_chain (#5779)
This PR fix #5778.

Easy test case to reproduce the problem:
https://play.golang.org/p/CAMdrOHT7C1

Since `certStr` is empty string during first iteration `strings.Join()`
will merge empty line with first CA cert.

Extra `strings.TrimSpace` call will remove that empty line, before
certificate will be return.
2018-12-12 15:38:35 -05:00
Jeff Mitchell e3c538e9cb Fix tests 2018-12-11 15:04:02 -05:00
Jeff Mitchell c178d05e07
Properly continue if cert entry is nil when tidying (#5933)
Fixes #5931
2018-12-11 11:28:14 -05:00
Jeff Mitchell 13b5c3be51
Fix SSH CA giving 500 if keys need to be deleted (#5897) 2018-12-04 13:29:11 -05:00
Calvin Leung Huang e6ec67fb8f
Use inclusive range on cert role diff comparison (#5737) 2018-11-08 12:15:12 -08:00
Jeff Mitchell fa26beeaed fmt 2018-11-07 16:52:01 -05:00
Becca Petrin 7bd22e6779
Run all builtins as plugins (#5536) 2018-11-06 17:21:24 -08:00
Calvin Leung Huang 20faa90ee3 Use Truncate instead of Round on duration diff (#5691) 2018-11-05 17:32:33 -05:00
Calvin Leung Huang 1a4e8fe53d Round time diff to nearest second to reduce flakiness (#5688) 2018-11-05 16:49:25 -05:00
Vishal Nayak 332e32294a
Remove namespace.TestContext and namespace.TestNamespace (#5682) 2018-11-05 11:11:32 -05:00
Nicolas Corrarello 0b44a55d22 Adding support for Consul 1.4 ACL system (#5586)
* Adding support for Consul 1.4 ACL system

* Working tests

* Fixed logic gate

* Fixed logical gate that evaluate empty policy or empty list of policy names

* Ensure tests are run against appropiate Consul versions

* Running tests against official container with a 1.4.0-rc1 tag

* policies can never be nil (as even if it is empty will be an empty array)

* addressing feedback, refactoring tests

* removing cast

* converting old lease field to ttl, adding max ttl

* cleanup

* adding missing test

* testing wrong version

* adding support for local tokens

* addressing feedback
2018-11-02 10:44:12 -04:00
Vishal Nayak 142a944bfd
Allow @ to be part of key name in TOTP secret engine (#5652)
* Allow @ to be part of key name in TOTP secret engine

* Allow @ for key name while generating the code
2018-10-31 12:57:18 -04:00
Jeff Mitchell 6c488921ff Fix website/path-help docs around pki/tidy 2018-10-30 21:33:30 -04:00
Balazs Nagy ca5c60642e Use tidy_revoked_certs instead of tidy_revocation_list (#5608) 2018-10-29 19:29:35 -04:00
andrejvanderzee 585911c79e Added role-option max_sts_ttl to cap TTL for AWS STS credentials. (#5500)
* Added role-option max_sts_ttl to cap TTL for AWS STS credentials.

* Allow for setting max_sts_ttl to 0 after it has been set already.

* Fixed message in error response for default_sts_ttl > max_sts_ttl.
2018-10-20 10:36:47 -04:00
Brian Kassouf e943a60041
Plugin version negotiation (#5434)
* Plugin version updates

* Update datatbase plugins

* Revert netRPC deletions

* Revert netRPC deletions

* Update plugins to serve both versions

* Update database plugins

* Add Initialize back in

* revert pointer changes

* Add deprecation warning

* Update tests

* Update go-plugin

* Review Feedback
2018-10-19 15:56:17 -07:00
Jeff Mitchell 5e2cc31cb6
Remove now-spurious ttl check and logic from sign-verbatim. (#5552)
This endpoint eventually goes through generateCreationBundle where we
already have the right checks.

Also add expiration to returned value to match output when using root
generation.

Fixes #5549
2018-10-19 11:13:59 -04:00
Chris Hoffman 6a462ea4d3
Only run cassandra test with VAULT_ACC set 2018-10-19 11:09:28 -04:00