This was causing failures when running `vault server -dev`:
> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x105c41c1c]
>
> goroutine 1 [running]:
> github.com/hashicorp/vault/command.(*ServerCommand).parseConfig(0x140005a2180)
> .../vault/command/server.go:429 +0x5c
Interestingly, we do not have a test case for running the dev
sever.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning about EA in FIPS mode
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* CLI: Add ability to display ListResponseWithInfos
The Vault Server API includes a ListResponseWithInfo call, allowing LIST
responses to contain additional information about their keys. This is in
a key=value mapping format (both for each key, to get the additional
metadata, as well as within each metadata).
Expand the `vault list` CLI command with a `-detailed` flag (and env var
VAULT_DETAILED_LISTS) to print this additional metadata. This looks
roughly like the following:
$ vault list -detailed pki/issuers
Keys issuer_name
---- -----------
0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7 n/a
35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0 n/a
382fad1e-e99c-9c54-e147-bb1faa8033d3 n/a
8bb4a793-2ad9-460c-9fa8-574c84a981f7 n/a
8bd231d7-20e2-f21f-ae1a-7aa3319715e7 n/a
9425d51f-cb81-426d-d6ad-5147d092094e n/a
ae679732-b497-ab0d-3220-806a2b9d81ed n/a
c5a44a1f-2ae4-2140-3acf-74b2609448cc utf8
d41d2419-efce-0e36-c96b-e91179a24dc1 something
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow detailed printing of LIST responses in JSON
When using the JSON formatter, only the absolute list of keys were
returned. Reuse the `-detailed` flag value for the `-format=json` list
response printer, allowing us to show the complete API response returned
by Vault.
This returns something like the following:
{
"request_id": "e9a25dcd-b67a-97d7-0f08-3670918ef3ff",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"key_info": {
"0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7": {
"issuer_name": ""
},
"35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0": {
"issuer_name": ""
},
"382fad1e-e99c-9c54-e147-bb1faa8033d3": {
"issuer_name": ""
},
"8bb4a793-2ad9-460c-9fa8-574c84a981f7": {
"issuer_name": ""
},
"8bd231d7-20e2-f21f-ae1a-7aa3319715e7": {
"issuer_name": ""
},
"9425d51f-cb81-426d-d6ad-5147d092094e": {
"issuer_name": ""
},
"ae679732-b497-ab0d-3220-806a2b9d81ed": {
"issuer_name": ""
},
"c5a44a1f-2ae4-2140-3acf-74b2609448cc": {
"issuer_name": "utf8"
},
"d41d2419-efce-0e36-c96b-e91179a24dc1": {
"issuer_name": "something"
}
},
"keys": [
"0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7",
"35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0",
"382fad1e-e99c-9c54-e147-bb1faa8033d3",
"8bb4a793-2ad9-460c-9fa8-574c84a981f7",
"8bd231d7-20e2-f21f-ae1a-7aa3319715e7",
"9425d51f-cb81-426d-d6ad-5147d092094e",
"ae679732-b497-ab0d-3220-806a2b9d81ed",
"c5a44a1f-2ae4-2140-3acf-74b2609448cc",
"d41d2419-efce-0e36-c96b-e91179a24dc1"
]
},
"warnings": null
}
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use field on UI rather than secret.Data
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Only include headers from visitable key_infos
Certain API endpoints return data from non-visitable key_infos, by
virtue of using a hand-rolled response. Limit our headers to those
from visitable key_infos. This means we won't return entire columns with
n/a entries, if no key matches the key_info key that includes that
header.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use setupEnv sourced detailed info
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix changelog environment variable
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix broken tests using setupEnv
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format
* VAULT-5885 Add changelog
* VAULT-5885 Update changelog as per PR comments
* VAULT-5885 Update changelog category to just 'auth'
* VAULT-5885 Hide useless token info in two-phase MFA case
* VAULT-5885 Update changelog to reflect token info now no longer present
* VAULT-5885 split up changelog into three blocks
* Use new parseutil helper: Safe variants
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update parseutil to v0.1.5
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix additional integer overflow in command/server
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.
Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.
I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
* Add server information as well as ability to collect metrics from DR secondary
* Update debug docs
Adding additional information around ability to gather metrics from DR secondary
* Fix broken link in updated doc
* Create 15316.txt
Create changelog entry
* Fix Formatting
* Update website/content/docs/commands/debug.mdx
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update changelog/15316.txt
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Trigger Build
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
This requires bumping https://github.com/mitchellh/go-testing-interface.
For this new version, we have to create a wrapper to convert
the stdlib `testing.TB` interface to the
`mitchellh/go-testing-interface` `T` interface, since it uses
`Parallel()` now, which is not supported by `testing.TB`. This had to be
added to a new package, `benchhelpers`, to avoid a circular dependency
in `testhelpers`.
We also have to *unbump* https://github.com/armon/go-metrics since
updating it breaks our usage of
https://github.com/google/go-metrics-stackdriver
I verified that the new `pkiCert` template function works with agent
injection using annotations like:
```yaml
vault.hashicorp.com/agent-inject-secret-sample.crt: "pki/issue/example-dot-com"
vault.hashicorp.com/agent-inject-template-sample.crt: |
{{ pkiCert "pki/issue/example-dot-com" "common_name=foo.example.com" "ttl=1h" }}
```
* WIP: output policy
* Outputs example policy HCL for given request
* Simplify conditional
* Add PATCH capability
* Use OpenAPI spec and regex patterns to determine if path is sudo
* Add test for isSudoPath
* Add changelog
* Fix broken CLI tests
* Add output-policy to client cloning code
* Smaller fixes from PR comments
* Clone client instead of saving and restoring custom values
* Fix test
* Address comments
* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError
* Print errors saved in buffer from preflight KV requests
* Unescape characters in request URL
* Rename methods and properties to improve readability
* Put KV-specificness at front of KV-specific error
* Simplify logic by doing more direct returns of strings and errors
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Use precompiled regexes and move OpenAPI call to tests (#15170)
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Make stderr writing more obvious, fix nil pointer deref
* add BuildDate to version base
* populate BuildDate with ldflags
* include BuildDate in FullVersionNumber
* add BuildDate to seal-status and associated status cmd
* extend core/versions entries to include BuildDate
* include BuildDate in version-history API and CLI
* fix version history tests
* fix sys status tests
* fix TestStatusFormat
* remove extraneous LD_FLAGS from build.sh
* add BuildDate to build.bat
* fix TestSysUnseal_Reset
* attempt to add build-date to release builds
* add branch to github build workflow
* add get-build-date to build-* job needs
* fix release build command vars
* add missing quote in release build command
* Revert "add branch to github build workflow"
This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.
* add changelog entry
* [VAULT-1618] Agent error log level is mismatched
`logLevelToStringPtr` translates `go-hclog`'s `ERROR` to `"ERROR"` for
Consul Template's runner, but that expects `ERR` and is quite strict
about it.
This will address https://github.com/hashicorp/vault-k8s/issues/223
after it is set as the default image in `vault-k8s`.
I didn't find a simple way to test this other than starting up a full
server and agent and letting them run, which is unfortunately fairly
slow.
I confirmed that this addresses the original issue by modifying the helm
chart with the values in this commit and patching the log level to `err`.
* VAULT-1618 Add changelog/14424.txt
* VAULT-1618 Update changelog/14424.txt based on @kalafut suggestion
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* VAULT-1618 Move cancel and server stop into defer in tests
* VAULT-1618 Triggering CircleCI tests
* VAULT-1618 Replace ioutil with os functions for agent template tests
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* Add explanation to help text and flag usage text
* KV get with new mount flag
* Clearer naming
* KV Put, Patch, Metadata Get + corresponding tests
* KV Delete, Destroy, Rollback, Undelete, MetadataDelete, MetadataPatch, MetadataPut
* Update KV-v2 docs to use mount flag syntax
* Add changelog
* Run make fmt
* Clarify deprecation message in help string
* Address style comments
* Warn on upper case in policy name
* Rename name variable to be less confusing
* Use more general solution for other string issues
* Clarify changelog
* Remove unnecessary check
* Don't throw CLI warning until after past errors
* Add before and after names with quotes to show spacing changes
* Update mholt/archiver to v3.5.0
* Bump archiver to 3.5.1
* Vendor dependencies
* Use newer go
* go mod tidy
* Remove vendor
* Rm vendor
* Revert api and sdk sums
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
These options must be specified multiple times in order to be properly
parsed. However, the present description suggests that a comma-separated
list would work as well, however this isn't the case and results in a
slice containing a single string (with all comma-separated values) in
the API request. Clarify the argument help text to make this clearer.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Full secret path in table output of get and put
* Add path output to KV patch and metadata get
* Add changelog
* Don't print secret path for kv-v1
* Make more readable
* Switch around logic to not swallow error
* Add test for secret path
* Fix metadata test
* Add unit test for padequalsigns
* Remove wonky kv get tests
* Add support for PROXY protocol v2 in TCP listener
I did not find tests for this so I added one trying to cover different
configurations to make sure I did not break something. As far as I know,
the behavior should be exactly the same as before except for one thing
when proxy_protocol_behavior is set to "deny_unauthorized", unauthorized
requests were previously silently reject because of https://github.com/armon/go-proxyproto/blob/7e956b284f0a/protocol.go#L81-L84
but it will now be logged.
Also fixes https://github.com/hashicorp/vault/issues/9462 by adding
support for `PROXY UNKNOWN` for PROXY protocol v1.
Closes https://github.com/hashicorp/vault/issues/3807
* Add changelog
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* interactive CLI for mfa login
* minor fixes
* bail if no input was inserted
* change label name
* interactive CLI when single methodID is returned from login request
* minor fix
* adding changelog
* addressing feedback
* a user with a terminal should be able to choose between interactive and non-interactive. A user without a terminal should not be able to use the interactive mode.
Co-authored-by: Josh Black <raskchanky@gmail.com>
* agent/azure: adds ability to use specific user assigned managed identity for auto auth
* add changelog
* change wording in error and docs
* Update website/content/docs/agent/autoauth/methods/azure.mdx
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Update website/content/docs/agent/autoauth/methods/azure.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* docs formatting
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
This patch adds a new /agent/v1/metrics that will return metrics on the
running Vault agent. Configuration is done using the same telemetry
stanza as the Vault server. For now default runtime metrics are
returned with a few additional ones specific to the agent:
- `vault.agent.auth.failure` and `vault.agent.auth.success` to monitor
the correct behavior of the auto auth mechanism
- `vault.agent.proxy.success`, `vault.agent.proxy.client_error` and
`vault.agent.proxy.error` to check the connection with the Vault server
- `vault.agent.cache.hit` and `vault.agent.cache.miss` to monitor the
cache
Closes https://github.com/hashicorp/vault/issues/8649
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* PR feedback
* duo to handle preventing passcode reuse
Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
* port SSCT OSS
* port header hmac key to ent and generate token proto without make command
* remove extra nil check in request handling
* add changelog
* add comment to router.go
* change test var to use length constants
* remove local index is 0 check and extra defer which can be removed after use of ExternalID
In future Vault Enterprise versions, we'll be building Vault with
FIPS-validated cryptography. To help operators understand their
environment, we'll want to expose information about their FIPS status
when they're running a FIPS version of Vault.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* adds development workflow to mirage config
* adds mirage handler and factory for mfa workflow
* adds mfa handling to auth service and cluster adapter
* moves auth success logic from form to controller
* adds mfa form component
* shows delayed auth message for all methods
* adds new code delay to mfa form
* adds error views
* fixes merge conflict
* adds integration tests for mfa-form component
* fixes auth tests
* updates mfa response handling to align with backend
* updates mfa-form to handle multiple methods and constraints
* adds noDefault arg to Select component
* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios
* adds tests
* flaky test fix attempt
* reverts test fix attempt
* adds changelog entry
* updates comments for todo items
* removes faker from mfa mirage factory and handler
* adds number to word helper
* fixes tests
* Revert "Merge branch 'main' into ui/mfa"
This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.
* format-ttl helper fix from main
* store version history as utc; add self-heal logic
* add sys/version-history endpoint
* change version history from GET to LIST, require auth
* add "vault version-history" CLI command
* add vault-version CLI error message for version string parsing
* adding version-history API and CLI docs
* add changelog entry
* some version-history command fixes
* remove extraneous cmd args
* fix version-history command help text
* specify in docs that endpoint was added in 1.10.0
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* enforce UTC within storeVersionTimestamp directly
* fix improper use of %w in logger.Warn
* remove extra err check and erroneous return from loadVersionTimestamps
* add >= 1.10.0 warning to version-history cmd
* move sys/version-history tests
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Add documentation for Managed Keys
- Add concept, sys/api and pki updates related to managed keys
* Review feedback
- Reworked quite a bit of the existing documentation based on feedback
and a re-reading
- Moved the managed keys out of the concepts section and into the
enterprise section
* Address broken links and a few grammar tweaks
In doing some testing I found that the listener clusteraddr isn't really used, or at least isn't as important as the top-level clusteraddr setting. As such, go-sockaddr templating needs to be implemented for the top-level `cluster_addr` setting or it's unusable for HA.
Also fix a nil pointer panic I discovered at the same time.
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata
* add kv metadata patch command
* add changelog entry
* success tests for kv metadata patch flags
* add more kv metadata patch flags tests
* add kv metadata patch cas warning test
* add kv-v2 key metadata patch API docs
* add kv metadata patch to docs
* prevent unintentional field overwriting in kv metadata put cmd
* like create/update ops, prevent patch to paths ending in /
* fix kv metadata patch cmd in docs
* fix flag defaults for kv metadata put
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata
* fix TestKvMetadataPatchCommand_Flags test
* doc fixes
* go get vault-plugin-secrets-kv@master; go mod tidy
* CLI changes for new mount tune config parameter allowed_managed_keys
* Correct allowed_managed_keys description in auth and secrets
* Documentation update for secrets and removed changes for auth
* Add changelog and remove documentation changes for auth
* removed changelog
* Correct the field description
* VAULT-1564 report in-flight requests
* adding a changelog
* Changing some variable names and fixing comments
* minor style change
* adding unauthenticated support for in-flight-req
* adding documentation for the listener.profiling stanza
* adding an atomic counter for the inflight requests
addressing comments
* addressing comments
* logging completed requests
* fixing a test
* providing log_requests_info as a config option to determine at which level requests should be logged
* removing a member and a method from the StatusHeaderResponseWriter struct
* adding api docks
* revert changes in NewHTTPResponseWriter
* Fix logging invalid log_requests_info value
* Addressing comments
* Fixing a test
* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core
* fixing go.sum
* minor refactoring
* protecting InFlightRequests from data race
* another try on fixing a data race
* another try to fix a data race
* addressing comments
* fixing couple of tests
* changing log_requests_info to log_requests_level
* minor style change
* fixing a test
* removing the lock in InFlightRequests
* use single-argument form for interface assertion
* adding doc for the new configuration paramter
* adding the new doc to the nav data file
* minor fix
Changes the error output for the second open of the persistent cache
file, to differentiate it from the c.UI.Error message for the initial
open of the cache file, just to make it easier to tell where a problem
occurred.
- Add the kms_library configuration stanza to Vault's command/server
- Provide validation of keys and general configuration.
- Add initial kms_library configuration documentation
- Attempt at startup to verify we can read the configured HSM Library
- Hook in KmsLibrary config into the Validate to detect typo/unused keys
* See what it looks like to replace "master key" with "root key". There are two places that would require more challenging code changes: the storage path `core/master`, and its contents (the JSON-serialized EncodedKeyringtructure.)
* Restore accidentally deleted line
* Add changelog
* Update root->recovery
* Fix test
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Native Login method, userpass and approle interfaces to implement it
* Add AWS auth interface for Login, unexported struct fields for now
* Add Kubernetes client login
* Add changelog
* Add a test for approle client login
* Return errors from LoginOptions, use limited reader for secret ID
* Fix auth comment length
* Return actual type not interface, check for client token in tests
* Require specification of secret ID location using SecretID struct as AppRole arg
* Allow password from env, file, or plaintext
* Add flexibility in how to fetch k8s service token, but still with default
* Avoid passing strings that need to be validated by just having different login options
* Try a couple real tests with approle and userpass login
* Fix method name in comment
* Add context to Login methods, remove comments about certain sources being inherently insecure
* Perform read of secret ID at login time
* Read password from file at login time
* Pass context in integ tests
* Read env var values in at login time, add extra tests
* Update api version
* Revert "Update api version"
This reverts commit 1ef3949497dcf878c47e0e5ffcbc8cac1c3c1679.
* Update api version in all go.mod files
* go get vault-plugin-secrets-kv@extend-kv-metadata-to-get-and-put
* test for custom_metadata in kv get, put, patch command output
* remove flagFormat-specific check from TestKVMetadataGetCommand
* rewrite custom metadata changelog entry
* go get vault-plugin-secrets-kv@master
* go mod tidy
* CLI makes request to incorrect URL when namespace is both provided as argument and part of the path
fixes#12675
* adding change log
* removing a switch and addressing a possibility of out of bound index
* operator generate-root -decode: allow token from stdin
Allow passing "-" as the value for -decode, causing the encoded token to
be read from stdin. This is intended to prevent leaking the encoded
token + otp into process logs in enterprise environments.
* add changelog entry for PR12881
* add check/test for empty decode value passed via stdin
Uses a bufconn listener between consul-template and vault-agent when
caching is enabled and either templates or a listener is defined. This
means no listeners need to be defined in vault-agent for just
templating. Always routes consul-template through the vault-agent
cache (instead of only when persistent cache is enabled).
Uses a local transportDialer interface in config.Cache{}.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
Dave May