Commit graph

1530 commits

Author SHA1 Message Date
Jason O'Donnell dd2ced661b
agent: add disable_idle_connections configurable (#15986)
* agent: add disable_keep_alives configurable

* Add empty test

* Add website doc

* Change to disable_idle_connections

* Update tests and doc

* Add note about env

* Changelog

* Change to slice

* Remove unused disable keep alive methods

* Add invalid value test
2022-06-16 18:06:22 -04:00
Josh Black d2ed39a04e
Correct drift between ENT and OSS (#15966) 2022-06-14 17:53:19 -07:00
Dave May 0f42131350
Fix debug bundle panic on Windows (#14399)
* Fix debug bundle panic on Windows

* Add changelog entry
2022-06-09 15:57:45 -07:00
akshya96 8f115a9904
Parse ha_storage in config (#15900)
* parsing values in config ha_storage

* adding changelog

* adding test to parse storage
2022-06-09 15:55:49 -07:00
Chris Capurso 75aa55eb57
add OSS noop check for valid ent storage (#15894) 2022-06-08 17:15:28 -04:00
Alexander Scheel 8d8a95cbf6
Add missing nil check to FIPS EA verification (#15883)
This was causing failures when running `vault server -dev`:

> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x105c41c1c]
>
> goroutine 1 [running]:
> github.com/hashicorp/vault/command.(*ServerCommand).parseConfig(0x140005a2180)
> 	.../vault/command/server.go:429 +0x5c

Interestingly, we do not have a test case for running the dev
sever.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 15:45:46 -04:00
Alexander Scheel dd6c339440
Add warning about EA in FIPS mode (#15858)
* Add warning about EA in FIPS mode

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 08:57:48 -04:00
davidadeleon 0026788d4b
api/monitor: Adding log format to monitor command and debug (#15536)
* Correct handling of "unspecified" log level

* Setting log-format default on monitor path

* Create changelog file

* Update website/content/api-docs/system/monitor.mdx

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-24 13:10:53 -04:00
Josh Black 416504d8c3
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
VAL c789fe3c8d
Set special output flags whether single or double hyphen (#15528) 2022-05-19 17:56:30 -07:00
Christopher Swenson e6fb16be9c
Remove spurious fmt.Printf calls including one of a key (#15344)
And add a semgrep for fmt.Printf/Println.
2022-05-19 12:27:02 -07:00
Alexander Scheel 9a3f34a41e
Vault CLI: show detailed information with ListResponseWithInfo (#15417)
* CLI: Add ability to display ListResponseWithInfos

The Vault Server API includes a ListResponseWithInfo call, allowing LIST
responses to contain additional information about their keys. This is in
a key=value mapping format (both for each key, to get the additional
metadata, as well as within each metadata).

Expand the `vault list` CLI command with a `-detailed` flag (and env var
VAULT_DETAILED_LISTS) to print this additional metadata. This looks
roughly like the following:

    $ vault list -detailed pki/issuers
    Keys                                    issuer_name
    ----                                    -----------
    0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7    n/a
    35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0    n/a
    382fad1e-e99c-9c54-e147-bb1faa8033d3    n/a
    8bb4a793-2ad9-460c-9fa8-574c84a981f7    n/a
    8bd231d7-20e2-f21f-ae1a-7aa3319715e7    n/a
    9425d51f-cb81-426d-d6ad-5147d092094e    n/a
    ae679732-b497-ab0d-3220-806a2b9d81ed    n/a
    c5a44a1f-2ae4-2140-3acf-74b2609448cc    utf8
    d41d2419-efce-0e36-c96b-e91179a24dc1    something

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow detailed printing of LIST responses in JSON

When using the JSON formatter, only the absolute list of keys were
returned. Reuse the `-detailed` flag value for the `-format=json` list
response printer, allowing us to show the complete API response returned
by Vault.

This returns something like the following:

    {
      "request_id": "e9a25dcd-b67a-97d7-0f08-3670918ef3ff",
      "lease_id": "",
      "lease_duration": 0,
      "renewable": false,
      "data": {
        "key_info": {
          "0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7": {
            "issuer_name": ""
          },
          "35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0": {
            "issuer_name": ""
          },
          "382fad1e-e99c-9c54-e147-bb1faa8033d3": {
            "issuer_name": ""
          },
          "8bb4a793-2ad9-460c-9fa8-574c84a981f7": {
            "issuer_name": ""
          },
          "8bd231d7-20e2-f21f-ae1a-7aa3319715e7": {
            "issuer_name": ""
          },
          "9425d51f-cb81-426d-d6ad-5147d092094e": {
            "issuer_name": ""
          },
          "ae679732-b497-ab0d-3220-806a2b9d81ed": {
            "issuer_name": ""
          },
          "c5a44a1f-2ae4-2140-3acf-74b2609448cc": {
            "issuer_name": "utf8"
          },
          "d41d2419-efce-0e36-c96b-e91179a24dc1": {
            "issuer_name": "something"
          }
        },
        "keys": [
          "0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7",
          "35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0",
          "382fad1e-e99c-9c54-e147-bb1faa8033d3",
          "8bb4a793-2ad9-460c-9fa8-574c84a981f7",
          "8bd231d7-20e2-f21f-ae1a-7aa3319715e7",
          "9425d51f-cb81-426d-d6ad-5147d092094e",
          "ae679732-b497-ab0d-3220-806a2b9d81ed",
          "c5a44a1f-2ae4-2140-3acf-74b2609448cc",
          "d41d2419-efce-0e36-c96b-e91179a24dc1"
        ]
      },
      "warnings": null
    }

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use field on UI rather than secret.Data

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Only include headers from visitable key_infos

Certain API endpoints return data from non-visitable key_infos, by
virtue of using a hand-rolled response. Limit our headers to those
from visitable key_infos. This means we won't return entire columns with
n/a entries, if no key matches the key_info key that includes that
header.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use setupEnv sourced detailed info

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix changelog environment variable

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix broken tests using setupEnv

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-18 13:00:50 -04:00
Hridoy Roy 679ccc81a5
Query and Precompute Non-Contiguous Segments in the Activity Log (#15352)
* query and precompute non-contiguous segments in the activity log

* changelog

* newline formatting

* make fmt

* report listener and storage types as found keys

* report listener and storage types as found keys

* Update vault/activity_log_test.go

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

* review comments

* merge conflict

* merge conflict

* merge conflict

* fix unchecked merge conflict

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-17 12:17:32 -07:00
akshya96 4e9e9b7eda
Vault-6037 making filesystem permissions check opt-in (#15452)
* adding env var changes

* adding changelog

* adding strcov.ParseBool
2022-05-17 11:34:31 -07:00
Violet Hynes 2c6bcbdeb5
VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format (#15428)
* VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format

* VAULT-5885 Add changelog

* VAULT-5885 Update changelog as per PR comments

* VAULT-5885 Update changelog category to just 'auth'

* VAULT-5885 Hide useless token info in two-phase MFA case

* VAULT-5885 Update changelog to reflect token info now no longer present

* VAULT-5885 split up changelog into three blocks
2022-05-17 14:03:02 -04:00
Alexander Scheel 8750512f9f
Fix integer overflows with new parseutil (#15437)
* Use new parseutil helper: Safe variants

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update parseutil to v0.1.5

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix additional integer overflow in command/server

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:57:19 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Christopher Swenson 0af0543bbe
VAULT-5935 agent: redact renew-self if using auto auth (#15380)
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.

Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.

I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
2022-05-12 09:25:55 -07:00
Hridoy Roy 0c955eb743
report listener and storage types as found keys (#15383)
* report listener and storage types as found keys

* changelog
2022-05-12 09:04:56 -07:00
AnPucel ba88b1da16
[VAULT-14990] Support retrieving kv secret paths with trailing spaces (#15188) 2022-05-10 14:07:45 -07:00
davidadeleon 9e869c52fa
Add DR Metric scraping capability to debug command (#15316)
* Add server information as well as ability to collect metrics from DR secondary

* Update debug docs

Adding additional information around ability to gather metrics from DR secondary

* Fix broken link in updated doc

* Create 15316.txt

Create changelog entry

* Fix Formatting

* Update website/content/docs/commands/debug.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update changelog/15316.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Trigger Build

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2022-05-06 16:04:08 -04:00
Christopher Swenson 0affe226ad
Update deps for consul-template 0.29.0 (#15293)
This requires bumping https://github.com/mitchellh/go-testing-interface.
For this new version, we have to create a wrapper to convert
the stdlib `testing.TB` interface to the
`mitchellh/go-testing-interface` `T` interface, since it uses
`Parallel()` now, which is not supported by `testing.TB`. This had to be
added to a new package, `benchhelpers`, to avoid a circular dependency
in `testhelpers`.

We also have to *unbump* https://github.com/armon/go-metrics since
updating it breaks our usage of
https://github.com/google/go-metrics-stackdriver

I verified that the new `pkiCert` template function works with agent
injection using annotations like:

```yaml
vault.hashicorp.com/agent-inject-secret-sample.crt: "pki/issue/example-dot-com"
vault.hashicorp.com/agent-inject-template-sample.crt: |
  {{ pkiCert "pki/issue/example-dot-com" "common_name=foo.example.com" "ttl=1h" }}
```
2022-05-05 10:30:40 -07:00
Chris Capurso 5f3d63a03a
clone vault agent client with headers (#15274) 2022-05-03 16:27:53 -04:00
Alexander Tischenko b89fc5af96
When running under systemd, send ready when server completed reloading config #7028 (#15041) 2022-05-03 08:34:11 -04:00
Jason O'Donnell e9535bda2f
agent/auto-auth: Add min_backoff to set first backoff value (#15204)
* Add initial_backoff to auto-auth method

* Disable retries in client

* Fix bug

* Thread initial backoff to CT

* Add comment

* Change to min_backoff

* changelog

* remove initial references, review

* fix test

* Thread max_backoff through

* Add doc note for max_backoff/templating
2022-04-29 12:31:32 -04:00
VAL 0ef529b710
Global flag that outputs minimum policy HCL required for an operation (#14899)
* WIP: output policy

* Outputs example policy HCL for given request

* Simplify conditional

* Add PATCH capability

* Use OpenAPI spec and regex patterns to determine if path is sudo

* Add test for isSudoPath

* Add changelog

* Fix broken CLI tests

* Add output-policy to client cloning code

* Smaller fixes from PR comments

* Clone client instead of saving and restoring custom values

* Fix test

* Address comments

* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError

* Print errors saved in buffer from preflight KV requests

* Unescape characters in request URL

* Rename methods and properties to improve readability

* Put KV-specificness at front of KV-specific error

* Simplify logic by doing more direct returns of strings and errors

* Use precompiled regexes and move OpenAPI call to tests

* Remove commented out code

* Remove legacy MFA paths

* Remove unnecessary use of client

* Move sudo paths map to plugin helper

* Remove unused error return

* Add explanatory comment

* Remove need to pass in address

* Make {name} regex less greedy

* Use method and path instead of info from retryablerequest

* Add test for IsSudoPaths, use more idiomatic naming

* Use precompiled regexes and move OpenAPI call to tests (#15170)

* Use precompiled regexes and move OpenAPI call to tests

* Remove commented out code

* Remove legacy MFA paths

* Remove unnecessary use of client

* Move sudo paths map to plugin helper

* Remove unused error return

* Add explanatory comment

* Remove need to pass in address

* Make {name} regex less greedy

* Use method and path instead of info from retryablerequest

* Add test for IsSudoPaths, use more idiomatic naming

* Make stderr writing more obvious, fix nil pointer deref
2022-04-27 16:35:18 -07:00
Tom Proctor b30e7d3545
Upgrade hashicorp/consul-template dependency (#15092)
* Includes sprig template functions
* Includes improvements to writeTo template function
* Add sprig functions test, improve failure message
2022-04-19 20:51:11 +01:00
Chris Capurso e69f89c279
Add build date (#14957)
* add BuildDate to version base

* populate BuildDate with ldflags

* include BuildDate in FullVersionNumber

* add BuildDate to seal-status and associated status cmd

* extend core/versions entries to include BuildDate

* include BuildDate in version-history API and CLI

* fix version history tests

* fix sys status tests

* fix TestStatusFormat

* remove extraneous LD_FLAGS from build.sh

* add BuildDate to build.bat

* fix TestSysUnseal_Reset

* attempt to add build-date to release builds

* add branch to github build workflow

* add get-build-date to build-* job needs

* fix release build command vars

* add missing quote in release build command

* Revert "add branch to github build workflow"

This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.

* add changelog entry
2022-04-19 14:28:08 -04:00
Vinny Mannello 6116903a37
[Vault-5248] MFA support for api login helpers (#14900)
* Add MFA support to login helpers
2022-04-15 11:13:15 -07:00
akshya96 87a2516e73
Fix nightly tests and debug (#14970)
* adding env var

* add fixes

* fixing debug

* removing umask from tests
2022-04-12 06:08:28 +05:30
akshya96 5e0ed27b87
adding env var (#14958) 2022-04-07 12:15:05 -07:00
Anton Averchenkov 23a3f950e4
Revert the WithContext changes to vault tests (#14947) 2022-04-07 15:12:58 -04:00
Christopher Swenson 0de8e8459e
Agent error log level is mismatched (#14424)
* [VAULT-1618] Agent error log level is mismatched

`logLevelToStringPtr` translates `go-hclog`'s `ERROR` to `"ERROR"` for
Consul Template's runner, but that expects `ERR` and is quite strict
about it.

This will address https://github.com/hashicorp/vault-k8s/issues/223
after it is set as the default image in `vault-k8s`.

I didn't find a simple way to test this other than starting up a full
server and agent and letting them run, which is unfortunately fairly
slow.

I confirmed that this addresses the original issue by modifying the helm
chart with the values in this commit and patching the log level to `err`.

* VAULT-1618 Add changelog/14424.txt

* VAULT-1618 Update changelog/14424.txt based on @kalafut suggestion

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

* VAULT-1618 Move cancel and server stop into defer in tests

* VAULT-1618 Triggering CircleCI tests

* VAULT-1618 Replace ioutil with os functions for agent template tests

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2022-04-07 11:03:38 -07:00
VAL 2113ae1021
Mount flag syntax to mitigate confusion from KV-v2 path discrepancies (#14807)
* Add explanation to help text and flag usage text

* KV get with new mount flag

* Clearer naming

* KV Put, Patch, Metadata Get + corresponding tests

* KV Delete, Destroy, Rollback, Undelete, MetadataDelete, MetadataPatch, MetadataPut

* Update KV-v2 docs to use mount flag syntax

* Add changelog

* Run make fmt

* Clarify deprecation message in help string

* Address style comments
2022-04-06 13:58:06 -07:00
Anton Averchenkov c74feaa6ac
Use WriteWithContext in auth helpers (#14775) 2022-04-06 11:20:34 -04:00
akshya96 796003ddda
Vault 3992 ToB Config and Plugins Permissions (#14817)
* updating changes from ent PR

* adding changelog

* fixing err

* fixing semgrep error
2022-04-04 09:45:41 -07:00
akshya96 9f03f86077
Vault 3999 Change permissions for directory/archive created by debug command (#14846)
* adding debug changes from ent

* adding changelog
2022-04-04 09:44:03 -07:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00
hghaf099 8f6a97b81b
Vault-4279 reporting redundant/unused keys in config (#14752)
* Vault-4279 reporting redundant/unused keys in config

* missing validate step

* CL

* Change the log level
2022-04-01 10:34:27 -04:00
Chris Capurso dbda6e6ec5
add kv write retry logic for TestKVPatchCommand_RWMethodPolicyVariations (#14812) 2022-03-31 17:18:14 -04:00
Chris Capurso 157a655632
Fix CLI panic caused by single backslash values (#14523)
* upgrade go-secure-stdlib/kv-builder to v0.1.2

* add kv test to ensure single backslash values no longer panic

* add changelog entry
2022-03-24 16:40:32 -04:00
VAL f9372145dd
Warn on upper case in policy name (#14670)
* Warn on upper case in policy name

* Rename name variable to be less confusing

* Use more general solution for other string issues

* Clarify changelog

* Remove unnecessary check

* Don't throw CLI warning until after past errors

* Add before and after names with quotes to show spacing changes
2022-03-24 13:29:11 -07:00
Anton Averchenkov 1222375d1a
Add context-aware functions to vault/api (#14388) 2022-03-23 17:47:43 -04:00
Lars Lehtonen 431c40629c
command: fix dropped errors (#14609) 2022-03-21 11:26:38 -06:00
Zach Stone b43ed904f2
Update mholt/archiver to v3.5.1 (#11584)
* Update mholt/archiver to v3.5.0

* Bump archiver to 3.5.1

* Vendor dependencies

* Use newer go

* go mod tidy

* Remove vendor

* Rm vendor

* Revert api and sdk sums

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2022-03-16 09:42:21 -05:00
hghaf099 361646ab26
add MFA validation support to vault login command (#14425)
* add MFA validation support to vault login command

* correctly report new totp code availability period
2022-03-14 15:54:41 -04:00
Alexander Scheel d9c1314552
Fix description of StringSliceVar options (#14439)
These options must be specified multiple times in order to be properly
parsed. However, the present description suggests that a comma-separated
list would work as well, however this isn't the case and results in a
slice containing a single string (with all comma-separated values) in
the API request. Clarify the argument help text to make this clearer.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-03-14 14:11:49 -04:00
VAL 94fcca09e3
Remove unneeded comments (#14423) 2022-03-09 11:37:18 -08:00
VAL 63a2ed296b
Output full secret path in certain kv commands (#14301)
* Full secret path in table output of get and put

* Add path output to KV patch and metadata get

* Add changelog

* Don't print secret path for kv-v1

* Make more readable

* Switch around logic to not swallow error

* Add test for secret path

* Fix metadata test

* Add unit test for padequalsigns

* Remove wonky kv get tests
2022-03-08 13:17:27 -08:00
Rémi Lapeyre e89bbd51d9
Add support for PROXY protocol v2 in TCP listener (#13540)
* Add support for PROXY protocol v2 in TCP listener

I did not find tests for this so I added one trying to cover different
configurations to make sure I did not break something. As far as I know,
the behavior should be exactly the same as before except for one thing
when proxy_protocol_behavior is set to "deny_unauthorized", unauthorized
requests were previously silently reject because of https://github.com/armon/go-proxyproto/blob/7e956b284f0a/protocol.go#L81-L84
but it will now be logged.

Also fixes https://github.com/hashicorp/vault/issues/9462 by adding
support for `PROXY UNKNOWN` for PROXY protocol v1.

Closes https://github.com/hashicorp/vault/issues/3807

* Add changelog
2022-03-08 12:13:00 -05:00