Commit graph

1098 commits

Author SHA1 Message Date
Tom Proctor fc378c0908
Event system alpha experiment (#18795) 2023-01-23 19:26:49 +00:00
Daniel Huckins fc6d13e29d
VAULT-12112: openapi response definitions: sys/audit (#18456)
* added audit-hash operations

* more audit paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added audit fields

* add changelog file

* dynamic fields should be nil

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* start to add test helper

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add tests for /sys/audit openapi paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <anton.averchenkov@hashicorp.com>
2023-01-20 11:09:33 -05:00
Josh Black fa1447cb3c
Add new clients into the monthly breakdown (#18766)
* Add new clients into the monthly breakdown

* add changelog
2023-01-19 09:12:17 -08:00
Jordan Reimer 2e44d2020a
Kubernetes Secrets Engine (#17893)
* Ember Engine for Kubernetes Secrets Engine (#17881)

* adds in-repo ember engine for kubernetes secrets engine

* updates kubernetes engine class name

* Kubernetes route plumbing (#17895)

* kubernetes route plumbing

* adds kubernetes role index route with redirect to details

* adds kubernetes as mountable and supported secrets engine (#17891)

* adds models, adapters and serializers for kubernetes secrets engine (#18010)

* adds mirage factories and handlers for kubernetes (#17943)

* Kubernetes Secrets Engine Configuration (#18093)

* moves RadioCard component to core addon

* adds kubernetes configuration view

* fixes tests using RadioCard after label for and input id changes

* adds confirm modal when editing kubernetes config

* addresses review comments

* Kubernetes Configuration View (#18147)

* removes configuration edit and index routes

* adds kubernetes configuration view

* Kubernetes Roles List (#18211)

* removes configuration edit and index routes

* adds kubernetes configuration view

* adds kubernetes secrets engine roles list view

* updates role details disabled state to explicitly check for false

* VAULT-9863 Kubernetes Overview Page (#18232)

* Add overview page view

* Add overview page tests

* Address feedback to update tests and minor changes

* Use template built in helper for conditionally showing num roles

* Set up roleOptions in constructor

* Set up models in tests and fix minor bug

* Kubernetes Secrets Engine Create/Edit Views (#18271)

* moves kv-object-editor to core addon

* moves json-editor to core addon

* adds kubernetes secrets engine create/edit views

* updates kubernetes/role adapter test

* addresses feedback

* fixes issue with overview route showing 404 page (#18303)

* Kubernetes Role Details View (#18294)

* moves format-duration helper to core addon

* adds kubernetes secrets engine role details view

* adds tests for role details page component

* adds capabilities checks for toolbar actions

* fixes list link for secrets in an ember engine (#18313)

* Manual Testing: Bug Fixes and Improvements (#18333)

* updates overview, configuration and roles components to pass args for individual model properties

* bug fixes and improvements

* adds top level index route to redirect to overview

* VAULT-9877 Kubernetes Credential Generate/View Pages (#18270)

* Add credentials route with create and view components

* Update mirage response for creds and add ajax post call for creds in adapter

* Move credentials create and view into one component

* Add test classes

* Remove files and update backend property name

* Code cleanup and add tests

* Put test helper in helper function

* Add one more test!

* Add code optimizations

* Fix model in route and add form

* Add onSubmit to form and preventDefault

* Fix tests

* Update mock data for test to be strong rather than record

* adds acceptance tests for kubernetes secrets engine roles (#18360)

* VAULT-11862 Kubernetes acceptance tests (#18431)

* VAULT-12185 overview acceptance tests

* VAULT-12298 credentials acceptance tests

* VAULT-12186 configuration acceptance tests

* VAULT-12127 Refactor breadcrumbs to use breadcrumb component (#18489)

* VAULT-12127 Refactor breadcrumbs to use Page::Breadcrumbs component

* Fix failing tests by adding breadcrumbs properties

* VAULT-12166 add jsdocs to kubernetes secrets engine pages (#18509)

* fixes incorrect merge conflict resolution

* updates kubernetes check env vars endpoint (#18588)

* hides kubernetes ca cert field if not defined in configuration view

* fixes loading substate handling issue (#18592)

* adds changelog entry

Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com>
2023-01-18 15:02:41 -06:00
Max Coulombe 553e1cfb0d
* added the new redis parameter documentation (#18752)
* added the new redis parameter documentation
* added changelog
2023-01-18 15:51:15 -05:00
Max Bowsher 4c5f583f39
OpenAPI generic_mount_paths follow-up (#18663)
* OpenAPI `generic_mount_paths` follow-up

An incremental improvement within larger context discussed in #18560.

* Following the revert in #18617, re-introduce the change from
  `{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
  otherwise paths from multiple plugins would clash - e.g. almost every
  auth method would provide a conflicting definition for
  `auth/{mountPath}/login`, and the last one written into the map would
  win.

* Move the half of the functionality that was in `sdk/framework/` to
  `vault/logical_system.go` with the rest; this is needed, as
  `sdk/framework/` gets compiled in to externally built plugins, and
  therefore there may be version skew between it and the Vault main
  code. Implementing the `generic_mount_paths` feature entirely on one
  side of this boundary frees us from problems caused by this.

* Update the special exception that recognizes `system` and `identity`
  as singleton mounts to also include the other two singleton mounts,
  `cubbyhole` and `auth/token`.

* Include a comment that documents to restricted circumstances in which
  the `generic_mount_paths` option makes sense to use:

	    // Note that for this to actually be useful, you have to be using it with
	    // a Vault instance in which you have mounted one of each secrets engine
	    // and auth method of types you are interested in, at paths which identify
	    // their type, and for the KV secrets engine you will probably want to
	    // mount separate kv-v1 and kv-v2 mounts to include the documentation for
	    // each of those APIs.

* Fix tests

Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.

* Add changelog

* Update 18663.txt
2023-01-17 23:07:11 -05:00
Jordan Reimer f58074a429
API Explorer Query Params (#18743)
* adds query params to api explorer test requests

* adds changelog entry
2023-01-17 16:37:07 -07:00
akshya96 6e04e4ede1
Prevent brute forcing : telemetry oss changes (#18718)
* Prevent brute forcing : telemetry oss changes

* adding changelog
2023-01-17 15:10:50 -08:00
akshya96 b2276a369a
Prevent Brute Forcing: Create an api endpoint to list locked users OSS changes (#18675)
* api to list lockedusers oss changes

* add changelog
2023-01-17 14:25:56 -08:00
Anton Averchenkov a4973bc45a
Remove timeout logic from ReadRaw functions and add ReadRawWithContext (#18708)
Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:

- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`

The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):

> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.

Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.

Resolves: #18658
2023-01-17 15:41:59 -05:00
Nick Cabatoff 07d1e26ff3
Fix changelog for #18401 (#18727) 2023-01-16 13:49:28 -05:00
Nick Cabatoff b5f19fffe9
Speculative fix for a panic that might arise during raft teardown (#18704) 2023-01-16 13:49:11 -05:00
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Peter Wilson 59450ecb82
Revert "Add new clients into the monthly breakdown (#18629)" (#18726)
This reverts commit d641bbc28e5e8cc12b81d409e5d5fc1f2cb7f66c.
2023-01-16 15:51:19 +00:00
Ben Ash 3ff530e001
auth/kubernetes: upgrade to v0.14.1 (#18716) 2023-01-13 19:00:18 -05:00
Ben Ash 02018f1d1d
Revert "auth/kubernetes: upgrade to v0.14.1 (#18711)" (#18715)
This reverts commit ed244a9263255affa797fe032a5b103d7ae41891.
2023-01-13 18:17:12 -05:00
Ben Ash 6bcd9f4458
auth/kubernetes: upgrade to v0.14.1 (#18711) 2023-01-13 17:15:35 -05:00
Anton Averchenkov 6ae09f3074
Add AppRole response schema validation tests (#18636)
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role  successful responses align with the declared response schema.

It also introduces a test helper in `sdk/helper/testhelpers`:

```go
func FindResponseSchema(t *testing.T, ...)
```

This test helper will be useful for all plugins that require similar response schema validation in tests.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 15:23:36 -05:00
Anton Averchenkov 9696600e59
Add response schema validation methods & test helpers (#18635)
This pull request adds 3 functions (and corresponding tests):

`testhelpers/response_validation.go`:

  - `ValidateResponse`
  - `ValidateResponseData`
  
field_data.go:

  - `ValidateStrict` (has the "strict" validation logic)

The functions are primarily meant to be used in tests to ensure that the responses are consistent with the defined response schema. An example of how the functions can be used in tests can be found in #18636.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 14:55:56 -05:00
Violet Hynes e8aa9c6429
VAULT-12542 Add info encouraging users to upgrade if agent version is different to server (#18684)
* VAULT-12542 Add info encouraging users to upgrade if agent version is different to server

* VAULT-12542 Changelog

* VAULT-12542 Language update
2023-01-13 09:49:36 -05:00
akshya96 78546af8fc
Vault 8308 Background thread to update locked user entries (#18673)
* background thread changes

* adding changelog

* fix changelog typo
2023-01-12 14:09:33 -08:00
claire bontempo 0f0b48eda4
ui: unload auth method when navigating away from form (#18651)
* unload record instead of rollback

* unload record instead of rollback

* add changelog

* add rollback attrs back if record is not new
2023-01-12 13:03:22 -08:00
Max Bowsher d1f2b101b5
Add option 'elide_list_responses' to audit backends (#18128)
This PR relates to a feature request logged through HashiCorp commercial
support.

Vault lacks pagination in its APIs. As a result, certain list operations
can return **very** large responses.  The user's chosen audit sinks may
experience difficulty consuming audit records that swell to tens of
megabytes of JSON.

In our case, one of the systems consuming audit log data could not cope,
and failed.

The responses of list operations are typically not very interesting, as
they are mostly lists of keys, or, even when they include a "key_info"
field, are not returning confidential information. They become even less
interesting once HMAC-ed by the audit system.

Some example Vault "list" operations that are prone to becoming very
large in an active Vault installation are:

    auth/token/accessors/
    identity/entity/id/
    identity/entity-alias/id/
    pki/certs/

In response, I've coded a new option that can be applied to audit
backends, `elide_list_responses`. When enabled, response data is elided
from audit logs, only when the operation type is "list".

For added safety, the elision only applies to the "keys" and "key_info"
fields within the response data - these are conventionally the only
fields present in a list response - see logical.ListResponse, and
logical.ListResponseWithInfo. However, other fields are technically
possible if a plugin author writes unusual code, and these will be
preserved in the audit log even with this option enabled.

The elision replaces the values of the "keys" and "key_info" fields with
an integer count of the number of entries. This allows even the elided
audit logs to still be useful for answering questions like "Was any data
returned?" or "How many records were listed?".
2023-01-11 16:15:52 -05:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Alexander Scheel 44c3b736bf
Allow tidy to backup legacy CA bundles (#18645)
* Allow tidy to backup legacy CA bundles

With the new tidy_move_legacy_ca_bundle option, we'll use tidy to move
the legacy CA bundle from /config/ca_bundle to /config/ca_bundle.bak.
This does two things:

 1. Removes ca_bundle from the hot-path of initialization after initial
    migration has completed. Because this entry is seal wrapped, this
    may result in performance improvements.
 2. Allows recovery of this value in the event of some other failure
    with migration.

Notably, this cannot occur during migration in the unlikely (and largely
unsupported) case that the operator immediately downgrades to Vault
<1.11.x. Thus, we reuse issuer_safety_buffer; while potentially long,
tidy can always be run manually with a shorter buffer (and only this
flag) to manually move the bundle if necessary.

In the event of needing to recover or undo this operation, it is
sufficient to use sys/raw to read the backed up value and subsequently
write it to its old path (/config/ca_bundle).

The new entry remains seal wrapped, but otherwise isn't used within the
code and so has better performance characteristics.

Performing a fat deletion (DELETE /root) will again remove the backup
like the old legacy bundle, preserving its wipe characteristics.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation about new tidy parameter

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for migration scenarios

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clean up time comparisons

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-11 12:12:53 -05:00
John-Michael Faircloth 847d40c4b3
db plugin: support multiline revoke stmt in postgres (#18632)
* db plugin: support multiline revoke stmt in postgres

* add changelong
2023-01-10 15:27:00 -06:00
Max Bowsher 6d6a726f9d
Fix HelpOperation on sudo-protected paths (#18568)
* Fix HelpOperation on sudo-protected paths

Fixes #18566

* Add changelog
2023-01-10 12:17:16 -06:00
Peter Wilson e4685c10ef
VAULT-9883: Agent Reloadable Config (#18638)
* Update command/agent.go
* Attempt to only reload log level and certs
* Mimicked 'server' test for cert reload in 'agent'

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Left out the `c.config` tweak that meant changes to lots of lines of code within the `Run` function of Agent command. :)
2023-01-10 17:45:34 +00:00
Alexander Scheel a18187c643
Correctly distinguish empty issuer names in PKI (#18466)
* Correctly distinguish empty issuer names

When using client.Logical().JSONMergePatch(...) with an empty issuer
name, patch incorrectly reports:

> issuer name contained invalid characters

In this case, both the error in getIssuerName(...) is incorrect and
patch should allow setting an empty issuer name explicitly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-10 10:04:30 -05:00
Josh Black d3f822a938
Add new clients into the monthly breakdown (#18629)
* Add new clients into the monthly breakdown

* add changelog
2023-01-09 15:26:11 -08:00
Mike Palmiotto 43a78c85f4
Mark deprecated builtins Removed (#18039)
* Remove logical database builtins

* Drop removed builtins from registry keys

* Update plugin prediction test

* Remove app-id builtin

* Add changelog
2023-01-09 09:16:35 -05:00
Chris Capurso 25d0afae23
VAULT-11830: Expand NodeStatusReporter with new fields (#18302)
* expand NodeStatusReporter with new fields

* only call IsRaftVoter if using raft storage

* add changelog entry

* fix listeners

* return LogLevel as enum

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* bump github.com/hashicorp/vault/vault/hcp_link/proto

* go mod tidy
2023-01-06 20:53:09 -05:00
Chris Capurso bb0c92afe7
VAULT-11829: Add cluster status handler (#18351)
* go get link proto @vault-11829-meta-get-cluster-status

* add HA status

* add HAEnabled method

* add raft config

* allocate HA nodes based on actual count

* add raft autopilot status

* add raft quorum warnings

* add ClusterID method

* add StorageType

* add ClusterID

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* fix raft config panic

* remove "Warning" quorum message prefix

* add error wrapping

* add Core.HAStateWithLock method

* reduce quorum warnings to single string

* fix HCP_API_HOST test env var check

* Revert "fix HCP_API_HOST test env var check"

This reverts commit 97c73c4798b77b84aea84f341f2c63c4d657914d.
2023-01-06 17:06:54 -05:00
Max Bowsher 5f8da0f6aa
Fix error in changelog template (#18572)
Too many newlines are stripped, which is responsible for the `FEATURES:`
heading in the current in-progress 1.13.0 changelog entry being
erroneously appended to the end of the last bullet point of the previous
`CHANGES:` section.
2023-01-06 14:32:22 +00:00
Josh Black c8a8c21cee
Account for mount counts when de-duplicating current and historical month data (#18598)
* Account for mount counts when de-duplicating current and historical month data

* add changelog
2023-01-05 09:34:05 -08:00
vinay-gopalan bbd8ac9bbf
Upgrade go.opentelemetry.io/otel from v0.20.0 to v1.11.2 (#18589) 2023-01-04 11:31:30 -08:00
Chris Capurso 0635d304de
only update SCADA metadata if status changes (#18585)
* only update SCADA metadata if status changes

* add changelog entry
2023-01-04 11:09:51 -05:00
Steven Clark cfd5b8a933
Resolve unrecognized parameter warnings on batch_input parameter in transit (#18299)
* Resolve unused warnings on batch_input parameter in transit

* Add cl

* Fix text in hmac batch_input parameter description
2023-01-04 09:15:48 -05:00
Theron Voran 49e97a09a6
secrets/kubernetes: updating to latest plugin (#18587)
go get github.com/hashicorp/vault-plugin-secrets-kubernetes@main
go mod tidy
2023-01-03 15:32:30 -08:00
Violet Hynes 0b15ad18a2
VAULT-12095 Support multiple config files for Vault Agent (#18403)
* VAULT-12095 Code changes for multi-config

* VAULT-12095 typo

* VAULT-12095 make vault non-nil during update

* VAULT-12095 docs

* VAULT-12095 small refactor

* VAULT-12095 typos
2023-01-03 12:50:19 -05:00
Milena Zlaticanin d9b8fb6877
MongoDB - Fix write_concern param (#18546)
* fix writeconcern defaulting to majority

* add changelog

* restart CI tests

* fix tests

* add package
2022-12-23 17:14:41 -06:00
akshya96 4126060d88
Prevent Brute Forcing: Create api endpoint to unlock users (#18279)
* code changes for unlock

* add test

* adding sys help

* adding sys help

* updating unlock user function

* edit test

* add changelog

* syshelp

* adding open api response definition

* removing response fields

* change path name
2022-12-19 14:24:42 -08:00
Alexander Scheel 3ccbddab0e
Add issuer reference info on JSON endpoint (#18482)
* Add issuer reference info on JSON endpoint

This endpoint is unauthenticated and shouldn't contain sensitive
information. However, listing the issuers (LIST /issuers) already
returns both the issuer ID and the issuer name (if any) so this
information is safe to return here.

When fetching /pki/issuer/default/json, it would be nice to know exactly
which issuer ID and name it corresponds to, without having to fetch the
authenticated endpoint as well.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-19 21:39:01 +00:00
Josh Black cd7d6d5761
De-duplicate namespaces when historical and current month data are mixed (#18452)
* De-duplicate namespaces when historical and current month data are mixed

* add changelog
2022-12-16 16:02:42 -08:00
davidadeleon 51b1b6d446
Approle: Fix CIDR validation for /32 masks on Token Bound CIDRs (#18145)
* Fix CIDR validation for /32 masks

* run go fmt

* add changelog
2022-12-16 12:09:05 -05:00
Nick Cabatoff 429916c135
Prevent panics in expiration invalidation, and make some changes for testing (#18401) 2022-12-15 18:09:36 +00:00
Mike Palmiotto 55e9555ec4
Bump go version to 1.19.4 (#18393) 2022-12-15 10:55:58 -05:00
claire bontempo d91e69d183
UI: update host to new doc link location (developer.hashicorp.com/) (#18374)
* change host for doc link

* add todo to LearnLink

* add changelog
2022-12-14 23:25:16 +00:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Chelsea Shaw 6d80ecdbdf
UI: Update database to connection name on role (#18350)
* Update database to connection name on role

* Add changelog
2022-12-14 10:23:59 -06:00