Commit graph

108 commits

Author SHA1 Message Date
Pratyoy Mukhopadhyay 62c09bc2be
oss changes (#15487)
* oss changes

* add changelog
2022-05-18 09:16:13 -07:00
Brian Kassouf b5472aadf3
Add list of granting policies audit logs (#15457)
* Add list of granting policies audit logs

* Add changelog
2022-05-16 16:23:08 -07:00
swayne275 6f15ac255a
add mount accessor to audit request and responses (#15342)
* add mount accessor to audit request and responses

* add changelog

* cleanup todo
2022-05-12 11:28:00 -06:00
Rémi Lapeyre cf8b5642f2
Add remote_port in the audit logs when it is available (#12790)
* Add remote_port in the audit logs when it is available

The `request.remote_port` field is now present in the audit log when it
is available:

```
{
  "time": "2021-10-10T13:53:51.760039Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:1304aab0ac65747684e1b58248cc16715fa8f558f8d27e90fcbcb213220c0edf",
    "accessor": "hmac-sha256:f8cf0601dadd19aac84f205ded44c62898e3746a42108a51105a92ccc39baa43",
    "display_name": "root",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2021-10-10T15:53:44+02:00"
  },
  "request": {
    "id": "829c04a1-0352-2d9d-9bc9-00b928d33df5",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:1304aab0ac65747684e1b58248cc16715fa8f558f8d27e90fcbcb213220c0edf",
    "client_token_accessor": "hmac-sha256:f8cf0601dadd19aac84f205ded44c62898e3746a42108a51105a92ccc39baa43",
    "namespace": {
      "id": "root"
    },
    "path": "sys/audit/file",
    "data": {
      "description": "hmac-sha256:321a1d105f8c6fd62be4f34c4da4f0e6d1cdee9eb2ff4af0b59e1410950fe86b",
      "local": false,
      "options": {
        "file_path": "hmac-sha256:2421b5bf8dab1f9775b2e6e66e58d7bca99ab729f3f311782fda50717eee55b3"
      },
      "type": "hmac-sha256:30dff9607b4087e3ae6808b4a3aa395b1fc064e467748c55c25ddf0e9b150fcc"
    },
    "remote_address": "127.0.0.1",
    "remote_port": 54798
  },
  "response": {
    "mount_type": "system"
  }
}
```

Closes https://github.com/hashicorp/vault/issues/7716

* Add changelog entry

* Empty commit to trigger CI

* Add test and explicit error handling

* Change temporary file pattern in test
2022-01-26 15:47:15 -08:00
Hridoy Roy 1d29b93eb5
fix adding clientID to request in audit log (#12919)
* fix adding clientID to request in audit log

* fix boolean statement

* use standard encoding for client ID instead of urlEncoding

* change encoding in tests

* add in client counts to request handling

* remove redundant client ID generation in request handling

* directly add clientID to req after handling token usage
2021-10-26 09:02:13 -07:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Lars Lehtonen d8f7dd364a
builtin: deprecate errwrap.Wrapf() throughout (#11430)
* audit: deprecate errwrap.Wrapf()

* builtin/audit/file: deprecate errwrap.Wrapf()

* builtin/crediential/app-id: deprecate errwrap.Wrapf()

* builtin/credential/approle: deprecate errwrap.Wrapf()

* builtin/credential/aws: deprecate errwrap.Wrapf()

* builtin/credentials/token: deprecate errwrap.Wrapf()

* builtin/credential/github: deprecate errwrap.Wrapf()

* builtin/credential/cert: deprecate errwrap.Wrapf()

* builtin/logical/transit: deprecate errwrap.Wrapf()

* builtin/logical/totp: deprecate errwrap.Wrapf()

* builtin/logical/ssh: deprecate errwrap.Wrapf()

* builtin/logical/rabbitmq: deprecate errwrap.Wrapf()

* builtin/logical/postgresql: deprecate errwrap.Wrapf()

* builtin/logical/pki: deprecate errwrap.Wrapf()

* builtin/logical/nomad: deprecate errwrap.Wrapf()

* builtin/logical/mssql: deprecate errwrap.Wrapf()

* builtin/logical/database: deprecate errwrap.Wrapf()

* builtin/logical/consul: deprecate errwrap.Wrapf()

* builtin/logical/cassandra: deprecate errwrap.Wrapf()

* builtin/logical/aws: deprecate errwrap.Wrapf()
2021-04-22 11:20:59 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Mark Gritter 8c67bed7ae
Send a test message before committing a new audit device. (#10520)
* Send a test message before committing a new audit device.
Also, lower timeout on connection attempts in socket device.
* added changelog
* go mod vendor (picked up some unrelated changes.)
* Skip audit device check in integration test.
Co-authored-by: swayne275 <swayne@hashicorp.com>
2020-12-16 16:00:32 -06:00
Scott Miller 883524c71c
Add backend type to audit logs (#9167)
Add a mount_type field to audit log requests and responses.
2020-06-16 07:22:33 -05:00
Scott Miller 12d704d97f
Provide token ttl and issue time in the audit log. (#9091)
* Populate a token_ttl and token_issue_time field on the Auth struct of audit log entries, and in the Auth portion of a response for login methods

* Revert go fmt, better zero checking

* Update unit tests

* changelog++
2020-05-29 12:30:47 -05:00
ncabatoff 23c13f24f0
Ensure that http_raw_body is always passed to the audit redaction system as a string
Before this it was passed as a []byte, which doesn't get HMAC'd.  The original non-HMACing behaviour can be obtained by adding "http_raw_body" to audit_non_hmac_response_keys. (#8130)
2020-02-03 11:53:02 -05:00
Jeff Mitchell 7b8c0b58f1
Call goimports as well as gofmt when doing a make fmt (#7148)
Closes #7147
2019-07-18 21:04:56 -04:00
ncabatoff d2beeefe79
Add support for hashing time.Time within slices (#6767)
Add support for hashing time.Time within slices, which unbreaks auditing of requests returning the request counters.  

Break Hash into struct-specific func like HashAuth, HashRequest. Move all the copying/hashing logic from FormatRequest/FormatResponse into the new Hash* funcs.  HashStructure now modifies in place instead of copying.

Instead of returning an error when trying to hash map keys of type time.Time, ignore them, i.e. pass them through unhashed.

Enable auditing on test clusters by default if the caller didn't specify any audit backends.  If they do, they're responsible for setting it up.
2019-07-02 18:18:40 -04:00
Jeff Mitchell 402ba1b0f0
Tokenhelper v2 (#6662)
This provides an sdk util for common token fields and parsing and plumbs it into token store roles.
2019-06-14 10:17:04 -04:00
Jeff Mitchell 78746c7ace Make fmt 2019-05-28 17:25:23 -04:00
Jeff Mitchell ff2e8053e8
Fully omitempty audit (#6727) 2019-05-28 16:24:30 -05:00
ncabatoff ad28263b69
Allow plugins to submit audit requests/responses via extended SystemView (#6777)
Move audit.LogInput to sdk/logical.  Allow the Data values in audited
logical.Request and Response to implement OptMarshaler, in which case
we delegate hashing/serializing responsibility to them.  Add new
ClientCertificateSerialNumber audit request field.

SystemView can now be cast to ExtendedSystemView to expose the Auditor
interface, which allows submitting requests and responses to the audit
broker.
2019-05-22 18:52:53 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Jeff Mitchell 6797e21f54
Migrate from SermoDigital go Square JOSE (#6445) 2019-03-20 14:54:03 -04:00
Jeff Mitchell 1d39e69d73 Add response warnings to audit logs (#6386) 2019-03-08 13:42:14 -08:00
Jeff Mitchell 5f249d4005
Add allowed_response_headers (#6115) 2019-02-05 16:02:15 -05:00
Jeff Mitchell bbc1d53a5d Revert "Refactor common token fields and operations into a helper (#5953)"
This reverts commit 66c226c593bb1cd48cfd8364ac8510cb42b7d67a.
2019-02-01 11:23:40 -05:00
Jeff Mitchell 85a560abba
Refactor common token fields and operations into a helper (#5953) 2019-01-30 16:23:28 -05:00
Jeff Mitchell a64fc7d7cb
Batch tokens (#755) 2018-10-15 12:56:24 -04:00
Jeff Mitchell 919b968c27
The big one (#5346) 2018-09-17 23:03:00 -04:00
Jeff Mitchell 5d44c54947
Changes the way policies are reported in audit logs (#4747)
* This changes the way policies are reported in audit logs.

Previously, only policies tied to tokens would be reported. This could
make it difficult to perform after-the-fact analysis based on both the
initial response entry and further requests. Now, the full set of
applicable policies from both the token and any derived policies from
Identity are reported.

To keep things consistent, token authentications now also return the
full set of policies in api.Secret.Auth responses, so this both makes it
easier for users to understand their actual full set, and it matches
what the audit logs now report.
2018-06-14 09:49:33 -04:00
Chris Hoffman a7ada08b3b
Core handling of TTLs (#4230)
* govet cleanup in token store

* adding general ttl handling to login requests

* consolidating TTL calculation to system view

* deprecate LeaseExtend

* deprecate LeaseExtend

* set the increment to the correct value

* move calculateTTL out of SystemView

* remove unused value

* add back clearing of lease id

* implement core ttl in some backends

* removing increment and issue time from lease options

* adding ttl tests, fixing some compile issue

* adding ttl tests

* fixing some explicit max TTL logic

* fixing up some tests

* removing unneeded test

* off by one errors...

* adding back some logic for bc

* adding period to return on renewal

* tweaking max ttl capping slightly

* use the appropriate precision for ttl calculation

* deprecate proto fields instead of delete

* addressing feedback

* moving TTL handling for backends to core

* mongo is a secret backend not auth

* adding estimated ttl for backends that also manage the expiration time

* set the estimate values before calling the renew request

* moving calculate TTL to framework, revert removal of increment and issue time from logical

* minor edits

* addressing feedback

* address more feedback
2018-04-03 12:20:20 -04:00
Jeff Mitchell 3e9539c1f5
Other audit log times are forced to UTC so make this behave the same way. (#4221) 2018-03-29 17:49:21 -04:00
Brian Kassouf 9dba3590ac
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Calvin Leung Huang e2fb199ce5
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell 2b78bc2a9b
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Calvin Leung Huang 6aa12999aa
Use RFC3339Nano for AuditRequestEntry.Time (#3551) 2017-11-07 18:09:54 -05:00
Jeff Mitchell 8e271dcbdc More syncing 2017-10-23 16:52:56 -04:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Calvin Leung Huang db9d9e6415 Store original request path in WrapInfo (#3100)
* Store original request path in WrapInfo as CreationPath

* Add wrapping_token_creation_path to CLI output

* Add CreationPath to AuditResponseWrapInfo

* Fix tests

* Add and fix tests, update API docs with new sample responses
2017-08-02 18:28:58 -04:00
Brian Rodgers e9b52ed928 Log auth info on permission denied due to ACL (#2754) 2017-06-05 18:04:31 -04:00
Jeff Mitchell 7cc72a9066 Delay salt initialization for audit backends 2017-05-23 20:36:20 -04:00
Jeff Mitchell 92e3758291 Don't hash time.Time values in return data maps, they may be useful for reconciling values and are not generally secret 2017-05-08 14:19:42 -04:00
Mitchell Hashimoto c29ee275ce audit: hash time.Time values in map fields (#2689)
This enables audit.Hash to hash time.Time values that may exist as
direct fields in the map. This will error (instead of panic) for any
time.Time values that don't occur within map values. For example, this
does not support a time.Time within a slice. If that needs to be
supported then modifications will need to be made.

This also requires an update to reflectwalk (included in this PR). This
is a minimal change that allows SkipEntry to signal to skip an entire
struct. We do this because we don't want to walk any of time.Time since
we handle it directly.
2017-05-08 14:06:08 -04:00
Brian Kassouf 657d433330 Update the ResponseWrapData function to return a wrapping.ResponseWrapInfo object 2017-04-24 12:15:01 -07:00
Vishal Nayak 5a6193a56e Audit: Add token's use count to audit response (#2437)
* audit: Added token_num_uses to audit response

* Fixed jsonx tests

* Revert logical auth to NumUses instead of TokenNumUses

* s/TokenNumUses/NumUses

* Audit: Add num uses to audit requests as well

* Added RemainingUses to distinguish NumUses in audit requests
2017-03-08 17:36:50 -05:00
Jeff Mitchell d7a6ec8d43 Add some repcluster handling to audit and add some tests (#2384)
* Add some repcluster handling to audit and add some tests

* Fix incorrect assumption about nil auth
2017-02-16 13:09:53 -05:00
Tommy Murphy ca06bc0b53 audit: support a configurable prefix string to write before each message (#2359)
A static token at the beginning of a log line can help systems parse
logs better. For example, rsyslog and syslog-ng will recognize the
'@cee: ' prefix and will parse the rest of the line as a valid json message.
This is useful in environments where there is a mix of structured and
unstructured logs.
2017-02-10 16:56:28 -08:00
Brian Kassouf 6701ba8a10 Configure the request headers that are output to the audit log (#2321)
* Add /sys/config/audited-headers endpoint for configuring the headers that will be audited

* Remove some debug lines

* Add a persistant layer and refactor a bit

* update the api endpoints to be more restful

* Add comments and clean up a few functions

* Remove unneeded hash structure functionaility

* Fix existing tests

* Add tests

* Add test for Applying the header config

* Add Benchmark for the ApplyConfig method

* ResetTimer on the benchmark:

* Update the headers comment

* Add test for audit broker

* Use hyphens instead of camel case

* Add size paramater to the allocation of the result map

* Fix the tests for the audit broker

* PR feedback

* update the path and permissions on config/* paths

* Add docs file

* Fix TestSystemBackend_RootPaths test
2017-02-02 11:49:20 -08:00
Jeff Mitchell 64fc18e523 When a JWT wrapping token is returned, audit the inner token both for
request and response. This makes it far easier to properly check
validity elsewhere in Vault because we simply replace the request client
token with the inner value.
2017-01-04 23:50:24 -05:00
Jeff Mitchell 3129187dc2 JWT wrapping tokens (#2172) 2017-01-04 16:44:03 -05:00
Vishal Nayak b3c805e662 Audit the client token accessors (#2037) 2016-10-29 17:01:49 -04:00