Commit graph

124 commits

Author SHA1 Message Date
Michael Golowka 1888323243
DBPW - Copy newdbplugin package to dbplugin/v5 (#10151)
This is part 1 of 4 for renaming the `newdbplugin` package. This copies the existing package to the new location but keeps the current one in place so we can migrate the existing references over more easily.
2020-10-15 13:20:12 -06:00
Jason O'Donnell 6cf7da3153
docker-dev: add go version as build arg (#10117) 2020-10-08 14:30:31 -04:00
Sam Salisbury 8f8c4b32c5
Packagespec0.1.5/master (#10090)
* packagespec upgrade -version 0.1.5

* remove meta.POST_PROCESSORS - no longer needed

* make ci-config: remove old prereq
2020-10-07 09:35:26 +01:00
Sam Salisbury 98fe5029b0
Packagespec v0.1.2/master (#9995)
* packagespec v0.1.2 - prep: remove unneeded code

* packagespec v0.1.2 - packagespec init

* packagespec v0.1.2 - make packages

* packagespec v0.1.3 - make packages
2020-09-28 13:53:39 +01:00
Mark Gritter 1b2c20e07c
Merge activity log work to date on enterprise back into oss. (#9900)
* Added stub class for activity logging. (#1435)
* Define activity fragments and starter methods for manipulating them. (#1441)
2020-09-08 14:22:09 -05:00
Michael Golowka acda64aa35
Add Database v5 interface with gRPC client & server (#9641)
* Add new Database v5 interface with gRPC client & server
This is primarily for making password policies available to the DB engine, however since there are a number of other problems with the current interface this is getting an overhaul to a more gRPC request/response approach for easier future compatibility.

This is the first in a series of PRs to add support for password policies in the combined database engine
2020-08-28 11:20:49 -06:00
Sam Salisbury c7d5b13b70 Makefile: delete old stage/publish targets 2020-08-12 13:27:40 +01:00
Sam Salisbury 25137a1702
ci/packagespec (#9653)
* add packagespec build system

- The majority of changes in this commit are files generated
  by packagespec (everything in the packages-oss.lock directory).

* add .yamllint

* update to packagespec@fd54aea4

* ci: bust packagespec cache

- Change to packagespec results in package IDs that can use
  git tag refs, not just commit refs.

* update to packagepsec@5fc121d0

- This busts all caches, because of a change to the way we
  no longer traverse from tag refs to commit refs, due to
  the potential confusion this can cause.
- See fd54aea482
  for the original change to packagespec necessitating this.

* update to packagespec@5e6c87b6

- This completes the change to allowing git tag refs
  to be used for source IDs, begun in f130b940a8fbe3e9398225b08ea1d63420bef7b6

* update to packagespec@4d3c9e8b

- This busts cache, needed to apply previous change.

* remove RELEASE_BUILD_WORKFLOW_NAME

* update packagespec, add watch-ci target

* fix package names (do not refer to EDITION)

* remove EDITION input from packages-oss.yml

* bump package cache, update packagespec

* update packagespec, add 'aliases' target

* update packagespec; less output noise

* ci: give release bundle file a better name

- When performing a release build, this will include the build ID
  as part of the name, making it easier to distinguish from other
  builds.

* ci: create aliases tarball artifact

* ci: cache package metadata files

* ci: add metadata artifact

* ci: bust circleci package cache

* Revert "ci: bust circleci package cache"

This reverts commit 1320d182613466f0999d63f5742db12ac17f8e92.

* ci: remove aliases artifact

* ci: use buildID not workflowName to id artifacts

* packages: add BUNDLE_NAME metadata

* do not cache package metadata with binaries

* ci: bump package cache

* ci: debugging

* ci: fix package cache; update packagespec

* ci: update packagespec to 10e7beb2

* ci: write package metadata and aliases

* ci: switch to .zip artifacts

* switch package bundle back to tar.gz (from zip)

- Because of the way zip works, the zip archive was over 2GB rather than under 750MB as with tar.gz.

* bump packagespec, adds list-staged-builds

* update packagespec

* add publish stub + general tidy up

* bump packagespec

* bump packagespec; add make publish-config

* Makefile: tidy up packagespec targets

* pass PRODUCT_REPO_ROOT to packagespec

* bump go to 1.14.6

* packages-oss.yml: use more explicit base image

* bump packagespec to b899b7c1

* bump packagespec to f040ce8f

* packages-oss.yml: pin base image to digest

- This digest is pointed to by debian:buster-20200720
- Using a specific digest ensures that builds use the same
  base image in all contexts

* add release-repo; bump packagespec

* remove BUILD_TAGS and -tags flag

* bump packagespec to e444f742

* bump to go1.14.7

* ci: bump to go1.14.7
2020-08-11 10:00:59 +01:00
Sam Salisbury ae3ab5ff28
bump to go1.14.7 (#9698) 2020-08-10 12:57:21 +01:00
Jason O'Donnell bbec78261b
Add docker dev build targets (#9426)
* Add docker dev build target

* Update scripts/docker/Dockerfile

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

* Add multi-stage build

* Add comment about multi-stage

* Add docker-dev-ui

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-07-20 14:11:34 -04:00
Jeff Mitchell 2f5f0049db
Switch bootstrap (except CI) over to using pinned versions from go.mod (#9000) 2020-05-14 13:45:12 -04:00
Brian Kassouf 826fb140e5 Fix Makefile 2020-04-30 13:13:46 +01:00
Sam Salisbury 5d1b7e38e6 Makefile: add goimports to CI tools
- This is needed for building the final release.
2020-04-29 21:35:35 +01:00
Austin Gebauer 98bea4517f
fix: removes broken golangci-lint external tool from bootstrap (#8846) 2020-04-24 17:52:23 -07:00
ncabatoff 35acc3f081
Use a more minimal bootstrap target when running in CI: just what we need to satisfy our job needs. Also remove govendor which we no longer use. (#8808) 2020-04-22 15:05:49 -04:00
Jeff Mitchell aeac23d59e Revert "Move encrypted blob info (#8357)"
This reverts commit 5a3c55b6dfe89d0348d78acafcb83b60cf614d38.
2020-02-14 17:02:33 -05:00
Jeff Mitchell e12351f263
Move encrypted blob info (#8357) 2020-02-14 16:45:37 -05:00
Sam Salisbury 7996b8f65a
upgrade to go1.13.7 (#8332) 2020-02-11 15:03:49 +00:00
Jeff Mitchell a0694943cc
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
Jim Kalafut 2c6be02579
Allow easier swapping of Go version (#8102) 2020-01-09 09:35:32 -08:00
Christian Frichot 803b29b2d9 UI update headless chrome flags to fix 'yarn run test:oss' (#8035)
* UI update headless chrome flags to fix ember/chrome crash in test

* UI update references from test-oss to test:oss

* UI removed disable-gpu flag from headless Chrome

* UI and also remove the comment
2019-12-18 10:12:44 -07:00
Clint 899277d216 revert a change introduced 13dbb3aac (#7979) 2019-12-07 18:01:11 -08:00
Steve Wills 7181da1ec1 Fix UI build in fresh repo (#7865)
Fixes #7863
2019-12-06 12:58:40 -06:00
Jeff Mitchell 708f164327 Update circle config 2019-10-17 15:48:45 -04:00
Matthew Irish 4a1013e915
Update ui dependencies (#7244)
* be more specific about node version, and specify a yarn version

* update ember, ember-cli, ember-data, ember-data-model-fragments

* use router handlers to access transition information

* fix shadowing of component helper

* update ivy-codemirror, ember-cli-inject-live-reload

* remove custom router service

* don't use transition.queryParams

* update ember-cli-deprecation-workflow

* refactor kv v1 to use 'path' instead of 'id' on creation

* fix auth-jwt-test and toolbar-link-test

* update ember composable helpers

* remove Ember.copy from test file

* no more deprecations in the workflow

* fix more secret tests

* fix remaining failed tests

* move select component to core because it's used by ttl-picker

* generate new model class for each test instead of reusing an existing one

* fix selectors on kmip tests

* refactor how control groups construct urls from the new transition objects

* add router service override back in, and have it be evented so that we can trigger router events on it

* move stories and markdown files to core if the component lives in core

* update ember-cli, ember-cli-babel, ember-auto-import

* update base64js, date-fns, deepmerge, codemirror, broccoli-asset-rev

* update linting rules

* fix test selectors

* update ember-api-actions, ember-concurrency, ember-load-initializers, escape-string-regexp, normalize.css, prettier-eslint-cli, jsdoc-to-markdown

* remove test-results dir

* update base64js, ember-cli-clipboard, ember-cli-sass, ember-cli-string-helpers, ember-cli-template-lint, ember-cli-uglify, ember-link-action

* fix linting

* run yarn install without restoring from cache

* refactor how tests are run and handle the vault server subprocess

* update makefile for new test task names

* update circle config to use the new yarn task

* fix writing the seal keys when starting the dev server

* remove optional deps from the lockfile

* don't ignore-optional on yarn install

* remove errant console.log

* update ember-basic-dropdown-hover, jsonlint, yargs-parser

* update ember-cli-flash

* add back optionalDeps

* update @babel/core@7.5.5, ember-basic-dropdown@1.1.3, eslint-plugin-ember@6.8.2

* update storybook to the latest release

* add a babel config with targets so that the ember babel plugin works properly

* update ember-resolver, move ember-cli-storybook to devDependencies

* revert normalize.css upgrade

* silence fetchadapter warning for now

* exclude 3rd party array helper now that ember includes one

* fix switch and entity lookup styling

* only add -root suffix if it's not in versions mode

* make sure drop always has an array on the aws role form

* fix labels like we did with the backport

* update eslintignore

* update the yarn version in the docker build file

* update eslint ignore
2019-08-19 15:45:39 -05:00
Jeff Mitchell 73ae1829d8 Add goimports to makefile external tools 2019-07-25 14:02:16 -04:00
Sam Salisbury e211a081ce
ci: remove travis config + all refs (#7122) 2019-07-25 11:10:31 +01:00
Jeff Mitchell 6c46813981 Make 1.12.7 required since we know of some important bugs fixed in it 2019-07-24 20:05:14 -04:00
Jeff Mitchell 19f669a52f Sync some changes that weren't properly pulled over 2019-07-22 13:11:00 -04:00
Jeff Mitchell ff706dc795 Remove extraneous call to gofmt in make fmt 2019-07-18 21:16:25 -04:00
Jeff Mitchell 7b8c0b58f1
Call goimports as well as gofmt when doing a make fmt (#7148)
Closes #7147
2019-07-18 21:04:56 -04:00
Sam Salisbury 6f79b6dfc6
ci pre-flight checks (#6960)
* ci: update Makefile

- Copied from c5c5149583/copythis.circleci/Makefile

* Makefile: make ci- targets phony and more explicit

* ci: add pre-flight checks

- Right now just validates ci config is up to date.

* ci: add --skip-update-check to circleci

* ci: remove cruft from Makefile

* ci: fix missing source dir error message

* ci: consolidate circleci path handling
2019-06-26 15:14:23 +01:00
Brian Kassouf ed14061578
Raft Storage Backend (#6888)
* Work on raft backend

* Add logstore locally

* Add encryptor and unsealable interfaces

* Add clustering support to raft

* Remove client and handler

* Bootstrap raft on init

* Cleanup raft logic a bit

* More raft work

* Work on TLS config

* More work on bootstrapping

* Fix build

* More work on bootstrapping

* More bootstrapping work

* fix build

* Remove consul dep

* Fix build

* merged oss/master into raft-storage

* Work on bootstrapping

* Get bootstrapping to work

* Clean up FMS and node-id

* Update local node ID logic

* Cleanup node-id change

* Work on snapshotting

* Raft: Add remove peer API (#906)

* Add remove peer API

* Add some comments

* Fix existing snapshotting (#909)

* Raft get peers API (#912)

* Read raft configuration

* address review feedback

* Use the Leadership Transfer API to step-down the active node (#918)

* Raft join and unseal using Shamir keys (#917)

* Raft join using shamir

* Store AEAD instead of master key

* Split the raft join process to answer the challenge after a successful unseal

* get the follower to standby state

* Make unseal work

* minor changes

* Some input checks

* reuse the shamir seal access instead of new default seal access

* refactor joinRaftSendAnswer function

* Synchronously send answer in auto-unseal case

* Address review feedback

* Raft snapshots (#910)

* Fix existing snapshotting

* implement the noop snapshotting

* Add comments and switch log libraries

* add some snapshot tests

* add snapshot test file

* add TODO

* More work on raft snapshotting

* progress on the ConfigStore strategy

* Don't use two buckets

* Update the snapshot store logic to hide the file logic

* Add more backend tests

* Cleanup code a bit

* [WIP] Raft recovery (#938)

* Add recovery functionality

* remove fmt.Printfs

* Fix a few fsm bugs

* Add max size value for raft backend (#942)

* Add max size value for raft backend

* Include physical.ErrValueTooLarge in the message

* Raft snapshot Take/Restore API  (#926)

* Inital work on raft snapshot APIs

* Always redirect snapshot install/download requests

* More work on the snapshot APIs

* Cleanup code a bit

* On restore handle special cases

* Use the seal to encrypt the sha sum file

* Add sealer mechanism and fix some bugs

* Call restore while state lock is held

* Send restore cb trigger through raft log

* Make error messages nicer

* Add test helpers

* Add snapshot test

* Add shamir unseal test

* Add more raft snapshot API tests

* Fix locking

* Change working to initalize

* Add underlying raw object to test cluster core

* Move leaderUUID to core

* Add raft TLS rotation logic (#950)

* Add TLS rotation logic

* Cleanup logic a bit

* Add/Remove from follower state on add/remove peer

* add comments

* Update more comments

* Update request_forwarding_service.proto

* Make sure we populate all nodes in the followerstate obj

* Update times

* Apply review feedback

* Add more raft config setting (#947)

* Add performance config setting

* Add more config options and fix tests

* Test Raft Recovery (#944)

* Test raft recovery

* Leave out a node during recovery

* remove unused struct

* Update physical/raft/snapshot_test.go

* Update physical/raft/snapshot_test.go

* fix vendoring

* Switch to new raft interface

* Remove unused files

* Switch a gogo -> proto instance

* Remove unneeded vault dep in go.sum

* Update helper/testhelpers/testhelpers.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* Update vault/cluster/cluster.go

* track active key within the keyring itself (#6915)

* track active key within the keyring itself

* lookup and store using the active key ID

* update docstring

* minor refactor

* Small text fixes (#6912)

* Update physical/raft/raft.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* review feedback

* Move raft logical system into separate file

* Update help text a bit

* Enforce cluster addr is set and use it for raft bootstrapping

* Fix tests

* fix http test panic

* Pull in latest raft-snapshot library

* Add comment
2019-06-20 12:14:58 -07:00
Sam Salisbury e2e5e16ff2 ci: break config into separate files (#6849)
* ci: break config into separate files

* Untangle githooks

* githooks: fix whitespace

* .hooks/pre-commit: add ui -> lint-staged check

- We no longer require dependency on husky with this change.

* ui: remove husky dependency and config

- The previous commit obviates the need for it.
- We will now have to manage these hooks by hand, but this removes
  the conflict between husky-installed hooks and those in the .hooks dir.

* ui: update yarn.lock with husky removed

* .hooks/pre-commit: always use subshell + docs

- Always use subshell means we consistently exit from the
  same place which feels less complex.
- Docs are necessary for horrible bash like this I think...

* Makefile: remove old husky githooks

- Husky has installed a handler for every single git hook.
- This causes warnings on every git operation.
- Eventually we can remove this, but better not to confuse
  people with these messages for now.

* ci: fix go build tags

* Makefile: improve compatibility of rm call

- Looks like the xargs in Travis does something different to the one
  on my mac, this more verbose call should be safe everywhere.

* ci: fix make target names

* ci: fix test-ui invocation

* Makefile: simplify husky hook cleanup

* ci: more focussed readme
2019-06-11 09:55:53 -05:00
Sam Salisbury 78c6668d80 ci: make ember-ci-test (browserstack) (#6751)
* ci: make ember-ci-test (browserstack) VLTES-28

* Update .circleci/config.yml

Co-Authored-By: Josh Freda <jfreda@users.noreply.github.com>

* Update .circleci/config.yml

Co-Authored-By: Josh Freda <jfreda@users.noreply.github.com>

* ci: rename test-ember -> test-ui-browserstack

* Makefile: ember-ci-test -> test-ui-browserstack

- Use the same name for this test everywhere to reduce cognitive load.

* browserstack: exit non-zero on failure to connect

* .travis.yml: ember-ci-test -> test-ui-browserstack

* browserstack: add vault bin to the path

* Makefile: browserstack: fail early w/clear msgs

This might save someone time later. The same checks could be applied
elsewhere too trivially.
2019-05-24 08:02:51 -04:00
Antonio M. Amaya Calvo c1363c716a Build: Respect the CGO_ENABLED environment variable (#6766) 2019-05-22 07:17:55 -05:00
Noelle Daley c7e5283b8b
Add Browserstack for IE11 testing (#6557)
* add browserstack

* check for data before removing root token

* fix root prefix and select by attributes for ie11

* use objectAt for ie11

* use blobs instead of files for ie11

* manually round cirucmference for ie11

* skip csp test on ie11

* skip tests in ie11

* include polyfill for CI

* remove on exit hooks

* update which browserstack tests are run

* remove ie check since we are not running these tests in ie

* remove ie check since we are not running these tests in ie
2019-05-03 15:20:14 -07:00
Jeff Mitchell 213b9fd1cf Update to api 1.0.1 and sdk 0.1.8 2019-04-15 14:10:07 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Jeff Mitchell b67c8a1198 Bump normal test timeout 2019-04-03 14:31:56 -04:00
nathan r. hruby b8cd0a6cf9 fix vet usage for newer go 2019-03-26 13:35:37 -07:00
Jim Kalafut 009fa70cab
Add UI asset file checks to Makefile (#6428) 2019-03-25 12:07:14 -05:00
Jeff Mitchell 0a78c00c9f Sync over 2019-03-25 12:43:16 -04:00
Jeff Mitchell 3bb381720f Allow devel in go version check and bump to 1.11 2019-01-29 11:27:04 -05:00
gitirabassi 1aaacda3ec small fixes to docs and indexes 2019-01-18 02:14:57 +01:00
Zac Medico 49eaa3d4ff Makefile: do not execute static-assets in parallel with deps (#6057)
The static-assets target has a dependency on *either* ember-dist or
ember-dist-dev, so these targets must not execute in parallel. Since
this is an either/or dependency, it cannot be expressed as a regular
dependency unless the targets are refactored somehow.

Fixes: 7a312d7c37bb ("Add Makefile/Dockerfile UI bits")
2019-01-17 18:19:54 -05:00
Matthew Irish 16a19ff9d1
UI husky (#5752)
* add husky and lint-staged as optional deps

* make sure make-driven builds don't install husky, and update the ui/README
2018-11-14 12:57:03 -06:00
Jim Kalafut 11d7f7eb13
Add memory profiling for custom builds (#5584) 2018-10-31 11:11:45 -07:00
Calvin Leung Huang a08ccbffa7
[Review Only] Autoseal OSS port (#757)
* Port awskms autoseal

* Rename files

* WIP autoseal

* Fix protobuf conflict

* Expose some structs to properly allow encrypting stored keys

* Update awskms with the latest changes

* Add KeyGuard implementation to abstract encryption/decryption of keys

* Fully decouple seal.Access implementations from sealwrap structs

* Add extra line to proto files, comment update

* Update seal_access_entry.go

* govendor sync

* Add endpoint info to configureAWSKMSSeal

* Update comment

* Refactor structs

* Update make proto

* Remove remove KeyGuard, move encrypt/decrypt to autoSeal

* Add rest of seals, update VerifyRecoveryKeys, add deps

* Fix some merge conflicts via govendor updates

* Rename SealWrapEntry to EncryptedBlobInfo

* Remove barrier type upgrade check in oss

* Add key to EncryptedBlobInfo proto

* Update barrierTypeUpgradeCheck signature
2018-10-19 14:43:57 -07:00
Jeff Mitchell 5363f1308f Makefile updates 2018-09-20 12:38:05 -04:00