open-vault/physical/cassandra
Jack Kleeman 1977305ffa Store less data in Cassandra prefix buckets (#7199)
* Store less data in Cassandra prefix buckets

The Cassandra physical backend relies on storing data for sys/foo/bar
under sys, sys/foo, and sys/foo/bar. This is necessary so that we
can list the sys bucket, get a list of all child keys, and then trim
this down to find child 'folders' eg food. Right now however, we store
the full value of every storage entry in all three buckets. This is
unnecessary as the value will only ever be read out in the leaf bucket
ie sys/foo/bar. We use the intermediary buckets simply for listing keys.

We have seen some issues around compaction where certain buckets,
particularly intermediary buckets that are exclusively for listing,
get really clogged up with data to the point of not being listable.
Buckets like sys/expire/id are huge, combining lease expiry data for
all auth methods, and need to be listed for vault to successfully
become leader. This PR tries to cut down on the amount of data stored
in intermediary buckets.

* Avoid goroutine leak by buffering results channel up to the bucket count
2019-08-19 11:50:00 -07:00
..
cassandra.go Store less data in Cassandra prefix buckets (#7199) 2019-08-19 11:50:00 -07:00
cassandra_test.go TestCassandraBackend times out pretty often on the ent build. Bump (#7130) 2019-07-23 13:26:36 -04:00