86fb5dcf8c
* backport * Update README.md Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>
121 lines
6 KiB
Plaintext
121 lines
6 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Key/Value (KV) Store Overview
|
|
description: >-
|
|
Consul includes a KV store for indexed objects, configuration parameters, and metadata that you can use to dynamically configure apps. Learn about accessing and using the KV store to extend Consul's functionality through watches, sessions, and Consul Template.
|
|
---
|
|
|
|
# Key/Value (KV) Store Overview
|
|
|
|
<Note>
|
|
The Consul KV API, CLI, and UI are now considered feature complete and no new feature development is planned for future releases.
|
|
</Note>
|
|
|
|
Consul KV is a core feature of Consul and is installed with the Consul agent.
|
|
Once installed with the agent, it will have reasonable defaults. Consul KV allows
|
|
users to store indexed objects, though its main uses are storing configuration
|
|
parameters and metadata. Please note that it is a simple KV store and is not
|
|
intended to be a full featured datastore (such as DynamoDB) but has some
|
|
similarities to one.
|
|
|
|
The Consul KV datastore is located on the servers, but can be accessed by any
|
|
agent (client or server). The natively integrated [RPC
|
|
functionality](/consul/docs/architecture) allows clients to forward
|
|
requests to servers, including key/value reads and writes. Part of Consul's
|
|
core design allows data to be replicated automatically across all the servers.
|
|
Having a quorum of servers will decrease the risk of data loss if an outage
|
|
occurs.
|
|
|
|
If you have not used Consul KV, complete this [Getting Started
|
|
tutorial](/consul/tutorials/interactive/get-started-key-value-store?utm_source=docs) on HashiCorp.
|
|
|
|
## Accessing the KV store
|
|
|
|
The KV store can be accessed by the [consul kv CLI
|
|
subcommands](/consul/commands/kv), [HTTP API](/consul/api-docs/kv), and Consul UI.
|
|
To restrict access, enable and configure
|
|
[ACLs](/consul/tutorials/security/access-control-setup-production).
|
|
Once the ACL system has been bootstrapped, users and services, will need a
|
|
valid token with KV [privileges](/consul/docs/security/acl/acl-rules#key-value-rules) to
|
|
access the the data store, this includes even reads. We recommend creating a
|
|
token with limited privileges, for example, you could create a token with write
|
|
privileges on one key for developers to update the value related to their
|
|
application.
|
|
|
|
The datastore itself is located on the Consul servers in the [data
|
|
directory](/consul/docs/agent/config/cli-flags#_data_dir). To ensure data is not lost in
|
|
the event of a complete outage, use the [`consul snapshot`](/consul/commands/snapshot/restore) feature to backup the data.
|
|
|
|
## Using Consul KV
|
|
|
|
Objects are opaque to Consul, meaning there are no restrictions on the type of
|
|
object stored in a key/value entry. The main restriction on an object is size -
|
|
the maximum is 512 KB. Due to the maximum object size and main use cases, you
|
|
should not need extra storage; the general [sizing
|
|
recommendations](/consul/docs/agent/config/config-files#kv_max_value_size)
|
|
are usually sufficient.
|
|
|
|
Keys, like objects are not restricted by type and can include any character.
|
|
However, we recommend using URL-safe chars - `[a-zA-Z0-9-._~]` with the
|
|
exception of `/`, which can be used to help organize data. Note, `/` will be
|
|
treated like any other character and is not fixed to the file system. Meaning,
|
|
including `/` in a key does not fix it to a directory structure. This model is
|
|
similar to Amazon S3 buckets. However, `/` is still useful for organizing data
|
|
and when recursively searching within the data store. We also recommend that
|
|
you avoid the use of `*`, `?`, `'`, and `%` because they can cause issues when
|
|
using the API and in shell scripts.
|
|
|
|
## Using Sentinel to apply policies for Consul KV
|
|
|
|
<EnterpriseAlert>
|
|
|
|
This feature requires
|
|
HashiCorp Cloud Platform (HCP) or self-managed Consul Enterprise.
|
|
|
|
</EnterpriseAlert>
|
|
|
|
You can also use Sentinel as a Policy-as-code framework for defining advanced key-value storage access control policies. Sentinel policies extend the ACL system in Consul beyond static "read", "write",
|
|
and "deny" policies to support full conditional logic and integration with
|
|
external systems. Reference the [Sentinel documentation](https://docs.hashicorp.com/sentinel/concepts) for high-level Sentinel concepts.
|
|
|
|
To get started with Sentinel in Consul,
|
|
refer to the [Sentinel documentation](https://docs.hashicorp.com/sentinel/consul) or
|
|
[Consul documentation](/consul/docs/agent/sentinel).
|
|
|
|
|
|
## Extending Consul KV
|
|
|
|
### Consul Template
|
|
|
|
If you plan to use Consul KV as part of your configuration management process
|
|
review the [Consul
|
|
Template](/consul/tutorials/developer-configuration/consul-template?utm_source=docs)
|
|
tutorial on how to update configuration based on value updates in the KV. Consul
|
|
Template is based on Go Templates and allows for a series of scripted actions
|
|
to be initiated on value changes to a Consul key.
|
|
|
|
### Watches
|
|
|
|
Consul KV can also be extended with the use of watches.
|
|
[Watches](/consul/docs/dynamic-app-config/watches) are a way to monitor data for updates. When
|
|
an update is detected, an external handler is invoked. To use watches with the
|
|
KV store the [key](/consul/docs/dynamic-app-config/watches#key) watch type should be used.
|
|
|
|
### Consul Sessions
|
|
|
|
Consul sessions can be used to build distributed locks with Consul KV. Sessions
|
|
act as a binding layer between nodes, health checks, and key/value data. The KV
|
|
API supports an `acquire` and `release` operation. The `acquire` operation acts
|
|
like a Check-And-Set operation. On success, there is a key update and an
|
|
increment to the `LockIndex` and the session value is updated to reflect the
|
|
session holding the lock. Review the session documentation for more information
|
|
on the [integration](/consul/docs/dynamic-app-config/sessions#k-v-integration).
|
|
|
|
Review the following tutorials to learn how to use Consul sessions for [application leader election](/consul/tutorials/developer-configuration/application-leader-elections) and
|
|
to [build distributed semaphores](/consul/tutorials/developer-configuration/distributed-semaphore).
|
|
|
|
### Vault
|
|
|
|
If you plan to use Consul KV as a backend for Vault, please review [this
|
|
tutorial](/vault/tutorials/day-one-consul/ha-with-consul?utm_source=docs).
|