226 lines
9.3 KiB
Plaintext
226 lines
9.3 KiB
Plaintext
---
|
|
layout: "docs"
|
|
page_title: "Commands: Snapshot Agent"
|
|
sidebar_current: "docs-commands-snapshot-agent"
|
|
---
|
|
|
|
# Consul Snapshot Agent
|
|
|
|
Command: `consul snapshot agent`
|
|
|
|
~> The [`agent`](/docs/commands/snapshot/agent.html) subcommand described here is
|
|
available only in [Consul Enterprise](https://www.hashicorp.com/products/consul/)
|
|
version 0.7.1 and later. All other [snapshot subcommands](/docs/commands/snapshot.html)
|
|
are available in the open source version of Consul.
|
|
|
|
The `snapshot agent` subcommand starts a process that takes snapshots of the
|
|
state of the Consul servers and saves them locally, or pushes them to an
|
|
optional remote storage service. This subcommand is only available in Consul
|
|
Enterprise 0.7.1 and later.
|
|
|
|
The agent can be run as a long-running daemon process or in a one-shot mode
|
|
from a batch job, based on the [`-interval`](#interval) argument. Snapshotting
|
|
a remote datacenter is only available in one-shot mode.
|
|
|
|
As a long-running daemon, the agent will perform a leader election so multiple
|
|
processes can be run in a highly available fashion with automatic failover. The
|
|
agent will also register itself with Consul as a service, along with health
|
|
checks that show the agent is alive ("Consul Snapshot Agent Alive") and able to
|
|
take snapshots ("Consul Snapshot Agent Saving Snapshots"). The latter check is
|
|
only added on agents who have become a leader, so it's possible for operators to
|
|
tell which instances are alive and on standby and which instance has become
|
|
leader and starting saving snapshots.
|
|
|
|
As snapshots are saved, they will be reported in the log produced by the agent:
|
|
|
|
```
|
|
2016/11/16 21:21:13 [INFO] Snapshot agent running
|
|
2016/11/16 21:21:13 [INFO] Waiting to obtain leadership...
|
|
2016/11/16 21:21:13 [INFO] Obtained leadership
|
|
2016/11/16 21:21:13 [INFO] Saved snapshot 1479360073448728784
|
|
```
|
|
|
|
The number shown with the saved snapshot is its ID, which is based on a UNIX
|
|
timestamp with nanosecond resolution, so collisions are unlikely and IDs are
|
|
monotonically increasing with time. This makes it easy to locate the latest
|
|
snapshot, even if the log data isn't available. The snapshot ID always appears
|
|
in the file name when using local storage, or in the object key when using
|
|
remote storage.
|
|
|
|
Snapshots can be restored using the
|
|
[`consul snapshot restore`](/docs/commands/snapshot/restore.html) command, or
|
|
the [HTTP API](/api/snapshot.html).
|
|
|
|
If ACLs are enabled, a management token must be supplied in order to perform
|
|
snapshot operations.
|
|
|
|
## Usage
|
|
|
|
Usage: `consul snapshot agent [options]`
|
|
|
|
#### API Options
|
|
|
|
<%= partial "docs/commands/http_api_options_client" %>
|
|
|
|
#### Config File Options:
|
|
|
|
* `-config-dir` - Directory to look for JSON config files. Files will be read in
|
|
alphabetical order and must end with the extension ".json". This won't
|
|
recursively descend directories. This can be specified multiple times on the
|
|
command line.
|
|
|
|
* `-config-file` - File to read JSON configuration from. Files must end with the
|
|
extension ".json". This can be specified multiple times on the command line.
|
|
|
|
Config files referenced using `-config-dir` and `-config-file` have the following
|
|
format (shown populated with default values):
|
|
|
|
```javascript
|
|
{
|
|
"snapshot_agent": {
|
|
"http_addr": "127.0.0.1:8500",
|
|
"token": "",
|
|
"datacenter": "",
|
|
"ca_file": "",
|
|
"ca_path": "",
|
|
"cert_file": "",
|
|
"key_file": "",
|
|
"tls_server_name": "",
|
|
"log": {
|
|
"level": "INFO",
|
|
"enable_syslog": false,
|
|
"syslog_facility": "LOCAL0"
|
|
},
|
|
"snapshot": {
|
|
"interval": "1h",
|
|
"retain": 30,
|
|
"stale": false,
|
|
"service": "consul-snapshot",
|
|
"deregister_after": "72h",
|
|
"lock_key": "consul-snapshot/lock",
|
|
"max_failures": 3
|
|
},
|
|
"local_storage": {
|
|
"path": "."
|
|
},
|
|
"aws_storage": {
|
|
"access_key_id": "",
|
|
"secret_access_key": "",
|
|
"s3_region": "",
|
|
"s3_bucket": "",
|
|
"s3_key_prefix": "consul-snapshot",
|
|
"s3_server_side_encryption":false,
|
|
"s3_static_snapshot_name":""
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
All fields are optional, and config files without a `snapshot_agent` object will
|
|
be ignored. At least one config file needs to have a `snapshot_agent` object, or the
|
|
snapshot agent will fail to start. The Consul agent is set up to ignore any
|
|
`snapshot_agent` object, so it's safe to use common config directories for both agents
|
|
if desired.
|
|
|
|
#### Snapshot Options
|
|
|
|
* <a name="interval">`-interval`</a> - Interval at which to perform snapshots
|
|
as a time with a unit suffix, which can be "s", "m", "h" for seconds, minutes,
|
|
or hours. If 0 is provided, the agent will take a single snapshot and then exit,
|
|
which is useful for running snapshots via batch jobs. Defaults to "1h"
|
|
|
|
* `-lock-key` - A prefix in Consul's KV store used to coordinate between
|
|
different instances of the snapshot agent order to only have one active instance
|
|
at a time. For highly available operation of the snapshot agent, simply run
|
|
multiple instances. All instances must be configured with the same lock key in
|
|
order to properly coordinate. Defaults to "consul-snapshot/lock".
|
|
|
|
* `-max-failures` - Number of snapshot failures after which the snapshot agent
|
|
will give up leadership. In a highly available operation with multiple snapshot
|
|
agents available, this gives another agent a chance to take over if an agent
|
|
is experiencing issues, such as running out of disk space for snapshots.
|
|
Defaults to 3.
|
|
|
|
* `-retain` - Number of snapshots to retain. After each snapshot is taken, the
|
|
oldest snapshots will start to be deleted in order to retain at most this many
|
|
snapshots. If this is set to 0, the agent will not perform this and snapshots
|
|
will accumulate forever. Defaults to 30.
|
|
|
|
|
|
#### Agent Options
|
|
|
|
* `-deregister-after` - An interval, after which if the agent is unhealthy it will be
|
|
automatically deregistered from Consul service discovery. This is a time with a
|
|
unit suffix, which can be "s", "m", "h" for seconds, minutes, or hours. If 0 is
|
|
provided, this will be disabled. Defaults to "72h".
|
|
|
|
* `-log-level` - Controls verbosity of snapshot agent logs. Valid options are
|
|
"TRACE", "DEBUG", "INFO", "WARN", "ERR". Defaults to "INFO".
|
|
|
|
* `-service` - The service name to used when registering the agent with Consul.
|
|
Registering helps monitor running agents and the leader registers an additional
|
|
health check to monitor that snapshots are taking place. Defaults to
|
|
"consul-snapshot".
|
|
|
|
* `-syslog` - This enables forwarding logs to syslog. Defaults to false.
|
|
|
|
* `-syslog-facility` - Sets the facility to use for forwarding logs to syslog.
|
|
Defaults to "LOCAL0".
|
|
|
|
#### Local Storage Options
|
|
|
|
* `-local-path` - Location to store snapshots locally. The default behavior
|
|
of the snapshot agent is to store snapshots locally in this directory. Defaults
|
|
to "." to use the current working directory. If an alternate storage option is
|
|
configured, then local storage will be disabled and this option will be ignored.
|
|
|
|
#### Amazon S3 Storage Options
|
|
|
|
* `-aws-access-key-id` and `-aws-secret-access-key` - These arguments supply
|
|
authentication information for connecting to AWS. These may also be supplied using
|
|
the following alternative methods:<br>
|
|
- `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables
|
|
- A credentials file (`~/.aws/credentials` or the file at the path specified by the
|
|
`AWS_SHARED_CREDENTIALS_FILE` environment variable)
|
|
- ECS task role metadata (container-specific)
|
|
- EC2 instance role metadata
|
|
|
|
* `-aws-s3-bucket` - S3 bucket to use. Required for S3 storage, and setting this
|
|
disables local storage.
|
|
|
|
* `-aws-s3-key-prefix` - Prefix to use for snapshot files in S3. Defaults to
|
|
"consul-snapshot".
|
|
|
|
* `-aws-s3-region` - S3 region to use. Required for S3 storage.
|
|
|
|
* `-aws-s3-server-side-encryption` - Enables saving snapshots to S3 using server side encryption with [Amazon S3-Managed Encryption Keys](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html)
|
|
|
|
* `-aws-s3-static-snapshot-name` - If this is given, all snapshots are saved with the same file name. The agent will not rotate or versionize snapshots, and will save them with the same name each time.
|
|
Use this if you want to rely on [S3's versioning capabilities](http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) instead of the agent handling it for you.
|
|
|
|
* `-aws-s3-enable-kms` - Enables using [Amazon KMS](https://aws.amazon.com/kms/) for encrypting snapshots.
|
|
|
|
* `-aws-s3-kms-key` - Optional Amazon KMS key to use, if this is not set the default KMS master key will be used. Set this if you want to manage key rotation yourself.
|
|
|
|
## Examples
|
|
|
|
Running the agent with no arguments will run a long-running daemon process that will
|
|
perform leader election for highly available operation, register itself with Consul
|
|
service discovery with health checks, take snapshots every hour, retain the last 30
|
|
snapshots, and save snapshots into the current working directory:
|
|
|
|
```
|
|
$ consul snapshot agent
|
|
```
|
|
|
|
To run a one-shot backup, set the backup interval to 0. This will run a single snapshot
|
|
and delete any old snapshots based on the retain settings, but it will not perform any
|
|
leader election or service registration:
|
|
|
|
```
|
|
$ consul snapshot agent -interval=0
|
|
```
|
|
|
|
Please see the [HTTP API](/api/snapshot.html) documentation for
|
|
more details about snapshot internals.
|