diff --git a/website/source/docs/glossary.html.md b/website/source/docs/glossary.html.md new file mode 100644 index 000000000..70999834a --- /dev/null +++ b/website/source/docs/glossary.html.md @@ -0,0 +1,85 @@ +--- +layout: "docs" +page_title: "Consul Glossary" +sidebar_current: "docs-glossary" +description: |- + This page collects brief definitions of some of the technical terms used in the documentation. +--- + +# Consul Glossary + +This page collects brief definitions of some of the technical terms used in the documentation for Consul and Consul Enterprise, as well as some terms that come up frequently in conversations throughout the Consul community. + +* [Agent](#agent) +* [Client](#client) +* [Server](#server) +* [Datacenter](#datacenter) +* [Consensus](#consensus) +* [Gossip](#gossip) +* [LAN Gossip](#lan-gossip) +* [WAN Gossip](#wan-gossip) +* [RPC](#rpc) + +## Agent + +An agent is the long running daemon on every member of the Consul cluster. +It is started by running `consul agent`. The agent is able to run in either *client* +or *server* mode. Since all nodes must be running an agent, it is simpler to refer to +the node as being either a client or server, but there are other instances of the agent. All +agents can run the DNS or HTTP interfaces, and are responsible for running checks and +keeping services in sync. + +## Client + +A client is an agent that forwards all RPCs to a server. The client is relatively +stateless. The only background activity a client performs is taking part in the LAN gossip +pool. This has a minimal resource overhead and consumes only a small amount of network +bandwidth. + +## Server + +A server is an agent with an expanded set of responsibilities including +participating in the Raft quorum, maintaining cluster state, responding to RPC queries, +exchanging WAN gossip with other datacenters, and forwarding queries to leaders or +remote datacenters. + +## Datacenter + +We define a datacenter to be a networking environment that is +private, low latency, and high bandwidth. This excludes communication that would traverse +the public internet, but for our purposes multiple availability zones within a single EC2 +region would be considered part of a single datacenter. + +## Consensus + +When used in our documentation we use consensus to mean agreement upon +the elected leader as well as agreement on the ordering of transactions. Since these +transactions are applied to a +[finite-state machine](https://en.wikipedia.org/wiki/Finite-state_machine), our definition +of consensus implies the consistency of a replicated state machine. Consensus is described +in more detail on [Wikipedia](https://en.wikipedia.org/wiki/Consensus_(computer_science)), +and our implementation is described [here](/docs/internals/consensus.html). + +## Gossip + +Consul is built on top of [Serf](https://www.serf.io/) which provides a full +[gossip protocol](https://en.wikipedia.org/wiki/Gossip_protocol) that is used for multiple purposes. +Serf provides membership, failure detection, and event broadcast. Our use of these +is described more in the [gossip documentation](/docs/internals/gossip.html). It is enough to know +that gossip involves random node-to-node communication, primarily over UDP. + +## LAN Gossip + +Refers to the LAN gossip pool which contains nodes that are all +located on the same local area network or datacenter. + +## WAN Gossip + +Refers to the WAN gossip pool which contains only servers. These +servers are primarily located in different datacenters and typically communicate +over the internet or wide area network. + +## RPC + +Remote Procedure Call. This is a request / response mechanism allowing a +client to make a request of a server. \ No newline at end of file diff --git a/website/source/docs/internals/anti-entropy.html.md b/website/source/docs/internals/anti-entropy.html.md index bd04681ee..d2c4e11f4 100644 --- a/website/source/docs/internals/anti-entropy.html.md +++ b/website/source/docs/internals/anti-entropy.html.md @@ -12,11 +12,6 @@ Consul uses an advanced method of maintaining service and health information. This page details how services and checks are registered, how the catalog is populated, and how health status information is updated as it changes. -~> **Advanced Topic!** This page covers technical details of -the internals of Consul. You don't need to know these details to effectively -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. - ### Components It is important to first understand the moving pieces involved in services and diff --git a/website/source/docs/internals/architecture.html.md b/website/source/docs/internals/architecture.html.md index d45314b0c..44bac85e0 100644 --- a/website/source/docs/internals/architecture.html.md +++ b/website/source/docs/internals/architecture.html.md @@ -12,61 +12,10 @@ Consul is a complex system that has many different moving parts. To help users and developers of Consul form a mental model of how it works, this page documents the system architecture. -~> **Advanced Topic!** This page covers technical details of -the internals of Consul. You don't need to know these details to effectively -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. +-> Before describing the architecture, we recommend reading the +[glossary](/docs/glossary) of terms to help +clarify what is being discussed. -## Glossary - -Before describing the architecture, we provide a glossary of terms to help -clarify what is being discussed: - -* Agent - An agent is the long running daemon on every node of the Consul cluster. -It is started by running `consul agent`. The agent is able to run in either *client* -or *server* mode. Since all nodes must be running an agent, it is simpler to refer to -them as being either a client or server. All -agents can run the DNS or HTTP interfaces, and are responsible for running checks and -keeping services in sync. - -* Client - A client is an agent that forwards all RPCs to a server. The client is relatively -stateless. The only background activity a client performs is taking part in the LAN gossip -pool. This has a minimal resource overhead and consumes only a small amount of network -bandwidth. - -* Server - A server is an agent with an expanded set of responsibilities including -participating in the Raft quorum, maintaining cluster state, responding to RPC queries, -exchanging WAN gossip with other datacenters, and forwarding queries to leaders or -remote datacenters. - -* Datacenter - We define a datacenter to be a networking environment that is -private, low latency, and high bandwidth. This excludes communication that would traverse -the public internet, but for our purposes multiple availability zones within a single EC2 -region would be considered part of a single datacenter. - -* Consensus - When used in our documentation we use consensus to mean agreement upon -the elected leader as well as agreement on the ordering of transactions. Since these -transactions are applied to a -[finite-state machine](https://en.wikipedia.org/wiki/Finite-state_machine), our definition -of consensus implies the consistency of a replicated state machine. Consensus is described -in more detail on [Wikipedia](https://en.wikipedia.org/wiki/Consensus_(computer_science)), -and our implementation is described [here](/docs/internals/consensus.html). - -* Gossip - Consul is built on top of [Serf](https://www.serf.io/) which provides a full -[gossip protocol](https://en.wikipedia.org/wiki/Gossip_protocol) that is used for multiple purposes. -Serf provides membership, failure detection, and event broadcast. Our use of these -is described more in the [gossip documentation](/docs/internals/gossip.html). It is enough to know -that gossip involves random agent-to-agent communication, primarily over UDP. - -* LAN Gossip - Refers to the LAN gossip pool which contains agents that are all -located on the same local area network or datacenter. - -* WAN Gossip - Refers to the WAN gossip pool which contains only servers. These -servers are primarily located in different datacenters and typically communicate -over the internet or wide area network. - -* RPC - Remote Procedure Call. This is a request / response mechanism allowing a -client to make a request of a server. ## 10,000 foot view diff --git a/website/source/docs/internals/consensus.html.md b/website/source/docs/internals/consensus.html.md index 29ab49202..48a49f9e2 100644 --- a/website/source/docs/internals/consensus.html.md +++ b/website/source/docs/internals/consensus.html.md @@ -14,11 +14,6 @@ The consensus protocol is based on ["Raft: In search of an Understandable Consensus Algorithm"](https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf). For a visual explanation of Raft, see [The Secret Lives of Data](http://thesecretlivesofdata.com/raft). -~> **Advanced Topic!** This page covers technical details of -the internals of Consul. You don't need to know these details to effectively -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. - ## Raft Protocol Overview Raft is a consensus algorithm that is based on diff --git a/website/source/docs/internals/coordinates.html.md b/website/source/docs/internals/coordinates.html.md index 0e5ef8593..26ac77aba 100644 --- a/website/source/docs/internals/coordinates.html.md +++ b/website/source/docs/internals/coordinates.html.md @@ -20,11 +20,6 @@ Serf's network tomography is based on ["Vivaldi: A Decentralized Network Coordin with some enhancements based on other research. There are more details about [Serf's network coordinates here](https://www.serf.io/docs/internals/coordinates.html). -~> **Advanced Topic!** This page covers the technical details of -the internals of Consul. You don't need to know these details to effectively -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. - ## Network Coordinates in Consul Network coordinates manifest in several ways inside Consul: diff --git a/website/source/docs/internals/gossip.html.md b/website/source/docs/internals/gossip.html.md index d380e50d1..f36fdd8c9 100644 --- a/website/source/docs/internals/gossip.html.md +++ b/website/source/docs/internals/gossip.html.md @@ -15,11 +15,6 @@ used by Serf is based on ["SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol"](http://www.cs.cornell.edu/info/projects/spinglass/public_pdfs/swim.pdf), with a few minor adaptations. There are more details about [Serf's protocol here](https://www.serf.io/docs/internals/gossip.html). -~> **Advanced Topic!** This page covers technical details of -the internals of Consul. You don't need to know these details to effectively -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. - ## Gossip in Consul Consul makes use of two different gossip pools. We refer to each pool as the diff --git a/website/source/docs/internals/index.html.md b/website/source/docs/internals/index.html.md index 21415d21c..524c44c56 100644 --- a/website/source/docs/internals/index.html.md +++ b/website/source/docs/internals/index.html.md @@ -8,8 +8,18 @@ description: |- # Consul Internals -This section covers some of the internals of Consul, such as the architecture, -consensus and gossip protocols, and security model. +This section covers some of the internals of Consul. Understanding the internals of Consul is necessary to successfully +use it in production. --> **Note:** Knowing about the internals of Consul is not necessary to successfully -use it. We document it here to be completely transparent about how Consul works. +Please review the following documentation to understand how Consul works. + +* [Architecture](/docs/internals/architecture.html) +* [Consensus Protocol](/docs/internals/consensus.html) +* [Gossip Protocol](/docs/internals/gossip.html) +* [Network Coordinates](/docs/internals/coordinates.html) +* [Sessions](/docs/internals/sessions.html) +* [Anti-Entropy](/docs/internals/anti-entropy.html) +* [Security Model](/docs/internals/security.html) + +You should also be familiar with [Jepsen testing](/docs/internals/jepsen.html), before deploying +a production datacenter. diff --git a/website/source/docs/internals/security.html.md b/website/source/docs/internals/security.html.md index feadde151..284b25fa7 100644 --- a/website/source/docs/internals/security.html.md +++ b/website/source/docs/internals/security.html.md @@ -28,11 +28,6 @@ This means Consul communication is protected against eavesdropping, tampering, and spoofing. This makes it possible to run Consul over untrusted networks such as EC2 and other shared hosting providers. -~> **Advanced Topic!** This page covers the technical details of -the security model of Consul. You don't need to know these details to -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. - ## Secure Configuration The Consul threat model is only applicable if Consul is running in a secure diff --git a/website/source/docs/internals/sessions.html.md b/website/source/docs/internals/sessions.html.md index 40fbb6d3a..1e271c62f 100644 --- a/website/source/docs/internals/sessions.html.md +++ b/website/source/docs/internals/sessions.html.md @@ -13,11 +13,6 @@ Sessions act as a binding layer between nodes, health checks, and key/value data They are designed to provide granular locking and are heavily inspired by [The Chubby Lock Service for Loosely-Coupled Distributed Systems](http://research.google.com/archive/chubby.html). -~> **Advanced Topic!** This page covers technical details of -the internals of Consul. You don't need to know these details to effectively -operate and use Consul. These details are documented here for those who wish -to learn about them without having to go spelunking through the source code. - ## Session Design A session in Consul represents a contract that has very specific semantics. diff --git a/website/source/layouts/docs.erb b/website/source/layouts/docs.erb index a7acafefa..449f4d23f 100644 --- a/website/source/layouts/docs.erb +++ b/website/source/layouts/docs.erb @@ -31,6 +31,10 @@ + > + Glossary + + > Internals