open-consul/website/content/docs/architecture/coordinates.mdx

101 lines
3.7 KiB
Plaintext
Raw Normal View History

2015-10-22 00:29:43 +00:00
---
2020-04-07 18:55:19 +00:00
layout: docs
page_title: Network Coordinates
sidebar_title: Network Coordinates
2020-07-08 23:09:00 +00:00
description: A Decentralized Network Coordinate System, with several improvements based on several follow-on papers.
2015-10-22 00:29:43 +00:00
---
# Network Coordinates
Consul uses a [network tomography](https://en.wikipedia.org/wiki/Network_tomography)
system to compute network coordinates for nodes in the cluster. These coordinates
allow the network round trip time to be estimated between any two nodes using a
2015-10-31 06:50:11 +00:00
very simple calculation. This allows for many useful applications, such as finding
2015-10-22 00:29:43 +00:00
the service node nearest a requesting node, or failing over to services in the next
closest datacenter.
2016-08-08 16:44:27 +00:00
All of this is provided through the use of the [Serf library](https://www.serf.io/).
2015-10-22 00:29:43 +00:00
Serf's network tomography is based on ["Vivaldi: A Decentralized Network Coordinate System"](http://www.cs.ucsb.edu/~ravenben/classes/276/papers/vivaldi-sigcomm04.pdf),
with some enhancements based on other research. There are more details about
2016-08-08 16:44:27 +00:00
[Serf's network coordinates here](https://www.serf.io/docs/internals/coordinates.html).
2015-10-22 00:29:43 +00:00
## Network Coordinates in Consul
Network coordinates manifest in several ways inside Consul:
- The [`consul rtt`](/commands/rtt) command can be used to query for the
2015-10-22 00:29:43 +00:00
network round trip time between any two nodes.
2020-04-09 23:46:54 +00:00
- The [Catalog endpoints](/api/catalog) and
[Health endpoints](/api/health) can sort the results of queries based
2015-10-22 00:29:43 +00:00
on the network round trip time from a given node using a "?near=" parameter.
2020-04-09 23:46:54 +00:00
- [Prepared queries](/api/query) can automatically fail over services
2017-11-01 04:23:51 +00:00
to other Consul datacenters based on network round trip times. See the
[Geo Failover](https://learn.hashicorp.com/tutorials/consul/automate-geo-failover) for some examples.
2020-04-09 23:46:54 +00:00
- The [Coordinate endpoint](/api/coordinate) exposes raw network
2015-10-22 00:29:43 +00:00
coordinates for use in other applications.
Consul uses Serf to manage two different gossip pools, one for the LAN with members
of a given datacenter, and one for the WAN which is made up of just the Consul servers
in all datacenters. It's important to note that **network coordinates are not compatible
between these two pools**. LAN coordinates only make sense in calculations with other
LAN coordinates, and WAN coordinates only make sense with other WAN coordinates.
## Working with Coordinates
Computing the estimated network round trip time between any two nodes is simple
once you have their coordinates. Here's a sample coordinate, as returned from the
2020-04-09 23:46:54 +00:00
[Coordinate endpoint](/api/coordinate).
2015-10-22 00:29:43 +00:00
```
"Coord": {
"Adjustment": 0.1,
"Error": 1.5,
"Height": 0.02,
"Vec": [0.34,0.68,0.003,0.01,0.05,0.1,0.34,0.06]
}
```
All values are floating point numbers in units of seconds, except for the error
term which isn't used for distance calculations.
Here's a complete example in Go showing how to compute the distance between two
coordinates:
```
import (
"math"
"time"
2017-05-05 10:36:47 +00:00
"github.com/hashicorp/serf/coordinate"
2015-10-22 00:29:43 +00:00
)
func dist(a *coordinate.Coordinate, b *coordinate.Coordinate) time.Duration {
// Coordinates will always have the same dimensionality, so this is
// just a sanity check.
if len(a.Vec) != len(b.Vec) {
panic("dimensions aren't compatible")
}
// Calculate the Euclidean distance plus the heights.
sumsq := 0.0
for i := 0; i < len(a.Vec); i++ {
diff := a.Vec[i] - b.Vec[i]
sumsq += diff * diff
}
rtt := math.Sqrt(sumsq) + a.Height + b.Height
// Apply the adjustment components, guarding against negatives.
adjusted := rtt + a.Adjustment + b.Adjustment
if adjusted > 0.0 {
rtt = adjusted
}
// Go's times are natively nanoseconds, so we convert from seconds.
const secondsToNanoseconds = 1.0e9
return time.Duration(rtt * secondsToNanoseconds)
}
```