* Support rate limiting and concurrency limiting CSR requests on servers; handle CA rotations gracefully with jitter and backoff-on-rate-limit in client
* Add CSR rate limiting docs
* Fix config naming and add tests for new CA configs
* Stared updaing links for the learn migration
* Language change cluster -> datacenter (#5212)
* Updating the language from cluster to datacenter in the backup guide to be consistent and more accurate.
* missed some clusters
* updated three broken links for the sidebar nav
Adds info about `k8stag` and `nodePortSyncType` options that were
added in consul-helm v0.5.0.
Additionally moves the k8sprefix to match the order in the Helm chart
values file, while also clarifying that it only affects one sync
direction.
If a user installs the default Helm chart Consul on a Kubernetes
cluster that is open to the internet, it is lacking some important
security configurations.
* Re-worked the ACL guide into two docs and an updated guide.
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Updating syntax based on amayer5125's comments.
* Missed one of amayer5125's comments
* found a bad link in the acl system docs
* fixing a link in the rules docs
This endpoint aggregates all checks related to <service id> on the agent
and return an appropriate http code + the string describing the worst
check.
This allows to cleanly expose service status to other component, hiding
complexity of multiple checks.
This is especially useful to use consul to feed a load balancer which
would delegate health checking to consul agent.
Exposing this endpoint on the agent is necessary to avoid a hit on
consul servers and avoid decreasing resiliency (this endpoint will work
even if there is no consul leader in the cluster).
* Adding the new backup guide
* Update website/source/docs/guides/backup.html.md
Looks good.
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/backup.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/backup.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/backup.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Updated the directions for the restore command.
* Update website/source/docs/guides/backup.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/backup.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* updated the token env
* Trying to make it extra clear where to run the commands.
* added not that list of backed up items isn't inclusive
There has been some confusion about the formating of multi-line
string variables in the Helm chart. This adds examples for these
situations, hopefully clarifying things for users.
* added a new section for adding servers, updated section titles, and added code snippets.
* Fixing typos
* fixing typos
* Addressing some of Paul's feedback.
* Updated the outage recovery recommendation
* Adding examples and a summary. Minor structure updates.
* Added a link to the deployment guide, but needed to remove a sentence referring to a guide that's not published yet.
* fixed typo
This changes with time so will opt to not hardcoding this. Investigated using the middleman helper to render
out the current configured version but that won't exist on learn.hashicorp.com so I think this
is the most future-proof way.
* Adding the new Consul Template Guide.
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Making updates based on Pauls feedback.
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Update website/source/docs/guides/consul-template.html.md
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Couple more updates based on Pauls feedback.
* updated the intro to include how template can replace api use
* updated the address
* Adding minikube guide to the index page.
* Adds images for Minikube guide
* Adds notes about Helm YAML configuration
* Embed Wistia video for Consul Minikube guide
* Cleans up menus, instructions, Helm steps
* Adds captions for all images
* Uses Docker images hosted on hashicorp org for demo
* Adds link to Minikube guide from menu and k8s index page
* Clarify steps in Minikube/Kubernetes guide
* Adds note about use of Fastly VCL for redirects
Some redirects are not possible with a redirect file, so we send traffic to
learn.hashicorp.com with custom VCL (Varnish) handlers using the Fastly web
admin.
Future migrations of content to learn.hashicorp.com should also use this
approach.
* adding file2consul listing
* minor edit
* modified wording added some punctuation
* Moved file2consul so it is listed in alpha order. Replaces Pull Request # #4942
* website: add multi-dc enterprise landing page
* website: switch all 1.4.0 alerts/RC warnings
* website: connect product wording
Co-Authored-By: pearkes <jackpearkes@gmail.com>
* website: remove RC notification
* commmand/acl: fix usage docs for ACL tokens
* agent: remove comment, OperatorRead
* website: improve multi-dc docs
Still not happy with this but tried to make it slightly more informative.
* website: put back acl guide warning for 1.4.0
* website: simplify multi-dc page and respond to feedback
* Fix Multi-DC typos on connect index page.
* Improve Multi-DC overview.
A full guide is a WIP and will be added post-release.
* Fixes typo avaiable > available
There is a fine line between making the helm chart easy and simple to
use and supporting lots of configurability. This documents options for
users who would like to extend the Helm chart beyond what is readily
available in the `values.yaml` file.
This adds two Helm chart values into the documentation with details
that have come up in several issues.
Additionally, it notes that persistent volumes and their claims need
to be removed manually because of current kubernetes and helm design.
* Adds redirects for Getting Started pages
* Uses correct links to resources at learn.hashicorp
* Reconfigures "Learn more" links to point to learn.hashicorp
* Links to learn.hashicorp on segmentation page
* Adds redirect for sample config file
* Fixes links to Getting Started guide on learn.hashicorp
* Remove getting started guide which is now on learn.hashicorp
* Corrects link to `consul/io` which should go to `consul.io`
* Revert "Remove getting started guide which is now on learn.hashicorp"
This reverts commit 2cebacf402f83fb936718b41ac9a27415f4e9f21 so a placeholder
message can be written here while we are transitioning content to
learn.hashicorp
* Adding a new page for getting started to direct users to learn.
* Added a note at the being of each doc to notify users about the temporary repo change.
* Revert "Added a note at the being of each doc to notify users about the temporary repo change."
This reverts commit 9a2a8781f9705028e4f53f758ef235e74b2b7198.
From conversation at https://github.com/hashicorp/consul/pull/4878
* Removes redirect from sample web.json demo file
* Removed typo
* Update the ACL API docs
* Add a CreateTime to the anon token
Also require acl:read permissions at least to perform rule translation. Don’t want someone DoSing the system with an open endpoint that actually does a bit of work.
* Fix one place where I was referring to id instead of AccessorID
* Add godocs for the API package additions.
* Minor updates: removed some extra commas and updated the acl intro paragraph
* minor tweaks
* Updated the language to be clearer
* Updated the language to be clearer for policy page
* I was also confused by that! Your updates are much clearer.
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Sounds much better.
Co-Authored-By: kaitlincarter-hc <43049322+kaitlincarter-hc@users.noreply.github.com>
* Updated sidebar layout and deprecated warning
* Fix partial rendering in service command (CLI) help
* Fix sample JSON to be a valid json for service registration
* Add missing id field to make the complete document complete.
This field was added back into the helm chart, but it was not added
back to the documentation. This adds it, then additionally fixes a
few typos in the same file.
* Implement CLI token cloning & special ID handling
* Update a couple CLI commands to take some alternative options.
* Document the CLI.
* Update the policy list and set-agent-token synopsis
Based on info from consul-helm issue 16, the formatting of the helm
chart value for joining an external cluster needs to be specified
as a yaml array. This updates the documentation to reflect this.
This PR is almost a complete rewrite of the ACL system within Consul. It brings the features more in line with other HashiCorp products. Obviously there is quite a bit left to do here but most of it is related docs, testing and finishing the last few commands in the CLI. I will update the PR description and check off the todos as I finish them over the next few days/week.
Description
At a high level this PR is mainly to split ACL tokens from Policies and to split the concepts of Authorization from Identities. A lot of this PR is mostly just to support CRUD operations on ACLTokens and ACLPolicies. These in and of themselves are not particularly interesting. The bigger conceptual changes are in how tokens get resolved, how backwards compatibility is handled and the separation of policy from identity which could lead the way to allowing for alternative identity providers.
On the surface and with a new cluster the ACL system will look very similar to that of Nomads. Both have tokens and policies. Both have local tokens. The ACL management APIs for both are very similar. I even ripped off Nomad's ACL bootstrap resetting procedure. There are a few key differences though.
Nomad requires token and policy replication where Consul only requires policy replication with token replication being opt-in. In Consul local tokens only work with token replication being enabled though.
All policies in Nomad are globally applicable. In Consul all policies are stored and replicated globally but can be scoped to a subset of the datacenters. This allows for more granular access management.
Unlike Nomad, Consul has legacy baggage in the form of the original ACL system. The ramifications of this are:
A server running the new system must still support other clients using the legacy system.
A client running the new system must be able to use the legacy RPCs when the servers in its datacenter are running the legacy system.
The primary ACL DC's servers running in legacy mode needs to be a gate that keeps everything else in the entire multi-DC cluster running in legacy mode.
So not only does this PR implement the new ACL system but has a legacy mode built in for when the cluster isn't ready for new ACLs. Also detecting that new ACLs can be used is automatic and requires no configuration on the part of administrators. This process is detailed more in the "Transitioning from Legacy to New ACL Mode" section below.
* agent/debug: add package for debugging, host info
* api: add v1/agent/host endpoint
* agent: add v1/agent/host endpoint
* command/debug: implementation of static capture
* command/debug: tests and only configured targets
* agent/debug: add basic test for host metrics
* command/debug: add methods for dynamic data capture
* api: add debug/pprof endpoints
* command/debug: add pprof
* command/debug: timing, wg, logs to disk
* vendor: add gopsutil/disk
* command/debug: add a usage section
* website: add docs for consul debug
* agent/host: require operator:read
* api/host: improve docs and no retry timing
* command/debug: fail on extra arguments
* command/debug: fixup file permissions to 0644
* command/debug: remove server flags
* command/debug: improve clarity of usage section
* api/debug: add Trace for profiling, fix profile
* command/debug: capture profile and trace at the same time
* command/debug: add index document
* command/debug: use "clusters" in place of members
* command/debug: remove address in output
* command/debug: improve comment on metrics sleep
* command/debug: clarify usage
* agent: always register pprof handlers and protect
This will allow us to avoid a restart of a target agent
for profiling by always registering the pprof handlers.
Given this is a potentially sensitive path, it is protected
with an operator:read ACL and enable debug being
set to true on the target agent. enable_debug still requires
a restart.
If ACLs are disabled, enable_debug is sufficient.
* command/debug: use trace.out instead of .prof
More in line with golang docs.
* agent: fix comment wording
* agent: wrap table driven tests in t.run()
In case `verify_server_hostname` is set in the configuration, Consul checks the certificate against `server.<datacenter>.<domain>`.
The name suggested by the guide generates errors like the following:
```
2018/10/10 12:42:20 [ERR] consul: Failed to confirm peer status for consul-3: rpc error getting client: failed to get conn: x509: certificate is valid for server.node.consul.labs, localhost, not server.consul.labs. Retrying in 16s...
```
Removing the `node` part from the certificate permits them to work also when that option is set.
* Add -enable-local-script-checks options
These options allow for a finer control over when script checks are enabled by
giving the option to only allow them when they are declared from the local
file system.
* Add documentation for the new option
* Nitpick doc wording
* Initial draft of Sidecar Service and Managed Proxy deprecation docs
* Service definition deprecation notices and sidecar service
* gRPC and sidecar service config options; Deprecate managed proxy options
* Envoy Docs: Basic envoy command; envoy getting started/intro
* Remove change that snuck in
* Envoy custom config example
* Add agent/service API docs; deprecate proxy config endpoint
* Misc grep cleanup for managed proxies; capitalize Envoy
* Updates to getting started guide
* Add missing link
* Refactor Envoy guide into a separate guide and add bootstrap reference notes.
* Add limitations to Envoy docs; Highlight no fixes for known managed proxy issues on deprecation page; clarify snake cae stuff; Sidecar Service lifecycle
* Refactor Service Definition ProxyDestination.
This includes:
- Refactoring all internal structs used
- Updated tests for both deprecated and new input for:
- Agent Services endpoint response
- Agent Service endpoint response
- Agent Register endpoint
- Unmanaged deprecated field
- Unmanaged new fields
- Managed deprecated upstreams
- Managed new
- Catalog Register
- Unmanaged deprecated field
- Unmanaged new fields
- Managed deprecated upstreams
- Managed new
- Catalog Services endpoint response
- Catalog Node endpoint response
- Catalog Service endpoint response
- Updated API tests for all of the above too (both deprecated and new forms of register)
TODO:
- config package changes for on-disk service definitions
- proxy config endpoint
- built-in proxy support for new fields
* Agent proxy config endpoint updated with upstreams
* Config file changes for upstreams.
* Add upstream opaque config and update all tests to ensure it works everywhere.
* Built in proxy working with new Upstreams config
* Command fixes and deprecations
* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...
TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.
* Fix translated keys in API registration.
≈
* Fixes from docs
- omit some empty undocumented fields in API
- Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.
* Documentation updates for Upstreams in service definition
* Fixes for tests broken by many refactors.
* Enable travis on f-connect branch in this branch too.
* Add consistent Deprecation comments to ProxyDestination uses
* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
* Add cache types for catalog/services and health/services and basic test that caching works
* Support non-blocking cache types with Cache-Control semantics.
* Update API docs to include caching info for every endpoint.
* Comment updates per PR feedback.
* Add note on caching to the 10,000 foot view on the architecture page to make the new data path more clear.
* Document prepared query staleness quirk and force all background requests to AllowStale so we can spread service discovery load across servers.
In designing a potential UI for a configuration of `enable_tag_override`,
I found the documentation confusing and lengthy. Here, I've made an
attempt at re-writing this section to be more concise and clear.
I also made a few small changes to the organization of this file to map
explanations to the order of the properties listing at the top. I find
it easier to scan docs when explanations appear in the same order they
are listed at the top. For explanations that span multiple paragraphs, I
provided a subheading, which also helps in linking from other pages.
Finally, I removed a duplicated paragraph from the documentation.
* Fill in gaps in semaphore guide
* Update to match that values come back b64 encoded
* Add that the value needs to be decoded
* Remove outdated reference to session1
* Fix some typos
* Clarify what is mean by a session having an active key
* Clarify requirements for lock holders in semaphore guide
* Adds Deployment Guide and update links
* Fixes releases link
* Re-organisation of content
* Cuts down "deployment" doc (which should focus on Reference Architecture) by moving raft and performance tuning to the Server Performance page which already covers some of this.
* Moves backups from "deployment" doc (which should focus on Reference Architecture) to "deployment-guide"
* Cleans up some notes and add single DC diagram
* Removes old link to deployment guide from nav
* Corrects minor styling, formatting, and grammar
* website: initial Kubernetes section with Helm information
* website: extraConfig for clients
* website: add more helm fields
* website: document extraVolumes
* website: document Consul DNS
* website: fix typos and show example of downward API
Fixes: #4578
Prior to this fix if there was an error binding to ports for the DNS servers the error would be swallowed by the gated log writer and never output. This fix propagates the DNS server errors back to the shell with a multierror.
* Implementation of Weights Data structures
Adding this datastructure will allow us to resolve the
issues #1088 and #4198
This new structure defaults to values:
```
{ Passing: 1, Warning: 0 }
```
Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.
* Implemented weights for DNS SRV Records
* DNS properly support agents with weight support while server does not (backwards compatibility)
* Use Warning value of Weights of 1 by default
When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.
Thus, we put a default value of 1 to be consistent with existing behaviour.
* Added documentation for new weight field in service description
* Better documentation about weights as suggested by @banks
* Return weight = 1 for unknown Check states as suggested by @banks
* Fixed typo (of -> or) in error message as requested by @mkeeler
* Fixed unstable unit test TestRetryJoin
* Fixed unstable tests
* Fixed wrong Fatalf format in `testrpc/wait.go`
* Added notes regarding DNS SRV lookup limitations regarding number of instances
* Documentation fixes and clarification regarding SRV records with weights as requested by @banks
* Rephrase docs
* Adds XL machine spec and notes on large deployments
* Clarifies machine sizes
* Fixes internal links within the document
* Moves datacenter size guidelines to "Single Datacenter" section
* Added log-file flag to capture Consul logs in a user specified file
* Refactored code.
* Refactored code. Added flags to rotate logs based on bytes and duration
* Added the flags for log file and log rotation on the webpage
* Fixed TestSantize from failing due to the addition of 3 flags
* Introduced changes : mutex, data-dir log writes, rotation logic
* Added test for logfile and updated the default log destination for docs
* Log name now uses UnixNano
* TestLogFile is now uses t.Parallel()
* Removed unnecessary int64Val function
* Updated docs to reflect default log name for log-file
* No longer writes to data-dir and adds .log if the filename has no extension
By default, the Consul agent listens on the local interface
at port 8500 for API requests. This change makes the API examples
using `curl` copy-pasteable for this default configuration.
* updates docs for agent options
trying to add a little more clarity to suggestion that folks should use
port 8302 for both LAN and WAN comms
* website: clarify language for serf wan port behavior
* New Providers added and updated vendoring for go-discover
* Vendor.json formatted using make vendorfmt
* Docs/Agent/auto-join: Added documentation for the new providers introduced in this PR
* Updated the golang.org/x/sys/unix in the vendor directory
* Agent: TestGoDiscoverRegistration updated to reflect the addition of new providers
* Deleted terraform.tfstate from vendor.
* Deleted terraform.tfstate.backup
Deleted terraform state file artifacts from unknown runs.
* Updated x/sys/windows vendor for Windows binary compilation
* Added explainations on how having a working configuration for Prometheus
Since Prometheus escapes by default query parameters, this paragraph explains
how having `format=prometheus` to be taken into account by prometheus.
* Rephrase Prometheus notes in documentation as requested by @pearkes
The GitHub repo for this library says that it is no longer maintained
and should not be used. The Ruby Diplomat library provides similar
functionality instead (and is already listed here).
The floating paragraph seems to need additional indentation to work correctly on the markdown parser middle man uses - GitHub got it right before in the preview but the website broke dumping the new config option inline.
Previous version of the documentation didn't mention this, which can
lead to confusion when experimenting with Connect. Many other features
of Consul work fine without `-dev` mode, but Connect needs certs in
order to proxy, which must be done with this flag or by generating certs
directly and passing them to Consul in configs.
The original snippet did not put the JSON output on its own line, which
made it look like it was part of the previous or next `curl` command.
This patch formats and highlights the command and output to appear as
they would in the terminal while also highlighting the multi-datacenter
elements of the output.
Original `discovery` snippet had a `curl` command that mentioned
multi-datacenter support. This removes part of the command that was
incorrect. It adds styling for the `dc2` section of the JSON output that
highlights the part of the query that relates to multiple data centers.
I was going to the public feedback on Connect-Native app integration and came across [this](https://twitter.com/relistan/status/1012263110403555329) thread. Added a few statements in the Connect-Native app integration page.
* website: add object-fit-image polyfill for ie compatibility
* website: make entire carousel cards linkable
* website: adjust hero background positioning
* website: responsive updates to the home hero
* website: update home hero js to work better with turbolinks
* website: a few font weight adjustments
* website: add an overlay to ui video
* update encryption doc and add guide for creating certificates in consul with cfssl
* add details about CLI and disabling HTTP
* delete $ symbols and add guide elements
* add missing periods and steps heading
No target for `$ make bootstrap` currently exists. If building from source you have to run `$ make` and then `$ make dev` to install one of the built binaries that is appropriate for your system.
Update docs a little
Update/add tests. Make sure all the various ways of determining the source IP work
Update X-Forwarded-For header parsing. This can be a comma separated list with the first element being the original IP so we now handle csv data there.
Got rid of error return from sourceAddrFromRequest
Currently the large image can load in Slack (or other
clients that expand links) and dominate the viewport.
This is just following the pattern from a recent
change to the Terraform website in https://github.com/hashicorp/terraform-website/pull/214
Previously, the documentation linked to Golang's source code, which
can drift from the list of cipher suites supported by Consul. Consul
has a hard-coded mapping of string values to Golang cipher suites, so
this is a more direct source of truth to help users understand which
string values are accepted in the `tls_cipher_suites` configuration
value.
This avoids a conflict with #datacenter later on the page. We're mixing
histroic manually specified anchors with generated anchors (via
redcarpet / middleman-hashicorp) so we have to manually override the
automatic generation here.
I was tempted to rewrite the old manual anchors to use the automatic
generation, but there is no way to maintain backwards compatibility,
so will leave that for a time when it is appropriate for us to break
links (or redirect them, etc).
Fixes#3916
These was spotted in issue #3709.
These two configuration elements were renamed "serf_lan" and "serf_wan" in commit 12216583a1
Update documentation to fit the new code.
Note that flags "-serf-lan-bind" and "-serf-wan-bind" were not renamed.
* config: refactor ReadPath(s) methods without side-effects
Return the sources instead of modifying the state.
* config: clean data dir before every test
* config: add tests for config-file and config-dir
* config: add -config-format option
Starting with Consul 1.0 all config files must have a '.json' or '.hcl'
extension to make it unambigous how the data should be parsed. Some
automation tools generate temporary files by appending a random string
to the generated file which obfuscates the extension and prevents the
file type detection.
This patch adds a -config-format option which can be used to override
the auto-detection behavior by forcing all config files or all files
within a config directory independent of their extension to be
interpreted as of this format.
Fixes#3620
This patch adds a /v1/coordinate/node/:node endpoint to get the network
coordinates for a single node in the network.
Since Consul Enterprise supports network segments it is still possible
to receive mutiple entries for a single node - one per segment.
DNS recursors can be added through go-sockaddr templates. Entries
are deduplicated while the order is maintained.
Originally proposed by @taylorchu
See #2932
* Updates the API client to support the current `ScriptArgs` parameter
for checks.
* Updates docs for checks to explain the `ScriptArgs` parameter issue.
* Adds mappings for "args" and "script-args" to give th API parity
with config.
* Adds checks on return codes.
* Removes debug logging that shows empty when args are used.
* Kill check processes after the timeout is reached
Kill the subprocess spawned by a script check once the timeout is reached. Previously Consul just marked the check critical and left the subprocess around.
Fixes#3565.
* Set err to non-nil when timeout occurs
* Fix check timeout test
* Kill entire process subtree on check timeout
* Add a docs note about windows subprocess termination
* agent: add option to discard health output
In high volatile environments consul will have checks with "noisy"
output which changes every time even though the status does not change.
Since the output is stored in the raft log every health check update
unblocks a blocking call on health checks since the raft index has
changed even though the status of the health checks may not have changed
at all. By discarding the output of the health checks the users can
choose a different tradeoff. Less visibility on why a check failed in
exchange for a reduced change rate on the raft log.
* agent: discard output also when adding a check
* agent: add test for discard check output
* agent: update docs
* go vet
* Adds discard_check_output to reloadable config table.
* Updates the change log.
* doc: document discrepancy between id and CheckID
* doc: document enable_tag_override change
* config: add TranslateKeys helper
TranslateKeys makes it easier to map between different representations
of internal structures. It allows to recursively map alias keys to
canonical keys in structured maps.
* config: use TranslateKeys for config file
This also adds support for 'enabletagoverride' and removes
the need for a separate CheckID alias field.
* config: remove dead code
* agent: use TranslateKeys for FixupCheckType
* agent: translate enable_tag_override during service registration
* doc: add '.hcl' as valid extension
* config: map ScriptArgs to args
* config: add comment for TranslateKeys
* Adds client-side retry for no leader errors.
This paves over the case where the client was connected to the leader
when it loses leadership.
* Adds a configurable server RPC drain time and a fail-fast path for RPCs.
When a server leaves it gets removed from the Raft configuration, so it will
never know who the new leader server ends up being. Without this we'd be
doomed to wait out the RPC hold timeout and then fail. This makes things fail
a little quicker while a sever is draining, and since we added a client retry
AND since the server doing this has already shut down and left the Serf LAN,
clients should retry against some other server.
* Makes the RPC hold timeout configurable.
* Reorders struct members.
* Sets the RPC hold timeout default for test servers.
* Bumps the leave drain time up to 5 seconds.
* Robustifies retries with a simpler client-side RPC hold.
* Reverts untended delete.
* Clean up handling of subprocesses and make using a shell optional
* Update docs for subprocess changes
* Fix tests for new subprocess behavior
* More cleanup of subprocesses
* Minor adjustments and cleanup for subprocess logic
* Makes the watch handler reload test use the new path.
* Adds check tests for new args path, and updates existing tests to use new path.
* Adds support for script args in Docker checks.
* Fixes the sanitize unit test.
* Adds panic for unknown watch type, and reverts back to Run().
* Adds shell option back to consul lock command.
* Adds shell option back to consul exec command.
* Adds shell back into consul watch command.
* Refactors signal forwarding and makes Windows-friendly.
* Adds a clarifying comment.
* Changes error wording to a warning.
* Scopes signals to interrupt and kill.
This avoids us trying to send SIGCHILD to the dead process.
* Adds an error for shell=false for consul exec.
* Adds notes about the deprecated script and handler fields.
* De-nests an if statement.
* config: provide stable config for /v1/agent/self (#3530)
This patch adds a stable subset of the previous Config struct to the
agent/self response. The actual runtime configuration is moved into
DebugConfig and will be documented to change.
Fixes#3530
* config: fix tests
* doc: update api documentation for /v1/agent/self
* Cleans up information about file extensions, now that they are required.
* Removes references to deprecated configuration options.
* Adds docs for multiple bind address support.
* metrics: replace statsite_prefix with service_prefix
The metrics prefix isn't statsite specific and is in fact used
for all metrics providers. Since we are deprecating fields
anyway we should fix this one as well.
Fixes#3293
* Updates docs and sorts telemetry section.
* Renames to "metrics_prefix" to disambiguate with Consul services.
* Updates the change log.
* agent: consolidate http method not allowed checks
This patch uses the error handling of the http handlers to handle HTTP
method not allowed errors across all available endpoints. It also adds a
test for testing whether the endpoints respond with the correct status
code.
* agent: do not panic on metrics tests
* agent: drop other tests for MethodNotAllowed
* agent: align /agent/join with reality
/agent/join uses PUT instead of GET as documented.
* agent: align /agent/check/{fail,warn,pass} with reality
/agent/check/{fail,warn,pass} uses PUT instead of GET as documented.
* fix some tests
* Drop more tests for method not allowed
* Align TestAgent_RegisterService_InvalidAddress with reality
* Changes API client join to use PUT instead of GET.
* Fixes agent endpoint verbs and removes obsolete tests.
* Updates the change log.
* Changes default Raft protocol to 3.
* Changes numPeers() to report only voters.
This should have been there before, but it's more obvious that this
is incorrect now that we default the Raft protocol to 3, which puts
new servers in a read-only state while Autopilot waits for them to
become healthy.
* Fixes TestLeader_RollRaftServer.
* Fixes TestOperator_RaftRemovePeerByAddress.
* Fixes TestServer_*.
Relaxed the check for a given number of voter peers and instead do
a thorough check that all servers see each other in their Raft
configurations.
* Fixes TestACL_*.
These now just check for Raft replication to be set up, and don't
care about the number of voter peers.
* Fixes TestOperator_Raft_ListPeers.
* Fixes TestAutopilot_CleanupDeadServerPeriodic.
* Fixes TestCatalog_ListNodes_ConsistentRead_Fail.
* Fixes TestLeader_ChangeServerID and adjusts the conn pool to throw away
sockets when it sees io.EOF.
* Changes version to 1.0.0 in the options doc.
* Makes metrics test more deterministic with autopilot metrics possible.
* new config parser for agent
This patch implements a new config parser for the consul agent which
makes the following changes to the previous implementation:
* add HCL support
* all configuration fragments in tests and for default config are
expressed as HCL fragments
* HCL fragments can be provided on the command line so that they
can eventually replace the command line flags.
* HCL/JSON fragments are parsed into a temporary Config structure
which can be merged using reflection (all values are pointers).
The existing merge logic of overwrite for values and append
for slices has been preserved.
* A single builder process generates a typed runtime configuration
for the agent.
The new implementation is more strict and fails in the builder process
if no valid runtime configuration can be generated. Therefore,
additional validations in other parts of the code should be removed.
The builder also pre-computes all required network addresses so that no
address/port magic should be required where the configuration is used
and should therefore be removed.
* Upgrade github.com/hashicorp/hcl to support int64
* improve error messages
* fix directory permission test
* Fix rtt test
* Fix ForceLeave test
* Skip performance test for now until we know what to do
* Update github.com/hashicorp/memberlist to update log prefix
* Make memberlist use the default logger
* improve config error handling
* do not fail on non-existing data-dir
* experiment with non-uniform timeouts to get a handle on stalled leader elections
* Run tests for packages separately to eliminate the spurious port conflicts
* refactor private address detection and unify approach for ipv4 and ipv6.
Fixes#2825
* do not allow unix sockets for DNS
* improve bind and advertise addr error handling
* go through builder using test coverage
* minimal update to the docs
* more coverage tests fixed
* more tests
* fix makefile
* cleanup
* fix port conflicts with external port server 'porter'
* stop test server on error
* do not run api test that change global ENV concurrently with the other tests
* Run remaining api tests concurrently
* no need for retry with the port number service
* monkey patch race condition in go-sockaddr until we understand why that fails
* monkey patch hcl decoder race condidtion until we understand why that fails
* monkey patch spurious errors in strings.EqualFold from here
* add test for hcl decoder race condition. Run with go test -parallel 128
* Increase timeout again
* cleanup
* don't log port allocations by default
* use base command arg parsing to format help output properly
* handle -dc deprecation case in Build
* switch autopilot.max_trailing_logs to int
* remove duplicate test case
* remove unused methods
* remove comments about flag/config value inconsistencies
* switch got and want around since the error message was misleading.
* Removes a stray debug log.
* Removes a stray newline in imports.
* Fixes TestACL_Version8.
* Runs go fmt.
* Adds a default case for unknown address types.
* Reoders and reformats some imports.
* Adds some comments and fixes typos.
* Reorders imports.
* add unix socket support for dns later
* drop all deprecated flags and arguments
* fix wrong field name
* remove stray node-id file
* drop unnecessary patch section in test
* drop duplicate test
* add test for LeaveOnTerm and SkipLeaveOnInt in client mode
* drop "bla" and add clarifying comment for the test
* split up tests to support enterprise/non-enterprise tests
* drop raft multiplier and derive values during build phase
* sanitize runtime config reflectively and add test
* detect invalid config fields
* fix tests with invalid config fields
* use different values for wan sanitiziation test
* drop recursor in favor of recursors
* allow dns_config.udp_answer_limit to be zero
* make sure tests run on machines with multiple ips
* Fix failing tests in a few more places by providing a bind address in the test
* Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder.
* Add porter to server_test.go to make tests there less flaky
* go fmt
* Updating maint.html to remove node maint
as discussed in #3268 maint mode does not have any affect on the node health.
* Update maint.html.markdown.erb
* Added rate limiting for agent RPC calls.
* Initializes the rate limiter based on the config.
* Adds the rate limiter into the snapshot RPC path.
* Adds unit tests for the RPC rate limiter.
* Groups the RPC limit parameters under "limits" in the config.
* Adds some documentation about the RPC limiter.
* Sends a 429 response when the rate limiter kicks in.
* Adds docs for new telemetry.
* Makes snapshot telemetry look like RPC telemetry and cleans up comments.
* Remove people from community section
This is going to be replaced with dynamic content from our CMS in the
future, but we agreed to remove it in the interim.
* Update deploy process
* Exit 2 if -child-exit-code and the child returned with an error.
* There is no platform independent way to check the exact return code of
* the child, so on error always return 2.
* Closes#947
* Closes#1503
* Update documentation for consul exec to mention all permissions it needs.
* Removed node write perm, not needed for consul exec
* Converts ACL note into a table like the API docs, adds ACL guide x-ref.
* Changes remote exec KV read to call GetTokenForAgent(), which can use
the acl_agent_token instead of the acl_token.
Fixes#3160.
* Fixes remote exec unit test with ACLs.
* Adds unhappy ACL path to unit tests for remote exec.
* Moves magic check and service constants into shared structs package.
* Removes the "consul" service from local state.
Since this service is added by the leader, it doesn't really make sense to
also keep it in local state (which requires special ACLs to configure), and
requires a bunch of special cases in the local state logic. This requires
fewer special cases and makes ACL bootstrapping cleaner.
* Makes coordinate update ACL log message a warning, similar to other AE warnings.
* Adds much more detailed examples for bootstrapping ACLs.
This can hopefully replace https://gist.github.com/slackpad/d89ce0e1cc0802c3c4f2d84932fa3234.
This patch adds an "http_config" object to the config file
and moves the "http_api_response_headers" option there.
"http_api_response_headers" is now deprecated in favor of
"http_config.response_headers"
Fix stale reads on server startup. Consistent reads will now wait for up to config.RPCHoldTimeout for the server to get past its raft log, before returning an error. Servers that are starting up will eventually catch up.
This fixes issue #2644
This patch adds support for a custom check id and name when
registering a service.
This is achieved by adding a CheckID and a Name field to the
CheckType structure which is used to register checks with a
service and when returning health check definitions.
CheckDefinition is a superset of CheckType which duplicates
some of the fields of CheckType. This patch decouples these
two structures by removing the embedding of CheckType in
CheckDefinition.
Fixes#3047
This patch stores the datacenter of a node in memdb
and exposes it via the API in all places where a Node
structure is returned.
* /catalog/nodes
* /catalog/node/:node
* /catalog/service/:service
* /health/service/:service
* /query/:uuid/execute
Rebecca Zanzig