This PR covers two sets of changes:
- Documenting the new `destination_peer` for proxy upstream definitions.
- Updating the exported-services config entry documentation.
Updates to the `exported-services` config entry include:
- As of 1.13.0 it is no longer only for Consul Enterprise
- A `PeerName` is now a possible consumer for an exported service.
- Added examples for OSS and Enterprise
- Linked to peering docs
* Docs - k8s - Webhook Certs on Vault
* Adding webhook certs to data-integration overview page
* marking items as code
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Updating prerequisites intro
* Updating prerequisites intro
* Updating `Create a Vault auth roles that link the policy to each Consul on Kubernetes service account that requires access` to `Link the Vault policy to Consul workloads`
* changing `Configure the Vault Kubernetes auth role in the Consul on Kubernetes helm chart` to `Update the Consul on Kubernetes helm chart`.
* Changed `Create a Vault PKI role that establishes the domains that it is allowed to issue certificates for` to `Configure allowed domains for PKI certificates`
* Moved `Create a Vault policy that authorizes the desired level of access to the secret` to the Set up per Consul Datacenter section
* Update website/content/docs/k8s/installation/vault/data-integration/webhook-certs.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Moving Overview above Prerequisites. Adding sentence where missing after page title.
* Moving Overview above Prerequisites for webhook certs page.
* fixing the end of the overview section that was not moved.
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Port some changes that were made to the backport branch but not in the original PR.
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
Change the `.tgz` file extension in the snapshot save and restore
examples on /api-docs/snapshot to `.snap`.
This is consistent with the file extension used in other example
snapshot save and restore commands, as well as the default extension
used by the Consul Snapshot Agent.
Path parameters, query parameters, and request body parameters are now shown in
separate sections rather than combined into one general parameters section.
This makes it much easier to understand quickly where a parameter should be
provided.
* update docs for single-dc-multi-k8s install
Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Modify node and service identities paragraphs on ACL index to better
conform with the style guide.
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Due to build changes in Consul 1.12.0 the `+ent` modifier is missing
from the version reported by `/v1/agent/self`.
Nomad looks for the `ent` modifier when determining whether to reconcile
services in non-default namespaces. Without the modifier Nomad will only
end up removing services from the default Consul namespace.
Commit 9333fad added JSON formatted examples for all ACL polices.
Most of these these examples are not valid JSON, and thus an error is
raised when attempting to create the example policies/rules in Consul.
This commit fixes the example JSON formatted ACL rules so that they
are valid JSON. This enables readers to use the policies as-is from
the documentation to successfully create policies in Consul.
It also removes unnecessary arrays from the example policies so that
the policies are easier for practitioners to read and write.
Remove empty CodeBlockConfig elements. These elements are not
providing any benefit for the enclosed code blocks. This PR removes
the elements so so that the source is easier to read.
* docs: Update Admin Partitions with more explicit commands by using shell variables
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
All Consul HTTP API endpoints now URL-decode resource names specified in the
path, enabling resource names containing URL-invalid characters to be used
in HTTP API requests if URL-encoded into the path.
Functionality implemented in Github Pull Requests:
- #11335
- #11957
- #12103
- #12190
- #12297
Also documents CLI accepting URL-invalid resource names.
All Consul HTTP API endpoints now URL-decode resource names specified in the
path, enabling resource names containing URL-invalid characters to be used
in HTTP API requests if URL-encoded into the path. The Consul HTTP API always
supported URL-decoding of query parameters.
The CLI automatically URL-encodes arguments which are inserted as resource
names in the URL path, enabling the CLI to also interact with resource names
that contain URL-invalid characters.
Changes include:
- Add diagrams of the operation of different consistency modes
- Note that only stale reads benefit from horizontal scaling
- Increase scannability with headings
- Document consistency mode defaults and how to override for
DNS and HTTP API interfaces
- Document X-Consul-Effective-Consistency response header
/docs/security/acl/acl-system was renamed in e9a42df from PR #12460 to
/docs/security/acl. A corresponding redirect was not added for this
page, resulting in a 404 being returned when accessing the old URL
path.
This commit redirects the former URL path to the new location, and
also updates all links on the site to point to the new location.
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* docs: Re-add config file content removed in PR #12562
Re-add agent config option content that was erroneously removed in #12562 with
commit f4c03d234.
* docs: Re-add CLI flag content removed in PR #12562
Re-add CLI flag content that was erroneously removed in #12562 with
commit c5220fd18.
* Update website/content/docs/agent/config/cli-flags.mdx
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
* Support vault namespaces in connect CA
Follow on to some missed items from #12655
From an internal ticket "Support standard "Vault namespace in the
path" semantics for Connect Vault CA Provider"
Vault allows the namespace to be specified as a prefix in the path of
a PKI definition, but our usage of the Vault API includes calls that
don't support a namespaced key. In particular the sys.* family of
calls simply appends the key, instead of prefixing the namespace in
front of the path.
Unfortunately it is difficult to reliably parse a path with a
namespace; only vault knows what namespaces are present, and the '/'
separator can be inside a key name, as well as separating path
elements. This is in use in the wild; for example
'dc1/intermediate-key' is a relatively common naming schema.
Instead we add two new fields: RootPKINamespace and
IntermediatePKINamespace, which are the absolute namespace paths
'prefixed' in front of the respective PKI Paths.
Signed-off-by: Mark Anderson <manderson@hashicorp.com>
* Adding documentation for WAN Federation with Vault as a secrets backend
* Reformatting systems integration
* fixing spacing and typos
* Fixing link to createFederactionSecret helm chart value
* More revisions in the Systems Integration section
* Systems Integration - fixing brok shell-session and adding paragraph.
* More formatting in data integration section
* Formatting consul config sections
* Fixing verbiage near helm installations.
* Changing refence to dc1 and dc2 to be primary datacenter(dc1) and secondary dataceneter (dc2)
* Apply suggestions from code review
Co-authored-by: Karl Cardenas <kcardenas@hashicorp.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
* fixing missing s in Kubernetes
* Providing reason in Usage section as to why someone would look at the Systems and Data Integration sections of the vault docs
* fixing highlighted linenumbers that got through off by deleting the comment line at the beginning.
* fixing indentation within order lists
* Add a validation step to the next steps section.
* making the data integration sections for dc1 and dc2 symmetrical
* PR Feedback
* Adding images
* Remove confusing references to Systems Integration and Data Integration pages.
* Updating images to be centered
* Removed confusing reference to federation secret.
Co-authored-by: Karl Cardenas <kcardenas@hashicorp.com>
Co-authored-by: David Yu <dyu@hashicorp.com>
Just like standard upstreams the order of applicability in descending precedence:
1. caller's `service-defaults` upstream override for destination
2. caller's `service-defaults` upstream defaults
3. destination's `service-resolver` ConnectTimeout
4. system default of 5s
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
* docs: Updating Gossip EncryptionKey Rotation page with Vault use case
* Adding a note to the vault instructions linking to the gossip key encryption using Vault page.
* Correcting Vault guide for storing the rotated gossip key.
* adding $ to shell sessions where it is missing on the gossip rotation page
* adding $ to more shell sessions where it is missing on the gossip rotation page
* Fixes a lint warning about t.Errorf not supporting %w
* Enable running autopilot on all servers
On the non-leader servers all they do is update the state and do not attempt any modifications.
* Fix the RPC conn limiting tests
Technically they were relying on racey behavior before. Now they should be reliable.
* k8s docs - ACLs refactor - Updating terminating gateway documentation to call out updating the role rather than the token with the policy
* Modifying role and policy names based on naming convention change.
The list of supported annotations for Consul service mesh were moved
from /docs/k8s/connect to /docs/k8s/annotations-and-labels in PR
#12323.
This commit updates various across the site to point to the new
URL for these annotations.
* Updating helm docs with additionalVault and ACLs refactor funtionality.
* PR Feedback corrections.
- Fix indentation.
- Fix description of secretName and secretKey to be consistent
- Change description of manageACLsRole to be more clear.
- Make the added vault role field descriptions consistent
* PR Feedback - correcting description for adminPartitionsRole
* Fixing broken shell sessions
* Fixing broken shell sessions by changing shell-session tobecloser tocomment marker
* add config watcher to the config package
* add logging to watcher
* add test and refactor to add WatcherEvent.
* add all API calls and fix a bug with recreated files
* add tests for watcher
* remove the unnecessary use of context
* Add debug log and a test for file rename
* use inode to detect if the file is recreated/replaced and only listen to create events.
* tidy ups (#1535)
* tidy ups
* Add tests for inode reconcile
* fix linux vs windows syscall
* fix linux vs windows syscall
* fix windows compile error
* increase timeout
* use ctime ID
* remove remove/creation test as it's a use case that fail in linux
* fix linux/windows to use Ino/CreationTime
* fix the watcher to only overwrite current file id
* fix linter error
* fix remove/create test
* set reconcile loop to 200 Milliseconds
* fix watcher to not trigger event on remove, add more tests
* on a remove event try to add the file back to the watcher and trigger the handler if success
* fix race condition
* fix flaky test
* fix race conditions
* set level to info
* fix when file is removed and get an event for it after
* fix to trigger handler when we get a remove but re-add fail
* fix error message
* add tests for directory watch and fixes
* detect if a file is a symlink and return an error on Add
* rename Watcher to FileWatcher and remove symlink deref
* add fsnotify@v1.5.1
* fix go mod
* do not reset timer on errors, rename OS specific files
* rename New func
* events trigger on write and rename
* add missing test
* fix flaking tests
* fix flaky test
* check reconcile when removed
* delete invalid file
* fix test to create files with different mod time.
* back date file instead of sleeping
* add watching file in agent command.
* fix watcher call to use new API
* add configuration and stop watcher when server stop
* add certs as watched files
* move FileWatcher to the agent start instead of the command code
* stop watcher before replacing it
* save watched files in agent
* add add and remove interfaces to the file watcher
* fix remove to not return an error
* use `Add` and `Remove` to update certs files
* fix tests
* close events channel on the file watcher even when the context is done
* extract `NotAutoReloadableRuntimeConfig` is a separate struct
* fix linter errors
* add Ca configs and outgoing verify to the not auto reloadable config
* add some logs and fix to use background context
* add tests to auto-config reload
* remove stale test
* add tests to changes to config files
* add check to see if old cert files still trigger updates
* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`
* fix to re add both key and cert file. Add test to cover this case.
* review suggestion
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* add check to static runtime config changes
* fix test
* add changelog file
* fix review comments
* Apply suggestions from code review
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* update flag description
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
* fix compilation error
* add static runtime config support
* fix test
* fix review comments
* fix log test
* Update .changelog/12329.txt
Co-authored-by: Dan Upton <daniel@floppy.co>
* transfer tests to runtime_test.go
* fix filewatcher Replace to not deadlock.
* avoid having lingering locks
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* split ReloadConfig func
* fix warning message
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* convert `FileWatcher` into an interface
* fix compilation errors
* fix tests
* extract func for adding and removing files
* add a coalesceTimer with a very small timer
* extract coaelsce Timer and add a shim for testing
* add tests to coalesceTimer fix to send remaining events
* set `coalesceTimer` to 1 Second
* support symlink, fix a nil deref.
* fix compile error
* fix compile error
* refactor file watcher rate limiting to be a Watcher implementation
* fix linter issue
* fix runtime config
* fix runtime test
* fix flaky tests
* fix compile error
* Apply suggestions from code review
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
* fix agent New to return an error if File watcher New return an error
* add a coalesceTimer with a very small timer
* extract coaelsce Timer and add a shim for testing
* set `coalesceTimer` to 1 Second
* add flag description to agent command docs
* fix link
* add Static runtime config docs
* fix links and alignment
* fix typo
* Revert "add a coalesceTimer with a very small timer"
This reverts commit d9db2fcb8213a81ac761f04b458091409c5fb1ee.
* Revert "extract coaelsce Timer and add a shim for testing"
This reverts commit 0ab86012a415ffeb452acf58e52c9f37c9f49254.
* Apply suggestions from code review
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
- `tls.incoming`: applies to the inbound mTLS targeting the public
listener on `connect-proxy` and `terminating-gateway` envoy instances
- `tls.outgoing`: applies to the outbound mTLS dialing upstreams from
`connect-proxy` and `ingress-gateway` envoy instances
Fixes#11966
* website: api-gateway helm install consul namespace
To mirror instructions at https://learn.hashicorp.com/tutorials/consul/kubernetes-api-gateway
* website(api-gateway): add notes on where to find available versions
* website(api-gateway): fixup link to more clearly indicate Consul Helm chart releases
* Update website/content/docs/api-gateway/api-gateway-usage.mdx
* tlsutil: initial implementation of types/TLSVersion
tlsutil: add test for parsing deprecated agent TLS version strings
tlsutil: return TLSVersionInvalid with error
tlsutil: start moving tlsutil cipher suite lookups over to types/tls
tlsutil: rename tlsLookup to ParseTLSVersion, add cipherSuiteLookup
agent: attempt to use types in runtime config
agent: implement b.tlsVersion validation in config builder
agent: fix tlsVersion nil check in builder
tlsutil: update to renamed ParseTLSVersion and goTLSVersions
tlsutil: fixup TestConfigurator_CommonTLSConfigTLSMinVersion
tlsutil: disable invalid config parsing tests
tlsutil: update tests
auto_config: lookup old config strings from base.TLSMinVersion
auto_config: update endpoint tests to use TLS types
agent: update runtime_test to use TLS types
agent: update TestRuntimeCinfig_Sanitize.golden
agent: update config runtime tests to expect TLS types
* website: update Consul agent tls_min_version values
* agent: fixup TLS parsing and compilation errors
* test: fixup lint issues in agent/config_runtime_test and tlsutil/config_test
* tlsutil: add CHACHA20_POLY1305 cipher suites to goTLSCipherSuites
* test: revert autoconfig tls min version fixtures to old format
* types: add TLSVersions public function
* agent: add warning for deprecated TLS version strings
* agent: move agent config specific logic from tlsutil.ParseTLSVersion into agent config builder
* tlsutil(BREAKING): change default TLS min version to TLS 1.2
* agent: move ParseCiphers logic from tlsutil into agent config builder
* tlsutil: remove unused CipherString function
* agent: fixup import for types package
* Revert "tlsutil: remove unused CipherString function"
This reverts commit 6ca7f6f58d268e617501b7db9500113c13bae70c.
* agent: fixup config builder and runtime tests
* tlsutil: fixup one remaining ListenerConfig -> ProtocolConfig
* test: move TLS cipher suites parsing test from tlsutil into agent config builder tests
* agent: remove parseCiphers helper from auto_config_endpoint_test
* test: remove unused imports from tlsutil
* agent: remove resolved FIXME comment
* tlsutil: remove TODO and FIXME in cipher suite validation
* agent: prevent setting inherited cipher suite config when TLS 1.3 is specified
* changelog: add entry for converting agent config to TLS types
* agent: remove FIXME in runtime test, this is covered in builder tests with invalid tls9 value now
* tlsutil: remove config tests for values checked at agent config builder boundary
* tlsutil: remove tls version check from loadProtocolConfig
* tlsutil: remove tests and TODOs for logic checked in TestBuilder_tlsVersion and TestBuilder_tlsCipherSuites
* website: update search link for supported Consul agent cipher suites
* website: apply review suggestions for tls_min_version description
* website: attempt to clean up markdown list formatting for tls_min_version
* website: moar linebreaks to fix tls_min_version formatting
* Revert "website: moar linebreaks to fix tls_min_version formatting"
This reverts commit 38585927422f73ebf838a7663e566ac245f2a75c.
* autoconfig: translate old values for TLSMinVersion
* agent: rename var for translated value of deprecated TLS version value
* Update agent/config/deprecated.go
Co-authored-by: Dan Upton <daniel@floppy.co>
* agent: fix lint issue
* agent: fixup deprecated config test assertions for updated warning
Co-authored-by: Dan Upton <daniel@floppy.co>
Introduces the capability to configure TLS differently for Consul's
listeners/ports (i.e. HTTPS, gRPC, and the internal multiplexed RPC
port) which is useful in scenarios where you may want the HTTPS or
gRPC interfaces to present a certificate signed by a well-known/public
CA, rather than the certificate used for internal communication which
must have a SAN in the form `server.<dc>.consul`.
* Update Kubernetes related YAML config examples to document supported
syntax in the latest version of the Helm chart.
* Fix syntax in JSON example configs.
Resolves#12403
Co-authored-by: mrspanishviking <kcardenas@hashicorp.com>
Recently there have been a handful of GitHub issues and Discuss posts
where users have expected the `consul` CLI to make use of config
options defined in the agent configuration files, and are confused
when it does not honor those config options.
This change clarifies that command-line and configuration file options
documented on the /agent/options page only apply to the Consul agent,
instead of the Consul CLI.
Nitya Dhanushkodi