Commit Graph

877 Commits

Author SHA1 Message Date
Aestek 260a9880ae [Security] Add finer control over script checks (#4715)
* Add -enable-local-script-checks options

These options allow for a finer control over when script checks are enabled by
giving the option to only allow them when they are declared from the local
file system.

* Add documentation for the new option

* Nitpick doc wording
2018-10-11 13:22:11 +01:00
Paul Banks 94332edf2e
[WIP] Initial draft of Sidecar Service and Managed Proxy deprecation docs (#4752)
* Initial draft of Sidecar Service and Managed Proxy deprecation docs

* Service definition deprecation notices and sidecar service

* gRPC and sidecar service config options; Deprecate managed proxy options

* Envoy Docs: Basic envoy command; envoy getting started/intro

* Remove change that snuck in

* Envoy custom config example

* Add agent/service API docs; deprecate proxy config endpoint

* Misc grep cleanup for managed proxies; capitalize Envoy

* Updates to getting started guide

* Add missing link

* Refactor Envoy guide into a separate guide and add bootstrap reference notes.

* Add limitations to Envoy docs; Highlight no fixes for known managed proxy issues on deprecation page; clarify snake cae stuff; Sidecar Service lifecycle
2018-10-11 10:44:42 +01:00
Pierre Souchay 42f250fa53 Added SOA configuration for DNS settings. (#4714)
This will allow to fine TUNE SOA settings sent by Consul in DNS responses,
for instance to be able to control negative ttl.

Will fix: https://github.com/hashicorp/consul/issues/4713

# Example

Override all settings:

* min_ttl: 0 => 60s
* retry: 600 (10m) => 300s (5 minutes),
* expire: 86400 (24h) => 43200 (12h)
* refresh: 3600 (1h) => 1800 (30 minutes)

```
consul agent -dev -hcl 'dns_config={soa={min_ttl=60,retry=300,expire=43200,refresh=1800}}'
```

Result:
```
dig +multiline @localhost -p 8600 service.consul

; <<>> DiG 9.12.1 <<>> +multiline @localhost -p 8600 service.consul
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 36557
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;service.consul.		IN A

;; AUTHORITY SECTION:
consul.			0 IN SOA ns.consul. hostmaster.consul. (
				1537959133 ; serial
				1800       ; refresh (30 minutes)
				300        ; retry (5 minutes)
				43200      ; expire (12 hours)
				60         ; minimum (1 minute)
				)

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed Sep 26 12:52:13 CEST 2018
;; MSG SIZE  rcvd: 93
```
2018-10-10 15:50:56 -04:00
Kyle Havlovitz 0cbd176a48 connect/ca: more OSS split for multi-dc 2018-10-10 12:17:59 -07:00
Kyle Havlovitz 475afd0300 docs: deprecate acl_datacenter and replace it with primary_datacenter 2018-10-10 12:16:47 -07:00
Paul Banks 92fe8c8e89 Add Proxy Upstreams to Service Definition (#4639)
* Refactor Service Definition ProxyDestination.

This includes:
 - Refactoring all internal structs used
 - Updated tests for both deprecated and new input for:
   - Agent Services endpoint response
   - Agent Service endpoint response
   - Agent Register endpoint
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Register
     - Unmanaged deprecated field
     - Unmanaged new fields
     - Managed deprecated upstreams
     - Managed new
   - Catalog Services endpoint response
   - Catalog Node endpoint response
   - Catalog Service endpoint response
 - Updated API tests for all of the above too (both deprecated and new forms of register)

TODO:
 - config package changes for on-disk service definitions
 - proxy config endpoint
 - built-in proxy support for new fields

* Agent proxy config endpoint updated with upstreams

* Config file changes for upstreams.

* Add upstream opaque config and update all tests to ensure it works everywhere.

* Built in proxy working with new Upstreams config

* Command fixes and deprecations

* Fix key translation, upstream type defaults and a spate of other subtele bugs found with ned to end test scripts...

TODO: tests still failing on one case that needs a fix. I think it's key translation for upstreams nested in Managed proxy struct.

* Fix translated keys in API registration.
≈

* Fixes from docs
 - omit some empty undocumented fields in API
 - Bring back ServiceProxyDestination in Catalog responses to not break backwards compat - this was removed assuming it was only used internally.

* Documentation updates for Upstreams in service definition

* Fixes for tests broken by many refactors.

* Enable travis on f-connect branch in this branch too.

* Add consistent Deprecation comments to ProxyDestination uses

* Update version number on deprecation notices, and correct upstream datacenter field with explanation in docs
2018-10-10 16:55:34 +01:00
Hannah Oppenheimer 29170340b3 Clarify enable_tag_override explanation
In designing a potential UI for a configuration of `enable_tag_override`,
I found the documentation confusing and lengthy. Here, I've made an
attempt at re-writing this section to be more concise and clear.

I also made a few small changes to the organization of this file to map
explanations to the order of the properties listing at the top. I find
it easier to scan docs when explanations appear in the same order they
are listed at the top. For explanations that span multiple paragraphs, I
provided a subheading, which also helps in linking from other pages.

Finally, I removed a duplicated paragraph from the documentation.
2018-10-09 09:55:44 -05:00
Chris Hoffman e7820bc9cb typo: missing space 2018-10-05 11:33:28 -05:00
danielehc 20ae959be8 Update options.html.md (#4753) 2018-10-04 19:28:26 +01:00
Mitchell Hashimoto 62ceacde67 website: document k8s go-discover (#4666)
This adds documentation for the `k8s` go-discover provider that will be part of 1.2.3.
2018-09-13 10:12:27 -04:00
Pierre Souchay 473e589d86 Implementation of Weights Data structures (#4468)
* Implementation of Weights Data structures

Adding this datastructure will allow us to resolve the
issues #1088 and #4198

This new structure defaults to values:
```
   { Passing: 1, Warning: 0 }
```

Which means, use weight of 0 for a Service in Warning State
while use Weight 1 for a Healthy Service.
Thus it remains compatible with previous Consul versions.

* Implemented weights for DNS SRV Records

* DNS properly support agents with weight support while server does not (backwards compatibility)

* Use Warning value of Weights of 1 by default

When using DNS interface with only_passing = false, all nodes
with non-Critical healthcheck used to have a weight value of 1.
While having weight.Warning = 0 as default value, this is probably
a bad idea as it breaks ascending compatibility.

Thus, we put a default value of 1 to be consistent with existing behaviour.

* Added documentation for new weight field in service description

* Better documentation about weights as suggested by @banks

* Return weight = 1 for unknown Check states as suggested by @banks

* Fixed typo (of -> or) in error message as requested by @mkeeler

* Fixed unstable unit test TestRetryJoin

* Fixed unstable tests

* Fixed wrong Fatalf format in `testrpc/wait.go`

* Added notes regarding DNS SRV lookup limitations regarding number of instances

* Documentation fixes and clarification regarding SRV records with weights as requested by @banks

* Rephrase docs
2018-09-07 15:30:47 +01:00
Siva Prasad 59dea9a31f
Adds a new command line flag -log-file for file based logging. (#4581)
* Added log-file flag to capture Consul logs in a user specified file

* Refactored code.

* Refactored code. Added flags to rotate logs based on bytes and duration

* Added the flags for log file and log rotation on the webpage

* Fixed TestSantize from failing due to the addition of 3 flags

* Introduced changes : mutex, data-dir log writes, rotation logic

* Added test for logfile and updated the default log destination for docs

* Log name now uses UnixNano

* TestLogFile is now uses t.Parallel()

* Removed unnecessary int64Val function

* Updated docs to reflect default log name for log-file

* No longer writes to data-dir and adds .log if the filename has no extension
2018-08-29 16:56:58 -04:00
Freddy 31c5e19e2f
Correct rpc telemetry docs (#4602) 2018-08-28 16:38:22 -04:00
Jack Pearkes 5c993a3f5f
Clarification for serf_wan documentation (#4459)
* updates docs for agent options

trying to add a little more clarity to suggestion that folks should use
port 8302 for both LAN and WAN comms

* website: clarify language for serf wan port behavior
2018-08-02 10:25:25 -07:00
Jeff Escalante 60e1450606 fix a couple html errors (#4456) 2018-07-26 16:30:24 -07:00
Christie Koehler fb4a902ca3 docs: Update links to ttl health check endpoints. (#4208)
* docs: Update links to ttl health check endpoints.

* remove absolute URLs
2018-07-26 16:14:44 -07:00
Matt Keeler 5c7c58ed26
Gossip tuneables (#4444)
Expose a few gossip tuneables for both lan and wan interfaces

gossip_nodes
gossip_interval
probe_timeout
probe_interval
retransmit_mult
suspicion_mult
2018-07-26 11:39:49 -04:00
Paul Banks 25628f0e69
Add config option to disable HTTP printable char path check (#4442) 2018-07-26 13:53:39 +01:00
Kyle Havlovitz ecc02c6aee
Merge pull request #4400 from hashicorp/leaf-cert-ttl
Add configurable leaf cert TTL to Connect CA
2018-07-25 17:53:25 -07:00
Siva Prasad a5ebab63e7 Vendoring update for go-discover. (#4412)
* New Providers added and updated vendoring for go-discover

* Vendor.json formatted using make vendorfmt

* Docs/Agent/auto-join: Added documentation for the new providers introduced in this PR

* Updated the golang.org/x/sys/unix in the vendor directory

* Agent: TestGoDiscoverRegistration updated to reflect the addition of new providers

* Deleted terraform.tfstate from vendor.

* Deleted terraform.tfstate.backup

Deleted terraform state file artifacts from unknown runs.

* Updated x/sys/windows vendor for Windows binary compilation
2018-07-25 16:21:04 -07:00
Pierre Souchay 7bf22a5091 Added explainations on how having a working configuration for Prometheus (#4397)
* Added explainations on how having a working configuration for Prometheus

Since Prometheus escapes by default query parameters, this paragraph explains
how having `format=prometheus` to be taken into account by prometheus.

* Rephrase Prometheus notes in documentation as requested by @pearkes
2018-07-25 16:17:38 -07:00
Sergey Shepelev e6dea80797 website: typo in node meta additional section (#4429) 2018-07-25 13:32:36 -07:00
Paul Banks 17de36c36e
Allow config-file based Service Definitions for unmanaged proxies and Connect-natice apps. (#4443) 2018-07-25 19:55:41 +01:00
Paul Banks feeea60dea
Ooops that was meant to be to a branch no master... EMORECOFFEE
Revert "Add config option to disable HTTP printable char path check"

This reverts commit eebe45a47b4df5c0271b17f0fd1bd85db8bdefca.
2018-07-25 15:54:11 +01:00
Paul Banks d6c16dd0ad
Add config option to disable HTTP printable char path check 2018-07-25 15:52:37 +01:00
Paul Banks 186987874c
Merge pull request #4353 from azam/add-serf-lan-wan-port-args
Make RPC, Serf LAN, Serf WAN port configurable from CLI
2018-07-24 12:33:10 +01:00
Kyle Havlovitz a125735d76
connect/ca: check LeafCertTTL when rotating expired roots 2018-07-20 16:04:04 -07:00
Mitchell Hashimoto 5c42dacef4
Merge pull request #4320 from hashicorp/f-alias-check
Add "Alias" Check Type
2018-07-20 13:01:33 -05:00
azam 5290d69cb3 Make Serf LAN & WAN port configurable from CLI
Make RPC port accessible to CLI

Add tests and documentation for server-port, serf-lan-port, serf-wan-port CLI arguments
2018-07-21 02:17:21 +09:00
Paul Banks a3d1d4af2e
Merge pull request #4352 from hashicorp/doc-dev-flag-generates-certs
Note that `-dev` generates certs for Connect.
2018-07-17 11:36:02 +01:00
Geoffrey Grosenbach 934d4e4477 Clarify Connect-related implications for `-dev` flag 2018-07-16 14:53:46 -07:00
Kyle Havlovitz ce29660245
website: update docs for leaf cert TTL option 2018-07-16 13:33:42 -07:00
Mitchell Hashimoto 3ff378ec60
website: fix some website typos 2018-07-12 21:18:00 -07:00
Mitchell Hashimoto 77e30d4fa5
website: document alias check 2018-07-12 21:14:36 -07:00
Matt Keeler 22c5951ec4
Merge pull request #4303 from pierresouchay/non_blocking_acl
Only send one single ACL cache refresh across network when TTL is over
2018-07-10 08:57:33 -04:00
Paul Banks dae66b1afc
Merge pull request #4038 from pierresouchay/ACL_additional_info
Track calls blocked by ACLs using metrics
2018-07-09 20:21:21 +01:00
Paul Banks 488c147ade
Merge pull request #4351 from hashicorp/doc-config-dir-reads-hcl-files
Notes that both "hcl" and "json" files are loaded from the config-dir.
2018-07-09 14:43:55 +01:00
Paul Banks aa1c0e04f8
Fix formatting issue in config docs.
The floating paragraph seems to need additional indentation to work correctly on the markdown parser middle man uses - GitHub got it right before in the preview but the website broke dumping the new config option inline.
2018-07-09 14:39:20 +01:00
Pierre Souchay 74d3e64cd5 Added new ACL blocked Metrics to telemetry.html 2018-07-09 11:36:33 +02:00
Geoffrey Grosenbach 361db8ebcf Note that `-dev` generates certs for Connect.
Previous version of the documentation didn't mention this, which can
lead to confusion when experimenting with Connect. Many other features
of Consul work fine without `-dev` mode, but Connect needs certs in
order to proxy, which must be done with this flag or by generating certs
directly and passing them to Consul in configs.
2018-07-06 12:26:15 -07:00
Geoffrey Grosenbach cc24a65d45 Notes that both "hcl" and "json" files are loaded from the config-dir.
The previous version only mentioned "json" but the behavior was to read
"hcl" files as well.
2018-07-06 12:22:18 -07:00
Siva Prasad 911f2a51f0
Website/Docs/Telemetry : Errata 2018-07-03 18:18:57 -04:00
Siva 1c2e0cc2ef Website/Docs/Telemetry: Added more raft and memberlist items. 2018-07-03 17:37:29 -04:00
Siva 9e0914e904 Website/Telemetry: Errata for snapshot.compact and reworded memberlist.health.score 2018-07-03 10:59:31 -04:00
Siva 5d65cb8f2a Website: Added more telemetry details for raft and memberlist. 2018-07-03 10:27:01 -04:00
Siva 8cbfb0e5ed Website: Added more telemetry metrics 2018-07-03 08:00:39 -04:00
Pierre Souchay d8a6571683 Improve doc for async-cache 2018-07-01 23:50:30 +02:00
Pierre Souchay 6dfbbf1350 Updated documentation and adding more test case for async-cache 2018-07-01 23:50:30 +02:00
Siva 1522171e53 Website: Errata in -bind option for the agent 2018-06-29 16:43:12 -04:00
Julien Pivotto db2c4b3447
Fix JSON in service configuration 2018-06-27 09:06:27 +02:00
Matt Keeler 685838ea0e
Fix layout issue with discovery_max_stale
It was indented when it shouldn't have been.
2018-06-26 15:09:14 -04:00
mkeeler 1da3c42867 Merge remote-tracking branch 'connect/f-connect' 2018-06-25 19:42:51 +00:00
Kyle Havlovitz 837f23441d connect/ca: hide the RotationPeriod config field since it isn't used yet 2018-06-25 12:26:18 -07:00
Kyle Havlovitz 76aa137ffc website: add Vault CA provider doc sections 2018-06-25 12:25:42 -07:00
Paul Banks e3cbbf4eed Add proxy telemetry to docs 2018-06-25 12:25:39 -07:00
Mitchell Hashimoto 4fa92e7d0c website: document proxy security settings 2018-06-25 12:25:11 -07:00
Kyle Havlovitz e401384194 docs: add agent config options for connect/CA 2018-06-25 12:25:10 -07:00
Mitchell Hashimoto db72f1018c website: fix typo 2018-06-25 12:24:15 -07:00
Mitchell Hashimoto 61c7e33a22 agent/config: move ports to `ports` structure, update docs 2018-06-25 12:24:15 -07:00
Mitchell Hashimoto 8dbe0017bb Starting Docs (#46)
* website: first stab at Connect docs

* website: lots more various stuff (bad commit messages)

* website: getting started page for Connect

* website: intentions

* website: intention APIs

* website: agent API docs

* website: document agent/catalog proxy kind service values

* website: /v1/catalog/connect/:service

* website: intention CLI docs

* website: custom proxy docs

* website: remove dedicated getting started guide

* website: add docs for CA API endpoints

* website: add docs for connect ca commands

* website: add proxy CLI docs

* website: clean up proxy command, add dev docs

* website: todo pages

* website: connect security
2018-06-25 12:24:05 -07:00
Jack Pearkes e8adbdb59b
Merge pull request #4226 from petems/monitoring_doc
Adds Monitoring Guide
2018-06-21 13:34:11 -07:00
petems e91d2d2bcb Adds extra thresholds and aggregates to Telemetry
* Thresholds and aggregates of metrics for monitoring in Consul
2018-06-21 20:45:13 +01:00
Matt Keeler 6ccc4f39db
Merge pull request #4216 from hashicorp/rpc-limiting
Make RPC limits reloadable
2018-06-20 09:05:28 -04:00
Matt Keeler 426211fad6
Merge pull request #4215 from hashicorp/feature/config-node-meta-dns-txt
Add configuration entry to control including TXT records for node meta in DNS responses
2018-06-20 08:53:04 -04:00
Matt Keeler b9d1e7042a Make filtering out TXT RRs only apply when they would end up in Additional section
ANY queries are no longer affected.
2018-06-19 10:08:16 -04:00
Omar Khawaja da12d8a88c
update encryption doc and add guide for creating certificates (#4238)
* update encryption doc and add guide for creating certificates in consul with cfssl

* add details about CLI and disabling HTTP

* delete $ symbols and add guide elements

* add missing periods and steps heading
2018-06-18 15:25:35 -04:00
Guido Iaquinti edd6a69541 Client: add metric for failed RPC calls to server 2018-06-13 12:35:45 +01:00
Matt Keeler c5d9c2362f Merge branch 'master' of github.com:hashicorp/consul into rpc-limiting
# Conflicts:
#	agent/agent.go
#	agent/consul/client.go
2018-06-11 16:11:36 -04:00
Matt Keeler 35ffa2276c Update docs about rpc limits being reloadable 2018-06-11 15:54:55 -04:00
Matt Keeler 6604828009 Add configuration entry to control including TXT records for node meta in DNS responses
If set to false, the only way to retrieve TXT records for node meta is to specifically query for TXT records.
2018-06-11 11:49:04 -04:00
Jack Pearkes 78ef912427
Merge pull request #4169 from rskrishnar/adds-a-link-to-agent-config-options
adds link to agent config options from basic.html
2018-06-01 09:25:58 -07:00
Jack Pearkes f9fec514a9
Merge pull request #4164 from hashicorp/raft-default-docs
website: add raft snapshot setting defaults
2018-06-01 09:06:00 -07:00
Nainterceptor 096c112d06
Fix given + mandatory fields 2018-05-30 23:03:40 +02:00
RoopSai Krishna 7f1c139639 adds link to agent config options from basic.html 2018-05-28 12:47:14 +05:30
Kyle Havlovitz d3c4d2c495
website: add raft snapshot setting defaults 2018-05-25 16:17:43 -07:00
Jack Pearkes 701c00ca8f
Merge pull request #4133 from rskrishnar/add-missing-telemetry
add missing metrics to telemetry documentation
2018-05-25 09:25:19 -07:00
RoopSai Krishna 6054e0a268 add missing metrics to telemetry documentation 2018-05-16 16:26:32 +05:30
Pierre Souchay 77ed8004a7 Error in documentation: `discovery_max_stale` is not in `dns_config`
Moved the entry at the top level instead of DNS
2018-05-15 14:59:15 +02:00
Jack Pearkes 36f4c760b3 website: fix broken link 2018-05-11 12:22:55 -07:00
Preetha Appan 5d3b267787
More docs and removed SnapShotInterval from raft timing struct stanza 2018-05-11 10:43:24 -05:00
Preetha Appan e28c5fbb4e
Also make snapshot interval configurable 2018-05-11 10:43:24 -05:00
Preetha Appan eb4bc79118
Make raft snapshot commit threshold configurable 2018-05-11 10:43:24 -05:00
Kyle Havlovitz 67167fd961
Merge pull request #4108 from hashicorp/vendor-go-discover
Update go-discover and add triton provider
2018-05-10 17:29:00 -07:00
Kyle Havlovitz c04ff88537
Move cloud auto-join docs to a separate page and add Triton 2018-05-10 17:15:41 -07:00
Jack Pearkes e611b1728a
Merge pull request #4097 from hashicorp/remove-deprecated
Remove deprecated check/service fields and metric names
2018-05-10 15:45:49 -07:00
Paul Banks 01358a6043
Update config docs with places sockaddr is supported 2018-05-10 14:56:48 +01:00
Kyle Havlovitz 60307ef328
Remove deprecated metric names 2018-05-08 16:23:15 -07:00
Kyle Havlovitz a480434517
Remove the script field from checks in favor of args 2018-05-08 15:31:53 -07:00
Kyle Havlovitz 36c5e59465
Remove support for EnableTagOverride in config files 2018-05-07 16:19:13 -07:00
Kyle Havlovitz fb3cd87c91
Remove support for CheckID field in service check definitions 2018-05-07 16:15:08 -07:00
Paul Banks ea731031d5
Merge pull request #4047 from pierresouchay/added_missing_meta_in_service_definition
[BUGFIX] Added Service Meta support in configuration files
2018-04-25 13:08:53 +01:00
Pierre Souchay e0ee7cd40f Changed a bit doc
(Fighting with Travis)
2018-04-24 22:55:39 +02:00
Pierre Souchay 6be9b6ca60 Improved documentation 2018-04-24 22:40:41 +02:00
Pierre Souchay 11bf3e4c71 Added documentation for meta 2018-04-24 21:56:35 +02:00
Paul Banks 06e1a62653
Merge pull request #4016 from pierresouchay/support_for_prometheus
Support for prometheus for metrics endpoint
2018-04-24 16:14:43 +01:00
Jack Pearkes 890cf47acb website: fix a broken link 2018-04-13 10:33:29 -07:00
Pierre Souchay e342ced97b Clearer documentation and comments for enabling Prometheus support 2018-04-09 13:16:45 +02:00
Pierre Souchay 2cccb8f36a Added documentation for telemetry{ prometheus_retention_time = duration } 2018-04-06 14:50:03 +02:00
Preetha 8fbe3dfceb
Adds discovery_max_stale (#4004)
Adds a new option to allow service discovery endpoints to return stale results if configured at the agent level.
2018-03-30 10:14:44 -05:00
Jack Pearkes 307deb63f5 website: document and warn users about disabling serf_wan port
See https://github.com/hashicorp/consul/pull/3984 for more.

This also fixes a link to a heading that existed twice in the document.
2018-03-29 08:19:21 -07:00