Commit Graph

449 Commits

Author SHA1 Message Date
Michael Schurter c82b14b0c4 core: add limits to unauthorized connections
Introduce limits to prevent unauthorized users from exhausting all
ephemeral ports on agents:

 * `{https,rpc}_handshake_timeout`
 * `{http,rpc}_max_conns_per_client`

The handshake timeout closes connections that have not completed the TLS
handshake by the deadline (5s by default). For RPC connections this
timeout also separately applies to first byte being read so RPC
connections with TLS enabled have `rpc_handshake_time * 2` as their
deadline.

The connection limit per client prevents a single remote TCP peer from
exhausting all ephemeral ports. The default is 100, but can be lowered
to a minimum of 26. Since streaming RPC connections create a new TCP
connection (until MultiplexV2 is used), 20 connections are reserved for
Raft and non-streaming RPCs to prevent connection exhaustion due to
streaming RPCs.

All limits are configurable and may be disabled by setting them to `0`.

This also includes a fix that closes connections that attempt to create
TLS RPC connections recursively. While only users with valid mTLS
certificates could perform such an operation, it was added as a
safeguard to prevent programming errors before they could cause resource
exhaustion.
2020-01-30 10:38:25 -08:00
Mahmood Ali 8e78fb15bf Update go-discover library
To pick up go.net change so we can cross compile to arm64
2020-01-08 14:49:04 -05:00
Mahmood Ali 0ec9532ab1
Merge pull request #6831 from hashicorp/add_inmemory_certificate
Add option to set certificate in-memory
2019-12-19 08:54:32 -05:00
Drew Bailey 31777c3fdb
update go-hclog to latest release 2019-12-18 12:53:33 -05:00
Michel Vocks 3864d91d03 Add option to set certificate in-memory via SDK 2019-12-16 10:59:27 +01:00
Mahmood Ali a7361612b6
Merge pull request #6556 from hashicorp/c-vendor-multierror-20191025
Update go-multierror library
2019-12-13 11:32:42 -05:00
Mahmood Ali ea30ab9c56 Update go-multierror to 72917a1
To pick up https://github.com/hashicorp/go-multierror/pull/28
2019-12-13 10:13:31 -05:00
Mahmood Ali a7d8513ced vendor: Update go-discover for AWS SDK change
Update go-discover to pull in latest AWS SDK change through its vendored
path, along with other provider changes.
2019-12-03 13:49:52 -05:00
Mahmood Ali 3f15326248 Update AWS SDK library to v1.25.41
Update github.com/aws/aws-sdk-go and github.com/hashicorp/go-discover to
pick up support for EC2 Metadata Instance Service v2 changes.

Follow up to https://github.com/hashicorp/go-discover/pull/128 .
2019-12-03 13:35:03 -05:00
Michael Schurter 98321f9729 vendor: update go-version to include NewSemver 2019-11-19 08:40:18 -08:00
Drew Bailey 37de02c8f7
updates consul template deps to v0.22.1
pin to v0.22.1
2019-11-14 08:58:34 -05:00
Charlie Voiselle 835831a3d8 Added service wrapper code (#6220)
This is the basic code to add the Windows Service Manager hooks to Nomad.

Includes vendoring golang.org/x/sys/windows/svc and added Docs:
* guide for installing as a windows service.
* configuration for logging to file from PR #6429
2019-11-11 15:16:07 -05:00
Drew Bailey da6229d704
update go-hclog dep
remove duplicate lock
2019-11-05 09:51:52 -05:00
Drew Bailey 3b9c33a5f0
new hclog with standardlogger intercept 2019-11-05 09:51:49 -05:00
Drew Bailey 786989dbe3
New monitor pkg for shared monitor functionality
Adds new package that can be used by client and server RPC endpoints to
facilitate monitoring based off of a logger

clean up old code

small comment about write

rm old comment about minsize

rename to Monitor

Removes connection logic from monitor command

Keep connection logic in endpoints, use a channel to send results from
monitoring

use new multisink logger and interfaces

small test for dropped messages

update go-hclogger and update sink/intercept logger interfaces
2019-11-05 09:51:49 -05:00
Drew Bailey 976c43157c
remove log_writer
prefix output with proper spacing

update gzip handler, adjust first byte flow to allow gzip handler bypass

wip, first stab at wiring up rpc endpoint
2019-11-05 09:51:48 -05:00
Drew Bailey 0de94466b2
Display error when remote side ended monitor
multisink logger

remove usage of logwriter
2019-11-05 09:51:48 -05:00
Drew Bailey f60e44afc7
Adds nomad monitor command
Adds nomad monitor command. Like consul monitor, this command allows you
to stream logs from a nomad agent in real time with a a specified log
level

add endpoint tests

Upgrade go-hclog to latest version

The current version of go-hclog pads log prefixes to equal lengths
so info becomes [INFO ] and debug becomes [DEBUG]. This breaks
hashicorp/logutils/level.go Check function. Upgrading to the latest
version removes this padding and fixes log filtering that uses logutils
Check
2019-11-05 09:51:47 -05:00
Drew Bailey b0184e2032
Adds AgentMonitor Endpoint
AgentMonitor is an endpoint to stream logs for a given agent. It allows
callers to pass in a supplied log level, which may be different than the
agents config allowing for temporary debugging with lower log levels.

Pass in logWriter when setting up Agent
2019-11-05 09:51:46 -05:00
Drew Bailey 8c3e1b382a
Update gziphandler to latest version
This version of gziphandler includes a fix that fixes GzipResponseWriter
to implement CloseNotifier

https://github.com/nytimes/gziphandler/pull/63
2019-11-05 09:51:46 -05:00
Mahmood Ali 1f267bad58 Update go-multierror library
Update multierror to latest as of now.  Our version is very old and
dates back to Sep 2015[1].  Here, we aim to pick up a panic fix found in
n https://github.com/hashicorp/go-multierror/pull/11 (Dec 2016).

This is a purely hygiene maintenance change.  I am unaware of any causes
of the panic in our current dependencies.  Though, some private internal
libraries did rely on the "recent" behavior of go-multierror, and I
aimed to update here to ease our adoption of other libraries later.

[1] d30f09973e
2019-10-25 09:25:42 -04:00
Mahmood Ali e1fbe94457 upgrade go-plugin to latest, 8091134
Upgrade go-plugin to latest to pick up Windows fix in
https://github.com/hashicorp/go-plugin/pull/125 .
2019-10-04 13:26:49 -04:00
Mahmood Ali 4c548b9c75 Fix hashicorp/go-msgpack import
Golang 1.13 is pickier with importpaths and aliasing and fails
compilation currently.

Here, for go-msgpack dependency, we use upstream ugorji/go with a single
change
23165f7bc3
.

For consistency and to ease noticing descripency, I made ugorji/go and
hashicorp/go-msgpack reference the same sha.

This is a dependency management update and has no functional change to
product.
2019-09-27 09:08:30 -04:00
Nick Ethier e440ba80f1
ar: refactor network bridge config to use go-cni lib (#6255)
* ar: refactor network bridge config to use go-cni lib

* ar: use eth as the iface prefix for bridged network namespaces

* vendor: update containerd/go-cni package

* ar: update network hook to use TODO contexts when calling configurator

* unnecessary conversion
2019-09-04 16:33:25 -04:00
Lang Martin a1936e3add
Merge pull request #6215 from hashicorp/f-upgrade-go-getter
upgrade go-getter, leave compiled protobuf at version 1.2
2019-08-28 11:01:31 -04:00
Nick Ethier cf014c7fd5
ar: ensure network forwarding is allowed for bridged allocs (#6196)
* ar: ensure network forwarding is allowed in iptables for bridged allocs

* ensure filter rule exists at setup time
2019-08-28 10:51:34 -04:00
Lang Martin 5ae153900f match pinned versions for sub-modules 2019-08-27 12:58:12 -04:00
Lang Martin c79eb24816 govendor fetch github.com/hashicorp/go-getter@f5101da, protobuf 1.2 2019-08-26 17:54:21 -04:00
Mahmood Ali 3791a70aa9
Merge pull request #5676 from hashicorp/f-b-upgrade-ugorji-dep-20190508
Update ugorji/go to latest
2019-08-23 18:29:49 -04:00
Lang Martin f807d9208f govendor fetch github.com/hashicorp/go-getter@6be654f 2019-08-23 15:59:03 -04:00
Chris Baker ee0bfe1926
Merge pull request #6152 from dwagin/master
Bump up github.com/hashicorp/go-syslog for better BSD support
2019-08-20 14:22:03 -04:00
Dmitry edde02c242 updated vendor/vendor.json 2019-08-20 13:34:36 +03:00
Dmitry d313ccaf93 added version github.com/hashicorp/go-syslog 2019-08-20 13:17:08 +03:00
Michael Schurter 7b2d5f601f vendor: update testify to v1.4.0 2019-08-19 15:36:04 -07:00
Dmitry f99f6d7384 Bump up github.com/hashicorp/go-syslog 2019-08-19 12:17:01 +03:00
Tim Gross bf8b27c4c7 update consul-template to latest version
pulls in configuration option for blacklisting template functions from:
https://github.com/hashicorp/consul-template/pull/1243
https://github.com/hashicorp/consul-template/pull/1246

pulls in configuration option for file sandboxing from:
https://github.com/hashicorp/consul-template/pull/1249
https://github.com/hashicorp/consul-template/pull/1254

pulls in vault KVv2 read fixes from:
https://github.com/hashicorp/consul-template/pull/1253
2019-08-12 16:34:48 -04:00
Michael Schurter dbd8badf31
Merge pull request #6009 from hmalphettes/f-update-go-getter
Update go-getter@1.1.0 to fix #3491
2019-08-01 12:59:09 -07:00
Nick Ethier 4433dc8435
vendor: add cni libs 2019-07-31 01:04:07 -04:00
Michael Schurter 5081e3f8a8
vendor: update consul/api@v1.5.1 for connect fields 2019-07-31 01:04:05 -04:00
Nick Ethier ab84630132
plugin/driver: fix tests and add new dep to vendor 2019-07-31 01:03:17 -04:00
hmalphettes bd904f8fe0
Update go-getter to v1.1.0 2019-07-26 06:54:23 +08:00
Mahmood Ali cd6f1d3102 Update consul-template dependency to latest
To pick up the fix in
https://github.com/hashicorp/consul-template/pull/1231 .
2019-07-18 07:32:03 +07:00
Danielle c6872cdf12
Merge pull request #5864 from hashicorp/dani/win-pipe-cleaner
windows: Fix restarts using the raw_exec driver
2019-07-02 13:58:56 +02:00
Danielle Lancashire 0ff27cfc0f
vendor: Use dani fork of go-winio 2019-06-28 13:47:18 +02:00
Chris Baker 0436f70975
Merge branch 'master' into b-5345-prometheus-metric-label-conflict 2019-06-19 12:50:03 -04:00
Mahmood Ali 4c3798c82a
Merge pull request #5844 from hashicorp/b-hcl-parse-unknown-vars
Upgrade hcl2 to validate arrays for unknown values
2019-06-19 10:44:21 -04:00
Mahmood Ali 0c8a1623f3 update logrus and go-windows-terminal-sequences 2019-06-18 14:55:19 -04:00
Mahmood Ali 3dd3195d45 Update github.com/opencontainers/selinux
Or github.com/opencontainers/selinux/go-selinux, a dependency of
libcontainer.
2019-06-18 14:49:11 -04:00
Mahmood Ali 0848f577bb Update libcontainer to 6cc5158
To pick up the fix in https://github.com/opencontainers/runc/pull/2023

Fixes https://github.com/hashicorp/nomad/issues/5836
2019-06-18 14:48:13 -04:00
Chris Baker 50b491564d metrics: upgraded prometheus http client to 0.9.4 to address label conflict in Nomad 0.9.x reported in #5345 2019-06-18 18:34:22 +00:00