Commit Graph

44 Commits

Author SHA1 Message Date
Michael Schurter d6d44a8214 test: assert fs endpoints are xss safe 2020-03-25 09:45:43 -04:00
Michael Schurter 63032917fc test: allow goroutine to exit even if test blocks 2020-02-07 15:50:53 -08:00
Mahmood Ali 88f8127820 tests: avoid using unnecessary pipe 2019-10-15 17:22:03 -04:00
Mahmood Ali 317e0f9e44 agent: report fs log errors as http errors
This fixes two bugs:

First, FS Logs API endpoint only propagated error back to user if it was
encoded with code, which isn't common.  Other errors get suppressed and
callers get an empty response with 200 error code.  Now, these endpoints
return  a 500 status code along with the error message.

Before
```
$ curl -v "http://127.0.0.1:4646/v1/client/fs/logs/qwerqwera?follow=false&offset=0&origin=start&region=global&task=redis&type=stdout"; echo
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 4646 (#0)
> GET /v1/client/fs/logs/qwerqwera?follow=false&offset=0&origin=start&region=global&task=redis&type=stdout HTTP/1.1
> Host: 127.0.0.1:4646
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Vary: Accept-Encoding
< Vary: Origin
< Date: Fri, 04 Oct 2019 19:47:21 GMT
< Content-Length: 0
<
* Connection #0 to host 127.0.0.1 left intact
```

After
```
$ curl -v "http://127.0.0.1:4646/v1/client/fs/logs/qwerqwera?follow=false&offset=0&origin=start&region=global&task=redis&type=stdout"; echo
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 4646 (#0)
> GET /v1/client/fs/logs/qwerqwera?follow=false&offset=0&origin=start&region=global&task=redis&type=stdout HTTP/1.1
> Host: 127.0.0.1:4646
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Vary: Accept-Encoding
< Vary: Origin
< Date: Fri, 04 Oct 2019 19:48:12 GMT
< Content-Length: 60
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host 127.0.0.1 left intact
alloc lookup failed: index error: UUID must be 36 characters
```

Second, we return 400 status code for request validation errors.

Before
```
$ curl -v "http://127.0.0.1:4646/v1/client/fs/logs/qwerqwera"; echo
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 4646 (#0)
> GET /v1/client/fs/logs/qwerqwera HTTP/1.1
> Host: 127.0.0.1:4646
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Vary: Accept-Encoding
< Vary: Origin
< Date: Fri, 04 Oct 2019 19:47:29 GMT
< Content-Length: 22
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host 127.0.0.1 left intact
must provide task name
```

After
```
$ curl -v "http://127.0.0.1:4646/v1/client/fs/logs/qwerqwera"; echo
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 4646 (#0)
> GET /v1/client/fs/logs/qwerqwera HTTP/1.1
> Host: 127.0.0.1:4646
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Vary: Accept-Encoding
< Vary: Origin
< Date: Fri, 04 Oct 2019 19:49:18 GMT
< Content-Length: 22
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host 127.0.0.1 left intact
must provide task name
```
2019-10-04 16:33:58 -04:00
Tim Gross 443ce3a831
api: add follow param to file stream endpoint (#6049)
The `/v1/client/fs/stream endpoint` supports tailing a file by writing
chunks out as they come in. But not all browsers support streams
(ex IE11) so we need to be able to tail a file without streaming.

The fs stream and logs endpoint use the same implementation for
filesystem streaming under the hood, but the fs stream always passes
the `follow` parameter set to true. This adds the same toggle to the
fs stream endpoint that we have for logs. It defaults to true for
backwards compatibility.
2019-08-01 08:32:43 -04:00
Michael Schurter 6bcf772f3c tests: test via ServeMux so http codes are set 2018-10-16 16:56:55 -07:00
Michael Schurter 1c9ccdeab5 tests: fix races caused by sharing a buffer
httptest.ResponseRecorder exposes a bytes.Buffer which we were reading
and writing concurrently to test streaming log APIs. This is a race, so
I wrapped the struct in a lock with some helpers.
2018-10-16 16:56:55 -07:00
Michael Schurter 39cef16c73 test: don't call t.Fatal from within a goroutine 2018-03-21 16:51:45 -07:00
Alex Dadgar aa98f8ba7b Enhance API pkg to utilize Server's Client Tunnel
This PR enhances the API package by having client only RPCs route
through the server when they are low cost and for filesystem access to
first attempt a direct connection to the node and then falling back to
a server routed request.
2018-02-15 13:59:03 -08:00
Alex Dadgar 14845bb918 Refactor determining the handler for a node id call 2018-02-15 13:59:02 -08:00
Alex Dadgar f5f43218f5 HTTP and tests 2018-02-15 13:59:02 -08:00
Alex Dadgar 8854b35b34 Agent logs 2018-02-15 13:59:02 -08:00
Alex Dadgar ca9379be09 Logs over RPC w/ lots to touch up 2018-02-15 13:59:01 -08:00
Michael Schurter 260b66bf52 Test all fs endpoints in a loop 2017-10-09 11:19:14 -07:00
Michael Schurter a7450c8633 FS HTTP API ACL enforcement
ACL enforcement for the filesystem HTTP APIs on clients.
2017-10-09 11:06:34 -07:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Luke Farnell f0ced87b95 fixed all spelling mistakes for goreport 2017-08-07 17:13:05 -04:00
Alex Dadgar 4e90d56098 More parallel 2017-07-20 09:36:34 -07:00
Alex Dadgar 9037693436 New test agent 2017-07-19 22:14:36 -07:00
Alex Dadgar 9faa98e13b Fix tests 2017-05-03 12:38:49 -07:00
Alex Dadgar ab330b96aa test 2017-01-23 16:04:50 -08:00
Michael Schurter 90f6ac7490 Fix tests post rebase 2017-01-06 11:39:13 -08:00
Alex Dadgar d82747bd33 Benchmark 2016-12-09 14:44:50 -08:00
Diptanu Choudhury 84722234b4 Fixed a bunch of TLS related failures 2016-10-26 14:08:46 -07:00
Michael Schurter 285e80ac0f Remove disk usage enforcement
Many thanks to @iverberk for the original PR (#1609), but we ended up
not wanting to ship this implementation with 0.5.

We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
2016-10-21 13:55:51 -07:00
Ivo Verberk 2a17895a83 Disk resource monitoring and enforcement 2016-08-18 07:59:03 +02:00
Cameron Davison 92ac9bb84c
duplicate check for deep equals of frame order 2016-08-13 22:26:53 -05:00
Alex Dadgar 4c77d9688f up timeouts 2016-08-11 18:30:46 -07:00
Alex Dadgar 2d5fa80ce7 Add travis multiplier 2016-08-11 18:16:10 -07:00
Diptanu Choudhury 471416ab20 Removing un-wanted dependencies 2016-07-25 14:17:52 -07:00
Alex Dadgar 1f600252e7 Handle skipping indexes 2016-07-25 11:16:54 -07:00
Alex Dadgar 120e9d14ee log tests 2016-07-25 11:16:54 -07:00
Alex Dadgar b55e986c9f Read from correct offset 2016-07-25 11:16:01 -07:00
Alex Dadgar c84964bf6e Fix buffer reuse 2016-07-25 11:16:01 -07:00
Alex Dadgar 0327109300 tests 2016-07-25 11:16:01 -07:00
Alex Dadgar 7dd14507ca Tests and handle conn close behavior better 2016-07-12 10:18:48 -06:00
Alex Dadgar e9ffadfdc6 initial comments 2016-07-11 10:58:18 -06:00
Alex Dadgar 281c1fbf39 StreamFramer tests 2016-07-10 13:57:20 -04:00
Alex Dadgar e100fc4422 Set up for testing 2016-07-10 13:57:04 -04:00
Diptanu Choudhury 39b263ed7f Refactoring some comments and test names 2016-01-14 15:07:24 -08:00
Diptanu Choudhury 95a9408e7d Renamed the tests 2016-01-14 13:39:30 -08:00
Diptanu Choudhury 7a93da0d3e Refactored the tests 2016-01-13 11:49:39 -08:00
Diptanu Choudhury 155845a46d Added a test for the readat api endpoint 2016-01-13 11:37:29 -08:00
Diptanu Choudhury 4f6dd60f3c Fixed the logic of extracting alloc 2016-01-13 11:19:45 -08:00