Commit Graph

47 Commits

Author SHA1 Message Date
Mahmood Ali 98c02851c8
use comment ignores (#9448)
Use targetted ignore comments for the cases where we are bound by
backward compatibility.

I've left some file based linters, especially when the file is riddled
with linter voilations (e.g. enum names), or if it's a property of the
file (e.g. package and file names).

I encountered an odd behavior related to RPC_REQUEST_RESPONSE_UNIQUE and
RPC_REQUEST_STANDARD_NAME.  Apparently, if they target a `stream` type,
we must separate them into separate lines so that the ignore comment
targets the type specifically.
2020-11-25 16:03:01 -05:00
Kris Hicks 9d03cf4c5f
protos: Update .proto files not to use Go package name (#9301)
Previously, it was required that you `go get github.com/hashicorp/nomad` to be
able to build protos, as the protoc invocation added an include directive that
pointed to `$GOPATH/src`, which is how dependent protos were discovered. As
Nomad now uses Go modules, it won't necessarily be cloned to `$GOPATH`.
(Additionally, if you _had_ go-gotten Nomad at some point, protoc compilation
would have possibly used the _wrong_ protos, as those wouldn't necessarily be
the most up-to-date ones.)

This change modifies the proto files and the `protoc` invocation to handle
discovering dependent protos via protoc plugin modifier statements that are
specific to the protoc plugin being used.

In this change, `make proto` was run to recompile the protos, which results in
changes only to the gzipped `FileDescriptorProto`.
2020-11-10 08:42:35 -08:00
Mahmood Ali c2d3c3e431
nvidia: support disabling the nvidia plugin (#8353) 2020-07-21 10:11:16 -04:00
Mahmood Ali 88cfe504a0 update grpc
Upgrade grpc to v1.27.1 and protobuf plugins to v1.3.4.
2020-03-03 08:39:54 -05:00
Mahmood Ali fb55717b0c
Regenerate Proto files (#5421)
Noticed that the protobuf files are out of sync with ones generated by 1.2.0 protoc go plugin.

The cause for these files seem to be related to release processes, e.g. [0.9.0-beta1 preperation](ecec3d38de (diff-da4da188ee496377d456025c2eab4e87)), and [0.9.0-beta3 preperation](b849d84f2f).

This restores the changes to that of the pinned protoc version and fails build if protobuf files are out of sync.  Sample failing Travis job is that of the first commit change: https://travis-ci.org/hashicorp/nomad/jobs/506285085
2019-03-14 10:56:27 -04:00
Michael Schurter d74755900e Generate files for 0.9.0-beta3 release 2019-02-26 09:44:49 -08:00
Alex Dadgar bc804dda2e Nomad 0.9.0-beta1 generated code 2019-01-30 10:49:44 -08:00
Mahmood Ali 698f0a1575
Merge pull request #5229 from hashicorp/r-grabbag-201901019
Grab bag of small changes
2019-01-23 13:06:51 -05:00
Mahmood Ali 651bbe5330 spell check 2019-01-23 10:54:52 -05:00
Michael Schurter 32daa7b47b goimports until make check is happy 2019-01-23 06:27:14 -08:00
Michael Schurter be0bab7c3f move pluginutils -> helper/pluginutils
I wanted a different color bikeshed, so I get to paint it
2019-01-22 15:50:08 -08:00
Alex Dadgar 6c2782f037 move catalog + grpcutils 2019-01-22 15:11:57 -08:00
Mahmood Ali 9909d98bee Track Basic Memory Usage as reported by cgroups
Track current memory usage, `memory.usage_in_bytes`, in addition to
`memory.max_memory_usage_in_bytes` and friends.  This number is closer
what Docker reports.

Related to https://github.com/hashicorp/nomad/issues/5165 .
2019-01-14 18:47:52 -05:00
Alex Dadgar c9825a9c36 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar 4c57d2ec4d Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Mahmood Ali 9e825f880c Use absolute path in example device plugin
deviceDir is used for specifying mount/device host paths, and those
should be absolute paths.
2018-12-06 15:46:35 -05:00
Preetha Appan f89dbcd9cc
modify fingerprint interface to use typed attribute struct 2018-11-28 10:01:03 -06:00
Mahmood Ali c5de71a424 Allow nullable fields in StatValues
In state values, we need to be able to distinguish between zero values
(e.g. `false`) and unset values (e.g. `nil`).

We can alternatively use protobuf `oneOf` and nested map to ensure
consistency of fields that are set together, but the golang
representation does not represent that well and introducing a mismatch
between representations.  Thus, I opted not to use it.
2018-11-14 14:41:19 -05:00
Mahmood Ali 713c9fe683 Move Stat{Object|Value} to plugins/shared/structs
Moving them as they may be useful for other packages/plugins besides
devices.
2018-11-14 09:01:26 -05:00
Mahmood Ali 1f4db08f42 Regenerate proto files with protoc-gen-go@v1.2.0 2018-11-14 09:01:26 -05:00
Mahmood Ali 1e92161f14
Merge pull request #4858 from hashicorp/b-fix-master-20181109
Fix some tests in master
2018-11-13 16:08:26 -05:00
Mahmood Ali ac3b4571eb Address review comments 2018-11-13 10:21:40 -05:00
Mahmood Ali fa146d9b85 fix plugin test 2018-11-13 10:21:40 -05:00
Alex Dadgar 693f244cce Plugin client's handle plugin dying
This PR plumbs the plugins done ctx through the base and driver plugin
clients (device already had it). Further, it adds generic handling of
gRPC stream errors.
2018-11-12 17:09:27 -08:00
Alex Dadgar c4f9e22aeb fix race 2018-11-07 12:22:07 -08:00
Alex Dadgar b4661df231 reserve uses donectx 2018-11-07 10:43:15 -08:00
Alex Dadgar f0c7a8159b tests 2018-11-07 10:43:15 -08:00
Alex Dadgar 204ca8230c Device manager
Introduce a device manager that manages the lifecycle of device plugins
on the client. It fingerprints, collects stats, and forwards Reserve
requests to the correct plugin. The manager, also handles device plugins
failing and validates their output.
2018-11-07 10:43:15 -08:00
Nick Ethier bda3b1d3b3
rename NomadConfig to ClientAgentConfig 2018-10-29 21:34:34 -04:00
Nick Ethier 65adb80ebf
plumb NomadConfig into plugins 2018-10-16 22:47:22 -04:00
Alex Dadgar 5fc9a95201 Use Attribute in device fingerprinting 2018-10-13 11:43:06 -07:00
Alex Dadgar 0183fb4e5c nvidia package restructue + build non-linux 2018-10-05 13:56:04 -07:00
oleksii.shyman 118e3fe7e9 Introduce nvidia-plugin reserve
- added reserve functionality that returns OCI compliant env variables
  specifying GPU IDs to be injected inside the container
2018-10-04 14:55:34 -07:00
oleksii.shyman 0ea1dc1776 Introduce Nvidia-plugin stats
- created go-nvml wrapper for stats
 - added stats feature to nvidia-plugin
2018-10-03 15:12:05 -07:00
oleksii.shyman b4a4b395e3 Introduce nvidia-plugin fingerprinting
- created go-nvml wrapper for fingerprinting
  - added fingerprinting feature to nvidia-plugin
2018-10-03 15:11:56 -07:00
Alex Dadgar 6a1db58593
Update README.md 2018-08-28 15:08:59 -07:00
Alex Dadgar b7f3e24391 launcher readme 2018-08-28 15:00:48 -07:00
Alex Dadgar 3e3ca1ac04 Add stats to example plugin 2018-08-28 13:59:57 -07:00
Alex Dadgar 83ad5afb69 Example device plugin and helpers
This PR introduces:
* An example device plugin that exposes files as devices for testing.
* Helpers for serving a device plugin
* A launcher binary that allows interacting with a device plugin without
needing a Nomad client.
2018-08-28 13:59:57 -07:00
Alex Dadgar cefa806519 gofmt -s 2018-08-28 10:32:03 -07:00
Alex Dadgar c394798106 add stats to device interface 2018-08-27 16:11:07 -07:00
Alex Dadgar 1eb043a46c statistics protos 2018-08-23 11:18:35 -07:00
Alex Dadgar 9ac7dcd1ee Describe public interface of Fingerprint 2018-08-20 15:19:08 -07:00
Alex Dadgar 98a67938a0 initial device go-plugin 2018-08-16 16:49:09 -07:00
Alex Dadgar 720cc4d90c Fix protos 2018-08-09 13:29:05 -07:00
Alex Dadgar ca121ce110 review feedback 2018-08-07 13:03:21 -07:00
Alex Dadgar 97ccf42c64 Base and device plugin and hclspec parser 2018-08-07 13:03:21 -07:00