Since the state store returns a pointer to the shared job structs in
memdb we must always copy it before mutating it and applying the new
version via raft. Otherwise if the rpc fails before the mutated job is
committed to raft (either due to validation, bug, crash, or other exit
condition), the leader server will have an updated copy of the job that
other servers will not have.
This PR adds some NSD check status output to the CLI.
1. The 'nomad alloc status' command produces nsd check summary output (if present)
2. The 'nomad alloc checks' sub-command is added to produce complete nsd check output (if present)
Before this change, Client had 2 copies of the config object: config and configCopy. There was no guidance around which to use where (other than configCopy's comment to pass it to alloc runners), both are shared among goroutines and mutated in data racy ways. At least at one point I think the idea was to have `config` be mutable and then grab a lock to overwrite `configCopy`'s pointer atomically. This would have allowed alloc runners to read their config copies in data race safe ways, but this isn't how the current implementation worked.
This change takes the following approach to safely handling configs in the client:
1. `Client.config` is the only copy of the config and all access must go through the `Client.configLock` mutex
2. Since the mutex *only protects the config pointer itself and not fields inside the Config struct:* all config mutation must be done on a *copy* of the config, and then Client's config pointer is overwritten while the mutex is acquired. Alloc runners and other goroutines with the old config pointer will not see config updates.
3. Deep copying is implemented on the Config struct to satisfy the previous approach. The TLS Keyloader is an exception because it has its own internal locking to support mutating in place. An unfortunate complication but one I couldn't find a way to untangle in a timely fashion.
4. To facilitate deep copying I made an *internally backward incompatible API change:* our `helper/funcs` used to turn containers (slices and maps) with 0 elements into nils. This probably saves a few memory allocations but makes it very easy to cause panics. Since my new config handling approach uses more copying, it became very difficult to ensure all code that used containers on configs could handle nils properly. Since this code has caused panics in the past, I fixed it: nil containers are copied as nil, but 0-element containers properly return a new 0-element container. No more "downgrading to nil!"
This PR fixes the circle workflow step on windows where we print
the go version. Like the other commands that use Go, we must
inject the install path into PATH first.
This PR is the first of several for cleaning up warnings, and refactoring
bits of code in the command package. First pass is over acl_ files and
gets some helpers in place.
* Initialized keyboard service
Neat but funky: dynamic subnav traversal
👻
generalized traverseSubnav method
Shift as special modifier key
Nice little demo panel
Keyboard shortcuts keycard
Some animation styles on keyboard shortcuts
Handle situations where a link is deeply nested from its parent menu item
Keyboard service cleanup
helper-based initializer and teardown for new contextual commands
Keyboard shortcuts modal component added and demo-ghost removed
Removed j and k from subnav traversal
Register and unregister methods for subnav plus new subnavs for volumes and volume
register main nav method
Generalizing the register nav method
12762 table keynav (#12975)
* Experimental feature: shortcut visual hints
* Long way around to a custom modifier for keyboard shortcuts
* dynamic table and list iterative shortcuts
* Progress with regular old tether
* Delogging
* Table Keynav tether fix, server and client navs, and fix to shiftless on modified arrow keys
Go to Optimize keyboard link and storage key changed to g r
parameterized jobs keyboard nav
Dynamic numeric keynav for multiple tables (#13482)
* Multiple tables init
* URL-bind enumerable keyboard commands and add to more taskRow and allocationRows
* Type safety and lint fixes
* Consolidated push to keyCommands
* Default value when removing keyCommands
* Remove the URL-based removal method and perform a recompute on any add
Get tests passing in Keynav: remove math helpers and a few other defensive moves (#13761)
* Remove ember math helpers
* Test fixes for jobparts/body
* Kill an unneeded integration helper test
* delog
* Trying if disabling percy lets this finish
* Okay so its not percy; try parallelism in circle
* Percyless yet again
* Trying a different angle to not have percy
* Upgrade percy to 1.6.1
[ui] Keyboard nav: "u" key to go up a level (#13754)
* U to go up a level
* Mislabelled my conditional
* Custom lint ignore rule
* Custom lint ignore rule, this time with commas
* Since we're getting rid of ember math helpers elsewhere, do the math ourselves here
Replace ArrowLeft etc. with an ascii arrow (#13776)
* Replace ArrowLeft etc. with an ascii arrow
* non-mutative helper cleanup
Keyboard Nav: let users rebind their shortcuts (#13781)
* click-outside and shortcuts enabled/disabled toggle
* Trap focus when modal open
* Enabled/disabled saved to localStorage
* Autofocus edit button on variable index
* Modal overflow styles
* Functional rebind
* Saving rebinds to localStorage for all majors
* Started on defaultCommandBindings
* Modal header style and cancel rebind on escape
* keyboardable keybindings w buttons instead of spans
* recording and defaultvalues
* Enter short-circuits rebind
* Only some commands are rebindable, and dont show dupes
* No unused get import
* More visually distinct header on modal
* Disallowed keys for rebind, showing buffer as you type, and moving dedupe to modal logic
willDestroy hook to prevent tests from doubling/tripling up addEventListener on kb events
remove unused tests
Keyboard Navigation acceptance tests (#13893)
* Acceptance tests for keyboard modal
* a11y audit fix and localStorage clear
* Bind/rebind/localStorage tests
* Keyboard tests for dynamic nav and tables
* Rebinder and assert expectation
* Second percy snapshot showing hints no longer relevant
Weird issue where linktos with query props specifically from the task-groups page would fail to route / hit undefined.shouldSuperCede errors
Adds the concept of exclusivity to a keycommand, removing peers that also share its label
Lintfix
Changelog and PR feedback
Changelog and PR feedback
Fix to rebinding in firefox by blurring the now-disabled button on rebind (#14053)
* Secure Variables shortcuts removed
* Variable index route autofocus removed
* Updated changelog entry
* Updated changelog entry
* Keynav docs (#14148)
* Section added to the API Docs UI page
* Added a note about disabling
* Prev and Next order
* Remove dev log and unneeded comments
ACL Policies aren't required to have any `namespace` blocks, and this is
particularly common with the anonymous policy. If a user visits the web UI
without a token already in their local storage and the anonymous policy has no
`namespace` blocks, the UI will hit unhandled exceptions when rendering the
sidebar or jobs page.
Filter for the case where there's no `namespace` block.
Similar to the deployment watcher fix in #14121 - the server code loves these mutable structs so we need to guard access to the struct fields with locks.
Capturing ch := b.capacityChangeCh is sufficient to satisfy the data race detector, but I noticed it was also possible to leak goroutines:
Since the watchCapacity loop is in charge of receiving from capacityChangeCh and exits when stopCh is closed, senders to capacityChangeCh also must exit when stopCh is closed. Otherwise they may block forever if capacityChangeCh is full because it will never be received on again. I did not find evidence of this occurring in my meager smattering of prod goroutine dumps I have laying around, but this isn't surprising as the chan has a buffer of 8096! I would imagine that is sufficient to handle "late" sends and then just get GC'd away when the last reference to the old chan is dropped. This is just additional safety/correctness.
* Handle conflict swith a cas qp on save and create
* Notify error and give them refresh or overwrite options
* Merge conflict missed, resolved
* Mirage fixture
* Integration test
* Bracket closed (thx jai)
* Adjust tests to account for number of variables with auto-conflicter
The HCL parser allows for labels that aren't needed, which makes it easy to
accidentally write a `secure_variable` block that has the intended path as the
label for that block instead of the innner `path` block. This can result in
silent failure to lock down variables if an incorrectly specified block was used
to reduce the scope of capabilities (for example, if another correctly-written
rule allows access to `*`).
We can't detect the extraneous label in the HCL API, but we can detect if we're
missing `path` blocks entirely. Use this to block obvious user errors.
HTTP API consumers that have network line-of-sight to client nodes can connect
directly for a small number of APIs. But in environments where the consumer
doesn't have line-of-sight, there's a long pause waiting for the
`api.ClientConnTimeout` to expire. Warn about this in the API docs so that
authors can avoid the extra timeout.
* Starting on namespaced id
* Traversal for variables uniqued by namespace
* Delog
* Basic CRUD complete w namespaces included
* Correct secvar breadcrumb joining and testfix now that namespaces are included
* Testfixes with namespaces in place
* Namespace-aware duplicate path warning
* Duplicate path warning test additions
* Trimpath reimplemented on dupe check
* Solves a bug where slash was not being passed to the can write check
* PR fixes
* variable paths integration test fix now uses store
* Seems far less hacky in retrospect
* PR feedback addressed
* test fixes after inclusion of path as local non-model var
* Prevent confusion by dropping namespace from QPs on PUT, since its already in .data
* Solves a harsh bug where you have namespace access but no secvars access (#14098)
* Solves a harsh bug where you have namespace access but no secvars access
* Lint cleanup
* Remove unneeded condition
The List RPCs only checked the ACL for the Prefix argument of the request. Add
an ACL filter to the paginator for the List RPC.
Extend test coverage of ACLs in the List RPC and in the `acl` package, and add a
"deny" capability so that operators can deny specific paths or prefixes below an
allowed path.
Move conflict resolution implementation into the state store with a new Apply RPC.
This also makes the RPC for secure variables much more similar to Consul's KV,
which will help us support soft deletes in a post-1.4.0 version of Nomad.
Reimplement quotas in the state store functions.
Co-authored-by: Charlie Voiselle <464492+angrycub@users.noreply.github.com>
