open-nomad/website/source/guides/web-ui/inspecting-the-cluster.html.md
2019-10-02 16:54:38 -07:00

6.5 KiB

layout page_title sidebar_current description
guides Inspecting the Cluster guides-web-ui-inspecting-the-cluster Learn how to inspect the state of the cluster from the Web UI.

Inspecting the Cluster

The Web UI can be a powerful tool for monitoring the state of the Nomad cluster from an operator's perspective. This includes showing all client nodes, showing driver health for client nodes, driver status, resource utilization, allocations by client node, and more.

Reviewing All Clients

From any page, the Clients List page can be accessed from the left-hand navbar. On narrow screens this navbar may need to opened from the top-right menu button. Here you see every client in the cluster. The table of clients is searchable, sortable, and filterable. Each client row in the table show basic information, such as the Node ID, name, state, address, datacenter, and how many allocations are running in it.

This view will also live-update as the state of client nodes change.

Clients List

Filtering Clients

If your Nomad cluster has many client nodes, it can be useful to filter the list of all client nodes down to only those matching certain facets. The Web UI has three facets you can filter by:

  1. Class: The node of the client, including a dynamically generated list based on the node class of each client node in the cluster.
  2. State: The state of the cluster, including Initializing, Ready, Down, Ineligible, and Draining.
  3. Datacenter: The datacenter the client node is in, including a dynamically generated list based on all the datacenters in the cluster.

Clients filters

Inspecting an Individual Client

From the Clients List page, clicking a client node in the table will direct you to the Client Detail page for the client node. This page includes all information about the client node is live-updated to always present up-to-date information.

Client Detail

Resource Utilization

Nomad as APIs for reading point-in-time resource utilization metrics for client nodes. The Web UI uses these metrics to create time-series graphics for the current session.

When viewing a client node, resource utilization will automatically start logging.

Client Resource Utilization

Allocations

Allocations belong to jobs and are placed on client nodes. The Client Detail page will list all allocations for a client node, including completed, failed, and lost allocations, until they are garbage-collected.

This is presented in a searchable table which can additionally be filtered to only preempted allocations.

Client Allocations

Client Events

Client nodes will also emit events on meaningful state changes, such as when the node becomes ready for scheduling or when a driver becomes unhealthy.

Client Events

Driver Status

Task drivers are additional services running on a client node. Nomad will fingerprint and communicate with the task driver to determine if the driver is available and healthy. This information is reported through the Web UI on the Client Detail page.

Client Driver Status

Attributes

In order to allow job authors to constrain the placement of their jobs, Nomad fingerprints the hardware of the node the client agent is running on. This is a deeply nested document of properties that the Web UI presents in a scannable way.

In addition to the hardware attributes, Nomad operators can annotate a client node with metadata as part of the client configuration. This metadata is also presented on the Client Detail page.

Client Attributes

When a Node is Draining

A routine part of maintaining a Nomad cluster is draining nodes of allocations. This can be in preparation of performing operating system upgrades or decommissioning an old node in favor of a new VM.

Drains are performed from the CLI but the status of a drain can be seen from the Web UI. A client node will state if it is actively draining or ineligible for scheduling.

Since drains can be configured in a variety of ways, the Client Detail page will also present the details of how the drain is performed.

Client Drain

Reviewing All Servers

Whereas client nodes are used to run your jobs, server nodes are used to run Nomad and maintain availability. From any page, the Servers List page can be accessed from the left-hand navbar.

Here you can see every server node. This will be a small list—typically three or five.

Servers List

Inspecting an Individual Server

Clicking a server node on the Servers List will expand the tags table for the server node.

Server Detail

Access Control

Depending on the size of your team and the details of you Nomad deployment, you may wish to control which features different internal users have access to. This includes limiting who has access to see and manage client nodes and see and manage server nodes. You can enforce this with Nomad's access control list system.

By default, all features—read and write—are available to all users of the Web UI. Check out the Securing the Web UI with ACLs guide to learn how to prevent anonymous users from having write permissions as well as how to continue to use Web UI write features as a privileged user.