2edbdfc8be
The e2e framework instantiates clients for Nomad/Consul but the provisioning of the actual Nomad cluster is left to Terraform. The Terraform provisioning process uses `remote-exec` to deploy specific versions of Nomad so that we don't have to bake an AMI every time we want to test a new version. But Terraform treats the resulting instances as immutable, so we can't use the same tooling to update the version of Nomad in-place. This is a prerequisite for upgrade testing. This changeset extends the e2e framework to provide the option of deploying Nomad (and, in the future, Consul/Vault) with specific versions to running infrastructure. This initial implementation is focused on deploying to a single cluster via `ssh` (because that's our current need), but provides interfaces to hook the test run at the start of the run, the start of each suite, or the start of a given test case. Terraform work includes: * provides Terraform output that written to JSON used by the framework to configure provisioning via `terraform output provisioning`. * provides Terraform output that can be used by test operators to configure their shell via `$(terraform output environment)` * drops `remote-exec` provisioning steps from Terraform * makes changes to the deployment scripts to ensure they can be run multiple times w/ different versions against the same host.
69 lines
2 KiB
Bash
Executable file
69 lines
2 KiB
Bash
Executable file
#!/bin/bash
|
|
# installs and configures the desired build of Nomad as a server
|
|
set -o errexit
|
|
set -o nounset
|
|
|
|
CLOUD="$1"
|
|
SERVER_COUNT="$2"
|
|
NOMAD_CONFIG="$3"
|
|
|
|
# Consul
|
|
CONSUL_SRC=/ops/shared/consul
|
|
CONSUL_DEST=/etc/consul.d
|
|
|
|
sed "s/SERVER_COUNT/$SERVER_COUNT/g" "$CONSUL_SRC/server.json" > /tmp/server.json
|
|
sudo mv /tmp/server.json "$CONSUL_DEST/server.json"
|
|
sudo cp "$CONSUL_SRC/base.json" "$CONSUL_DEST/"
|
|
sudo cp "$CONSUL_SRC/retry_$CLOUD.json" "$CONSUL_DEST/"
|
|
sudo cp "$CONSUL_SRC/consul_$CLOUD.service" /etc/systemd/system/consul.service
|
|
|
|
sudo systemctl enable consul.service
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart consul.service
|
|
sleep 10
|
|
|
|
# Vault
|
|
VAULT_SRC=/ops/shared/vault
|
|
VAULT_DEST=/etc/vault.d
|
|
|
|
sudo cp "$VAULT_SRC/vault.hcl" "$VAULT_DEST"
|
|
sudo cp "$VAULT_SRC/vault.service" /etc/systemd/system/vault.service
|
|
|
|
sudo systemctl enable vault.service
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart vault.service
|
|
|
|
# Add hostname to /etc/hosts
|
|
echo "127.0.0.1 $(hostname)" | sudo tee --append /etc/hosts
|
|
|
|
# Add Docker bridge network IP to /etc/resolv.conf (at the top)
|
|
DOCKER_BRIDGE_IP_ADDRESS=$(/usr/local/bin/sockaddr eval 'GetInterfaceIP "docker0"')
|
|
echo "nameserver $DOCKER_BRIDGE_IP_ADDRESS" | sudo tee /etc/resolv.conf.new
|
|
cat /etc/resolv.conf | sudo tee --append /etc/resolv.conf.new
|
|
sudo mv /etc/resolv.conf.new /etc/resolv.conf
|
|
|
|
# Nomad
|
|
|
|
NOMAD_SRC=/ops/shared/nomad
|
|
NOMAD_DEST=/etc/nomad.d
|
|
NOMAD_CONFIG_FILENAME=$(basename "$NOMAD_CONFIG")
|
|
|
|
# assert Nomad binary's permissions
|
|
if [[ -f /usr/local/bin/nomad ]]; then
|
|
sudo chmod 0755 /usr/local/bin/nomad
|
|
sudo chown root:root /usr/local/bin/nomad
|
|
fi
|
|
|
|
sudo cp "$NOMAD_SRC/base.hcl" "$NOMAD_DEST/"
|
|
|
|
sed "s/3 # SERVER_COUNT/$SERVER_COUNT/g" "$NOMAD_SRC/$NOMAD_CONFIG" \
|
|
> "/tmp/$NOMAD_CONFIG_FILENAME"
|
|
sudo mv "/tmp/$NOMAD_CONFIG_FILENAME" "$NOMAD_DEST/$NOMAD_CONFIG_FILENAME"
|
|
|
|
# enable as a systemd service
|
|
sudo cp "$NOMAD_SRC/nomad.service" /etc/systemd/system/nomad.service
|
|
|
|
sudo systemctl enable nomad.service
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart nomad.service
|