27 KiB

Raw Blame History

layout	page_title	sidebar_current	description
guides	Secure Multi-Tenancy with Namepaces - Guides	guides-operations-multi-tenant	This guide provides guidance in creating a multi-tenant environment.

Secure Multi-Tenancy with Namespaces

~> Enterprise Only: The namespaces feature is a part of Vault Enterprise Pro.

Everything in Vault is path-based, and often uses the terms path and namespace interchangeably. The application namespace pattern is a useful construct for providing Vault as a service to internal customers, giving them the ability to implement secure multi-tenancy within Vault in order to provide isolation and ensure teams can self-manage their own environments.

Reference Material

Estimated Time to Complete

10 minutes

Personas

The scenario described in this guide introduces the following personas:

operations is the cluster-level administrator with privileged policies
org-admin is the organization-level administrator
team-admin is the team-level administrator

Challenge

When Vault is primarily used as a central location to manage secrets, multiple organizations within a company may need to be able to manage their secrets in a self-serving manner. This means that a company needs to implement a Vault as a Service model allowing each organization (tenant) to manage their own secrets and policies. The most importantly, tenants should be restricted to work only within their tenant scope.

Solution

Create a namespace dedicated to each team, organization, or app where they can perform all necessary tasks within their tenant namespace.

Each namespace can have its own:

Policies
Mounts
Tokens
Identity entities and groups

~> Tokens are locked to a namespace or child-namespaces. Identity groups can pull in entities and groups from other namespaces.

Prerequisites

To perform the tasks described in this guide, you need to have a Vault Enterprise environment.

-> NOTE: The creation of namespaces should be performed by a user with a highly privileged token such as root to set up isolated environments for each organization, team, or application.

Steps

Scenario: In this guide, you are going to create a namespace dedicated to the Education organization which has Training and Certification teams. Delegate operational tasks to the team admins so that the Vault cluster operators won't have to be involved.

In this guide, you are going to perform the following steps:

Create namespaces
Write policies
Setup entities and groups
Test the organization admin user
Test the team admin user

Step 1: Create namespaces

(Persona: operations)

CLI command

To create a new namespace, run: vault namespace create <namespace_name>

Create a namespace dedicated to the education organizations:
```
$ vault namespace create education
```

Create child namespaces called training and certification under the education namespace:

$ vault namespace create -namespace=education training

$ vault namespace create -namespace=education certification

List the created namespaces:

$ vault namespace list
education/

$ vault namespace list -namespace=education
certification/
training/

API call using cURL

To create a new namespace, invoke sys/namespaces endpoint:

$ curl --header "X-Vault-Token: <TOKEN>" \
       --request POST \
       <VAULT_ADDRESS>/v1/sys/namespaces/<NS_NAME>

Where <TOKEN> is your valid token, and <NS_NAME> is the desired namespace name.

Create a namespace for the education organization:

$ curl --header "X-Vault-Token: ..." \
       --request POST \
       http://127.0.0.1:8200/v1/sys/namespaces/education

Now, create a child namespace called training and certification under education. To do so, pass the top-level namespace name in the X-Vault-Namespace header.

# Create a training namespace under education
# NOTE: Top-level namespace is in the API endpoint
$ curl --header "X-Vault-Token: ..." \
       --header "X-Vault-Namespace: education" \
       --request POST \
       http://127.0.0.1:8200/v1/education/sys/namespaces/training

# Create a certification namespace under education
# NOTE: Pass the top-level namespace in the header
$ curl --header "X-Vault-Token: ..." \
       --header "X-Vault-Namespace: education" \
       --request POST \
       http://127.0.0.1:8200/v1/sys/namespaces/certification

List the created namespaces:

$ curl --header "X-Vault-Token: ..." \
       --request LIST
       http://127.0.0.1:8200/v1/sys/namespaces | jq
{
   ...
   "data": {
     "keys": [
       "education/"
     ]
   },
   ...


$ curl --header "X-Vault-Token: ..." \
       --request LIST
       http://127.0.0.1:8200/v1/education/sys/namespaces | jq
 {
   ...
   "data": {
     "keys": [
       "certification/",
       "training/"
     ]
   },
   ...

Web UI

Open a web browser and launch the Vault UI (e.g. http://127.0.01:8200/ui) and then login.
Select Access.
Select Namespaces and then click Create a namespace.
Enter education in the Path field.
Click Save.
To create child namespaces, select the down-arrow on the upper left corner of the UI, and select education under CURRENT NAMESPACE.
Under the Access tab, select Namespaces and then click Create a namespace.
Enter training in the Path field, and click Save.
Select Create a namespace again, and then enter certification in the Path field, and click Save.

Step 2: Write Policies

(Persona: operations)

In this scenario, there is an organization-level administrator who is a superuser within the scope of the education namespace. Also, there is a team-level administrator for training and certification.

Policy for education admin

Requirements:

Create and manage namespaces
Create and manage policies
Enable and manage secret engines
Create and manage entities and groups
Manage tokens

edu-admin.hcl

# Manage namespaces
path "sys/namespaces/*" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Manage policies via API
path "sys/policies/*" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Manage policies via CLI
path "sys/policy/*" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# List policies via CLI
path "sys/policy" {
   capabilities = ["read", "update", "list"]
}

# Enable and manage secrets engines
path "sys/mounts/*" {
   capabilities = ["create", "read", "update", "delete", "list"]
}

# List available secret engines
path "sys/mounts" {
  capabilities = [ "read" ]
}

# Create and manage entities and groups
path "identity/*" {
   capabilities = ["create", "read", "update", "delete", "list"]
}

# Manage tokens
path "auth/token/*" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

Policy for training admin

Requirements:

Create and manage child-namespaces
Create and manage policies
Enable and manage secret engines

training-admin.hcl

# Manage namespaces
path "sys/namespaces/*" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Manage policies via API
path "sys/policies/*" {
   capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# Manage policies via CLI
path "sys/policy/*" {
  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}

# List policies via CLI
path "sys/policy" {
  capabilities = ["read", "update", "list"]
}

# Enable and manage secrets engines
path "sys/mounts/*" {
   capabilities = ["create", "read", "update", "delete", "list"]
}

# List available secret engines
path "sys/mounts" {
  capabilities = [ "read" ]
}

Now, let's deploy the policies!

CLI command

To target a specific namespace, you can do one of the following:

Set VAULT_NAMESPACE so that all subsequent CLI commands will be executed against that particular namespace
```
$ export VAULT_NAMESPACE=<namespace_name>
$ vault policy write <policy_name> <policy_file>
```

Specify the target namespace with -namespace flag

$ vault policy write -namespace=<namespace_name> <policy_name> <policy_file>

Since you have to deploy policies onto "education" and "education/training" namespaces, use "-namespace" flag instead of environment variable.

Create edu-admin and training-admin policies.

# Create edu-admin policy under 'education' namespace
$ vault policy write -namespace=education edu-admin edu-admin.hcl

# Create training-admin policy under 'education/training' namespace
$ vault policy write -namespace=education/training training-admin training-admin.hcl

API call using cURL

To target a specific namespace, you can do one of the following:

Pass the target namespace in the X-Vault-Namespace header
Prepend the API endpoint with namespace name (e.g. <namespace_name>/sys/policies/acl)

Create edu-admin and training-admin policies.

# Create a request payload
$ tee edu-payload.json <<EOF
{
  "policy": "path \"sys/namespaces/education/*\" {\n  capabilities = [\"create\", \"read\", \"update\", \"delete\", \"list\", \"sudo\"]\n } ... "
}
EOF

# Create edu-admin policy under 'education' namespace
$ curl --header "X-Vault-Token: ..." \
       --header "X-Vault-Namespace: education" \
       --request PUT \
       --data @edu-payload.json \
       https://vault.rocks/v1/sys/policies/acl/edu-admin

# Create a request payload
$ tee training-payload.json <<EOF
{
 "policy": "path \"sys/namespaces/education/training/*\" {\n  capabilities = [\"create\", \"read\", \"update\", \"delete\", \"list\", \"sudo\"]\n  }  ... "
}
EOF

# Create training-admin policy under 'education/training' namespace
# This example directs the target namespace in the API endpoint
$ curl --header "X-Vault-Token: ..." \
       --request PUT \
       --data @training-payload.json \
       https://vault.rocks/v1/education/training/sys/policies/acl/training-admin

Web UI

In the Web UI, make sure that the CURRENT NAMESPACE is set to education in the upper left menu.
Click the Policies tab, and then select Create ACL policy.
Toggle Upload file sliding switch, and click Choose a file to select your edu-admin.hcl file you authored. This loads the policy and sets the Name to be edu-admin.
Click Create Policy to complete.
Set the CURRENT NAMESPACE to be education/training in the upper left menu.
In the Policies tab, select Create ACL policy.
Toggle Upload file, and click Choose a file to select your training-admin.hcl file you authored.
Click Create Policy.

Step 3: Setup entities and groups

(Persona: operations)

Bob who is an organization-level administrator (superuser) has two accounts: bob and bsmith. You will create an entity, Bob Smith to associate those two accounts.

Also, you are going to create a group for the team-level administrator, Team Admin, and add Bob Smith entity as a group member so that Bob can inherit the training-admin policy to manage the child namespace if he ever has to take over.

-> This step only demonstrates CLI commands and Web UI to create entities and groups. Refer to the Identity - Entities and Groups guide if you need the full details. Also, read the Additional Discussion section for an example of setting up external groups.

CLI Command

# First, you need to enable userpass auth method
$ vault auth enable -namespace=education userpass

# Create a user 'bob'
$ vault write -namespace=education \
        auth/userpass/users/bob password="password"

# Create an entity for Bob Smith with 'edu-admin' policy attached
# Save the generated entity ID in entity_id.txt file
$ vault write -namespace=education -format=json identity/entity name="Bob Smith" \
        policies="edu-admin" | jq -r ".data.id" > entity_id.txt

# Get the mount accessor for userpass auth method and save it in accessor.txt file
$ vault auth list -namespace=education -format=json \
        | jq -r '.["userpass/"].accessor' > accessor.txt

# Create an entity alias for Bob Smith to attach 'bob'
$ vault write -namespace=education identity/entity-alias name="bob" \
        canonical_id=$(cat entity_id.txt) mount_accessor=$(cat accessor.txt)

# Create a group, "Training Admin" in education/training namespace
$ vault write -namespace=education/training identity/group \
        name="Training Admin" policies="training-admin" \
        member_entity_ids=$(cat entity_id.txt)

# Enable userpass auth method in training namespace
$ vault auth enable -namespace=education/training userpass

# Create a user 'bsmith'
$ vault write -namespace=education/training \
        auth/userpass/users/bsmith password="password"

# Get the mount accessor for userpass auth method and save it in accessor2.txt file
$ vault auth list -namespace=education/training -format=json \
        | jq -r '.["userpass/"].accessor' > accessor2.txt

# Add 'bsmith' to Bob Smith entity as its alias
$ vault write -namespace=education identity/entity-alias name="bsmith" \
        canonical_id=$(cat entity_id.txt) mount_accessor=$(cat accessor2.txt)

Web UI

In the Web UI, make sure that the CURRENT NAMESPACE is set to education in the upper left menu.
Click the Access tab, and select Enable new method.
Select Username & Password from the Type drop-down menu.
Click Enable Method.
Click the Vault CLI shell icon (>_) to open a command shell. Enter the following command to create a new user, bob.
```
vault write auth/userpass/users/bob password="password"
```
Click the icon (>_) again to hide the shell.
From the Access tab, select Entities and then Create entity.
Enter Bob Smith in the Name field, and edu-admin in the Policies field.
Click Create.
Select Add alias. Enter bob in the Name field and select userpass/ (userpass) from the Auth Backend drop-down list.
Click Create.
Click the Access tab and select Entities.
Select the bob-smith entity and copy its ID displayed under the Details tab.
Now, set the CURRENT NAMESPACE to education/training.
In the Access tab, select Groups, and select Create group.
Paste in the entity ID in the Member Entity IDs field you copied.
Enter Training Admin in the Name field, training-admin in the Policies field, and click Create.
Click the Access tab, and select Enable new method.
Select Username & Password from the Type drop-down menu.
Click Enable Method. Copy the mount accessor value which you will user later.
Click the Vault CLI shell icon (>_) to open a command shell. Enter the following command to create a new user, bsmith.
```
vault write auth/userpass/users/bsmith password="password"
```
Set the CURRENT NAMESPACE back to education.
In the command shell, enter the following command. Be sure to replace the <Bob_Smith_entity_id> with the value you copied at step 13, and <mount_accessor> with the value you copied at step 20.
```
vault write identity/entity-alias name="bsmith" \
        canonical_id=<Bob_Smith_entity_id> mount_accessor=<mount_accessor>
```

Step 4: Test the organization admin user

(Persona: org-admin)

CLI Command

$ vault login -namespace=education -method=userpass username="bob" password="password"

Key                    Value
---                    -----
token                  5ai0qpQeCdRHALzEY4Q8sW.28dk2
token_accessor         9xXQmdx6Aq6zw1KX4gpzb.28dk2
token_duration         768h
token_renewable        true
token_policies         ["default"]
identity_policies      ["edu-admin"]
policies               ["default" "edu-admin"]
token_meta_username    bob

Notice that the user, bob only has default policy attached to his token (token_policies); however, he inherited the edu-admin policy from the Bob Smith entity (identity_policies).

Test to make sure that bob can create a namespace, enable secrets engine, and whatever else that you want to verify.

# Set the target namespace as an env variable
$ export VAULT_NAMESPACE="education"

# Create a new namespace called 'web-app'
$ vault namespace create web-app
Success! Namespace created at: education/web-app/

# Enable key/value v2 secrets engine at edu-secret
$ vault secrets enable -path=edu-secret kv-v2
Success! Enabled the kv-v2 secrets engine at: edu-secret/

Optionally, you can create new policies to test that bob can perform the operations as expected. When you are done testing, unset the VAULT_NAMESPACE environment variable.

$ unset VAULT_NAMESPACE

API call using cURL

$ curl --header "X-Vault-Namespace: education" \
       --request POST \
       --data '{"password": "password"}' \
       http://127.0.0.1:8200/v1/auth/userpass/login/bob | jq
{
   ...
   "auth": {
     "client_token": "5ai0qpQeCdRHALzEY4Q8sW.28dk2",
     "accessor": "9xXQmdx6Aq6zw1KX4gpzb.28dk2",
     "policies": [
        "default",
        "edu-admin"
      ],
      "token_policies": [
        "default"
      ],
      "identity_policies": [
        "edu-admin"
      ],
      "external_namespace_policies": {
        "9dKXw": [
          "training-admin"
        ]
      },
     "metadata": {
       "username": "bob"
     },
     ...
   }
}

Notice that the user, bob only has default policy attached to his token (token_policies); however, he inherited the edu-admin policy from the Bob Smith entity (identity_policies). Also, training-admin policy is listed under external_namespace_policies due to its membership to the Training Admin group in education/training namespace.

Verify that bob can perform the operations permitted by the edu-admin policy.

# Create a new namespace called 'web-app'
# Be sure to use generated bob's client token
$ curl --header "X-Vault-Token: 5ai0qpQeCdRHALzEY4Q8sW.28dk2" \
       --request POST \
       http://127.0.0.1:8200/v1/education/sys/namespaces/web-app

# Enable key/value v2 secrets engine at edu-secret
$ curl --header "X-Vault-Token: 5ai0qpQeCdRHALzEY4Q8sW.28dk2" \
       --request POST \
       --data '{"type": "kv-v2"}' \
       http://127.0.0.1:8200/v1/education/sys/mounts/edu-secret

Web UI

Open a web browser and launch the Vault UI (e.g. http://127.0.01:8200/ui). If you are already logged in, sign out.
At the Sign in to Vault, set the Namespace to education.
Select the Userpass tab, and enter bob in the Username field, and password in the Password field.
Click Sign in. Notice that the CURRENT NAMESPACE is set to education in the upper left corner of the UI.
To add a new namespace, select Access.
Select Namespaces and then click Create a namespace.
Enter web-app in the Path field, and then click Save.
Select Secrets, and then Enable new engine.
Select KV from the Secrets engine type drop-down list, and enter edu-secret in the Path field.
Click Enable Engine to finish.

Step 5: Test the team admin user

(Persona: team-admin)

CLI Command

$ vault login -namespace=education/training -method=userpass username="bsmith" password="password"

Key                    Value
---                    -----
token                  5YNNjDDl6D8iW3eGQIlU0q.9dKXw
token_accessor         6TVkDhdvEQXO2JaD64TVLv.9dKXw
token_duration         768h
token_renewable        true
token_policies         ["default"]
identity_policies      ["training-admin"]
policies               ["default" "training-admin"]
token_meta_username    bsmith

Notice that the user, bsmith inherited the training-admin policy from the Training Admin group (training_admin) which Bob Smith entity is a member of.

Verify that bsmith can perform the operations permitted by the training-admin policy.

# Set the target namespace as an env variable
$ export VAULT_NAMESPACE="education/training"

# Create a new namespace called 'vault-training'
$ vault namespace create vault-training
Success! Namespace created at: education/training/vault-training/

# Enable key/value v1 secrets engine at team-secret
$ vault secrets enable -path=team-secret -version=1 kv
Success! Enabled the kv secrets engine at: team-secret/

When you are done testing, unset the VAULT_NAMESPACE environment variable.

$ unset VAULT_NAMESPACE

API call using cURL

$ curl --header "X-Vault-Namespace: education/training" \
       --request POST \
       --data '{"password": "password"}' \
       http://127.0.0.1:8200/v1/auth/userpass/login/bsmith | jq
{
   ...
   "auth": {
     "client_token": "5YNNjDDl6D8iW3eGQIlU0q.9dKXw",
      "accessor": "6TVkDhdvEQXO2JaD64TVLv.9dKXw",
      "display_name": "education-training-auth-userpass-bsmith",
      "policies": [
        "default",
        "training-admin"
      ],
      "token_policies": [
        "default"
      ],
      "identity_policies": [
        "training-admin"
      ],
      "external_namespace_policies": {
        "28dk2": [
          "edu-admin"
        ]
      },
      "metadata": {
        "username": "bsmith"
      },
     ...
   }
}

Notice that the user, bsmith inherited the training-admin policy from the Training Admin group which Bob Smith entity is a member of. Also, edu-admin policy is listed under external_namespace_policies.

Verify that bsmith can perform the operations permitted by the training-admin policy.

# Create a new namespace called 'vault-training'
# Be sure to use generated bsmith's client token
$ curl --header "X-Vault-Token: 5YNNjDDl6D8iW3eGQIlU0q.9dKXw" \
       --request POST \
       http://127.0.0.1:8200/v1/education/training/sys/namespaces/web-app

# Enable key/value v1 secrets engine at team-secret
$ curl --header "X-Vault-Token: 5YNNjDDl6D8iW3eGQIlU0q.9dKXw" \
       --request POST \
       --data '{"type": "kv"}' \
       http://127.0.0.1:8200/v1/education/training/sys/mounts/edu-secret

Web UI

Open a web browser and launch the Vault UI (e.g. http://127.0.01:8200/ui). If you are already logged in, sign out.
At the Sign in to Vault, set the Namespace to education/training.
Select the Userpass tab, and enter bsmith in the Username field, and password in the Password field.
Click Sign in.
To add a new namespace, select Access.
Select Namespaces and then click Create a namespace.
Enter vault-training in the Path field, and then click Save.
Select Secrets, and then Enable new engine.
Select KV from the Secrets engine type drop-down list, and enter team-secret in the Path field.
Click Enable Engine to finish.

~> Summary: As this guide demonstrated, each namespace you created behaves as an isolated Vault environment. Once you sign into a namespace, there is no visibility into other namespaces regardless of its hierarchical relationship. Tokens, policies, and secrets engines are tied to its namespace; therefore, each client must acquire a valid token for each namespace to access their secrets.

Additional Discussion

For the simplicity, this guide used the username and password (userpass) auth method which was enabled at the education namespace. However, most likely, your organization uses LDAP auth method which is enabled in the root namespace instead.

In such as case, here are the steps to create the "Training Admin" group as described in this guide.

Enable and configure the desired auth method (e.g. LDAP) in the root namespace.

$ vault auth enable ldap

$ vault write auth/ldap/config \
        url="ldap://ldap.example.com" \
        userdn="ou=Users,dc=example,dc=com" \
        groupdn="ou=Groups,dc=example,dc=com" \
        groupfilter="(&(objectClass=group)(member:1.2.840.113556.1.4.1941:={{.UserDN}}))" \
        groupattr="cn" \
        upndomain="example.com" \
        certificate=@ldap_ca_cert.pem \
        insecure_tls=false \
        starttls=true

Create an external group in the root namespace.

# Get the mount accessor for ldap auth method and save it in accessor.txt file
$ vault auth list -format=json \
        | jq -r '.["ldap/"].accessor' > accessor.txt

# Create an external group and save the generated group ID in group_id.txt
$ vault write -format=json identity/group name="training_admin_root" \
        type="external" \
        | jq -r ".data.id" > group_id.txt    

# Create a group alias - assuming that the group name in LDAP is "ops_training"
$ vault write -format=json identity/group-alias name="ops_training" \
        mount_accessor=$(cat accessor.txt) \
        canonical_id=$(cat group_id.txt)

In the education/training namespace, create an internal group which has the external group (training_admin_root) as its member.

$ vault write -namespace=education/training identity/group \
        name="Training Admin" \
        policies="training-admin" \
        member_group_ids=$(cat group_id.txt)

Next steps

Refer to the Sentinel Policies guide if you need to write policies that allow you to embed finer control over the user access across those namespaces.

27 KiB Raw Blame History

Secure Multi-Tenancy with Namespaces

Reference Material

Estimated Time to Complete

Personas

Challenge

Solution

Prerequisites

Steps

Step 1: Create namespaces

CLI command

API call using cURL

Web UI

Step 2: Write Policies

Policy for education admin

Policy for training admin

CLI command

API call using cURL

Web UI

Step 3: Setup entities and groups

CLI Command

Web UI

Step 4: Test the organization admin user

CLI Command

API call using cURL

Web UI

Step 5: Test the team admin user

CLI Command

API call using cURL

Web UI

Additional Discussion

Next steps

27 KiB

Raw Blame History