New to KubeVault? Please start here.

Deploy VaultServer with Amazon S3 Backend

You can easily deploy and manage OpenBao in the Kubernetes cluster using KubeVault operator. In this tutorial, we are going to deploy Vault on the Kubernetes cluster using KubeVault operator, with Amazon S3 (or any S3-compatible object storage) as the storage backend.

To keep everything isolated, we are going to use a separate namespace called demo throughout this tutorial.

$ kubectl create ns demo
namespace/demo created

Deploy VaultServer with Amazon S3 Backend

Create beforehand: The S3 bucket. Vault does not create it.

Prepare the S3 Bucket

Vault stores its data as encrypted objects in an S3 bucket. Use Amazon S3 or an S3-compatible object storage, such as MinIO, that is reachable from the Vault pods. This example uses MinIO at http://minio.demo.svc:9000 with the bucket vault-bucket. The bucket must exist before Vault starts. Vault does not create it.

Create Credential Secret:

The credential secret holds the access key pair in the keys access_key and secret_key. Here, we use the MinIO credentials.

$ kubectl create secret generic s3-cred -n demo \
    --from-literal=access_key=minioadmin \
    --from-literal=secret_key=minioadmin
secret/s3-cred created

Demo only: minioadmin/minioadmin are MinIO’s default credentials. In production, use an access key pair that is limited to the Vault bucket.

To use Amazon S3, create the bucket first, remove endpoint and forcePathStyle from the VaultServer, set region to the region of the bucket and put the access key pair of an IAM user in the secret. The IAM user needs permission to list the bucket and to read, write and delete the objects in it. If you use temporary credentials, add the session token to the secret in the key session_token.

Deploy Vault using KubeVault

We’re going to use Kubernetes secret to store the unseal-keys & root-token. A sample VaultServer with S3 backend manifest file may look like this:

apiVersion: kubevault.com/v1alpha2
kind: VaultServer
metadata:
  name: vault
  namespace: demo
spec:
  version: sigilr-2.6.1.2
  replicas: 1
  allowedSecretEngines:
    namespaces:
      from: All
  backend:
    s3:
      endpoint: "http://minio.demo.svc:9000"
      bucket: "vault-bucket"
      region: "us-east-1"
      credentialSecretRef:
        name: s3-cred
      forcePathStyle: true
  unsealer:
    secretShares: 5
    secretThreshold: 3
    mode:
      kubernetesSecret:
        secretName: vault-keys
  terminationPolicy: WipeOut

forcePathStyle makes the client address the bucket in the path of the URL (http://minio.demo.svc:9000/vault-bucket), which most S3-compatible storages need. The S3 storage backend does not support high availability, so the VaultServer runs with a single replica.

Now, let’s deploy the VaultServer:

$ kubectl apply -f vaultserver.yaml
vaultserver.kubevault.com/vault created

KubeVault operator will create a AppBinding CRD on VaultServer deployment, which contains the necessary information to take backup of the Vault instances. It’ll have the same name & be created on the same namespace as the Vault. Read more about AppBinding here.

$ kubectl get appbinding -n demo vault -oyaml
apiVersion: appcatalog.appscode.com/v1alpha1
kind: AppBinding
metadata:
  name: vault
  namespace: demo
spec:
  appRef:
    apiGroup: kubevault.com
    kind: VaultServer
    name: vault
    namespace: demo
  clientConfig:
    service:
      name: vault
      port: 8200
      scheme: http
  parameters:
    apiVersion: config.kubevault.com/v1alpha1
    backend: s3
    backupTokenSecretRef:
      name: vault-backup-token
    deploymentMode: Local
    kind: VaultServerConfiguration
    kubernetes:
      serviceAccountName: vault
      tokenReviewerServiceAccountName: vault-k8s-token-reviewer
      usePodServiceAccountForCSIDriver: true
    path: kubernetes
    stash:
      addon:
        backupTask:
          name: vault-backup-1.10.3
          params:
            - name: keyPrefix
              value: k8s.62e86b47-6e7c-4849-89fb-88062a22f451.demo.vault
        restoreTask:
          name: vault-restore-1.10.3
          params:
            - name: keyPrefix
              value: k8s.62e86b47-6e7c-4849-89fb-88062a22f451.demo.vault
    unsealer:
      mode:
        kubernetesSecret:
          secretName: vault-keys
      secretShares: 5
      secretThreshold: 3
    vaultRole: vault-policy-controller
  type: VaultServer
  version: sigilr-2.6.1.2

Now, let’s wait until all the vault pods come up & VaultServer phase becomes Ready.

$ kubectl get pods -n demo -l app.kubernetes.io/instance=vault
NAME      READY   STATUS    RESTARTS   AGE
vault-0   2/2     Running   0          2m8s
$ kubectl get vaultserver -n demo
NAME    REPLICAS   VERSION          STATUS   AGE
vault   1          sigilr-2.6.1.2   Ready    2m50s

At this stage, we’ve successfully deployed Vault using KubeVault operator with S3 as the storage backend.

Let’s write some data in a KV secret engine. Let’s export the necessary environment variables & port-forward from vault service or exec into the vault pod in order to interact with it.

$ export VAULT_TOKEN=$(kubectl vault root-token get vaultserver vault -n demo --value-only)
$ export VAULT_ADDR='http://127.0.0.1:8200'
$ kubectl port-forward -n demo svc/vault 8200

Now check whether Vault server can be accessed (run this in another terminal):

$ vault status
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         2.6.1
Storage Type    s3
Cluster Name    vault-cluster-3f8a1c92
Cluster ID      56c3a681-f799-2310-9813-d67877d25450
HA Enabled      false

We can see the currently enabled list of secret engines.

$ vault secrets list
Path                                       Type         Accessor              Description
----                                       ----         --------              -----------
cubbyhole/                                 cubbyhole    cubbyhole_bb7c56f9    per-token private secret storage
identity/                                  identity     identity_fa8431fa     identity store
k8s.kubevault.com.kv.demo.vault-health/    kv           kv_5129d194           n/a
sys/                                       system       system_c7e0879a       system endpoints used for control, policy and debugging

Let’s enable a KV type secret engine:

$ vault secrets enable kv
Success! Enabled the kv secrets engine at: kv/

Write some dummy data in the secret engine path:

$ vault kv put kv/name name=appscode
Success! Data written to: kv/name

Verify data written in KV secret engine:

$ vault kv get kv/name
==== Data ====
Key     Value
---     -----
name    appscode

Cleanup

To clean up the Kubernetes resources created by this tutorial, run:

$ kubectl delete vaultserver -n demo vault
$ kubectl delete secret -n demo s3-cred
$ kubectl delete ns demo

With terminationPolicy: WipeOut, the operator connects to the bucket to remove the objects Vault stored there, so the endpoint has to be reachable until the VaultServer is gone.

To see all the fields you can set under spec.backend.s3, please check the S3 storage backend concept.

For more details on how to interact with the vault server, please check the Vault Server guide.