When managing Kubernetes applications with ArgoCD and Helm, securing sensitive data such as passwords, API keys, and other secrets is crucial. Bitnami Sealed Secrets provides a powerful way to encrypt secrets that can be safely stored in Git and used within your ArgoCD and Helm workflows.
This guide will cover how to integrate Sealed Secrets with ArgoCD and Helm to securely manage secrets in your values.yaml files for Helm charts.
Overview
ArgoCD allows you to deploy and manage applications in Kubernetes using GitOps principles, where the desired state of your applications is stored in Git repositories. Helm, on the other hand, is a package manager for Kubernetes that simplifies application deployment through reusable templates (Helm charts).
Bitnami Sealed Secrets provides a way to encrypt your Kubernetes secrets using a public key, which can only be decrypted by the Sealed Secrets controller running in your Kubernetes cluster. This allows you to safely store and version-control encrypted secrets.
1. Prerequisites
Before you begin, ensure you have the following set up:
- Kubernetes Cluster: A running Kubernetes cluster.
- ArgoCD: Installed and configured in your Kubernetes cluster.
- Helm: Installed on your local machine.
- Sealed Secrets: The Sealed Secrets controller installed in your Kubernetes cluster.
- kubeseal: The Sealed Secrets CLI tool installed on your local machine.
2. Setting Up Sealed Secrets
If you haven’t already installed the Sealed Secrets controller, follow these steps:
Install the Sealed Secrets Controller
Using Helm:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install sealed-secrets-controller bitnami/sealed-secretsOr using kubectl:
kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.20.2/controller.yaml3. Encrypting Helm Values Using Sealed Secrets
In this section, we’ll demonstrate how to encrypt sensitive values in a Helm values.yaml file using Sealed Secrets, ensuring they are securely managed and version-controlled.
Step 1: Identify Sensitive Data in values.yaml
Suppose you have a Helm chart with a values.yaml file that contains sensitive information:
# values.yaml
database:
username: admin
password: my-secret-password # Sensitive data
host: db.example.comStep 2: Create a Kubernetes Secret Manifest
First, create a Kubernetes Secret manifest for the sensitive data:
# my-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: my-database-secret
namespace: default
type: Opaque
data:
password: bXktc2VjcmV0LXBhc3N3b3Jk # base64 encoded 'my-secret-password'Step 3: Encrypt the Secret Using kubeseal
Use the kubeseal CLI to encrypt the secret using the public key from the Sealed Secrets controller:
kubeseal --format yaml < my-secret.yaml > my-sealedsecret.yamlThis command generates a SealedSecret resource that is safe to store in your Git repository:
# my-sealedsecret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: my-database-secret
namespace: default
spec:
encryptedData:
password: AgA7SyR4l5URRXg... # Encrypted dataStep 4: Modify the Helm Chart to Use the SealedSecret
In your Helm chart, modify the values.yaml file to reference the Kubernetes Secret instead of directly embedding sensitive values:
# values.yaml
database:
username: admin
secretName: my-database-secret
host: db.example.comIn the deployment.yaml template of your Helm chart, reference the secret:
# templates/deployment.yaml
env:
- name: DB_USERNAME
value: {{ .Values.database.username }}
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.database.secretName }}
key: passwordThis approach keeps the sensitive data out of the values.yaml file, instead storing it securely in a SealedSecret.
Step 5: Apply the SealedSecret to Your Kubernetes Cluster
Apply the SealedSecret to your cluster:
kubectl apply -f my-sealedsecret.yamlThe Sealed Secrets controller will decrypt the SealedSecret and create the corresponding Kubernetes Secret.
4. Deploying the Helm Chart with ArgoCD
Step 1: Create an ArgoCD Application
You can create an ArgoCD application either via the ArgoCD UI or using the argocd CLI. Here’s how to do it with the CLI:
argocd app create my-app \
--repo https://github.com/your-org/your-repo.git \
--path helm/my-app \
--dest-server https://kubernetes.default.svc \
--dest-namespace defaultIn this command:
--repo: The URL of the Git repository where your Helm chart is stored.--path: The path to the Helm chart within the repository.--dest-server: The Kubernetes API server.--dest-namespace: The namespace where the application will be deployed.
Step 2: Sync the Application
Once the ArgoCD application is created, ArgoCD will monitor the Git repository for changes and automatically synchronize the Kubernetes cluster with the desired state.
- Auto-Sync: If auto-sync is enabled, ArgoCD will automatically deploy the application whenever changes are detected in the Git repository.
- Manual Sync: You can manually trigger a sync using the ArgoCD UI or CLI:
argocd app sync my-app5. Example: Encrypting and Using Multiple Secrets
In more complex scenarios, you might have multiple sensitive values to encrypt. Here’s how you can manage multiple secrets:
Step 1: Create Multiple Kubernetes Secrets
# db-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: db-secret
namespace: default
type: Opaque
data:
username: YWRtaW4= # base64 encoded 'admin'
password: c2VjcmV0cGFzcw== # base64 encoded 'secretpass'
# api-key-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: api-key-secret
namespace: default
type: Opaque
data:
apiKey: c2VjcmV0YXBpa2V5 # base64 encoded 'secretapikey'Step 2: Encrypt the Secrets Using kubeseal
Encrypt each secret using kubeseal:
kubeseal --format yaml < db-secret.yaml > db-sealedsecret.yaml
kubeseal --format yaml < api-key-secret.yaml > api-key-sealedsecret.yamlStep 3: Apply the SealedSecrets
Apply the SealedSecrets to your Kubernetes cluster:
kubectl apply -f db-sealedsecret.yaml
kubectl apply -f api-key-sealedsecret.yamlStep 4: Reference Secrets in Helm Values
Modify your Helm values.yaml file to reference these secrets:
# values.yaml
database:
secretName: db-secret
api:
secretName: api-key-secretIn your Helm chart templates, use the secrets:
# templates/deployment.yaml
env:
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: {{ .Values.database.secretName }}
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.database.secretName }}
key: password
- name: API_KEY
valueFrom:
secretKeyRef:
name: {{ .Values.api.secretName }}
key: apiKey6. Best Practices
- Environment-Specific Secrets: Use different SealedSecrets for different environments (e.g., staging, production). Encrypt and store these separately.
- Backup and Rotation: Regularly back up the SealedSecrets and rotate the keys used by the Sealed Secrets controller.
- Audit and Monitor: Enable logging and monitoring in your Kubernetes cluster to track the use of SealedSecrets.
When creating a Kubernetes Secret, the data must be base64 encoded before you can encrypt it with Sealed Secrets. This is because Kubernetes Secrets expect the values to be base64 encoded, and Sealed Secrets operates on the same principle since it wraps around Kubernetes Secrets.
Why Base64 Encoding?
Kubernetes Secrets require data to be stored as base64-encoded strings. This encoding is necessary because it allows binary data (like certificates, keys, or complex strings) to be stored as plain text in YAML files.
Steps for Using Sealed Secrets with Base64 Encoding
Here’s how you typically work with base64 encoding in the context of Sealed Secrets:
1. Base64 Encode Your Secret Data
Before creating a Kubernetes Secret, you need to base64 encode your sensitive data. For example, if your secret is a password like my-password, you would encode it:
echo -n 'my-password' | base64This command outputs the base64-encoded version of my-password:
bXktcGFzc3dvcmQ=2. Create the Kubernetes Secret Manifest
Create a Kubernetes Secret YAML file with the base64-encoded value:
apiVersion: v1
kind: Secret
metadata:
name: my-secret
namespace: default
type: Opaque
data:
password: bXktcGFzc3dvcmQ= # base64 encoded 'my-password'3. Encrypt the Secret Using kubeseal
Once the Kubernetes Secret manifest is ready, encrypt it using the kubeseal command:
kubeseal --format yaml < my-secret.yaml > my-sealedsecret.yamlThis command creates a SealedSecret, which can safely be committed to version control.
4. Apply the SealedSecret
Finally, apply the SealedSecret to your Kubernetes cluster:
kubectl apply -f my-sealedsecret.yamlThe Sealed Secrets controller in your cluster will decrypt the SealedSecret and create the corresponding Kubernetes Secret with the base64-encoded data.
Summary
- Base64 Encoding: You must base64 encode your secret data before creating a Kubernetes Secret manifest because Kubernetes expects the data to be in this format.
- Encrypting with Sealed Secrets: After creating the Kubernetes Secret manifest with base64-encoded data, use Sealed Secrets to encrypt the entire manifest.
- Applying SealedSecrets: The Sealed Secrets controller will decrypt the SealedSecret and create the Kubernetes Secret with the correctly encoded data.
Conclusion
By combining ArgoCD, Helm, and Sealed Secrets, you can securely manage and deploy Kubernetes applications in a GitOps workflow. Sealed Secrets ensure that sensitive data remains encrypted and safe, even when stored in a version control system, while Helm provides the flexibility to manage complex applications. Following the steps outlined in this guide, you can confidently manage secrets in your Kubernetes deployments, ensuring both security and efficiency.