GitOps Secrets Management for Production Kubernetes

NetworkPolicy Enforcement Deep Dive: eBPF (Cilium) vs iptables (Calico)

A deep technical comparison of NetworkPolicy enforcement between eBPF-based CNIs (Cilium) and iptables-based CNIs (Calico). Covers implementation internals, real performance benchmarks at scale, L7 policy trade-offs, migration strategies, and production debugging patterns.

GitOps promises a single source of truth for infrastructure: all configuration in Git, automatically synced to production. Secrets break this promise. You cannot store database passwords, API tokens, and TLS certificates in plain text in a Git repository, but you also cannot manage them outside Git without losing the reproducibility and audit trail that GitOps provides.

This article covers five production-proven patterns for managing secrets in GitOps workflows, ranked by complexity and capability.

Pattern 1: Sealed Secrets (Bitnami)

Sealed Secrets allow you to encrypt Kubernetes Secrets into SealedSecret custom resources that are safe to store in Git. Only the controller running in your cluster can decrypt them.

How It Works

1. Install the Sealed Secrets controller in your cluster
2. Use kubeseal to encrypt a Secret into a SealedSecret
3. Commit the SealedSecret YAML to Git
4. ArgoCD syncs the SealedSecret to the cluster
5. The controller decrypts it into a regular Secret

Usage

kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# Encrypt a literal value
echo -n "my-db-password" | kubeseal --raw \
  --name db-password \
  --namespace production \
  --from-file=/dev/stdin

Production Considerations

Pattern 2: External Secrets Operator

External Secrets Operator synchronizes secrets from external providers (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault) into Kubernetes Secrets.

Usage

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: secret/production/database
        property: password

Pattern 3: SOPS with Age/GPG

SOPS (Secrets OPerationS) is a file-level encryption tool. You encrypt individual values in YAML files while keeping the structure readable:

apiVersion: v1
kind: Secret
metadata:
  name: api-credentials
type: Opaque
stringData:
  api_key: ENC[AES256_GCM,data:encrypted-base64,iv:...,tag:...]

Sops encrypts specific values in your YAML, and ArgoCD plugins or CI pipelines decrypt during sync. This keeps the full secret manifest in Git while protecting sensitive values.

Pattern 4: HashiCorp Vault with ArgoCD

For teams already running Vault, ArgoCD's Vault plugin (argocd-vault-plugin) substitutes Vault secrets during sync:

apiVersion: v1
kind: Secret
metadata:
  name: app-config
  annotations:
    avp.kubernetes.io/path: "secret/data/production/app"
type: Opaque
stringData:
  db_password: <vault:secret/data/production/app#db_password>

Pattern 5: CSI Secrets Store Driver

The CSI Secrets Store Driver mounts secrets as volumes — never as Kubernetes Secrets. This is the most secure pattern because secrets never enter etcd:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: vault-db
spec:
  provider: vault
  parameters:
    vaultAddress: https://vault.example.com
    roleName: app-role
    objects: |
      - objectName: db_password
        secretPath: secret/data/production/database
        secretKey: password

Choosing the Right Pattern

For most teams starting out, Sealed Secrets provides the best balance of security, simplicity, and Git-native workflow. As your infrastructure scales, adding External Secrets Operator for dynamic secrets (database passwords that rotate, API tokens with expiry) is the natural next step.

Production Checklist

• Encrypted secrets are reviewed in pull requests
• Secret rotation is automated (not manual)
• Controller/operator backups are tested
• Access to the encryption key is audited
• Cluster recovery includes secret restoration
• Secrets are scoped per namespace and environment
• Sync fails if secrets cannot be decrypted (fail secure)