Helmfile Deployment

Deploy the complete Kindo stack using Helmfile with pure Helm values. This approach automates the manual steps from the Installation Guide into a single declarative workflow.

Overview

Multi-namespace — Each application runs in its own namespace
Pure Helm — Per-app values files match chart structure exactly
Declarative — Single helmfile sync deploys everything
Dependency ordering — Peripheries → Secrets → Applications

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      helmfile.yaml.gotmpl                       │
│                                                                 │
│  environments/default.yaml     values/*.yaml                    │
│  (enable/disable apps)         (per-app Helm values)            │
└──────────────────────────────────┬──────────────────────────────┘
                                   │
           ┌───────────────────────┼───────────────────────┐
           │                       │                       │
           ▼                       ▼                       ▼
┌─────────────────────┐  ┌─────────────────┐  ┌─────────────────────┐
│   PERIPHERIES       │  │  kindo-secrets  │  │   APPLICATIONS      │
│                     │  │  (kindo-system) │  │                     │
│ - unleash (unleash) │  │                 │  │ - api               │
│ - unleash-edge      │  │ Creates secrets │  │ - next              │
│ - qdrant (qdrant)   │  │ in all app      │  │ - litellm           │
│ - presidio          │  │ namespaces      │  │ - llama-indexer     │
│ - speaches          │  │                 │  │ - credits           │
└─────────────────────┘  └─────────────────┘  │ - external-sync     │
                                              │ - external-poller   │
                                              │ - audit-log-exporter│
                                              │ - cerbos            │
                                              │ - ssoready          │
                                              │ - prisma-migrations │
                                              └─────────────────────┘

Prerequisites

Kubernetes 1.32+
Helm 3.8+
Helmfile 0.150+
helm-diff plugin
Access to registry.kindo.ai
External infrastructure:
- PostgreSQL (databases: kindo, litellm, ssoready)
- Redis
- RabbitMQ
- S3-compatible storage

See Infrastructure Requirements for full details.

Quick Start

1. Install Helmfile

brew install helmfile
helm plugin install https://github.com/databus23/helm-diff

HELMFILE_VERSION="0.162.0"
curl -fsSL -o helmfile.tar.gz \
  "https://github.com/helmfile/helmfile/releases/download/v${HELMFILE_VERSION}/helmfile_${HELMFILE_VERSION}_linux_amd64.tar.gz"
tar -xzf helmfile.tar.gz helmfile
sudo mv helmfile /usr/local/bin/
rm helmfile.tar.gz

helm plugin install https://github.com/databus23/helm-diff

HELMFILE_VERSION="0.162.0"
curl -fsSL -o helmfile.tar.gz \
  "https://github.com/helmfile/helmfile/releases/download/v${HELMFILE_VERSION}/helmfile_${HELMFILE_VERSION}_linux_arm64.tar.gz"
tar -xzf helmfile.tar.gz helmfile
sudo mv helmfile /usr/local/bin/
rm helmfile.tar.gz

helm plugin install https://github.com/databus23/helm-diff

Verify installation:

make check-prereqs

2. Generate Secrets

make generate-secrets
vi values/secrets.yaml

Required configuration in values/secrets.yaml:

domains:
  base: "your-domain.com"

postgres:
  main:
    connectionString: "postgresql://user:pass@host:5432/kindo"
  litellm:
    connectionString: "postgresql://user:pass@host:5432/litellm"
  ssoready:
    connectionString: "postgresql://user:pass@host:5432/ssoready"

redis:
  connectionString: "redis://host:6379"

rabbitmq:
  connectionString: "amqp://user:pass@host:5672"

storage:
  accessKey: "your-access-key"
  secretKey: "your-secret-key"
  bucketName: "your-bucket"
  region: "us-east-1"

3. Configure Environment

Edit environments/default.yaml to set the app version and enable/disable services:

appVersion: "2025.10.1-rc.1"
storageClass: ""  # Leave empty for cluster default

api:
  enabled: true
next:
  enabled: true
litellm:
  enabled: true
llama-indexer:
  enabled: true
credits:
  enabled: true
external-sync:
  enabled: true
external-poller:
  enabled: true
audit-log-exporter:
  enabled: true
cerbos:
  enabled: true
ssoready:
  enabled: true
prisma-migrations:
  enabled: true

4. Setup Helm and Registry

make setup-repos

helm registry login registry.kindo.ai

./scripts/setup-registry-credentials.sh registry.kindo.ai \
  'robot$your_user' 'your-password'

5. Deploy

Deploy peripheries (Unleash, Qdrant, Presidio, Speaches):
Terminal window
```
make peripheries
```
Import Unleash feature flags — Port-forward Unleash and create a Personal Access Token:
Terminal window
```
kubectl port-forward svc/unleash 4242:4242 -n unleash
```
Open http://localhost:4242, log in, create a Personal API token from your profile settings, then import:
Terminal window
```
make import-flags TOKEN='user:your-token-here'
```
Deploy secrets and applications:
Terminal window
```
make secrets
make applications
```

To deploy everything at once (if feature flags are already imported):

make all

Deploy a single application:

make applications APP=api

Preview changes before deploying:

make diff
make diff APP=api

6. Operations

# Rollout restart all application deployments
make restart

# Restart a single app
make restart APP=api

# Check deployment status
make status

# Uninstall everything
make uninstall

How It Works

Helmfile Orchestration

Helmfile (helmfile.yaml.gotmpl) is the orchestration layer on top of Helm. It defines which charts to deploy, where (namespace), and in what order via needs dependencies.

# Example release definition
- name: api
  chart: oci://registry.kindo.ai/kindo-helm/api
  version: "{{ .Values.appVersion }}"
  namespace: api
  labels:
    group: applications
  condition: api.enabled
  needs:
    - kindo-system/kindo-secrets
  values:
    - values/api.yaml.gotmpl

Values Files

Plain YAML (.yaml) — Static configuration
Go Template (.yaml.gotmpl) — Dynamic configuration that reads from secrets.yaml

The .gotmpl extension tells Helmfile to process the file as a Go template before passing to Helm.

Secrets Flow

generate-secrets.sh → values/secrets.yaml → .gotmpl files + kindo-secrets chart

.gotmpl values files read secrets for peripheries (Unleash credentials, etc.)
The kindo-secrets chart reads secrets.yaml and creates Kubernetes Secrets in each app namespace

Deployment Order

Helmfile ensures correct ordering via needs:

Peripheries (no dependencies) — Unleash, Qdrant, Presidio, Speaches
kindo-secrets (needs peripheries) — Creates secrets in all app namespaces
Applications (needs kindo-secrets) — API, Next, LiteLLM, and all other services

Directory Structure

kindo-e2e/
├── Makefile                      # CLI entry point
├── helmfile.yaml.gotmpl          # Helmfile orchestration
├── generate-secrets.sh           # Secret generator
├── charts/
│   ├── kindo-secrets/            # Creates K8s Secrets for all apps
│   ├── presidio/
│   ├── speaches/
│   └── feature_flags.json        # Unleash bootstrap
├── environments/
│   └── default.yaml              # App versions & enable/disable
├── scripts/
│   └── setup-registry-credentials.sh
└── values/
    ├── secrets.yaml              # Your config (edit this!)
    ├── secrets.yaml.template
    ├── unleash.yaml.gotmpl       # Periphery values
    ├── api.yaml.gotmpl           # Application values
    ├── next.yaml
    └── ...                       # One file per service

Make Commands

Command	Description
`make help`	Show help and available commands
`make generate-secrets`	Generate secrets file from template
`make check-prereqs`	Verify prerequisites are installed
`make setup-repos`	Add Helm repositories
`make peripheries`	Deploy peripheral services
`make import-flags TOKEN='...'`	Import Unleash feature flags
`make secrets`	Deploy secrets only
`make applications`	Deploy all applications
`make applications APP=<name>`	Deploy single application
`make all`	Deploy everything
`make diff`	Preview changes before deploying
`make restart`	Rollout restart all deployments
`make restart APP=<name>`	Rollout restart single app
`make status`	Show deployment status
`make template`	Render templates locally (debug)
`make uninstall`	Uninstall everything

Common Workflows

Adding a new environment variable to an app:

# Option 1: Edit the secret template
stringData:
  MY_NEW_VAR: {{ .Values.mySection.myValue | quote }}

# Option 2: Use serviceOverrides in secrets.yaml
serviceOverrides:
  api:
    MY_NEW_VAR: "my-value"

Changing app version:

appVersion: "2025.10.2"

Disabling an app:

prisma-migrations:
  enabled: false

Namespace Reference

Service	Namespace
Unleash + Edge	`unleash`
Qdrant	`qdrant`
Presidio	`presidio`
Speaches	`speaches`
API	`api`
Next.js	`next`
LiteLLM	`litellm`
Llama Indexer	`llama-indexer`
Credits	`credits`
External Sync	`external-sync`
External Poller	`external-poller`
Audit Log Exporter	`audit-log-exporter`
Cerbos	`cerbos`
SSOReady	`ssoready`
Prisma Migrations	`prisma-migrations`
Secrets Chart	`kindo-system`

Troubleshooting

Check Status

make status
kubectl get pods -A
kubectl get pods -n api

View Logs

kubectl logs -l app.kubernetes.io/name=api -n api
kubectl logs -l app.kubernetes.io/name=litellm -n litellm
kubectl logs -n unleash -l app.kubernetes.io/name=unleash

Verify Secrets

kubectl get secrets -A | grep "\-env"
kubectl get secret api-env -n api -o jsonpath='{.data}' | jq 'keys'
kubectl get secret api-env -n api -o jsonpath='{.data.DATABASE_URL}' | base64 -d

Common Issues

Secrets not found:

make secrets
kubectl get secret api-env -n api

Pods stuck in ImagePullBackOff:

kubectl get secret registry-credentials -n <namespace>
kubectl describe pod <pod-name> -n <namespace>
./scripts/setup-registry-credentials.sh registry.kindo.ai 'robot$user' 'pass'

Pod CrashLoopBackOff:

kubectl logs <pod-name> -n <namespace>
kubectl logs <pod-name> -n <namespace> --previous
kubectl describe pod <pod-name> -n <namespace> | grep -A5 "Limits:"

Healthcheck failures (pod restarts after ~5 min):

kubectl describe deployment <name> -n <namespace> | grep -A5 "Liveness:"

Debug Helm/Helmfile

helmfile template -l name=api
helm template kindo-secrets ./charts/kindo-secrets -f values/secrets.yaml
helmfile --debug sync