AWS Applications Deployment Guide

This guide provides a streamlined approach for deploying Kindo application services using the kindo-applications module. Users only need to modify the terraform.tfvars file.

Table of Contents

1. Overview
2. Prerequisites
3. Quick Start
4. Configuration
5. Deployment
6. Verification
7. Troubleshooting

Overview

The kindo-applications module deploys the core Kindo services:

• API Service: Backend REST API (Node.js)
• Next.js Frontend: Web application UI
• LiteLLM: AI model proxy and router
• Llama Indexer: Document indexing service
• External Poller: Background job processor
• External Sync: Data synchronization service
• Credits Service: Usage tracking and billing
• Audit Log Exporter: Compliance and logging
• Cerbos: Authorization policy engine
• Task Worker TS: TypeScript-based task processing service
• Nango: Integration platform for third-party APIs
• Hatchet: Workflow orchestration engine

Prerequisites

Before deploying applications, ensure you have:

1. ✅ Completed infrastructure deployment (01-infrastructure-setup)
2. ✅ Deployed secrets management (02-secrets-management)
3. ✅ Installed peripheries (03-peripheries-deployment)
4. ✅ Access to the EKS cluster

Verify prerequisites:

# Get cluster name from infrastructurecd ../infra-aws
CLUSTER_NAME=$(terraform output -json infra_outputs | jq -r '.eks_cluster_name')
REGION=$(terraform output -json infra_outputs | jq -r '.region')
# Update kubeconfig
aws eks update-kubeconfig --name $CLUSTER_NAME --region $REGION
# Verify cluster access
kubectl get nodes
# Check External Secrets Operator
kubectl get clustersecretstore
# Should show: aws-secrets-manager Ready
# Check ALB Ingress Controller
kubectl get ingressclass
# Should show: alb

Configuration

terraform.tfvars Reference

The stack comes with a pre-configured main.tf, provider.tf, and other necessary files. You only need to modify terraform.tfvars:

# ======================================================================
# KINDO APPLICATIONS CONFIGURATION
# ======================================================================
# Only modify this file - all other files are pre-configured
# ======================================================================

# -----------------------------
# CORE CONFIGURATION
# -----------------------------
# These values should match your infrastructure deployment
project_name     = "example-project"
environment_name = "dev"
aws_region       = "us-west-2"

# -----------------------------
# REGISTRY CONFIGURATION
# -----------------------------
# Credentials for Kindo Helm charts
registry_username = "your-registry-username"
registry_password = "your-registry-password"

# -----------------------------
# APPLICATION ENABLEMENT
# -----------------------------
# Enable/disable individual applications
enable_api                = true
enable_next               = true
enable_litellm            = true
enable_llama_indexer      = true
enable_credits            = true
enable_external_sync      = true
enable_external_poller    = true
enable_audit_log_exporter = true
enable_cerbos             = true
enable_task_worker_ts     = true
enable_nango              = true
enable_hatchet            = true

# -----------------------------
# APPLICATION VERSIONS
# -----------------------------
# Helm chart versions for each application
api_chart_version                = "0.0.15"
next_chart_version               = "0.0.15"
litellm_chart_version            = "0.0.15"
llama_indexer_chart_version      = "0.0.15"
credits_chart_version            = "0.0.15"
external_sync_chart_version      = "0.0.15"
external_poller_chart_version    = "0.0.15"
audit_log_exporter_chart_version = "0.0.15"
cerbos_chart_version             = "0.0.15"
task_worker_ts_chart_version     = "0.0.15"
nango_chart_version              = "0.0.15"
hatchet_chart_version            = "0.0.15"

# -----------------------------
# APPLICATION SCALING
# -----------------------------
# Number of replicas for each service
api_replica_count                = 2
next_replica_count               = 2
litellm_replica_count            = 2
llama_indexer_replica_count      = 1
credits_replica_count            = 1
external_sync_replica_count      = 1
external_poller_replica_count    = 1
audit_log_exporter_replica_count = 1
cerbos_replica_count             = 2
task_worker_ts_replica_count     = 2
nango_replica_count              = 1
hatchet_replica_count            = 1

# -----------------------------
# ADVANCED CONFIGURATION (Optional)
# -----------------------------
# Override default resource requests/limits if needed
# api_resource_overrides = {
#   "resources.limits.cpu"       = "2000m"
#   "resources.limits.memory"    = "4Gi"
#   "resources.requests.cpu"     = "1000m"
#   "resources.requests.memory"  = "2Gi"
# }

# next_resource_overrides = {
#   "resources.limits.cpu"       = "1000m"
#   "resources.limits.memory"    = "2Gi"
#   "resources.requests.cpu"     = "500m"
#   "resources.requests.memory"  = "1Gi"
# }

# litellm_resource_overrides = {
#   "resources.limits.cpu"       = "4000m"
#   "resources.limits.memory"    = "8Gi"
#   "resources.requests.cpu"     = "2000m"
#   "resources.requests.memory"  = "4Gi"
# }

Configuration Details

Core Configuration

• project_name: Must match the project name used in infrastructure deployment
• environment_name: Must match the environment used in infrastructure deployment
• aws_region: AWS region where your infrastructure is deployed

Registry Configuration

• registry_username: Your Kindo registry username
• registry_password: Your Kindo registry password

Application Enablement

Set to true or false to enable/disable each application.

Application Versions

Specify the Helm chart version for each application. Contact Kindo support for latest versions.

Application Scaling

Set the number of replicas for each service based on your load requirements.

Deployment

Step 1: Initialize Terraform

terraform init

Step 2: Review the Plan

terraform plan

Review the output to ensure:

• Correct number of resources will be created
• Namespaces will be created for each application
• Secrets will be referenced correctly

Step 3: Deploy Applications

terraform apply
# Type 'yes' when prompted

Deployment typically takes 5-10 minutes.

Step 4: Monitor Deployment

# Watch pod creation
kubectl get pods -A -w | grep -E "api|next|litellm|credits|cerbos|task-worker|nango|hatchet"

4. Run initial migrations

kubectl exec -n api deployment/api -- npx prisma migrate dev --name initial_migration --schema /app/backend/api/node_modules/.prisma/client/schema.prisma

Verification

1. Check Pod Status

# Quick status check
kubectl get pods -A | grep -v "kube-system\|external-secrets\|unleash"
# All pods should show Running status

2. Verify External Secrets

# Check if secrets are synced
kubectl get externalsecrets -A
# All should show SecretSynced = True

3. Test Application Endpoints

# Get the load balancer URL
ALB_URL=$(kubectl get ingress -n api -o jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}')
# Test API health
curl -s https://api.$(terraform output -raw domain_name)/health | jq .
# Test frontend
curl -I https://app.$(terraform output -raw domain_name)
# Test LiteLLM
curl -s https://litellm.$(terraform output -raw domain_name)/health | jq .

4. Check Application Logs

# API logs
kubectl logs -n api -l app.kubernetes.io/name=api --tail=50
# Frontend logs
kubectl logs -n next -l app.kubernetes.io/name=next --tail=50

Troubleshooting

Common Issues and Solutions

Pods Stuck in Pending

# Check node capacity
kubectl describe nodes | grep -A 5 "Allocated resources"
# Solution: Scale up node groups or reduce resource requests in terraform.tfvars

ImagePullBackOff

# Check registry credentialskubectl describe pod -n api <pod-name> | grep -A 10 "Events"# Solution: Verify registry_username and registry_password in terraform.tfvars

External Secrets Not Syncing

# Check External Secrets Operator logskubectl logs -n external-secrets deployment/external-secrets
# Check secret storekubectl describe clustersecretstore aws-secrets-manager
# Solution: Verify IAM permissions and secret names match

Application Not Starting

# Check pod logskubectl logs -n api <pod-name># Check environment variableskubectl exec -n api <pod-name> -- env | grep -E "DATABASE|REDIS|RABBITMQ"# Solution: Verify all required secrets are present in AWS Secrets Manager

Ingress Not Creating ALB

# Check ALB controller logskubectl logs -n kube-system deployment/aws-load-balancer-controller
# Check ingress statuskubectl describe ingress -n api
# Solution: Verify ALB controller is running and has correct IAM permissions

Quick Diagnostics Script

Create a diagnose.sh script:

#!/bin/bashecho "=== Checking Application Health ==="# Check podsecho -e "\n📦 Pod Status:"kubectl get pods -A | grep -E "api|next|litellm|credits|cerbos" | grep -v Running || echo "✅ All pods running"# Check secretsecho -e "\n🔐 External Secrets:"kubectl get externalsecrets -A | grep -v SecretSynced || echo "✅ All secrets synced"# Check ingressecho -e "\n🌐 Ingress Status:"kubectl get ingress -A | grep -v ADDRESS || echo "⚠️  No ingress found"# Check recent errorsecho -e "\n❌ Recent Errors:"kubectl get events -A --field-selector type=Warning --sort-by='.lastTimestamp' | tail -5

Scaling Applications

To scale applications after deployment:

# Scale using Terraform (recommended)# Edit terraform.tfvars and change replica counts, then:terraform apply
# Or scale directly with kubectl (temporary)kubectl scale deployment api -n api --replicas=3
kubectl scale deployment next -n next --replicas=3

Updating Applications

To update application versions:

1. Edit terraform.tfvars with new chart versions
2. Run terraform apply

# Example: Update API to new version
# Edit terraform.tfvars:
# api_chart_version = "0.0.16"
terraform apply

Clean Up

To remove all applications:

# Destroy all application resources
terraform destroy
# Verify cleanup
kubectl get namespaces | grep -E "api|next|litellm|credits|cerbos"

Unleash Feature Flags Configuration

Disabling Pusher When Not Configured

If Pusher is not configured in your environment, disable it via Unleash:

1. Access Unleash UI at https://unleash.yourdomain.com
   1. Obtain credentials in peripheries stack output:
      terraform output unleash_tokens
      default user admin
2. Navigate to the feature flag: PUSHER_ENABLED
3. Set the flag to false for your environment

Base Models Configuration

Important: Order of Operations

1. First: Create model endpoints via API (see Model Management section)
2. Then: Configure Unleash feature flags with the returned model IDs

Required Model Feature Flags

Configure these model feature flags in Unleash with strategy variants containing model IDs:

Optional Model Feature Flags

Model Management

Adding Models via API

Before configuring Unleash, add models using the API:

# Example: Add Claude 3.5 Haiku
curl -X POST https://api.yourdomain.com/internal/openapi/admin/model/new \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <UM_INTERNAL_API_KEY>' \
  -d '{
    "orgId": "<YOUR_ORG_ID>",
    "userId": "<YOUR_USER_ID>",
    "displayName": "Claude 3.5 Haiku",
    "modelProviderDisplayName": "Anthropic",
    "type": "CHAT",
    "contextWindow": 32768,
    "metadata": {
      "type": "Text Generation",
      "costTier": "MEDIUM",
      "usageTag": "Chat + Agents",
      "description": "Fast model for quick responses",
      "modelCreator": "Anthropic"
    },
    "litellmModelName": "claude-3-5-haiku",
    "litellmParams": {
      "model": "anthropic/claude-3-5-haiku-latest",
      "api_key": "<ANTHROPIC_API_KEY>"
    }
  }'

Save the returned model ID for Unleash configuration.

Configuring Unleash Variants

After creating models, configure Unleash feature flags:

{
  "stickiness": "default",
  "variants": [{
    "name": "models",
    "weight": 1000,
    "payload": {
      "type": "json",
      "value": "[\"model-id-from-api\"]"
    }
  }]
}

Replacing Models

To replace an existing model:

curl -X POST https://api.yourdomain.com/internal/openapi/admin/model/delete-with-replacement \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <UM_INTERNAL_API_KEY>' \
  -d '{
    "deletingModelId": "<OLD_MODEL_ID>",
    "replacementModelId": "<NEW_MODEL_ID>",
    "orgId": "<YOUR_ORG_ID>",
    "userId": "<YOUR_USER_ID>"
  }'

Next Steps

After successful deployment:

1. Configure monitoring: Set up Prometheus and Grafana dashboards
2. Set up CI/CD: Configure automated deployments
3. Performance tuning: Adjust resource allocations based on metrics
4. Security hardening: Implement network policies and RBAC
5. Backup strategy: Set up database backups and disaster recovery

Support

For issues or questions:

• Check application logs: kubectl logs -n <namespace> <pod-name>
• Review Terraform state: terraform show
• Contact Kindo support with diagnostic output