Skip to main content

AI Service Deployment

The background-tasks (FastAPI) service deploys to AWS ECS Fargate via GitHub Actions. After Phase 7 the SSH-into-EC2 deploy + :previous tag rollback are gone — replaced by ECS APIs and the deployment circuit breaker.

End-to-end deploy time is ~3-4 minutes (build ~1-2 min, ECS rolling deployment ~2 min).

Environments​

EnvironmentECS serviceTask familyImage tagReachability
Productionbg-tasks-prod (cluster aiqlick)aiqlick-bg-tasks-prodbackground-tasks:latestPrivate — backend → bg-prod.aiqlick.local
Developmentbg-tasks-dev (cluster aiqlick)aiqlick-bg-tasks-devbackground-tasks:dev-latestPrivate — backend → bg-dev.aiqlick.local

The BG service has no public ALB target. Per the architectural invariant established in Phase 3.6 of the AWS-native migration and preserved in Phase 7, the frontend must reach BG only via the NestJS gateway (subscribeAi / queryAi / mutateAi), which proxies through the internal ALB.

Deployment Pipeline​

Workflow file: .github/workflows/aws-deploy.yml Runner: runs-on: ubuntu-latest (the [self-hosted, aiqlick-bg] runner referenced in older docs was already removed in late April).

Branch-to-Environment Mapping​

BranchImage tag prefixECS service
devdev-bg-tasks-dev
main(none)bg-tasks-prod

Docker Build​

Single-stage Dockerfile (unchanged from EC2):

PropertyValue
Base imagepython:3.11-slim
System depslibpq5, curl, poppler-utils, libsndfile1, ffmpeg
Entrypointuvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 1
Healthcheckcurl -f http://localhost:8000/health (30s interval)
Image size~700 MB
Build time~2 minutes

Auto-Rollback (deployment circuit breaker)​

ECS Fargate has a native deployment circuit breaker — no :previous tag dance needed. The service is configured with:

deploymentCircuitBreaker = { enable: true, rollback: true }
maximumPercent           = 200
minimumHealthyPercent    = 100
healthCheckGracePeriod   = 120s

If new tasks fail ALB health checks, ECS automatically reverts to the previous task-definition revision. This replaces the manual docker tag :previous && docker run :previous flow that lived in the EC2 deploy script.

Health verification​

background-tasks has no public URL after Phase 7. Health is verified via the backend's /health/ai-gateway endpoint, which proxies a connectivity check from api(.dev).aiqlick.com to bg-{env}.aiqlick.local through the internal ALB.

curl -fsS https://api-dev.aiqlick.com/health/ai-gateway | jq .
# Expected: { "ok": true, "upstream": { "httpClientReady": true, ... } }

Manual deployment​

# Force re-deploy with current task def
aws ecs update-service --profile Administrator-842697652860 --region eu-north-1 \
  --cluster aiqlick --service bg-tasks-prod --force-new-deployment

# Pin to a previous revision
aws ecs update-service --profile Administrator-842697652860 --region eu-north-1 \
  --cluster aiqlick --service bg-tasks-prod \
  --task-definition aiqlick-bg-tasks-prod:42

Troubleshooting​

ProblemCommands
Backend reports ok=false for ai-gatewayCheck BG task health: aws elbv2 describe-target-health --target-group-arn $(aws elbv2 describe-target-groups --names tg-bg-prod --query 'TargetGroups[0].TargetGroupArn' --output text)
Container crashes on bootaws logs tail /ecs/aiqlick-bg-tasks-prod --follow --since 10m
Bedrock calls failingVerify task role has bedrock:InvokeModel* and bedrock-runtime VPC endpoint is healthy
Cross-region (Rekognition / Transcribe) failingCheck NAT gateway aiqlick-nat-eu-north-1a status — those calls go through it