NestJS AI Gateway

The AI gateway is a NestJS module that proxies every AI operation from the frontend through to background-tasks. It's the only path the frontend uses to reach the AI service — direct frontend → background-tasks calls are forbidden by the architectural invariant introduced in the AWS-native migration.

Why a gateway

Before May 2026 the frontend opened 4 separate connections to ai.aiqlick.com:

• wss://ai.aiqlick.com/graphql for 8+ AI subscriptions (CV extraction, job parsing, agent chat, voice, …)
• https://ai.aiqlick.com/graphql for HTTP CRUD (useAIAgent, useAgentDocuments, …)
• wss://ai.aiqlick.com/voice/ws/<conversationId> for binary PCM voice streaming
• wss://ai.aiqlick.com/api/v1/jitsi/events for transcription

That meant: 4 environment variables on Amplify, JWT injection in 8+ hooks, billing context (userId + companyId) hand-stamped client-side, two ALB targets, two SSL certs, two CORS configurations.

After the migration:

• Frontend opens one Apollo client + WS to api.aiqlick.com. That's it.
• All 10 AI subscriptions go through one of three NestJS resolvers: subscribeAi, queryAi, mutateAi.
• Voice WS proxies through NestJS at /voice/ws/:conversationId (raw frame forwarder).
• Backend stamps billing context server-side from the JWT — no more withBillingContext() calls in the frontend.
• Background-tasks runs in a private VPC; the public ai-dev.aiqlick.com ALB is locked down (only the backend's SG can reach port 8000).

Architecture

Operation registries

Every proxied AI operation has a single-line entry in one of two registries. Adding a new operation is one entry — no resolver, no DTO, no test scaffolding.

AI_OPERATIONS (subscriptions)

aiqlick-backend/src/ai-gateway/operations/subscription.registry.ts

{
  name: 'extractCv',
  stampBillingContext: true,
  featureFlag: 'ai-gateway-via-extractCv',
  planLimitKey: 'maxCvParsesPerMonth',
  document: /* GraphQL */ `
    subscription ProxyExtractCv($input: CVExtractionInput!) {
      extractCv(input: $input) {
        __typename
        ... on CVExtractionProcessing { id status message progress }
        ... on CVExtractionSuccess { id status parsed { ... } rawText fileUrl }
        ... on CVExtractionFailure { id status error { code message recoveryHint } }
      }
    }
  `,
}

Live entries (10): extractCv, parseJob, faceExtraction, agentChat, textRewrite, generateMeetingInsight, addCandidate, userJobMatching, bulkCvImport, documentProcessing.

AI_HTTP_OPERATIONS (queries + mutations)

aiqlick-backend/src/ai-gateway/operations/http.registry.ts

Same shape as AI_OPERATIONS plus an isMutation: boolean discriminator so queryAi rejects mutation-registered ops and vice versa.

Live entries (22):

Op	Kind	Used by
getAgent, listAgents, createAgent, updateAgent, enableAgent, disableAgent, deleteAgent	query / mutation	useAIAgent
listConversations, getConversationMessages, startConversation, archiveConversation, rateConversation	query / mutation	useConversationManager
listAgentDocuments, uploadAgentDocument, deleteAgentDocument, enableDocumentRag, disableDocumentRag, reprocessDocument	query / mutation	useAgentDocuments
listToolDefinitions, updateToolDefinition, toggleToolDefinition	query / mutation	useToolDefinitions
createBulkCvImport, getBulkCvImportStatus	mutation / query	useBulkCvImport

The Phase 3.7d-era AiPassthroughController + AI_HTTP_ALLOWED_ROOT_FIELDS allowlist were retired once every FE caller migrated to aiGatewayRequest(). There is no general-purpose GraphQL forwarding endpoint anymore — every operation must have a typed registry entry.

HTTP CRUD ops set stampBillingContext: false because the FE explicitly passes ownership context (companyId/userId/createdById) on every CRUD call. Stamping is only appropriate for streaming credit operations (see AI_OPERATIONS above). A side effect of having neither stampBillingContext nor planLimitKey is that the gateway treats the op as non-billable and skips the active-subscription preflight (see "Subscription preflight" below).

Spec field reference

Field	Purpose
name	Background-tasks operation name (must match the upstream resolver)
document	Full GraphQL document text proxied to background-tasks
stampBillingContext	When true, NestJS injects userId + companyId into the upstream input. Set false for ops whose BG input does not have those fields (agentChat, userJobMatching, addCandidate, generateMeetingInsight) — BG derives billing context from the JWT in those cases. Doubles as a billable-op marker — see "Subscription preflight" below.
featureFlag	Optional AppConfig flag name. When the flag is OFF, the gateway returns FEATURE_FLAG_OFF immediately.
planLimitKey	Optional plan-limit key (e.g. maxCvParsesPerMonth). The gateway runs PlanLimitsService.checkLimit() before forwarding. Also marks the op as billable for the subscription preflight.
variableShape	'wrapped' (default) for subscription Foo($input: FooInput!) signatures; 'spread' for scalar args (e.g. subscription bulkCvImport($importId: String!)).
isMutation (HTTP only)	true for mutations, false for queries.

Subscription preflight (billable-op rule)

Before forwarding to background-tasks, both gateway resolvers run AiPolicyService.check() for three things: feature flag, active subscription, plan limit. The subscription branch is gated on whether the op consumes paid resources:

requiresActiveSubscription: Boolean(op.stampBillingContext || op.planLimitKey)

So:

Op shape	Requires active subscription?	Examples
stampBillingContext: true	Yes	extractCv, parseJob, textRewrite, faceExtraction, bulkCvImport
planLimitKey set	Yes	agentChat (maxAgentMessagesPerMonth)
Neither flag	No	listAgents, getAgent, listAgentDocuments, listToolDefinitions, listConversations, getConversationMessages, archiveConversation, rateConversation, etc.

The non-billable case matters: an expired/cancelled company can still reach the chatbot picker, knowledge base, tool config, and conversation history — they just can't use anything that costs credits. Before this rule existed, listAgents rejected with SUBSCRIPTION_INACTIVE and the chatbot picker rendered "No agents available" even though the company had agents in the DB.

When you add a new op to either registry, the subscription requirement comes for free from the flags you already declare. Do not pass requiresActiveSubscription by hand from new resolver call sites — the Boolean(stampBillingContext || planLimitKey) derivation lives in ai-http.resolver.ts and ai-gateway.resolver.ts and is the single source of truth.

Selection-set rule (subtle, broke production once)

Background-tasks return types are Strawberry unions (Processing | Success | Failure). Selecting a scalar directly on a union is invalid GraphQL — every leaf must live inside an inline fragment:

... on FooProcessing { id status message progress }
... on FooSuccess { id status <real success fields> }
... on FooFailure { id status error { code message recoveryHint } }

The original registry forgot the inline fragments and shipped invalid GraphQL — every gateway-routed subscription failed when AppConfig flipped the flags ON. Fixed in commit efe1349. Always include __typename so the frontend can discriminate.

Billing context stamping (HTTP)

For 'wrapped' ops the upstream signature is mutation Foo($input: FooInput!) so userId/companyId must be injected inside the input object — top-level vars are silently ignored by GraphQL. For 'spread' ops they go top-level. AiHttpResolver handles both:

if (op.stampBillingContext) {
  if (shape === 'wrapped' && variables.input) {
    variables.input = { ...variables.input, userId: user.id, companyId: user.selectedCompanyId };
  } else {
    variables.userId = user.id;
    variables.companyId = user.selectedCompanyId;
  }
}

Internal-service JWT

The gateway authenticates to background-tasks via a short-lived HS256 JWT signed with INTERNAL_SERVICE_SECRET (in Secrets Manager). Background-tasks validates with the same secret and trusts the actor claims — no DB lookup needed.

Claim	Type	Meaning
iss	string	"aiqlick-backend" literal
aud	string	"aiqlick-background-tasks" literal
sub	string	"internal:" + actorUserId
actorUserId	uuid	The end-user the operation acts on behalf of
actorCompanyId	uuid | null	Selected company at request time
actorRoles	string[]	Effective UCR roles
traceId	uuid	Distributed trace ID; logged on both sides
iat / exp	unix s	exp = iat + 300 (5 min TTL)
jti	uuid	Unique token id

Two-secret rotation supported via INTERNAL_SERVICE_SECRET_PREVIOUS — both are tried during validation so the rotation script can write both, wait, then drop the previous with zero downtime.

User JWTs no longer reach background-tasks at all on the gateway path. The legacy user-JWT validator is still in BG behind ALLOW_USER_JWT=false (default off) as an emergency fallback.

Voice WS proxy

Voice can't fit the GraphQL model (binary PCM + low-latency requirements), so it gets a dedicated raw-WS gateway at /voice/ws/:conversationId?token=<userJwt> that:

1. Validates the user JWT (query-param token, since browsers can't set headers on a WS upgrade).
2. Looks up selectedCompanyId from the User table (the JWT carries only sub/email, not company context).
3. Mints an internal-service JWT and opens an upstream WS to background-tasks.
4. Pipes frames bidirectionally — no parsing, no buffering. Text JSON control + binary 16kHz PCM both pass through unchanged.
5. On either-side close, closes the other side with the same code; backpressure cutoff at 1 MB buffered.

background-tasks/app/api/voice_ws.py validates the internal-service JWT before await websocket.accept(). Closes 1008 on missing or invalid token. Pre-Phase 3.5 this endpoint accepted any unauthenticated connection — fixed in commit 1e0ce9c.

Health endpoint

GET /health/ai-gateway returns the full wiring state — useful for ops smoke tests + the CloudWatch Synthetics canary that watches it.

{
  "ok": true,
  "upstream": {
    "aiBackendGraphQlUrl": "http://172.31.17.4:8000/graphql",
    "aiBackendGraphQlWsUrl": "ws://172.31.17.4:8000/graphql",
    "aiBackendVoiceWsUrl": "ws://172.31.17.4:8000/voice/ws",
    "httpClientReady": true,
    "wsClientInFlight": 0
  },
  "internalToken": { "signerReady": true },
  "appConfig": {
    "enabled": true,
    "flags": {
      "ai-gateway-via-extractCv": true,
      "ai-gateway-via-parseJob": true,
      "ai-gateway-via-faceExtraction": true,
      "ai-gateway-via-agentChat": true,
      "ai-gateway-via-textRewrite": true,
      "ai-gateway-via-meetingInsight": true,
      "ai-gateway-via-addCandidate": true,
      "ai-gateway-via-userJobMatching": true,
      "ai-gateway-via-bulkCvImport": true,
      "ai-gateway-via-documentProcessing": true
    }
  },
  "operations": {
    "streaming": [...],
    "http": [...]
  }
}

No auth required — surfaces only plumbing state, not user data.

Frontend usage

The frontend opens one Apollo subscription to wss://api.aiqlick.com/graphql via the split-link transport and uses three documents:

import { subscribeAiOp, type AiGatewaySubscription } from '@lib/ai/aiGateway';

const sub = subscribeAiOp<CVExtractionUpdate>(
  apollo,
  'extractCv',
  { id, fileUrl } as Record<string, unknown>,
  {
    onUpdate: (update) => {
      switch (update.__typename) {
        case 'CVExtractionProcessing': /* update progress */ break;
        case 'CVExtractionSuccess':    /* finalise */ break;
        case 'CVExtractionFailure':    /* error UI */ break;
      }
    },
    onGatewayError: (msg) => /* gateway refused: flag off, plan limit, etc */ ,
    onError: (err) => /* transport drop */ ,
  },
);
// later: sub.unsubscribe();

All AI hooks (useCvExtractionGraphQL, useFaceExtraction, useJobParsingSubscription, useAIAgentChat, useAgentDocuments, useTextRewrite, useBulkCvImport, useUserJobMatching, useMeetingInsights, useVoiceChat, useCvProcessorGraphQL) use this helper. The deleted clients (useAISubscriptionClient, useAgentWebSocket, useTextRewriteClient) are gone, along with the four NEXT_PUBLIC_AI_* env vars.

Adding a new AI operation

1. Define the resolver in background-tasks/app/graphql/... like any Strawberry op.
2. Add a one-line entry to AI_OPERATIONS (subscriptions) or AI_HTTP_OPERATIONS (HTTP).
3. If the op needs a feature flag, add it to AppConfig (zatlet7 profile) and set featureFlag in the registry entry.
4. If stampBillingContext: true, make sure the BG input type has companyId: Optional[str] and userId: Optional[str] fields.
5. The frontend calls it through subscribeAiOp / aiGatewayRequest — no per-op TypeScript generation needed.

Related

• AWS-Native Migration
• AI Services Overview
• Voice Service