Skip to main content

Real-time delivery (SSE)

Notifications stream from the backend to the browser over Server-Sent Events (SSE). Latency from notification.create → toast in the user's tab is typically <500ms, replacing the 30-second worst-case lag of the polling fallback.

Why SSE (and not WebSocket)

The team previously removed WebSocket subscriptions because they drop silently behind corporate proxies and reconnect logic gets ugly when the JWT rotates. SSE was chosen for the next iteration because:

  • It's one-way — matches the actual data direction (server → client). No need for a bidirectional protocol.
  • Runs over plain HTTP/HTTPS — passes through proxies, ALBs, and CDNs without special handling.
  • EventSource has built-in auto-reconnect with Last-Event-ID resume semantics.
  • Polling stays in place as a safety net — dedup via notification ID makes double-delivery harmless, so SSE failures degrade gracefully.

Architecture

The Redis-backed pub/sub was retired with Phase 4 of the AWS-native migration. Two parallel paths now run:

  • In-process pub/subNotificationPubSubService uses graphql-subscriptions's native PubSub for sub-second SSE fan-out within the same backend pod. Feeds the event: notification to open browser streams.
  • SQS-backed channel deliveryNotificationService.queueForDelivery() enqueues { notificationId, channels } to aiqlick-notifications-delivery-*. SqsNotificationConsumer drains the queue and runs multi-channel fan-out (IN_APP DB write, EMAIL via SES, PUSH via SNS once mobile lands). Survives restarts.

When backend scales beyond one pod, the in-process pub/sub will gain a SNS+SQS adapter so events from any pod reach every SSE client. Until then prod must run with ALB sticky sessions so SSE traffic for a given user lands on the same pod that emits its notifications.

Listener events must actually be emitted

The notification listeners only fire if the corresponding domain event is emitted. The auth.service.ts requestPasswordReset / resetPassword / verifyEmail paths historically forgot to emit user.password.reset.requested / .completed / user.email.verified — emails went out (because MailService was called directly) but in-app SSE notifications were silently dropped. Fixed in commit 7fb62c0. When you add a new domain action that should produce a notification, emit the event AND verify the listener actually catches it via the SSE stream.

Endpoint contract

PropertyValue
MethodGET
Path/notifications/stream
AuthJWT in ?token=... query param (browsers cannot set headers on EventSource)
Content-Typetext/event-stream
Wire formatStandard SSE (event: / id: / data: lines)

Event types

event:PayloadWhen
open{ ok: true, sessionId }Immediately on stream open
notificationFull NotificationOutput JSONWhenever any notification fires for the authenticated user
ping{ ts: <epoch> }Every 25s to keep the connection alive

Heartbeats

The controller emits a ping every 25 seconds because the AWS ALB closes idle connections at 60s. Without heartbeats the connection would drop silently after a minute of inactivity. Browsers ignore unknown event types, so the heartbeat is invisible to UI code.

Auth

SseAuthGuard accepts the JWT from ?token= first, falling back to Authorization: Bearer ... for non-browser callers (curl, tests). It is scoped only to this route — the rest of the platform's HTTP/GraphQL surface uses header-based auth via JwtAuthGuard.

The JWT in the query string can appear in proxy access logs. This is the standard SSE trade-off and is documented as accepted risk; a future ticket may switch to short-lived dedicated SSE tokens.

Frontend integration

const url = `${API_URL}/notifications/stream?token=${encodeURIComponent(token)}`;
const es = new EventSource(url);

es.addEventListener("open", () => {
setSseConnected(true);
startPolling(60000); // relax polling cadence while SSE is healthy
});

es.addEventListener("notification", (evt: MessageEvent) => {
const n = JSON.parse(evt.data) as Notification;
if (toastedIdsRef.current.has(n.id)) return;
toastedIdsRef.current.add(n.id);
setLastNotification(n);
refetchUnreadCount();
});

es.addEventListener("error", () => {
setSseConnected(false);
startPolling(30000); // tighten polling while SSE is reconnecting
// EventSource auto-reconnects with exponential backoff; no manual retry needed
});

Source: aiqlick-frontend/contexts/NotificationContext.tsx.

Cold-load behaviour: backlog stays silent

A pre-existing unread notification is not a delivery event — replaying it as a toast on every page load surfaces stale "Support Reply on SUP-XXXXX" popups even on logged-out marketing pages. The mount-time flow on the frontend treats the first network response specifically as the backlog baseline:

Because the polling/SSE effect early-returns while isInitialLoadRef.current === true, neither the cached count=0 from a previous session (stale Apollo cache) nor the fresh network emission can be misclassified as a brand-new arrival.

When a real-time notification surfaces a toast (post-DND guard), the toast handler fires MARK_NOTIFICATION_AS_READ($notificationId) and refetches UnreadNotificationCount so the badge stays in sync without the user needing to click. DND short-circuits before the mutation, so backlog and DND-suppressed items both stay unread.

See also: Frontend notifications page for the React-level details.

Polling-fallback design

StatePolling cadenceWhy
SSE connected60sSSE owns delivery; polling is just a safety canary
SSE error / disconnected30sPolling becomes the primary delivery path until SSE reconnects
Logged out / token missingDisableduseQuery({ skip: true })

Both paths funnel through the same setLastNotification state, and toastedIdsRef (cap 200) ensures a notification arriving on both channels only renders one toast. This makes SSE additive — if it fails for any reason the user still gets notifications, just with the existing 30s lag.

Failure modes & detection

The SSE pipeline can fail in several ways:

FailureSymptomDetection signal
Redis pub/sub brokenNotifications created but never streamednotificationsCreatedLastHour > 0 && deliveriesLastHour == 0 && activeSessions > 0 (red banner on the Observability page)
Iterator throwsConnection ends with SERVER_ERROR reasonerrorDisconnectsLast5Min > 0 KPI
Reconnect stormSame user reconnects ≥3× per minute"Reconnect storms" panel on Observability page
Stuck-open socketHeartbeat fails silentlyEventSource fires error after ~30s; client falls back to polling

For the operator-facing diagnosis flow, see Observability.