Notification Routing

Backend architecture for multi-channel notification delivery.

Runtime model: Container-first (persistent Bun process), with Vercel serverless as compatible fallback.

Routing Flow

┌─────────────────────────────────────────────────────────────────────────────┐
│                         NotificationService.send()                           │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  Input: { userId, type, title, body, entityRef?, actionUrl?, groupKey? }    │
│                                                                              │
│  1. Create in-app notification record (always)                              │
│  2. Load user settings + connected channels                                  │
│  3. Apply routing rules:                                                     │
│     - Security type → force email (cannot disable)                          │
│     - Check settings matrix for each channel                                │
│     - Skip disconnected/inactive channels                                   │
│  4. Create delivery records in outbox                                       │
│  5. routeExternal (fire-and-forget) writes outbox records:                  │
│     - Container: in-process worker polls and delivers                       │
│     - Vercel: /api/cron/[job] sweep delivers pending records                │
│                                                                              │
│  Output: { notificationId, queuedChannels[] }                               │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Provider Interface

Each channel provider implements a common interface:

Delivery Result

Provider Implementations

Email Provider (Resend)

Telegram Provider (Bot API via fetch)

Discord Provider

Critical: Discord rate limits are NOT hard-coded. Always parse:

• X-RateLimit-Limit
• X-RateLimit-Remaining
• X-RateLimit-Reset

Web Push Provider

Unlike the other three providers, Web Push never goes through the outbox — see below.

Rate Limiting Strategy

Per-Provider Limits

Implementation Options

Recommendation: Database-backed rate tracking for container (simple, persistent). Inngest's rateLimit for Vercel deployments. Both use the same outbox table.

Outbox Pattern

Store pending deliveries in database before async processing:

Container runtime:
┌─────────────┐     ┌─────────────────────┐     ┌──────────────────┐
│  Sync       │────▶│ notification_       │────▶│  In-process      │
│  Handler    │     │ deliveries (outbox) │     │  Worker          │
└─────────────┘     └─────────────────────┘     │  (setInterval)   │
      │                       │                  └──────────────────┘
      ▼                       ▼                       │
 In-app notif           Delivery          External providers
   created              persisted            called async

Vercel serverless:
┌─────────────┐     ┌─────────────────────┐     ┌──────────────────┐
│  Sync       │────▶│ notification_       │────▶│  /api/cron/[job] │
│  Handler    │     │ deliveries (outbox) │     │  sweep           │
└─────────────┘     └─────────────────────┘     └──────────────────┘

Why Outbox?

Delivery Processing (Runtime-Adaptive)

The outbox table is the contract. Who processes it depends on the runtime.

Container: In-Process Worker (Primary)

The delivery worker runs on its own setInterval in src/lib/server/jobs/delivery-scheduler.ts (separate from the 3-hourly scheduler.ts). It is gated on platform.persistent and ticks at DEFAULT_DELIVERY_INTERVAL_MS (15s). The worker logic lives in src/lib/server/jobs/notification-delivery.ts:

The claim, the fence token, and the reaper are specified in architecture/workers.md — this table is only the routing view of them.

service.ts calls routeExternal directly (fire-and-forget) after the outbox write — there is no runtime branch and no event emit.

Vercel: Cron Sweep (Serverless)

On serverless (no persistent process), notification-delivery is a registered job (jobs/index.ts) scheduled in vercel.json at /api/cron/notification-delivery (dispatched through the generic /api/cron/[job] route). This cron is what drains pending Telegram / Discord / email deliveries on Vercel — the interval scheduler never runs there.

Inngest is design-intent only. It is not a dependency and is never imported. Async delivery uses the in-process worker (container) plus the cron sweep (serverless).

Retry Configuration

Policy lives in $lib/server/notifications/backoff.ts (pure) with constants in $lib/server/config.ts. The delay is applied against the database clock, so a skewed app server cannot shift the queue.

Terminal states are split deliberately: failed means the provider says this will never work (403, bad address, no recipient) so a Retry button is pointless; dead means a retryable fault outlived the budget and a human should look. Only dead and failed rows are eligible for the admin retry action.

Web Push Bypasses the Outbox

Push never writes a notification_deliveries row. service.ts partitions 'push' out of the router's channel list before createDeliveries runs, then calls sendPushNow() directly: it renders a generic localized category line (renderNotification('notif_push_{type}', {}, recipientLocale)) and hands it to WebPushProvider fire-and-forget. No outbox row means no retry and no dead-letter state — a push send either reaches the browser's push service on the first attempt or it doesn't. See ../pwa.md for the full payload contract and subscription lifecycle.

Error Handling

Error Classification

Channel Deactivation

When a channel fails permanently:

1. Set is_active = false on the account row (user_telegram_accounts / user_discord_accounts)
2. Error detail is recorded per-delivery in notification_deliveries.error_code / error_message — the account row stores no error string
3. Send notification via other channels: "Your {channel} is disconnected"
4. Surface in settings UI with "Reconnect" button

Delivery Tracking

notification_deliveries Table

Retention Policy

Not implemented. No job prunes notification_deliveries by status. The per-status table below is aspirational.

The existing jobs/notification-cleanup.ts only touches the notifications table: it archives at NOTIFICATION_ARCHIVE_DAYS (30) and hard-deletes at NOTIFICATION_DELETE_DAYS (90).

Module Structure

src/lib/server/notifications/
├── index.ts                # Re-exports public API
├── service.ts              # NotificationService.send() — single entry point (NotificationType inline union)
├── router.ts               # Preference resolution, channel selection
├── outbox.ts               # Delivery record management
├── stream.ts               # SSE: notifyUser() + connection registry (container only)
├── crypto.ts               # AES-GCM token encryption (ENCRYPTION_KEY)
├── health.ts               # Channel health stats
├── render-message.ts       # Notification → channel message body
├── telegram.ts             # Telegram link/verification helpers
└── providers/
    ├── index.ts            # Provider registry (getProvider by channel)
    ├── types.ts            # Provider interface
    ├── email.ts            # Resend provider
    ├── telegram.ts         # Raw fetch() to Bot API
    ├── discord.ts          # Discord REST provider
    └── web-push.ts         # WebPushProvider (VAPID, per-device fan-out)

DB read/write helpers live under $lib/server/db/notifications/{queries,mutations,admin-queries}.ts.

The delivery worker is $lib/server/jobs/notification-delivery.ts, driven by $lib/server/jobs/delivery-scheduler.ts. There is no Inngest function — Inngest is not wired.

Runtime Detection

service.ts does not branch on runtime. After the outbox write it calls routeExternal directly (fire-and-forget). The persistent container's delivery-scheduler.ts worker (gated on platform.persistent) picks up pending records on its own interval; on serverless the /api/cron/[job] route sweeps them. No inngest.send call exists.

Related

• ./channels.md - Channel connection flows
• ./schema.md - Database tables
• ../middleware.md - Hooks integration
• ../../stack/notifications/ - Provider details
• ../pwa.md - Web push design record (payload contract, subscription lifecycle, delivery-mode verdict)