Architecture

Strum VOD is a self-hosted VOD (Video on Demand) platform. The monorepo (pnpm workspaces) is split into five apps — apps/api (Fastify REST), apps/worker (Node: bridge + AI + analytics), apps/transcoder (Go: the FFmpeg ladder), and the two frontends apps/dashboard and apps/player — plus six shared packages: packages/db (Drizzle schema + Postgres factory), packages/email (OTP emails), packages/subtitles (VTT/SRT generation), packages/workbench (BullMQ dashboard), packages/player-ui (player components), and packages/design-tokens (brand CSS variables).

System Overview

Browser
  ├── Dashboard (CF Worker + Assets, via wrangler)  → REST →  apps/api (Fly.io / Docker)
  └── Player app (CF Pages, via wrangler)           → REST →  apps/api

Upload paths — both land the original file in R2 (strum-vod):
  Standard:  Browser ──presigned PUT──────────────────────────▶ R2
  Resumable: Browser ──TUS──▶ apps/api ──S3 multipart──────────▶ R2

Transcode pipeline (Redis is the shared backbone — BullMQ + Streams):
  apps/api ──enqueue BullMQ "transcode"──▶ Redis
  apps/worker  startTranscodeBridge  (BullMQ "transcode" → XADD go:transcode:jobs)
  apps/transcoder (Go)  XREADGROUP go:transcode:jobs
      └─▶ resolve source (shared volume / yt-dlp / S3) → FFmpeg HLS ladder 360p–4320p
          → thumbnails + audio → upload to R2 → mark asset ready (Postgres)
          → XADD go:ai:dispatch + go:webhook:dispatch
  apps/worker  startAiDispatchBridge  (go:ai:dispatch → BullMQ "ai-process")
      └─▶ AI worker: transcribe (local/Modal/Deepgram) → subtitles → chapters → highlights → R2
  apps/worker  startWebhookDispatchBridge  (go:webhook:dispatch → BullMQ "webhook-delivery")
      └─▶ apps/api webhook worker → customer webhook URL

Data stores:
  Postgres (Neon)  — collections, assets, renditions, jobs, ai_jobs, highlights, analytics, outbox/webhooks, ...
  Redis            — BullMQ queues (transcode, ai-process, webhook-delivery) + Redis Streams
                     (go:transcode:jobs, go:ai:dispatch, go:webhook:dispatch)
  Cloudflare R2    — 3 buckets: strum-videos (sources + playback), attachments, backups

Everything on Cloudflare (R2 buckets, the Dashboard and Player Workers) is deployed with wrangler — see apps/*/wrangler.toml. R2 bucket CORS is applied from the repo with pnpm r2:cors:set (config in infra/r2/); it is a bucket-level setting that must be re-applied if the bucket is ever recreated. Frontends run in local dev on Vite (http://localhost:1337 dashboard, http://localhost:1338 player).

Diagrams

Static renders of the repo's mermaid diagrams. The source of truth is the mermaid blocks in CLAUDE.md (and infra/modal-whisper/README.md), which GitHub renders directly — these PNG/SVG copies exist for contexts that can't (PDFs, printed docs, issue embeds). Keep the mermaid source canonical and regenerate rather than editing the images.

Transcode → AI → webhook pipeline

The full flow from upload (presigned PUT / TUS) to HLS delivery on R2, AI processing — including the async Modal transcription loop (dispatch → signed callback → resume) — and asset.ready/asset.error domain webhooks. Queue/stream names are the env defaults from apps/worker/src/env.ts.

SVG: transcode-pipeline.svg

Async Modal transcription

The dispatch → 202 {callId} → cold-start/inference (spawned call) → HMAC-signed callback → resume/retry/reaper sequence for the self-hosted Modal Whisper provider (infra/modal-whisper/app.py).

SVG: modal-whisper-sequence.svg

Asset lifecycle

The state machine of an asset: created → uploaded → queued → processing → ready | error, with the hard-delete terminal and the error-retry path (POST /process has no status guard).

SVG: asset-lifecycle.svg

Regenerating the images

# Requires @mermaid-js/mermaid-cli (headless Chrome via puppeteer):
#   npm i -g @mermaid-js/mermaid-cli
# GitHub Actions runners also need --no-sandbox:
#   echo '{"args":["--no-sandbox","--disable-setuid-sandbox"]}' > puppeteer.json

# 1. Extract the ```mermaid block to a .mmd file:
#    CLAUDE.md block 1 → transcode-pipeline, block 2 → modal-whisper-sequence,
#    block 3 → asset-lifecycle
awk -v T=1 '/^```mermaid$/{n++; f=(n==T); next} /^```$/{if(f) exit} f' CLAUDE.md > pipeline.mmd
awk -v T=2 '/^```mermaid$/{n++; f=(n==T); next} /^```$/{if(f) exit} f' CLAUDE.md > sequence.mmd
awk -v T=3 '/^```mermaid$/{n++; f=(n==T); next} /^```$/{if(f) exit} f' CLAUDE.md > asset.mmd

# 2. Render SVG + PNG (pass -w/-H from the SVG's viewBox so wide diagrams
#    aren't cropped to mermaid-cli's default 800x600 viewport)
mmdc -i pipeline.mmd -o docs/diagrams/transcode-pipeline.svg  -p puppeteer.json
mmdc -i pipeline.mmd -o docs/diagrams/transcode-pipeline.png  -p puppeteer.json -w 4247 -H 762
mmdc -i sequence.mmd -o docs/diagrams/modal-whisper-sequence.svg -p puppeteer.json
mmdc -i sequence.mmd -o docs/diagrams/modal-whisper-sequence.png -p puppeteer.json -w 785 -H 675
mmdc -i asset.mmd    -o docs/diagrams/asset-lifecycle.svg    -p puppeteer.json
mmdc -i asset.mmd    -o docs/diagrams/asset-lifecycle.png    -p puppeteer.json -w 991 -H 932

The modal sequence diagram is duplicated (intentionally) in CLAUDE.md and infra/modal-whisper/README.md; both render to the same image.

---

Apps

apps/api — REST API

Stack: Fastify + TypeScript · DB: Postgres (Neon) · Deploy: Fly.io / Docker

The central entry point for all asset management. Handles CRUD, generates presigned upload URLs, serves the resumable TUS upload endpoint, enqueues transcode jobs, serves playback info, and runs the webhook delivery worker.

Key files:

• src/routes/assets.ts — asset endpoints: upload-url, upload-token, upload-complete, upload, import, process (no status guard — retry from error works), hard-delete (row + R2 objects, FK CASCADE), search, thumbnail upload/reset, downloads, audio, transcript/chapters/highlights
• src/routes/collections.ts — collections (video folders) CRUD + per-collection counts
• src/routes/tus.ts — resumable upload (/upload/videos) via @tus/server + @tus/s3-store; verifies the upload-token JWT on every request
• src/routes/ai.ts — AI job management + POST /v1/ai/whisper-callback (receives the HMAC-signed async transcription result from Modal)
• src/routes/webhooks.ts — customer webhook endpoints; the webhook delivery worker (src/queue.ts) consumes the BullMQ webhook-delivery queue and POSTs domain events (asset.ready, asset.error, asset.deleted, ai.completed, ai.failed)
• src/routes/playback.ts — public /v1/playback/:playbackId resolution
• src/routes/analytics.ts / stats.ts — analytics ingestion + aggregation reads
• src/routes/orgs.ts, auth.ts, billing.ts, settings.ts, comments.ts, backups.ts, bull-board.ts, workbench.ts, health.ts
• src/db.ts — Postgres pool + raw SQL migrations (CREATE TABLE IF NOT EXISTS…)
• src/env.ts — Zod-validated env (incl. SHARED_AUTH_SECRET for TUS JWTs)

apps/worker — Node bridge + AI + analytics

Stack: Node + BullMQ + Redis Streams · Deploy: Fly.io / Docker

The Node worker no longer transcodes — the FFmpeg ladder lives in apps/transcoder (Go). It has six jobs:

1. Transcode bridge (src/bridge.ts::startTranscodeBridge) — BullMQ Worker('transcode', …) that does no transcoding; it XADDs the job onto the go:transcode:jobs Redis Stream and returns. Completion is tracked via Postgres, not BullMQ's result.
2. AI-dispatch bridge (startAiDispatchBridge) — consumes go:ai:dispatch (XREADGROUP, written by the Go transcoder) and relays each message onto the BullMQ ai-process queue.
3. AI worker (src/ai-worker.ts) — downloads ai/audio.mp3 + archived source from R2 and runs src/ai/process.ts: transcription (local OpenAI-compatible Whisper, Modal async dispatch + signed webhook callback with retry/reaper, or Deepgram), subtitle, chapter, and highlight-clip generation (ffmpeg still shelled out only for clip cutting).
4. Webhook bridge (startWebhookDispatchBridge) — relays go:webhook:dispatch → BullMQ webhook-delivery (consumed by apps/api's webhook worker).
5. Analytics worker (analytics-worker.ts) — aggregates player events into analytics_daily / analytics_asset_stats (retention curve, engagement score, peak hour — the metrics too heavy to compute on every read). Quality distribution and the heatmap are not aggregated here anymore — apps/api computes both live from analytics_events; see Real-Time Analytics.
6. DB backups (db-backup.ts) — pg_dump → R2 backups bucket (S3_BACKUP_BUCKET, e.g. strum-vod-backups).

Env defaults (src/env.ts): stream keys go:transcode:jobs / go:ai:dispatch / go:webhook:dispatch; queues transcode / ai-process / webhook-delivery; AI config (TRANSCRIPTION_PROVIDER ∈ local|modal|deepgram, WHISPER_API_URL, API_PUBLIC_URL for the Modal callback target, LLM_* for chapters/highlights).

apps/transcoder — Go transcode ladder

Stack: Go + ffmpeg/ffprobe/yt-dlp · Deploy: Fly.io, runs continuously (min_machines_running=1)

Consumes go:transcode:jobs via a Redis Streams consumer group (XREADGROUP/XACK, with XCLAIM-based recovery of orphaned messages). Per job:

• Source resolution (3-tier): local shared volume → yt-dlp URL import → S3 fallback
• HLS ladder — 7 renditions 360p–4320p (see Transcoding Profiles), each filtered to the source's short side (FilterLadder, portrait-safe), H.264/HEVC, CPU or VA-API hardware acceleration, encoded in a single ffmpeg pass (decode once, filter_complex split, parallel HLS outputs)
• Thumbnails — poster thumbnail.jpg (25% duration) + tiled sprite sheets thumbnails/sprite_%03d.jpg (~100 thumbs per file, one ffmpeg pass)
  • thumbnails/thumbnails.vtt scrub-bar index referencing each tile
• Audio — playback audio.m4a (AAC 128k, 48kHz, stereo) and AI ai/audio.mp3 (mono 16kHz, 64k)
• Downloadable MP4 — fast remux of each rendition (download.mp4, no re-encode)
• Uploads everything to R2, marks the asset ready + archives the source, then dispatches go:ai:dispatch and go:webhook:dispatch

Hardware-adaptive: auto-detects CPU/RAM (cgroup-aware) to size concurrency, ffmpeg threads, and pool sizes.

apps/dashboard — Management Dashboard

Stack: React + Vite + Tailwind CSS v4 · Deploy: Cloudflare Worker + Assets (SPA) via wrangler

Authenticated SPA for managing assets, AI configuration, and monitoring transcoding. Local dev: http://localhost:1337.

• Upload videos via presigned URL or resumable TUS (toggle in NewVideoPage.tsx)
• Import videos from external URLs
• Monitor transcoding status (polls every 5s) + AI job per-step status
• Full video management: title, description, thumbnail, AI transcript editing
• Organize videos into collections (folders)
• Settings: AI provider config (transcription provider local/Modal/Deepgram, LLM provider, API keys — stored in settings.ai_config)
• Installable PWA (manifest + Workbox SW, safe-area-aware layout)
• Also serves /embed/:playbackId and /watch/:playbackId (and /player/*) on the same origin for convenience — the canonical public embeds live in the player app

apps/player — Public Embeddable Player

Stack: React + Vite + hls.js · Deploy: Cloudflare Pages via wrangler (pnpm deploy:player)

Standalone SPA for public video playback. No authentication, no dashboard chrome. Deployed independently so embed URLs remain stable regardless of dashboard changes. Local dev: http://localhost:1338. Installable PWA (manifest + Workbox SW).

Routes:

• /embed/:playbackId — Embeddable iframe player (minimal chrome)
• /watch/:playbackId — Full-page public watch page

Features:

• Adaptive quality selector (auto, 360p … 4320p as transcoded)
• Thumbnail seek preview (from VTT sprite)
• Subtitle/caption toggle, chapters, reactions/comments
• Fullscreen support
• Analytics heartbeat (batched, best-effort)

Embed in any page:

<iframe
  src="https://player.strum-vod.dev/embed/p1b2c3d4e5f6g7h8"
  width="100%" height="450"
  frameborder="0" allowfullscreen>
</iframe>

packages/db — Shared Database Layer

Stack: Drizzle ORM (pg-core) + postgres-js · DB: Postgres (Neon)

Shared schema and connection factory used by the API, the Worker, and the Go transcoder (which mirrors the constants).

• createDb(databaseUrl, poolConfig?) — postgres-js pool; auto-enables SSL for neon.tech / sslmode=require URLs, and exposes a pool.query(sql, params) compatibility wrapper that translates ? → $n placeholders
• Tables: collections, assets, renditions, jobs, ai_jobs, highlights, analytics_events, analytics_daily, analytics_asset_stats, settings, comments, reactions, users, organizations, org_members, api_keys, admin_audit_log, domain_events, webhook_deliveries, db_backups, otp_codes
• Column names use snake_case in Postgres, camelCase in TypeScript

packages/email — Transactional email (OTP)

Resend / SMTP / console providers. Sends magic login codes from services/otp.ts via sendOtpEmail.

packages/subtitles — VTT/SRT generation

generateVtt() / generateSrt() from transcript segments — used by the PATCH /v1/assets/:id/transcript route to regenerate subtitle files.

packages/workbench — BullMQ admin dashboard

Third-party queue dashboard mounted by apps/api at /v1/admin/workbench (superadmin-only).

packages/player-ui & packages/design-tokens

Shared player components (VideoHeatmap, …) and brand design tokens (CSS custom properties, Tailwind @theme).

---

Upload Flows

Standard (presigned URL)

Browser                    API                      R2/S3
  │                         │                         │
  │ POST /assets/:id/        │                         │
  │   upload-url ───────────►│                         │
  │                         │ GetSignedUrl            │
  │◄─────── { uploadUrl } ──┤                         │
  │                         │                         │
  │ PUT <uploadUrl> ────────────────────────────────►  │
  │                         │                         │
  │ POST /assets/:id/        │                         │
  │   upload-complete ──────►│ HeadObject             │
  │                         │──────────────────────► │
  │◄─── { status: uploaded } ┤                         │

Resumable (TUS)

Served by apps/api itself (src/routes/tus.ts, @tus/server + @tus/s3-store) — there's no separate upload worker. Every TUS request, not just the initial POST, carries the same bearer JWT, so an in-progress upload can't be resumed/appended to without a token valid for that exact object key:

• Algorithm: HS256
• Signed and verified with SHARED_AUTH_SECRET (one secret, one process)
• aud: videos
• sub: R2 object key (e.g. sources/{assetId}/input.mp4) — also becomes the TUS upload id directly, so the finished object lands at the same key the rest of the pipeline expects, no move step
• maxLen: maximum upload size in bytes, checked against Upload-Length

Browser                    API (apps/api)                     R2
  │                         │                                  │
  │ POST /assets/:id/        │                                  │
  │   upload-token ─────────►│                                  │
  │                         │ Sign JWT                          │
  │◄──── { token, key } ────┤                                  │
  │                         │                                  │
  │ POST /upload/videos ─────►│                                  │
  │   (Authorization: Bearer <token>)                           │
  │                         │ Verify JWT, create S3 multipart ─►│
  │◄──── 201 + Location ────┤                                  │
  │                         │                                  │
  │ PATCH /upload/videos/... ►│                                  │
  │   (chunk data)          │ Verify JWT, upload part ─────────►│
  │◄──── 204 ────────────────┤                                  │
  │         (repeat)        │                                  │
  │                         │                                  │
  │ POST /assets/:id/        │                                  │
  │   upload-complete ──────►│ HeadObject ──────────────────────►│
  │◄─── { status: uploaded } ┤                                  │

---

Database Schema (Postgres)

assets

Column	Type	Description
id	VARCHAR(36)	Primary key, nanoid(12)
org_id	VARCHAR(36)	Owning organization
collection_id	VARCHAR(36)	FK → collections (SET NULL on delete)
status	VARCHAR(32)	Lifecycle state (created → uploaded → queued → processing → ready/error)
source_type	VARCHAR(32)	upload or url
source_key	VARCHAR(512)	R2 path for uploads
source_url	VARCHAR(2048)	URL for imports
title	VARCHAR(255)	Display name
playback_id	VARCHAR(64)	Unique playback ID, nanoid(16)
metadata / custom_metadata / public_settings	JSONB	Flexible payloads
custom_thumbnail_key	VARCHAR(512)	Override thumbnail
duration_sec	INT	Duration in seconds
error_message	VARCHAR(1024)	Error details
audio_codec / audio_bitrate_kbps / audio_sample_rate / audio_channels / audio_path / audio_file_size_bytes	—	Playback audio track metadata
ai_audio_path	VARCHAR(1024)	Extracted Whisper input (ai/audio.mp3)
created_at / updated_at	TIMESTAMP	Timestamps

renditions

Column	Type	Description
id	VARCHAR(36)	Primary key
asset_id	VARCHAR(36)	FK → assets (CASCADE)
quality	VARCHAR(32)	360p, 480p, 720p, 1080p, 1440p, 2160p, 4320p
width / height	INT	Resolution
bitrate_kbps	INT	Video bitrate
file_size_bytes	INT	Output size
codec	VARCHAR(32)	h264 (or hevc with VA-API)
hw_accel	INT	Whether VA-API was used
playlist_path	VARCHAR(1024)	R2 path to the rendition playlist

jobs

Column	Type	Description
id	VARCHAR(36)	Primary key, nanoid(12)
asset_id	VARCHAR(36)	FK → assets (CASCADE)
type	VARCHAR(32)	transcode, ai_process
status	VARCHAR(32)	queued, processing, completed, failed
current_step	VARCHAR(64)	Progress step (probing, transcoding_*, upload, AI, …)
attempts	INT	Retry count
error_message	VARCHAR(1024)	Failure details
worker_id / worker_kind / hostname / region	—	Fleet attribution (node vs go)
cpu_cores / total_mem_gb / hw_accel_used	—	Hardware metadata
started_at / duration_ms	—	Runtime metrics

ai_jobs (AI pipeline)

Per-asset AI processing with per-step status columns: transcription_status, subtitles_status, chapters_status, highlights_status (each pending → processing → completed/failed/skipped). Output paths in R2: transcript_path (ai/transcript.json), subtitles_path (ai/subtitles.vtt), srt_subtitles_path, chapters_path, plus transcript_text/transcript_data for inline serving. For the Modal async provider: provider_job_id (correlates the signed callback, guards replay), transcription_dispatched_at (reaper input), transcription_attempts (max 3).

Other tables

• collections — video folders (name, color, org_id; assets reference it via collection_id)
• highlights — AI highlight clips (title, time range, clip_path/thumbnail_path, dimensions)
• analytics_events / analytics_daily / analytics_asset_stats — raw player events + hourly/daily rolls + per-asset aggregates
• settings — org platform settings incl. ai_config JSONB (AI provider config edited from the dashboard)
• comments / reactions — public engagement on playback_id
• users / organizations / org_members / api_keys — auth + multi-tenancy
• domain_events / webhook_deliveries — transactional outbox + delivery log for webhooks
• db_backups — backup runs (pg_dump → R2 backups)
• otp_codes — magic-link / email OTP codes

---

Storage Layout (R2)

strum-videos/
├── sources/
│   └── {assetId}/
│       └── input.mp4              ← original upload (archived after transcode)
├── playback/
│   └── {assetId}/
│       ├── master.m3u8            ← HLS master playlist (all renditions; VERSION 6,
│       │                           ←   INDEPENDENT-SEGMENTS, FRAME-RATE + measured AVERAGE-BANDWIDTH per variant)
│       ├── {quality}/             ← 360p, 480p, 720p, 1080p, 1440p, 2160p, 4320p
│       │   ├── index.m3u8
│       │   ├── segment_000.ts ...
│       │   └── download.mp4       ← fast-remux MP4 (no re-encode)
│       ├── thumbnail.jpg          ← poster at 25% duration
│       ├── thumbnails/
│       │   ├── sprite_000.jpg     ← tiled sheets, ~100 thumbs each
│       │   ├── sprite_001.jpg     ←   (5×20 grid, 5s interval, 160px thumbs)
│       │   └── thumbnails.vtt     ← scrub-bar seek preview
│       ├── audio.m4a              ← playback audio track (AAC 128k stereo)
│       └── ai/
│           ├── audio.mp3          ← mono 16kHz extract for Whisper
│           ├── transcript.json    ← Whisper/Deepgram/Modal transcript
│           ├── subtitles.vtt
│           ├── subtitles.srt
│           ├── chapters.json      ← AI-generated chapters
│           └── highlights/        ← AI highlight clips
attachments/                        ← org uploads (logos, custom thumbnails)
backups/                            ← pg_dump snapshots

• sources/ — Private. Only accessible via presigned URLs or TUS upload token.
• playback/ — Public read. No per-object ACLs — R2 bucket-level public access serves all files.
• S3_PUBLIC_BASE_URL controls the URL prefix prepended to HLS manifest paths.

---

Transcoding Profiles

Defined in apps/transcoder/internal/ladder/ladder.go (Ladder) — hand-mirrored from the retired apps/worker/src/transcoding.ts (no generated single source yet; keep the copies in sync by hand). FilterLadder keeps only renditions at/below the source's short side (portrait-safe), always retaining at least the lowest:

Quality	Resolution	Video Bitrate	Profile/Level	Codec
360p	640 × 360	1,000 kbps	main / 3.0	H.264
480p	854 × 480	1,800 kbps	main / 3.0	H.264
720p	1280 × 720	3,000 kbps	main / 3.1	H.264
1080p	1920 × 1080	6,000 kbps	high / 4.0	H.264
1440p	2560 × 1440	10,000 kbps	high / 5.0	H.264
2160p	3840 × 2160	20,000 kbps	high / 5.1	H.264
4320p	7680 × 4320	40,000 kbps	high / 6.0	H.264

• Segment duration: 6 seconds · Playlist type: VOD
• Keyframes: forced every 2s (force_key_frames gte(n_forced*2))
• Single pass: the whole ladder runs as one ffmpeg process (TranscodeLadder/buildLadderArgs) — the source decodes once and a filter_complex split fans the frames to one scaled branch per rendition, each writing its own HLS output
• Configurable: RENDITION_CODEC (h264|hevc), MAX_RENDITION_HEIGHT (cap the tallest rung, 0 = full 360p–4320p), HEVC_MIN_HEIGHT (hybrid ladder — rungs at/above this height encode in HEVC while lower rungs keep the base codec), FFMPEG_HWACCEL (auto|vaapi|disabled)
• Audio: one shared AAC track (128k / 48kHz / stereo, AUDIO_PLAYBACK_*) encoded in the same pass to audio/index.m3u8 and declared in master.m3u8 as an EXT-X-MEDIA AUDIO group; renditions are video-only (-an). Sources without audio (probe HasAudio=false) skip it. audio.m4a (the public download file) is derived from that track by stream copy (-c copy), so only one AAC encode happens per asset
• Encoding: libx264 (CRF 23) / libx265 (CRF 28) on CPU, or VA-API (h264_vaapi/hevc_vaapi) when hardware acceleration is available
• Timeout per ladder: max(5min, duration×4s); concurrency/threads are hardware-adaptive

---

Domain Events & Webhooks

The Go transcoder never talks HTTP webhooks — it XADDs onto go:webhook:dispatch, the worker's bridge relays to BullMQ webhook-delivery, and apps/api's webhook worker POSTs to the customer URL. Delivery is backed by the domain_events (outbox) + webhook_deliveries (log) tables with retry/attempt accounting.

Event types: asset.ready, asset.error, asset.deleted, ai.completed, ai.failed.

---

Real-Time Analytics

The dashboard's per-asset analytics tab (AssetAnalytics.tsx) mixes three freshness tiers, all read from the same GET /v1/assets/:id/analytics call plus one dedicated live channel:

Metric	Source	Freshness
Views, watch time, time series, hourly breakdown	Live SQL over analytics_events	Always fresh (query re-runs every poll)
Quality distribution	Live SQL over analytics_events (GROUP BY quality_height)	Always fresh — previously came from the daily batch job (analytics_asset_stats.quality_distribution), which is no longer written
Heatmap ("most replayed")	Live SQL over analytics_events, heartbeat events only	Always fresh
Retention curve, engagement score, peak hour	analytics_asset_stats	Batch — refreshed once/day by analytics-worker.ts's aggregateDaily
Live viewer count	Redis sorted set, pushed via SSE	~3s

Heatmap bucketing — getHeatmap() in apps/api/src/services/analytics.ts splits the asset's duration into a fixed 100 buckets (bucket_size = duration_sec / 100, YouTube-style — same bar count regardless of video length) and counts heartbeat events per bucket (current_time column). Only heartbeat counts as "watched here": seek marks a jump target, not dwell time, and view_start/view_end are just boundaries. No new storage — it's a GROUP BY over the same analytics_events table every other real-time stat already reads, respecting the same period filter (7d/30d/90d/all). Rendered by VideoHeatmap (@strum-vod/player-ui).

Live viewer count (SSE) — the first push-based channel in the project (everything else is REST/poll). Chosen over WebSocket because the data flow is one-directional (server → client only):

• Presence — apps/api/src/services/analytics.ts's updatePresence() runs inline in insertAnalyticsEvents (best-effort, never blocks ingestion). A per-asset Redis sorted set (asset:{id}:live, member = sessionId, score = last-seen epoch seconds) is ZADDed on heartbeat/view_start and ZREMed immediately on pause/view_end/error, so the count drops the moment a viewer actually stops instead of waiting for a timeout. The key carries a TTL (2× the window) so an asset nobody's watching doesn't leak a Redis key forever. A 20s window (2× the player's 10s heartbeat interval) is only a safety net for a connection that dies without firing a final event.
• Routes — GET /v1/assets/:id/live-count (JSON snapshot) and GET /v1/assets/:id/live-stream (SSE, text/event-stream via reply.hijack() + reply.raw, same raw-response pattern as the TUS route). The stream doesn't use Redis pub/sub — each connection just polls ZCOUNT server-side every 3s and writes a data: frame. That sidesteps cross-instance fan-out entirely (Redis is already the single source of truth reachable from any API instance) at the cost of ~3s latency, which is invisible for a viewer-count widget.
• Client — apps/dashboard/src/hooks/useLiveViewerCount.ts uses @microsoft/fetch-event-source instead of the native EventSource, because the dashboard authenticates with an Authorization: Bearer header, which EventSource cannot send. Falls back to polling live-count every 5s if the stream can't be established after a few retries, so the number never just goes stale.

---

ID Conventions

Entity	Generator	Length
Asset ID	nanoid(12)	12 chars
Playback ID	nanoid(16)	16 chars
Job ID	nanoid(12)	12 chars
AI Job ID	nanoid(12)	12 chars
Highlight ID	nanoid(12)	12 chars
Analytics session	nanoid(20)	20 chars
Analytics event	nanoid(16)	16 chars
User / Org / Member / Settings	nanoid(12)	12 chars
API key	random (prefix + hash stored)	32 chars
Comment / Reaction	nanoid(16) / nanoid(12)	—
Domain event / Webhook delivery / DB backup	nanoid(16)	16 chars