Skip to content

Exercise Video & Thumbnail Rendering

This document explains how exercise thumbnails and videos are rendered from different video sources across the Okta-Health platform (web and mobile).

Overview

The platform supports multiple video hosting providers for exercise demonstration videos. When displaying thumbnails or playing videos, the system uses a priority-based fallback system to determine which source to use.

Media handling is now consolidated into a shared module per platform — the API builds a single structured ExerciseMedia object on every exercise response, and clients consume that object instead of re-deriving it from the raw fields each time. The legacy flat fields (cloudflareStreamId, kinescopeVideoId, videoUrl, photoUrl, etc.) are still returned for backward compatibility.

Supported Video Sources

  1. Personalized Video — Therapist-recorded per-patient demos (Cloudflare Stream)
  2. Cloudflare Stream — Primary video hosting for new content
  3. Kinescope — Legacy video hosting (Russian market)
  4. YouTube — External videos via URL
  5. S3 — Therapist-uploaded videos stored in DigitalOcean Spaces, served via presigned playback URL
  6. Static Photo — Fallback for exercises without video

Structured ExerciseMedia Type

All three platforms now share the same shape:

type ExerciseVideoSource =
  | { provider: "cloudflare"; scope: "default" | "personalized"; streamId: string }
  | { provider: "kinescope"; scope: "default"; videoId: string }
  | { provider: "youtube"; scope: "default"; url: string }
  | { provider: "s3"; scope: "default"; storageKey: string; playbackUrl: string };

type ExercisePhotoSource =
  | { provider: "external"; url: string }
  | { provider: "s3"; storageKey: string; url: string };

interface ExerciseMedia {
  videoSources: ExerciseVideoSource[];   // ordered by playback priority
  photoSource: ExercisePhotoSource | null;
}

The videoSources array is pre-sorted by playback priority when the API builds it. Clients pick videoSources[0] for playback and use the getThumbnailUrl() helper for thumbnails (which has its own priority order — see below).

Priority Tables

Playback Priority (matches videoSources array order)

Rank Source Identifier field
1 Cloudflare Stream — Personalized personalizedVideo.cloudflareStreamId
2 Cloudflare Stream — Default cloudflareStreamId
3 Kinescope kinescopeVideoId
4 YouTube videoUrl
5 S3 (uploaded) uploadedVideoKey → presigned playbackUrl

Thumbnail Priority (used by getThumbnailUrl())

Rank Source Notes
1 Cloudflare Stream — Personalized customer-…cloudflarestream.com/{id}/thumbnails/thumbnail.jpg?time=1s
2 Kinescope kinescope.io/{id}/poster.jpg — fast and reliable
3 Cloudflare Stream — Default Cloudflare thumbnail at 1s
4 YouTube img.youtube.com/vi/{id}/hqdefault.jpg
5 Static photo (photoSource.url) External URL or signed S3 URL

Note: Playback prefers Cloudflare Stream over Kinescope (better playback experience). Thumbnails prefer Kinescope over default Cloudflare (faster, more reliable poster image). Personalized videos always win on both.

API Layer (OktaPT-API)

Module: src/media/exerciseMedia.ts

attachExerciseMedia(exercise, requestContext?) / attachExerciseMediaList(exercises, requestContext?)

Adds a media: ExerciseMedia field to any object that already has the raw media fields (videoUrl, photoUrl, uploadedVideoKey, uploadedPhotoKey, cloudflareStreamId, kinescopeVideoId, optional personalizedVideo). This is the standard way endpoints expose media to clients — call it once per response and the client gets the structured object alongside the legacy fields.

buildExerciseMedia(rawMedia, options?)

The lower-level builder. Constructs the prioritized videoSources array and resolves the photo source. For S3-backed videos and photos, it lazily signs storage GET URLs through the request context.

createExerciseMediaRequestContext(options?)

Returns an ExerciseMediaRequestContext that wraps S3 signing with an in-memory Map<storageKey, Promise<signedUrl>> cache. Pass a single context per request so multiple exercises in the same response don't pay for duplicate getSignedUrl calls.

exerciseMediaSelect

A Prisma.ExerciseSelect constant listing the columns required to build ExerciseMedia. Use it in Prisma select clauses so endpoints fetch exactly the fields they need.

buildLegacyExerciseMediaPayload(rawMedia, requestContext?)

Backward-compatibility helper. Returns the old-shape payload (videoStreamId, videoPlaybackUrl, photoSignedUrl, videoProvider, plus video/photo aliases) used by routes that haven't migrated to the structured type yet.

Helper functions exported by the module

  • getExercisePlaybackSource(media) — returns videoSources[0] (or null).
  • getExerciseThumbnailUrl(media) — applies thumbnail priority and returns a URL string or null.
  • hasPlayableExerciseVideo(media) — returns true if videoSources is non-empty.
  • getVideoPlaybackUrl(source) — turns a single ExerciseVideoSource into a playable URL (Cloudflare iframe, Kinescope page, raw YouTube URL, or already-signed S3 URL).
  • Provider-specific URL builders: getCloudflarePlaybackUrl, getCloudflareThumbnailUrl, getKinescopePlaybackUrl, getKinescopeThumbnailUrl, getYouTubeVideoId, getYouTubeThumbnailUrl.

Mobile (Okta-Mobile)

Module: lib/exerciseMedia.ts

The mobile module mirrors the API types so client code can consume ExerciseMedia directly. Because some endpoints still return the legacy flat shape, the mobile module also ships adaptLegacyExerciseMedia(carrier) which inspects the raw fields and synthesizes a structured ExerciseMedia value with the same priority order.

Key exports:

  • adaptLegacyExerciseMedia(carrier) — converts a legacy exercise object into structured ExerciseMedia.
  • getPlaybackSource(input) / getPlaybackSources(input) — returns the top source (or all sources). Accepts either a structured ExerciseMedia or a legacy carrier — resolveExerciseMedia picks the right path automatically.
  • getThumbnailUrl(input) — applies the thumbnail-priority chain.
  • hasPlayableVideo(input) — convenience for "should I show a play overlay?".
  • getCloudflareVideoThumbnail(streamId), getKinescopeThumbnail(videoId), getYouTubeThumbnail(videoId), getYouTubeVideoId(url).

Web (OktaPT-FE)

Module: lib/utils/exercise.ts (existing) — still home to getExerciseThumbnailUrl, hasVideoSource, getVideoSource, and the YouTube/Kinescope/Cloudflare URL helpers used by the web players.

New helper: lib/utils/enrichExerciseMedia.ts — enrichExerciseWithMedia(ex, detailsMap) fills in missing media fields on an exercise that arrived without them (e.g. an AI-generated exercise that started life as just { id, name }). It looks the exercise up in a details map (typically built from useExerciseDetails) and returns a clone with the media fields populated. Used by the plan-creation flow so thumbnails render in InitialPlanCreationModal, EditPlanModal, DayWorkoutPreview, PlanComparisonView, and SelectedExerciseCard.

Video Playback Implementation

Web (OktaPT-FE)

Component: components/patient/VideoPreviewModal.tsx

The web video player resolves the highest-priority source via the shared getVideoSource() helper and renders the matching player:

  1. Personalized Cloudflare — @cloudflare/stream-react <Stream> component
  2. Default Cloudflare — <Stream> component
  3. Kinescope — embedded via iframe
  4. YouTube — embedded via iframe with autoplay
const source = getVideoSource(exercise.exercise);
switch (source?.type) {
  case "cloudflare":
    return <Stream controls src={source.streamId} autoplay />;
  case "kinescope":
    return <iframe src={`https://kinescope.io/${source.videoId}?autoplay=true`} />;
  case "youtube":
    return <iframe src={getYouTubeEmbedUrl(source.url)} />;
}

Mobile (Okta-Mobile)

Component: components/workout/WorkoutVideoPlayer.tsx

The mobile player builds a sources array via getPlaybackSources() and steps through it on error (sourceIndex state). Each source uses the appropriate native renderer:

Provider Renderer
Cloudflare (any scope) WebView pointing at iframe.cloudflarestream.com/{id}
Kinescope WebView pointing at the in-app embed page (/embed/k/{id}) hosted on web
YouTube react-native-youtube-iframe
S3 Native <Video> component using the presigned playback URL

Kinescope Embed Page: OktaPT-FE/pages/embed/k/[id].tsx — minimal Next.js page that wraps the Kinescope iframe so the mobile WebView avoids Kinescope's mobile-detection issues.

Thumbnail Display Components

Web: components/patient/ExerciseThumbnailRow.tsx

  • 84×84 px, rounded corners
  • Spinner while the image loads, placeholder SVG on error
  • Semi-transparent play overlay (with a play icon) when the exercise has any playable video source
  • Tap opens VideoPreviewModal for video sources, no-op for photo-only

Web: components/ExerciseSelector/SelectedExerciseCard.tsx

The "selected exercises" list in the workout builder also renders a 56×56 px thumbnail with a play icon overlay, using the shared getExerciseThumbnailUrl and hasVideoSource helpers. Clicking the thumbnail opens the exercise detail dialog.

Mobile: components/workout/ExerciseThumbnailRow.tsx (and exercise lists)

Mobile uses the shared getThumbnailUrl() from lib/exerciseMedia.ts with React Native <Image> components.

PDF Export

When generating PDF workout plans, thumbnails need special handling because of CORS restrictions in @react-pdf/renderer.

Key file: OktaPT-FE/lib/pdf/imageUtils.ts

  1. Proxy fetch — images go through /api/image-proxy to bypass CORS.
  2. Base64 encoding — images convert to base64 data URLs for PDF embedding.
  3. Batch processing — prefetchImagesToBase64() handles multiple thumbnails in parallel.
  4. Caching — a Map<url, base64> avoids duplicate fetches across pages.
const thumbnailUrls = exercises.map((e) => getExerciseThumbnailUrl(e.exercise));
const thumbnailCache = await prefetchImagesToBase64(thumbnailUrls);
const base64 = thumbnailCache.get(thumbnailUrl);

Database Audit

Script: OktaPT-API/scripts/auditExerciseMedia.ts (run via ts-node against a database connection)

The audit script:

  1. Reads every active Exercise row and tallies the unique combinations of populated video provider columns (e.g. cloudflare+kinescope, youtube, none).
  2. Flags exercises that have multiple video providers populated (potential ambiguity) and exercises with only an uploadedVideoKey (the new S3 path, used to confirm migration progress).
  3. Reads PersonalizedExerciseVideo rows and reports any COMPLETED && isActive rows missing a cloudflareStreamId (broken processing).
  4. Hits a curated list of read endpoints and asserts that responses include the structured media field with the expected source mix.

Use it after migrations or large media-related changes to confirm no rows or endpoints regressed.

Adding a New Video Provider

1. Database schema

Add the new ID column to the Exercise model in OktaPT-API/prisma/schema.prisma:

model Exercise {
  // ...existing fields
  newProviderId String?
}

2. API: extend ExerciseVideoSource and the builder

In OktaPT-API/src/media/exerciseMedia.ts:

  • Add a new variant to the ExerciseVideoSource union with the appropriate fields.
  • Add the column to exerciseMediaSelect and RawExerciseMediaFields.
  • In toExerciseMediaPersistenceInput and buildExerciseMedia, push the new source at the correct playback-priority position.
  • Add a thumbnail branch to getExerciseThumbnailUrl and a playback branch to getVideoPlaybackUrl.

3. Mobile: mirror the changes

Apply the same union-extension and priority updates to Okta-Mobile/lib/exerciseMedia.ts (adaptLegacyExerciseMedia, getPlaybackSource, getThumbnailUrl).

Also add a renderer branch in components/workout/WorkoutVideoPlayer.tsx.

4. Web: add a renderer branch

Update components/patient/VideoPreviewModal.tsx and the getVideoSource resolver in lib/utils/exercise.ts.

5. Update the audit script

Add the new provider to buildVideoProviderSignature in scripts/auditExerciseMedia.ts so combinations get reported.

Key Files

File Role
OktaPT-API/src/media/exerciseMedia.ts Shared API module — types, builders, attach helpers, S3 signing
OktaPT-API/src/media/__tests__/exerciseMedia.test.ts Unit tests for the shared API module
OktaPT-API/scripts/auditExerciseMedia.ts DB + endpoint media audit
Okta-Mobile/lib/exerciseMedia.ts Mobile mirror — types, adaptLegacyExerciseMedia, thumbnail/playback helpers
Okta-Mobile/lib/__tests__/exerciseMedia.test.ts Mobile unit tests
Okta-Mobile/components/workout/WorkoutVideoPlayer.tsx Multi-provider mobile video player
OktaPT-FE/lib/utils/exercise.ts Web utility functions (thumbnail/video URL helpers, getVideoSource)
OktaPT-FE/lib/utils/enrichExerciseMedia.ts Hydrate AI-generated exercises with media for plan-creation flows
OktaPT-FE/components/patient/VideoPreviewModal.tsx Web video player modal
OktaPT-FE/components/patient/ExerciseThumbnailRow.tsx Web thumbnail row
OktaPT-FE/components/ExerciseSelector/SelectedExerciseCard.tsx Selected-exercise card with 56×56 thumbnail
OktaPT-FE/pages/embed/k/[id].tsx Kinescope embed page (used by mobile WebView)
OktaPT-FE/lib/pdf/imageUtils.ts PDF image proxy + base64 cache

URL Patterns Quick Reference

Video embed URLs

Provider URL pattern
Kinescope https://kinescope.io/{id}?autoplay=true&muted=true
Cloudflare iframe https://iframe.cloudflarestream.com/{id}
YouTube https://www.youtube.com/embed/{id}?autoplay=1&modestbranding=1&rel=0
S3 Presigned GET URL (TTL 1 hour, regenerated per request)

Thumbnail URLs

Provider URL pattern
Kinescope https://kinescope.io/{id}/poster.jpg
Cloudflare https://customer-j8qsy7zjqvqbtuqp.cloudflarestream.com/{id}/thumbnails/thumbnail.jpg?time=1s
YouTube https://img.youtube.com/vi/{id}/hqdefault.jpg