Clinical Assessment (Mobile)¶

Overview¶

Clinical Assessment lets a therapist film a patient performing a movement test on the phone and get back a measured, AI-assisted report a few minutes later. It runs on the doctor side of the mobile app (Start Test on an in-clinic patient card, or the assessment screen directly) and needs app version 2.1.0 or later from the app stores, because the on-device motion tracking is part of the native app.

What the September 2026 release changed, in short:

Tests are structured: a test type, a side, and for range-of-motion tests a movement. Shoulder tests offer five movements, and All shoulder tests records all of them for both sides in one session.
A setup screen shows where to stand and where to film from before the camera opens, with an animated demo of the movement.
The camera tracks the patient live: a skeleton for most tests, and for range-of-motion tests just the tested limb with a goniometer arc and a live angle, plus hints when something is out of frame.
Recording starts after a hold-still countdown, every clip can be reviewed and retaken before anything is sent, and unusable videos come back with a plain reason and a one-tap retake instead of an error.
Results lead with measured range of motion per side and let you watch the clip back with the skeleton drawn on top.

What the October 2026 release added (app version 2.2.0, a store update: the passive tests read the phone's motion sensors):

More regions on camera: elbow flexion; hip flexion, extension and abduction; ankle dorsiflexion as the knee-to-wall lunge; neck flexion, extension and lateral flexion; trunk flexion and lateral flexion. They work exactly like the shoulder tests: live overlay, server measurement, playback, run table and report.
Passive tests with the phone on the body: lay the phone on the patient's limb, head or trunk, hold still to zero it, then move the segment to the end of the range and back. The phone's sensors give the degrees, like a digital inclinometer. Every region has them, including the neck and the back; they are kept apart from the camera's active range everywhere (a "Passive" tag, their own rows in the session).

What the mid-October 2026 update added (an app update, no store release needed):

Measure first, name the test after: two free goniometers under Quick measure on the assessment screen. Phone on the body reads the range with the phone on any segment, measurement after measurement, whether you move the segment (passive) or the patient does with the phone held on (active); Camera reads the angle of whichever joint you tap on the live picture. Each measurement is then named from a searchable list of every range-of-motion movement, active or passive (or given a name of your own), and saved like a test of the catalog.
A faster start. The phone on the body zeroes itself after about a second of stillness, and a hand's tremor no longer restarts the count; the camera starts recording the moment you press Start when the patient already stands still, and otherwise counts 2-1 over one second.

For the engineering reference (data model, endpoints, pipeline, engines) see internal/clinical-video-assessment.md.

Tests and movements¶

The picker lists body regions head to feet. Each region shows its camera tests (the patient moves, the phone films and measures) and its passive tests (you move the patient with the phone laid on the segment). Camera tests are pre-checked; passive tests are opt-in, except in regions that only have passive tests (wrists).

Test	Side	Movements	Camera position
Gait Analysis	—	—	From the side while the patient walks across the frame
ROM - Shoulder	Left / Right / Both	Abduction (front), Flexion (side), Extension (side), External rotation (side, elbow at the side bent 90°), Internal rotation (behind, hand reaches up the back)	Shown on the setup screen per movement
ROM - Knee	Left / Right / Both	Flexion	Side
ROM - Elbow	Left / Right / Both	Flexion	Side of the tested arm
ROM - Hip	Left / Right / Both	Flexion (side), Extension (side), Abduction (front)	Shown on the setup screen per movement
ROM - Ankle	Left / Right / Both	Dorsiflexion as the knee-to-wall lunge (the shank's forward inclination)	Side
ROM - Neck	Flexion and extension once; lateral flexion Left / Right / Both	Flexion (side), Extension (side), Lateral flexion (front)	Shown on the setup screen per movement
ROM - Trunk	Flexion once; lateral flexion Left / Right / Both	Flexion (side), Lateral flexion (front)	Shown on the setup screen per movement
Squat Assessment	—	—	Front
Balance Test	Left / Right / Both	—	Front
Posture Screen	—	—	Front
Custom test	—	free text	—

All shoulder tests runs ten steps (both sides × five movements) ordered by camera position — front, right side, left side, behind — so you move the phone as little as possible. Any step can be skipped, retaken, or picked from the step pills at the top of the camera. The same ordering applies to every region: front view first, then the side, then from behind.

Neck flexion, neck extension and trunk flexion are measured once (the head and the trunk bend as one), so they have no side and appear in the session under "other tests" with a single value rather than in the right-vs-left table.

Passive tests (phone on the body)¶

Region	Passive tests	Patient position	Where the phone goes
Neck	Flexion, extension, lateral flexion (L/R), rotation (L/R)	Seated, looking straight ahead	Flat on top of the head, long side front to back
Shoulders	Flexion (supine), abduction (seated), external and internal rotation (supine, arm out 90°, elbow 90°)	As listed	Outer upper arm; back of the forearm for the rotations
Elbows	Flexion (supine), pronation and supination (seated, elbow 90°)	As listed	Back of the forearm; flat in the palm for pronation/supination
Wrists	Flexion, extension	Forearm on a table, palm down, hand over the edge	Back of the hand
Back	Flexion, extension, lateral flexion (L/R) standing; rotation (L/R) seated	As listed	Flat against the chest, long side vertical
Hips	Flexion (supine), extension (prone), abduction (side-lying), external and internal rotation (seated, knee 90°)	As listed	Front / back / outer thigh; front of the shin for the rotations
Knees	Flexion	Prone	Front of the shin
Ankles	Dorsiflexion, plantarflexion	Seated, knee 90°, foot hanging free	Top of the foot

Positions are chosen so the movement happens in a vertical plane (the phone tilts with the segment) or about the vertical (rotations, read by the gyroscope). The passive screen names the position and the placement for each test.

Running a test¶

Pick the test (and side, and movement). The step list appears for the shoulder protocol.
Set up the recording. The setup screen shows one framing rule ("Step back until every joint this test needs stays in frame for the whole movement — including at the very top"), the camera position for this exact movement, the start position where one matters, and a looping stick-figure demo seen from the camera's viewpoint. Press Open Camera.
Frame the patient. The live overlay shows what the phone sees. For range-of-motion tests you get the tested limb, a goniometer arc, a dashed guide for where the arm must be at 180°, and a Current / Peak readout. Hints tell you when to step back, turn the patient side-on or to face the camera, or when a joint is not visible. The readout is a live estimate; the analysed result is the record.
Start test. The phone watches the start position while you frame the shot. If the patient already stands still with every joint the test needs in view, recording begins the moment you press Start ("Go!" and a haptic tap, nothing to wait for). Otherwise the patient gets in position and holds still for one second while a 2-1 count runs (with a haptic tick); any movement or a hidden joint restarts it, and recording begins at "Go!". Tell the patient to start moving once you see "Go!": the analysis reads the still first second of the clip as the start position.
Review. The clip replays on a loop with Peak, Tracking and Length, a warning if joints were hidden for much of the clip, and a Retake button. Press Next: … to move on, Skip this test to leave a step out, or Finish & send at the end. Nothing is uploaded until you finish, so retakes are free. Leaving early asks whether to discard the recorded clips.
Processing. A single test shows the analysis progress; a protocol run returns to Recent Tests where each clip is analysed in turn. You also get an email when a report is ready.

If the selection includes passive tests, they follow once the recordings are sent (Continue to passive tests). A selection with only passive tests skips the camera entirely.

Running a passive test¶

The screen names the patient's position, where the phone goes and what to do. Put the patient in position, lay the phone on the segment and press Start test.
Hold still. After about a second of stillness the start position becomes the zero (a haptic tap). The zero is the average of that still second, so a hand's tremor or a knock on the phone does not restart it; sliding the phone into place does. A bar shows the second filling.
Move slowly to the end of the range and back. There is no need to stop at the top. The live angle is shown large, with the peak under it. On the way back, once the segment has come back past about half of its peak, the peak is captured by itself (another haptic tap), before you pick the phone up. A tap or a knock on the phone does not count as the peak, and neither does a quick out and back of under a second, such as a re-grip. Done (available once the segment has moved at least 5°) ends the measurement at any moment with the peak so far.
Review. The peak; Redo measures again, Next moves on, and in a sitting of several tests the pills at the top go back to any of them. Nothing is sent until Finish & send, after which the session opens with every measurement in place (there is no analysis to wait for).

If the phone was moved fast or shaken during a measurement the result carries the caveat "The phone moved fast or was shaken during the measurement — value may be unreliable"; in the session report that caveat makes a side difference a re-test. Rotation tests (neck, trunk) read the gyroscope, whose drift is measured during the hold-still count and taken out. Devices without motion sensors (the iOS simulator, some tablets) cannot run passive tests and say so.

Measure first, name the test after (Quick measure)¶

Two free goniometers sit under Quick measure at the bottom of the assessment screen, for the moments when you want a number before deciding which test it was, or for a test the catalog does not have.

Phone on the body. Choose Tilt (the default: any movement in a vertical plane) or Turn (rotations about the vertical, read by the gyroscope), press Start, lay the phone on the segment and hold it there. After about a second of stillness the start position is the zero; move slowly to the end of the range and back, and the peak is kept by itself (or press Done). The phone is then ready for the next segment at once: lay it on, hold still, move. Measure as many as you need, the patient moving on their own or you moving the segment, then press Stop (a movement still under way keeps its peak, as Done would). Each measurement is now a numbered row with its peak; Name 3 measurements opens Which test was it? for each in turn: a search field, region chips (Neck, Shoulders, Elbows, Wrists, Back, Hips, Knees, Ankles), every range-of-motion movement as Active (the patient moved, the phone held on) and as Passive (you moved the segment) with Right / Left on the sided ones, your last picks on top, and, for anything the list does not have, Save as “…” with the name you typed and whether it was active or passive. The pick says who moved; nothing else is asked. A row can be renamed by tapping it or removed with its cross, named or not; a capture taken by mistake (the phone moved between segments) can also go with Remove while it is being named, and Finish & send posts the named ones, as one session when there are several. Active phone measurements get their own rows ("Hip · Flexion · Active (phone)") and the joint-angle typical ranges, apart from the camera's rows and the passive ones; a test saved under its own name is listed with the other tests, with its peak.

Camera. The whole body is tracked and every joint it sees gets a ring: tap one and its angle is read live, with the current value and the furthest the movement went. The measure follows the joint and the view: a shoulder or a hip reads flexion from the side and abduction facing the camera, the ears read neck flexion from the side and lateral flexion facing the camera, the mid-trunk reads trunk flexion or lateral flexion the same way; knees, elbows and ankles have one measure each. A measure that counts from a start position (the neck, the trunk) sets its zero itself once the patient holds still ("Hold still to set the zero…"); a movement in the other direction reads negative on the same measure. Press Start (the recording starts at once when the patient is still), Stop, review the clip, then Name the test: the tapped joint's test is the first suggestion, the full list and a name of your own follow. The clip is then analysed by the server exactly like that test recorded from the catalog.

If a video cannot be analysed reliably you see "Video needs a retake" with the reason (no person detected, person only partly visible, the joints for this test hidden or out of frame, not enough usable footage, or a file that could not be read), a hint, and a Retake Video button that reopens the setup screen for the same test. Recent Tests marks these with an amber Retake badge.

Reading the results¶

Video quality (good / acceptable / poor) and AI confidence (high / moderate / low) badges in the header. A poor video keeps the confidence low on purpose.
Measured range of motion per side: the peak in degrees (or a percentage of trunk length for the hand-behind-back reach), how the peak was found ("Peak of the movement", "Peak at the very end of the recording — the clip may have stopped early", or "No clear movement detected — estimated value"), how much of the clip was tracked, a "possibly up to" value when tracking was patchy, the left/right difference, and caveats such as "Joint was hidden at the end of the range — peak likely under-read" or "Movement was not perpendicular to the camera". These numbers are measured from the video in the camera plane, like a goniometer; the written report interprets them.
The video, replayed in-app with the skeleton drawn on top (toggle with Show overlay / Hide overlay), a scrub bar and a Peak button that jumps to the measured peak. Older assessments recorded before this release play without the overlay.
Frames: the peak frame is shown as the hero image; the rest sit behind All frames (tap to seek the video, long-press to open the still).
Overall summary, key findings, detailed assessment, recommendations and areas for further evaluation, as before.
Patient: assign or reassign the assessment to a patient from the header chip; tests started from an in-clinic patient card are assigned automatically.
Passive tests open with a "Passive · phone on the body" badge, the measured passive range (the peak of the movement) and how it was measured (tilt or turn, the position and the placement). They have no video and no written AI analysis. A free measurement (Quick measure) says that no position or placement was prescribed; one the patient moved for carries "Active · phone on the body" instead, and one saved under its own name shows that name and its "Measured range".

In the session (run) screen, passive measurements have their own rows marked "Passive" next to the camera rows of the same movement, so active and passive range sit side by side, and active measurements taken with the phone on the body have theirs, marked "Active (phone)"; the report keeps them apart the same way (keys ending in :passive and :active).

Motion tracking settings¶

The gear on the assessment screen opens Motion tracking, the model that follows the patient on screen while you record. Options depend on the phone: MediaPipe (33 joints, iPhone and Android), Apple Vision (built into iOS, fastest, 17 joints, the iOS default) and YOLO11 Pose (17 joints, iPhone only). Try it on camera opens a live view with the chosen engine without creating an assessment. The choice is per device.

Live tracking needs a real device: it is not available in the iOS simulator, in Expo Go, or on Android versions before 8.0. Without it the camera still records, the countdown is a plain three-second timer, and the analysis still runs on the server.

Recent Tests¶

Three groups, newest first: Needs Attention (not yet uploaded, failed, or needing a retake), In Progress, and Past Assessments. Tapping an item resumes the upload, retries processing, opens the retake flow, or opens the results.

Key Files¶

File	Role
`Okta-Mobile/app/(doctor)/(clinical-assessment)/index.tsx`	Test, side and movement selection; shoulder protocol; settings gear
`Okta-Mobile/app/(doctor)/(clinical-assessment)/setup.tsx`	Framing rule, camera position, movement demo
`Okta-Mobile/app/(doctor)/(clinical-assessment)/record.tsx`	Camera session with live overlay, countdown, review/retake, step list, background uploads; hands over to the passive screen
`Okta-Mobile/app/(doctor)/(clinical-assessment)/passive.tsx`	Passive tests: position and placement, hold-still zero, live angle, the peak captured on the way back, send
`Okta-Mobile/app/(doctor)/(clinical-assessment)/measure.tsx`	Quick measure, phone on the body: free goniometer, the picker, the list of named measurements, send
`Okta-Mobile/app/(doctor)/(clinical-assessment)/measure-camera.tsx`	Quick measure, camera: tap a joint, live angle, record, name, upload
`Okta-Mobile/components/clinical/TestPickerSheet.tsx`, `Okta-Mobile/lib/clinical/freeTests.ts`	"Which test was it?": search, region chips, sides, recent picks, a name of your own
`Okta-Mobile/lib/clinical/freeMeasure.ts`, `Okta-Mobile/components/clinical/TapTargetsOverlay.tsx`	The tap → joint → measure mapping and the rings on the tappable joints
`Okta-Mobile/hooks/useInclinometer.ts`, `Okta-Mobile/lib/clinical/inclinometer.ts`	The phone as an inclinometer: the still-second zero (tremor-tolerant), accelerometer tilt, gyroscope turn, the peak and the way back, the posted measurement
`Okta-Mobile/lib/clinical/sendSensor.ts`	Posting a sitting's sensor measurements (shared by the passive screen and Quick measure)
`Okta-Mobile/app/(doctor)/(clinical-assessment)/processing.tsx`	Progress, failure retry, quality-retake card
`Okta-Mobile/app/(doctor)/(clinical-assessment)/results.tsx`	Measured-ROM card, video playback with overlay, findings
`Okta-Mobile/app/(doctor)/(clinical-assessment)/motion-cap.tsx`	Engine try-out screen
`Okta-Mobile/components/clinical/`	Overlays, countdown, review card, movement demo, measured-ROM card, settings sheet
`Okta-Mobile/lib/clinical/`	Test steps, angle math, smoothing, start-position detection
`Okta-Mobile/services/clinicalAssessmentService.ts`	API client
`OktaPT-API/src/routes/clinical-assessment/`	Assessment endpoints
`OktaPT-API/src/services/assessmentPipelineService.ts`	Analysis pipeline