Dynalytix

The Gap

Running and field sports have decades of motion capture research. Climbing has very little, and what exists almost never records what the climber was attempting or what they felt while doing it. A camera can recover joint angles. It cannot recover intent, outcome, or a twinge in the left shoulder on the third move. Those have to come from the climber, and they have to be written down in a form that lines up with the pose data. That is what this project builds.

The Labeling Framework

A move is described through three lenses at once. They are parallel descriptions of the same move, not steps in a pipeline. Sensations sit inside the outcome lens and attach to individual frames.

Why a Grid and Not a List

The first schema had a flat list of twelve move types: static, deadpoint, dyno, lock-off, gaston, undercling, and so on. It broke on real footage. A single move is often a bump and a deadpoint at once, and a flat list forces one label. It also mixed categories: a gaston is a kind of hold, not a kind of move.

The rebuilt schema splits those apart. Hold types moved to the environment lens. Moves became a cell in an approach × tags grid with an independent size, so "bump + deadpoint" is simply dynamic × bump. The static versus dynamic line is defined by center-of-mass motion, which the pose data can check. Taylor Reed's feedback drove the three-hold model and the tag list, and the form-quality anchors reference published qualitative indicators of climbing movement.

A Labeling Session

• Upload. A climber uploads a video of a boulder or route attempt. It is stored privately under their account.
• Extract. Pose runs on the video: 33 landmarks per frame, with real frame-rate detection so a 60 fps phone clip keeps correct timestamps.
• Define. The labeler scrubs the video and marks the start and end frame of each move.
• Describe. A side panel, with the video still visible, walks through environment, strategy, and outcome. Every option has a definition one click away.
• Tag. The labeler steps to any frame and adds a sensation: type, side, location, level.
• Export. One table, one row per frame.

What Comes Out

Each row of the export is a single frame: frame number and timestamp, 33 landmark positions, 12 joint angles, per-landmark velocities, and center of mass, joined to the environment, strategy, and outcome of the move that frame belongs to, plus any sensation tagged on it. A researcher can filter to every frame where a climber reported a level 6 or higher sensation in the right shoulder and read the elbow and shoulder angles directly.

• Two papers planned, on purpose. The first would be the pipeline, the schema, and the labeled dataset, as a resource for other researchers. A model would come second, only once there is enough data to support real performance claims.
• Reliability first. The next build adds a study mode where several labelers rate the same video independently, so agreement between them can be measured before the schema is called stable.
• Release. A de-identified pose and label dataset is planned alongside the paper. Original videos stay private and are retained so pose can be re-extracted later with whole-body models.

Technical Highlights

• Built solo: FastAPI on Railway, Supabase Postgres with row-level security and per-user scoping, video and exports on Cloudflare R2
• 197 automated tests (125 frontend, 72 backend against a scratch Postgres); infrastructure sized for 20 concurrent labelers
• Pose extractor rewritten around play-through capture with real frame-rate detection, so labels and landmarks share one clock
• Schema v3 cutover (Sep 2026) moved the platform from SQLite and a GitHub-sync datastore to Postgres and object storage in a single chained merge
• Hold detection ships off on purpose: every published climbing-hold YOLO model is AGPL or unlicensed, so labeler-drawn boxes become training data for a detector I can license cleanly
• Next: GPU-backed pose extraction on a serverless worker, and a multi-rater reliability study for the paper

Also in Dynalytix: the Scoring Platform

An earlier part of the project is a movement scoring prototype. It takes a front and a side video of a deep squat, extracts pose in the browser, and scores the movement from 0 to 3 with explicit rules. Each rule traces to a published threshold (Butler 2010, Heredia 2021), so a score can be checked by hand, and it is not a black box. It is paused while the climbing dataset takes priority, and it stays online as a demonstration.

• Dual-angle assessment: front and side views are scored independently and merged per criterion
• Five biomechanical criteria, with left and right differences flagged frame by frame
• Two report views from one assessment, with the raw joint angles included so a reader can verify the score
• Designed for human review: every automated score is a draft that a person can override before approving

Demo and walkthrough → · How the scoring works → · Open the prototype →

How It Got Here

• 2023. Started as an OpenCV / MediaPipe climbing tracker I built at 16, in my last year of high school (original repo). Taylor Reed, VP of the International Rock Climbing Research Association, started advising it that year.
• 2025. After setting the project down for a while, picked it back up with Taylor again and rebuilt it as a multi-user labeling platform.
• Before the rebuild. Built the movement scoring prototype described above. That work is paused; the pose pipeline carried over.
• 2026. Brought the three-lens labeling framework to Taylor. He rejoined as co-author on the manuscript in preparation and has shaped the schema through multiple feedback rounds.
• Now. The platform is live on its current schema. Pilot beginning, so there is no dataset to cite yet. Manuscript in preparation.

Stack