Landing Page
Movement Detection Page (Contributor)
Contributions Page (Contributor)
Stats Page (Contributor)
Business Dashboard (Business)
Data Analysis Integration (in Dark Mode, Business)
Developer Integration (in Dark Mode, Business)
Live API Fetch Example

OpenMotion

Inspiration

Robotics companies like Tesla, Figure AI, and Boston Dynamics spend hundreds of thousands of dollars hiring employees to manually record repetitive tasks for robot training data. Inspired by platforms like Poke.com and Scale AI that crowdsource text labeling, we asked: what if we could do the same thing for motion data? We wanted to make training data collection accessible, affordable, and fair — putting money directly in the hands of everyday workers.

What it does

OpenMotion is a crowdsourced marketplace connecting robotics companies with everyday contributors. Our platform has 4 main features:

Task posting - Businesses create tasks (e.g., "warehouse picking," "folding laundry"), set prices, and define requirements
Real-time pose capture - Contributors record themselves using just their webcam while TensorFlow BlazePose extracts full 3D skeletal data (33 body keypoints) — no special hardware needed
AI verification - Gemini Vision API automatically verifies videos match task requirements before submission
Data access - Businesses download complete datasets (video + JSON pose data), access a developer API, and analyze patterns using built-in Jupyter notebooks
User Incentive - Users are paid when they submit videos of doing tasks

Additional features

SMS integration via poke.com for accessibility to workers without computers
Privacy-first design with client-side face blurring before upload
Real-time 3D avatar visualization that mirrors user movements
Track earnings and browse available tasks

How we built it

Our tech stack combines modern tools for a robust, scalable solution:

Frontend: Next.js 16 with TypeScript, Tailwind CSS, and Framer Motion for smooth animations
Backend: Firebase for authentication, Firestore for data storage, and Firebase Storage for video files
AI Processing: TensorFlow.js with BlazePose for client-side pose detection and Google Gemini Vision API for video verification
3D Visualization: Three.js with React Three Fiber for the live skeleton avatar that mirrors user movements
Developer API: Custom REST endpoints with API key authentication
Data Analytics: Built-in Jupyter-style notebook environment for pose data analysis
SMS Service: MCP server (FastMCP/Python) powering poke.com text-based platform interaction

Challenges we ran into

Browser Performance: Loading MediaPipe + TensorFlow.js dynamically while maintaining real-time skeleton overlay
Canvas Recording: Recording from canvas (not raw video) so face blur and skeleton are baked into the file required canvas.captureStream() synchronization
Data Quality: Sanitizing pose frames for NaN/Infinity values before JSON serialization
Rate Limits: Redesigning Gemini verification flow to avoid automatic rate limit hits

Accomplishments that we're proud of

Full 3D pose capture running entirely in the browser — no server processing, downloads, or special hardware
Built a complete end-to-end pipeline from task creation to API access in 36 hours
Real-time 3D avatar provides immediate visual feedback while recording
Privacy-first with client-side face blurring
Made the platform accessible via SMS for workers without computers

What we learned

Browser-based ML is incredibly powerful — TensorFlow.js tracks 33 3D keypoints in real time without lag
Designing for two different user types (contributors and businesses) requires careful role-based UI decisions
AI verification adds huge quality control value but needs rate limit consideration
Accessibility matters — SMS integration opened the platform to excluded users
Edge case handling is critical for production-ready applications

What's next for OpenMotion

Future enhancements we're planning:

1. Mobile & Advanced Detection

Mobile-native recording app with optimized camera controls
Multi-person pose detection for collaborative tasks
Marketplace matching based on skills and location

2. Payments & Quality

Stripe/PayPal integration for real money transfers
Automatic quality scoring based on keypoint confidence
Advanced analytics with pre-built ML model templates

3. Enterprise Features

Dedicated data pipelines and SLAs
Priority contributor pools
Custom training workflows

Our vision is to make OpenMotion the go-to platform for crowdsourced motion data, making robot training more efficient and accessible for everyone.