System Architecture
Top View
Loading page
3d view
3d view
Simulation Loading Page
Stats

MetaPitch: Unleashing Spatial Awareness ⚽️

Tagline: Revolutionizing sports analytics with Gemini 3. MetaPitch renders match data in 3D and uses AI to simulate and predict optimal player trajectories in real time.

Overview

MetaPitch is a 3D sports trajectory simulation and analysis platform that ingests real or mock match data and renders players on a holographic, FIFA-standard pitch.
At its core is Metavision, an AI layer powered by the Gemini 3 API that performs spatial reasoning on live field states to predict optimal movement paths, uncover scoring opportunities, and simulate high-stakes “awakening” scenarios.

Inspiration

The spark for MetaPitch came from the anime Blue Lock, where strikers unlock “Spatial Awareness,” the ability to perceive the entire field and choose the best path to score.
We asked: What if we could build an AI that sees the game like Isagi Yoichi?
That idea clicked perfectly with Gemini 3’s reasoning and low-latency capabilities, and MetaPitch was born.

What It Does

MetaPitch helps users analyze soccer plays with an interactive, visual workflow:

3D Holographic Pitch: Visualize player and ball movement on a stylized pitch.
Timeline Scrubbing: Move frame-by-frame through a play and inspect positioning.
AI Trajectory Prediction: Generate and render predicted runs and responses.
Opportunity Detection: Highlight passing lanes, spaces, and “best path” options.
Stats View: Show xG and goal probability (if included in your pipeline).

Gemini Integration (Detailed)

MetaPitch uses the Gemini 3 API as the core AI engine for spatial reasoning, trajectory prediction, media extraction, and camera understanding. The integration is designed around two models with complementary strengths:

gemini-3-flash-preview for low-latency, interactive predictions during live exploration.
gemini-3-pro-preview for deeper reasoning, constraint validation, and higher-fidelity analysis.

1) Forward Simulation Engine (Metavision)

When a user clicks Analyze, MetaPitch sends the current field state to Gemini, including:

Player positions (x, y), estimated velocities, and team identities
Ball position and possession context
Pitch bounds, frame timestamp, and scenario metadata

Primary path (real-time):
MetaPitch calls gemini-3-flash-preview to generate predicted movement for the next 10+ frames. The prompt is inspired by Blue Lock’s “Metavision” and instructs Gemini to produce structured, render-ready trajectories rather than descriptive text.

Optional second pass (higher fidelity):
MetaPitch can call gemini-3-pro-preview to validate and refine the flash output:

Enforce constraints (stay in-bounds, realistic speeds, avoid collisions)
Re-rank candidate routes based on spatial advantage
Produce a short rationale describing why the chosen path increases scoring odds

Outputs are constrained to a strict JSON schema (waypoints, timestamps, confidence, and ranked candidates). The frontend converts these waypoints into splines and animates them in Three.js, allowing users to scrub through time and compare alternate outcomes.

2) Video / Image Extraction (Fun Mode)

In “Fun Mode,” MetaPitch uses Gemini to convert unstructured media into structured match state:

gemini-3-pro-preview analyzes an uploaded field image or a video URL to detect jersey colors, approximate player locations, and the ball position.
The system reconstructs a formation and generates a normalized coordinate representation that can be rendered on the 3D pitch.

This feature allows users to bring in non-tracking footage and still generate a playable, analyzable scene.

3) Camera Pose Estimation

MetaPitch uses Gemini to infer camera geometry so that overlays align to the pitch:

gemini-3-pro-preview reasons over broadcast angles and field markings to estimate a perspective transform (pose + calibration hints).
This enables perspective-correct overlays and improves mapping between 2D broadcast frames and pitch coordinates.

4) Technical Implementation

SDK: @google/genai
Models: gemini-3-flash-preview, gemini-3-pro-preview
Routing: All calls flow through the Express backend to keep the API key secure and to standardize error handling and telemetry.
- POST /api/predict → trajectory + forward simulation (Flash, optionally Pro refinement)
- POST /api/extract → image/video → player/ball coordinates (Pro)
- POST /api/camera-pose → pose estimation + geometry alignment (Pro)

This architecture transforms MetaPitch from a static visualizer into an intelligent analytics platform that “sees” the field like a world-class striker.

How We Built It

Layer	Technology	Purpose
Frontend	React 19, Three.js, Vite, Zustand	Interactive 3D pitch, playback controls, state management
Backend	Node.js, Express.js	API orchestration, secure Gemini calls, logging
Database	SQLite	Local-first match/frame storage and fast reads
AI Engine	Gemini 3 API (`gemini-3-flash-preview`, `gemini-3-pro-preview`)	Spatial reasoning, prediction, extraction, pose estimation
Data Pipeline	Python	Mock match generation and scenario scripting
IDE	Antigravity	Rapid prototyping and iteration

Challenges We Faced

Real-time Rendering: Balancing fidelity with stable FPS in Three.js required careful scene optimization and efficient updates.
Spatial Prompt Engineering: Getting structured, machine-usable coordinates (not essays) needed schema constraints and iterative prompting.
Data Interoperability: Designing a schema that bridges professional tracking formats and 3D visualization without losing key semantics.

What We Learned

Reasoning as a Renderable Output: The best AI output is the one the UI can directly consume.
Low Latency vs Deep Reasoning: Flash powers interactive exploration; Pro improves correctness and strategic depth.
Clean Pipelines Win: A modular data-to-visual pipeline reduces bugs and makes iteration faster.

What’s Next

Pro League Integration: Move from mock data to real match feeds and richer event context.
Prediction Challenges: Multiplayer mode where users compete against Metavision predictions.
Multi-sport Expansion: Extend the engine to basketball and American football.