PuckSense

PuckSense is an AI-automated website that is tailored to help hockey players and coaches to identify gaps in the game. Simply upload a gameplay video and PuckSense will break it down in full detail.

Comment

Inspiration

While brainstorming a way to “revolutionize learning,” our team realized that learning extends beyond traditional academics. Hockey is a technical sport; however, many people neglect the importance of the mental aspect. Many aspiring athletes practice hard and show a strong “want” factor, yet still fall short. Our goal in creating PuckSense is to bridge the gap in gamesense to help the athlete who dreams big make his dreams come true.

What It Does: PuckSense — AI Tactical Video Analyzer for Hockey

PuckSense is an AI-powered website that focuses on identifying the strengths and weaknesses during gameplay. PuckSense can be applicable in several contexts - from helping players identify their next steps to helping coaches grasp the gaps in overall team strategy. After receiving a video, PuckSense pinpoints the frames with the most play and conducts a full analysis, ensuring that no aspect of the game goes unnoticed.

As the user scrolls through the video, the website shows exactly where the key frames are with its pop-up feature. This feature ensures that the user is able to quickly identify where the AI’s criticism is focused on in the video.

Two-Stage AI Analysis

  1. Frame Screening : Extracts frames at 2-second intervals to isolate key moments.
  2. Tactical Analysis : Each selected frame is analyzed by LLaVA (via Ollama) to assess:
    • Puck carrier decisions
    • Player positioning
    • Defensive coverage
    • Offensive setups and opportunities

Holistic Summary Generation

After analyzing all frames, PuckSense compiles a comprehensive tactical report, identifying:

  • Offensive and defensive patterns
  • Missed opportunities
  • Key plays and transitions
  • Actionable coaching recommendations

The final output includes both per-frame breakdowns and a game-sequence summary, delivered as structured JSON to the frontend for visualization.

How We Built It: Frame-by-Frame, Decision-by-Decision

  • Frontend: Cursor and a load of figuring things out.
  • Backend (Flask): Handles video uploads, frame extraction (every 2 seconds), and AI analysis using LLaVA via Ollama.
  • AI Pipeline:
    1. Stage 1 :Detects active play frames.
    2. Stage 2 :Performs detailed tactical reasoning on each frame.
    3. Final Stage : Aggregates results into an overall summary using multi-frame contextual analysis.
      Output: Structured JSON reports containing frame-level insights and holistic summaries for frontend visualization.

Challenges We Ran Into

Performance Bottleneck

Our first version analyzed 1 frame per second, taking 7–8 minutes for a short 10–20 second clip. We optimized to 1 frame every 2 seconds, drastically cutting inference time without losing key tactical context.

Prompt Consistency

Getting reliable tactical insights from multimodal models was tricky. The model occasionally produced irrelevant or vague results. We solved this by iteratively refining our system prompts and guiding the model to focus on hockey-specific reasoning.

Connectivity Issues

Wi-Fi instability during testing slowed down uploads and model inference — a classic hackathon challenge.

Accomplishments We’re Proud Of

  • End-to-end build in under 20 hours: We joined late and had midterms, yet still built a full AI-powered analysis platform : backend, AI logic, and frontend.
  • Meaningful tactical insights: The model successfully identified player positioning, puck carrier options, and defensive gaps.
  • Team resilience: Despite sleep deprivation and tight deadlines, we transformed our passion for hockey into a working product.

What We Learned

  • Building vision-language pipelines requires precise prompt control for domain relevance.
  • Efficient video analysis means balancing frame rate, context depth, and latency.
  • UI/UX decisions : like how to visualize plays, drastically affect how coaches interpret insights.

We also learned that sometimes, less is more: analyzing fewer frames at higher reasoning quality yields better tactical comprehension.

What’s Next for PuckSense

  • Player Tracking Integration: Add positional heatmaps and player movement trails using object detection.
  • Live Game Support: Near real-time tactical feedback during live matches.
  • Model Fine-Tuning: Train LLaVA with hockey-specific data for sharper domain understanding.
  • Coaching Dashboard: Build interactive visual analytics for comparing players and games.

What we have currently is only the start for PuckSense. PuckSense will grow to become a more personalized and interactive platform. We plan to have PuckSense generate performance reports that focus on the trend of specific weaknesses over multiple games. PuckSense will be able to generate customized drill suggestions to target the weak areas.

The website will also generate scenarios similar to the ones identified in the game analysis and quiz the reader on the best course of action. PuckSense will focus on not only identifying the issue but also providing personalized suggestions and resources to guide the user towards becoming a better player.

Model Overview

$$ \text{Video} \rightarrow \text{Frame Extractor} \rightarrow \text{Vision Model (LLaVA)} \rightarrow \text{Aggregator} \rightarrow \text{Tactical Summary} $$

Built With

Share this project:

Updates