a11Yum

Inspiration: A TikTok creator adapting a recipe for a follower with a disability, highlighting the need for accessible cooking instructions.
Problem: Traditional recipes often assume full physical, sensory, and cognitive abilities, excluding many people.
Solution: An AI-powered platform that automatically adapts any recipe for various accessibility needs by providing alternatives for steps, ingredients, and tools.

Recipe Input:
- URL Parsing: Extracts and adapts recipes from any provided URL.
- Text Generation: Generates new accessible recipes based on a user's text description.
AI Backend: A FastAPI backend with Google's Agent Development Kit (ADK) and Gemini 2.0 Flash model processes requests. WebSockets provide real-time updates.
Frontend App: A cross-platform app for iOS, Android, and web built with React Native and Expo.

Motor Disabilities:
- No-chop alternatives (e.g., pre-chopped ingredients, food processors).
- One-handed techniques and tools.
- Alternatives to heavy lifting (e.g., smaller batches).
- Recommendations for ergonomic and adaptive tools.
Physical Disabilities:
- Alternative cooking methods (microwave, oven, slow cooker).
- Steps adapted for seated/wheelchair cooking.
- Energy-saving steps with built-in rest periods.
Cognitive Disabilities:
- Instructions broken down into simple, single-action steps.
- Visual aids and clear progress tracking.
- In-built memory support and reminders.
Vision Impairments:
- Detailed audio descriptions for steps.
- Suggestions for using tactile or auditory cues (touch, sound).
- High-contrast UI and prominent safety warnings.

Voice Accessibility: Integrating cross-platform speech-to-text and text-to-speech in React Native was more complex than anticipated. The UI is ready, but full implementation is pending.
AI Agent Tuning: Fine-tuning the AI to provide safe, practical, and context-aware accessibility suggestions required significant prompt engineering.
Data Synchronization: Managing user data and preferences (via Auth0 metadata) consistently across platforms, especially with offline support, was complex.
Recipe Parsing: Accurately scraping and structuring data from diverse website formats proved challenging.

Functional MVP Delivered: Successfully built a working end-to-end product with URL parsing, AI generation, user accounts, and a cross-platform mobile app.
Advanced AI Integration: Implemented Google's ADK and Gemini model to create a sophisticated, accessibility-focused agent.
Comprehensive Data Model: Designed a robust recipe structure where every ingredient, tool, and step can have multiple accessibility-friendly alternatives and safety warnings.
Modern Tech Stack Mastery: Successfully learned and deployed a modern stack including Google ADK, FastAPI, and React Native with Expo.

AI/ML: Prompt engineering and maintaining user context are critical for developing a useful and safe AI agent.
Accessibility: True accessibility requires providing multiple, meaningful alternatives for every action and prioritizing user safety.
Cross-Platform Dev: React Native and Expo are powerful for rapid development but require careful management of platform-specific differences and performance.
Backend: FastAPI with async patterns is highly effective for building scalable, real-time applications.
Teamwork: Clear communication, task prioritization, and flexibility are essential for success in a time-constrained environment like a hackathon.

Complete Voice Control: Fully implement speech-to-text, text-to-speech, and voice navigation for a hands-free experience.
Advanced AI Features:
- Automatic nutritional analysis and allergy management.
- Adaptations for specific diets (e.g., vegan, keto).
- Real-time cooking assistance and troubleshooting.
Global Expansion:
- Add multi-language support.
- Adapt recipes for regional cuisines and ingredient availability.
- Ensure compliance with global accessibility standards (WCAG).