Inspiration Our inspiration for ReelEngine AI came from the Twelve Labs Challenge, which highlighted the common challenges faced by video editors and independent creators. We wanted to tackle the time-consuming tasks of searching, organizing, and tagging video footage by using AI to streamline the process. What it Does ReelEngine AI offers:
Natural language search: Users can search for clips using simple text queries. Automatic indexing and tagging: Uploaded clips are indexed and tagged with descriptions, making it easy to retrieve relevant footage. Built-in narration and translation: The platform generates narrations based on the user’s script and can automatically translate and dub them into multiple languages using MyShell OpenVoice models. How We Built It We integrated Twelve Labs’ Marengo and Pegasus models for indexing, searching, and organizing video clips. We used MyShell OpenVoice models directly for multilingual text-to-speech (TTS) and voice cloning to enhance the audio experience. MoviePy was used for video processing and editing. Python served as the core language for backend development and orchestrating these services. Challenges We Ran Into Implementing smooth transitions between multilingual voiceovers and ensuring audio sync across different clips was a key technical hurdle. Balancing automated clip selection with user-defined creativity while maintaining a coherent trailer structure. Accomplishments That We're Proud Of Automating the entire process of trailer creation, from video clip indexing to final editing and exporting. Leveraging AI for seamless multilingual translation and narration using voice cloning. What We Learned We gained insights into how AI can dramatically reduce time in video editing workflows and learned how to effectively integrate AI models to handle multilingual video production. We also tackled challenges in managing audio synchronization for voiceovers. What's Next for ReelEngine AI Adding interactive features, like customizable endings or alternate scene choices. Expanding language support and refining voice cloning for even more realistic dubbing. Incorporating advanced video effects to further streamline the trailer-making process. Technologies Used Languages: Python Frameworks: FastAPI Platforms: Twelve Labs for video indexing and storage Video Processing: MoviePy AI Models: Twelve Labs Marengo and Pegasus, MyShell OpenVoice for TTS and voice cloning
Log in or sign up for Devpost to join the conversation.