[U19] - Kairosity

Kairosity unlocks learning Using AI tags and spaced repetition, it turns your space into an interactive environment, making learning fun, contextual, and efficient. Boost retention across any field.

Comment

Inspiration

  • We are a team passionate about language learning. We know a lot of language learning apps and we often the tool ANKI to learn languages.
  • Moreover, our personal experiences working abroad, in specific contexts such as robotics, and travelling to specific events (for example, a friend's wedding in Japan) has lead us to think how to learning and retaining vocabulary for specific and familar contexts quickly and effectively.
  • With this background, we thought how could we improve the main issues with ANKI by using mixed reality. The two main issues are the following:
    • ANKI lacks context. The images had to be taken manually by me, and creating the deck is tedious. If I have a familiar and personalized context, I can optimize my experience.
    • ANKI has a high entry barrier. User-created decks often have an enormous amount of cards, and if you want to create the deck personally, you must invest a lot of time in it. We intent to improve this creation phase by using AI and a simple UX based on natural gestures.

What it does

Kairosity integrates spaced repetition and AI-generated tags to enhance memory retention and engagement. By transforming the physical space into an interactive learning environment, the app enhances recall through contextual and immersive experiences. Whether it's language learning, professional skills training, or personal development, the challenges allow the user to make its learning process fun and efficient.

The app usage cycle fundamentally consists of two phases: creation and execution. The user generates an initial group of cards they want to learn and later can start a session to solve the cards scheduled for that moment, based on the assimilation of the same.

In detail, the experience unfolds as follows:

  1. The first time you enter the app, you are prompted to create the group of cards. The user can position the cards in space, using both automatic object recognition through AI or manually, inserting a voice command with the word or concept in their native language.
  2. Once all the cards are created, the group of cards can be executed so that the challenges are positioned in the corresponding places. At this point, the user passes through the points and completes the challenges. Depending on the quality of the response in each challenge, the card is scheduled for a future session (soon, if not answered well, or later, if the concept is already assimilated).
  3. When all the challenges are completed, the total time spent is anecdotally notified and the session ends until the scheduled cards are available again for play.

How we built it

  • First, we brainstormed. We analyzed how existing applications that use spaced repetition technology like ANKI or Kanji Study (in the case of Japanese) work. Additionally, we focused on the specific use case of languages and asked professionals and language learning enthusiasts to validate the idea.
  • Iteration of the idea:
    • Initially, we thought about covering the room with personalized post-its and being able to return to it whenever the user wanted. But the intelligent factor of the application was missing. Having a space always full of information is overwhelming and would be a pain point, counterproductive for the user.
    • It is better if only the elements you remember least appear in each daily session. At the exact moment. This is how we improved the concept of simply tagging a space. It is not just a tagged space, but it is also intelligent and adapts to each user's learning rhythm through a spaced repetition algorithm. This allows optimizing learning sessions, reducing the time per session, and maximizing the correct retention of concepts.
  • Prototyping and design document:
    • We made some physical and virtual prototypes to validate the app's flow.
    • We used collaborative online tools to create flowcharts and class diagrams to improve communication among team members.
  • Implementation:
    • For project production management, we used a kanban board to distribute tasks effectively.
    • Initially, we implemented the core features: AI integration, challenge mechanics with Meta SDKs interactions, and the overall architecture of the app flow. Later, we moved to the bug-fixing and polishing phase.

Challenges we ran into

  • Network instability caused problems when integrating and testing the various APIs our project uses, such as ChatGPT, Google Translate, or Datamuse.
  • Implementation of both AI-Kits:
    • Real-Time Object Detection & CV Kit by Lukas Moro. We managed to integrate the entire flow with Python and OBS linked to the HMD feed but for some reason, the refresh rate was very slow (on the CPU instead of the GPU?) and did not recognize objects well. We decided not to use it.
    • We integrated the Image Capture AI Kit for Meta Quest package by Roberto Coviello and also had problems with the ChatGPT API Key, but we finally managed to get it working correctly and integrate it organically into our application.

Accomplishments that we're proud of

  • Thanks to the excellent Image Capture AI Kit package we have managed to implement cutting-edge image recognition functionalities and integrate them organically to overcome an entry barrier to a system like the creation of spatialized cards.
  • We believe we have developed a very useful app for long-term concept retention and improved an existing concept in an unimaginable way to date.

What we learned

  • Analysis and implementation of the spatial repetition algorithm.
  • Integration of AI tools in a mixed reality application.
  • Combined use of the various capabilities of the Meta Presence Platform SDKs & Meta Building Blocks.

What's next for Kairosity

  • Creation of different decks.
  • Include a few settings to configure sessions and further personalize learning.
  • Insertion of two modes: classic (no time limit) and challenge (timed session).
  • Difficulty levels in the source according to the assigned score. The cards will become more difficult as the concept is learned.
  • Use of the MR Stylus from Logitech for card resolution through graphical inputs, such as kanjis.
  • Integration of improvements in the automatic object recognition software for even faster card creation.

Built With

Share this project:

Updates