-
-
-
Buttery Biscuit Base, the star of Wotsitbot
-
Our fantastic UML diagram
We wanted to make something hardware based, user-centric and fun! So we made a teleoperational robot that lets you collect 'things' in the real world, whilst simultaneously educating it's users!
We really tried to make a hardware project scalable, so did alot of things by servers (see our architecture diagram!), and also tried to solve a real world problem: kids having fun in a physio-digital world.
We had to make a video streaming standard for GMS2, as there is almost no support for loading images from a stream, let alone streaming.
Inspiration
We really wanted to build a robot this time, and thought that doing something where a use controls a robot and finds stuff would be fun!
What it does
Allows a user (presumably a young learner) to drive around a small robot, and click on things they can see in the world, allowing them to learn what they are, and a little more about them
How we built it
One of the heaviest projects we've ever undertaken!
We built a home-made differential drive robot, along with a camera and a tilt servo. The entire thing is powered by a pi3 (thank you Eva!!).
Then a game interface is produced using GameMaker Studio 2, which allows the user to control the robot, and see what it sees, which is all done through websockets and a UDP server. I never want to see another async in my life.
When a user selects an object in the world, we run a local Segment Anything model to mask the object, and then pass it into OpenAI to retrieve what the object is. Information and pictures about the object are then scraped from the internet, and redisplayed to the user!
Challenges we ran into
RACE CONDITIONS, turns out that concurrency is really annoying! Building the robot was quite tricky, and our robot (Buttery Biscuit Base) had to be rebuilt 4 damned times. Getting the camera to publish an image was like Sisyphus pushing the boulder
Accomplishments that we're proud of
What we learned
Building a robot from 0, and all the bs networking it requires to work
Log in or sign up for Devpost to join the conversation.