Gregor

• 

  Greggy with some flowers :)

• 

  electronic components in Gregor

• 

  Gregor identifying plants

• 

  Gregor put together

Inspiration

Coming into the hackathon, we already knew we wanted to do a hardware hack. We brought Gregor with us, and all the wirings we would need to give him image recognition and speech functionality. When the BlackRock team announced their challenge in partnership with The Green Team, reading more about them across their website made us realise we could make the perfect product to help them meet their goals!

The Green Team is all about getting young people out into nature and discovering the world around them, to help create and promote a more environmentally stable society. We decided to turn Gregor into a tool The Green Team could utilise on their many programmes for young people, to make them even more exciting.

Tailored towards younger children (e.g. on the Skylarks and Green Explorers programmes), Gregor is a great way for students to learn and experience the world firsthand. His environmental know-how and endearing character are sure to win over children’s hearts, and make him an integral part of their outdoor adventures!

What it does

Gregor is cute, customisable, friendly and a specialist in all things environment - your all-in-one adventure buddy! He loves being taken around and exploring the outdoors alongside you. Point him at a plant and give him a thumbs-up, and he will tell you exactly what plant you’re looking at along with some facts!

Gregor loves documenting his adventures, so he keeps track of all the plants you find together. He’ll celebrate with you when you hit discovery milestones, and even let you know if you’ve come across a rare find! If you happen to stumble across a plant you’ve already found before, Gregor will ask you a fun trivia question about the environment to test your knowledge. Give him a thumbs-up to answer true, or a thumbs-down for false.

Exploring the outdoors is a wonderful way to not only learn more about the world we live in, but also to boost your own mental health and morale. Gregor allows you to do all this with your own little companion - make sure to wave to him to say hi!

How we built it

The components inside Gregor include a Raspberry Pi, speaker, and camera to detect what it sees. We arranged the camera to be placed in the middle of his eyes, to give him the most natural vision, and the speaker to come out underneath him. By packing all the electronic components within him, we have been able to create a very compact, all-in-one robot.

The computer vision aspects involve two main features: hand movement detection, and image recognition. We have implemented these using various computer vision libraries, including mediapipe and pla@ntnet.

As all code is hosted on the Raspberry Pi within Gregor, he is able to perform actions completely independently of an external computer. As long as he has internet and is connected to a power source, he’s good to go!

Challenges we ran into

The computer vision was slow and unreliable at times, but we improved this by making optimisations to our recognition algorithms. Aside from this, our project was relatively smooth-sailing. We managed to get all the various image recognition components working well with one another, and got it hosted on the Raspberry Pi quite simply. All the additional features of Gregor (e.g. milestones, quizzes etc.) are developed to be stored locally, so this data was not difficult to transfer across devices.

Accomplishments that we're proud of

We’re very proud of creating a compact, portable and fully working outdoor companion, that is more than capable of supporting The Green Team in its goals to educate and promote environmental sustainability to the younger generation. We believe that Gregor would make a great addition to The Green Team’s existing programmes, becoming a well-loved character of teams and keeping children even more engaged.

We are also proud to complete this all in the space of the 24 hours, taking Gregor from an ordinary rubber duck to the environmental extraordinaire he is today! We think Gregor is a solid prototype that could be taken so much further with further development.

What's next for Gregor

With added time and interest from The Green Team, we would love to make Gregor completely portable and easy to take around - currently, Gregor is connected to an external power source, but finding some smaller form of battery (and other, more streamlined hardware components) would make him far more simple to take on adventures!

We also have many ideas to improve on Gregor’s functionality, more than we can implement in 24 hours! One feature we’d love to see in Gregor is some sort of class/team leaderboard, where children on The Green Team programmes can compete with their friends to find the most plants. This would bring a healthy sense of competition to their outdoor expeditions, making them even more eager to get outside and discover new things!

Built With

• computer-vision
• hardware