SoLoMoRo

SoLoMo Robot virtual assistant for elderly people that provides emotional recognition and telepresence access

Comment 1

Inspiration

Social Local Mobile Robot (SoLoMoRo) Silicon Valley (HBO TV show); robots singing, robots helping sick people, 2001 a Space Odyssey, Virtual Reality, autonomous AI, IoT, and cloud messaging,

What it does

SoLoMo Robot is a personal assistant for the elderly that recognizes emotional states and automatically alerts the patient's care giver. This device is the first to combine Virtual Reality and Internet of Things to track the patient's position and provide an accurate location of the subject. After the robot triggers an alert, the care giver will be able to see the patient via telepresence and perform diagnostic triage.

How I built it

SoLoMoRo uses the APIs bellow:

  • PubNub Connecting HTC Vive positioning detector with our robot assistant for automatic patient's tracking via the PubNub IoT API.
  • Autonomous: Using the robot's API to automatically control the patient's movement and provide telepresence capability.
  • Nexmo: Sending notifications via SMS API to alert the patient's doctor.
  • Affectiva: Detecting the patient's emotions and analyze its emotional state.

Extra Technologies:

  • HTC Vive: Tracks accurate patient's position relative to the robot device.

Challenges I ran into

Getting the robot to start and stop by itself with automatic distance tracking and then sing a happy song after seeing a sad face was challenging Getting the robot to sync with the Vive motion sensors and match the VR world view on the headset

Accomplishments that I'm proud of

Combining APIs: Unity VR, Android SDK (phone and tablet), Autonomous AI Android robot, Nexmo, Pubnub remote device control for IOT, and cloud latency with position tracking in VR

What I learned

Many new APIs and integrations for IoT control, mood detection, and sms messaging integrated with VIVE, Android, and Autonomous AI

What's next for solomoro

Working to integrate more autonomous messaging, voice and position tracking into the application to help the caregiver to make an accurate diagnosis

Built With

Share this project:

Updates