
GuideSight
A harness sensor and phone app that warn guide-dog walkers about obstacles ahead, built at HackCamp 2026
Team
Donna Li
Jake Fung
Sam
My role
Hardware
Backend
Frontend
Tools
ESP32
HC-SR04
Flask
React
Gemini Vision
ElevenLabs
Duration
Hackathon · HackCamp 2026
Overview
GuideSight is an assistive walking companion for people who use guide dogs. A sensor on the harness and a phone app work together to give obstacle alerts and spoken guidance, combining hardware sensing with AI scene analysis.
Problem
A guide dog steers around obstacles, but it can't tell its handler what's ahead, how far, or how urgent it is. Curbs, drop-offs, and oncoming traffic all get the same silence.
No distance warning
A dog stops or swerves, but the handler gets no sense of how close an obstacle is until they're on top of it.
No hazard severity
A curb, a staircase, and a moving car all call for different responses, and none of that distinction reaches the handler.
No hands-free alert
A phone alone can't be checked mid-walk, and a cane-style buzzer alone can't describe a scene.
Hardware
An ESP32 and an ultrasonic sensor ride on the harness.
An HC-SR04 ultrasonic sensor measures distance to whatever is ahead, and an ESP32 microcontroller reads it and sends it over Wi-Fi to the backend. A buzzer and an optional vibration motor sit on the harness handle, so the beep itself tells the handler how close something is.
- Distance thresholds trigger progressive warnings: the beep speeds up as the obstacle gets closer.
- The ESP32 streams readings to the Flask backend over Wi-Fi rather than processing hazards on-device.
Software
A Flask backend with WebSocket support (flask-sock) takes the sensor stream and the phone's camera frames, backed by Firestore. The walker-facing app is React, Vite, and Tailwind. Google Maps' Places, Routes, and Geocoding APIs handle navigation alongside the obstacle alerts.
Gemini Vision
The phone's camera becomes a second sensor.
Every 15 seconds, or every 6 seconds once the harness senses something nearby, the phone sends a camera frame to Gemini Vision. Its scene description feeds a hazard classifier on the server, which sorts what it sees into trip hazards, drop-offs, or traffic danger.
Obstacle warning system
The hazard tier decides how hard the system interrupts the walk:
- Trip hazards, like curbs or potholes, get a spoken warning without stopping navigation.
- Drop-offs and stairs always get an urgent response, with a stop screen inside one meter.
- Dangers like red lights or moving traffic trigger an immediate stop screen regardless of distance.
Next steps
The harness sensor, Gemini Vision pipeline, and tiered alerts work end to end for the hackathon demo. Next would be testing the distance thresholds and hazard tiers with an actual guide-dog handler, and moving hazard classification closer to the harness to cut latency.
Thank you
To Jake and Sam, built together at HackCamp 2026.
Behind the scenes
Wiring the sensor into the harness pocket, testing the walking mode and the accessibility settings on the phone, the app running next to its code, and the three of us heads-down at HackCamp 2026.

