🖥️ DeskHUD

A full-screen, always-on heads-up display dashboard designed for desktop monitors or Raspberry Pi screens. Black background, frosted-glass widgets, edge glow effects, and touchless hand-gesture interaction. Built at an Entrepreneur First hackathon in London.

The system is a three-tier architecture: a Next.js 15 / React 19 frontend locked to 1920x1080 with no scrolling, and two Python WebSocket backends for real-time sensor processing.

Hand gesture control: A USB camera feeds into MediaPipe for hand landmark detection. Pinch your thumb and index finger together over a widget to "grab" it, move your hand to drag it, and release to drop. A One-Euro filter adapts smoothing to movement speed, and a calibrated homography maps camera coordinates to screen space. Sticky grab with grace periods prevents flicker from camera glitches.

Directional audio: A 4-channel USB mic array captures audio at 48kHz. GCC-PHAT cross-correlation computes time-difference-of-arrival across all 6 mic pairs, a trained linear regressor maps those to a 2D direction vector, and the frontend renders a glowing arc along the screen edge nearest to the sound source. Optional CLAP-based sound classification labels what it hears (doorbell, speech, glass breaking, etc.).

All widgets are draggable with 50px grid snapping and AABB collision detection so they can't overlap. Widgets include a digital clock, countdown timer, date display, uptime counter, A3 document viewer, and A4 paper scanner. Both audio direction and hand pose use per-setup calibration rather than assuming fixed hardware geometry, making the system hardware-agnostic.

Built with Tomas. Still adding widgets and features whenever we get the chance.