Short Course: Building AI Assistants with On-Device Memory in partnership with Qdrant
Qdrant Edge Memory Robot

The Qdrant Edge Memory Robot sees an object, learns its name from your voice, and remembers where and when it saw it. Detection, speech recognition, embeddings, and vector search all run on the device, no cloud service, API key, or large language model required.
This project is the physical continuation of the DeepLearning.AI short course Building AI Assistants with On-Device Memory, which builds the memory layer in the notebooks. This repository connects that layer to a camera, microphone, object detector, and browser interface.
Start Here
You do not need to print the enclosure or own a Jetson to explore the code.
What the Robot Does
- Hold TEACH and say, "This is my chair." The robot stores the object, its name, and the current place and time.
- Show the object again. The robot recognizes it and saves a sighting.
- Open MEMORY to review, rename, or forget objects and individual views.
- Hold ASK and ask, "When did you last see my chair?"

Recognize objects
|

Recall sightings
|

Browse memories
|

Manage memories
|
Quick Start
You need Windows, macOS, or Linux, Python 3.12 or newer, uv, a webcam, and a microphone. Windows support targets 64-bit Windows on x86 processors.
uv sync
cp .env.example .env
uv run python -m robot.app
On Windows PowerShell, replace the copy command with:
Copy-Item .env.example .env
Open http://127.0.0.1:8765. The first run downloads about 1.5 GB of model files and takes longer than later starts.
For the first interaction, hold an object near the center of the camera view. When its box becomes steady, hold TEACH, say "This is my object," and release the button. See Get Started for the full walkthrough and common setup issues.
How It Works
camera -> detect and track -> crop -> CLIP image vector
|
voice -> Whisper -> name -> Nomic text vector
|
Qdrant Edge shard
|
recognize and recall
YOLOE finds and tracks objects, but the application discards its class names. The memories you teach determine an object's name. Qdrant Edge stores two named vectors on each taught view: a CLIP image vector for recognition and a Nomic text vector for spoken recall.
Repository Map
robot/
brain/ course concepts: detection, embeddings, memory, and recall
device/ camera, microphone, browser server, and interface
app.py command-line entry point and image replay
config.py settings loaded from .env
docs/ setup, build, architecture, and calibration guides
deploy/ optional headless Jetson service and Wi-Fi setup
hardware/ printable enclosure files
testdata/ sample images and the calibration check
tests/ unit tests for platform-specific helpers
Start reading at robot/brain/core.py. The Robot.process_frame method connects detection, embedding, recognition, and sighting storage. The architecture guide provides a guided reading order.
Runtime memories are stored in edge-data/, which Git ignores. Use --reset to start with an empty memory.
License
The code and documentation are licensed under Apache 2.0.
The calibration photographs in testdata/ come from Wikimedia Commons under CC BY-SA 4.0. testdata/CREDITS.json names each photographer and links the original file.