The Machine
That Remembers
An assistant that runs entirely on a graphics card in the next room — with a memory of past conversations, a face on the desktop, and a voice that answers when called.
With the household server running, its graphics card spent most of the day doing nothing. The obvious use was a language model — but a chatbot in a browser tab is not an assistant. An assistant should be present, should remember, and should answer when spoken to.
Evo is the result: a language model running on local hardware, wrapped in a custom interface that docks to the edge of the screen, backed by a memory service that recalls earlier conversations by meaning, and reachable by voice from anywhere on the desktop with a single keystroke.
No request leaves the house. No subscription is paid. The trade is capability for privacy: a seven-billion-parameter model is a capable junior rather than an oracle, but it is a junior that knows my hardware, my configuration and what I asked it last week.
The Assistant, Reproduced
A faithful copy of the interface that docks to the right of my desktop. The original talks to a language model on my own hardware; this one answers from a short script.
How the Memory Works
A language model's weights do not change as you talk to it; on consumer hardware, retraining is not realistic. What produces the sensation of memory is retrieval.
Each statement worth keeping is converted into an embedding — a numerical fingerprint of its meaning — and stored. Before every reply, the incoming question is embedded the same way and compared against the store, and the closest matches are folded into the model's instructions. It arrives already informed. The service was written for this project in Python, backed by SQLite, and runs alongside the model.
The subtlety is what to store. Storing entire exchanges seems obvious and is a trap; the account of that failure is below.
From Keystroke to Answer
What Runs, and Why
| Component | Purpose & Reasoning |
|---|---|
| Ollama | Serves a seven-billion-parameter coding model on an RTX 3060 Ti. Chosen at that size deliberately: it fits in eight gigabytes of video memory alongside the speech models and still responds quickly. |
| Memory service | Written for this project. A small Python API storing embeddings in SQLite, retrieving by cosine similarity. Shared by the sidebar, the voice assistant and the login greeting. |
| Custom interface | A single-file application in plain HTML, CSS and JavaScript — streaming replies, conversation history, model switching, and a visible indicator of which memories informed each answer. |
| Whisper | Speech to text, primed with a vocabulary list so it recognises names and technical terms it would otherwise mangle. |
| Kokoro | Text to speech. Replaced an earlier, more robotic engine; markdown and code are stripped before synthesis so nothing reads syntax aloud. |
| Local proxy | Written to make every service same-origin, eliminating an entire class of browser security failure in one step. Also serves the interface. |
Three Problems Worth Reporting
I. The Assistant That Learned Its Own Mistakes
Once memory was working, a question about my desktop environment returned generic advice — run this command to find out — despite the correct answer sitting in the database. The retrieval log explained it. The three highest-scoring matches were the assistant's own earlier wrong answers to the same question, each outranking the fact itself, because near-identical text matches near-identical text.
It had been faithfully learning its own errors and reinforcing them with every repetition. The fix was to stop storing the model's replies and remember only what the user says: statements of fact are worth keeping, a model's guesses are not. The answers corrected themselves immediately.
II. Silence With No Explanation
Retrieval scored every result below the acceptance threshold, returning nothing at all. The cause was a documented requirement I had overlooked: the embedding model expects stored passages and search queries to be marked differently before encoding.
Adding those two prefixes moved the same query, against the same stored fact, from below the threshold to comfortably above it. Nothing about the architecture was wrong; one detail of a dependency's interface had been skipped.
III. The Window That Would Not Sit Still
The interface refused to dock, ignoring every window rule aimed at it. The rules were sound; the target was wrong. Under Wayland the browser ignores the flag that sets a window's class and derives one from the page address instead — so the rules had been matching a window that never existed.
Asking the compositor to list its live windows produced the real name in seconds. The failure was one of diagnosis rather than implementation, which makes it the more useful of the two to have made.
The Machine Underneath
On Building Something Smaller Than the Alternative
Evo will not out-argue a model running in a datacentre, and pretending otherwise would be dishonest. What it offers instead is ownership: it knows my systems because it has been given the context to know them, it costs nothing per query, and nothing I say to it leaves the building.
The engineering interest was never in calling a model — that part is a single request. It was in everything around it: what to remember and what to discard, how to make retrieval actually surface the right thing, how to move speech through four services without stuttering, and how to make it all survive a reboot without attention.
Every part was written, broken, diagnosed and repaired by hand. The failures above are included deliberately; they are the part of the work that taught me something.