The microwave wants to leave the house
Building AI systems with privacy in-mind
This is a start of a series on building a product on top of AI. For a primer on some of my approaches, I suggest you start with the posts covering AI memory and AI pre-game routines. Also there is more on memory and retrieval. But you can also just start here.
Talking about privacy is free. Building it costs you decisions you can’t take back later, which is why so few products do it and why the ones that wait end up with something that only looks private.
I learned how expensive those decisions are from one bug. A while back I sent my personal chat agent, Microwave, a health question. It made the right decision. It recognized the question as medical and routed it through the secure channel, the restricted path where nothing identifying information is transmitted from the device. Subsequently, I responded with, “It’s not me; I do have a picture, though,” and the entire process unraveled.
The router evaluated each message independently. Initially, the message appeared to be medical, while the second one seemed like a man sending a picture. The router had no idea that they were the same conversation, so the privacy decision made in the first message silently expired in the second. Asking an AI to manage privacy is like trusting a friend who excels at keeping secrets for a limited time, such as up to three drinks.
I delved deeper into that thread and stumbled upon a product. After a few months of using Microwave, I discovered which feature I used the most. Among its various capabilities, reading my files, monitoring my cameras, and running errands, the one I utilized the most was health. This wasn’t because the health feature was superior. Instead, I comprehended the precise level of privacy it offered, and that understanding influenced my willingness to request it.
That’s the product. For a tool that directly impacts my daily life, “take it on faith” is insufficient. It turns out that my behavior changes the moment faith ceases to be a prerequisite.
The Sealed Room and the Importance of Private AI
Nearly every health app on the market operates on a similar principle. You provide your data, which is then sent to their servers and becomes their property. This isn’t malicious behavior; it’s simply the business model. Your data is the asset, and you’re essentially renting access to your own life. If you stop paying, your account will be terminated, and your records will remain with the company. Even with a privacy policy there is a lot of opportunities for bad behavior with your data.
So for this product, I wanted a different approach. I wanted everything that could remain on the device to do so. The user should be in-control of the data. When a question genuinely requires more than the local phone data can handle, the system carefully steps outside and utilizes the most secure path available to handle sensitive information.
Additionally, by have the app log every routing decision, it allows me to audit its actions. This is because after the sticky-routing bug incident, I stopped accepting the excuse of “it probably did the right thing” from my own software. As I wrote previously, AI models are made to lie to you, so to assume they will be honest about the routing is foolish.
That bug incident established a design principle for everything that followed: when the convenient path and the safe path diverge, prioritize the safe path. Then, make the safe path feel like the convenient one. Users should never be forced to choose between privacy and a good experience, as they’ll inevitably lose that battle. I understand this, as I was the user who initiated the follow-up.
The challenge lies in the moment when a question surpasses the capabilities of a phone. Certain questions require a powerful model in the cloud, which is precisely when your private information is traditionally logged by someone’s server.
The analogy I used earlier remains relevant. Imagine sliding your question through a slot into a soundproof room where a machine responds to it. No one, including the building’s owner, can access the room or listen in. This is a trusted execution environment. What sets it apart from a mere metaphor is attestation: the hardware cryptographically verifies that the room was sealed before you inserted anything. “Trust us, we won’t look” transforms into “we can’t look, and here’s the proof.”
My sealed room operates on NEAR AI. I didn’t select it solely based on a comparison chart. I spent months thoroughly testing their agent, IronClaw. By the time I needed a confidential compute layer for a real product, I had invested enough time in this one to trust it with my own health questions. That’s my benchmark. I’m a user zero.
There are two primary functionalities. Everyday questions are routed through NEAR Cloud, where identifying information is removed before reaching a large provider. The upstream provider receives a de-identified question, not “Joe from account X asked this.” Sensitive questions are processed through open-weight models running inside NEAR’s TEEs, which is essentially a soundproof room where the host running the hardware couldn’t peek if they wanted to.
Here’s the part that I find genuinely amusing. Large health platforms can’t replicate this, and the reason has nothing to do with engineering talent. Their revenue depends on seeing your data. You can’t sell privacy as a feature when visibility is the business. Any of them could build a sealed room tomorrow, but none of them can afford the financial implications.
A product built from the outset with confidential compute never encounters this conflict because it was never designed to see anything. Privacy isn’t a policy I’m committed to upholding; it’s an inherent property of the architecture, and the architecture will demonstrate this through the math. Policies can change when companies are acquired, but properties remain constant.
Another point I frequently return to is that the person this protects doesn’t necessarily need to be aware of it. Most users will never check an attestation or read a routing log. They’ll simply notice, as I did with Microwave, that there’s one tool they keep reaching for with the questions they wouldn’t type anywhere else. They won’t be able to explain why. The architecture will know.