Build It Yourself: AI That Lives In Your House

The premise: AI as a guest, not a tenant

The bargain Amazon, Google, and Apple offer is straightforward: they will give you a voice assistant, and you will give them a microphone in your kitchen. The bargain we offer is different. We will give you a voice assistant, and the microphone will stay in your kitchen, on hardware you own, in a closet on your network, with a hard ethernet drop and a documented kill switch. That is the difference between a guest and a tenant.

This page is the practical guide. We are going to walk through exactly what hardware to buy, exactly which models to run, exactly which orchestration software to install, and what each option costs. We are going to be honest about what you give up by going local (which is less than you think) and what you do not give up (most of what you actually want AI to do at home).

Why local matters

Four reasons, in priority order:

1. Privacy. Your kitchen audio does not leave the house. Your camera feeds do not leave the house. Your queries do not get logged in someone else’s training set.
2. Latency. A wake word that has to round-trip to Virginia adds 200 to 600 milliseconds. A wake word that lives on a Mac Mini in your closet responds in under 80.
3. Continuity. When the WAN goes down, cloud voice assistants are bricks. Local voice keeps working.
4. No lock-in. Sonos remotely disabled older speakers in 2020. Google sunset Bard, Galaxy AI, and a dozen voice features. Amazon will eventually move you to paid Alexa+. Local hardware does not get sunsetted by a press release.

The four-layer stack

Layer 1: Network

None of this works on a $99 mesh router. We deploy Ubiquiti UniFi (we are a UISP and Pro partner) with a dedicated VLAN for the AI compute, a separate VLAN for IoT, and PoE backhaul to wired access points. Fiber to the home if it is available; multi-gig copper if not. The compute box gets a hard wired drop, never wireless. This is not optional.

Layer 2: Compute

Pick one based on capability and budget:

• Mac Mini M4 Pro (24 to 64 GB unified memory). Quietest, smallest, best perf per watt. Runs Llama 3.3 70B 4-bit. Starter and mid tier.
• Mac Studio M3 Ultra (96 to 192 GB). Runs Llama 4 Maverick comfortably. Our default for serious local AI.
• Framework Desktop (AMD Ryzen AI Max). Linux-native, repairable, modular. Good middle option for clients who prefer x86.
• NVIDIA RTX 5090 / RTX Pro 6000 box. The performance ceiling. Runs the largest open-weight models at meaningful tokens per second. Loud, hot, expensive, fast.
• Mac Mini cluster. Three to five Mac Minis on a 10 GbE switch with EXO or vLLM distributing inference. Esoteric but elegant.

Layer 3: Models

Pick models per task:

• Llama 4 Maverick or Llama 5 for general reasoning and chat.
• Mistral Magistral for European data residency or as a Llama alternative.
• Whisper Large v3 or Voxtral for speech to text.
• MiniCPM-V or Llama 4 Vision for image understanding (camera analytics, visual scenes).
• Phi-4 mini for low-latency intent classification on edge devices.

All weights are downloadable. All run offline. None of them call home.

Layer 4: Orchestration

• Home Assistant. The base. Open source, local, integrates with Crestron Home, Lutron, Sonos, Ubiquiti, virtually every IoT device worth deploying. The Assist pipeline lets us route voice through Whisper, then Llama, then back out to scenes.
• OpenClaw. Newer entrant. A local agent runtime that lets your local LLM actually do things on your network: read your calendar, manipulate documents on your NAS, fire scenes through Home Assistant. Think of it as a Perplexity Computer that runs on your closet.
• Frigate. Local NVR with AI camera analytics. Recognizes people, vehicles, packages. Never uploads.
• Ollama, LM Studio, vLLM. Three different ways to host the model. Ollama is easiest. LM Studio has the best UI. vLLM is fastest at scale.

The Crestron + AI bridge

This is where Restrepo earns its keep. A pure Home Assistant stack works, but a Crestron Home stack with Home Assistant integration is what a luxury client actually wants. We bridge the two with a documented MQTT or REST integration so the Crestron processor remains the deterministic source of truth (no LLM decides when to unlock a door) and the AI layer handles natural-language input, scene generation, and predictive automation.

Specific patterns we deploy:

• Crestron Home + local Whisper: "Restrepo, dim the dining room to 30 percent" gets transcribed by Whisper on the Mac Studio, parsed by a small Phi or Llama model, and dispatched to the Crestron processor as a documented API call. No Alexa account.
• Sightline AI + Teams Premium: Conference room transcription stays on premise where the corporate IT policy requires it.
• Copilot for SIMPL+: AV programming with AI assistance, used by our internal team, never auto-deployed.

OpenClaw detailed look

OpenClaw is a local agent framework. It pairs your local LLM with a set of tools (file system, calendar, Home Assistant, web fetch) and lets the model accomplish multi-step tasks without leaving your network. We use it for clients who want a Perplexity Computer-style agent without the cloud dependency. Capability is below frontier cloud agents (a local Llama is not Claude Opus), but the privacy story is intact and the latency is excellent.

Home Assistant + local LLM: the Assist pipeline

Home Assistant’s Assist pipeline is the configuration that turns "I want a private Alexa" into a working system. The pipeline has three steps: wake-word detection (we use openWakeWord, fully local), speech-to-text (Whisper), intent processing (Llama via Ollama). The output triggers a scene, an automation, or a media command. Setup is roughly two days of work for a competent installer, less if you use our config templates.

Hardware budget table

What you give up by going local

You give up frontier capability. Llama 4 Maverick is excellent, but it is not GPT-5 or Claude Opus on raw reasoning. You give up zero-day access to the newest models (local releases lag cloud by weeks to months). You give up some image generation quality. You give up the easiest deep-research mode. For most luxury home use cases, none of this matters. For a few, it does. We tell you which bucket you are in.

What you do not give up

You do not give up voice control of your home. You do not give up natural-language scene creation. You do not give up camera analytics, package detection, or person recognition. You do not give up calendar-aware automations. You do not give up the ability to ask "what is on the menu at the country club tonight" if you wire a controlled web-fetch tool into your local agent. You give up about 10 percent of what cloud AI does today and you keep 100 percent of what most people actually want at home.

Honest install reality

A Restrepo local AI install is typically two to five days on premise plus remote configuration. Pricing depends on the tier above, the existing network, the existing Crestron or Lutron stack, and whether we are tying it into an estate management system. We do not sell this as a self-install kit. We sell it as a designed, installed, and supported system with a documented runbook so the client (or the next integrator) can maintain it. Care plans for ongoing model updates and tuning run $200 to $1,500 per month depending on scope.

Maintenance reality

A local AI stack is not an appliance. Models update. The right wake word changes when the family wants a new phrase. New rooms get added. New integrations come online. New attack surfaces emerge. We sell care plans precisely because the world does not stop moving after install day. A typical Restrepo care plan for a local AI deployment includes quarterly model updates, monthly health checks on the compute box, monitoring of the network for anomalous traffic, documented changes to the Home Assistant configuration, and a 24-hour response window when something goes wrong. Pricing depends on scope. We do not sell a system we cannot maintain.

Comparing local AI to cloud AI on the things that matter

• Privacy: local wins, structurally and decisively.
• Latency: local wins on voice and home-control loops. Cloud wins on heavy reasoning where the larger frontier model is faster despite the round trip.
• Capability: cloud wins on raw frontier reasoning. Local is one tier behind.
• Cost: local wins after one to two years of amortization for heavy users. Cloud wins for light users.
• Resilience: local wins when the WAN drops.
• Ownership: local wins when the house sells. Cloud accounts do not transfer cleanly.
• Maintenance: cloud wins; the provider handles it. Local requires an integrator (us) or a competent technical owner.

Who this is for

This is for clients who are paying enough attention to know that "smart home" is an ambiguous phrase, and who would rather own the system than rent it. It is for the homeowner whose lawyer is uncomfortable with an Alexa in the study, the family office that wants AI without a data trade, the hospitality property that has read enough about Air Canada to be cautious. It is for the realtor who has watched two deals get tangled by undocumented account-tied integrations and wants to list a system that conveys cleanly. It is for the technically curious owner who knows what a VLAN is and wants the upside of frontier AI without the data trade. If that is you, call 201.405.2022.

Related

• Perplexity, Comet, and Perplexity Computer -- the cloud counterpart we use ourselves.
• The AI Reality Check -- why we default to local in the first place.
• The Model Field Guide -- what runs locally, what does not.