Most “AI agent” setups involve sending your data to a corporation and paying by the subscription/token. This one doesn’t. Everything below runs on a second-hand 64GB M1 Mac Studio sitting in my house.
Why I Actually Did This
There are three reasons I built it, and they matter in this order.
To learn. Running a model through an API teaches you the API (chat windows don’t really count). Running one locally teaches you how it actually works, model installation, the tradeoffs between architectures, what MCP servers are really doing, where the latency comes from, how to build your own vector database so the knowledge base belongs to you rather than a vendor. Second-hand because waste is terrible and new machines are damn expensive.
To keep my data at home. Every document I’d want an agent to read is one I’d rather not upload. A local stack means I can plan bigger integrations over time without the question “but where does the data go?” needing a new answer each time.
Cost, eventually. The initial outlay is real. But LLM pricing is heading in a predictable direction in my opinion: list/subscription prices rise while the free tier and usage gets quietly rationed, because the big labs have to recover what they’ve spent. A rig you already own doesn’t care what OpenAI charges next quarter.
Design Principles
Two rules. Every decision traces back to one of them.
Local and secure by default. Inference happens on hardware I own, on data that never leaves the house. For anything genuinely heavy, I’m considering MCP protocols that can selectively offload to a cloud model (something that I may address in the future). The key word is selectively. The default is home.
Always on. It needs to be available and reasonably responsive. I expect the model to be reasonably performant but most importantly on and available. Both macOS sleep states and Ollama’s default memory behaviour work against this, and both needed fixing.
The Foundation
The M1 Mac Studio is worth naming naming because after much consideration, it struck the right balance of performance and cost. You don’t need the latest silicon to run the Gemma 4 31B dense model. You need the memory. Back Market had exactly the right spec at roughly half the current retail price.
Ollama runs natively on macOS, headless. No container between the model and the hardware. Direct access to Apple Silicon’s unified memory and GPU cores, which is the entire reason to run this on a Mac.
Gemma 4 31B Dense is the model. I bypassed the faster Mixture-of-Experts (26B A4B) options in favour of a dense architecture. 31B documentation states deep reasoning, reliable logic chains, and solid tool use. For an agent that has to decide what to do not just generate fluent text. I also care about the right decision over lots of speed.
OrbStack is the container runtime. Lighter than Docker Desktop, faster to start, much kinder to a Mac’s fans. OpenClaw runs inside it as the agent gateway and orchestrator, connecting out to Ollama on the host via host.docker.internal. The containerised agent logic uses the native Mac hardware without Ollama itself being containerised.
Tailscale handles secure remote access. The rig is reachable from my phone without punching a hole in the home network.
Telegram is the primary interface and easy to set up.
Knowledge Base: The Read-Only Contract
An agent without context is a chatbot. The context here is my Obsidian vault.
The vault lives inside a Google Drive shared folder, which macOS CloudStorage syncs locally. That local sync folder is then bind-mounted into the OpenClaw container.
This has already earned its keep on something as mundane as task management. The agent reads my vault, understands what I’m actually tracking, and helps me shape the list rather than starting from scratch every time. The value isn’t intelligence in the abstract. It’s intelligence applied to my notes. When I asked if it could help with any tasks it genuinely surprised me and delivered a half decent writeup.
The Two Bottlenecks That Nearly Killed It
Getting the model to run was the easy part. Getting it to respond instantly when pinged from Telegram took two weekends, because macOS aggressively prioritises energy efficiency over standby readiness.
Ollama’s Memory Drop
Ollama unloads models after five minutes of inactivity to free up RAM. On a 31B dense model, that’s a cold start on every query roughly 15 seconds reloading 34GB of weights before the agent can even begin thinking.
The fix is one line:
launchctl setenv OLLAMA_KEEP_ALIVE "-1"Set once, survives reboots, done. RAM usage goes up permanently, latency drops permanently. Right trade when you have 64GB to spend.
The macOS Sleep Throttle
Even with the model locked in memory, locking the Mac Studio’s screen triggers a deep sleep state. The CPU and GPU cores spin down to draw under 3W. When a webhook arrives, the processor wakes from a dead stop and latency spikes.
The fix is to decouple screen lock from logic-board sleep. macOS Energy settings (or Amphetamine, for more granular control) let the display power off while the performance cores stay warm. The Mac stays locked and secure. It just doesn’t go to sleep.
Together, these two changes turn a clunky experiment into something that responds in under two seconds, reliably, from a locked screen. That gap 15 seconds to under 2 is the difference between “interesting prototype” and “tool I actually use.”
The Stack at a Glance
| Layer | Component | Role |
|---|---|---|
| Hardware | Mac Studio M1 (64GB, second-hand) | Inference host |
| Model | Gemma 4 31B Dense | Reasoning engine |
| Runtime | Ollama (native macOS) | Model serving |
| Container | OrbStack | Lightweight virtualisation |
| Orchestrator | OpenClaw | Agent + MCP gateway |
| Context | Obsidian vault via Google Drive (:ro mount) | Read-only knowledge base |
| Remote access | Tailscale | Secure tunnel |
| Interface | Telegram | Daily-use entry point |
Final Thoughts
This is all about learning but I do believe in building tools that actually work for you. Will this be as powerful as cloud-based agents in a year? No. But, it will be as I add more tools, more context, and push for efficiency. It also means I’m willing to experiment more without token fear.
The hardware seems pretty comfortable and can handle 31B. What separates a clunky experiment from something you actually use is the unglamorous last 10%. The Obsidian vault & local GDrive folder share which allows OpenClaw secure access to personal data. OpenClaw and Gemma 4 31B’s reliability and reasoning has actually been impressive so far.
Next steps include CrewAI for multi-agent orchestration and a self-built Qdrant vector database for proper semantic search over the knowledge base. The point of building it rather than using a hosted service is the same as the point of everything else here: to understand what’s actually happening, not to outsource the understanding.