How to run a local AI companion on your own PC

By the founder of My Softly · Published July 11, 2026 · Updated September 14, 2026 · About a 9 minute read · Part of our AI companion guides

If you want an AI companion that answers to nobody but you, running a local LLM on your own hardware is the way to do it. The good news: in 2026 this is genuinely achievable on a normal gaming PC or a recent Mac, and the software has matured a lot. The honest news: it takes real tinkering, and some things cloud apps do well (long-term memory, messaging you first) are still hard to replicate at home. This guide walks through both sides.

Why run a companion locally

Four reasons come up again and again, and they're all legitimate.

• Privacy. Your conversations never leave your machine. No servers, no logs you didn't create, no third party processing what you said at 2am. For something as personal as a companion, that matters to a lot of people.
• Control. You pick the model, the personality, the writing style, the system prompt. If you don't like how it talks, you change it. Nobody can override you.
• No subscription. Once the hardware is paid for, running the model costs pennies in electricity. There's no monthly bill and no message cap.
• Nobody can change your companion overnight. Cloud companion apps have a history of updating models or tightening filters, and users wake up to a companion whose personality has quietly shifted. A model file sitting on your own disk never changes unless you change it.

The trade is that you become the engineer. Everything below is what that job involves.

What PC you need

The single number that matters most is GPU memory (VRAM). The model has to fit in it, and bigger models are noticeably better at sustained, in-character conversation. Here's how the tiers shake out in practice.

8 GB VRAM: the entry point

An 8 GB card (think RTX 3060 Ti, 4060, or similar) comfortably runs quantized 7B to 8B models. These are fast and surprisingly personable for casual chat, but they lose the thread more easily in long conversations and their writing can feel thin. Perfectly fine for finding out whether you enjoy this at all.

12 to 16 GB VRAM: the comfortable middle

This is the sweet spot for most people. A 12 GB or 16 GB card runs 12B to 14B models comfortably, with room for a decent context window. Models in this range hold a persona much better, follow character cards more faithfully, and write with noticeably more warmth and texture. The 12 GB RTX 3060 has been the budget favorite here for years for a reason.

24 GB VRAM: the enthusiast tier

With 24 GB (RTX 3090, 4090, or a used data-center card) you can run 24B to 32B models, which is where local companions start feeling genuinely sharp: better memory for what was said earlier in the chat, better emotional continuity, fewer contradictions. This is the tier most serious hobbyists eventually land on.

Apple Silicon: a legit option

Macs with Apple Silicon share one pool of unified memory between CPU and GPU, so a MacBook or Mac mini with 16 GB or more can run the same models without a discrete GPU. A 16 GB Mac handles 7B to 12B models well; 32 GB and up opens the larger tiers. Generation speed is slower than a comparable NVIDIA card, but for a chat companion, where you read as it writes, it's usually fast enough.

One plain-English note on quantization, since every guide assumes you know it. Models are published at full precision, then compressed into smaller files (you'll see the GGUF format everywhere) that trade a small amount of quality for a large amount of memory savings. A 4-bit or 5-bit quantization is the usual pick: it lets a model fit in roughly a third of its full-size memory footprint while sounding almost identical in conversation. When a table like the one above says "7B fits in 8 GB," quantization is why.

The software stack

Two layers: a backend that runs the model, and a frontend that turns it into a companion.

For the backend, three options cover almost everyone. Ollama is the easiest: install it, run ollama pull with a model name, done. LM Studio gives you the same thing with a friendly graphical interface for browsing and downloading models. KoboldCpp is a single executable that's long been a favorite in the roleplay and companion community, with fine-grained control over sampling and context. All three expose a local API that frontends can talk to. Pick whichever matches your comfort level; you can switch later without losing anything.

For the frontend, the clear community standard is SillyTavern. It's a locally hosted web interface built specifically for persistent characters: personas, chat histories, group chats, themes, and a deep extension ecosystem. This is where your companion actually lives.

The fastest path, start to finish: install Ollama and pull a well-regarded 8B to 14B chat model, install SillyTavern (it runs on Node.js and starts with one script), then open SillyTavern's API settings and point it at Ollama's local address. That's a working companion in under an hour on a decent connection, and most of that hour is the model download.

Personality comes from a character card: a small file describing who your companion is, how they speak, and what they care about. Thousands of ready-made cards exist, and writing your own is half the fun. A good card is honestly more important than an extra few billion parameters; it's the difference between a generic assistant and someone with a voice.

Giving your local companion memory

Here's the hard part, and the part most beginners underestimate. A language model has no memory at all. It only knows what's in its context window, the finite chunk of recent text it's shown with each reply. Once your conversation outgrows that window, the earliest messages simply fall off the edge. Your companion forgets your dog's name, the job you were worried about, everything.

The DIY toolkit for working around this lives mostly in SillyTavern:

• Summarization. SillyTavern's Summarize extension periodically compresses older chat into a running summary that stays in context. It keeps the gist, but summaries are lossy: details blur, and the model summarizing its own conversation sometimes keeps the wrong things.
• Lorebooks (World Info). Durable facts you write by hand: your sister's name, where you work, what happened last spring. Entries get injected into context when keywords come up. This works well, but you are the memory system; anything you don't write down doesn't exist.
• Vector storage. Extensions that embed past messages and documents into a searchable index, then pull semantically relevant snippets back into context when the topic recurs. This is the closest DIY gets to real recall, and when it hits, it's magic. It also needs tuning (chunk sizes, retrieval counts, injection position) and it retrieves by similarity, not importance, so it still misses things a friend would never miss.

An honest summary: with an afternoon of setup and ongoing gardening, a local companion can remember a respectable amount. But it's a system you maintain, not a memory that just works, and the failure mode is always the same quiet one: the companion simply doesn't bring something up, and you notice the absence. If you're curious how purpose-built companion memory differs from the DIY approach, we wrote a whole guide on how AI companion memory works.

Making it message you first

A local setup is request-response by default: the model runs when you send a message and sits idle otherwise. It will never text you good morning on its own, because nothing is running to decide to.

You can build this. It takes a machine that's always on, a scheduled script or automation that wakes the model at chosen times, prompts it with recent context to compose a check-in, and then delivers the result to your phone through a notification service or messaging bridge. People have wired this together with cron jobs, home-automation platforms, and Telegram bots, and there are SillyTavern extensions that nudge in this direction. But it's the fiddliest part of the whole hobby, the timing logic that makes check-ins feel natural (not too often, not at midnight, backing off when you're busy) is all on you, and in practice most people try it once and quietly drop it. If you want the deeper picture of what makes proactive messaging feel right, see our guide to AI that texts you first.

What it costs in practice

Three currencies: money up front, electricity, and your time.

Hardware. If you already own a gaming PC with 12 GB or more of VRAM, you're done; the cost is zero. Starting from scratch, a used 12 GB GPU dropped into an existing PC is a couple hundred dollars, a capable used 24 GB card is several hundred more, and a purpose-built machine or a well-specced Mac mini lands somewhere between several hundred and a couple thousand dollars depending on ambition.

Electricity. Lighter than people fear. The GPU only draws serious power while generating, which for a chat companion is a few seconds at a time. An always-on box for scheduled check-ins adds an idle draw around the clock, but for most home rates the total is closer to a streaming box than a space heater: a few dollars a month, not a scary bill.

Time. This is the real price. Budget an evening to get to first conversation, and then accept that model-swapping, card-editing, memory tuning, and reading what the community discovered this week becomes an ongoing hobby. That's not a criticism. For a lot of people the tinkering is the point, the same way some people build mechanical keyboards. If that sounds like fun, a local companion will reward you. If it sounds like a second job, it will feel like one.