Usage-based pricing killing your vibe - here's how to roll your own local AI coding agents

Take those token limits and shove them by vibe coding with a local LLM With model devs pushing more aggressive rate limits, raising prices, or even abandoning subscriptions for usage-based pricing, that vibe-coded hobby project is about to get a whole lot more expensive.

Fortunately, you're not without cost-saving options. It just so happens that Alibaba recently dropped Qwen3. 6-27B , which the cloud and e-commerce giant boasts packs "flagship coding power" into a package small enough to run on a 32 GB M-series Mac or 24 GB GPU.

This isn't the first time we've looked at local code assistants. Previously we explored using Continue's VS Code extension for tasks such as code completion and generation.

At the time, the models and software stack were quite immature, making them useful tools, but not necessarily good enough to compete with larger frontier models. Since then, model architectures and agent harnesses have improved dramatically.

"Reasoning" capabilities allow small models to make up for their size by "thinking" for longer, mixture-of-experts models mean you don’t need terabytes a second of memory bandwidth for an interactive experience, and vastly improved function and tool calling capabilities mean that these models can actually interact with code bases, shell environments, and the web.

In this hands on, we'll be looking at how to deploy and configure local models like Qwen3. 6-27B, for coding on your computer, and explore some of the agent frameworks you can use with them. Note: Older M-series Macs may struggle with the large context lengths required for agentic coding.

You may have better luck with an inference engine like oMLX , which can take better advantage of Apple's hardware accelerators, but your mileage may vary. Running LLMs locally is a dead simple process these days. Install your favorite inference engine. Download the model, and connect your app via the API.

However, for code assistants in particular, there are a couple of parameters we need to dial in, otherwise the model is apt to churn out garbage and broken code. Some models require specific hyper-parameters to function properly in different applications, and Qwen3. 6-27B is no exception. When using Qwen3.

6-27B for vibe coding, Alibaba recommends setting the following parameters: We also need to set the model's context window as large as we can fit in memory. If you're not familiar, a model's context window defines how many tokens the model can keep track of for any given request.

When working with large code bases containing thousands of lines of code, this adds up quickly. What's more, the system prompts used by many agent frameworks can be quite large, so we want to set our context window as high as possible. Qwen3.

6-27B supports a 262,144 token context window, but unless you have a high-end Mac or a workstation GPU, you probably don't have enough memory to take advantage of all of that, at least not at 16-bit precision. The good news is that we don't need to store the key-value caches, which track the model state, at 16-bits.

We can get away with lower precisions without too much performance and quality degradation. To maximize our context window, we'll be compressing the key value pairs to 8-bits. Finally, we'll want to make sure prefix caching is turned on.

For workloads where large sections of the prompt are going to be reprocessed over and over again, like a system prompt or code base, this will speed up inference by ensuring only new tokens are processed. In newer builds of Llama. cpp this should be enabled by default, but we'll call those flags just in case.

With all that out of the way, here's the launch command we're using for a 24GB Nvidia RTX 3090 TI, but the same code command should work just fine if you're using an AMD or Intel GPU or are running Llama. cpp on a Mac.

If you're running this on a machine with more memory, try bumping up the context window to 131,072 or 262,144. If you're planning on running Llama. cpp and accessing it on another machine, you'll also want to add --host 0. 0. 0. 0 to the command, which will expose it to your local area network. If Llama.

cpp is running in a VPC, you'll want to configure your firewall rules before passing this flag for the sake of security. Now that our model is up and running, we need to connect it to an agentic coding harness.

On their own, models can generate code, but they have no way to implement, test, or debug it without an active development environment. Part of what has helped vibe coding take off where other AI ventures have struggled, is that code is verifiable. It either runs or compiles, or it doesn't.

To keep things simple we'll be looking at three popular options: Claude Code, Pi Coding Agent, and Cline. We'll kick things off with Claude Code. Despite what you might think, you don't have to use Claude Code with Anthropic's models.

The framework works just fine with local models, assuming you've got enough resources to run them. Install Claude Code as you normally would. You can find Anthropic's one-liner here .

Next, we'll need to tell Claude Code we want to use the model running locally on our machine rather than a Claude account or Anthropic's API services. This is done by setting a few shell variables before launching Claude Code. These will need to be run each time you launch Claude from a new session.

Now when you start Claude, it'll connect directly to your local model. Claude Code itself continues to function as it normally would. Let's say you not only want to use your own local models, but would prefer an open source harness as well. If you like Claude Code, you'll probably like the Pi Coding Agent .

And just like Claude Code, it's not picky about what model you use with it. One of the main attractions of Pi Coding Agent is how lightweight it is. Long input sequences can be extremely taxing on lower end or older GPUs or accelerators.

Claude Code and Cline both have system prompts that can bring less capable hardware to a crawl. By comparison, Pi Coding Agent's default system prompt is short enough to keep things snappy, especially with prompt-caching enabled.

However, that speed comes at the expense of many of the guardrails and safety features we see on other coding agents. This is one you'll probably want to spin up in a virtual machine, container, or even a Raspberry Pi.

Much like Claude, the Pi Coding Agent can be installed using the appropriate one liner for your system. After that, all that's required is a little bit of JSON telling the agent harness where to find your model. If you've been following along, the setup is fairly simple.

Using your preferred text editor, create the following file: Next, paste in the following template. If you've set an API key, replace no_API_key_required with your key. The rest of these will depend on what model and port you're using. You'll also want to adjust the contextWindowSize to match what you set in Llama.

cpp. With that out of the way, we can navigate to our working directory, launch Pi Coding Agent, and get to work vibe coding our next hobby project.

Claude Code integrates directly with popular integrated development environments (IDEs) like VS Code, but if you're going this route, we also recommend checking out another open source app called Cline.

Installing Cline is as simple as finding it in VS Code's — or a supported IDE's — extension manager and adding it to your library. Next, we'll point Cline at our Llama.

cpp server and adjust a few hyperparameters like temperature and context size: Once it is configured, you can interact with Cline through its chat interface. Any files or edits will appear in VS Code as they're generated.

One of Cline's more useful features is the ability to switch between a pure planning mode and an action mode. If you've ever gotten frustrated because Claude interpreted a question as a call to action when what you really want to do is workshop a problem, this is a huge help. So can Qwe