Software for an Audience of One

For most of the history of software, an application existed because enough people wanted it to pay for the time it took to build. A tool that only one person needed usually didn’t get written, unless that person happened to be a programmer with a lot of spare evenings. Everyone else adapted to whatever the market offered.

That has changed. With modern LLM coding tools, a person with a clear picture of the application they want can build it, even if they will be its only user. I think of it as custom software on demand: not a product with a market, but a tool made for one person’s way of working. I’ve been building software this way for most of this year, and I think it is one of the more important things these tools make possible.

Running Claude Code with Local LLMs via vLLM and LiteLLM

GPU, &, Ai

Every query to Claude Code means sending my source code to Anthropic’s servers. For proprietary codebases, that’s a non-starter. With vLLM and LiteLLM, I can point Claude Code at my own hardware - keeping my code on my network while maintaining the same workflow.

The Architecture#

The trick is that Claude Code expects the Anthropic Messages API, but local inference servers speak OpenAI’s API format. LiteLLM bridges this gap. It accepts Anthropic-formatted requests and translates them to OpenAI format for my local vLLM instance.