Rocm on Hullabalooing

I Pushed Local LLMs Harder. Here's What Two Models Actually Did.

Mon, 02 Mar 2026 00:00:00 -0700

In Part 1 of this series, I set up Claude Code against local LLMs on dual MI60 GPUs and watched it scaffold a Flask application from scratch. Small tasks worked. Complex ones did not. I ended with three ideas I wanted to test: running a dense model, trying Claude Code’s agent teams feature, and building a persistent memory layer for coding sessions.

I started the experiment that mattered most: giving a local LLM a project with real scope and seeing what happened. I ran the same project against two different models. The results were instructive, and not in the direction I expected.

ComfyUI on MI60

Wed, 21 Jan 2026 00:00:00 -0700

Node-based Stable Diffusion UI with AMD MI60 GPU acceleration via ROCm.

Overview#

ComfyUI runs as a containerized service using a custom ROCm 5.7 image built for the MI60’s gfx906 architecture. It can run standalone on both GPUs or alongside vLLM in the image-chat configuration.

Quick Start#

# Build the container image
cd comfyui
make build

# Start ComfyUI (standalone, both GPUs)
make up

# Or use the image-chat config (vLLM on GPU0, ComfyUI on GPU1)
curl -X POST -H "Content-Type: application/json" \
  -d '{"config":"image-chat"}' http://localhost:9100/switch

Access the UI at http://localhost:8188.

vLLM Inference on MI60

Wed, 21 Jan 2026 00:00:00 -0700

Production inference using vLLM with tensor parallelism across dual MI60 GPUs.

Why vLLM?#

After evaluating Ollama, llama.cpp, and vLLM for MI60 inference, vLLM emerged as the best choice:

Tensor Parallelism: Native support for splitting large models across multiple GPUs. With dual MI60s (64GB total), I can run 70B parameter models that wouldn’t fit on a single GPU.
PagedAttention: More efficient memory management, allowing higher GPU utilization (90%) without OOM errors.