https://cruver.ai/tags/project/ Recent content in Project on Hullabalooing Hugo en Mon, 02 Mar 2026 00:00:00 -0700 https://cruver.ai/gpu-ai/ Mon, 02 Mar 2026 00:00:00 -0700 https://cruver.ai/gpu-ai/ <p>Running local AI inference on AMD Instinct MI60 GPUs - a budget-friendly path to 70B parameter models at home.</p> <h2 id="overview">Overview<a class="anchor" href="#overview">#</a></h2> <p>The AMD Instinct MI60 is a datacenter GPU from 2018 that remains surprisingly capable for local AI workloads. With 32GB of HBM2 memory per card, a dual-MI60 setup provides 64GB of VRAM, enough to run 70B parameter models with quantization.</p> <p>This section documents my setup: hardware modifications for cooling, software configuration for ROCm and vLLM, and the workflows that make it practical for daily use.</p>