vLLM Inference on MI60

Calendar January 21, 2026
GPU/AI
Gpu, Vllm, Inference, Llm, Mi60, Rocm

Production inference using vLLM with tensor parallelism across dual MI60 GPUs.

Why vLLM?#

After evaluating Ollama, llama.cpp, and vLLM for MI60 inference, vLLM emerged as the best choice:

  1. Tensor Parallelism: Native support for splitting large models across multiple GPUs. With dual MI60s (64GB total), I can run 70B parameter models that wouldn’t fit on a single GPU.

  2. PagedAttention: More efficient memory management, allowing higher GPU utilization (90%) without OOM errors.