Dynamic switching between GPU configurations based on workload needs.
Overview#
Rather than running a single fixed configuration, I dynamically switch between GPU configurations. A Python service manages the transitions via HTTP API.
Available Configurations#
| Config | Model | Use Case |
|---|---|---|
big-chat |
Llama 3.3 70B (TP=2) | Maximum quality chat |
coder |
Qwen3 32B (TP=2) | Coding tasks with thinking mode |
dual-chat |
Qwen3 8B + 14B | Two models, different tasks |
image-chat |
Qwen 7B + ComfyUI | Text + image generation |
Configuration files are stored as compose YAML files.