Files

koko210Serve c708770266 reorganize: consolidate all documentation into readmes/

- Moved 20 root-level markdown files to readmes/
- Includes COMMANDS.md, CONFIG_README.md, all UNO docs, all completion reports
- Added new: MEMORY_EDITOR_FEATURE.md, MEMORY_EDITOR_ESCAPING_FIX.md,
  CONFIG_SOURCES_ANALYSIS.md, MCP_TOOL_CALLING_ANALYSIS.md, and others
- Root directory is now clean of documentation clutter

2026-03-04 00:19:49 +02:00

2.8 KiB

Raw Permalink Blame History

ROCm Docker Image Update - February 2026

Issue Summary

The custom-built ROCm container for llama-swap failed to build due to changes in the llama-swap repository structure:

UI directory changed from ui to ui-svelte
Build output changed from ui/dist to proxy/ui_dist
Go build was failing with "pattern ui_dist: no matching files found"

Resolution

Switched to using official llama.cpp ROCm Docker images which are now available since PR #18439 was merged on December 29, 2025.

What Changed

Base Image: Now using ghcr.io/ggml-org/llama.cpp:server-rocm instead of building llama.cpp from source
UI Build Path: Updated to use ui-svelte directory (was ui)
Simpler Build: Removed custom llama.cpp compilation stage - official image already includes optimized ROCm build

Benefits of Official Image

Maintained by upstream: Always up-to-date with latest llama.cpp ROCm optimizations
Pre-compiled: Faster build times, no need to compile llama.cpp from source
Tested: Official builds are tested by llama.cpp CI/CD
Smaller Dockerfile: Reduced from 4 stages to 3 stages

Updated Dockerfile Structure

Stage 1: UI Builder (Node.js)

Clones llama-swap repository
Builds UI from ui-svelte directory
Outputs to proxy/ui_dist

Stage 2: Binary Builder (Go)

Copies llama-swap source with built UI
Compiles Go binary with GOTOOLCHAIN=auto

Stage 3: Runtime (Official ROCm Image)

Based on ghcr.io/ggml-org/llama.cpp:server-rocm
Includes llama-server with ROCm support
Adds llama-swap binary
Creates non-root user with GPU access groups
Sets environment variables for AMD RX 6800 (gfx1030)

Environment Variables

HSA_OVERRIDE_GFX_VERSION=10.3.0  # RX 6800 compatibility
ROCM_PATH=/opt/rocm
HIP_VISIBLE_DEVICES=0

GPU Access Groups

GID 187: hostrender group (render access)
GID 989: hostvideo group (kfd/video access)

References

Upstream Issue: https://github.com/ggml-org/llama.cpp/issues/11913
Fix PR: https://github.com/ggml-org/llama.cpp/pull/18439
llama-swap Repo: https://github.com/mostlygeek/llama-swap
Official ROCm Images: ghcr.io/ggml-org/llama.cpp:server-rocm

Testing

After building, verify GPU detection:

docker compose up llama-swap-amd -d
docker compose exec llama-swap-amd rocminfo

Expected output should show AMD RX 6800 (gfx1030) device.

Migration Notes

If you had the old custom-built container running:

Stop the old container: docker compose down llama-swap-amd
Remove old image: docker rmi miku-discord-llama-swap-amd
Rebuild with new Dockerfile: docker compose build llama-swap-amd
Start new container: docker compose up llama-swap-amd -d

Configuration files (llama-swap-rocm-config.yaml) remain unchanged.

2.8 KiB Raw Permalink Blame History