reorganize: consolidate all documentation into readmes/
- Moved 20 root-level markdown files to readmes/ - Includes COMMANDS.md, CONFIG_README.md, all UNO docs, all completion reports - Added new: MEMORY_EDITOR_FEATURE.md, MEMORY_EDITOR_ESCAPING_FIX.md, CONFIG_SOURCES_ANALYSIS.md, MCP_TOOL_CALLING_ANALYSIS.md, and others - Root directory is now clean of documentation clutter
This commit is contained in:
81
readmes/ROCM_UPDATE_FEB2026.md
Normal file
81
readmes/ROCM_UPDATE_FEB2026.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# ROCm Docker Image Update - February 2026
|
||||
|
||||
## Issue Summary
|
||||
The custom-built ROCm container for llama-swap failed to build due to changes in the llama-swap repository structure:
|
||||
- UI directory changed from `ui` to `ui-svelte`
|
||||
- Build output changed from `ui/dist` to `proxy/ui_dist`
|
||||
- Go build was failing with "pattern ui_dist: no matching files found"
|
||||
|
||||
## Resolution
|
||||
Switched to using **official llama.cpp ROCm Docker images** which are now available since PR #18439 was merged on December 29, 2025.
|
||||
|
||||
### What Changed
|
||||
|
||||
1. **Base Image**: Now using `ghcr.io/ggml-org/llama.cpp:server-rocm` instead of building llama.cpp from source
|
||||
2. **UI Build Path**: Updated to use `ui-svelte` directory (was `ui`)
|
||||
3. **Simpler Build**: Removed custom llama.cpp compilation stage - official image already includes optimized ROCm build
|
||||
|
||||
### Benefits of Official Image
|
||||
|
||||
- **Maintained by upstream**: Always up-to-date with latest llama.cpp ROCm optimizations
|
||||
- **Pre-compiled**: Faster build times, no need to compile llama.cpp from source
|
||||
- **Tested**: Official builds are tested by llama.cpp CI/CD
|
||||
- **Smaller Dockerfile**: Reduced from 4 stages to 3 stages
|
||||
|
||||
## Updated Dockerfile Structure
|
||||
|
||||
### Stage 1: UI Builder (Node.js)
|
||||
- Clones llama-swap repository
|
||||
- Builds UI from `ui-svelte` directory
|
||||
- Outputs to `proxy/ui_dist`
|
||||
|
||||
### Stage 2: Binary Builder (Go)
|
||||
- Copies llama-swap source with built UI
|
||||
- Compiles Go binary with `GOTOOLCHAIN=auto`
|
||||
|
||||
### Stage 3: Runtime (Official ROCm Image)
|
||||
- Based on `ghcr.io/ggml-org/llama.cpp:server-rocm`
|
||||
- Includes llama-server with ROCm support
|
||||
- Adds llama-swap binary
|
||||
- Creates non-root user with GPU access groups
|
||||
- Sets environment variables for AMD RX 6800 (gfx1030)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
HSA_OVERRIDE_GFX_VERSION=10.3.0 # RX 6800 compatibility
|
||||
ROCM_PATH=/opt/rocm
|
||||
HIP_VISIBLE_DEVICES=0
|
||||
```
|
||||
|
||||
## GPU Access Groups
|
||||
|
||||
- GID 187: `hostrender` group (render access)
|
||||
- GID 989: `hostvideo` group (kfd/video access)
|
||||
|
||||
## References
|
||||
|
||||
- **Upstream Issue**: https://github.com/ggml-org/llama.cpp/issues/11913
|
||||
- **Fix PR**: https://github.com/ggml-org/llama.cpp/pull/18439
|
||||
- **llama-swap Repo**: https://github.com/mostlygeek/llama-swap
|
||||
- **Official ROCm Images**: `ghcr.io/ggml-org/llama.cpp:server-rocm`
|
||||
|
||||
## Testing
|
||||
|
||||
After building, verify GPU detection:
|
||||
```bash
|
||||
docker compose up llama-swap-amd -d
|
||||
docker compose exec llama-swap-amd rocminfo
|
||||
```
|
||||
|
||||
Expected output should show AMD RX 6800 (gfx1030) device.
|
||||
|
||||
## Migration Notes
|
||||
|
||||
If you had the old custom-built container running:
|
||||
1. Stop the old container: `docker compose down llama-swap-amd`
|
||||
2. Remove old image: `docker rmi miku-discord-llama-swap-amd`
|
||||
3. Rebuild with new Dockerfile: `docker compose build llama-swap-amd`
|
||||
4. Start new container: `docker compose up llama-swap-amd -d`
|
||||
|
||||
Configuration files (`llama-swap-rocm-config.yaml`) remain unchanged.
|
||||
Reference in New Issue
Block a user