koko210Serve
1fc3d74a5b
Features: - Built custom ROCm container for AMD RX 6800 GPU - Added GPU selection toggle in web UI (NVIDIA/AMD) - Unified model names across both GPUs for seamless switching - Vision model always uses NVIDIA GPU (optimal performance) - Text models (llama3.1, darkidol) can use either GPU - Added /gpu-status and /gpu-select API endpoints - Implemented GPU state persistence in memory/gpu_state.json Technical details: - Multi-stage Dockerfile.llamaswap-rocm with ROCm 6.2.4 - llama.cpp compiled with GGML_HIP=ON for gfx1030 (RX 6800) - Proper GPU permissions without root (groups 187/989) - AMD container on port 8091, NVIDIA on port 8090 - Updated bot/utils/llm.py with get_current_gpu_url() and get_vision_gpu_url() - Modified bot/utils/image_handling.py to always use NVIDIA for vision - Enhanced web UI with GPU selector button (blue=NVIDIA, red=AMD) Files modified: - docker-compose.yml (added llama-swap-amd service) - bot/globals.py (added LLAMA_AMD_URL) - bot/api.py (added GPU selection endpoints and helper function) - bot/utils/llm.py (GPU routing for text models) - bot/utils/image_handling.py (GPU routing for vision models) - bot/static/index.html (GPU selector UI) - llama-swap-rocm-config.yaml (unified model names) New files: - Dockerfile.llamaswap-rocm - bot/memory/gpu_state.json - bot/utils/gpu_router.py (load balancing utility) - setup-dual-gpu.sh (setup verification script) - DUAL_GPU_*.md (documentation files)
41 lines
924 B
Docker
41 lines
924 B
Docker
FROM python:3.11-slim
|
|
|
|
WORKDIR /app
|
|
|
|
COPY requirements.txt .
|
|
RUN pip install -r requirements.txt
|
|
RUN playwright install
|
|
|
|
# Install system dependencies
|
|
# ffmpeg: video/audio processing for media handling
|
|
# libgl1: OpenGL library required by opencv-contrib-python
|
|
# libglib2.0-0: GLib library (common dependency)
|
|
RUN apt-get update && apt-get install -y \
|
|
ffmpeg \
|
|
libgl1 \
|
|
libglib2.0-0 \
|
|
&& apt-get clean \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY bot.py .
|
|
COPY server_manager.py .
|
|
COPY command_router.py .
|
|
COPY utils /app/utils
|
|
COPY commands /app/commands
|
|
COPY memory /app/memory
|
|
COPY static /app/static
|
|
COPY globals.py .
|
|
COPY api.py .
|
|
COPY api_main.py .
|
|
COPY miku_lore.txt .
|
|
COPY miku_prompt.txt .
|
|
COPY miku_lyrics.txt .
|
|
COPY evil_miku_lore.txt .
|
|
COPY evil_miku_prompt.txt .
|
|
COPY evil_miku_lyrics.txt .
|
|
COPY MikuMikuBeam.mp4 .
|
|
COPY Miku_BasicWorkflow.json .
|
|
COPY moods /app/moods/
|
|
|
|
CMD ["python", "-u", "bot.py"]
|