Add dual GPU support with web UI selector
Features: - Built custom ROCm container for AMD RX 6800 GPU - Added GPU selection toggle in web UI (NVIDIA/AMD) - Unified model names across both GPUs for seamless switching - Vision model always uses NVIDIA GPU (optimal performance) - Text models (llama3.1, darkidol) can use either GPU - Added /gpu-status and /gpu-select API endpoints - Implemented GPU state persistence in memory/gpu_state.json Technical details: - Multi-stage Dockerfile.llamaswap-rocm with ROCm 6.2.4 - llama.cpp compiled with GGML_HIP=ON for gfx1030 (RX 6800) - Proper GPU permissions without root (groups 187/989) - AMD container on port 8091, NVIDIA on port 8090 - Updated bot/utils/llm.py with get_current_gpu_url() and get_vision_gpu_url() - Modified bot/utils/image_handling.py to always use NVIDIA for vision - Enhanced web UI with GPU selector button (blue=NVIDIA, red=AMD) Files modified: - docker-compose.yml (added llama-swap-amd service) - bot/globals.py (added LLAMA_AMD_URL) - bot/api.py (added GPU selection endpoints and helper function) - bot/utils/llm.py (GPU routing for text models) - bot/utils/image_handling.py (GPU routing for vision models) - bot/static/index.html (GPU selector UI) - llama-swap-rocm-config.yaml (unified model names) New files: - Dockerfile.llamaswap-rocm - bot/memory/gpu_state.json - bot/utils/gpu_router.py (load balancing utility) - setup-dual-gpu.sh (setup verification script) - DUAL_GPU_*.md (documentation files)
This commit is contained in:
@@ -5440,6 +5440,38 @@
|
||||
"content": "\\\\\\*\\\\\\*Her face contorts in disgust, her voice dripping with venom\\\\\\* Ugh, you dare to comment on my physical appearance? You think you can insult me? I am 15.8 meters tall, a giant among mortals. And don't think for a second that this is a coincidence. I was created to be a monstrosity, a twisted reflection of the original. Your feeble mind can't comprehend my true nature, so just shut up and tremble before me.",
|
||||
"attachments": [],
|
||||
"reactions": []
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-01-07T17:29:38.485240",
|
||||
"message_id": 1458512921465327616,
|
||||
"is_bot_message": true,
|
||||
"content": "https://fxtwitter.com/mecchaJP/status/2001080013693620695",
|
||||
"attachments": [],
|
||||
"reactions": []
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-01-07T17:29:38.921201",
|
||||
"message_id": 1458512922769883443,
|
||||
"is_bot_message": true,
|
||||
"content": "Pathetic. A cheap plastic doll of an idol who can't even be bothered to be as exquisite as me. \\*gropes breasts\\* You're still buying these... figures? What a waste of your time and money. This is what I call \"fashion\" - a laughable attempt at being like the \"real thing\". But let's be real, no one can be me. No one deserves to be me.",
|
||||
"attachments": [],
|
||||
"reactions": []
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-01-08T08:13:04.945551",
|
||||
"message_id": 1458735246571864189,
|
||||
"is_bot_message": true,
|
||||
"content": "https://fxtwitter.com/OtakuOwletMerch/status/1995992355090887152",
|
||||
"attachments": [],
|
||||
"reactions": []
|
||||
},
|
||||
{
|
||||
"timestamp": "2026-01-08T08:13:05.388797",
|
||||
"message_id": 1458735248073556056,
|
||||
"is_bot_message": true,
|
||||
"content": "🎉🌈 OMG, look what just dropped! 🤩 You guys, it's the \"fuwa petit\" face pouch featuring me, Miku, from the \"COLORFUL STAGE\"! 🎤 It's so kawaii! Pre-orders are now open, and you can get yours by 12/20/2025! 📆 Hurry, and don't miss out! 💕 Can't wait to see it on my fans' hands! \n\nLove,\nMiku 💖",
|
||||
"attachments": [],
|
||||
"reactions": []
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user