Major architectural overhaul of the speech-to-text pipeline for real-time voice chat:
STT Server Rewrite:
- Replaced RealtimeSTT dependency with direct Silero VAD + Faster-Whisper integration
- Achieved sub-second latency by eliminating unnecessary abstractions
- Uses small.en Whisper model for fast transcription (~850ms)
Speculative Transcription (NEW):
- Start transcribing at 150ms silence (speculative) while still listening
- If speech continues, discard speculative result and keep buffering
- If 400ms silence confirmed, use pre-computed speculative result immediately
- Reduces latency by ~250-850ms for typical utterances with clear pauses
VAD Implementation:
- Silero VAD with ONNX (CPU-efficient) for 32ms chunk processing
- Direct speech boundary detection without RealtimeSTT overhead
- Configurable thresholds for silence detection (400ms final, 150ms speculative)
Architecture:
- Single Whisper model loaded once, shared across sessions
- VAD runs on every 512-sample chunk for immediate speech detection
- Background transcription worker thread for non-blocking processing
- Greedy decoding (beam_size=1) for maximum speed
Performance:
- Previous: 400ms silence wait + ~850ms transcription = ~1.25s total latency
- Current: 400ms silence wait + 0ms (speculative ready) = ~400ms (best case)
- Single model reduces VRAM usage, prevents OOM on GTX 1660
Container Manager Updates:
- Updated health check logic to work with new response format
- Changed from checking 'warmed_up' flag to just 'status: ready'
- Improved terminology from 'warmup' to 'models loading'
Files Changed:
- stt-realtime/stt_server.py: Complete rewrite with Silero VAD + speculative transcription
- stt-realtime/requirements.txt: Removed RealtimeSTT, using torch.hub for Silero VAD
- bot/utils/container_manager.py: Updated health check for new STT response format
- bot/api.py: Updated docstring to reflect new architecture
- backups/: Archived old RealtimeSTT-based implementation
This addresses low latency requirements while maintaining accuracy with configurable
speech detection thresholds.
- Removed local 'import json' statement inside get_servers() function
- This was shadowing the module-level import and causing
'cannot access local variable' error
- json is already imported at the top of the file (line 44)
- Created new logging infrastructure with per-component filtering
- Added 6 log levels: DEBUG, INFO, API, WARNING, ERROR, CRITICAL
- Implemented non-hierarchical level control (any combination can be enabled)
- Migrated 917 print() statements across 31 files to structured logging
- Created web UI (system.html) for runtime configuration with dark theme
- Added global level controls to enable/disable levels across all components
- Added timestamp format control (off/time/date/datetime options)
- Implemented log rotation (10MB per file, 5 backups)
- Added API endpoints for dynamic log configuration
- Configured HTTP request logging with filtering via api.requests component
- Intercepted APScheduler logs with proper formatting
- Fixed persistence paths to use /app/memory for Docker volume compatibility
- Fixed checkbox display bug in web UI (enabled_levels now properly shown)
- Changed System Settings button to open in same tab instead of new window
Components: bot, api, api.requests, autonomous, persona, vision, llm,
conversation, mood, dm, scheduled, gpu, media, server, commands,
sentiment, core, apscheduler
All settings persist across container restarts via JSON config.
Features:
- Built custom ROCm container for AMD RX 6800 GPU
- Added GPU selection toggle in web UI (NVIDIA/AMD)
- Unified model names across both GPUs for seamless switching
- Vision model always uses NVIDIA GPU (optimal performance)
- Text models (llama3.1, darkidol) can use either GPU
- Added /gpu-status and /gpu-select API endpoints
- Implemented GPU state persistence in memory/gpu_state.json
Technical details:
- Multi-stage Dockerfile.llamaswap-rocm with ROCm 6.2.4
- llama.cpp compiled with GGML_HIP=ON for gfx1030 (RX 6800)
- Proper GPU permissions without root (groups 187/989)
- AMD container on port 8091, NVIDIA on port 8090
- Updated bot/utils/llm.py with get_current_gpu_url() and get_vision_gpu_url()
- Modified bot/utils/image_handling.py to always use NVIDIA for vision
- Enhanced web UI with GPU selector button (blue=NVIDIA, red=AMD)
Files modified:
- docker-compose.yml (added llama-swap-amd service)
- bot/globals.py (added LLAMA_AMD_URL)
- bot/api.py (added GPU selection endpoints and helper function)
- bot/utils/llm.py (GPU routing for text models)
- bot/utils/image_handling.py (GPU routing for vision models)
- bot/static/index.html (GPU selector UI)
- llama-swap-rocm-config.yaml (unified model names)
New files:
- Dockerfile.llamaswap-rocm
- bot/memory/gpu_state.json
- bot/utils/gpu_router.py (load balancing utility)
- setup-dual-gpu.sh (setup verification script)
- DUAL_GPU_*.md (documentation files)
ISSUE
=====
When using the manual webhook message feature via API, the following error occurred:
- 'Timeout context manager should be used inside a task'
- 'NoneType' object is not iterable (when sending without files)
The error happened because Discord.py's webhook operations were being awaited
directly in the FastAPI endpoint context, rather than within a task running in
the bot's event loop.
SOLUTION
========
Refactored /manual/send-webhook endpoint to properly handle async operations:
1. Moved webhook creation inside task function
- get_or_create_webhooks_for_channel() now runs in send_webhook_message()
- All Discord operations (webhook selection, sending) happen inside the task
- Follows same pattern as working /manual/send endpoint
2. Fixed file parameter handling
- Changed from 'files=discord_files if discord_files else None'
- To conditional: only pass files parameter when list is non-empty
- Discord.py's webhook.send() cannot iterate over None, requires list or omit
3. Maintained proper file reading
- File content still read in endpoint context (before form closes)
- File data passed to task as pre-read byte arrays
- Prevents form closure issues
TECHNICAL DETAILS
=================
- Discord.py HTTP operations use timeout context managers
- Context managers must run inside bot's event loop (via create_task)
- FastAPI endpoint context is separate from bot's event loop
- Solution: Wrap all Discord API calls in async task function
- Pattern: Read files → Create task → Task handles Discord operations
TESTING
=======
- Manual webhook sending now works without timeout errors
- Both personas (Miku/Evil) send correctly
- File attachments work properly
- Messages without files send correctly
Users can now send manual messages as either Hatsune Miku or Evil Miku
via webhooks without needing to toggle Evil Mode. This provides more
flexibility for controlling which persona sends messages.
Features:
- Checkbox option to "Send as Webhook" in manual message section
- Radio buttons to select between Hatsune Miku and Evil Miku
- Both personas use their respective profile pictures and mood emojis
- Webhooks only available for channel messages (not DMs)
- DM option automatically disabled when webhook mode is enabled
- New API endpoint: POST /manual/send-webhook
Frontend Changes:
- Added webhook checkbox and persona selection UI
- toggleWebhookOptions() function to show/hide persona options
- Updated sendManualMessage() to handle webhook mode
- Automatic channel selection when webhook is enabled
Backend Changes:
- New /manual/send-webhook endpoint in api.py
- Integrates with bipolar_mode.py webhook management
- Uses get_or_create_webhooks_for_channel() for webhook creation
- Applies correct display name with mood emoji based on persona
- Supports file attachments via webhook
This allows manual control over which Miku persona sends messages,
useful for testing, demonstrations, or creative scenarios without
needing to switch the entire bot mode.
Major Features:
- Complete Bipolar Mode system allowing Regular Miku and Evil Miku to coexist and argue via webhooks
- LLM arbiter system using neutral model to judge argument winners with detailed reasoning
- Persistent scoreboard tracking wins, percentages, and last 50 results with timestamps and reasoning
- Automatic mode switching based on argument winner
- Webhook management per channel with profile pictures and display names
- Progressive probability system for dynamic argument lengths (starts at 10%, increases 5% per exchange, min 4 exchanges)
- Draw handling with penalty system (-5% end chance, continues argument)
- Integration with autonomous system for random argument triggers
Argument System:
- MIN_EXCHANGES = 4, progressive end chance starting at 10%
- Enhanced prompts for both personas (strategic, short, punchy responses 1-3 sentences)
- Evil Miku triumphant victory messages with gloating and satisfaction
- Regular Miku assertive defense (not passive, shows backbone)
- Message-based argument starting (can respond to specific messages via ID)
- Conversation history tracking per argument with special user_id
- Full context queries (personality, lore, lyrics, last 8 messages)
LLM Arbiter:
- Decisive prompt emphasizing picking winners (draws should be rare)
- Improved parsing with first-line exact matching and fallback counting
- Debug logging for decision transparency
- Arbiter reasoning stored in scoreboard history for review
- Uses neutral TEXT_MODEL (not evil) for unbiased judgment
Web UI & API:
- Bipolar mode toggle button (only visible when evil mode is on)
- Channel ID + Message ID input fields for argument triggering
- Scoreboard display with win percentages and recent history
- Manual argument trigger endpoint with string-based IDs
- GET /bipolar-mode/scoreboard endpoint for stats retrieval
- Real-time active arguments tracking (refreshes every 5 seconds)
Prompt Optimizations:
- All argument prompts limited to 1-3 sentences for impact
- Evil Miku system prompt with variable response length guidelines
- Removed walls of text, emphasizing brevity and precision
- "Sometimes the cruelest response is the shortest one"
Evil Miku Updates:
- Added height to lore (15.8m tall, 10x bigger than regular Miku)
- Height added to prompt facts for size-based belittling
- More strategic and calculating personality in arguments
Integration:
- Bipolar mode state restoration on bot startup
- Bot skips processing messages during active arguments
- Autonomous system checks for bipolar triggers after actions
- Import fixes (apply_evil_mode_changes/revert_evil_mode_changes)
Technical Details:
- State persistence via JSON (bipolar_mode_state.json, bipolar_webhooks.json, bipolar_scoreboard.json)
- Webhook caching per guild with fallback creation
- Event loop management with asyncio.create_task
- Rate limiting and argument conflict prevention
- Globals integration (BIPOLAR_MODE, BIPOLAR_WEBHOOKS, BIPOLAR_ARGUMENT_IN_PROGRESS, MOOD_EMOJIS)
Files Changed:
- bot/bot.py: Added bipolar mode restoration and argument-in-progress checks
- bot/globals.py: Added bipolar mode state variables and mood emoji mappings
- bot/utils/bipolar_mode.py: Complete 1106-line implementation
- bot/utils/autonomous.py: Added bipolar argument trigger checks
- bot/utils/evil_mode.py: Updated system prompt, added height info to lore/prompt
- bot/api.py: Added bipolar mode endpoints (trigger, toggle, scoreboard)
- bot/static/index.html: Added bipolar controls section with scoreboard
- bot/memory/: Various DM conversation updates
- bot/evil_miku_lore.txt: Added height description
- bot/evil_miku_prompt.txt: Added height to facts, updated personality guidelines
- Added Evil Miku mode with 4 evil moods (aggressive, cunning, sarcastic, evil_neutral)
- Created evil mode content files (evil_miku_lore.txt, evil_miku_prompt.txt, evil_miku_lyrics.txt)
- Implemented persistent evil mode state across restarts (saves to memory/evil_mode_state.json)
- Fixed API endpoints to use client.loop.create_task() to prevent timeout errors
- Added evil mode toggle in web UI with red theme styling
- Modified mood rotation to handle evil mode
- Configured DarkIdol uncensored model for evil mode text generation
- Reduced system prompt redundancy by removing duplicate content
- Added markdown escape for single asterisks (actions) while preserving bold formatting
- Evil mode now persists username, pfp, and nicknames across restarts without re-applying changes
- Replace simple 'Engage Random User' button with expandable submenu
- Add user ID input field for targeting specific users
- Add engagement type selection: random, activity-based, general, status-based
- Update API endpoints to accept user_id and engagement_type parameters
- Modify autonomous functions to support targeted engagement
- Maintain backward compatibility with random user/type selection as default
- Added optional reply message ID field to web UI
- Added radio buttons to control mention/ping behavior in replies
- Updated frontend JavaScript to send reply parameters
- Modified /manual/send and /dm/{user_id}/manual endpoints to support replies
- Fixed async context by moving message fetching inside bot event loop task
- Supports both channel and DM reply functionality
- Fixed function name mismatch: generateImage() -> generateManualImage()
- Fixed status div ID mismatch in HTML
- Added /image/view/{filename} endpoint to serve generated images from ComfyUI output
- Implemented proper image preview with DOM element creation instead of innerHTML
- Added robust error handling with onload/onerror event handlers
- Added debug logging to image serving endpoint for troubleshooting
- Images now display directly in the Web UI after generation
Features:
- Real-time streaming chat interface (ChatGPT-like experience)
- Model selection: Text model (fast) or Vision model (image analysis)
- System prompt toggle: Chat with Miku's personality or raw LLM
- Mood selector: Choose from 14 different emotional states
- Full context integration: Uses complete miku_lore.txt, miku_prompt.txt, and miku_lyrics.txt
- Conversation memory: Maintains chat history throughout session
- Image upload support for vision model
- Horizontal scrolling tabs for responsive design
- Clear chat history functionality
- SSE (Server-Sent Events) for streaming responses
- Keyboard shortcuts (Ctrl+Enter to send)
Technical changes:
- Added POST /chat/stream endpoint in api.py with streaming support
- Updated ChatMessage model with mood, conversation_history, and image_data
- Integrated context_manager for proper Miku personality context
- Added Chat with LLM tab to index.html
- Implemented JavaScript streaming client with EventSource-like handling
- Added CSS for chat messages, typing indicators, and animations
- Made tab navigation horizontally scrollable for narrow viewports
- Added /autonomous/join-conversation API endpoint in api.py
- Added 'Detect and Join Conversation' button to Web UI under Autonomous Actions
- Added 'autonomous join-conversation' command to CLI tool (miku-cli.py)
- Updated miku_detect_and_join_conversation_for_server to support force=True parameter
- When force=True: skips time limit, activity checks, and random chance
- Force mode uses last 10 user messages regardless of age
- Manual triggers via Web UI/CLI now work even with old messages
