Fix vision pipeline: route images through Cat, pass user question to vision model

- Fix silent None return in analyze_image_with_vision exception handler
- Add None/empty guards after vision analysis in bot.py (image, video, GIF, Tenor)
- Route all image/video/GIF responses through Cheshire Cat pipeline (was
  calling query_llama directly), enabling episodic memory storage for media
  interactions and correct Last Prompt display in Web UI
- Add media_type parameter to cat_adapter.query() and forward as
  discord_media_type in WebSocket payload
- Update discord_bridge plugin to read media_type from payload and inject
  MEDIA NOTE into system prefix in before_agent_starts hook
- Add _extract_vision_question() helper to strip Discord mentions and bot-name
  triggers from user message; pass cleaned question to vision model so specific
  questions (e.g. 'what is the person wearing?') go directly to the vision model
  instead of the generic 'Describe this image in detail.' fallback
- Pass user_prompt to all analyze_image_with_qwen / analyze_video_with_vision
  call sites in bot.py (image, video, GIF, Tenor, embed paths)
- Fix autonomous reaction loops skipping messages that @mention the bot or have
  media attachments in DMs, preventing duplicate vision model calls for images
  already being processed by the main message handler
- Increase vision max_tokens: images 300->800, video/GIF 400->1000 (no VRAM
  impact; KV cache is pre-allocated at model load time)
This commit is contained in:
2026-03-05 21:59:27 +02:00
parent ae1e0aa144
commit d5b9964ce7
5 changed files with 144 additions and 20 deletions

View File

@@ -814,6 +814,9 @@ async def miku_autonomous_reaction_for_server(guild_id: int, force_message=None,
# Skip bot's own messages
if message.author == globals.client.user:
continue
# Skip messages that directly addressed Miku (handled by main handler)
if globals.client.user and globals.client.user in message.mentions:
continue
# Skip messages we've already reacted to
if message.id in _reacted_message_ids:
continue
@@ -979,6 +982,11 @@ async def miku_autonomous_reaction_for_dm(user_id: int, force_message=None):
# Skip bot's own messages
if message.author == globals.client.user:
continue
# Skip messages with media attachments in DMs — these are always directed at
# Miku and already processed by the main on_message handler, so analyzing them
# again here would trigger a redundant vision model call
if message.attachments:
continue
# Skip messages we've already reacted to
if message.id in _reacted_message_ids:
continue