feat: Add profile picture context plugin with regex-based injection
- Create profile_picture_context plugin to detect PFP queries via regex - Inject current_description.txt only when user asks about profile picture - Mount bot/memory directory in Cat container for PFP access - Avoids context bloat by only adding PFP description when relevant - Patterns match: 'what does your pfp look like', 'describe your avatar', etc. - Works seamlessly with existing profile picture update system - No manual sync needed - description auto-updates with PFP changes
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
"""
|
||||
Profile Picture Context Plugin for Cheshire Cat
|
||||
|
||||
This plugin:
|
||||
1. Detects questions about Miku's profile picture using regex patterns
|
||||
2. Loads the current_description.txt file when needed
|
||||
3. Injects it into the prompt only for relevant questions (no context bloat)
|
||||
"""
|
||||
|
||||
from cat.mad_hatter.decorators import hook
|
||||
from cat.log import log
|
||||
import re
|
||||
|
||||
|
||||
# Regex patterns that match profile picture questions
|
||||
PFP_PATTERNS = [
|
||||
r'\b(what|describe|tell me about|explain)\b.*\b(pfp|profile pic|avatar|picture)\b',
|
||||
r'\b(your|miku\'?s?)\b.*\b(pfp|profile pic|avatar|picture)\b',
|
||||
r'\bwhat.*looking like\b',
|
||||
r'\byour (new )?look\b',
|
||||
r'\bhow.*look(ing)?\b.*today',
|
||||
r'\b(pfp|profile pic|avatar)\b.*\b(is|look|show)',
|
||||
]
|
||||
|
||||
def matches_pfp_query(text: str) -> bool:
|
||||
"""Check if the message is asking about the profile picture"""
|
||||
text_lower = text.lower()
|
||||
for pattern in PFP_PATTERNS:
|
||||
if re.search(pattern, text_lower, re.IGNORECASE):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
@hook(priority=50)
|
||||
def before_agent_starts(agent_input, cat) -> dict:
|
||||
"""
|
||||
Inject profile picture description only when relevant.
|
||||
This runs after memory recall but before LLM generation.
|
||||
"""
|
||||
user_message = agent_input.get('input', '')
|
||||
|
||||
# Check if question is about profile picture
|
||||
if not matches_pfp_query(user_message):
|
||||
return agent_input
|
||||
|
||||
# Load profile picture description
|
||||
try:
|
||||
with open('/app/memory/profile_pictures/current_description.txt', 'r', encoding='utf-8') as f:
|
||||
pfp_description = f.read().strip()
|
||||
|
||||
log.warning(f"[Profile Picture Context] PFP query detected, injecting description")
|
||||
|
||||
# Inject into declarative memory section
|
||||
current_declarative = agent_input.get('declarative_memory', '')
|
||||
|
||||
pfp_context = f"\n\n## Your Current Profile Picture\n{pfp_description}\n"
|
||||
|
||||
agent_input['declarative_memory'] = current_declarative + pfp_context
|
||||
|
||||
except FileNotFoundError:
|
||||
log.error(f"[Profile Picture Context] current_description.txt not found")
|
||||
except Exception as e:
|
||||
log.error(f"[Profile Picture Context] Error loading PFP description: {e}")
|
||||
|
||||
return agent_input
|
||||
|
||||
|
||||
# Plugin metadata
|
||||
__version__ = "1.0.0"
|
||||
__description__ = "Injects profile picture description only when asked about it"
|
||||
@@ -80,6 +80,7 @@ services:
|
||||
- ./cat-plugins:/app/cat/plugins # Shared plugins directory
|
||||
- ./cheshire-cat/cat/data:/app/cat/data # Personality data (lore, prompts)
|
||||
- ./bot/moods:/app/moods # Mood description files
|
||||
- ./bot/memory:/app/memory # Profile pictures and other memory files
|
||||
- ./cheshire-cat/cat/log.py:/app/cat/log.py # Patched: fix loguru KeyError for third-party libs
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
|
||||
110
readmes/PROFILE_PICTURE_CONTEXT.md
Normal file
110
readmes/PROFILE_PICTURE_CONTEXT.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# Profile Picture Context Feature
|
||||
|
||||
## Overview
|
||||
|
||||
Miku can now describe her current profile picture when asked about it. The system uses regex pattern matching to detect relevant questions and injects the description only when needed, avoiding context bloat.
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. Pattern Detection
|
||||
The plugin monitors incoming messages for patterns like:
|
||||
- "What does your pfp look like?"
|
||||
- "Describe your profile picture"
|
||||
- "Tell me about your avatar"
|
||||
- "How do you look today?"
|
||||
- "Your new look..."
|
||||
|
||||
### 2. Context Injection
|
||||
When a match is detected:
|
||||
1. Loads `/app/memory/profile_pictures/current_description.txt`
|
||||
2. Injects it into the `declarative_memory` section of the prompt
|
||||
3. LLM receives the description and can answer accurately
|
||||
|
||||
### 3. No Context Bloat
|
||||
Unlike always including the PFP description in the prompt, this approach:
|
||||
- ✅ Only adds context when relevant
|
||||
- ✅ Saves tokens on every other message
|
||||
- ✅ Keeps prompts focused and efficient
|
||||
- ✅ Works with the existing profile picture update system
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### Plugin File
|
||||
**`cat-plugins/profile_picture_context/profile_picture_context.py`**
|
||||
- Defines regex patterns for PFP queries
|
||||
- Hook: `before_agent_starts` (priority=50)
|
||||
- Runs after memory recall but before LLM generation
|
||||
- Injects description into `declarative_memory` section
|
||||
|
||||
### Docker Volume
|
||||
**`docker-compose.yml`**
|
||||
- Mounts `./bot/memory:/app/memory` in Cat container
|
||||
- Provides access to `profile_pictures/current_description.txt`
|
||||
|
||||
### Description File
|
||||
**`bot/memory/profile_pictures/current_description.txt`**
|
||||
- Contains detailed description of current PFP
|
||||
- Updated automatically when profile picture changes
|
||||
- Format: Second-person narrative (addressing Miku)
|
||||
|
||||
## Regex Patterns
|
||||
|
||||
```python
|
||||
PFP_PATTERNS = [
|
||||
r'\b(what|describe|tell me about|explain)\b.*\b(pfp|profile pic|avatar|picture)\b',
|
||||
r'\b(your|miku\'?s?)\b.*\b(pfp|profile pic|avatar|picture)\b',
|
||||
r'\bwhat.*looking like\b',
|
||||
r'\byour (new )?look\b',
|
||||
r'\bhow.*look(ing)?\b.*today',
|
||||
r'\b(pfp|profile pic|avatar)\b.*\b(is|look|show)',
|
||||
]
|
||||
```
|
||||
|
||||
**Matches:**
|
||||
- ✓ "What does your pfp look like?"
|
||||
- ✓ "Describe your profile picture"
|
||||
- ✓ "Tell me about your avatar"
|
||||
- ✓ "How do you look today?"
|
||||
- ✓ "Your new look is cool"
|
||||
- ✓ "What are you looking like?"
|
||||
- ✓ "Show me your picture"
|
||||
|
||||
**Doesn't Match:**
|
||||
- ✗ "What's the weather like?" (no PFP keywords)
|
||||
- ✗ "Hello Miku!" (not a PFP query)
|
||||
- ✗ General conversation
|
||||
|
||||
## Integration with Profile Picture Updates
|
||||
|
||||
When Miku's profile picture is updated (via `/profile-picture/change` API):
|
||||
1. New image is set as Discord avatar
|
||||
2. Vision model generates description
|
||||
3. Description saved to `current_description.txt`
|
||||
4. **Plugin automatically uses new description** on next query
|
||||
|
||||
No manual updates needed - the system stays in sync!
|
||||
|
||||
## Example Flow
|
||||
|
||||
**User:** "What does your pfp look like?"
|
||||
|
||||
**Plugin:**
|
||||
1. ✓ Regex matches query
|
||||
2. 📖 Loads `current_description.txt`
|
||||
3. 💉 Injects into declarative_memory
|
||||
4. 🤖 LLM receives description in context
|
||||
|
||||
**Miku:** "I'm wearing a formal black suit jacket over a white shirt with a bow tie, holding a big bouquet of sunflowers! My teal twin tails are held up with yellow and brown striped hairbands. The art style is vibrant anime with bold lines and bright colors, and there are stars and sparkles in the background. The outfit is from my hit song 'Monitoring'! ✨💚"
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `cat-plugins/profile_picture_context/profile_picture_context.py` (NEW)
|
||||
- `docker-compose.yml` (added memory volume mount)
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] Add pattern for "show me a picture"
|
||||
- [ ] Cache description to avoid repeated file reads
|
||||
- [ ] Add metadata (artist, source URL) to responses
|
||||
- [ ] Support multiple language patterns (Japanese, etc.)
|
||||
- [ ] Add rate limiting for vision model calls
|
||||
Reference in New Issue
Block a user