koko210Serve
eb557f655c
- Create profile_picture_context plugin to detect PFP queries via regex - Inject current_description.txt only when user asks about profile picture - Mount bot/memory directory in Cat container for PFP access - Avoids context bloat by only adding PFP description when relevant - Patterns match: 'what does your pfp look like', 'describe your avatar', etc. - Works seamlessly with existing profile picture update system - No manual sync needed - description auto-updates with PFP changes
111 lines
3.8 KiB
Markdown
111 lines
3.8 KiB
Markdown
# Profile Picture Context Feature
|
|
|
|
## Overview
|
|
|
|
Miku can now describe her current profile picture when asked about it. The system uses regex pattern matching to detect relevant questions and injects the description only when needed, avoiding context bloat.
|
|
|
|
## How It Works
|
|
|
|
### 1. Pattern Detection
|
|
The plugin monitors incoming messages for patterns like:
|
|
- "What does your pfp look like?"
|
|
- "Describe your profile picture"
|
|
- "Tell me about your avatar"
|
|
- "How do you look today?"
|
|
- "Your new look..."
|
|
|
|
### 2. Context Injection
|
|
When a match is detected:
|
|
1. Loads `/app/memory/profile_pictures/current_description.txt`
|
|
2. Injects it into the `declarative_memory` section of the prompt
|
|
3. LLM receives the description and can answer accurately
|
|
|
|
### 3. No Context Bloat
|
|
Unlike always including the PFP description in the prompt, this approach:
|
|
- ✅ Only adds context when relevant
|
|
- ✅ Saves tokens on every other message
|
|
- ✅ Keeps prompts focused and efficient
|
|
- ✅ Works with the existing profile picture update system
|
|
|
|
## Technical Implementation
|
|
|
|
### Plugin File
|
|
**`cat-plugins/profile_picture_context/profile_picture_context.py`**
|
|
- Defines regex patterns for PFP queries
|
|
- Hook: `before_agent_starts` (priority=50)
|
|
- Runs after memory recall but before LLM generation
|
|
- Injects description into `declarative_memory` section
|
|
|
|
### Docker Volume
|
|
**`docker-compose.yml`**
|
|
- Mounts `./bot/memory:/app/memory` in Cat container
|
|
- Provides access to `profile_pictures/current_description.txt`
|
|
|
|
### Description File
|
|
**`bot/memory/profile_pictures/current_description.txt`**
|
|
- Contains detailed description of current PFP
|
|
- Updated automatically when profile picture changes
|
|
- Format: Second-person narrative (addressing Miku)
|
|
|
|
## Regex Patterns
|
|
|
|
```python
|
|
PFP_PATTERNS = [
|
|
r'\b(what|describe|tell me about|explain)\b.*\b(pfp|profile pic|avatar|picture)\b',
|
|
r'\b(your|miku\'?s?)\b.*\b(pfp|profile pic|avatar|picture)\b',
|
|
r'\bwhat.*looking like\b',
|
|
r'\byour (new )?look\b',
|
|
r'\bhow.*look(ing)?\b.*today',
|
|
r'\b(pfp|profile pic|avatar)\b.*\b(is|look|show)',
|
|
]
|
|
```
|
|
|
|
**Matches:**
|
|
- ✓ "What does your pfp look like?"
|
|
- ✓ "Describe your profile picture"
|
|
- ✓ "Tell me about your avatar"
|
|
- ✓ "How do you look today?"
|
|
- ✓ "Your new look is cool"
|
|
- ✓ "What are you looking like?"
|
|
- ✓ "Show me your picture"
|
|
|
|
**Doesn't Match:**
|
|
- ✗ "What's the weather like?" (no PFP keywords)
|
|
- ✗ "Hello Miku!" (not a PFP query)
|
|
- ✗ General conversation
|
|
|
|
## Integration with Profile Picture Updates
|
|
|
|
When Miku's profile picture is updated (via `/profile-picture/change` API):
|
|
1. New image is set as Discord avatar
|
|
2. Vision model generates description
|
|
3. Description saved to `current_description.txt`
|
|
4. **Plugin automatically uses new description** on next query
|
|
|
|
No manual updates needed - the system stays in sync!
|
|
|
|
## Example Flow
|
|
|
|
**User:** "What does your pfp look like?"
|
|
|
|
**Plugin:**
|
|
1. ✓ Regex matches query
|
|
2. 📖 Loads `current_description.txt`
|
|
3. 💉 Injects into declarative_memory
|
|
4. 🤖 LLM receives description in context
|
|
|
|
**Miku:** "I'm wearing a formal black suit jacket over a white shirt with a bow tie, holding a big bouquet of sunflowers! My teal twin tails are held up with yellow and brown striped hairbands. The art style is vibrant anime with bold lines and bright colors, and there are stars and sparkles in the background. The outfit is from my hit song 'Monitoring'! ✨💚"
|
|
|
|
## Files Modified
|
|
|
|
- `cat-plugins/profile_picture_context/profile_picture_context.py` (NEW)
|
|
- `docker-compose.yml` (added memory volume mount)
|
|
|
|
## Future Enhancements
|
|
|
|
- [ ] Add pattern for "show me a picture"
|
|
- [ ] Cache description to avoid repeated file reads
|
|
- [ ] Add metadata (artist, source URL) to responses
|
|
- [ ] Support multiple language patterns (Japanese, etc.)
|
|
- [ ] Add rate limiting for vision model calls
|