reorganize: consolidate all documentation into readmes/
- Moved 20 root-level markdown files to readmes/ - Includes COMMANDS.md, CONFIG_README.md, all UNO docs, all completion reports - Added new: MEMORY_EDITOR_FEATURE.md, MEMORY_EDITOR_ESCAPING_FIX.md, CONFIG_SOURCES_ANALYSIS.md, MCP_TOOL_CALLING_ANALYSIS.md, and others - Root directory is now clean of documentation clutter
This commit is contained in:
334
readmes/MEMORY_EDITOR_FEATURE.md
Normal file
334
readmes/MEMORY_EDITOR_FEATURE.md
Normal file
@@ -0,0 +1,334 @@
|
||||
# Memory Editor UI - Comprehensive CRUD Interface
|
||||
|
||||
## Overview
|
||||
|
||||
Implemented comprehensive memory management UI in the Web Control Panel (Tab 9: Memories) with full CRUD operations. Users can now manually edit, create, delete, and search memories through an intuitive interface.
|
||||
|
||||
## Features
|
||||
|
||||
### 1. Edit Existing Memories
|
||||
- **Edit Button (✏️)** next to each memory/fact
|
||||
- Modal dialog with textarea for content editing
|
||||
- Source field for tracking where the memory came from
|
||||
- Preserves vector embeddings when only metadata changes
|
||||
- Re-embeds content when text is modified
|
||||
|
||||
### 2. Create New Memories
|
||||
- **"➕ Add Fact"** button for declarative memories
|
||||
- **"➕ Add Memory"** button for episodic memories
|
||||
- Modal dialog with:
|
||||
- Content textarea
|
||||
- Optional User ID (for user-specific memories)
|
||||
- Source field (defaults to "manual")
|
||||
- Automatically generates proper embeddings and metadata
|
||||
|
||||
### 3. Search/Filter Memories
|
||||
- Search input above each memory list
|
||||
- Real-time filtering as you type
|
||||
- Case-insensitive search across content and metadata
|
||||
- Shows/hides matching items dynamically
|
||||
|
||||
### 4. Delete Operations
|
||||
- Individual delete (🗑️ button) - single confirmation
|
||||
- Delete All - multi-step safety flow (unchanged)
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Backend API (bot/api.py)
|
||||
|
||||
**New Pydantic Models:**
|
||||
```python
|
||||
class MemoryEditRequest(BaseModel):
|
||||
content: str
|
||||
metadata: Optional[dict] = None
|
||||
|
||||
class MemoryCreateRequest(BaseModel):
|
||||
content: str
|
||||
collection: str # "declarative" or "episodic"
|
||||
user_id: Optional[str] = None
|
||||
source: Optional[str] = None
|
||||
metadata: Optional[dict] = None
|
||||
```
|
||||
|
||||
**New Endpoints:**
|
||||
- `PUT /memory/point/{collection}/{point_id}` - Edit existing memory
|
||||
- `POST /memory/create` - Create new memory point
|
||||
|
||||
### Cat Client (bot/utils/cat_client.py)
|
||||
|
||||
**New Methods:**
|
||||
|
||||
**`update_memory_point(collection, point_id, content, metadata=None)`:**
|
||||
1. Fetches existing point from Qdrant (includes vector)
|
||||
2. Checks if content changed
|
||||
3. If content changed: Re-embeds via Cat's `/embedder` endpoint
|
||||
4. If only metadata changed: Preserves existing vector
|
||||
5. Updates point in Qdrant with new payload
|
||||
6. Returns `True` on success
|
||||
|
||||
**`create_memory_point(collection, content, user_id=None, source=None)`:**
|
||||
1. Generates UUID for new point_id
|
||||
2. Embeds content via Cat's `/embedder` endpoint
|
||||
3. Builds payload with metadata:
|
||||
- `source`: Tracking field (e.g., "manual", "discord", "autonomous")
|
||||
- `when`: Unix timestamp
|
||||
- `user_id`: For declarative memories, identifies user
|
||||
4. Inserts into Qdrant collection
|
||||
5. Returns `point_id` on success, `None` on failure
|
||||
|
||||
**Key Details:**
|
||||
- Uses Qdrant REST API: `http://cheshire-cat-vector-memory:6333`
|
||||
- Leverages Cat's `/embedder` endpoint for consistent vector generation
|
||||
- Proper error handling with try/except blocks
|
||||
- Async operations with aiohttp
|
||||
|
||||
### Frontend UI (bot/static/index.html)
|
||||
|
||||
**Modal Dialogs Added:**
|
||||
|
||||
**Edit Memory Modal:**
|
||||
```html
|
||||
<div id="edit-memory-modal">
|
||||
<textarea id="edit-memory-content"></textarea>
|
||||
<input id="edit-memory-source" placeholder="Source">
|
||||
<button onclick="saveMemoryEdit()">Save Changes</button>
|
||||
<button onclick="closeEditMemoryModal()">Cancel</button>
|
||||
</div>
|
||||
```
|
||||
|
||||
**Create Memory Modal:**
|
||||
```html
|
||||
<div id="create-memory-modal">
|
||||
<textarea id="create-memory-content"></textarea>
|
||||
<input id="create-memory-user-id" placeholder="User ID (optional)">
|
||||
<input id="create-memory-source" value="manual">
|
||||
<button onclick="saveNewMemory()">Create Memory</button>
|
||||
<button onclick="closeCreateMemoryModal()">Cancel</button>
|
||||
</div>
|
||||
```
|
||||
|
||||
**JavaScript Functions Added:**
|
||||
|
||||
- `showEditMemoryModal(collection, pointId, memoryData)` - Open edit modal with data
|
||||
- `closeEditMemoryModal()` - Close edit modal
|
||||
- `saveMemoryEdit()` - PUT request to update memory
|
||||
- `showCreateMemoryModal(collection)` - Open create modal for collection type
|
||||
- `closeCreateMemoryModal()` - Close create modal
|
||||
- `saveNewMemory()` - POST request to create memory
|
||||
- `filterMemories(listId, searchTerm)` - Real-time search/filter
|
||||
|
||||
**UI Enhancements:**
|
||||
- Added `memory-item` class to all memory divs for filtering
|
||||
- Edit button (✏️) styled with blue color (#5599cc)
|
||||
- Delete button (🗑️) kept in red (#993333)
|
||||
- Search inputs with `oninput` handlers for live filtering
|
||||
- Flex layout for button groups
|
||||
|
||||
## Usage Guide
|
||||
|
||||
### Editing a Memory
|
||||
|
||||
1. Navigate to **Tab 9: Memories** in Web UI
|
||||
2. Load facts or episodic memories
|
||||
3. Click the **✏️ Edit** button next to any memory
|
||||
4. Modal opens with current content and source
|
||||
5. Modify the content and/or source
|
||||
6. Click **"Save Changes"**
|
||||
7. Memory is updated in Qdrant and list refreshes
|
||||
|
||||
### Creating a New Memory
|
||||
|
||||
**For Declarative Facts:**
|
||||
1. Click **"➕ Add Fact"** button
|
||||
2. Enter the fact content (e.g., "Miku's favorite color is cyan")
|
||||
3. Optionally add User ID (for user-specific facts)
|
||||
4. Set source (defaults to "manual")
|
||||
5. Click **"Create Memory"**
|
||||
6. Fact is embedded and stored in declarative collection
|
||||
|
||||
**For Episodic Memories:**
|
||||
1. Click **"➕ Add Memory"** button
|
||||
2. Enter the memory content (conversation context)
|
||||
3. Optionally add User ID
|
||||
4. Set source (e.g., "manual", "test", "import")
|
||||
5. Click **"Create Memory"**
|
||||
6. Memory is embedded and stored in episodic collection
|
||||
|
||||
### Searching Memories
|
||||
|
||||
1. Type in the search box above the memory list
|
||||
2. Results filter in real-time
|
||||
3. Clear the search box to see all memories again
|
||||
4. Search matches content and metadata
|
||||
|
||||
### Testing the Feature
|
||||
|
||||
**Create a Test Fact:**
|
||||
```javascript
|
||||
// In browser console or through UI:
|
||||
// Click "Add Fact" button, enter:
|
||||
Content: "Miku's favorite color is cyan"
|
||||
User ID: (leave empty for general fact)
|
||||
Source: "test"
|
||||
```
|
||||
|
||||
**Verify Through Chat:**
|
||||
```
|
||||
User: "What is your favorite color?"
|
||||
Miku: "My favorite color is cyan!" (should recall the fact)
|
||||
```
|
||||
|
||||
**Edit the Fact:**
|
||||
```javascript
|
||||
// Click ✏️ next to the fact, change:
|
||||
Content: "Miku's favorite color is teal"
|
||||
Source: "test_edit"
|
||||
```
|
||||
|
||||
**Delete the Fact:**
|
||||
```javascript
|
||||
// Click 🗑️ button, confirm deletion
|
||||
```
|
||||
|
||||
## Technical Architecture
|
||||
|
||||
### Data Flow
|
||||
|
||||
**Edit Memory:**
|
||||
```
|
||||
User clicks ✏️
|
||||
→ showEditMemoryModal() populates form
|
||||
→ User edits and clicks "Save"
|
||||
→ saveMemoryEdit() → PUT /memory/point/{collection}/{id}
|
||||
→ api.py validates with MemoryEditRequest
|
||||
→ cat_adapter.update_memory_point()
|
||||
→ Fetches existing point from Qdrant
|
||||
→ Re-embeds if content changed
|
||||
→ Updates point in Qdrant
|
||||
→ Returns success
|
||||
→ UI reloads memory list
|
||||
```
|
||||
|
||||
**Create Memory:**
|
||||
```
|
||||
User clicks ➕ Add Fact
|
||||
→ showCreateMemoryModal('declarative')
|
||||
→ User fills form and clicks "Create"
|
||||
→ saveNewMemory() → POST /memory/create
|
||||
→ api.py validates with MemoryCreateRequest
|
||||
→ cat_adapter.create_memory_point()
|
||||
→ Generates UUID
|
||||
→ Embeds content via Cat's /embedder
|
||||
→ Builds payload with metadata
|
||||
→ Inserts into Qdrant
|
||||
→ Returns point_id
|
||||
→ UI reloads memory list and refreshes stats
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
**Backend:**
|
||||
- Validates collection name (only "declarative" and "episodic" allowed)
|
||||
- Checks for empty content
|
||||
- Handles Qdrant connection errors
|
||||
- Returns proper error messages in JSON
|
||||
|
||||
**Frontend:**
|
||||
- Shows user-friendly notifications
|
||||
- Disables buttons during operations
|
||||
- Reverts button text on error
|
||||
- Validates required fields before submission
|
||||
|
||||
## Metadata Structure
|
||||
|
||||
### Declarative Memories
|
||||
```json
|
||||
{
|
||||
"source": "manual",
|
||||
"when": 1734567890.123,
|
||||
"user_id": "user_discord_id_here" // Optional, for user-specific facts
|
||||
}
|
||||
```
|
||||
|
||||
### Episodic Memories
|
||||
```json
|
||||
{
|
||||
"source": "manual",
|
||||
"when": 1734567890.123
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** `user_id` is only used in declarative memories to associate facts with specific users.
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Potential Features:
|
||||
1. **Batch Operations**
|
||||
- Select multiple memories with checkboxes
|
||||
- Bulk delete selected
|
||||
- Bulk export selected
|
||||
|
||||
2. **Import/Export**
|
||||
- Export memories as JSON file
|
||||
- Import memories from JSON
|
||||
- Bulk memory migration
|
||||
|
||||
3. **Advanced Filtering**
|
||||
- Filter by date range
|
||||
- Filter by source
|
||||
- Filter by user_id
|
||||
- Sort by similarity score
|
||||
|
||||
4. **Memory History**
|
||||
- Track edit history
|
||||
- Show who edited and when
|
||||
- Ability to revert changes
|
||||
|
||||
5. **Memory Insights**
|
||||
- Show most recalled memories
|
||||
- Identify unused/stale memories
|
||||
- Suggest memory consolidation
|
||||
|
||||
6. **Rich Text Editing**
|
||||
- Markdown support in memory content
|
||||
- Syntax highlighting for code snippets
|
||||
- Preview mode
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [x] Backend API endpoints respond correctly
|
||||
- [x] Cat client methods communicate with Qdrant
|
||||
- [x] Edit modal opens and populates correctly
|
||||
- [x] Edit saves and updates memory in Qdrant
|
||||
- [x] Create modal opens for both fact and memory types
|
||||
- [x] Create generates proper embeddings and stores in Qdrant
|
||||
- [x] Search/filter works in real-time
|
||||
- [x] Delete still works as expected
|
||||
- [x] UI refreshes after operations
|
||||
- [x] Error notifications display properly
|
||||
- [ ] Test with actual chat queries to verify LLM recalls edited facts
|
||||
- [ ] Test user-specific facts with user_id parameter
|
||||
- [ ] Performance test with large memory sets (1000+ items)
|
||||
|
||||
## Related Files
|
||||
|
||||
**Backend:**
|
||||
- `/bot/api.py` - Lines 2778-2788 (models), 2918-2968 (endpoints)
|
||||
- `/bot/utils/cat_client.py` - Lines 382-541 (new methods)
|
||||
|
||||
**Frontend:**
|
||||
- `/bot/static/index.html` - Lines 1613-1781 (UI sections and modals), 5268-5685 (JavaScript functions)
|
||||
|
||||
**Documentation:**
|
||||
- `/readmes/MEMORY_EDITOR_FEATURE.md` - This file
|
||||
- `/readmes/AUTONOMOUS_V2_DECISION_LOGIC.md` - Memory usage in autonomous system
|
||||
|
||||
## Notes
|
||||
|
||||
- Container must be restarted after API changes: `docker restart miku-bot`
|
||||
- Modals use flexbox centering with semi-transparent backdrop
|
||||
- Memory items use `class="memory-item"` for filtering
|
||||
- Edit button uses ✏️ emoji with blue color (#5599cc)
|
||||
- Create buttons use ➕ emoji
|
||||
- All operations show loading states (button text changes)
|
||||
- Success/error notifications use existing `showNotification()` function
|
||||
Reference in New Issue
Block a user