Twitter changed the JS bundle structure from the old single-map format
(e=>e+"."+{...}[e]+"a.js") to a new two-map format
(u.u=e=>""+(({name})[e]||e)+"."+({hash})[e]+"a.js"), breaking
x-client-transaction-id generation.
This caused IndexError: list index out of range, which twscrape
interpreted as an account timeout (15-min lockout), preventing Miku
from fetching/sharing tweets.
The fix adds:
- A robust multi-pattern parser that tries known formats in order
- The _js_obj_to_dict helper from PR #303 for handling unquoted numeric
keys and scientific notation in JS object literals
- Debug logging to capture the JS snippet when ALL patterns fail,
making future breakage easier to diagnose
References:
- https://github.com/vladkens/twscrape/issues/302
- https://github.com/vladkens/twscrape/pull/303
- Created new logging infrastructure with per-component filtering
- Added 6 log levels: DEBUG, INFO, API, WARNING, ERROR, CRITICAL
- Implemented non-hierarchical level control (any combination can be enabled)
- Migrated 917 print() statements across 31 files to structured logging
- Created web UI (system.html) for runtime configuration with dark theme
- Added global level controls to enable/disable levels across all components
- Added timestamp format control (off/time/date/datetime options)
- Implemented log rotation (10MB per file, 5 backups)
- Added API endpoints for dynamic log configuration
- Configured HTTP request logging with filtering via api.requests component
- Intercepted APScheduler logs with proper formatting
- Fixed persistence paths to use /app/memory for Docker volume compatibility
- Fixed checkbox display bug in web UI (enabled_levels now properly shown)
- Changed System Settings button to open in same tab instead of new window
Components: bot, api, api.requests, autonomous, persona, vision, llm,
conversation, mood, dm, scheduled, gpu, media, server, commands,
sentiment, core, apscheduler
All settings persist across container restarts via JSON config.
Twitter changed their JavaScript response format to include unquoted keys in JSON objects, which breaks twscrape's parser. This fix applies a monkey patch that uses regex to quote the unquoted keys before parsing.
This resolves the issue preventing figurine notifications from being sent for the past several days.
Reference: https://github.com/vladkens/twscrape/issues/284
