Twitter changed the JS bundle structure from the old single-map format
(e=>e+"."+{...}[e]+"a.js") to a new two-map format
(u.u=e=>""+(({name})[e]||e)+"."+({hash})[e]+"a.js"), breaking
x-client-transaction-id generation.
This caused IndexError: list index out of range, which twscrape
interpreted as an account timeout (15-min lockout), preventing Miku
from fetching/sharing tweets.
The fix adds:
- A robust multi-pattern parser that tries known formats in order
- The _js_obj_to_dict helper from PR #303 for handling unquoted numeric
keys and scientific notation in JS object literals
- Debug logging to capture the JS snippet when ALL patterns fail,
making future breakage easier to diagnose
References:
- https://github.com/vladkens/twscrape/issues/302
- https://github.com/vladkens/twscrape/pull/303
This commit completes a major refactoring of the Miku control panel from a single 7,191-line monolithic HTML file to a modern modular architecture:
CHANGES:
- Extracted 872 lines of CSS into css/style.css
- Created 10 specialized JavaScript modules (4,964 lines total):
* core.js: Global state, utilities, initialization, polling system
* servers.js: Server management and mood handling
* modes.js: Evil mode, GPU selection, bipolar mode, scoreboard
* actions.js: Autonomous/manual actions, custom prompts, reactions
* image-gen.js: Image generation system
* status.js: Status display and statistics
* dm.js: DM user management and conversation analysis
* chat.js: LLM chat interface with streaming and voice calls
* memories.js: Cheshire Cat memory integration (episodic/declarative/procedural)
* profile.js: Profile picture, album gallery, activities editor
- Cleaned index.html to 1,351 lines (structure only, zero inline JS/CSS)
- Removed 12 duplicate variable declarations
- Maintained strict script load order for dependency resolution
- Added backup comment to index.html.bak for historical reference
VERIFICATION COMPLETED:
✓ All 191 functions/variables from original accounted for
✓ Cross-referenced with backup to ensure nothing lost
✓ All onclick handlers and modal systems validated
✓ No circular dependencies or broken references
✓ HTML structure integrity verified (11 tabs, all buttons/modals intact)
✓ CropperJS CDN links preserved
The refactored code is production-ready with improved maintainability and clear separation of concerns.
Critical fixes:
- Add threading.Lock for all shared mutable state (override, cache, current activity)
- Atomic YAML writes (temp file + os.replace) to prevent corruption on crash
- Deep-copy cache on reads to prevent callers from mutating shared state
High-severity fixes:
- Validate entries in pick_activity_for_mood() — skip/log malformed instead of KeyError
- Log warning on unrecognized activity type fallback
- Normalize empty-string state to None (avoid 'None' display)
- release_manual_override() now uses force=True so bot always shows activity
- Add try/except in release_manual_override() to handle failures gracefully
Medium fixes:
- Remove dead 'test' mood from activities.yaml
- Validate name length (128 char Discord limit) in CRUD and manual set
- Validate streaming entries have URL in CRUD path
- Add JSON parse error handling in API routes
- on_ready preserves active manual override instead of overwriting
- Log override expiry timestamp (HH:MM:SS) for easier debugging
- exc_info=True on presence update errors for full stack traces
Low fixes:
- JS activitySetFromEntry() shows notification on parse error
Each activity in the mood lists now has a 🎯 Set button that immediately
sets it as the bot's current Discord activity (30-min manual override),
so users can pick from existing entries instead of typing manually.
- Rewrite utils/activities.py with mood energy-driven activity probability
(high-energy moods like excited/bubbly show activity ~80-85% of the time,
low-energy moods like sleepy/melancholy only ~15-25%)
- Add manual override system with 30-min auto-expiry for Web UI control
- Support all 5 Discord activity types: listening, playing, watching,
competing, streaming (with purple LIVE badge via discord.Streaming)
- Add current activity tracking (get_current_activity)
- Add force=True param to update_bot_presence for on_ready (bot.py)
- Add 4 new API routes for manual override:
GET/POST/DELETE /activities/current, POST /activities/current/auto
- Expand activities.yaml from 139 to 157 entries, adding watching,
competing, and streaming entries across 11 moods
- Update Web UI: activity type dropdown with all 5 types, conditional
URL field for streaming, 'Current Activity' override panel with
set/clear/auto controls, type-aware icons and labels
- Add 'state' field to all 139 activity entries in activities.yaml
- Songs: state shows artist (e.g. 'by kz (livetune)')
- Games: state shows genre (e.g. 'Rhythm Game', 'Sandbox', 'FPS')
- Update pick_activity_for_mood() to return 3-tuple (type, name, state)
- Update update_bot_presence() to pass state to discord.Activity()
- Add state validation in set_activities_for_mood() (optional string)
- Update Web UI editor: view shows state, edit form has state input
- State is fully optional — backward compatible, no breaking changes
The 'state' field appears as a secondary text line in Discord profile
popup, the richest display possible for bot accounts (full Rich Presence
with cover art/buttons is server-side restricted to OAuth applications).
Collapsible section in the Status tab with:
- Normal and Evil mood sections, each collapsible
- Per-mood expandable rows showing songs (🎵) and games (🎮)
- Inline editing: change type, name, weight
- Add/remove entries per mood
- Save via API with client-side validation
- Reload from disk button
- Lazy-loads data only when section is expanded
- In on_ready(), set presence based on current mood (evil or normal)
after all state is restored
- When LLM-detected mood shift is applied, update presence immediately
New endpoints:
- GET /activities — full data (normal + evil)
- GET /activities/{section}/{mood} — per-mood activities
- POST /activities/{section}/{mood} — update activities with validation
- POST /activities/reload — force reload from disk
New module that loads activities.yaml and provides:
- Weighted random activity selection per mood
- Discord presence update (Listening/Playing)
- File mtime caching for hot-reload
- Validation for CRUD operations
- Fallback for moods with no activities defined
Curated list of Vocaloid/Miku songs and real game titles for each
normal mood (13 moods, excluding asleep) and each evil mood (10 moods).
Each entry has type (listening/playing), name, and weight for
weighted random selection. Editable via this file or the Web UI.
- 217 error returns across 18 route files + api.py now use JSONResponse
with appropriate HTTP status codes instead of returning HTTP 200
- Status code distribution: 500 (121), 400 (39), 503 (28), 404 (24), 409 (3), 502 (2)
- Fixed language.py tuple-return bug (was serializing as JSON array)
- Fixed bare except clauses in bipolar_mode.py and voice.py
- Body-level error schemas preserved (status/error + success/error patterns)
so web UI continues working without changes
- chat.py (SSE) unchanged: errors sent within stream protocol
- All 170 tests pass
- Change misleading 'Unused, kept for structural completeness' to
'TODO: implement angry-wakeup mechanic or remove field'
- Field is dead code: never read or written in any Python code
Previously gpu_router.py had its own module-level PREFER_AMD_GPU constant
that was frozen at import time. The config API wrote to globals.PREFER_AMD_GPU
which didn't exist, so runtime GPU preference changes never took effect.
Now globals.py owns PREFER_AMD_GPU and gpu_router reads it from there.
21 tests across 6 groups:
A. Config loading & persistence (runtime path, YAML schema, overrides)
B. Runtime state (live globals reading, /config/set sync, restore)
C. Reset (full reset, single-key reset)
D. Server manager (zero-server default, corrupt handling, CRUD, no dead code)
E. GPU deduplication (delegates to config_manager, correct URL switching)
F. Clean imports (no dead os/Union/GUILD_SETTINGS)
Run: ./bot/tests/run_tests.sh (builds + runs in Docker container)
Removed _create_default_config() which hardcoded a specific guild ID
(759889672804630530) as a fallback. Now:
- Missing servers_config.json → starts with empty servers dict
- Corrupt JSON → logs error, starts with empty servers dict
- Servers are added via the API/dashboard, not by magic defaults
All code that iterates server_manager.servers handles empty dicts safely.
The server_memories dict and its methods (get_server_memory,
set_server_memory) plus API endpoints (GET/POST /servers/{guild_id}/memory)
were never called by any bot logic, command, or frontend code.
All per-server state is stored as ServerConfig dataclass fields and
persisted via servers_config.json. The generic key-value store was an
unfinished scaffolding feature superseded by the dataclass approach.
get_server_config() and set_server_config() in ConfigManager had zero
callers — every part of the codebase already uses the server_manager
singleton. Removing them eliminates the risk of a stale write that
bypasses the in-memory cache in ServerManager.
server_manager is now the sole owner of servers_config.json.
Previously only 4 of 5+ settings were synced to globals when set via
the generic /config/set endpoint. Added:
- memory.use_cheshire_cat -> globals.USE_CHESHIRE_CAT
- runtime.mood.dm_mood -> globals.DM_MOOD + DM_MOOD_DESCRIPTION
- Uses same _GLOBALS_SYNC mapping pattern as restore_runtime_settings
reset_to_defaults() previously only cleared the runtime_config dict and
saved config_runtime.yaml, but never touched the actual globals that
control runtime behavior. After a reset, LANGUAGE_MODE, AUTONOMOUS_DEBUG,
VOICE_DEBUG_MODE, USE_CHESHIRE_CAT, PREFER_AMD_GPU, and DM_MOOD all kept
their current in-memory values until the next restart.
Now reset_to_defaults() also resets the corresponding globals to their
default values from CONFIG (the static config loaded from config.yaml).
Both full reset and single-key reset are supported. The default values
come from the Pydantic AppConfig schema, ensuring consistency.
Tested: set non-default values, full reset -> all back to defaults,
single-key reset -> only that key back to default, runtime_state property
reflects the reset immediately.
config_manager.runtime_state was a plain dict initialized with hardcoded
defaults (dm_mood='neutral', evil_mode=False, etc.) that were never updated
by any code path except current_gpu. The /config/state endpoint and
get_full_config() both returned this stale dict, so the API always reported
neutral mood and english mode regardless of actual state.
Replaced the static dict with a @property that reads live values from
globals (DM_MOOD, EVIL_MODE, BIPOLAR_MODE, LANGUAGE_MODE) on every access.
GPU state is still managed via _current_gpu and persisted to gpu_state.json.
get_state() and set_state() continue to work for the GPU path.
Removed the Config Manager Integration block and all 19 backward-compat
variable re-exports (LLAMA_URL, CHESHIRE_CAT_URL, LANGUAGE_MODE, etc.)
from config.py. These were dead code because:
1. Circular import: config.py tried to import config_manager at module
level, but config_manager.py imports from config.py first, so
HAS_CONFIG_MANAGER was always False and _get_config_value() was a
no-op that always returned the static value.
2. Frozen snapshots: Even if the circular import worked, the values were
assigned to module-level names at import time and never updated. Other
modules importing 'from config import LLAMA_URL' would get a stale
snapshot, not a live value.
3. Nothing imports them: The entire codebase uses globals.py for mutable
runtime state, not these config.py copies. Only ERROR_WEBHOOK_URL was
imported (by error_handler.py), so it is kept as a simple re-export
from SECRETS.
Also cleaned up unused imports: Any, field_validator.
Japanese mode is NOT affected — LANGUAGE_MODE and JAPANESE_TEXT_MODEL live
in globals.py and are untouched.
config.yaml nested cheshire_cat and face_detector under the 'services' key,
and llama URLs under 'services.llama'. But AppConfig expects:
- services -> {url, amd_url} (llama endpoints directly)
- cheshire_cat -> top-level key
- face_detector -> top-level key
Because Pydantic silently ignores extra fields, ServicesConfig received
{llama: {...}, cheshire_cat: {...}, face_detector: {...}} and none matched
its 'url'/'amd_url' fields, so ALL service config from YAML was silently
ignored and Pydantic defaults were always used instead.
Flattened services to contain url/amd_url directly, and moved cheshire_cat
and face_detector to top-level keys matching the AppConfig model. Verified
both AppConfig(**yaml_data) and config_manager dot-path traversal work.
config_runtime.yaml was written to the container root (/) because the path
resolved via Path(__file__).parent.parent from /app/config_manager.py = /.
This location is not volume-mounted, so all runtime config changes (language,
debug flags, Cheshire Cat toggle, mood, GPU preference) were lost on every
container restart.
Moved runtime_config_path to memory/config_runtime.yaml, which lives inside
the volume-mounted ./bot/memory:/app/memory directory and persists across
restarts. Also reordered __init__ so memory_dir is initialized before
runtime_config_path depends on it.
Relocated the button from the top action row to below the '512x512
displayed as circle' label for more intuitive placement next to the
avatar it acts on.
- Backend: album storage in memory/profile_pictures/album/{uuid}/ with
original.png, cropped.png, and metadata.json per entry
- add_to_album/add_batch_to_album with efficient resource management
(vision model + face detector kept alive across batch)
- set_album_entry_as_current auto-archives current PFP before replacing
- manual/auto crop album entries without applying to Discord
- Disk usage tracking, single & bulk delete
- API: full CRUD endpoints under /profile-picture/album/*
- Frontend: collapsible album grid in tab11 with thumbnail cards,
multi-select checkboxes for bulk delete, detail panel with crop
interface (Cropper.js), description editor, set-as-current action
- profile_picture_manager.py:
- Add ORIGINAL_PATH constant; save full-res original before every crop
- Add skip_crop param to change_profile_picture() for manual crop workflow
- Add manual_crop(x,y,w,h) method with Discord avatar update + role color sync
- Add auto_crop_only() to re-run face-detection crop on stored original
- Add update_description() with Cheshire Cat declarative memory re-injection
- Add regenerate_description() via vision model
- Skip crop step if image is already at/below 512x512
- api.py:
- GET /profile-picture/image/original — serve full-res original (no-cache)
- GET /profile-picture/image/current — serve current cropped avatar (no-cache)
- POST /profile-picture/change-no-crop — acquire image, skip auto-crop
- POST /profile-picture/manual-crop — apply crop coords {x,y,width,height}
- POST /profile-picture/auto-crop — re-run intelligent crop on original
- POST /profile-picture/description — save freeform description + Cat inject
- POST /profile-picture/regenerate-description — re-generate via vision model
- GET /profile-picture/description — fetch current description text
- index.html:
- Add new tab11 '🖼️ Profile Picture Management'
- Remove PFP + role color sections from Actions tab (tab2)
- Add Cropper.js 1.6.2 via CDN for manual square crop
- Tab layout: action buttons, file upload, auto/manual crop toggle,
Cropper.js interface, side-by-side original/cropped previews,
role color management, freeform description editor, metadata box (bottom)
- Wire switchTab hook for tab11 → loadPfpTab()
- All new JS functions: pfpChangeDanbooru, pfpUploadCustom, pfpRestoreFallback,
pfpShowCropInterface, pfpApplyManualCrop, pfpApplyAutoCrop, pfpSaveDescription,
pfpRegenerateDescription, pfpRefreshPreviews, setCustomRoleColor, resetRoleColor
When Evil Mode activates, the bot's Discord account avatar is changed to evil_pfp.png.
Previously, get_persona_avatar_urls() would read this swapped avatar and pass it to
the Miku webhook, causing both webhooks to display Evil Miku's pfp.
Now caching the regular Miku CDN URL before Evil Mode changes the bot's avatar.
When Evil Mode is active, the cached URL is used instead of reading from the bot
account. Discord CDN URLs remain valid after avatar changes, so this reliably
preserves the correct pfp for both regular and Evil Miku webhooks during arguments.
- Added MIKU_NORMAL_AVATAR_URL global in bot/globals.py
- Updated get_persona_avatar_urls() to cache and return the cached URL
- Save the normal avatar URL before Evil Mode switches the bot's avatar
- Fix silent None return in analyze_image_with_vision exception handler
- Add None/empty guards after vision analysis in bot.py (image, video, GIF, Tenor)
- Route all image/video/GIF responses through Cheshire Cat pipeline (was
calling query_llama directly), enabling episodic memory storage for media
interactions and correct Last Prompt display in Web UI
- Add media_type parameter to cat_adapter.query() and forward as
discord_media_type in WebSocket payload
- Update discord_bridge plugin to read media_type from payload and inject
MEDIA NOTE into system prefix in before_agent_starts hook
- Add _extract_vision_question() helper to strip Discord mentions and bot-name
triggers from user message; pass cleaned question to vision model so specific
questions (e.g. 'what is the person wearing?') go directly to the vision model
instead of the generic 'Describe this image in detail.' fallback
- Pass user_prompt to all analyze_image_with_qwen / analyze_video_with_vision
call sites in bot.py (image, video, GIF, Tenor, embed paths)
- Fix autonomous reaction loops skipping messages that @mention the bot or have
media attachments in DMs, preventing duplicate vision model calls for images
already being processed by the main message handler
- Increase vision max_tokens: images 300->800, video/GIF 400->1000 (no VRAM
impact; KV cache is pre-allocated at model load time)
- Change bot/memory/*.json to bot/memory/** to properly ignore all
subdirectories (dms/, dm_reports/, profile_pictures/)
- Untrack bot/memory/ files from index (DMs, profile pics, dm reports)
- Untrack cheshire-cat discord_bridge __pycache__/*.pyc from index
- These files are runtime/user data that should never be in version control
Voice conversion pipeline (Soprano TTS → RVC) with Docker support.
Previously tracked as bare gitlink; removed .git/ directories and
absorbed into main repo for unified tracking.
Includes: Soprano TTS, RVC WebUI integration, Docker configs,
WebSocket API, and benchmark scripts.
Updated .gitignore to exclude large model weights (*.pth, *.pt, *.onnx, *.index).
287 files (3.1GB of ML weights properly excluded via gitignore).
UNO card game web app (Node.js/React) with Miku bot integration.
Previously an independent git repo (fork of mizanxali/uno-online).
Removed .git/ and absorbed into main repo for unified tracking.
Includes bot integration code: botActionExecutor, cardParser,
gameStateBuilder, and server-side bot action support.
37 files, node_modules excluded via local .gitignore.
- Moved 20 root-level markdown files to readmes/
- Includes COMMANDS.md, CONFIG_README.md, all UNO docs, all completion reports
- Added new: MEMORY_EDITOR_FEATURE.md, MEMORY_EDITOR_ESCAPING_FIX.md,
CONFIG_SOURCES_ANALYSIS.md, MCP_TOOL_CALLING_ANALYSIS.md, and others
- Root directory is now clean of documentation clutter
- Moved 8 root-level test scripts + 2 from bot/ to tests/
- Moved run_rocinante_test.sh runner script to tests/
- Added tests/README.md documenting each test's purpose, type, and requirements
- Added test_pfp_context.py and test_rocinante_comparison.py (previously untracked)
