Koko210/miku-discord
1
0
Fork 0
Code Issues 34 Pull Requests Actions Packages Projects Releases Wiki Activity
205 Commits 1 Branch 0 Tags
cfd5eb16f7506511f8a070bc7447cea9f714b894
Commit Graph

5 Commits

Author SHA1 Message Date
koko210Serve
9d2c14fa0b Fix vision pipeline: ffmpeg removal by autoremove, increase vision timeout, reduce frame count, add Discord activity awareness 
- bot/Dockerfile: Add ffmpeg to reinstall line after apt-get autoremove
  (autoremove was sweeping up ffmpeg as 'no longer needed' after playwright install)
- bot/utils/image_handling.py: Increase video analysis timeout 120s→300s, 6→3 for Tenor GIFs (GTX 1660 VRAM constraint)
- bot/utils/activities.py: Add _activity_changed_at timestamp tracking,
  get_current_activity_label() and get_current_activity_fresh() with 30-min decay
- bot/utils/cat_client.py: Pass current Discord activity to Cheshire Cat pipeline
- bot/utils/llm.py: Inject current Discord activity into system prompt
- cat-plugins/*: Forward Discord activity through working_memory to personality plugins
- bot/persona/*/preamble.txt: Add Discord status usage guidelines for character prompts
- llama-swap-rocm-config.yaml: Add qwen3.5 model entry for ComfyUI prompt generation
- AGENTS.md: New project documentation file
 2026-05-27 01:18:12 +03:00
koko210Serve
eafab336b4 feat: add Traefik proxy, custom chat template, improve Cheshire Cat memory 
docker-compose.yml:
- Add Traefik proxy network + labels for miku.panel domain
- Connect miku-bot service to proxy network

llama-swap-config.yaml / llama-swap-rocm-config.yaml:
- Add --chat-template-file flag to disable Llama 3.1 built-in tool
  calling (was causing malformed responses)
- ROCm config: add Rocinante-X 12B model entry for comparison testing

cheshire-cat discord_bridge plugin:
- Increase declarative memory recall (k=3→10, threshold 0.7→0.5)
  for better factual retrieval
- Add agent_prompt_prefix hook to enforce factual accuracy from
  declarative memories
- Add before_agent_starts debug logging for memory inspection
- Add passthrough hooks for message/suffix pipeline
 2026-03-04 00:48:58 +02:00
koko210Serve
c0aaab0c3a Disabled KV cache offloading on llama-server and enabled Flash Attention. Performance gains in the tens. 2026-01-27 19:11:49 +02:00
koko210Serve
fe0962118b Implemented new Japanese only text mode with WebUI toggle, utilizing a llama3.1 swallow dataset model. Next up is Japanese TTS. 2026-01-23 15:02:36 +02:00
koko210Serve
1fc3d74a5b Add dual GPU support with web UI selector 
Features:
- Built custom ROCm container for AMD RX 6800 GPU
- Added GPU selection toggle in web UI (NVIDIA/AMD)
- Unified model names across both GPUs for seamless switching
- Vision model always uses NVIDIA GPU (optimal performance)
- Text models (llama3.1, darkidol) can use either GPU
- Added /gpu-status and /gpu-select API endpoints
- Implemented GPU state persistence in memory/gpu_state.json

Technical details:
- Multi-stage Dockerfile.llamaswap-rocm with ROCm 6.2.4
- llama.cpp compiled with GGML_HIP=ON for gfx1030 (RX 6800)
- Proper GPU permissions without root (groups 187/989)
- AMD container on port 8091, NVIDIA on port 8090
- Updated bot/utils/llm.py with get_current_gpu_url() and get_vision_gpu_url()
- Modified bot/utils/image_handling.py to always use NVIDIA for vision
- Enhanced web UI with GPU selector button (blue=NVIDIA, red=AMD)

Files modified:
- docker-compose.yml (added llama-swap-amd service)
- bot/globals.py (added LLAMA_AMD_URL)
- bot/api.py (added GPU selection endpoints and helper function)
- bot/utils/llm.py (GPU routing for text models)
- bot/utils/image_handling.py (GPU routing for vision models)
- bot/static/index.html (GPU selector UI)
- llama-swap-rocm-config.yaml (unified model names)

New files:
- Dockerfile.llamaswap-rocm
- bot/memory/gpu_state.json
- bot/utils/gpu_router.py (load balancing utility)
- setup-dual-gpu.sh (setup verification script)
- DUAL_GPU_*.md (documentation files)
 2026-01-09 00:03:59 +02:00