From 37063b7f859d0d121c4334c426162b1550b80037 Mon Sep 17 00:00:00 2001
From: koko210Serve <koko.yordanov@proton.me>
Date: Sun, 7 Dec 2025 17:29:24 +0200
Subject: [PATCH] Add comprehensive README with features, setup, and
 documentation

---
 README.md | 547 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 547 insertions(+)
 create mode 100644 README.md

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..fe48ac3
--- /dev/null
+++ b/README.md
@@ -0,0 +1,547 @@
+# 🎤 Miku Discord Bot 💙
+
+<div align="center">
+
+![Miku Banner](https://img.shields.io/badge/Virtual_Idol-Hatsune_Miku-00CED1?style=for-the-badge&logo=discord&logoColor=white)
+[![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge&logo=docker&logoColor=white)](https://www.docker.com/)
+[![Python](https://img.shields.io/badge/python-3.10+-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
+[![Discord.py](https://img.shields.io/badge/discord.py-2.0+-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discordpy.readthedocs.io/)
+
+*The world's #1 Virtual Idol, now in your Discord server! 🌱✨*
+
+[Features](#-features) • [Quick Start](#-quick-start) • [Architecture](#️-architecture) • [API](#-api-endpoints) • [Contributing](#-contributing)
+
+</div>
+
+---
+
+## 🌟 About
+
+Meet **Hatsune Miku** - a fully-featured, AI-powered Discord bot that brings the cheerful, energetic virtual idol to life in your server! Powered by local LLMs (Llama 3.1), vision models (MiniCPM-V), and a sophisticated autonomous behavior system, Miku can chat, react, share content, and even change her profile picture based on her mood!
+
+### Why This Bot?
+
+- 🎭 **14 Dynamic Moods** - From bubbly to melancholy, Miku's personality adapts
+- 🤖 **Smart Autonomous Behavior** - Context-aware decisions without spamming
+- 👁️ **Vision Capabilities** - Analyzes images, videos, and GIFs in conversations
+- 🎨 **Auto Profile Pictures** - Fetches & crops anime faces from Danbooru based on mood
+- 💬 **DM Support** - Personal conversations with mood tracking
+- 🐦 **Twitter Integration** - Shares Miku-related tweets and figurine announcements
+- 🎮 **ComfyUI Integration** - Natural language image generation requests
+- 🔊 **Voice Chat Ready** - Fish.audio TTS integration (docs included)
+- 📊 **RESTful API** - Full control via HTTP endpoints
+- 🐳 **Production Ready** - Docker Compose with GPU support
+
+---
+
+## ✨ Features
+
+### 🧠 AI & LLM Integration
+
+- **Local LLM Processing** with Llama 3.1 8B (via llama.cpp + llama-swap)
+- **Automatic Model Switching** - Text ↔️ Vision models swap on-demand
+- **OpenAI-Compatible API** - Easy migration and integration
+- **Conversation History** - Per-user context with RAG-style retrieval
+- **Smart Prompting** - Mood-aware system prompts with personality profiles
+
+### 🎭 Mood & Personality System
+
+<details>
+<summary>14 Available Moods (click to expand)</summary>
+
+- 😊 **Neutral** - Classic cheerful Miku
+- 😴 **Asleep** - Sleepy and minimally responsive
+- 😪 **Sleepy** - Getting tired, simple responses
+- 🎉 **Excited** - Extra energetic and enthusiastic
+- 💫 **Bubbly** - Playful and giggly
+- 🤔 **Curious** - Inquisitive and wondering
+- 😳 **Shy** - Blushing and hesitant
+- 🤪 **Silly** - Goofy and fun-loving
+- 😠 **Angry** - Frustrated or upset
+- 😤 **Irritated** - Mildly annoyed
+- 😢 **Melancholy** - Sad and reflective
+- 😏 **Flirty** - Playful and teasing
+- 💕 **Romantic** - Sweet and affectionate
+- 🎯 **Serious** - Focused and thoughtful
+
+</details>
+
+- **Per-Server Mood Tracking** - Different moods in different servers
+- **DM Mood Persistence** - Separate mood state for private conversations
+- **Automatic Mood Shifts** - Responds to conversation sentiment
+
+### 🤖 Autonomous Behavior System V2
+
+The bot features a sophisticated **context-aware decision engine** that makes Miku feel alive:
+
+- **Intelligent Activity Detection** - Tracks message frequency, user presence, and channel activity
+- **Non-Intrusive** - Won't spam or interrupt important conversations
+- **Mood-Based Personality** - Behavioral patterns change with mood
+- **Multiple Action Types**:
+  - 💬 General conversation starters
+  - 👋 Engaging specific users
+  - 🐦 Sharing Miku tweets
+  - 💬 Joining ongoing conversations
+  - 🎨 Changing profile pictures
+  - 😊 Reacting to messages
+
+**Rate Limiting**: Minimum 30-second cooldown between autonomous actions to prevent spam.
+
+### 👁️ Vision & Media Processing
+
+- **Image Analysis** - Describe images shared in chat using MiniCPM-V 4.5
+- **Video Understanding** - Extracts frames and analyzes video content
+- **GIF Support** - Processes animated GIFs (converts to MP4 if needed)
+- **Embed Content Extraction** - Reads Twitter/X embeds without API
+- **Face Detection** - On-demand anime face detection service (GPU-accelerated)
+
+### 🎨 Dynamic Profile Picture System
+
+- **Danbooru Integration** - Searches for Miku artwork
+- **Smart Cropping** - Automatic face detection and 1:1 crop
+- **Mood-Based Selection** - Filters by tags matching current mood
+- **Quality Filtering** - Only uses high-quality, safe-rated images
+- **Fallback System** - Graceful degradation if detection fails
+
+### 🐦 Twitter Features
+
+- **Tweet Sharing** - Automatically fetches and shares Miku-related tweets
+- **Figurine Notifications** - DM subscribers about new Miku figurine releases
+- **Embed Compatibility** - Uses fxtwitter for better Discord previews
+- **Duplicate Prevention** - Tracks sent tweets to avoid repeats
+
+### 🎮 ComfyUI Image Generation
+
+- **Natural Language Detection** - "Draw me as Miku swimming in a pool"
+- **Workflow Integration** - Connects to external ComfyUI instance
+- **Smart Prompting** - Enhances user requests with context
+
+### 📡 REST API Dashboard
+
+Full-featured FastAPI server with endpoints for:
+- Mood management (get/set/reset)
+- Conversation history
+- Autonomous actions (trigger manually)
+- Profile picture updates
+- Server configuration
+- DM analysis reports
+
+### 🔧 Developer Features
+
+- **Docker Compose Setup** - One command deployment
+- **GPU Acceleration** - NVIDIA runtime for models and face detection
+- **Health Checks** - Automatic service monitoring
+- **Volume Persistence** - Conversation history and settings saved
+- **Hot Reload** - Update without restarting (for development)
+
+---
+
+## 🚀 Quick Start
+
+### Prerequisites
+
+- **Docker** & **Docker Compose** installed
+- **NVIDIA GPU** with CUDA support (for model inference)
+- **Discord Bot Token** ([Create one here](https://discord.com/developers/applications))
+- At least **8GB VRAM** recommended (4GB minimum)
+
+### Installation
+
+1. **Clone the repository**
+   ```bash
+   git clone https://github.com/yourusername/miku-discord.git
+   cd miku-discord
+   ```
+
+2. **Set up your bot token**
+   
+   Edit `docker-compose.yml` and replace the `DISCORD_BOT_TOKEN`:
+   ```yaml
+   environment:
+     - DISCORD_BOT_TOKEN=your_token_here
+     - OWNER_USER_ID=your_discord_user_id  # For DM reports
+   ```
+
+3. **Add your models**
+   
+   Place these GGUF models in the `models/` directory:
+   - `Llama-3.1-8B-Instruct-UD-Q4_K_XL.gguf` (text model)
+   - `MiniCPM-V-4_5-Q3_K_S.gguf` (vision model)
+   - `MiniCPM-V-4_5-mmproj-f16.gguf` (vision projector)
+
+4. **Launch the bot**
+   ```bash
+   docker-compose up -d
+   ```
+
+5. **Check logs**
+   ```bash
+   docker-compose logs -f miku-bot
+   ```
+
+6. **Access the dashboard**
+   
+   Open http://localhost:3939 in your browser
+
+### Optional: ComfyUI Integration
+
+If you have ComfyUI running, update the path in `docker-compose.yml`:
+```yaml
+volumes:
+  - /path/to/your/ComfyUI/output:/app/ComfyUI/output:ro
+```
+
+### Optional: Face Detection Service
+
+Start the anime face detector when needed:
+```bash
+docker-compose --profile tools up -d anime-face-detector
+```
+
+Access Gradio UI at http://localhost:7860
+
+---
+
+## 🏗️ Architecture
+
+### Service Overview
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                        Discord API                          │
+└───────────────────────┬─────────────────────────────────────┘
+                        │
+                        ▼
+┌─────────────────────────────────────────────────────────────┐
+│                     Miku Bot (Python)                       │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
+│  │   Discord    │  │   FastAPI    │  │  Autonomous  │     │
+│  │  Event Loop  │  │   Server     │  │    Engine    │     │
+│  └──────────────┘  └──────────────┘  └──────────────┘     │
+└───────────┬────────────────┬────────────────┬──────────────┘
+            │                │                │
+            ▼                ▼                ▼
+┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐
+│   llama-swap    │ │   ComfyUI       │ │ Face Detector│
+│  (Model Server) │ │ (Image Gen)     │ │  (On-Demand) │
+│                 │ │                 │ │              │
+│  • Llama 3.1    │ │  • Workflows    │ │  • Gradio UI │
+│  • MiniCPM-V    │ │  • GPU Accel    │ │  • FastAPI   │
+│  • Auto-swap    │ │                 │ │              │
+└─────────────────┘ └─────────────────┘ └──────────────┘
+         │
+         ▼
+   ┌──────────┐
+   │  Models  │
+   │  (GGUF)  │
+   └──────────┘
+```
+
+### Tech Stack
+
+| Component | Technology |
+|-----------|-----------|
+| **Bot Framework** | Discord.py 2.0+ |
+| **LLM Backend** | llama.cpp + llama-swap |
+| **Text Model** | Llama 3.1 8B Instruct |
+| **Vision Model** | MiniCPM-V 4.5 |
+| **API Server** | FastAPI + Uvicorn |
+| **Image Gen** | ComfyUI (external) |
+| **Face Detection** | Anime-Face-Detector (Gradio) |
+| **Database** | JSON files (conversation history, settings) |
+| **Containerization** | Docker + Docker Compose |
+| **GPU Runtime** | NVIDIA Container Toolkit |
+
+### Key Components
+
+#### 1. **llama-swap** (Model Server)
+- Automatically loads/unloads models based on requests
+- Prevents VRAM exhaustion by swapping between text and vision models
+- OpenAI-compatible `/v1/chat/completions` endpoint
+- Configurable TTL (time-to-live) per model
+
+#### 2. **Autonomous Engine V2**
+- Tracks message activity, user presence, and channel engagement
+- Calculates "engagement scores" per server
+- Makes context-aware decisions without LLM overhead
+- Personality profiles per mood (e.g., shy mood = less engaging)
+
+#### 3. **Server Manager**
+- Per-guild configuration (mood, sleep state, autonomous settings)
+- Scheduled tasks (bedtime reminders, autonomous ticks)
+- Persistent storage in `servers_config.json`
+
+#### 4. **Conversation History**
+- Vector-based RAG (Retrieval Augmented Generation)
+- Stores last 50 messages per user
+- Semantic search using FAISS
+- Context injection for continuity
+
+---
+
+## 📡 API Endpoints
+
+The bot runs a FastAPI server on port **3939** with the following endpoints:
+
+### Mood Management
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/servers/{guild_id}/mood` | GET | Get current mood for server |
+| `/servers/{guild_id}/mood` | POST | Set mood (body: `{"mood": "excited"}`) |
+| `/servers/{guild_id}/mood/reset` | POST | Reset to neutral mood |
+| `/mood` | GET | Get DM mood (deprecated, use server-specific) |
+
+### Autonomous Actions
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/autonomous/general` | POST | Make Miku say something random |
+| `/autonomous/engage` | POST | Engage a random user |
+| `/autonomous/tweet` | POST | Share a Miku tweet |
+| `/autonomous/reaction` | POST | React to a recent message |
+| `/autonomous/custom` | POST | Custom prompt (body: `{"prompt": "..."}`) |
+
+### Profile Pictures
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/profile-picture/change` | POST | Change profile picture (body: `{"mood": "happy"}`) |
+| `/profile-picture/revert` | POST | Revert to previous picture |
+| `/profile-picture/current` | GET | Get current picture metadata |
+
+### Utilities
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/conversation/reset` | POST | Clear conversation history for user |
+| `/logs` | GET | View bot logs (last 1000 lines) |
+| `/prompt` | GET | View current system prompt |
+| `/` | GET | Dashboard HTML page |
+
+### Example Usage
+
+```bash
+# Set mood to excited
+curl -X POST http://localhost:3939/servers/123456789/mood \
+  -H "Content-Type: application/json" \
+  -d '{"mood": "excited"}'
+
+# Make Miku say something
+curl -X POST http://localhost:3939/autonomous/general
+
+# Change profile picture
+curl -X POST http://localhost:3939/profile-picture/change \
+  -H "Content-Type: application/json" \
+  -d '{"mood": "flirty"}'
+```
+
+---
+
+## 🎮 Usage Examples
+
+### Basic Interaction
+
+```
+User: Hey Miku! How are you today?
+Miku: Miku's doing great! 💙 Thanks for asking! ✨
+
+User: Can you see this? [uploads image]
+Miku: Ooh! 👀 I see a cute cat sitting on a keyboard! So fluffy! 🐱
+```
+
+### Mood Changes
+
+```
+User: /mood excited
+Miku: YAYYY!!! 🎉✨ Miku is SO EXCITED right now!!! Let's have fun! 💙🎶
+
+User: What's your favorite food?
+Miku: NEGI!! 🌱🌱🌱 Green onions are THE BEST! Want some?! ✨
+```
+
+### Image Generation
+
+```
+User: Draw yourself swimming in a pool
+Miku: Ooh! Let me create that for you! 🎨✨ [generates image]
+```
+
+### Autonomous Behavior
+
+```
+[After detecting activity in #general]
+Miku: Hey everyone! 👋 What are you all talking about? 💙
+```
+
+---
+
+## 🛠️ Configuration
+
+### Model Configuration (`llama-swap-config.yaml`)
+
+```yaml
+models:
+  llama3.1:
+    cmd: /app/llama-server --port ${PORT} --model /models/Llama-3.1-8B-Instruct-UD-Q4_K_XL.gguf -ngl 99
+    ttl: 1800  # 30 minutes
+    
+  vision:
+    cmd: /app/llama-server --port ${PORT} --model /models/MiniCPM-V-4_5-Q3_K_S.gguf --mmproj /models/MiniCPM-V-4_5-mmproj-f16.gguf
+    ttl: 900   # 15 minutes
+```
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `DISCORD_BOT_TOKEN` | *Required* | Your Discord bot token |
+| `OWNER_USER_ID` | *Optional* | Your Discord user ID (for DM reports) |
+| `LLAMA_URL` | `http://llama-swap:8080` | Model server endpoint |
+| `TEXT_MODEL` | `llama3.1` | Text generation model name |
+| `VISION_MODEL` | `vision` | Vision model name |
+
+### Persistent Storage
+
+All data is stored in `bot/memory/`:
+- `servers_config.json` - Per-server settings
+- `autonomous_config.json` - Autonomous behavior settings
+- `conversation_history/` - User conversation data
+- `profile_pictures/` - Downloaded profile pictures
+- `dms/` - DM conversation logs
+- `figurine_subscribers.json` - Figurine notification subscribers
+
+---
+
+## 📚 Documentation
+
+Detailed documentation available in the `readmes/` directory:
+
+- **[AUTONOMOUS_V2_IMPLEMENTED.md](readmes/AUTONOMOUS_V2_IMPLEMENTED.md)** - Autonomous system V2 details
+- **[VOICE_CHAT_IMPLEMENTATION.md](readmes/VOICE_CHAT_IMPLEMENTATION.md)** - Fish.audio TTS integration guide
+- **[PROFILE_PICTURE_FEATURE.md](readmes/PROFILE_PICTURE_FEATURE.md)** - Profile picture system
+- **[FACE_DETECTION_API_MIGRATION.md](readmes/FACE_DETECTION_API_MIGRATION.md)** - Face detection setup
+- **[DM_ANALYSIS_FEATURE.md](readmes/DM_ANALYSIS_FEATURE.md)** - DM interaction analytics
+- **[MOOD_SYSTEM_ANALYSIS.md](readmes/MOOD_SYSTEM_ANALYSIS.md)** - Mood system deep dive
+- **[QUICK_REFERENCE.md](readmes/QUICK_REFERENCE.md)** - Ollama → llama.cpp migration guide
+
+---
+
+## 🐛 Troubleshooting
+
+### Bot won't start
+
+**Check if models are loaded:**
+```bash
+docker-compose logs llama-swap
+```
+
+**Verify GPU access:**
+```bash
+docker run --rm --runtime=nvidia nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi
+```
+
+### High VRAM usage
+
+- Lower the `-ngl` parameter in `llama-swap-config.yaml` (reduce GPU layers)
+- Reduce context size with `-c` parameter
+- Use smaller quantization (Q3 instead of Q4)
+
+### Autonomous actions not triggering
+
+- Check `autonomous_config.json` - ensure enabled and cooldown settings
+- Verify activity in server (bot tracks engagement)
+- Check logs for decision engine output
+
+### Face detection not working
+
+- Ensure GPU is available: `docker-compose --profile tools up -d anime-face-detector`
+- Check API health: `curl http://localhost:6078/health`
+- View Gradio UI: http://localhost:7860
+
+### Models switching too frequently
+
+Increase TTL in `llama-swap-config.yaml`:
+```yaml
+ttl: 3600  # 1 hour instead of 30 minutes
+```
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome! Here's how you can help:
+
+1. **Fork the repository**
+2. **Create a feature branch** (`git checkout -b feature/amazing-feature`)
+3. **Commit your changes** (`git commit -m 'Add some amazing feature'`)
+4. **Push to the branch** (`git push origin feature/amazing-feature`)
+5. **Open a Pull Request**
+
+### Development Setup
+
+For local development without Docker:
+
+```bash
+# Install dependencies
+cd bot
+pip install -r requirements.txt
+
+# Set environment variables
+export DISCORD_BOT_TOKEN="your_token"
+export LLAMA_URL="http://localhost:8080"
+
+# Run the bot
+python bot.py
+```
+
+### Code Style
+
+- Use type hints where possible
+- Follow PEP 8 conventions
+- Add docstrings to functions
+- Comment complex logic
+
+---
+
+## 📝 License
+
+This project is provided as-is for educational and personal use. Please respect:
+- Discord's [Terms of Service](https://discord.com/terms)
+- Crypton Future Media's [Piapro Character License](https://piapro.net/intl/en_for_creators.html)
+- Model licenses (Llama 3.1, MiniCPM-V)
+
+---
+
+## 🙏 Acknowledgments
+
+- **Crypton Future Media** - For creating Hatsune Miku
+- **llama.cpp** - For efficient local LLM inference
+- **mostlygeek/llama-swap** - For brilliant model management
+- **Discord.py** - For the excellent Discord API wrapper
+- **OpenAI** - For the API standard
+- **MiniCPM-V Team** - For the amazing vision model
+- **Danbooru** - For the artwork API
+- **Fish.audio** - For TTS integration (optional)
+
+---
+
+## 💙 Support
+
+If you enjoy this project:
+- ⭐ Star this repository
+- 🐛 Report bugs via [Issues](https://github.com/yourusername/miku-discord/issues)
+- 💬 Share your Miku bot setup in [Discussions](https://github.com/yourusername/miku-discord/discussions)
+- 🎤 Listen to some Miku songs!
+
+---
+
+<div align="center">
+
+**Made with 💙 by a Miku fan, for Miku fans**
+
+*"The future begins now!" - Hatsune Miku* 🎶✨
+
+[⬆ Back to Top](#-miku-discord-bot-)
+
+</div>