# ๐ค Miku Discord Bot ๐
[](https://www.docker.com/)
[](https://www.python.org/)
[](https://discordpy.readthedocs.io/)
*The world's #1 Virtual Idol, now in your Discord server! ๐ฑโจ*
[Features](#-features) โข [Quick Start](#-quick-start) โข [Architecture](#๏ธ-architecture) โข [API](#-api-endpoints) โข [Contributing](#-contributing)
---
## ๐ About
Meet **Hatsune Miku** - a fully-featured, AI-powered Discord bot that brings the cheerful, energetic virtual idol to life in your server! Powered by local LLMs (Llama 3.1), vision models (MiniCPM-V), and a sophisticated autonomous behavior system, Miku can chat, react, share content, and even change her profile picture based on her mood!
### Why This Bot?
- ๐ญ **14 Dynamic Moods** - From bubbly to melancholy, Miku's personality adapts
- ๐ค **Smart Autonomous Behavior** - Context-aware decisions without spamming
- ๐๏ธ **Vision Capabilities** - Analyzes images, videos, and GIFs in conversations
- ๐จ **Auto Profile Pictures** - Fetches & crops anime faces from Danbooru based on mood
- ๐ฌ **DM Support** - Personal conversations with mood tracking
- ๐ฆ **Twitter Integration** - Shares Miku-related tweets and figurine announcements
- ๐ฎ **ComfyUI Integration** - Natural language image generation requests
- ๐ **Voice Chat Ready** - Fish.audio TTS integration (docs included)
- ๐ **RESTful API** - Full control via HTTP endpoints
- ๐ณ **Production Ready** - Docker Compose with GPU support
---
## โจ Features
### ๐ง AI & LLM Integration
- **Local LLM Processing** with Llama 3.1 8B (via llama.cpp + llama-swap)
- **Automatic Model Switching** - Text โ๏ธ Vision models swap on-demand
- **OpenAI-Compatible API** - Easy migration and integration
- **Conversation History** - Per-user context with RAG-style retrieval
- **Smart Prompting** - Mood-aware system prompts with personality profiles
### ๐ญ Mood & Personality System
14 Available Moods (click to expand)
- ๐ **Neutral** - Classic cheerful Miku
- ๐ด **Asleep** - Sleepy and minimally responsive
- ๐ช **Sleepy** - Getting tired, simple responses
- ๐ **Excited** - Extra energetic and enthusiastic
- ๐ซ **Bubbly** - Playful and giggly
- ๐ค **Curious** - Inquisitive and wondering
- ๐ณ **Shy** - Blushing and hesitant
- ๐คช **Silly** - Goofy and fun-loving
- ๐ **Angry** - Frustrated or upset
- ๐ค **Irritated** - Mildly annoyed
- ๐ข **Melancholy** - Sad and reflective
- ๐ **Flirty** - Playful and teasing
- ๐ **Romantic** - Sweet and affectionate
- ๐ฏ **Serious** - Focused and thoughtful
- **Per-Server Mood Tracking** - Different moods in different servers
- **DM Mood Persistence** - Separate mood state for private conversations
- **Automatic Mood Shifts** - Responds to conversation sentiment
### ๐ค Autonomous Behavior System V2
The bot features a sophisticated **context-aware decision engine** that makes Miku feel alive:
- **Intelligent Activity Detection** - Tracks message frequency, user presence, and channel activity
- **Non-Intrusive** - Won't spam or interrupt important conversations
- **Mood-Based Personality** - Behavioral patterns change with mood
- **Multiple Action Types**:
- ๐ฌ General conversation starters
- ๐ Engaging specific users
- ๐ฆ Sharing Miku tweets
- ๐ฌ Joining ongoing conversations
- ๐จ Changing profile pictures
- ๐ Reacting to messages
**Rate Limiting**: Minimum 30-second cooldown between autonomous actions to prevent spam.
### ๐๏ธ Vision & Media Processing
- **Image Analysis** - Describe images shared in chat using MiniCPM-V 4.5
- **Video Understanding** - Extracts frames and analyzes video content
- **GIF Support** - Processes animated GIFs (converts to MP4 if needed)
- **Embed Content Extraction** - Reads Twitter/X embeds without API
- **Face Detection** - On-demand anime face detection service (GPU-accelerated)
### ๐จ Dynamic Profile Picture System
- **Danbooru Integration** - Searches for Miku artwork
- **Smart Cropping** - Automatic face detection and 1:1 crop
- **Mood-Based Selection** - Filters by tags matching current mood
- **Quality Filtering** - Only uses high-quality, safe-rated images
- **Fallback System** - Graceful degradation if detection fails
### ๐ฆ Twitter Features
- **Tweet Sharing** - Automatically fetches and shares Miku-related tweets
- **Figurine Notifications** - DM subscribers about new Miku figurine releases
- **Embed Compatibility** - Uses fxtwitter for better Discord previews
- **Duplicate Prevention** - Tracks sent tweets to avoid repeats
### ๐ฎ ComfyUI Image Generation
- **Natural Language Detection** - "Draw me as Miku swimming in a pool"
- **Workflow Integration** - Connects to external ComfyUI instance
- **Smart Prompting** - Enhances user requests with context
### ๐ก REST API Dashboard
Full-featured FastAPI server with endpoints for:
- Mood management (get/set/reset)
- Conversation history
- Autonomous actions (trigger manually)
- Profile picture updates
- Server configuration
- DM analysis reports
### ๐ง Developer Features
- **Docker Compose Setup** - One command deployment
- **GPU Acceleration** - NVIDIA runtime for models and face detection
- **Health Checks** - Automatic service monitoring
- **Volume Persistence** - Conversation history and settings saved
- **Hot Reload** - Update without restarting (for development)
---
## ๐ Quick Start
### Prerequisites
- **Docker** & **Docker Compose** installed
- **NVIDIA GPU** with CUDA support (for model inference)
- **Discord Bot Token** ([Create one here](https://discord.com/developers/applications))
- At least **8GB VRAM** recommended (4GB minimum)
### Installation
1. **Clone the repository**
```bash
git clone https://github.com/yourusername/miku-discord.git
cd miku-discord
```
2. **Set up your bot token**
Edit `docker-compose.yml` and replace the `DISCORD_BOT_TOKEN`:
```yaml
environment:
- DISCORD_BOT_TOKEN=your_token_here
- OWNER_USER_ID=your_discord_user_id # For DM reports
```
3. **Add your models**
Place these GGUF models in the `models/` directory:
- `Llama-3.1-8B-Instruct-UD-Q4_K_XL.gguf` (text model)
- `MiniCPM-V-4_5-Q3_K_S.gguf` (vision model)
- `MiniCPM-V-4_5-mmproj-f16.gguf` (vision projector)
4. **Launch the bot**
```bash
docker-compose up -d
```
5. **Check logs**
```bash
docker-compose logs -f miku-bot
```
6. **Access the dashboard**
Open http://localhost:3939 in your browser
### Optional: ComfyUI Integration
If you have ComfyUI running, update the path in `docker-compose.yml`:
```yaml
volumes:
- /path/to/your/ComfyUI/output:/app/ComfyUI/output:ro
```
### Optional: Face Detection Service
Start the anime face detector when needed:
```bash
docker-compose --profile tools up -d anime-face-detector
```
Access Gradio UI at http://localhost:7860
---
## ๐๏ธ Architecture
### Service Overview
```
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Discord API โ
โโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Miku Bot (Python) โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โ
โ โ Discord โ โ FastAPI โ โ Autonomous โ โ
โ โ Event Loop โ โ Server โ โ Engine โ โ
โ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโ
โ โ โ
โผ โผ โผ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ
โ llama-swap โ โ ComfyUI โ โ Face Detectorโ
โ (Model Server) โ โ (Image Gen) โ โ (On-Demand) โ
โ โ โ โ โ โ
โ โข Llama 3.1 โ โ โข Workflows โ โ โข Gradio UI โ
โ โข MiniCPM-V โ โ โข GPU Accel โ โ โข FastAPI โ
โ โข Auto-swap โ โ โ โ โ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโ
โ Models โ
โ (GGUF) โ
โโโโโโโโโโโโ
```
### Tech Stack
| Component | Technology |
|-----------|-----------|
| **Bot Framework** | Discord.py 2.0+ |
| **LLM Backend** | llama.cpp + llama-swap |
| **Text Model** | Llama 3.1 8B Instruct |
| **Vision Model** | MiniCPM-V 4.5 |
| **API Server** | FastAPI + Uvicorn |
| **Image Gen** | ComfyUI (external) |
| **Face Detection** | Anime-Face-Detector (Gradio) |
| **Database** | JSON files (conversation history, settings) |
| **Containerization** | Docker + Docker Compose |
| **GPU Runtime** | NVIDIA Container Toolkit |
### Key Components
#### 1. **llama-swap** (Model Server)
- Automatically loads/unloads models based on requests
- Prevents VRAM exhaustion by swapping between text and vision models
- OpenAI-compatible `/v1/chat/completions` endpoint
- Configurable TTL (time-to-live) per model
#### 2. **Autonomous Engine V2**
- Tracks message activity, user presence, and channel engagement
- Calculates "engagement scores" per server
- Makes context-aware decisions without LLM overhead
- Personality profiles per mood (e.g., shy mood = less engaging)
#### 3. **Server Manager**
- Per-guild configuration (mood, sleep state, autonomous settings)
- Scheduled tasks (bedtime reminders, autonomous ticks)
- Persistent storage in `servers_config.json`
#### 4. **Conversation History**
- Vector-based RAG (Retrieval Augmented Generation)
- Stores last 50 messages per user
- Semantic search using FAISS
- Context injection for continuity
---
## ๐ก API Endpoints
The bot runs a FastAPI server on port **3939** with the following endpoints:
### Mood Management
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/servers/{guild_id}/mood` | GET | Get current mood for server |
| `/servers/{guild_id}/mood` | POST | Set mood (body: `{"mood": "excited"}`) |
| `/servers/{guild_id}/mood/reset` | POST | Reset to neutral mood |
| `/mood` | GET | Get DM mood (deprecated, use server-specific) |
### Autonomous Actions
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/autonomous/general` | POST | Make Miku say something random |
| `/autonomous/engage` | POST | Engage a random user |
| `/autonomous/tweet` | POST | Share a Miku tweet |
| `/autonomous/reaction` | POST | React to a recent message |
| `/autonomous/custom` | POST | Custom prompt (body: `{"prompt": "..."}`) |
### Profile Pictures
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/profile-picture/change` | POST | Change profile picture (body: `{"mood": "happy"}`) |
| `/profile-picture/revert` | POST | Revert to previous picture |
| `/profile-picture/current` | GET | Get current picture metadata |
### Utilities
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/conversation/reset` | POST | Clear conversation history for user |
| `/logs` | GET | View bot logs (last 1000 lines) |
| `/prompt` | GET | View current system prompt |
| `/` | GET | Dashboard HTML page |
### Example Usage
```bash
# Set mood to excited
curl -X POST http://localhost:3939/servers/123456789/mood \
-H "Content-Type: application/json" \
-d '{"mood": "excited"}'
# Make Miku say something
curl -X POST http://localhost:3939/autonomous/general
# Change profile picture
curl -X POST http://localhost:3939/profile-picture/change \
-H "Content-Type: application/json" \
-d '{"mood": "flirty"}'
```
---
## ๐ฎ Usage Examples
### Basic Interaction
```
User: Hey Miku! How are you today?
Miku: Miku's doing great! ๐ Thanks for asking! โจ
User: Can you see this? [uploads image]
Miku: Ooh! ๐ I see a cute cat sitting on a keyboard! So fluffy! ๐ฑ
```
### Mood Changes
```
User: /mood excited
Miku: YAYYY!!! ๐โจ Miku is SO EXCITED right now!!! Let's have fun! ๐๐ถ
User: What's your favorite food?
Miku: NEGI!! ๐ฑ๐ฑ๐ฑ Green onions are THE BEST! Want some?! โจ
```
### Image Generation
```
User: Draw yourself swimming in a pool
Miku: Ooh! Let me create that for you! ๐จโจ [generates image]
```
### Autonomous Behavior
```
[After detecting activity in #general]
Miku: Hey everyone! ๐ What are you all talking about? ๐
```
---
## ๐ ๏ธ Configuration
### Model Configuration (`llama-swap-config.yaml`)
```yaml
models:
llama3.1:
cmd: /app/llama-server --port ${PORT} --model /models/Llama-3.1-8B-Instruct-UD-Q4_K_XL.gguf -ngl 99
ttl: 1800 # 30 minutes
vision:
cmd: /app/llama-server --port ${PORT} --model /models/MiniCPM-V-4_5-Q3_K_S.gguf --mmproj /models/MiniCPM-V-4_5-mmproj-f16.gguf
ttl: 900 # 15 minutes
```
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `DISCORD_BOT_TOKEN` | *Required* | Your Discord bot token |
| `OWNER_USER_ID` | *Optional* | Your Discord user ID (for DM reports) |
| `LLAMA_URL` | `http://llama-swap:8080` | Model server endpoint |
| `TEXT_MODEL` | `llama3.1` | Text generation model name |
| `VISION_MODEL` | `vision` | Vision model name |
### Persistent Storage
All data is stored in `bot/memory/`:
- `servers_config.json` - Per-server settings
- `autonomous_config.json` - Autonomous behavior settings
- `conversation_history/` - User conversation data
- `profile_pictures/` - Downloaded profile pictures
- `dms/` - DM conversation logs
- `figurine_subscribers.json` - Figurine notification subscribers
---
## ๐ Documentation
Detailed documentation available in the `readmes/` directory:
- **[AUTONOMOUS_V2_IMPLEMENTED.md](readmes/AUTONOMOUS_V2_IMPLEMENTED.md)** - Autonomous system V2 details
- **[VOICE_CHAT_IMPLEMENTATION.md](readmes/VOICE_CHAT_IMPLEMENTATION.md)** - Fish.audio TTS integration guide
- **[PROFILE_PICTURE_FEATURE.md](readmes/PROFILE_PICTURE_FEATURE.md)** - Profile picture system
- **[FACE_DETECTION_API_MIGRATION.md](readmes/FACE_DETECTION_API_MIGRATION.md)** - Face detection setup
- **[DM_ANALYSIS_FEATURE.md](readmes/DM_ANALYSIS_FEATURE.md)** - DM interaction analytics
- **[MOOD_SYSTEM_ANALYSIS.md](readmes/MOOD_SYSTEM_ANALYSIS.md)** - Mood system deep dive
- **[QUICK_REFERENCE.md](readmes/QUICK_REFERENCE.md)** - Ollama โ llama.cpp migration guide
---
## ๐ Troubleshooting
### Bot won't start
**Check if models are loaded:**
```bash
docker-compose logs llama-swap
```
**Verify GPU access:**
```bash
docker run --rm --runtime=nvidia nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi
```
### High VRAM usage
- Lower the `-ngl` parameter in `llama-swap-config.yaml` (reduce GPU layers)
- Reduce context size with `-c` parameter
- Use smaller quantization (Q3 instead of Q4)
### Autonomous actions not triggering
- Check `autonomous_config.json` - ensure enabled and cooldown settings
- Verify activity in server (bot tracks engagement)
- Check logs for decision engine output
### Face detection not working
- Ensure GPU is available: `docker-compose --profile tools up -d anime-face-detector`
- Check API health: `curl http://localhost:6078/health`
- View Gradio UI: http://localhost:7860
### Models switching too frequently
Increase TTL in `llama-swap-config.yaml`:
```yaml
ttl: 3600 # 1 hour instead of 30 minutes
```
### Development Setup
For local development without Docker:
```bash
# Install dependencies
cd bot
pip install -r requirements.txt
# Set environment variables
export DISCORD_BOT_TOKEN="your_token"
export LLAMA_URL="http://localhost:8080"
# Run the bot
python bot.py
```
### Code Style
- Use type hints where possible
- Follow PEP 8 conventions
- Add docstrings to functions
- Comment complex logic
---
## ๐ License
This project is provided as-is for educational and personal use. Please respect:
- Discord's [Terms of Service](https://discord.com/terms)
- Crypton Future Media's [Piapro Character License](https://piapro.net/intl/en_for_creators.html)
- Model licenses (Llama 3.1, MiniCPM-V)
---
## ๐ Acknowledgments
- **Crypton Future Media** - For creating Hatsune Miku
- **llama.cpp** - For efficient local LLM inference
- **mostlygeek/llama-swap** - For brilliant model management
- **Discord.py** - For the excellent Discord API wrapper
- **OpenAI** - For the API standard
- **MiniCPM-V Team** - For the amazing vision model
- **Danbooru** - For the artwork API
---
## ๐ Support
If you enjoy this project:
- โญ Star this repository
- ๐ Report bugs via [Issues](https://github.com/yourusername/miku-discord/issues)
- ๐ฌ Share your Miku bot setup in [Discussions](https://github.com/yourusername/miku-discord/discussions)
- ๐ค Listen to some Miku songs!
---
**Made with ๐ by a Miku fan, for Miku fans**
*"The future begins now!" - Hatsune Miku* ๐ถโจ
[โฌ Back to Top](#-miku-discord-bot-)