diff --git a/CLAUDE.md b/CLAUDE.md index 6b77033..141dabb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -Telegram bot built with TypeScript and [Telegraf](https://telegraf.js.org/). Uses Markov chains with trigrams to generate sentences from learned messages. Optional KI/LLM integration via OpenAI-compatible API. Each chat has its own isolated data. +Telegram bot built with TypeScript and [Telegraf](https://telegraf.js.org/). Uses Markov chains with bigrams to generate sentences from learned messages. Optional KI/LLM integration via OpenAI-compatible API. Each chat has its own isolated data. ## Commands @@ -20,52 +20,54 @@ docker compose up -d --build # Docker deployment ``` src/ ├── index.ts # Bot entry point, commands, setup wizard -├── markov.ts # Markov chain with trigram support -├── database.ts # SQLite persistence layer +├── markov.ts # Markov chain with bigram support (order 1) +├── database.ts # SQLite persistence layer (optimized) └── ai.ts # OpenAI-compatible client ``` **Data flow (Markov):** -1. Message received → `chain.learn(text)` → SQLite +1. Message received → `chain.learn(text)` → `updateChain(learned)` → SQLite (incremental) 2. Trigger check (reply/mention/random) -3. If triggered → `chain.generate()` → reply +3. If triggered → `chain.generate()` → sanitize (@-removal) → reply **Data flow (AI):** 1. `/ask-ai` or trigger word detected -2. Load last 50 messages as context +2. Load last 20 messages as context 3. Build prompt (global + group prompt) -4. Call OpenAI-compatible API → reply +4. Call OpenAI-compatible API → sanitize (@-removal) → reply **Per-chat isolation:** - Each chat has separate Markov chain data - Each chat has separate AI settings -- Each chat has separate message context (50 messages) +- Each chat has separate message context (20 messages) ## Key Implementation Details ### Markov Chain (markov.ts) -- Uses **trigrams** (order=2): "word1 word2" → "word3" +- Uses **bigrams** (order=1): "word1" → "word2" for higher creativity - Weighted random selection based on frequency -- Generates sentences up to 20 words +- No character filtering (keeps emojis/punctuation) ### Database (database.ts) - SQLite with `better-sqlite3` (synchronous API) +- **Optimized storage:** `updateChain` uses `ON CONFLICT` for incremental updates (no full rewrites) - Tables: - `chat_settings` - Markov probability - `markov_transitions` - Word transitions - `markov_starts` - Sentence starts - `ai_settings` - KI configuration per chat - - `message_context` - Last 50 messages per chat + - `message_context` - Last 20 messages per chat - `setup_sessions` - Setup wizard state - Automatic cleanup every 24h ### AI Client (ai.ts) - OpenAI-compatible API (works with OpenAI, Ollama, OpenRouter, etc.) +- Parameters: `temperature: 0.8`, `presence_penalty: 0.6`, `frequency_penalty: 0.6` - Prompt system: - Global: `data/system-prompt.txt` → `.env` fallback → default - Group-specific: stored in DB per chat - API key masking for security -- Context: Last 50 messages +- Context: Last 20 messages ### Setup Wizard - Started via `/ai setup` in group @@ -73,6 +75,12 @@ src/ - API keys never shown in full - Steps: Provider → API Key → Model → URL → Trigger/Prob → Group Prompt +### Performance Features +- **Admin Cache:** 5-minute cache for chat administrators to reduce API calls +- **Bot Info Cache:** Cached bot username and ID +- **Incremental DB:** Markov updates don't delete existing data +- **Typing Status:** AI responses show "typing" status while generating + ## Environment ```env @@ -80,46 +88,14 @@ BOT_TOKEN= # Telegram bot token (required) AI_DEFAULT_API_KEY= # Default API key (optional) AI_DEFAULT_BASE_URL= # Default API URL (default: OpenAI) AI_DEFAULT_MODEL= # Default model (default: gpt-4o-mini) -AI_SYSTEM_PROMPT= # Global system prompt (fallback, see below) +AI_SYSTEM_PROMPT= # Global system prompt (fallback) ``` **System Prompt Loading:** 1. `data/system-prompt.txt` (persistent, editable without rebuild) 2. `AI_SYSTEM_PROMPT` env variable (fallback) 3. Hardcoded default (last resort) -AI_DEFAULT_MODEL= # Default model (default: gpt-4o-mini) -AI_SYSTEM_PROMPT= # Global system prompt -``` -## Database Schema +## Response Triggers & Sanitization -```sql --- Markov -chat_settings (chat_id, probability) -markov_transitions (chat_id, key, next_word, count) -markov_starts (chat_id, key, count) - --- AI -ai_settings (chat_id, enabled, trigger_word, random_prob, group_prompt, provider, base_url, model, api_key) -message_context (chat_id, role, content, timestamp) -setup_sessions (user_id, chat_id, step, data) -``` - -## Response Triggers - -### Markov (always active) -1. Reply to bot's message → always respond -2. @username mention → always respond -3. Random probability (configurable per chat) - -### AI (when enabled) -1. `/ask-ai [text]` command -2. Trigger word (configurable, default: "ask-ai") -3. Random probability (configurable, default: 0%) - -## Security - -- API keys stored in database, never logged -- API keys masked in `/ai status` output: `sk-***...***xyz` -- Setup only via private chat -- Admin-only configuration commands \ No newline at end of file +- **@-Ping Protection:** All responses (AI & Markov) have `@` symbols removed before sending to prevent user notifications. \ No newline at end of file diff --git a/README.md b/README.md index 015c6a2..df9197f 100644 --- a/README.md +++ b/README.md @@ -5,11 +5,12 @@ Ein Telegram-Bot, der mithilfe von Markov Chains neue Sätze aus vorherigen Nach ## Features - Lernt von allen Text-Nachrichten in einer Gruppe -- Generiert grammatikalisch sinnvolle Sätze mit Trigrammen +- Generiert kreative Sätze mit Bigrammen (hohe Diversität) +- **Automatischer @-Ping-Schutz** (entfernt @ vor dem Senden) - Antwortet bei Reply, Erwähnung (@username) oder zufällig - Pro-Chat-Wahrscheinlichkeit einstellbar (Admins) - **KI/LLM-Integration** (OpenAI, Ollama, OpenRouter, etc.) -- Persistente SQLite-Datenbank +- **Leistungsoptimiert:** Inkrementelle SQLite-Updates statt kompletter Rewrites - Docker-Support ## Schnellstart @@ -154,8 +155,8 @@ src/ └── ai.ts # OpenAI-kompatibler Client ``` -- **Markov Chain** mit Trigrammen für bessere Grammatik -- **SQLite** für persistente Speicherung +- **Markov Chain** mit Bigrammen für hohe Kreativität und Abwechslung +- **SQLite** mit performanten ON CONFLICT Updates - Pro Chat separate Daten/Lernkurve ### Datenbank-Cleanup diff --git a/src/index.ts b/src/index.ts index d7bc6f5..f7166b0 100644 --- a/src/index.ts +++ b/src/index.ts @@ -56,14 +56,20 @@ function initSystemPrompt(): string { // Load from persistent file try { - const prompt = readFileSync(PERSISTENT_PROMPT_FILE, 'utf-8').trim(); + let prompt = readFileSync(PERSISTENT_PROMPT_FILE, 'utf-8').trim(); + + // Fallback to .env if file is empty + if (!prompt && process.env.AI_SYSTEM_PROMPT) { + prompt = process.env.AI_SYSTEM_PROMPT.trim(); + } + if (!prompt) { - console.warn('system-prompt.txt is empty, AI will use minimal default'); + console.warn('system-prompt.txt and AI_SYSTEM_PROMPT env are empty, AI will use minimal default'); } return prompt || 'Du bist ein freundlicher Chat-Bot.'; } catch (error) { - console.warn('Could not load system-prompt.txt:', error); - return 'Du bist ein freundlicher Chat-Bot.'; + console.warn('Could not load system-prompt.txt, checking .env fallback:', error); + return process.env.AI_SYSTEM_PROMPT || 'Du bist ein freundlicher Chat-Bot.'; } }