From f15459bb9956012a8caf1ec5cdc312de142256d1 Mon Sep 17 00:00:00 2001 From: dahyeon Date: Mon, 18 May 2026 20:08:31 -0700 Subject: [PATCH] Add bilingual README --- README.md | 385 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 331 insertions(+), 54 deletions(-) diff --git a/README.md b/README.md index f176c75..a8cc79e 100644 --- a/README.md +++ b/README.md @@ -1,98 +1,375 @@ # lolympus-agent -## Overview +`lolympus-agent`는 카카오톡 기반으로 운영되는 롤림푸스 커뮤니티의 운영 자동화를 위한 NATS 메시지 드리븐 멀티 에이전트 시스템입니다. 신규 유저 온보딩과 커뮤니티 규칙을 묻는 메시지에 대한 응답을 지원합니다. -lolympus-agent is an LLM-based onboarding agent for the lolympus gaming community. -It receives onboarding events, checks the user's current state, and replies with a -short helpful message through the community's backend messaging pipeline. +English version: [English](#english) -## Purpose +## 개요 -I created this service to automate repetitive onboarding guidance while keeping -the response quality close to a human community staff member. The agent can answer -common questions, validate onboarding progress, and route more complex cases -through LLM-powered reasoning and tool calls. +카카오톡 운영방에는 닉네임 형식, 하트 인증, 디스코드 프로필, 커뮤니티 규칙처럼 반복해서 들어오는 질문이 많습니다. 이 서비스는 그런 메시지를 받아 현재 상태를 확인하고, 필요한 경우 짧은 안내 메시지를 생성합니다. -## Functionalities +응답하기 애매하거나 자동화하기 어려운 요청은 supervisor review queue로 넘겨 운영자가 확인할 수 있게 합니다. -**1. Onboarding responses:**
-    * Handles new member, nickname, photo, progress, timeout, and completion events
-    * Generates concise responses for user messages
-    * Falls back to deterministic templates when an LLM response is not needed
+## 목적 -**2. Nickname and progress checks:**
-    * Validates community nickname format
-    * Checks onboarding session progress from Redis/Garnet
-    * Looks up room context when MongoDB is configured
+반복적인 온보딩 안내와 규칙 질의 응답을 자동화하되, 운영자가 직접 답하는 톤과 안전성을 최대한 유지하는 것이 목표입니다. -**3. Tool-aware LLM runtime:**
-    * Supports planner, responder, critic, and tool-loop execution modes
-    * Routes requests across local, proxy, or OpenRouter-compatible LLM providers
-    * Uses circuit breaking, health checks, and response validation
+단순한 케이스는 template이나 가벼운 LLM 응답으로 처리하고, 더 복잡한 케이스는 tool 호출, handoff, review queue를 통해 다룹니다. -**4. Evaluation pipeline:**
-    * Captures golden and shadow evaluation cases
-    * Runs rule, LLM, process, attribution, and PRM-style judges
-    * Supports review queues, calibration, drift checks, promotion, and tournaments
+## 기능 -**5. Background worker:**
-    * Processes shadow evaluation jobs asynchronously
-    * Archives evaluation results and publishes alerts
-    * Keeps real-time responses separate from slower analysis work
+**1. 온보딩 응답** -## Used technologies +- 신규 멤버, 닉네임, 사진 인증, 진행 상태, timeout, 완료 이벤트를 처리합니다. +- 사용자 메시지에 맞춰 짧은 안내를 생성합니다. +- LLM이 필요 없는 케이스는 deterministic template로 빠르게 응답합니다. -    * TypeScript
-    * Node.js
-    * NATS
-    * Redis / Garnet
-    * MongoDB / Mongoose
-    * OpenTelemetry
-    * Pino
-    * Zod
-    * Vitest
-    * ESLint
-    * Prettier
+**2. 닉네임과 진행 상태 확인** -## How to run +- 커뮤니티 닉네임 형식을 검사합니다. +- Redis/Garnet에 저장된 온보딩 세션 진행 상태를 확인합니다. +- MongoDB가 설정되어 있으면 room context를 함께 참고합니다. -1. Install dependencies: +**3. 멀티 에이전트 라우팅** + +- onboarding, moderation, supervisor skill을 분리해 다룹니다. +- `src/agent/dispatch.ts`의 공통 dispatch loop가 skill 간 handoff를 처리합니다. +- transition matrix와 budget guardrail로 잘못된 handoff나 loop를 막습니다. + +**4. Tool-aware LLM runtime** + +- planner, responder, critic, tool-loop 실행 모드를 지원합니다. +- local, proxy, OpenRouter 호환 LLM provider를 사용할 수 있습니다. +- circuit breaker, health check, response validation을 포함합니다. + +**5. Evaluation pipeline** + +- golden case와 shadow evaluation case를 수집합니다. +- rule, LLM, process, handoff judge를 실행합니다. +- review queue, calibration, drift check, promotion, tournament를 통해 prompt와 routing 개선 근거를 만듭니다. + +**6. Background worker와 운영 알림** + +- shadow eval과 archive 작업을 실시간 응답 경로 밖에서 처리합니다. +- eval 결과를 MongoDB에 저장하고, 필요한 경우 NATS/Slack alert를 보냅니다. +- supervisor review queue로 운영자가 확인해야 하는 케이스를 모읍니다. + +## 사용 기술 + +- TypeScript +- Node.js +- NATS +- Redis / Garnet +- MongoDB / Mongoose +- Infisical +- OpenTelemetry +- Pino +- Zod +- Vitest +- ESLint +- Prettier + +## 실행 방법 + +1. 의존성을 설치합니다. ```bash npm install ``` -2. Create a local environment file: +2. 환경 변수를 준비합니다. ```bash cp env-vars/env.example env-vars/dev.env ``` -3. Fill in the required NATS, Redis/Garnet, MongoDB, and LLM provider values in - `env-vars/dev.env`. +`env-vars/env.example`이 없는 환경이라면 아래 [환경 변수](#환경-변수) 섹션을 기준으로 `env-vars/dev.env`를 직접 만듭니다. -4. Run checks: +3. 타입과 테스트를 확인합니다. ```bash npm run typecheck npm test ``` -5. Start the agent: +4. 개발 환경으로 실행합니다. ```bash npm run start:dev ``` +PM2로 agent와 worker를 함께 실행하려면 다음 명령을 사용합니다. + +```bash +npm run pm2:dev +``` + +## 환경 변수 + +Infisical 연동을 위한 환경 변수가 필요합니다. + +```env +INFISICAL_CLIENT_SECRET=your_client_secret +INFISICAL_CLIENT_ID=your_client_id +INFISICAL_PROJECT_ID=your_project_id +NODE_ENV=dev|test|prod +``` + +### 환경 변수 로드 우선순위 + +환경 변수는 다음 순서로 로드되며, 먼저 로드된 값이 우선합니다. + +1. **LocalMachine 전역 환경 변수** + - Linux/macOS: `~/source/global-env-vars.txt` + - Windows: `%USERPROFILE%\source\global-env-vars.txt` + - 모든 프로젝트에서 공통으로 사용하는 전역 설정입니다. +2. **프로젝트별 환경 변수 파일** + - `./env-vars/{NODE_ENV}.env` + - 프로젝트 특화 설정입니다. 예: `dev.env`, `test.env`, `prod.env` +3. **쉘 환경 변수** + - 현재 쉘 세션에서만 쓰는 임시 설정입니다. + +동일한 키가 여러 곳에 정의되어 있으면 우선순위가 높은 값이 사용됩니다. 앱 내부 dotenv 로더는 `override: false`로 동작하므로, 이미 설정된 값은 `env-vars/{NODE_ENV}.env`가 덮어쓰지 않습니다. + +### IMDS + +Infisical client id/secret은 환경 변수로 직접 넣을 수도 있고, IMDS 호환 local metadata service에서 읽을 수도 있습니다. + +기본 후보는 개발 환경에서 `http://127.0.0.1:8675`, 운영 환경에서 `http://169.254.169.254`입니다. 필요하면 `METADATA_BASE_URL`, `IMDS_BASE_URL`, `MACHINE_ENV_BASE_URL`로 바꿀 수 있고, `IMDS_DISABLED=true`로 비활성화할 수 있습니다. + +### Infisical Secrets + +다음 시크릿이 Infisical에 설정되어 있어야 합니다. + +| Secret | 설명 | +| --------------------------- | ------------------------- | +| `NATS_ENDPOINT` | NATS 서버 엔드포인트 | +| `NATS_USER` | NATS 사용자명 | +| `NATS_PASS` | NATS 비밀번호 | +| `GARNET_ENDPOINT` | Garnet/Redis 엔드포인트 | +| `GARNET_USERID` | Garnet/Redis 사용자명 | +| `GARNET_PASSWORD` | Garnet/Redis 비밀번호 | +| `MONGODB_CONNECTION_STRING` | MongoDB 연결 문자열 | +| `MONGODB_PROJECT_ID` | MongoDB project id | +| `SLACK_ALERT_WEBHOOK_URL` | Slack alert webhook, 선택 | + +MinIO를 함께 사용하는 배포라면 아래 값도 Infisical에 둡니다. + +| Secret | 설명 | +| ------------------ | --------------------- | +| `MINIO_ENDPOINT` | MinIO 서버 엔드포인트 | +| `MINIO_ACCESS_KEY` | MinIO 액세스 키 | +| `MINIO_SECRET_KEY` | MinIO 시크릿 키 | + +## 자주 쓰는 명령 + +```bash +npm run build +npm run typecheck +npm run lint +npm test +npm run eval -- --skip-llm-judge +npm run eval:moderation -- --skip-llm-judge +npm run eval:handoff -- --skip-llm-judge +npm run eval:report -- --worst=20 --since=7d +npm run eval:router-shadow-report -- --since=7d +npm run review:list -- --limit=10 +``` + +운영 환경에서는 실시간 agent(`src/main.ts`)와 background worker(`src/workers/background.ts`)를 함께 실행합니다. + +## 문서 + +- [docs/multi-agent-architecture.md](docs/multi-agent-architecture.md): 전체 구조 +- [docs/service-evaluation-operations.md](docs/service-evaluation-operations.md): 운영 runbook +- [docs/optimization-roadmap.md](docs/optimization-roadmap.md): 고도화 시작 조건 +- [docs/resolution-hash-sunset-procedure.md](docs/resolution-hash-sunset-procedure.md): supervisor resolution hash 정리 절차 + +--- + +## English + +# lolympus-agent + +`lolympus-agent` is a NATS-driven multi-agent system for automating operations in the KakaoTalk-based Lolympus community. It supports new-user onboarding and replies to messages that ask about community rules. + +## Overview + +KakaoTalk operation rooms get many repeated questions about nickname format, heart verification, Discord profiles, and community rules. This service receives those messages, checks the current state, and produces short guidance when it can. + +Unclear or risky requests are sent to the supervisor review queue so an operator can check them. + +## Purpose + +The goal is to automate repetitive onboarding and rule-answering work while keeping the tone and safety close to a human operator. + +Simple cases use templates or lightweight LLM responses. More complex cases go through tool calls, handoff, and review queues. + +## Functionalities + +**1. Onboarding responses** + +- Handles new member, nickname, photo, progress, timeout, and completion events. +- Generates short replies for user messages. +- Falls back to deterministic templates when an LLM response is not needed. + +**2. Nickname and progress checks** + +- Validates the community nickname format. +- Checks onboarding session progress from Redis/Garnet. +- Looks up room context when MongoDB is configured. + +**3. Multi-agent routing** + +- Separates onboarding, moderation, and supervisor skills. +- Handles skill-to-skill handoff through the shared dispatch loop in `src/agent/dispatch.ts`. +- Uses a transition matrix and budget guardrails to block invalid handoffs and loops. + +**4. Tool-aware LLM runtime** + +- Supports planner, responder, critic, and tool-loop execution modes. +- Can use local, proxy, or OpenRouter-compatible LLM providers. +- Includes circuit breaking, health checks, and response validation. + +**5. Evaluation pipeline** + +- Captures golden and shadow evaluation cases. +- Runs rule, LLM, process, and handoff judges. +- Supports review queues, calibration, drift checks, promotion, and tournaments to guide prompt and routing improvements. + +**6. Background worker and alerts** + +- Processes shadow eval and archive jobs outside the real-time response path. +- Stores eval results in MongoDB and sends NATS/Slack alerts when needed. +- Collects cases that need operator review in the supervisor review queue. + +## Used technologies + +- TypeScript +- Node.js +- NATS +- Redis / Garnet +- MongoDB / Mongoose +- Infisical +- OpenTelemetry +- Pino +- Zod +- Vitest +- ESLint +- Prettier + +## How to run + +1. Install dependencies. + +```bash +npm install +``` + +2. Prepare environment variables. + +```bash +cp env-vars/env.example env-vars/dev.env +``` + +If `env-vars/env.example` is not available, create `env-vars/dev.env` using the [Environment variables](#environment-variables) section below. + +3. Run checks. + +```bash +npm run typecheck +npm test +``` + +4. Start the dev service. + +```bash +npm run start:dev +``` + +To run the agent and worker together with PM2: + +```bash +npm run pm2:dev +``` + +## Environment variables + +Infisical integration needs these variables. + +```env +INFISICAL_CLIENT_SECRET=your_client_secret +INFISICAL_CLIENT_ID=your_client_id +INFISICAL_PROJECT_ID=your_project_id +NODE_ENV=dev|test|prod +``` + +### Loading priority + +Environment variables are loaded in this order. Earlier values win. + +1. **LocalMachine global environment variables** + - Linux/macOS: `~/source/global-env-vars.txt` + - Windows: `%USERPROFILE%\source\global-env-vars.txt` + - Shared settings used across local projects. +2. **Project environment file** + - `./env-vars/{NODE_ENV}.env` + - Project-specific values such as `dev.env`, `test.env`, and `prod.env`. +3. **Shell environment variables** + - Temporary values for the current shell session. + +If the same key is defined in several places, the higher-priority value is used. The internal dotenv loader uses `override: false`, so `env-vars/{NODE_ENV}.env` does not overwrite values that already exist. + +### IMDS + +Infisical client id/secret can be provided directly as environment variables or fetched from an IMDS-compatible local metadata service. + +The default candidates are `http://127.0.0.1:8675` in development and `http://169.254.169.254` in production. Override them with `METADATA_BASE_URL`, `IMDS_BASE_URL`, or `MACHINE_ENV_BASE_URL`; disable IMDS with `IMDS_DISABLED=true`. + +### Infisical Secrets + +These secrets should exist in Infisical. + +| Secret | Description | +| --------------------------- | ---------------------------- | +| `NATS_ENDPOINT` | NATS server endpoint | +| `NATS_USER` | NATS username | +| `NATS_PASS` | NATS password | +| `GARNET_ENDPOINT` | Garnet/Redis endpoint | +| `GARNET_USERID` | Garnet/Redis username | +| `GARNET_PASSWORD` | Garnet/Redis password | +| `MONGODB_CONNECTION_STRING` | MongoDB connection string | +| `MONGODB_PROJECT_ID` | MongoDB project id | +| `SLACK_ALERT_WEBHOOK_URL` | Optional Slack alert webhook | + +Deployments that also use MinIO should keep these in Infisical as well. + +| Secret | Description | +| ------------------ | --------------------- | +| `MINIO_ENDPOINT` | MinIO server endpoint | +| `MINIO_ACCESS_KEY` | MinIO access key | +| `MINIO_SECRET_KEY` | MinIO secret key | + ## Useful commands ```bash npm run build +npm run typecheck npm run lint -npm run eval -npm run eval:report -npm run eval:review +npm test +npm run eval -- --skip-llm-judge +npm run eval:moderation -- --skip-llm-judge +npm run eval:handoff -- --skip-llm-judge +npm run eval:report -- --worst=20 --since=7d +npm run eval:router-shadow-report -- --since=7d +npm run review:list -- --limit=10 ``` -The service is designed to run as two processes in production: the real-time -agent (`src/main.ts`) and the background worker (`src/workers/background.ts`). +In production, run both the real-time agent (`src/main.ts`) and the background worker (`src/workers/background.ts`). + +## Documentation + +- [docs/multi-agent-architecture.md](docs/multi-agent-architecture.md): full architecture +- [docs/service-evaluation-operations.md](docs/service-evaluation-operations.md): operations runbook +- [docs/optimization-roadmap.md](docs/optimization-roadmap.md): optimization triggers +- [docs/resolution-hash-sunset-procedure.md](docs/resolution-hash-sunset-procedure.md): supervisor resolution hash cleanup procedure