Move onboarding intents, prompt data, and tools behind skill-local sources. Register skills through factories and add a moderation skill PoC.
lolympus-agent
Overview
lolympus-agent is an LLM-based onboarding agent for the lolympus gaming community. It receives onboarding events, checks the user's current state, and replies with a short helpful message through the community's backend messaging pipeline.
Purpose
I created this service to automate repetitive onboarding guidance while keeping the response quality close to a human community staff member. The agent can answer common questions, validate onboarding progress, and route more complex cases through LLM-powered reasoning and tool calls.
Functionalities
1. Onboarding responses:
* Handles new member, nickname, photo, progress, timeout, and completion events
* Generates concise responses for user messages
* Falls back to deterministic templates when an LLM response is not needed
2. Nickname and progress checks:
* Validates community nickname format
* Checks onboarding session progress from Redis/Garnet
* Looks up room context when MongoDB is configured
3. Tool-aware LLM runtime:
* Supports planner, responder, critic, and tool-loop execution modes
* Routes requests across local, proxy, or OpenRouter-compatible LLM providers
* Uses circuit breaking, health checks, and response validation
4. Evaluation pipeline:
* Captures golden and shadow evaluation cases
* Runs rule, LLM, process, attribution, and PRM-style judges
* Supports review queues, calibration, drift checks, promotion, and tournaments
5. Background worker:
* Processes shadow evaluation jobs asynchronously
* Archives evaluation results and publishes alerts
* Keeps real-time responses separate from slower analysis work
Used technologies
* TypeScript
* Node.js
* NATS
* Redis / Garnet
* MongoDB / Mongoose
* OpenTelemetry
* Pino
* Zod
* Vitest
* ESLint
* Prettier
How to run
- Install dependencies:
npm install
- Create a local environment file:
cp env-vars/env.example env-vars/dev.env
-
Fill in the required NATS, Redis/Garnet, MongoDB, and LLM provider values in
env-vars/dev.env. -
Run checks:
npm run typecheck
npm test
- Start the agent:
npm run start:dev
Useful commands
npm run build
npm run lint
npm run eval
npm run eval:report
npm run eval:review
The service is designed to run as two processes in production: the real-time
agent (src/main.ts) and the background worker (src/workers/background.ts).