This commit is contained in:
dahyeon committed 2026-05-14 14:28:59 -07:00
1 parent da76af3826
commit e4f7a5df48
72 files changed
+7077 -113

No files matched your search

+20 -11
View File
@@ -848,25 +848,34 @@ pnpm run eval:report --slice tier=0
## 12. 작업 체크리스트 (codex가 PR 단위로 진행)
### Phase α (Week 1-2)
- [ ] PR-A1: TASK-α-2-01 (Hybrid Early Abstention) + TASK-α-FLAG-01 (RuntimeFlagManifest)
- [ ] PR-A2: TASK-α-8a-01 (Structured Tool Error Recovery)
- [ ] PR-A3: TASK-α-4a-01 (Programmatic Tool Calling 시범)
- [x] PR-A1: TASK-α-2-01 (Hybrid Early Abstention) + TASK-α-FLAG-01 (RuntimeFlagManifest)
- [x] PR-A2: TASK-α-8a-01 (Structured Tool Error Recovery)
- [x] PR-A3: TASK-α-4a-01 (Programmatic Tool Calling 시범)
- [ ] Phase α Exit 게이트 검증 (Section 5)
### Phase β (Week 3-5)
- [ ] PR-B1: TASK-β-1-01 (SKILL.md L1/L2/L3) — *skill 추가 결정에 따라 최우선*
- [ ] PR-B2: TASK-β-5-01 (PRM + Attribution)
- [ ] PR-B3: TASK-β-3-01 (Router v1 룰)
- [ ] PR-B4: TASK-β-6-01 (User Memory + Adaptive Few-shot)
- [ ] PR-B5: TASK-β-3-02 (Router v2 학습) — *PRM 라벨 충분 시*
- [x] PR-B1: TASK-β-1-01 (SKILL.md L1/L2/L3) — *skill 추가 결정에 따라 최우선*
- [x] PR-B2: TASK-β-5-01 (PRM + Attribution)
- [x] PR-B3: TASK-β-3-01 (Router v1 룰)
- [x] PR-B4: TASK-β-6-01 (User Memory + Adaptive Few-shot)
- [x] PR-B5: TASK-β-3-02 (Router v2 학습) — *PRM 라벨 충분 시*
- [ ] Phase β Exit 게이트 검증
### Phase γ (Week 6+)
- [ ] PR-C1: TASK-γ-7-01 (Trust & Governance)
- [ ] PR-C2: TASK-γ-4b-01 (Tool Search Tool) — *skill/tool 증가 시점*
- [ ] PR-C3: TASK-γ-8b-01 (Multi-step Self-Critic)
- [x] PR-C1: TASK-γ-7-01 (Trust & Governance)
- [x] PR-C2: TASK-γ-4b-01 (Tool Search Tool) — *skill/tool 증가 시점*
- [x] PR-C3: TASK-γ-8b-01 (Multi-step Self-Critic)
- [ ] Phase γ Exit 게이트 검증
### Verification Run Log
- [x] 2026-05-13T06:35Z local static gates: `npm run typecheck`, `npm run lint`, `npm test` 통과 (43 files / 168 tests).
- [x] 2026-05-13T06:02Z golden eval without LLM judge: passRate=1, ruleAverageScore=1, averageLatencyMs=1.4ms. Report: `eval-reports/2026-05-13T06-02-46-431Z.json`.
- [x] 2026-05-13T06:04Z all-new-flags smoke eval without LLM judge: passRate=1, ruleAverageScore=1, averageLatencyMs=1.8ms. Flags: abstention, programmatic tool calling, SKILL.md mode, user memory, tool search, critic iterations=2, static router. Report: `eval-reports/2026-05-13T06-04-40-217Z.json`.
- [x] 2026-05-13T06:33Z fixed OpenRouter `/api/v1` endpoint normalization to `/api/v1/chat/completions`; added `llm-client.test.ts`. `npm run typecheck`, `npm run lint`, targeted LLM client tests 통과.
- [ ] 2026-05-13T06:03Z full golden eval with LLM judge: blocked/degraded by LLM endpoint health. cliproxy timed out, OpenRouter returned HTML instead of JSON, llama timed out intermittently. Report: `eval-reports/2026-05-13T06-03-54-205Z.json` (not accepted as a quality gate).
- [ ] 2026-05-13T06:32Z full golden eval retry after URL fix: OpenRouter URL path is now correct, but gate is still blocked/degraded by provider health/model availability. cliproxy returned unknown-provider for the judge model, OpenRouter free models returned upstream 429 rate-limit, and many generation calls produced empty fallback responses. Report: `eval-reports/2026-05-13T06-32-45-317Z.json` (not accepted as a quality gate).
- [ ] Aggregator/report/drift/tournament operational gates: blocked locally because `MONGODB_CONNECTION_STRING` is not available in current eval environment.
---
## 13. 비고