.
This commit is contained in:
1 parent
da76af3826
commit
e4f7a5df48
72 files changed
+7077
-113
No files matched your search
+20
-11
@@ -848,25 +848,34 @@ pnpm run eval:report --slice tier=0
|
||||
## 12. 작업 체크리스트 (codex가 PR 단위로 진행)
|
||||
|
||||
### Phase α (Week 1-2)
|
||||
- [ ] PR-A1: TASK-α-2-01 (Hybrid Early Abstention) + TASK-α-FLAG-01 (RuntimeFlagManifest)
|
||||
- [ ] PR-A2: TASK-α-8a-01 (Structured Tool Error Recovery)
|
||||
- [ ] PR-A3: TASK-α-4a-01 (Programmatic Tool Calling 시범)
|
||||
- [x] PR-A1: TASK-α-2-01 (Hybrid Early Abstention) + TASK-α-FLAG-01 (RuntimeFlagManifest)
|
||||
- [x] PR-A2: TASK-α-8a-01 (Structured Tool Error Recovery)
|
||||
- [x] PR-A3: TASK-α-4a-01 (Programmatic Tool Calling 시범)
|
||||
- [ ] Phase α Exit 게이트 검증 (Section 5)
|
||||
|
||||
### Phase β (Week 3-5)
|
||||
- [ ] PR-B1: TASK-β-1-01 (SKILL.md L1/L2/L3) — *skill 추가 결정에 따라 최우선*
|
||||
- [ ] PR-B2: TASK-β-5-01 (PRM + Attribution)
|
||||
- [ ] PR-B3: TASK-β-3-01 (Router v1 룰)
|
||||
- [ ] PR-B4: TASK-β-6-01 (User Memory + Adaptive Few-shot)
|
||||
- [ ] PR-B5: TASK-β-3-02 (Router v2 학습) — *PRM 라벨 충분 시*
|
||||
- [x] PR-B1: TASK-β-1-01 (SKILL.md L1/L2/L3) — *skill 추가 결정에 따라 최우선*
|
||||
- [x] PR-B2: TASK-β-5-01 (PRM + Attribution)
|
||||
- [x] PR-B3: TASK-β-3-01 (Router v1 룰)
|
||||
- [x] PR-B4: TASK-β-6-01 (User Memory + Adaptive Few-shot)
|
||||
- [x] PR-B5: TASK-β-3-02 (Router v2 학습) — *PRM 라벨 충분 시*
|
||||
- [ ] Phase β Exit 게이트 검증
|
||||
|
||||
### Phase γ (Week 6+)
|
||||
- [ ] PR-C1: TASK-γ-7-01 (Trust & Governance)
|
||||
- [ ] PR-C2: TASK-γ-4b-01 (Tool Search Tool) — *skill/tool 증가 시점*
|
||||
- [ ] PR-C3: TASK-γ-8b-01 (Multi-step Self-Critic)
|
||||
- [x] PR-C1: TASK-γ-7-01 (Trust & Governance)
|
||||
- [x] PR-C2: TASK-γ-4b-01 (Tool Search Tool) — *skill/tool 증가 시점*
|
||||
- [x] PR-C3: TASK-γ-8b-01 (Multi-step Self-Critic)
|
||||
- [ ] Phase γ Exit 게이트 검증
|
||||
|
||||
### Verification Run Log
|
||||
- [x] 2026-05-13T06:35Z local static gates: `npm run typecheck`, `npm run lint`, `npm test` 통과 (43 files / 168 tests).
|
||||
- [x] 2026-05-13T06:02Z golden eval without LLM judge: passRate=1, ruleAverageScore=1, averageLatencyMs=1.4ms. Report: `eval-reports/2026-05-13T06-02-46-431Z.json`.
|
||||
- [x] 2026-05-13T06:04Z all-new-flags smoke eval without LLM judge: passRate=1, ruleAverageScore=1, averageLatencyMs=1.8ms. Flags: abstention, programmatic tool calling, SKILL.md mode, user memory, tool search, critic iterations=2, static router. Report: `eval-reports/2026-05-13T06-04-40-217Z.json`.
|
||||
- [x] 2026-05-13T06:33Z fixed OpenRouter `/api/v1` endpoint normalization to `/api/v1/chat/completions`; added `llm-client.test.ts`. `npm run typecheck`, `npm run lint`, targeted LLM client tests 통과.
|
||||
- [ ] 2026-05-13T06:03Z full golden eval with LLM judge: blocked/degraded by LLM endpoint health. cliproxy timed out, OpenRouter returned HTML instead of JSON, llama timed out intermittently. Report: `eval-reports/2026-05-13T06-03-54-205Z.json` (not accepted as a quality gate).
|
||||
- [ ] 2026-05-13T06:32Z full golden eval retry after URL fix: OpenRouter URL path is now correct, but gate is still blocked/degraded by provider health/model availability. cliproxy returned unknown-provider for the judge model, OpenRouter free models returned upstream 429 rate-limit, and many generation calls produced empty fallback responses. Report: `eval-reports/2026-05-13T06-32-45-317Z.json` (not accepted as a quality gate).
|
||||
- [ ] Aggregator/report/drift/tournament operational gates: blocked locally because `MONGODB_CONNECTION_STRING` is not available in current eval environment.
|
||||
|
||||
---
|
||||
|
||||
## 13. 비고
|
||||
|
||||
Reference in new issue
Block a user