fix: harden result polling and Redis 8.10 operations

Preserve tournament result jobs through lock contention and transient failures,
atomically schedule retries before acknowledgement, and publish initial polls
after releasing the preparation lock.

Fix Redis connection blocking, durability Hash cleanup, duplicate auction
request ordering, and scheduled-job promotion. Use native Redis 8.10 stream,
lock, counter, and field-expiring cache operations with invalidation fencing.

Repair 90 test type errors and add a repeatable API typecheck command.

Validation: 704 API tests, 129 web tests, 31 real Redis integration tests,
3 dependency checks, workspace type checks and builds, and changed-file lint.
This commit is contained in:
dahyeon committed 2026-11-09 17:33:13 +09:00
1 parent 0c4c4239ec
commit 4b8ccf0077
49 files changed
+2214 -371

No files matched your search

+62
View File
@@ -0,0 +1,62 @@
# Redis 8.10 correctness and optimization
## Scope and invariants
- Fix blocking reads sharing a client, Hash payload cleanup, duplicate action-order entries, and non-atomic scheduled-job promotion.
- Preserve high-before-low consumption, the 30-minute pending recovery threshold, business processing locks, replay fingerprints, 500-request limits, and auction lifecycle fencing.
- Use XACKDEL ACKED, XREADGROUP CLAIM, conditional string deletion/renewal, and INCREX. Never trim unacknowledged work.
- Move community response caches to hashes with independent field TTLs: 300 seconds normally, 86400 for complete details, 60 for incomplete details. Preserve response shapes and in-flight coalescing. Fence cache fills across invalidation.
- Do not implement conditional Compact Hash, JSON, Array, or request-retention redesign proposals in this change.
## Implementation sequence
1. Add regression coverage and fix the four correctness defects.
2. Upgrade stream consumption/acknowledgement and protect pending entries during trimming.
3. Replace lock scripts and token-unsafe dedup deletion with Redis conditional commands.
4. Replace failure-counter increment/expiry with INCREX.
5. Introduce versioned community hash caches and generation-aware fills/invalidation.
## Validation
- Run existing API tests before changes and record baseline failures.
- Exercise actual Lua and Redis 8.10 commands in an isolated local Redis process, including concurrent promotion, pending recovery, foreign-group retention, stale lock ownership, lifecycle-specific cleanup, and cache invalidation races/field expiry.
- Add unit regressions for worker connection lifecycle, command response handling, caller cache scopes, and interrupted operations.
- Run the full API suite, API build, changed-file lint, and dependency checks. Distinguish mock-based tests from real Redis integration results.
## Environment
The repository's named workflow skills are absent. Work follows the above explicit specification, incremental implementation, regression tests, and final review. Windows has no Docker/WSL; a temporary loopback-only Redis 8.10.3 Windows community build is used for integration validation, without accessing application Redis credentials.
## Results (2026-11-09)
| Check | Result |
| --- | --- |
| API unit/regression suite | 105 suites, 704 tests passed |
| Web regression suite | 129 tests passed |
| Isolated Redis 8.10.3 integration suite | 2 suites, 31 tests passed |
| API, web, and contracts builds | Passed (`pnpm -r --workspace-concurrency=1 build`) |
| ESLint on the 20 repaired test files | Passed with zero warnings |
| Dependency regression checks | 3 tests passed |
| Integration runner syntax and missing-executable failure cleanup | Passed; no leaked temporary directory |
| Git whitespace check | Passed |
| Full API TypeScript check including test files | Passed; all 90 pre-existing diagnostics repaired |
| Contracts and web application TypeScript checks | Passed (web check runs as part of its build) |
The initial API baseline was 685 passing tests and one failure caused by Windows path separators in the performance report. The report now emits platform-independent paths and its existing test passes. The original Redis change exposed 90 pre-existing test-type diagnostics, confirmed against HEAD sources through an in-memory compiler host. The requested follow-up repairs all 90 across 20 test files. Workspace dependencies were installed from the frozen lockfile without changing package versions.
The type repairs preserve test assertions and scenarios: public service types are separated from private-method test accessors, partial database mocks are typed at the harness boundary, generic helpers retain their full element types, and Riot/Match fixtures provide required fields. Position fixtures use the domain's `SUP`/`JGL` values, optional lookup assertions still fail when the expected object is missing, and scheduler teardown is awaited. No tests were removed or skipped, no TypeScript settings were relaxed, and no suppression directives were added. The final suite also includes the concurrent tournament polling/retry changes present in the shared workspace.
Run `pnpm --filter api typecheck` to check API source and tests without emitting files or relying on incremental state. Run `pnpm -r --if-present typecheck`, `pnpm --filter api test -- --runInBand`, `pnpm --filter web test`, `pnpm --filter api test:redis`, `pnpm test:dependency-security`, and `pnpm -r --workspace-concurrency=1 build` to reproduce the workspace checks above. The web build reports a non-failing bundle-size warning.
Actual Redis coverage includes dedicated blocking reads alongside normal commands, stale pending recovery, acknowledgement retention across consumer groups, safe stream trimming, foreign-owner lock protection, compare-and-delete dedup tokens, concurrent failure-counter increments, scheduled-job failure/retry and concurrent promotion, rescheduling fences, malformed scheduled entries and batch limits, Hash/legacy payload cleanup, action reservation/apply/replay, the 500-request cap and five-minute retention, independent cache field expiry, scope isolation, and stale cache fills across invalidation. The detail-response invalidation race also exercises the real CommunityStatsService and Redis cache with only the underlying dataset loader stubbed.
MongoDB/NATS/Riot end-to-end execution, Linux production-runtime validation, and production-scale latency/memory benchmarks were not run. No measured production speedup is claimed.
## Rollout and operating behavior
- Requires Redis 8.10.x and the existing node-redis 5.12.x client. The integration runner verifies the server version before executing tests.
- Switch API instances together: old String-cache invalidation and new Hash-cache invalidation target different namespaces. Rolling with both versions serving traffic can leave the other version's cache stale. Rollback has the same constraint.
- Old cache values expire on their existing TTL, at most 24 hours. New hashes populate on cache misses; this can temporarily increase database reads after deployment. A persistent generation token per community fences slow responses across invalidation.
- Pending/unread stream entries are retained even past the approximate 10,000-entry target. Monitor stream length and pending entries; an abandoned consumer group can retain entries until that group is deliberately retired.
- Existing action-order duplicates are not rewritten during deployment. The fix prevents new duplicates; existing entries age out through the existing five-minute replay retention and count-based cleanup.
- `pnpm --filter api test:redis` starts only its own loopback server, uses a random port/password, disables persistence, and cleans up afterward. Set `REDIS_TEST_SERVER` to an installed executable if it is not on PATH.