docs(01): add validation strategy
This commit is contained in:
124
.planning/phases/01-level-up-pf2e-regelkonform/01-VALIDATION.md
Normal file
124
.planning/phases/01-level-up-pf2e-regelkonform/01-VALIDATION.md
Normal file
@@ -0,0 +1,124 @@
|
||||
---
|
||||
phase: 1
|
||||
slug: level-up-pf2e-regelkonform
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: 2026-04-27
|
||||
---
|
||||
|
||||
# Phase 1 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution.
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | Jest 30.0.0 + ts-jest (server only — `server/package.json:88-104`) |
|
||||
| **Config file** | inline in `server/package.json` — `testRegex: ".*\\.spec\\.ts$"`, `rootDir: "src"` |
|
||||
| **Quick run command** | `cd server && npm test -- --testPathPattern=leveling` |
|
||||
| **Full suite command** | `cd server && npm test` |
|
||||
| **Estimated runtime** | ~10s quick / ~30s full |
|
||||
|
||||
**Client test framework:** None today. Phase 1 deliberately does NOT introduce vitest on the client — wizard UI is verified manually + via the integration test that exercises the full commit path through the API.
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** Run `cd server && npm test -- --testPathPattern=leveling`
|
||||
- **After every plan wave:** Run `cd server && npm test`
|
||||
- **Before `/gsd-verify-work`:** Full suite must be green; integration test for atomic commit MUST pass
|
||||
- **Max feedback latency:** ~10 seconds (quick) / ~30 seconds (full)
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
||||
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
||||
| 1-W0-01 | W0 | 0 | infra | — | N/A | unit | `cd server && npm test -- apply-attribute-boost.spec.ts` | ❌ W0 | ⬜ pending |
|
||||
| 1-W1-01 | W1 | 1 | LVL-02 | — | `applyAttributeBoost(17) = 19` | unit | `cd server && npm test -- apply-attribute-boost.spec.ts` | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-02 | W1 | 1 | LVL-02 | — | `applyAttributeBoost(18) = 19` (cap) | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-03 | W1 | 1 | LVL-02 | — | `applyAttributeBoost(20) = 21` (above cap) | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-04 | W1 | 1 | LVL-02 | — | `isValidBoostSet(['STR','STR','CON','INT']) = false` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-05 | W1 | 1 | LVL-02 | — | `isValidBoostSet(['STR','DEX','CON','INT']) = true` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-06 | W1 | 1 | LVL-06 | — | `canIncreaseSkill('TRAINED', 2) = false` | unit | `cd server && npm test -- skill-increase-cap.spec.ts` | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-07 | W1 | 1 | LVL-06 | — | `canIncreaseSkill('TRAINED', 3) = true` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-08 | W1 | 1 | LVL-06 | — | `canIncreaseSkill('EXPERT', 6) = false` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-09 | W1 | 1 | LVL-06 | — | `canIncreaseSkill('EXPERT', 7) = true` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-10 | W1 | 1 | LVL-06 | — | `canIncreaseSkill('MASTER', 14) = false` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-11 | W1 | 1 | LVL-06 | — | `canIncreaseSkill('MASTER', 15) = true` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-12 | W1 | 1 | LVL-09 | T-1-Tampering-prereq | Pure skill-rank evaluates `{ ok: true }` when met | unit | `cd server && npm test -- prereq-evaluator.spec.ts` | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-13 | W1 | 1 | LVL-09 | T-1-Tampering-prereq | Same prereq, untrained → `{ ok: false }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-14 | W1 | 1 | LVL-09 | — | Disjunctive prereq with one match → `{ ok: true }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-15 | W1 | 1 | LVL-09 | — | Conjunctive `; …` with one missing → `{ ok: false }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-16 | W1 | 1 | LVL-09 | — | Bare feat-name with feat present → `{ ok: true }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-17 | W1 | 1 | LVL-09 | — | Heritage ref with matching heritage → `{ ok: true }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-18 | W1 | 1 | LVL-09 | — | Class ref → `{ ok: true }` when class matches | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-19 | W1 | 1 | LVL-09 | — | Spellcasting ref → `{ unknown: true, raw: ... }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-20 | W1 | 1 | LVL-09 | — | Deity ref → `{ unknown: true, raw: ... }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-21 | W1 | 1 | LVL-09 | — | Empty/null prereq → `{ ok: true }` | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-22 | W1 | 1 | LVL-10 | — | `recompute().hpMax = ancestryHP + (classHP + conMod) × level` | unit | `cd server && npm test -- recompute-derived-stats.spec.ts` | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-23 | W1 | 1 | LVL-10 | — | Recompute respects boost-cap-at-18 | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-24 | W1 | 1 | LVL-10 | — | Recompute applies `proficiencyChanges` from ClassProgression | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-25 | W1 | 1 | LVL-10 | — | Recompute does NOT mutate `hpCurrent` (Pitfall #9) | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-26 | W1 | 1 | LVL-01 | — | `computeApplicableSteps(L=5, Fighter, FA=false, isCaster=false)` includes boost+skill+feat-class+feat-skill+feat-ancestry | unit | `cd server && npm test -- compute-applicable-steps.spec.ts` | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-27 | W1 | 1 | LVL-01 | — | At L4 no boost step | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-28 | W1 | 1 | LVL-01,LVL-13 | — | With FA enabled, includes feat-archetype step | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W1-29 | W1 | 1 | LVL-01,LVL-14 | — | With caster class includes spellcaster step | unit | same | ❌ W1 | ⬜ pending |
|
||||
| 1-W2-01 | W2 | 2 | LVL-08 | — | Seed populates `ClassProgression` for 16 classes × 20 levels (≥320 rows) | manual | `cd server && npm run db:seed:class-progression && psql -c 'SELECT className, COUNT(*) FROM "ClassProgression" GROUP BY className'` | N/A | ⬜ pending |
|
||||
| 1-W3-01 | W3 | 3 | LVL-12 | T-1-Tampering-commit | Commit transaction is atomic — mid-tx throw rolls back ALL writes | integration | `cd server && npm test -- leveling.service.spec.ts` | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-02 | W3 | 3 | LVL-12 | — | Commit creates one `LevelUpHistory` row with snapshot + choices | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-03 | W3 | 3 | LVL-12 | T-1-WS-injection | Commit broadcasts `level_up_committed` exactly once (mock gateway) | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-04 | W3 | 3 | LVL-11 | — | DELETE `/level-up/:sessionId` removes DRAFT, leaves character untouched | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-05 | W3 | 3 | LVL-11 | T-1-Race-double-commit | Partial unique index allows new DRAFT after previous committed | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-06 | W3 | 3 | LVL-14 | — | Commit applies `spellSlotIncrement` for casters | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-07 | W3 | 3 | LVL-14 | — | Commit does NOT add slots for non-casters | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-08 | W3 | 3 | LVL-15 | — | Translation pipeline call hits existing `TranslationsService.getTranslationsBatch` | integration | same | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-09 | W3 | 3 | LVL-03,LVL-04,LVL-05,LVL-07 | — | `feat-filter.service.ts` returns only `{ok:true}`/`{unknown:true}` feats, respects source filters | integration | `cd server && npm test -- feat-filter.service.spec.ts` | ❌ W3 | ⬜ pending |
|
||||
| 1-W3-10 | W3 | 3 | LVL-04 | T-1-Access | Endpoints invoke `checkCharacterAccess(..., requireOwnership=true)` (Owner OR GM) | integration | `cd server && npm test -- leveling.controller.spec.ts` | ❌ W3 | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
The codebase has Jest infrastructure but zero unit-test files alongside source. Wave 0 establishes the file convention and proves the runner works on a leveling-module test:
|
||||
|
||||
- [ ] Create `server/src/modules/leveling/lib/apply-attribute-boost.ts` + `.spec.ts` (smallest possible — one function, four tests). Run `npm test`. Confirms ts-jest compiles, the testRegex picks it up, the run completes.
|
||||
- [ ] Add `server/jest.config.cjs` ONLY if the inline `package.json` Jest config has surprises with the new test files. Default: trust the inline config.
|
||||
- [ ] Add `db:seed:class-progression` script to `server/package.json` scripts pointing at `tsx prisma/seed-class-progression.ts`.
|
||||
- [ ] Add `.gitignore` entry for `server/prisma/data/foundry-pf2e/` (the dev-cloned source).
|
||||
- [ ] Add unit-test pattern documentation to `.planning/codebase/TESTING.md` once the Wave-1 modules land — first real pattern in the codebase.
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||||
|----------|-------------|------------|-------------------|
|
||||
| Wizard UI renders steps mobile-first per `01-UI-SPEC.md` | LVL-01, LVL-11 | No client test framework yet (deferred to a later cross-cutting phase) | Open `/characters/:id`, click "Stufe steigen", walk through wizard on Chrome DevTools mobile (375×667). Verify each step matches UI-SPEC. |
|
||||
| Pathbuilder import banner appears for prereq violations | LVL-09, D-06 | Requires real Pathbuilder JSON with violations | Import a prepared Pathbuilder JSON containing a feat whose prereq is unmet; verify the character-header banner lists the violations. |
|
||||
| Pathbuilder FA auto-detect (D-09) | LVL-13, A1 | Requires real FA-enabled Pathbuilder export to verify heuristic | Import a Pathbuilder character known to have Free Archetype enabled; verify `Character.freeArchetype = true` after import. NestJS log line `PathbuilderImport: detected FA=true for character X based on Y` appears. |
|
||||
| Real-time WebSocket broadcast lands on a second client | LVL-12, LVL-14 | Multi-client coordination | Open two browser windows on the same character; commit a level-up in window 1; window 2 updates HP-Max + level + stats within 1s without reload. |
|
||||
| Free-Archetype slot shows multi-archetype talents after Dedication | LVL-13, D-07 | UI verification of correct filter behavior | Pick a Dedication in the FA slot at L2; advance to L4; FA slot at L4 should list talents from any archetype, not only the dedicated one. |
|
||||
| Spellcaster Repertoire-Increment step appears for spontaneous casters only | LVL-14, D-18 | Requires casting-class character | Level-up a Sorcerer (spontaneous): Repertoire step appears. Level-up a Cleric (prepared): Repertoire step does NOT appear. |
|
||||
| Translation cache hit on subsequent prereq display | LVL-15 | Cache verification | Open a feat with German prereq text; close; reopen — second open hits cached translation, no Claude API call (verify via NestJS log). |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
||||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 30s
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Reference in New Issue
Block a user