Add runtime validation gate and known issues baseline
This commit is contained in:
@@ -229,3 +229,50 @@ This validator checks text-based effect assets in `Effect`:
|
||||
- `.msf` `BombEffect`
|
||||
- `.msf` `AttachFile`
|
||||
- derived `.mss` sound scripts and their referenced `.wav` files
|
||||
|
||||
Runtime release gate:
|
||||
|
||||
```bash
|
||||
python3 scripts/validate_runtime_gate.py \
|
||||
--runtime-root /tmp/m2dev-client-runtime-http
|
||||
```
|
||||
|
||||
Strict runtime release gate:
|
||||
|
||||
```bash
|
||||
python3 scripts/validate_runtime_gate.py \
|
||||
--runtime-root /tmp/m2dev-client-runtime-http \
|
||||
--strict-known-issues
|
||||
```
|
||||
|
||||
The gate runs these validators together:
|
||||
|
||||
- `scripts/validate_runtime_scenarios.py`
|
||||
- `scripts/validate_actor_scenarios.py`
|
||||
- `scripts/validate_effect_scenarios.py`
|
||||
|
||||
By default they load the shared baseline:
|
||||
|
||||
- `known_issues/runtime_known_issues.json`
|
||||
|
||||
Result semantics:
|
||||
|
||||
- `known_issue_ids`: currently accepted historical content issues
|
||||
- `unexpected_issue_ids`: new issues that fail the gate
|
||||
- `stale_known_issue_ids`: baseline entries not observed anymore
|
||||
|
||||
Default behavior:
|
||||
|
||||
- known issues are reported but do not fail the gate
|
||||
- only unexpected issues fail the gate
|
||||
|
||||
Strict behavior:
|
||||
|
||||
- unexpected issues fail the gate
|
||||
- stale known-issue entries also fail the gate
|
||||
|
||||
Current baseline on the real runtime:
|
||||
|
||||
- `world`: `0`
|
||||
- `actor`: `5`
|
||||
- `effect`: `12`
|
||||
|
||||
Reference in New Issue
Block a user