m2pack-secure/docs/testing.md

# Testing

## Linux headless

`m2pack-secure` can be validated headless on a Linux VPS without GUI support.

This covers the archive toolchain and release artifacts, not the full Windows
client runtime.

Run:

```bash
cmake -S . -B build
cmake --build build -j
python3 scripts/headless_e2e.py
```

Or through npm:

```bash
npm run headless:e2e
```

The headless E2E test covers:

- `keygen`
- `build`
- `list`
- `verify`
- `diff`
- `export-client-config`
- `export-runtime-key` in `json`
- `export-runtime-key` in `blob`
- `extract`
- byte-for-byte comparison of extracted files

## What it does not cover

- Windows SDK compilation of the client
- Direct3D startup
- shared-memory launcher bootstrap on Windows
- in-game runtime behavior of the client loader

## Full client runtime

To test the actual client runtime you still need one of these:

- a Windows build machine
- a Windows VM
- or a Linux host with `wine` plus the required runtime dependencies

The Linux VPS path is now validated as well.

Confirmed runtime path on this host:

- Linux-hosted Windows build of `m2dev-client-src`
- headless execution through `xvfb-run + wine`
- runtime key delivery through:
  - `M2PACK_MASTER_KEY_HEX`
  - `M2PACK_KEY_ID`

Confirmed `.m2p` smoke tests:

- `root.m2p` only, with `root.pck` removed
- `root.m2p` + `patch1.m2p`, with both legacy `.pck` files removed
- `root.m2p` + `patch1.m2p` + `season3_eu.m2p`, with all three legacy `.pck`
  files removed
- `root.m2p` + `patch1.m2p` + `patch2.m2p` + `season3_eu.m2p`, with all four
  legacy `.pck` files removed
- `root.m2p` + `patch1.m2p` + `patch2.m2p` + `season3_eu.m2p` +
  `metin2_patch_snow.m2p` + `metin2_patch_snow_dungeon.m2p` +
  `metin2_patch_etc_costume1.m2p` + `metin2_patch_pet1.m2p`, with all eight
  matching legacy `.pck` files removed
- `root.m2p` + `patch1.m2p` + `patch2.m2p` + `season3_eu.m2p` +
  `metin2_patch_snow.m2p` + `metin2_patch_snow_dungeon.m2p` +
  `metin2_patch_etc_costume1.m2p` + `metin2_patch_pet1.m2p` +
  `metin2_patch_pet2.m2p` + `metin2_patch_ramadan_costume.m2p` +
  `metin2_patch_flame.m2p` + `metin2_patch_flame_dungeon.m2p`, with all
  twelve matching legacy `.pck` files removed
- the same twelve-pack startup set plus:
  - `locale.m2p`
  - `uiscript.m2p`
  - `uiloading.m2p`
  - `ETC.m2p`
  with all sixteen matching legacy `.pck` files removed
- the same sixteen-pack startup and UI bootstrap set plus:
  - `item.m2p`
  - `effect.m2p`
  - `icon.m2p`
  - `property.m2p`
  with all twenty matching legacy `.pck` files removed
- the same twenty-pack set plus:
  - `terrain.m2p`
  - `tree.m2p`
  - `zone.m2p`
  - `outdoora1.m2p`
  - `outdoora2.m2p`
  - `outdoorb1.m2p`
  - `outdoorc1.m2p`
  - `outdoorsnow1.m2p`
  with all twenty-eight matching legacy `.pck` files removed
- the same twenty-eight-pack set plus:
  - `pc.m2p`
  - `pc2.m2p`
  - `guild.m2p`
  - `npc.m2p`
  - `monster2.m2p`
  - `sound.m2p`
  - `sound_m.m2p`
  - `sound2.m2p`
  with all thirty-six matching legacy `.pck` files removed
- the same thirty-six-pack set plus:
  - `monster.m2p`
  - `npc2.m2p`
  - `textureset.m2p`
  - `outdoora3.m2p`
  - `outdoorb3.m2p`
  - `outdoorc3.m2p`
  - `outdoordesert1.m2p`
  - `outdoorflame1.m2p`
  - `outdoorfielddungeon1.m2p`
  with all forty-five matching legacy `.pck` files removed

In every confirmed case the client reached the login bootstrap path under Wine
and completed:

- `PackInitialize`
- Python bootstrap through `prototype.py`
- `app.Create(...)`
- `app.SetCamera(...)`
- `MainStream.SetLoginPhase()`

This is now enough to treat the `.m2p` path as validated for the current core
startup, UI bootstrap, and shared login-adjacent content pack set used by the
client runtime.

At this point the login-phase smoke test stops being a strong validator for
additional packs such as world, NPC, monster, and late-load gameplay content.
Those should be validated with map loads or in-game scenario coverage rather
than startup-only checks.

The world, audio, actor, and gameplay-adjacent packs listed above are therefore
validated only as
startup-time regression smoke coverage. They do prove that the additional
`.m2p` archives do not break bootstrap or early asset resolution, but they do
not yet prove full map streaming, actor loading, or gameplay correctness.

Current non-fatal runtime issues on the VPS:

- missing `Tahoma` font mapping in the client runtime
- headless ALSA / audio decoder warnings

Those do not currently block `.m2p` loading or the login-phase smoke test.

Example runtime command:

```bash
M2PACK_MASTER_KEY_HEX="$(cat /home/mt2.jakubkadlec.dev/.secrets/m2pack-secure/2026-04-14/master.key)" \
M2PACK_KEY_ID=1 \
M2_TIMEOUT=20 \
WINEDEBUG=-all \
/home/mt2.jakubkadlec.dev/metin/repos/m2dev-client-src/scripts/run-wine-headless.sh \
  /home/mt2.jakubkadlec.dev/metin/repos/m2dev-client-src/build-mingw64-lld/bin
```

Automated smoke runner for selected packs:

```bash
python3 scripts/runtime_smoke_wine.py \
  --runtime-root /tmp/m2dev-client-runtime-http \
  --client-repo /home/mt2.jakubkadlec.dev/metin/repos/m2dev-client-src \
  --master-key /home/mt2.jakubkadlec.dev/.secrets/m2pack-secure/2026-04-14/master.key \
  --key-id 1 \
  --pack root \
  --pack patch1 \
  --pack patch2 \
  --pack season3_eu \
  --pack locale \
  --pack uiscript \
  --pack uiloading \
  --pack ETC \
  --json
```

World/map scenario validator:

```bash
python3 scripts/validate_runtime_scenarios.py \
  --runtime-root /tmp/m2dev-client-runtime-http \
  --json
```

This validator checks real cross-pack world references:

- each selected `Outdoor*` map pack contains `setting.txt`
- each selected `Outdoor*` map pack contains `mapproperty.txt`
- `TextureSet` targets from `setting.txt` exist in `textureset`
- `Environment` targets from `setting.txt` exist in `ETC/ymir work/environment`

Actor/content scenario validator:

```bash
python3 scripts/validate_actor_scenarios.py \
  --runtime-root /tmp/m2dev-client-runtime-http \
  --json
```

This validator checks local actor integrity for `Monster`, `NPC`, and `PC`:

- `motlist.txt` motion files exist
- each motion resolves to a valid `.gr2`, including redirected `MotionFileName`
  targets in `.msa`
- `.msm` base model targets resolve against the runtime asset set
- `.msm` effect script targets resolve against the runtime asset set
- `.msm` default hit effect targets resolve against the runtime asset set
- `.msm` default hit sound targets resolve against the runtime asset set

Effect graph validator:

```bash
python3 scripts/validate_effect_scenarios.py \
  --runtime-root /tmp/m2dev-client-runtime-http \
  --json
```

This validator checks text-based effect assets in `Effect`:

- `.mse` particle `TextureFiles`
- `.mse` mesh `meshfilename`
- `.msf` `BombEffect`
- `.msf` `AttachFile`
- derived `.mss` sound scripts and their referenced `.wav` files

Runtime release gate:

```bash
python3 scripts/validate_runtime_gate.py \
  --runtime-root /tmp/m2dev-client-runtime-http
```

Strict runtime release gate:

```bash
python3 scripts/validate_runtime_gate.py \
  --runtime-root /tmp/m2dev-client-runtime-http \
  --strict-known-issues
```

The gate runs these validators together:

- `scripts/validate_runtime_scenarios.py`
- `scripts/validate_actor_scenarios.py`
- `scripts/validate_effect_scenarios.py`
- `scripts/validate_audio_scenarios.py`

By default they load the shared baseline:

- `known_issues/runtime_known_issues.json`

Result semantics:

- `known_issue_ids`: currently accepted historical content issues
- `unexpected_issue_ids`: new issues that fail the gate
- `stale_known_issue_ids`: baseline entries not observed anymore

Default behavior:

- known issues are reported but do not fail the gate
- only unexpected issues fail the gate

Strict behavior:

- unexpected issues fail the gate
- stale known-issue entries also fail the gate

Current baseline on the real runtime:

- `world`: `0`
- `actor`: `0`
- `effect`: `10`
- `audio`: `0`

The remaining ten effect baseline entries were checked against the current
`m2dev-client` git history and still have no matching source asset files. They
are currently treated as real content gaps rather than easy alias/path fixes.

Audio scenario validator:

```bash
python3 scripts/validate_audio_scenarios.py \
  --runtime-root /tmp/m2dev-client-runtime-http \
  --json
```

This validator checks the runtime audio script layer:

- all `*.mss` files under the real client runtime
- every `SoundDataNN` reference to `wav/mp3`
- resolution against the effective virtual audio namespace used by the client

Current real-runtime findings no longer show any audio content issues. The
audio validator is now expected to pass cleanly on the current runtime.

Built-in CI:

- `.gitea/workflows/ci.yml` builds `m2pack` on Linux
- the CI workflow runs `scripts/headless_e2e.py`
- the real client runtime gate remains a separate step because it depends on
  external client assets not stored in this repository

Self-hosted runtime CI:

- `.gitea/workflows/runtime-self-hosted.yml`
- `scripts/self_hosted_runtime_ci.py`

This path is meant for a dedicated runner that already has:

- a real client runtime checkout
- a Linux-built `m2dev-client-src`
- Wine runtime dependencies
- access to the runtime master key file

Expected runner environment:

- `M2_RUNTIME_ROOT`
- `M2_CLIENT_REPO`
- `M2_MASTER_KEY_PATH`
- `M2_KEY_ID`
- optional `M2_RUNTIME_SMOKE_PACKS`
- optional `M2_TIMEOUT`

The self-hosted orchestrator runs, in order:

1. `scripts/headless_e2e.py`
2. `scripts/validate_runtime_gate.py`
3. `scripts/runtime_smoke_wine.py`

Default Wine smoke coverage uses the known startup-safe `.m2p` pack set:

- `root`
- `patch1`
- `patch2`
- `season3_eu`
- `metin2_patch_snow`
- `metin2_patch_snow_dungeon`
- `metin2_patch_etc_costume1`
- `metin2_patch_pet1`
- `metin2_patch_pet2`
- `metin2_patch_ramadan_costume`
- `metin2_patch_flame`
- `metin2_patch_flame_dungeon`
- `locale`
- `uiscript`
- `uiloading`
- `ETC`
- `item`
- `effect`
- `icon`
- `property`