Wolfenstein Modloader Buy me a coffeeKaffee spendieren

Wiki / Research briefsForschungsaufträge

Research brief: Idclwriter

OpenOffen Original page in the GitHub wikiOriginalseite im GitHub-Wiki

In plain wordsIn einfachen Worten

This is the task description given to a research worker for Wolfenstein II (TNC). It asks for a tool that writes the four hash fingerprints of game archives, which were already worked out but not yet written by any code. The tool must prove itself by rewriting original archives identically and by repairing four wrong fingerprints in a custom map bundle.

  • The bundle had 7,123 of 7,127 correct entries; the four wrong ones were the newly made files.
  • With a certain checking setting on, a wrong fingerprint is a fatal error, so this is a hidden crash risk.
  • Required: a byte-identical round-trip on at least three original archives of different sizes.
  • Required: negative tests, such as a flipped byte or hashes written in the wrong order, must be caught.
  • Rules: no game launch, no new compressor, and no changes to the other team's files.

Dies ist die Aufgabenbeschreibung für einen Forschungs-Worker zu Wolfenstein II (TNC). Gefordert ist ein Werkzeug, das die vier Hash-Fingerabdrücke der Spielarchive schreibt, die schon hergeleitet, aber noch von keinem Code geschrieben wurden. Das Werkzeug muss sich beweisen, indem es Original-Archive identisch neu schreibt und vier falsche Fingerabdrücke in einem eigenen Karten-Paket repariert.

  • Das Paket hatte 7.123 von 7.127 korrekten Einträgen; die vier falschen waren die neu erstellten Dateien.
  • Bei aktivierter Prüfeinstellung ist ein falscher Fingerabdruck ein fataler Fehler, also ein verstecktes Absturzrisiko.
  • Gefordert: ein byte-gleicher Roundtrip bei mindestens drei Original-Archiven unterschiedlicher Größe.
  • Gefordert: Negativtests, etwa ein gekipptes Byte oder Hashes in falscher Reihenfolge, müssen erkannt werden.
  • Regeln: kein Spielstart, kein neuer Kompressor und keine Änderungen an Dateien des anderen Teams.

Technical detailsTechnische Details

Der technische Teil ist auf Englisch – so, wie er im Wiki steht. Unterstrichene Begriffe haben eine Erklärung auf Deutsch.

Worker w-idclwriter-a1; task idclwriter; implement. Read .agent-cache/WORKER.md. Bus: python "%USERPROFILE%/.claude/skills/agent-cache-orchestrator/scripts/acbus.py" Parallel: w-texdbkey-a1, w-cvartable-a1, w-savestructure-a1. The other master (codex/Astra) is at its host session limit and is not running.

Objective

Turn today's five solved hashes into a working writer, and prove it on a real defect.

Everything a self-built TNC container needs is now derived and verified:

entry +0x50 MurmurHash64B(decompressed payload, len = usize@0x48, seed 0xDEADBEEF) -> tools/probe_datahash.py::murmur64b entry +0x60 == +0x50 for every type except image/extkisclule/cfile/compfile; no reader recomputes it, but the same value must go into dep+0x18 of every dependency record referencing the entry header +0x20 FarmHash64(metadata from +0x78 through the trailing IDCL marker) header +0x08 FarmHash64(header[0x10:0x78]) + 3-round kMul finalizer mixed with the version -> tools/probe_idclhash.py

Nothing in the repository writes them yet. wolfsdk/idcl.py reads only.

The concrete defect to fix

Master verified the other master's bundle read-only: re_probes/map_alias_v3/bundle_headerfixed/base/chunk_22.resources (151,702,019 bytes, 7128 entries). 7123/7127 entries carry a correct +0x50. Exactly four do not - the four whose payloads were authored or newly created:

compfile:maps/game/dlc/c02/c2v1_custom.entities usize=256319 comp=0 extkisclule:generated/decls/maps/game/dlc/c02/c2v1_custom_gameplay/ kiscules/sacramento_gameplay_kiscule_combat_sim_screen_fx_893872923.decl usize=1643 comp=0 json:maps/game/dlc/c02/c2v1_custom/devmapinfo.json usize=304 comp=0 file:generated/buildgame/game/dlc/c02/c2v1_custom.mapresources usize=2150169 comp=2

They kept the hash of the original payload. With cvar resourceStorage_checkDataCheckSum at its default 0 this is never checked, so the bundle loads; set to 1 it is a FatalError, not a warning. Latent crash.

Reproduce the finding first, before changing anything: python tools/probe_datahash.py re_probes/map_alias_v3/bundle_headerfixed/base/chunk_22.resources

Ownership

wolfsdk/idclwrite.py (new - the writer), tools/probe_idclwriter.py (new - derivation checks and the repair driver), re_probes/idclwriter/ (new dir, ALL output including the corrected container), re_probes/agent_reports/idclwriter.md.

Do NOT modify re_probes/map_alias_v3/ in any way. It belongs to the other master and is its only copy. Read it, write your corrected container into re_probes/idclwriter/. Copying 151 MB is acceptable; destroying someone else's artifact is not. wolfsdk/idcl.py is read-only for you - import it, do not edit it. Its docstring already documents all four derivations; keep your writer consistent with it.

Forbidden / do not touch

No game launch, no process attach, no writes under the Steam install or "%USERPROFILE%/Saved Games". Do not touch tools/probe_idclhash.py, tools/probe_datahash.py, tools/probe_cmdtable.py, tools/probe_saveverify.py, tools/probe_texdbkey.py, tools/probe_cvartable.py, tools/tnc_.py, tools/build_tnc_map_alias_v3.py, tools/verify_tnc_map_alias_v3.py, tools/vtpixel.py, re_probes/{map_alias_v3,custom_save_candidates,idclhash,datahash,cmdtable,saveverify,texdbkey,cvartable,vtrest}/, docs/, config.json. No pip install.

Hard constraints

Python 3.11, stdlib only (the repository has no third-party dependencies and must keep it that way - see README). Oodle/Kraken decompression already exists in wolfsdk/oodle.py; reuse it. Do not add a compressor: for a changed payload, store it uncompressed if you cannot reproduce the original codec exactly, and say so in the report.

Acceptance criteria

  1. wolfsdk/idclwrite.py recomputes all four hashes in the correct order: entry table finalized -> +0x20 -> +0x08 over the finished header.
  2. Round-trip control: reading a retail container and writing it back unchanged must produce a byte-identical file. Do this for at least three retail archives of different sizes. If it is not byte-identical, the writer is wrong - find out why before going further.
  3. The repaired container in re_probes/idclwriter/ passes python tools/probe_datahash.py <that file> with 7127/7127 (one entry is compression mode 4 and has no decoder - it stays skipped), and python tools/probe_idclhash.py <that file> with +0x08 and +0x20 OK.
  4. Every payload except the four repaired entries must be byte-identical to the source container. Prove it, do not assert it.
  5. Negative controls, each must FAIL: flip one payload byte without recomputing and show the checker catches it; write the header hashes in the wrong order (+0x08 before +0x20) and show the result is rejected; skip the dep+0x18 update for a changed +0x60 and show the stale check would fire.

Verification expectations

The round-trip in (2) is the load-bearing check: a writer that cannot reproduce a retail file exactly has no business writing a modified one. Run it first. State sample sizes and counts everywhere; today's standard on this project is n/n with named negative controls.

Failure honesty

If a payload's original compression cannot be reproduced and storing it uncompressed changes the container layout in a way you cannot verify, say so and stop - a half-verified 151 MB container is worse than none. Report what blocked you with offsets.

Coordination / deliverable

Checkpoint at milestones (writer round-trips; four entries repaired; controls pass). Publish a bus fact when the writer round-trips byte-exactly - that is the moment self-built containers become real rather than theoretical. Concise German report in re_probes/agent_reports/idclwriter.md. Finish the bus task.