Wolfenstein Modloader Buy me a coffeeKaffee spendieren

Wiki / Research briefsForschungsaufträge

Research brief: Idclhash

OpenOffen Original page in the GitHub wikiOriginalseite im GitHub-Wiki

In plain wordsIn einfachen Worten

This page is a task description for Wolfenstein II: The New Colossus. The game's archive files (IDCL) have a value at position +0x08 in their header that nobody could explain, so every custom map bundle so far had to copy an original archive. The worker was asked to find where the game code reads this value and whether a self-built archive can produce a correct one. Earlier tries ruled out several common checksums; the brief demands proof from the code, not guessing via hash sweeps.

  • Field +0x08 was the last blocker for building a custom archive chunk.
  • Known: it is not a checksum of the metadata; two files share it while differing elsewhere.
  • Positive control required: re-derive the already understood field +0x20 for at least 3 archives.
  • Negative control: analysis against a shuffled or shifted image must fail.
  • Game launch and changes to the Steam installation were forbidden.

Diese Seite ist eine Aufgabenbeschreibung für Wolfenstein II: The New Colossus. Die Archivdateien des Spiels (IDCL) haben an Position +0x08 im Header einen Wert, den niemand erklären konnte, deshalb musste jedes bisherige eigene Kartenpaket ein Originalarchiv kopieren. Der Worker sollte finden, wo der Spielcode diesen Wert liest und ob ein selbst gebautes Archiv einen gültigen Wert erzeugen kann. Frühere Versuche hatten mehrere gängige Prüfsummen ausgeschlossen; der Auftrag verlangt Belege aus dem Code statt Raten per Hash-Durchlauf.

  • Feld +0x08 war die letzte Hürde für einen selbst gebauten Archiv-Chunk.
  • Bekannt: Es ist keine Prüfsumme der Metadaten; zwei Dateien teilen den Wert, obwohl sie sich anderswo unterscheiden.
  • Positivkontrolle gefordert: das bereits verstandene Feld +0x20 für mindestens 3 Archive nachrechnen.
  • Gegenprobe: Die Analyse gegen ein verschobenes oder gemischtes Abbild muss fehlschlagen.
  • Spielstart und Änderungen an der Steam-Installation waren verboten.

Technical detailsTechnische Details

Der technische Teil ist auf Englisch – so, wie er im Wiki steht. Unterstrichene Begriffe haben eine Erklärung auf Deutsch.

Worker w-idclhash-a1; task idclhash; research (read-only RE). Read .agent-cache/WORKER.md. Bus: python "%USERPROFILE%/.claude/skills/agent-cache-orchestrator/scripts/acbus.py" Parallel workers on other hosts: astra-map-v3 (map bundle), astra-save-v3 (save serialization), w-cmdtable-a1 (command table), w-saveverify-a1 (.verify crypto). w-cmdtable-a1 and w-saveverify-a1 read the same image file read-only. Stay in your lane.

Objective

Settle the IDCL header field +0x08 from CODE, not from hash sweeps. It is the last blocker for building a custom TNC archive chunk: without a correct +0x08 no self-built chunk can be trusted, which is why every custom map bundle so far had to copy or hardlink a retail chunk.

Answer three questions with evidence:

  1. Where does the engine READ +0x08, and what does it compare it against?
  2. Is the value computed from the container, supplied by the mount manifest (packagemapspec/dlcinfo), or a fixed package identity?
  3. Given that, can a self-built chunk produce an accepted +0x08? If yes, give the exact derivation and a working implementation. If no, name the input the engine expects and where it comes from.

What is already measured - do not contradict it without evidence

re_probes/agent_reports/hash08.md is the prior static attempt. Its key result is a DIFFERENTIAL COUNTEREXAMPLE, and it is solid: chunk_26_hash1 and chunk_26_oracle share the identical +0x08 (3533df45c7f9b1d2) while differing at +0x20 and in metadata bytes [0x698,0x69c). So +0x08 is NOT a checksum of the metadata. Excluded by that report: unseeded FarmHash64/CityHash64/XXH64, MD5/SHA truncations, filename/path spellings. NOT excluded: seeded/keyed variants, CRC64 forms, and a manifest/VFS-supplied identifier. +0x20 is known to be FarmHash over the metadata - use it as a positive control for your tooling, because you must be able to re-derive a value that is already understood before claiming anything about one that is not.

Sources / starting points

  • re_probes/runtime_87400/module.bin - 92,188,672 bytes, read-only capture of the RUNNING, SteamStub-unpacked NewColossus_x64vk.exe. Layout "virtual RVA": file offset == RVA. manifest.json lists all sections (.text rva 0x1000, .rdata rva 0x1E62000, .data rva 0x2E39000, .pdata rva 0x50E2000). Image base is in manifest.json; for an absolute pointer, RVA = VA - base. This is the decisive new capability: previous work only had the packed EXE.
  • .pdata is a RUNTIME_FUNCTION table - use it to enumerate function boundaries cheaply instead of linear-sweeping 31 MB of .text.
  • Entry point: the four-byte magic IDCL (0x4C434449) appears as an immediate in the header validation code. Find those compares, and you are inside the archive header parser; from there follow the loads at struct offsets +0x08 and +0x20. Cross-check with the known FarmHash call for +0x20.
  • capstone 5.0.7 is vendored: PYTHONPATH=re_probes/python_deps python ...
  • wolfsdk/idcl.py (our parser, read-only), re_probes/hash08_scratch/inspect.py (prior probe, read-only), docs/moddingflaeche-tnc.md.
  • Retail archives to test against, READ ONLY: /steamapps/common/Wolfenstein.II.The.New.Colossus

Ownership

tools/probe_idclhash.py (new), re_probes/idclhash/ (new dir), and re_probes/agent_reports/idclhash.md (new).

Forbidden / do not touch

No game launch, no process attach, no writes anywhere under the Steam install or "%USERPROFILE%/Saved Games". Do not touch files owned by others: tools/probe_cmdtable.py, re_probes/cmdtable/, tools/probe_saveverify.py, re_probes/saveverify/, tools/build_tnc_map_alias_v3.py, tools/verify_tnc_map_alias_v3.py, re_probes/map_alias_v3/, tools/build_custom_save_candidate.py, tools/probe_details_checksum.py, tools/verify_custom_save_candidate.py, re_probes/custom_save_candidates/, tools/vtpixel.py, docs/, wolfsdk/, config.json. No pip install.

Hard constraints

Python 3.11, stdlib plus vendored capstone. Reproducible from the command line, no running game required.

Acceptance criteria

  • A named function RVA in .text that provably touches header+0x08, with the disassembly excerpt that shows it and the path by which you reached it.
  • A statement of what the value IS, backed by that code.
  • Positive control: your tooling re-derives the already-understood +0x20 FarmHash for at least three distinct retail archives. If it cannot reproduce the KNOWN field, no claim about the unknown field is credible.
  • Negative control: the same analysis run against a wrong section base or a shuffled image must fail to produce the result, not produce a different plausible one.
  • If the answer is "supplied by the manifest", say exactly which file and field, and verify it against at least three retail chunks with different IDs.

Verification expectations

Every important assurance needs a check designed to FAIL if the assurance is false. A byte pattern that merely "looks like" a hash constant table is not evidence; a cross-reference from the IDCL validation path is. Do not promote a plausible candidate function to "the" function without an xref chain.

Failure honesty

A clean negative with exact offsets and a named obstacle is a good result and is more useful to the other workers than an optimistic guess. Report failed hypotheses explicitly. Do not repeat the brute-force hash sweep that hash08.md already exhausted - that path is closed, go through the code.

Coordination / deliverable

Checkpoint at milestones (header parser located; +0x08 read site found; controls passed). Publish a bus fact as soon as the verdict is known, because astra-map-v3 is building a chunk bundle and this decides whether a self-built chunk is possible at all. Concise German report in re_probes/agent_reports/idclhash.md. Finish the bus task when done.