Wolfenstein Modloader Buy me a coffeeKaffee spendieren

Wiki / Research briefsForschungsaufträge

Research brief: Matpipe

OpenOffen Original page in the GitHub wikiOriginalseite im GitHub-Wiki

In plain wordsIn einfachen Worten

This is the assignment given to a research worker, not a result. The goal is to show a texture made by the modder on a surface in a Wolfenstein II: The New Colossus map. The worker must decode how a material names its images, check whether game images can be read and written back unchanged, and build the full chain from image to texture database entry to material. Because the game is not started, the brief asks to report this only as 'structurally valid, rendering unconfirmed'.

  • Already solved beforehand: the lookup key of the texture database, correct for 25,930 of 25,930 entries.
  • Required: a material reader that rebuilds at least 200 retail materials byte for byte.
  • Required: an image round-trip for at least 200 retail images, checked against the texture database sizes.
  • Negative controls: unsorted rows, a wrongly masked key and a wrong mip count must all be caught.
  • If the pixel format needs an encoder the project does not have, the worker must say so instead of adding a dependency.

Dies ist der Auftrag an einen Forschungs-Worker, kein Ergebnis. Ziel ist, eine selbst gemachte Textur auf einer Fläche in einer Karte von Wolfenstein II: The New Colossus zu zeigen. Der Worker soll entschlüsseln, wie ein Material seine Bilder benennt, prüfen, ob Spielbilder sich lesen und unverändert zurückschreiben lassen, und die ganze Kette vom Bild über den Eintrag in der Texturdatenbank bis zum Material bauen. Da das Spiel nicht gestartet wird, soll das Ergebnis nur als 'strukturell gültig, Darstellung unbestätigt' gemeldet werden.

  • Vorher schon gelöst: der Suchschlüssel der Texturdatenbank, korrekt für 25.930 von 25.930 Einträgen.
  • Verlangt: ein Material-Leser, der mindestens 200 Original-Materialien byte-genau nachbaut.
  • Verlangt: ein Roundtrip für mindestens 200 Originalbilder, abgeglichen mit den Größen in der Texturdatenbank.
  • Gegenproben: unsortierte Zeilen, ein falsch maskierter Schlüssel und eine falsche Mip-Anzahl müssen erkannt werden.
  • Braucht das Pixelformat einen Encoder, den das Projekt nicht hat, soll der Worker das sagen, statt eine Abhängigkeit hinzuzufügen.

Technical detailsTechnische Details

Der technische Teil ist auf Englisch – so, wie er im Wiki steht. Unterstrichene Begriffe haben eine Erklärung auf Deutsch.

Worker w-matpipe-a1; task matpipe; research/implement. Read .agent-cache/WORKER.md. Bus: python "%USERPROFILE%/.claude/skills/agent-cache-orchestrator/scripts/acbus.py" Parallel: w-meshfmt-a1, w-cmfmt-a1, w-entitypipe-a1, w-mapmanifest-a1. You own surfaces: material -> image -> texdb, end to end.

Objective

Close the visual loop: put a texture we authored onto a surface in a TNC map. 1462 image and 388 material entries per map, and the lookup key that used to block this was solved today.

Answer and deliver:

  1. The material payload format - how a material names its images and which slot is the albedo/diffuse.
  2. The TNC image payload (.bimage, BIM\x0e): can we author one, and in which pixel formats? Round-trip a retail image first.
  3. The full chain: author an image, write the matching .texdb rows, bind it through a material, so a named surface in a map would use our pixels.

What is already solved - do not redo it

  • The TexDB key is solved and master-verified today: key = ((entry+0x60 & 0x0FFF_FFFF_FFFF_FFFF) << 4) | (mip_base - mip_level), 25930/25930 across all 28 .texdb. The asset NAME does not enter the key. Read re_probes/agent_reports/texdbkey.md in full before you start - it also decoded the .bimage mip table (header 0x32 bytes, then 32-byte records: level, 0, w, h, raw_size, streamed/inline, stored_size, offset), and it establishes that MountDatabase checks only the magic and that a new chunk_N.resources auto-mounts its chunk_N.texdb.
  • Crucially it also says: entry+0x60 for images is assigned by the build tool and no reader recomputes it, so we may choose it freely - and 570 retail hashes are shared between image names, i.e. the engine itself deduplicates. Reusing a retail key gives retail pixels; own pixels need own texdb rows, written SORTED or the binary search fails.
  • Container writing is finished: wolfsdk/idclwrite.py, byte-exact round-trip.
  • Part 1 image code exists and is read-only for you: wolfsdk/bim.py, wolfsdk/image.py (BIM decode/encode, PNG in/out), tools/probe_texdb.py, tools/verify_image.py. Part 1 is BIM of a different version - check, do not assume.

Sources / starting points

  • re_probes/runtime_87400/module.bin - read-only capture of the RUNNING, unpacked executable (file offset == RVA, sections in manifest.json). The image/material loaders are in there; today's five wins all came from reading loaders rather than sweeping data.
  • docs/texdb.md, docs/virtualtextures.md, docs/ui-assets.md (part 1 image replacement already works there, image-export/image-import).
  • Retail corpus READ ONLY: /steamapps/common/Wolfenstein.II.The.New.Colossus
  • capstone 5.0.7 vendored: PYTHONPATH=re_probes/python_deps

Ownership

tools/probe_matpipe.py (new), re_probes/matpipe/ (new dir), re_probes/agent_reports/matpipe.md (new). Everything else read-only, including other workers' tools and dirs.

Forbidden / do not touch

No game launch, no process attach, no writes under the Steam install or "%USERPROFILE%/Saved Games". Do not edit wolfsdk/, docs/, config.json, tools/probe_texdbkey.py or any other worker's file. No pip install.

Hard constraints

Python 3.11, stdlib only (plus vendored capstone for disassembly). Reuse wolfsdk/oodle.py. No new image library - if a block-compressed format needs an encoder we do not have, say so rather than adding a dependency.

Acceptance criteria

  • material parser with a byte-exact round-trip, n/n over at least 200 retail materials across 3 archives.
  • image payload round-trip, n/n over at least 200 retail images, and the mip table cross-checked against the sizes the .texdb actually stores.
  • End-to-end demo: an authored image (a flat, unmistakable colour is ideal - it is obvious on screen and trivially verifiable) plus its texdb rows plus a material that points at it, all inside a container that passes probe_datahash n/n and probe_idclhash OK.
  • Negative controls: unsorted texdb rows, a key with the top nibble left unmasked, and a mip count that disagrees with the bimage header must each be caught by your checker.

Verification expectations

Nothing offline shows that a pixel reaches the screen. Say "structurally valid by these checks, rendering unconfirmed". Keep "the lookup will find our block" and "the GPU will decode it" strictly apart - the second depends on the pixel format, which is NOT yet established (texdbkey.md says so explicitly).

Failure honesty

If the pixel format turns out to be a block-compressed form we cannot encode without a new dependency, that is the answer: report which formats appear, with counts, and whether any uncompressed form is accepted. Part 1 has exactly this limitation documented - do not repeat its mistake of shipping an encoder whose output nobody ever looked at.

Coordination / deliverable

Checkpoint at milestones. Publish a bus fact when the image round-trip works - w-meshfmt-a1 needs materials for any authored geometry. Concise German report in re_probes/agent_reports/matpipe.md. Finish the bus task.