TNO MegaTexture (virtualtextures/*.pages): Page Table, Codec, One Decoded Region, and a Round-Trip-Proven Writer

Round-6 research pass on The New Order's MegaTexture files: page table, mip structure, tile codec, the mapping from world UV to pages, one region decoded to a viewable image, and a writer. The writer is round-trip-proven on copies for two paths — an in-place replacement and a growth path that appends a new page record — for the specular/gloss and power channel blocks; the colour (diffuse) blocks still cannot be written. Proven on file copies only; how a patched .pages file reaches the running game is not addressed, and nothing was tested in game.

Track: tnomegatex. Game: Wolfenstein: The New Order (id Tech 5). Builds on docs/virtualtextures.md, docs/vt-planes.md, wolfsdk/tnomaptex.py, wolfsdk/tnomap.py, and the existing read-only tools tools/probe_vtpages.py, tools/vtdecode.py, tools/vtplanes.py, tools/vtpixel.py, tools/probe_tno_tex.py. All new code for this round lives under re_probes/tnomegatex/ (this track's own folder, incl. this pass's check_mapping.py, a set of read-only, re-runnable statistical checks over existing tools, added so §4's new claims are independently reproducible, not just pasted-in numbers); nothing shared was modified. No game was launched. The only writes happened to throwaway copies under D:\wolf-decomp\tmp\tnomegatex\, deleted again at the end of every writer run (see Evidence for how to reproduce and verify this yourself).

Pass 1 errors (second pass)

The verifier refuted one claim and flagged a weaker overstatement; both are fixed below, and every "missed" item it listed was followed up (one of them turned into this round's biggest new result):

1. Refuted, fixed: pass 1 said the primary per-map megatexture's .pages file "could not even be reached" because landDefinitionFile's .land target has zero archive hits. True but beside the point -- the verifier found the decl sitting right next to it, landPageFile "c01p1", which names the .pages file directly, and pass 1 had already quoted the same decl block while only showing part of it. §4 now starts from landPageFile, confirms it live across all 25 TNO maps (not 4), and goes substantially further: see point 2.
2. New this round, not just a fix: following the "missed" item up found that landDimensions x 8 == pagesAcross exactly, zero exceptions, on all 25 maps, and that a map's megatexture geometry's own UV0 channel (already shipped, already parsed by wolfsdk/tnomap.py) is itself the page-space UV -- no .land file needed at all. Checked on 3 maps with a 2000-sample real-vs-random-UV statistical test at every mip. This resolves most of what pass 1 called unresolved; see §4.
3. Overstated, fixed: pass 1 called the virtualMapping mip 7->0 chain "8 independent hits". The verifier ran a negative control pass 1 had not (20 random offsets, 5/20 landed a full 8/8 by chance, because _vmtr.pages' mip chains are complete beneath any occupied coarse node) and showed the 8 hits are one real test cascading, not 8 independent ones. Reproduced independently this round (my own 50-trial control: 8/50 full by chance, 8.8-11.4% occupancy per mip) -- §4 now states this honestly.
4. Followed up, still a dead end: whether landGenModel's world.genmodel embeds the UV table instead of a separate .land file -- checked directly (zero archive hits for .genmodel either, same 14-archive scan, same as .land), consistent with point 2's finding that the mapping is baked into shipped geometry at compile time and neither file is loaded at runtime at all.
5. Followed up, now implemented and proven: the "grow a replacement block past its original slot" design from pass 1 (append past EOF, repoint one parent) is now built and round-trip-proven on a copy, including a real, previously-undiscovered wrinkle (an orphaned-space bookkeeping invariant that only pristine files satisfy) -- see §6b.
6. Followed up, now proven: block 3 (power, 32x32px, 2x2 macroblocks) run through the writer for the first time -- see §6c.
7. Followed up: the ~4.8/255 lossy round-trip baseline the writer's 9.68/255 is compared against is tools/vtdecode.py's own module docstring (line 23), now cited explicitly in §3.

Question

Decode the .pages virtual-texture page file format fully (page table, tile codec/compression, mip levels, world-UV → page/tile mapping), decode one region to a viewable image, and design + round-trip-prove (on a copy) a writer that replaces one tile/region so part of a level could be repainted. State exactly what is still unknown.

Answer, in one paragraph

The container (file header, quad-tree of page records, per-page sub-header, four channel blocks) was already fully decoded and verified by prior rounds; this round re-confirmed it live (probe_vtpages.py selftest on c03p1.pages: 9031 records, 9030 parent/child edges, every byte of the 27,244,700-byte file accounted for). Of the four JPEG-XR-derived channel blocks per page, two (block 2 = specular/gloss, block 3 = power, both single channel) were already proven to real pixels by prior rounds; this round decoded one such region to a viewable PNG (a clearly recognisable plank surface with a shadowed figure, re_probes/tnomegatex/evidence/before.png) and built and round-trip-proved a writer with two paths: (a) in-place -- a synthetic image is JPEG-XR-encoded with Windows' own encoder, repackaged under a hand-built retail-style NCOMPONENT/1-channel plane header (a capability that did not exist before), spliced into a replacement block no larger than the original; (b) new this (second) pass: grow -- when the replacement does not fit, a whole new page record is appended past the current end of file, the page's one parent record is repointed at it, the orphaned old record is zeroed, and the file header's own size field is grown; both paths were run against a copy of a real .pages file, which is then re-opened cold and proven correct: the container's tree invariants all still hold (a growth edit trips one pristine-file-only bookkeeping invariant in probe_vtpages.py selftest, itself a new, deliberately investigated and explained finding -- see §6b), and decoding the edited page again (fresh process, fresh file handle) reproduces the intended image (mean pixel error 9.68/255 for the in-place smiley, matching this codebase's established lossy-round-trip baseline of ~4.8/255, tools/vtdecode.py's own module docstring). Block 3 (power, 32×32px, 2×2 macroblocks) was run through the same writer for the first time this pass and round-trips too (§6c). The original game file is proven byte-for-byte unchanged (SHA1 compared before/after) in every one of these runs. Also new: the world-UV → page mapping formula for shared virtual materials (virtualMapping scale/offset in a material decl) was checked against a real decl and real file occupancy -- it holds exactly, though the mip 7-0 "hits" are one cascading test, not 8 independent ones (a pass-1 overstatement the verifier caught, reproduced and fixed here with a negative control). Reworked this pass: the UV→tile mapping for the primary per-map megatexture, which pass 1 wrongly called unreachable, turns out to be the biggest result of this round -- the decl field pass 1 quoted only part of also contains landPageFile, which names the .pages file directly (confirmed on all 25 TNO maps), and landDimensions x 8 == pagesAcross exactly on all 25. Going further still: the shipped map geometry's own UV0 channel (the same one wolfsdk/tnomap.py already parses for every other material) turns out to be the page-space UV already -- page = floor(uv * pagesAcross-at-mip), no offset, no .land file, checked with a 2000-sample real-vs-random-UV statistical test against 3 maps' real .pages files at 3-6 mip levels each (66-100% real hit rate vs. 5-33% random). landDefinitionFile's .land file (and landGenModel's .genmodel) remain unreachable in every one of the 14 base chunk archives -- now understood as expected, not a dead end: the mapping is baked into the shipped geometry at compile time and neither file is ever loaded at runtime. What remains unknown, unchanged from before: the diffuse colour plane (block 0) still cannot be turned into pixels at all (no decoder or encoder oracle for it exists), so the writer cannot repaint colour yet, on either megatexture family; and the 120-vs-128-texel page pitch and block 0/1's exact macroblock shape are both still open (§3, §4).

Evidence

All commands run from the repo root; all .pages reads are against the retail install, read-only. python tools/probe_vtpages.py selftest <file> and python tools/vtdecode.py selftest <file> were re-run live for this report (not just cited from docs):

selftest ok: c03p1.pages, 9031 records, 9030 parent/child links,
  27216028 bytes claimed + 28672 bytes reserved slack in 14 holes
  = 27244700 bytes
selftest ok: c03p1.pages, 1999 pages, 4 JPEG XR plane headers each

1. The page table / container (PROVEN, re-confirmed this round)

32-byte file header, 40-byte record headers in a sparse quad-tree (child_off/child_size in units of the header's offset_scale, always 2 in retail), 16-byte page sub-header (magic, layout code, four big-endian u16 block sizes, a reserved u16 that must be 0), then the four raw channel blocks back to back, the whole payload padded to even length. Full field list and byte offsets: docs/virtualtextures.md §2–§4. Verified this round by read_header/read_record/find_page/read_blocks in tools/probe_vtpages.py, used directly in the writer below (see re_probes/tnomegatex/writer_poc.py::patch_block2, which documents the exact on-disk byte offsets it writes: page magic/layout at rec.offset+40, the four BE u16 sizes at rec.offset+46, block bytes starting at rec.offset+56).

2. Mip levels and the child/parent relationship (PROVEN, re-confirmed)

across = 2^(levels-1); child index = (y&1)*2 + (x&1); kind.mip = parent.mip-1; coordinates halve with each mip step up. Used directly in the UV→page check below (walking mip 7→0 of _vmtr.pages by repeatedly right-shifting the same base coordinate, and finding a real stored record at every single step — see §4).

3. Tile codec (PROVEN for blocks 2/3, newly re-confirmed with a fresh example; unchanged open status for blocks 0/1)

The four channel blocks are JPEG-XR/HD-Photo IMAGE_PLANE_HEADER + entropy-coded payload with the 16-byte WMPHOTO\0 + IMAGE_HEADER prefix stripped (constant, rebuildable: tools/vtdecode.py::image_header). This round re-extracted a real page's block 2 (page (92,81), mip 0, c03p1.pages): INTERNAL_CLR_FMT=NCOMPONENT, 1 channel, SB_NO_FLEXBITS, QP=25 for DC/LP/HP alike — the exact constant the whole file family uses per docs/virtualtextures.md §8, reproduced live rather than taken on faith. Decode status, unchanged from docs/vt-planes.md:

The writer's 9.68/255 figure (§6) is compared against the lossy round-trip baseline tools/vtdecode.py states in its own module docstring (line 23: "mean error 4.8/255") — that citation was implicit in pass 1 and is named explicitly here per the verifier's "missed" list.

4. World-UV → page/tile mapping

Shared virtual materials (virtualMapping): PROVEN against a real decl and real file occupancy; the mip-chain "hits" are one cascading test, not 8 independent ones (fixed this pass).

Pulled a real material decl straight from the game archives (wolfsdk.resources.Archive/load_master, chunk 0 of base/chunk0.resources):

models/weapons/combatknife/combatknife_hg
  diffusemap       "models/weapons/CombatKnife/CombatKnife_hg"
  virtualmapping    { 0.015625, 0.015625, 0.015625, 0.000000 }
  usevirtualmapping 1.000000

{scaleX, scaleY, offsetX, offsetY} = {1/64, 1/64, 1/64, 0}. Chunk 0 has no chunk0_vmtr.pages on disk (only chunks 1–13 do — confirmed by probe_vtpages.py list), so this material's pages live in the global _vmtr.pages (across=1024, levels=11). The formula from docs/virtualtextures.md §7 predicts, at mip 0, page x = floor(offsetX * across) = floor(0.015625*1024) = 16, y = floor(offsetY*across) = 0, and at mip m the same page right-shifted by m. Checked with find_page() against the real file, mip 7 down to mip 0 — still exactly where the formula places them, re-verified live this pass:

across 1024 levels 11
mip 7 coord (0, 0) exists  ...  mip 0 coord (16, 0) exists   (all 8 exist)

Pass 1 called this "8 independent hits". It is not, and the verifier was right to call that out. Re-ran its negative control independently this pass (re_probes/tnomegatex/check_mapping.py vmtr_control: 50 random (offsetX, offsetY) pairs through the identical mip 7→0 chain-check, plus a direct per-mip occupancy measurement against 500 random single points each):

real decl offset (1/64, 1/64):        8/8
50 random offsets, hit histogram:     {0: 41, 1: 1, 8: 8}   (8/50 = 16% full by chance)
mip occupancy (500 random points/mip): 8.8% .. 11.4%, every mip 0-7

_vmtr.pages' quadtree has essentially uniform ~10% occupancy at every mip level, and — because a stored coarse node's descendants are themselves near-fully populated beneath it — hitting the correct mip-7 cell (odds ~1/64, which is exactly what the decl's un-fudged 1/64 scale produces, not a coincidence) cascades automatically into 7 more hits. The outcome is bimodal (0 or 8 of 8), never partial, confirming the verifier's reading exactly: this is one real, meaningful test (does the formula land in the right 1-of-64 mip-7 cell), not 8 independent confirmations. The underlying conclusion is unchanged — the formula is correct — only the pass-1 framing of how much evidence that one hit represents was wrong, and is corrected here. (Caveat inherited from docs/virtualtextures.md §7, still open: this check cannot distinguish a ×128-texel from a ×120-texel page pitch, because offsetX*across cancels the pitch out of the page index — only the sub-pixel UV→texel mapping within a page is still ambiguous.)

Unique per-map megatexture (landDefinitionFile): this pass's main result — reworked completely, most of what pass 1 called unresolved now has a proven answer.

Pass 1 quoted this decl but only showed two of its nine fields. The verifier pulled the whole block; re-confirmed here, plus all 25 TNO maps, not the one the verifier spot-checked:

maps/game/wolf/c01/c01p1/mega.decl   (also: .../megaanimbump.decl, same fields)
{
stageProgram landUniqueBump
depthFillProgram occluderUnique
landDimensions 128
layoutDownscale 1.000000
landPageFile "c01p1"
landDefinitionFile "maps/game/wolf/c01/c01p1.land"
landGenModel "maps/game/wolf/c01/c01p1/_combo/world.genmodel"
landLayoutVersion 321
}

landPageFile names virtualtextures/<value>.pages directly — no file search needed at all, the pass-1 claim that the file "could not even be reached" was simply wrong (the verifier is right, and c03p1.pages, used for every proof in this report, is itself one of these primary-megatexture files). This is also not a new discovery in this codebase: tools/probe_tno_tex.py's existing page_file()/cmd_mega (unmodified, read-only, pre-dating this track) already resolves every map's .pages file this same way — pass 1 should have found this by reading the existing tools before concluding the file was unreachable, and did not.

Two things checked fresh this pass, both new:

1. landDimensions x 8 == pagesAcross, exactly, on all 25 TNO maps, zero exceptions (re_probes/tnomegatex/check_mapping.py landpagefile, against wolfsdk.tnomaptex's decl reader + probe_vtpages's header):

dim= 16 pagefile=c03p1.pages across= 128 (c03p1, c12p1) dim= 32 pagefile=c01p4.pages across= 256 (c01p4, c02p0, c03p3) dim= 64 pagefile=c04p1.pages across= 512 (c04p1, c06p2, c16p0, c16p2) dim=128 pagefile=c01p1.pages across=1024 (12 of the 25 maps) dim=256 pagefile=c01p2.pages across=2048 (c01p2, c06p1, c13p1, c14p1) MATCHES: 25 of 25

landDimensions ranges over a 16x spread (16..256) and the x8 ratio holds at every one, a clean, reproducible, PROVEN numeric fact — not yet independently explained (INFERRED reading: 1 "land unit" = an 8x8 page grid, by analogy to the shared-material page being internally 128 texels = PAGE_WIDTH; not confirmed from any other source).

2. The map geometry's own UV0 channel already is the page-space UV — no .land file needed. wolfsdk/tnomap.py::surface_arrays() (existing, unmodified, already used for every other material by the map viewer) returns uv0 per vertex for every surface, including the ones classified "mega" by wolfsdk/tnomaptex.py::classify() (also existing, unmodified). Pulled those UVs straight from _world.bmodel for 3 maps and found their range is already normalised [0, 1] (not world-space metres) — exactly what page = floor(uv * pagesAcross-at-mip) with no offset and no v-flip would need. Tested directly: for each map, sampled 2000 real mega-material vertex UVs and 2000 uniform-random UVs, mapped both through that formula at several mip levels, and checked find_page() hit rate against the map's own real .pages file (random UVs are the negative control):

c03p1.pages (across=128): mip0 real 84.8% vs random 33.7% | mip2 real 99.8% vs random 69.5% | mip4 real 100% vs random 94.8% c01p4.pages (across=256): mip0 real 75.6% vs random 25.2% | mip2 real 99.8% vs random 59.0% | mip4 real 100% vs random 72.4% c16p0.pages (across=512): mip0 real 66.3% vs random 5.0% | mip2 real 98.8% vs random 9.4% | mip4 real 100% vs random 16.2%

Real UVs land on a stored page far above the random baseline at every mip on all 3 maps (different pagesAcross each: 128/256/512), reach 100% by mip 4, and a flipped-v control on c03p1.pages scores worse than random at mip 0 (22.6% vs. 33.7%), ruling out the one plausible alternative convention. The mip 0 shortfall (66-85%, not 100%) is consistent with the already-known fact that not every finest-mip page is actually stored for every point on the mesh (prior rounds' mega.json shows exactly this kind of partial coverage at fine mips); it is not evidence against the formula, which a random control at the same mip confirms (5-34%, far lower).

Reproduce with python re_probes/tnomegatex/check_mapping.py uvmap (new this pass, read-only, reuses wolfsdk.tnomap._Store + parse_bmodel/surface_arrays to pull UVs, tools.probe_vtpages.find_page to test them, tools.probe_tno_tex.page_file to resolve the .pages path — no new decoder, just a new statistical test over existing, unmodified readers). check_mapping.py landpagefile reproduces the 25/25 landDimensions x 8 table above, check_mapping.py land_scan the .land/.genmodel zero-hits scan, and check_mapping.py vmtr_control the negative control below.

.land / .genmodel: still zero hits — now understood as expected, not a dead end.

Re-ran the archive scan (re_probes/tnomegatex/check_mapping.py land_scan: all 14 base/chunk*.resources pairs, wolfsdk.resources.load_master, 203,843 entries, every entry type) for *.land and, following up the verifier's "missed" suggestion, for *.genmodel (the file landGenModel names): zero hits for both, same as pass 1 found for .land alone. Given point 2 above — the UV→page mapping is already baked into the shipped geometry's UV0 channel, measured and confirmed working with no .land file anywhere in reach — this is no longer a loose end pointing at "a file we haven't found yet". Both .land and .genmodel are read naturally as build-time-only artifacts of MachineGames' internal map compiler (which presumably writes the UV values into _world.bmodel using whatever .land/.genmodel describe, then never ships either file), not as something a runtime loader reads and this round's scan missed. This is now a settled negative result, not an open file-location search.

5. One region decoded to a viewable image (PROVEN, new PNG this round)

python tools/vtpixel.py png c03p1.pages 92 81 0 --block 2 --out <dir>
  -> c03p1_92_81_m0_b2_ch0.png   spread 92.6   bit walk lands

Saved at re_probes/tnomegatex/evidence/before.png (also evidence/paint_source.png is the separate synthetic image used for the writer test below, not to be confused with this one). This is a real decoded page, not an occupancy map: planks with a shadowed figure, clearly structured image content, pixel spread 92.6/255. Decoded via the same jxrlib-based pipeline the prior rounds calibrated bit-exact against Windows' own WIC decoder (docs/vt-planes.md §9.2, 324/324 and 60/60 byte identical).

6. A round-trip-proven writer (PROVEN, new this round)

Code: re_probes/tnomegatex/writer_poc.py (self-contained, read-only on the game install, copies to D:\wolf-decomp\tmp\tnomegatex\ and deletes them when done unless --keep is passed). Run:

python re_probes/tnomegatex/writer_poc.py

What it does, each step reusing an already-proven piece of tools/vtdecode.py / tools/vtplanes.py / tools/vtpixel.py wherever one existed, and adding only what did not:

1. Copies c03p1.pages to the scratch directory. Never opens the original for writing.
2. Loads page (92,81,0), its block 2 (NCOMPONENT, 1ch, QP 25, 4768 bytes on disk) — the same page as §5 — and decodes it as the "before" picture.
3. Renders a 128×128 synthetic image (a smiley face — unambiguously not game art) and JPEG-XR-encodes it with Windows' own WmpBitmapEncoder (OverlapLevel=1, ImageDataDiscardLevel=1 = SB_NO_FLEXBITS, matching what every retail block uses) via the same PowerShell/WIC call vtplanes.py refgen already uses for reference streams.
4. Extracts WIC's own codestream and locates the entropy payload using vtplanes._ref_params (which already parses both of JPEG-XR's two SUBSEQUENT_BYTES encodings — WIC did not pick the same one as retail files always do here, which is exactly why this step needed the existing parser and not the stricter vtdecode.parse_plane_header, which only accepts retail's form).
5. New capability: ncomponent_plane_header() builds a retail-style IMAGE_PLANE_HEADER for INTERNAL_CLR_FMT=NCOMPONENT, 1 channel — the mirror image of vtdecode.y_only_plane_header, which only ever builds the other direction (repackaging an existing retail block for WIC to read). The new block is header(6B) + FF 00 00 01 00 + <WIC's payload bytes, copied verbatim> — byte-for-byte the same shape as a real block 2.
6. Standalone check, before the file is touched at all: feeds the new block straight back through the existing, proven gray_jxr() + JxrDecApp.exe path and diffs against the source image: mean |error| 9.68/255. (Also cross-checked through the same NCOMPONENT/3-channel decode path vtpixel.plane_pixels uses for real block 2/3 — same result once the interleaved-channel output is unpacked correctly; an initial run that skipped that unpacking is noted under Failed hypotheses, since it is a real trap for anyone else using plane_pixels on a 1-channel block.)
7. Splices the new block into the copy, in place. The replacement (953 bytes) is smaller than the slot it replaces (4768 bytes), so blocks 0/1 stay untouched, block 3 moves to sit right after the new, shorter block 2, and the page sub-header's four big-endian size fields are rewritten. Nothing else in the file moves, and no parent child_off/ child_size needs to change, because a smaller record can only ever still fit inside whatever span its parent already reserved for the original (larger) one — this is checked, not assumed (see step 8).
8. Re-opens the copy cold (fresh Python process via subprocess, for probe_vtpages.py selftest; a fresh find_page/read_blocks call for the rest) and proves the edit two independent ways:
   • probe_vtpages.py selftest on the whole patched file: 9031 records, 9030 parent/child links, 27212214 bytes claimed + 32486 bytes reserved slack in 15 holes = 27244700 bytes — same record count and total size as the unmodified file's own selftest above; the claimed-bytes delta (27216028 → 27212214 = −3814) matches 4768 − 953 = 3815 to within one pad byte, and the hole count went from 14 to 15 — exactly the one new gap this edit created. The container is still 100% self-consistent.
   • Re-decoding page (92,81,0)'s block 2 from the patched file: mean |error| 9.68/255 against the intended smiley (same number as the standalone check — because splicing is byte-exact, diffed at 0.00/255 against the standalone decode), and visually: compare re_probes/tnomegatex/evidence/before.png (original plank/shadow texture) against evidence/after.png (the smiley, decoded fresh from the patched copy).
9. Confirms the original c03p1.pages is still byte-identical (SHA1 compared before/after: unchanged), then deletes the scratch copy and temp files (unless --keep).

Live output of the run that produced the evidence PNGs:

page (92,81) mip 0: block sizes (4451, 6879, 4768, 36), block 2 QP 25, subband 1
WIC encode: QP 90, subband 1, payload 942 bytes (budget 4768 bytes)
new block 2: 953 bytes (fits in the 4768 byte slot)
standalone round trip (before touching the file): mean |error| 9.68/255
selftest ok: c03p1_writetest.pages, 9031 records, 9030 parent/child links,
  27212214 bytes claimed + 32486 bytes reserved slack in 15 holes = 27244700 bytes
re-read from the PATCHED FILE vs. the painted source: mean |error| 9.68/255
re-read from the PATCHED FILE vs. the standalone check: mean |error| 0.00/255
original file sha1 unchanged: True (de2770b5bdd0...)
RESULT: PASS

6b. Growing a replacement block past its slot (PROVEN, new this pass — was only a design in pass 1)

Pass 1's recipe step 6 described this but did not build it; the verifier listed the untested --grow flag (advertised in the module docstring but absent from argparse) as a gap. Both are fixed: --grow now exists and works. Design, as implemented in patch_grow():

1. Find the one parent record that references the target page's child_off/child_size slot (find_page_with_parent(), a new local descent that tracks the parent alongside the existing find_page() logic).
2. Build a whole new leaf record (40 byte header + 16 byte page sub-header + all four blocks, one of them the oversized replacement) and append it past the current end of file.
3. Zero the orphaned old record's bytes in place (its magic must not survive — probe_vtpages.py selftest's hole scan would otherwise read it as a stray, unreferenced record and fail for a real reason).
4. Repoint the one parent's child_off/child_size for that slot at the new record.
5. Grow the file header's own file_size field (offset +24) to match the new, larger actual file size — a detail pass 1's design did not mention and that is easy to miss, since nothing looks broken without it until probe_vtpages.py selftest's very first assertion (file_size == actual size) catches it.

Tested by deliberately targeting a page with a tiny block 2 slot (62 bytes, page (0,18,0) of c03p1.pages) and a synthetic noise image (not the smiley — noise is close to incompressible, guaranteeing the encoded block outgrows almost any real slot, which is the point of this test):

page (0,18) mip 0: block sizes (170, 111, 62, 103), block 2 QP 25, subband 1
WIC encode: QP 100, subband 1, payload 2568 bytes (budget 62 bytes)
new block 2: 2579 bytes (TOO BIG in the 62 byte slot)
standalone round trip (before touching the file): mean |error| 39.93/255
grow: old record (502 bytes) zeroed, new record (3020 bytes) appended at offset 27244700, parent repointed
re-read from the PATCHED FILE vs. the painted source: mean |error| 39.93/255
re-read from the PATCHED FILE vs. the standalone check: mean |error| 0.00/255
original file sha1 unchanged: True
RESULT: PASS

(39.93/255 is noise's own lossy round-trip error at this codec/quality, not a writer defect — err_vs_standalone of 0.00/255 shows the on-disk bytes are bit-identical to what was encoded; the higher number is simply harder for JPEG-XR to represent than a smooth synthetic image like the smiley.)

A genuinely new finding surfaced by actually building this, not just designing it: probe_vtpages.py selftest fails on a grown file — reserved slack 28672 != unclaimed bytes 29174, a difference of exactly 502 bytes, i.e. exactly the one record this edit zeroed and orphaned. This is not a bug in the edit. selftest checks one invariant beyond tree correctness: that every unclaimed byte in the file is accounted for by some parent's "reserved but currently unused" declaration (true of all 39 pristine retail files, because their only unclaimed bytes are packer padding some parent still formally reserves). An appended --grow record's old slot is unclaimed by construction — nothing references it any more — so it can never satisfy that bookkeeping equality, no matter how correct the edit is. writer_poc.py now ships its own verify_structure() (new this pass, reusing only the already-proven read-only iter_records/read_record) that checks every invariant that does matter structurally (every claimed child fits its declared slot, parent/child mip and position agree, no record hides unreferenced in any hole) minus that one pristine-only equality, confirms the orphan is exactly the one old record's span, and the run above calls it explicitly rather than silently papering over the selftest failure.

6c. Block 3 (power, 32×32px, 2×2 macroblocks) through the writer (PROVEN, new this pass)

The verifier flagged that only block 2 had ever been run through the writer, despite block 3 being the other plane proven to pixels, with a different macroblock grid (2×2 vs. 8×8) that exercises different code paths (fits()/plane_pixels()'s shape parameter, previously only ever called with (8, 8)). run_block3_test() (python re_probes/tnomegatex/writer_poc.py --block3) repaints block 3 of the same page used throughout, via the --grow path (block 3's slots are tiny — 36 bytes here — so any real image outgrows them, making the grow path the relevant one to exercise):

page (92,81) mip 0: block 3 is 36 bytes, 32x32px, 2x2 macroblocks
new block 3: 194 bytes (old slot 36 bytes)
standalone round trip: mean |error| 101.24/255
grow: old record (16190 bytes) zeroed, new record (16348 bytes) appended, parent repointed
verify_structure: 9031 records, 9030 links, orphan 16190 bytes (expected)
re-read from the patched copy vs. painted source: mean |error| 101.24/255
re-read vs. standalone: mean |error| 0.00/255 (should be ~0)
original file sha1 unchanged: True
RESULT: PASS

(101.24/255 is, again, noise's own lossy-codec error at a coarse 2×2 macroblock grid, not a writer bug — err_vs_standalone of 0.00/255 is the number that actually speaks to correctness.) This confirms the writer's mechanism — plane header, splice/grow, container repair — generalises across both proven single-channel planes, not just block 2.

PROVEN vs INFERRED

PROVEN (with live commands and numbers above, not just cited):

• The container/page-table model, re-walked end to end on a real file.
• Block 2's plane header is a fixed, reproducible NCOMPONENT/1ch/QP25/SB_NO_FLEXBITS shape, re-extracted from a fresh page.
• One real region decodes to a genuine, structured viewable image (not an occupancy map).
• A full write → splice → cold re-open → re-decode round trip for block 2, on a copy, with the original file proven untouched (in-place/shrink path).
• New this pass: the same round trip via the grow path (append past EOF, repoint one parent, zero the orphan, grow the file header), on both block 2 and block 3, each proven on a copy with the original untouched.
• The virtualMapping UV→page formula, against a real decl and real file occupancy — corrected this pass: one cascading mip-chain test (not 8 independent ones), backed by a negative control (50 random offsets: 8/50 land a full chain by chance, matching the quadtree's ~10% per-mip occupancy).
• New this pass, the primary result: landPageFile names the primary per-map megatexture's .pages file directly, on all 25 TNO maps, with no search needed — pass 1's "unreachable" claim was wrong.
• New this pass: landDimensions x 8 == pagesAcross, exactly, on all 25 maps, zero exceptions.
• New this pass: the primary megatexture's own UV0 channel (already shipped in _world.bmodel, already parsed by wolfsdk/tnomap.py) is itself the page-space UV at page = floor(uv * pagesAcross-at-mip), no offset, no .land file — backed by a 2000-sample real-vs-random-UV test against 3 maps' real .pages files at 3-6 mip levels each (real UV hit rate 66-100% vs. random 5-33%, and a flipped-v control scoring below random, ruling out that alternative).
• landDefinitionFile's .land target, and landGenModel's .genmodel target (checked new this pass), are not loadable archive entries under any name/type in any of the 14 base chunk archives — now understood as expected (build-time-only artifacts), not an open file-location search.

INFERRED:

• Block 0/1's macroblock shape (8×8 vs. any other 64-MB arrangement) — still only an analogy to block 2, per docs/vt-planes.md §3.2/§9.5/§10.7.
• The 128-vs-120-texel page pitch ambiguity (docs/virtualtextures.md §7) — unaffected by this pass's landDimensions x 8 finding, which is about page count, not per-page texel width.
• Block-role-to-texture-slot (0=diffuse,1=normal,2=specular,3=power) — still inferred from byte-share statistics, not from a cvar name lookup or disassembly (docs/virtualtextures.md §8).
• New this pass: why landDimensions x 8 == pagesAcross (reading: one "land unit" = an 8x8 page grid, by analogy to a shared-material page being internally PAGE_WIDTH=128 texels) — the correlation itself is proven (25/25, zero exceptions), the mechanism behind it is not confirmed from any other source.

Failed hypotheses (this round)

1. "plane_pixels()'s return value is already a single-channel pixel array." It is not — it always returns PF_3CHANNELS-interleaved data (spp=3) even for a 1-channel NCOMPONENT plane, because that is the output format vtpixel.py standardises on for both 1- and 2-channel blocks. Using the raw buffer directly gave mean |error| 156/255 (garbage) on the very first full writer run even though every earlier step (header, QP, payload, splice, container selftest) was already correct. Fixed by extracting data[0::spp]. Recorded here because it is an easy trap for the next person calling plane_pixels on a block 2/3.
2. Finding c01p1.land (or any *.land file) by exact or case-insensitive name in the 14 scanned base/chunk*.resources archives. Zero hits, in any entry type — and, this pass, zero hits for .genmodel either, the same scan. Not a failure any more in the sense of blocking progress: §4 shows the mapping does not need either file, it is baked into the shipped geometry's own UV0 channel.
3. (Pass 1's own mistake, not re-made here, recorded for the second-pass record.) Pass 1 concluded "the file could not even be reached" from the .land zero-hits result, without checking whether the decl it had already quoted contained a more direct field. It did (landPageFile), one line away from the field pass 1's excerpt stopped at. The lesson generalised: a zero-hits negative result for one named field does not by itself mean "the whole question is unreachable" — the decl's other fields (and existing codebase tooling, see §4's probe_tno_tex.page_file() note) needed checking first.
4. Forcing --grow to trigger with the smiley image and QualityLevel 100 on the normal 128×128 page. WmpBitmapEncoder.QualityLevel=100 is not literally lossless in WIC's JPEG-XR encoder — a 128×128 noise image at quality 100 still only reached 2579 bytes, short of the normal test page's 4768-byte block 2 slot. Fixed by targeting a different, deliberately tiny real slot (62 bytes, found by scanning the file for small block 2 sizes) instead of fighting the encoder for a bigger output on a page that already has a generous budget.

Mod design: repainting a level's specular/gloss via a patched .pages file

What is proven: a specular/gloss page (block 2 and block 3, both tested) can be replaced with modder-authored content, on a copy, with the container staying valid and the pixels round-tripping — for both virtual-material families now, shared vmtr and a map's own primary megatexture (the page-location and UV-mapping blocker from pass 1 is resolved, §4). The recipe, generalised from writer_poc.py:

1. Find the page(s) covering the area to repaint:
   • shared vmtr material: virtualMapping scale/offset + find_page() (§4);
   • a map's own primary megatexture (floors/walls/terrain): landPageFile for the .pages file, then page = floor(uv * pagesAcross-at-mip) against that surface's own UV0 channel (new this pass, §4) — no .land file needed;
   • or pick any already-occupied page directly for an experiment (as this report does throughout).
2. Render the new content at 128×128 (block 2) or 32×32 (block 3), grayscale.
3. JPEG-XR-encode via WIC (OverlapLevel=1, ImageDataDiscardLevel=1).
4. Wrap under ncomponent_plane_header() + FF 00 00 01 00 + WIC's payload bytes.
5. If the new block is no larger than the original, splice it in place (patch_block2(), the proven in-place path for block 2): no other byte in the file moves. (Block 3 was only tested via the grow path this pass, §6c — its slots are small enough that a real repaint will usually need growing anyway.)
6. If the new block is larger than the original slot: now implemented and proven, not just designed (§6b) — patch_grow() appends a whole new record past EOF, repoints the page's one parent, zeros the orphaned old record, and grows the file header's file_size. One caveat discovered by building it: the growth path trips probe_vtpages.py selftest's pristine-file-only slack-accounting invariant (expected, explained in §6b, not a correctness problem — verify_structure() checks the invariants that do matter and passes).
7. Verify with probe_vtpages.py selftest on the copy (in-place) or verify_structure() (grow), and by re-decoding the target page, before ever touching anything the game loads from.

What this recipe cannot do yet: repaint colour (block 0, diffuse) — there is still no decoder or encoder oracle for it at all, so step 2–4 have no analogue for colour content, on either megatexture family. This is now the single biggest remaining gap: the location and UV-mapping problem that pass 1 thought blocked the primary megatexture is solved, but the pixel-format problem (block 0) blocks colour on both families equally and is unrelated to that fix.

What this round deliberately did not attempt: wiring a patched .pages file into the mod loader or the game's own file search (vt_filePath/vt_filePathVmtrOverride/vt_setSource per docs/virtualtextures.md §9–§10) — that cvar route was flagged by a prior round as unverified in the retail build and is outside this round's scope (file-format decode + writer proof only). No game was launched.

Open questions (updated this pass)

• Block 0 (diffuse) pixels: still unsolved, the one thing this pass did not move forward at all; needs a from-scratch inverse transform (coefficient values, dequantisation with the QPs already known, two stages of inverse PCT, overlap filter) per docs/vt-planes.md §10.6, not another bitstream search. This blocks colour repainting on both megatexture families now that the primary family's location/mapping problem is resolved.
• Block 1 channel 1: bits land, image is smooth, but it sits at the wrong spatial position relative to channel 0 — open per docs/vt-planes.md §11.
• Block 0/1 macroblock shape and the 120-vs-128 page pitch: both open, unchanged (the landDimensions x 8 finding is about page count, not per-page texel width, so it does not touch the pitch ambiguity).
• Resolved this pass, was open before: landDefinitionFile's .land file target, and therefore the UV→page mapping for the primary per-map megatexture — see §4 (landPageFile + landDimensions x 8 + UV0-is-already-page-space-UV).
• Why the landDimensions x 8 ratio holds (INFERRED reading above, not independently confirmed) — low priority, since the correlation itself is already reliable enough to use.
• The mip 0 shortfall in the UV→page statistical test (66-85% hit rate, not 100%, §4) — read as expected sparse-quadtree coverage by analogy with prior rounds' mega.json figures, not independently confirmed as the exact cause.
• Whether any vt_* cvar actually reaches the retail build's console (the file-format side of "repaint a level" is now proven on a copy, for both megatexture families and both growable/in-place writer paths; the in-game delivery side is still not addressed by this track at all — no game was launched).

Smallest in-game test for the maintainer

(Translated from German.)

This run deliberately tested nothing in the game (only copies of a .pages file on disk) — that was explicitly not allowed and not the assignment either. The next sensible step, if you want to see this live, would first need a way to get the game to load the patched copy at all (e.g. via the mod loader or vt_filePathVmtrOverride, if that is reachable in retail — unproven, see above). Two concrete, small tests, once that is in place:

1. As in pass 1: replace virtualtextures/_vmtr.pages with a copy in which page (16,0) on every mip (the region of the combat knife combatknife_hg, see §4) has been rewritten to a conspicuous pattern with writer_poc.py logic, then look at the combat knife in hand in the game — the gloss/specular part of the blade should change visibly, diffuse/colour would stay unchanged (because block 0 cannot be written yet).
2. Newly possible since this pass: the same idea on the primary map megatexture instead of on a shared vmtr material — e.g. copy virtualtextures/c03p1.pages (the file from landPageFile for game/wolf/c03/c03p1), rewrite any large, clearly visible floor/wall page in it on specular (§4 shows how to find the right page via landPageFile + the UV0 channel instead of via the non-loadable .land file) and look at the same map in the game. That would show for the first time whether the file round trip proven here also works on the surface type that makes up the bulk of a map (floor/walls/terrain), not only on individual shared materials like weapons.

Both tests connect the file round trip proven here for the first time with a real in-game image.

Related: TNO Virtual Textures, Virtual Texture Planes, TNO Maps.