Wolfenstein Modloader Buy me a coffeeKaffee spendieren

Wiki / Research reportsForschungsberichte

Why the edited save state was rejected — Structure of checkpoint.dat

Partly solvedTeilweise gelöst Original page in the GitHub wikiOriginalseite im GitHub-Wiki

In plain wordsIn einfachen Worten

A modified save game for Wolfenstein II: The New Colossus, with a renamed map, was rejected by the game. Reading the game's code showed the cause: the map name got 7 characters longer, but the number stored in front of it that says how long the name is was not changed. A second error was a wrong checksum line in game.details. A corrected version (v3) fixes both and passes 33 of 33 offline checks, but whether the game actually loads it has not been tested.

  • The length field before the map name said 17 but should say 24 (and 14 instead of 21 in another file).
  • The game then read part of the name as a count of about 1.6 billion entries.
  • The checksum in game.details was also wrong (2007466474 instead of 499283768).
  • A version rebuilt from the decoded code is byte-identical to candidate v3 in all 14 files.
  • Whether the game accepts v3 is still unconfirmed; it cannot be checked without starting the game.

Ein veränderter Spielstand für Wolfenstein II: The New Colossus mit umbenannter Karte wurde vom Spiel abgelehnt. Das Lesen des Programmcodes zeigte die Ursache: Der Kartenname wurde 7 Zeichen länger, aber die Zahl davor, die die Länge des Namens angibt, blieb gleich. Ein zweiter Fehler war eine falsche Prüfsummenzeile in game.details. Eine korrigierte Version (v3) behebt beides und besteht 33 von 33 Offline-Prüfungen; ob das Spiel sie wirklich lädt, ist aber nicht getestet.

  • Das Längen-Feld vor dem Kartennamen zeigte 17 statt 24 (in einer anderen Datei 14 statt 21).
  • Das Spiel las dadurch einen Teil des Namens als Anzahl von rund 1,6 Milliarden Einträgen.
  • Auch die Prüfsumme in game.details war falsch (2007466474 statt 499283768).
  • Eine aus dem Programmcode neu gebaute Version ist in allen 14 Dateien bytegleich mit Kandidat v3.
  • Ob das Spiel v3 annimmt, ist noch unbestätigt; ohne Spielstart lässt es sich nicht prüfen.

Technical detailsTechnische Details

Der technische Teil ist auf Englisch – so, wie er im Wiki steht. Unterstrichene Begriffe haben eine Erklärung auf Deutsch.

Source of grammar: re_probes/runtime_87400/module.bin (running, extracted NewColossus_x64vk.exe, file offset == RVA). Tool: tools/probe_save_structure.py. Corpus: re_probes/save_backups (5 real save states) and the three candidates, all read-only. Writing was done exclusively to re_probes/save_structure/.


Summary

  1. The defect is a stale length prefix. Candidates v1 and v2 have extended the card name, but left the preceding u32 length unchanged. At checkpoint.dat offset 0xdf, there stands 17; it should stand 24. In checkpoint_mapstart.dat, this occurs six times (0x14, 0x8d, 0x106: 17 instead of 24; 0x44, 0xbd, 0x136: 14 instead of 21).
  2. Second, independent defect in v1/v2: the checksum line in game.details remained at 2007466474; the correct value is 499283768.
  3. v3 fixes both. A candidate reconstructed from scratch based on the grammar read from code is byte-identical to v3 in all 14 files. 33 of 33 checks pass green.
  4. Not claimed: that v3 loads. "File Corrupt" is not observable offline. The reliable statement is: structurally consistent according to these tests, runtime unconfirmed.
  5. Correction on the task: Branch (a) from saveverify.md (Read(rumpf) != length-4 at 0x8b47c4) is a pure short-read test on the file and cannot trigger here — the file delivers exactly Length-4 bytes. The consumption error lies one level deeper, in the component reader. See Section 5.

1. The Grammar — from code, not guessed

Three functions carry everything necessary:

RVA Role
0x11b9980 idGameLocal::RestoreCheckpointInfo — header of checkpoint.dat
0x11a5610 / 0x11a5710 idGameMapMemoryManager::ReadFromFile — card block
0x11db420 Writer of the tag-free block in checkpoint_mapstart.dat

The primitives (calls via serializer vtable):

[vtbl+0x28]  Read(puffer, n)      Raw bytes
[vtbl+0x30]  Write(puffer, n)
[vtbl+0xe8]  Serialize(idStr&)    u32-LE length, then characters, no NUL
[vtbl+0xf0]  SerializeCheck(tag, file, line)   writes/checks exactly such an idStr

Endianness: Integers and floats are big-endian — the reader rotates them after each manual Read(4) (0x11a57d4–0x11a57f1, pattern 0↔3, 1↔2). The length prefixes of idStr are little-endian, they are written without rotation (0x66ab58–0x66ab77). This exact mixture confuses the format at first glance.

checkpoint.dat, header (0x11b99e4–0x11b9e4e)

u32 LE   saveFormatVersion       must be <= 0x444D0005, otherwise FatalError
u32 LE   Subversion              here 0x26
u8       Flag                    must be 1 (0x11b9dee, otherwise FatalError)
u8       Flag
  if Subversion >= 0x26:  idStr, idStr        (Stand ID, SHA1-Hex)
  if Subversion > 0x21:  6 x u32
tag "idGameLocal_CheckpointInfo:10"
<idGameMapMemoryManager::ReadFromFile>
tag "idGameLocal_CheckpointInfo:11"
... from here, opaque component stream

idGameMapMemoryManager::ReadFromFile

tag "idGameMapMemoryManager:10"
u32 BE   Magic  == 0x0060FFCD     otherwise "Invalid Magic Number in idGameMapMemoryManager"
u32 BE   Version
u32 BE   Count of card entries
per entry:
    tag "idGameMapMemoryManager:20"
    idStr    Card name          <== the edited location
    u32 BE   Count of sub-entries
    ... sub-entries (0x30 bytes each; in backup always 0 — unmeasured)
    tag "idGameMapMemoryManager:30"

In the original from SLOT0, this fits perfectly: Tag at 0xb6… Magic 0xb6, Version 0xba, Count 0xbe = 1, Tag :20 at 0xc2, Card name at 0xdf (Length 17, characters 0xe3–0xf3), Sub-count 0xf4 = 0, Tag :30 at 0xf8.

checkpoint_mapstart.dat — complete

This file is tag-free and fully parseable:

4-Byte header, followed by k identical blocks:
  u32 BE  Magic 0x0060FFCD
  u32 BE  Version (3)
  u32 BE  u32 BE
  idStr   Card Name
  idStr   idStr   idStr   idStr        (empty in inventory)
  u32 BE  n   +  n x idStr             Layer
  u32 BE  m   +  m x idStr
  u32 BE  u32 BE  u32 BE  u32 BE
  float BE  float BE  float BE

Block size at SLOT0: 107 bytes, three blocks, 4 + 3*107 = 325 = file size. At SLOT2 it is 451 bytes, also exact. This is the branch where the parser can definitively state how many bytes the reader has consumed.


2. The Defect, Named

File Offset Is Present Should Be Present
checkpoint.dat 0xdf 11 00 00 00 (17) 18 00 00 00 (24)
checkpoint_mapstart.dat 0x14, 0x8d, 0x106 17 24
checkpoint_mapstart.dat 0x44, 0xbd, 0x136 14 21
game.details checksum= 2007466474 499283768

The build step replaced game/dlc/c02/c2v1 (17 bytes) with game/dlc/c02/c2v1_custom (24 bytes) and correctly grew the file by 7 bytes — only the u32 preceding it remained at the old value. The 4-byte MD5 folding header was then recalculated, so the file branch (b) of the load path parses without issue. A valid header over an inconsistent body.

What the Reader Does With It

idGameMapMemoryManager::ReadFromFile reads length 17 at 0xdf, takes game/dlc/c02/c2v1, and then becomes _cus…. The next read operation is the BE32 count of sub-entries — it reads the ASCII bytes 5f 63 75 73 as 1 600 353 651. The game subsequently attempts to reserve a list with 1.6 billion elements × 48 bytes (roughly 77 GB). The parser reports exactly this value when v2 is presented:

@0xf0: 1600353651 Sub-entries - Grammar unmeasured

In checkpoint_mapstart.dat it fails even earlier: there, the field behind the card name itself is again an idStr, and the reader retrieves the "length" 1 937 072 991 (_cus as u32 LE).


3. The Test Tool and Its Counter-tests

python tools/probe_save_structure.py parse <file.dat>
python tools/probe_save_structure.py check <folder> [--original <folder>]
python tools/probe_save_structure.py build --source … --target … \
       --alt-map … --new-map … --alt-layer … --new-layer …
python tools/probe_save_structure.py --selftest

The self-test is constructed so that it can fail:

  • Positive: all 5 unchanged game states from re_probes/save_backups are accepted; checkpoint_mapstart.dat consumes exactly the file length there (325 or 4×451).
  • Negative, real: v1 and v2 are rejected with the specific location in the message. This is the exact counter-test requested by the order — if the tester accepts both, it measures nothing.
  • Negative, synthetic:
    1. Card name extended, prefix stale, header recalculated → header is valid (branch (b) notices nothing), structure parse fails.
    2. a flipped byte → header fails.
    3. an inserted null byte before the tag :30 → tag comparison fails.
    4. Identity renaming (same name) → byte-identical file.

Result: 0 errors, Exit code 0.

During renaming, the parser accesses the idStr fields at their parsed offsets and writes prefix and characters together. The defect of v1/v2 is thus constructively excluded, not just checked retroactively.


4. The Corrected Candidate

re_probes/save_structure/FC2-WOLF0-c2v1_custom_struct, built from re_probes/save_backups/SLOT0_before_c2custom. 33 tests, all green:

Test Scope
MD5 Header (Branch b) all 7 .dat files
Structure parsing checkpoint.dat, checkpoint_mapstart.dat, both _alt.dat files
Consumption == File Length everywhere where the grammar is complete
game.details CRC32 yes
Strings == Original + Renaming per file, element-wise
Unparsed End == Original checkpoint.dat, from byte 317 or 310 onwards
No framed old path anymore all .dat files
.verify unchanged all 7

It is byte-for-byte identical to candidate v3 of the other Masters in all 14 files. This is an independent confirmation: v3 was created via framed text replacement, while this one was created via the grammar read from disassembly — two paths leading to the same result.

Regarding the Unparsed Remainder of checkpoint.dat

Of the 476,231 bytes parsed by the tool, 317 are consumed and 475,914 remain unparsed; the grammar of the component stream behind it (1,036 length-prefixed strings, 577 idQuickSaveStream tags with Magic 0x3832E21A, Havok Cloth blocks) is not measured. Instead of an excuse, there is a test: the remainder is byte-for-byte identical to that of the original, and the parser proves that the reader arrives at exactly the same position there (317 instead of 310, i.e., exactly +7). The original loads successfully. Therefore, the remainder does not need its own grammar.


5. What Has Been Refuted

  • "Branch (a) at 0x8b47c4 is the trigger." No. That location only checks whether Read(rumpf) provides as many bytes as Length()-4 — a short read test on the file. The candidate provides exactly Length-4. That location cannot fire here. The consumption error sits one level deeper, in idGameMapMemoryManager::ReadFromFile, which can set its own error code at [stream+0x184] or via [vtbl+0xe0] — a path that the scan after mov dword [r15+0x270], 4 does not see. The direction of the instruction ("the deserializer consumes a different number of bytes than the file provides") was correct, only the specific code location was wrong.
  • "Somewhere there is still an old total length / an offset table." No. In checkpoint.dat, no u32 and no u64 exist at any position, neither LE nor BE, holding the value of the file or buffer length (476,224 / 476,220, old or new) — 0 hits. The 52 values in the range 470,000–480,000 are, except for ten outliers, a monotonically increasing float series at 0x5d5c5, and the file end (last 256 bytes) consists of Havok floats, not a table. The block grammar of the map storage has no size field — the length of the name stands exactly once, in its own prefix.
  • "The map name is additionally stored as a hash in the file." No. Checked against game/dlc/c02/c2v1: CRC32, Adler32, FNV-1a 32/64, idStr-Hash, MD5/SHA-1/SHA-256 (first 4 or 8 bytes respectively), each LE and BE. 0 hits in checkpoint.dat. For checkpoint_mapstart.dat, the question becomes moot: there every byte is covered by the grammar.
  • "All map names in a savegame must be identical." No — measured at SLOT2: the map storage in checkpoint.dat regularly also carries the previous map (game/wolf/c01/c01p1 next to game/wolf/c02/c02p0), and GAME-TRANSITION even has three mapstart blocks with two different maps. Such a consistency check would be incorrect; instead, the tool compares against the original plus renaming.
  • .verify remains untouched — confirmed by saveverify.md, here only rechecked to ensure the files were byte-identically copied over.

6. Open

  • Runtime Proof. Whether the game FC2-WOLF0-c2v1_custom_struct (= v3) accepts is undecidable offline. To determine this, one would need to save the state in %USERPROFILE%\Saved Games and launch the game — both of which are excluded here. Before attempting such a test: the backups under re_probes/save_backups/ are the only copies of the real states.
  • Sub-entries in the Card Storage. The branch with Anzahl > 0 (elements at 0x30 bytes) does not exist in the entire inventory; the parser explicitly rejects it rather than guessing.
  • Component Stream from Byte ~0x2142. idQuickSaveStream with Magic 0x3832E21A before each day, non-fatal tag comparison (idQuickSaveStream::SerializeCheck: tag mismatch: is: %s, should be %s, 0x66aa60) and compressed heap sections. Excessive — whoever intends to modify munitions or upgrades there must start here.