Recently, I completely revamped the save system in the gob engine. To explain why that was necessary and what that means for old saves, I have to start with how script variables are handled, how saving works in the original games and how the old save system worked.
Script variables
From the engine’s viewpoint, the whole of the script variables is just one huge blob of memory. The scripts can then ask the engine to write into or read out of this memory area either:
- a 4 byte value at an offset divisible by 4
- a 2 byte value at an offset divisible by 2
- a 1 byte value at any offset
- a string at any offset
- a multi-dimensional array of any of these above
Moreover, some variables are also indirectly accessed by the engine, by having some structs overlain with the variables memory. For example, the objects’ animation parameter structs are directly mapped onto the variables memory, so that both the engine and the scripts can easily change the current animation frame of any object.
No index for which offsets are used how exists, so the engine only knows the type of a variable when it’s accessed.
Saving in the original engine
Like the variables organisation, saving is almost exclusively handled by the scripts. The scripts assemble the save name, which saves are already there and the like and give simple read and write orders to the engine. All save slots are written one after the other into one file. All screen shots, if the game made them, into another file.
Saving commonly looks like that then:
- Write 500 bytes out of variables offset 1234 into file “CAT.INF” offset 0 (header, containing stuff like used joker count
- Write 1200 bytes out of variables offset 5678 into file “CAT.INF” offset 500 (slot names)
- Write 50000 bytes out of variables offset 0 into file “CAT.INF” offset 51700 (save the whole variables blob, second slot)
The old save system in ScummVM
Since for ScummVM, we don’t want all slots pasted into one big file, but one file per slot, including screen shots and the like, I had to hook into the reading and writing opcodes. Generally, ScummVM now looks for know file names, caches the header and index, calculates the slot number from the whole block dump offset and then saves everything into a slot file.
Special care is given for reading the index, which the original already loads for determining which slots are filled. ScummVM simply iterates over all slot files and reads the name.
I messed up there, though. I didn’t think I had to change the format any time in the future, so I didn’t add an own header into the slot files. For Gobliins 2, they’re simple the slot name and the variables afterwards, for example.
The first problem I faced was that of big endian systems. The single variables are always written in native endian into the variables memory, so it was not possible to exchange saves made on differently endianed systems. So make that work, I encapsulated variables access, noting the size of each variable. Then, at saving time, I used these sizes to force all variables to little endian. This seemed to work, at first.
The second problem I noticed was that of Mac and Amiga versions of gob games, i.e. ones that run natively on big endian systems. Sometimes, the scripts depend on variables being written in big endian, breaking support in ScummVM on little endian systems. I then saw that it’s better to change variables access to always write them in the endianness of the original game. Not wanting to break save game compatibility, I still kept the variable sizes.
Unfortunately, I must have messed up there as well, since these two systems didn’t work together correctly, creating save files that couldn’t be exchanged between differently endianed systems anymore. Which was the main reason for me to finally create a better, non-broken save system.
The new save system
The new save system brings in some big changes:
- Save parts (variables, name, …) are now in an IFF-like container
- There’s a header, noting the game and game’s endianness
- Variable sizes don’t exist anymore
Basically, the new system fixes all problem that existed in the old ones. Saves can now be freely exchanged between different systems again. It’s quite a lot bigger in code size, though.
Old saves in the new system
And what does it mean for old saves? I’ve included a converter, that tried to convert old saves on-the-fly while loading. The old saves aren’t touched, but most of them can be loaded and then saved in the new format. Did I just say “most” of them? Yes, unfortunately, saves made on big endian systems are fundamentally broken and can’t be used anymore. There’s no header in the old saves, so I can’t even tell whether it’s broken beforehand and try to fix it too.
The marginally good side is that you can’t accidentally keep using a broken save: if it’s broken, it’s broken completely and you can’t even access the menus anymore. You will have to restart ScummVM.
In conclusion, I’m sorry for the inconvenience brought in by my short-sightedness when I implemented saving at first. I should be fixed now, and if there’s again something broken, it can be more easily changed without breaking compatibility.