discovery Modifying .sc4 savegames: It IS possible!

[smf_16 1,835]

Looking at the code, it looks like the error originates here, indicating that the tool somehow can't find the SimGrid where the zone data is stored. This is required to allow the lot to be redeveloped later on - which wasn't possible with the initial version of the tool.

What do you see when you open the ZoneView in game? Does it show something strange there?

[justforfun 7,802]

21 hours ago, smf_16 said:

What do you see when you open the ZoneView in game? Does it show something strange there?

Thanks for looking into this. The zone view looks to me the same as in the other cities. I see no RCI colours at all as I plopped 100% of the buildings and none was growified, so it just shows the transport network and all blank terrain which is what I see on plopped zones in the other maps. 

[smf_16 1,835]

Well, my guess then is that the required SimGrid entry hasn't been created yet by the game because it didn't need it yet in the simulation. I would run a single game month, save again and then see if it works now. Make sure to keep a copy of the old city where the simulation has not run yet though in case something goes wrong!

It should be possible to modify the program so that it adds the SimGrid file if it does not exist yet, but that would require a bit of efforts, for which I currently don't have the time unfortunately.

[justforfun 7,802]

18 hours ago, smf_16 said:

I would run a single game month, save again and then see if it works now.

Done it, and unfortunately I got the very same error message.   It's weird because the train station is being used. 

[CorinaMarie 29,605]

@justforfun What about zone something like a 1 x 2 industrial, grow it, then save, and try again?

[justforfun 7,802]

26 minutes ago, CorinaMarie said:

@justforfun What about zone something like a 1 x 2 industrial, grow it, then save, and try again?

Hi, sure I can try however I growified with no problems other maps that had never been run and only featured a few plopped residentials. I can't fathom what the problem might be with this one.

UPDATE: done that too, and still the same error as I feared. 

[Wiimeiser 444]

I think this might be the point where we send the city in and compare the internal data.

[justforfun 7,802]

19 hours ago, Wiimeiser said:

I think this might be the point where we send the city in and compare the internal data.

I can send it to anyone kind enough to analyse it 

[bladeberkman 714]

Here's an interesting little growify experiment. You may be familiar with Reddonquixote's Eureka Tower. There are two versions of Eureka Tower on the STEX: A ploppable residential and a non-functional landmark. You may also remember that this was among his early BATs and he didn't realize that a plopped residential building would inevitably abandon (he would later upload the landmark version as a separate download). I plopped the ploppable residential version and growified it. A residential zone now appears beneath the building and Sims are able to leave the building and find jobs. This is a building that was specifically modded as a ploppable residential building. Amazing! The only strangeness: When you hover the route query tool over the building, the query is still the usual landmark orange and it says "Workers" instead of "Commuters."

-Blade

[justforfun 7,802]

Ok, here I am with a new bug!

This time the central map of my region no longer opens. When I click on it to run it, I get a CTD.  In the past I sometimes had CTDs as soon as the city opened, and the problem was due to my tinkering with some building exemplars. But now the CTD happens from the region view just a couple of seconds after the city starts loading. When I try and open it through SC4 Startup Manager, the program just freezes. 

So I tried growifying the city to check whether there was something unusual about it, and indeed this is the message I got:

Perhaps this gives hints as to why I can't open it? Thanks in advance to @smf_16 or anyone with insights. 

[rsc204 21,808]

3 hours ago, justforfun said:

and indeed this is the message I got

So why not make a Backup (I'd do this for the entire Region at this point), then see if running that function helps, worth a shot right?

[smf_16 1,835]

Based on the error, it looks like the game has somehow corrupted one of the SimGrid entries when saving. It errors on the decompression stage which indicates that the compressed input is malformed. I'm not sure which version you're running of the tool, but in later versions I've rewritten the qfs decompression algorithm and ported it to JavaScript. You could try using the newest version and see if the error is a bit more specific. However, if I'm not mistaken, the newest version requires a Node version that doesn't run anymore on Windows 7 iirc, so before that make sure you know your current version by running sc4 --version so you can downgrade again later to that version using npm install -g sc4@0.0.x.

[justforfun 7,802]

50 minutes ago, rsc204 said:

So why not make a Backup (I'd do this for the entire Region at this point), then see if running that function helps, worth a shot right?

Hi, which function should I run then?

@smf_16 I have node 13.14.0   x64

Suppose I manage to growify the city with the newest version, do you think this would fix the savefile so that I could then open the city? 

  Edited June 19, 2022 by justforfun  

[smf_16 1,835]

7 minutes ago, justforfun said:

Suppose I manage to growify the city with the newest version, do you think this would fix the savefile so that I could then open the city? 

Theoretically yes, but the new version doesn't really do anything different, it just doesn't use compiled C++ code anymore to decompress, but a pure JS implementation that I wrote. In the most optimistic scenario there was a bug in the original C++ decompression which is no longer present in the JS implementation, but I'm 99.99% sure that won't be the case since you're experiencing CTD's both in the game and the SC4 Startup Manager, meaning those algorithms would be bugged as well.

That said, trying to decompress with the pure JS implementation may provide a more detailed error of where the decompression exactly fails and maybe that points somewhere.

[justforfun 7,802]

1 hour ago, smf_16 said:

However, if I'm not mistaken, the newest version requires a Node version that doesn't run anymore on Windows 7 iirc, so before that make sure you know your current version by running sc4 --version so you can downgrade again later to that version using npm install -g sc4@0.0.x.

You are right that I have Windows 7, so how do I run the Node version that your newest upgrade requires? Running the command you indicated gives me 0.0.24

[smf_16 1,835]

Apparently v0.0.24 is the latest version already - glad to know that it still runs on Windows 7. Unfortunately, it means that there's no more detailed error message, so it's not clear how it got corrupted.

[THijSCiTy 141]

wow I go download your program and see if it works for my project, did you also make any programs that can modify the map files / savegame files?

[THijSCiTy 141]

I've been working on breaking the CRC algorithm, and got success!!

The method seems to be named CRC32 Unreflected, this method was used in late 90s and early 2000's for doing integrity checks in game data to make sure they are correct.
Also online sources citate that only the first 250 000 bytes of a section are calculated and the CRC is calculated over the Memory address + data field.

Now, a python script has been put together which can commit edits to each section in the savegame file!!!

 

 

def crc32_unreflected(data_bytes, poly=0x04c11db7, init=0xffffffff, xor=0): crc = init for byte in data_bytes: crc ^= (byte

# Example: Insert your memory + data hex string here (no spaces) hex_string = "1C18E01C0C00020000000000B206C20500000000010000003600000001000000360000005F8C2B0001000500000054524F4245080000004E657720436974790D0000005448494A53204C55494B454E530500000054524942450000000000000000B81EF442D7A317420000384200000000000080450000804500008041000080410000803D0000803D00010000000100009C17002836A5EF89DC67F01C8AC390099C9F012893BD90491887CD1C4D968F095C14002810BD90A91C14E01C2D9B8F2920860928019B8F4920860928019B8F491C140028C06290294456A40F478FC4891C050A2F3BCDA6E99C27AB2AAED409EA2028AB2AA14920AAA028AB2A280A23EAE46D761FBCE63FAA1CF8CA2AE0BF90A91CF8062842C1902918FC0628E5C190291CCEA90FD8BD90098C36A10F01BE90E9204D07287ABE90899C1E002880C0908920F6CA2A9AC090891C590728F2BF902924F606285AC090291C060A2F45C090E91C58072834C090C95C503F2D22C090A91C070A2F8EC190E91C29AB2AA4C090C99C29AB2A8EC090A91CCCA90F75C0900924F4CA2A4FC09069201CE01C65C090891C04E01C05C090091C10E01C13C09049BC3F8C03AAC1906920FA0628BCC190291C4874298134410A20A509284E661B4A20030A2F03CA14CA78DA8E03B2820F6AD4F48B033B9D8FA9FC62160F179D8FC96C368E034A9D8FC9D437130F289D8FA92488092816D60D4A242AAB2AFB3638AA1CF4062825958FE91C080A2F29F9E7A7B85A3F2D36BAD4EA68E690038143B10B1C090A2F46BD90C901" # From your new dump data_bytes = bytes.fromhex(hex_string) new_crc = crc32_unreflected(data_bytes) print(f"New CRC: 0x{new_crc:08X} (Little-endian bytes: {' '.join(f'{byte:02X}' for byte in [(new_crc >> (i*8)) & 0xFF for i in range(4)])})")

with this code I changed my city name from TRIBE to TROBE!! wohooo, next thing Im gonna do is making edits to the map size, as that was my inspiration

[THijSCiTy 141]

so what I try to do next is make a blank city and shift the dimensions from 64x64 to 63x65, this is a trick I used in the past to shift map sizes without needing to edit the data size

[THijSCiTy 141]

I will provide a step by step tutorial how to make the program work:
Step 1: Install etc. python if necessary
Step 2: Save this code as python file named CRCCALCULATORRAW.py 
 

import sys

def crc32_unreflected(data_bytes, poly=0x04c11db7, init=0xffffffff, xor=0):
    crc = init
    for byte in data_bytes:
        crc ^= (byte << 24)
        for _ in range(8):
            if crc & 0x80000000:
                crc = ((crc << 1) ^ poly) & 0xffffffff
            else:
                crc = (crc << 1) & 0xffffffff
    return crc ^ xor

def main():
    if len(sys.argv) != 2:
        print("Usage: python sc4_crc_calculator_raw.py <binary_file_path>")
        sys.exit(1)
    
    file_path = sys.argv[1]
    try:
        with open(file_path, 'rb') as f:  # Binary mode for raw data
            data_bytes = f.read()
    except FileNotFoundError:
        print(f"Error: File '{file_path}' not found.")
        sys.exit(1)
    
    # Cap at 250,000 bytes
    if len(data_bytes) > 250000:
        data_bytes = data_bytes[:250000]
        print("Warning: Input truncated to 250,000 bytes.")
    
    crc_value = crc32_unreflected(data_bytes)
    
    print(f"Computed CRC: 0x{crc_value:08X}")
    little_endian_bytes = ' '.join(f'{byte:02X}' for byte in [(crc_value >> (i*8)) & 0xFF for i in range(4)])
    print(f"Little-Endian Bytes: {little_endian_bytes}")

if __name__ == "__main__":
    main()

Step 3: Go with a hex editor inside your savegame file after you made backups of them and select the data you want to compute the CRC on and save the file named as RAWHEXDUMP.HEX
Note: Be sure to select the data part where the memory index number begins right after the block size and crc.

Step 4: If you are in windows, make a windows batch file called "Run Calculator.bat"

Step 5: Put inside the following lines:
python CRCCALCULATORRAW.py RAWHEXDUMP.HEX
PAUSE
CRCCALCULATORRAW.py Should call your saved python file
RAWHEXDUMP.HEX Should open your saved hex data
PAUSE is just to prevent the window from automatically closing so that you see the computed CRC

The little endian bytes are the output crc code.

Step 6: You can make updates to the hex dump and/or savegame file to update the new crc, please note that if the size of the data changes, that you must also update the size header also. 

Info: the code in my previous post directly calculates the crc checksum of a ascii string hex dump in the input field.

Hopefully someone can incorporate this method into the SimCity exemplar reader program, then we can make REAL edits to savegames haha!!
For now I have to do with the manual mode, well after 22 years I feel happy to have publicly shared a way how to edit savegames in SimCity!! wohoooo