Sprite extraction - XYZ.dat

[Danny50205 71]

Hey alll

 

I have done some searching and nothing has come up. so i thought i might ask!. 

 

Is there any program or any way to extract the games original sprites from the .dat files?. 

I have tried Wattos game extractor. but it sees the files as  "Close Zoom Image"(czi) & "Far Zoom Image"(fzi), Which helps none. as the files don't show any extension information.

So i was wondering if anyone knows of a old tool or a viewer to see the sprites. i am not worried about nitty gritty of importing the files. i am just looking to view or extract them into a viewable format. 

I included some fzi & czi for reference or if someone knows how to decrypt them? 

Thanks

D.

Roads.dat.rar

[TheMurderousCricket 5,107]

I'm afraid it cannot be accomplished without the source code, or at least dev tools.  Otherwise we would have already been there. 

[asasnat 222]

the .DAT files within the Sprites folder contain the letters "BM" (in ASCII) in quite a lot of places. If you don't know, that's the header field for a .BMP file. Those instances of BM also tend to appear near similar or the same patterns of neighboring letters. It is entirely possible that the files are just huge libraries worth of bitmap image data. Hopefully EA won't sue me for revealing that piece of information.

(Sorry if this was hard to understand, my posts always end up like this when I try to explain something ultra nerdy.)

On 11/22/2021 at 8:32 PM, TheMurderousCricket said:

I'm afraid it cannot be accomplished without the source code, or at least dev tools.  Otherwise we would have already been there. 

 

[CorinaMarie 29,894]

2 hours ago, Bruh Man said:

Those instances of BM also tend to appear near similar or the same patterns of neighboring letters.

If you copy out the hex code between instances of BM, does that then conform to the BMP file format specification? I presume you've tried that saving as a separate file with a .bmp extension and then checked if it can be viewed with image software.

[asasnat 222]

46 minutes ago, CorinaMarie said:

I presume you've tried that saving as a separate file with a .bmp extension and then checked if it can be viewed with image software.

nope, I'm just making a speculation. Maybe I should do more research into this.

EDIT: I eventually concluded that it can't be stored in the .BMP format. The math just doesn't add up. The header field specifies a size for the file, but it never was correct, often being larger than the file it was stored in. Once again I tried and failed.

  Edited January 14, 2022 by Bruh Man  

[CorinaMarie 29,894]

5 hours ago, Bruh Man said:

The header field specifies a size for the file, but it never was correct, often being larger than the file it was stored in.

But wait, what if that size is a decompressed value? Like the data following the header could be all smooshed up in DBPF format or such.

 

5 hours ago, Bruh Man said:

Once again I tried and failed.

Just because you discovered a specific way it isn't laid out doesn't mean failure IMO.

[asasnat 222]

On 1/15/2022 at 12:48 AM, CorinaMarie said:

But wait, what if that size is a decompressed value? Like the data following the header could be all smooshed up in DBPF format or such.

No chance. The reported size was a thousand MB or so. A stupid size for a file, especially considering that it was stored in a file 100x smaller than that, and also that the game was supposed to fit on a CD.

[asasnat 222]

Hold on! Does anyone remember that landmark converter utility for SC3U?

If we can reverse-engineer how it packs up the files into something the game can read, we might be able to extract the many sprites hidden within the game. Not guaranteeing that it will work for sure, but it might at least work for the buildings.

[Cyclone Boom 17,031]

1 minute ago, Bruh Man said:

Does anyone remember that landmark converter utility for SC3U?

Maybe this one?
 

 

[asasnat 222]

1 minute ago, Cyclone Boom said:

Maybe this one?
 

 

Yes! Exactly that!

[Alan Doherty 13]

On 1/14/2022 at 8:01 PM, Bruh Man said:

nope, I'm just making a speculation. Maybe I should do more research into this.

EDIT: I eventually concluded that it can't be stored in the .BMP format. The math just doesn't add up. The header field specifies a size for the file, but it never was correct, often being larger than the file it was stored in. Once again I tried and failed.

I did a lot of looking into this, including getting *some* source code from some of the old dev's (see this post). I thought the difficulty in reading the format was due to some compression, which turned out to be a dead end. My goal was to extract the sprites for fun and some hobby projects.

The database format isn't off the shelf as far as I can tell, I remember some suggestion to buy some commercial IBM software. It's part of the EA Rizzo/Gonzo engine which the game uses, references to the muppets are intended. For reverse engineering, the Linux, UK and Unlimited versions all provide some crossover and I remember finding a golden copy that had a lot of debugging symbols included.

The databases do contain bitmaps, used by the UI graphics - but the sprites for buildings and such are spread across multiple "entries" in the database. One containing metadata possibly, another the actual rendering data. Some very old developer blogs talk about how the engine started as 3D, and was moved to 2D after a certain amount of development had already taken place. My theory was that the sprites weren't just being loaded, they were being rendered - possibly by combining smaller sprites together. Unfortunately the "block placeholder" sprite you see before the sprites pop in might just be because the engine reads sprite entries as needed, rather than because it's needing to render them to a flat graphic on demand.

I also found some archaic FTP server from EA, which talked about the file formats for Sims Online/something similar. The database was either similar or the same codebase.

[asasnat 222]

I knew that SimCity 3000 was originally supposed to be a 3D game, but I didn't know that the rendering method the finished product uses is more or less the same as back before the game was downgraded by one dimension. Certainly interesting indeed! (And yes, I know this is all just speculation)

[Alan Doherty 13]

This motivated me to do more digging, it looks like some form of run length compression is used, I've attached a project I was using to open the IXF files. With some tweaking of the `Program.cs` and changing to a newer .NET, you'll be able to extract databases entries at will.

Each sprite has two entries in the database, one for some kind of metadata and one with the actual sprite data. Sprite "attributes" or something similar is a grouping of many sprites, typically providing 20 sprites. One for each rotation level (1-4) and zoom level (1-5), with support for more frames when the sprite is animated. These sprite attribute entries are text, so much easier to read. They point to the database entries by type, group and instance in hexadecimal. There's building sprites, and effect sprites as well as some other categories.

If you have access to a debugger/reverse engineering tools, it looks like the actual decoding of the sprites into graphical data occurs in SIMSPR.dll/SIMSPR.so. If you use the Linux version you'll indeed have very detailed debugging symbols, but running it is quite difficult and I haven't been able to. Debugging the Linux version would be ideal.

sc3-redux-master (1).zip