Modding General Discussion

[oppie 487]

Since the game hasn't even been released yet, this discussion is going to be private for now. This thread is intended for a discussion about the focus for the current modding effort. Personally, my goal is to first have a working tool to read and edit the package files with, and hopefully from there we'll be able to modify textures and eventually 3d models. Since we don't even know how much modding (if any) is supported in the game, we'll probably have to keep our expectations and goals simple for now.

Also, I want to stress that I do absolutely not intend to use any of my discoveries to circumvent the "always online" requirement of the game. I know some people are clamoring for an offline mode and are mainly looking at the modders to accomplish this, but since that kind of cracking is illegal, I want no part of it (and I hope neither do any of you).

[warrior 169]

Do you own the simcitymodding account on twitter? If you do, could you let someone like Ocean know about 2 spelling mistakes I found?

 

 

TGI:0x0a98eaf0, 0x02fabf01, 0xb38a0dc3

"0x0ec897a7" : "Disaster arleady in progress.",

TGI: 0x0a98eaf0, 0x02fabf01, 0x6c969dee

"0x0eb0bace" : "Place inside a city box to incluence freeway attach point."

 

In SimCityData\Locale\en-us\Data.package

 

Also it appears Type 03E421F0 is do with terrain, as it appears in TerrainTemp.package and what appear to be something to with cities in SimCityUserData\Packages. I

And from looking on the internet it seems to be some sort of image histogram as the the files look very similar to part of some NASA files which are labelled as image histograms, but I'm unsure how a histogram can apply to a map, and how to read them.

[oppie 487]

Yeah, the simcitymodding Twitter account is mine. I've contacted Ocean Quigley and Dan Moskowitz to make sure that mods weren't going to be blocked by the game (like you experienced when you tried to modify something). To the credit of the SimCity dev team, they replied very quickly with https://twitter.com/oceanquigley/status/299566347438936064 and https://twitter.com/moskow23/status/299571797895028736 - both of which are very promising in terms of what we're allowed to mod.

 

We should make a list of things like spelling errors and other mistakes/bugs that we find so that we can report them all at once. Maybe it's better to wait until next week's beta as well, because maybe they've already fixed them.

 

Good discovery! I was wondering whether the city/region maps might be stored in the .package files, and maybe that's what the 03e421F0 type represents. I'd imagine that they have to store their maps (height, forest, oil and so on) somewhere, and those might be it.The _Game.package also has a few PNG's that look suspiciously like height maps.

[warrior 169]

I found these strings: 

Your cities and settings are saved online but all add-ons and bonuses in the game installation directory will be removed. Are you sure you want to uninstall 

 "Someone in the region has new content, we need to migrate your game. This will take just a moment."

 

And also references elsewhere to validating the game files, so I'm leaning towards: we can create our own packages but cant modify the existing ones. which I thought seemed pointless because you can just override subfiles based on the TGI, but then realised that by overriding with our own packages, the game can choose to use the original for online play, and the modified for sandbox.

 

The prop files in the SimCityUserData\Packages files, the IIDs are like this, but the tool didn't fill the table(?) so I just saw about two words in each file:

 

0x02843a04 - ore map

0x06f082fb - forest map

0x0a20698b - "terrainEditor"??

0x196fc73d - soil map

0x277bd429 - desirability map

0x2ffd7eed - ocean simulation? (tessendorf: http://graphics.ucsd.edu/courses/rendering/2005/jdewall/tessendorf.pdf

0x3babb8db - "heightMap"?

0x494f8678 (GID same as filename) - "environment"??

0x51e7a18d - "region"??

0x6682666b - desirabilityTwo map??

0x9b2bb418 - oil map

0xad03ba05 - coal map

0xae71a653 - water table map

0xfd3d2ed9 - "grass"?

 

(I has two packages in that directory, and the IIDs were the same in both)

 

 

And when I was trying to open the file with Type: 0x024a0e52, the tool crashed. This is potentially an interesting file because the number 0x024a0e52 in spore has come up in connection with property names.

 

And Type 0x0a98eaf0 are JSON files. (in the _App package there are normal JSON files, but in the locale package there are used to define strings)

[oppie 487]

You should be able to correctly open those specific files(0x024a0e52) with the new version of the tool. They don't contain anything particulary interesting though.

Also, yeah, I've noticed those property files. They also exist in the _Game package and seem to be representative of all the different 'maps' in the game. The most interesting to me is the 'heightmap' which might allow us to reintroduce terraforming.

The existance of a 'soil' map is also an interesting tidbit - which seems to be meant originally for farming. It seems like they were orignally planning to have more "specialization" industry types, because I've found several references to a 'grain' and 'lumber' resource (among references to known resource types like coal and ore).

For this weekend's second beta, I'm planning to prepare a package file that contains an override for a texture. I have a feeling that placing it in the \SimCityUserData folder might work, but we'll see.

[warrior 169]

In _App there is a prop file (IID: 01c7ac81) which lists Type IDs and their name Though the word creature in it suggests its left from spore?

[oppie 487]

I've checked out the file and I think it's mostly leftovers from Spore. A few of the typeIds mentioned don't even exist in any of the packages.

What I find interesting is that a lot of the 'key' properties point to non-existant type ids, a lot of times even to typeId 0x00000000, which makes no sense. I wonder if that means there's still a problem with how the key properties are being read (which I wouldn't rule out), or that the key properties don't necessarily have to point to an existing TGI combination.

[warrior 169]

I found that with some keys myself, but I haven't tried it with latest version in which you said you fixed some of the properties.

I also don't understand how some RW4 files have textures in them, but when you switch to hex view for the texture section there is only about 10 bytes. Are the textures stored in the RW4 files are the texture section merely references to other file (in which case such a small number of bytes is understandably), to me it doesn't make sense to include the textures in the RW4, what if different models share the same textures? (say a metallic chimney) it wouldnt be efficient for each model to have its own copy?

Also the raster images look very wierd, like the colours are wrong, and they seem quite small considering there is such a large number of them.

[oppie 487]

I think that the weird key types may point to a 'known type' (where the property will always point to the same type and only the instance is important) or something that's not designated by a TGI combination altoghether. I should search some of those numbers as instances, maybe that'll clear some things up.

The RW4 files are still mostly a mystery to me as well. I can explain the texture mystery though - the Texture section (20003) is only a descriptor that defines the compression, pixelformat, size and such for the textures and a pointer to a TextureBlob (10030) section that contains the actual data. Although TextureBlob is a bit of a misnomer for the 10030 sections because they also contain the vertex data for the meshes. Basically every RW4 section points to another section in some way. While I managed to pretty much trial-and-error my way to a working mesh viewer, this is definitely an area where someone with 3d programming experience could probably do a much better job than me - I know what vertices and normals are, but that's as far as my knowledge reaches.

The raster images decoding was mainly a test to see if the python script I found was any good - the RGBA values are probably jumbled, but since the files at least make some form of sense, at least I'm on the right track with them. I don't know why they're so small, I expected them to be bigger as well, but maybe that's a mipmap thing. Currently I'm still focusing at so many areas at once that many of them are still very much unfinished.

[warrior 169]

Have you got a key for the beta? I have two so if you need one,

I know from SC4 that textures were external to models and referenced by Instance only, also in RUL files only instances were used. I think the majority of references were by instance only, so perhaps it's the same here?

Also in SC4 models, each point would have 5 coordinates, 3 for the model (x,y,z) and 2 for the texture (u,v).

[oppie 487]

I have a key for the beta, so don't worry. I've been doing some tests, but the results were not quite what I was hoping for. Here are the things I've tried:

New package file with modified png file

I figured that the most simple thing to start with was to export one single TGI index to a new .package file and modify one of the menu item pngs to see if the game allows overriding TGI's. This gave me the "cannot create city" error when starting the game (something which I believe you saw as well during the first beta). I put this file in a multitude of locations (almost every folder related to the game) but no luck. However, this might be because the .png compression in the package files is different from what photoshop generates (haven't tried all the different settings yet)

New package file with non-modified RW4 file

When I was testing the 'export to package' function, I made one where I exported one single RW4 model. Considering that the game apparently reads all the package files in any of the game folders (judging from the fact that putting my modified one in almost all of the locations made the game crash), I wondered if putting a non-modified one in there was going to have the same result. Interestingly enough - it didn't - the game loaded fine, which leads me to believe that there's still something about the package format export that I'm not getting right.

New package with modified TGI information

So, with the end of the beta in sight, there weren't many things I could try anymore. I had hoped to do a texture replacement test, but as you noted in the other thread, that's still very buggy. The final option to see if TGI overriding worked was to use the working .package file and modify its TGI value to override another (in this case, I tried overriding one menu icon with another, I only changed the instance id). The game loaded fine, but nothing was being overridden.

Anyway, I don't want to draw too many conclusions from this because one day was really too short to try anything substantial. There are some tests I wish I could have done (like switching around two TGI values in the _Game.package itself) for which the tool is still too unstable, and I still suspect that overrides only work in sandbox mode (which can only be tested on the release date).

I know from SC4 that textures were external to models and referenced by Instance only, also in RUL files only instances were used. I think the majority of references were by instance only, so perhaps it's the same here?

Also in SC4 models, each point would have 5 coordinates, 3 for the model (x,y,z) and 2 for the texture (u,v).

Yeah, in SC4 things were a lot simpler. I don't know why they did it this way, but yeah, I'm assuming that a lot of the references are by instance only.

The 3d models are a lot different. Not only is there the xyz and uv information, but there's also normals, "bone weights" and lots of other stuff stored in the vertex information alone. The different types of models (there's a clear difference between buildings and vehicles for example) even have differently sized vertex data. Only the 'xyz' information (and the normals information for the buildings) seems to be completely reliable for now.

Speaking of interesting stuff I've noticed during the beta - have you noticed how most of the buildings seem to have 'modeled interiors'? Well, they don't - it's some kind of effect they managed to pull off simply by using textures (with some kind of displacement mapping).

[warrior 169]

Strange that your game crashed. I tried overriding some strings from the Data.package in the Locale folder, I extracted file into it's own package, and put it everywhere twice, (one where the filename started with a and the other with z as I know in SC4 the fils were loaded alphabetically). The game didn't crash but nothing changed.

I tried extracted the file using both your tool and Sims3PE, it made no effect.


I thought maybe it was just because it was strings. So I tried PNG, extracted it, modified it, put it in it's own package (tried using both s3pe and your tool). No effect, and no crashing.

I even tried naming the override package "SimCity_Overide.package" incase the game only likes that prefix. No effect.

Finally I decided to just modify the original package, and replace the PNG with my edited one directly. And it worked. The game didn't crash and the image was replaced. So thats Existing package with modified png file 

 

 

If I remember correctly this seems to contradict what I tried in the first beta, where I modified the package and it got stuck loading the city as if it had checked to see if the file had been modified. 

[oppie 487]

Finally I decided to just modify the original package, and replace the PNG with my edited one directly. And it worked. The game didn't crash and the image was replaced. So thats Existing package with modified png file 

Then I guess congratulations are in order for being the first person to create a working SimCity mod

 

That's pretty good news - I was worried that Maxis was blocking all modified files after all, but at least this proves it's going to be possible (although I hope there's going to be a way that doesn't require having to change the original package files). I hope we'll be able to override files when the game is released (in a "downloads" folder or something), or maybe this is what Maxis means when they say they "won't support modding at launch". We'll see - at least this proves that there's nothing on the server side that's blocking mods.

 

 

I think the reason my game crashed might have something to do with the .png I used being in the wrong format (compression/interlaced might be off), because it only crashed when using that file.

[mazman34340 1]

This might not be a good thing. This easy modification is great but EA and Maxis won't be happy that their system has been 'compromised'. Remember the Battlefield bans for the color mod?

It just takes one troll.

 

Best thing to do is be relatively quiet and when Maxis greenlights modding, we already know what we need to do.

[Ma8thew 77]

This might not be a good thing. This easy modification is great but EA and Maxis won't be happy that their system has been 'compromised'. Remember the Battlefield bans for the color mod?

It just takes one troll.

 

Best thing to do is be relatively quiet and when Maxis greenlights modding, we already know what we need to do.

Come off it. All of this is completely legal. Maxis never gave any official help nor approval to the Simcity 4 era mods.

[w0lfalot 18]

Finally I decided to just modify the original package, and replace the PNG with my edited one directly. And it worked. The game didn't crash and the image was replaced. So thats Existing package with modified png file 

Then I guess congratulations are in order for being the first person to create a working SimCity mod

 

That's pretty good news - I was worried that Maxis was blocking all modified files after all, but at least this proves it's going to be possible (although I hope there's going to be a way that doesn't require having to change the original package files). I hope we'll be able to override files when the game is released (in a "downloads" folder or something), or maybe this is what Maxis means when they say they "won't support modding at launch". We'll see - at least this proves that there's nothing on the server side that's blocking mods.

 

 

I think the reason my game crashed might have something to do with the .png I used being in the wrong format (compression/interlaced might be off), because it only crashed when using that file.

Hey, I know this thread is old and all, but since you were on topic I wanted to ask you up front:

 

Are there some types or individual resources that can't be overriden with a .package file ? Some .js won't be changed (for SimCity_App.package in my case) and can only be hardcoded ...

 

Or am I missing some detail ?