SimCityPak - Modding Tool

615 posts in this topic

Posted

It all depends on how much of the game we'll be allowed to modify (will there be checks server side?) but I'm optimistic that sandbox mode will provide the freedom we're looking for. From what I've gathered looking through the game files, this opens up a lot of interesting possibilities. With a bit of luck, we might even be able to mod in terraforming (the tree tool writes to the 'forestheightmap' - all we have to do is get it to write to the normal 'heightmap' instead).

Anyway, if you've been using SimCityPak, you've probably noticed that some areas work better than others. I thought I'd write up a short summary of the current state of the game file deciphering progress for the most important file types:

Package files:

The DBPF structure is completely deciphered and I have yet to get any error there. While I haven't tried writing to a package file yet, the Spore object model I'm using seems stable enough to work for SimCity.

Property files:

There are some major differences between how these worked in Spore and SimCity. While I've got most of the specific property types to work, ArrayProperties are still a matter of concern. The problem is that if one property fails to read, all subsequent properties will also fail (because I can only pick up reading where the previous property left off). This means that to get properties to work a 100% of the time, all of them need to be read 100% correctly. This means analyzing the file byte by byte, and while I've made some progress there, I wouldn't at all call it 'stable'. Nevertheless, while there might be missing properties in a lot of the files, a lot can be done to analyze them. For example, I've managed to figure out how the game uses TextsProperties to link to a certain localization string, something I hope to integrate into the tool at some point.

RW4 Files:

Besides the textures, not much of these is readable. While all the individual sections can be read, the contents are either unknown or outdated for the Spore object model I'm using.

RASTER Files:

These contain textures (for buildings, I'm assuming), but are still unreadable. I've got a python script that is supposed to be able to translate them into something viewable, but I haven't yet had enough time to look at it.

Share this post


Link to post

Posted

One thing that confuses me is that I cant see much (if any) difference between the latest DBPF 2.x and the new DBPF 3. I wonder why they increased the version number at all?

 

And I tried access the source code on codeplex, but it says its not published, my account's username is jon889.

 

And what is the difference between TGIRegistry and TypeRegistry?

Share this post


Link to post

Posted

One thing that confuses me is that I cant see much (if any) difference between the latest DBPF 2.x and the new DBPF 3. I wonder why they increased the version number at all?

 

And I tried access the source code on codeplex, but it says its not published, my account's username is jon889.

 

And what is the difference between TGIRegistry and TypeRegistry?

 

Yeah, I can't see much difference there either - I don't know what warranted the version change but I don't mind - makes it easier for us.

 

For now the project hasn't been published yet. I've given your account developer permissions  so you should be able to access it now.

 

TGIRegistry was a previous attempt of mine to save information about the types/groups and instances that didn't work out as well - only TypeRegistry.xml is used right now.

 

Also: I'll be uploading new versions of the tool often, so check this thread and the changelog for updates.

Hellken likes this

Share this post


Link to post

Posted

I've uploaded a new version - I managed to find out why the RW4 mesh information wasn't being read correctly, so I was able to add a 3D viewer for the meshes. This should make identifying which property file represent which in-game object a lot easier!

mesh1.jpg

Shengkaijan, JoeOh and Hellken like this

Share this post


Link to post

Posted

Once again I've uploaded a new version of the tool.

Also, feel free to post any comments you might have; bugs, ideas for the user interface, interesting observations - anything is welcome!

Hellken likes this

Share this post


Link to post

Posted

The link to 0.005 doesn't work. 

 

It would be good if you could add the ability to export a subfile, I've added this myself but I cant for the life of me get Team Explorer/CodePlex to work with Visual Studio Express.

 

Also I find it impossible to open large files, it eventually stops completely, and I've got out of memory exceptions, so a size column would be good to have too.

 

Also I can't build the project because WriteableBitmap has no method "SetPixel" (and also there is no XNAControl, though I have the XNA framework installed).

 

The ability to search through the whole package would also be good, even if it isn't quick.

Hellken likes this

Share this post


Link to post

Posted

Thanks for the heads-up! I've fixed the link to the latest version.

The reason why you're getting build errors is that some of the referenced dll's pointed to locations specific to my system. I've done a quick update of the code to remove all unnecessary references (like the xnacontrol) and moved the others to a location that can be referenced as a relative path. I've also included the referenced libraries and the folder they're in in the Team Foundation Server. In the checked-in code, I've also added the 'export' context menu option and managed to get all the individual properties to read correctly. (only very few files still have an error). A search function is also something I'd like to have, but for now I'm going to try to focus on saving/loading package files and importing/exporting textures so that I hopefully have something to test with for the beta this weekend.

What kind of troubles are you having with Team Explorer? I'm using the built-in version from Visual Studio 2010 premium - but I read somewhere that for Express you have to use some kind of stand-alone version?

Hellken likes this

Share this post


Link to post

Posted

When I make a change in VSExpress, and then go back to the stand alone Team Explorer, it doesn't show any changes for me to act on.

I've just tried replacing a texture in a RW4, when importing I couldn't see any files (including .dds files) so I had to remove the filter from the open dialog. Then when I did import the new texture I get a NotSupportedException in ViewTexture.xaml.cs on Line 234 (textureSection.Write(null, section, strm) ;)

Hellken likes this

Share this post


Link to post

Posted

Hmmm, strange - I haven't really worked with the standalone version of Team Explorer a lot, but I'd assume that it'd compare the files in TSF with the ones in your local working folder and update the changes accordingly.

The texture import function doesn't really work yet. I wanted to get it to work for a test in the beta, but I was a bit too optimistic as to how easy it would be to implement.

Hellken likes this

Share this post


Link to post

Posted

I think it's because you have used write() rather than read(), on the import texture function. Write() means write the data already in the rw4 file to a stream. And Read() takes a stream and "reads" it into the rw4 file. (I think anyway)

I changed the write to read, and it worked a little more, but then it didn't like that the first parameter was null. But I don't know enough about WPF to know how to get the RW4Model from the parent MeshView to the TextureView. (the whole DataContext and WPF confuses me)

Hellken likes this

Share this post


Link to post

Posted

I think it's because you have used write() rather than read(), on the import texture function. Write() means write the data already in the rw4 file to a stream. And Read() takes a stream and "reads" it into the rw4 file. (I think anyway)

I changed the write to read, and it worked a little more, but then it didn't like that the first parameter was null. But I don't know enough about WPF to know how to get the RW4Model from the parent MeshView to the TextureView. (the whole DataContext and WPF confuses me)

 

Yeah, I already changed that in my own local version, but like you said, it still doesn't work - it's because the RW4Model splits some of the header info that defines what kind of texture it is (directx compression, size) from the actual texture data, which it then stores in different sections of the RW4 file.

 

WPF is pretty cool once you get the hang of how it works. I use it for most of my own projects these days. It's especially great when combined with the MVVM pattern, but in this case I went with a much more quick and dirty approach because I wanted to crack those package files open. Speaking of quick and dirty - while I'm trying to refactor a lot of older code, some parts of the application are still a bit of a mess. Eventually I'd like to have a simple and straight-forward object model for package and TGI manipulation that people can build on.

Hellken likes this

Share this post


Link to post

Posted

I've uploaded a new version (to codeplex this time, because my own site seems to be down for the moment, let me know if you can't access it). I've adopted a new (more sensible) version numbering, so this version is called 0.1.0.0. The most exciting new features of this version are the addition of a texture viewer to the 3D mesh preview (this only works for vehicle models for now):

ztoFMOZ.jpg

Also, to help identify properties, you can now load your locale files from the new Settings... dialog. This will load the localized strings into memory and will display them appropriately in the PROP files. This should make identifying those a lot easier.

NOTE: I will be on vacation from 23-feb to 11-mar

This not only means that I'm sadly not around for the game's release, but also that as of today, I won't be able to do any more work on the tool. Warrior, I've given you coordinator permissions on the codeplex project so that if needed, you can manage that. I've also given simcool developer permissions, so if he wants to he can join in on the development as well. Feel free to make any changes you think are necessary.

Like I said, I also won't be around on the game release date. I was originally hoping to make the tool open source on that same day - and honestly that could still be possible. From what I can see on CodePlex, warrior should be able to publish the project (which should be done before march 5th because otherwise it will be deleted) and if Dirk unhides this forum (I'm not sure what the plan is for release date), everyone should be able to find it.

Hellken and JoeOh like this

Share this post


Link to post

Posted

Wow thats brilliant, strange that it only works for vehicle models though? Are they in a different format or something? And the localized settings sounds interesting,

 

sorry I haven't replied much, I'll publish the project, but I'm not sure about making the forum public on the release day, at least not without seeing to what extent modifications can be made, not sure how happy Maxis will be if suddenly the game breaks because everyone is fiddling with it :) 

 

If it's that kind of vacation, I hope you have fun :)

Hellken likes this

Share this post


Link to post

Posted

It only works for vehicles because the RW4 meshes are different for different types of models - the information stored in a vertex, mainly. While the x, y, and z data is always stored in the same bytes, the uv mapping, normals and bone weights seem to be different from model to model - and even then there's more data in them that I just ignore for now. Besides that, the vehicles are the only objects where the textures are reliably stored in the RW4 file itself.

...I'm not sure about making the forum public on the release day, at least not without seeing to what extent modifications can be made, not sure how happy Maxis will be if suddenly the game breaks because everyone is fiddling with it :)

I've thought about this, but I don't think it can do much "damage" to make this forum public. First of all, the tool currently doesn't really support any fiddling with the files, because it's still much more of a viewer than an editor (speaking of which, making the tool able to save packages and the readable file types should be a priority for the next new feature). I've experienced that finding people who are willing to participate is very hard when you're doing it in secret, and I'm hoping that the more people are experimenting with the tool, the more we'll find out about the modability of the game. Besides, Maxis already knows that we're working on modding, and I'm sure they're well prepared for it. But since I'm gone on the release date, I'm also fine with waiting a week or so.

If it's that kind of vacation, I hope you have fun :)

Thanks! I'm going to South Africa, I'm sure it's going to be amazing!
Hellken and jarichp like this

Share this post


Link to post

Posted

I've downloaded the latest version and the loading from locale file makes everything a lot clearer :)

Though you would seem to have a different locale file than me, because the tool crashes when it came across a comment in the JSON files in the locale package, I had to add a check for comments in the code, which implies you didn't come into that problem? Windows says my locale package is 450,408 bytes (not "bytes on disk"), from beta 2 files (the crash also happened on beta 1 files).

 

There is one more thing I'm curious about, in the screenshot below, it says the value refers to a JS file:

 

K5fchZ9.png

 

But when I search the blue number as an instance ID no file is found (in any of the SimCity_* packages), so how did the tool know it will be a javascript file?

Hellken likes this

Share this post


Link to post

Posted

I have Dutch locale files, so that's probably why it didn't crash for me.

The key properties sometimes contain a full TGI link, from which I can identify the file type the key should be pointing to. When you click the link, it searches the current package for that exact TGI instance. If it's not found, nothing happens..

Hellken likes this

Share this post


Link to post

Posted

I'm not sure if you'll see this, and I feel bad for saying on your vacation. But only the project owner can set the license, which has to be done in order to publish it.

 

Otherwise, I managed to get the Team Explorer working by using a trial of VS 2012 (horrible UI though), and I committed a change that makes the property links work in the ViewWindow.

Hellken likes this

Share this post


Link to post

Posted

No worries, I have wifi from time to time. Could you try to find out which license would be best for our project?

Hellken likes this

Share this post


Link to post

Posted

I've picked a license and published the project so it doesn't get deleted.

I see you've done some checkins, keep up the good work!

Hellken likes this

Share this post


Link to post

Posted

It's been quite addictive, but I haven't the skills to do anything more complicated. But I hope I've made it easier to work thing out, at least with the PROP files.

Share this post


Link to post

Posted

You guys are cool.  Your interest in this (modding SC13) is highly appreciated. I was trying to read your progress, but I dont understand much.  If you ever figure out how to bring the joy of SC4 to SC13, devs should pay you because you'll keep this game around forever like modders did for SC4.  Good Luck. 

 

Oh a few ideas for future thinking:

1. Allow/change game to upload/save to local drive

2. Expand city size limit, or build outside the dredged white dotted line.

3. Farms

4. Self dependant city

 

I know that any of that stuff is probably impossible (you dont have to explain why), but seriously to us simple people - modders are the last hope for enjoying something new like the SC13.

Share this post


Link to post

Posted

2. Expand city size limit, or build outside the dredged white dotted line..

 

If they could do this it'd be amazing.

Share this post


Link to post

Posted

While I'm not sure if you could focus on it with this tool, I am sure that two of the biggest things would be the ability to create your own buildings and the ability to save to your own computer, offline. It sounds like you've all got some work done on the former, so kudos to you guys! It's projects like these that provide the best chances for salvaging this game. great work! :)

Share this post


Link to post

Posted

Expanding the city size is unlikely. while it's possible that the city size is stored in the package files, creating a region with these larger tiles seems highly unlikely. Perhaps it will be possible in sandbox mode, and I'm 99% certain we can't expand the city sizes for multiplayer and single player, as Maxis will have put checks in place for modifications to preserve the integrity of the leaderboards.

 

There's more chance we'll be able to change things like the textures of the roads (Euro mod for example), but this may not work because the game may not allow any changes, included pure cosmetic ones in the non sandbox modes. 

 

Saving locally is heading into the piracy side of things, and isn't something that would be done by a tool for package files.

 

I'm in Europe so I don't get the game until Friday, and Oppie is on vacation atm, but when I get the game I'll be seeing what we can modify in what modes (if someone hasn't already).

 

If we are able to mod things, then the amount of what we can change looks like more than SimCity 4 (the UI seems less connected to the EXE than in SC4 for example)

Share this post


Link to post

Posted

God I can't wait for modding tools to be available.. 

Share this post


Link to post

Posted

Once modding does become possible, most likely the only things that will be able to be changed are textures and skins for various objects or buildings.  Adding completely new content probably won't be possible due to the EA server connection.  And of course changing any of the physical abilities of the game is unlikely.

 

But amazing work so far!  It took years for SC4 to be modifiable; who would have thought someone would start on this game in a few days?  Bravo!

Share this post


Link to post

Posted

Well, that was quick.

 

I'll probably get in touch with some of the others and ask their opinion on rejoining dev work, but otherwise, I'm in as usual.

Ma8thew likes this

Share this post


Link to post

Posted

Been a long time since I posted here.

 

My college has Maya, UDK, Z-Brush, and Photoshop on all their computers. While I don't know anything about character rigging (yet) or programming, I would like to start making content for the game for the sake of my portfolio, as a hopefull future game designer. Joining a mod team would be great too. I haven't purchased simcity yet, but I may this summer.

 

Would you mind explaining what exactly can be done with the program, other than previewing buildings and textures?

Also, will you write a short user guide anytime soon?

Share this post


Link to post

Posted

Just seeing the names Oppie and Karybdis in this thread took me back 10 years. We are in good hands here.

jigsaw likes this

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.