SimCityPak - Modding Tool

[potorium 17]

these wont scan in but are either valid hash or properly named ids

moved to xml

[potorium 17]

   added to xml

[warrior 169]

I believe you can now get rid of the hidden attribute, I think the filter list is filtered based on the files in the open DBPFs?

also there are some instances that are the same but need different names for different Types and Groups

[potorium 17]

the ones on this page are mostly fnv hashes used as property values. if you scanned them in from csv they arn't found but will show up as data on a prop view. At least if using vanilla xml from download

 

 added to xml

[trgs 27]

I believe you can now get rid of the hidden attribute, I think the filter list is filtered based on the files in the open DBPFs?

also there are some instances that are the same but need different names for different Types and Groups

 

I still have to see what the use for the hidden attribute is (no idea atm), but the Type and Group attributes are useless as far as I can see (already removed them). The RegistryInstance xml is now only used to link between a FNV hash and it's corresponding human readable name.

 

Currently i'm implementing the importing of FNV hashes for prop values, although I'm a bit in doubt if we should just make one big XML for all FNV name hashes...

 

these wont scan in but are either valid hash or properly named ids

 <Instance Id="0x3198301f" DisplayName= "Zoning" Hidden="false" />

 

When calculating FNV for "Zoning" I'm getting a different result? -> 0xae7a2be2

[potorium 17]

yeah, its weird in a lot of prop values. but i basically gave up trying to find the correct hash and just read it from the array index names which are nicely stored in an array too

 

not able to load up to get the full id to check, but check filter the insance id with 0x622b and the two props had half fill values. the value hash has the names in array below them.

 

i'm currently building a 2mill xml from the word dictionaries i scanned and borrowed to load ! am going to see what pops up in the prop values.

 

these wont scan in but are either valid hash or properly named ids

 

as in true hash value from fnv or simply a pseudo name as the names in js or props don't always match. _ - camelcase, pre and post fixes all a pita. and i also noticed some ID's are masked. they are the correct value fnv hash + a mask for the region groups i was playing with yesterday. the mask values may well be in one those two prop files 0x622b's as there is a list of hash values that struck me as crafted for that purpose.

[trgs 27]

Committed new version with prop value importer.

[potorium 17]

0x00b1b104 - 0x09878a01- 0x454041b4 = 140 odd prop values to write up. scroll to bottom of prop , is a list matches the array above with corresponding entries,

 

cant copy past from the right window

 

oo just seen your post will dl now

[potorium 17]

0x00b1b104-0x40800200-0x622b9cd7

 

0x0977f11c  is flora.

[warrior 169]

I still have to see what the use for the hidden attribute is (no idea atm), but the Type and Group attributes are useless as far as I can see (already removed them). The RegistryInstance xml is now only used to link between a FNV hash and it's corresponding human readable name.

Currently i'm implementing the importing of FNV hashes for prop values, although I'm a bit in doubt if we should just make one big XML for all FNV name hashes...

There are some Instances that aren't the hash of their name, I say this because some of them are sequential and the changes of getting sequential hashes is effectively impossible, so are they still in the registry

I put the hidden tag in to prevent the Instance filter list from getting overcrowded with instances that only appear as property values and not as IDs of files. But this is done properly now so it can be removed.

I don't know what you mean by the second paragraph. I had started implementing something that recorded all the names and hashes when entered into the FNV tool, but wasn't able to finish it (I 'shelved' it in Visual Studio, so if that what you mean then it might be on the codeplex server as a starting point)

Also perhaps it would be easier for sharing entries if instead of one XML file. There is a folder and any XML files in that folder are loaded.

[oppie 487]

I'm pretty sure that some instance/property ID's are not the result of FNV hashes - like warrior said, there are many sequential numbers (for example, most array properties used in tables follow this pattern) which we won't be able to decode from FNV hashes. Nevertheless, seems like we've made some great progress already.

[trgs 27]

I'm pretty sure that some instance/property ID's are not the result of FNV hashes - like warrior said, there are many sequential numbers (for example, most array properties used in tables follow this pattern) which we won't be able to decode from FNV hashes. Nevertheless, seems like we've made some great progress already.

 

I'm quite sure it's (at least) 50/50. Did some checks using a 5 char long rainbow table and I did manage to get some results that actually made sense.

I agree that the sequential values cannot ever be hashed (human readable) names.. hopefully we can find some other way of matching the sequential ones..

[warrior 169]

 

I'm pretty sure that some instance/property ID's are not the result of FNV hashes - like warrior said, there are many sequential numbers (for example, most array properties used in tables follow this pattern) which we won't be able to decode from FNV hashes. Nevertheless, seems like we've made some great progress already.

 

I'm quite sure it's (at least) 50/50. Did some checks using a 5 char long rainbow table and I did manage to get some results that actually made sense.

I agree that the sequential values cannot ever be hashed (human readable) names.. hopefully we can find some other way of matching the sequential ones..

 

We already have the names for some of the sequential/obviously-not-hashes, are they still included in the registry?

 

Also what is the difference between the TypeRegistry.xml and TypeRegistry.clean.xml? The clean one seems to be missing quite a few entries?

[trgs 27]

 

We already have the names for some of the sequential/obviously-not-hashes, are they still included in the registry?

 

Also what is the difference between the TypeRegistry.xml and TypeRegistry.clean.xml? The clean one seems to be missing quite a few entries?

 

 The idea for the *.clean.* XML's was to include only FNV valid hash names. TypeRegistry.XML contains multiple types of registries, so only the ones that apply to the importing of hashes have been cleaned up (Groups and Properties). I'm going to remove them again as soon as I made a setting available to filter or mark non-FNV-valid names. [edit]The FNV importer will overwrite old (user defined) DisplayNames with a valid one (if found), the old value will be put into the new one's comments[/edit]

 

I've also double checked that all hashes are based on lowercase strings only (was already implemented like that in SimCityPak). Did a check on a case sensitve and a uppercased rainbow table (500k entries) no valid hashes where found (on lower that 5k+ valid ones). I will convert all FNV valid names to lowercase in XML, since that's closest to reality (or would that be inconvenient?).[edit]bad idea...[/edit]

 

@potorium, In regard to the XML snippets you posted on this page, please use the proper <property> tags's. I will add them as soon as someone seperated the instances from the properties (I don't feel like checking all of them by hand )

  Edited June 16, 2013 by trgs  

[potorium 17]

mm they not properties, they

 

 added to xml

[potorium 17]

the override means we can import any cvs we want and as long as we import our own custom cvs last, we keep our own display names ?

 

edit. read from codeplex first. read you post above so valid > user names for importing

[potorium 17]

awesome work trgs!! So nice to be able to get rid of all the random id names and for them not to keep being overwritten.

[potorium 17]

a load more instance id;s. they wont scan in with csv because there is no instances of them (yet?). but they all fill out prop values

  added to xml

  Edited July 1, 2013 by potorium  

[trgs 27]

 

a load more instance id;s. they wont scan in with csv because there is no instances of them (yet?). but they all fill out prop values

I will import all of those as soon as I've implemented a property value scanner for the FNV importer. Unfortunately I have to rewrite a bunch of code to get it to work at decent speeds. Next commit will include this feature (will most likely not be today ).

 

However, I'm a bit in doubt about these prop values popping up in the InstanceRegistry and I do want to change that.

 

We now have 2 files:

InstanceRegistry.xml, contains:

 - Instances (eg. package entities)

 - Instances (whom are actually property values)

 

TypeRegistry, contains:

 - Instance Types (JS JSON PNG etc)

 - GroupId's

 - Property Value entity ID's

 

The current grouping into the XML's does not make a lot of sense.. I would prefer to separate the GroupId's into a new file and to make a separate file for all Property ID's (Instance names + Prop values ID's + Prop values).

[potorium 17]

i agree the tgi needs renamed / breaking up a bit more maybe. but the values are instance ids. the props are templates alot of the time, and thus the prop value must simply hold a link to the relevant container type instance groups.

 

 

clicking the region (0x00b1b104-0x8b7d9365-0x51e7a18d) throws up the region 'view'/container data. the prop values are instance id's of groups.

 

clicking cities prop name id 0xfb7a85a0 pops up window group of all regional city tile containers.

 

naming the groups allows easy navigaion using the prop value links but we  need to be able to hide groups from the drop down group list because there are hundreds!

 

snippet of region tile containers that name the pop up regional tile container window

<!-- City tiles container -->
<Group id="0x83831425" Abbreviation="Edgewater R_City tiles" DisplayName="Edgewater Bay City tiles container" Comment="" Hidden="true" />
<Group id="0xf0e2e39c" Abbreviation="Cape Trinity R_City tiles" DisplayName="Cape Trinity City tiles container" Comment="" Hidden="true"/>
<Group id="0xce729093" Abbreviation="R_City tiles" DisplayName="R_City tiles container" Comment="" Hidden="true"/>
<Group id="0xe37a36c3" Abbreviation="Sunrunner BayR_City tiles" DisplayName="Sunrunner Bay City tiles container" Comment="" Hidden="true"/>
<Group id="0xce70523b" Abbreviation="Reflection R_City tiles" DisplayName="Reflection Atoll City tiles container" Comment="" Hidden="true"/>
<Group id="0xe4dd8a2b" Abbreviation="Whitewater R_City tiles" DisplayName="Whitewater Valley City tiles container" Comment="" Hidden="true"/>
<Group id="0xcb1178df" Abbreviation="Titan Gorge R_City tiles" DisplayName="Titan Gorge City tiles container" Comment="" Hidden="true"/>
<Group id="0xd27e9966" Abbreviation="Horizon R_City tiles" DisplayName="Horizon Archipelago City tiles container" Comment="" Hidden="true"/>
<Group id="0xb5fdd017" Abbreviation="Viridian R_City tiles" DisplayName="Viridian Woods City tiles container" Comment="" Hidden="true"/>
<Group id="0xcb1d7b64" Abbreviation="Discovery R_City tiles" DisplayName="Discovery Delta City tiles container" Comment="" Hidden="true"/>

PROP (0x00b1b104) Edgewater R_City tiles (0x83831425) CITIES True 191 False PROP (0x00b1b104) Cape Trinity R_City tiles (0xf0e2e39c) CITIES True 135 False PROP (0x00b1b104) R_City tiles (0xce729093) CITIES True 114 False PROP (0x00b1b104) Sunrunner BayR_City tiles (0xe37a36c3) CITIES True 114 False PROP (0x00b1b104) Reflection R_City tiles (0xce70523b) CITIES True 196 False PROP (0x00b1b104) Whitewater R_City tiles (0xe4dd8a2b) CITIES True 162 False PROP (0x00b1b104) Titan Gorge R_City tiles (0xcb1178df) CITIES True 309 False PROP (0x00b1b104) Horizon R_City tiles (0xd27e9966) CITIES True 247 False PROP (0x00b1b104) Viridian R_City tiles (0xb5fdd017) CITIES True 313 False PROP (0x00b1b104) Discovery R_City tiles (0xcb1d7b64) CITIES True 310 False

 

 

if we can hide groups displaying in the drop down but  being displayed as is everywhere else, it would make the group drop down list a lot more useable as an entry point to reach what the user is looking for. ie, don't need to list every region tile container in the drop down, because its easy to reach by going through the regions container.

 

We could add a user definable group xml ?

<Group Id="numbers" Abbreviation="" DisplayName="" Comment="" Hidden="" UserDefined="True" />

if UserDefined="True" reads the TGI 's to display from an xml. And is up to us then to maintain our own user defined xml files. and we can just drop these into the group drop down by adding one line to the Typeregistry

 

no idea if this is easy or hard as my programing skills are about 2/10 at best and only if using php

[potorium 17]

 

 

However, I'm a bit in doubt about these prop values popping up in the InstanceRegistry and I do want to change that.

 

 I currently have 45k lines in mine if i can build up the motivation to fight excel i might try load up a 2mill line instance registry to see what turns up. excel kept crashing trying to manipulate a load of cvs and dict files so i might break it down into smaller chunks;

[potorium 17]

gid's probably need better naming but these work if people dont have them yet

added to xml

[warrior 169]

However, I'm a bit in doubt about these prop values popping up in the InstanceRegistry and I do want to change that.

I currently have 45k lines in mine if i can build up the motivation to fight excel i might try load up a 2mill line instance registry to see what turns up. excel kept crashing trying to manipulate a load of cvs and dict files so i might break it down into smaller chunks;

So you want to split Instance IDs that only appear as property values from Instance IDs that are of files?

How about just making an Instance folder anthem it just reads any XML file, as i suggested before, but call one file PropertyValues.xml and the other InstanceRegistry.xml (the names are technically irrelevant to the program but to the user looking at the raw xml files it shows the separation)

Also we don't need to worry about cluttering up filter lists as the filter list themselves are filtered based on the files currently in the main list. (At least the instance filter does this, and so should the group and type)

[potorium 17]

<Group Id="0x02393c07" Abbreviation="Mouse Cursors" DisplayName="Mouse Cursors" Comment="" />
   <Group Id="0x098a44f2" Abbreviation="Container - Alert Icons" DisplayName="Container Alert Icon" Comment="" />
   <Property Id="0x0d8c29c3" DisplayName="Container - Model" Comment="" />

**renamed**  also contains streetlights among other things

<Property Id="0x0d897169" DisplayName="Unit(?) Model" Comment="" />

[potorium 17]

we have hundreds of groups with like 3 in a group. leaving them un-named is one option but they dont drop their display name to the files we view.

 

makes no odds to me as i'm naming the groups myself but i'm thinking of new users. if we can hide the none entry point groups life will be much easier for them.

 

ie clicking on the main region(by name) group to see all region eco stuff, or the main global containers, layer containers etc

 

just be nice to be able to hide all the junk like the menu pallettes ones for new users and just leave the easier to navigate ids. Can add a simple check box option to view all groups or view only none hidden fagged

<Group Id="0x09a4ba02" Abbreviation="Menus Container" DisplayName="Menu Palettes Container" Comment="to list of layer menu containers? " />

 

[potorium 17]

edited to put list in one place

 

 added to xml

[Buggi 684]

I'd just like to throw something out there regarding the foundational DBPF format. As some of you may know, this system was developed by Paul Pedriana (who later gave me the extracheats DLL and source). It is my hunch that the DBPF stands for DataBase Page File. Rather then think of the data contained therein, as a series of separate entities think of it like a database. There are tables rows and columns.

 

Using the old Type/Group/Instance nomenclature:

Think of it like this: a Type could be considered a table. So all entries with the same Type ID would be found in the same table. The data contained in that table would be expressed by the Group (Row) and Instance (Column). These data can be many things, an image, a string, an array of values. This might be how you find ID's that are sequential as they would merely be the next row.

 

Now I could be off a little on the concept of Row and Column expression, but I think I'm dead on in thinking the overall format is expressed like a database. There is simply too much data to be organized like individual files. I haven't read up recently on other database file formats, but I'm guessing they're pretty similar.

 

Also, when the game loads the different packages, it uses those ID's as indexes into the data it already has. This is how SC4 mods overwrite certain items. It isn't overwriting a FILE, but an entry in a database.

 

Granted, the database could be all one table, and the ID's you've been finding could be used as indexes into that table for fast access. So a sequential ID would be handled internally as (BaseID)+1.

 

Thoughts?

[potorium 17]

i think the sequential property ids are as you say. but i can't for the life of me figure out the bases. i'll assume its to do with the directory structure though.

a fw more instance id's. some are true fnv hashes for instances, others are for populating the property values. just use trgs scanner and importer if you only want instance id's and not those that are type/groupless

 

[ added to xml

[potorium 17]

just a few more instance (true cvs importable) id's

 added to xml

  Edited July 1, 2013 by potorium  

[AlexArtAndros 0]

W3C HTML5 & CSS3 quick validation button.

https://chrome.google.com/webstore/detail/w3c-html5-css3-validator/kobpbiokkobnmnaefmpcodeeficgbfkg

* It's a shortcut button for quick validation, not a offline or local validator.