SC4, Skylines, SC2013, in order
82 members have voted
I've taken the liberty of starting this topic because I have just posted a major upgrade to DataNode, and to date there has not been a single place where support questions could be posted. DataNode is the sort of tool that is actually very easy to use, but overwhelming for many because of the massive amount of data it presents. The Manual included with it is there for a reason - to make this use of this data easier to understand and put to your own ends.
Its really an audit tool aimed at a couple of tasks as a player:
Checking that the load order of mods in plugins leads to the result you want - a fast loading, clean SC4 plugins folder. Checking that you have the dependencies needed for a mod As a modder it can help in verifying that your mod is correct and clean, and works as you expect if it overrides game content.
Latest Version of SC4DataNode
DataNode has been around for a while (since 2012) , and from time to time gets a mention when people encounter bugs or discuss how they manage their plugins. Today I've released a new version v1.06 that is well worth getting especially if you have a large plugin folder.
After some forum discussion about what SC4Datanode shows of the files in the Plugins folder, I’ve decided to make all the files touched in the loading process visible. This slows the program a little bit but means it is now possible to see all files that SC4 ignores when loading those it recognises. These do no harm (generally) but slow the loading process as SC4 must look at every file to see if it is relevant.
Here are the main changes:
in the SC4 file view files that are coloured with a beige background and orange text, with zzz in the Dead column, are sleepers or zombies – they are examined by SC4 but only contribute to the time taken to start up the game. These are mostly documentation files – text and images - but may include all sorts of files included by authors of game material. You don’t have to remove these but if you do the game will start faster. By default, these go onto the scanning Log unless suppressed. In the Preferences panel one can suppress the logging (and display) of non-SC4 Files. This is useful if you have a lot of documentation files but want to leave them there and just concentrate on dependencies. If a file ends in .dat, .sav, .SC4, .SC4Desc, .SC4Lot, or .SC4Model, and the File is empty it will be flagged with an Apricot background, and Orange text. It will be put on the log as it may be damaged if it should be a SC4 file. But, for example, the uninst000.dat in SC4PLUGINS is a file that isn’t a real SC4 .dat. Occasionally installers leave .dll or .exe files in the Plugins – the ones in SimCity:Plugins and NAM are valid – others may just be installers for mods and can (should) be removed after installation. They are marked as IsOK in the Dead Column of the SC4 File display – just in case. SC4Datanode will now also show all files that could be loaded by SC4, LE or PIM-x even if they have no extensions or other wild names. These are treated as if they were SC4.Model files, that is, are loaded in phase 1 of the loading sequence. These should probably be renamed to have a .dat extension for easy identification in future but you’ll need to judge by its contents. When a path name becomes so long (more than 260 characters) that Windows refuses to handle it, SC4Datanode now shows you where it is and will stop running. At this point you will need to fix the problem by rearranging the folders so that the structure is not so deep. The CSV reports by view now produce one file per view – named according to the view – some reports have extra properties common to all items. The Q column has the value “=” when it is the only or final exemplar after overrides are accounted for. FileView and TGI views all produce a report. The program has had its Error Handling strengthened and should also run faster on x64 machines. Memory management has been a focus of testing. It is configured to adapt to x32 and x64 systems thus helping memory constraints if the plugins folder is very, very large. The Manual has been updated to reflect these changes. I recommend you (re)read the pages 10-15 to understand the new features.
Memory and Performance
During testing of this new version it became apparent that the old version would start to show memory problems on a large plugin set. This was due to the program being compiled for x32 machines. This program wades through a lot of data and uses up to 40% of the size of your plugins total during the scan. It has now been compiled to use x64 when on 64bit machines, and x86 on 32bit machines. Because pointers are double the size on x64 it will use about 50% more memory on x64 but have no ceiling other than physical memory (assuming no disk paging). On x32 the addressable memory is usually closer to 2GB. I have no idea if the Extended Flag works on .NET code - if so the limit may be 3.6GB.
on x32 – 2GB addressable implies a maximum plugin size of about 4GB before memory may start causing issues. on x64 – 4GB memory implies about 7GB max plugin size, 8GB memory about 14GB, and 16GB should not give problems until 28GB of plugins. The memory used is shown on the status bar. A lot of work has been done to minimize use of memory. However, if you do exceed physical memory it may start disk-memory paging and become very slow or even CTD in the worst case (out of program control).
Performance is very dependent on disk speed – the best times have come from SSD, and 10,000 rpm iron disks. The program is I/O bound and will use only one processor for itself by design. The OS (under the carpet) then can use as many cores as it wants for I/O. .NET is quite efficient in I/O, so the program is quite snappy – obviously the age of machine, OS, disk type and size of plugins affect performance.
It is worth noting that if you run SC4Datanode directly after a reboot its run time may be quite a bit slower than subsequent rescans or scans. This is because Windows is filling its disk caches for the first time as it goes. These get used next time round leading to much faster loading times. This also holds for SC4. The more headroom in memory the more impressive this caching is. Also keep in mind that Windows can easily use 4GB (x64) on its own these days.
These restrictions hold even more for SC4 – if you feed it a plugin collection in excess of 4GB its going to be slower and slower starting up and may actually run into problems. Its always more pleasant and responsive to run SC4 with a subset of your plugin archives. SC4Datanode will allow you to manage a much larger plugin set than SC4 can comfortably run.
Finally this version would not be solid as it is without the efforts of @twalsh102 who now holds the record for SC4DataNode with a Plugins folder size of 14.1 GB with 29,961 files which was eventually scanned and analysed in 2 minutes 53 seconds settling on 2.36 GB of memory. This was achieved on a 2018 Win10 machine with 16GB of memory. This is nearly twice as large and fast as the 2012 testing figures held by Doc Rorlach which were achieved on a Win7 machine of that time namely 7.34GB of plugins in 2m38s.
Please use this place to can ask/comment/suggest about DataNode. Like any program it can be improved.
TL-Nexis and DM Waterfront Parks and Promenade Modular Set
I blame Nexis (mrbisonn) for this set. He sent me a nice model of an anchor and… I let it ran away and make babies with Don Miguel Waterfront Promenade lots. Here’s the result. Six parks, five of which work as waterfront pieces
TL-DM WTF Park Paths
TL-DM WTF Addons Paths
That you can variously combine with :
TL-DM WTF Large Promenade 11x4
and its modular versions :
TL-DM WTF PromenadePark East
TL-DM WTF PromenadePark Center 5x5
TL-DM WTF PromenadePark West
and with : TL-Nexis WTF Anchor Park 6x5
and its fully waterfronty version (to be combined with the paths):
TL-Nexis WTF Anchor Park 6x5 Open
The original Don Miguel sets are here : Waterfront Promenade and Waterfront Promenade Set 2
And go have a look at both Nexis and Don Miguel’s Content, why don’t you ?
Many thanks to my technical gurus, Rsc204, Kosc and Dreadnought. And especially Rivit, who lighted my SC4world and is always generously ready to help. And the guys and gals on the forums (Twalsh, Tyberius06, Matias93, Fantozzi, Catty-cb, Corinamarie…). We need help when we begin peeking under the hood of SC4. That the community is always there for us noobies is very comforting.
TL-DM Waterfront Parks :
BSC MegaProps CARCH vol.1
BSC MegaProps MJB vol2
BSC MegaProps Misc vol.1
BSC MegaProps - SG vol.1
JENXPARIS_Prop_Bridge_Light_Cone_R1.dat (in NAM 34 and up)
PEG Texture Pack vol.1 (PEG_Overlay textures_vol.1.dat)
Simfox 8 Chestnuts
Tariel-Rivit’s Essentials (must load after the NAM)
Twrecks Maxis HD Tree Replacement
***Girafe trees always Necessary for Twrecks Maxis Trees HD replacement ***:
Ashes - Beeches - Canary date Palms - Cypresses - Lindens - Rowans - VIP orange fagus Seasonal Flora
wmp_power_props01.dat, in WMP Power Lots.
Girafe Trees : beeches, chestnuts, cypresses, elms, lindens, Norway maples, rowans,
TL-Nexis Anchor Park : sames deps, plus :
NetworkAddonod_Props.dat (in NAM 34 & up)
SM2 Roundabout Tiles_Tex.dat, in SM2 Roundabout Sculptures Set
and Moar Girafe Trees : birches, common spruces, conifers !
Same deps for all the the paths.
General warnings :
(1) This is for SC4 Rush Hour.
(2) I play with NAM36, RRW-EU selection (it shouldn’t interfere here, but you never know !)
(3) I also play with Twrecks Maxis HD Tree Replacement (if you don’t use it, your trees might look a bit different [not these trees !] ; perhaps you should use it…)
(4) I play Nights. Depending on your using or not a Light Replacement Mod, the lighting will be slightly different, but There Will Be Light. If you dont want it, you can either tweak my lights in the Lot Editor or take them out of my Essentials folder dep.
(5) I like trees. I make intensive use of trees. Especially Girafe tree collection on the LEX ; so here’s the link, once for all ; just follow it into the woods, when indicated.
Recently Browsing 0 members
No registered users viewing this page.