tool SC4DataNode Support

[rivit 3,391]

I've taken the liberty of starting this topic because I have just posted a major upgrade to DataNode, and to date there has not been a single place where support questions could be posted. DataNode is the sort of tool that is actually very easy to use, but overwhelming for many because of the massive amount of data it presents. The Manual included with it is there for a reason - to make this use of this data easier to understand and put to your own ends.

Its really an audit tool aimed at a couple of tasks as a player:

• Checking that the load order of mods in plugins leads to the result you want - a fast loading, clean SC4 plugins folder.
• Checking that you have the dependencies needed for a mod

As a modder it can help in verifying that your mod is correct and clean, and works as you expect if it overrides game content.

Latest Version of SC4DataNode

 

DataNode has been around for a while (since 2012) , and from time to time gets a mention when people encounter bugs or discuss how they manage their plugins. Today I've released a new version v1.06 that is well worth getting especially if you have a large plugin folder.

After some forum discussion about what SC4Datanode shows of the files in the Plugins folder, I’ve decided to make all the files touched in the loading process visible. This slows the program a little bit but means it is now possible to see all files that SC4 ignores when loading those it recognises. These do no harm (generally) but slow the loading process as SC4 must look at every file to see if it is relevant.

Here are the main changes:

• in the SC4 file view files that are coloured with a beige background and orange text, with zzz in the Dead column, are sleepers or zombies – they are examined by SC4 but only contribute to the time taken to start up the game. These are mostly documentation files – text and images - but may include all sorts of files included by authors of game material. You don’t have to remove these but if you do the game will start faster. By default, these go onto the scanning Log unless suppressed.
• In the Preferences panel one can suppress the logging (and display) of non-SC4 Files. This is useful if you have a lot of documentation files but want to leave them there and just concentrate on dependencies.
• If a file ends in .dat, .sav, .SC4, .SC4Desc, .SC4Lot, or .SC4Model, and the File is empty it will be flagged with an Apricot background, and Orange text. It will be put on the log as it may be damaged if it should be a SC4 file. But, for example, the uninst000.dat in SC4PLUGINS is a file that isn’t a real SC4 .dat.
• Occasionally installers leave .dll or .exe files in the Plugins – the ones in SimCity:Plugins and NAM are valid – others may just be installers for mods and can (should) be removed after installation. They are marked as IsOK in the Dead Column of the SC4 File display – just in case.
• SC4Datanode will now also show all files that could be loaded by SC4, LE or PIM-x even if they have no extensions or other wild names. These are treated as if they were SC4.Model files, that is, are loaded in phase 1 of the loading sequence. These should probably be renamed to have a .dat extension for easy identification in future but you’ll need to judge by its contents.
• When a path name becomes so long (more than 260 characters) that Windows refuses to handle it, SC4Datanode now shows you where it is and will stop running. At this point you will need to fix the problem by rearranging the folders so that the structure is not so deep.
• The CSV reports by view now produce one file per view – named according to the view – some reports have extra properties common to all items.The Q column has the value “=” when it is the only or final exemplar after overrides are accounted for. FileView and TGI views all produce a report.
• The program has had its Error Handling strengthened and should also run faster on x64 machines. Memory management has been a focus of testing. It is configured to adapt to x32 and x64 systems thus helping memory constraints if the plugins folder is very, very large.

The Manual has been updated to reflect these changes. I recommend you (re)read the pages 10-15 to understand the new features.

Memory and Performance

During testing of this new version it became apparent that the old version would start to show memory problems on a large plugin set. This was due to the program being compiled for x32 machines. This program wades through a lot of data and uses up to 40% of the size of your plugins total during the scan. It has now been compiled to use x64 when on 64bit machines, and x86 on 32bit machines. Because pointers are double the size on x64 it will use about 50% more memory on x64 but have no ceiling other than physical memory (assuming no disk paging). On x32 the addressable memory is usually closer to 2GB. I have no idea if the Extended Flag works on .NET code - if so the limit may be 3.6GB.

So:

• on x32 – 2GB addressable implies a maximum plugin size of about 4GB before memory may start causing issues.
• on x64 – 4GB memory implies about 7GB max plugin size, 8GB memory about 14GB, and 16GB should not give problems until 28GB of plugins.

The memory used is shown on the status bar. A lot of work has been done to minimize use of memory. However, if you do exceed physical memory it may start disk-memory paging and become very slow or even CTD in the worst case (out of program control).

Performance is very dependent on disk speed – the best times have come from SSD, and 10,000 rpm iron disks. The program is I/O bound and will use only one processor for itself by design. The OS (under the carpet) then can use as many cores as it wants for I/O. .NET is quite efficient in I/O, so the program is quite snappy – obviously the age of machine, OS, disk type and size of plugins affect performance.

It is worth noting that if you run SC4Datanode directly after a reboot its run time may be quite a bit slower than subsequent rescans or scans. This is because Windows is filling its disk caches for the first time as it goes. These get used next time round leading to much faster loading times. This also holds for SC4. The more headroom in memory the more impressive this caching is. Also keep in mind that Windows can easily use 4GB (x64) on its own these days.

These restrictions hold even more for SC4 – if you feed it a plugin collection in excess of 4GB its going to be slower and slower starting up and may actually run into problems. Its always more pleasant and responsive to run SC4 with a subset of your plugin archives. SC4Datanode will allow you to manage a much larger plugin set than SC4 can comfortably run.

Finally this version would not be solid as it is without the efforts of @twalsh102 who now holds the record for SC4DataNode with a Plugins folder size of 14.1 GB with 29,961 files which was eventually scanned and analysed in 2 minutes 53 seconds settling on 2.36 GB of memory. This was achieved on a 2018 Win10 machine with 16GB of memory. This is nearly twice as large and fast as the 2012 testing figures held by @DocRorlach which were achieved on a Win7 machine of that time namely 7.34GB of plugins in 2m38s.

 

Please use this place to can ask/comment/suggest about DataNode. Like any program it can be improved.

[CorinaMarie 29,575]

@rivit

As one of the most important and comprehensive tools for SC4, I feel this topic is worthy of being pinned up top of the forum. If you'd like we can do this. May I also suggest editing in a linky to SC4DataNode v1.06 at the top of your post?

[rivit 3,391]

thanks Cori - I obviously was getting a bit fatigued - have now edited the link into the post.

As for where the post should be (which forum etc)  - I didn't really know so took a guess.

I would suggest that if there is a way to clump together - managing plugins, dependency discussions, loading order discussions and DataNode that would probably bring most utility for players. Now there is so much content available, it seems even newly starting SC4 players are getting straight in at the deep end - a bit like foreign tourists in the surf at Bondi - and a few of them drown.

~~

BTW your discovery of that Developer thingy is quite inetersting - doing some experiments with it. The reason Reader 0.93 cant see it is because it has a bug reading properties that have UInt16 format - so the rest of the exemplar becomes unreadable to it.

[CorinaMarie 29,575]

At the risk of derailing the intent of your topic I'll reply here. Rest assured we can hide these posts later so they don't distract from the thread.

13 minutes ago, rivit said:

As for where the post should be (which forum etc)  - I didn't really know so took a guess.

Unless we find a new topic totally out of place we prefer the author choses were they feel it best fits. Should you decide somewhere else would be better any of the staff can move it for you. (It's just a couple of mouse clicks to do that.)

 

14 minutes ago, rivit said:

I would suggest that if there is a way to clump together - managing plugins, dependency discussions, loading order discussions and DataNode that would probably bring most utility for players.

Would the idea be to have a sub forum for those particular topics? With CB's help I can create one using admin tools. In general, my philosophy is to help those who know more than I do to get the job done when it requires hidden button pushing. This could easily be a separate discussion if that's the idea.

 

18 minutes ago, rivit said:

BTW your discovery of that Developer thingy is quite interesting - doing some experiments with it.

I look forward to whatever you discover about it. I'm very much in favor of its use and had it been the default toggle of false in the original game release, I believe a large handful of topics where the middle and high wealth Sims wreck havoc due to supplanting the lower wealth would not need the standard answer of: Mark everything you want to preserve historical.

[rivit 3,391]

OK,  (a) lets leave it where it is,  (b) think about the clumping topics thing for a while, and (c) see how the third experiment works out.

Then hide these extra bits

[CorinaMarie 29,575]

10 minutes ago, rivit said:

Then hide these extra bits

I leave it up to you to say when and then all but your first post can be hidden.

[jeffryfisher 2,176]

I've just downloaded and installed DN 1.06. After fighting AVG for supremecy, I'm even allowed to run it. Now, puttin' on my dusty ol' QA hat...

1) Currently employing a RAM-drive, I use a custom plugins folder. When DataNode starts up, it is blank. I must always "rescan", and then it picks up everything. Is that my error? A setting I can change? Or could DN detect that the first time in and load data?

(File View)

2) A few of my mods are tinted brown for having multiple directories. Should I be worried? How can I fix them?

(TGI View)

3) Listing "faulty" TGIs shows things like SINT32 when expecting UINT32, but the value is x00031083, which works the same either way. How does DN detect a type incompatibility, and how can I fix it? Is there a way to turn off "who cares" and "doesn't make a difference" data type disagreements to reduce the haystack?

4) S3D: When I filter TGI view for "missing dependencies", I get a haystack of about 2000 TGIs that have missing S3Ds. I'm not sure what S3D means, but 3D suggests the red-blue 3d viewing graphics that nobody uses. I wish I could ignore missing S3Ds so I could better see the one or two missing props and/or models. PS: It's like LTEXT, which I am already grateful to ignore.

 

 

  Edited September 21, 2018 by jeffryfisher  
S3D

[rivit 3,391]

thanks for the questions @jeffryfisher 

1) In preferences you can make Datanode start scanning by default - if the path for the Ram drive is predictable then you should be able to put that in one of the preferred folder names.

2) Some files do have more than one DIR. As far as I can tell the point of a DIR file is to predict how much memory to allocate when decompressing the entries (that is all TGIs that are marked as compressed in Reader should be mentioned in a DIR). However I conclude SC4 doesn't actually take any notice as files without, or having more than one DIR, still work. Ideally the DIRs should be deleted and reconstructed in Reader (Rebuild,Reindex)

So in general nothing to worry about - we have found files with >50 DIRS - this caused by concatenating several smaller files without rebuilding DIR.

3) The format to be expected is in a list of properties (see Resources in the program Folder, also underneath Reader). As SC4 has been defensively built any values that are convertible between formats are just used by SC4. Sometimes this may go wrong but we have no idea what SC4 then does - it doesn't generally lead to crashes, but can give odd logic e.g. if a pollution radius was negative what does that mean?.

For these - its a mental ignore - theres no ignore in the program as yet - if you're really fussy then the way to fix them is with Reader - add the property again into the exemplar with the same value, (it will default the correct format) then delete the line with wrong format  (the exemplars are loadable into Reader in one hit using Batch Load and the _Readerloadlist.txt) 

4) There are a large number of NAM Highway Exemplars that point to models (S3D = a 3D model) that are now considered superceded content. One day these will probably be cleared up, but can be ignored (I just cant find a way to do this elegantly in DataNode). All others are probably real.

If you run the CSV report (into Excel or the like) of these missing dep exemplars you can filter them out and be left with those to action. In fact any real divide and conquer searching for stuff works quite well on the CSV reports in conjunction with the Program or Reader.

 

 

[carlosmarcelo 973]

Hi, all! I can´t install sc4datanode because norton antivirus (symantec) blocked it as "WS.Reputation.1": norton moves the msi file to quarantine, after open zip file. How I use my computer to work too, I can't get risk of any problem with trojans, backdors or virus.

Are there a other way get the installed files to I do "manual instalation"? I believe that problem with norton would be with .msi file, and not with program itself.

[Cyclone Boom 16,852]

Hi @carlosmarcelo, thanks for asking.

As a fellow Norton Security user myself, this is something I'm all too aware of when downloading SC4 content.

The WS.Reputation.1 warning is because Norton sees the executable file as "unproven" due to lack of usage on the global scale. Basically until enough people use a file around the world, it's considered to have a low reputation. It doesn't mean there is a guaranteed security risk, and far from it at that. Instead it can be what's known as a false positive because there's not necessarily any risk whatsoever. In the case of DataNode, we can assure you it's absolutely 100% safe.

For how to restore such files, this article explains the steps for moving a file from Quarantine and excluding it from future detections.

Also this video might be useful for a visual overview:

(It just doesn't show the excluding from future detections part.)

Hope this helps!

[PaPa-J 778]

So, if I have an entry that shows up like this -

And it says that it is overridden by another file like this -

Is it safe to say that I can get rid of the file that is being overridden?

[Cyclone Boom 16,852]

1 hour ago, PaPa-J said:

Is it safe to say that I can get rid of the file that is being overridden?

Yes, I believe so, but there are considerations. From your screenshots there, it seems the file overriding is the DBSSYMN Vol02 "MEGA" prop pack.

What this override means is there's the same Instance IDs used as the individual SC4Model file. For whether the MEGA pack should override as seen, it depends on whether other content you've got relies on this and what's contained within. Should Lyndhurst Manor be the only item that needs the model for the building, you could safely keep the SC4Model and the accompanying SC4Desc file. Then the Vol02 MEGA pack can be removed since the other props within wouldn't be needed as dependencies.

[rivit 3,391]

Broadly I agree. The Individual Lyndhurst Model may have been eventually packaged up into the Megapack. Only by knowing something of the history of the Model file could you know for sure - or perhaps if theres a cleanitol file that the megapack uses to remove redundant stuff.

You can test this by temporarily removing the Modelfile after plopping it in a test  city. If it still shows correctly afterwards - no box - then its almost certainly redundant.  But being a small file its not really something to spend a lot of time on.

[PaPa-J 778]

Thanks for the quick response guys!

[jeffryfisher 2,176]

One thing to watch out for is the word "file". There are files on disk and also "files" within some of those files. You can only safely remove a disk file when all of the data "files" inside it are redundant.

Whenever you see an override, think about which should be overriding which. Is it going in the right direction? In general, more recent mods and mega-collections should override older versions and separates. However, there are exceptions (read the docs!). Sometimes DataNode is alerting you to a load-order problem, not a redundancy problem. After sorting the load order, then you may have a redundancy you can move (never delete unless you already have a copy somewhere else).

[PaPa-J 778]

Very good point, thanks for pointing that out.

[carlosmarcelo 973]

On 25/04/2019 at 6:45 PM, Cyclone Boom said:

It doesn't mean there is a guaranteed security risk, and far from it at that. Instead it can be what's known as a false positive because there's not necessarily any risk whatsoever. In the case of DataNode, we can assure you it's absolutely 100% safe

Thanks about informations, @Cyclone Boom! I already know about exclude itens os quarantine and I trust in the simtropolis files, but I discovered a way to tell to norton that I believe this file is a false positive. After 3 days the Symantec answared me this: 

"In relation to submission 146762.

Upon further analysis and investigation we have verified your submission and, as such, the detection(s) for the following file(s) will be removed from our products:

    File name: SC4DataNode_1.06.msi
    MD5: C372E1F3C5B9020DA8FCA2E70E9B0459
    SHA256: FC7AC6048EDC79CDEF91B61A4FABADF7FE80E7E3948A4CF8F95D570316713F58
    Note: Whitelisting may take up to 24 hours to take effect via Live Update"

Now is official: SC4DataNode is not a risk to Symantec Norton.

How I think there are other simtropolis tools with same "problem", I will put here the steps to users and developers sinalize to all Norton Users that the file is a good and reliable "person"! 

===============

1. Access https://submit.symantec.com/false_positive/
2. Fill the fields.
3. Submit file.
4. Just! The end. 

Or you can submit within Norton's own antivirus interface, as soon as it detects the false positive and shows you options on what to do: see attached image (sorry, I only have the image in Portuguese).

I hope it this informations help the SC4 Comunity.

[rivit 3,391]

Thankyou carlosmarcelo - you've been very thorough and persistent on this.

[carlosmarcelo 973]

Hi again! I don't know if I can ask questions about how I use this excellent tool. If I'm wrong, just tell me to change topic.
I get some industrial growable builds from the sc4devotion, but they don't grew.
Seeing with SC4DataNode, I realized this files are "DIR" equals 2. I read in manual about but I can't understand whats means and what I can do resolv the problem (and if this is the problem).
Can you help me?

  Edited May 5, 2019 by carlosmarcelo  
Add image.

[Cyclone Boom 16,852]

1 hour ago, carlosmarcelo said:

I don't know if I can ask questions about how I use this excellent tool. If I'm wrong, just tell me to change topic.

You can, you have, and you're right to use this topic for asking anything about the SC4DataNode tool or file overrides. It's the perfect place.

It's ideally suited for anyone to ask about the features of this valuable program, and being here in the General Discussion forum means it's more likely other SC4 players can see the discussion. Then maybe those reading might learn a few tips and tricks along the way.

 

1 hour ago, carlosmarcelo said:

I get some industrial growable builds from the sc4devotion, but they don't grew.

This would depend on a number of factors. For instance the industrial demand, the growth stage, total occupants, and also the lot zoning parcels. With some custom growables, it can be worth using precision zoning methods by holding the Ctrl key while laying down the parcels. This is no certainty for them to grow, and sometimes the zoned area gets split into smaller cells. But it can encourage development when there are certain sizes of lots.

Perhaps an RCI Maxis blocking mod like this one might be worth using. Then once installed for industrials (and any other developer types), it would mean the only lots which would be allowed to grow are the custom ones you've installed.

 

1 hour ago, carlosmarcelo said:

Seeing with SC4DataNode, I realized this files are "DIR" equals 2.

That is interesting, and I know how DIR files are directories for referencing the Type Group and Instance (TGI) values for SC4 data files (including Exemplars). These are for compressed items, and non-compressed don't require a DIR file.

Having more than one of these indexes might not be desired, hence why DataNode is flagging those 2 entries up in the highlighted colour. If opening in Reader, I've found how a DIR can be removed from the entry. Then click the Rebuild Directory file option and it should create a new individual entry. Finally it's good practice to then click ReIndex before saving, just so everything is ordered properly. Though I believe in Reader 0.9.3 how this happens automatically upon Save of the file.

[rsc204 21,806]

1 hour ago, Cyclone Boom said:

Having more than one of these indexes might not be desired, hence why DataNode is flagging those 2 entries up in the highlighted colour. If opening in Reader, I've found how a DIR can be removed from the entry. Then click the Rebuild Directory file option and it should create a new individual entry. Finally it's good practice to then click Reindex before saving, just so everything is ordered properly. Though I believe in Reader 0.9.3 how this happens automatically upon Save of the file.

I don't think 2 DIR files will prevent something from working, unless one of those is corrupted, that causes havoc.

I would make sure you delete both DIR files, Reindex, then Save to fix these files. Also, you really want to ensure you are using Reader 0.9.3 to do such operations. It's very possible that Reader can crash if you don't save/close right after such changes.

However, I doubt that this is really the cause of your problems... to the point that I downloaded the PepsiPlant myself to check things. SimGoober was one of the most prolific and knowledgeable modders for a very long time in this community. Between that and the sheer amount of downloads of these mods, without reports of problems, straight away I am sure they work just fine.

Sure enough, I tested these in a city and got them to build eventually (without removing DIR files), but they do not grow easily. This is very typical of any lot over 6x6 in size, the game will allow them to grow, but it will frequently prefer to grow two or more smaller lots in such a space, over one big one. There are many reasons for this, but if you persevere, you will be rewarded with their appearance eventually. It's much easier to Plop larger lots if you really want to see them in a specific place. You can mod it to be a ploppable, although here SG does include such Plop lots, found in the Landmark Menu. Otherwise it's often easier to use either the "Building Plop" or "Lot Plop" cheats when you want to manually place something. I believe to use either "cheat", you need the following download installed:

Note the following VERY IMPORTANT information regarding plopping lots. You need sufficient demand if you don't want to potentially screw up your cities.

[rivit 3,391]

On DIRS - I suspect that although they are defined for compressed items to indicate the uncompressed memory needed for them that the Exe doesn't actually make use of that info - I've seen files with 50 dirs  - the result of concatenating multiple small automata files with some program - and they all load fine. Also .dats with compressed items do load properly even if there is no DIR.

I flag them because 1 is the standard and multiple could be a cause of strangeness.

[carlosmarcelo 973]

Thanks everyone! 

Let me see if I understood about DIR: it is a "index reference" for others TGI files, therefore, when there is 2 or more "index" in the same SC4file is strange. I'm correct?

I agreed with @rsc204 and I went to read the link of topic "same buildings" send by @Cyclone Boom . There are logical explications about this game behavior what is that I really want to solve: break repetions boring and bring more diversity in my cities. In this link I saw a lot of suggest to download set of comercial, industrial and residencial lots.

The @rsc204 's ideia is excelent (use lotplop in extra cheats plugin) because I do not have to wait for a build to grow or even use the CAM to have one or two amazing builds. But, I understood about demand and I will use this method with temperance. 

About coca cola and pespi builds: I can with lotplop and the lots works well, offers jobs to my city. This same will happens with comercial and residencial lots?

I have other doubt about SC4DataNode, but I will post next weekend because I need get some prints and time! 

[rsc204 21,806]

13 hours ago, carlosmarcelo said:

Let me see if I understood about DIR: it is a "index reference" for others TGI files, therefore, when there is 2 or more "index" in the same SC4file is strange. I'm correct?

Yes and no, as Rivit pointed out, this tends to happen when multiple files have been merged together. Each file may have a DIR file, ideally those should be merged together, leaving only one behind. However, this didn't always happen for whatever reason.

13 hours ago, carlosmarcelo said:

This same will happens with comercial and residencial lots?

The crux of how this works comes down to how the game "links" roads to lots. Only when a building grows correctly can anything exit the lot. In practise this means that you can plop Commercial without problem (provided you have demand), because sims only need to enter those lots, which works. The same is true for Industrial, with the exception that freight can not leave the lot. However, for Residential, this presents a bigger problem, because sims can not leave the building to find work. Long and short, plopped Residential will never work and always abandon eventually. Even if it appears to be working, it almost certainly won't be.

[rivit 3,391]

Investigating the SimCity Soundscape

In recent work to upgrade SC4Datanode, version 1.0.10 arriving soon, to see more dependencies I have had reason to delve into the workings of the Sound system underneath Simcity4 in order to understand why certain files could not be found.

As it turns out the soundscape is made up of at least three main components driven from within the exe, probably some of the LUA scripts and a large number of files containing sound data and instructions on how to use them. Most of the data is kept in the Sound.dat (not surprisingly) but some are also in the Simcity1.dat disguised as INI files. In the following discussion the mp3 Music files in the Simcity4\Radio folder are not mentioned.

The main group of 1608 samples are stored as XA files - a form of WAV file. These are referred to in different ways. Most SFX properties in the Exemplars refer to TRK files (they must have unique Instances) which in turn refer to LOGICOBJ, HLS and XA files. The LogicObj files seem to be used to combine/invoke sound files in a generic way, HLS files list a number of XA files which are randomly chosen to supply for the sound effect. This works like the Family property in Props and Buildings allowing for variation in sound. There are 870 TRK files, 215 HLS files, and 12 LOGICOBJ files.

The second group is a collection of 43 sound banks (AB files) and their directory (AE File) which are hidden in INI files in Simcity1.dat. These AB files comprise a structured/compressed form of sound data that is used as the source for the sounds made when User Driving Automata. This is the SFX Occupant Sound Property. One of the AB files has no Sound Bank data and is not used, another referred to by all Sailing Automata seems to be missing entirely.

The Directory also hints at 632 files, probably samples, and 21 other files which may well be internally defined. I haven't been able to pin these down, but there are TRK files with no corresponding HLS or XA files that may use the sound banks. In fact all those with missing XA and HLS files refer through two specific LOGICOBJ files and may be an internal link to Sound Banks.

A third group of files used are called LEV files. These consist what looks like an Text version of a TRK file of which there are 374, and the rest are XA samples. All sorts of parameters for these sounds are controlled through LEV files. These files also refer to alternates or chains of sounds, often used in the ambient soundscape. I cannot see how these are invoked, probably directly though the exe. They do not seem to be used directly from exemplars.

So all up there is a pretty vast collection of sounds. Some exist at 5 levels of relative volume to account for zoom - most are tied to Buildings, Automata, Props and Animations. In fact pretty much any item you could click on or zoom into may have a sound file. Think of them as sound props.

How can we make use of this info:

Firstly it looks like the AB files are entirely beyond us to make without having specialised tools. Also there seems to be a hard compiled limit of 43 in the AE Directory. Similarly, the LEV files don't seem to be addressable by user properties. They might accept different XA files though, and substitutes for the originals would probably work - eg. use Currawong or Magpie sounds in place of Crows. The XA files would need to have consistent properties, though.

This leaves us with the TRK files, and the XA or WAV files. Taking the TRK files first - there is a good collection that can be used as the basis of Plop, Query, Activation and Action sounds for buildings etc. In fact because nearly all exemplars have been derived/cloned from MAXIS buildings or automata they all refer to the original TRK files. However All civic buildings (and their derivatives) refer to one of 2 missing TRK files to be used as Activation effects, but, as usual, SimCity just ignores them like missing props, and we have never heard them to be missing. I may try to make a fix for this, even if just to get rid of the Missing Error.

XA or WAV files are quite easy to make - they are mostly 22050Hz, mono, 16bit samples. Some are 8K,mono,8bit files. Any good wave editor (I still use CoolEdit) can make them from stuff around the net or even recorded yourself. They need to match quality, volume and duration with the samples they are meant to emulate - so a 4 second query sample should be matched in duration. If you do the matching well then it should be possible to clone a suitable TRK file, give it a new UNIQUE instance and refer to the new sample (which should also have a Unique Instance). Also like above, replacing an existing sound should also work. I know this can work and there are a few spread in the SPAM Mod and other places.

Thirdly there are a large number of TRKs that I have never heard in-game. I assume they are used somewhere or else why have them in the files? You obviously need to click around a bit to hear them. e.g. The mayor's house is a popping champagne cork and a yay. Some of them are in Simlish and are quite amusing to listen to. 

~~~~~~
Below is a reference to a zip containing an Excel spreadsheet with all of the data about the game files and their relationships as I have found them so far. It took a few elapsed days of pottering all up and I'm now over it.. All KEY references are the last 7 hex digits of the Instance, as the first digit is a key access randomizer. The TRK-AUDIO Tab shows the available TRKs and their structures. Their Instances should be used in experiments. There are a few duplicate uses of Sounds i.e. multiple TRKs referring to the same Audio. The AUDIO tab lists all of the XA(WAV) files that are to be found and what sounds they represent (as I have identified them, that is). Other tabs show the data relationships between files. Some of the files look incomplete and can't, so far, be fathomed. You may have more luck.

SC4Audio.zip

Also below there is a zip of a MP3 I made of a slab of Simlish samples - quite funny to listen to, I've absolutely no idea of what they're on about. :-)

SC4Simlish.zip

[CorinaMarie 29,575]

21 minutes ago, rivit said:

probably some of the LUA scripts

If it's any help, there is this block in 0xFFE2858B (_sfx.lua).

SFX = {                 ----  			sfx = {SFX.Automata_StrikeSmall},
   Automata_Panic          = "0xa9f14c47",
   Automata_ArmyDrill      = "0xea55b3ca",	-- Begin Added 08/05/02 by Marc Farly
   Automata_BooLarge1      = "0xca55b40d",
   Automata_BooMedium      = "0xca55b6f2",
   Automata_BooSmall       = "0x8a55b719",
   Automata_CheerLarge1    = "0xca55b733",
   Automata_CheerMedium    = "0xaa55b748",
   Automata_CheerSmall     = "0xaa55b75a",
   Automata_RiotLarge      = "0x2a55b78a",
   Automata_RiotMedium     = "0xea55b7ac",
   Automata_RiotSmall      = "0x0a55b7cc",
   Automata_StrikeLarge    = "0x8a55b7de",
   Automata_StrikeMedium   = "0xaa55b75a",
   Automata_StrikeSmall    = "0xaa55b80b",
   Automata_ThemeparkWalla = "0x6a55b892",	-- "Walla" is nonspecific crowd dialog, as heard during a party
   Automata_WallaLarge     = "0xea55b81f",
   Automata_WallaMedium    = "0x0a44b832",
   Automata_WallaSmall     = "0xea55b845",
   Automata_ZooWalla       = "0xaa55b8b6",	-- End Added 08/05/02 by Marc Farly
   Animal_Chimp		   = "0xca777589",
   Prison_Siren		   = "0xea777468",
   Automata_KidsPlay	   = "0x6a55b28a",		-- Can be used for children arriving to and leaving school
   Automata_Park_Crowd	   = "0xeaa70cd3",	-- Use for park crowds. Added 10/03/02 by Marc Farly
   }

And prolly less useful is each Lua that assigns SFX to some automata. In the following, just ignore the first 3 digits code as that's our own reference number. Then the Hex Code is the IID if you'd want to look them up, and the rest is the text containing "SFX.". Ones preceded by two or more dashes are commented out and not used, but I included them for thoroughness.

Spoiler

101 - FF89227D - attractors.lua:         --~ sfx = { SFX.Automata_Panic},
109 - FF0B6EAD - apebreak.lua:           sfx = { SFX.Automata_Panic, SFX.Animal_Chimp},
110 - FF4B1CB6 - armybase.lua:			 sfx = {SFX.Automata_ArmyDrill},
111 - FFC78D3B - armybase_peds.lua:      --~ sfx = { SFX.Automata_WallaMedium},
112 - FF0B25FD - armybase_jumpjacks.lua: --~ sfx = {SFX.Automata_ArmyDrill},  --- For the Generator itself
112 - FF0B25FD - armybase_jumpjacks.lua: sfx = { SFX.Automata_ArmyDrill},
115 - FFC8BBDC - biz.lua:                sfx = {SFX.Automata_WallaSmall},
116 - FFD7FB99 - bus_stop.lua:           sfx = {SFX.Automata_WallaSmall},
119 - FFE75A90 - convention.lua:         sfx = {SFX.Automata_WallaSmall, SFX.Automata_WallaMedium},
122 - FFDFB44E - crowd.lua:              sfx = {SFX.Automata_WallaMedium},
124 - FF6F5F68 - crowd_sm.lua:           sfx = { SFX.Automata_Park_Crowd},
126 - FFEA6C30 - crowd_sm_stand.lua:     sfx = { SFX.Automata_WallaSmall},
128 - FFB41119 - disasters.lua:          sfx = { SFX.Automata_Panic },
128 - FFB41119 - disasters.lua:          sfx = { SFX.Automata_Panic },
128 - FFB41119 - disasters.lua:          sfx = { SFX.Automata_Panic },
130 - FFE000DB - dmv.lua:                sfx = {SFX.Automata_WallaMedium},
131 - FFD0A790 - fire.lua:               sfx = { SFX.Automata_Panic },
132 - FFD5B679 - fireworks_crowd.lua:    sfx = {SFX.Automata_CheerLarge1},
132 - FFD5B679 - fireworks_crowd.lua:    --~ sfx = {SFX.Automata_CheerLarge1},
132 - FFD5B679 - fireworks_crowd.lua:    sfx = {SFX.Automata_CheerLarge1},
132 - FFD5B679 - fireworks_crowd.lua:    sfx = {SFX.Automata_CheerLarge1},
135 - FFCECECC - jailbreak.lua:          sfx = { SFX.Prison_Siren },
135 - FFCECECC - jailbreak.lua:          --~ sfx = { SFX.Prison_Siren }, --- This is for the generator only - should be a one shot.
135 - FFCECECC - jailbreak.lua:          --~ sfx = { SFX.Prison_Siren }, --- This is for the generator only - should be a one shot.
135 - FFCECECC - jailbreak.lua:          sfx = { SFX.Prison_Siren }, 
136 - FF07E94C - kidcrowd.lua:           sfx = { SFX.Automata_KidsPlay},
137 - FF20477B - landmark_ogle.lua:      --~ sfx = {SFX.Automata_WallaSmall},
138 - FF91DC12 - landmark_queue.lua:     sfx = {SFX.Automata_WallaSmall, SFX.Automata_WallaMedium},
139 - FFB77452 - maxis.lua:              sfx = {SFX.Automata_WallaSmall,},
142 - FF9BD3D0 - niteclub.lua:           --~ sfx = {SFX.Automata_WallaMedium},
142 - FF9BD3D0 - niteclub.lua:           sfx = {SFX.Automata_WallaSmall,},
142 - FF9BD3D0 - niteclub.lua:           sfx = {SFX.Automata_WallaSmall, SFX.Automata_CheerSmall, SFX.Automata_RiotSmall},                  
143 - FF1404C5 - strikers.lua:			 sfx = {SFX.Automata_StrikeSmall},
143 - FF1404C5 - strikers.lua:			 sfx = {SFX.Automata_StrikeSmall},
143 - FF1404C5 - strikers.lua:			 sfx = {SFX.Automata_StrikeSmall},
144 - FF76B38D - reward.lua:             sfx = {SFX.Automata_CheerSmall,},
144 - FF76B38D - reward.lua:             sfx = {SFX.Automata_CheerLarge1,},
144 - FF76B38D - reward.lua:             sfx = {SFX.Automata_CheerLarge1,},
145 - FFB58A07 - nuclear.lua:  			 sfx = {SFX.Automata_StrikeMedium,SFX.Automata_BooLarge1,SFX.Automata_StrikeMedium},
145 - FFB58A07 - nuclear.lua:  			 sfx = {SFX.Automata_StrikeSmall, SFX.Automata_BooSmall},
150 - FFDC382C - riots.lua:			     sfx = {SFX.Automata_RiotSmall},
150 - FFDC382C - riots.lua:			     sfx = {SFX.Automata_RiotLarge},
150 - FFDC382C - riots.lua:			     sfx = {SFX.Automata_RiotMedium},
151 - FF5971D3 - simcrowd.lua:           sfx = { SFX.Automata_ThemeparkWalla},
152 - FFF47979 - schools.lua:            sfx = { SFX.Automata_KidsPlay},
152 - FFF47979 - schools.lua:            sfx = {SFX.Automata_KidsPlay}, 
152 - FFF47979 - schools.lua:            sfx = {SFX.Automata_KidsPlay},
154 - FF211CBB - stadium.lua:            sfx = {SFX.Automata_WallaSmall},
155 - FF8BB157 - statefair.lua:          sfx = { SFX.Automata_ThemeparkWalla},
162 - FFD51175 - zoomanji.lua:           sfx = { SFX.Automata_Panic, SFX.Automata_ZooWalla, SFX.Automata_ZooWalla },
162 - FFD51175 - zoomanji.lua:           sfx = { SFX.Automata_Panic, SFX.Automata_ZooWalla, SFX.Automata_ZooWalla },
163 - FF270DA2 - mayorlimo.lua:          sfx = {SFX.Automata_CheerSmall,},
163 - FF270DA2 - mayorlimo.lua:          sfx = {SFX.Automata_CheerLarge1,},
163 - FF270DA2 - mayorlimo.lua:          sfx = {SFX.Automata_CheerSmall,},
163 - FF270DA2 - mayorlimo.lua:          sfx = {SFX.Automata_CheerLarge1,},
163 - FF270DA2 - mayorlimo.lua:          sfx = { SFX.Automata_Panic },
163 - FF270DA2 - mayorlimo.lua:          sfx = { SFX.Automata_Panic },

 

 

[rivit 3,391]

3 hours ago, CorinaMarie said:

If it's any help, there is this block in 0xFFE2858B (_sfx.lua).

Thanks Cori - all of these reference TRK files in my lists. 2 of them seem to be of the Sound bank variety i.e. do not reference loose XA files

[carlosmarcelo 973]

7 hours ago, rivit said:

Also below there is a zip of a MP3 I made of a slab of Simlish samples - quite funny to listen to, I've absolutely no idea of what they're on about. :-)

It's really funny!  Perhaps Simlish conversations were to be used with MySims.   This is my guess. 

[CorinaMarie 29,575]

9 hours ago, carlosmarcelo said:

Perhaps Simlish conversations were to be used with MySims.   This is my guess. 

The MySims in the game do speak Simlish and some of that does sound familiar from them.

[rivit 3,391]

Quote

Automata_WallaMedium = "0x0a44b832",

FYI Cori, this entry from the SFX.lua actually doesn't exist - however 0x0a55b832 does and it is a crowd noise - seems like a typo but could have been a fast fix.