Game framework compatible DLL loading (and other modding discoveries)

[simmaster07 1,220]

Hi everyone,

I just finished up exams, which means it's time for the ~biannual check-in~. Multi-threading and the graphics engine proof-of-concept are on hold to focus on smaller but more immediately impactful changes. In the meantime, let's talk about Mac.

I got a secondhand Mac Mini recently, and I wanted to see if Macs can be modded using the equivalent of DLLs on Macs (aka dylibs). As it turns out, despite Maxis already writing the code to handle loading libraries, Aspyr stripped this function from their port. Thankfully, though, all the other components for loading dylibs (searching the Plugins folder for them and attempting to actually load them) seem to be there, so all that needs to be done is rewriting this one function that tries to get the operating system to load the library.

macOS supports this pretty cool thing called function interposition, which means that even if a program doesn't load a library you can:

1. Force it to load a specific library anyway, and
2. Use that library to replace functions in the program with your own implementation.

The next few days are gonna be busy as I pack up and get ready to go back home for the holidays but by mid-January I want to accomplish three main goals that I'll keep you all posted on:

1. Fix SC4 on Mac so it actually tries to load game-framework libraries.
2. Port the extra cheats DLL to Mac.
3. Rewrite SC4Fix to work on both Mac and Windows.

[matias93 7,535]

You are truly a wizard!! Just mentioning the multithreading and the graphic fixes on passing, as if they were simple bugs to smash away, that's mindboggling!

On the Mac initiative, the only thing I think could be an obstacle is the announcement, by part of Apple, of the upcoming end of support for 32 bit applications, which could make the Aspyr port unusable. I think you should check the dedicated thread on this...

[simmaster07 1,220]

32 minutes ago, matias93 said:

You are truly a wizard!! Just mentioning the multithreading and the graphic fixes on passing, as if they were simple bugs to smash away, that's mindboggling!

On the Mac initiative, the only thing I think could be an obstacle is the announcement, by part of Apple, of the upcoming end of support for 32 bit applications, which could make the Aspyr port unusable. I think you should check the dedicated thread on this...

I've looked at the thread and will still be moving forward with it. Personally I'm betting Aspyr will recompile most, if not all of their desktop games for 64-bit when the time to do so approaches, at least so that their publishers won't get mad that they will get literally zero return on investment the moment that 32-bit is deprecated and users upgrade if they don't do anything. For the most part the effort involved in recompiling a 32-bit Intel app for 64-bit Intel is minimal, and retargeting well-crafted code (like SimCity 4, as far as I'm aware anyway) shouldn't be much harder than changing a few options in the compiler toolchain and maybe fixing a few things here and there.

[T Wrecks 10,209]

This is really exciting news! If Macs were able to use the Extra Cheats DLL, this would mean we'd no longer need ploppable RCIs... at all.

And if I was a ruthless dictator in a futuristic world, I'd make sure to have you cloned a few times to accelerate the process.

[matias93 7,535]

59 minutes ago, T Wrecks said:

If Macs were able to use the Extra Cheats DLL, this would mean we'd no longer need ploppable RCIs... at all.

I didn't understand why this would be the case, ploppable commercial and industrial buildings are useful not only on the Aspyr port. 

[T Wrecks 10,209]

Plopping commercial and industrial can be very useful, yes. I totally agree. This is precisely why I welcomed the Extra Cheats DLL so much: it allows plopping of each and every building that you have, no matter what category, no matter whether it's inherently growable (e.g. mansion) or ploppable (e.g. hospital) - call up the "buildingplop" cheat, select, plop.

This eliminates the need for additional ploppable lots, which are a royal pain in certain body cavities to make: even if PIM-X makes it possible to avoid the tedious process of creating a growable .sc4desc first and coyping some properties over to the ploppable-to-be, you still end up having to add a custom icon, a menu name and description. Not to mention menus - pretty crowded already due to the amount of custom content - clogged with dozens of ploppables, endless scrolling...

Using the DLL instead, you make the growable - a really staright-forward process - and anyone can plop it anytime, anywhere, however often they want, as they see fit. Anyone but Mac users, that is. Or was. Or is going to be a thing of the past if @simmaster07 succeeds.

We'd keep the same possibilities, but Mac users would get a far superior tool, and creators would be relieved of the task to use kludges like ploppable lots if they want to treat Mac users equally. Everybody wins.

[matias93 7,535]

wow, I didn't consider that, but you are clearly right on this!

Now, if lotplop is poised to become the main way to plop lots in the game, maybe some tweaks would be useful: for example, if the function can be scripted, one could add a string of text when running the command and make the code use that string as a query. For example, entering lotplop RTMT would display the list of lots that contain the RTMT string on their name, allowing for convenient filters.

Eventually, if this is easy enough to be done, several functions now existent on the DAMN menus could be implemented directly to the lotplop cheat, or maybe an hybrid approach could be used, but that sounds even harder to code :/

  Edited December 23, 2017 by matias93  
that last line was to another thread, weird, no?

[T Wrecks 10,209]

22 hours ago, matias93 said:

Now, if lotplop is poised to become the main way to plop lots in the game, maybe some tweaks would be useful: for example, if the function can be scripted, one could add a string of text when running the command and make the code use that string as a query. For example, entering lotplop RTMT would display the list of lots that contain the RTMT string on their name, allowing for convenient filters.

This is not a bad idea! Dunno if it's possible, but it would certainly be helpful for people with larger plugins folders.

Using the buildingplop cheat also highlights another area that has been pretty much under the radar so far: building exemplar names. I've had a little exchange with @Francis90b via PM about this very topic. Ideally, the exemplar names (these are what appears in the buildingplop list as names of the assets) would be such that they make it easy for users to see what they are about to plop, or at least easy to find what they are looking for.

However, some great uploads come with names that make perfect sense when referring to that one building they relate to, but don't help a user find out what they are - RCI type, wealth level, author, any set or pack they belong to? No idea.

Should we get around to fixing up STEX content and bundling it into MODPACCs, this would therefore be another aspect to bear in mind, I think.

[simmaster07 1,220]

So that's goals 1 and 2 down for the most part. I've posted a prerelease of the injector on GitHub (which also has the source code and some instructions on how to use it in its current state). It's a useful proof-of-concept, but I'm not really happy with the effort required to set it up and use it, which is why it's not on the STEX yet. I'm going to work on packaging it in a .pkg file or something with a simple installer before uploading the injector and some Mac-ported mods (extra cheats, extra extra cheats, growable city hall, etc.). After that I'll start investigating puzzle piece/TE lot crashes on Mac to see if I can rewrite SC4Fix to not be specific to Windows patch 641.

[CorinaMarie 29,605]

Tis nothing short of amazing what you are doing!

[simmaster07 1,220]

Soooooo I was going to upload this to the STEX with a new package installer but I started getting errors when I tried to upload the file. I'll probably try again tonight to see if this isn't a persistent issue, it's just a bit awkward since I already posted the support thread so I could put a link to it in the installer. Anyway, here's prerelease 2. Tested and works with the Steam version of SC4 Mac, but the App Store version is, as always, extremely iffy. Since it's installed from the App Store it's a lot more strict about making sure any code that gets injected is signed, although I haven't been able to test it since my App Store copy corrupted itself and refuses to run even after reinstalling it, so I'm not 100% sure it works there.

[CorinaMarie 29,605]

2 hours ago, simmaster07 said:

I'll probably try again tonight to see if this isn't a persistent issue,

Yes, please do try again. We have been having intermittent errors with our files host. Hopefully they will get it all resolved.

[mattb325 28,880]

Thankyou for everything that you've given this old game and the community. Everyone of your uploads has been nothing short of completely game-changing 

[Adolfff 14]

On 28.10.2016 at 3:12 PM, simmaster07 said:

EDIT: It's on the STEX now.

The game (1.1.641) does not start after installing this file. This is strange, but SC4Fix works completely. My old PC is rather weak, WinXPpsp3, Pantium D, 2 RAM, GF9600gt.

[Sabretooth78 808]

On 12/7/2018 at 6:00 AM, Adolfff said:

The game (1.1.641) does not start after installing this file. This is strange, but SC4Fix works completely. My old PC is rather weak, WinXPpsp3, Pantium D, 2 RAM, GF9600gt.

Seems it might be an issue with WinXP, because I'm noticing the same behavior.  If I view the task manager processes list, SC4 appears but disappears very quickly.  Doesn't make a difference whether I place it in the My Documents or Program Files plugin folders.

I'm running a WinXP (fully updated) virtual machine in Parallels 14; macOS 10.13.6 High Sierra.  The DLL works fine in my Win7 virtual machine.  I suppose that's good enough for me - I didn't plan on creating disasters but if I have a nuclear incident I know I can clean the city up in the alternate OS - but that doesn't help the OP.

[doldrum 67]

May I offer an opinion? I am not a programmer in any way, shape, or form. I can do hex conversions in my head, and can write a reasonably useful basic BASIC program, if lucky, spot a pattern in a hex dump, BUT I am not a programmer. Still, I remember the color and feel of the copier-resistant paper the first copy protection for DOS SimClassic came with. I've been around awhile.

Kudos to you, for what you are doing. You advance our knowledge and capabilities, and even if you were to disappear tomorrow, we would be better for you having been here.

But I've seen this play before. Despite your desire to give useful things to us,  both the Windows & Mac communities, please admit to yourself that the biggest reason you are doing this is to expand your knowledge & expertise, to satisfy your own innate curiosity. You are young, and pursuing what may be a fruitful career path in this field. No shame in that, and I look up to you and admire your talent.

Still, one day we will look around and say "Where is simmaster07?" You will have moved on to bigger and brighter things, deeper puzzles & challenges to conquer. Again, no shame in that, it is the way of the world, the Circle of Life.

But we, the SC4 community, we are selfish. We want you to stay right here, forever, and give each of us that certain one thing we want in this little corner of the world. When you inevitably move on, we will moan and groan until we find another bright, young, eager mind.

Much of what your doing is over my head, though I recognize you are trying to lay a foundation for a skyscraper, not build one from scratch.

On another site, I expressed my preferences to an artist. Still, I qualified it. I pointed out that it was HIS art, and he must do the art he wants to do.

Let me offer this proposal. Once you've laid the foundation to your satisfaction, ignore the calls for big, grand mods that will "revolutionize" the game. Go small and practical. There are so many unanswered questions about things we already know of. Given that it is commonly believed that Buggi's cheats were intended as developer's shortcuts, tools to do useful things within the game, pursue those. So many have unknown parameters & formats. Do what you can to lay those out.

In the long run, I believe that will do more for this community. The way it is heading, a few people with the requisite skills will be able to write DLLs. That may be good for programming as a life choice. I think, instead, letting many people have their curiosity piqued by being able to do, ON THEIR OWN, some small task, enabled by widespread knowledge of using those tools, would be far more fruitful.

Ok, I'm done. Thanks for listening.

 

[doldrum 67]

    I'd like to expand on my prior post with an example from my own personal play style. I am heavily into the civil engineering aspect of the game. As such, I use the 'terrainquery' cheat regularly. One of the advantages of the cheat is that it reports the elevation of a cell, i.e "x=520.0, y=270.0, z=520.0", where "y" is the ELEVATION, and "z" is the corresponding "y" of an (x,y) coordinate. (The "x" & "z" coordinates are meters from (0,0) measured from the NW corner of the tile.) It is VERY useful, but only reports the elevation to one decimal place, while an actual cell elevation may be any  real number whose precision is unknown. The 'terrainquery' rounds it up or down to the nearest tenth. Getting an exact elevation with the terrain tools can be tricky. A half of a tenth can make the difference as to whether a land bridge will or won't be built.

    In one of the original lists outlining Buggi's cheats was the quote:

"...except for these, which I don't know how to get to work...
        ...
    SetCellAltitude
    SetAltitude
        ..."

    MAYBE! I don't know what experiments he undertook, but 'SetAltitude' sort of works, it's just not immediately visible.

--------
    A  short diversion on formats - it is common practice to describe a coordinate format as 'x1,y1' 'x2,y2' This carries through in various descriptions of the cheats (resulting in some confusion). Maxis programmers chose to use 'space' as the delimiter, so

"SimDate [Value] - changes the in-game date to the value you entered, this is useful for testing seasonal props. Use a date after 12/31/1999 in the format SimDate 12 31 1999."

or

"PlaceNetwork [type(0-11)] [x1] [y1] [x2] [y2]" (In this case, the 'x' & 'y' values are cell coordinates - i.e., (4,5), not meters from the NW corner, so 'placenetwork 3 4 6 7 6' is the only format that works. )

where 'spaces' are used between parameters rather than commas.

    It seems logical, then, that the same format is used for the 'SetCellAltitude' & 'SetAltitude' cheats.
--------

    Like Buggi, I have been unable to make the 'SetCellAltitude' cheat manifest any result. I have found though, that the 'SetAltitde' cheat works. Sort of.

    I made some assumptions about the format, and tested same. It appears it must be at least 3 parameters, i.e., 'setaltitude 100 200 350', where the format is 'setaltitude x=100 z=200 y=350'. Any attempt to use a delimiter other than a 'space', results in the 1st & 2nd coordinates counted as one parameter, the elevation as the second, and the final parameter as an unstated zero. Result: a zero elevation hole. Some pics to illustrate:

    I can live with it rounding down to the NW vertex. The elevation change is exact. There is just one fly in the ointment: No matter what I do, the second parameter, the 'z' coordinate, is ALWAYS set equal to the first parameter, the 'x'coordinate. In effect, you can only set elevations for the NW vertex of cells 0,0 1,1 2,2 3,3 etc.

    We learn a few things from this: the terraforming function works, the game maintains a separate elevation map from the terrain tile map, and the parser for this function is bugged.

    I can think of no rational reason for a programmer to write such a limited function. It is, to my mind, a bug.

    So, my question, to you and others working on .dlls for the game - is it possible to write a .dll that hooks in and bypasses the bugged parser, sending correct parameters to the terraforming function? Just food for thought to illustrate what I said in my prior post about, what I think, are fruitful directions for more research.

 

[asasnat 216]

I wonder if this could be used to add in new disasters to the game. I'd like to see a tsunami disaster mod. I did manage to modify the volcano disaster to create a lot more lava, and even tried creating a custom disaster that sets off a nuclear explosion when activated. I ended up failing at the latter.

Here's a video of the more lava volcano: 

 

 

[TogaMasterJohn 705]

On 12/4/2021 at 12:11 PM, Bruh Man said:

I wonder if this could be used to add in new disasters to the game. I'd like to see a tsunami disaster mod. I did manage to modify the volcano disaster to create a lot more lava, and even tried creating a custom disaster that sets off a nuclear explosion when activated. I ended up failing at the latter.

Here's a video of the more lava volcano: 

 

 

That really is cool!

 

[asasnat 216]

Oh, there's another thing that could be possible with this. Remember how in SimCity 3000 you could transport garbage to your landfills/incinerators/recycling centers/outside the city via rail? That function was unceremoniously removed in SimCity 4. However, with the game's ability to load in DLL files, that feature could be brought back from the eliminated.

I know there are some lots available that supposedly let you get trash out of your city via rail. However, they're not really functional. All they are is just an incinerator that doesn't produce pollution (and lets you hook up rail to it). With DLLs, we can make that actually functional, not to mention but rail transport of trash within the city borders (without a neighbor connection) would be possible with this, while it wouldn't be possible with these lots. Sorry if this was hard to read, it always ends up like this whenever it's more than a few lines long.

EDIT: Also, diagonal bridges.

EDIT2: Diagonal bridges are already a thing the community's been working on. Nevermind.

[asasnat 216]

For anyone who is curious enough, I found that the test DLL in the original post also works on SimCity 3000 Unlimited. This confirms my hypothesis that it uses a game engine similar to SimCity 4.

Or at least it partially works. It doesn't display any events when the game quits, which I think it is supposed to do when loaded into SimCity 4. Still better than nothing though.

[simmaster07 1,220]

On 4/22/2022 at 11:24 AM, Bruh Man said:

For anyone who is curious enough, I found that the test DLL in the original post also works on SimCity 3000 Unlimited. This confirms my hypothesis that it uses a game engine similar to SimCity 4.

Or at least it partially works. It doesn't display any events when the game quits, which I think it is supposed to do when loaded into SimCity 4. Still better than nothing though.

This hadn't even crossed my mind but it makes sense. In theory it should also work on The Sims 2 since cGZFramework also shows up in crash logs for the Mac version.

[Kopachris 7]

Still a long way to go, but I've at least discovered the interface ID for a BuildingOccupant is 0x87dda3a9, so I can at least get (tested) and probably set (not tested yet, it's late at night already) building names when they grow. Now I just need to be able to reasonably categorize buildings so I can have my dll read a file of potential names to choose from. Seems like that should be easy enough by checking against occupant group IDs that are already well-known... For now it's just popping up the building name in a message box on an InsertBuildingOccupant message, but still, nice!

Other things that seem possible: setting the window as borderless on load without relying on a whole separate .exe to set the flags on the window handle, making a cheat to mark all buildings as historical, creating a built-in (possibly quiet) autosave instead of (again) relying on a separate .exe which this time just sends the Ctrl-S keystroke, and maybe even custom keyboard shortcuts even for tools/commands that don't normally have one.

@simmaster07 if you've come up with any more documentation or usable GZCOM source code since... (checks GitHub again) early 2018... I'd be thrilled to see it, but thanks for the work you've done already! (I've forked your gzcom-dll repo.)

[Propfam 2,288]

Hi @simmaster07, I heard that @OrSpeeder (aka speeder in Devotion) made discoveries in unused UI components such as pie chart. I hope we can explore it deeper. And also, his attempt to RDTSC.

[asasnat 216]

SimCity_1.dat includes three unused God mode tools: Spring, Rainfall and Lake. They allow you to place bodies of water above sea level. Teirusu's mod enables Rainfall and Lake. Unfortunately the water that you place down is completely stoic and doesn't save. Also, the Spring tool isn't enabled because it doesn't work.

I was looking at November 2002 preview footage when I noticed an above-sea-level body of water being placed:

Not only that, you can clearly see it sloshing around as more water is added - kind of like how Cities: Skylines simulates bodies of water.

Would it be viable (or even possible at all) to locate the code that handles the water physics and enable it with a DLL? Assuming - of course - that there isn't an option somewhere deep in the .dat files that needs to be ticked to enable it, or that the code for this is even present in the final build at all.

Sorry if this is the wrong place to talk about this. I noticed the water physics at 3:00 AM while discussing SimCity 4 prerelease footage and have no idea where to talk about this. We can make this a separate thread if the admins so desire.

[matias93 7,535]

On 6/27/2023 at 11:06 AM, asasnat said:

Would it be viable (or even possible at all) to locate the code that handles the water physics and enable it with a DLL? Assuming - of course - that there isn't an option somewhere deep in the .dat files that needs to be ticked to enable it, or that the code for this is even present in the final build at all.

Sorry if this is the wrong place to talk about this. I noticed the water physics at 3:00 AM while discussing SimCity 4 prerelease footage and have no idea where to talk about this. We can make this a separate thread if the admins so desire.

From all what we know, the whole water simulation was abandoned inside the game code because of time pressures, and there are several other vestigial elements, like the moisture simulation and the fact that water towers and water pumps mention a "water source" that could presumably vary if the simulation was working.

Making the water simulation work would probably require more than a hotfix like the ones used for the Extra Cheats or the Prop Pox cure, because it would entail reconnecting all the elements that were meant to work alongside the simulation, to find the simulator inside the game code, and if it's not there, to create it from scratch. It "should" be possible, but as I read an unrelated gamedev some weeks ago, "I'm always amazed at game modders, because if we consider impossible to include extra features on the second third of development, they are doing it on the fourth third, and going in blind"

[Tarkus 6,932]

Since Rain and Lake work (I wasn't even aware of Spring) in terms of actually being able to place above-sea-level water in-game, it would seem to me that the real crux of the matter is getting the water data to be stored in the savegame.  My guess is that it isn't, or, at least, not fully/properly.  I'm going to tag @smf_16, who is our resident savegame expert, to see if he has any insight on that side of the equation.

-Tarkus

[TogaMasterJohn 705]

On 6/27/2023 at 11:06 AM, asasnat said:

I was looking at November 2002 preview footage when I noticed an above-sea-level body of water being placed:

Not only that, you can clearly see it sloshing around as more water is added - kind of like how Cities: Skylines simulates bodies of water.

Would it be viable (or even possible at all) to locate the code that handles the water physics and enable it with a DLL? Assuming - of course - that there isn't an option somewhere deep in the .dat files that needs to be ticked to enable it, or that the code for this is even present in the final build at all.

Cori and CB did manage to get a river tool (similar to what is seen above) working with a simple modification without needing a DLL mod or anything major. As of now, it is not perfect, but it is quite cool to use! The tool is still a work-in-progress. Right now, it does not modify the terrain around it (though that functionality could possibly be implemented).

The functionality is in the game... it was just never fully finished!

Does the water save? Unlike the rain and lake, it sort of does. Each time you open the tile, the water will start to re-flow from where you first plopped it.

[SIM-ple Jack 17,444]

20 hours ago, TogaMasterJohn said:

Cori and CB did manage to get a river tool (similar to what is seen above) working with a simple modification

Any idea where it might be in that massive thread? I made it to about page 34 reading backwards, found a few tricks.

[asasnat 216]

21 hours ago, TogaMasterJohn said:

Cori and CB did manage to get a river tool (similar to what is seen above) working with a simple modification without needing a DLL mod or anything major.

Oh, I recognize that one. Accidentally found it when playing around with Cori's LUA execution thingamajig! I didn't think it was the originally intended river tool, because I couldn't bridge over it (although that's probably just a limitation of having to spawn the effect in manually by using cheats or scripting)

EDIT: Another thing. Unused parameters in the terrain tuning INI file suggest that water streams would pick up sediment from the ground, turning a different color. As it goes further downstream, it would deposit it back into the ground, leading to slow, gradual, hydraulic erosion. I wonder if this would work with the stream effect?