Game framework compatible DLL loading (and other modding discoveries)

[SecretMbr_735754 2,157]

The interesting thing about the behavior of this mod so far is that with the City Hall Unlock removed, and with the Reward Unlock still installed, the upgrade works as intended with the 2nd city hall appearing in the reward menu, and it does so automatically without any messaging.

[CorinaMarie 29,605]

8 hours ago, simmaster07 said:

EDIT: It's on the STEX now.

Nice work! I'm bumping this for the peeps who'd already read your post but wouldn't know about your edit. I wouldn't have known except I got beeped when @MushyMushy liked my post here and I clicked the link to find this new info.

[simmaster07 1,220]

For what it's worth, after getting city hall upgrades to work I also managed to fix a pesky CTD-on-exit in the DLL I made to demonstrate adding custom cheat codes:

https://github.com/nsgomez/gzcom-dll/releases/tag/poc-newcheats

Again, that DLL adds the cheats "Chernobyl" and "SievertBeGone" which makes the map fully radioactive or non-radioactive, respectively. The latter also has the interesting side effect of removing radiation from lots that generate it, such as the Advanced Research Center. I'm not entirely sure about uploading it to the STEX though since it seems underwhelming to only have two new cheats that are pretty niche.

[APSMS 1,892]

5 hours ago, simmaster07 said:

I'm not entirely sure about uploading it to the STEX though since it seems underwhelming to only have two new cheats that are pretty niche.

Pffft. I'm sure some users will find it handy over having to use a cheat lot that sends their radiation skyward if they accidentally bulldoze it. This way you can blow up cities with Nuclear powerplants with indemnity, especially if you accidentally saved the city and don't want to scrap it entirely after the explosion.

[MushyMushy 3,617]

I'd say go ahead and upload it. The ability of the second one to remove radiation from lots actually sounds handy for those who "play" the game.

[SecretMbr_735754 2,157]

UpdateCityHall shows up in the cheat window so can be applied at any time.  I couldn't resist the temptation so I went and got the nuke dll.  I just nuked a large city tile.  All the stats immediately went red, the advisors screaming, the whole tile irradiated.  Good thing we have virtual reality to play with. 

[fbstj 3]

heck!! this is such a groundbreaking and incredible improvement to the game I am really loving how much time you're putting into all this thank you for sharing your progress and making such cool things!!! 

[simmaster07 1,220]

Re: uploading the DLL with the radiation cheats to the STEX, I'll get on that once I've finished a couple of additions to it. In the code repository I have for this project, I already added a couple of cheat codes for spawning riots and toxic spills at will. Right now this only tries in random locations, though, so I'm currently trying to make it so that they spawn on the tile that the mouse is pointed at. I'd be content with uploading it once that's done.

[Cyclone Boom 16,875]

Exciting stuff! Great job with your continued progress.

Given this is uncharted territory for SC4 modding, we'd all appreciate any new 'bonuses' which might result from this, however big or small.

  Edited November 2, 2016 by Cyclone Boom  
Topic pinned & featured.

[simmaster07 1,220]

As promised, the "extra extra cheats" DLL with the radiation and disaster cheats has been updated to take advantage of the mouse's position when spawning disasters, rather than trying to shove them into random tiles. The STEX upload is over here.

While researching game methods for that, I came across a class called cGZWinLuaScriptConsole that appears to provide a debugging interface for Lua scripts. I don't think this is even enabled by Buggi's Extra Cheats DLL. If anyone knows if this has been implemented before I won't really bother with it, but otherwise I can try and add a cheat code that attempts to re-enable it. Aside from that, I do want to move on to creating some properly organized documentation for all of this before doing any deeper dives to make it easier for the select few people in this community who do know C++ to work with this kind of modding.

  Edited November 4, 2016 by simmaster07  
Forgot to edit in the class name

[CorinaMarie 29,605]

8 hours ago, simmaster07 said:

cGZWinLuaScriptConsole that appears to provide a debugging interface for Lua scripts.

When I was poking around I found a few which might be related to debugging:

• SetDebugLevel
• SetGZLogLevel
• GZLog
• CreateGZLog

Then after some Googling I found that "CreateGZLog filename" would create a zero byte file outside of the game. I wonder if "GZLog message" might be how to write to said file, but I didn't experiment with it.

I'm sure you already know all this and more tho.

There was also FX_00_SkyWriting somewhere. Perhaps that would also accept a message and then maybe an airplane would write said message in game? Thing is, I wrote myself limited notes and when I realized it was more involved than I would pursue, I just moved on to things I could actually do something with.

[Buggi 684]

I've always had the source to the extra cheats DLL. I thought I posted it when I sent out the DLL file.

Anyway, it's attached.

 

extracheats.cpp

[simmaster07 1,220]

11 hours ago, Buggi said:

I've always had the source to the extra cheats DLL. I thought I posted it when I sent out the DLL file.

Anyway, it's attached.

 

extracheats.cpp

I did use that source code as a reference, but it's incomplete since it requires a declaration for the cRZCOMDllDirector class, as well as the interfaces for cISC4App and cIGZFramework, so it wouldn't compile. The main breakthrough is the reconstruction of those interfaces.

[tigerbuilder 131]

This sounds amazing. Thankyou for working in this area. 

[rsc204 21,808]

I don't know what your plans are for further enhancements, but I do wonder if there is any way to make TAB-Rings, used in the NAM work without the NAM.

To clarify what I mean, with a Tab-Ring you can have multiple pieces that are cycled between with the TAB key. But, in order for them to work, RUL code must be included in the NAM controller. I'm not even sure if it technically works with lots right now, but that's a moot point, because adding third party objects like this to the NAM codebase is pretty much a non-starter.

But, if we had a way to enable such Tab-Rings without the main RUL code having to be used, perhaps we could apply it to other menus for groups of lots and modular sets? It doesn't matter if it had to remain RUL-based, just that the RUL could be read from a file independent of the main RUL files in the NAM controller. I'm thinking of some system that would enable anyone to make it work for existing and new content. Much like how I could make an MML for a set of existing lots and distribute it for others to make use of. There just needs to be a way for the game to interact with some sort of configuration file/additional RUL file and actually enable the Tab-Rings themselves.

This could be a killer blow to the problem of highly-congested menus. Whilst MMLs and DAMN are two potential solutions, I still don't feel either of them are really ideal. Since we have Tab Ring functionality, it would be great if we could better utilise it. Does this sound remotely possible @simmaster07? Its still hard to wrap ones head around what will and won't be possible utilising your breakthroughs with DLLs. I find myself wishing I'd paid more attention to C programing and it's derivatives like C++ now.

[simmaster07 1,220]

19 hours ago, rsc204 said:

This could be a killer blow to the problem of highly-congested menus. Whilst MMLs and DAMN are two potential solutions, I still don't feel either of them are really ideal. Since we have Tab Ring functionality, it would be great if we could better utilise it. Does this sound remotely possible @simmaster07? Its still hard to wrap ones head around what will and won't be possible utilising your breakthroughs with DLLs. I find myself wishing I'd paid more attention to C programing and it's derivatives like C++ now.

It's honestly too soon to tell. I don't think I've gotten through even a quarter of the game's interfaces so it's hard to say if the game's behavior can be modified to that extent.

[simmaster07 1,220]

A couple of quick new notes:

First, I figured out that you can actually override game behavior completely by changing a single parameter in the DLL source code to indicate you have a more recent version of whatever you're trying to override. However, you can't override individual functions — you have to reimplement and override the entire class containing what you're trying to modify, from top to bottom. For complex interfaces like the lot developer this is somewhat of a nightmare.

Secondly, I found out that (just like everything else in the game) the graphics subsystem is also made up of code similar to a DLL plugin, just baked into the game. What's more interesting is that "SimCity 4.ini" allows you to load custom graphics drivers if you know their unique identifiers (and they exist and implement the graphics interface). This means that the old DirectX 7 renderer (which is 18 years old and not optimized for modern hardware) or OpenGL (which has been broken) can be replaced with new, modern implementations.

As a proof-of-concept I wrote a dummy DLL with the same graphics interface but no functionality:
https://github.com/nsgomez/SimGLDx11/releases/tag/experiment-0

If you drop this in your Plugins folder, you should get a "Hello!" message. However, adding the following lines to "<Install Directory>\Apps\SimCity 4.ini" will force the graphics system to use the new driver and give an "Init" message before exiting. (None of the graphics functionality is actually implemented, so this is fine).

[Graphics Drivers]
OpenGL =SimGLRef.dll, C4554841
Software=SimGLSoft.dll,7ACA35C6
D3D7 =SimGLDx7.dll, BADB6906
D3D11 =SimGLDx11.dll, 5896AB0D

Removing the D3D11 line changes the game back to its standard settings and allows it to load again without having to delete the DLL itself.

It's also possible to override the D3D7 driver entirely instead of creating a new one, but if the new driver gets deleted this makes it slightly harder to get the game back to a working state since it starts associating DirectX rendering with a driver that no longer exists.

[CorinaMarie 29,605]

3 hours ago, simmaster07 said:

"SimCity 4.ini" allows you to load custom graphics drivers if you know their unique identifiers (and they exist and implement the graphics interface).

Whoa! That could be extremely useful.

 

On a related note of DLL overrides here's a couple ideas that may not be possible. I'll suggest them so you can shoot them down.

1. Override the code that causes the save file to be compressed. (The routine that quits working when the save file reaches a certain size.) My reasoning is that if it isn't compressed from the beginning then the sudden appearance of a poxed city would not occur.
2. Override the addition of the checksum to the save file or override the reading of same when loading a city. Here that'd then give us the possibility of editing the save game files directly and still being able to load them.

 

[jeffryfisher 2,176]

On 8/1/2017 at 11:29 PM, simmaster07 said:

What's more interesting is that "SimCity 4.ini" allows you to load custom graphics drivers if you know their unique identifiers

I think some of us have been doing something like that using  "Graphics Rules.sgr" according to the modern hardware thread, but the process can be daunting to non-power-users.

On 8/2/2017 at 2:36 AM, CorinaMarie said:

...here's a couple ideas that may not be possible.

Ditto:

• Enable larger screen resolution without messing up the region view.
• Debug the "Save & Exit" command to do what careful players always do: Save first, and then exit without saving.
• Override the disk-check so that the disk version can launch in Win-10 (keep this separate so we XP users can skip it).

[matias93 7,535]

3 minutes ago, jeffryfisher said:

Override the disk-check so that the disk version can launch in Win-10 (keep this separate so we XP users can skip it).

I think this one is illegal if done by a third party, but I'm not sure

[jeffryfisher 2,176]

Just now, matias93 said:

4 minutes ago, jeffryfisher said:

Override the disk-check so that the disk version can launch in Win-10 (keep this separate so we XP users can skip it).

I think this one is illegal if done by a third party, but I'm not sure

Cracking the program to skip the check might be illegal, but it should be okay to repair the disk-check so it works as intended. Right now, the program calls some ancient system command that Win-10 doesn't support. An override would call a more modern replacement function (if one exists).

I see it as an adapter, not as a security crack. As long as we just call the the right function and don't poke our heads inside, then we should be OK.

[rsc204 21,808]

1 hour ago, jeffryfisher said:

Override the disk-check so that the disk version can launch in Win-10 (keep this separate so we XP users can skip it).

1 hour ago, jeffryfisher said:

Cracking the program to skip the check might be illegal, but it should be okay to repair the disk-check so it works as intended. Right now, the program calls some ancient system command that Win-10 doesn't support. An override would call a more modern replacement function (if one exists).

Actually since this requires modifying the executable, it's totally verboten and is well outside the scope of .dll modding.

In fact it's even less simple, since the O/S handles such DRM systems, allowing the disk to be read, you'd need to either modify the .exe akin to a NoCD crack that skips looking for the disk or modify the disk itself to remove the protection from it. Both acts are pretty much explicitly illegal, more than just simply a violation of the EULA.

There is already a solution, since this is simply DRM handled by having a system file installed/running, which verifies the encryption allowing the disk to run. Simply find the original SecDrv.sys file from a previous version of Windows and manually Digitally Sign it to work on Windows 10. Then make a script that calls the file starting the service, runs SC4 and once it's past the check, stops the service once more. Of course this could potentially cause all sorts of security problems, but it would work. There is really no need to resort to more drastic solutions, it's just by default Win Vista-8.1 systems now have this file removed. Windows 10 never had one, hence the need to digitally sign it before the O/S will allow it to run.

 

[Tarkus 6,932]

It's always like Christmas when there's news in this thread--thanks for continuing to boldly go where no one has gone before here!

If we are talking about DLL possibilities, before he disappeared, @OrSpeeder had apparently found the key to making it such that the RHW network could actually have a proper monthly cost.  

-Tarkus

 

  Edited August 9, 2017 by Tarkus  
Darn autocorrect

[jeffryfisher 2,176]

21 hours ago, rsc204 said:

Actually since this requires modifying the executable, it's totally verboten

If indeed it must change the EXE, then it would be off limits. But I don't know enough about how disk verification works. I thought Win-10 had a new way to do that, and that a DLL could provide an interface to an obsolete function and map it to a new function. Such a DLL would be a compatibility interface for Win-10, which would help any and all old programs with this problem. It would not be a "crack".

21 hours ago, rsc204 said:

There is already a solution, since this is simply DRM handled by having a system file installed/running, which verifies the encryption allowing the disk to run. Simply find the original SecDrv.sys file from a previous version of Windows and manually Digitally Sign it to work on Windows 10. Then make a script that calls the file starting the service, runs SC4 and once it's past the check, stops the service once more.

OK, Though it has more steps than just dropping a DLL into a system folder, that's close enough to what I had in mind. As XP - Win7 machines die, eventually every SC4 on-disk player will need to get over this compatibility hump. To wrap it up with a bow, maybe someone could write an installer for it so the right files go to the right places.

I'll search for threads discussing the details (like where the SecDrv.sys file lives in a Win-10 machine, and can one signed copy be passed around, or does each computer need to do its own signing?) I want to bookmark the solution against the day I eventually need it.

 

 

[rsc204 21,808]

1 hour ago, jeffryfisher said:

To wrap it up with a bow, maybe someone could write an installer for it so the right files go to the right places.

Honestly, if you don't know how to do what I listed here, you probably shouldn't be doing it in the first place. Because the key changes in Windows 10 designed to keep the OS more secure are basically those things you are fighting here.

1 hour ago, jeffryfisher said:

I'll search for threads discussing the details (like where the SecDrv.sys file lives in a Win-10 machine, and can one signed copy be passed around, or does each computer need to do its own signing?) I want to bookmark the solution against the day I eventually need it.

Another reason why an installer won't work, you must sign it for your own machine/OS, to sign it for every machine requires a different process and maybe even MS certification, which is simply not going to happen for this particular file.

Note also this issue is in no way unique to SC4, but covers all disks using Safe Disc 2 DRM. All of this was explained here:

and here:

 

[narukaze 1]

Just out of curiosity, would it be possible for someone to use a DLL to reimplement all of the game's simulators to use multithreading properly? At this point, quad-core CPUs are pretty much standard, so the game's inability to use more than one core without destabilizing seems... inefficient. In addition, the DLL could add hooks into certain parts of the simulator, like freight, so others could expand the simulator's capabilities, using other DLLs. (At least, I get the impression that's possible from the documentation @simmaster07 has made.)

[matias93 7,535]

Multithreading would require to rewrite the entire executable and surely many DLL libraries, which is highly illegal and also a huge task. So that's a big no, at least until the game becomes legally abandonware.

About hooks, I also remember having read @simmaster07 saying that it is possible to do. The tiny detail there is that he's the only member of the community that knows how to write DLL files for the game, so we are dependent exclusively on his work to see more functionalities added.

[narukaze 1]

 

2 hours ago, matias93 said:

Multithreading would require to rewrite the entire executable and surely many DLL libraries, which is highly illegal and also a huge task. So that's a big no, at least until the game becomes legally abandonware.

About hooks, I also remember having read @simmaster07 saying that it is possible to do. The tiny detail there is that he's the only member of the community that knows how to write DLL files for the game, so we are dependent exclusively on his work to see more functionalities added.

The multithreading idea was based on the fact that according to simmaster, you can override default game behavior with a DLL.

On 8/2/2017 at 2:29 AM, simmaster07 said:

A couple of quick new notes:

First, I figured out that you can actually override game behavior completely by changing a single parameter in the DLL source code to indicate you have a more recent version of whatever you're trying to override. However, you can't override individual functions — you have to reimplement and override the entire class containing what you're trying to modify, from top to bottom. For complex interfaces like the lot developer this is somewhat of a nightmare.

So if I'm right, you would not actually have to modify any game code. Also, he does have a GitHub repo, under the Source Code link in the first post, which could be used by anyone as a base. Since DLLs are actually separate executables, there's nothing stopping one from running its task in a separate thread. I could be wrong, though; I'm not an expert programmer, merely a hobbyist. So if I'm truly wrong in what I'm saying, please correct me.

[jeffryfisher 2,176]

On 9/25/2017 at 2:57 PM, narukaze said:

would it be possible for someone to use a DLL to reimplement all of the game's simulators to use multithreading properly?

Multi-threading requires synchronization between processes that prep data and processes that use that data. Since we probably can't rewrite the whole game to synchronize its data consumption, multi-threading is almost certainly beyond us.

That said, I must agree that it would be super-cool to have an independent path-finding thread running full time on its own processor whenever a city's clock was running. Then we'd have both perfect path-finding and short turn-around times. I'm tired of seeing new houses labeled with no-road zots for a year or more before they notice the road they've always been facing.

[narukaze 1]

On 9/27/2017 at 3:08 PM, jeffryfisher said:

Multi-threading requires synchronization between processes that prep data and processes that use that data. Since we probably can't rewrite the whole game to synchronize its data consumption, multi-threading is almost certainly beyond us.

Fair point. But if all we're talking about is simulators, it might be possible. Based on what @simmaster07 has said, it appears that in order to override any part of a simulator, you would have to reimplement the whole thing. If that's already necessary, then it shouldn't be a problem to implement multithreading in the "preparing data" phase. Again, though, it's just a thought I, an amateur programmer, had, so take it with a grain of salt. (Also, that says nothing as to the feasibility of someone actually doing it.)

On 9/27/2017 at 3:08 PM, jeffryfisher said:

That said, I must agree that it would be super-cool to have an independent path-finding thread running full time on its own processor whenever a city's clock was running. Then we'd have both perfect path-finding and short turn-around times. I'm tired of seeing new houses labeled with no-road zots for a year or more before they notice the road they've always been facing.

Oh yeah, that is SO annoying. Seriously, just... I dunno, maybe check more than once per decade? It'd also be nice to have residential zoning rotate to face the road, so if you place a bus stop you don't accidentally cut the thing off. (I've done that a number of times. So aggravating...)