Let's write our own SC4 hardware renderer

[Turnstyled 486]

On 4/16/2023 at 5:44 PM, Ulisse Wolf said:

Nvidia has just released RTX Remix that allows you to create mods that introduce new RTX technologies such as DLSS 3 and Ray Tracking.

https://github.com/NVIDIAGameWorks/rtx-remix

As it is, RTX Remix is for games that use DirectX 8, or DirectX 9 with fixed-function pipelines. That's what the documentation states. I'm stating this as someone who doesn't know how to integrate it into a mod.

I tried getting the wrapper included in the link that you provided to work on my 4K display. I tried using launch arguments to do that. It did not work. I did not try a supported resolution, such as 1080p.

The documentation for RTX Remix on GitHub states:

Quote

While support for D3D9 is included in Remix out of the box, games that use other graphics APIs can also be made to work by utilizing translation layers that target D3D9. You will have to acquire these separately. For example, D3D8 games can be supported through D3D8to9

As far as I can see, there is no Direct3D 7 to Direct3D 9 wrapper. The following screenshot is a list of wrappers on the PC Gaming Wiki. The link can be found here: https://www.pcgamingwiki.com/wiki/Glossary:Display_wrapper. The Emulation General wiki also maintains a list of wrappers that can be found here: https://emulation.gametechwiki.com/index.php/Wrappers

I apologize if I got any of the technical details wrong. I don't know if SimCity 4 uses DirectDraw 7 or Direct3D 7. I hate to potentially spread misinformation. Also, my short time on Mastodon made me aware of individuals who use screen readers. If it is inappropriate for me to submit a screenshot full of text, please let me know.

  Edited April 19, 2023 by Turnstyled  
Changed "material" to first instance of "documentation"

[Ulisse Wolf 4,412]

50 minutes ago, Turnstyled said:

As it is, RTX Remix is for games that use DirectX 8, or DirectX 9 with fixed-function pipelines. That's what the documentation states. I'm stating this as someone who doesn't know how to integrate it into a mod.

I tried getting the wrapper included in the link that you provided to work on my 4K display. I tried using launch arguments to do that. It did not work. I did not try a supported resolution, such as 1080p.

The documentation for RTX Remix on GitHub states:

As far as I can see, there is no Direct3D 7 to Direct3D 9 wrapper. The following screenshot is a list of wrappers on the PC Gaming Wiki. The link can be found here: https://www.pcgamingwiki.com/wiki/Glossary:Display_wrapper. The Emulation General wiki also maintains a list of wrappers that can be found here: https://emulation.gametechwiki.com/index.php/Wrappers

I apologize if I got any of the technical details wrong. I don't know if SimCity 4 uses DirectDraw 7 or Direct3D 7. I hate to potentially spread misinformation. Also, my short time on Mastodon made me aware of individuals who use screen readers. If it is inappropriate for me to submit a screenshot full of text, please let me know.

Technically you have to use dx8to9 on SC4 to make RTX Remix work. The problem currently that RTX Remix has too many problems (it just came out with the launch of RTX 4070). Also people are starting to test RTX Nvidia now and there are games with Directx 7.0 or older versions as in the case of Gothic (DirectX 7.0) or Star Wars Racer, Re-Volt etc ..

https://www.reddit.com/r/RTXRemix/comments/10b9hg7/want_to_know_if_a_game_works_heres_the_remix_game/

 

[Turnstyled 486]

@Ulisse Wolf Oh, wow! Thank you so much for the information and the link!

[matias93 7,535]

On 4/19/2023 at 1:28 PM, Turnstyled said:

Also, my short time on Mastodon made me aware of individuals who use screen readers. If it is inappropriate for me to submit a screenshot full of text, please let me know.

There's no rule here about alt text (probably because the game itself isn't accessible that way), but if you want to include it, the forum software allows for it. After inserting an image, you can double click on it on the text editor, and it will open this window:

On the second field, "Image Title", you can replace the file name with the adequate alt text, and any screen reading software will read from it. This window also allows to make hyperlinks out of images, to resize and to align them with the text.

[Turnstyled 486]

@matias93 Thanks so much for the walkthrough!

Edit: I saw on the RTX Remix community Discord server, which can be found at the link that @Ulisse Wolf provided earlier in this thread, that people have been trying to get SimCity 4 working with RTX Remix. One user was able to open the RTX Remix menu, though I am certain that that user is mistaken and that their game was running in software rendering mode rather than hardware rendering mode. There is not currently a DirectX 7 to DirectX 9 wrapper. The users of that server have not deemed SimCity 4 a priority due to the limited use cases of using RTX Remix with it.

Now, I hope I haven't completely derailed this thread from its purpose of discussing SCGL!

[Ulisse Wolf 4,412]

13 minutes ago, Turnstyled said:

@matias93 Thanks so much for the walkthrough!

Edit: I saw on the RTX Remix community Discord server, which can be found at the link that @Ulisse Wolf provided earlier in this thread, that people have been trying to get SimCity 4 working with RTX Remix. One user was able to open the RTX Remix menu, though I am certain that that user is mistaken and that their game was running in software rendering mode rather than hardware rendering mode. There is not currently a DirectX 7 to DirectX 9 wrapper. The users of that server have not deemed SimCity 4 a priority due to the limited use cases of using RTX Remix with it.

Now, I hope I haven't completely derailed this thread from its purpose of discussing SCGL!

I'm not surprised that SC4 and The Sims 2 have the same problems. Both games have the same game engine with some modifications. Which makes me think there's something in the game engine that is interfering with RTX Nvidia.

[rsc204 21,808]

Could it be that advanced 3D tools just aren’t all that suitable for a 2.5d isometric game like SC4?

I mean in the best case scenario what is it you hope to gain from using such a tool? SC4 still looks sweeter and crisper than any 3D city builder I’ve come across. That’s the advantage of 2D, it’s very efficient to use pre-rendered assets. Whilst trying to render them all in real-time still takes down the most powerful gear most of us can’t afford to buy anyhow. 

[Pakistani890 122]

I wanted to say this that umm it's gonna be based on Opengl 3 right? So will it use Direct X also because older systems with older GPU's don't have opengl 3 support or will it use the latest Direct X? (I am sorry if this sounds dumb, i am new to this stuff..)

[anonblahacco 1]

The 4k experience with a wrapper (I use dgvoodoo) is already pretty good. If this new utility was simply able to render out and upscale the lower resolution (1080p) region view to 4k so we didn't get the empty blue bands, you might not have to rewrite the whole thing but just package up the wrappers and fixes into one download and menu for modern users.

[simmaster07 1,220]

Long time no chat! Life got pretty busy after I posted my last update, so I didn't have much time to work on side projects like this.

On 4/27/2023 at 4:54 AM, Pakistani890 said:

I wanted to say this that umm it's gonna be based on Opengl 3 right? So will it use Direct X also because older systems with older GPU's don't have opengl 3 support or will it use the latest Direct X? (I am sorry if this sounds dumb, i am new to this stuff..)

No, DirectX is a different way of writing graphics code. I'm also trying to bring this part of the game up to modern standards, which necessarily means dropping older GPUs with no OpenGL 3 support. (If it works, it works, but if it doesn't work, I have no plans to fix it.) If you need support for older GPUs, stick with the DirectX hardware renderer that the game came with.

On 5/13/2023 at 9:28 PM, anonblahacco said:

The 4k experience with a wrapper (I use dgvoodoo) is already pretty good. If this new utility was simply able to render out and upscale the lower resolution (1080p) region view to 4k so we didn't get the empty blue bands, you might not have to rewrite the whole thing but just package up the wrappers and fixes into one download and menu for modern users.

I don't think we'll be able to upscale the region view from this project, but also, there's not much fun in just packaging dgVoodoo. I'm doing this out of my own personal curiosity to drill into the game internals more than anything else.

Anyway, I think I'm settled in again now, so let's revisit this, because this is the second-most important graphics optimization in the game:

On 1/17/2023 at 8:15 AM, simmaster07 said:

There's also an extension for "buffer regions" to boost performance which, again, is not standard in OpenGL. The game does try to provide them as an extension, but it seems this does not work for Intel graphics.

Consider your typical scene:

How much effort does it take to render this? It depends on your zoom level and some graphics settings, but the high-level steps would (and I'm really oversimplifying here) look like this:

1. Clear the screen.
2. Draw the background (i.e. the gridlines on the edges of the map).
3. Draw the terrain.
4. Paint lots, roads, train tracks, and everything else "on the ground."
5. Draw sims and vehicles.
6. Draw the buildings and props.
7. Draw the visual effects.
8. Draw the UI.

This isn't a real-time shooter or anything, so you can get by with a "cinematic" 24fps, but drawing all these lots is still going to be a struggle. Every "thing" on the screen -- bits of terrain, lots, buildings, props -- requires the CPU to send a separate message to the GPU saying "hello, please draw these triangles on the screen with these textures", and when you're zoomed out or working with a dense city, the overhead of sending these messages can get overwhelming.

You know what would be nice? If the CPU could batch all these small messages together into one big message, so the GPU can just go through a whole list of draw calls at a time with way less overhead. Unfortunately, you are a graphics developer in the year 2002 and there won't be any useful way of doing that for another 10+ years, and you're stuck targeting a GeForce 256 from 1999. Even then, you need a software rendering mode for kids and parents who don't know how to read the system requirements on the box.

You're already doing 2.5D rendering like in SimCity 3000, and that takes a huge load off since almost everything is now a pre-rendered box, which is the #1 most important optimization you could make. But it's still a lot of effort to render every lot in a loop. This is where the dirty rectangle system comes into play.

Conceptually, dirty rectangles are a very fancy screenshot system. After the scene is rendered once, it copies the rendered result to an invisible region of graphics memory (a.k.a. a buffer region). Then whenever the game needs to render the same scene, it just copies what it computed from that off-screen buffer region. If the game needs to change something on screen -- for example, because a lot is being redeveloped -- then that part of the screen is marked with a rectangle to indicate to the graphics engine that it's dirty (hence the name):

And then only what's in those rectangles is re-drawn, and the result gets merged with that saved image:

The same thing happens with scrolling:

This isn't as effective if the player is using the minimap to jump around parts of the city, if they're zooming and unzooming, or if they're scrolling very fast, since those actions cause almost everything to be redrawn anyway, which is why you might see severe frame drops when scrolling. But this helps a lot with the typical case of "just looking at a city and planning your next move," and that's all it really needs to do.

As far as SCGL is concerned, this doesn't require a lot of implementation work. The graphics engine takes care of calculating, drawing, and merging dirty rectangles further up the chain of command. All our little driver has to tell it is that we have support for OpenGL buffer regions and expose the graphics API for it. This has sort of been supported in SCGL using an obsolete OpenGL extension that only NVIDIA supports now. Just to dip my toes back into this, I replaced that implementation with a better-supported one that should work on Intel and AMD too.

Next step: the core rendering loop is still kinda busted, but I did narrow it down to when certain props are on screen, so I have some hope that it's debuggable.

[Fantozzi 4,067]

You know, this is the reason why those april fools downloads always work on me. Shure on first thought I say - ha, this isn't real, this can't be working, they try to fool me. And even the first picture in the starting post - a simple minded guy like me what will he think? "Look, some funny piece of art, another joke." But then, what I thought to be a prank @simmaster07 turns it into something real. So - how should I know these are aprils fools downloads in the exchange if he on the other hand does such things? 

Well, next april make me april's fool again. *Sigh* Thanks to Simmaster07's wizardry I'll believe everything you tell me. 

[TheMurderousCricket 5,055]

On 28.08.2023 at 10:46 AM, simmaster07 said:

Conceptually, dirty rectangles are a very fancy screenshot system. After the scene is rendered once, it copies the rendered result to an invisible region of graphics memory (a.k.a. a buffer region). Then whenever the game needs to render the same scene, it just copies what it computed from that off-screen buffer region. If the game needs to change something on screen -- for example, because a lot is being redeveloped -- then that part of the screen is marked with a rectangle to indicate to the graphics engine that it's dirty (hence the name)

Pardon my ignorance, but wouldn't it mean that memory will be red-hot from saving and remembering where all of the dirty rectangles generated in the game session are?

We all know that SC4 as it stands now, takes immediate dislike to whatever is interrupting with a city's save process. This is why I am afraid that storing additional bits of information on a given city and then hitting save button will result in new city and save corruptions.

[nos.17 5,120]

7 hours ago, TheMurderousCricket said:

Pardon my ignorance, but wouldn't it mean that memory will be red-hot from saving and remembering where all of the dirty rectangles generated in the game session are?

We all know that SC4 as it stands now, takes immediate dislike to whatever is interrupting with a city's save process. This is why I am afraid that storing additional bits of information on a given city and then hitting save button will result in new city and save corruptions.

Think about the alternative - constantly repainting every single pixel on the screen. A simplistic example might be if you are writing a paper and want to change a word, instead of whiting-out and writing over the replaced word (the targeted dirty rectangle) you write the whole chapter over again with the changed word. Copying and recalling the bits to memory is literally lightning fast - the slow part is in the calculating and rendering what should be there in the first place, so if you can minimize the rendering, you are better off. Tangentially, this is also a very common way to compress and encode videos - the sections that are static are essentially referenced from previous frames and only the changed pixels are stored. Thus you can get much smaller file size with minimal visual differences. (this is probably not technically accurate but at a high enough level, it should get the idea across)

About the save process, this is actually a different issue that is not really related. In the save process, the game is basically dumping everything into a database (the savegame file). The more things there are in your city, the more time it takes to save a record of what is there. Maxis devs imposed a hard-coded timeout to the save which would have been longer than any vanilla game would be expected to take to save, probably as some sort of failsafe. However, lots of custom content can cause the save process to take longer than this timeout limit, and the save process being interrupted mid way → potential corruption. In short, the frame buffers are basically being constantly stored and replaced and overwritten at many times per second in memory, which is totally independent and different from the game save process saving to disk.

Also, SC4 very already uses these techniques to render the screen, which is why you have frame drops on camera rotations and large zooms and not with slow to moderate camera panning, as simmaster mentioned.

[simmaster07 1,220]

Got some updates, but first a quick word on RTX Remix since people have been asking and I don't think I ever responded. I haven't tried it yet, and it requires DirectX 9 instead of OpenGL, so it's not likely I'll write another renderer to test it. I'm also really skeptical that it'll work for raising the fidelity of SimCity 4. Path-traced lighting depends on there being some kind of 3D geometry to simulate light against. The problem is that buildings are pre-lit and pre-rendered by the BAT, and essentially flattened into a picture to be copied and pasted onto the screen. I suspect the lighting simulation will suffer a lot as a result, and not produce the results people are looking for.

---------------------------------------------------------------------

As for the main topic, let's start with the last showstopper bug. I found what was causing textures to get drawn in the wrong places, as I showed back in January:

This is now mostly fixed:

We also have working night mode now:

I have a rough roadmap for making this a stable mod, which looks like this:

1. Polish up the rendering engine.
   1. Retaining walls and terrain edges are still broken, which you can see above.
   2. Some transparency effects are also broken.
2. Fix other bugs with how SC4 deals with SCGL.
   1. Screenshots and CustomResolution still crash the game.
3. Make it fast.
   1. The focus so far has been entirely on correctness and not speed, so SCGL runs slower than DirectX. I have some ideas on how to close that gap.

Also, thanks to @TorchedSammy, you can get unstable builds of SCGL from GitHub, which get compiled as soon as the source code is updated. I don't recommend doing that quite yet, but it is there if you're interested in the novelty of it. You will need to use one of the game's "known good" resolutions (800x600; 1024x768; 1280x1024; 1600x1200). You also need the latest Visual C++ 2022 redistributable.

Finally, I rigged my own custom Graphics Rules.sgr file that skips parsing Video Cards.sgr and just assumes the fastest possible hardware. That probably has some value even if you're just using the DirectX hardware renderer, so check that if you're interested.

[ebones5 36]

Let me preface by saying I have no idea if this is possible, but would it be feasible for SCGL to make use of multi-core setups by way of a companion program or other means? Since the problem with SC4 in multi-core is when simulator instructions get "out of sync", can the graphics engine have, say, SCGLhelper.exe running in the background that handles frame queueing and the like? As I said, it may not be possible (and this is an idea coming from someone with basic programming knowledge but nothing too extensive, e.g., a layman for all intents and purposes), and I'm not too sure how much of a benefit (if any) it would yield, especially if the overhead consumed by the CPU components of rendering is small.

[Propfam 2,288]

15 hours ago, ebones5 said:

Let me preface by saying I have no idea if this is possible, but would it be feasible for SCGL to make use of multi-core setups by way of a companion program or other means? Since the problem with SC4 in multi-core is when simulator instructions get "out of sync", can the graphics engine have, say, SCGLhelper.exe running in the background that handles frame queueing and the like? As I said, it may not be possible (and this is an idea coming from someone with basic programming knowledge but nothing too extensive, e.g., a layman for all intents and purposes), and I'm not too sure how much of a benefit (if any) it would yield, especially if the overhead consumed by the CPU components of rendering is small.

I don't know about that but I remember @OrSpeeder said it's about RDTSC. So according to him, the best way to enjoy SimCity 4 is to use 2003ish processors.

Correct me.

[ebones5 36]

13 hours ago, elfrjz said:

I don't know about that but I remember @OrSpeeder said it's about RDTSC. So according to him, the best way to enjoy SimCity 4 is to use 2003ish processors.

Correct me.

Actually this provides a better explanation of (or at least part of) the multicore problem (though if it was solely RDTSC, would hyper-threading not be safe? I've tried experiments with hyper-threading to no avail, could be Intel's out-of-order execution). My proposal is less a solution and more of a workaround to the problem at hand, while we may not be able to properly "enable" multicore (in a stable manner) without source (or without massive backwards engineering), it may be possible to off-load some "independent" tasks onto another core by way of a helper and some .dll magic, since it's unlikely that there are other things dependent on the renderer, and since there is a CPU component to the rendering, if we could move those instructions to another core (even if only one) it would save some cycles and make that one core work just a little bit better.

Though again, this is all conjecture by a layman, this may not be possible with the tools at hand, or the gains could be negligible and not worth the effort. Though I'm fairly certain only our lord and savior @simmaster07 could answer for sure :p.

[simmaster07 1,220]

I would have to dig deeper into how the game uses the rdtsc instruction to know what impact it has. It's not like the instruction just counts time on modern processors though. My observation is that it increments at a constant rate, but the rate it increments at is still roughly based on your processor speed.

The problem is that modern CPUs change clock frequency all the time to optimize power usage, so using raw clock data to measure how fast a processor is also causes timing issues, since time could speed up or slow down essentially at random. Now it's fixed at your processor's base clock, which is... probably fine for SimCity 4? As far as I know there are rules in the game that use CPU speed to judge your hardware capabilities, but beyond that everything is based on timers and increments of time, not increments of clock cycles.

As for multithreading, you wouldn't want a completely separate process since the cost of inter-process communication is unreasonably high. I guess in theory you could make a separate thread that handles rendering commands, and SCGL just dumps commands into a queue for that thread and hands control back to the game. But DirectX and OpenGL already queue commands to the GPU asynchronously, so the benefit of that is also questionable. You would have to tear out a lot of code further up the stack to get a reasonable multithreading model going.

[Ulisse Wolf 4,412]

On 7/9/2023 at 5:23 AM, simmaster07 said:

I would have to dig deeper into how the game uses the rdtsc instruction to know what impact it has. It's not like the instruction just counts time on modern processors though. My observation is that it increments at a constant rate, but the rate it increments at is still roughly based on your processor speed.

The problem is that modern CPUs change clock frequency all the time to optimize power usage, so using raw clock data to measure how fast a processor is also causes timing issues, since time could speed up or slow down essentially at random. Now it's fixed at your processor's base clock, which is... probably fine for SimCity 4? As far as I know there are rules in the game that use CPU speed to judge your hardware capabilities, but beyond that everything is based on timers and increments of time, not increments of clock cycles.

As for multithreading, you wouldn't want a completely separate process since the cost of inter-process communication is unreasonably high. I guess in theory you could make a separate thread that handles rendering commands, and SCGL just dumps commands into a queue for that thread and hands control back to the game. But DirectX and OpenGL already queue commands to the GPU asynchronously, so the benefit of that is also questionable. You would have to tear out a lot of code further up the stack to get a reasonable multithreading model going.

I wonder if with this OpenGL edit you found something related to UI and save system

[Propfam 2,288]

An elephant in the room. How do we use the mod? Is it as simple as dragging over to Plugins? Or should we add -d argument? What's after -d? OpenGL? scgl?

[TheMurderousCricket 5,055]

5 hours ago, elfrjz said:

An elephant in the room. How do we use the mod? Is it as simple as dragging over to Plugins? Or should we add -d argument? What's after -d? OpenGL? scgl?

I think that once the final and stable version lands we won't have to add any arguments anymore.  At least in the way I understand it.

[simmaster07 1,220]

On 9/10/2023 at 8:30 PM, elfrjz said:

An elephant in the room. How do we use the mod? Is it as simple as dragging over to Plugins? Or should we add -d argument? What's after -d? OpenGL? scgl?

Drag the DLL to the Plugins folder, add -d:OpenGL to your game launch options.

On 9/8/2023 at 10:27 AM, Ulisse Wolf said:

I wonder if with this OpenGL edit you found something related to UI and save system

Nah, that's further up the game logic, can't do anything with that at the level we're working at.

[Propfam 2,288]

5 hours ago, simmaster07 said:

Drag the DLL to the Plugins folder, add -d:OpenGL to your game launch options.

Thanks for the info!  Also I have a problem regarding your another mod in another topic. May I ask for your help there?

Also an idea for the deactivated water and weather systems, anyone?

 

[grebe48 27]

Hello! I complied the package from GitHub in Visual Studio 2022. 

Then, i placed the compiled DLL into the /Gog Games/Simcity4/Plugins directory.

I then opened the GraphicsOptions.ini, and changed the renderer to SCGL.

Then when I open SC4 (running as administrator), I get the first message. click ok, then I get the 2nd message. No matter what option I choose, the game then crashes.

Is there a step by step tutorial on how to install SCGL? Because my programming skill is useless in 2025. Thanks!

[TogaMasterJohn 705]

16 hours ago, grebe48 said:

Hello! I complied the package from GitHub in Visual Studio 2022. 

Then, i placed the compiled DLL into the /Gog Games/Simcity4/Plugins directory.

I then opened the GraphicsOptions.ini, and changed the renderer to SCGL.

Then when I open SC4 (running as administrator), I get the first message. click ok, then I get the 2nd message. No matter what option I choose, the game then crashes.

Is there a step by step tutorial on how to install SCGL? Because my programming skill is useless in 2025. Thanks!

You may want to give this DLL plugin a try if you want to test SCGL out:

I'm assuming that you're wanting to test SCGL out (which is cool!), because if you downloaded this for serious gameplay, you should probably use the DirectX renderer instead (and use dgVoodoo2 or another wrapper if deemed necessary).  Last I knew, SCGL is still a work in progress and is not entirely complete yet.

[simmaster07 1,220]

SCGL is actually almost feature complete, there's just one rendering bug involving alpha transparency that I've been racking my brain on for... ever. You're still welcome to try it out if you're cool with cloud shadows sometimes showing up as black splotches, or cars not fading in and out right.

@grebe48 I think the assert is because you built the DLL in Debug mode. It's trying to write some OpenGL debug messages to a log file at C:\temp\cGDriver.opengl.log, so if the C:\temp folder doesn't exist, it'll fail out like that. Creating that folder should make the asserts go away; or, you could change the path in cGDriver_Viewport.cpp under the LogGLMessage function; or you could switch to a Release build.

[aspct 7]

Hi simmaster, I've been following this project for a while now and after learning that it's almost feature complete I decided to try it out. So far it has almost doubled my fps and eliminated all stutters, especially when scrolling compared to using dgvoodo2 wrapper so I can't thank you enough! 

Only problem I've encountered so far is that diagonal track pieces never switch to night textures (or the effect is never applied, don't know how sc4 handles night time lighting). Picture in spoiler. (I color matched the tracks with their daytime counterpart and its a 1:1 match, meaning the lighter looking pieces look the same as if it's daytime)

Both NAM and Simfox Day and Nite are installed as per attached picture.

Spoiler