simmaster07

Worth noting that STLport is a fork of SGI STL that worked on making it portable to different platforms, and doing general maintenance when SGI stopped maintaining their library. MSVC was also unreliable at the time since the string class in VC6 had bad bugs like corrupting memory when multi-threading and being slow at string assignment (Paul Pedriana called this out in his GDC talk). My best guess is that SC4 Deluxe uses a pre-release version of STLport 4.6, but I also don't think the changes since 4.5.1 or 4.5.3 were major enough to really break function ID. I opened a PR to add FID databases for STLport 4.6 and Lua 5.0, built with Visual C++ 2003, and I get pretty solid coverage on SC4.exe v641 from that. Unfortunately a lot of the STL template types are both inlined and only generated at compile time with those types, which was necessary for performance but makes the FIDBs fairly limited. I'm also looking at getting FIDBs set up for libpng and zlib, but I suspect those were built with a different compiler than the game was using, because the function prologues are a lot less optimized than VC6 or VC7, and I can't find a combination of compiler flags that reproduces that. (For example, libpng functions always copy registers edi, esi and ebx to the stack and preserve them, even if those registers aren't used by that function.)

That's my bad, I was aiming for LGPL-2.1 to be more permissive (e.g. allowing GPLv2 projects to use it), but I didn't mean to restrict the use of LGPL-3.0. I just pushed a couple of commits to SCGL and Scion to relicense as LGPL-2.1-or-later (and add the notice headers to SCGL), and if I were going to relicense gzcom-dll, it would also be under LGPL-2.1-or-later.

re: gzcom-dll, I think you and I are the two main contributors on that project, and one contributor was making an automated fix to a solution file, so as long as @memo is okay with relicensing to LGPL to enforce source code sharing then we should be fine.

Fair point - from a licensing perspective it'd be fine to vendor those classes separately and link them with the MIT code, but for simplicity's sake I'll probably just relicense Scion to MIT when I get home tonight. I'll also have to check out the BaseMultiPackedFile implementation, that should be a big help for wrapping up the GZResourceD implementation. Thankfully Aspyr at least included retail symbols for SC4 and TS2 in their Mac ports, and a lot of the stuff that isn't platform specific is reused between those two games and SC3K, so the work is still doable, just takes additional time to correlate those symbols with the Windows ship builds. Might be worth setting up a Ghidra server at some point for that.

What is this? Scion is a (very work-in-progress) reference implementation of the Gonzo-Rizzo game engine used by Maxis in the late '90s and 2000s. You can find the code here: https://github.com/nsgomez/scion Who is this for? This project is mainly for programmers either developing DLL mods, or digging into the internals of 3D-era Maxis games. While this framework version is specifically based on SimCity 4 Deluxe, it might still be helpful for looking at different versions of the framework that were used in other Maxis projects, like: SimCity 3000 The Sims The Sims 2 Spore Since this is an SDK and not a standalone mod, it won't immediately benefit players. How is this different from gzcom-dll? gzcom-dll is a basic SDK for writing a DLL that the game engine will load, and for plugging into different parts of the game. It's mostly meant for making as many SC4 game interfaces accessible as possible. Scion is the game engine that would be loading DLLs and tying them all together. How does it work? (This section summarizes a lot of stuff that's been written about the game engine by myself, speeder, and Null 45, so if you're already familiar with DLL modding, this probably isn't news to you.) Gonzo-Rizzo is named for Muppets characters Gonzo the Great and Rizzo the Rat. As best I can tell, Gonzo ("GZ" in code), like his name suggests, is responsible for a lot of the main functionality: the core framework, graphics, sound, resource management, and so on. Rizzo ("RZ") mostly handles utilities like string parsing, filesystem access, thread synchronization, and reusable helpers. The game engine is modeled after Microsoft's Component Object Model (COM). At the core of the engine is the Framework, which provides: A GZCOM class registry, for registering component classes and creating new instances of registered classes; A registry of system services, which are singleton classes that can be used by components and do work on game ticks; Coordination for game ticks and framework hooks. Above the COM are directors, which act as framework hooks and can (and usually do) have their own registered classes, as well as child directors. The engine comes with prefabricated directors for doing graphics, sound, UI, and resource management, among other things. If you ever checked at the Apps folder in SimCity 3000 and saw a bunch of GZ*D files, that's what those are: GZSoundD.DLL, for example is the GZ Sound Director. Games build on top of the engine by doing the basic framework initialization and registering their own directors. For example, SimCity 4 has its own cSC4COMDirector to register its 3D renderer, simulator, lot developers, etc. SC3K has a bunch of SIM* DLLs that serve the same purpose, like SimTransit.dll, a GZCOM DLL director that implements transit networks. The game also takes responsibility for loading plugin DLLs, which is where expansion functionality comes in. (Aside: for any enterprising SC3K modders, those built-in DLLs might be a useful entry point for modding that game.) How far along is Scion? As of posting, the Scion implementation has a core GZCOM implementation that's mostly done with some scattered TODO items, and a mostly-done implementation of DBPF for resource management. There's still some more work to be done on the resource manager, and no work has started on the other core directors. The README on GitHub has a table showing the latest status.

SCGL is actually almost feature complete, there's just one rendering bug involving alpha transparency that I've been racking my brain on for... ever. You're still welcome to try it out if you're cool with cloud shadows sometimes showing up as black splotches, or cars not fading in and out right. @grebe48 I think the assert is because you built the DLL in Debug mode. It's trying to write some OpenGL debug messages to a log file at C:\temp\cGDriver.opengl.log, so if the C:\temp folder doesn't exist, it'll fail out like that. Creating that folder should make the asserts go away; or, you could change the path in cGDriver_Viewport.cpp under the LogGLMessage function; or you could switch to a Release build.

The degree of improvement seems like it'd depend mostly on the type of drive the Plugins folder is on, and the amount of space that's taken up by non-DBPF files. A 10GB plugins folder on an NVMe drive (~7500 MB/s read speed) that had DATPacker and Cleanitol run on it might not see much benefit. The same size plugins folder on a 7200rpm HDD (~100MB/s read speed) with a bunch of images and readme files would load a lot faster.

There's interest in it and some efforts have popped up occasionally, but there hasn't been a coordinated push for it, and the work that has been done is coming from a lot of different angles: @Null 45 has been contributing back to the gzcom-dll project and writing their own DLLs, which work from the "top down," modifying game memory and inspecting SC4's interfaces to make changes to the game logic. I, personally, have mostly pivoted to "bottom up" work by reverse engineering the game engine and framework components (Scion) before getting at the SC4 bits built on top of that. (And I don't exactly have the free time I did in college, so the time commitment's been pretty shaky for years now.) Others still have been taking a complete clean room approach (OpenSC4) and rewriting as much as they can in game engines like Godot. I've also gotten emails from maintainers on "Open Sims 2"-like projects -- even though it's a different game, SC4 and TS2 share a lot of internals, so there's mutual interest in whatever's happening between the two. There's still a lot of quality-of-life changes that can be made without making a whole OpenSC4 project, so it's probably not something worth prioritizing right now. But having an active DLL modding scene helps with building up the kind of knowledge and expertise that can push a project like that forward, and I predict it'll gain more traction when other avenues for improving the game are exhausted. It's also a project that would require a lot of manpower, since SimCity 4 is a bigger game with a more complicated engine than projects like TTD and RCT2. There would also have to be ground rules established on what a project like that should look like. For example, even if no assets like DAT files are distributed, it might not be okay to create a straight-up EXE replacement, since that could be used to circumvent copy protection, which is what got projects like Bnetd in trouble. It would also be a bad idea to try porting the game to mobile, as FreeSO tried with The Sims Online before getting a cease-and-desist. FreeSO's port would've competed with The Sims Mobile; doing that for SC4 would compete with SimCity BuildIt. EA backed off of FreeSO when that port was cancelled.

Drag the DLL to the Plugins folder, add -d:OpenGL to your game launch options. Nah, that's further up the game logic, can't do anything with that at the level we're working at.

Biffa's also got a 30 minute let's play:

I would have to dig deeper into how the game uses the rdtsc instruction to know what impact it has. It's not like the instruction just counts time on modern processors though. My observation is that it increments at a constant rate, but the rate it increments at is still roughly based on your processor speed. The problem is that modern CPUs change clock frequency all the time to optimize power usage, so using raw clock data to measure how fast a processor is also causes timing issues, since time could speed up or slow down essentially at random. Now it's fixed at your processor's base clock, which is... probably fine for SimCity 4? As far as I know there are rules in the game that use CPU speed to judge your hardware capabilities, but beyond that everything is based on timers and increments of time, not increments of clock cycles. As for multithreading, you wouldn't want a completely separate process since the cost of inter-process communication is unreasonably high. I guess in theory you could make a separate thread that handles rendering commands, and SCGL just dumps commands into a queue for that thread and hands control back to the game. But DirectX and OpenGL already queue commands to the GPU asynchronously, so the benefit of that is also questionable. You would have to tear out a lot of code further up the stack to get a reasonable multithreading model going.

Got some updates, but first a quick word on RTX Remix since people have been asking and I don't think I ever responded. I haven't tried it yet, and it requires DirectX 9 instead of OpenGL, so it's not likely I'll write another renderer to test it. I'm also really skeptical that it'll work for raising the fidelity of SimCity 4. Path-traced lighting depends on there being some kind of 3D geometry to simulate light against. The problem is that buildings are pre-lit and pre-rendered by the BAT, and essentially flattened into a picture to be copied and pasted onto the screen. I suspect the lighting simulation will suffer a lot as a result, and not produce the results people are looking for. --------------------------------------------------------------------- As for the main topic, let's start with the last showstopper bug. I found what was causing textures to get drawn in the wrong places, as I showed back in January: This is now mostly fixed: We also have working night mode now: I have a rough roadmap for making this a stable mod, which looks like this: Polish up the rendering engine. Retaining walls and terrain edges are still broken, which you can see above. Some transparency effects are also broken. Fix other bugs with how SC4 deals with SCGL. Screenshots and CustomResolution still crash the game. Make it fast. The focus so far has been entirely on correctness and not speed, so SCGL runs slower than DirectX. I have some ideas on how to close that gap. Also, thanks to @TorchedSammy, you can get unstable builds of SCGL from GitHub, which get compiled as soon as the source code is updated. I don't recommend doing that quite yet, but it is there if you're interested in the novelty of it. You will need to use one of the game's "known good" resolutions (800x600; 1024x768; 1280x1024; 1600x1200). You also need the latest Visual C++ 2022 redistributable. Finally, I rigged my own custom Graphics Rules.sgr file that skips parsing Video Cards.sgr and just assumes the fastest possible hardware. That probably has some value even if you're just using the DirectX hardware renderer, so check that if you're interested.

Long time no chat! Life got pretty busy after I posted my last update, so I didn't have much time to work on side projects like this. No, DirectX is a different way of writing graphics code. I'm also trying to bring this part of the game up to modern standards, which necessarily means dropping older GPUs with no OpenGL 3 support. (If it works, it works, but if it doesn't work, I have no plans to fix it.) If you need support for older GPUs, stick with the DirectX hardware renderer that the game came with. I don't think we'll be able to upscale the region view from this project, but also, there's not much fun in just packaging dgVoodoo. I'm doing this out of my own personal curiosity to drill into the game internals more than anything else. Anyway, I think I'm settled in again now, so let's revisit this, because this is the second-most important graphics optimization in the game: Consider your typical scene: How much effort does it take to render this? It depends on your zoom level and some graphics settings, but the high-level steps would (and I'm really oversimplifying here) look like this: Clear the screen. Draw the background (i.e. the gridlines on the edges of the map). Draw the terrain. Paint lots, roads, train tracks, and everything else "on the ground." Draw sims and vehicles. Draw the buildings and props. Draw the visual effects. Draw the UI. This isn't a real-time shooter or anything, so you can get by with a "cinematic" 24fps, but drawing all these lots is still going to be a struggle. Every "thing" on the screen -- bits of terrain, lots, buildings, props -- requires the CPU to send a separate message to the GPU saying "hello, please draw these triangles on the screen with these textures", and when you're zoomed out or working with a dense city, the overhead of sending these messages can get overwhelming. You know what would be nice? If the CPU could batch all these small messages together into one big message, so the GPU can just go through a whole list of draw calls at a time with way less overhead. Unfortunately, you are a graphics developer in the year 2002 and there won't be any useful way of doing that for another 10+ years, and you're stuck targeting a GeForce 256 from 1999. Even then, you need a software rendering mode for kids and parents who don't know how to read the system requirements on the box. You're already doing 2.5D rendering like in SimCity 3000, and that takes a huge load off since almost everything is now a pre-rendered box, which is the #1 most important optimization you could make. But it's still a lot of effort to render every lot in a loop. This is where the dirty rectangle system comes into play. Conceptually, dirty rectangles are a very fancy screenshot system. After the scene is rendered once, it copies the rendered result to an invisible region of graphics memory (a.k.a. a buffer region). Then whenever the game needs to render the same scene, it just copies what it computed from that off-screen buffer region. If the game needs to change something on screen -- for example, because a lot is being redeveloped -- then that part of the screen is marked with a rectangle to indicate to the graphics engine that it's dirty (hence the name): And then only what's in those rectangles is re-drawn, and the result gets merged with that saved image: The same thing happens with scrolling: This isn't as effective if the player is using the minimap to jump around parts of the city, if they're zooming and unzooming, or if they're scrolling very fast, since those actions cause almost everything to be redrawn anyway, which is why you might see severe frame drops when scrolling. But this helps a lot with the typical case of "just looking at a city and planning your next move," and that's all it really needs to do. As far as SCGL is concerned, this doesn't require a lot of implementation work. The graphics engine takes care of calculating, drawing, and merging dirty rectangles further up the chain of command. All our little driver has to tell it is that we have support for OpenGL buffer regions and expose the graphics API for it. This has sort of been supported in SCGL using an obsolete OpenGL extension that only NVIDIA supports now. Just to dip my toes back into this, I replaced that implementation with a better-supported one that should work on Intel and AMD too. Next step: the core rendering loop is still kinda busted, but I did narrow it down to when certain props are on screen, so I have some hope that it's debuggable.