thiekus

Updated to v0.0.2, which now able to extract sprites from *.sjs file generated by SimCity 3000 Unlimited Rendering Kit and fix quirk of SJS file itself which store sprites as resource type same as image buffer. Now I'm wondering if I could recreate my own rendering kit program (after I implements sprites compression and bitmap importer functions of course), because current SC3URK didn't play nicely with modern Windows.

Yeah, that's also my concern regarding minilzo licensing. That's why I give minilzo as license exception on repo readme, though most of my other parts keep as MIT license. It's possible to build without GPL'd minilzo after stripped SC3K decompression part (obviously you'll missing ability to decompress SC3K sprites, only SC3U), though I didn't implement compiler switch to do so for this time. Since I did expose all source code needed for build under open source license, I think I didn't violate GPL. For future ongoing "IXF Explorer" probably I'll use GPLv3 or later for GUI parts (and keep core classes/functions as MIT) because I prefer more restrictive license for bigger project. Edit: I give some clarification on readme and some of dependent unit code which states built program is licensed as GPL v2 or later as well when linking minilzo code to comply this situation, while keep most code still under MIT.

So, In my spare time, I decided to decode SimCity 3000 *.IXF file (internally known as Indexed File) by reverse engineer SC3U Windows (which has PDB debug info embedded on retail SC3U.ICD, though only main executable and not complete) and SC3U Linux libSimSpr.so (not debug info but most functions and class methods were exposed). There's attempts to extracting SC3K sprites here and the same guy did share clues code which responsible to extracting SC3K from late Paul Pedriana (RIP). But so far, no one in has given working sprite extractor. Sprite format isn't that complicated. It's actually small header that contains sprite size and color type (usually 16-bit RGB 565 or RGB 555), followed by compressed data stream. Compression used is different between SC3K and later SC3U re-release, which in SC3K sprites using MiniLZO in RGB 555 color format, and later SC3U using older iteration of RefPack (or known as Qfs in SimCity 4 and The Sims 2 community) in RGB 565 color format (some were still at RGB 555). After data stream were decompressed, there's another header contains size and transparent color used (or just straght to indexed data for sprite masks) followed by array of strides infomations. Each strides has variable length data that skips transparent color of stride beginnings. There's still unknown part which stride width often has 0x8000 flag which I ignore this time and sprites still rendered just fine. In addition it's also able to extracting buffer images which used mostly for UI elements. My code is written on Free Pascal flavours of Object Pascal used in Lazarus. This because this project were subset of my bigger GUI application using Lazarus which I can't finish it for now due to current circumstances (hopefully at later time). There's including RefPack decompression routine which I ported Null's DBPFSharp from C#. I'm also port MiniLZO decompression function from C. This sprite extractor source code in this Github repo or you can also use prebuilt exe from release page here. Now you can start see you favorites building sprites directly or just data mine of possibly unused sprites.

I have idea to move mangled symbols from Aspyr SC4 Mac to SC4 Windows (probably) more faster. As turns out, Ghidra understands compiler Map (*.map files) besides to other debug info formats like PDB. AFAIK, map files usually generates from compilers like older versions of Visual C++ and Borland compilers (C++ and Delphi). Unlike PDB, Map files is plain text, so it's easy to create and edit. Ghidra usually didn't allow to demangle GCC mangled names if you input Visual C++ exe, but there's trick: Prepare attached crafted mapfile, save as .map extension. It's simple format so no need more explanations. Import new "SimCity 4.exe" as usual. Did auto analysis as usual. After analysis finished, select menu File -> Add to Program. Select the crafted map info. If that valid map file, format will be "Program Mapfile (MAP)", otherwise mapfile format is wrong. If succeed, defined function will be renamed as GCC mangled name as defined in mapfile. To able use GCC demangler, you need temporarily change compiler spec by select menu Edit -> Options for 'SimCity 4.exe'. Then select "Program Information". From there, change Compiler from "visualstudio:unknown" to "gcc:unknown". Click Ok to proceed. Save and close active CodeBrowser, then open it again. In analysis options, you now have "Demangler GNU" that not available before. Do demangler analysis only by menu Analysis -> One Shot -> Demangler GNU. Now you don't waste time to manually edit many functions, demangler will did the job After done, change back compiler spec back to original "visualstudio:unknown". Save and close CodeBrowser, then open again. Every you have new symbol entries, you just load map file again. You only need change compiler spec when you need demangle them all at once. There's small caveat, every you reload map file after demangle process, demangled static data/variable symbols will be renamed to mangled name back (demangled functions isn't affected). I don't see anyone did this tricks before (or did exists but I don't know yet), and it would be more ideal if mangled GCC names can be converted to MSVC mangled name. I didn't see another side effects but take this tricks at your risk. Here my sample mapfile for SC4 build 641 Address Publics by Value ; ; Formats are: ; <section:RVA> <symbol name> <absolute address> ; ; Ghidra only cares 2nd and 3rd parameters (absolute address), it don't care about RVA in 1st param. ; Name and origin address comes from SC4 Rush Hour Mac universal binary x86 beta. ; Comments are using semicolon prefix, that entries can't be have blank lines (Ghira will skipped it). ; ; [0x00008302] SC4AppStart(cRZCmdLine const&, bool) 0000:00000000 __Z11SC4AppStartRK10cRZCmdLineb 0044c170 ; [0x0002456c] SC4DisplayResourceError 0000:00000000 __Z23SC4DisplayResourceErrorm 005f8d40 ; ; [0x00ab71e8] cGZFramework::mpFrameWork 0000:00000000 __ZN12cGZFramework11mpFrameWorkE 00b540ac ; [0x00ab71e0] cGZFramework::mpApp 0000:00000000 __ZN12cGZFramework5mpAppE 00b540b4 ; [0x00ab71ec] cGZFramework::mnReturnCode 0000:00000000 __ZN12cGZFramework12mnReturnCodeE 00b540a8 ; ; [0x0001cc0c] cGZFramework::Main(cRZCmdLine const&, bool) 0000:00000000 __ZN12cGZFramework4MainERK10cRZCmdLineb 0087b0b9 ; [0x0001c81c] cGZFramework::sInit(cRZCmdLine const&, bool) 0000:00000000 __ZN12cGZFramework5sInitERK10cRZCmdLineb 0087af13 ; [0x0001a67c] cGZFramework::sRun() 0000:00000000 __ZN12cGZFramework4sRunEv 0087957a ; [0x0001cbf8] cGZFramework::sShutdown() 0000:00000000 __ZN12cGZFramework9sShutdownEv 0087ab07 ; [0x0001a448] cGZFramework::sSetApplication(cIGZApp*) 0000:00000000 __ZN12cGZFramework15sSetApplicationEP7cIGZApp 008793dd ; [0x0001a41e] cGZFramework::sSetFrameWork(cGZFramework*) 0000:00000000 __ZN12cGZFramework13sSetFrameWorkEPS_ 008793bd ; [0x0001c7aa] cGZFramework::HookPreFrameWorkInit() 0000:00000000 __ZN12cGZFramework20HookPreFrameWorkInitEv 0087a286 ; [0x0001c738] cGZFramework::HookPreAppInit() 0000:00000000 __ZN12cGZFramework14HookPreAppInitEv 0087a2d6 ; [0x0001c6c6] cGZFramework::HookPostAppInit() 0000:00000000 __ZN12cGZFramework15HookPostAppInitEv 0087a326 ; ; [0x00007ba4] cSC4App::cSC4App(char const*) 0000:00000000 __ZN7cSC4AppC1EPKc 0044acd0 ; ; [0x0001120c] cRZAppCmdLine::FixCommandLine(cIGZCmdLine&, cIGZString const*) 0000:00000000 __ZN13cRZAppCmdLine14FixCommandLineER11cIGZCmdLinePK10cIGZString 0091479d

I was able to past from license check and compile libpng and zlib using that same Intel C++ Compiler. You really need VC6 or VC 2002 to really use this version of ICC, it wouldn't integrate to any newer VS. I've did PR for this FIDB. I'd tried to build IJG libjpeg 6b database and only detects very few jpeg functions. Turns out it's use Intel Jpeg Library (IJL) statically which nowhere to found (DLL versions is more common, it used by The Sims 1). So I'm not put them on PR, but attached is built jpeg 6b static lib for reference. Edit: seems SC4 actually load IJL dynamically at 0x00a0ee50. libjpeg.lib

Makes sense since from the SGI STL FAQ, it's header only library implementation, and while STLPort fork is evolved to their own, may still have traces from old SGI STL. Before I caught SC3U debug info, I've tried to create stub program that purposely calls to any STL functions I can call compiles with VC6, then put resulted OBJ to Ghidra and generate FID from there. It got some functions detected but I stopped doing that for SC3U immediately after I found exe debug info. Attached code is my crude attempt to generate sc3u STL signatures. I agree that VC6 is notoriously joke that even defying the C++ standards that should be correct. I looked at Internet Archive and found interesting "Intel® Software Development Products Evaluation CD" dated from 2002. It contains 30-days of various Intel development tool products including Intel C++ Compiler 6.0. I didn't try to install yet but after I extracted icl.exe, it use the same version & revision that used in SC4 compiler option strings. Look promising. Edit: it requires valid FlexLM license after I install and need explicitly request trial to Intel, the request link from the CD isn't work anymore. Some part of Gimex source code likely published as well on CnC Generals source code. stlimpl.cpp

VC Toolkit is barebone cli compiler with minimal CRT library, so as I said before, you need Platform SDK for complete toolchain like nmake, etc. I was able to reproduce STLPort 4.6 that detected in SC4 build 641 using VC Toolkit v1.01 and Windows Platform SDK 2003 SP1(from same link I mentioned before) since compiling STLPort only need nmake. First, you need open "Windows Server 2003 32-bit Build Environment" from PSDK 2003 start menu (XP 32bit may works too although not tested yet). Then apply VC Toolkit 2003 env by copying absolute vcvars32.bat to command window. For 32 bit Windows "C:\Program Files\Microsoft Visual C++ Toolkit 2003\vcvars32.bat" For 64 bit Windows (not tested yet) C:\Program Files (x86)\Microsoft Visual C++ Toolkit 2003\vcvars32.bat And test by cl /?, make sure you execute the right compiler. If not, probably you need check vcvars32.bat environments in VC Toolkit directory is correct. Build static build stlport with nmake -f vc7.mak as usual. It works on STLPort 4.6 but didn't in older 4.5.x. Forgot to mention, you should not check at analysis options when analyze Function ID when building own FID. Also sometime Ghidra has annoying bug that makes Function ID is missing from analysis options, just close and open new instance of Ghidra should solve this problem. I need check further if this is correct fidb and STLPort version.

You don't need whole Visual Studio 2003 IDE to install, which is pain to install and use even in Windows 7 in my experience, as turn out there's weird issue with DWM enabled. But the good news is, Microsoft at one point give VC 2003 compiler tools only as freeware as Visual C++ Toolkit 2003 and some basic libraries (AFAIK, you need another platform SDK for Windows 2003 to use WinAPI headers & libraries). There's 2 version, v1.00 (build 3052) and v1.01 (build 3077 same as VC 2003 RTM) which according to Microsoft Q&A here is doesn't matter (but v1.01 is preferable to ensure match the VC 2003 RTM). There's someone who posting both version (v1.01 is signed with newer June 2004), take for risk consideration as Microsoft have remove original download link for long time ago. Here's comparison between my VC 2003 RTM and VC 2003 Toolkit 2003 v1.01 on my WinXP VM.

Hi, nice to see you working again for this project! Although I have done some fixups to SC3U exe based what I found in embedded debug info (I surprised why they did it), and did some unfinished boilerplate code of interfaces targets for VC6 SP3 (for build 949 Unlimited, some later UK edition with slightly newer build 966 is using VC6 SP4). At that time, I decided to create from scratch rather use scion as base because what I found is SC3U & SC4 have some differences how base app framework were created, and so on. Unfortunately I didn't have much time for now as still busy for ongoing thesis. However I would like to contribute to scion instead if I have spare time later. Some tidbits: SC3U debug info were only on on main exe, but it still contains valuable since it provides more detailed how Maxis create functions like arguments name, enums, structs and interfaces from external modules. Most Internet functions that isn't essential still lives there, and still ping to EA's SimCity.com when idle (didn't figure yet how disable this elegantly, there's also lot undocumented ini options for this internet feature). SC3K and SC3U apperently didn't use STLPort or MSVC standard STL, but instead use SGI STL v3.13 (they may not update this more often because it was integrated on rizzo framework base on debug info). Loki port build is use SGI STL too instead of GCC stdc++ and this why Linux build is bigger because of how GCC aggresively inlined most of STL functions. The Sims 1 actually have leaked custom build with Edith development tool, it has release & unoptimzed debug build with MSVC assertions & fraction of RTTI. Sadly both didn't include PDB. While SC4 Rush Hour EP built with MSVC 2003, older SC4 base game up to build 272 apparently built on VC6 SP4 or SP5 with older Lua 4.x. This may affects decompilation as VC 2003 allegedly have more optimized code generation than VC6. To reduce guessing function names, you can use Ghidra Function ID feature. That's work by: compiling known library source code with exact version (e.g Lua 5.0) to the exact same compiler that exe game used to decompile (e.g SC4 Rush Hour is using MSVC 2003 RTM without SP) as static library, then create Ghidra new project, import lib to extracts OBJs. Open all OBJs by select .lib file, then select batch, make sure you set language to `x86:LE:32:default:windows` (sometimes Ghidra misrepresenting as gcc), make sure check both "strip leading path" & "strip container path". Open all objs by selecting all obj, right click -> open with -> code browser. Don't analyze now, instead select No, then go to menu Analysis -> Analyze all open. Then use build FID as usual I not mentioned here but there's much basic tutorial to build Ghidra FID. You can do this for MSVC 2003 libraries like libcmt.lib an others as SC4 is built CRT staticaly & Ghidra built in MSVC FIDB is quite bad. Attached file for example to detect Lua 5.0 function in latest SC4. Not perfect (some not recognized) but still time saver.

In spare time, I'm reversing SC3U based on retail CD release build that contains embedded debug information on sc3u.icd which I have told before on Simtropolis Discord chat. Not much I can do for now, but by using debugger I managed to force invoke sc3u integrated internal web browser which unused in final sc3u. It's slow and barely render HTML, no window resizing, loading over HTTP is often not work, HTTPS is out of question. No wonder why this feature along some internet feature like IRC were cut on sc3u and left behind in game executable.

If you save on "C:\Program Files\*" and the game isn't run as administrator, chances is saved on Windows Virtual Store instead at "C:\Users\<your username>\AppData\Local\VirtualStore" because Windows (since Vista and up) won't permit ordinary user to write on Program Files by default without raising privilege to Administrator.

Hi, I'm recently playing SC3U Linux Loki port. Yes, it's easier to install Steam version with Proton or even set up Wine manually. But I managed to try old Linux version on my 64-bit Ubuntu 24.04, and made some progress. With help sc3u-nptl, I was able run Linux version without loki-compat library (albeit still require really old stdc++ from old Ubuntu/Debian). Then I tried to resolve missing sound issue without installing OSSPD by creating OpenAL compat for old Loki games because there's ABI breakage between pre-1.0 OpenAL with later OpenAL (I'll explain in my repo why it can't simply use current OpenAL API). At least it has sound with some annoying problems. On modern distro while playable, there's issue crashing when you exiting (not occurred in old Ubuntu 6.06) but I cant play for hours just fine. Also it relied to old Creative OpenAL that still have Loki specifics OpenAL extension which SC3U Linux needs, and it has their own problems (for example crashing when you switching or minimize window when you use SDL as OpenAL backend. For next time, I may hacking around Linux version executable along with recent Windows executable because while it's nice to have Linux executable with mostly exposed class methods unlike windows exe, most SGI STL functions in Linux exe were simply inlined by GCC unlike windows exe (and yes, both version seems use SGI STL, I have confirmed ). Here my OpenAL compat in action. It was done about 2 months ago because recently I'm still busy. But fell free to check my repo for more details and how this fix works.