Jump to content


  • Content Count

  • Joined

  • Last Visited

  • Most Liked  


simmaster07 last won the day on
January 21

simmaster07 had the most liked content!
View Past Leaders

About simmaster07

  • Rank
    Sim Master

Contact Methods

Profile Information

  • Gender
  • Location
    New York, NY
  • Interests
    Systems programming, reverse engineering, jazz-influenced rap
  • City-building game(s)
    SimCity 4
    Cities: Skylines

Recent Profile Visitors

5,314 Profile Views
  1. https://arstechnica.com/gadgets/2018/04/apple-is-exploring-macs-running-its-own-cpus-but-that-dream-is-a-long-way-off/ Not only is Apple going to drop support for Intel 32-bit programs, it looks like they're going to try to replace Intel chips outright with an in-house solution. If this ends up happening, it'll probably be an ARM-based chip like their A11 chip for iPhones, and if that's the case then those machines will have no support for even the rebuilt 64-bit applications that developers are putting out. I don't see this working out for anything more powerful than their entry-level Macbook line. ARM chips have lower power consumption but are far slower, and trying to push this onto their pro models is completely unviable. I also wonder if this'll just fracture their Mac ecosystem.
  2. It is possible. You can set up two partitions, one running the latest macOS and the other running an older version, and dual boot between the two. You might need rEFInd for this.
  3. SC4Fix: third-party patches for SC4

    @jeffryfisher DLL files can be placed in any plugins folder just like any other plugin. If you rename your user plugin folder often that'd be a reason to prefer placing the DLL in the Program Files plugin folder, but otherwise there's no difference. I've tested the DLL in the Documents plugin folder and that's also worked fine.
  4. Revisiting Prop Pox and Prop Theory

    Thanks for the feedback and kind words, everyone. I just fixed the experimental DLL leaking memory on every save — it was a small amount, maybe 1-2MB each time, but I wanted to fix it before marking this as stable. Given the feedback so far I've decided to mark this as stable now and upload a new version of SC4Fix to the STEX: (If you're still using the experimental DLL, you're probably going to want to grab this one for the memory leak fix.) We've certainly come a long way from April Fool's pranks
  5. Revisiting Prop Pox and Prop Theory

    I don't believe that a prop being disabled intrinsically causes prop corruption, but I also can't see what in the game internals causes a prop to become disabled. At a glance the code for serializing stuff like prop entries to disk doesn't seem to change its behavior based on the visibility/enabled flags; the game might disable props if they have invalid properties, but I honestly have no clue. I'd have to look at this later. If prop pox is the consequence of a bug in the serializer, it might also be responsible for other savefile corruption issues. The implementation of savegame writing in the digital and CD versions of the game are definitely the same, though. --- Anyway, an experimental DLL for Windows is done and has been rolled into a prerelease build of SC4Fix, which you can grab here. I really need to stress that even if this is shown to be a valid fix it is not ready for primetime and you should not use this on any cities you don't have backups for. One big issue is this will probably cause memory to leak when you save the game. The reason I'm calling this out as a potential fix is because of some experimenting I did with this city: Test City, prop pox timebomb Without my DLL this city will immediately show buffer corruption in wouanagaine's Savegame Explorer the first time it's loaded and saved. This behavior's been pretty consistent on both Mac and Windows. With the DLL, though, a few props are marked as disabled but the prop subfile hasn't been outright corrupted after playing and saving it for a bit. Feel free to try it out for yourself and see what happens. I think I'd actually be relieved to hear that it doesn't work because I still have absolutely no idea how my DLL is more stable than the game's normal implementation. --- Side note: the main reason getting this done for Windows took as long as it did is that unlike the Mac port, SC4 on Windows does use EASTL, but it uses a far older version of EASTL from when Paul Pedriana was first working on it, which means that publicly available builds weren't 100% compatible, plus there are some private implementation details specific to SC4 that I can't reimplement. This means things are somewhat hacked together and the code is ugly, but it works. ¯\_(ツ)_/¯ And a fun fact on that note: I remember reading people complaining that the Mac port of SC4 runs pretty slowly. One reason why is that Aspyr didn't use EASTL, which is optimized for games with large memory requirements to run as smoothly as possible by reserving memory in large pools and reducing the number of function calls. The standard library that Aspyr used is just what shipped with their compiler and isn't purpose-built or highly optimized. I wish I knew why they did this.
  6. Update 2018/01/21 SC4Fix has been updated with a stable build that fixes prop pox. Download on the STEX: --- I like a good challenge, so I decided to take this one up. Context Prop pox is a bug that causes decorative props in some cities to begin disappearing. The problem is exacerbated if the city continues to be played and saved, and persists even after adjusting graphics settings for prop detail or restarting the game. The prevailing hypothesis is bap's prop theory, which makes these main claims: In a normal game installation with absolutely no plugins, prop pox will never occur, even after updating. Prop pox is actually caused by plugins that redefine Maxis prop exemplars that add properties that were not originally in the exemplars. Prop pox will only manifest itself if one of these plugins is installed and the city's prop subfile* is sufficiently large. * bap refers to the prop subfile as the network subfile in his post, which seems to be a mistake. Testing this is pretty straightforward as all we need to do is clear our plugins, develop a city until the prop subfile is large enough (around 6MB, according to bap) and then use a plugin that causes pox by improperly redefining exemplars. For this I made a mostly flat large tile city with sprawling low-density developments to add as many props as possible in a short timeframe. When the prop subfile for the city was just under 6MB I made a copy of it and then allowed it to develop further until it crossed that limit. Why is this 6MB limit significant? The game determines whether a subfile should be compressed mainly by its filesize (along with a few other properties); subfiles smaller than 6MB can be compressed, but anything larger than that is saved decompressed in order to save time and computational resources when saving very large subfiles. Prop pox apparently begins most commonly when crossing this compression limit. With no plugins installed, the city was able to develop past the 6MB prop subfile limit without any pox or save corruption occurring, no matter how many times I saved or how long the simulation ran for, which I tested using wouanagaine's SC4 Savegame Explorer. If prop theory is correct, then pox should occur if we install a bad plugin and try to develop the city past that limit again. For this, bap used SimPEG's Beach Development Kit 1: In particular, you only need to extract PEG-OWW2_BDK_RESOURCE.dat as this contains the exemplar redefinitions. After installing this plugin, I loaded the test city, added some new zones to force new props to develop, ran the simulation for a bit and saved the city a couple of times. This immediately resulted in the city developing prop pox according to the savegame explorer. For your convenience for reproducing this, here are the savegame files I used: Test City, prop subfile under 6MB, no plugins and no pox Test City, prop subfile over 6MB, no plugins and no pox Test City, prop subfile under 6MB, BDK_RESOURCE installed and pox present Explaining the buffer corruption Since the savegame explorer reports buffer corruption at a particular location in the savegame, we should probably check out what's happening there to see if this improves our understanding of what causes prop pox. First we need to extract the prop subfile from the savegame using iLive's Reader: The pox-infected .sc4 save can be opened in the Reader. From there we need to extract the entry with type ID 0x2977aa47 (which is marked as Unknown, and that's fine). We also want to make sure we have the decompressed version of the subfile, so we'll save the decoded file and open it in an external hex editor since the one built into the Reader isn't great. I prefer using HxD. If we jump to the reported offset of corruption in the hex editor, we can read the bytes of the prop subfile as base-16 hex values, and the structure of the prop subfile is given in the SC4D wiki. (Note that DWORDs are four bytes large, WORDs are two bytes, and BYTEs are one.) Here's what the corrupted prop entry looks like in the hex editor: And here's what this looks like when filling it into the prop structure given on the SC4D wiki: DWORD Size: 0x00000058 DWORD CRC: 0x006C2462 DWORD Memory: 0x4CE00000 ??? : 0x59E6 ??? : 0xA084 ??? : 0x0F06 WORD Major: 0x0006 WORD Minor: 0x0004 WORD Zot: 0x0000 BYTE Unknown: 0x00 BYTE Appearance: 0x04 DWORD : 0xA823821E BYTE Min Tract X : 0x42 BYTE Min Tract Z : 0x6E BYTE Max Tract X: 0x42 BYTE Max Tract Z: 0x6E WORD X Tract Size: 0x0002 WORD Z Tract Size: 0x0002 DWORD Property Ct.: 0x00000001 SGPROP #1 DWORD Property Name: 0x89A1C16C DWORD Property Name: 0x89A1C16C DWORD : 0x00000000 BYTE Data Type: 0x03 BYTE Key Type: 0x00 WORD: 0x0000 DWORD Rep Count: 0x00000000 DWORD GID: 0xC977C536 DWORD TID: 0x6534284A DWORD IID Exemplar: 0x29000000 DWORD IID Given: 0x29000000 FLOAT32 Min X: 0x4308558C FLOAT32 Min Y: 0x43870000 FLOAT32 Min Z: 0x45386FA3 FLOAT32 Max X: 0x430A558C FLOAT32 Max Y: 0x43880000 FLOAT32 Max Z: 0x45389FA3 BYTE Orientation: 0x03 BYTE State: 0x00 BYTE Start Time: 0x00 BYTE Stop Time: 0x00 BYTE Count: 0x00 BYTE Rand. Chance: 0x64 BYTE Lot Type: 0x02 DWORD Object ID: 0x2A46EEFF BYTE Conditional: 0x00 Immediately something is wrong: there are six extra bytes after the first three pieces of data that don't correspond to anything in the prop file structure. Moreover, this entry gives its size as 0x58 (88) bytes but the hex editor says this is 0x72 (114) bytes in it. Where are the extra 26 bytes coming from? Well, first we can discount the seemingly random six bytes that appeared at the top. Next we can scrutinize the SGPROP since that's only an optional part of the prop subfile, and sure enough four DWORDs (16 bytes), a WORD (2 bytes) and two bytes add up to 20. And just out of curiosity, let's see what exemplar IID 0x29000000 maps to: Sure enough, it's a redefined residential beach umbrella exemplar from SimPEG's BDK resources. The main change in PEG's exemplar is that the Nighttime State Change entry (one byte large) in the Maxis exemplar was replaced by a Prop Time of Day entry (two Float32s, or eight bytes large). This is similar to what RippleJet observed when testing this. Because this prop entry is corrupted, closing this save and loading it will cause the game to believe that the entry really is only 88 bytes long, it will read several garbage values in the process, and after 88 bytes have been read it will attempt to read what it thinks is another prop entry. That prop entry will have a nonsensical size entry because it's being misinterpreted due to the corruption of the save, and that prop and all props that follow it in the savefile will not be present. As for why savefiles tend to continuously shrink on every save: When loading a file with corrupted props, incorrect size data and other malformed information will cause the corrupted prop and all props that follow it in the file to be misread. These misread props are discarded by the game, causing them to disappear from the city. Since the malformed props were discarded when loading, they will not appear when saving again, causing the filesize to shrink. On this new save, however, other malformed props could corrupt the savefile again, repeating the process. Towards a vaccine It's worth pointing out that bap originally speculated that prop pox was caused by a buffer overflow or some kind of memory error in the game that caused data for a modded prop to spill past the end of its buffer in memory. To test this, I installed the game on Fedora Linux 26 using WINE 3.0-rc6 and ran it under Valgrind, a memory debugging tool for Linux. However, I found no difference in the number of memory errors reported before and after installing PEG's beach resources. Since some reported instances of prop pox involve parts of the prop subfile being overwritten, we have to look at how the game actually writes data to disk. Since other prop poxxed cities appear to have prop definitions overwriting the middle of other prop definitions, it's possible that the game is incorrectly properly calculating where it should be writing when dealing with these overridden exemplars. Unfortunately this is where things go off the rails: in order to better determine what's going on in this process, I rewrote the function responsible for writing savegame objects, and this link shows what that looks like. Since the Mac version of the game contains the debugging symbols for this to work, I've only tested this on my Mac, and porting this to Windows is nontrivial. In another insane twist, I can't reproduce prop pox using my rewritten Mac serialization code, and can reproduce it immediately when not injecting that code, but I can't confidently say I fixed it since I can't demonstrate this to very many people without a usable Windows DLL, so that's my project for the time being. I also have no idea why my code doesn't cause prop pox to occur since I tried to recreate the original implementation of savegame serialization as closely as possible. For now, I'd speculate that prop pox is caused by a quirk in how the game saves records. Since the game may not know the size of all records in advance when saving a game, it has to resort to rewinding and fast-forwarding to arbitrary offsets in order to save data. For instance, when saving a prop record, the game will: Write the number 0 as a placeholder for the size in bytes. Write the number 0 as a placeholder for the CRC checksum. Write an identifier and the record itself. Rewind to the first zero. Calculate and rewrite the size based on how many more bytes there are in the file after writing the record. Fast-forward to the record itself and read the data in it back to a buffer. Rewind to the second zero and calculate and write the CRC checksum. Fast-forward to the end of the output stream to allow more data to be written. I think that miscalculated offsets when rewinding and fast-forwarding is causing previously written prop records to be overwritten by malformed props, corrupting the entire subfile in the process. How exactly does that relate to the redefinition of the prop exemplar? ¯\_(ツ)_/¯ I'm working on getting my serialization code ported to Windows ASAP to see what happens there. In the meantime, if you have a Mac and want to try it out and report if you're still seeing prop pox: Grab this dylib which was built from the source code linked above (and again here). Go to Utilities in Finder and open up a terminal. I used Steam, so I used these commands: cd "$HOME/Library/Application Support/Steam/steamapps/common/SimCity 4 Deluxe/SimCity 4 Deluxe Edition.app/Contents/MacOS" echo 24780 > steam_appid.txt DYLD_INSERT_LIBRARIES="$HOME/Downloads/libinjector.dylib" ./Sim\ City\ 4\ Deluxe\ Editionsub Your game should now be running with some output labeled COMSerialize, particularly when saving the game. And if you want to help me out, just send me PMs with prop poxxed savegames for me to download so I can look at them.
  7. My take on this is more technical. When distributing an operating system that supports both 32-bit and 64-bit systems, you have to compile core system libraries, utilities and applications twice — once for 32-bit compatibility and once for native 64-bit — which effectively doubles the size of the operating system and increases the number of binaries that need to be tested. Since the release of Mountain Lion though there are no Macs that can run that OS or later that don't support 64-bit, and this was five years ago. There really hasn't been a reason to support 32-bit this entire time aside from compatibility with apps built for the 32-bit-only Macs that were released before then. What really makes the situation convoluted is that when Apple made the switch from PowerPC to Intel for their Macs, they produced a few models in 2006 that only supported 32-bit. Later that same year, though, many of those products were replaced with models that had 64-bit processors, so had they waited another couple of quarters and not jumped the shark on the Intel conversion there would've been no reason to have 32-bit builds of OS X in the first place. They also made the situation worse by waiting as long as they did to deprecate 32-bit, and if they announced that there'd be no 32-bit support starting with Mountain Lion there'd be far fewer apps that would need to be rebuilt and the damage would be less severe. I don't blame them for trying to walk back 32-bit compatibility, though. There are some performance advantages1 with making everything native 64-bit, the switch reduces the complexity of producing macOS and it can save on the amount of space required by the OS. 1 The biggest improvement that 64-bit processors have to offer there are more data registers offered by the processor. Data can be stored in those registers without having to go to RAM, which is far faster, and can help speed up compute-intensive tasks. All 64-bit processors also support SSE2 instructions which are useful for parallel processing, but 32-bit processors may not necessarily support these instructions.
  8. SC4MacInjector - Support Thread

    Hey @SimsofLasVegas, I uploaded a new version to fix the issues you're seeing. Can you redownload and try again?
  9. Buggi's Extra Cheats for Mac

    Version 1.0.0


    What does this do Enables Buggi's extra cheats for SimCity 4 on Mac copies of the game. Dependencies You MUST have SC4MacInjector installed or else nothing will happen. You must also have a Mac - if you're on Windows, use Buggi's extra cheats DLL. Installation Copy ExtraCheatsMac.so to your plugins folder. On Steam and disc copies this is in ~/Documents/SimCity 4/Plugins On Mac App Store copies this is in ~/Library/Containers/com.aspyr.simcity4.appstore/Data/Documents/SimCity 4/Plugins Create these directories if they do not exist yet. Relevant resources Development Thread Source Code
  10. Version 0.9.1


    Prerelease note This mod is prerelease software and has only been tested on Steam and Mac App Store copies of SimCity 4 on Mac OS X El Capitan 10.11.6. It is recommended that you back up your regions (and, especially if using an App Store copy of SC4, your plugins folder) before using this mod. What does this do SC4MacInjector allows the Mac equivalent of .dll mods to be loaded on Aspyr's port of SimCity 4 for Intel Macs. Note: you will not be able to use Windows .dll mods directly. They need to be targeted specifically for Macs as .so files. Requirements This mod requires an Intel Mac copy of SimCity 4 Deluxe/Rush Hour to already be installed. Installation Run the .pkg installer. If you do not have a Steam or App Store copy, or if the App Store copy was moved out of /Applications, the installer will ask you to manually locate your game executable, and this can get somewhat hairy if it's a relocated App Store copy. Relevant resources Technical Support Development Thread Source Code
  11. Soooooo I was going to upload this to the STEX with a new package installer but I started getting errors when I tried to upload the file. I'll probably try again tonight to see if this isn't a persistent issue, it's just a bit awkward since I already posted the support thread so I could put a link to it in the installer. Anyway, here's prerelease 2. Tested and works with the Steam version of SC4 Mac, but the App Store version is, as always, extremely iffy. Since it's installed from the App Store it's a lot more strict about making sure any code that gets injected is signed, although I haven't been able to test it since my App Store copy corrupted itself and refuses to run even after reinstalling it, so I'm not 100% sure it works there.
  12. What is SC4MacInjector? SC4MacInjector is a mod for Mac copies of SimCity 4 Deluxe that enables the use of dynamic library mods (the equivalent of DLLs on Windows) for Macs. General discussion and development updates are posted in the game framework discoveries thread – this thread serves to consolidate any potential support requests in one purpose-specific place. General troubleshooting steps The injector responsible for enabling these mods may be overwritten by game updates or checking game cache integrity in Steam. Try re-running the installer if mods like the extra cheats DLL are suddenly disabled. Also make sure you've obtained the latest version from the STEX. Reporting issues One very helpful thing to provide is a detailed installer log. When running the installer, bring up the log: Change the detail level to All Logs: Then click the Save button on the installer log in the top right corner of the window. Attach this log to your post. Another extremely helpful thing to provide, if present, is the injection log, which you can find at ~/Documents/SimCity 4/libinjector.log. Attach this too if possible. Lastly, try to provide a thorough description of your problem, when it started occurring, and what version of the game you use (e.g. Steam, App Store, disc copy).
  13. So that's goals 1 and 2 down for the most part. I've posted a prerelease of the injector on GitHub (which also has the source code and some instructions on how to use it in its current state). It's a useful proof-of-concept, but I'm not really happy with the effort required to set it up and use it, which is why it's not on the STEX yet. I'm going to work on packaging it in a .pkg file or something with a simple installer before uploading the injector and some Mac-ported mods (extra cheats, extra extra cheats, growable city hall, etc.). After that I'll start investigating puzzle piece/TE lot crashes on Mac to see if I can rewrite SC4Fix to not be specific to Windows patch 641.
  14. I've looked at the thread and will still be moving forward with it. Personally I'm betting Aspyr will recompile most, if not all of their desktop games for 64-bit when the time to do so approaches, at least so that their publishers won't get mad that they will get literally zero return on investment the moment that 32-bit is deprecated and users upgrade if they don't do anything. For the most part the effort involved in recompiling a 32-bit Intel app for 64-bit Intel is minimal, and retargeting well-crafted code (like SimCity 4, as far as I'm aware anyway) shouldn't be much harder than changing a few options in the compiler toolchain and maybe fixing a few things here and there.
  15. Hi everyone, I just finished up exams, which means it's time for the ~biannual check-in~. Multi-threading and the graphics engine proof-of-concept are on hold to focus on smaller but more immediately impactful changes. In the meantime, let's talk about Mac. I got a secondhand Mac Mini recently, and I wanted to see if Macs can be modded using the equivalent of DLLs on Macs (aka dylibs). As it turns out, despite Maxis already writing the code to handle loading libraries, Aspyr stripped this function from their port. Thankfully, though, all the other components for loading dylibs (searching the Plugins folder for them and attempting to actually load them) seem to be there, so all that needs to be done is rewriting this one function that tries to get the operating system to load the library. macOS supports this pretty cool thing called function interposition, which means that even if a program doesn't load a library you can: Force it to load a specific library anyway, and Use that library to replace functions in the program with your own implementation. The next few days are gonna be busy as I pack up and get ready to go back home for the holidays but by mid-January I want to accomplish three main goals that I'll keep you all posted on: Fix SC4 on Mac so it actually tries to load game-framework libraries. Port the extra cheats DLL to Mac. Rewrite SC4Fix to work on both Mac and Windows.