CTD with Boston Big Dig mod

[jonnyquest 71]

Does anyone still use the road > subway traffic technique for tunnels these days? I'm getting crashing every time I try. There's an immediate CTD as soon as I even try to plop any of the pieces from the Boston Big Dig mod here: https://www.sc4devotion.com/csxlex/lex_filedesc.php?lotGET=942

It's an old plugin so I''m assuming this is most likely a compatibility issue with more recent NAM versions? Or is there perhaps some other conflicting mod that might be causing this? I want to play around with some RHW intersections by having slip roads go underground and connect and the flexibility of the subway technique seems to me the easiest way to do this

[rivit 3,391]

Have you got SC4Fix.dll installed (see link on main page Simtropolis). If you don't and these are transit enabled lots then you could be triggering the TE Lot CTD caused when two TE'd lots touch. If you do have it installed then I would assume its a bit more exotic.  

[jonnyquest 71]

7 minutes ago, rivit said:

Have you got SC4Fix.dll installed (see link on main page Simtropolis). If you don't and these are transit enabled lots then you could be triggering the TE Lot CTD caused when two TE'd lots touch. If you do have it installed then I would assume its a bit more exotic.  

This could very well be the problem! I should've mentioned I'm running the Mac version of the game so I suppose this is yet another reason to switch as I don't believe there's a Mac equivalent of SC4Fix

[CorinaMarie 29,608]

Just now, jonnyquest said:

I don't believe there's a Mac equivalent of SC4Fix

There was, but for the 32 bit version of the game. Now that the Mac version is 64 bit the Mac Injector method is no longer viable.

[Cyclone Boom 16,884]

And here's a link to that:

 

But alas, sadly it's no longer available even for 32-bit Mac systems.

The source code can be found here though in case it might be useful for developing a 64-bit version. That seems unlikely to be easily feasible however.

[rsc204 21,808]

8 hours ago, jonnyquest said:

There's an immediate CTD as soon as I even try to plop any of the pieces from the Boston Big Dig mod

So there are two types of data formats that the 64-bit version (Mac) dislikes, however since these issues only presented themselves after it's release, there remains much content on the exchanges that may have the problem. A thread exists here listing all the content that's so far come to light, with fixes for Mac users. I've added fixed versions of the SC4Lot files for this set to this post.

In short one will prevent the game from starting altogether, CTDing when loading a city. The other only CTDs when you try to place a lot in-game, such as you're experiencing here.

Note that the CTDs from hovering Puzzle Pieces/Starters/FlexPieces over TE Lots such as these, very much exists as a problem. However these CTDs don't come from placing TE Lots, they only happen when selecting Network-Related items. Sadly you can't use the SC4Fix anymore, so you need to exercise caution when using such items.

5 hours ago, Cyclone Boom said:

The source code can be found here though in case it might be useful for developing a 64-bit version.

The basic problem is the concept of 'injecting' code is simply impossible under Catalna and has nothing to do with the removal of 32-bit support. It's related to the way in which MacOS sandboxes apps and underlying security of the OS kernel had made it impossible for this kind of .dll modding to work. Hence the project was abandoned, since there is no realistic method available to replicate how this works anymore. Nothing short of a rewritten .exe would be sufficient to fix this now, since the OS won't allow code to be altered during runtime.

[jonnyquest 71]

13 hours ago, rsc204 said:

So there are two types of data formats that the 64-bit version (Mac) dislikes, however since these issues only presented themselves after it's release, there remains much content on the exchanges that may have the problem. A thread exists here listing all the content that's so far come to light, with fixes for Mac users. I've added fixed versions of the SC4Lot files for this set to this post.

In short one will prevent the game from starting altogether, CTDing when loading a city. The other only CTDs when you try to place a lot in-game, such as you're experiencing here.

Note that the CTDs from hovering Puzzle Pieces/Starters/FlexPieces over TE Lots such as these, very much exists as a problem. However these CTDs don't come from placing TE Lots, they only happen when selecting Network-Related items. Sadly you can't use the SC4Fix anymore, so you need to exercise caution when using such items.

The basic problem is the concept of 'injecting' code is simply impossible under Catalna and has nothing to do with the removal of 32-bit support. It's related to the way in which MacOS sandboxes apps and underlying security of the OS kernel had made it impossible for this kind of .dll modding to work. Hence the project was abandoned, since there is no realistic method available to replicate how this works anymore. Nothing short of a rewritten .exe would be sufficient to fix this now, since the OS won't allow code to be altered during runtime.

Very interesting. Perhaps a patch of the app could be developed, along the lines of the 4gb patch for Windows users?

[jonnyquest 71]

On 9/11/2021 at 12:36 AM, rsc204 said:

So there are two types of data formats that the 64-bit version (Mac) dislikes, however since these issues only presented themselves after it's release, there remains much content on the exchanges that may have the problem. A thread exists here listing all the content that's so far come to light, with fixes for Mac users. I've added fixed versions of the SC4Lot files for this set to this post.

In short one will prevent the game from starting altogether, CTDing when loading a city. The other only CTDs when you try to place a lot in-game, such as you're experiencing here.

Note that the CTDs from hovering Puzzle Pieces/Starters/FlexPieces over TE Lots such as these, very much exists as a problem. However these CTDs don't come from placing TE Lots, they only happen when selecting Network-Related items. Sadly you can't use the SC4Fix anymore, so you need to exercise caution when using such items.

The basic problem is the concept of 'injecting' code is simply impossible under Catalna and has nothing to do with the removal of 32-bit support. It's related to the way in which MacOS sandboxes apps and underlying security of the OS kernel had made it impossible for this kind of .dll modding to work. Hence the project was abandoned, since there is no realistic method available to replicate how this works anymore. Nothing short of a rewritten .exe would be sufficient to fix this now, since the OS won't allow code to be altered during runtime.

At risk of resurrecting a dying thread: I'm wondering how difficult the process is of adapting these transit lots to overcome this quirk in the 64-bit version? I've designed an entire city around a monorail station that's not working on my Mac version of the game: 

  Also a GHSR to subway converter I enjoy using 

 

[rsc204 21,808]

12 hours ago, jonnyquest said:

At risk of resurrecting a dying thread: I'm wondering how difficult the process is of adapting these transit lots to overcome this quirk in the 64-bit version?

It's not really a dead thread, but in the time it's been there you are like the 3rd person to report problems. Fixing them is very, very easy if you have access to Reader, the Windows tool needed and understand what you are doing. I have (somewhere in the web of all this), uploaded full instructions with screenshots on how to fix the problem. We really need a thread to report the problem, I didn't want to do that where the fixes are, right now it's neat and easy to follow, which will bear fruit if the thread get's long over time. I don't like the idea of having tons of posts interspersed between the important stuff. For now, if someone asks, generally I just take care of it.

At it's core, these problems occur when a Float32 data type is defined with a Rep count of 1. This is because the correct count for a single rep is 0. Until the 64-bit Mac version came along, this never caused an issue, but the 64-bit code changes something that means it must now be altered. The fix is essentially finding those errors and changing a 1 to a 0, before saving the file. A rep is like a list, each item in the list would need it's own Rep. Note one item of data = 0 Rep, >1 items of data = that number of Reps, it's only 1 that is special here.

You first need to work out which of the two errors a file has, whilst the both relate to the Rep count, they are subtly different:

• So if the game CTDs when selecting an item, likely the issue is with the transit properties, Transit Switch Entry Cost and Transit Switch Traffic Capacity. The tool modders use to make TE Lots (literally the only one we have), sadly has a bug that upon every save, set's these as 1 again. If a modder doesn't know or isn't careful, it's real easy for these to get into the wild.
• If however, the game CTD's whilst loading, that tells you the problem is in some Exemplar. You just need to get through every one from the mod, only checking the Float32 data, there will be one or more that has only a single Rep but has 1 instead of the desired 0 in-place.

It's kind of a moot point, since either way, the fix is essentially the same, the latter problem is far less common and despite a statement to the contrary by some, the fact is we've no idea how this data is being messed up or which tool might be behind it. I can say this with 100% certainty, since two of my own files were affected and I know the exact, step-by-step history of how I put them together. Repeating that process does not lead to these issues, so it's a mystery but thankfully not too common.

On 11/09/2021 at 8:26 PM, jonnyquest said:

Perhaps a patch of the app could be developed, along the lines of the 4gb patch for Windows users?

Sadly not, we don't have a patch for the Windows edition either, which would be legally speaking, impossible to release such without EA's blessing.

The 4GB patch uses a function of Windows to set a flag the OS can see, it's like a tickbox you can't see, once that's ticked, the OS will allow a 32-bit app access to more memory than it otherwise would. Were it not for this being in 64-bit editions of Windows, we'd all be buggered without a specific 64-bit update of SC4, since the a full NAM install NEEDS that extra RAM now. I know of other games that even unmodded fail without applying the same patch.

Aspyr have done more updating than Maxis/EA ever bothered with, but it should be noted they only ever do what's needed to keep the game running after some OS changes. When you see the stupid bugs left behind and how they simply don't respond to these issues, let alone fix them, I wouldn't hold my breath. I mean, this is the guys that instead of starting with the Deluxe edition, used a copy of the original SC4 with the Rush Hour addon (see splash screen on startup), then never bothered to add the two Maxis official patches. In short, expectations re making the game better, should be really low here, the Mac edition does not have to be such a mess. Given they still want $20 for it, there is no good reason not to fix these annoying shortfalls IMHO, but why spend the money when people buy the games anyhow, right?

[jonnyquest 71]

1 hour ago, rsc204 said:

It's not really a dead thread, but in the time it's been there you are like the 3rd person to report problems. Fixing them is very, very easy if you have access to Reader, the Windows tool needed and understand what you are doing. I have (somewhere in the web of all this), uploaded full instructions with screenshots on how to fix the problem. We really need a thread to report the problem, I didn't want to do that where the fixes are, right now it's neat and easy to follow, which will bear fruit if the thread get's long over time. I don't like the idea of having tons of posts interspersed between the important stuff. For now, if someone asks, generally I just take care of it.

At it's core, these problems occur when a Float32 data type is defined with a Rep count of 1. This is because the correct count for a single rep is 0. Until the 64-bit Mac version came along, this never caused an issue, but the 64-bit code changes something that means it must now be altered. The fix is essentially finding those errors and changing a 1 to a 0, before saving the file. A rep is like a list, each item in the list would need it's own Rep. Note one item of data = 0 Rep, >1 items of data = that number of Reps, it's only 1 that is special here.

You first need to work out which of the two errors a file has, whilst the both relate to the Rep count, they are subtly different:

• So if the game CTDs when selecting an item, likely the issue is with the transit properties, Transit Switch Entry Cost and Transit Switch Traffic Capacity. The tool modders use to make TE Lots (literally the only one we have), sadly has a bug that upon every save, set's these as 1 again. If a modder doesn't know or isn't careful, it's real easy for these to get into the wild.
• If however, the game CTD's whilst loading, that tells you the problem is in some Exemplar. You just need to get through every one from the mod, only checking the Float32 data, there will be one or more that has only a single Rep but has 1 instead of the desired 0 in-place.

It's kind of a moot point, since either way, the fix is essentially the same, the latter problem is far less common and despite a statement to the contrary by some, the fact is we've no idea how this data is being messed up or which tool might be behind it. I can say this with 100% certainty, since two of my own files were affected and I know the exact, step-by-step history of how I put them together. Repeating that process does not lead to these issues, so it's a mystery but thankfully not too common.

Sadly not, we don't have a patch for the Windows edition either, which would be legally speaking, impossible to release such without EA's blessing.

The 4GB patch uses a function of Windows to set a flag the OS can see, it's like a tickbox you can't see, once that's ticked, the OS will allow a 32-bit app access to more memory than it otherwise would. Were it not for this being in 64-bit editions of Windows, we'd all be buggered without a specific 64-bit update of SC4, since the a full NAM install NEEDS that extra RAM now. I know of other games that even unmodded fail without applying the same patch.

Aspyr have done more updating than Maxis/EA ever bothered with, but it should be noted they only ever do what's needed to keep the game running after some OS changes. When you see the stupid bugs left behind and how they simply don't respond to these issues, let alone fix them, I wouldn't hold my breath. I mean, this is the guys that instead of starting with the Deluxe edition, used a copy of the original SC4 with the Rush Hour addon (see splash screen on startup), then never bothered to add the two Maxis official patches. In short, expectations re making the game better, should be really low here, the Mac edition does not have to be such a mess. Given they still want $20 for it, there is no good reason not to fix these annoying shortfalls IMHO, but why spend the money when people buy the games anyhow, right?

Well I do appreciate the performance boost of the 64-bit version at least. Dealing with the rest of the quirks they don't bother to fix is a pain.

Much thanks as always for the explanation! If you turn up the instructions on how to fix the lots maybe I'll give it a shot with Reader, as I still have Wine lying around to fire it up with..

[rsc204 21,808]

Actually it turns out it was the link on the first post of the thread about the patches, which is here. But if you come across something, even if you fix it for yourself, please do pass the information on at least. If we keep everything together, hopefully one day at worst we'll have a bunch of functioning patches so these things don't have to be fixed multiple times.

[jonnyquest 71]

2 hours ago, rsc204 said:

Actually it turns out it was the link on the first post of the thread about the patches, which is here. But if you come across something, even if you fix it for yourself, please do pass the information on at least. If we keep everything together, hopefully one day at worst we'll have a bunch of functioning patches so these things don't have to be fixed multiple times.

I'm definitely going to look into this and whatever lots I end up fixing I'll upload for the community.

Maybe we could start a thread of mods specifically patched for Mac users, like some custom lots that have been updated with nitelights :3 From my cursory understanding of the situation with that, it might be good to upload nitelight fixes along with the specific ID we use for the fix, so that people know not to repeat it when using or editing other lots?

[CorinaMarie 29,608]

3 hours ago, jonnyquest said:

Maybe we could start a thread of mods specifically patched for Mac users, like some custom lots that have been updated with nitelights :3

Very good idea and it almost happened a couple or three years ago, but the Mac player who was involved sort of faded away and then the project did too. With active interest again, this would go well.

 

3 hours ago, jonnyquest said:

From my cursory understanding of the situation with that, it might be good to upload nitelight fixes along with the specific ID we use for the fix, so that people know not to repeat it when using or editing other lots?

Yep. True too.

@rsc204 started a spreadsheet with exactly that in mind which also precluded reserved IIDs that NAM (and prolly other important things) use. Then he and @Cyclone Boom tweaked how it worked. As I recall, just as that was getting polished to be ready for a dedicated thread is when the need sort of evaporated when the interested person disappeared.

[rsc204 21,808]

18 hours ago, jonnyquest said:

Maybe we could start a thread of mods specifically patched for Mac users, like some custom lots that have been updated with nitelights

Specifically in relation to the Nitelighting, there are a few considerations. First as it stands right now, there are just a small handful of such models that exist. Maxis soon realised after creating the SC4 BAT tool, there would be a problem with the existing ID system, so they patched the game (NiteLighting patch) to expand the available IDs. As such, since everything would have been created with these 'Windows' tools, outside of the one or two creators that tinkered to add it, there isn't a lot of content that exists right now.

In theory, within the specific ID range Mac's must use, there are 65,000 potential free IDs. However, that includes everything Maxis already included in the game, along with other ranges that simply can't be used for one reason or another. We made that ID sheet, cross-checking every 'Mac compatible' model we knew of. Even so, given the lack of demand, I think there are a sufficient number of IDs to be useful. The question is how to manage the process for the greatest benefit. I think the tags we can add to STEX entries would make a lot of sense to begin with, since you can search everything with that tag very easily. I'd go with Mac Nite or something simple, with Mac Compatible being the equivalent tag to confirm it has been tested or modified to prevent CTDs for Mac users. I personally prefer the idea of separating out the two, since nothing prevents both from being added.

14 hours ago, CorinaMarie said:

As I recall, just as that was getting polished to be ready for a dedicated thread is when the need sort of evaporated when the interested person disappeared.

Yeah, that pretty much sums this up, we only really had interest from one member here, I got a bit busy and whilst all the building blocks were sorted behind the scenes, it never got any further. However the ID list is online and we could start issuing ID ranges in blocks of 240 (roughly) IDs at any time we liked.

18 hours ago, jonnyquest said:

it might be good to upload nitelight fixes along with the specific ID we use for the fix, so that people know not to repeat it when using or editing other lots?

This is doing things the wrong way around if you ask me, much better to use a range that's been expressly reserved for you before starting, this was certainly the solution we were looking to implement. There are only a few areas of the game that require dedicated IDs, typically where the number that exist is limited. So right now if you wanted to make/release Textures, MMPs or Prop/Building Families, you must first apply for a range that only you are allowed to use. Note this doesn't cover personal use, if not releasing things you can use any IDs you like. In terms of the number of IDs available, the NiteLighting system is the most restrictive of all those four. But this isn't some attempt at controlling things, every user has the right to be assigned a range on request. For example, I've known banned members get a range for example, it simply needs to be managed to prevent conflicts and provide the maximum benefits to all. As such, right now if anyone wants to ask for a range, they can simply contact one of the ST staff and we'll make sure the request is dealt with. Again, since no one ever did, setting up a thread sort of got forgotten, having one or not is rather a moot point at this time, until there is some actual demand for IDs. A thread with no responses quickly get's lost unless you make it 'sticky', on the flip side sticky threads can become clutter too, there's a need to balance things, although we can adapt things as the situation changes.

The next question is what to do with such files, to try and ensure they are available for others to take advantage of. You could make a thread similar to the one for the CTD-fixes I've done, especially if users need a special download or "patch", it would be nice to keep everything in one place. It wouldn't be a good idea to start converting the models en-masse on the exchanges to use the Mac-Compatible GID, this is better kept as specific Mac versions/patches. Otherwise we're essentially undoing the fix Maxis implemented all those years ago. Also there are other potential problems for those not using the Mac edition by doing this for existing content.

Then comes the issue of permission, because this requires the re-distribution of other user's content, ideally you'd get their permission first. Practically speaking, that's not usually simple, many creators have moved on for example. Generally speaking I believe it was agreed we'd allow such things, but if the user of the content is active, you should try to obtain permission first. The other consideration is to do with packaging and dependencies, models aren't always stand-alone and may be re-used for multiple purposes. Let's imagine someone makes a house model, it ends up in a MegaPack dat or some such, you create a patch that alters the ID in use, but are only aware of one Lot that uses it. Because such ID changes, require cascading changes to all Exemplars that reference it, that has the potential to cause problems. Thankfully the sort of thing we're probably looking at here, is likely to be single-models in SC4Model files, which lessens the issue.

I think I would like to see one stickied thread, which explains the problems both of NiteLights and the CTD issues, where people can report them or request Mac patches. This ensures the problem is prominent enough and everything is in one place. We can then keep separate threads where the fixes are posted, but I am of the view that these are better managed by the site rather than left open. I hope you can see how having all the Mac CTD Patches in one location, without interruptions or fluff, is a much more user friendly resource than your typical forum thread. But on the other side, only ST Mods/Admins can add/edit posts there, so it means regular users need to get one of us involved to contribute, which isn't always ideal. However, say you did makes some patches, just forward them to us, ideally with links to the originals, then we can ensure they'll be in the relevant thread and that you are properly credited. Since we can actively fix anything on the STEX directly, saving the need for patches and making the process seamless to new users. Of course this doesn't work in the realm of NiteLite ID fixes, something which I firmly believe should only be done on a patch basis.

[CorinaMarie 29,608]

7 hours ago, rsc204 said:

I would like to see one stickied thread, which explains the problems both of NiteLights and the CTD issues, where people can report them or request Mac patches.

We currently have 3 stickied in the SC4 Mac Users forum so that would work well without being clutter.

As you are both completely knowledgeable about the Mac related issues and on staff to have permissions to create such a thread, feel free to take the bull by the horns if you have time.

-Cori & CB

[jonnyquest 71]

Just an update on my end, I've been able to modify some transit lots to fix the CTD issue. I still need to wrap my head around how the nitelight thing works but I want to start doing that asap. A little work on the front-end to get the ball rolling on something like this but I imagine this could be a fairly popular project once we do.

One question that just occurred to me: would I need to bulldoze any affected buildings in my cities after we fix the nitelight thing in their files and re-plop or allow them to re-grow to actually reflect the fix? I'd be slightly annoyed to have to demolish a bunch of skyscrapers etc although I suppose it could be done gradually

[CorinaMarie 29,608]

13 minutes ago, jonnyquest said:

I still need to wrap my head around how the nitelight thing works but I want to start doing that asap.

I scrolled up to see if this was linked before and didn't notice it: Night Lights on Mac

@rsc204 explains the complete process in that thread. I was able to follow along and I did one tho it's been long enough ago I've completely forgotten the exact steps.

[rsc204 21,808]

18 hours ago, jonnyquest said:

would I need to bulldoze any affected buildings in my cities after we fix the nitelight thing in their files

Thankfully no, the fix is dynamic, after all such additional textures must already exist as part of the model, most but not all modellers add some form of NiteLighting.

Every single object (asset) used by SC4, must have a unique ID, this is how the game references everything. Prior to the last official patch, niteliteing only gets triggered for models where one specific Group ID (0x1ABE787D) is used, the one all of Maxis' content used in fact. This patch unlocked the ability to utilise over 4 billion unique IDs, which is tied in with how the SC4 BAT tool, used to make models to SC4 standards, ID's new models.

So the process of forcing a model's nitelights to be 'switched on', is essentially just altering the ID such that it falls under the same scheme as Maxis used. That's why these 'fixes' are really only recommended for Mac users, since it greatly increases the odds for conflicts and there could conceivably be issues that pop up, since this hasn't been tested beyond a proof of concept. But it shouldn't be anything worse than having to switch to another ID or ID range that does work, the game's code reserves certain resources in ways we can't know, for sure with other objects certain IDs are off limits. Of course it may not turn out to be an issue at all, but it's something worthy of keeping in mind. Some testing would certainly be a good idea, although the ID ranges filtered out any ID's we know Maxis used from the SimCity_# dats and related files at least.