SC4 for Mac 64-bit

[Tarkus 6,932]

IIRC, when @memo did the initial work determining what was causing the crashes with the Mac port after it went to Universal Binary, it was primarily due to the encoding on some of the exemplar files in the Additional Bridges.  My initial guess is that there's something similarly obscure afoot here with the files in question, related to whatever Aspyr did in the process of converting the game to 64-bit. 

memo hasn't been particularly active in some time now, and because there pretty much are no mod tools on par with ilive's Reader or Tropod's SC4Reader that work on Mac, he was having to use his own custom (unreleased) tools he built from scratch with his jDBPF library.  I'm not sure this is going to be particularly easy to find without that level of expertise.

-Tarkus

[mmprog 104]

xxdita - regarding the spa St Paul building: it's specifically "CO$$4_1x2_St. Paul's Building Right Corner_e1dadca6.SC4Lot" that causes the CTD. If I load the left corner SC4Lot instead, along with all the SC4Model/SC4Desc files, no crash. Attaching a screen shot showing the two SC4Lot files in Reader.

I also investigated spa's Vogue Building in depth. For that, "CS$$3_1x2_Vogue Building_d02b1795.SC4Lot" causes a CTD. The other SC4Lot file and model/desc files are fine. https://www.sc4devotion.com/csxlex/lex_filedesc.php?lotGET=3081

Tarkus - 
I've been using iLivesReader on my Mac, using WineBottler. It took a bit of tweaking to get it to work though, and it's not working on Catalina, only on Mojave. I haven't yet spent any trying to get it to run on Catalina. But anyway, if it's ok to do so, I can share my copy of iLivesReader with anyone who wants it for a Mac that isn't on Catalina.

And I'm willing to help test or debug things as needed with my machines (I currently have both a Catalina and a Mojave Mac, each with a copy of SC4).

[MI-Native 7]

12 hours ago, mmprog said:

Yeah, the only files I found that cause a CTD are Mass Transit Lots/High Speed Rail Stations/Modern Arched GHSR Station and Modern Arched HSR Station. I can load everything else within z___NAM without a crash.

I did this as well, and mine is working now without crashing to the desktop.  However, small detail that I felt was worth noting.  When trying to run streets diagonally, there appears to be a glitch.  It doesn't show the new street at all, and only shows the ground beneath it.  Anyone else encountering this issue?

[c4bl3fl4m3 405]

3 hours ago, mmprog said:

I've been using iLivesReader on my Mac, using WineBottler. It took a bit of tweaking to get it to work though, and it's not working on Catalina, only on Mojave. I haven't yet spent any trying to get it to run on Catalina. But anyway, if it's ok to do so, I can share my copy of iLivesReader with anyone who wants it for a Mac that isn't on Catalina.

OMG this is a GAMECHANGER. (Can this become its own thread?) We don't have to have Windows users do iLiveReader stuff for us anymore!

Please share your copy of iLiveReader with the community. And/or write a thing up on what you did to get it to work so that other people can do it too.

[c4bl3fl4m3 405]

On 20/02/2020 at 11:28 AM, Tarkus said:

As of yesterday, the App Store version of SimCity 4: Deluxe Edition is now 64-bit, per Aspyr's page .  It doesn't look like the Steam version has been, however, as it's listed under "planned 64-bit updates".  Will be curious to see how it performs.

-Tarkus

Can we get this as a sticky at the top of the post now that it's out?

  Edited February 21, 2020 by Cyclone Boom  
Good idea. I've added it as a Recommended post in the top panel.

[c4bl3fl4m3 405]

FWIW, since there's CTD bugs (AND I have other programs that are 32 bit only), I'm going to hold off on updating to the 64 bit version, at least for now.

Those who have updated, once you get it to stop CTD, how does it run? Any better/faster? Is it worth upgrading to Catalina for? And did they patch the Mac nightlighting & transit bugs?

[MI-Native 7]

8 minutes ago, c4bl3fl4m3 said:

FWIW, since there's CTD bugs (AND I have other programs that are 32 bit only), I'm going to hold off on updating to the 64 bit version, at least for now.

Those who have updated, once you get it to stop CTD, how does it run? Any better/faster? Is it worth upgrading to Catalina for? And did they patch the Mac nightlighting & transit bugs?

I see no difference in game play, but I haven't dived in too deeply either. I posted a follow up upthread regarding how, with NAM installed and the folders causing CTD removed, the streets cannot be built diagonally without glitching out.  Everything else seems okay at first glance.

[Bghgfungerhbfhjhfyy 1]

Based on what I have read this will work with just pathfinding installed? I prefer vanilla SC4

[CorinaMarie 29,605]

3 hours ago, c4bl3fl4m3 said:

OMG this is a GAMECHANGER. (Can this become its own thread?) We don't have to have Windows users do iLiveReader stuff for us anymore!

Please share your copy of iLiveReader with the community. And/or write a thing up on what you did to get it to work so that other people can do it too.

It sounds to me like a similar concept to running Reader with Wine on Linux. In other words, the Reader program itself doesn't need to be changed, but rather the environment in which it is run.

And I agree, a new topic by @mmprog explaining how to set that up on a Mac would be ideal.

 

3 hours ago, c4bl3fl4m3 said:

Can we get this as a sticky at the top of the post now that it's out?

Yep. I see @Cyclone Boom recommended Alex's post so it does show up top.

[MI-Native 7]

1 hour ago, Bghgfungerhbfhjhfyy said:

Based on what I have read this will work with just pathfinding installed? I prefer vanilla SC4

The vanilla 64-bit game seems to work just fine.  You should be good to go.

[ryaboisse 0]

Additional Maxis Buildings not showing up for me since updating. Anyone else have this problem?

[xxdita 1,097]

1 hour ago, Bghgfungerhbfhjhfyy said:

Based on what I have read this will work with just pathfinding installed? I prefer vanilla SC4

So far, the only issues reported regarding NAM have been a couple of stations. You should be fine running the included traffic simulator. 

7 hours ago, mmprog said:

xxdita - regarding the spa St Paul building: it's specifically "CO$$4_1x2_St. Paul's Building Right Corner_e1dadca6.SC4Lot" that causes the CTD. If I load the left corner SC4Lot instead, along with all the SC4Model/SC4Desc files, no crash. Attaching a screen shot showing the two SC4Lot files in Reader.

I also investigated spa's Vogue Building in depth. For that, "CS$$3_1x2_Vogue Building_d02b1795.SC4Lot" causes a CTD. The other SC4Lot file and model/desc files are fine. https://www.sc4devotion.com/csxlex/lex_filedesc.php?lotGET=3081

Tarkus - 
I've been using iLivesReader on my Mac, using WineBottler. It took a bit of tweaking to get it to work though, and it's not working on Catalina, only on Mojave. I haven't yet spent any trying to get it to run on Catalina. But anyway, if it's ok to do so, I can share my copy of iLivesReader with anyone who wants it for a Mac that isn't on Catalina.

And I'm willing to help test or debug things as needed with my machines (I currently have both a Catalina and a Mojave Mac, each with a copy of SC4).

Thanks, but I still don't see any real differences in these lots. I just realized that the lot named left corner is set as right corner, and vice versa, but that shouldn't be causing the CTD. 

There may be a way to further pinpoint the issue, via trial and error, if you're up to it. I've attached a brand new test lot with the only dependency being BSC essentials for the query, and of course the original model file. The test lot is free of all props, and only uses the Maxis textures PIMX assigns as standard, but let's work on function before we worry about making it pretty. 

CO$$4_1x2_St. Paul's Building Right test1_2bd592c7.zip

[mmprog 104]

6 hours ago, CorinaMarie said:

And I agree, a new topic by @mmprog explaining how to set that up on a Mac would be ideal.

I'll see what I can do.

4 hours ago, xxdita said:

There may be a way to further pinpoint the issue, via trial and error, if you're up to it.

Nice, I'll test that lot either tonight or tomorrow.

[mmprog 104]

5 hours ago, xxdita said:

The test lot is free of all props, and only uses the Maxis textures PIMX assigns as standard, but let's work on function before we worry about making it pretty. 

CO$$4_1x2_St. Paul's Building Right test1_2bd592c7.zip

I was able to run the game with this lot, and even got it to grow. Occupancy and all other functionality was as expected. No CTD

[Tarkus 6,932]

@mmprog, that's awesome news on running the Reader via WineBottler.  Being able to have the heavy-hitting mod tools like that work on Macs is absolutely huge, so thank you for sharing your findings there. I'd imagine the Catalina issues are either with WineBottler or the Reader itself.  The Reader is almost certainly a 32-bit app

Based on the results with the test lot, my initial guess is that it's potentially an exemplar for one of the props that might have the issue.  The actual issue memo found that was causing the CTDs with some of the NAM Bridges with the last big Aspyr update was the actual encoding on the exemplars themselves--something which requires hex editing to fix.  He had quite a series of posts on it over at SC4D (some of which, IIRC, are on the NAM Private Board) back in 2014.  I'm going to see if I can find them and provide more details about just what he fixed.

I do actually have a Mac Mini at my disposal--once Aspyr updates the Steam version (I'm not going to pay $20 to buy the App Store version, when I already own the Steam version), I might be able to do some investigation myself (time permitting--that being a huge caveat for me right now).

-Tarkus

Edit: @MI-Native, just how exactly are the Diagonal Streets glitching out?  I'd be curious to see.  Aspyr did actually briefly engage with the NAM Team after the Steam release, so it may be we can make contact with them again if there's something serious afoot.

[MI-Native 7]

4 minutes ago, Tarkus said:

Edit: @MI-Native, just how exactly are the Diagonal Streets glitching out?  I'd be curious to see.  Aspyr did actually briefly engage with the NAM Team after the Steam release, so it may be we can make contact with them again if there's something serious afoot.

So it seems this issue may have been caused but the NAM files I was trying to carry over to the Mac.

I was using older NAM files from a previously saved Windows SC4 game.  For whatever reason, it left out some of the essential .dat files (i.e. the diagonal streets .dat) that caused the aforementioned glitch.

After I re-downloaded fresh NAM 36 on the Windows machine and brought them over to my Mac plugins folder, it resolved the issue with the glitching diagonal streets.

Hope that helps clarify that aspect of this discussion.

[xxdita 1,097]

1 minute ago, Tarkus said:

@mmprog, that's awesome news on running the Reader via WineBottler.  Being able to have the heavy-hitting mod tools like that work on Macs is absolutely huge, so thank you for sharing your findings there. I'd imagine the Catalina issues are either with WineBottler or the Reader itself.  The Reader is almost certainly a 32-bit app

Based on the results with the test lot, my initial guess is that it's potentially an exemplar for one of the props that might have the issue.  The actual issue memo found that was causing the CTDs with some of the NAM Bridges with the last big Aspyr update was the actual encoding on the exemplars themselves--something which requires hex editing to fix.  He had quite a series of posts on it over at SC4D (some of which, IIRC, are on the NAM Private Board) back in 2014.  I'm going to see if I can find them and provide more details about just what he fixed.

I do actually have a Mac Mini at my disposal--once Aspyr updates the Steam version (I'm not going to pay $20 to buy the App Store version, when I already own the Steam version), I might be able to do some investigation myself (time permitting--that being a huge caveat for me right now).

-Tarkus

That's great news. I'll do some digging as well. I'll wear coveralls and everything.  

And yeah, since the fresh lot & desc works, the next logical step would be to add the textures to it, then the props, stopping at the point we can replicate the CTD. Then we'll know where the problem(s) is(are). I would've just made a new prop desc, but hexing is probably better, if memo's notes can provide the key to which hexes need to be changed. 

@mmprog, I'll get the remaining test lots going, and probably add them to a single post. I'll be using the same lot, so you'll need to demolish any prior instances for the changes to take effect. 

[xxdita 1,097]

Ok, so I've found the fix. Now, if I understand what I read, then we can skip over any other testing. 

@mmprog I'm attaching the original spa right corner lot, with the changes memo suggested in the private NAM thread. It's not my post/board to share though, so I'll leave that up to @Tarkus. 

If this works, then I'll be able to correct the Vogue Building just as easily. 

Also, @mmprog, are you using a fresh install of NAM 36? I would recommend trying that, as at least one of the stations you're having problems with were corrected for the same compatibility issues introduced in the original 32bit Aspyr release. 

St. Paul's Building Right Corner-MACfix - hopefully.zip

[Bghgfungerhbfhjhfyy 1]

Currently only OSX Catalina, tried downloading NAM from moddb and working through some of the debugging tips suggested from high sierra, but couldn't get the NAM installer to open yet. 

[mmprog 104]

11 hours ago, xxdita said:

 I'm attaching the original spa right corner lot, with the changes memo suggested in the private NAM thread.

This file works!

[xxdita 1,097]

1 hour ago, mmprog said:

This file works!

Excellent! 

Ok, to put it in English, the MAC version seems to have a stuttering problem with any and all Float 32's with a Rep Count other than 0. 

This has been a known issue for transit stations, and a fairly easy fix, though hasn't caused CTD's. But for some reason, the Float 32's with Rep1 are occurring on these Lot exemplars for all properties related to Slopes. 

0x699B08A4 LotConfigPropertyMinSlopeAllowed

0x88EDC792 LotConfigPropertyMaxSlopeBeforeLotFoundation

0xE99B068C LotConfigPropertyMaxSlopeAllowed

So I'll tag @CorinaMarie & @Cyclone Boom so they'll know what to look for as more issues are reported. I'll also tag @c4bl3fl4m3 so that he is also aware of the issue and the fix. The more awareness there is, the better. Hopefully one of them can get the corrected spa St Paul's Building added to the STEX as well, as I just noticed I seem to be missing buttons. I'm also going to tag @rivit as he may be able to offer a solution to finding problematic lots with DataNode. 

The new lot will be functional on Windows as well as MAC. The one currently on the STEX will cause a CTD for MAC 64bit players, but still work fine for Windows. From everything I've read, these lots will also cause problems for MAC 32bit SC4, but I don't know how severe. 

I have no explanation for how this happens, or what program is causing it. If we could get to the bottom of that, we could maybe fix the problem at the source and be done with it. Going forward at least. 

Let me get a fix going for spa's Vogue Building. That will be updated on the LEX shortly,. And then as long as @Tarkus doesn't mind, I'll start on checking out all lots included with the NAM 36. It should be quick and easy, and me doing it doesn't take away a NAM member from what they're working on. 

[mmprog 104]

Great work @xxdita! I'm not surprised it's a Rep problem that got more severe with the 64-bit change. I've fixed the station capacity bug in a few transit station lots so I'm familiar with the sort of problems unexpected Rep values can cause on macOS.

[xxdita 1,097]

1 minute ago, mmprog said:

Great work @xxdita! I'm not surprised it's a Rep problem that got more severe with the 64-bit change. I've fixed the station capacity bug in a few transit station lots so I'm familiar with the sort of problems unexpected Rep values can cause on macOS.

That's a very good thing. Please remember that if you have a problem with a specific lot, then it's quite likely that all other MAC users will as well. So if you can share these lots as you find them & fix them, that would be a tremendous help for the MAC SC4 Community. 

There are just so very few MAC users that can access the modding tools available. 

[mmprog 104]

2 minutes ago, xxdita said:

So if you can share these lots as you find them & fix them

Yeah, I'll put that on my todo list.

[rivit 3,391]

2 hours ago, xxdita said:

I'm also going to tag @rivit as he may be able to offer a solution to finding problematic lots with DataNode. 

Its a bit of a catch-22 this one. Normally an array property type will have NRreps explicitly coded so it needs to be read after the type, but non array properties assume NReps=0  - as a sort of shorthand and it is interpreted as 1. This means that an entry like FLOAT32, 1, 345.89 will read the 1 as part of a float, which might not be a valid number, and read alignment for the rest of the exemplar is often screwed too. I would say its undefined on Windows exactly what is read.  

However:  there are two stored forms of Exemplar - Text and Binary. In a Text form the NReps is explicitly encoded, in a Binary its assumed. So its possible that a conversion from Text to Binary form - the Packers do this, Reader may too - could inadvertently include the erroneous 1.  

While in fact a check the other way exists - any array variable with rep count 0 is flagged, checking for a NReps after a non-array variable won't work for the 99.999% of all exemplars that are in Binary Form.

@xxditaWhat did DataNode say for the ones you pinned down? 

[xxdita 1,097]

So far, there have been 4 lots reported. 2 NAM stations and 2 of spa's buildings. I've corrected them all on my machine just using Reader. And I've gone through the majority of NAM stations to make sure that no others are affected, but all of this has been in Reader. Now that I know what I'm looking at, I can see the Float32's with Rep=1 for all 3 Slope related properties, but it doesn't stand out. There seems to be nothing of note for any of the effected lots according to DataNode. 

It was easier to go through and eyeball the NAM stations in DataNode vs Reader, for sure. But even then, I knew the 2 lots that needed to be looked at specifically, so Reader was the faster fix. 

As I said, the 3 Slope-related properties are all Float32 and all have a Rep Count of 1, as opposed to Rep 0. spa's lots are right corner or left corner, and one will work, while the other causes CTD on Macs. 

With only 4 lots total being reported so far, this probably isn't a major problem. But somehow I doubt that only 4 such lots exist. I was hoping there could be an easier way to find them, before they actually became a problem. 

[Cyclone Boom 16,875]

@xxdita

Well done looking into the Mac compatibility issues!

About this:

4 hours ago, xxdita said:

I just noticed I seem to be missing buttons.

Oops. I can explain...

When you didn't reply back to the post where I'd granted you those permissions, 3 weeks later I checked the logs and saw they were unused. As such I presumed this was a burden I was placing on you at the time. Thus I set your account back to normal standard permissions, with the idea to give them back to you if asking.

I've reinstated those tools again now which you can use if you'd like to. It's wonderful when people volunteer to help, as @Tyberius06 along with @c4bl3fl4m3 both agreed and actively assist with fixing things on the STEX. As admins, we believe these extra tools are best given to those who will use them to improve our site and we'd be grateful if that's something you'll help with too. If not though, it's completely fine and no pressure whatsoever.

[xxdita 1,097]

While I certainly won't be taking on the task of the entire STEX, as a BSC Custodian, it seems necessary to be able to make this sort of fix when it presents itself, for our team members that aren't active now. It will also make it easier on me, as quite often BSC files are on both the LEX and STEX. Having an outdated and problematic file on one, when I've fixed the other would just be exhausting. Really, even adding the corrected file without being able to remove the problematic one from circulation completely may present more headaches, but we'll see how that goes. 

Honestly, my main obligations and priorities lay with the LEX and to the files of BSC members. I have no problem helping with tech support on almost any file from around the world, but there is a massive difference in correcting a file that leads to CTD vs lowering the park or landmark effect. So I won't be remodding the STEX willy-nilly, but I will update files that prove to cause real problems. 

[simmaster07 1,220]

On 2/20/2020 at 3:28 AM, Tarkus said:

As of yesterday, the App Store version of SimCity 4: Deluxe Edition is now 64-bit, per Aspyr's page .  It doesn't look like the Steam version has been, however, as it's listed under "planned 64-bit updates".  Will be curious to see how it performs.

-Tarkus

Now this is exciting! And also terrifying. I'm curious if they've taken this opportunity to try adding any other patches (or at least fixing the performance a bit), but I would've also expected a whole bunch of new issues to crop up considering how the game was written 17 years ago with the basic assumption that every computer is 32-bit (or at least 32-bit compatible). And reading the comments so far it looks like that's exactly what happened.

Has the Steam version been updated yet? Might have to boot up my Mac Mini...

[xxdita 1,097]

4 minutes ago, simmaster07 said:

Now this is exciting! And also terrifying. I'm curious if they've taken this opportunity to try adding any other patches (or at least fixing the performance a bit), but I would've also expected a whole bunch of new issues to crop up considering how the game was written 17 years ago with the basic assumption that every computer is 32-bit (or at least 32-bit compatible). And reading the comments so far it looks like that's exactly what happened.

Has the Steam version been updated yet? Might have to boot up my Mac Mini...

I'm not seeing any new bugs reported, just worsening of an old one. 

No word on the Steam version yet. "Sooooooon".