So that theory that we knew what causes Prop Pox...

[rsc204 21,836]

Look at what I just got:

Yes our old friend Prop Pox has come to visit the creatively named A1 city of my current test region. So I should clarify that whilst I am using SC4Fix, I have deliberately stuck with the R3 version, which does NOT have the Prop Pox fix included. The reason being, I wanted this to happen, for many years I've wondered if without forcing things, I could 'catch' the Pox and this evening I got it... or did I?

There is a problem and a big one, my last backup was made on the 13th Feb and I had not actually saved this city since April last year either. However by looking at my previous save file, you can clearly see the Prop Subfile had already been compressed:

Why is this important?, well we were told the cause of Prop Pox was data written to the save file in an improper format. This data doesn't cause a problem until this particular subfile is compressed, at which point any data that was written in places it shouldn't be, gets reordered such that the data becomes corrupted. Well that didn't happen here, 100% for certain, because if the Prop file was compressed, somehow the 'error' has uncompressed the previously compressed prop subfile? That to my knowledge, isn't supposed to be possible?

Anything from here out is conjecture until such times as I can really dig into the problem, but let's start with what I do know:

• There is no way any props (or flora) have been altered between RKT1 and RKT4 (or other RKT) states. Everything I'm doing involves new, unique, ID systems, there are literally no changes I made in Plugins that fall under the previous explanation of the problem.
• Every other developed city in the region has a compressed Prop Subfile (5 pretty heavily developed large tiles), none of them show Prop Pox. This includes the one I used to test everything in the PEG CDK3 collection last year, which includes a fixed version of the Beach Development Kit, the file originally fingered as the cause.
• The only changes I've made to the Plugins is to add a bunch of T21s as per recent postings (OK, a lot of them - up from 13k to 122k).
• The only changes to my Plugins folder were adding the Flora, Flora Families and T21s, most of which is tried and tested stuff.
• Maxis used Flora Families and they used them on T21s too, so I'm pretty sure what I've done isn't unintended stuff. Although it's possible using RKT4 flora with many more Reps is itself a part of the problem.

But honestly, I suspect there may be some upper limit on the number of objects, in a very quick time the number of Props has grown dramatically and that's when the problems started. I was noticing some terrible performance issues, especially in Zooms 1-3 earlier, I figured that was the T21 mod. So I went and added the ApperanceZoomFlag property to Girafe's signs so they only appear in zoom 4 and 5. I then added this property to all the trees for the mod, so they only appear in zooms 3-5. This should have removed any additional overhead in zoom 1 & 2 completely, but it didn't seem to stem the performance problem. I then realised the issues were focused on a certain part of my city, the low density residential area to the North, when I was elsewhere on the map, things quickly returned to normal for what is a sparsely modded region. The city in question is just shy of 200k residents and maybe 150k jobs, but one of the smaller ones, it was the founding city in the region.

Now of course, the addition of a previously unused ApperanceZoomFlag property with existing objects can not be ignored as a potential cause. However, since the difference between my backup and the broken save, is simply that I had yet to add any of these T21s, means I'll be easily able to repeat the process with a save file where that flag has never not been present. If this backup corrupts as well, we could pretty much rule that out, I'll update on that one.

But the one salient fact that is clear, it seems 'Prop Pox' is just another name for file corruption, SC4 can be quite finicky in this regard. As I have always believed, the problem is much less simple that one type of edit, not to mention that the way SC4 'dumps from RAM data into a file', just isn't ideal. We know of many different things that when changed, don't dynamically refresh, which can lead to a multitude of problems. The way I see it, it's quite easy under various scenarios for this data to go wrong and when it does corrupt, the way data is handled can lead to a runaway problem where things get worse. Speaking of which, I'm sure I noticed this pretty early on, but it wasn't long before being in the 'bad' area of my city, started causing instability in addition to the slow down. It was after a few CTDs that I then checked the save file using the recommended tool.

Personally I have the feeling some of this stems from pushing the game to it's limits with the number of props. Whilst I am totally happy to release the files for this region, I seriously doubt anyone could use them due to the heavily customised setups I'm using. The city in question has the census repository, CAL's canal system and some other mods, but cosmetic stuff aside it's mostly vanilla. I'll run some checks on what happens without my Plugins suite present and see what the best way forward is there.

But the main point is that for certain there is more to this bug than has been documented.

[TheMurderousCricket 5,061]

So correct me if I'm wrong but does this mean that SC4Fix is useless? Or it is but it simply fixes just certain type of bugs?

Increasing the number of plugins and objects in a city can indeed slow the game down and statistically increase the chance of something going wrong. Makes sense to me.

I'm just not sure what to think about it... How are we supposed to tell how much is too much?

[Tyberius06 12,209]

Just now, TheMurderousCricket said:

So correct me if I'm wrong but does this mean that SC4Fix is useless? Or it is but it simply fixes just certain type of bugs?

Increasing the number of plugins and objects in a city can indeed slow the game down and statistically increase the chance of something going wrong. Makes sense to me.

I'm just not sure what to think about it... How are we supposed to tell how much is too much?

He did NOT use the version of the SC4Fix which fixes the Prop Pox issue. He used an earlier version of the fix. 

Quote

So I should clarify that whilst I am using SC4Fix, I have deliberately stuck with the R3 version, which does NOT have the Prop Pox fix included. 

The SC4 Fix originally meant to fix the transit enabled lots over network Puzzle Pieces CTD issue. The Prop Pox fix came much later at the begining of 2019 (or end of 2018).

[rsc204 21,836]

1 minute ago, TheMurderousCricket said:

does this mean that SC4Fix is useless?

Not at all, I have to run tests, including seeing if the r7 or later release, which first included the Prop Pox fix can resolve or otherwise prevent the problem. But the importing thing to consider is that I did not have that version, i.e. no protection against Prop Pox.

[redfox85 659]

This actually fits perfectly with a scenario where I suspected an overload in props caused pox in a city (that sc4 fixed). In my case .. I made a giant container port in lot editor (I think I created around 50-60 lots for the port), and did not hollow out the container stacks. I figured there might upwards of 10-15,000 container props in that port... combined with everything else in that large tile (although the tile is about half water atleast). Also ..yes .. that tile does take a while to load (althought I've seen load times go from 15 mins on skylake to 6 minutes on alder lake). Also .. this all reminds me I prolly forgot to hollow out the stacks for a much smaller container port in my current region.

[rsc204 21,836]

Well this is a real bummer:

All I did was take the previous backup, pause on the 1st Sep, then re-click all networks (no small task), to add the T21s which currently only cover streets and avenue. As you can see, a lot less (T21) props this time too, yet I could once again feel real performance issues. Something I can't explain, given I have set everything the T21s do such that they don't display in Zooms 1&2, why is there still such a dramatic performance problem in Zoom3 and lower views? I know from before I set the flag, you could still see trees in Zoom2, whereas the signs are invisible either way in Zoom3 already. So the property is working, something doesn't add up here, because clearly the game is having to work a lot harder.

This time the save file remains compressed, but it was compressed when it broke again, so that's another mystery.

But this really does fit my theory that it is possible to overload a city with too many objects, although that still might be specific in relation to T21s or the use of Flora and/or RKT 4 props. I'm going to switch back to Prop Based trees, albeit with less variety and run some tests, when the T21 prop count goes up, does this consistently cause problems or is this only happening with the Flora.

Let's look at B2, the central city in a 3x3 region of large tiles, also by far the most developed with 510k residents and 340k jobs. Perhaps, due to the higher density of development here, things work out better, but you can see the overall number of props is well in excess of the corrupted file. So clearly it's not as simple as just 'too many props' either. Still much to do, including tests on the 64-bit version and doing the same with the newer SC4Fix installed.

But two things are conclusive:

• You can still corrupt a city after the Prop Subfile is Compressed
• You can break your savefile in a manner that SC4Save deems to be 'Prop Pox', without doing anything that's previously been associated as a cause/trigger.

[Tyberius06 12,209]

What's interesting in your latest post among other things, that it marks the city Prop Poxed, yet it doesn't have any disabled props. In your first post, the poxed city had plenty of disabled props, which is what I had when I had truely poxed cities. Bunch of growing number of disabled props. However your non-poxed city already shows 6 disabled props. That something which I experienced too, though I don't know the reason. 

[Barroco Hispano 26,785]

15 hours ago, redfox85 said:

This actually fits perfectly with a scenario where I suspected an overload in props caused pox in a city (that sc4 fixed). In my case .. I made a giant container port in lot editor (I think I created around 50-60 lots for the port), and did not hollow out the container stacks. I figured there might upwards of 10-15,000 container props in that port... combined with everything else in that large tile (although the tile is about half water atleast). Also ..yes .. that tile does take a while to load (althought I've seen load times go from 15 mins on skylake to 6 minutes on alder lake). Also .. this all reminds me I prolly forgot to hollow out the stacks for a much smaller container port in my current region.

The game slows down because even unseen containers cast a shadow. I decided to include as many props in a building to avoid overloading the game with individual props.

@rsc204 Can you upload a picture of a T21 lot to see how many props the lot has?

[rsc204 21,836]

7 hours ago, Barroco Hispano said:

The game slows down because even unseen containers cast a shadow.

First thing I thought of was turning them to low. If I compare that to a heavily forested city from a seasonal tree controller, also flora, the game suddenly becomes more responsive. In this case it makes no difference whatsoever. 

7 hours ago, Barroco Hispano said:

Can you upload a picture of a T21 lot to see how many props the lot has?

Well they aren’t really lots, but I can tell you most have just one or two items, the tree Flora Family and a light prop. In fact most of what I’m doing is editing Maxis T21s, often not adding to the amount of props you would otherwise have by much but cumulatively it really adds up.

So the T-Junctions have 4 Props, 5 if a light is present, the 4-way intersections have 6 or 8.

Whereas the diagonals shown here comprise of 4 T21s, each with either a single tree or a single light, in other words just a single prop (or Flora) entry.

Sure, I've gone a little overkill with the more elaborate setups like Ave intersections, but nothing too crazy. After all, simply using both LRM and SRM mods does add a lot of props, so would a more traditional T21 mod like the SFBT one, that surely adds as many props as I've tried too.

[rsc204 21,836]

Found some time to look into this further today and honestly I have two big questions I can't definitively answer.

1. Why am I getting disabled props and so many of them, where do these come from?
2. Then what is actually going on with my save file, because with a little help from the updated SC4Fix.DLL, I can tell you for certain a few things. If I update to 1.0.7 and restore my Prop Poxed file, I can continue to save the file without Prop Pox occurring. This seems to remain true for subsequent saves, but it will take some time to see if the disabled props get worse, right now (without developing the city further and saving), they do not.
   Image comparing the poxed city (right) with a subsequent save after installing the updated SC4Fix.DLL:

I can say with some certainty that without SC4 Fix this city quickly becomes so broken you can't even load it any more (the game CTDs). Not that I think it was in dispute, but this absolutely backs up all the other data re: a fix for Prop Pox/Save File corruption, that it can both fix and prevent these issues.

In this specific case I've got two things working for me, first this file is literally the first 'Poxed' save, so the worst has not yet occurred in terms of data destruction. Actually a cursory glance suggests I could keep going with this save using the updated SC4Fix without much trouble. Since I have a backup which is really just before the T21 and Flora Modding that seems to have ended so badly, I also have the option of returning to a completely clean save file.

This brings me back to those disabled Props, what does that really mean and what is causing them? In my view it's not a big leap to think that the corruption is happening somehow as a result of so many Props making the save process potentially unstable. Again very early days for conclusions, but I think the altered method which SC4Fix employs for saving files is correcting for such potential problems, such that everyone should be using this fix, because it could very well save your save files from an otherwise catastrophic fate.

An interesting observation I had comes from the fact that my backup is clean, in that is has 0 disabled props (see first post/2nd image). Yet, when restoring it and removing the T21s/Flora Families from Plugins, a subsequent save was showing 6 disabled props. However this remains stable at 6, so I tried to dig further and removed yet more from Plugins that had changed, now I have this figure at just 1 disabled prop. So it's clear that I've edited something that is behind this, again supporting the theory any change in Plugins (except new files) is just not a good idea. I need to see if I can reset the Plugins to match the state from last Apr (the date of the backup). Because whilst I hadn't in all that time played this particular city, I had been messing about in this region and saving other tiles. If I can work out which files are causing the Disabled Props, maybe I can find the exact cause. But if you notice from the 'saved' city (image above), in doing so we've gone from 24,916 to 39,706 disabled props. Here there were no changes in Plugins, but of course could be a consequence of fixing the problem. Again I need to do mores tests, will this value increase further, does that destabilise things more? Still just starting down this rabbit hole I guess.

[TheMurderousCricket 5,061]

5 hours ago, rsc204 said:

So it's clear that I've edited something that is behind this, again supporting the theory any change in Plugins (except new files) is just not a good idea.

Perhaps there is also a difference in the nature of changes one makes to their Plugin folder?

At one point I decided to completely overhaul my file structure and went on a renaming spree to segregate my content into geographical groups. Many new folder names appeared, many files were moved. I did not notice any adverse effects to the game. I'm not sure how SC4Fix was exactly written but would it be safe to assume that it also prevents data corruption resulting from moving / removing the "DLCs"?

I am also pretty certain that I also have some objects in my cities that are not exactly "healthy". I once edited out the whole park set and I'm pretty positive that I failed to remove some of the existing instances within that set from my cities. Still, I have never noticed anything wrong going with my cities, though I'm not sure if this is a matter of sheer luck or again the SC4Fix's.

[Tyberius06 12,209]

@rsc204

If I recall correctly I got increased number of disabled props, when I bulldozed a massive portion of one of my cities because of network replanning. Many houses and parks etc... with massive amount of props got bulldozed at once and I noticed a higher number of disabled props after this procedure. 

I remember I had a city which got semi-corrupted, almost like it had Prop Pox, but bulldozing the affected lots and replopping/growing them and saving the city did not cause the spread of the disappearing props. Even in the state of with disappeared props from lots, it did not show up as Prop Poxed city. However after the rebuilding and saving etc... it did show an increased amount of disabled props. Well later on this city got poxed, but that's a different story. In short I used a mod which provided bunch of timed props, then an update in that mod was faulty and the timed props were not in the mod anymore. When I noticed the error it got fixed, but when I bulldozed the empty lots and replopped them with the now fixed mod, that's triggered the Prop Pox. IF I did not bulldoze the empty lots I could have been playing with the city forever (I tested it at that time), but when I bulldozed even just one of the affected LOTs and saved the city, with the next entry the Prop Pox started spreading. So I gave up on the city and restarted the whole stuff and a few months later the Prop Pox Fix arrived. I returned to the old version just to fix the empty lots with the fixed lots, but then completely let the old version go. 

Anyway. In both cases the disabled props started showing up, when I started bulldozing a big chunk of the already plopped/grown lots in the cities.

[rsc204 21,836]

On 17/03/2023 at 6:36 PM, TheMurderousCricket said:

I'm not sure how SC4Fix was exactly written but would it be safe to assume that it also prevents data corruption resulting from moving / removing the "DLCs"?

Honestly at this point I wouldn't assume anything, whilst it does appear the fix does more than simply prevent Prop Pox, how far that holds up, especially considering its intended purpose, is not clear. But it alters the way data is saved at it's core and this seems to have wider benefits / protection at first glance.

When you say removing the DLCs, do you mean the official Maxis addons?, there shouldn't be a problem removing them, provided you don't load and then save a city where they have been built, having removed the files. But doing such things regardless of the source of the files being removed has always been considered a bad idea.

23 hours ago, Tyberius06 said:

If I recall correctly I got increased number of disabled props, when I bulldozed a massive portion of one of my cities because of network replanning. Many houses and parks etc... with massive amount of props got bulldozed at once and I noticed a higher number of disabled props after this procedure...

...

Anyway. In both cases the disabled props started showing up, when I started bulldozing a big chunk of the already plopped/grown lots in the cities.

That's the confusing thing, because there does seem to be a link between the huge number of suddenly disabled props and my save file being toast. However, it also seems that such disabled props can be a normal part of the process according to the explanation here. Again it seems it might be better to give things time to settle if you delete or alter a large part of a city, before you actually save it.

But the question remains, are disabled props actually bad or just wasteful in terms of space? If no, then were the disabled props in my city just a coincidence or side effect of a different issue? Or could it be that beyond a point, disabled props can actually be self-destructive to the save data? SC4Fix doesn't seem to alter anything in regards to such disabled props, so it certainly isn't needed to restore a working save file.

One interesting note, I realised that having followed the Prop Pox fix instructions for my save, all the low density houses now have for sale signs on the lawn, literally I can't find one without it. So whatever happened, my sims are obviously not happy about it . But I wonder if this is just an unintended side effect, I honestly can't ever recall seeing such signs before? But switching between Low and High City Detail seems to have made them a new feature of 'let's get the hell outta here as fast as we can-land', as my sims now nicknamed the city. That said, I'm not seeing any negative effects in terms of demand and actual occupation, but it is a bit annoying.

I won't get too into this, because I absolutely intend to restore the last working backup and use that moving forwards for peace of mind.

[TheMurderousCricket 5,061]

16 hours ago, rsc204 said:

When you say removing the DLCs, do you mean the official Maxis addons?

Ah, no. I meant all the custom plugins. 

These sale signs really do look suspicious. Come to think of it, I have never seen any myself.

[rsc204 21,836]

51 minutes ago, TheMurderousCricket said:

Ah, no. I meant all the custom plugins. 

Really, really bad idea, once something is in your Plugins folder unless you can be certain it isn't in any save file, it should neither be removed nor altered. If you don't adhere to this rule, then you are opening up Pandora's 'Poxed' Box to some extent.

Personally I have two types of regions, those I am using to develop content and test things like new NAM features. Then I have those I use for serious playing (well I used to, I hope to actually play the game again sometime), the Plugins for which I do my best not to alter. But in much the same way, if you are playing a heavily modded game and messing with things, you really don't want to do that in cherished cities. Be certain you want a thing and that it's working how you want it, before you add it to your Plugins folder.

[CorinaMarie 29,772]

Wow! And here I thought this was eradicated with the timed beach umbrella (and such) props fixes.

 

On 3/18/2023 at 11:55 AM, rsc204 said:

all the low density houses now have for sale signs on the lawn

I'm sure you already know it's Prop: 6534284a-c977c536-1ee40000 with property 0x6A5E0727 (Active Lot State) set to 0x04 (Vacant).

And from Ingred.ini:

; Lot states for props (Active Lot State 0x6A5E0727)
[Uint8ValueMap_0x6a5e0727]
0=Created
1=Ready
2=Constructing
3=Occupied
4=Vacant
5=Destructing
6=Destroyed

0x04 Vacant also includes the two (or is it three?) levels of Distress. You can go to any of your low density residential areas and make them mad and these will pop up as the homes turn gray, grayer, and finally black. (Zone a landfill and let it collect some garbage in an all Maxis R$$$ low density area. )

My guess is something about the pox and repair testing somehow altered the recorded Lot State Prop Flag.

Anyhow, this reminded me that not long before I disappeared I was working on creating a version of that prop which would be super tall with the idea one could see them from afar to tell if things were starting to go wrong. I don't have any of my tools atm to continue this experiment, but tweaking (or creating a new) prop is right up your alley and it'd be cool if you had a chance to see if there is any practical use for the concept.

I can imagine it like a skull and crossbones flag with the s3ds as overrides and one could either drop the .dat file in plugins to look around a troubled city, or even just use it all the time as an early warning system.