Processor HQ: File Processing Info & Discussion

[Cyclone Boom 17,016]

  Processor HQ

File Processing Info & Discussion

Hi all!

As part of your ST Recovery Team privileges, you have editing access to all STEX files which were lost. By now a number of these have already been restored, but there are still numerous which we need help to restore. Additionally, there might be some existing restorations which need an extra file, or more images, etc. For any file which has been restored and yet to be verified, you're most welcome to edit.

I'll walk you through a series of steps to help with editing STEX files...

Stage A: Preliminary Check

1) First determine what needs restoring:

• One or more downloadable files
• Any files and images
• Only images (no files)

Note: You can use the Restoration Tracker to identify this per file (using the "All Missing" filter).

 

Stage B: For Files (to be downloaded)

1) On a file entry, inside the "File Actions" menu, choose Upload New Version.

2) Keep the checkbox checked for saving the previous version.

3) Add a brief note into the changelog such as: Restored file entry for the Simtropolis Recovery Project.

4) Click Delete on any "0 bytes" files shown attached to the file. These would be old entries representing missing files. By deleting them from there, this ensures broken files won't appear under the Download button.
Important Note: Keep any files which are intact and still worth being available.

5) If needing to restore images as well as files, proceed to Stage C below.

6) Otherwise, click Save to process the new version.

 

Stage C: For Screenshots (images on the file entry)

1) Since you are uploading a new version, you can add images on the same upload page.

2) Click to Delete any images which have a missing main pic (thumbnail only), or for any images not displaying entirely.

3) Upload the new images in place. These can be originals from the upload, or ones you've gathered in game showing the item of content.

4) Once everything is uploaded, click Save to finalise the new version.

 

Stage D: File Description Details

1) This won't always be necessary, but what we're looking for here are major oversights or omissions in a file's description.

2) Click Edit Details and open the editor, which allows you to edit the description.

3) Make any amendments as needed to the description text.

4) You may also replace attachments in this step, by deleting them at the bottom of the editor, and then re-uploading and inserting them in place.

5) Usually there won't be need to correct the title, but the option is there if necessary. Also you may add/remove tags too.

6) Once everything is finalised, click Save to confirm. You can repeat this process as needed, if worth editing again afterwards.

 

Quick-ish Questions

I want to add/replace images on a STEX file. Do I choose to Edit Details or Upload New Version?

If only images are missing, Edit Details is the best option. A few weeks before we lost the old server, I added a new feature which allows managing images via the Edit Details page. This allows replacing images, and adding new ones, without needing the updated timestamp to change.

The advantage of edit details is it becomes a silent edit, which is desirable unless we need to add a physical update to the files available to download.

 

What's the difference between a thumbnail and a main image?

Thumbnails show in the square boxes above a file's description. Main images appear when clicking a thumbnail, where they load in larger form. If a thumbnail only exists on a STEX file, it'll initially look fine. However, the placeholder image will show in place once viewing the main pic.

 

I accidently created a duplicated version. What should I do?

Yikes! That would be an absolute utter disaster.

So in this scenario, there is only one option...
You must quickly panic by running around in circles. Then as fast as you can, flee for the hills!

Spoiler

No worries! Post below in this thread and I (or another staff member) will help tidy it up for you.

 

I might not have much time to help due to RL stuff, but I can upload a few files now and then. Is this acceptable?

Yes, that's completely fine!

In fact, we imagine it's the case for many of you. There is no obligation of minimum needed to help out at all with anything. We appreciate every individual effort, which collectively will combine to completing the overall jigsaw puzzle. Even if you only want to post and discuss the logistics, that's fine too.

 

Is there a timescale for when files must be restored by?

No, there aren't any deadlines here. We also emphasise how speed isn't essential with the restoration process. It's more important to do an accurate job than a fast job, even though we've been amazed at the rapid process already from staff and file authors combined.

So consider this a relaxed do-it-at-your-own-pace type project.

 

Is there any limit to how many files I can restore?

Yes, the exact limit precisely corresponds to the total files which need restoring.

In other words, of the total available, there is no limit at all. The more the merrier.

 

What about elsewhere than the STEX? Is there a way to restore forum image attachments for instance?

There will be in a subsequent phase in the project.

For now our focus is primarily on STEX files, to restore as many as we can within the next few months.

However, attachment images and files will be a key priority. Any of your own attachments you're welcome to individually restore (see your list here). Understandably though, this process would be very tedious to do. As such in due course, my hope is to add controls onto the attachments list which allow you to easily replace files and images, without needing to individually edit items of content (e.g. forum posts or CJs).

----------

I'll stop rambling now. I think that's about everything.

Do reply if you've any questions about the above (or anything related to uploading content).

Thanks!

[Cyclone Boom 17,016]

A little tip for removing message banners... 

 

Using the checkboxes on the right side, and then use the Clear Banner option. They'll then be removed in one click.

[Kel9509 1,476]

The Restoration Tracker knows about missing files prior to February 28, 2026.  

Is there a thread or tracker that we can use to upload files after February 28, 2026?  For instance, @kaimai uploaded some SimGoober W2W relots and I downloaded that on April 15, 2026, right before the site was wiped.  I don't have any pictures of that relot set but I do have the download file.  Is there a way to upload or track those missing files as well?

[CaptCity 4,876]

23 minutes ago, Kel9509 said:

Is there a thread or tracker that we can use to upload files after February 28, 2026?

I've been going through my files and sorting the ones after Feb. 28th into a OneDrive folder similarly to what I did with my other backups that I shared previously. I haven't posted it since I wasn't sure what the plans are to handle those yet. As before, these are mostly as they were uploaded. I do have a habit, at times, of making readmes if they didn't contain one, but I very rarely saved any images.

ST Feb-Apr 2026

P.S...  As with the others, these are only SC4 files, and some are from the 'Found' club postings and not on the general STEX per se...

[Kel9509 1,476]

16 minutes ago, CaptCity said:

I do have a habit, at times, of making readmes is they don't come with one, but I very rarely saved any images.

Yes I noticed that you've been adding readme files to the archives.  It makes it difficult to verify files because the size of the archive is slightly different than the original file and so I need to look much closer.  I've been verifying the files as ok if everything else matches, though.

[CaptCity 4,876]

3 minutes ago, Kel9509 said:

It makes it difficult to verify files because the size of the archive is slightly different than the original file and so I need to look much closer.

Yes, it would do that. especialy if I made a Word doc or PDF rather than just a text file (although that is rare). I usually focus on the date listed as submitted or last updated for the appropriate file. I almost always downloaded the files within a day or two of being uploaded.

[CaptCity 4,876]

Another question rolling around in my head (or just food for thought... or I'm just confused...), are we holding off on removing the banners to make the downloads available until every part of the entries have been replaced? The files, obviously, and the details (information, dependency links, etc.) definitely need to be correct, but I can see trying to get images replaced may be a hold up. I've noticed some banners haven't been removed even after the file is available but maybe not all images are present, so I was wondering at what point should we 'release' the file - when the file has been uploaded and information is verified or when everything is replaced, including images, and it has gone through the verification process? 

Just looking for input... 

[Kel9509 1,476]

8 minutes ago, CaptCity said:

are we holding off on removing the banners to make the downloads available until every part of the entries have been replaced?

I've held off on removing the banner if the images are not there, even for files I've uploaded.  I think that's the intent based on the Verifier HQ thread, which says:

Quote

1) Check all the images at the top. Click them each and use the arrow keys to navigate to them all. Ensure they each load fully.

2) Read the description and look for any missing images. If there are any attachment files, make sure they download. A description which has broken attachments won't prohibit us from verifying files, but if able to fix them we should attempt to do so.

3) Make sure any attachments aren't meant to be the actual files under the Download button. Otherwise we'll need to upload them properly. This applies to anything in the description, or even in the Changelog section.

4) Remove the message banner (if not already). This can be done via the Restoration Tracker under the "File Actions" menu. Alternatively you can remove message banners using the batch controls. Message banners may only be removed once a file has been restored to the entry.

So steps 1 and 2 are image related.  After that, it says to remove the message banner.  Although maybe it can be removed earlier since it also notes that the banner might already be removed.

I certainly did not save any image files for others downloads, although for my own I put each image I upload to the STEX inside my download archive to make it identical to the STEX entry.  I've uploaded some files but unless the images are out there and can be restored somehow, they may need to be recreated.  Maybe in that case we can remove the banner but keep tracking it until it's fully restored?

[CorinaMarie 29,788]

Just my 2¢ for this part.  

As soon as the file is restored we've been removing the banner so peeps can download it. That's what the Delete Me thread was for prior to getting the Team assembled and CB posting his fancy tracker. And that's still fine.

The verifying is a separate pair of steps and it's there that we want to hold off giving the OK if anything is missing.

Edit: It does seem some peeps aren't aware of that other thread to request banner removal. That'll explain ones with a restored file while still having said banner. For those, feel free to nix it.

[CaptCity 4,876]

I certainly hope that we can get things back as close to what they were before, and the tracker is an excellent tool for reaching that goal. I do, on the other hand, think that the image thing might create sort of a bottleneck if we make it a determining factor in 'releasing' the items. While it would definitely be a positive to have the original creator's material available, it may be difficult to get. Getting images for the thumbnails, at least, might be doable by loading content into the game to make new ones. I had to do that a bit in other efforts to restore some older content. And yes, it is easier said than done - maybe we need a new committee...

Anyway, just thinking out loud...

[CaptCity 4,876]

32 minutes ago, CorinaMarie said:

It does seem some peeps aren't aware of that other thread to request banner removal. That'll explain ones with a restored file while still having said banner. For those, feel free to nix it.

Ah, thanks. Just saw your edit. That was part of the reason I raised the question. I wasn't sure what to do if others were providing their own files but maybe hadn't re-posted images.

[Cyclone Boom 17,016]

(By the way, I've moved all our posts here to the Processing thread. This one had no replies in yet, so was feeling lonely!)

 

2 hours ago, Kel9509 said:

Is there a thread or tracker that we can use to upload files after February 28, 2026?

I think initially we can have a new thread where any info about files after Feb 28 can be compiled. These will be the most challenging, since there's no database record of them, and unfortunately the Wayback Machine hasn't archived many ST pages from March or April. A few authors have re-uploaded content since, but we'll have to try and identify any gaps. Perhaps I could develop a slightly different tracker once we have some info.

That'll be a big help @CaptCity for the collection of files you downloaded. That will build strong foundations of a files list.

 

1 hour ago, CaptCity said:

Another question rolling around in my head (or just food for thought... or I'm just confused...), are we holding off on removing the banners to make the downloads available until every part of the entries have been replaced?

Once there are no more missing downloads, at that point it's fine to remove message banners, to make the download button available to click. Images might take longer indeed to fully restore. Longer term the more images we can get added the better, but I think the key is visually communicating what a file is so downloaders are aware. As long as we achieve that with what's seen (for one or several images), that'll be a success I believe.

 

1 hour ago, Kel9509 said:

Maybe in that case we can remove the banner but keep tracking it until it's fully restored?

Yeah, that sounds reasonable. The verification can only be completed once there are no missing downloads or images. As Cori said, the idea with the banners is to prevent people clicking Download and running into File Not Found errors. So this way whenever seeing the Download button, it gives confidence it should work.

 

22 minutes ago, CaptCity said:

I do, on the other hand, think that the image thing might create sort of a bottleneck if we make it a determining factor in 'releasing' the items. While it would definitely be a positive to have the original creator's material available, it may be difficult to get.

Very much so, and we ought to be mindful about that. While it'd be ideal to restore all images we can, the key is presenting file entries in a well enough way for the downloader to fully appreciate the creation prior to downloading. Of course for many very old STEX files, they only have tiny thumbnail images. I believe that was due to the small file size limit at the time, or perhaps something got lost for the originals?

For files which have a main pic but missing thumbnail, it'll be because the Wayback Machine restoration only found one and not both. On entries with a full sized image, I'm inclined to create something to fix those up in batch. So I think for now, best focusing on replacing missing files and full sized images.

 

22 minutes ago, CaptCity said:

I had to do that a bit in other efforts to restore some older content. And yes, it is easier said than done - maybe we need a new committee...

It might be a future project to add better images to older content. But for sure, one thing at a time...

[Cyclone Boom 17,016]

Just to note, the system only allows clearing message banners once all recorded downloadable file entries are restored (excluding images).

Another quick tip for clearing message banners...

Firstly, filter by With Message Banner. This sorts first any which can have banners removed.

Then click the corner checkbox:

 

Then click Clear Banner from:

 

And finally the confirmation of how many will actually be cleared:

 

It sure saves a few clicks this way.

[simmering 2,706]

2 hours ago, Kel9509 said:

By the way, on images is it ok that we start recreating them? 

 

2 hours ago, CorinaMarie said:

Yep. GFI

And if original authors return later and want to restore their own, that's fine too. For now it's all about damage remediation so it's all being done in good faith to be as true to the original intent as possible.

When recreating images, should we not add the recreations as new ones and leave the old placeholder images? That way creators can still see the name of their missing images (unless this is not necessary as the information is kept somewhere else, like in the tracker).

[CorinaMarie 29,788]

12 minutes ago, simmering said:

When recreating images, should we not add the recreations as new ones and leave the old placeholder images?  That way creators can still see the name of their missing images ...

Oh! Now that is a very good point. 

For the verifying integrity, CB has set it so the pics either need replaced exactly or the 0 byte image files removed. The latter will also remove the image file names. So, one option would be to keep the text of the file names. Maybe put them in the change log?

But if we do need them later he's got a spare copy of the database as it was on day one when he and Dirk brought the site back. So from that we could make a table or .csv with the File ID and list of its pics.

[Golan0 279]

Also, how many images we should recreate? Some files have multiple missing images, with some random descriptions. One is enough for now? We can make multiple, but it will make restoring files longer, and usually we don't know what they were originally. 

[CorinaMarie 29,788]

4 minutes ago, Golan0 said:

Also, how many images we should recreate?

This'll be a tough one to decide. 

Let's go with: What views and quantity of images do you feel would showcase their work well? If you met them next week and chatted, would you feel comfy explaining your decisions?

If they included one or two pics and you do the same, that seems justified. If they included eight, I believe providing only one would feel inadequate.

[Golan0 279]

Ok, so something like:

"We assume old images are lost forever. So new ones must be good enough to replace them" rather then "We do some placeholder until old ones are restored"

[CorinaMarie 29,788]

3 minutes ago, Golan0 said:

Ok, so something like:

"We assume old images are lost forever. So new ones must be good enough to replace them" rather then "We do some placeholder until old ones are restored"

Yes. And then we are also hoping the original authors pop in sometime and can provide the pics they used if they want to.

[TogaMasterJohn 711]

2 hours ago, CorinaMarie said:

Let's go with: What views and quantity of images do you feel would showcase their work well? If you met them next week and chatted, would you feel comfy explaining your decisions?

If they included one or two pics and you do the same, that seems justified. If they included eight, I believe providing only one would feel inadequate.

I agree. In fact, I was thinking along the same lines when I re-uploaded @G.F Jay Cosis's Oceanic Metropolis tile set. None of the pictures survived the incident, and it seemed that the Wayback Machine did not save them, either (though, now that I think about it, I didn't think to check! ). Given how much work he had put into the tile set, I wanted to feature as many highlights as I could within a reasonable number of screenshots (I may have gone a little overboard, I don't know). They were taken across the three cities he also included within the package. Taking just a few screenshots didn't seem right, and if my memory serves me right, he submitted five or six detailed high-resolution screenshots... The laptop I was/am doing all of this work on only has a screen capable of 1366x768. So, I did what I thought was best in order to honor and respect both the work in question and the original poster.

I may have rambled a bit here, but hopefully my point comes across!  It's definitely well worth taking the user's work into consideration.

[CaptCity 4,876]

As @Handyman mentioned in the note...

Quote

Dependencies:
This link to this dependency "The Italian Collection: This plugin contains all building graphics."
("https://community.simtropolis.com/files/file/28443-the-italian-collection/") works but the 'Download' button & the "C_Fixed_Files_2020_06_29.zip (132 kB)" link on that page result in "404 Page not found"

The dependency file for this set that had been available here at one point, is no longer accessible through the Download button or the link mentioned in the listing.. Apparently, it was stored externally. I'm guessing it may have been due to the size limitations at the time (it's 236MB))...? I don't see anything referencing the issue in the comments, so maybe it did not survive the restoration...?

I do have the 2.3 version of the Italian Collection that I can upload so that the dependency file will be available, but I wanted to be sure there was not some other issue involved with taking the file off.

[CaptCity 4,876]

Didn't know where else to ask this, but...

Was wondering if we were able to restore those broken attachment links in the forums as we come across them. This question here made me realize that some of us may have those files that had been attached but are now gone. Staff could go in and re-attech them if we want to go that route. Or should we hold off until a formal process is set up. Just wondering...

[CorinaMarie 29,788]

50 minutes ago, CaptCity said:

Was wondering if we were able to restore those broken attachment links in the forums as we come across them.

Yep! Please do.  

And if this ever causes any trouble with your attachments storage quota, send a PM to CB and me and he can prolly write a script to disassociate them from your count (or such).

It gets a little tricky for the next post where it's quoted. To fix that you have to jump thru a few hoops.  

1. Fix the attachment in the original post first.
2. Now quote in the same way as the next person did.
3. Change to Source in the editor.
4. Copy it all (then erase it there).
5. Edit the next post.
6. Change to Source.
7. Replace the borked quote with your copied version.
8. Save.

And if it's a silly long quote when it doesn't need to be, feel free to chop it down to a proper size.

If you aren't comfy with HTML Source Editing, let me know on a case by case basis after you fix the attachment and I'll tweak any quotes which also include the linky.