STEX Restoration Tracker: Info & Discussion

[Cyclone Boom 17,031]

(The above does not represent the current progress. It is a hypothetical representation only!)

  STEX Restoration Tracker

Info & Discussion

Good news! The real-time page to track restoration progress is now (finally) completed and available.

Preview pic:

It can be viewed at the following URL...
https://community.simtropolis.com/restoration/stex/

This topic serves as a mini guide, and also as a place where you can ask questions about it, report any issues, or suggest improvements. It's taken a good couple of weeks to develop this, and during this time we've seen a remarkable number of files already been restored. This is actually a very good thing though, because what the tracker provides is a real-time overview of what's been completed, and what still needs to be done.

The info on the tracker will be updated every 5 minutes, so it'll be an accurate snapshot of the current restoration efforts.

 

Layout Overview

Firstly at the top, there are summary panels which total how many files have been:

• Restored
• Verified
• Still Missing

And then the "Tracked Total" which includes any STEX file uploaded or updated after June 17, 2024. (Which was the date of our last full STEX backup.)

Pretty self-explanatory for those, and then we have several filter options:

STEX Tracker Search & Filter Options

Filter / Control	Description
Author	Search for an individual author's file entries.
Category	Filter by a specific STEX category (the main ones are listed first).
Restored by	A dropdown menu to filter by any files restored by specific staff or volunteers.
Thumbnail	Allows choosing between Small, Medium (default), and Large thumbnail images, which show in the table.
Display	Switch between Full (default), or Compact for the table, which allows fewer columns to appear.
Search	Allows searching for any particular file entries. You can also use the id:# prefix to find a file by its ID number (e.g. entering id:26793 will find the NAM).

"Filter by" Specific Criteria

Sub-Filter	Description
Any Missing	Entries which have anything missing on them (files or images).
Missing Files	Entries with only files missing. This is the default option.
Missing Images	Entries that have only images missing.
Duplicate Files	Lists any entries which are duplicates of others, or have the same file uploaded.
All Restored	Lists all files which have been restored (whether verified or not).
Restored (Unverified)	Lists only files which haven't been verified.
Restored (Verified)	Lists only files which have been verified.
With Message Banner	Entries which have a "file missing" message banner still.
With File Notes	Lists any entries in the tracker which have 1 or more notes added.
With Files Claims	Lists any entries in the tracker which have 1 or more file claims added.
All Tracked Files	Lists the complete selection of entries, corresponding to the overall Tracked Total.

Note: See more info below about File Notes and Claims.

 

Table Columns

Each of the columns are sortable when clicking the headings. This allows viewing entries by most files/images missing, or other combinations.

Here is a summary of the individual columns and what they represent:
 

• File Entry: The title, author, date uploaded, and status text of what's needs restoring.
   
• Files List: Totals of how many files and images there are. These will be colour coded for red if missing, orange for partially missing, and green once everything is orderly and restored. The Notes total will be orange if there are any which need responding to. Click each of these to view a dialog which show the precise files assigned to the entry. This lets us see what's present or still missing.
   
• Found: The total number of File Claims. This represents how many people have a file, who can help with restoring if needed. Clicking this opens up a dialog with a summary of who has claimed what (files and/or images). We may wish to contact any of these people to assist with sharing them.
   
• Files: Indicates with a green checkmark or red crossmark, whether downloadable files for an entry have been restored. If partially restored, it will represent the current live status for how many are available.
   
• Images: Similarly will represent the status of how many screenshots on the entry are valid or missing.
   
• Verified: Whether the file has been approved or not. Shows a green checkmark once confirmed.
   
• Updated: The date when the file has been restored. Hover over to see the precise time.
   
• Restored By: Displays who restored each file. This will only be shown once a file has all entries in place, since we anticipate partial restorations will represent a work in progress. If the author does this, it will say File Author as the name.
  
  Volunteer / Staff Only column:
• Manage: There is a "File Tools" option with a range of functions. This includes copying the file ID number, adding notes, marking files as verified, and clearing message banners. Options will be greyed out if unavailable, with a message shown when hovering over each. There are also checkboxes beside each row, which then allows performing batch actions on applicable file listings.

 

Other Options

There are also a few extras to mention...

• Sticky Header: Allows the table columns to always remain in view when scrolling down. (Default is checked)
• Export CSV: Saves the current filtered data to a CSV file, which allows it to be imported into a spreadsheet. Some of you might be interested in this.
• Activity Log: Opens a dialog window with a list of actions as part of the restoration efforts.
• Reset Filters: Clears any filters you've set or table sorting, so everything returns to the initial display view (filtered by Missing Files).

 

File Notes

On the "Files List" column there is a total of notes for each file.

This allows anyone to add a comment about the file in question. Then staff or anyone part of the ST Recovery Team can "check off" a note to acknowledge what it says. Notes can only be added on files which haven't yet been verified. Also, files can only be verified once all notes have been checked off.

In effect this system is a way to mention any important details about each file. Perhaps there are additional entries which need adding. Maybe there is a Dark Nite version the author originally added. Perhaps something in the description isn't quite correct. It's a way to make anything like this known, so then we can collectively fix up issues as part of the process.

 

File Claims

No, this is not an insurance claim.   (It's a way to express how you have content which can be restored.)

Sometimes certain files might be especially difficult to locate. Maybe you have an older version which is worth restoring to the previous versions. Perhaps you've got saved copies of images which used to be on the STEX file entry, or have taken some screenshots yourself.

For any files you might have locally and wish to help restore, feel free to submit a claim.

Upon clicking the button:

It'll open up a dialog like this:

Then check any files you might have, and click the Confirm button. Even files which are "Present" you can check too, since sometimes there might only be lower resolution images or thumbnails. Once doing this, your username will be added to a list at the bottom.

You can always return to the dialog and uncheck any files, to then remove your claim to have them.

 

Verifications

What does it involve for a file to be Verified?

It means 2 people as part of the ST Recovery Team (including staff) have checked over to make sure a file has been restored properly. We'll be following a process to do this the best we can, which adds assurance to the restoration efforts. A file which has been verified means at least 2 people have downloaded all files on an entry, and checked out to ensure everything is in tip-top shape.

In the event a file still needs something addressing after being verified, please let us know and we'll take another look.

----------

So that concludes a brief overview of the Restoration Tracker. Given the first phase of the project is focused around the STEX specifically, for now we're only keeping track of missing file entries. In time I'll expand this for an Attachments Tracker and perhaps other site areas like the Gallery, etc.

Feel free to reply with any comments or questions you may have, and I hope this will be a useful resource for the restoration efforts.

[CaptCity 4,893]

Wow, this is way beyond what was expected.   It will certain help the process proceed smoothly. 

Thanks for an incredible tool! 

[CorinaMarie 29,857]

All volunteers have been added to the new Recovery Team user group and given permissions based on what y'all said you'd like to be involved with. (Ofc, if we missed anyone or any others would like to pitch in, just let us know.)

Here's a fancy spreadsheet CB created showing everyone in the new group:

Note: Staff aren't listed since they already have moderator tools and can help out as they wish.

All y'all now have this shiny new user group badge:

[Kel9509 1,476]

Wow.  This looks really impressive but more importantly it might make it very easy to restore everything.  I've been holding off reuploading my items but I'll try to do it this weekend (been a bit busy)!  

[simmering 2,723]

Is there a post explaining how to add a file to the existing latest version (i.e. update it)? It seems I am creating new versions with the same version number.

[Cyclone Boom 17,031]

31 minutes ago, simmering said:

Is there a post explaining how to add a file to the existing latest version (i.e. update it)? It seems I am creating new versions with the same version number.

I've posted a general overview in the File Processing topic here:

Adding a file would involve clicking "Upload New Version" and proceeding to upload it via the upload form.

If keeping previous files, check they're valid and not 0 bytes, or it would still retain broken files which we don't want under the Download button. If unsure about which to keep, feel free to ask in the Processing thread, or add a File Note onto the file in question (via the tracker).

And no worries about version numbering at this stage. I'll run a cleanup routine which will remove duplicate versions later on.

[Propfam 2,295]

Thanks for trusting me to help the recovery process. But I still really don't know what I'm going to do and how to do it. Handing some STEX mod tools to a group of us needs more communication.

Edit: I'm so illiterate. I'm proud of it. I hate it.

[carlosmarcelo 982]

Hi. I didn't find the original images that I had generate when I added submenu. 

But, fortunately, I had yet the same city, so, that I redid the images and they are attach in this post. 

I sinalized in recovery page too. 

 

  Edited June 14 by carlosmarcelo  
I removed the attached because after I discovered that I self can update the images.

[Ernestmaxis 231]

Same here, how do i restore? Where can i find the files to upload them?

[CaptCity 4,893]

21 hours ago, Propfam said:

But I still really don't know what I'm going to do and how to do it.

 

1 hour ago, Ernestmaxis said:

Same here, how do i restore? Where can i find the files to upload them?

You can read through these posts for what is involved for each of the three tasks...

Sharing....................If you have files that you can share with others

Processing.............If you have access to files to upload

Verifying................Checking to see if restored listings are accurate and complete.

 

[simmering 2,723]

1 hour ago, Ernestmaxis said:

Same here, how do i restore? Where can i find the files to upload them?

The details on how to restore files are described in this post:

 

As for where to find files, there is, to my knowledge, no central repository. I think most, like myself, work from their own downloads, at least for now.

The Calling All Volunteers! thread has post where some members provide a list of their downloads. Some provide public access to downloads:

---------------------------------

EDIT: The Sharer HQ: File Sharing Collaboration thread has links to publicly accessible repositories.

  Edited June 14 by simmering  
update

[Cyclone Boom 17,031]

Great to see all the continued progress being made.

Yes, the 3 main topics should cover things as a starting point, and for any specific questions do let me know.

There are no new "mod tools" involved with this, beyond what the tracker page allows for volunteers and staff. For editing files, it's the same method as doing so for your own STEX files. I hope the page UI is intuitive enough, but I'm still tweaking things here and there. For example, on the Activity Log it now jumps to the file when clicking the links.

To find files by author, the box on the left allows this:

 

I've also just added a new My Alerts feature, with the button on the top right:

This will be highlighted when there are any alerts to view. Alerts can include replies to a File Note, new claims on a file, or when any of your files are fully verified.

[nos.17 5,224]

Quick note, the confirmation dialogs are treating the html as plain text. 

[Cyclone Boom 17,031]

@nos.17 

Thanks, good catch!

I've fixed it now:

And also on a few other confirmation dialogs which likewise weren't rendering the bold formatting.

[Handyman 785]

When I select the "One More Verification" filter, I wonder if their might be a way to filter out the files where I am verifier '1' of '1/2'.
I find I am spending a lot of time checking each file manually.

And when selecting the "Restored (Unverified)" filter, they are mixed with "One More Verification" files.

Also, as mentioned previously, I do need to refresh the tracker page to enact the "Mark Verified" selection, but this is not a major thing.

Overall the Tracker is a great tool. Thank you.

 

[CorinaMarie 29,857]

@Handyman 

It will be a while (prolly a week at the soonest) before CB can look into your suggestions.

(There's a heatwave going on in England where he lives and homes do not have air conditioning. Worse, they were built to trap heat in the upper rooms when that was the more desirable option years and years ago. His comp is upstairs and he is taking refuge downstairs which while also quite hot is less so than the sauna above.)

[CorinaMarie 29,857]

@Handyman 

Good news! There was a bit of a reprieve so CB went upstairs to his comp, cranked up his fan, and added the features you requested.  

He also added a silent auto reload of the tracker (this takes place in the background), so it shouldn't need manually refreshing. Give it a whirl now and let us know.  

[Handyman 785]

@CorinaMarie @Cyclone Boom

Sorry for making you sweat it out, but it works great, saves lots of time.
Excellent work as usual, Thank you.

[Cyclone Boom 17,031]

Thanks everyone for the continued help with restoring STEX content, it's greatly appreciated!

We just reached 90% restored and almost 75% of files are now fully verified:

For the final few dozens of STEX files to restore, the process might slow and that's to be expected. Some files will be harder to locate, and at which point we might need to make community appeals for specific items. Excellent progress made, and it's been even faster than I anticipated.

Next will be a separate previous versions tracker, followed by attachments. I'm hoping to get at least one of these up and running before the next heatwave strikes...

[Handyman 785]

I have begun experiencing broken links to a page authors BAT thread . At first thought it was just a problem with one author (BurroDiablo) as I found 2 such links on their pages, but now I have found a broken BAT thread link and a broken link to "SC4DOT Sign Workshop" on 2 pages by MandelSoft.
This new today, all previously have been fine. Has something changed that could affect them.

Also, the drop down button on the far right of the tracker has all but disappeared. Not a big thing as I can see just enough to click on it.

[CorinaMarie 29,857]

1 minute ago, Handyman said:

Has something changed that could affect them.

CB was indeed tweaking the tracker code last night so it's quite possible that's the cause.  

[Handyman 785]

@CorinaMarie   

I have decided to add a note and verify the pages in the meanwhile.

What do you think?

[CorinaMarie 29,857]

8 minutes ago, Handyman said:

I have decided to add a note and verify the pages in the meanwhile.

My best guess is yes, GFI.  

[Handyman 785]

@CorinaMarie

It appears the link problem may have been a coincidence, as I discovered the links on the MandeSoft pages were actually broken / repairable.

Still not sure about the BurroDiablo links. I did message him and added a note to his files.

[Cyclone Boom 17,031]

16 hours ago, Handyman said:

Also, the drop down button on the far right of the tracker has all but disappeared. Not a big thing as I can see just enough to click on it.

I've fixed this now. As Cori said, it was a bug caused by me modifying the column widths. I've made sure they don't keep changing width between each page of the table now. In the Activity Log, I've added a few extra filters such as "Note Related" which lists all new notes and replies. I figured that might help to check those in one place.

Also I created a new "File Logs" option inside the File Actions menu. That allows viewing all logs from a given file entry.

For the links issue, it might be they were already broken before the site crash. We can always fix those if worthwhile while at it.