[Released!] SC4 Backup Checkpoint Tool - SC4Ckpt

[MarkShot 631]

STATUS UPDATE

* I did create 2 work around EXEs depending on whether you want to run as ADMIN or not ADMIN.

* I am waiting to hear back from the individual who reported the problem as he/she encountered it in configuration.

* I have not heard from CB (perhaps he is offgrid).

* As this is not an evolved version of the SC4Chkpt.exe, I think I am going to create a separate STEX entry and a link to the workaround.  This should minimize downloader confusion as the situation is very rare.  I am not sure if I can edit the v1.01 upload text, and may have to wait for an admin's assistance.

The fix you might need is at this link, my business server, if you find yourself as the last person left alive in the world or with electricity and internet service wanting to play SC4.

10/28/22 - An official STEX link is now up for the work around.  Also, it is cross linked in STEX.  No need to bookmark this post.

https://community.simtropolis.com/files/file/35316-domain-fix-for-sc4ckpt/

Again, if you are up and running already, you are FINE.  If you plan to test it out, this is a very unlikely problem that likely only occurs for those using office PCs and/or company laptops.  However, there is a work around, because I don't want to be labeled as a key factor behind inflation.

  Edited October 28, 2022 by MarkShot  
Link replaced with official STEX link

[MarkShot 631]

A good example of how no code is ever wasted.  The following code segment will create career checkpoints when a save is done in Silent Hunter 3.  80% of the code was adapted from existing code here.  Now to begin the testing.  Most of the larger structure not shown has already been validated.

; ----- Begin Processing ---

        AddExtender("WWZIP44I.DLL")

        While WinExist(GameName2)

; ------- Screen Shot Processing -----
         
          ErrorMode(@OFF)
          FileMove ("D:\User\MK\Tmp\SnagItCapture\*.jpg", "E:\FRAPS\ScreenFrap\SH3Fraps\", @False)
          ErrorMode(@CANCEL)
          CurrentDir=DirGet()
          DirChange("E:\FRAPS\ScreenFrap\SH3Fraps\")
          ShotFileName=FileItemize("SNAG-*.jpg")
          If ShotFileName<>"" Then
            Base=StrSub(ShotFileName, 14, 19)
            TimePrefix=TimeYmdHms()
            TimePrefix=StrClean(TimePrefix, ":", "", @False, 1)
            TimePrefix=StrSub(TimePrefix, 1, 14)
            TimePrefix=StrCat(StrSub(TimePrefix, 1, 8), " ", StrSub(TimePrefix, 9, 6))
            NewFileName=StrCat(TimePrefix, ".jpg")
            FileRename(ShotFileName, NewFileName)
          End IF
          DirChange(CurrentDir)

; ------- Check Point Processing -----

          ZIP=@False

          CurrentDir=DirGet()
          TimeNow=TimeYmdHms() ; what's the clock?

          CareerFullPath="E:\Games\SH3-ALEX GAME Data\SH3\data\cfg\Careers"
          DirChange(CareerFullPath)
 
          CareersCount=0
          Careers=DirItemize(E:\Games\SH3-ALEX GAME Data\SH3\data\cfg\Careers) ; get the regions
          CareersCount=ItemCount(Careers, @TAB)

          For I=1 to CareersCount ; scan the careers
            CareerDir=ItemExtract(I, Careers, @Tab)
            CareerDirFull=StrCat(CareerFullPath, "\", CareerDir)

            PatrolsCount=0
            Patrols=DirItemize(CareerDirFull)
            PatrolsCount=ItemCount(Patrols, @TAB)

            For J=1 to PatrolsCount                   ; scan the patrols (we don't need to find the nth patrol
              PatrolDir=ItemExtract(J, Patrols, @Tab) ; just a patrol with has been modified in the last few seconds,
              PatrolDirFull=StrCat(CareerFullPath, "\", PatrolDir, "\") ; a patrol where a save just took place

              TimeLastModified=FileTimeGetEx(PatrolDirFull, 2)
              If TimeDiffSecs(TimeNow, TimeLastModified)<Relative Then ; Did the patrol just update?
                TargetDirFullTemplate=StrCat(CareerDirFull, "\*.*") ; Our Zip path
                Base=TimeNow ; Start forming the checkpoint name
                NewBase1=StrSub(Base, 3, 17)
                NewBase2=StrReplace(NewBase1, ":", "")
                NewBase3=StrCat(CareerDir, " ", StrSub(NewBase2, 1, 6), "-", StrSub(NewBase2, 7, 6))
                NewBase4=StrCat(CareerFullPath, "\", NewBase3)
                ZipPath=StrCat(CareerDirFull,"\*")
                Delay(Stabilize)
                ResultCode=zZipFiles("S", NewBase4, ZipPath, "") ; calling to the zip lib
                ResultStrip=StrSub(ResultCode, 1, 1)
                If !(ResultStrip==0) Then
                  Abort=""
                  Abort=StrCat(Abort, "FATAL ERROR ZIPPING.", @CR)
                  Abort=StrCat(Abort, "ResultCode=", ResultCode, @CR)
                  Abort=StrCat(Abort, "ABORTING NOW!")
                  UserCancel=@False
                  Goto Abort
                Else
                  ZIP=@True
                  Beep
                  Delay(1)
                  Beep
                  Delay(CrossWindow)
                End If
                If ZIP Then Break ; only 1 patrol can update in game
              End If
            Next ; J (Patrol)

            If ZIP Then Break ; one 1 career can update in game

          Next ; I (Career)

          Delay(Interval)

        End While

[MarkShot 631]

Coding is so rewarding.  Repurposed in 3 hours.  No CTDs for me!

I really wanted to set her down on the bottom, but I don't think Herr Doenitz would appreciate if I bring back to France with the paint scratched.  Test drives are so restrictive!

[MarkShot 631]

The official STEX work around for domain servers has been published with help from CB while preserving the sanctity of Cori Commemoration, v1.01.  Thanks, CB!

There is an important lessons from the above.  Doing this for myself was 4-6 hours and doing this for distribution properly was 4-7 days.

* Being to able to code is very cost effective skill when it comes to your own quality of life in the world.  Your personal productivity 10X greater than commercial development.

* I have known indie game developers and 2/3s are self taught.  This is one reason why they need EARLY ACCESS on Steam, because they don't have the training to deliver what they promise on their own.  When involving (time and money) with new game projects, you should really ask people their backgrounds.  I was an investor in AGEOD, and our main programmer, Philippe Malacher, had formal training and a CV with the French State hospital system.  His coding was superb.  We had very well thought out UI, and he had modular a design that allowed new titles feature releases to be backported to older games (frequently - 3 months at no charge).  Further he fully knew our development IDE DELPHI had poor performance, but was high productivity; ideal for a one man show.

* So, CS graduates especially going BS->MS->PHD become more problematic fitting them into commercial computing environments.  Self taught is easy, if you can get appropriate management and mentorship.  Higher level education is great when doing advanced designs, but often poses deadline problems.  You get what we used to call "Analysis Paralysis".  Each day is a new paper new perspectives and possibilities.  A good solution deployed on schedule beats a superb solution deployed months late.

So, yes, you could say the above was nothing but an exercise in cut & paste programming on the surface, but at deeper levels there were lessons to draw.  (Yes, the sub metaphor/pun was very intentional.)

Gute jagd, Alles!  (Good hunting, Everyone.)

PS:  End of support in '23 does not mean abandonment, just a promise transforms to goodwill.

[MarkShot 631]

I just had my first kill from a convoy with the stern tube, the crew are now war veterans!

It also dawned me that better code is possible for this.  Basically, the algorithm (means computer code with a concept; not something evil about Google or YouTube) asks if the REGION or CAPTAIN/PATROL recently changed, then checkpoint.  The main problem is the use of absolute time windows.  Too small a window, and it fails to detect "recently".  Too large a window, and the same "recently" yields multiple detections.  But I just came up with a fail safe that can use a a very large "recently" window.  The test still uses the system clock, but the filename does not use the clock, but the TLM of the directory.  Before creating the ZIP, you check if one is on disk already.  If it is, then; no need, and there is your fail safe.  A few lines of code (just put it in and tested it for my sub check points).  Maybe SC4Ckpt.exe for Christmas.

[MarkShot 631]

News from an unnamed source at UBoat Research Park in Hamburg

The fail safe timing window solution for checkpoints has led to yet another innovation.  Which is how to deal with the multiple buffer flush problem?

BUFFER = Tech speak for an area of memory which holds data from your game to be flushed to disk as a save.

FLUSH = The act of writing said buffer to disk.

TLM = Time last modified

MULTIPLE BUFFER FLUSH PROBLEM = How to correctly handle TLMs for the first and nth. write

An improved design of FAIL SAFE Checkpointing has been implemented and validated for Silent Hunter III.  Sie fragen "Was ist?" ...

* The first TLM which occurs is sufficient to trigger the checkpoint code path.

* But it is the nth TLM which is actually used to both create the checkpoint and name it (the fail safe).  No partial data captured/corruption.

* Data fidelity is the key value proposition in checkpointing.

REAL WORLD SILENT HUNTER III RESULTS

First, I now understand of what players complain about.  It is not a single patrol or save which gets corrupted, but the entire captain/career.  In SC4, this would be not be the city tile, but the entire region.  Once this happens the captain/career/campaign will not load and crashes the game.

Much superstition has grown among players, as one would imagine over a game released in 2005 which has been to YingYang and back with overhauls mods of overhaul mods:

* Don't save under water.

* Don't save in 40km of another ship.

* Don't save in the external view.

* Don't save until the captain's log credits your tonnage.

* on and on and on .....................

Personally, being in systems I consider that one reason no one has nailed the cause in 15 years:

* It is multi-threaded.  If I have a single threaded 10 statement program.  There are 10 bug states to investigate.  If 2 threads, 100 bug states to investigate.  If 3 threads, 1000 bug states to investigate.  You get the picture.  This would account for all the voodoo and a lot of smart players failing to nail it.

* I do believe the game is multi-threaded.  Why?  It is blazing fast saving and multi-buffered despite the amount of data it needs to save.  The only way there could incur so little delay in game play is multi-threading the save.

* Although this problem occurs to most players, it is neither repeatable nor happens more than 10% of saves.

So, given that the coders are probably collecting Social Security ... it ain't going to be debugged.  Voodoo player beliefs will continue.

What I have is the one click (like SC4Chkpt.exe) tool to easily create  lots of checkpoints.  Some may be corrupted (non-loading), but most are fine.  I have already encountered the dreaded load crash.  A simple unzip worked fine, and all I lost was about 3 Km of surface cruising.

Also, I have saved submerged, next to other ships, next to dead ships I torpedoed ... no problem with unzip/load.

There is another general lesson for software types here.  Porting projects can often lead to significant insights and improvements in code.

Gute jagd, Alles!  (Good hunting, Everyone.)

[MarkShot 631]

EYES ONLY/NO COPY:  U-44 conducting trials of checkpointing in the North Atlantic sent an ENIGMA message that the checkpointing device had simply stopped working.  This message was decrypted and rapidly made its way from the small receiving station on the coast of France to the highest levels in Berlin.

A few days later, another fascinating result is released from Hamburg's Special NTFS Division.  SH3 checkpointing just stopped working like that.  Of course, all weekend leaves were canceled as such is a matter of great urgency during war time.  This is what what the mathematicians and programmers found.

TLM for directories stop auto updating approximately when the directory contains 1,000 file items.  Of course, this breaks the entire checkpoint trigger sequence which is initiated by the player doing a SAVE.

But, it was observed that the same TLM will be pushed up if the directory (patrol directory of saves is scanned).

So, what we have is for small directories, TLM is pushed up automatically, and for large directories it can be forced up by by a directory scan.  A short excerpt of code has gone from theoretical physics to experimental physics of NTFS (Window's file system).  Here is the validated code segment which accounts for:

* Fail safe (time invariant checkpoints)

* Multi-buffer writes

* NTFS big vs small directory handling

I have high lighted some lines which correspond to these new revelations since SC4Ckpt.exe was being coded.

I should note, although working, you will observe that the solution involves practically a total enumeration of all SH3 save files once every second.  Note, a big issue if you have multiple cores, but the inefficiency bothers me.  We may soon be looking for some form of cache heuristics that keeps the scanning to drive TLMs to the parent directory  bounded in some fashion.  For those who not known:

Cache = programming technique to preserve information rather than repeatedly recalculate it.  In this case, we have a form file locality in that there is only one patrol being updated most likely for weeks.  This could reduce work load by one or two orders of magnitude and keep the Arctic from melting.

            For J=1 to PatrolsCount                   ; scan the patrols (we don't need to find the nth patrol
              PatrolDir=ItemExtract(J, Patrols, @Tab) ; just a patrol with has been modified in the last few seconds,
              PatrolDirFull=StrCat(CareerDirFull, "\", PatrolDir) ; a patrol where a save just took place

              DirChange(PatrolDirFull) ; Force update of TLM if patrol has more than 1,000 items
              Delay(1)
              Dummy=DirItemize("*")
              DirChange(CareerFullPath)

              TimeNow=TimeYmdHms() ; what's the clock?
              TimeLastModified=FileTimeGetEx(PatrolDirFull, 2)
              If TimeDiffSecs(TimeNow, TimeLastModified)<Relative Then ; Did the patrol just update?
                Delay(Stabilize)
                TimeLastModified=FileTimeGetEx(PatrolDirFull, 2) ; get the updated time on multiple buffers
                TargetDirFullTemplate=StrCat(CareerDirFull, "\*.*") ; Our Zip path
                Base=TimeLastModified ; Start forming the checkpoint name
                NewBase1=StrSub(Base, 3, 17)
                NewBase2=StrReplace(NewBase1, ":", "")
                NewBase3=StrCat(CareerDir, " ", StrSub(NewBase2, 1, 6), "-", StrSub(NewBase2, 7, 6))
                NewBase4=StrCat(CareerFullPath, "\", NewBase3)
                ZipPath=StrCat(CareerDirFull,"\*")

                CurrentDir=DirGet()
                DirChange(CareerFullPath)
                ZipFileName=StrCat(NewBase3, ".zip")
                If !FileExist(ZipFileName) Then ; Failsafe preventing redundant checkpoints
                  ZipPath=StrCat(CareerDir, "\*")
                  ResultCode=zZipFiles("S R", ZipFileName, ZipPath, "") ; calling to the zip lib

Today's latest discover will not have any immediate on SC4Ckpt.exe as the number files involved in SC4's save structure is fairly minimal.

[MarkShot 631]

Yes, it is amazing where porting code can take you.

Late yesterday, while tweaking/playing/tweaking/playing ... I FOUND THE EXACT BUG THAT ACCOUNTS FOR SAVE CRASHES AND UNLOADABLE CAREERS.  How can I be so sure?  I can reproduce it at will now that I understand it.

It comes down to the game cannot handle more than X number of save records and they must be sequential.  This can be totally prevented by a back ground program a la SC4Ckpt.exe to prevent the game from overflowing its resources while writing checkpoints.  How could as many a million eyes fail to catch this in 15 years?  They were all seeking context dependent causes versus structural causes.  As you know, SC4Ckpt.exe is not really an SC4 utility at heart like the NAM Traffic Manager, but a file system utility (structural).  The implementation was in the structure, not the context.

I am coding for myself, but personally, I don't want spend 4-6 month coding a checkpoint/fix utility for the SH3 Community.  (I am not so selfish.  But my health is precarious, and I want to play ... not do substantial projects and publicize what I found.)

---

I will bury the nugget of knowledge here.  Perhaps some Googler may find it and from there and it will spread ...

Subject:  How to prevent the career load/save crash in Silent Hunter III (SH3) --- base game bug; mod independent ...

Declaration:  This has been tested.  Crashes and resolution of crash are fully reproducible in all contexts.  Submerged, ports, combat, ...

Background:  The bug is not dependent on content of the save like being submerged or close to home port.  The bug is dependent on file system structure.  Each patrol, gets a numbered patrol directory which counts up from 0 (not 1).  Each save generates 10 files.  The root name matches the save for 9 files, but different extensions.  One file has a fixed base name, but save name is saved as part of the suffix.

Cause:  The game becomes unstable when managing careers loads and patrol saves when it is in the area of 100 saves for a single patrol (structural 1000 files in any patrol directory).  This is easy to do if one simply saves 4 times per patrol day during a realistic length patrol.

Solution:  Keep the file count well below 1000 in the patrol directory.  I will do so coordinated with checkpointing such that first 10 saves are always kept along with then the most recent 20.  Thus, the last 20 will operate in a cyclic fashion which should be sufficient to handle any sequence of combat.  Also, the checkpoints (automatic zips) will still cover everything; and allow full recovery to any save.  With just 30 saves kept for a patrol, there will be no more than 300 files for the core patrol directory.  The resources of the SH3.EXE will not be stressed.  If 1000 files is the key number, then why set the limit of saves to just 30?  Because one does not know if the SH3.EXE or one of its popular mods suffers from a member leak.  This provides more head room to the player; just in case.

Sample of the patrol save structure:

Thank you SC4 players and moderators for allowing me to leave this non-SC4 nugget of information.  SH3 (2005) like SC4 is a legacy game.  Hopefully, someone will find this post through the magic of google and write a tool the community greatly needs.  The tools mechanism is not a foregone conclusion.  It could be background like SC4Ckpt.exe or it could be an offline tool.  There are a ton of options; another reason why I don't want to begin to write one for the community.  (The community already has a plethora of tools besides out right excellent overhaul and single mods.)

Thank you for your indulgence.  Future discussion will now return to SC4 after this post about checkpointing REGION/CITY.

[MarkShot 631]

Just a month left in 22.  What a year.

About 9,000 views, and yet but for maybe 10, I don't know you folks.  Yet, perhaps, I can make some educated guesses just based on statistics.

Most of you share a lot with me beyond just gaming.

I labored much of my life, and still have the good fortune to retire.  You may say ... oh "middle class" or "upper middle class".  Not at all I was "working class".  My guess whether you are software engineers, doctors, taxi drivers ... you will be/are/were "working class too".  It matters little if you speak English, Korean, Arabic, German, Ukraine, Russian, Navaho ... we are the salt of the Earth, we produce ... it matters not if you are white, male, female, brown, black, Han, Zulu ...  It matter not whether you pray to Jesus, submit to Allah ... I know you are my brother and sister.

The people who pushed death right up to the Russian border and Chinese coast ... can call themselves Americans, but I share nothing with them.  I worked for a living and did not live off of Raytheon stock nor was part of a government of functionaries rewarded by payoffs or insider trades.  I am not that kind of American; hardly any are.  I regret the "win-lose" education I got ... that means Ukrainians must die for the 1% so they can enact Lebensraum and colonize Russia for resources.  I regret the death; all those deaths in Iraq, Afghanistan, Vietnam, Ukraine ... all around the world supposedly done in my name and with taxes forcibly taken from me.  I regret that Europeans are facing harsh condition and choices for they are no more valuable to the Empire than Ukrainians; just pawns.  I regret streets I have walked in NYC and Philadelphia are full of drugged and hungry homeless.  Billions for weapons every week; nothing to feed real Americans.

Christmas is very special to me.  A dream came true.  I was adopted Christmas Eve 1970.  I hope this Christmas you might reflect on who we are ... there is no reason to hate and kill one another.  Yes, "working class people" can do you great harm, but they do it not for themselves ... we have been raised with hate to preserve the wealth of a small few.  It's time to look beyond.  We near extinction for the obscene wealth of a handful.  This may be our last chance.

Before WWI broke out, it was hoped it could be avoided.  After all what made workers across Europe so unique that they should fill trenches and butcher one another.  Absolutely nothing, but the forces of elitist nationalism and propaganda prevailed and millions were cut down in the prime of life.  It was truly horrible; our first World War.  But the stakes are far higher this time ... my life is near done with no children, but still I don't want to leave knowing extinction is on the way.

2023 ... comes ... think who benefits from the flare up of hostility against one another across the globe.  Who benefited in 2003?  Who benefited in 2008?  Who benefited in 2020 ... the odds say it is not the thousands of you who have dropped by that I don't know.  Remember in almost all our religious/social traditions, it is wrong to take a life or cause suffering.  Don't let the media find an excuse to make you into a monster.  You were born with far more potential than that.

A happy holiday to one and all; make it happen for yourself; make it happen for the rest of humanity.  Giving someone the power to kill or get killed has never created love and harmony.

Become the better angels of our nature.

God bless, you (the 10 and 8990 others)!

[MarkShot 631]

Since this thread has become a show place of all different ways one can code checkpointing game quicksaves, I have come up with yet another new method.

This pertains to Paradoxes' Crusader Kings II.  Note, there is an option for save compression which is nothing more than zipping, and I am using that.  Otherwise, your save is just a huge text file.

I have programmed two sections of code:

(1)  A co-resident utility which time stamps checkpoints with sys time and date.

(2)  A game UI interface that does a quick save with a single key press.

What is the issue?  The writes grows and UI delay grows during the life of the game.  I have settled on 5Mb requires 1 second; and there is always 1 second pad; Which takes care of the case of being below 5Mb to begin with.

So, the co-resident code measures the save and communicates the delay to the UI via an INI.  As a general rule, INI is an easy way to interface different languages running under Windows, since everything handles INI as a standard.  And it is also used in SC4Ckpt.exe to make values persist between runs.

From the co-resident utility which does a lot more than just checkpoint, just a segment of WinBatch (we are using the dirty file bit to catch new saves which are just written; in SC4Ckpt.exe we were using relative times, since I didn't want mess up anyone's ancient backup technology that uses dirty bits).  I have highlighted the INI communication in red.

          If FirstSaveFileName<>"" Then
            ATTR=FileAttrGet(FirstSaveFileName)
            If StrSub(ATTR, 2, 1) == "A" Then
              Base=FileYmdHms(FirstSaveFileName)
              NewBase1=StrSub(Base, 3, 17)
              NewBase2=StrReplace(NewBase1, ":", "")
              NewBase3=StrCat(StrSub(NewBase2, 1, 6), "-", StrSub(NewBase2, 7, 6))
              NewBase4=StrCat(NewBase3,"-")
              NewFileName=StrCat(NewBase4, FirstSaveFileName)

              ; dynamically scale the delay for UI actions
              ActualSize=FileSize(FirstSaveFileName) ; actual bytes
              ScaledSize=Ceiling(ActualSize/1000000) ; MB trunc
              ScaledSize=Floor(ScaledSize/5)+1 ; Seconds needed
              ScaledSize=ScaledSize*1000 ; Millisec needed
              ScaledSize=Int(ScaledSize) ; Back to INT
              IniWritePvt("STATES", "SaveMSec", ScaledSize, CK2INIFile)

              FileRename(FirstSaveFileName, NewFileName)
              FileAttrSet(NewFileName, "rash")
              Beep
              Delay(1)
              Beep
            End If
          End IF

Here is the whole code for the quicksave and delay for the CK2 UI done in AHK:

CK2INIFile:="D:\User\MK\Software\Games\CK2\Utility\CK2.INI"

MouseGetPos, OutputVarX, OutputVarY,

; Menu
MouseMove, 1719, 1062
Sleep, 100
Click
Sleep, 100

; Save Game
MouseMove, 960, 480
Sleep, 100
Click
Sleep, 100

; Confirm
MouseMove, 1015, 774
Sleep, 100
Click
Sleep, 100

; Time to write
IniRead, SleepTime, %CK2INIFile%, States, SaveMSec, 10000
Sleep, %SleepTime%

; Close
MouseMove, 965, 756
Sleep, 100
Click
Sleep, 100

MouseMove, %OutputVarX%, %OutputVarY%,

[MarkShot 631]

A sample of what this yields.  We get both sys and game date/time in the filename.  The game date is the Paradox engine, since I have converted a general save through scripting into a quick save.  Of course, we have the system date/time through attributes, but it is just more convenient to have it in the filename.

[Quasi-Detective 0]

@MarkShot

I'm assuming this project is dead now, but in case it isn't... I dunno if I'm missing something, but when I run it and do all the config, SC4Ckpt just says "Please wait. WinBatch processing..." and does nothing at all. Well, that's not entirely true; at some point it made a single backup of a single region. It didn't do this upon manual save, so I'm not sure when it did it, exactly.

It does not ding at me when I save. It does not ding at me to remind me every 10 minutes to save, as I set it to. I saved again 20 minutes ago and it didn't make another backup. None of its subprocesses even start. "WinBatch" just keeps processing... forever... I don't know what to do. Only thing the program advertised that it actually does for me is kill itself when I close SC4. At this point I may as well just set a timer on my phone to remind me to save. I was really hoping this program would work

[CorinaMarie 29,899]

4 hours ago, Quasi-Detective said:

I'm assuming this project is dead now,

Yeah, very likely as we don't expect Mark to drop back in, but then again, maybe he will.

In the meantime you could check out @Null 45's SC4 Auto Save in the DLL section of the STEX. (I've not personally used this one, but have used several other DLLs they've created.)