GMAX: Anti-Aliasing Render + Post processing

@CorinaMarie do you think there is a possibility to script step 3. of this process?

Entering all S3D files and setting the AlphaFunc value of all FSH files to greater? I think that is something that could be done in a short script but I have no idea of this file format and am a bit short of time to get into this. Additionally my programming skills are rather limited.

However something that I might need weeks for could maybe be done by someone with the right experience in a few hours. Just a thought... or maybe you could hint me to someone how I could approach with this request.

I think @Barroco Hispano would also be highly grateful for a script that could open an sc4model and perform this task. For the larger models there are over 60 FSH files per view per direction, changing all those manually is rather terrible

 

@AP I always end up with a sore wrist. 

1 hour ago, AP said:

@CorinaMarie do you think there is a possibility to script step 3. of this process?

I can answer with a definite maybe.

It would, ofc, require Reader 1.5.4 and then Lua coding. The Script in Reader thread over on Devotion is the main info for this. Member toja (in that topic) found four different functions which are bugged with regard to S3Ds, but none of those seem to be the ones needed for altering the Alpha thinger to "Greater" so that's encouraging.

Besides the standard code of looping thru each entry in a .dat file (which I already understand and have coded my own slightly modified version for exemplars), it looks like we can use s3dmat_GetInfo(block) to get the data with the key elements being the alphafunc and depthfunc. From there we change one or both (not sure on this until I'd read the values) and then set them (or it) to 4 (greater). Next using the s3dmat_SetInfo()  function (passing all the appropriate parameters), set them with the updated value for said "Greater".

The whole shebang would be in a sub loop of the main entries loop anytime the main flag = 2 (S3D) and if it works it'd run in prolly less than a second.

One caveat with 1.5.4 is this should be done in a .dat which does not contain any LText entries as those sometimes get borked with that version of Reader. So, if those are needed they should be added via 0.9.3 after the script and updating is done and saved from 1.5.4

In order to test this, I'd need before and after .dat files which used your method. (Yes, I could prolly do some manually following the tutorial, but I'm sure y'all already have those and would know the after is exactly right.)

Thanks for the prompt reply! And thank you even more for wanting to look into this!

I will prepare some files and send them over to you in a PM! Do the models have to be in .dat files or would it work with the .sc4model files?

2 minutes ago, AP said:

Do the models have to be in .dat files or would it work with the .sc4model files?

Oh, I imagine they can be any file extension. I'm just used to working with .dat files.

 

6 minutes ago, AP said:

I will prepare some files and send them over to you in a PM!

Sounds good. Please add @Cyclone Boom to the PM since we do almost all Lua coding together.

Confirmed that .sc4model files are perfectly fine.

I am, however, confuzzled about something. I've pulled up half a dozen random model files scattered throughout my plugins and in Reader every single one is already set to "greater" for the Alpha Func. Is that because they haven't been run thru the other steps in your guide? In other words is it the anti-aliasing or the transparency steps which causes them to change to "always"?

Back to the script idea: I have code written where it goes thru and sets the Alpha Func for all materials in all S3D entries to any of the acceptable values so I believe we'll be good to go on that once I have before and after sample files to test with and compare to.

 

10 minutes ago, CorinaMarie said:

I am, however, confuzzled about something. I've pulled up half a dozen random model files scattered throughout my plugins and in Reader every single one is already set to "greater" for the Alpha Func. Is that because they haven't been run thru the other steps in your guide? In other words is the the anti-aliasing or the transparency steps which causes them to change to "always"?

That is absolutely correct! The 2nd step sets them to "always" as a side effect which then requires the correctoin of step 3. I am preparing files that are exactly before and after step 3.

Since the number of FSH files per S3D file changer with the size of the model you probably need to read out the number of entries per S3D file first to use that variable to set the length of your loop to change the AlphaFunc parameter. (If this coding works at all like I imagine it).

2 minutes ago, AP said:

(If this coding works at all like I imagine it).

Yep. You imagine it exactly right.

There are three loops in the code. The first one goes thru the entries list (the left hand column visually in Reader) and checks if they are S3D (flag = 2). Then for only those we get the number of FSH (materials list) with count = reader:s3d_Count(s3d, 4) where the 4 is the block number. Then we do the sub loop starting at index 0 going up to the count (minus one). Within that loop we have the third sub loop which goes thru block positions one by one and changes the alphafunc to value 4. For each one changed we then set the new block info. When all are processed in that 3rd nested loop we then call the function to encode the entry and cycle back to the main loop to get the next S3D entry.

@CorinaMarie

--------------------------------------------------------------------------------------

@Barroco Hispano

Thanks. Those are different settings in your Maxis example compared to what @AP sent as a before file (as in all of theirs were already correct except the Alpha Func = greater was set to "always"). No worries tho. If your 2nd pic is how you always want them, I can simply force all 5 check boxes to be enabled and the 4 settings to be those you've listed. If any are already correct, it won't hurt them.

If you have time, send us one of your before files and I'll run my code on it and send it back so you can check it.

Edit: See this post below for an updated v0.09 which leaves zooms 1 & 2 as-is.

Note: Instructions for use are still in this post.
 

Ok. Y'all can cancel sending me additional files as we've got the code ready and it seems to be fine when CB and I test it.

Spoiler

-- Lua S3D Script for AP and Barroco Hispano - v0.07.lua
-- Code by Cori and CB --

function reader.main (this)                               -- This is a requirement and must be named exactly like this.

-- Main loop starts here.
    print("Cori is looking for S3D entries . . .")        -- Announce we've started.
    pos = reader:entries_GetHeadPosition()                -- Set position pointer to top of file entries.
    while pos ~= 0                                        -- Start looping thru each entry one by one.
    do
      -- Entries Loop
      entry, pos = reader:entries_GetNext(pos)            -- Read current Entry.
      flag       = reader:entry_GetFlag(entry)            -- Tells what kind of entry it is. See Annex 1 in "ilive_script.txt".

      if flag == 2                                        -- We only care about flag 2 which is S3D entry.
      then
        -- Let's not go crazy with scratch memory usage.
        if collectgarbage("count") * 1024 > 2147483647    -- This was used when reading large files like SimCity_1.dat. Prolly not needed here.
        then
          collectgarbage("collect")
        end

        s3d    = reader:s3d_Decode(entry)	              -- Make a table of the entire S3D entry.
        count  = reader:s3d_Count(s3d, 4)                 -- How many material IDs are there in the list?

        -- Loop thru each index in the S3D entry.
        for i = 0, count - 1                              -- The index itself starts at zero so increment to one less than the count.
        do
          block  = reader:s3d_GetBlock(s3d, 4, i)         -- We only care about block 4. Each iteration of i is the next index.
          subpos = reader:s3dtext_GetHeadPosition(block)  -- Now get the sub position within that block.
          while subpos ~= 0                               -- Start looping thru these one by one.
          do
            s3dtext, subpos = reader:s3dtext_GetNext(block, subpos) -- This fetches the info for the textures at the current index.
            -- This is the part we are after which is the settings per material ID:
            flags, alphafunc, depthfunc,srcBlendFactor,dstBlendFactor,alphaThreshold = reader:s3dmat_GetInfo(block)

            -- Force these variables as AP and Barroco Hispano want.
            flags          = 59 -- This is actually a bit mask = 111011b and sets all 5 checkboxes enabled
            alphafunc      =  4 -- Value for "greater" (We are changing from "always" code 7.
            srcBlendFactor =  4 -- "source alpha"
            dstBlendFactor =  5 -- "one minus source alpha"

            -- Now set those in this index of this block.
            reader:s3dmat_SetInfo(block, flags, alphafunc, depthfunc,srcBlendFactor, dstBlendFactor,alphaThreshold)
          end -- End of: subpos loop
          reader:s3d_SetBlock(s3d, 4, i, block)           -- We are done in the for loop, so set the full block of info.
        end -- End of: for loop
        reader:s3d_Encode(entry,s3d)                      -- This writes the entire S3D entry.
      end -- End of: if flag == 2
    end -- End of: Entries while loop (by "pos")

    print(" . . Cori made the requested alterations. Be sure to save the file now.")
    reader:refresh()                                      -- Updates everything so it's now safe to manually save the file.

end -- End of function reader.main(this)

-- EOF

 

 

To use this, first open your file that contains all the S3D entries which need updated in Reader 1.5.4. (Note: It's the first linky that is simply "v1.5.4 update" in that post.)

Then do the following:

Step 1: Click the New Script button:

 

Step 2: Paste our code in the big white box:

 

Step 3: Click the little Run button:

 

Step 4: Save the file.

Step 5: Say: Yay!

 

Here's the same code, but as a download if you want it: Lua S3D Script for AP and Barroco Hispano - v0.07.lua

Special thanks to CB for his help testing and code reviewing.

Edit: See this post below for an updated v0.09 which leaves zooms 1 & 2 as-is.

18 hours ago, CorinaMarie said:

Those are different settings in your Maxis example compared to what @AP sent as a before file (as in all of theirs were already correct except the Alpha Func = greater was set to "always"). No worries tho. If your 2nd pic is how you always want them, I can simply force all 5 check boxes to be enabled and the 4 settings to be those you've listed. If any are already correct, it won't hurt them.

I assume this discrepancy comes from the fact that I sent a file that is already processed through step #2 (applying transparency) while Barroco sent a file that is Maxis standard.

I will give this a try as soon as possible and report back!

OMG! I just ran it on a massive file! it seems to all have worked perfectly! This is very exciting! I will run further tests and let you know if I discover any anomalies!

THANK YOU SO MUCH @Cyclone Boom and @CorinaMarie !

@Barroco Hispano I hope it produces the same results when you test it?

Edit: See this post below for an updated v0.09 which leaves zooms 1 & 2 as-is.

Note: Instructions for use are still in this post above.

 

On 10/30/2021 at 3:43 PM, AP said:

OMG! I just ran it on a massive file! it seems to all have worked perfectly!

Woot!

 

On 10/30/2021 at 3:43 PM, AP said:

I will run further tests and let you know if I discover any anomalies!

Sounds good. Let us know.

 

I did notice I'd left out the forced setting of the Depth Func. In all y'all's examples that's already set as "less than or equal", but for completeness I've added that to the force values section of the code. Then I tweaked a wee bit adding a missing parenthesis in a comment and some better spacing in parameters lists.

Here's the new code:

Spoiler

-- Lua S3D Script for AP and Barroco Hispano - v0.08.lua
-- Code by Cori and CB --

function reader.main (this)                               -- This is a requirement and must be named exactly like this.

-- Main loop starts here.
    print("Cori is looking for S3D entries . . .")        -- Announce we've started.
    pos = reader:entries_GetHeadPosition()                -- Set position pointer to top of file entries.
    while pos ~= 0                                        -- Start looping thru each entry one by one.
    do
      -- Entries Loop
      entry, pos = reader:entries_GetNext(pos)            -- Read current Entry.
      flag       = reader:entry_GetFlag(entry)            -- Tells what kind of entry it is. See Annex 1 in "ilive_script.txt".

      if flag == 2                                        -- We only care about flag 2 which is S3D entry.
      then
        -- Let's not go crazy with scratch memory usage.
        if collectgarbage("count") * 1024 > 2147483647    -- This was used when reading large files like SimCity_1.dat. Prolly not needed here.
        then
          collectgarbage("collect")
        end

        s3d    = reader:s3d_Decode(entry)                 -- Make a table of the entire S3D entry.
        count  = reader:s3d_Count(s3d, 4)                 -- How many material IDs are there in the list? (Block 4)

        -- Loop thru each index in the S3D entry.
        for i = 0, count - 1                              -- The index itself starts at zero so increment to one less than the count.
        do
          block  = reader:s3d_GetBlock(s3d, 4, i)         -- We only care about block 4. Each iteration of i is the next index.
          subpos = reader:s3dtext_GetHeadPosition(block)  -- Now get the sub position within that block.
          while subpos ~= 0                               -- Start looping thru these one by one.
          do
            s3dtext, subpos = reader:s3dtext_GetNext(block, subpos) -- This fetches the info for the textures at the current index.
            -- This is the part we are after which is the settings per material ID:
            flags, alphafunc, depthfunc, srcBlendFactor, dstBlendFactor, alphaThreshold = reader:s3dmat_GetInfo(block)

            -- Force these variables as AP and Barroco Hispano want.
            flags          = 59 -- This is actually a bit mask = 111011b and sets all 5 checkboxes enabled
            alphafunc      =  4 -- Value for "greater" (We are changing from "always" code 7.)
            depthfunc      =  3 -- "less than or equal"
            srcBlendFactor =  4 -- "source alpha"
            dstBlendFactor =  5 -- "one minus source alpha"

            -- Now set those in this index of this block.
            reader:s3dmat_SetInfo(block, flags, alphafunc, depthfunc, srcBlendFactor, dstBlendFactor, alphaThreshold)
          end -- End of: subpos loop
          reader:s3d_SetBlock(s3d, 4, i, block)           -- We are done in the for loop, so set the full block of info.
        end -- End of: for loop
        reader:s3d_Encode(entry,s3d)                      -- This writes the entire S3D entry.
      end -- End of: if flag == 2
    end -- End of: Entries while loop (by "pos")

    print(" . . Cori made the requested alterations. Be sure to save the file now.")
    reader:refresh()                                      -- Updates everything so it's now safe to manually save the file.

end -- End of function reader.main(this)

-- EOF

 

 

The v0.07 will likely be completely fine, but please do make this v0.08 your standard to use.

Attached: Lua S3D Script for AP and Barroco Hispano - v0.08.lua

Edit: See this post below for an updated v0.09 which leaves zooms 1 & 2 as-is.

Note: Instructions for use are still in this post above.

@CorinaMarie @AP It works very well!   

I am wondering if it is possible to make the script only change the parameters of the FSH's 400 (Zoom 6 & 5), 300 (Zoom 4) and 200 (Zoom 3).

I have a silly question that has nothing to do with this; Does the number of folders in the plugin folder hurt performance? I like that my BATs have their respective folders to be able to update them better.

 

2 hours ago, Barroco Hispano said:

I am wondering if it is possible to make the script only change the parameters of the FSH's 400 (Zoom 6 & 5), 300 (Zoom 4) and 200 (Zoom 3).

Sure thing. Does this mean leave the others (zooms 1 and 2) completely as-is? Or do they still need all check boxes toggled on? And/or do they need any individual changes to the other four settings?

 

2 hours ago, Barroco Hispano said:

Does the number of folders in the plugin folder hurt performance?

Only very, very minimally as in each folder might add a microsecond or such to the initial game load time, but no further effect once you've arrived at the region view. I'd personally say to keep using all the sub folders you want for better organization like you do.

@CorinaMarie 1. Yes, leave them with the maxis parameters (Maybe a very tall building with our parameters can conflict with the clouds but I cannot confirm it yet). 

2. That's good news for me, anyway, it seems that when you datpacking your plugins the tool "removes" the folders (By remove I mean there are no folders in the .dat file). 

@AP I forgot to warn you never to use our parameters with a BAT that has part of the LOD under water like for example; a rock, since the part that remains under the water will disappear. 

 

On 10/31/2021 at 7:00 PM, Barroco Hispano said:

1. Yes, leave them with the maxis parameters

Cool.

Give this a test run please:

Official Final Version:

-- Lua S3D Script for AP and Barroco Hispano - v0.09.lua
-- Added filter to process only zooms 3, 4, 5/6. (aka only 200s, 300s, and 400s of the Instance ID)
-- Code by Cori and CB --

function reader.main (this)                               -- This is a requirement and must be named exactly like this.

-- Main loop starts here.
    print("Cori is looking for S3D entries . . .")        -- Announce we've started.
    pos = reader:entries_GetHeadPosition()                -- Set position pointer to top of file entries.
    while pos ~= 0                                        -- Start looping thru each entry one by one.
    do
      -- Entries Loop
      entry, pos = reader:entries_GetNext(pos)            -- Read current Entry.
      flag       = reader:entry_GetFlag(entry)            -- Tells what kind of entry it is. See Annex 1 in "ilive_script.txt".
      T, G, I    = reader:entry_GetTGI(entry)             -- Fetch the Type, Group, and Instance IDs.

      -- Next 3 lines convert the T, G, and I to a text string of 8 Hex digits (pre-pending zeroes as needed).
      Hex_T      = s3dDec2Hex(T, 8)                       -- Don't need, but do in case there's a reason to have it later.
      Hex_G      = s3dDec2Hex(G, 8)                       -- Don't need, but do in case there's a reason to have it later.
      Hex_I      = s3dDec2Hex(I, 8)                       -- This is the one we want to check the 3rd digit from the right (position 6).

      -- Here we check if the 6th digit of the Instance ID is 2 or 3 or 4 so we can process only zooms 3, 4, and 5/6.
      if (string.sub(Hex_I, 6, 6) == "2") or (string.sub(Hex_I, 6, 6) == "3") or (string.sub(Hex_I, 6, 6) == "4")
      then
        Do_Process_This_Zoom = true
      else
        Do_Process_This_Zoom = false
      end

      if flag == 2 and Do_Process_This_Zoom               -- We only care about flag 2 which is S3D entry AND zooms 3, 4, 5/6.
      then
        -- Let's not go crazy with scratch memory usage.
        if collectgarbage("count") * 1024 > 2147483647    -- This was used when reading large files like SimCity_1.dat. Prolly not needed here.
        then
          collectgarbage("collect")
        end

        s3d    = reader:s3d_Decode(entry)                 -- Make a table of the entire S3D entry.
        count  = reader:s3d_Count(s3d, 4)                 -- How many material IDs are there in the list? (Block 4)

        -- Loop thru each index in the S3D entry.
        for i = 0, count - 1                              -- The index itself starts at zero so increment to one less than the count.
        do
          block  = reader:s3d_GetBlock(s3d, 4, i)         -- We only care about block 4. Each iteration of i is the next index.
          subpos = reader:s3dtext_GetHeadPosition(block)  -- Now get the sub position within that block.
          while subpos ~= 0                               -- Start looping thru these one by one.
          do
            s3dtext, subpos = reader:s3dtext_GetNext(block, subpos) -- This fetches the info for the textures at the current index.
            -- This is the part we are after which is the settings per material ID:
            flags, alphafunc, depthfunc, srcBlendFactor, dstBlendFactor, alphaThreshold = reader:s3dmat_GetInfo(block)

            -- Force these variables as AP and Barroco Hispano want.
            flags          = 59 -- This is actually a bit mask = 111011b and sets all 5 checkboxes enabled
            alphafunc      =  4 -- Value for "greater" (We are changing from "always" code 7.)
            depthfunc      =  3 -- "less than or equal"
            srcBlendFactor =  4 -- "source alpha"
            dstBlendFactor =  5 -- "one minus source alpha"

            -- Now set those in this index of this block.
            reader:s3dmat_SetInfo(block, flags, alphafunc, depthfunc, srcBlendFactor, dstBlendFactor, alphaThreshold)
          end -- End of: subpos loop
          reader:s3d_SetBlock(s3d, 4, i, block)           -- We are done in the for loop, so set the full block of info.
        end -- End of: for loop
        reader:s3d_Encode(entry,s3d)                      -- This writes the entire S3D entry.
      end -- End of: if flag == 2
    end -- End of: Entries while loop (by "pos")

    print(" . . Cori made the requested alterations. Be sure to save the file now.")
    reader:refresh()                                      -- Updates everything so it's now safe to manually save the file.

end -- End of function reader.main(this)

-- Converts a decimal number to Hex and pre-pends zeroes as padding based on the cmLen parameter (typically 8) passed.
function s3dDec2Hex(IN,cmLen)
    OUT = string.rep("0", (cmLen - (string.len(string.format("%x", IN))))) .. string.format("%x", IN)
    return OUT
end

-- EOF

Note: Instructions for use are still in this post above.

Attached File: Lua S3D Script for AP and Barroco Hispano - v0.09.lua <-- Official Final Version

@CorinaMarie The script works fine! thanks!

@CorinaMarie I subjected your script to a stress test with BATs that have more than 160 FSHs and it worked perfectly! 

Your script makes BATs look a whole lot better but maybe that's just my impression.

2 minutes ago, Barroco Hispano said:

I subjected your script to a stress test with BATs that have more than 160 FSHs and it worked perfectly! 

Woot!

 

3 minutes ago, Barroco Hispano said:

Your script makes BATs look a whole lot better but maybe that's just my impression.

It'll be y'all's technique which does that. All the script does is automate what used to be a tedious manual process.

@AP and @Barroco Hispano

Excellent, we're really pleased the automation script works and glad it'll be a big timesaver for your BAT creations. Cori did the vast majority of the Lua coding, and I just helped with a few little things and for internal testing along the way.

It's the absolute freaking best, I use it every day!  It saves SO much time! I can't thank you enough!

1 hour ago, AP said:

It's the absolute freaking best, I use it every day! 

Woot!

Btw, have you considered updating your tutorial to include instructions for using the script instead of the manual method? (Maybe leave the manual instructions marked as an alternative.) And if you do update it, feel free to use my pics and verbiage if you want.

Here is an example of what the script is capable of doing automatically:

(Focus on the edges of the tree.)

1. Maxis parameters:                                                                             2. Our parameters:

       

On 18.11.2021 at 9:30 PM, CorinaMarie said:

Btw, have you considered updating your tutorial to include instructions for using the script instead of the manual method? (Maybe leave the manual instructions marked as an alternative.)

Yes, that is a good idea! It has been on my list for weeks now, but I just have so many other things going on. I am still planning to do it!

 

On 18.11.2021 at 9:30 PM, CorinaMarie said:

And if you do update it, feel free to use my pics and verbiage if you want.

Thanks for that, that's great!

@CorinaMarie @Cyclone Boom The script modifies something related to the alpha threshold?

I changed the parameters that the script assigned and I set the default parameters of maxis and now the tree has an ugly halo. 

 

 

21 minutes ago, Barroco Hispano said:

The script modifies something related to the alpha threshold?

The script modifies the five check boxes and 4 drop down choices to be as shown in your pic in the green box you drew:

 

So, if any of those are what are being changed and should be left alone, that's an easy fix in the code.

Show me your new test pic of these choices as they really should be and then also what is happening after running the Lua Script so I can see what occurs.

Ok, I found the problem.

It seems that the script sets an alpha threshold of 255 which makes the BAT use the entire alpha channel correctly (Even the thinnest wire can be seen). This parameter works very well with ships or other BATs that do not interfere with others. The problem begins when the LOD of your BAT interferes with others (ex: trees), an alpha threshold of 255 makes the transparency of the edges erase what is behind or it can also make a halo appear with the background color (As you can see in the picture).

This is what it looks like when parameters are assigned manually.

Maxis default parameters + alpha threshold: 32767 (

Maxis default parameters + alpha threshold: 255

I tried to assign the alpha threshold of 33767 manually but for some reason it doesn't save. now I don't know how to fix the trees that the script assigned our parameters to.