SOLVED: Alpha Bleeding (White Edges) BAT4MAX 3DS MAX

[Barroco Hispano 26,670]

I will show you the only way to correct in less than two minutes the annoying white edges that cause a weird effect in our BATs 

Anti Bleeding Background:

(Don't put 100% black background, you might end up with some issues while exporting the models).

-

Alpha Bleeding Examples:

If this method works for you please leave a comment 

@korver @Odainsaker @Jasoncw @kellydale2003 @JP Schriefer @Bombardiere @MushyMushy @rsc204 @Aaron Graham @Diego Del Llano @Cyclone Boom @Reddonquixote @gutterclub @Toothless Stitch @mattb325

[JP Schriefer 5,590]

Interesting to have another method. Usually I just change the background color to something near the building's edges color, but as you might think it doesn't always work. I'll test it with an image as you suggested next time I deal with this problem. 

[kellydale2003 2,275]

I do adjust them for the day renders (mainly with trees) but never really thought to apply it for the night renders too (I guess because I never use night and never noticed such an issue). I use a rich black equivalent which is like 99% black/dark green.

Thanks for bringing that to light.

[Jasoncw 11,839]

I've done a test doing what you've described.

To the left is the normal way. It looks good in this case, because it's white teapots against the grey pavers, so the grey matting blends in with the pavers. I probably should have changed the background color to red or something so that it would stand out more.

To the right is following your instructions to replace the background with the .png you provided. In my version of max I didn't have the option for #12 (use system default gamma) and instead had the choice of manually setting it, and "automatic" which is what I went with. I imagine automatic set the gamma to 2.2 since that's what the rest of the gamma settings are set to.

Because the grey matting blends in so well, I took the image above and overlayed the two on top of each other, and then zoomed in and toggled the layers on and off. The black AA pixels on the right corresponded with grey AA pixels on the left. So for my case at least, this just had the effect of replacing the grey matting with black matting.

If you do more testing on your end though maybe we can learn more about it.

[Barroco Hispano 26,670]

 

With this method I managed to reduce the file size by 75.8% 

 

 @Odainsaker @Jasoncw @kellydale2003 @JP Schriefer @Bombardiere @MushyMushy @rsc204 @Aaron Graham @Diego Del Llano @Cyclone Boom @Reddonquixote @gutterclub @Toothless Stitch @mattb325

[Jasoncw 11,839]

You have a dark plane against the dark ground, so making the background dark would make the alpha bleeding blend in better anyway. And it looks like changing the sun direction is making the bottom edge of the plane be in shadow, which makes that edge darker which makes it blend in better with the darker background. 

Can you make the plane bright white, and then put the plane against a bright ground? Then we can see if the alpha bleeding is actually gone, or if it's just black instead.

 

I personally have no idea why the alpha bleeding happens. I've compared the alphas of old BATs where there's no bleeding and new BATs where there is, and there doesn't seem to be a difference. The alpha bleeding happens because of grey alpha pixels but the old BATs don't have correct alphas either.

[rivit 3,391]

1 hour ago, Jasoncw said:

I personally have no idea why the alpha bleeding happens. I've compared the alphas of old BATs where there's no bleeding and new BATs where there is, and there doesn't seem to be a difference. The alpha bleeding happens because of grey alpha pixels but the old BATs don't have correct alphas either.

I've always thought that it depends on how well the LOD texture coordinates are mapped into the Model files - if the S3D texture coordinates are off then you will get bleeding (part pixels) and distortions - not necessarily at the edges but it shows there most. The rendering process produces a given quality, but if the coordinates mapping is slightly off (and just comparing the coordinates in the model with the original is illuminating) then what was clean can now be fringed. I've seen the same effect in automata texture mapping if the feature edges are not spot on.

[Barroco Hispano 26,670]

On 27/1/2019 at 2:27 PM, Jasoncw said:

I personally have no idea why the alpha bleeding happens. 

@rivit

This video perfectly explains why this happens but it is in Spanish.

 

You can also read this article:

https://knowledge.autodesk.com/support/3ds-max/learn-explore/caas/CloudHelp/cloudhelp/2016/ENU/3DSMax/files/GUID-E49782BE-6486-4B9D-929A-7A02535F1829-htm.html

Unfortunately, alpha channels and AntiAliasing do not work correctly in SimCity 4...

[rivit 3,391]

2 hours ago, Goldman Sachs said:

Unfortunately, alpha channels and AntiAliasing do not work correctly in SimCity 4...

Can you explain this please? There are constraints imposed in many parts of the process from model->render->store->retrieve->display

[Jasoncw 11,839]

I mean I know why alpha bleeding happens. But I don't know why it happens with new BATs, but doesn't (very minorly at least) with old BATs, even though the renders are the same. BATs have always had AA edges, even though the game doesn't use AA, but those older BATs rarely had alpha bleeding problems.

On 1/27/2019 at 3:53 PM, rivit said:

I've always thought that it depends on how well the LOD texture coordinates are mapped into the Model files - if the S3D texture coordinates are off then you will get bleeding (part pixels) and distortions - not necessarily at the edges but it shows there most. The rendering process produces a given quality, but if the coordinates mapping is slightly off (and just comparing the coordinates in the model with the original is illuminating) then what was clean can now be fringed. I've seen the same effect in automata texture mapping if the feature edges are not spot on.

If the texture mapping is off in the S3D, wouldn't the entire .FSH be offset? To make the alpha bleeding, you would need the diffuse and alpha to be offset by different amounts, but they should be sharing the same mapping. And if this was a problem it would affect all BATs, even old Gmax BATs which don't have AA edges, and have never even had minor alpha bleeding.

Something about one of the 3ds Max, Mental Ray, or .FSH making updates caused the problem.

But even just looking at the .FSH, after all of those other programs have done their work, the alphas look the same whether they have alpha bleeding or not.

3 hours ago, rivit said:

Can you explain this please? There are constraints imposed in many parts of the process from model->render->store->retrieve->display

I think he means that buildings naturally have AA turned off in the game. If it wasn't for the graphical problems we could just start setting AA to on when we make buildings and the problem would be solved.

[rivit 3,391]

4 hours ago, Jasoncw said:

If the texture mapping is off in the S3D, wouldn't the entire .FSH be offset?

Not necessarily  - I'm thinking more of precision errors in some coords rather than uniform misalignment like you describe. Sometimes we see seams on a model in-game too.

Does the bleeding show in Reader as well as in-Game?.

How the FSH are made can have a pretty big influence in demolishing good antialiasing in the render. When a DXT1 FSH is encoded, any alpha >=0.5 becomes alpha=1 and the rest alpha=0 (or should according to the spec). This is how the cutout works. Any that pixel with alpha=0 will be black afterwards. The color components of the AA are incorporated into the colors of the body of the render, but only three colors (A,B, (A+B)/2) are available to render any 4x4 pixel block that has any alpha=0. This can lead to color errors, and mismatches between the resultant stepped alpha and the original AA in the colors. Below such an example

 

The color (on grey 128) and the alpha were made the same way (AA) in separate layers. The combined result then FSH encoded and then undone. The alpha now has hard edges. You'll need to (download and) zoom to see the artefacts in the color.

Now how this is rendered in a game it will depend on where the coordinate centres are and what the texture filtering is. If inside the boundary centerline you may see some grey fringe, outside the centrelines it might show some black. In anycase rendering against the dominant color will help the FSH quality, but it will still depend on the game engine what happens in-game. 

So coordinates, and the method by which the fsh are made, and the game rendering engine could affect the result.

Traditional MAXIS renders used DXT1 (as above) for Day and DXT3 (16 alpha values) for nights. It looks like the BAT4Max uses DXT3 for both Day and Night (but uses them as if they were DXT1), but all other things being equal this should have led to fewer fringes (beter retention of rendered AA).

In all cases the texture filtering is "nearest" so no interpolation occurs at the edges, but to be frank I'm not certain this has no side-effects if the texture has multiple alpha values (DXT3).

  Edited January 30, 2019 by rivit  
Additional info

[Barroco Hispano 26,670]

@Jasoncw @rivit

this is the alpha channel that I used:

[Jasoncw 11,839]

@rivit

@Goldman Sachs

So DXT3 results in better retention of AA in the alpha (16 alpha values)? And DXT1 makes the alpha purely black and white (2 alpha values)?

And then in the game there are the Alpha Function, Depth Function, Source Blend, and Destination Blend, which control how the game uses the alpha (interpreting the 16 alpha values as either "opaque" or "transparent"). iirc, checking "framebuffer blending" makes grey alpha pixels transparent, making the AA functional, but there's glitches if two models with this setting overlap each other (if this could be fixed, everything would be great).

If the game is only going to use 2 alpha values, opaque or transparent, then why are we using DXT3?

Also, this is a small speculation, but do HD zoom 6 exports use DXT1? That would explain why Goldman Sach's exports look so much better. They still have alpha bleeding. With this new example, zooming in and comparing them, I can see exactly which edge pixel has become "opaque", and I can see that the black matting is showing through just like always. But if it's using DXT1, and if DXT1 has better results, then changing the background color would improve the appearance enough that it would be barely noticeable. If DXT1 has small file sizes it might have been preferred for HD zoom 6.

Quote

Does the bleeding show in Reader as well as in-Game?

The image quality isn't good enough in the reader to be able to tell. The alphas do have grey pixels though.

[rivit 3,391]

21 hours ago, Jasoncw said:

So DXT3 results in better retention of AA in the alpha (16 alpha values)? And DXT1 makes the alpha purely black and white (2 alpha values)?

Yes - the red pickup from 3dsmax (which I presume is BAt4Max) has smoother lines along the edges (but no shadow?) , the SimCity 4 (I presume gmax) is very jaggy by comparison. Also you cant do fades with gmax at all. However if you coulld extend the edge colors in the render beyond the alpha you could improve the FSH more. 

I wonder if changing the texture filtering from Nearest to Bilinear in the S3ds of the Bat4max Model will be enough to improve the in-game render even more.  That wont work for DXT1 though - makes the black edge worse.

21 hours ago, Jasoncw said:

If the game is only going to use 2 alpha values, opaque or transparent, then why are we using DXT3?

The game seems quite capable of using all of the extra alphas - I just don't think the filtering is optimal.  I don't know why DXT3 format was chosen for Bat4MAx but I would hazard better edges and less blocky FSH as some of the color gradients in DarkNight are steep and non-linear. The price for the improved quality is that all the FSH are twice as big.

21 hours ago, Jasoncw said:

HD zoom 6 exports use DXT1?

Don't know - but I think gmax does and Bat4max doesn't.    You can tell immediately in Reader if it tells you the textures have alpha and color its DXT3, else DXT1

  Edited January 31, 2019 by rivit  
tpyos

[Barroco Hispano 26,670]

Unfortunately, this problem only happens in the game...

(I opened the FSH file in photoshop).

[Jasoncw 11,839]

2 hours ago, rivit said:

The game seems quite capable of using all of the extra alphas - I just don't think the filtering is optimal. 

I've never seen the game do this except for when the "Frambuffer Blending" setting is turned on.

Otherwise they always look like...

...which is DXT3 and which has a smooth gradient alpha. The game takes the alpha and then either displays the pixels opaque or transparent. A 60% grey pixel in the alpha doesn't become 60% opaque, it becomes 100% opaque. 20% grey becomes transparent rather than 20% opaque.

[rivit 3,391]

@Jasoncw My apologies - you're quite right - I misinterpreted Goldman Sachs images which I thought were all in-game.

After testing some HD props in-game its clear the game treats model object boundaries as if they had been DXT1 - ie jagged - even if they were DXT3 to start with. Changing the filtering to Bilinear only worked in sizeof zooms and exaggerated the tile seams.

All of which implies the DXT3 Daytime Bat4Max FSH are wasting a lot of space for no benefits. And we are stuck with what we have.

 

[Barroco Hispano 26,670]

@rivit @Jasoncw

I think I found a way to eliminate the black edges.

[Jasoncw 11,839]

Framebuffer blending will fix it, but if you have models overlapping that have that setting on, there's graphical problems.

[Barroco Hispano 26,670]

2 hours ago, Jasoncw said:

Framebuffer blending will fix it, but if you have models overlapping that have that setting on, there's graphical problems.

I have not had any problems so far .

Download:

PICKUP_Dodge_RAM_Red_2019_90_R$$-0x5ad0e817_0x69450acc_0x30000.SC4Model

PICKUP_Dodge_RAM_Red_2019_90_R$$-0x6534284a-0xb88fdcdb-0x694ba41c.SC4Desc

 

If this method does not work, I'm afraid I'm going to leave SimCity 4...  

[Jasoncw 11,839]

!!

So it does work! I plopped it around, as well as one of my own AA BATs (https://community.simtropolis.com/files/file/20064-bsp-the-pylon/)

The problem is that it does interfere with some other objects, but only the wind power plant. It seems like other AA buildings, automata, and animated props are all fine. It also interferes with demolition effects, and smoke effects. And it changes how the building gets highlighted (for example when you're hovering over it to demolish it).

Since your props are short it's unlikely that they would cause any serious problems.

But it would still be worth testing to see if there are any other objects that AA models interfere with.

[thawking 28]

Wait, so @Goldman Sachs, what's the program you're using in the post where you say "I think I found a way to eliminate the black edges"? And ... how does that method work? I'm having a heap of trouble w alpha bleeding on this BAT i'm trying to render, and I'd love to know how to get rid of it. (I've used both of the solutions that Mattb325 describes here, and the problem remains.)

[Barroco Hispano 26,670]

@thawking

If you want to keep a little more detail on the stairs I recommend a minimum of 4 or 16 samples per pixel.

After making the changes do not forget to press "apply" and save.
 

[Raymond7cn 6,772]

Hiya experts, I have a question about background color: @Jasoncw @Barroco Hispano

If i want to apply a HDRI background map, is it possible set the background color at the meantime? 

Besides, Another question is that you can tell from the front cover, why dose the reflection look differently from different perspective?

Thank you for the tips about these.

Sincerely,

-- Raymond

-- Edit --

When i set the HDRI's mapping to "screen", The second issue was solved, reflection became to normal in every view. But rendering background became to HDRI map, The white edge is very obvious. Still couldn't figure out how to apply a environment map for reflecting but with a pure color background for rendering.

[Jasoncw 11,839]

I don't know how. The way we export our buildings for SC4 is not the way that renders are normally done. Normally when exporting, you use "Render Elements" to isolate different parts of the rendering, and then you edit them afterwards in Photoshop. So normally it doesn't matter what the background color is because normally if someone wanted it to be different they could just edit it in Photoshop or they could save the files as filetypes with the correct types of transparency for compositing, or they could use different render elements to help them edit the image how they want it.

 

But like I recommended before, if the goal is to give something for upward facing materials to reflect, the best way to do it is to go into "Advanced Rendering Options" of the Arch & Design material of your reflective materials, check "Max Distance" (and give a distance that is bigger than your BAT is. For this car, something like 20 meters would be fine), check "Fade to end color" and put in your sky image there, and then for the sky bitmap it needs to be bright enough to show up in the reflections, which you can do by increasing the "Output Amount" of the bitmap.

This way, you get something for your materials to reflect without changing BAT4Max's standardized lighting. And this way you can also change the background to whatever you'd like, just like you normally would.

[Raymond7cn 6,772]

Dear Jason @Jasoncw, Thank you for your elaborating on this, As you know, I just shifted the render from scanline to mental ray recently, so after learning the Arch & Design from Autodesk, i couldn't find out other way to solve it, so indeed i'm still using the reflection of "Advanced Rendering Options" method you taught me before, Thank you !

But i have run into another problem, it almost drove me crazy these days, Looking forward your tips there in your thread.

Sincerely,

-- Raymond

[Barroco Hispano 26,670]

@Raymond7cnI never use hdris in mental ray 

[Raymond7cn 6,772]

2 hours ago, Barroco Hispano said:

@Raymond7cnI never use hdris in mental ray 

By the way, this is my car material library:

Cars_Materials.mat

Thank you Alejo, HDRI is really hard to handle with, Frankly, i thought i worried too much about this, since those details would hard to be observed in SC4, LOL, I will go to learn your magic car painting.

[Barroco Hispano 26,670]

@Raymond7cn Chrome is one of the hardest materials to make it look good on large surfaces, but on small cars I think the material I sent you ("Chrome_Principal") is pretty decent.

by the way, you must use a 0.1m ambient occlusion on the chrome material because I set it 4.0m by mistake (4.0m AO for the chrome parts of a small car is totally unnecessary).

[Jasoncw 11,839]

A 0.1m radius is effectively the same as turning ambient occlusion off.