Forces of Light and the Geometry of Shadows

[SimFox 804]

Greetings, all simtopians and visitors!

Despite the title, this thread will have nothing to do with Babylon5, no matter how marvelous it was. This will be a forum where technical details of BAT could be discussed. Keeping it here will hopefully let other forums stay clean of techno talk and will make this information easy to find for those who will be interested in it. In 3D modeling Light and Shadows it casts are of crucial importance in recreating believable world. And practically all and every development in the 3D software and especially rendering technology is centered on this pair. So I think it will be fitting to put them in the title of this thread. I’ll try to keep a title list in this first message so that any topic raised here could be easily found. I think most subject discussed here will be relevant mostly for those using MAX to create their BATs, but some information may be relevant to the community at large. Anyway I invite everyone to take part in the discussion and share knowledge and views freely, but respectfully. Remember everyone is entitled to his/her views, but it is a sign of respect to others if you not simply state them but take a time to explain your position. 1. Shadows and the ways to get them. pg.1 right after this message Review of various shadow types, their quality and speed with MAX’s default Scanline rendering engine 2. Brightness of colors. pg. 1 scroll down a bit you'll notice it - it's bright. Test your perseption of this often desputed subject 3. Let there be Light pg. 2 little tutorial on how to create new lighting rig that fixes problems of the default one.

[SimFox 804]

So, first subject will be SHADOWS and different ways to create them.

This is to a degree relevant to all those using GMAX as well, since it gives you choice of ray traced shadows and those generated with shadow maps.
All renders were made with MAX and numbers given are for MAX as well, but I believe it will hold for GMAX too.

SHADOW MAPS

Let’s start with shadow maps. These work as textures and as such they could become HUGE drain on memory. If you keep them at small default values you’ll get something amorphous with fringed borders and often odd splotches.

Shadow Map’s saving grace is the fact that they’re rather easy on CPU. But in today’s world where CPU speed had gone loooong way Shadow Maps hold little if any benefit anymore.

Memory is presently also cheap. But if you want your shadow maps equal in quality ray traced shadows you may realize that limit is set by your OS. X86 version of Windows can handle only so much and can allocate to one given process even less.

 

RAY TRACED SHADOWS

Ray traced shadows work by tracing rays of light source (no surprise here). Process is very CPU intensive, but produces accurate shadow line. Calculations can be speed up by process called Quadtree. I’ll show you effect of this on speed of render later on.

 

AREA SHADOWS (not available on GMAX)

Area shadows are, in fact a subdivision of Raytraced shadow and are SLOWEST of all. Speed and look of area shadows depends on your choice of the shape of the light source and the size you assign to it. The smaller it is the more area shadows will look like regular ray traced shadows.

In reality ALL shadows are Area Shadows. But given the burden they place on CPUs they are often substituted by regular ray traced ones when small and very far away light sources (like the Sun) are emulated.

---

To illustrate that brief info and to show quality vs. time effect here are few renders of common teapot. All renders are done with default Scanline render. And none have any GI what-so-ever. I’ll write about that subject later.

Notice:
1. All pictures are PNG files. I will not use JPG as compression may interfere with picture itself, making comparison pointless. If you can not see then – damp IE and get yourself Firefox!
2. All following is true for Scanline renderer (default option in MAX). Second rendering engine that comes with MAX – Mental Ray has it’s own peculiarities and not considered here.

 

Let start with Shadow Map, the way BAT creates it:

 

This is with default black as a Shadow color. This sort of shadows may be seen on Moon but not on Earth! I will go in to more detail why on next show-and-tell about GI. But MAX has nice little trick – absolutely cheat one, but sweet none the less: you can change shadow color and give them much more natural look:

 

you can clearly see what is wrong with this type of shadows. This result you get with default value of 512 for the shadow map size. At this size your memory usage for rendering this scene is about 2,5Mb. And quality you can see from the picture. Render time is 1 sec.

Here is for a comparison how same scene looks with Ray traced shadow:

 

this is done with default settings for this particular shadow type. Render time is 21 sec. Memory usage is little under 1 Mb.

 

With Shadow Map Increasing map size leads to better quality , but also higher memory usage.
With map size of 4096 Shadow mapped shadow is still far away from quality of Ray traced one:

 

but memory usage rises to 67Mb.

 

Raising Shadow Map size to 8192 brings us to some approximation of the quality of ray traced shadow

 

but it requires whopping 264Mb of memory!!! And this is PER light source!

 

Reading all this you may ask ‘Why is any of this important? Why should I care?’ Well this sloppy nature of Shadow Maps is the reason for that peculiar situation when Roof junk appears to float in mid air in stead of being firmly put on the roof. Decorative elements on walls look like they are meters apart from those, or at all placed at the different height then in reality.

 

 

Max Quadtree Depth Parameter.

In Ray Traced Shadow parameters you’ll find dial for Max Quadtree Depth. This is used to speed up calculations of Ray traced shadows. It is as I gather some sort of interpolation method. Default value is 7. Dialing it down to 1 will increase render time to 12min 25sec – 745sec !!! But dialing this up to 10 – max value – will bring render timer back to the same 2 sec as with shadow map. Theoretically quality and precision of shadow may suffer, however on our teapot result is indistinguishable.

Quadtree 1

 

 

Quadtree 10

 

 

So we have best of both worlds – speed of Shadow map and memory consumption of a ray trace.

 

Area Shadows.

Area Shadows produce effect commonly seen in real world – shadows borders get more blur and washed out the farther the shadow is from the point of origin. In fact all shadows we see around us are Area Shadows.
Amount of this effect depends in MAX on size and shape of the light source. For sun it would be natural to chose sphere or a disk.

Area shadow of a sphere of size of 10 with other settings on default will produce this image at 27sec. Memory usage is 2mb

 

Dialing size down to 5x5x5m makes shadow border more defined and reduce render time to 20 sec.

 

 

Changing shape of light source to the disk. Affect shadows quite a bit:

 

Boarders of the shadow are much more blurred then with Sphere, but this blur is coming with significant noise. Render time is 12 sec. To fix this noise we can dial up Shadow Integrity value from default 2 to 5. Doing so gives us this result:

At the same time render time goes from 12 sec to 50 sec.

 

You can experiment with other light source shapes and quality settings to get the feel how Area Shadows work

 

Although Area Shadows may look nice I believe one can skip them on a slower machine or larger structures because:

1. Sun (on a clear day) produce rather sharp, well defined shadows. When seen from the distances approximate of those in Simcity (even at zoom5) area effect is a minor one. Nice if it is there, but if that means days of render time I doubt it is worth it.

 2. Shadow degradation in real world doesn’t depend on the distance – in space for instance it never happens. But on the nature of the environments surrounding shadow and surface on which shadow is cast. More rough and bumpy this surface is, the more degradation you’ll see. Reflective qualities of surroundings will also contribute to the process the more light is bounced around the faster shadow will degrade. Algorithms in Area shadow just some sort of approximation of an average environment.

[bixel 1,228]

Very helpful, I didn't know that shadow map was just bad shadow alignment on the default settings...all the shadows were so soft and non attached to any object in the scene I though it was a sort fake GI.

[mntoes 271]

Very usefull information. Will you be showing us how to make use of these? I have no idea about the use of shadows in Gmax but would like to learn. 

Thanks

Mntoes

[SimFox 804]

mntoes: I myself use MAX, but I'll try to answer questions on GMAX as well. Remember though, that GMAX has very much limited capabilities and power - it seems it was done on purpose as not to create free competitor to full blown MAX.

So if you have some question – ask.

Next little info will be on REFLECTIONS as it seems to be a buzz word nowadays.

Tune-in in a day or so!

[moganite 4]

I had better find out how to do this stuff as it appears to be at the core of the buildings etc. While i know some of he basics I am yet to get it all done properly yet. I look fowrads to more

in your second point in your first post I would dissagree. But it is only becasue of difraction that occurs and gravity lensing bend light and making the edges fuzzy. Also it works only on large scales so it isn't important for our purposes.

I have done some simple reflection mapping just a plane inclined to the sun.

[SimFox 804]

With wich point exactly do you disagree?? About Area Shadows?? i said that all shadows are area shadows meaning in normal environment for us people... Gravity wells and some such exotic phenomena aside, just take a look at the shadow of a lamp pole! there it is- nice area shadow.

[cowkilla 12]

Originally posted by: moganite I had better find out how to do this stuff as it appears to be at the core of the buildings etc. While i know some of he basics I am yet to get it all done properly yet. I look fowrads to more

in your second point in your first post I would dissagree. But it is only becasue of difraction that occurs and gravity lensing bend light and making the edges fuzzy. Also it works only on large scales so it isn't important for our purposes.

I have done some simple reflection mapping just a plane inclined to the sun.quote>

area shadows are caused by a light emiting object being greater in size then the shadow casting object. giving it a fuzzy look that is exponental with distance. not from gravity lensing. maybe in a black hole. but not on earth.

sim fox. good luck with your teaching program

[moganite 4]

cowkilla

you missed the bit were I said it wasnt important for our purposes, Gravity lensing that is.

The fuzz bit is actually due to also the wave nature of light and it will "bend" round an object the phenomena is diffraction and the reason why only a small area of the planet actually experiences a solar eclips whn the moon passes between the sun and earth.

Also any object with sufficent mass has a lensing effect the sun has it and porbably the plane earth has it but is would not be detectable. On Earth an in an atmosphere you have back scatter from the gas molecules and dust particles present in any atmosphere that would wash out a shadow.

Also it depends so the size of the source and the ditsance from the source. remember the sun is 150 miilion km from here and All shadows are going to be parrallel becuse it is far away and has small angular size. if you compare that to the light you get from an ordinary light the shadows are no wehere nears as sharp as the source has a large angular size and what you are seing is many shadows being summed from many tiny point sources. Infact that is true for alot of lights just that by distyance from the source and proximity to the surface being project on. The is some interesting science going on.

I agree the Project is a good idea after all it is the wave properties of light that matter here.

[SimFox 804]

Cowkilla!!! great to hear from you!!! You haven't been very active lately... RL bytes? Moganite & Cowkilla. Guys you are both right! you just speak at somewhat different perspective 3D graphics and physics. Ideally all shadows should be treated this way but sometimes cost in our still slow computer time is prohibitive. But day is coming! Moganite: I would be honored to see such a learned fellow here often! You know this bits and pieces of science given here and there go a long way on improving overall knowledge!! One thing though... shadows from the sun on a bright day will be sharper then from almost all commonly accessible light source on Earth! Guys I'll be next writing about reflections and I invite you both to come here and correct me where I'll be wrong and share your knowledge as 3d artist and physics aficionado!

[mntoes 271]

Whoosh..... that's the science bit going beyond me... lol.

 I just want to know how so many of the great Bats here manage to get reflections on them . For example these building have benches, plants etc at their bases. Low and behold you can see them reflected in the building. How? Am I asking too much? Is this thread about the science bit which is interesting and greatly appreciated. will you show us by example? I'm not pushing you to show us. But if you could I'm keen to understand this.

thanks for the info so far

Mntoes

[SimFox 804]

mntoes: sure, that's what this thread is here for, to share knowledge for the betterment of community!

Next topic will be reflections.

It will take few days for me to sort out some computer issues and then I'll continue with help of other interested parties.

But one thing I can say right away, reflections - true ones, not faked with texturing are only possible with MAX, not GMAX.

[cowkilla 12]

sun shadows close to the surface are indeed sharp. but because the sun is many many miles accross it creates a fuzy effect. the same reason why an eclips is not a harsh line that goes across a planet but a very soft halo effect. this is not gravity bending its just the way light works

[moganite 4]

True thats why if you want sharp shadows you need small lights and why comerical lights tend to be as small as possible while maintaining as much brightness as possible. we can ignore gravity lensing for our purposes what we cant ignore is refractive lensing or refraction through glass.

The relative angular size of your object to your source at the surface of prjections is extremely important to. The other extremely imprtant variable is how divergent the rays of light are or the tightness of the beam. Intensity for light is like any other wave is inversly proprtional to the square of the distance from the source. For spotlights it is the same idea but you have to have a virtual distance for virtual source of an intensity that is very much higher than I will need to draw an image and look up the equations.

While I have an understanding of the pyhsics involved, I am lacking on the controls in gmax etc. I was going to make a few industrial light using omnilights with 100 percent reflection and see the result. most materials only reflect less than-40 percent light. most of my bats dont have night lights but I have a few planned. I dont know if i will relse them but it might be possible.

[cowkilla 12]

...what in gods name are you talking about?

not only are the first two paragraphs mostly nonsense. but you cannot have an omni lite have reflection and area lights dont exist in gmax.

[moganite 4]

Maybe I am lousy at explaing things but also maybe you dont evenunder stand why  objects get bigger as they get closer. Also if things wernt clear there are better ways of clarifying than calling someones words nonsense like you did. You could of asked a question or re explained the idea. But instead you called it nonsense  and refused to  elaborate and/or explain better. In other words i really doubt thet you have any understanding of the subject at all.

To under stand shadows you need to understand the concept of angular size. hopefull the pictures I have drawn explains things alot better. If you cant under stand it i sugest you look it up in a high school text book  before you make yourself look stupid.

I have deliberatedly left all of the equations out as alot of people would not be able to understand it and you really need some understanding of maths and calculus to go alot further.

The first image explains the idea of angular size and how it affects shadows.

The circle on the right is the source of light and the black rectangle is an opaque object.

A, B,C are imaginary planes that light passes through

Plane A :  Shadow has a defined shape and is relatively sharp. It is possible for the oject to completely obscure/occult the source and you have shadow. Angular size of the object is greater than the source often much greater. there may be some fringing at the extremites.

Plane B: At this plane you have a fuzzy indistinct shadow   and possibly a tiny hard shadow clse to this plane. at point D the angular size of the object and the source are identical and you get some bleed through of light washing out the shadow.

Plane C: At this Plane the shadow is indisinct and fuzzy or there is no shadow as there is enough light to illuminate it. The angular size of the object is Smaller than the angular size of the source.Light from  the source object obscures the object and makes it less vissible  resulting in a silloheute of the object surrounded by glare or light  from the source you alsu may get some diffuse/specular reflection from the object interacting with the light from the source.

Why does Sunlight make sharper shadows than commercial lights? it is best expained by the diagram below

 The situations here is  all objects when they radiate light they do so in all directions therefore it is nessecary to use reflectors to guide light.

The reason light from the sun gices sharp shadow is that all of its rays are more or less parallel to each other and the main reason for shadows getting more diffuse is the angular size of the object getting smaller with increasing distance.

This is because of the size of the sun is mitigated by the distance and the inverse square law that states that  as the distance doubles the intensity/flux of the light quarters. With commercial lights  only the object that is desired to be lit  has useful light falling on it. to increase the efficency of lights you add a reflector to increase the intensity and/or increase the tightness of the beam.

Even so because of the high curvature of the light and its proximity to the object being lit the rays are  also not parralell but in a mish mash of directions aimed at an object. Again relative angular size of the object  being lit at the surface the shadow is cast being tthe other infulece of softening of shadows.

finally there how human eyes function in a bright  sunny day you dont see the fuzz as it  to dim to be noticed as it is washed out by the sun. wheras the fuzz is readily noticable as most lights are not  more than a few percent of the brightness of the sun. 

As for area lights i never used the phrase in that post and if i did can you quote it i have bee a bit busy in the last few days.  Why cant omni lights reflect have you done it before? all you would need is a reflective surface the right shape and you could make a spotlight of sorts. I was palnning on using an omnilight inconjuction with a reflective surface and see what the result would be. Finally just because someone says something cant be done doesn't mean that it cant. I have already done a few things that are considered sort of difficult.

Also I like to experiment with this sort of stuff pity alot of other people dont.

[mntoes 271]

Thanks Simfox.  Guess i'll have to download a trial verion of Max. Moganite:- like the diagrams, I need to  re-read this to try and get my head arround it though. Never could ungerstand calculus.Thanks.

[bixel 1,228]

WhooOOooo! lots O' action here..we should move on to how we are going to deal with textures and light

for instance check out the neat effect of light has on this new building..can we replicate this?

[SimFox 804]

ha ha ha

 

yeeees, right

I just LOVE Chelsea Court

 

Well this neat effect is called REFLECTION. Glass of the building is reflecting the SKY. reflections are very glossy - so you know glass is shiny and very smooth. Walls also reflect light - everything does. Just their reflections are NOT glossy (glossiness less , much less then 1.0) Actually this reflection of light is the reason why colors are appear to be "bleached" by the sun.

I'm in the middle of preparing material on Reflection, but it advances very slow since I have to do it for the system I don't use... but I'm half way there.

 

BTW see what kind of heated debate shadows can steer!! I’m telling you! There are moving between us. Planting seeds of chaos and conflict!! I just wonder where does Vorlon Empire look to?

[SimFox 804]

There is a lot of talk about things being too bright going on around.

Often some of the coments leave me completely bewildered as of what exactly author had tried to say. So, to clarify this, I’ve decided to make this series of very simple tests. Take a look at them and let people know which cube is a brightest in each of the tests. Please use ONLY your eyes!!!

 

TEST I:

 

TEST II:

 

TEST III:

 

TEST IV:

 

 

TEST V:

 

[T Wrecks 10,210]

Hehe, that's fun. May I? May I?

I say in Test I 3 is brightest, 1 has just highest saturation.

In Test II: Cube 2, I'd say

In Test III: They look equally bright to me.

In Test IV: Number one.

In Test V: Equal I'd guess, but it's damn hard to judge in front of those different backgrounds.

[paulvmontfort 617]

3, 2, 3, 1, 2

[moganite 4]

Test 1:  1 looks brighter has that dayglow look, but i know 3 is in brighter light

Test 2: 3 or 1 looks brighter but 2 is in brighter light

Test 3:   3, 1 and 2 are less bright

Test  4:   1 , 2 and 3 are identical

Test 5:   2 looks brighter due to is back ground 1 looks a bit faded cause of the gray

Its down to the way we see colour and well I do come from  the southern hemisphere in the land that has the brightes sun.

I  didn't think it was that heated.

[Skyfire2275 23]

I - 3, 1 is most saturated

II - 2, 2 and 3 are just more saturated

III - same?

IV - 1, difference in saturation again?

V - 2, but it may jyst be the background.

I assume you will eventually tell us the answers.

[bixel 1,228]

2, 2, 2, 2, & 2.

[SimFox 804]

Bixel!! You’re remarkably consistent!!

The truth shall be revealed tomorrow afternoon!

[SimFox 804]

Well interesting results!!

 

The correct answers are:

 Test I

Cube#1 and #3 –they are of exactly same brightness.

 

Test II

This is the most interesting one!!!

Cubes #1 and #2 and #3 – brightness is same for all. Only difference is in saturation level.

 

Test III

Cube #1. Second cube is 15% less bright then first and third is 30% less bright then first.

 

Test IV

1^st cube is a brightest one. In this test Cube number 2 was used as sort of etalon. In case of cube number 1 saturation was reduces, but brightness increased. In cube number 3 other way around – saturation increased and brightness reduced.

 

Test V

Cube #1. The point of this test was to show the effect the black background is having on the perception of brightness. That’s why it may be beneficial to present WIP renderings not on black background but on something with more neutral brightness – something similar of in game average.

 

Important Note:

These tests are not about winning or loosing.

Their purpose is to improve understanding of key concepts in color management. And that way bring discussion on brightness etc., to common ground, so that when one respected member says to another respected member - Dude, like your building, like too bright, dude - both and also people reading this exciting conversation precisely understand each other.

So when presented with cube like this: (HSV: 255,255,255)
Request to make it less bright means that result  like this expected: (HSV 255,255,128)
And not like this: (HSV 255,128,255)

To wrap it up I’ll make a little piece on numerical color representation that may be helpful to someone…

As always I invite everyone to take a critical look and if some problems and inconsistencies are discovered bring them to light!

[ardecila 125]

Yea, people desaturating their colors annoys me. SC4 is meant to be much more saturated than a real city.

BTW, SimFox - I'd recommend applying some of this same knowledge to your garden BAT.  It is way too bright, and the saturation is too low.  A combination of lowering brightness and raising saturation should solve that problem.

[SimFox 804]

Ardecila:

well, the point of everything said here is not to tell people what to do, but just bring knowledge. It is anybodies own choice how to apply it. I for one do not agree that Simcity was meant to be so saturated as it has become nowadays. if you look through original textures, very few of them are actually saturated. It is all the modes that had significantly darken and oversaturated it - in my opinion rather to detriment, then to the benefit of game’s appearance. My garden was in a way play over that concept - that's why it was called CandyGarden. And I didn't even pretend that colors be natural. I do agree that grass needs more work. Color of foliage was a bit more complicated issue

BTW your own creations are neither too dark nor too saturated. Actually that's the reason I like them!

[ardecila 125]

Thanks!  I didn't know it was a candy garden, sorry.  Excellent modelling on that.

BTW, would you mind responding to my post in the MAXBAT help thread?