Recently at work I was chatting with a colleague, and the topic of energy conservation for specular reflections came up. This reminded me that I’ve been sitting on a blog post for a while about just this subject, so I thought it was time to finish it.
First of all, I’d like to start by looking at the standard diffuse reflection model. In games, the typical formula for calculating diffuse reflection from a particular light is:
Where Cd is the diffuse material color, Li is the light color, N is the normal, and L is the normalized direction to the light. What’s the problem with this? Well, it’s not energy conserving. In itself, this isn’t really a problem since we don’t calculate multiple bounces of light in games, so we’re not adding energy to the scene as light bounces around like would happen in a ray tracer. It’s a good starting point for discussion though.
Energy Conservation
As the name suggests, energy conservation is a restriction on the reflection model that requires that the total amount of reflected light cannot be more than the incoming light. It sounds sensible, but it’s often not practiced. A more formal way of stating this restriction is:
The function Ï represents the bidirectional reflection distribution function (BRDF), and could be anything from a simple Lambertian diffuse model, to a complicated microfacet model. Either way, the energy conservation restriction still stands.
Diffuse Energy Conservation
I’m going to show why the diffuse lighting equation above isn’t energy conserving, and how to make it so. Let’s start by replacing the BRDF with the constant diffuse color:
The incoming light direction is fixed here, and we are integrating over outgoing directions. Because of this, both Cd and Li are constant over the integral, and can be pulled outside. Also, the incoming light appears on both sides of the inequality, so we can divide by Li leaving:
This integral can be solved analytically. First of all, we need to rewrite it as a double integral of the two polar coordinates φ and θ:
The extra sin θ may seem a little bit confusing at first, but it’s necessary to take into account the smaller area towards the polar region. Using the double angle trigonometric identity, this is the same as:
Now we can start using some trigonometric integrals to integrate, firstly over θ:
This integral completely disappears down to nothing:
So now, integrate over φ:
This gives us the final inequality:
Assuming that we want to keep our diffuse material color in the range [0,1], all this says is that we need to divide it by π in order to remain energy conserving. Since this is just a constant scale, it might not be worth doing in a game if the only lighting model it uses is diffuse. Most games use a more sophisticated reflection model though, at least including specular reflections.
Specular Energy Conservation
The standard Blinn-Phong specular model is also not energy conserving. In fact, in some ways it is even worse than the diffuse model, because as you increase the specular power, you lose more and more energy. A manifestation of this problem can be that artists find it hard to get a really tight specular highlight.
Here’s an example of a sphere rendered at three different specular powers. Notice how much light there appears to be on the left image, and how it seems to have disappeared on the right, even though it’s supposed to be more focused. It’s at this point that artists start to ask to ramp the specular reflection color over one to compensate. This isn’t a good idea!
Instead of boosting the specular reflection color, you can switch to an energy conserving specular model. If you do this, the same spheres with the same specular powers now look like this:
The specular reflection on sphere on the left is actually dimmer than the non-conserving model. This is because the non-conserving specular model reflected too much light in this case. As the specular power increases, this time we compensate for the energy loss, and you get a really nice and tight hotspot on the right.
The typical Blinn-Phong reflection model is:
Somebody who knows more than me has worked out that to ensure energy conservation, the normalization factor is:
[Edit: Thanks to Fabian “ryg” Giesen for showing that this is actually the normalization factor for the regular Phong specular model, not Blinn-Phong. A commonly used normalization factor (according to Real-Time Rendering) for Blinn-Phong is shown below.]
So this makes the energy conserving function for Blinn-Phong specular reflection:
I’d love to know how this was derived, but I haven’t found anything so far that explains it. The integral for raising a cosine function to a power gets pretty hairy very quickly. All I know is that it appears to work. If anyone reading this knows why, then please let me know!
[Edit: Fabian “ryg” Giesen very kindly posted up a derivation of the normalization factor for both the regular Phong, and Blinn-Phong specular reflection models here. Interestingly he doesn’t come up with the exact same answer for the Blinn-Phong normalization factor as shown in Real-Time Rendering. Check out the comments below to find out more about this.]
Combined Diffuse and Specular
Once you have energy-conserving models for diffuse and specular, it’s easy to make sure that the combined model is also energy conserving. You just need to make sure that your diffuse and specular material colors don’t sum to more than one:
This means that if you want your material to have more specular, you may have to reduce the diffuse. Here’s a range of variations of the same material, going from 100% diffuse to 100% specular:
Is It Worth It For Games?
Clearly, the specular model benefits significantly from being energy conserving so I think most people would say that it’s worth it. Switching to a model where the diffuse and specular have to compete for energy might not be though, since it’s can be harder to tweak. I personally use this kind of model for my projects at home, but that’s because I don’t have artists to please.
One thing I do like about an energy conserving reflection model is that it enforces some kind of reasonable limits to the material reflections. This might help to make materials created by different people sit better together.
Interesting read, thanks a lot!
Note though, that making specular energy-conserving can be as easily achieved by allowing specular intensity to go in [1, (n+2)/pi] 🙂 so allowing intensity to grow past 1 is good enough if artists adjust it everywhere where specular highlights look noticeably bad.
I have also seen the specular energy conservation term as (n + 8)/8.
However, I, too, have no idea if it’s from an analytical derivation, or if someone was just eye-balling the cos power graph and came up with these numbers.
I could see where the divide by 2pi comes in, since that’s the result of the integral of the function over phi, so this would mean that the integral over theta is approximated by 1 / (n + 2).
I’ve also seen that other form, but again I don’t know of the derivation. I hope someone comes along who does.
great post!
On the normalization factor: This is the normalization factor for the “classic” Phong model, not Blinn-Phong, actually. I put up a derivation here: http://www.farbrausch.de/~fg/stuff/phong.pdf – I haven’t done Blinn-Phong yet, maybe later.
Oh, and a small addition: As per Real-Time Rendering, 3rd Edition (page 257), the normalization factor for Blinn-Phong is (m+8)/8pi, but the reference cited is “Personal communication”, so no derivations coming from that direction I fear.
Erm, okay, apparently my first post didn’t arrive? Okay, short version then: that normalization factor is for the “classic” Phong model and the derivation isn’t that hard, I put it up at http://www.farbrausch.de/~fg/stuff/phong.pdf in case anyone’s interested.
Okay, apparently I can’t post any URLs in the text, at least I tried twice and both comments didn’t arrive. Spambot protection I guess, oh well. Anyway, the (n+2)/2pi normalization factor is actually for the “classic” Phong model, not Blinn-Phong, and I’ve put up a derivation at the URL I listed as my webpage URL, in case anyone’s still interested.
Thanks for clearing that one up! I had to approve those comments with the links apparently.
That derivation was simpler than I thought it was going to be. I didn’t realise it was just for the original Phong model though. It still bugs me that the n+8/8pi hasn’t been explained though.
Okay, I’ve updated the PDF with a half-complete derivation of the Blinn-Phong normalization factor; this includes the complete evaluation of the integral, but is (currently) without a proof that the maximum amount of reflected energy occurs when L=N. There’s probably an elegant way to prove this, but I didn’t find it, and showing it directly from the integral seemed to messy to seriously pursue it. I did however do some numerical experiments and it does seem to check out.
The actual integration proceeds in a pretty straightforward fashion – one substitution (which yields a surprisingly neat term), one integration by parts to calculate the valuee (which turns it into a big mess), and then a bunch of algebra to simplify it down again (I’ve kept more intermediate steps of algebraic simplifications than I normally would in a printed document, but after all the whole point of this exercise was to have a complete derivation).
Anyway, the whole story’s got a punchline: the normalization factor I got is (n+2)(n+4) / (8pi * (2^(-n/2) + n)), not (n+8)/8pi. If you plot this factor against n, you’ll notice that it really is close to linear in n, but not quite there.
I was pretty surprised about this result to say the least. I cross-checked the integral with Mathematica, and I also wrote a small test program that computes integrals of functions over the hemisphere using stratified Monte Carlo integration to be sure; I didn’t want to rely on the analytical angle-based formulations too much, since it’s very easy to mess up and accidentally take the wrong angle.
The result, using both analytical computations and the Monte Carlo integration test program, were that with n=16, the overall amount of reflected energy with L=N and the (n+8)/8pi normalization term is about 1.06 times L_i, while the (n+2)(n+4)/… variant derived in the PDF is 1.0 to within integration accuracy. Other values of n yield the same qualitative result: (n+8)/8pi is slightly too large.
After finding this out last weekend, I wrote a mail to Naty Hoffman (who apparently did the derivation for Real-Time Rendering 3rd Ed.), but I didn’t get an answer yet, and I wanted to put this online before I lose my notes with the derivation, so here goes. 🙂
That’s really interesting. I followed it through and it all makes sense to me. I doubt I would have come up with that myself, so thanks for showing the complete derivation.
I wonder about the explanation of (n+8/8pi) even more now. I’d be interested in hearing what Naty says about this when he gets back to you. I expect I’ll meet him at GDC since we’re both Activision employees now, so I’ll try and bug him about this if I don’t hear anything.
I used Maxima to graph the difference between the two functions ((n+8)/8pi)/((n+2)(n+4) / (8pi * (2^(-n/2) + n))), and it seems like it peaks very early (n = 8, error is 7.5%) and then asymptotes 0% error. It takes a long time to get down to anything close to 0% error though, so even at n=100, the difference is still 1.8%.
I wonder if the n+8/8*pi normalization factor is just a relatively cheap-to-compute conservative approximation of the version you derived?
I’ll update my initial post with a link to your derivation if you don’t mind.
(n+2)(n+4) = n² + 6n + 8, so for medium-large n a more accurate cheap approximation would be (n+6)/8pi, not (n+8)/8pi. Also, (n+8)/8pi is not conservative; after all, it makes the BRDF integrate to more than 1, not less. I would however still very much like to know how it was derived.
The L=N variant is really the only special case of Blinn-Phong that’s reasonably easy to write down in terms of angles analytically. For general L, the plane spanned by L and V (which contains H) doesn’t contain N, so there’s no easy way to write (N.H) in terms of the angles you know; I’d like to see the (n+8)/8pi derivation because I’d very much like to see how that issue is treated there.
Feel free to link to my derivation; however, I clean out my /stuff directory more or less regularly, so this is not a good candidate for a permanent link. I’ve put up a copy of the derivation at (same URL)/articles/phong.pdf where it’s safe.
Ah yes. I forgot that the constants weren’t the result of the integral, but the inverse. So I mistakenly thought that numbers above one indicated a conservative approximation, when it’s the exact opposite.
(n + 6) / (8 * pi) seems good since it is conservative and is within 5% of your normalization constant for n >= 10, but the error ramps up to 25% at n = 0. It looks like it produces a better error on average (n between 0 and 100) than ( n + 8 ) / (8 * pi), but a worse worst case. I suppose the effect of using this would mean duller than expected specular reflections at very low specular powers.
I’ll update the link to your derivation. Thanks for the help.
I did manage to talk to Naty about this at GDC. He said that a few people have asked him about the derivation of the specular factor in the book, and that he had gone through it himself and got the exact same answer as Fabian. The value they mention in the book is just an approximation of that result.
About the approximation we chose, we were not trying to be strictly conservative (that is important for multi-bounce GI solutions to converge, but not for rasterization). We were trying to choose a cheap approximation which is close to 1, and we thought it more important to be close for low specular powers. Low specular powers have highlights that cover a lot of pixels and are unlikely to be saturating past 1.
[…] Driscoll had a good post in his blog a few months ago reminding everyone about energy conservation in lighting models. […]
Nice piece of work – and good images to show the results – definitely worth implementing !
I did notice however that your intermediate integral is wrong (although actually you still got the right answer for the next stage by ignoring the error)
-1/2 cos (2 theta) is :
theta = 0 : -1/2 x 1 = -0.5
theta = pi/2 : -1/2 x -1 = +0.5
so the integral is 0.5 – (-0.5) = 1.0
of course then your next integral starts to make sense as the integral of one with respect to phi is indeed phi. (the integral of zero is zero over all ranges, so clearly otherwise this makes no sense)
Right you are! I’m assuming that’s a typo when I was copying the math over from paper, since like you say, it doesn’t make sense. Thanks for pointing that one out.
[…] [3] Giesen, http://www.farbrausch.de/~fg/stuff/phong.pdf [4] Hoffman, comment on the post http://www.rorydriscoll.com/2009/01/25/energy-conservation-in-games/ [5] Cook, Torrance, A Reflectance Graphics Model for Computer graphic […]
[…] http://www.rorydriscoll.com/2009/01/25/energy-conservation-in-games/ based, conserving, diffuse, energy, physically, rendering, […]
My Blinn-Phong specular function uses this:
specularLight=specFactor*pow(max(dot(N,H),0),shininess);
So I take the dot between normal and halfvector. In the formula above you use dot(H,L); is that similar? I added the normalization for diffuse (divide by pi) and specular ( (n+6)/8pi ) but now I’m not too sure it’s right, given dot(N,H).
Ruud – that was in fact an embarrassing mistake on my part. Thank you for pointing it out! Yours is absolutely correct.
For reference, here’s a simple energy-conserving shader I wrote which works in Stephen Hill’s shader sandbox.
Sandbox: http://www.selfshadow.com/sandbox/gloss.html
Shader: https://gist.github.com/1394734
Nice to know I’m not going mad. 😉
I notice in the sandbox example (nice site btw) that you took out the division by PI; probably since pi is present in both factors, you can multiply your light color instead.
Interesting stuff!
No, the division by pi is still in there, it’s just hidden. The DiffuseConservation() still has the obvious divide by pi.
The SpecularConservation() function still has it, but not explicitly. I used Mathematica to fit a line to (n+8)/(8 * pi) since that’s a little bit cheaper in the shader, and almost as good.
Check out the graphs: http://www.wolframalpha.com/input/?i=%28%28n%2B8%29%2F%288*pi%29%29+vs+%280.0397436+*+n+%2B+0.0856832%29+for+n+from+0+to+1000
How about this?
((power+8)/25.13274)*2.793
So at a power of 1 it’s at the same level of diffuse lighting.
This way you can easily compute the final light using:
float diff = max(dot(N,L),0);
float spec = max(dot(N,H),0);
spec = spec*((power+8)/25.13274)*2.793;
float result = lightcolor*lerp(diff, spec, reflection);
It would seem to me this way the total energy always adds up to the original input color energy.
While potentially more intuitive for artists, that formula is no longer energy conserving. As mentioned above, this isn’t necessarily a problem for games.
Remember that total energy out is spread over the entire hemisphere, so the intuition that a a certain value in gives the same value out is not generally correct.
Yes, I added a command ‘analyse shaders’ in my game which checks that diff+spec<=1. A black non-specular object might have a low number, which is fine; the rest is 'heat' (1-diff-spec), which isn't simulated in most graphics engines I would think.
Note that the line fit from rory’s post at Dec 14, 8:24am seems ok for n=0..1000, but for n=0 it’s a bit off. We have quite a bit of low-specular materials.
At n=0, (n+8)/8*PI becomes 1/PI. So for the linear approximation, why not this:
http://www.wolframalpha.com/input/?i=%28%28n%2B8%29%2F%288*pi%29%29+vs+%280.0397436+*+n+%2B+0.3183%29+for+n+from+0+to+100
This changes the offset at n=0 to exactly 1/PI and seems more correct for specular=0..100.
Also (sorry for flooding); the (n+8)/(8*PI) seems an exact linear line. The derivative of that formula is 1/(8*PI), so 0.0397887357729839. Why not use that? It seems this is an exact match to (n+8)*(8*PI).
So in Wolfram, you get:
((n+8)/(8*pi)) vs (0.0397887357729839 * n + 0.3183) for n from 0 to 10
Oh, sorry, I should have mentioned that (n+8)/(8*pi) is itself an approximation of the real result of the integral. The real conservation factor is not linear. Take a look at the second page of Fabian’s derivation: http://www.farbrausch.de/~fg/stuff/phong.pdf
The line in the shader is a fit of a linear function to the real conservation factor.
A note on ‘is it worth it for games?’. I’m using it in a game (see racer.nl), but the reason that I’m using it is because I think it’s easier to get realistic render results if you try to stick with plausible models.
My lighting is done using klux (kilo-lux) values, so you can in fact measure lighting amounts outside, of the sun, inside rooms, to get actual klux values (klux instead of lux btw because lux values are so large that you get floating point trouble; kilolux fits floating point buffers quite nicely).
Pushing actual klux values into the lighting time-of-day system then should more easily give realistic lighting if energy is conserved as above. So I’d still recommend it for games, if only to make coming up with a more realistic result becomes easier this way.
[…] a more restricted topic Rory Driscoll briefly explained the problem of energy conservation, a first but important step toward PBR, in a convincing and straight to the point manner. In […]
[…] [3] “Energy conservation in game” http://www.rorydriscoll.com/2009/01/25/energy-conservation-in-games/ [4] Hill, “Righting wrap” http://blog.selfshadow.com/2011/12/31/righting-wrap-part-1/ […]
[…] http://graphics.stanford.edu/papers/envmap/ [6] “Energy conservation in game” http://www.rorydriscoll.com/2009/01/25/energy-conservation-in-games/ Advertisement LD_AddCustomAttr("AdOpt", "1"); LD_AddCustomAttr("Origin", "other"); […]
FYI, better linear approximations exist. But they are domain specific. The standard approximation (m+8)/(8*pi) is the best over the entire theoretical dynamic range 0<m<Infinity. I do this work for living and have found linear approximations for the domain of our implementation (0,255] that reduce the maximum relative error by more than 50% at the expense of a roughly 10% higher average relative error which we justify by preferring to always be close to right than closer on average. By the way, I am still waiting for that proof that the maximum energy case occurs when L=N, Fabian! Cheers!
Furthermore, what about when L!=N? Is the normalization still appropriate, or should it really be a function of L?
Nice article and discussion. There is one point in Fabian’s derivation that I hope I can get clarification on. In the discussions of radiosity I’ve seen (RTR and PBR books and other papers) the cos(theta) term is the projected area; that is N.L where theta is the angle between the light vector and the normal and this accounts for the spreading of irradiance across surface area as L deviates from N.
Fabian describes theta as the angle between the view vector and the normal. Why is this valid? I thought cos(theta) should be N.L, which would equal 1 with his L=N max energy constraint.
I found some other discussion here, but this point is not discussed: http://comments.gmane.org/gmane.games.devel.algorithms/21394
I appreciate any help understanding this point. Thanks.
I completely understand the concept and reasons for this blog entry. It is good to read and I’m sure to some people it may be enlightening (no pun intended)
But if it is to be used to enhance realism where that is important, then it’s already failed because these materials were generally never designed for realism in the first place. Ie, photo-realism in game engines.
Realistic response, yes, but without being able to actually reflect something when they get shiny then they are already disobeying energy conservation.
Put that ball in a room with a single light bulb. The room now imparts energy to the ball as well as the light bulb.
You can control the ambient, diffuse and spec terms to get a realistic response from those specific terms yes…
…but that won’t look realistic because a shiny ball reflects it’s environment, and in doing so it would reflect the light bulb, and that would double up the energy (you’d have high value reflection of bulb + specular simulating reflection from bulb)
The middle ground example where it’s a slightly rough ball is harder still. Do you blur the reflection, or blur the reflection, use 50% of it, and use 50% of the specular?
Energy conservation makes sense with those old materials, but using those old methods for a realistic appearance will never really work.
For example, what about response due to reflection angle (IOR), or other such properties that are not special cases by universal.
So we can control energy realistically and automatically this way, on a model that is immediately not very useful for any realistic representation in a real-time graphics engine except objects lit with a single light in a black room that absorbs all light.
I use the real-time gfx engine Ruud van Gaal mentioned earlier and it really is still a mixture of mixing intelligence and understanding with the maths.
Ie, my race course is a large matt surface, but the sky still creates a million small specular responses on the surface that needs to be present for it to look realistic.
Suddenly you need to fake that energy by say multiplying the ambient light intensity by the specular level and specular control texture map.
But how do we then conserve energy or know what is right or wrong?
An alternative may be to remove specular altogether and use a HDR reflection cube map to do the job instead, and apply IOR properties too.
To get these material types to look realistic for the roles they play today we need to be creative with them, to the point that limiting values or automating them can actually work against you getting a realistic response.
As long as we are mixing methods which you will always do with those old type materials, you need to be free to balance values how you see fit to actually achieve a realistic look.
Thanks
Dave
[…] 原始的Blinn-Phong高光并ä¸ç¬¦åˆèƒ½é‡å®ˆæ’，看下é¢ä¸¤å¼ 图，上é¢æ˜¯æ²¡æœ‰Normalized的，下é¢æ˜¯ç›¸åŒçš„高光å‚数下符åˆèƒ½é‡å®ˆæ’的效果。å¯è§åŸºäºŽç‰©ç†çš„模型更容易调整效果，而ä¸ç”¨å€ŸåŠ©å„ç§å¥‡æ€ªçš„å‚数。 […]
Hmmmm, this all makes sense as soon as you want any single relatively homogeneous material but with varying gloss levels across it.
The example imagery doesn’t really do the example justice, because three discrete materials can be managed independently, and may well each be their own unique material.
When we have one material (shader) then we need to use energy conservation.
Thanks
Dave
[…] Driscoll had a good post in his blog a few months ago reminding everyone about energy conservation in lighting models. […]
[…] to read about physically-based shading, and discover the idea of energy conservation (something I wrote about […]
[…] Energy conservation is an important aspect of a BRDF, a great introduction from Rory Driscoll again. Energy conservation is worth pursuing even if you use non realistic lighting models like wrapped-diffuse. On the topic of normalisation, Fabian Giesen shows how the normalisation factor for Phong specular is derived and  The Blinn-​Phong Normalization Zoo discusses the various options for the Phong and Blinn-Phong specular. […]
Good post. I learn something new and challenging on websites I stumbleupon on a daily basis.
It’s always interesting to read content from other writers and use a little something from their web
sites.