Energy Conservation In Games
Recently at work I was chatting with a colleague, and the topic of energy conservation for specular reflections came up. This reminded me that I’ve been sitting on a blog post for a while about just this subject, so I thought it was time to finish it.
First of all, I’d like to start by looking at the standard diffuse reflection model. In games, the typical formula for calculating diffuse reflection from a particular light is:
Where Cd is the diffuse material color, Li is the light color, N is the normal, and L is the normalized direction to the light. What’s the problem with this? Well, it’s not energy conserving. In itself, this isn’t really a problem since we don’t calculate multiple bounces of light in games, so we’re not adding energy to the scene as light bounces around like would happen in a ray tracer. It’s a good starting point for discussion though.
Energy Conservation
As the name suggests, energy conservation is a restriction on the reflection model that requires that the total amount of reflected light cannot be more than the incoming light. It sounds sensible, but it’s often not practiced. A more formal way of stating this restriction is:
The function ρ represents the bidirectional reflection distribution function (BRDF), and could be anything from a simple Lambertian diffuse model, to a complicated microfacet model. Either way, the energy conservation restriction still stands.
Diffuse Energy Conservation
I’m going to show why the diffuse lighting equation above isn’t energy conserving, and how to make it so. Let’s start by replacing the BRDF with the constant diffuse color:
The incoming light direction is fixed here, and we are integrating over outgoing directions. Because of this, both Cd and Li are constant over the integral, and can be pulled outside. Also, the incoming light appears on both sides of the inequality, so we can divide by Li leaving:
This integral can be solved analytically. First of all, we need to rewrite it as a double integral of the two polar coordinates φ and θ:
The extra sin θ may seem a little bit confusing at first, but it’s necessary to take into account the smaller area towards the polar region. Using the double angle trigonometric identity, this is the same as:
Now we can start using some trigonometric integrals to integrate, firstly over θ:
This integral completely disappears down to nothing:
So now, integrate over φ:
This gives us the final inequality:
Assuming that we want to keep our diffuse material color in the range [0,1], all this says is that we need to divide it by π in order to remain energy conserving. Since this is just a constant scale, it might not be worth doing in a game if the only lighting model it uses is diffuse. Most games use a more sophisticated reflection model though, at least including specular reflections.
Specular Energy Conservation
The standard Blinn-Phong specular model is also not energy conserving. In fact, in some ways it is even worse than the diffuse model, because as you increase the specular power, you lose more and more energy. A manifestation of this problem can be that artists find it hard to get a really tight specular highlight.
Here’s an example of a sphere rendered at three different specular powers. Notice how much light there appears to be on the left image, and how it seems to have disappeared on the right, even though it’s supposed to be more focused. It’s at this point that artists start to ask to ramp the specular reflection color over one to compensate. This isn’t a good idea!
Instead of boosting the specular reflection color, you can switch to an energy conserving specular model. If you do this, the same spheres with the same specular powers now look like this:
The specular reflection on sphere on the left is actually dimmer than the non-conserving model. This is because the non-conserving specular model reflected too much light in this case. As the specular power increases, this time we compensate for the energy loss, and you get a really nice and tight hotspot on the right.
The typical Blinn-Phong reflection model is:
Somebody who knows more than me has worked out that to ensure energy conservation, the normalization factor is:
[Edit: Thanks to Fabian "ryg" Giesen for showing that this is actually the normalization factor for the regular Phong specular model, not Blinn-Phong. A commonly used normalization factor (according to Real-Time Rendering) for Blinn-Phong is shown below.]
So this makes the energy conserving function for Blinn-Phong specular reflection:
I’d love to know how this was derived, but I haven’t found anything so far that explains it. The integral for raising a cosine function to a power gets pretty hairy very quickly. All I know is that it appears to work. If anyone reading this knows why, then please let me know!
[Edit: Fabian "ryg" Giesen very kindly posted up a derivation of the normalization factor for both the regular Phong, and Blinn-Phong specular reflection models here. Interestingly he doesn't come up with the exact same answer for the Blinn-Phong normalization factor as shown in Real-Time Rendering. Check out the comments below to find out more about this.]
Combined Diffuse and Specular
Once you have energy-conserving models for diffuse and specular, it’s easy to make sure that the combined model is also energy conserving. You just need to make sure that your diffuse and specular material colors don’t sum to more than one:
This means that if you want your material to have more specular, you may have to reduce the diffuse. Here’s a range of variations of the same material, going from 100% diffuse to 100% specular:
Is It Worth It For Games?
Clearly, the specular model benefits significantly from being energy conserving so I think most people would say that it’s worth it. Switching to a model where the diffuse and specular have to compete for energy might not be though, since it’s can be harder to tweak. I personally use this kind of model for my projects at home, but that’s because I don’t have artists to please.
One thing I do like about an energy conserving reflection model is that it enforces some kind of reasonable limits to the material reflections. This might help to make materials created by different people sit better together.
18 comments18 Comments so far
Leave a reply
Interesting read, thanks a lot!
Note though, that making specular energy-conserving can be as easily achieved by allowing specular intensity to go in [1, (n+2)/pi]
so allowing intensity to grow past 1 is good enough if artists adjust it everywhere where specular highlights look noticeably bad.
I have also seen the specular energy conservation term as (n + 8)/8.
However, I, too, have no idea if it’s from an analytical derivation, or if someone was just eye-balling the cos power graph and came up with these numbers.
I could see where the divide by 2pi comes in, since that’s the result of the integral of the function over phi, so this would mean that the integral over theta is approximated by 1 / (n + 2).
I’ve also seen that other form, but again I don’t know of the derivation. I hope someone comes along who does.
great post!
On the normalization factor: This is the normalization factor for the “classic” Phong model, not Blinn-Phong, actually. I put up a derivation here: http://www.farbrausch.de/~fg/stuff/phong.pdf – I haven’t done Blinn-Phong yet, maybe later.
Oh, and a small addition: As per Real-Time Rendering, 3rd Edition (page 257), the normalization factor for Blinn-Phong is (m+8)/8pi, but the reference cited is “Personal communication”, so no derivations coming from that direction I fear.
Erm, okay, apparently my first post didn’t arrive? Okay, short version then: that normalization factor is for the “classic” Phong model and the derivation isn’t that hard, I put it up at http://www.farbrausch.de/~fg/stuff/phong.pdf in case anyone’s interested.
Okay, apparently I can’t post any URLs in the text, at least I tried twice and both comments didn’t arrive. Spambot protection I guess, oh well. Anyway, the (n+2)/2pi normalization factor is actually for the “classic” Phong model, not Blinn-Phong, and I’ve put up a derivation at the URL I listed as my webpage URL, in case anyone’s still interested.
Thanks for clearing that one up! I had to approve those comments with the links apparently.
That derivation was simpler than I thought it was going to be. I didn’t realise it was just for the original Phong model though. It still bugs me that the n+8/8pi hasn’t been explained though.
Okay, I’ve updated the PDF with a half-complete derivation of the Blinn-Phong normalization factor; this includes the complete evaluation of the integral, but is (currently) without a proof that the maximum amount of reflected energy occurs when L=N. There’s probably an elegant way to prove this, but I didn’t find it, and showing it directly from the integral seemed to messy to seriously pursue it. I did however do some numerical experiments and it does seem to check out.
The actual integration proceeds in a pretty straightforward fashion – one substitution (which yields a surprisingly neat term), one integration by parts to calculate the valuee (which turns it into a big mess), and then a bunch of algebra to simplify it down again (I’ve kept more intermediate steps of algebraic simplifications than I normally would in a printed document, but after all the whole point of this exercise was to have a complete derivation).
Anyway, the whole story’s got a punchline: the normalization factor I got is (n+2)(n+4) / (8pi * (2^(-n/2) + n)), not (n+8)/8pi. If you plot this factor against n, you’ll notice that it really is close to linear in n, but not quite there.
I was pretty surprised about this result to say the least. I cross-checked the integral with Mathematica, and I also wrote a small test program that computes integrals of functions over the hemisphere using stratified Monte Carlo integration to be sure; I didn’t want to rely on the analytical angle-based formulations too much, since it’s very easy to mess up and accidentally take the wrong angle.
The result, using both analytical computations and the Monte Carlo integration test program, were that with n=16, the overall amount of reflected energy with L=N and the (n+8)/8pi normalization term is about 1.06 times L_i, while the (n+2)(n+4)/… variant derived in the PDF is 1.0 to within integration accuracy. Other values of n yield the same qualitative result: (n+8)/8pi is slightly too large.
After finding this out last weekend, I wrote a mail to Naty Hoffman (who apparently did the derivation for Real-Time Rendering 3rd Ed.), but I didn’t get an answer yet, and I wanted to put this online before I lose my notes with the derivation, so here goes.
That’s really interesting. I followed it through and it all makes sense to me. I doubt I would have come up with that myself, so thanks for showing the complete derivation.
I wonder about the explanation of (n+8/8pi) even more now. I’d be interested in hearing what Naty says about this when he gets back to you. I expect I’ll meet him at GDC since we’re both Activision employees now, so I’ll try and bug him about this if I don’t hear anything.
I used Maxima to graph the difference between the two functions ((n+8)/8pi)/((n+2)(n+4) / (8pi * (2^(-n/2) + n))), and it seems like it peaks very early (n = 8, error is 7.5%) and then asymptotes 0% error. It takes a long time to get down to anything close to 0% error though, so even at n=100, the difference is still 1.8%.
I wonder if the n+8/8*pi normalization factor is just a relatively cheap-to-compute conservative approximation of the version you derived?
I’ll update my initial post with a link to your derivation if you don’t mind.
(n+2)(n+4) = n² + 6n + 8, so for medium-large n a more accurate cheap approximation would be (n+6)/8pi, not (n+8)/8pi. Also, (n+8)/8pi is not conservative; after all, it makes the BRDF integrate to more than 1, not less. I would however still very much like to know how it was derived.
The L=N variant is really the only special case of Blinn-Phong that’s reasonably easy to write down in terms of angles analytically. For general L, the plane spanned by L and V (which contains H) doesn’t contain N, so there’s no easy way to write (N.H) in terms of the angles you know; I’d like to see the (n+8)/8pi derivation because I’d very much like to see how that issue is treated there.
Feel free to link to my derivation; however, I clean out my /stuff directory more or less regularly, so this is not a good candidate for a permanent link. I’ve put up a copy of the derivation at (same URL)/articles/phong.pdf where it’s safe.
Ah yes. I forgot that the constants weren’t the result of the integral, but the inverse. So I mistakenly thought that numbers above one indicated a conservative approximation, when it’s the exact opposite.
(n + 6) / (8 * pi) seems good since it is conservative and is within 5% of your normalization constant for n >= 10, but the error ramps up to 25% at n = 0. It looks like it produces a better error on average (n between 0 and 100) than ( n + 8 ) / (8 * pi), but a worse worst case. I suppose the effect of using this would mean duller than expected specular reflections at very low specular powers.
I’ll update the link to your derivation. Thanks for the help.
I did manage to talk to Naty about this at GDC. He said that a few people have asked him about the derivation of the specular factor in the book, and that he had gone through it himself and got the exact same answer as Fabian. The value they mention in the book is just an approximation of that result.
About the approximation we chose, we were not trying to be strictly conservative (that is important for multi-bounce GI solutions to converge, but not for rasterization). We were trying to choose a cheap approximation which is close to 1, and we thought it more important to be close for low specular powers. Low specular powers have highlights that cover a lot of pixels and are unlikely to be saturating past 1.
[...] Driscoll had a good post in his blog a few months ago reminding everyone about energy conservation in lighting models. [...]
Nice piece of work – and good images to show the results – definitely worth implementing !
I did notice however that your intermediate integral is wrong (although actually you still got the right answer for the next stage by ignoring the error)
-1/2 cos (2 theta) is :
theta = 0 : -1/2 x 1 = -0.5
theta = pi/2 : -1/2 x -1 = +0.5
so the integral is 0.5 – (-0.5) = 1.0
of course then your next integral starts to make sense as the integral of one with respect to phi is indeed phi. (the integral of zero is zero over all ranges, so clearly otherwise this makes no sense)
Right you are! I’m assuming that’s a typo when I was copying the math over from paper, since like you say, it doesn’t make sense. Thanks for pointing that one out.