Mikkelsen is apparently well-versed in academic obfuscation (tsk!), so the paper itself can be a little hard to read. If you’re interested in reading it, then I would recommend first reading Jim Blinn’s original bump mapping paper to understand some of the derivations.

But Wait! What’s Wrong with Normal Maps?

Nothing really. But if something comes along that can improve quality, performance or memory consumption then it’s worth taking a a look.

A Quick Detour into Gradients

Given a scalar height field (i.e. a two-dimensional array of scalar values), the gradient of that field is a 2D vector field where each vector points in the direction of greatest change. The length of the vectors corresponds to the rate of change.

The contour map below represents the scalar field generated from the function . The vector field shows the gradient of that scalar field. Note how each vector points towards the center, and how the vectors in the center are smaller due to the lower rate of change.

Derivative Maps

The main premise of the paper is that we can project the gradient of the height field onto an underlying surface and use it to skew the surface normal to approximate the normal of the height-map surface. We can do all of this without requiring tangent vectors.

As with the original bump-mapping technique, it’s not exact due to some terms being dropped due to their relatively small influence, but it’s close.

There are really only two important formulae to consider from the paper. The first shows how to perturb the surface normal using the surface gradient. Don’t confuse the surface gradient with the gradient of the height field mentioned above! As you’ll see shortly, they’re different.

Here, represents the perturbed normal, is the underlying surface normal, and is the surface gradient. So basically, this says that the perturbed normal is the surface normal offset in the negative surface gradient direction.

So how do we calculate the surface gradient from the height field gradient? Well, there’s some fun math in there which I don’t want to repeat, but if you’re interested, I would recommend reading Blinn’s paper first, then Mikkelsen’s paper. You eventually arrive at:

In addition to the symbols defined previously, and are the partial derivatives of the surface position, and and are the partial derivatives of the height field. The derivative directions and are not explictly defined here.

It’s easiest to think of this as the projection of the 2D gradient onto a 3D surface along the normal. Intuitively, this says that the surface gradient direction is pushed out on orthogonal vectors to the s/n and t/n planes by however much the gradient specifies. The denominator term is there to scale up the result when the and are not orthogonal, or are flipped.

Implementation

Implementing this technique is fairly straightforward once you realise the meaning of some of the variables. Since we’re free to choose the partial derivative directions and , it’s convenient for the shader to use screen-space x and y. The value is the position, and the value is the height field sample.

// Project the surface gradient (dhdx, dhdy) onto the surface (n, dpdx, dpdy)
float3 CalculateSurfaceGradient(float3 n, float3 dpdx, float3 dpdy, float dhdx, float dhdy)
{
	float3 r1 = cross(dpdy, n);
	float3 r2 = cross(n, dpdx);

	return (r1 * dhdx - r2 * dhdy) / dot(dpdx, r1);
}

// Move the normal away from the surface normal in the opposite surface gradient direction
float3 PerturbNormal(float3 n, float3 dpdx, float3 dpdy, float dhdx, float dhdy)
{
	return normalize(normal - CalculateSurfaceGradient(normal, dpdx, dpdy, dhdx, dhdy));
}

So far, so good. Next we need to work out how to calculate the partial derivatives. The reason why we chose screen-space x and y to be our partial derivative directions is so that we can use the ddx and ddy shader instructions to generate the partial derivatives of both the position and the height.

Given a position and normal in the same coordinate-space, and a height map sample, calculating the final normal is straighforward:

// Calculate the surface normal using screen-space partial derivatives of the height field
float3 CalculateSurfaceNormal(float3 position, float3 normal, float height)
{
	float3 dpdx = ddx(position);
	float3 dpdy = ddy(position);

	float dhdx = ddx(height);
	float dhdy = ddy(height);

	return PerturbNormal(normal, dpdx, dpdy, dhdx, dhdy);
}

Note that in shader model 5.0, you can use ddx_fine/ddy_fine instead of ddx/ddy to get high-precision partial derivatives.

So how does this look? At a medium distance, I would say that it looks pretty good:

But what about up close?

Uh oh! What’s happening here? Well, there are a couple of problems…

The main problem is that the height texture is using bilinear filtering, so the gradient between any two texels is constant. This causes large blocks to become very obvious when up close. There are a couple of options for alleviating this somewhat.

One option is to use bicubic filtering. I haven’t tried it, but I would expect this to make a good difference. The problem is that it will incur an extra cost. Another option, suggested in the paper, is to add a detail bump texture on top. This helps quite a lot, but again it adds more cost.

In the image below I’ve just tiled the same texture at 10x frequency over the top. It would be better to apply some kind of noise function as in the original paper.

The second problem is more subtle. We’re getting some small block artifacts because of the way that the ddx and ddy shader instructions work. They take pairs of pixels in a pixel quad and subtract the relevant values to get the derivative. In the case of the height derivatives, we can alleviate this by performing the differencing ourselves with extra texture samples.

The first problem is pretty much a killer for me. I would rather not have to cover up a fundamental implementation issue with extra fudges and more cost.

What Now?

It’s unfortunate that this didn’t make it into the original paper, but Mikkelsen mentions in a blog post that you can increase the quality by using precomputed height derivatives. This method requires double the texture storage (or half the resolution) of the ddx/ddy method, but produces much better results.

You’re probably wondering how you can possibly precompute screen-space derivatives. We don’t actually have to. Instead we can use the chain rule to transform a partial derivative from one space to another. In our case we can transform our derivatives from uv-space to screen-space if we have the partial derivatives of the uvs in screen-space.

To calculate dhdx you need dhdu, dhdv, dudx and dvdx:

To calculate dhdy you need dhdu, dhdv, dudy and dvdy:

The hlsl for this is very simple:

float ApplyChainRule(float dhdu, float dhdv, float dud_, float dvd_)
{
	return dhdu * dud_ + dhdv * dvd_;
}

Assuming that we have a texture that stores the texel-space height derivatives, we can scale this up in the shader to uv-space by simply multiplying by the texture dimensions. We can then use the screen space uv derivatives and the chain rule to transform from dhdu/dhdv to dhdx/dhdy.

// Calculate the surface normal using the uv-space gradient (dhdu, dhdv)
float3 CalculateSurfaceNormal(float3 position, float3 normal, float2 gradient)
{
	float3 dpdx = ddx(position);
	float3 dpdy = ddy(position);

	float dhdx = ApplyChainRule(gradient.x, gradient.y, ddx(uv.x), ddx(uv.y));
	float dhdy = ApplyChainRule(gradient.x, gradient.y, ddy(uv.x), ddy(uv.y));

	return PerturbNormal(normal, dpdx, dpdy, dhdx, dhdy);
}

So how does this look? Well, it’s pretty much the same at medium distance.

But it’s way better up close, since we’re now interpolating the derivatives.

Conclusions

In order to really draw any conclusions about this technique, I’m going to need to compare the quality, performance and memory consumption to that of normal mapping. That’s a whole other blog post waiting to happen…

But in theory, the pros are:

• Less mesh memory: We don’t need to store a tangent vector, so this should translate into some pretty significant mesh memory savings.
• Fewer interpolators: We don’t need to pass the tangent vector from the vertex shader to the pixel shader, so this should be a performance gain.
• Possible less texture memory: At worst this method requires two channels in a texture. At best, a normal map takes up two channels.
• Easy scaling: It’s easy to change the height scale on the fly by scaling the height derivatives. This isn’t quite so easy to get right when using normal maps. See here. As Stephen Hill points out in the comments below, this is a pretty weak argument, so I’m removing it.

And the cons are:

• More ALU: It’s going to be interesting to see the actual numbers, but this is probably the only thing that could put the nail in the coffin for derivative maps. The extra cost for ALU might be compensated partially by the fewer interpolators, but we’ll have to see.
• Less flexible: A normal map can represent any derivative map, but the reverse is not true. I’m not sure that this is a significant problem in practice though.
• Worse quality? I’m not sure about this one, but it’ll be interesting to see if the quality holds up.

I was impressed when I ran the demo with how smooth his UI looked. It turns out that he’s using a little trick (which I’d never seen before, but I’m sure is old to many) to smooth of the edges of his UI elements.

Basically, the trick is to create a ring of extra vertices by extruding the edges of the polygon out by a certain amount. These extra vertices take the same color as the originals, but their alpha is set to zero. Mikko calls this ‘feathering’.

In my case, I found that I got good results by feathering just one pixel. Here’s a quick before/after comparison of the my IMGUI check box at 800% zoom:

And here’s a 1-to-1 example showing rounded button corners:

It’s a pretty nice improvement for a very simple technique! If you’re interested in what the code looks like, then either take a look at Mikko’s IMGUI implementation, or you can find the code I use to feather my convex polygons below.

My implementation is a little less efficient since I recalculate each edge normal twice, but I chose to keep it simple for readability.

void CalculateEdgeNormal(float& nx, float& ny, float x0, float y0, float x1, float y1)
{
	const float x01 = x1 - x0;
	const float y01 = y1 - y0;

	const float length = Sqrt(x01 * x01 + y01 * y01);

	const float dx = x01 / length;
	const float dy = y01 / length;

	nx = dy;
	ny = -dx;
}

void FeatherConvexPolygon(Primitives& primitives, const Vertex* vertices, int count, float amount, const Texture* texture)
{
	Vertex* extruded = Memory::Allocate<Vertex>(Memory::Temp, sizeof(Vertex) * count);

	for (int i = 0; i < count; ++i)
	{
		const Vertex& previous = vertices[(i + count - 1) % count];
		const Vertex& current = vertices[i];
		const Vertex& next = vertices[(i + 1) % count];

		float nx0, ny0, nx1, ny1;

		CalculateEdgeNormal(nx0, ny0, previous.x, previous.y, current.x, current.y);
		CalculateEdgeNormal(nx1, ny1, current.x, current.y, next.x, next.y);

		float nx = (nx0 + nx1) * 0.5f;
		float ny = (ny0 + ny1) * 0.5f;

		extruded[i] = Vertex(current.x + nx * amount, current.y + ny * amount, Color(current.r, current.g, current.b, 0.0f));
	}

	for (int i = 0; i < count; ++i)
	{
		const int j = (i + 1) % count;
		AddQuad(primitives, vertices[i], extruded[i], extruded[j], vertices[j], texture);
	}

	Memory::Free(extruded);
}

Well, you could point out a number of things to answer that question. There’s some pretty obvious aliasing, a random pixel on the ground which should be in shadow but isn’t, it’s noisy, boring etc. But that’s not my point. The point is: It’s too dark!

I know it’s too dark because I know how I rendered it, and I rendered it wrong. It still kind of looks acceptable (well to me at least) though. I’m not sure that I would say that it’s implausibly dark if I didn’t know it.

How many bounces are enough?

I rendered this image using a Monte Carlo estimator with two bounces of indirect light. Each bounce estimated the irradiance at the intersection point using 256 rays in a cosine-weighted stratified-sampling pattern. It’s the two bounce part that makes it wrong. Any light that bounced more than twice before heading toward the camera is totally ignored. Since this approach doesn’t converge on the correct solution to the rendering equation, it’s classified as ‘biased’.

How much does light that bounced more than twice really contribute to a scene? Of course that depends on the materials in the scene quite a bit, but in this case, there’s a noticeable difference. I rendered the same exact scene using the Monte Carlo estimator for the first bounce, but for the subsequent bounces I used a path tracer. By using Russian Roulette (a topic unto itself) to terminate the path, you can get an unbiased approximation of the irradiance.

Ok, great. It’s brighter, but is it actually correct? I was wondering about this, then I happened to come across an idea while reading Physically Based Rendering to compare the results of my integrators with something that has an analytical solution to the rendering equation.

I now present an analytical solution to the Rendering Equation!

(… in a very simple case)

Solving the rendering equation analytically for most scenes is just impossible, that’s why we have to rely on numerical methods like Monte Carlo Estimation. However, the book suggests a very simple scene for which it can be solved. The suggested setup is that of light bouncing around the inside of a sphere. The sphere emits light internally, and reflects it diffusely to other points on the inside of the sphere. Since the sphere is rotationally invariant and the reflections are diffuse, every point on the sphere reflects and emits the same radiance in all directions.

So how do you solve the rendering equation for this situation? It’s fairly easy. Recall the rendering equation:

In this setup, we are using a diffuse BRDF with reflectance d. It’s also normalized to maintain energy conservation

Also, as mentioned previously, the outgoing radiance in all directions is the same as the incoming radiance, so this reduces the rendering equation to:

Solving this equation for L is now pretty easy. First we have to convert from an integral over solid angles, to a double-angle version. The important thing to remember when doing this is to introduce the new sine term:

There’s a double angle trigonometric identity that makes this easier:

So we just need to integrate the following:

Integrate over theta:

This integral over theta is just 1, so now integrate over phi:

This integral is of course just 2 Π. So we’re left with a very simple equation:

Solving for L gives us the final expect radiance at every point in the sphere:

This intuitively makes sense. As d grows, so does L. When d is 1, all hell breaks loose, and when d is 0 all we are left with is the emitted light. Obviously d can never be greater than 1 or the energy conservation rule would have been broken.

The test application

Alright, so all I need to do now is make a test application that fires a bunch of rays around the inside of a sphere and compare the results to the analytical solution. Well… So I thought. Due to the curvature of the inside of the sphere, I found that a good number of rays I fired near the horizon were escaping the sphere.

Currently I apply an epsilon to the minimum ray intersection to try and prevent self-intersections and this causes problems. For now (and just for these tests), instead I’m pushing the ray starting point away from the intersection a small amount in the direction of the intersection normal. I made the sphere really big too. I’d welcome any better ideas for alleviating this problem.

For the tests below, I set the emitted light value to 1, and the diffuse reflectance, d, to 0.5, meaning that the outgoing radiance, L, should be 2.

Multi-bounce Monte Carlo results

You can work out from the radiance equation how different numbers of bounces of light will affect the final solution. In this case, I just ran my Monte Carlo integrator from 0 to 16 bounces and produced the following graph showing the percentage (of the expected result) absolute error.

You can see that the error starts off really high, but drops off pretty quickly as more bounces are added. Each subsequent bounce has less and less effect on the error reduction, as expected.

What you have to remember though, is that each bounce adds exponentially to the number of rays that have to be cast to achieve that error. In real-world situations we need to cast a lot of rays over the hemisphere in order to get a correct solution. To get under 5% error you’d need four bounces with this method. Even using a very modest number of rays per estimation quickly becomes unruly: 256 rays with 4 bounces = over 4 billion rays needed! Per pixel! Without anti-aliasing!

Path Tracer results

Ok, so how about the path tracer? In theory the path tracer should average out to zero error, but can have potentially very high variance. Here’s the average error over 1000 runs with different survival probabilities:

Well, it’s not zero everywhere, but it’s a pretty close in most cases. I think that the results obtained from this kind of method probably depend quite a bit on the quality of the random number generator used. I’m just using the stdlib version, so I’ll try switching it up at some point to see how it affects things. I haven’t shown it here, but the variance at low survival probabilities is incredibly high. This surely accounts for how far off the results are under about 10% survival.

The good thing about the path tracer is that it doesn’t require exorbitant numbers of rays. Yes, the variance can be high, but it can be reduced by tracing more paths. The net result is that it produces low error images much quicker than using standard Monte Carlo integration.

The End

Well, the point of this post really was: How do you know your integrator is correct? Comparing it to something simple that can be calculated analytically is not the be-all and end-all, but it’s a good start. There are many other things that could still be wrong even if your integrator gets good results against this simple test, like how you’re calculating sample directions, your PDF etc.

For me, being able to compare my integrators to a known solution definitely helped me to make sure that my path tracer was getting correct results.

Things have changed a lot with Direct3D 11. The rendering API has been separated from the factory functions into a separate object called the device context. The factory functions on the device are all free threaded, meaning that they can be called from any thread. The device context functions are designed to be called from the same thread.

The basic idea behind multithreading in Direct3D 11 is that you create an immediate device context on the main thread. Then, for each thread on which you’d like to be able to render, you create a deferred context. As you can probably guess from the names, commands executed on the immediate context get executed immediately, but those on the deferred context just get saved off into a command list. You then execute the deferred command lists on the main thread using the immediate device context. Sounds easy enough.

Thread Pools

Given that you can submit draw calls to deferred contexts on multiple threads, it makes sense to ditch the single rendering thread concept and switch to using something like a thread pool for issuing the draw calls. This scales far better than a dedicated rendering thread. It’s also pretty easy to set up a simple thread pool, and give each worker thread a deferred render context.

There are plenty of places on the internet to read about thread pools so I’m not going to get into it here, but one thing I can’t stress enough is to make sure that you get your synchronization right! In my initial implementation, I used my normal queue data structure, but wrapped it up in mutexes (mutices?) to make sure it was thread-safe. This worked out well since I was very confident that things were working correctly, but a quick foray into VTune told me that I was spending 40% of the time waiting on synchronization points!

After some quick digging around, I came across a few articles that Herb Sutter wrote for Dr Dobb’s Journal about producer/consumer queues. I implemented the low-lock queue recommended by Sutter, and got a good speedup of at least 30% (that number is off the top of my head, but I remember it was a lot). The relevant articles I read are single producer/consumer queue, generalized concurrent queue, and measuring performance. I still use events for sending the worker threads to sleep when there is nothing left to work on, and to wake them up when data is added to the queue.

My application already stores up all of the state needed for a draw call in an object called a RenderContext, so instead of passing off this render context to the renderer on the main thread, instead it just gets enqueued to be rendered by one of the threads in the thread pool. When the worker thread gets to it, it passes the render context off to a thread-local renderer object initialized with a deferred device context. This renderer sets all of the changed state and issues the final draw call.

Finally, back on the main thread, it waits until all of the render contexts have been submitted to the deferred device contexts, and then executes each of these on the immediate device context.

Test Scenario

In order to stress my renderer a bit, I fabricated a scenario with 10,000 models. Each model has a sphere and a ground plane with their own material. I use a loose octree for culling out the models outside of the frustum, but I don’t do any sorting of any kind. This means that the alternating materials that get rendered for the sphere and then the ground put a fair amount of stress on the CPU side of the renderer.

My single threaded renderer took about 50 ms to render the intial view of the scene. By switching to using the thread pool, this went down to about 30 ms. A nice improvement, that’s for sure. Obviously, as fewer objects are visible, the gains of using the multithreaded renderer disappeared.

Profiling

I was happy that the multithreading appeared to be doing its job, but I wasn’t quite satisfied because I couldn’t really tell how well it was doing. Time for some profiling!

There appear to be quite a few CPU profilers out there. First of all I downloaded an evaluation of Intel VTune. It’s pretty overwhelming, but it gave me a lot of pertinent information. The bugger is that you have to pay a hefty sum for it, so I tossed it out of the window. I also tried out Microsoft xperf. This sampling profiler gave me a pretty good overview of what was expensive with the standard inclusive/exclusive view. It was a great help for quickly tracking down some areas of the code that I could very easily improve. I still use this.

The trouble with most of the sampling profilers is that they don’t know about frames. They just add up all of the samples over the given time period which gives you an idea on average what is happening. I wanted to get information about what was happening within the frame, so I implemented a really simple frame profiler.

Frame Profiler

An in-game profiler is a really handy tool to have. It lets you see in real-time exactly how your CPU time is being spent in one frame on each of your threads. It’s also pretty easy to set up.

First of all, I created a class called ThreadProfiler. As the name suggests, the ThreadProfiler class is responsible for recording events on a specific thread. This class has functions to notify it of the beginning and end of the frame, as well as when a profiling event begins and ends. All it really does is to record the name of the event, a color for display, and the timestamps when the event begins and ends. The events can be nested, so it maintains a stack of active events and records the depth of the stack for each event.

Next I created the singleton FrameProfiler class. The idea for this class is to hold all of the ThreadProfiler objects, and to forward events onto those classes based on the current thread ID. Threads are required to register their thread ID with the frame profiler in order for events to be recorded.

        class FrameProfiler : public Core::Singleton<FrameProfiler>
        {
        public:

            FrameProfiler();

            void RegisterThread(int threadId);

            void BeginFrame(bool enabled);
            void EndFrame();

            void BeginEvent(int threadId, const Core::String& name, uint32 color);
            void EndEvent(int threadId);

            DataStructures::ArrayList<ThreadProfiler>& GetThreadProfilers();
            const DataStructures::ArrayList<ThreadProfiler>& GetThreadProfilers() const;

        private:

            DataStructures::ArrayList<ThreadProfiler> m_threadProfilers;
        };

The final piece is a really simple macro which grabs the function name and creates an object which tells the FrameProfiler when it is created and destroyed. This is the macro that I place into whatever function or loop I’d like to profile.

class ScopedProfileEvent
{
public:

        ScopedProfileEvent(const Core::String& name, uint32 color)
        {
                if (FrameProfiler::IsCreated())
                {
                        FrameProfiler::Instance().BeginEvent(Core::Platform::GetCurrentThreadId(), name, color);
                }
        }

        ~ScopedProfileEvent()
        {
                if (FrameProfiler::IsCreated())
                {
                        FrameProfiler::Instance().EndEvent(Core::Platform::GetCurrentThreadId());
                }
        }
};
#define PROFILE(X) const Profile::ScopedProfileEvent event__LINE__(String(__FUNCTION__), (uint32)X)

Unlike a sampling profiler, this kind of profiling has a certain amount of processing overhead. There are a couple of quick things you can do to help with this though. The first is just to make sure that you don’t always have the overhead, and compile it out for your final builds. It’s important to do your profiling on an optimized build, so I would recommend debug, release, and final configurations or something similar. The second thing you can do is to just not run it every frame. I have it set on a key press so that I can get to the area I’d like to profile without the overhead, then hit the button to profile the next frame and display the results.

I’m not sure about how accurate this would be, but you could probably compare the previous frame’s duration to the profiled frame to get a rough estimate of the overhead that the profiling functions added. I wouldn’t rely on that though.

There’s actually quite a bit of information that can be gleaned from these profiling events, but the first thing I did was to render out the events as rectangles on a timeline. In the image below, I have two threads running. The main thread at the bottom has three levels of nested events being shown, and the top worker thread just has one.

Ok, there’s no legend right now, but I’m working on it. Each black/grey bar in the background represents one millisecond of frame time.

The bottom row on the main thread represents the update in green, the render in blue, and the call to Device::Present in red. Given the long red bar, I’d say I’m GPU limited in this scene.

The row above represents the breakdown of the render function from the bottom row. The cyan sliver is shadow rendering (actually I’m not rendering any shadows which is why it’s tiny). The huge magenta bar is the model rendering, and the yellow bar is post-processing.

The top row in the bottom thread represents the breakdown of the model rendering function. The green slivers are models being found in the octree and the red blocks are models being prepared for rendering. The large white bar is actually the command list from the worker thread being executed on the immediate device context. I was pretty surprised to see this segment so large, since I didn’t notice it in the other profilers at all.

Experiments

Now that I have a frame profiler, I can really experiment with my thread pool setup to see how it affects the frame. My computer has a dual core processor, so based on Sutter’s articles, I was expecting that one main thread and one worker would be the best setup. Even so, I tried running a variety of numbers of worker threads to see how it looked. Here’s what four threads looks like:

The first thing I noticed was just how much worse all of the threads fared. Each worker thread appeared to perform a tiny bit of work, and then get swapped out for another thread. The main thread really suffered due to this too. This is a great example of how visualizing this data is really illuminating. The scene was already GPU bound, so even though the rendering code was performing far worse, the frame rate actually stayed the same.

Another experiment I wanted to run was just how much other applications could affect the frame rate of my application. In this case, I just had sysinternals process exlporer running and polling the system processes every half second. It only took me a few tries to hit a frame where I could see the effect:

Notice the scale of the millisecond bars now – this frame took over twice as long to run as my first example with the exact same setup. You can see a big gap on the worker thread where another process stole its time. Event when it did get some time, it appears to be running very slowly.

Also, you can now see a large grey bar in the middle row of the main thread which shows the main thread waiting for the worker thread to finish.

The execution of the command list is pretty consistently taking up three and half milliseconds or so. This is much higher than I had thought it would be. I really hope that this time gets reduced with newer drivers or hardware.

One last thing I’ve done to investigate what is happening in my application is to display the frame rate history. I use a moving average to calculate the frame time, so I have the last 100 frames stored anyway. It’s a simple enough task to just display this.

You can see how varied the frame times are even though the camera isn’t moving. This is probably due to other processes on my computer interfering I’d imagine.

Final Thoughts

It was a fun adventure porting my code to Direct3D 11, particularly implementing a multithreaded renderer using a thread pool. I would recommend trying it out to those of you who have Direct3D 10 engines at the moment.

The jump from Direct3D 10 to 11 is nowhere near as bad as the previous jump from 9 to 10. It took me about three hours to change my rendering code to deal with the changes. The most awkward part was probably having to pass in the device context to functions which need to map buffers, since these functions are no longer on the buffers themselves.

Visualizing profiling data in real time can be a real eye-opener for understanding how your code is actually running rather than how you think it may be running. It has really helped me identify good candidates for moving to using the thread pool as well as pointing out areas of the code that are taking a surprisingly large amount of frame time.

First of all, I’d like to start by looking at the standard diffuse reflection model. In games, the typical formula for calculating diffuse reflection from a particular light is:

Where Cd is the diffuse material color, Li is the light color, N is the normal, and L is the normalized direction to the light. What’s the problem with this? Well, it’s not energy conserving. In itself, this isn’t really a problem since we don’t calculate multiple bounces of light in games, so we’re not adding energy to the scene as light bounces around like would happen in a ray tracer. It’s a good starting point for discussion though.

Energy Conservation

As the name suggests, energy conservation is a restriction on the reflection model that requires that the total amount of reflected light cannot be more than the incoming light. It sounds sensible, but it’s often not practiced. A more formal way of stating this restriction is:

The function ρ represents the bidirectional reflection distribution function (BRDF), and could be anything from a simple Lambertian diffuse model, to a complicated microfacet model. Either way, the energy conservation restriction still stands.

Diffuse Energy Conservation

I’m going to show why the diffuse lighting equation above isn’t energy conserving, and how to make it so. Let’s start by replacing the BRDF with the constant diffuse color:

The incoming light direction is fixed here, and we are integrating over outgoing directions. Because of this, both Cd and Li are constant over the integral, and can be pulled outside. Also, the incoming light appears on both sides of the inequality, so we can divide by Li leaving:

This integral can be solved analytically. First of all, we need to rewrite it as a double integral of the two polar coordinates φ and θ:

The extra sin θ may seem a little bit confusing at first, but it’s necessary to take into account the smaller area towards the polar region. Using the double angle trigonometric identity, this is the same as:

Now we can start using some trigonometric integrals to integrate, firstly over θ:

This integral completely disappears down to nothing:

So now, integrate over φ:

This gives us the final inequality:

Assuming that we want to keep our diffuse material color in the range [0,1], all this says is that we need to divide it by π in order to remain energy conserving. Since this is just a constant scale, it might not be worth doing in a game if the only lighting model it uses is diffuse. Most games use a more sophisticated reflection model though, at least including specular reflections.

Specular Energy Conservation

The standard Blinn-Phong specular model is also not energy conserving. In fact, in some ways it is even worse than the diffuse model, because as you increase the specular power, you lose more and more energy. A manifestation of this problem can be that artists find it hard to get a really tight specular highlight.

Here’s an example of a sphere rendered at three different specular powers. Notice how much light there appears to be on the left image, and how it seems to have disappeared on the right, even though it’s supposed to be more focused. It’s at this point that artists start to ask to ramp the specular reflection color over one to compensate. This isn’t a good idea!

Instead of boosting the specular reflection color, you can switch to an energy conserving specular model. If you do this, the same spheres with the same specular powers now look like this:

The specular reflection on sphere on the left is actually dimmer than the non-conserving model. This is because the non-conserving specular model reflected too much light in this case. As the specular power increases, this time we compensate for the energy loss, and you get a really nice and tight hotspot on the right.

The typical Blinn-Phong reflection model is:

Somebody who knows more than me has worked out that to ensure energy conservation, the normalization factor is:

[Edit: Thanks to Fabian "ryg" Giesen for showing that this is actually the normalization factor for the regular Phong specular model, not Blinn-Phong. A commonly used normalization factor (according to Real-Time Rendering) for Blinn-Phong is shown below.]

So this makes the energy conserving function for Blinn-Phong specular reflection:

I’d love to know how this was derived, but I haven’t found anything so far that explains it. The integral for raising a cosine function to a power gets pretty hairy very quickly. All I know is that it appears to work. If anyone reading this knows why, then please let me know!

[Edit: Fabian "ryg" Giesen very kindly posted up a derivation of the normalization factor for both the regular Phong, and Blinn-Phong specular reflection models here. Interestingly he doesn't come up with the exact same answer for the Blinn-Phong normalization factor as shown in Real-Time Rendering. Check out the comments below to find out more about this.]

Combined Diffuse and Specular

Once you have energy-conserving models for diffuse and specular,  it’s easy to make sure that the combined model is also energy conserving. You just need to make sure that your diffuse and specular material colors don’t sum to more than one:

This means that if you want your material to have more specular, you may have to reduce the diffuse. Here’s a range of variations of the same material, going from 100% diffuse to 100% specular:

Is It Worth It For Games?

Clearly, the specular model benefits significantly from being energy conserving so I think most people would say that it’s worth it. Switching to a model where the diffuse and specular have to compete for energy might not be though, since it’s can be harder to tweak. I personally use this kind of model for my projects at home, but that’s because I don’t have artists to please.

One thing I do like about an energy conserving reflection model is that it enforces some kind of reasonable limits to the material reflections. This might help to make materials created by different people sit better together.

Gradients

The gradient of a function represents both the direction and rate of change of that function as the inputs vary. For a one dimensional function this is simply the derivative of the function. As you move into higher dimensions, you need to consider which coordinate system the inputs for the function are specified in, as this will change how you need to calculate the gradient.

For now, I’m just going to focus on calculating the gradient of a function defined using normalized spherical coordinates. Unfortunately, there’s no real standard way to define spherical coordinates, and despite similar looking symbols, the values are often interchanged. I’m going to define the spherical coordinates on the unit sphere as azimuthal value φ [0, π), and polar value θ [0, 2π).

When dealing with multiple dimensions, you can calculate the gradient by splitting the gradient calculation into multiple partial derivatives and summing them with appropriate vector weights. For normalized spherical coordinates the gradient is:

The function f represents a scalar field, so the gradient is a vector field. Each vector points in the direction of greatest increase. Much like the integrals of functions described using spherical coordinates, you have to take care to weight the azimuthal contribution by the sine of the polar angle.

As a real-world example of using gradients, let’s calculate the gradient for a simple function:

The derivatives of the function for each argument are easy to calculate:

Combining these together with the previous definition, you can calculate the gradient vector at any point on the unit sphere for f using:

There’s lots more to be learned about gradients, and a good start would be Wikipedia, and also this page on the MIT website.

Rotational Irradiance Gradient

The irradiance contribution from a direction on the hemisphere about the surface normal, specified using spherical coordinates φ [0, 2π) and θ [0, π / 2) is:

Where L is the incident radiance in the supplied direction. So to calculate the gradient vector at any point on the hemisphere, you just need to evaluate:

This just calculates the gradient in a specific direction, but for the irradiance gradient we need to calculate the average gradient over the entire hemisphere. We can do this at the same time as we calculate the irradiance by using a similar Monte Carlo estimator. We want to share the sampling strategy between the irradiance calculation and the rotational gradient calculation, so we’re stuck using the same pdf:

So the estimator for the irradiance gradient becomes:

Which collapses down to:

The vector v is a unit vector on the plane of the hemisphere pointing in the perpendicular direction to the angle φ. There are two perpendicular vectors to φ, and which one you decide to use depends on the order you do the cross product on when evaluating the gradient. Using the left hand rule for rotation, I’m doing a clockwise rotation of φ by ninety degrees.

Using the Rotational Irradiance Gradient

The irradiance estimate at a point is defined by a weighted sum of irradiance cache entries:

Now we have the rotational gradient, we can use it to improve the estimate. The cross product of the surface normal and cache entry normal represents both a direction and magnitude of rotational difference. We then project this difference onto the irradiance gradient to calculate how much the irradiance is changing in that direction:

Note that conceptually this calculation needs to be performed once for each color channel, since the gradient of the irradiance is really the gradient of three scalar fields – red, green and blue. In practice it’s easier to assume that the gradient is a three dimensional vector of colors, rather than scalars.

Implementation

The implementation is pretty straightforward, but there are a couple of optimizations that can be made because we are using a unit hemisphere for sampling. Assuming that we have the sample direction in Cartesian coordinates in local space, where the z-axis points in the polar direction, we can get the cos weighting from the z coordinate:

Also, we can get the sine weighted projected unit vector by simply setting the z value to zero, since:

So, to get the local space tan weighted perpendicular vector, we just need to use the following:

Note that the x and y values were swapped, and the x value negated to get the perpendicular vector.

Results

I’ve rendered out a before and after shot, showing just the indirect irradiance. There are no translational gradients being used at the moment, so there are still some artefacts. Here’s the before shot:

And here’s the same render, but with rotational gradients this time:

Note that the time to render each frame is almost exactly the same, yet the rotational gradients provide a much smoother result. Next I’ll implement translational gradients and hopefully the image will look considerably better.

One interesting observation made by Greg Ward in his Siggraph ’88 paper is that contrary to direct lighting, where shadows and lights can cause harsh changes, the indirect lighting on a surface tends to vary relatively slowly. One way to picture why this is, is to imagine the computing average color from the what you can see from each of your eyes. Even though each eye has a slightly different view on the world, the images they see are nearly similar, and so the average color is also nearly the same.

The image below shows the same scene from my previous post with just the indirect irradiance, and it’s pretty clear that for each surface, the lighting varies in a very smooth fashion.

Ward proposed using this knowledge to reduce the number of times that the Monte Carlo estimator was evaluated by interpolating between nearby previously calculated values. At the time he just called it ‘lazy evaluation’, which I personally think is a good way to picture the idea. Later it became known as irradiance caching.

Irradiance Caching

The basic concept for irradiance caching is really simple: For each point on a surface at which you want to evaluate irradiance, if the cache contains any valid entries then interpolate between them. Otherwise, calculate a new irradiance entry, and add it to the cache.

A cache entry contains the position and normal for the point on the surface where the irradiance was evaluated as well as the irradiance value itself. One important additional piece of information that the cache requires is the range over which the entry is considered potentially valid. This range could be calculated in a number of ways, but the most common one is to use the harmonic mean of the hit distance of the rays used for the estimator. For n estimator samples, each with hit distance d, the harmonic mean is simply:

Using the harmonic mean distance makes the cache entry distribution very dense in corners and crevices, and sparse in open spaces. This matches up very well with where the indirect irradiance is likely to be changing the fastest. To get an idea of how the cache entry distribution looks, here’s the scene above with the cache entry positions shown as red dots:

Once you can add entries into the cache, you need to know how to find whether or not a particular cache entry can be used for interpolating the irradiance at a sample point. There are potentially quite a few ways that you can discard invalid cache entries depending on how fancy you want to get. For now, I’m using three simple tests.

Discard the entry if any of the following are true

• It is out of range of the sample point.
• It has a normal that is too different than the sample normal.
• It is in front of the sample point.

Once you have a valid cache entry, you need to calculate a weight for that entry, then carry on looking for other entries that are potentially valid. As you come across each valid cache entry, you need to keep the sum of the weighted irradiance values, and the sum of the weights themselves. From these two sums, you can calculate the final interpolated irradiance:

The weight for a particular cache entry is another part of the algorithm that can potentially be calculated in many different ways. For now, I’m using the weight that Ward proposes, but there’s some interesting information about the weights used at Dreamworks in this paper. Here’s Ward’s initial weighting function:

Note that you have to be a little bit wary of this function, since it is unbounded. When the sample point lies exactly at the same point as the cache entry then there you will get a divide by zero.

Typically, you would also discard cache entries that are below some weight threshold as specified by the user. This effectively scales the density of the cache entries and allows the user to make the trade off between speed and quality.

Implementation

I’ve made a very bare bones implementation of irradiance caching as outlined above. At the moment I’m not using a quad tree to store the cache entries, so each cache check requires iterating through an array of entries. Clearly this is a very slow way to process the cache entries, but for now it does a decent enough job to allow me to focus on the irradiance caching algorithm itself. Here are the results:

Not very impressive, or smooth, is it? I was hoping that the simple implementation I have made would provide better results than this, but apparently not. At the moment there’s one crucial improvement to the algorithm that my implementation is missing though – Irradiance Gradients. Irradiance Gradients basically give a better clue as to how to interpolate the irradiance cache entries, both positionally and rotationally. I’m hoping that they will significantly reduce the artefacts visible at the moment.

One problem that can occur when using an irradiance cache is that later cache entries don’t contribute to previously rendered pixels. When this happens, you can see blocky artefacts where the irradiance values have been interpolated differently. Something like this:

One thing you can do to avoid this situation is to perform an irradiance gathering pass before doing the final render. When you perform the final render, you should have no cache misses. In my case, I am using a progressive renderer, so the cache is actually fairly well primed before rendering the 1×1 pixel size.

Improvements

In addition to irradiance gradients, there have been a load of improvements made to irradiance caching since the inital paper. The course notes for the Siggraph 2008 course provide details of many of these. I’ll post up some screenshots when I’ve added the irradiance gradients.

My approach for generating the ambient occlusion rays was to generate uniform random samples over the hemisphere about the normal. Based on two random numbers in the range [0,1), I calculate the normalized sample direction using the following function:

Vector3 Sample::UniformSampleHemisphere(float u1, float u2)
{
	const float r = Sqrt(1.0f - u1 * u1);
	const float phi = 2 * kPi * u2;

	return Vector3(Cos(phi) * r, Sin(phi) * r, u1);
}

This generates points on a hemisphere from uniform variables u1 and u2, where each point has equal probability of being selected. The following image was generated with 256 random uniform samples:

It looks pretty noisy, that’s for sure. Part of the trouble comes from the fact that there’s no way to ensure that there’s an even distribution of the rays. A common way to alleviate this problem is to do stratified sampling instead of fully random sampling. The idea of stratified sampling is to split up the domain into evenly sized segments, and then to pick a random point from within each of those segments. You still get some randomness, but the points are more evenly distributed, which in turn reduces the variance. Less variance means less noise. Here’s the scene again, using 256 rays, but this time using stratified sampling:

As expected, it’s much less noisy, and for the same amount of computation!

Sampling for Diffuse Monte Carlo Estimator

The stratified sampler helps out with the indirect diffuse lighting calculation too, but one other thing you can do to reduce noise for the Monte Carlo estimator is to choose random values that have a similar ‘shape’ to the integral you are estimating. Looking at the integral for diffuse reflections, you will see the familiar cosine term inside the integral:

Where c is the diffuse material color, Li is the incoming radiance, and pi is the energy conservation constant.

Rather than wasting samples on areas of the integral where they will get mulitiplied out by the cosine term, why not just choose proportionally fewer samples in those areas?

Recall that the Monte Carlo estimator for an the integral of the function f(x), with probability density function p(x) is:

The probability density function is just a function that returns the probability that a particular value will be chosen. For the uniform hemisphere sampling function above, the pdf is just a constant, (1 / (2 * pi)). This makes the Monte Carlo estimator for the diffuse integral:

Rather than mutliply by the cosine term above, we just want to generate proportionally fewer rays at the bottom of the hemisphere. The integral of the pdf over the hemisphere must equal one, so by switching to a cosine-weighted sample distribution, the pdf becomes (cos(theta) / pi).

This makes the estimator:

Which cleans up rather nicely to:

Normally I would post a couple of images up for comparison’s sake, but in this case, the difference is pretty difficult to perceive without being able to compare one on top of the other. The difference is small, but it is definitely worth it!

The common way to generate a cosine weighted hemisphere sampler is to generate uniform points on a disk, and then project them up to the hemisphere. Here’s some code:

Vector3 Sample::CosineSampleHemisphere(float u1, float u2)
{
	const float r = Sqrt(u1);
	const float theta = 2 * kPi * u2;

	const float x = r * Cos(theta);
	const float y = r * Sin(theta);

	return Vector3(x, y, Sqrt(Max(0.0f, 1 - u1)));
}

Just by doing these two small steps, I’ve been able to clean up my images significantly. Here’s the scene from above again, this time with single bounce final gather with 256 rays, stratified cosine-sampled:

Next on my list is to take a look at path tracing, followed by irradiance caching (wasn’t that the point of all this?). This should allow me to get fairly cheap multi-bounce diffuse lighting.

What have I been doing? Well, as I was approaching the break, I read through the course notes from the Practical Global Illumination with Irradiance Caching course at Siggraph last year. I thought the course itself was really good, and very clearly presented. After blitzing through the notes again, I thought I’d have a go at writing a ray tracer. It seemed simple enough at the time, but like most things, the devil is in the details.

The first thing I did was to set up a really simple single-threaded ray tracer that just displayed the color of the surface it hit. This was fairly quick to get up and running once I had written a few supporting classes for the cameras and shapes. It’s not very glamorous, but it’s a start:

Next, I added point lights and directional lights, and wrote a new integrator to calculate direct diffuse lighting. Once you have a function to trace rays around the scene, it’s really easy to add hard-edged shadows. It looks a lot better than the solid color integrator I first used, but it still not very impressive.

Here’s the scene with a single directional light and hard-edged shadows:

I wanted to flex the ray tracer a little bit, so and easy next step was to add an ambient occlusion integrator. Initially, I just used a function generate random uniform rays on the hemisphere around the hit normal, and used the ratio of misses to hits as the occlusion value. I found that this was really pretty noisy, so I tried using the length of the ray hits to weight the occlusion values. This definitely improved things, but it’s still pretty noisy. The obvious way to reduce the noise is to use more rays, but I’d like to find a cheaper way to do this if possible.

Here’s the scene rendered with the ambient occlusion integrator using 4096 rays per hit:

The first time I tried to render this scene using 4096 ambient occlusion rays per pixel, it took about thirteen minutes. I’ve never really used release builds at home and the settings weren’t great, so I tweaked some of the project settings, and defined out asserts. This got the time down to about ten minutes. I’m running these renders on my Macbook Pro, so I have a whole other core just sitting there doing nothing. Switching to using a multithreaded renderer basically sped the renders up by a factor of two.

Combining some of the concepts of the ambient occlusion integrator, and the direct diffuse integrator, I created a multi-bounce diffuse integrator. Like the direct diffuse integrator, it calculates the direct diffuse lighting at the hit point. Additionally though, it uses a Monte Carlo estimator to approximate the diffuse lighting integral over the hemisphere about the normal of the hit point. It can handle any number of bounces of indirect light, but the render time increases exponentially with each bounce added. Like the ambient occlusion integrator, it requires a large number of sample rays to get an acceptable level of noise.

Here’s the scene again with one bounce of indirect light, and 4096 rays per hit:

When a ray misses the scene, it looks up an environment color, which you can see in the background. Most of the indirect rays actually miss the scene, so this background color actually has a huge effect over the look of the scene. I should mention as well that I’m using a really simple tone mapping operator to map the HDR ray tracer values down to the 8 bit per channel texture.

While working on the ray tracer, I would often be playing around with the objects and lights in the scene. I quickly found out that it’s really not very fun to wait for the ray trace to complete before getting some feedback. I can reduce the number of indirect rays to make things quicker, but even at relatively low values, it can take a while to render the final scene.

I had already split the rendering of the scene into 32 by 32 blocks when I switched to a multi-threaded ray tracer, so it was a really simple extension to change the resolution in each of these blocks on the fly. I basically start things off by rendering with each ray covering 32 by 32 pixels, then when that completes, I immediately kick off another render at 16 by 16, and so on. Each successive render takes four times as long as the previous render, so if the 1 by 1 render takes about a minute, then you get the 8 by 8 render in about a second!

Here’s the scene rendered using 512 indirect samples, paused at the 4 by 4 resolution:

And here’s the scene at the conclusion of rendering (note that the time is cumulative of all the previous renders):

It’s pretty clear at the 4 by 4 resolution how the render is going to look, and it only took four seconds to get there, whereas the final scene took nearly a minute. The 1 by 1 resolution actually took only 40 seconds of that minute to render, but still, having the feedback within a tenth of the final render time seems worth the extra wait at the end.

That’s basically as far as I got over the past couple of weeks. Like many things I do, there seems to be more to do now than at the beginning. One of the things I’d really like to do is to be able to render out the lighting to radiosity normal maps. This would allow me to combine the static precomputed lighting in my DirectX10 engine. I could also output spherical harmonic coefficients for light probes which would allow me to render dynamic objects using the precomputed lighting.

Well, work starts back up in a couple of days, so the amount of time I can spend on this is going to be limited again, but I’ll post any significant updates. I have another article about the the lighting calculation on the the way, but it’s competing for my time!

As seemed to be typical for the time, my introduction to the industry was pretty much a trial by fire. I quickly found out that I couldn’t hope to truly understand every single new thing I encountered, so I learned to just accept some things as the truth. For example, I was told that the dot product of two normalized vectors yields the cosine of the angle between them. I just accepted this, and only took the time to find out why later on.

One of the things I accepted at the beginning was the maths used to perform the lighting of our models during rendering. While I could understand how the equations appeared to yield decent looking results, I never understood where they came from, and why they worked.

After a while I began to wonder about this… Where did the equations for diffuse and specular reflections come from? What are the units of brightness we use for lights? What are the units for the pixels that get rendered?

It took lot of reading, re-reading, and re-re-reading, for me to really understand some of these things, so now that I have a blog, I thought I would share what I learned just in case anyone else is wondering about these things too.

The Rendering Equation

When light hits a point on a surface, some of it might get absorbed, reflected or possibly even refracted. Also, there may be additional light being emitted from that point by a power source, or perhaps scattered in from another point on the surface. Things can get complicated pretty quickly!

Luckily for us, some smart guys came up with something called the rendering equation to deal with these factors. The rendering equation can produce incredibly realistic-looking images, but in its original form, it can also be very costly to evaluate.

I’ve included a slightly simplified version of the rendering equation below. If you compare this to the version currently on Wikipedia, you’ll see that I’ve removed a couple of parameters, t and lamba.

Typically if you have something like the power output of a light varying over time, it’s a better to idea to evaluate it before trying to solve the rendering equation. By doing this, you can assume time is constant, and so you can ignore it.

The lambda symbol in the original equation represents a dependency on the wavelength of the light. Without this, everything would be greyscale, so it’s an important property. Rather than dealing with wavelength explicitly, we can just treat the red, green and blue color channels independently and solve the rendering equation once for each channel. Note that in practice we end up using per-component vector mathematics to solve the equation for all three channels at the same time.

At first, this may seem a little bit intimidating, but actually it’s fairly simple. I’m going to break down each part and explain what it means.

Outgoing Light

This says that the rendering equation is a function which gives you the outgoing light in a particular direction w from a point x on a surface.

Emitted Light

This is any light that is being emitted from the point. Most surfaces don’t emit light, so normally you don’t see any contribution here.

Integral

This says that the enclosed functions need to be integrated over all directions w’ in the hemisphere above x. The orientation of the hemisphere is determined by the normal, n.

BRDF

This is the bidirectional reflectance distribution function (BRDF). It’s a fancy name for the ratio of the amount of light reflected in a particular direction w, to the amount received from another direction w’. The BRDF warrants its own discussion, but for now it can just be thought of as the reflection amount.

Incoming Light

This is the incoming light at the point x from the direction w’. Note that the incoming light doesn’t have to come from a light source (direct light). It may have been reflected or refracted from another point in the scene (indirect light).

Normal Attenuation

This attenuates the incoming light at x based on the cosine of the angle between the normal n and the incoming light direction w’.

Making it Real-Time

We use the rendering equation to perform lighting calculations in games, albeit in a simpler form. The most obvious problem for evaluating the rendering equation in a pixel shader is the integral, so we need to find a way to approximate it. One thing we can do is to split the way we deal with direct light and indirect light.

Since the indirect light is the harder of the two to deal with, we can approximate it. There are various different ways you might want to approximate the indirect light, from a simple ambient color, to more complex forms like spherical harmonics. Typically you would modulate your indirect light approximation with the diffuse part of your BRDF, since you don’t have a directional component to use in direction-dependent BRDFs.

By approximating the indirect light, we only have to worry the direct light in the rendering equation, so we can replace the integral with a simple sum over k light sources. Also, we typically model the change in the BRDF over space by using textures mapped over the surface, so x is no longer needed for that function.

I’ve changed up the notation here to match more closely what you see in games literature. The vector v is the normalized direction from the point x to the camera. The vector l_i is the normalized vector from light source i to the point x.

It’s not explicitly written here, but the dot product now needs to be clamped to zero to avoid negative light values.

This may not look much like the ‘diffuse plus specular’ equations you commonly see in games, but it’s actually pretty close. In fact, imagine removing the emitted light, and swapping the BRDF function with a constant color value, and you’ll see that we have the equation for calculating diffuse lighting.

The multiply symbol inside the circle just represents the component-wise multiplication of the red, green and blue channels of the color with the corresponding channels of the incoming light.

Units…

You may notice that I’ve been using fuzzy terms like ‘incoming light’ as if it’s something we all know how to measure. A good question at this point might be “what are the units being used for light?”.

Well, there are well defined units at play here, but I’m going to say right now that the units don’t really matter. I don’t know of any game engines where they attribute real-world units to lighting and material values, since those kinds of things tend to be driven more by the appearance than the physical correctness. The relative intensities of, say, a candle and a 100W light bulb are probably more important than the absolute values.

Having said all that, here’s the information anyway: The light is measured in units of radiance (Wsr^-1m^-2). Looking back at the integral in the rendering equation, you can see that the radiance is multiplied by the differential solid angle. This converts the radiance into irradiance (Wm^-2).

Remember that the BRDF is the ratio of light reflected to light received? Well that’s a ratio of radiance to irradiance, so the BRDF has units of sr^-1.

One thing to be careful of is that the incoming light in the simplified version of the rendering equation (using the sum over the lights) is measured in units of irradiance (Wm^-2). We have to use irradiance directly here since the typical lights we use (point, spot, directional) don’t have any area.

That’s It For Now

I hope what I’ve explained so far was fairly clear, but I know I’ve left some pretty big holes here. I’m going to be looking at some of the following at a later date:

• What is the difference between irradiance and radiance?
• What is the BRDF?
• What is energy conservation, and does it matter for games?