Rendering in Retrospect

Rendering in Retrospect · #3 of 12

Shadows Are a Budget Problem, Not a Quality Problem

Retrospective — written in 2026, looking back.

The Wrong Question

For a long time, when a shadow looked bad in a personal project, I asked: how do I make this shadow higher quality? The answers that invites are all expensive — a bigger shadow map, an extra filtering pass, more frequent shadow updates. Each treats quality as something purchased with resolution, and each worked just well enough on a small test scene to keep me from asking a better question.

The better question is: where should my shadow budget be spent? A shadow map is a finite resource — a fixed number of texels covering a fixed area. Every texel assigned to a part of the scene the camera cannot see is taken from a shadow edge the viewer is staring at. Most bad shadows I have produced were not under-resourced. They were misallocated.

What a Shadow Map Actually Is

A shadow map is a depth image rendered from the light's point of view. For a directional light, you place an orthographic camera at the light, aim it along the light direction, and record for each texel the distance to the closest surface the light can see. Later, shading from the main camera, each fragment is transformed into light space. If its distance from the light exceeds the stored distance, something closer blocked the light, and the fragment is in shadow.

// GLSL-style excerpt — illustrative shadow lookup
uniform sampler2D uShadowMap;
uniform mat4 uLightSpaceMatrix;

float shadowFactor(vec3 worldPos) {
    vec4 lightSpace = uLightSpaceMatrix * vec4(worldPos, 1.0);
    vec3 proj = lightSpace.xyz / lightSpace.w;  // perspective divide
    proj = proj * 0.5 + 0.5;                      // NDC to [0,1]
    float closestDepth = texture(uShadowMap, proj.xy).r;
    float bias = 0.0015; // illustrative value, see below
    return (proj.z - bias > closestDepth) ? 0.0 : 1.0;
}

The algorithm is simple. Its famous artifacts — acne, peter-panning, shimmering, blocky edges — follow from one fact: a continuous world is represented by a discrete grid of depth samples, and that grid almost never lines up with the camera's pixels.

The useful measure is texel density: how much world-space area each shadow texel covers, relative to how much area a screen pixel covers at that distance. If a shadow texel covers more world area than a screen pixel, several pixels share one depth sample and edges go blocky. If it covers far less, you spent resolution where no viewer can perceive it. Good shadow work is mostly keeping texel density roughly matched to screen pixels, where it matters, within a total texel count you can afford.

Where the Budget Leaks

In my early experiments, the directional shadow camera was set up the laziest way: an orthographic box large enough to contain the entire scene, at a resolution picked because it was a round number. Every part of that leaked budget.

Covering what the camera cannot see. If the view shows a small part of a large test scene, a scene-sized frustum spreads texels over the whole scene anyway. Tightening the frustum to the region around the camera — a cheap approximation of intersecting the light's view with the camera's view — can multiply effective density several times over without changing map size. Same budget, better spent.

Covering empty space. A loosely fitted box includes large empty margins. Fitting it tightly to actual casters and receivers, and aligning it to texel boundaries, costs a little code and pays constantly. Alignment matters more than it sounds: if the frustum moves in sub-texel increments, the world-to-texel mapping shifts every frame and edges shimmer even in a static scene. Quantizing the frustum position to whole texels makes the mapping stable — not higher quality, but the same quality every frame, which reads as much better.

Uniform allocation over distance. A single map from camera to horizon gives nearby geometry, where a viewer can see individual texels, the same density as geometry far away. That is the problem cascaded shadow maps exist to solve.

Cascades, Conceptually

Cascaded shadow maps split the camera frustum into several depth ranges — near, middle, and far, as an illustrative structure — and give each its own shadow map fitted to that slice. The near map covers a small area, so its texels are dense and nearby shadows crisp. The far map covers a huge area with the same texel count, so its shadows are coarse — but far away, a screen pixel covers a large area of world anyway. Density ends up roughly matched to perception across the view.

Retain the allocation principle, not just the implementation: you spend the same texels non-uniformly, deliberately, where the viewer can tell the difference. Split distances are the dial, and splits that grow with distance usually match perspective better than even splits, because perspective is non-linear — much of the visible detail lives close to the camera.

Cascades also multiply cost: each is another render of the casters. In a personal project that must be justified per scene. A small indoor test scene rarely needs cascades — one tight map wins. A long outdoor view is where a single map collapses and two or three cascades earn their passes back. Choosing not to use a technique for a specific scene is also allocation.

Acne and Peter-Panning: One Dial, Two Failures

The two classic artifacts are not independent problems. They are opposite ends of one dial, and seeing that saved me from treating each new artifact as a new bug.

Shadow acne is self-shadowing error. A surface at an angle to the light samples discrete texels; the stored depth is the depth at the texel center, the fragment's depth is its own, and discretization makes a lit surface compare as slightly farther than the stored value — so it shadows itself, in stripes following the texel grid. It worsens at grazing light angles, where depth changes rapidly across one texel.

Peter-panning is the opposite: shadows detach from their casters, so objects look like they float. It appears when you push the comparison too far the other way and light leaks past a caster's base.

The dial is bias. Subtracting a small bias in the lookup says: count a fragment as shadowed only if it is meaningfully behind the occluder, not behind by discretization noise. Too little bias, acne. Too much, peter-panning. A common refinement scales bias by the surface's angle to the light — through the dot product of normal and light direction — because the error itself scales with that angle. Flat-facing surfaces get a tiny bias; grazing surfaces get more.

A bigger map does not eliminate either artifact. It shrinks the error, so the dial gets less touchy, but the trade-off remains at every resolution. That turned my debugging from hunting a magic bias number — there is none; it is scene- and scale-dependent — into setting bias proportional to texel size and surface slope, then checking both failures at the extremes. An illustrative starting point in my experiments is a bias on the order of one texel's depth footprint, scaled per cascade: a far cascade with huge texels needs proportionally more than the near one, a detail I missed for an embarrassingly long time after adding cascades.

Choosing a Budget Per Scene

In my personal engine experiments I now set shadows up in this order, writing the choices down per scene instead of burying them as constants:

  1. 1. What is this light for? A key directional light that defines the scene's look gets the budget. A decorative light often gets no shadow map at all — unshadowed light is a legitimate allocation, not a missing feature.
  2. 2. What area must be covered? Fit the frustum to the visible, relevant region, aligned to texels, before touching resolution.
  3. 3. How far does the view extend? Short view, one map. Long view, consider cascades — and count the extra caster renders in the price.
  4. 4. Only then, what resolution? Raise it if near-field density is still below screen-pixel density after allocation is sane. Lower it, without guilt, if density is well above what the screen can show.

Filtering — sampling neighboring texels and averaging, in the style of percentage-closer filtering — belongs at the end of that list. It softens the appearance of limited resolution; it does not create information. After allocation is fixed, it is nice polish. Before, it is expensive blur hiding a budget problem.

The General Lesson

Shadows taught me a pattern I now see everywhere in rendering. A resource is finite: texels, bandwidth, milliseconds. The naive response to poor results is to request more of it. The effective response, usually cheaper and almost always available, is to audit where the current resource goes and stop spending it where nobody is looking.

My shadows did not improve when I could afford bigger maps. They improved when I treated every texel as spent money, and asked whether each part of the scene had earned its share.