Blog » gg, weeks 6 and 7: weather, TAA, and a flag day for Vulkan

Weather driven off the scene clock, TAA after a month of arguing against it, the SDL fork retired along with SDL_GPU, and the first half of water.

Two weeks in this one, since the jam post was a one-off and there was no good place to split the fortnight after it. It's also the biggest change to the engine since the project started. In the week 4 post I said the engine builds against my own fork of SDL and that the fork was mandatory. The fork is gone now, along with SDL_GPU, and the engine draws through a bindless device layer of its own on Vulkan 1.4.

Before that happened there was weather, foliage that bends in the wind, TAA, and the golden images quietly eating the repository. After it came clouds I still don't like, a day of planning, and the first half of water.

TL;DR: If you're only here for the Vulkan story, skip down to A flag day for Vulkan.

Weather, from the clock

Week 3's sky derives everything from an hour, a date and a place, and weather now hangs off the same clock. weather_at(authored, hours) is a pure function of the scene's time, so the same hour always gives the same sky. Weather stays out of the save file and the tick inputs, and a replay reproduces a storm by reproducing the clock.

the village under a storm, dark sky, the trees leaning

Storm. Weather is ten conditions, all of them zero by default.

The obvious alternative was a Markov chain stepped on the tick. That gets you hysteresis and easy event coupling, but it loses every one of those properties, so I rejected it. Scripted weather is a set of override windows, and a transition is one too: asking for "storm, over two hours" writes the storm into the current state and puts the previous weather in a window that's already ramping out.

Wetness and snow cover follow the same rule. They're what precipitation leaves behind, so each is an integral looking back over the pattern. That costs a few dozen noise evaluations a query, and weather stays stateless.

On the rendering side, the cloud trace, the shadow march and everything downstream all read one composed weather map, and precipitation comes from its coverage, so the dark cloud is overhead exactly where the rain is.

the village at dusk after rain, wet roofs and a puddled path reflecting the lantern

Wet, with Lagarde's recipe unmodified. At full wetness a rough surface's albedo drops to a fifth of what it was, which is stronger than I expected, but it's also what the paper says.

the village under snow, roofs and ground white, walls bare

Snow, just a dusting. The grains are hashed in world space so they don't crawl with the camera.

Both read how sheltered a surface is off the sky-visibility field that was already there, and the whole surface response costs less than I can measure.

Two things I would have got wrong without measuring:

  • Puddles were going to be a mask off the curvature the terrain splat already computes, but the demo basin's hills are sinusoids hundreds of metres across. A mask loose enough to catch those hollows floods whole hillsides and a tighter one catches nothing. Puddles need metre-scale relief or hand placement, and either way that's a content decision. So, no puddles yet.
  • The haze layer used earth's Mie scale height of 1200 m, and at that height haze reddens the sun long before it hazes the view: two hundredths of view-side optical depth cost the sun 56% of its red. At 60 m it costs the sun nothing measurable, so the village has a shallow haze layer now.

Trees that bend like trees

Wind came in at the same time. It started out as a property of the cloud layer, which I only discovered when I couldn't find it in the editor, and since plants need wind whether or not there are clouds, it got hoisted up to the sky component.

the forest floor, undergrowth and flowers in long afternoon shadows

The wood after the wind work. An actual editor screenshot this time, taken while I was fiddling with the wind speed.

The bend is computed in the one function every pass uses to get an instance's position, so the scene, the prepass, the shadows and the sky-visibility field all agree about it. Otherwise you get a tree casting the shadow of a tree standing still next to it.

Wood and leaves bend differently. Wood stiffens toward the model's vertical axis so the trunk holds its line and the branch tips do the moving. In the first version a whole tree sheared over as one curve and looked like rubber. Grass needed a second go too. Its first attempt hashed the flutter from the whole vertex position and looked like seaweed underwater, and now a blade moves as one piece.

How far a plant bends is a property of the plant, set in the model's import sidecar, and it defaults to rigid. The cook can't tell a sapling from a boulder, and a swaying rock is worse than a still tree.

A rendering lab

The demo village had collected a bunch of oddly placed lights that existed only to give the golden images something to test, plus a row of material spheres that have no business in a village. All of it moved out to lab.scene, along with the goldens that use it.

a flat white plane under an overcast sky, with material spheres, a small house and a stone wall

The lab. Nothing here is trying to look nice.

Obvious in hindsight: a test scene can be as ugly as it likes, and the demo scene gets to go back to being content.

The goldens were eating the repo

A push that touched the golden baselines had started taking minutes, and so had a fresh clone. When I finally looked, .git was 815 MB, and about a third of that was baseline images at around seven revisions each. Not ideal.

Three things changed:

  • I rewrote the history to strip every superseded baseline revision, and .git is 35 MB now. I don't want to do that again, and there's a written plan for the images to leave git entirely, but I haven't decided on it yet.
  • PNG encoding goes through libdeflate now, underneath stb's writer, because stb's deflate emits a single fixed-Huffman block whatever the compression level. That took 41.6% off the golden set.
  • The baseline recorder holds any case whose frame moved by less than the comparator can see. A re-record used to rewrite every case whose bytes differed at all, and one change that shifted every frame by a sub-quantum rewrote 20 MiB of images nobody could tell apart.

Pro-tip: If your golden images live in git, don't let byte-exactness decide when one gets re-recorded. If a human can't tell two frames apart they shouldn't cost a megabyte of history. Your comparator already knows what "can't tell apart" means, because that's the threshold it fails on.

TAA, after all

The plans had developed a habit: every few pages there'd be a line like "no TAA means..." followed by a workaround. That was never a decision I made. It fell out of the early planning, when everything was sized against a demo scene and one art style, and I'd pushed back on it more than once without settling it. So I weighed it up properly, and then read elopezr's post on TAA, which made the case a lot stronger.

The short version is that MSAA's cost scales with samples times pixels, and the machine the quality tiers exist for is a Meteor Lake laptop with an iGPU that can't afford four samples at any resolution worth playing at. A single history can also serve every pass that wants one: the cloud reconstruction already kept a private one, and each new stochastic pass would have wanted another. And a canopy of alpha-tested leaf cards at one sample degenerates into a binary test that chequers. Temporal integration fixes that.

So, settled. The resolve weighs luminance on exposed values, because a moonlit frame is four orders of magnitude darker than noon and a weight on raw values does nothing at night. At one sample, MASK materials swap alpha-to-coverage for a hashed alpha test seeded per frame, and the history integrates that back into a coverage. On its own, that removes a third of the chequering.

three crops of pine needles against cloud: jagged, smooth, smooth

The same pine at one sample, at 4x MSAA, and at one sample with the temporal pass, in 2x nearest-neighbour crops. The third is cheaper than the second.

the village with everything static black and the foliage picked out in yellow

The taa_motion debug view, showing the motion a surface reports beyond the camera's. In a still frame the swaying canopy is the only thing writing velocity.

On the 3070 at 1080p, one sample with the pass on is 0.72 ms cheaper than four samples with it off, and the resolve costs the same 0.18 ms at either sample count, since it runs over the resolved image. That's the trade the quality tiers make now. Every tier has TAA on, low and medium render one sample, and high keeps two for the hard edges a history can't help with.

Three things came out of tuning it.

First was the jitter period. I measured it wrong twice, in ways that looked like results. An unjittered one-sample frame is identical to the MSAA one on every fully covered pixel, and that's 99% of them, so over whole frames any jitter scores as pure loss. Measured only over the edges, the pass removes about 45% of the aliasing, and past eight frames the edges stop improving while the texture blur keeps growing. Eight it is.

Second was the sharpener. Early on, the one-sample temporal frame measured worse for chequering than the plain one. RCAS runs at native resolution whenever TAA is on, and it was using the setting tuned for the upscaling path, where EASU has already softened the frame. At native there's nothing to recover, so it was undoing the antialiasing the pass had just done. Changing the setting from 0.2 to 1.0 took the chequering from twice the plain frame's to below it.

And finally the overlays. The debug grid and lines were being drawn inside the history, where a line averaged over eight sub-pixel positions loses a third of its peak contrast, so they draw after the tonemap now.

A knob tuned for one frame is wrong for another, and nothing tells you. The sharpener had a perfectly good default. It was the default for a frame that had been through a different pipeline, and it had a measurement behind it, so nobody questioned it.

A flag day for Vulkan

While writing the water plan I kept hitting the same wall. SDL_GPU's binding model is slot-bound at DX11 counts, and the shared surface header was already at fifteen of sixteen samplers and eight of eight storage buffers. The terrain's layer table was a uniform block because there is no ninth storage buffer, and the water plan was spending a page working out how to fit its field, its caustic map and its reflection march into the one slot left.

There were other walls too. You can't read a multisampled texture at all, which is why the frame stored depth three times and the fork carried a depth-resolve patch, and the fork itself was four patches to rebase onto every SDL bump. SDL_GPU also has no indirect count and no draw parameters, so every cull and every LOD selection was CPU work over every resident entity. That's the work week 4 threw a job system at.

The week before all this, Sebastian Aaltonen had put out NoGraphicsAPI, and it's exactly the shape I wanted: memory as GPU pointers, one application-owned descriptor heap, a push-constant root per draw, global barriers and timeline completion. It's also a prototype, as its README says, and it needs four extensions beyond core 1.4, three of which neither of my GPUs has. So I copied the shape and wrote my own.

The decisions took one evening. The backend would be our own, a library between the platform layer and the engine, and it would go in as a flag day with no SDL_GPU fallback. It uses volk and VMA, because a loader and a suballocator are the two parts of a device layer that always cost more than they look. macOS got dropped, and the floor is Vulkan 1.4, since both my drivers were already there. 1.4 asks nothing of the hardware that 1.3 didn't, so it's really a driver floor: roughly everything from early 2025 on.

The shape of it

Every storage buffer and uniform block is memory now, reached through a typed pointer that the same C++ struct mirrors, so none of it costs a binding. A texture is a 32-bit index into one descriptor heap that every pipeline shares, and each draw gets a 256-byte push-constant block of pointers: what SDL_GPU needed forty-six uniform pushes for is now one struct per draw. Even a vertex is a load through a pointer, and a draw list is plain memory that a compute cull writes and one indirect draw per pipeline consumes. Barriers are global, and there's one queue, with every destruction deferred to the timeline value that last used it.

Stage A: the layer, and the tests that found it

The layer went in first as a library nothing linked yet. Every test case creates its own device and fails on any validation error or warning, with synchronisation validation on. Nine bugs came out on the way to zero, six of them in the layer, and the heap descriptor set, for one, was never bound: vkCmdBindDescriptorSets didn't appear anywhere in the library, and every test passed anyway, because none of them had read through the heap. The first one that did segfaulted.

None of those would have produced a wrong image on my hardware. The next round found seven more, and four of those no validation layer can see, like a depth clear and a depth test that cancelled each other out so the test passed everything. Fun.

Pro-tip: A device layer's tests should create a real device and fail on validation warnings. Most of the thirteen defects above would have rendered a perfectly plausible image on the hardware they were written on. Validation is the only oracle that doesn't need a wrong pixel to point at.

Stage B: the port

The port itself took four days of checkpoint commits, squashed once the goldens were green. The renderer, the engine, every shader, the CPU cull and the editor's imgui backend all moved together, so at no point did the tree hold two binding models, and the rule for the whole stretch was "forget SDL_GPU ever existed, including the silly workarounds". Reverse-Z went in under cover of the rewrite, along with an imgui backend of our own over the heap, because the stock Vulkan one submits and idles the whole queue every time imgui rasterises a glyph.

two crops of the village path, the top with the ground texture smeared into streaks, the bottom correct

One of the port's bugs: the terrain layers got a clamping sampler where the old backend had used a repeating one, so every ground texture past its first tile smeared into streaks. A golden image catches this in seconds.

The golden suite was the method. By the end of the port one defect accounted for 27 of the last 29 failures: a band of blown-out pixels down the right edge of every lavapipe frame, whose value depended on device memory the image didn't own. The margin of a render target outside its drawn sub-rect had never been defined, and defining it fixed the band. It also turned up depth images being cleared through a colour-clear call that lavapipe implements as a null function pointer. Lovely.

The fork got retired partway through, once the layer covered what its patches did, and the SDL recipe now pins stock SDL at the exact commit the fork had been rebased onto. Nothing in the tree patches a dependency any more.

Stage C: the frame the GPU drives

Stage C is the reason for doing the port at all, since it's everything the old binding model made impossible.

A compute cull now writes every draw list in the frame, with slots assigned by prefix sum, so the draw order is a pure function of the input and doesn't depend on which workgroup ran first. Two-phase occlusion culling runs in the same cull, against last frame's depth pyramid and then this frame's.

The village couldn't test any of this. Its frame is fragment-bound, the frustum already keeps most of what's resident, and even a perfect occluder would only remove a fraction of the triangles. So, a new scene:

a grassy shelf below a bare ridge, the crest hiding a wooded basin behind it

ridge.scene, built for occlusion, with a wooded basin just behind the crest. My first version had the ridge 60 m high, and it measured the same 3.9% as the village, because it only hid things far enough away that the LOD chain had already made them cheap.

From the shelf, occlusion takes 21.6% off the frame and draws about a sixth of the instances, and the image difference is smaller than the difference between two runs of the same arm.

Then clusters. Each cooked LOD level is cut into clusters of at most 124 triangles, and a surviving cluster reaches the rasteriser either through a compute pass that copies its triangles into the index arena or through a mesh shader that emits it from one word. On the ridge's shelf both paths take the frame from 8.6 ms with plain index ranges to under 7.5, and the mesh shader wins on the cull, because writing one word per cluster is much cheaper than writing its hundred-odd indices. On the village neither one pays off, since the triangles a cull removes were dying to early-z anyway.

The terrain clipmap's patch selection moved to the GPU as well, and making its draws indirect produced the best bug of the fortnight. An indirect draw may not use a non-zero firstInstance unless the device was created with that feature, which it never is. The scene's job came first in the buffer with a base of zero, so every scene image was correct, and only the sky-visibility map and the shadow cascades drew whichever patches they felt like. Two goldens caught it and the other 86 couldn't see it, and neither can validation, because the value is written into device memory by a shader.

The pipeline cache turned out to be a cost as well. It had grown to a 688 MiB blob that made startup slower than having no cache at all. Oh. It has a size limit now.

What a 4K frame costs

With the CPU out of the frame, the next question was what the GPU spends it on. At 3840x2160 the answer is shading: the demo's scene pass is 9.98 ms shaded and 1.50 unshaded. Of the shaded 10 ms the shadows are a third and the lighting maths about 15%, and the largest single item is what's left after every term, about 3.1 ms of material sampling (the texture fetches, the tangent frame rebuilt per fragment, and the weather terms on top). I suspect that's where the next real win is. Next time.

Clouds, again

The cloud layer from week 4 got two fixes and one verdict. The horizon was being sampled 45 times more coarsely than the zenith, because a grazing ray spans the full 90 km reach and a vertical one only crosses the layer. The march budget scales with the ray's span now. And a render-scale change was resetting the cloud history 36 times over six steps. Once, now.

sunset clouds over a dark treeline, the village lanterns just visible

Late evening, before either fix. I still think they look like a skybox.

The verdict is that they're not good enough. They look like a painted dome from most angles and worse at the horizon, and other engines have real volumetric clouds at good frame rates. The early passes at this were sized to the demo scene, which is the mistake this project keeps making. There's a proper plan now, and it's next on the renderer's list after water is drawn.

A day of planning

Between the port and the water, one whole day went on plans. Most of the remaining work orders (water, foliage, weather, precipitation, the scale-and-photoreal survey and what was left of stylization) had been sized to the demo scene without anyone saying so. Each got fleshed out until it could be built without redoing the research, and then I settled every open question in one sitting.

Water, part 1

The first customer for water is a lake, seen from its bank and from a glide, with the village on the shore and a river feeding it. Part 1 is everything a body of water needs before any of it gets drawn.

Bodies of water are part of the terrain document, next to its material and scatter layers, because the carve, the depth masks and the placement rules all need the ground and the water in one pass. The cook derives a field from them on the heightfield's grid, with the surface height, the current, the depth and the distance to the shore.

There are three kinds of body, each authored with the least information that determines it. A lake is a seed point and a level, a sea is a level, and a river is a list of points along a spline, each with a height, a width, a depth and a speed. The cook carves the river's bed above the base raster and below the sculpt layers, so a ford you raised by hand sits on top of a carved bed.

a small square of terrain floating beside the demo basin, with a channel carved down it into a round bowl

The water fixture, stood next to the demo basin, with the stream bed carved down the valley into the pond bowl and no water in either of them yet.

Placement rules take a depth window and a shore window now: reeds grow in the shallows, and a wood keeps a few metres back from a bank. The picker's ray marches the water field the same way it marches the ground, the character controller wades and swims, and water_at crossed the C ABI waist. My favourite number: a ray cast onto the pond lands 1.997 m above where the same ray lands with water ignored, which is exactly the depth the field reports at that point. Two code paths that share nothing past the field.

Nothing is drawn yet and no golden moved, and a swim replays digest for digest. Part 2 is the surface.

Where it stands

The engine has weather, a temporal pass, its own device layer with a GPU-driven frame, and half of water. Next is water's surface, then the clouds, then skinning in compute. The traversal slice is still what all of this is for, and it still hasn't started.

Credits

Everything in these shots that isn't engine is bought or borrowed:

  • The buildings, props, trees, foliage and rocks are Quaternius kits: the Medieval Village and Stylized Nature MegaKits, bought as the paid source versions, plus the free Ultimate Stylized Nature set. All of it is released CC0, and he takes Patreon support.
  • The ground materials are Poly Haven textures, CC0.
  • The type is Inter by Rasmus Andersson, under the SIL Open Font License.