MSAA Fragment Merging Explained

Samples and fragments

Key idea. MSAA decides coverage per sample but shades per pixel: each triangle that covers any of a pixel's samples gets one fragment there, and that fragment writes the samples its triangle covers.

With 4x MSAA, a pixel holds four samples, each with its own depth and colour. The rasteriser tests each sample against each triangle. The fragment shader then runs once per pixel per triangle, not once per sample.

Praxis's main pass draws at 4x MSAA (ADR 0019). The coverage tests place its four samples where Vulkan's standard positions put them, at (0.375, 0.125), (0.875, 0.375), (0.125, 0.625) and (0.625, 0.875) within the pixel, and wherever nothing merges the M4 Pro agrees with them at every sample. Where an edge crosses a pixel, one triangle covers some of its samples and its neighbour covers the rest. Each gets a fragment, and each fragment's output lands only in its own triangle's samples. That is what antialiases the edge: the resolve averages the four samples, so the pixel blends both triangles in proportion to their coverage.

A fragment's inputs come from the vertex shader's outputs. Most are smooth: interpolated across the triangle, so their value says where on the triangle the fragment lies. A flat input, declared @interpolate(flat) in WGSL, is not interpolated. Every fragment of a triangle reads the value one vertex wrote. The lit pass's Material ID is one. Chapter 2 shows why that difference turned out to matter on the M4 Pro.

Indices matter too. A Meshlet's triangles share vertices, and two triangles that meet along an edge name the same two vertex indices. Praxis's packed indices carry the Meshlet record in their high bits, so triangles of different Meshlets or Instances never share an index, even where their positions coincide.

What the M4 Pro does

Key idea. When the fragment stage reads no flat input, the M4 Pro can merge two index-sharing triangles' fragments in a pixel both cover. The samples covered stay exact; which triangle's inputs they receive does not. Reading any flat input stops it.

The rule, as ADR 0019 now records it: under 2x or 4x MSAA, when the fragment stage reads no flat-interpolated input, the fragments of nearby triangles of one draw that share vertex indices can be merged in a pixel. One fragment, with one triangle's interpolated inputs and primitive_index, writes the samples of both.

How it was measured

src/render/coverage_tests.rs draws quads with sides of 0.3 to 1.5 px into a 256×256 multisampled ID target. The fragment stage writes each sample's triangle ID, and a compute pass reads the target back per sample. The flat scene's 4,096 quads do not overlap, so an analytic oracle knows every sample's owner. It leaves out samples within 0.01 px of an edge, where rounding could go either way. Each quad is drawn as the cooker draws one, 0 1 2 then 0 2 3, so its two triangles, partners, share two indices. Each quad also carries a weight that only its first triangle interpolates and one that only its partner does, so a fragment's smooth inputs alone say which triangle they were interpolated over.

Samples whose ID departs from the oracle, a selection of the rows in docs/performance.md. Every departure went to the partner, never to another quad or to an empty sample, and every count was identical over three runs.

• Fragments, not just the index. IDs built from smooth inputs alone depart exactly where primitive_index does. So a merged fragment carries one triangle's interpolated inputs onto the other's samples, not only its index.
• A flat input stops it. Every row that reads a flat input departs nowhere, even where the flat value is the same for both triangles. What matters is that the stage reads one, not what it holds.
• What triggers it. No flat input, shared vertex indices, nearness in the draw, and more than one sample per pixel. One triangle between partners still merges most; a partner thousands of triangles later, or in another draw call, never. Sample-rate shading does not stop it.
• Coverage is exact. The samples the two triangles cover together do not change. Only their attribution moves, and how it moves depends on what else the draw holds: remove every other quad and 1,421 becomes 760.

That last point corrects the reading #296 had made. Its guide said a triangle's coverage depended on what else the draw held. It does not: a triangle drawn alone covers exactly its own samples. What depends on the draw is which triangle a sample is credited to.

What it resembles

Fatahalian and colleagues proposed something like it in 2010, as a way to cut shading cost on small triangles: gather the fragments of adjacent triangles of a mesh and shade them once. Their merge conditions are a shared screen location, no sample covered by both, the same facing, and source triangles connected by edges. A merged fragment then shades both triangles' samples with inputs chosen from one of them. Their paper says nothing about flat inputs. As far as #348 could find, Apple's documentation says nothing about it: #348 searched Metal's documentation, Apple's talks, the developer forums and wgpu's issues and found nothing. The resemblance is a reading, not a source for what the M4 Pro does. A plausible explanation of the flat-input rule, which no source confirms, is that inputs interpolated across a triangle can be evaluated for a neighbour, while a flat input belongs to one primitive and cannot be shared.

Option B's lost samples

Key idea. An ID pass that reads no flat input credits some of a triangle's samples to its partner. Drawing only the triangles it names then drops the uncredited triangle, and its samples go missing; drawing whole Meshlets keeps both.

Option B, the visibility pre-pass that Visibility Pre-pass Explained describes, wrote each sample's triangle ID in an ID pass and then drew what those IDs named: the named triangles alone in its triangle-granularity variant, and their whole Meshlets as recorded. Its fragment stage read primitive_index and no flat input, so the M4 Pro merged partners' fragments. Where merged fragments carried the partner's ID, a triangle all of whose samples had gone to its partner was named by none. The draw of named triangles left it out, and the partner, whose coverage is exact, does not cover its samples, so nothing did.

#348 reproduced the prototype's loss in the harness. On #296's tilted scene, 7,871 quads 0.3 m wide at 40 m, an ID pass at depth Equal after a pre-pass, followed by a draw of the triangles it named, lost 697 samples and credited 554 to a partner. With whole quads kept, it lost none, though 368 samples were still credited to a partner. With each triangle given its own indices, neither happened. The prototype branch itself showed the same 697 and 554.

For Praxis's ADR 0019 the rule became: a pass that decides from which triangle a sample shows, as an ID pass or a visibility buffer does, reads a flat input, which costs nothing, or keeps whole Meshlets, the groups that share indices. Culling decided from geometry, as #347's Meshlet culling is, never asks which triangle a sample shows, so it is unaffected.

a_draw_of_whole_index_sharing_groups_keeps_every_sample stays on main as the guard. It checks that drawing whole quads loses nothing, a check any conformant rasteriser should pass. On the M4 Pro, it also checks that drawing the named triangles alone still loses samples. If a driver change makes that check fail, the merging has stopped, and ADR 0019's line is due for review.

The shadow mask's normals

Key idea. Two passes that must agree on a sample's triangle disagreed because only one of them read a flat input. The depth passes wrote one triangle's normals into an edge sample that the lit pass then shaded as its neighbour.

The passes that matter here, in order, with shadow rays and culling on:

1. culling phase one
2. depth pre-pass
3. depth pyramid
4. culling phase two
5. phase two's depth pass
6. shadow mask pass
7. lit pass

Since #350, the depth passes, marked orange above, write the Surface normals with a fragment shader, fs_surface_normals. The shadow mask pass reads them to decide, per light, whether to hide it with no ray, trace a ray from outside the surface, or trace one from inside. The lit pass then tests each sample at depth Equal, shades the fragments that pass and reads the mask's bits.

fs_surface_normals read no flat input. The lit pass's fs_main reads the flat Material ID to find its Material. So on the M4 Pro the depth passes' fragments could merge and the lit pass's never did. At a sample beside a shared edge, the normals could come from triangle A while the lit pass shaded triangle B there, at the same depth, so B passed the Equal test. Across a crease, A may face away from a light that B faces, and the mask hides that light where the shading says lit.

How culling exposed it

Which fragments merge depends on what the draw holds. With culling on, the depth passes draw the two culling phases' Instances in two draws, while the lit pass draws them all in one. With culling off, phase two is empty, and the depth pre-pass makes the same draw as the lit pass. The depth passes' fragments could merge in both modes, so in both, some edge samples held a neighbour's normals. But the two modes merged different pairs, so their images differed at a few pixels, and that difference is what a test could see. A pixel wrong the same way in both modes was invisible to that comparison; the four Goldens the fix changed (chapter 5) show where the culled image had been wrong.

That is how the defect was found. Culling must not change the image, so tests/goldens.rs compares town culled and unculled at eight frames. With #350's normals, six of those frames differed by 1 to 6 pixels at 640×360, by up to 69 steps, and PR #366 tolerated up to 8, as HARDWARE_UNCULLED_PIXELS, filing #369 to bring it back to 0. Probes on #366's branch narrowed it down. A lit pass that wrote a hash of each fragment's depth and normal matched in both modes, and constant surface normals matched too. Neither depth pass's tie rule changed a single pixel. Later, #347's Meshlet culling test made helmet differ by 23 pixels, and by none with shadow rays off.

The fix

Key idea. The depth passes now read a flat input too, so neither pass's fragments merge and both credit every sample to the triangle that covers it. The read has to survive the compiler, so it is used, to no effect.

// main.wgsl, PR #373: the depth passes' vertex output gains the flat Material ID.
struct DepthNormalsOutput {
    @builtin(position) @invariant clip_position: vec4<f32>,
    @location(0) camera_relative_position: vec3<f32>,
    @location(1) normal: vec3<f32>,
    @location(2) @interpolate(flat) material_id: u32,
}

// The end of fs_surface_normals.
let encoded_normals = vec4<f32>(octahedral_encode(triangle_normal), octahedral_encode(interpolated_normal))
    * 0.5 + 0.5;
return select(encoded_normals, vec4<f32>(0.0), in.material_id == NO_MATERIAL - 1u);

An input the shader never reads can be dropped by the compiler, and then the stage reads no flat input after all. So fs_surface_normals compares the Material ID with NO_MATERIAL - 1, which is u32::MAX - 1. Material IDs index a table of 256 Materials (MATERIAL_CAPACITY), so the comparison never holds and the normals pass through unchanged. The compiler cannot bound a value it reads from a vertex output, so it has to keep the read. That pattern comes from #348's coverage tests.

What changed

• The check. HARDWARE_UNCULLED_PIXELS went from 8 to 0, and town's eight frames match culled and unculled on Metal. On #347's prototype branch, helmet went from 23 differing pixels to 0.
• The Goldens. Four Metal Goldens changed at single edge pixels, where the old image held a merged fragment's normals: normal-tangent 50 pixels, pbr-sweep 3, shadows 1, town 1. The owner accepted them.
• The cost. On town, in three rounds interleaved with a build of main: GPU frame +13 to +24 µs, depth_prepass +23 to +26 µs, and shadow_mask_pass −13 to −34 µs. The counters did not move, so no Baseline was re-recorded.

From PR #373, against the recorded Baseline. The interleaved comparison with main, quoted above, is the fairer measure of the change. Either way it is tens of microseconds in a frame of about 15 ms.

Why invariance could not do it

Before choosing, #369 read the published practice, as the owner asks of every defect. The obvious tool is invariance, and Praxis already uses it: clip_position is @invariant in every main-pass entry point, and wgpu turns on Metal's preserveInvariance. But WGSL's @invariant applies only to the position output, and Apple's preserveInvariance likewise guarantees only that position is computed the same way in each pass. That is what makes depth Equal sound, and depth was already equal in #369. The Vulkan specification's invariance appendix, the most explicit of the three, promises identical fragments only for identical pipelines, or across a listed set of state changes. A different fragment shader is not among them. No source found says when two pipelines with different fragment shaders resolve a shared-edge sample to the same triangle.

The options

1. Read a flat input in the depth passes, as shipped. Almost free, and it matches #348's measurement that a fragment stage reading a flat input is never merged. It rests on that measurement, not on a published rule.
2. Write the normals in one draw of both phases at depth Equal, the ticket's first candidate, at about one more geometry pass, roughly 1.5 to 2 ms on town. Both passes would then make the same draw, but Vulkan's rule for identical pipelines would still not cover two different fragment shaders, so it is no safer and costs more.
3. Write the normals in the lit pass's own draw. Impossible: the shadow mask pass reads them before the lit pass runs.
4. Store a triangle ID per sample and derive the normals from it, as Burns and Hunt's visibility buffer does. Exact by construction and portable, since no later pass resolves ownership again, but a larger change. It is the fallback if the flat-input behaviour fails on another GPU.

ADR 0019 already carried the rule the fix follows: two passes that must agree on a sample's triangle both read a flat input.

What is still open

Key idea. The fix is sound on the one GPU that was measured. Its guards are tests on that GPU, and the per-sample triangle ID is the way out if another GPU behaves differently.

• One GPU. Only the M4 Pro was measured. Other Apple GPU families and later macOS releases may merge under other conditions, or regardless of a flat input. Nobody has measured them yet.
• No vendor documentation. PR #370 drafted a report for Apple's Feedback Assistant, describing the behaviour and a reproducer. Filing it needs the owner's Apple ID, so it is the owner's action, and it blocks nothing.
• The guards. On Metal, the culled-vs-unculled check at 0 pixels catches the read being removed, and the coverage test's M4 Pro check catches the merging stopping. Lavapipe, which CI runs, would not catch the flat read being deleted: a shader test asserting that fs_surface_normals reads a flat input was deferred until the read is removed or the normals pass is restructured.
• Every new pass. Any later pass that decides from which triangle a sample shows, such as a visibility buffer (#297), reads a flat input or keeps whole Meshlets.

Check yourself

Reading list

Grouped by source, each with what it is good for here. Links into the code are pinned to 7525b7d, or to PR #373's merge for the fix.

Glossary

Meshlet, Material ID, Surface normals, Shadow mask and Golden are CONTEXT.md's definitions. The rest are this guide's own.