Canvas: clip() does not clip to the path (needs real stencil-based path clipping)

[bkaradzic-microsoft]

Follow-up to #1824 (see the clip() discussion there) and split out of the follow-up work in #1843 / #1844.

Problem

CanvasRenderingContext2D.clip() does not clip to the path. Context::Clip has two branches, and neither is correct:

• Rectangular path -- intersects an nvgScissor with the path's bounding box. Correct only when the path really is an axis-aligned rectangle.
• Non-rectangular path (m_pathHasNonRect) -- leaves the path current and lets the next fill draw it, then leaves the enclosing scissor untouched. This does not clip at all; it just draws the clip path.

roundRect() is a third case: it deliberately does not set m_pathHasNonRect, so it takes the scissor branch and silently loses its rounded corners. That is deliberate, and the comment in Context::RoundRect explains why -- routing it into the emulation is much worse, because nanovg fills the union of subpaths rather than their intersection:

ctx.roundRect(x, y, w, h, 20);
ctx.clip();
ctx.fillRect(0, 0, 1000, 1000);   // paints the whole fillRect, not the rounded region

Measured on the "Native Canvas" visual test: the scissor's square bounding box gives 1.850% pixel difference, routing roundRect into the union emulation gives 20.980%. The bounding box is the smaller error because it can only ever clip too little, whereas the union failure is unbounded.

save()/restore() also do not currently rewind a clip region, which the spec requires.

Why it is not a small fix

The good news is that no framebuffer work is needed: both the canvas framebuffer (Canvas.cpp) and the filter-stack pool (FrameBufferPool.cpp) already allocate D24S8, so a stencil buffer is present everywhere nanovg draws.

The obstacle is that bgfx's stencil state has no write mask -- setStencil offers TEST_*, FUNC_REF, FUNC_RMASK and the three op slots, but no equivalent of glStencilMask. That rules out the usual "reserve a high bit for the clip" scheme, because the existing concave-fill cover pass in glnvg__fill zeroes the entire stencil value:

gl->encoder->setStencil(0
        | BGFX_STENCIL_TEST_NOTEQUAL
        | BGFX_STENCIL_FUNC_RMASK(0xff)
        | BGFX_STENCIL_OP_FAIL_S_ZERO
        | BGFX_STENCIL_OP_FAIL_Z_ZERO
        | BGFX_STENCIL_OP_PASS_Z_ZERO
        );

so it would wipe the reserved clip bit along with the winding count. The winding passes themselves are fine (INCR/DECR never carry into bit 7 for realistic winding numbers); it is only the zeroing cover pass that cannot be made selective.

Proposed design

Reserve stencil bit 0x80 as the clip mask and re-render the clip path before each draw rather than trying to preserve the bit across draws. Correctness over speed, which seems the right trade for canvas 2D.

1. Clip pass. Render the clip path's winding into the low bits, then a cover pass over its bounding box with TEST_NOTEQUAL, REF=0, RMASK=0x7f, OP_PASS_Z_REPLACE with FUNC_REF(0x80) and OP_FAIL_S_ZERO. Result: stencil is 0x80 inside the clip region and 0x00 outside.

2. Draw pass. The existing winding passes are unchanged. The cover pass changes its test to TEST_GREATER with REF=0x80, RMASK=0xff, which passes exactly when the clip bit is set and the winding count is non-zero. Its existing OP_*_ZERO ops stay as they are and wipe both the winding bits and the clip bit, which is fine given step 3.

3. Re-render. Because step 2 destroys the mask, the clip pass runs again before the next draw while a clip is active.

Work involved:

• nanovg.h / nanovg.cpp -- a public nvgClipPath() / nvgResetClip(), clip path storage in NVGstate (so nvgSave/nvgRestore rewind it, which also fixes the missing save/restore of the clip region), and a new call type.
• nanovg_babylon.cpp -- the new clip pass, plus updated stencil state on every draw path (glnvg__fill, glnvg__convexFill, glnvg__stroke, glnvg__triangles).
• Context.cpp -- Clip() uses it; RoundRect can then set m_pathHasNonRect and keep its corners; m_isClipped / ResetPathState emulation and the FillRect special case all disappear.

Risk

This changes the stencil state of every nanovg draw, so it affects all five graphics backends, and only D3D11/D3D12/Vulkan can be verified locally -- Metal and OpenGL/ES would need CI or device coverage. The intersection of clipping with the filter stack (which renders through intermediate framebuffers) needs explicit testing too.

Happy to pick this up; filing it rather than rushing it, since it is a rendering-correctness change with a wide blast radius.