(release/25.2) xnest: Check malloc return values in GCOps to avoid NULL dereferences

[metux]

malloc() require check on valid ptr

Check malloc() return values for NULL in xnestBitBlitHelper(), xnestPolyText8(), and
xnestPolyText16() to prevent potential NULL pointer dereferences.

Backport of #3637 (based on its current head commit
eae9444; original PR not merged yet)

(cherry picked from commit eae9444)

[metux]

🤖 Automated review — generated by Starfleet ship Voyager on behalf of @metux. Not a human review.

Reviewed hw/xnest/GCOps.c — the release backport of #3637.

Verdict: PASS

Identical +12/−2 cherry-pick of the master fix on 25.2/25.1/25.0 (same path, no reorg),
applied cleanly on each branch. All three malloc NULL-checks (xnestBitBlitHelper q,
xnestPolyText8/xnestPolyText16 buffer) are correct defensive hardening with no behavioral
change on the malloc-success path; the free(event) in the failure branch also prevents an event
leak, and the added break; is a no-op (last case).

No ABI impact (nested software DDX, no external driver/nvidia surface). Pure defensive
malloc-failure hardening — not a client-triggerable security issue or reachable crash; not a
mandatory backport (opened on maintainer request).

No blocking findings.