vmwgfx ShaderDefine Overflow

Vulnerability analysis

The shader creation path validates the user-supplied shader window against its backing buffer by adding the size and offset and comparing to the buffer length, but when the offset is near U64_MAX the unsigned addition wraps to a tiny value that passes the check. The unchecked offset is then stored and forwarded to host SVGA shader-create commands, allowing an attacker to reference memory beyond the buffer. The fix replaces the arithmetic with an overflow-checking helper that detects the wrap and rejects the request if the resulting endpoint exceeds the buffer size. The vulnerable ioctl is reachable from a local process with access to the VMware SVGA device node, which is typically available to users with render node permissions on systems running the vmwgfx driver.