Skip to content

DXMT: keep indexed draw base vertex signed#758

Open
moreaki wants to merge 1 commit into
VirtualBox:mainfrom
moreaki:agent/dxmt-signed-base-vertex
Open

DXMT: keep indexed draw base vertex signed#758
moreaki wants to merge 1 commit into
VirtualBox:mainfrom
moreaki:agent/dxmt-signed-base-vertex

Conversation

@moreaki

@moreaki moreaki commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Keep indexed-draw base-vertex values signed across the bundled DXMT/winemetal command boundary.

BaseVertexLocation is signed in D3D11, and Metal's indexed-draw baseVertex argument is signed. The two intermediate command fields are currently uint32_t; this patch changes only those fields to int32_t. Their size and command layout remain unchanged.

Deterministic reproduction

Use an indexed draw with BaseVertexLocation = -1.

  1. MTLD3D11DeviceContextImpl::DrawIndexed receives the correct signed INT value.
  2. Assigning it to DXMT_DRAW_INDEXED_ARGUMENTS::BaseVertex currently stores 0xffffffff in an unsigned field.
  3. Assigning that to wmtcmd_render_draw_indexed::base_vertex keeps it unsigned.
  4. Objective-C then promotes the field for Metal's signed baseVertex parameter as positive 4294967295, not -1.

Keeping both fields signed preserves -1 through the existing command path.

Validation

  • Rebased independently on current origin/main (5ac81eda).
  • git diff --check passes.
  • Cherry-picked on top of the LLVM 15 build prerequisite from DXMT: fix the supported LLVM 15 build #748 and built kmk VBoxDxMt on macOS/Arm with LLVM 15.0.7.
  • Included in the clean integrated macOS/Arm full build, which completed with exit 0 and produced VBoxDxMt.dylib.

Closes #754.

Signed-off-by: Roberto Nibali rnibali@gmail.com

@oracle-contributor-agreement

Copy link
Copy Markdown

Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
The following contributors of this PR have not signed the OCA:

To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application.

When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated.

If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public.

@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Required At least one contributor does not have an approved Oracle Contributor Agreement. label Jul 8, 2026
@moreaki

moreaki commented Jul 9, 2026

Copy link
Copy Markdown
Author

DXMT v0.80 audit update:

This is already addressed in DXMT v0.80: the corresponding D3D11 indexed draw command state uses a signed int32_t base-vertex field. Updating the vendored DXMT tree to v0.80 would include the equivalent fix.

So this PR is best understood as a narrow backport for VirtualBox's currently vendored DXMT snapshot.

D3D11 indexed draw calls take BaseVertexLocation as a signed INT, and Metal's baseVertex parameter is signed as well. Keep the DXMT indirect argument and winemetal command fields signed so negative base-vertex offsets are not reinterpreted as large unsigned values.

Signed-off-by: Roberto Nibali <rnibali@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Required At least one contributor does not have an approved Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DXMT: keep indexed draw base vertex signed

1 participant