Component
GPU attestation with the current C++ SDK CLI (nvattest). The trigger involves CC mode switching with gpu-admin-tools (Host Tools side), so the root cause may be in the GPU/driver state rather than the verifier. Filing here for triage; happy to move this to attestation-sdk or gpu-admin-tools if more appropriate.
Describe the bug
After toggling CC mode on → off → on on an RTX PRO 6000 Blackwell Server Edition (single-GPU passthrough to an Intel TDX guest), local GPU attestation via nvattest consistently fails with result_code = 12 (Overall Attestation Result is False).
The failure is a single driver-source measurement mismatch at index 52. The runtime value at index 52 is byte-for-byte identical across guest VM reboots, GPU FLR, and further CC mode toggles (each performed with --reset-after-cc-mode-switch), i.e. the GPU deterministically keeps returning a measurement that no longer matches the driver RIM golden value.
The only action that recovered attestation was a full host reboot. This contradicts the gpu-admin-tools README ("the system as a whole does not need resetting or rebooting to enable/disable CC-mode") and goes beyond the attestation troubleshooting guide (reset device, reload driver, retry), which we followed without success.
A second, identically configured GPU (same driver 580.159.03, VBIOS 98.02.8D.00.01) in another TD on the same host was not toggled and kept attesting successfully the whole time — this rules out RIM-service-, driver-, or verifier-side changes.
To Reproduce
- Initial state: CC mode
on, GPU passed through to a TDX guest (QEMU/libvirt, vfio-pci). nvattest attest --device gpu --verifier local succeeds (result_code = 0, measres = success).
- Shut down the guest VM.
nvidia_gpu_tools.py --gpu-bdf=<bdf> --set-cc-mode=off --reset-after-cc-mode-switch- Boot the guest. As expected, the R580 open driver refuses to initialize a CC-off GPU inside a TDX guest (
confComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled → RmInitAdapter failed). This step was part of a deliberate negative test.
- Shut down the guest VM.
nvidia_gpu_tools.py --gpu-bdf=<bdf> --set-cc-mode=on --reset-after-cc-mode-switch- Boot the guest. Driver initializes normally;
nvidia-smi conf-compute -f → CC status: ON.
- Run
nvattest attest --device gpu --verifier local --format json → fails as described.
The following did not recover attestation (same index-52 mismatch, same runtime value after each):
- Guest VM reboots (multiple)
- GPU FLR from the host while the VM was shut off (
echo 1 > /sys/bus/pci/devices/<bdf>/reset)
- Another full
--set-cc-mode=off/on --reset-after-cc-mode-switch round-trip
- Config drift check:
--query-cc-settings (enable=1, devtools=0, bar0-filter=0), --query-prc-knobs, ECC enabled, MIG disabled, nvidia-persistenced --uvm-persistence-mode active — all identical to the healthy GPU
Recovery that worked: shut down guests → reboot the host → start guests → nvattest succeeds again (result_code = 0, measres = success) with no other changes.
Expected behavior
Attestation succeeds after a CC mode transition once the GPU has been reset with --reset-after-cc-mode-switch (per the gpu-admin-tools README, no system reboot should be required), or at least after the documented remediation of device reset + driver reload.
Environment
nvtrust package version: N/A (using the C++ SDK CLI: nvattest / libnvat 1.2.2.1780962352-1, CUDA repo ubuntu2604)- Python version: N/A (C++ CLI; guest system Python is 3.14.4)
- NVIDIA Driver Version: 580.159.03 (
nvidia-driver-580-server-open, open kernel modules; GSP firmware 580.159.03)
- Host OS: Ubuntu 26.04 (guest TD: Ubuntu 26.04)
- Kernel Version: 7.0.0-27-generic (host and guest)
- CPU Architecture:
x86_64 (Intel Xeon 6530P, Intel TDX)
- GPU Model(s): RTX PRO 6000 Blackwell Server Edition (PCI
10de:2bb5, hwmodel: GB20X A01 GSP BROM)
- GPU Firmware Version: VBIOS 98.02.AF.00.01 (affected GPU; the unaffected control GPU is 98.02.8D.00.01)
- Deployment: Bare-metal host + Intel TDX CVM guest (QEMU/libvirt, single-GPU vfio passthrough)
- gpu-admin-tools: v2025.11.21
Component
GPU attestation with the current C++ SDK CLI (
nvattest). The trigger involves CC mode switching with gpu-admin-tools (Host Tools side), so the root cause may be in the GPU/driver state rather than the verifier. Filing here for triage; happy to move this toattestation-sdkorgpu-admin-toolsif more appropriate.Describe the bug
After toggling CC mode
on → off → onon an RTX PRO 6000 Blackwell Server Edition (single-GPU passthrough to an Intel TDX guest), local GPU attestation vianvattestconsistently fails withresult_code = 12 (Overall Attestation Result is False).The failure is a single driver-source measurement mismatch at index 52. The runtime value at index 52 is byte-for-byte identical across guest VM reboots, GPU FLR, and further CC mode toggles (each performed with
--reset-after-cc-mode-switch), i.e. the GPU deterministically keeps returning a measurement that no longer matches the driver RIM golden value.The only action that recovered attestation was a full host reboot. This contradicts the gpu-admin-tools README ("the system as a whole does not need resetting or rebooting to enable/disable CC-mode") and goes beyond the attestation troubleshooting guide (reset device, reload driver, retry), which we followed without success.
A second, identically configured GPU (same driver 580.159.03, VBIOS 98.02.8D.00.01) in another TD on the same host was not toggled and kept attesting successfully the whole time — this rules out RIM-service-, driver-, or verifier-side changes.
To Reproduce
on, GPU passed through to a TDX guest (QEMU/libvirt, vfio-pci).nvattest attest --device gpu --verifier localsucceeds (result_code = 0,measres = success).nvidia_gpu_tools.py --gpu-bdf=<bdf> --set-cc-mode=off --reset-after-cc-mode-switchconfComputeConstructEngine_IMPL: GPU confidential compute capability is not enabled→RmInitAdapter failed). This step was part of a deliberate negative test.nvidia_gpu_tools.py --gpu-bdf=<bdf> --set-cc-mode=on --reset-after-cc-mode-switchnvidia-smi conf-compute -f→CC status: ON.nvattest attest --device gpu --verifier local --format json→ fails as described.The following did not recover attestation (same index-52 mismatch, same runtime value after each):
echo 1 > /sys/bus/pci/devices/<bdf>/reset)--set-cc-mode=off/on --reset-after-cc-mode-switchround-trip--query-cc-settings(enable=1, devtools=0, bar0-filter=0),--query-prc-knobs, ECC enabled, MIG disabled,nvidia-persistenced --uvm-persistence-modeactive — all identical to the healthy GPURecovery that worked: shut down guests → reboot the host → start guests →
nvattestsucceeds again (result_code = 0,measres = success) with no other changes.Expected behavior
Attestation succeeds after a CC mode transition once the GPU has been reset with
--reset-after-cc-mode-switch(per the gpu-admin-tools README, no system reboot should be required), or at least after the documented remediation of device reset + driver reload.Environment
nvtrustpackage version: N/A (using the C++ SDK CLI:nvattest/libnvat1.2.2.1780962352-1, CUDA repo ubuntu2604)nvidia-driver-580-server-open, open kernel modules; GSP firmware 580.159.03)x86_64(Intel Xeon 6530P, Intel TDX)10de:2bb5,hwmodel: GB20X A01 GSP BROM)