fix(ort): CUDA detection was gated behind the TensorRT-EP build flag

detect_ort_provider() only tested for CUDAExecutionProvider inside
#ifdef SAE_ORT_WITH_TRT_EP, so any build that did not also opt into the
TensorRT execution provider could never select CUDA and fell straight
through to the CPU. The two are independent: the TRT EP needs the
headers and profile plumbing and is rightly an opt-in, CUDA is a plain
ORT provider and is not.

The failure is silent rather than loud, which is why it survived --
inference runs on the CPU and every answer is still correct, just far
slower. Measured on the VR-012 study: 0.32 s/crop against 0.0021 s/crop
once a GPU backend is actually used, with the card sitting at 212 MiB
and 0% utilisation throughout.

Only the TensorrtExecutionProvider line stays inside the guard.
This commit is contained in:
2026-07-31 22:25:00 +02:00
parent 1ae88376e1
commit 41d30395da
+12 -1
View File
@@ -23,12 +23,23 @@
enum class OrtProvider { CPU, CUDA, ROCm, TensorRT }; enum class OrtProvider { CPU, CUDA, ROCm, TensorRT };
inline OrtProvider detect_ort_provider() { inline OrtProvider detect_ort_provider() {
// ORT returns these in its own preference order (TensorRT, CUDA, ..., CPU
// last), so the first recognised entry is the best available and the loop
// returns on it.
auto available = Ort::GetAvailableProviders(); auto available = Ort::GetAvailableProviders();
for (const auto& p : available) { for (const auto& p : available) {
// Only the TensorRT *EP* is a build-time opt-in — it needs the headers
// and the profile plumbing below. CUDA is not: it is a plain ORT
// provider, and gating its detection on the TRT flag (as this did) made
// the CUDA branch unreachable in every build that did not also ask for
// TensorRT. The symptom is silent rather than loud — inference simply
// runs on the CPU and everything still returns correct answers — which
// is why it survived: a 300-actor VR-012 grid cell took 76 s on the CPU
// with the GPU idle at 212 MiB.
#ifdef SAE_ORT_WITH_TRT_EP #ifdef SAE_ORT_WITH_TRT_EP
if (p == "TensorrtExecutionProvider") return OrtProvider::TensorRT; if (p == "TensorrtExecutionProvider") return OrtProvider::TensorRT;
if (p == "CUDAExecutionProvider") return OrtProvider::CUDA;
#endif #endif
if (p == "CUDAExecutionProvider") return OrtProvider::CUDA;
if (p == "ROCMExecutionProvider") return OrtProvider::ROCm; if (p == "ROCMExecutionProvider") return OrtProvider::ROCm;
} }
return OrtProvider::CPU; return OrtProvider::CPU;