RHEL AI 3.3 with B300 GPU is certified with NVLink limitation on SuperServer SYS-822GS-NB3RT

Solution Verified - Updated -

Issue

The vLLM inference fails with NVIDIA B300 GPU due to a CUDA initialization error. This occurs because RHEL AI 3.3 does not include the DOCA/OFED InfiniBand drivers required by the NVIDIA Fabric Manager service. Fabric Manager manages NVLink/NVSwitch communication between GPUs, and without it, CUDA cannot initialize on multi-GPU B300 NVSwitch-based platforms.

/usr/local/lib/python3.12/dist-packages/torch/cuda/__init__.py:180: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 802: system not yet initialized (Triggered internally at /opt/pytorch/pytorch/c10/cuda/CUDAFunctions.cpp:119.)

Environment

  • Red Hat Enterprise Linux AI (RHEL AI) 3.3
  • Supermicro SuperServer SYS-822GS-NB3RT
  • NVIDIA B300 GPU

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content