AI System Card — Red Hat AI Inference

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Purpose

Red Hat AI Inference is an enterprise AI model-serving runtime and orchestration platform. It enables organizations to deploy, serve, and manage AI/ML models at production scale on their own infrastructure. The platform provides optimized inference serving (vLLM), model compression and validation (Model Toolkit / llm-compressor), and distributed inference orchestration across GPU clusters (llm-d).

Red Hat AI Inference is classified as Option B (AI-supporting infrastructure) in the AIMS. It does not make AI decisions itself — it serves the models that downstream applications use. Impact assessment (A.5) and use controls (A.9) are carried by the dependent products (Ask Red Hat, RHEL Lightspeed).

Technical Information

Components

Data Provenance and Pedigree

Red Hat AI Inference does not train, fine-tune, or curate AI models. It serves pre-trained models provided by third parties or customers.

Validated model repository: Red Hat publishes pre-optimized versions of supported models on Hugging Face and as signed OCI artifacts in Red Hat Registry. Each model undergoes:

• Adversarial red-teaming via Garak (prompt injection, jailbreaking, PII leakage)
• Compression and quantization via llm-compressor
• Task-based performance benchmarking across validated hardware configurations
• Cryptographic signing via Red Hat Trusted Software Supply Chain with AIBOM and in-toto provenance attestations

Model providers and their licenses:

• Meta (Llama) — Llama Community License
• Mistral — Apache 2.0
• IBM (Granite) — Apache 2.0
• Alibaba (Qwen) — Apache 2.0
• DeepSeek — MIT License
• Google (Gemma) — Gemma Terms of Use
• NVIDIA (Nemotron) — NVIDIA Open Model License
• Microsoft (Phi) — MIT License

BYOM (Bring-Your-Own-Model): Customers may deploy their own models. RHAII does not validate BYOM model integrity, licensing, or safety. SafeTensors format is strongly recommended to mitigate unsafe deserialization risks.

Security and Safety of the AI System

Intent and use:

Red Hat AI Inference is designed for enterprise platform engineering teams to deploy and serve AI models at production scale on their own infrastructure — on-premises, air-gapped, or sovereign cloud. It is foundational AI infrastructure for regulated industries including financial services, healthcare, government, and defense. Downstream applications depend on the accuracy, integrity, and availability of its inference outputs.

Scope:

RHAII serves models — it does not make autonomous decisions, generate content independently, or interact directly with end users. Its scope is limited to:

• Receiving inference requests via API
• Routing requests to the appropriate model and GPU
• Returning model outputs to the requesting application
• Providing operational telemetry (no prompt/completion content in metrics)

RHAII does not process, store, or transmit customer inference data to Red Hat. All data remains within the customer's control boundary.

Security considerations:

• SDLC threat model completed: 29 countermeasures identified via SD Elements, all addressed and closed (INFERENG-614)
• Security Architecture Review (SAR) completed: Full report available (INFERENG-689)
• ProdSec penetration testing completed: Findings tracked under INFERENG-1056. ProdSec release sign-off granted (INFERENG-708)
• Upstream vulnerability management: Active, ongoing (INFERENG-1960). Daily automated CVE scanning. Monthly z-stream releases + out-of-band critical patches.
• Supply chain security: Hermetic builds via AIPCC pipeline. Container image scanning, SBOM generation, cryptographic signing. Fast restriction mechanism blocks vulnerable package versions within minutes.
• Incident response: ProdSec/InfoSec notified within 1 hour, leadership within 2 hours. Compromised package response runbook tested against real incidents (LiteLLM March 2026, lightning April 2026).
• Export classification: ECCN 5D002.b(1) — encryption (TLS, SSH, IPSec inherited from RHEL 9)

Security flaws and Safety hazards:

Governance

Any security or safety issues related to the AI system should be reported to security@redhat.com.

System owner: Erwan Gallen (egallen@redhat.com), Product Manager, Red Hat AI. ProdSec Security Engineer: Mihail Milev (mmilev@redhat.com). Upstream vulnerability management lead: Russell Bryant (rbryant@redhat.com).

SDLC Feature: SDLC-5157 — Red Hat Inference Server 3.0 (Closed). AIPCC Security Policy Compliance: RHAISTRAT-2046 (In Progress).

References

• Red Hat AI Inference Product Definition: https://docs.google.com/presenta tion/d/1uB2oWmHr5eniUY8Pp9NVKMNDItFiaTr78Vn7wm7uow8/edit
• RHAII Architecture Document: https://docs.google.com/docum ent/d/1Ia25oeo8FAoRWa_IwzabqXRaoXFCMB6ppa5z0o3AfTQ/edit?tab=t.z2zhlf53lzuo
• vLLM Project: https://github.com/vllm-project/vllm
• llm-d Project: https://github.com/llm-d
• llm-compressor: https://github.com/vllm-project/llm-compressor
• Red Hat AI Model Repository on Hugging Face: https://huggingface.co/RedHatAI
• SD Elements Threat Model Tracker: http s://redhat.sdelements.com/bunits/psse-secure-development/group-e-emergent-business-priorities/red-hat-inference-server/
• Red Hat Secure Software Development Lifecycle: https://www.redhat.com/en/topics/ security/software-development-lifecycle-security

Previous System Card Releases

• None — this is the initial version (v1.0).

Attachments

• Red Hat AI Inference Shared Responsibility Model