RHAIIS: vLLM based model inferencing is failing with error "huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars"

Solution Verified - Updated -

Issue

  • RHAIIS: vLLM based model inferencing is failing with error
# podman run --rm -it --device nvidia.com/gpu=all --security-opt=label=disable --shm-size=4GB -p 8000:8000 --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" --env "HF_HUB_OFFLINE=0" --env=VLLM_NO_USAGE_STATS=1 --userns=keep-id:uid=1001 -v ./rhaiis-cache:/opt/app-root/src/.cache registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.0.0 --model RedHatAI/llama-3-3-70b-instruct-fp8-dynamic:1.5 --tensor-parallel-size 1
.
.
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'RedHatAI/llama-3-3-70b-instruct-fp8-dynamic:1.5'.
.
.
ValueError: Invalid repository ID or local directory specified: 'RedHatAI/llama-3-3-70b-instruct-fp8-dynamic:1.5'.
Please verify the following requirements:
1. Provide a valid Hugging Face repository ID.
2. Specify a local directory that contains a recognized configuration file.
   - For Hugging Face models: ensure the presence of a 'config.json'.
   - For Mistral models: ensure the presence of a 'params.json'.

Environment

  • Red Hat AI Inference Server 1.5

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content