RHAIIS: vLLM based model inferencing is failing with error "huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars"
Issue
- RHAIIS: vLLM based model inferencing is failing with error
# podman run --rm -it --device nvidia.com/gpu=all --security-opt=label=disable --shm-size=4GB -p 8000:8000 --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" --env "HF_HUB_OFFLINE=0" --env=VLLM_NO_USAGE_STATS=1 --userns=keep-id:uid=1001 -v ./rhaiis-cache:/opt/app-root/src/.cache registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.0.0 --model RedHatAI/llama-3-3-70b-instruct-fp8-dynamic:1.5 --tensor-parallel-size 1
.
.
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'RedHatAI/llama-3-3-70b-instruct-fp8-dynamic:1.5'.
.
.
ValueError: Invalid repository ID or local directory specified: 'RedHatAI/llama-3-3-70b-instruct-fp8-dynamic:1.5'.
Please verify the following requirements:
1. Provide a valid Hugging Face repository ID.
2. Specify a local directory that contains a recognized configuration file.
- For Hugging Face models: ensure the presence of a 'config.json'.
- For Mistral models: ensure the presence of a 'params.json'.
Environment
- Red Hat AI Inference Server 1.5
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.