RHEL AI: How much disk space will be required for single and multi phase LLM training?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux AI 1.2

Issue

  • RHEL AI: How much disk space will be required for single and multi phase LLM training?

Resolution

1. LLM MODELS (For Inference Only)

  • granite-7b-redhat-lab : 13 GB

Note: Total 13 GB space will be required for LLM downloads.

2. LLM Models (For Synthetic Data Generation and Training)

  • skills-adapter-v3 : 111 MB
  • knowledge-adapter-v3 : 111 MB
  • granite-7b-starter : 13 GB
  • prometheus-8x7b-v2-0 : 88 GB
  • mixtral-8x7b-instruct-v0-1 : 88 GB

Note: Total 187 GB space will be required for LLM downloads.

3. Single Phase Training

  • Default epoch value from single phase training is 10 as per file ~/.config/instructlab/config.yaml
train:

num_epochs: 10
  • For each epoch run RHEL AI generates following amount of data
    • Dataset size per qna.yaml : 2 GB
    • Each epoch checkpoint (~/.local/share/instructlab/checkpoints/full_state) : 88 GB
    • Each epoch based LLM (~/.local/share/instructlab/checkpoints/hf_format) : 13 GB

Note: Tentatively around 1200 GB space will be required for single phase knowledge training.

4. Multi-Phase Training

  • Default epoch value from multipe phase training per file ~/.config/instructlab/config.yaml
    • Phase 1: 7
    • Phase 2: 10
train:

phased_phase1_num_epochs: 7
phased_phase2_num_epochs: 10

  • In Phase 1, for each epoch run RHEL AI generates following amount of data

    • Dataset size per qna.yaml : 2 GB
    • Each epoch checkpoint (~/.local/share/instructlab/phased/phase1/checkpoints/full_state) : 88 GB
    • Each epoch based LLM (~/.local/share/instructlab/phased/phase1/checkpoints/hf_format) : 13 GB

  • In Phase 2, for each epoch run RHEL AI generates following amount of data

    • Dataset size per qna.yaml : 2 GB
    • Each epoch checkpoint (~/.local/share/instructlab/phased/phase2/checkpoints/full_state) : 88 GB
    • Each epoch based LLM (~/.local/share/instructlab/phased/phase2/checkpoints/hf_format) : 13 GB

Note: Tentatively around 1910 GB space will be required for multi phase training.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments