Select Your Language

Infrastructure and Management

Cloud Computing

Storage

Runtimes

Integration and Automation

  • Comments
  • failed to get device already provisioned by ceph-volume raw: osd.1: belonging to a different ceph cluster

    Posted on

    We're trying to recreate the Ceph cluster inside Kubernetes, but we're still encountering the same issue.
    Here’s what we’ve already tried:

    Ran sudo sgdisk --zap-all /dev/sda

    Ran sudo dd if=/dev/zero of=/dev/sda bs=1M count=10

    Despite these steps, the problem persists.

    ERROR:
    2025-05-22 06:11:30.973751 E | cephosd: failed to determine if the encrypted block "/mnt/ocs-deviceset-odflvs-0-data-3zlqsz" is from our cluster. failed to dump LUKS header for disk "/mnt/ocs-deviceset-odflvs-0-data-3zlqsz". Device /mnt/ocs-deviceset-odflvs-0-data-3zlqsz is not a valid LUKS device.: exit status 1
    2025-05-22 06:11:30.973988 D | exec: Running command: stdbuf -oL ceph-volume --log-path /tmp/ceph-log raw list /mnt/ocs-deviceset-odflvs-0-data-3zlqsz --format json
    2025-05-22 06:11:31.508337 D | cephosd: {
    "6ac6e9bc-cfbe-4740-8a8b-7967c6732af1": {
    "ceph_fsid": "1d330653-4f95-4e7c-9e6c-501d68ae3dce",
    "device": "/dev/sda",
    "osd_id": 3,
    "osd_uuid": "6ac6e9bc-cfbe-4740-8a8b-7967c6732af1",
    "type": "bluestore"
    }
    }
    2025-05-22 06:11:31.508493 I | cephosd: skipping device "/mnt/ocs-deviceset-odflvs-0-data-3zlqsz": failed to detect if there is already an osd. osd.3: "6ac6e9bc-cfbe-4740-8a8b-7967c6732af1" belonging to a different ceph cluster "1d330653-4f95-4e7c-9e6c-501d68ae3dce".
    2025-05-22 06:11:31.520136 I | cephosd: configuring osd devices: {"Entries":{}}
    2025-05-22 06:11:31.520171 I | cephosd: no new devices to configure. returning devices already configured with ceph-volume.
    2025-05-22 06:11:31.520180 D | exec: Running command: pvdisplay -C -o lvpath --noheadings /mnt/ocs-deviceset-odflvs-0-data-3zlqsz
    2025-05-22 06:11:31.552469 W | cephosd: failed to retrieve logical volume path for "/mnt/ocs-deviceset-odflvs-0-data-3zlqsz". exit status 5
    2025-05-22 06:11:31.552516 D | exec: Running command: lsblk /mnt/ocs-deviceset-odflvs-0-data-3zlqsz --bytes --nodeps --pairs --paths --output SIZE,ROTA,RO,TYPE,PKNAME,NAME,KNAME,MOUNTPOINT,FSTYPE
    2025-05-22 06:11:31.555531 D | sys: lsblk output: "SIZE=\"536870912000\" ROTA=\"1\" RO=\"0\" TYPE=\"disk\" PKNAME=\"\" NAME=\"/dev/sda\" KNAME=\"/dev/sda\" MOUNTPOINT=\"\" FSTYPE=\"\""
    2025-05-22 06:11:31.555574 D | exec: Running command: stdbuf -oL ceph-volume --log-path /tmp/ceph-log lvm list --format json
    2025-05-22 06:11:31.835940 D | cephosd: {}
    2025-05-22 06:11:31.836015 I | cephosd: 0 ceph-volume lvm osd devices configured on this node
    2025-05-22 06:11:31.836029 D | exec: Running command: cryptsetup luksDump /mnt/ocs-deviceset-odflvs-0-data-3zlqsz
    2025-05-22 06:11:31.841994 E | cephosd: failed to determine if the encrypted block "/mnt/ocs-deviceset-odflvs-0-data-3zlqsz" is from our cluster. failed to dump LUKS header for disk "/mnt/ocs-deviceset-odflvs-0-data-3zlqsz". Device /mnt/ocs-deviceset-odflvs-0-data-3zlqsz is not a valid LUKS device.: exit status 1
    2025-05-22 06:11:31.842041 D | exec: Running command: stdbuf -oL ceph-volume --log-path /tmp/ceph-log raw list /mnt/ocs-deviceset-odflvs-0-data-3zlqsz --format json
    2025-05-22 06:11:32.362022 D | cephosd: {
    "6ac6e9bc-cfbe-4740-8a8b-7967c6732af1": {
    "ceph_fsid": "1d330653-4f95-4e7c-9e6c-501d68ae3dce",
    "device": "/dev/sda",
    "osd_id": 3,
    "osd_uuid": "6ac6e9bc-cfbe-4740-8a8b-7967c6732af1",
    "type": "bluestore"
    }
    }

    by

    points

    Responses

    Red Hat LinkedIn YouTube Facebook X, formerly Twitter

    Quick Links

    Help

    Site Info

    Related Sites

    © 2026 Red Hat