Regarding the issue with the setup for RHEV 3.0 Beta

Latest response

Kindly update if you need any other logs for the same as we are doing the POC and would request for the response at the earliest.

Responses

Dear Team,

 

I have an active account with RHN and also registered the machine on RHN. We are doing the POC for one of our customer and facing while trying to add RHEL6 machine as the host for RHEV 3.0 Manager. The machine is already subscribed to the required channels.

 

Kindly find the attached logs and the snapshot for the error received.

 

Mon, 12 Dec 2011 04:22:40 DEBUG
Mon, 12 Dec 2011 04:22:40 DEBUG    <BSTRAP component='VDS PACKAGES' status='OK' result='libselinux-python' message='libselinux-python-2.0.94-5.2.el6.x86_64 '/>
Mon, 12 Dec 2011 04:22:40 DEBUG    Basic configuration found, skipping this step
Mon, 12 Dec 2011 04:22:40 DEBUG    <BSTRAP component='CreateConf' status='OK' message='Basic configuration found, skipping this step'/>
Mon, 12 Dec 2011 04:22:40 DEBUG    <BSTRAP component='CheckLocalHostname' status='OK' message='Local hostname is correct.'/>
Mon, 12 Dec 2011 04:22:40 DEBUG    Bridge rhevm already exists.
Mon, 12 Dec 2011 04:22:40 DEBUG    Bridge rhevm already exists. Skipping bridge creation.
Mon, 12 Dec 2011 04:22:40 DEBUG    <BSTRAP component='SetNetworking' status='OK' message='Bridge rhevm already exists. Skipping bridge creation.'/>
Mon, 12 Dec 2011 04:22:40 DEBUG    getAddress Entry. url=http://manager.rhevdemo.com:8080/Components/vds/
Mon, 12 Dec 2011 04:22:40 DEBUG    getAddress return. address=manager.rhevdemo.com port=8080
Mon, 12 Dec 2011 04:22:40 DEBUG    getRemoteFile start. IP = manager.rhevdemo.com port = 8080 fileName = "/rhevm.ssh.key.txt"
Mon, 12 Dec 2011 04:22:40 DEBUG    /rhevm.ssh.key.txt failed in HTTPS. Retrying using HTTP.
Traceback (most recent call last):
  File "/tmp/deployUtil.py", line 1272, in getRemoteFile
    conn.sock = getSSLSocket(sock, certPath)
  File "/tmp/deployUtil.py", line 1116, in getSSLSocket
    cert_reqs=ssl.CERT_REQUIRED)
  File "/usr/lib64/python2.6/ssl.py", line 338, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "/usr/lib64/python2.6/ssl.py", line 118, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 185090050] _ssl.c:328: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib
Mon, 12 Dec 2011 04:22:40 DEBUG    getRemoteFile end.
Mon, 12 Dec 2011 04:22:40 DEBUG    handleSSHKey start
Mon, 12 Dec 2011 04:22:40 DEBUG    handleSSHKey: failed to chmod authorized_keys
Traceback (most recent call last):
  File "/tmp/deployUtil.py", line 545, in handleSSHKey
    selinux.restorecon(P_ROOT_AUTH_KEYS)
  File "/usr/lib64/python2.6/site-packages/selinux/__init__.py", line 80, in restorecon
    status, context = matchpathcon(path, mode)
OSError: [Errno 2] No such file or directory
Mon, 12 Dec 2011 04:22:40 DEBUG    handleSSHKey end
Mon, 12 Dec 2011 04:22:40 DEBUG    <BSTRAP component='SetSSHAccess' status='FAIL' message='Failed to write server~s SSH key.'/>
Mon, 12 Dec 2011 04:22:40 ERROR    setSSHAccess test failed
Mon, 12 Dec 2011 04:22:40 DEBUG    <BSTRAP component='RHEV_INSTALL' status='FAIL'/>
Mon, 12 Dec 2011 04:22:40 DEBUG    **** End VDS Validation ****

Can you verify this is resolvable from the RHEL 6 host you are trying to add?

 

IE from the host you are trying yo install please ping the RHEV Manager address an do nslookup on it.

 

If it does not exist, please add it to your DNS or /etc/hosts

Dear Simon,

 

We are able to resolve frmo manager to RHEL6 machine:

 

[root@manager ~]# nslookup host2.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   host2.rhevdemo.com
Address: 10.10.10.202
[root@manager ~]# nslookup host2.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   host2.rhevdemo.com
Address: 10.10.10.202
 

 

and also able to resolve from RHEL6 machine to Manager:

 

[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 

 

[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 

 

[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 
 
However still getting the same error while trying to add the host in RHEV manager.
 
Thanks,
Ashish
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 

 

 

[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200
 

 

 

[root@host2 tmp]# nslookup manager.rhevdemo.com
Server:         10.10.10.212
Address:        10.10.10.212#53
 
Name:   manager.rhevdemo.com
Address: 10.10.10.200

apologies for the unsorted data.

Let me send the snapshot again for the same:

 

 

Dear Simon,

We are able to resolve frmo manager to RHEL6 machine:

 

[root@manager ~]# nslookup host2.rhevdemo.com
 
Server:         10.10.10.212
 
Address:        10.10.10.212#53
 
 
Name:   host2.rhevdemo.com
 
Address: 10.10.10.202
 
 
and also able to resolve from RHEL6 machine to Manager:
 
[root@host2 tmp]# nslookup manager.rhevdemo.com
 
Server:         10.10.10.212
 
Address:        10.10.10.212#53
 
 
Name:   manager.rhevdemo.com
 
Address: 10.10.10.200
 
 
Thanks,
Ashish

Replace my rhevm address with yours.

 

wget http://rhevm.demo.redhat.com:8080/rhevm.ssh.key.txt
--2011-12-22 15:09:23--  http://rhevm.demo.redhat.com:8080/rhevm.ssh.key.txt

Resolving rhevm.demo.redhat.com... 23.2.2.10

Connecting to rhevm.demo.redhat.com|23.2.2.10|:8080... connected.

HTTP request sent, awaiting response... 200 OK

Length: 219 [text/plain]

Saving to: “rhevm.ssh.key.txt”
 

 

[root@manager ~]# login as: root
root@10.10.10.200's password:
Last login: Thu Dec 22 14:36:00 2011 from 10.10.10.118
[root@manager ~]# wget http://manager.rhevdemo.com:8080/rhevm.ssh.key.txt
--2011-12-22 17:47:49--  http://manager.rhevdemo.com:8080/rhevm.ssh.key.txt
Resolving manager.rhevdemo.com... 10.10.10.200
Connecting to manager.rhevdemo.com|10.10.10.200|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 219 [text/plain]
Saving to: ârhevm.ssh.key.txtâ
 
100%[===========================================================================================>] 219         --.-K/s   in 0s
 
2011-12-22 17:47:49 (27.2 MB/s) - ârhevm.ssh.key.txtâ
[root@manager ~]# login as: root
root@10.10.10.200's password:
Last login: Thu Dec 22 14:36:00 2011 from 10.10.10.118
[root@manager ~]# wget http://manager.rhevdemo.com:8080/rhevm.ssh.key.txt
--2011-12-22 17:47:49--  http://manager.rhevdemo.com:8080/rhevm.ssh.key.txt
Resolving manager.rhevdemo.com... 10.10.10.200
Connecting to manager.rhevdemo.com|10.10.10.200|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 219 [text/plain]
Saving to: ârhevm.ssh.key.txtâ
 
100%[===========================================================================================>] 219         --.-K/s   in 0s
 
2011-12-22 17:47:49 (27.2 MB/s) - ârhevm.ssh.key.txtâ
 

That is fails here:

    if fReturn:
        try:
            import selinux
            os.chmod(P_ROOT_AUTH_KEYS, 0644)
            selinux.restorecon(P_ROOT_AUTH_KEYS)
        except:
            fReturn = False
            logging.debug("handleSSHKey: failed to chmod authorized_keys",
                          exc_info=True)

Can it be that selinux is disabled on your host?

 

The reason that I'm asking is because it looks  it did pass the previous stage

    if fReturn:
        try:
            addSSHKey(P_ROOT_AUTH_KEYS, strKey)
        except:
            fReturn = False
            logging.debug("handleSSHKey: failed to write authorized_keys!",
                          exc_info=True)
 

So the key was created however it fails when trying to apply SELinux rules.

yes, selinux is disabled in the grub.conf file as if it is not disabled there it gives the error "kernel panic - not syncing.

 

Kindly suggest.

 

 

Thanks,

Ashish

Can you post your grub.conf and a console output for the panic? RHEV hosts require SELinux for secure operation, and it needs to be turned on

If I let the selinux enabled, I get the following error while loading:

"kernel panic - not syncing: attempted to kill init !

panic occoured, switching back to text console

 

kindky find my grub file output:

 

[root@host2 ~]# cat /etc/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda2
#          initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-220.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=UUID=a2b54d53-f379-481d-acbb-bad709c24467 rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet elevator=deadline processor.max_cstate=1
        initrd /initramfs-2.6.32-220.el6.x86_64.img
title Red Hat Enterprise Linux (2.6.32-131.0.15.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-131.0.15.el6.x86_64 ro root=UUID=a2b54d53-f379-481d-acbb-bad709c24467 rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet
        initrd /initramfs-2.6.32-131.0.15.el6.x86_64.img
 
 
kindly suggest.

Hi All, 

 

Any update on the same?

Yes, can you please turn SELinux on, and reset it to defaults? Then try to add the host again.

Dear Dan, 

 

Whenever I trun the Selinux on, I get the below mentioned error as Kernel panic at the boot prompt:

 

"kernel panic - not syncing: attempted to kill init !

panic occoured, switching back to text console"

 

 

That's why I have turned off the selinux at the grub promot so that I can boot my machine.

 

Kindly suggest. 

 

This is a labeling problem.  If you change the machine to permissive mode you should be able to fix the labeling.

Kernel parameters "enforcing=0 autorelabel"  Should cause the machine to boot in permissive mode and trigger a relabel.

Hi Team, 

 

Even after making the changes in grub file, recieved the same error:

 

 Mon, 26 Dec 2011 01:57:45 DEBUG    Bridge rhevm already exists.

Mon, 26 Dec 2011 01:57:45 DEBUG    Bridge rhevm already exists. Skipping bridge creation.
Mon, 26 Dec 2011 01:57:45 DEBUG    <BSTRAP component='SetNetworking' status='OK' message='Bridge rhevm already exists. Skipping bridge creation.'/>
Mon, 26 Dec 2011 01:57:45 DEBUG    getAddress Entry. url=http://manager.rhevdemo.com:8080/Components/vds/
Mon, 26 Dec 2011 01:57:45 DEBUG    getAddress return. address=manager.rhevdemo.com port=8080
Mon, 26 Dec 2011 01:57:45 DEBUG    getRemoteFile start. IP = manager.rhevdemo.com port = 8080 fileName = "/rhevm.ssh.key.txt"
Mon, 26 Dec 2011 01:57:45 DEBUG    /rhevm.ssh.key.txt failed in HTTPS. Retrying using HTTP.
Traceback (most recent call last):
  File "/tmp/deployUtil.py", line 1272, in getRemoteFile
    conn.sock = getSSLSocket(sock, certPath)
  File "/tmp/deployUtil.py", line 1116, in getSSLSocket
    cert_reqs=ssl.CERT_REQUIRED)
  File "/usr/lib64/python2.6/ssl.py", line 338, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "/usr/lib64/python2.6/ssl.py", line 118, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 185090050] _ssl.c:328: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib
Mon, 26 Dec 2011 01:57:45 DEBUG    getRemoteFile end.
Mon, 26 Dec 2011 01:57:45 DEBUG    handleSSHKey start
Mon, 26 Dec 2011 01:57:45 DEBUG    handleSSHKey: failed to chmod authorized_keys
Traceback (most recent call last):
  File "/tmp/deployUtil.py", line 545, in handleSSHKey
    selinux.restorecon(P_ROOT_AUTH_KEYS)
  File "/usr/lib64/python2.6/site-packages/selinux/__init__.py", line 80, in restorecon
    status, context = matchpathcon(path, mode)
OSError: [Errno 2] No such file or directory
Mon, 26 Dec 2011 01:57:45 DEBUG    handleSSHKey end
Mon, 26 Dec 2011 01:57:45 DEBUG    <BSTRAP component='SetSSHAccess' status='FAIL' message='Failed to write server~s SSH key.'/>
Mon, 26 Dec 2011 01:57:45 ERROR    setSSHAccess test failed
Mon, 26 Dec 2011 01:57:45 DEBUG    <BSTRAP component='RHEV_INSTALL' status='FAIL'/>
Mon, 26 Dec 2011 01:57:45 DEBUG    **** End VDS Validation ****
 
 
also attaching the output for my grub.conf file:
=================================================================================
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda2
#          initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-220.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=UUID=a2b54d53-f379-481d-acbb-bad709c24467 rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet elevator=deadline processor.max_cstate=1 enforcing=0 autorelabel
        initrd /initramfs-2.6.32-220.el6.x86_64.img
title Red Hat Enterprise Linux (2.6.32-131.0.15.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-131.0.15.el6.x86_64 ro root=UUID=a2b54d53-f379-481d-acbb-bad709c24467 rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet
        initrd /initramfs-2.6.32-131.0.15.el6.x86_64.img