On RHN Satellite v5.4.1, receiving traceback with error, 'ORA-03135: connection lost contact'

Solution Verified - Updated -

Environment

  • Red Hat Network(RHN) Satellite 5.4.1

Issue

  • On RHN Satellite v5.4.1, receiving traceback with error, 'ORA-03135: connection lost contact'

Resolution

  • Ensure that network connectivity is restored between clients and servers, and servers are operating within the recommended resource allocation.

Root Cause

This error is usually associated with clients losing connection to the RHN Satellite database:

  • Typically attributed to a networking problem between the client and the satellite server. Clients on different subnets from the Satellite server may experience loss of connection during partial networking outages or disconnects.

  • Recommended resource allocation for the satellite server not being met may cause the database to deny new connections or lose existing connections. Please click the following if also receiving the error "ORA-04031: unable to allocate 32 bytes of shared memory"

  • The RHN Satellite server is unavailable for another reason (ie. planned maintenance)

Diagnostic Steps

  • Received following traceback via email from the satellite client system:

    Exception reported from xx.xx.xx.xx
    Time: Sat Oct 22 02:49:56 2011
    Exception type server.rhnSQL.sql_base.SQLConnectError
    Request object information:
    URI: /XMLRPC
    Remote Host: 161.154.134.226
    Server Name: www.example.com:443
    Headers passed in:
         Accept-Encoding: identity
         Content-Length: 2166
         Host: www.example.com:443
         content-type: text/xml
         user-agent: rhn.rpclib.py/$Revision: 190936 $
         x-client-version: 1
         x-info: RPC Processor (C) Red Hat, Inc (version 195664)
    
         x-rhn-client-capability: 
    packages.verifyAll(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1,packages.verifyAll(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1
         x-rhn-transport-capability: follow-redirects=3
         x-transport-info: Extended Capabilities Transport (C) Red Hat, Inc (version 190936)
         x-up2date-version: 0.4.20-56.el5
    
    Exception Handler Information
    Traceback (most recent call last):
      File "/usr/share/rhn/server/apacheHandler.py", line 84, in headerParserHandler
        rhnSQL.initDB()
      File "/usr/share/rhn/server/rhnSQL/__init__.py", line 121, in initDB
        __init__DB(backend, host, port, username, password, database)
      File "/usr/share/rhn/server/rhnSQL/__init__.py", line 52, in __init__DB
        __DB.connect()
      File "/usr/share/rhn/server/rhnSQL/driver_cx_Oracle.py", line 385, in connect
        raise apply(sql_base.SQLConnectError, err_args)
    SQLConnectError: (3135, 'ORA-03135: connection lost contact\n', '<CENSORED!>', 'Connection_Connect(): begin session')
    
    Local variables by frame
    Frame connect in /usr/share/rhn/server/rhnSQL/driver_cx_Oracle.py at line 385
                            e = <type 'instance'> ORA-03135: connection lost contact
    
                        errno = <type 'int'> 3135
                         self = <type 'instance'> <server.rhnSQL.driver_cx_Oracle.Database instance at 0x2afc0cdedf80>
    
                          ret = <type 'tuple'> (3135, 'ORA-03135: 
    connection lost contact\n', 'Connection_Connect(): begin session')
                    reconnect = <type 'int'> 1
    
                     err_args = <type 'list'> ['<CENSORED!>', 3135, 
    'ORA-03135: connection lost contact\n', 'Connection_Connect(): begin 
    session']
                       errmsg = <type 'str'> ORA-03135: connection lost contact
    
    
    Frame __init__DB in /usr/share/rhn/server/rhnSQL/__init__.py at line 52
                     username = <type 'str'> <CENSORED!>
                     db_class = <type 'classobj'> server.rhnSQL.driver_cx_Oracle.Database
                     database = <type 'str'> <CENSORED!>
                         host = <type 'str'> localhost
                     password = <type 'str'> <CENSORED!>
                         port = <type 'NoneType'> None
                      backend = <type 'str'> oracle
    
    Frame initDB in /usr/share/rhn/server/rhnSQL/__init__.py at line 127
                     username = <type 'str'> <CENSORED!>
                     database = <type 'str'> <CENSORED!>
                          dsn = <type 'str'> <CENSORED!>
                         host = <type 'str'> localhost
                     password = <type 'str'> <CENSORED!>
                         port = <type 'NoneType'> None
                      backend = <type 'str'> oracle
    
    Frame headerParserHandler in /usr/share/rhn/server/apacheHandler.py at line 86
                         self = <type 'instance'> <server.apacheHandler.apacheHandler instance at 0x2afc0cbc6200>
                          req = <type 'mp_request'> <mp_request object at 0x2afbf659d050>
    
                      options = <type 'mp_table'> {'RHNComponentType': 
    'server.xmlrpc', 'SERVER': 'xmlrpc', 'RootDir': '/usr/share/rhn'}
                          ret = <type 'int'> 0
    
    Frame HandlerDispatch in /usr/lib64/python2.4/site-packages/mod_python/apache.py at line 299
    
                      newpath = <type 'list'> ['/usr/lib64/python24.zip', 
    '/usr/lib64/python2.4', '/usr/lib64/python2.4/plat-linux2', 
    '/usr/lib64/python2.4/lib-tk', '/usr/lib64/python2.4/lib-dynload', 
    '/usr/lib64/python2.4/site-packages', 
    '/usr/lib64/python2.4/site-packages/Numeric', 
    '/usr/lib64/python2.4/site-packages/gtk-2.0', 
    '/usr/lib/python2.4/site-packages', '/usr/share/rhn']
    
    object = <type 'instancemethod'> <bound method 
    apacheHandler.headerParserHandler of 
    <server.apacheHandler.apacheHandler instance at 
    0x2afc0cbc6200>>
                   pathstring = <type 'str'> sys.path+['/usr/share/rhn']
                         self = <type 'instance'> <mod_python.apache.CallBack instance at 0x2afbf6595b90>
                          req = <type 'mp_request'> <mp_request object at 0x2afbf659d050>
                            l = <type 'list'> ['server.apacheServer', 'HeaderParserHandler']
                       module = <type 'module'> <module 'server.apacheServer' from '/usr/share/rhn/server/apacheServer.pyc'>
    
                        hlist = <type 'mp_hlist'> 
    {'handler:'server.apacheServer::HeaderParserHandler','directory':'/XMLRPC/','silent':0}
                   object_str = <type 'str'> HeaderParserHandler
                        debug = <type 'int'> 0
                  module_name = <type 'str'> server.apacheServer
                       config = <type 'mp_table'> {'PythonInterpreter': 'rhn.server', 'PythonPath': 'sys.path+['/usr/share/rhn']'}
                       result = <type 'int'> 500
    
    
    Environment for PID=22229 on exception:
    LANG = C
    NLS_LANG = english.AL32UTF8
    ORACLE_HOME = /opt/oracle
    PATH = /sbin:/usr/sbin:/bin:/usr/bin
    PERL_BADFREE = 0
    PWD = /
    SHLVL = 2
    TERM = xterm
    _ = /usr/sbin/httpd
    

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments