`radosgw-agent` enters an infinite loop, and the replication fails with the message "IndexError: list index out of range", why?

Solution In Progress - Updated -

Issue

  • Zone replications between two RGW nodes stall, with radosgw-agent logging the following messages in the logs:
Wed, 04 May 2016 14:32:56 GMT
/<access>/<path>
2016-05-04 16:32:56,178 248448 [boto][DEBUG ] Signature:
AWS system-access:eC+mL+ndaPVSbUf3QqLoxVtRjMA=
2016-05-04 16:32:56,178 248448 [boto][DEBUG ] Final headers: {'Date': 'Wed, 04 May 2016 14:32:56 GMT', 'Content-Length': 0, 'Authorization': u'AWS system-access:eC+mL+ndaPVSbUf3QqLoxVtRjMA=', 'User-Agent': 'Boto/2.32.1 Python/2.6.6 Linux/3.10.94-1.el6.x86_64'}
2016-05-04 16:32:56,187 248448 [boto][DEBUG ] Response headers: [('date', 'Wed, 04 May 2016 14:32:56 GMT'), ('content-length', '2'), ('content-type', 'application/json')]
2016-05-04 16:32:56,188 248448 [radosgw_agent.worker][DEBUG ] op state is []
2016-05-04 16:32:56,188 248448 [radosgw_agent.worker][DEBUG ] error geting op state: list index out of range
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/radosgw_agent/worker.py", line 275, in wait_for_object
    state = state[0]['state']
IndexError: list index out of range
  • Restarting the radosgw-agent process does not solve the problem.

Environment

  • Upstream Ceph Hammer (0.94.x)

  • radosgw-agent version 1.2.3

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.