`radosgw-agent` enters an infinite loop, and the replication fails with the message "IndexError: list index out of range", why?
Issue
- Zone replications between two RGW nodes stall, with
radosgw-agentlogging the following messages in the logs:
Wed, 04 May 2016 14:32:56 GMT
/<access>/<path>
2016-05-04 16:32:56,178 248448 [boto][DEBUG ] Signature:
AWS system-access:eC+mL+ndaPVSbUf3QqLoxVtRjMA=
2016-05-04 16:32:56,178 248448 [boto][DEBUG ] Final headers: {'Date': 'Wed, 04 May 2016 14:32:56 GMT', 'Content-Length': 0, 'Authorization': u'AWS system-access:eC+mL+ndaPVSbUf3QqLoxVtRjMA=', 'User-Agent': 'Boto/2.32.1 Python/2.6.6 Linux/3.10.94-1.el6.x86_64'}
2016-05-04 16:32:56,187 248448 [boto][DEBUG ] Response headers: [('date', 'Wed, 04 May 2016 14:32:56 GMT'), ('content-length', '2'), ('content-type', 'application/json')]
2016-05-04 16:32:56,188 248448 [radosgw_agent.worker][DEBUG ] op state is []
2016-05-04 16:32:56,188 248448 [radosgw_agent.worker][DEBUG ] error geting op state: list index out of range
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/radosgw_agent/worker.py", line 275, in wait_for_object
state = state[0]['state']
IndexError: list index out of range
- Restarting the
radosgw-agentprocess does not solve the problem.
Environment
-
Upstream Ceph Hammer (0.94.x)
-
radosgw-agentversion 1.2.3
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.