Ceph - After upgrade to RHCS 3.3z1 RGW daemon experiences segmentation fault
Issue
- After upgrading the Red Hat Ceph Storage Cluster to 3.3z1 (12.2.12-74.el7cp or 12.2.12-67redhat1xenial) the RGW daemons experiences segmentation faults.
- Errata release with issue is RHBA-2019:3173
- Segmentation fault will appear similar to this output:
2019-10-24 10:35:56.813839 7f9a20fd3700 1 civetweb: 0x55a720ef2000: 10.134.32.28 - - [24/Oct/2019:10:35:56 +0200] "GET /cc-droplets/?prefix=buildpack_cache%2Fb7%2F09%2Fb70908c0-5e3e-4fd4-8489-69c3407b996a HTTP/1.1" 200 849 - fog-core/1.43.0
2019-10-24 10:35:56.816960 7f9a207d2700 -1 *** Caught signal (Segmentation fault) **
in thread 7f9a207d2700 thread_name:civetweb-worker
ceph version 12.2.12-74.el7cp (6c4a9c2235eb0c7e3d61719cdc1d6b7b2dcbdea9) luminous (stable)
1: (()+0x2d82a1) [0x55a71e13f2a1]
2: (()+0xf630) [0x7f9a5719e630]
3: (ceph_str_hash_rjenkins(char const*, unsigned int)+0x20) [0x7f9a4e9c4500]
4: (pg_pool_t::hash_key(std::string const&, std::string const&) const+0xc0) [0x7f9a4e95e980]
5: (OSDMap::map_to_pg(long, std::string const&, std::string const&, std::string const&, pg_t*) const+0x75) [0x7f9a4e917975]
6: (OSDMap::object_locator_to_pg(object_t const&, object_locator_t const&, pg_t&) const+0xa6) [0x7f9a4e917a66]
7: (()+0xcc447) [0x7f9a58159447]
8: (()+0xd9c59) [0x7f9a58166c59]
9: (()+0xe6df8) [0x7f9a58173df8]
10: (()+0xe707a) [0x7f9a5817407a]
11: (librados::IoCtxImpl::aio_operate(object_t const&, ObjectOperation*, librados::AioCompletionImpl*, SnapContext const&, int, blkin_trace_info const*)+0x1a1) [0x7f9a58124191]
12: (librados::IoCtx::aio_operate(std::string const&, librados::AioCompletion*, librados::ObjectWriteOperation*)+0x53) [0x7f9a580e6d33]
13: (RGWRados::cls_bucket_list_ordered(RGWBucketInfo&, int, cls_rgw_obj_key const&, std::string const&, unsigned int, bool, std::map<std::string, rgw_bucket_dir_entry, std::less<std::string>, std::allocator<std::pair<std::string const, rgw_bucket_dir_entry> > >&, bool*, cls_rgw_obj_key*, bool (*)(std::string const&))+0x13f1) [0x55a71e2cb501]
14: (RGWRados::Bucket::List::list_objects_ordered(long, std::vector<rgw_bucket_dir_entry, std::allocator<rgw_bucket_dir_entry> >*, std::map<std::string, bool, std::less<std::string>, std::allocator<std::pair<std::string const, bool> > >*, bool*)+0x3fa) [0x55a71e2cbc4a]
15: (RGWListBucket::execute()+0x25c) [0x55a71e22481c]
16: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, bool)+0x188) [0x55a71e25a8b8]
17: (process_request(RGWRados*, RGWREST*, RGWRequest*, std::string const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSocket*, int*)+0xb88) [0x55a71e25b678]
18: (RGWCivetWebFrontend::process(mg_connection*)+0x3a2) [0x55a71e0bdaa2]
19: (()+0x2c8517) [0x55a71e12f517]
20: (()+0x2c9dd2) [0x55a71e130dd2]
21: (()+0x2ca5a8) [0x55a71e1315a8]
22: (()+0x7ea5) [0x7f9a57196ea5]
23: (clone()+0x6d) [0x7f9a4b6348cd]
Environment
- Upgrade to Red Hat Ceph Storage 3.3z1
- Red Hat Ceph Storage 3.3z1
- Ceph 12.2.12-74.el7cp
- Ceph 12.2.12-67redhat1xenial
- Errata RHBA-2019:3173
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.