System I/O hang with 'seos' module loaded

Solution Unverified - Updated -

Issue

  • System I/O hang with seos module loaded

  • I/O hung and processes blocked. Oldest 5 processes:

crash> ps -m | grep UN | tail -n5
[  0 02:21:00.721] [UN]  PID: 19181  TASK: ffff881539e76040  CPU: 9   COMMAND: "oracle"
[  0 02:21:00.765] [UN]  PID: 29194  TASK: ffff8805882bd540  CPU: 16  COMMAND: "ora_lgwr_sisid0"
[  0 02:21:00.764] [UN]  PID: 19175  TASK: ffff885010ea6ae0  CPU: 1   COMMAND: "oracle"
[  0 02:21:00.863] [UN]  PID: 12922  TASK: ffff881284ea8080  CPU: 5   COMMAND: "ora_j000_sisid0"
[  0 02:21:00.870] [UN]  PID: 28918  TASK: ffff8801c253aae0  CPU: 11  COMMAND: "ora_dbw2_sisid0"
  • Processes hung trying to get I/O flushed to disk:
crash> bt ffff8801c253aae0
PID: 28918  TASK: ffff8801c253aae0  CPU: 11  COMMAND: "ora_dbw2_sisid0"
 #0 [ffff8802941bba18] schedule at ffffffff81529970
 #1 [ffff8802941bbaf0] schedule_timeout at ffffffff8152a8c5
 #2 [ffff8802941bbba0] wait_for_common at ffffffff8152a543
 #3 [ffff8802941bbc30] wait_for_completion at ffffffff8152a65d
 #4 [ffff8802941bbc40] __blkdev_issue_flush at ffffffff8127531d
 #5 [ffff8802941bbca0] blkdev_issue_flush at ffffffff81275366
 #6 [ffff8802941bbcb0] ext4_sync_file at ffffffffa02a70a5 [ext4]
 #7 [ffff8802941bbd00] vfs_fsync_range at ffffffff811c07f1
 #8 [ffff8802941bbd50] generic_write_sync at ffffffff811c089b
 #9 [ffff8802941bbd60] generic_file_aio_write at ffffffff811269e6
#10 [ffff8802941bbdb0] ext4_file_write at ffffffffa02a6e08 [ext4]
#11 [ffff8802941bbdf0] aio_rw_vect_retry at ffffffff811daf44
#12 [ffff8802941bbe20] aio_run_iocb at ffffffff811dc904
#13 [ffff8802941bbe50] do_io_submit at ffffffff811ddd31
#14 [ffff8802941bbf70] sys_io_submit at ffffffff811de3d0
#15 [ffff8802941bbf80] system_call_fastpath at ffffffff8100b072
    RIP: 0000003fc8200667  RSP: 00007ffffd8e2678  RFLAGS: 00000206
    RAX: 00000000000000d1  RBX: ffffffff8100b072  RCX: 0000800000010000
    RDX: 00007ffffd8db2e0  RSI: 0000000000000001  RDI: 00007f415a719000
    RBP: 00007ffffd8dd310   R8: 00007ffffd8ddbb8   R9: 00000002ca1b82d0
    R10: 0000000000000001  R11: 0000000000000246  R12: ffffffff811de3d0
    R13: ffff8802941bbf78  R14: 0000000000000001  R15: 00007ffffd8ddbb8
    ORIG_RAX: 00000000000000d1  CS: 0033  SS: 002b

Environment

  • Red Hat Enterprise Linux 6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.