System got rebooted while running Oracle dataguard process

Solution Unverified - Updated -

Issue

  • While transferring the asm dataguard from one server to other, the server got rebooted with following call traces:

    Request for unknown module key 'Oracle Linux RHCK Module Signing Key: dd995b155c19b3a7c3ef7707b969e25f9639666e' err -11
    ACFSK-0037: Module load succeeded. Build information:   (LOW DEBUG) USM_19.10.0.0.0ACFSRU_LINUX.X64_201219 2020/12/22 08:07:58
    Request for unknown module key 'Oracle Linux RHCK Module Signing Key: dd995b155c19b3a7c3ef7707b969e25f9639666e' err -11
    AFDK-00001: Module load succeeded. Build information:  (LOW DEBUG) - USM_19.10.0.0.0ACFSRU_LINUX.X64_201219 built on 2020/12/22 09:39:43.
    nr_pdflush_threads exported in /proc is scheduled for removal
    warning: `turbostat' uses 32-bit capabilities (legacy support in use)
    ------------[ cut here ]------------
    kernel BUG at drivers/scsi/virtio_scsi.c:521!
    invalid opcode: 0000 [#1] SMP 
    Modules linked in: oracleafd(POE) oracleacfs(POE) bmhook(OE) tmhook(OE) oracleadvm(POE) oracleoks(POE) dsa_filter(POE) dsa_filter_hook(OE) iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_rng virtio_balloon i2c_piix4 cbsensor(OE) auth_rpcgss binfmt_misc sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_console failover virtio_scsi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
    CPU: 3 PID: 9817 Comm: oracle_9817_+as Kdump: loaded Tainted: P           OE  ------------   3.10.0-1160.36.2.el7.x86_64 #1
    Hardware name: Red Hat RHEL/RHEL-AV, BIOS 1.14.0-1.module+el8.4.0+8855+a9e237a9 04/01/2014
    task: ffff9e53f3c2d280 ti: ffff9e61a4a94000 task.ti: ffff9e61a4a94000
    RIP: 0010:[<ffffffffc02f6222>]  [<ffffffffc02f6222>] virtscsi_queuecommand+0x1f2/0x230 [virtio_scsi]
    RSP: 0018:ffff9e61a4a976f0  EFLAGS: 00010283
    RAX: 000000000000007e RBX: ffff9e61a494c540 RCX: ffff9e633bb8e760
    RDX: ffff9e61a494c540 RSI: ffff9e633b7da948 RDI: ffff9e633b7da740
    RBP: ffff9e61a4a97708 R08: 0000000000000020 R09: 0000000000000004
    R10: ffff9e5376599800 R11: ffff9e61a494c540 R12: ffff9e61a494c540
    R13: ffff9e633b7da000 R14: ffff9e63394e0848 R15: ffff9e61a494c540
    FS:  00007f9093bfd400(0000) GS:ffff9e633f2c0000(0000) knlGS:0000000000000000
    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: 00007f908cd5c000 CR3: 0000000e600fe000 CR4: 00000000001606e0
    Call Trace:
     [<ffffffffc02f6297>] virtscsi_queuecommand_single+0x37/0x40 [virtio_scsi]
     [<ffffffffb0ae4810>] scsi_dispatch_cmd+0xb0/0x240
     [<ffffffffb0aedf3c>] scsi_request_fn+0x4ac/0x680
     [<ffffffffb0978b9e>] ? deadline_add_request+0x6e/0x90
     [<ffffffffb0953ee9>] __blk_run_queue+0x39/0x50
     [<ffffffffb0957c93>] blk_queue_bio+0x3b3/0x400
     [<ffffffffc060efd1>] afd_mkrequest_fn+0x261/0x810 [oracleafd]
     [<ffffffffb0953d9f>] ? generic_make_request_checks+0x27f/0x390
     [<ffffffffb0955b77>] generic_make_request+0x147/0x380
     [<ffffffffb0955e20>] submit_bio+0x70/0x150
     [<ffffffffc0611ef9>] safd_bio_submit+0x109/0x190 [oracleafd]
     [<ffffffffc0607216>] afdq_batch_submit+0x1166/0x1450 [oracleafd]
     [<ffffffffb06d7402>] ? check_preempt_curr+0x92/0xa0
     [<ffffffffc05f40d4>] ? afduhLookup+0x84/0x120 [oracleafd]
     [<ffffffffc05f80f3>] afdc_io+0xb13/0xf70 [oracleafd]
     [<ffffffffb07e1038>] ? zone_statistics+0x88/0xa0
     [<ffffffffc05f86ab>] afdc_dispatch_ioctl+0x15b/0x18d0 [oracleafd]
     [<ffffffffc05fa18c>] afdc_execute_ioctl+0x36c/0x470 [oracleafd]
     [<ffffffffb07e9807>] ? anon_vma_interval_tree_insert+0x97/0xa0
     [<ffffffffb083ece2>] ? __mem_cgroup_commit_charge+0xe2/0x2f0
     [<ffffffffb07ce09e>] ? lru_cache_add+0xe/0x10
     [<ffffffffc060e9cc>] afd_ioctl+0x18c/0x350 [oracleafd]
     [<ffffffffb09685ea>] blkdev_ioctl+0x28a/0xa20
     [<ffffffffb088f131>] block_ioctl+0x41/0x50
     [<ffffffffb0863a80>] do_vfs_ioctl+0x3a0/0x5b0
     [<ffffffffb0863d31>] SyS_ioctl+0xa1/0xc0
     [<ffffffffb0d95f92>] system_call_fastpath+0x25/0x2a
    Code: 00 c6 00 00 48 8d 78 01 b2 bf e9 a3 fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 31 f6 48 83 c7 02 83 ea 02 66 89 77 fe e9 91 fe ff ff <0f> 0b 48 8b 82 d0 00 00 00 48 8b 0a 48 c7 c7 00 84 2f c0 44 0f 
     [ 1992.311757] RIP  [<ffffffffc02f6222>] virtscsi_queuecommand+0x1f2/0x230 [virtio_scsi]
     [ 1992.312912]  RSP <ffff9e61a4a976f0>
    

Environment

  • Red Hat Enterprise Linux 7.9
  • Virtual machine configured in RHV
  • VirtIO SCSI disks
  • Oracle ASM Filter Driver

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content