NVMe drives fail to discover

Issue

• During boot, NVMe drives fail to discover.

• The following messages can be seen in /var/log/messages. PCI memory assignment failure associated with the NVMe device:

Dec  8 15:53:51 host kernel: pci 0000:e3:00.0: BAR 0: can't assign mem (size 0x4000)
Dec  8 15:53:51 host kernel: pci 0000:e3:00.0: BAR 0: can't assign mem (size 0x4000)
Dec  8 15:53:51 host kernel: pci 0000:e8:00.0: BAR 0: can't assign mem (size 0x4000)

• Backtrace with nvme_dev_start failure:

Dec  8 15:53:51 host kernel: ------------[ cut here ]------------
Dec  8 15:53:51 host kernel: WARNING: at arch/x86/mm/ioremap.c:110 __ioremap_caller+0x35f/0x390() (Not tainted)
Dec  8 15:53:51 host kernel: Hardware name: UCSC-C480-M5
Dec  8 15:53:51 host kernel: Modules linked in: fnic(+) libfcoe libfc scsi_transport_fc scsi_tgt usb_storage sd_mod crc_t10dif sr_mod cdrom megaraid_sas(U) nvme xhci_hcd ahci wmi dm_mirror dm_region_hash dm_log dm_mod
Dec  8 15:53:51 host kernel: Pid: 843, comm: events/72 Not tainted 2.6.32-642.1.1.el6.x86_64 #1
Dec  8 15:53:51 host kernel: Call Trace:
Dec  8 15:53:51 host kernel: [<ffffffff8107c911>] ? warn_slowpath_common+0x91/0xe0
Dec  8 15:53:51 host kernel: [<ffffffffa009138c>] ? nvme_dev_start+0xec/0x6f0 [nvme]
Dec  8 15:53:51 host kernel: [<ffffffff8107c97a>] ? warn_slowpath_null+0x1a/0x20
Dec  8 15:53:51 host kernel: [<ffffffff8105286f>] ? __ioremap_caller+0x35f/0x390
Dec  8 15:53:51 host kernel: [<ffffffff8145bb1a>] ? pcibios_set_master+0x7a/0xa0
Dec  8 15:53:51 host kernel: [<ffffffff810529b7>] ? ioremap_nocache+0x17/0x20
Dec  8 15:53:51 host kernel: [<ffffffffa009138c>] ? nvme_dev_start+0xec/0x6f0 [nvme]
Dec  8 15:53:51 host kernel: [<ffffffff810a0711>] ? __queue_work+0x41/0x50
Dec  8 15:53:51 host kernel: [<ffffffffa0091b70>] ? nvme_async_probe+0x0/0x550 [nvme]
Dec  8 15:53:51 host kernel: [<ffffffffa0091ba3>] ? nvme_async_probe+0x33/0x550 [nvme]
Dec  8 15:53:51 host kernel: [<ffffffffa0091b70>] ? nvme_async_probe+0x0/0x550 [nvme]
Dec  8 15:53:51 host kernel: [<ffffffff8109fdc0>] ? worker_thread+0x170/0x2a0
Dec  8 15:53:51 host kernel: [<ffffffff810a6ac0>] ? autoremove_wake_function+0x0/0x40
Dec  8 15:53:51 host kernel: [<ffffffff8109fc50>] ? worker_thread+0x0/0x2a0
Dec  8 15:53:51 host kernel: [<ffffffff810a662e>] ? kthread+0x9e/0xc0
Dec  8 15:53:51 host kernel: [<ffffffff8100c28a>] ? child_rip+0xa/0x20
Dec  8 15:53:51 host kernel: [<ffffffff810a6590>] ? kthread+0x0/0xc0
Dec  8 15:53:51 host kernel: [<ffffffff8100c280>] ? child_rip+0x0/0x20
Dec  8 15:53:51 host kernel: ---[ end trace a207f15839f38433 ]---

• Device shown as not ready:

Dec  8 15:53:51 host kernel: nvme 0000:e3:00.0: Device not ready; aborting reset
Dec  8 15:53:51 host kernel: nvme 0000:e3:00.0: Device not ready; aborting reset
Dec  8 15:53:51 host kernel: nvme 0000:e8:00.0: Device not ready; aborting reset
Dec  8 15:55:38 host kernel: nvme 0000:e8:00.0: Device not ready; aborting reset