What does the message "NMI: IOCK error" mean?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 5

Issue

  • The following message is seen on the console

        NMI: IOCK error (debug interrupt?)
        CPU 0 
        Modules linked in: ipt_MASQUERADE iptable_nat ip_nat xt_state ip_conntrack nfnetlink ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge mptctl mptbase bonding be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic ipv6 xfrm_nalgo crypto_api uio cxgb3i cxgb3 8021q libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi dm_round_robin dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec i2c_core dell_wmi wmi button battery asus_acpi acpi_memhotplug ac parport_pc lp parport joydev sr_mod cdrom hpilo bnx2 serio_raw shpchp pcspkr sg dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod usb_storage qla2xxx scsi_transport_fc ata_piix libata cciss sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
        Pid: 0, comm: swapper Not tainted 2.6.18-194.17.4.el5 #1
        RIP: 0010:[<ffffffff8019d550>]  [<ffffffff8019d550>] acpi_processor_idle_simple+0x14c/0x30e
        RSP: 0018:ffffffff803fbf58  EFLAGS: 00000046
        RAX: 0000000000d4d87e RBX: ffff81061e10a160 RCX: 0000000000000908
        RDX: 0000000000000915 RSI: 0000000000000003 RDI: 0000000000000000
        RBP: 0000000000d4d87e R08: ffffffff803fa000 R09: 0000000000000039
        R10: ffff810001005710 R11: 0000000000000000 R12: 0000000000000000
        R13: ffff81061e10a000 R14: 0000000000000000 R15: 0000000000000000
        FS:  0000000000000000(0000) GS:ffffffff803ca000(0000) knlGS:0000000000000000
        CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
        CR2: 0000000009013954 CR3: 000000060799d000 CR4: 00000000000006e0
        Process swapper (pid: 0, threadinfo ffffffff803fa000, task ffffffff80308b60)
        Stack:  ffff81061e10a000 ffffffff8019d404 0000000000000000 ffffffff8019d404
         0000000000090000 0000000000000000 0000000000000000 ffffffff8004923a
         0000000000200800 ffffffff80405807 0000000000090000 0000000000000000
        Call Trace:
         [<ffffffff8019d404>] acpi_processor_idle_simple+0x0/0x30e
         [<ffffffff8019d404>] acpi_processor_idle_simple+0x0/0x30e
         [<ffffffff8004923a>] cpu_idle+0x95/0xb8
         [<ffffffff80405807>] start_kernel+0x220/0x225
         [<ffffffff8040522f>] _sinittext+0x22f/0x236

        Code: 89 ca ed ed 41 89 c4 41 8a 45 1c 83 e0 30 3c 30 75 15 f0 ff

Resolution

  • Use vendor hardware diagnostics software to analyse system health.

  • Contact the hardware manufacturer for further assistance.

  • Under RHEL6, the kernel.panic_on_io_nmi = 1 sysctl can be set to have the system panic when an I/O NMI is received. Refer to How to set sysctl variables on Red Hat Enterprise Linux? for details.

Root Cause

  • Under x86_64, I/O port 0x61 (System Control Port B) holds the reason of an NMI (NonMaskable Interrupt).

  • For a detailed description of NMIs, refer e.g. to the Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3A: System Programming Guide, Part 1.

  • An NMI is a hardware-driven interrupt and it is usually reserved to report serious hardware errors, to a CPU.

File: include/asm-x86_64/nmi.h

    #define get_nmi_reason() inb(0x61)

  • In the event of an NMI it is possible to consult the aforementioned port to obtain an indication of what caused the error.

  • In the default_do_nmi() function we see why io_check_error() is called and consequently why the NMI: IOCK error (debug interrupt?) message is displayed on the console:

File: arch/x86_64/kernel/traps.c

    asmlinkage __kprobes void default_do_nmi(struct pt_regs *regs)
    {
        unsigned char reason = 0; 
        int cpu; 

        cpu = smp_processor_id();
        ...

        if (!cpu)
            reason = get_nmi_reason();

        ...
        if (reason & 0x40)
            io_check_error(reason, regs);
    }

    static __kprobes void
    io_check_error(unsigned char reason, struct pt_regs * regs)
    {
        printk("NMI: IOCK error (debug interrupt?)\n");
        show_registers(regs);
        ...
  • As illustrated above, when the 6-th bit (1 << 6 == 0x40) is set, it reports that an I/O Channel Check error (IOCHK) has occurred.
    • Parity and uncorrectable hardware errors are examples of why an IOCHK error could be raised.
    • Most hardware errors should however be reported through the MCE (Machine Check Exception) mechanism. An MCE indicates that the CPU detected an internal machine error or a bus error, or that an external agent detected a bus error. Normally the hardware manufacturer will be able to provide further details.

Diagnostic Steps

  • Check dmesg and lspci output to figure out the cause of the NMI

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments