How to Manually Initiate a Vmcore Dump from a Power System on LPAR

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 8
  • POWER9

Issue

How can I manually initiate a vmcore dump for a logical partition (LPAR) on my Power system?

Resolution

Manual vmcore dumps for logical partitions (LPARs) can be initiated from the LPAR restart partition restart options in the Hardware Management Console (HMC).

The following screenshot shows how a manual dump was initiated for LPAR 7 from the HMC GUI tool:

LPAR restart options for partitions
Choose the 'dump' restart option to manual initiate a vmcore dump.

This method dumps the vmcore and restarts the logical partition.

Note: The above screenshot was taken from a POWER9 system. Should you need further assistance finding the above menu to initiate the dump, we recommend reaching out to your hardware vendor for further assistance.

Diagnostic Steps

  1. kdump Deployment

    • Ensure the kexec-tools package that provides kdump is installed and that the kdump service is active:

      # rpm -qa | grep kexec
kexec-tools-2.0.20-34.el8_3.1.ppc64le

    • For Red Hat Enterprise Linux 7 higher:

      # systemctl status kdump
● kdump.service - Crash recovery kernel arming
   Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled)
   Active: active (exited) since Fri 2021-01-01 02:21:00 EST; 2s ago
[...]

    • For additional information and guidance on kdump deployment, please see the following article:

  2. Initiate the Dump

    • From the HMC GUI tool, initiate the dump from the restart options of the restart partition menu.

  3. Setting Up the crash Environment

  4. vmcore Observations

    • Note that in crash the process context is set to the idle swapper thread of CPU 0:

      crash> bt
PID: 0      TASK: c0000000015f2b80  CPU: 0   COMMAND: "swapper/0"
 #0 [c00000000166fd50] plpar_hcall_norets at c0000000000ebcfc
 #1 [c00000000166fdc0] cpuidle_enter_state at c000000000a75bf4
 #2 [c00000000166fe20] do_idle at c0000000001992c0
 #3 [c00000000166fea0] cpu_startup_entry at c000000000199588
 #4 [c00000000166fed0] rest_init at c0000000000103d8
 #5 [c00000000166ff00] start_kernel at c0000000010b4228
 #6 [c00000000166ff90] start_here_common at c00000000000ad7c

    • Additionally there is no panic back trace seen in the kernel ring buffer:

      crash> log | tail
[   46.113455] ibmvnic 30000003: Partner initialization complete
[   46.113507] ibmvnic 30000003: Partner protocol version is 256
[   47.637864] ibmvnic 30000002: Backing device failover detected
[   47.638013] ibmvnic 30000002: Partner initialized
[   47.716198] ibmvnic 30000002: Partner initialization complete
[   47.716233] ibmvnic 30000002: Partner protocol version is 256
[   48.889598] (v 108868) GUARD-02: 179824 ktap_stap_ioctl: ktap: ktap daemon starting (179824), flags 0x2 (line 6370)
[  878.033460] Sending IPI to other CPUs
[  878.044016] IPI complete
[  878.083846] kexec: Starting switchover sequence.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments