Occasionally it is useful to debug a running guest VM’s kernel. Setup your host machine for virtual machine hosting with QEMU/KVM, and add ddebs to your host system using this wiki.
Next, compile your own qemu with --enable-debug. Setup your environment to
use this binary (if using libvirt), or call it directly using qemu.
Ensure you have vmlinux of the L1 guest somewhere in your L0 machine.
First, setup your L1 guest. If running with qemu add the -s flag. With
libvirt you can add the following entry to your machine description:
# Modify this:
<domain type='kvm'>
# To look like this:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<qemu:commandline>
<qemu:arg value='-s'/>
</qemu:commandline>
Start the guest and find the start address of the text section:
echo 0x$(sudo cat /proc/kallsyms | egrep -e "T _text$" | awk '{print $1}')
Next attach a gdb session on the running guest. Add the symbol file for vmlinux
using the proper start address. Then add a breakpoint where you want to detect
the failure. In this example we’ll use sysrq_handle_crash which can be
triggered easily with /proc/sysrq-trigger.
gdb /usr/bin/qemu-system-x86_64
(gdb) target remote localhost:1234
(gdb) add-symbol-file vmlinux 0xffffffff81000000
(gdb) b sysrq_handle_crash
(gdb) c
In the VM run the following:
echo 'c' | sudo tee /proc/sysrq-trigger
Now you’ll notice gdb has stopped at the breakpoint:
(gdb) c
Continuing.
[New Thread 2]
[Switching to Thread 2]
Breakpoint 1, sysrq_handle_crash (key=99) at drivers/tty/sysrq.c:136
136 drivers/tty/sysrq.c: No such file or directory.