2017-10-26 21:20:01 +00:00
= How to debug Dracut problems
*Foreword*
If you are experiencing a problem with system initialization due to
Dracut, please see the link:Bugs/Common[common bugs] document before
filing a bug. Some easy configuration tweaks that fix a wide range of
issues may be listed there. If the problem you are seeing is not listed
there or none of the workarounds seem to help, please consider filing a
bug to help us make Fedora run better on your hardware.
Be prepared to include some information (logs) about your system as
well. These should be complete (no snippets please), not in an archive,
uncompressed, with MIME type set as text/plain.
[[identifying-your-problem-area]]
2020-10-18 07:39:57 +00:00
== Identifying your problem area
2017-10-26 21:20:01 +00:00
2021-01-09 11:33:12 +00:00
. Remove `rhgb` and `quiet` from the kernel command line
. Add `rd.shell` to the kernel command line. This will present a shell
2020-10-18 07:39:57 +00:00
in case dracut is unable to locate your root device
2021-01-09 11:33:12 +00:00
. Add `rd.shell rd.debug log_buf_len=1M` to the kernel command line so
2020-04-18 21:27:08 +00:00
that dracut shell commands are printed as they are executed
2021-01-09 11:33:12 +00:00
. Inspect the system logs:
2017-10-26 21:20:01 +00:00
....
# less /run/initramfs/rdsosreport.txt
# journalctl -a
# dmesg
# less /run/initramfs/init.log
....
[[information-to-include-in-your-report]]
2020-10-18 07:39:57 +00:00
== Information to include in your report
2017-10-26 21:20:01 +00:00
[[all-bug-reports]]
2020-10-18 07:39:57 +00:00
=== All bug reports
2017-10-26 21:20:01 +00:00
In all cases, the following should be mentioned and attached to your bug
report:
* The exact kernel command-line used. Typically from the bootloader
2020-10-18 07:39:57 +00:00
configuration file (e.g. `/etc/grub.conf`) or from `/proc/cmdline`
* A copy of your disk partition information from `/etc/fstab`
2017-10-26 21:20:01 +00:00
* A device listing from device-mapper. This can be obtained by running
2020-10-18 07:39:57 +00:00
the command `dmsetup ls -- tree`
2017-10-26 21:20:01 +00:00
* A list of block device attributes including vol_id compatible mode.
2020-10-18 07:39:57 +00:00
This can be obtained by running the commands `blkid` and `blkid -o udev`
* Turn on dracut debugging (see link:How_to_debug_Dracut_problems#Debugging[the 'debugging dracut'
2020-04-18 21:27:08 +00:00
section]), and attach all relevant information from the boot log. This
2020-10-18 07:39:57 +00:00
can be obtained by running the command `dmesg|grep dracut`
* If you use a dracut configuration file, please include `/etc/dracut.conf`
2017-10-26 21:20:01 +00:00
[[logical-volume-management-related-problems]]
2020-10-18 07:39:57 +00:00
=== Logical Volume Management related problems
2017-10-26 21:20:01 +00:00
As well as the information from
link:How_to_debug_Dracut_problems#AllInfo[the 'all bug reports'
section], include the following information:
2020-10-18 07:39:57 +00:00
* Include physical volume information by running the command: `lvm pvdisplay`
* Include volume group information by running the command: `lvm vgdisplay`
* Include logical volume information by running the command: `lvm lvdisplay`
2017-10-26 21:20:01 +00:00
[[software-raid-related-problems]]
2020-10-18 07:39:57 +00:00
=== Software RAID related problems
2017-10-26 21:20:01 +00:00
As well as the information from
link:How_to_debug_Dracut_problems#AllInfo[the 'all bug reports'
section], include the following information:
2020-10-18 07:39:57 +00:00
* If using software RAID disk partitions, please include the output of `/proc/mdstat`
2017-10-26 21:20:01 +00:00
[[network-root-device-related-problems]]
Network root device related problems
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section details information to include when experiencing problems
on a system whose root device is located on a network attached volume
(e.g. iSCSI, NFS or NBD). As well as the information from
link:How_to_debug_Dracut_problems#AllInfo[the 'all bug reports'
section], include the following information:
* Please include the output of
[[debugging-dracut]]
2020-10-18 07:39:57 +00:00
== Debugging dracut
2017-10-26 21:20:01 +00:00
[[configure-a-serial-console]]
2020-10-18 07:39:57 +00:00
=== Configure a serial console
2017-10-26 21:20:01 +00:00
Successfully debugging dracut will require some form of console logging
during the system boot. This section documents configuring a serial
console connection to record boot messages. To enable serial console
output for both the kernel and the bootloader, follow the procedure
below.
2021-01-09 11:33:12 +00:00
. Open the file `/etc/grub.conf` for editing. Below the line _timeout=5_, add the following:
+
2017-10-26 21:20:01 +00:00
....
serial --unit=0 --speed=9600
terminal --timeout=5 serial console
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. Also in `/etc/grub.conf`, add the following boot arguments to the _kernel_ line:
+
2017-10-26 21:20:01 +00:00
....
console=tty0 console=ttyS0,9600
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. When finished, `/etc/grub.conf` should look similar to the example below:
2020-10-18 07:39:57 +00:00
2017-10-26 21:20:01 +00:00
....
default=0
timeout=5
serial --unit=0 --speed=9600
terminal --timeout=5 serial console
title Fedora (2.6.29.5-191.fc11.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.29.5-191.fc11.x86_64 ro root=/dev/mapper/vg_uc1-lv_root console=tty0 console=ttyS0,9600
initrd /dracut-2.6.29.5-191.fc11.x86_64.img
....
More detailed information on how to configure the kernel for console
output can be found at
http://www.faqs.org/docs/Linux-HOWTO/Remote-Serial-Console-HOWTO.html#CONFIGURE-KERNEL[1].
[[using-the-dracut-shell]]
2020-10-18 07:39:57 +00:00
=== Using the dracut shell
2017-10-26 21:20:01 +00:00
Dracut offers a shell for interactive debugging in the event dracut
fails to locate your root filesystem. To enable the shell:
2021-01-09 11:33:12 +00:00
. Add the boot parameter `rd.shell` to your bootloader configuration
2020-10-18 07:39:57 +00:00
file (e.g. `/etc/grub/conf`)
2021-01-09 11:33:12 +00:00
. Remove the boot arguments `rhgb` and `quiet`
2017-10-26 21:20:01 +00:00
2020-10-18 07:39:57 +00:00
A sample `/etc/grub.conf` bootloader configuration file is listed below:
2017-10-26 21:20:01 +00:00
....
default=0
timeout=5
serial --unit=0 --speed=9600
terminal --timeout=5 serial console
title Fedora (2.6.29.5-191.fc11.x86_64)
root (hd0,0)
2019-03-20 22:42:41 +00:00
kernel /vmlinuz-2.6.29.5-191.fc11.x86_64 ro root=/dev/mapper/vg_uc1-lv_root console=tty0 rd.shell
2017-10-26 21:20:01 +00:00
initrd /dracut-2.6.29.5-191.fc11.x86_64.img
....
If system boot fails, you will be dropped into a shell as seen in the
2020-10-18 07:39:57 +00:00
example below:
2017-10-26 21:20:01 +00:00
....
No root device found
Dropping to debug shell.
sh: can't access tty; job control turned off
2019-03-20 22:42:41 +00:00
#
2017-10-26 21:20:01 +00:00
....
Use this shell prompt to gather the information requested above (see
link:How_to_debug_Dracut_problems#AllInfo[the 'all bug reports'
section]).
[[accessing-the-root-volume-from-the-dracut-shell]]
2020-10-18 07:39:57 +00:00
=== Accessing the root volume from the dracut shell
2017-10-26 21:20:01 +00:00
From the dracut debug shell, you can manually perform the task of
locating and preparing your root volume for boot. The required steps
will depend on how your root volume is configured. Common scenarios
include:
2020-10-18 07:39:57 +00:00
* A block device (e.g. `/dev/sda7`)
* A LVM logical volume (e.g. `/dev/VolGroup00/LogVol00`)
* An encrypted device (e.g. `/dev/mapper/luks-4d5972ea-901c-4584-bd75-1da802417d83`)
* A network attached device (e.g. `netroot=iscsi:@192.168.0.4::3260::iqn.2009-02.org.fedoraproject:for.all`)
2017-10-26 21:20:01 +00:00
The exact method for locating and preparing will vary. However, to
continue with a successful boot, the objective is to locate your root
2020-10-18 07:39:57 +00:00
volume and create a symlink `/dev/root` which points to the file system. For example, the following example demonstrates accessing and booting a root volume that is an encrypted LVM Logical volume.
2021-01-09 11:33:12 +00:00
. Inspect your partitions using `parted`:
+
2017-10-26 21:20:01 +00:00
....
# parted /dev/sda -s p
Model: ATA HTS541060G9AT00 (scsi)
Disk /dev/sda: 60.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 10.8GB 107MB primary ext4 boot
2 10.8GB 55.6GB 44.7GB logical lvm
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. You recall that your root volume was a LVM logical volume. Scan and
2020-10-18 07:39:57 +00:00
activate any logical volumes:
2021-01-09 11:33:12 +00:00
+
2017-10-26 21:20:01 +00:00
....
# lvm vgscan
# lvm vgchange -ay
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. You should see any logical volumes now using the command `blkid`:
+
2017-10-26 21:20:01 +00:00
....
# blkid
2019-03-20 22:42:41 +00:00
/dev/sda1: UUID="3de247f3-5de4-4a44-afc5-1fe179750cf7" TYPE="ext4"
/dev/sda2: UUID="Ek4dQw-cOtq-5MJu-OGRF-xz5k-O2l8-wdDj0I" TYPE="LVM2_member"
/dev/mapper/linux-root: UUID="def0269e-424b-4752-acf3-1077bf96ad2c" TYPE="crypto_LUKS"
/dev/mapper/linux-home: UUID="c69127c1-f153-4ea2-b58e-4cbfa9257c5e" TYPE="ext3"
/dev/mapper/linux-swap: UUID="47b4d329-975c-4c08-b218-f9c9bf3635f1" TYPE="swap"
2017-10-26 21:20:01 +00:00
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. From the output above, you recall that your root volume exists on an
2020-04-18 21:27:08 +00:00
encrypted block device. Following the guidance disk encryption guidance
from the
http://docs.fedoraproject.org/install-guide/f%7B%7BFedoraVersion%7D%7D/en-US/html/apcs04s04.html[
Installation Guide], you unlock your encrypted root volume.
2021-01-09 11:33:12 +00:00
+
2017-10-26 21:20:01 +00:00
....
UUID=$(cryptsetup luksUUID /dev/mapper/linux-root)
cryptsetup luksOpen /dev/mapper/linux-root luks-$UUID
Enter passphrase for /dev/mapper/linux-root:
2019-03-20 22:42:41 +00:00
Key slot 0 unlocked.
2017-10-26 21:20:01 +00:00
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. Next, make a symbolic link to the unlocked root volume
+
2017-10-26 21:20:01 +00:00
....
ln -s /dev/mapper/luks-$UUID /dev/root
....
2020-10-18 07:39:57 +00:00
2021-01-09 11:33:12 +00:00
. With the root volume available, you may continue booting the system
2020-04-18 21:27:08 +00:00
by exiting the dracut shell
2020-10-18 07:39:57 +00:00
2017-10-26 21:20:01 +00:00
....
exit
....
[[summary-of-dracut-kernel-command-line-options]]
2020-10-18 07:39:57 +00:00
=== Summary of dracut kernel command line options
2017-10-26 21:20:01 +00:00
A selection of the most common debugging related dracut options:
2020-10-18 07:39:57 +00:00
==== `*rd.shell*`
Drop to a shell, if the initramfs fails.
==== `*rd.debug*`
Set -x for the dracut shell.
==== `*rd.break=[cmdline|pre-udev|pre-trigger|initqueue|pre-mount|mount|pre-pivot|cleanup]*`
Drop the shell on defined breakpoint (use
2017-10-26 21:20:01 +00:00
`egrep 'rd.?break' /usr/lib/dracut/modules.d/99base/init.sh` to find the
2020-10-18 07:39:57 +00:00
breakpoints supported by your dracut version)
==== `*rd.udev.info*`
Set udev to loglevel info
==== `*rd.udev.debug*`
Set udev to loglevel debug
2017-10-26 21:20:01 +00:00
See the `dracut.cmdline(7)`
http://man7.org/linux/man-pages/man7/dracut.cmdline.7.html[man page] for
the complete reference.
Category:Debugging[D] Category:How_to[Category:How to]
'''
See a typo, something missing or out of date, or anything else which can be
2017-11-10 15:16:19 +00:00
improved? Edit this document at https://pagure.io/fedora-docs/quick-docs.
