
Boot Process
Boot Process Related Development Notes
GRUB[edit]
grub-install command responsibility[edit]
Who should run the grub-install
command? SystemBuildTools or Debian package maintainer scripts?
As it is currently designed, it seems SystemBuildTools are supposed to execute the
grub-install
command.
calamares
installer runs grub-install
. live-build
has extensive code to set up GRUB and other bootloaders. mkosi
uses grub-mkimage
.
It's the system build tool that is responsible for setting up the bootloader, which requires running bootloader installation commands.
Don't call grub-install on fresh install of grub-pc. It's the job of installers to do that after a fresh install.
grub2
package, Debian changelog, Colin Watson Nov 2020
Core Bootloader Packages[edit]
Kicksecure uses different metapackages to provide the bootloader for different systems. grub-cloud is used on Kicksecure VMs, while grub-efi and grub-pc-bin are used by the ISO.
grub-cloud package[edit]
You don't want to use this package outside of cloud images.
grub-cloud-amd64
package, Debian
grub-cloud-amd64
package and /etc/default/grub
file inclusion:
/etc/default/grub
List of files
Non-issue: grub-cloud
, while it has "cloud" in its name, and while it may be suitable for installation on cloud servers, has no additional networking or cloud features not found in "standard" GRUB packages. grub-cloud
does not "interact with the cloud". It does not boot from the cloud or have other problematic cloud features. Such features are not planned either. Its source code is minimal and consists only of Debian packaging files and a /etc/default/grub
configuration file. The grub-cloud
package is a workaround for the lack of grub-pc
and grub-efi
co-installability, a workaround for Debian bug grub-efi-amd64: Allow concurrent installation of grub-pc and grub-efi-amd64.
Source code references:
- grub-cloud source code
- AMD64
/etc/default/grub
- ARM64
/etc/default/grub
- AMD64
postinst
- ARM64
postinst
AMD64 /etc/default/grub
contents:
# If you change this file, run 'update-grub' afterwards to update # /boot/grub/grub.cfg. # For full documentation of the options in this file, see: # info -f grub -n 'Simple configuration' GRUB_DEFAULT=0 GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="" GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,115200 earlyprintk=ttyS0,115200 consoleblank=0" GRUB_TERMINAL_OUTPUT="gfxterm serial" GRUB_SERIAL_COMMAND="serial --speed=115200"
- Potential issues with
grub-cloud
managing/etc/default/grub
:- Running
debsums --changed --config
would list/etc/default/grub
as a changed configuration file. - Setting
GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,115200 earlyprintk=ttyS0,115200 consoleblank=0"
can cause issues:- Security concerns?
- Systemd log spam inside VirtualBox:
- Running
serial-getty@ttyS0.service: Succeeded. serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart. serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 625. Stopped Serial Getty on ttyS0. Started Serial Getty on ttyS0. /dev/ttyS0: not a tty serial-getty@ttyS0.service: Succeeded. serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart. serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 626. Stopped Serial Getty on ttyS0. Started Serial Getty on ttyS0. /dev/ttyS0: not a tty
- VirtualBox: Adding a virtual disconnected serial console does not help either. This causes:
- GRUB boot menu becoming invisible.
- No console output for a long time.
- Extremely slow boot times.
The serial console-related issues were encountered ~5 years ago when considering "why not enable a serial console by default inside VM images."
- Possible solution: If using a
grub-cloud
-based solution, it may be better to undo the serial console setup. - Architectural limitations:
grub-cloud
currently supports only a limited set of architectures (Intel/AMD64 and ARM64 at the time of writing). Depending on your plans for multi-architecture support (as Debian is the universal operating system), this may be a limitation.
Related Debian issues:
- bug report:
grub-pc
andgrub-efi
co-installability: grub-efi-amd64: Allow concurrent installation of grub-pc and grub-efi-amd64 - bug report: grub-cloud-amd64: Ships /etc/default/grub, which installers need to be able to modify
- bug report: grub-cloud-amd64: not co-installable with grub-pc due to incompatible /etc/default/grub handling
Related Debian pull requests:
grub-efi and grub-pc[edit]
- Debian for grub-pc with grub-efi co-install-ability feature request: Allow concurrent installation of grub-pc and grub-efi-amd64
[edit]
The following packages directly affect the bootloader or bootloader configuration used by Kicksecure.
live-config-dist[edit]
- Purpose: Used to provide installer and live ISO configuration.
- Effects on bootloader:
- Sets the distro name and version shown on the boot menu of the live ISO.
- Ensures a menu entry for accessing UEFI firmware settings is added to the live ISO.
- Ensures the GRUB fallback bootloader is properly installed.
- Assists with initial bootloader installation on machines installed from the ISO.
dist-base-files[edit]
- Provides base configuration used by both Kicksecure and Whonix.
- Effects on bootloader:
- Provides customized versions of the grub-mkconfig scripts in order to reorganize the bootloader menu so that normal boot modes appear at the top, and "Advanced options" boot modes appear at the bottom.
- Provides common files for the Kicksecure and Whonix GRUB themes.
- For VM images (not ISO-installed systems), overrides non-ideal GRUB bootloader settings from grub-cloud, putting the kernel in quiet mode and disabling the serial console.
grub-live[edit]
- Provides a live boot mode. Changes made to the root filesystem in this mode are ephemeral and will be lost on reboot.
- Effects on bootloader:
- Adds entries to the boot menu for booting in live mode.
- Adds additional debugging info to the output of grub-mkconfig.
serial-console-enable[edit]
- Adds a TTY that can be accessed via the serial console.
- Effects on bootloader:
- Enables GRUB bootloader serial console output.
- Adds kernel parameters to the Linux kernel command line to enable a TTY on the serial console.
security-misc[edit]
- Enables a plethora of hardening features to increase the security of Kicksecure and Whonix.
- Effects on bootloader:
- Enables strong CPU vulnerability mitigations via the kernel command line.
- Enables several general kernel hardening features via the kernel command line.
- Puts the kernel into quiet logging mode via kernel parameters to avoid leaking sensitive info on the console during boot.
- Disables Dracut-based recovery features via kernel parameters to make it more difficult to get a root shell improperly.
usability-misc[edit]
- Provides miscellaneous usability improvements for Kicksecure.
- Effects on bootloader: Sets the default display resolution during early boot to 1024x768. (Note that this is NOT a hard limit; the end-user can set their resolution to whatever they want once the system is booted.)
debug-misc[edit]
- Enables a wide variety of debugging features. Not installed by default and should NOT be installed on systems where security is a concern.
- Effects on bootloader:
- Removes kernel parameters that would otherwise disable message printing on the console during early boot.
- Enables verbose debugging output in initramfs-tools, dracut, systemd, and the Linux kernel via kernel parameters.
- Disables SELinux enforcement via a kernel parameter. Kicksecure itself doesn't use SELinux by default, but debug-misc may be used on some other distro or a user might enable SELinux later, which could interfere with debugging.
kicksecure-base-files[edit]
- Provides base configuration specific to Kicksecure.
- Effects on bootloader:
- Sets the distro name shown on the boot menu of installed systems.
- Provides the Kicksecure-specific components of the GRUB theme.
- Sets the GRUB theme in GRUB itself.
- Sets the screen resolution for the GRUB menu to 1280x720 on EFI systems, and 1024x768 on BIOS systems.
user-sysmaint-split[edit]
- Prevents standard user accounts from using privilege escalation tools to obtain root and provides a special sysmaint boot mode in which root access can be obtained.
- Effects on bootloader:
- Adds a boot entry for booting into sysmaint mode.
- Adds a boot entry for uninstalling user-sysmaint-split quickly and with minimal effort.
whonix-base-files[edit]
- Whonix-only. Provides base configuration specific to Whonix.
- Effects on bootloader: Sets the distro name shown on the boot menu of installed systems to a generic "Whonix" value. This is usually overridden by one of anon-ws-base-files or anon-gw-base-files.
anon-ws-base-files[edit]
- Whonix-only. Provides base configuration specific to Whonix-Workstation.
- Effects on bootloader:
- Sets the distro name shown on the boot menu of installed systems.
- Provides the Whonix-Workstation-specific components of the GRUB theme.
- Sets the GRUB theme in GRUB itself.
- Sets the screen resolution for the GRUB menu to 1280x720 on EFI systems, and 1024x768 on BIOS systems.
anon-gw-base-files[edit]
- Whonix-only. Provides base configuration specific to Whonix-Gateway.
- Effects on bootloader:
- Sets the distro name shown on the boot menu of installed systems.
- Provides the Whonix-Gateway-specific components of the GRUB theme.
- Sets the GRUB theme in GRUB itself.
- Sets the screen resolution for the GRUB menu to 1280x720 on EFI systems, and 1024x768 on BIOS systems.
Live ISO GRUB configuration[edit]
derivative-maker sets a custom GRUB configuration for Kicksecure live ISOs. This configuration is stored under derivative-maker/live-build-data/grub-config. The files in this directory are enumerated below, along with the job each one performs.
- config.cfg
- Provides base GRUB config setup. Loads fonts, video drivers, and the theme for GRUB, among other things.
- grub.cfg
- Template configuration file into which live-build inserts boot menu information. Provides menu entries for live boot, debian-installer (if applicable - currently this is not applicable to Kicksecure's ISOs), and launchers for utilities like memtest, firmware setup, and boot media checksumming.
- install_gui.cfg
- Only applicable when debian-installer is enabled (currently it is not for Kicksecure). Provides boot modes that launch either the GUI or text-mode Debian installer when debian-installer is enabled and GUI mode is selected.
- install_start_gui.cfg
- Vestigial, copied from the base live GRUB config in live-build. Unused by Kicksecure even if debian-installer is enabled.
- install_start_text.cfg
- Vestigial, copied from the base live GRUB config in live-build. Unused by Kicksecure even if debian-installer is enabled.
- install_text.cfg
- Only applicable when debian-installer is enabled (currently it is not for Kicksecure). Provides boot modes that launch the text-mode Debian installer when debian-installer is enabled and GUI mode is disabled.
- memtest.cfg
- Provides boot modes for launching Memtest86+.
- splash.svg
- Provides the background image for the GRUB splash screen used on the live ISO.
- theme.cfg
- Loads the GRUB theme from live-theme/theme.txt. Also provides a fallback default theme if this fails for some reason.
- live-theme/theme.txt
- Provides dynamic parts of the GRUB theme. Specifies the colors and positions of UI elements, and includes a progress bar indicating how much time the user has to react before GRUB automatically boots the first boot mode listed in the ISO's boot menu.
GRUB Upstream[edit]
GRUB Slow Upstream[edit]
We all know and love GRUB2. It is a good boot loader. It is also big, complex, rich, massive and tends to move slow on the development side.openSUSE blog post Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS
talking about their motivation to add support for systemd-boot
The openSUSE package for this boot loader contains more than 200 patches. Some of those patches are there for the last 5, 6 … 10 years. That is both an indication of the talent of the maintainers, but also can signal an issue in how slow the upstream contribution process can be.openSUSE blog post Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS
talking about their motivation to add support for systemd-boot
GRUB Feature Richness[edit]
GRUB2 supports all the relevant systems, including mainframes, arm or powerpc. Multiple types of file systems, including btrfs or NTFS. It contains a full network stack, an USB stack, a terminal, can be scripted … In some sense, it is almost a mini OS by itself.openSUSE blog post Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS
talking about their motivation to add support for systemd-boot
GRUB Full Disk Encryption[edit]
Kicksecure doesn’t use GRUB to unlock encrypted disks. This is because we use Debian’s GRUB, and Debian’s GRUB only has very bad LUKS support (only supports LUKS1, can’t handle non-US keyboard layouts, ugly, slow, only gives you one shot to unlock the drive, and then the Linux kernel has to unlock the drive again once it boots). Instead, we use an unencrypted /boot partition and let the initramfs handle decrypt. This lets us use more secure encryption, provides a better user interface for decryption, works with multiple keyboard layouts, and works faster.https://forums.kicksecure.com/t/installing-fde-luks-with-detached-luks-header-option/907/2
See also:
Calamares[edit]
Multiple Bootloader Maintenance Burden[edit]
Supporting another boot loader comes with a cost.openSUSE blog post Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS
talking about their motivation to add support for systemd-boot
systemd-boot[edit]
systemd-boot - Limited Architecture Support[edit]
At time of writing, systemd-boot as can be soon on https://packages.debian.org/testing/systemd-boot supported only the following architectures:
amd64
arm64
armhf
i386
riscv64
systemd-boot - random seed[edit]
- https://uapi-group.org/specifications/specs/boot_loader_specification/
- https://systemd.io/BOOT_LOADER_INTERFACE/
LoaderSystemToken
systemd-boot - SecureBoot Support[edit]
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033725
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996202
- TODO: What is the latest status? Does systemd-boot in Debian support SecureBoot yet?
RPi[edit]
misc[edit]
- https://wiki.archlinux.org/title/Talk:GRUB#Custom_keyboard_layout
- https://cryptsetup-team.pages.debian.net/cryptsetup/encrypted-boot.html
keyboard layout issue[edit]
- https://github.com/calamares/calamares/issues/1772
- https://github.com/calamares/calamares/issues/1726
- https://github.com/calamares/calamares/issues/1203
- https://superuser.com/questions/974833/change-the-keyboard-layout-of-grub-in-stage-1
- https://cryptsetup-team.pages.debian.net/cryptsetup/encrypted-boot.html
Kicksecure Specific[edit]
GRUB - File Names[edit]
-
Click = Copy
Copied to clipboard!
-
Click = Copy
Copied to clipboard!
has been forked by
dist-base-files
-
Click = Copy
Copied to clipboard!
has been forked by
- Click = Copy Copied to clipboard!
/etc/default/grub.d/20_dist-base-files.cfg[edit]
File Click = Copy Copied to clipboard! is used to undo the opinionated default configuration set by the Debian package grub-cloud package.
Why is the folder /usr/share/derivative-base-files
used? Why is the file copied using derivative-maker during the build process? Why not simply ship the file as /etc/default/grub.d/20_dist-base-files.cfg
as part of package dist-base-files
? Because it is not applicable to all image creation and installation methods. This should only be done when building a VM image that uses grub-cloud (which is utilized by grml-debootstrap
).
VM images:
File /usr/share/derivative-base-files/20_dist-base-files.cfg
is copied by derivative-maker during the build process to
/etc/default/grub.d/20_dist-base-files.cfg
.
Kicksecure's ISO:
The ISO does not need /etc/default/grub.d/20_dist-base-files.cfg
because it does not use grub-cloud
. (The ISO is build using live-build
, not grml-debootstrap
.)
Calamares:
The installer used by Kicksecure's ISO, Calamares, edits the file /etc/default/grub
by adding rd.luks.uuid
to GRUB_CMDLINE_LINUX_DEFAULT
. For example:
GRUB_CMDLINE_LINUX_DEFAULT='quiet rd.luks.uuid=dc1f531b-eea8-47b0-86f2-a841d6d61a4e'
If the file /etc/default/grub.d/20_dist-base-files.cfg
were shipped unconditionally, it might break the boot process.
grub config file - calamares - grub unlocks full disk encrypted hard drive[edit]
dracut bug log[edit]
Debian bug report: unbootable system after installing dracut on a standard Debian installation
sudo dracut -f
dracut: Executing: /usr/bin/dracut -f dracut: dracut module 'mksh' will not be installed, because command 'mksh' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command 'coredumpctl' could not be found! dracut: dracut module 'systemd-coredump' will not be installed, because command '/usr/lib/systemd/systemd-coredump' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command 'portablectl' could not be found! dracut: dracut module 'systemd-portabled' will not be installed, because command '/usr/lib/systemd/systemd-portabled' could not be found! dracut: dracut module 'systemd-resolved' will not be installed, because command 'resolvectl' could not be found! dracut: dracut module 'systemd-resolved' will not be installed, because command '/usr/lib/systemd/systemd-resolved' could not be found! dracut: dracut module 'systemd-timesyncd' will not be installed, because command '/usr/lib/systemd/systemd-timesyncd' could not be found! dracut: dracut module 'dbus-broker' will not be installed, because command 'dbus-broker' could not be found! dracut: dracut module 'rngd' will not be installed, because command 'rngd' could not be found! dracut: dracut module 'lvmmerge' will not be installed, because command 'lvm' could not be found! dracut: dracut module 'lvmthinpool-monitor' will not be installed, because command 'lvm' could not be found! dracut: dracut module 'dmraid' will not be installed, because command 'dmraid' could not be found! dracut: dracut module 'lvm' will not be installed, because command 'lvm' could not be found! dracut: dracut module 'mdraid' will not be installed, because command 'mdadm' could not be found! dracut: dracut module 'multipath' will not be installed, because command 'multipath' could not be found! dracut: dracut module 'pcsc' will not be installed, because command 'pcscd' could not be found! dracut: dracut module 'tpm2-tss' will not be installed, because command 'tpm2' could not be found! dracut: dracut module 'nvmf' will not be installed, because command 'nvme' could not be found! dracut: dracut module 'biosdevname' will not be installed, because command 'biosdevname' could not be found! dracut: dracut module 'memstrack' will not be installed, because command 'memstrack' could not be found! dracut: memstrack is not available dracut: If you need to use rd.memdebug>=4, please install memstrack and procps-ng dracut: *** Including module: systemd *** dracut: *** Including module: systemd-initrd *** dracut: *** Including module: modsign *** dracut: *** Including module: console-setup *** dracut: *** Including module: i18n *** dracut: *** Including module: drm *** dracut: *** Including module: plymouth *** dracut: *** Including module: btrfs *** dracut: *** Including module: crypt *** dracut: *** Including module: dm *** dracut: Skipping udev rule: 10-dm.rules dracut: Skipping udev rule: 13-dm-disk.rules dracut: Skipping udev rule: 64-device-mapper.rules dracut: *** Including module: kernel-modules *** dracut: *** Including module: kernel-modules-extra *** dracut: *** Including module: nvdimm *** dracut: *** Including module: overlay-root *** dracut: *** Including module: qemu *** dracut: *** Including module: lunmask *** dracut: *** Including module: resume *** dracut: *** Including module: rootfs-block *** dracut: *** Including module: terminfo *** dracut: *** Including module: udev-rules *** dracut: Skipping udev rule: 40-redhat.rules dracut: Skipping udev rule: 91-permissions.rules dracut: Skipping udev rule: 80-drivers-modprobe.rules dracut: *** Including module: virtiofs *** dracut: *** Including module: dracut-systemd *** dracut: *** Including module: usrmount *** dracut: *** Including module: base *** dracut: *** Including module: fs-lib *** dracut: *** Including module: shutdown *** dracut: *** Including modules done *** dracut: *** Installing kernel module dependencies *** dracut: *** Installing kernel module dependencies done *** dracut: *** Resolving executable dependencies *** dracut: *** Resolving executable dependencies done *** dracut: *** Hardlinking files *** dracut: Mode: real dracut: Method: sha256 dracut: Files: 2226 dracut: Linked: 211 files dracut: Compared: 0 xattrs dracut: Compared: 3762 files dracut: Saved: 18.82 MiB dracut: Duration: 0.203010 seconds dracut: *** Hardlinking files done *** dracut: *** Generating early-microcode cpio image *** dracut: *** Constructing AuthenticAMD.bin *** dracut: *** Constructing GenuineIntel.bin *** dracut: *** Store current command line parameters *** dracut: *** Stripping files *** dracut: *** Stripping files done *** dracut: *** Creating image file '/boot/initrd.img-6.1.0-10-amd64' *** dracut: Using auto-determined compression method 'gzip' dracut: *** Creating initramfs image file '/boot/initrd.img-6.1.0-10-amd64' done ***
Footnotes[edit]

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 12 year success story and maybe DONATE!