Debian encrypted root partition, systemd and crypttab

I have a simple server with encrypted disks running debian. I had trouble setting up a fully encrypted system, so I thought I’d share the details of my setup.

/dev/sda1 SSD: unencrypted boot partition
/dev/sda2 SSD: encrypted lvm partition containing system root partition. This is a luks partition.
/dev/sdb  conventional 3TB encrypted hard disk with zfs mirrored raid
/dev/sdc  conventional 3TB encrypted hard disk with zfs mirrored raid
/dev/<more-raid-disks>

When booting, grub loads the kernel in the boot partition, and the initramfs prompts for the decryption passphrase. Once I enter it, the initramfs scripts mount an lvm partition containing my encrypted root partition.

Then, systemd prompts for the passphrase again several times for each entry in my crypttab. This is a pain in the butt, although I don’t really have to reboot very often.

There is a simple solution to this: you create a second luks key that decrypts the raid array, and store it somewhere on the root partition (like /etc/luks-key1). Then you make a crypttab file like so:

# <target name>	<source device>		<key file>	<options>
cryptroot UUID=<UUID>   	        none            luks
cryptzfs1  /dev/disk/by-id/<DEVICEID>   /etc/luks-key1  luks
cryptzfs2  /dev/disk/by-id/<DEVICEID>   /etc/luks-key1  luks

The initramfs loads the encrypted root partition, and systemd creates mount units for each crypttab entry using a generator. See

man systemd-cryptsetup-generator

for more details about this. Unfortunately, all my raid disks are configured to be plain dm-crypt, and such a keyfile does not work with systemd. I did it this was because I was told that the luks header would cause alignment issues with zfs that would degrade filesystem performance. Well, plain dm-crypt is real pain in the ass, and I’d rather take the minor performance hit in the future. My solution is to use debian’s keyscript, decrypt_keyctl to not have to re-enter the password a billion times. However, this comes with issues too, since systemd does not support the keyscript option in crypttab! I have a simple workaround that others might find useful that I’ll detail below.

Initramfs setup

Copy the following files

cp -a /usr/share/initramfs-tools/scripts/local-top/cryptroot /etc/initramfs-tools/scripts/local-top/cryptroot
cp -a /usr/share/initramfs-tools/scripts/hooks/cryptroot /etc/initramfs-tools/scripts/hooks/cryptroot
cp -a /usr/share/initramfs-tools/scripts/local-block/cryptroot /etc/initramfs-tools/scripts/local-block/cryptroot
cp -a /usr/share/initramfs-tools/hooks/cryptkeyctl /etc/initramfs-tools/scripts/local-block/cryptroot

I don’t think all of these files are required, but copy them anyway. Then, edit the file /etc/initramfs-tools/conf.d/cryptroot and add a line

CRYPTOPTS=target=cryptroot,source=/dev/<device of root partition>,key=none,lvm=ssd1-debianroot

Do not run update-initramfs after you have all of this setup.

Crypttab

Next thing to do is to setup crypttab.

    # <target name>	<source device>		<key file>	<options>
    cryptroot UUID=<UUID>		        none            luks
    cryptzfs1  /dev/disk/by-id/<DEVICEID>   zfs_raidstore  plain,cipher=aes-xts-plain64,hash=sha512,offset=0,size=512,keyscript=decrypt_keyctl,initramfs
    cryptzfs2  /dev/disk/by-id/<DEVICEID>   zfs_raidstore  plain,cipher=aes-xts-plain64,hash=sha512,offset=0,size=512,keyscript=decrypt_keyctl,initramfs

The key is the keyscript=decrypt_keyctl line. This makes it store a key in the memory using the /bin/keyctl command. You need the keyutils package installed:

apt install keyutils

The zfs_raidstore identifies which of the crypttab entries have the same passphrase.

The third ingredient is the initramfs option, which tells the initramfs to load these crypttab entries. Usually the initramfs would only load the root partition. If you didn’t have this hook here, systemd would load it instead. And systemd does not currently have support for the keyscript line in crypttab, as mentioned earlier.

Therefore, systemd crypttab generators have to be disabled with the following line in /etc/default/grub.cfg

GRUB_CMDLINE_LINUX_DEFAULT="quiet luks.crypttab=no"

More options like luks.crypttab=no may be found in man systemd-cryptsetup-generator. You can test your crypttab setup with

cryptdisks_start <name in crypttab>

This is a required step before you run update-initramfs, since it appears to need the encrypted disks to be mounted. Backup your current initramfs (by adding a backup line in /etc/initramfs-tools/update-initramfs.conf for example) and then run

update-initramfs -k <version> -u -v

and you should look for lines like

copied /bin/keyctl
calling hook cryptkeyctl

and so on. Then run

update-grub

and reboot. Hopefully this works for you.

Problems

The problem is that the root encrypted partition is called by cryptroot, which doesn’t seem to listen to the keyscript option. Therefore, I have to enter the password twice, once for the raid array and once for the root partition.

The way to fix this is to have and have extra luks keys for the raid partitions as mentioned earlier. Unfortunately, my raid disks use plain dm-crypt. There are other solutions to this, of course: I could use a different keyscript that simply reads a file in a root partition and outputs the password. The complication with this is that you have to specify the device of the decrypted root device the keyscript lives on —I have not had much success with this— or store the key unecrypted in the initramfs.

Thus, this was my slightly suboptimal solution. However, if I end up having 10 disks in the future, this will be quite useful.