Gentoo Linux: Building/rebuilding a kernel and Intel CPU microcode in an installation with initramfs

In a 2014 post I explained how to update the Intel CPU microcode in a Gentoo Linux installation with an initramfs (I use sys-kernel/genkernel to build the kernel in the installation on my Compal NBLB2 laptop, which is running the Testing Branch of Gentoo Linux although the branch is not important). The initscript method (Method 1 in that post) for updating the CPU microcode is no longer valid, and the behaviour of the tool sys-apps/iucode_tool for updating the CPU microcode (Method 2 in that post) has changed, hence this update.

Although not essential I normally perform the microcode upgrade procedure when I either rebuild or upgrade the Linux kernel, therefore I explain both procedures contiguously here.

These days the grub-mkconfig command edits the file /boot/grub/grub.cfg to add a line to the GRUB menu entries, to load the CPU microcode at boot, but nevertheless I prefer to follow a slightly different method that works reliably for me.

Below is the procedure I follow to build/rebuild the kernel and the Intel CPU microcode. Others may have a different approach, but this has always worked well for me, even if some of the steps are sometimes nugatory.

If they are not already installed, you need to merge a couple of packages before starting the main procedure:

root # emerge app-arch/lzma # Needed to build bzImage.
root # emerge iucode_tool

1. Mount the boot directory if it is on a separate partition

root # mount /dev/sda3 /boot

2. Check which kernel sources are installed and which of those sources is currently selected

root # eselect kernel list

3. Make a back-up configuration file for the current running kernel

root # zcat /proc/config.gz > /usr/src/config

4. Select the kernel sources I want to build

root # eselect kernel set <n>

5. Build the kernel image and the initramfs image

root # genkernel --kernel-config=/usr/src/config --clean --menuconfig --microcode=intel --no-splash --module-rebuild all

I have configured the following kernel options relating to the early loading of the Intel CPU microcode (see later):

root # grep CONFIG_BLK_DEV_INITRD /usr/src/linux/.config
CONFIG_BLK_DEV_INITRD=y
root # # grep CONFIG_MICROCODE /usr/src/linux/.config
CONFIG_MICROCODE=y
CONFIG_MICROCODE_INTEL=y
# CONFIG_MICROCODE_AMD is not set
# CONFIG_MICROCODE_OLD_INTERFACE is not set
# grep CONFIG_INITRAMFS_SOURCE /usr/src/linux/.config
CONFIG_INITRAMFS_SOURCE=""

6. Rebuild the X Windows Server and X Windows drivers

I always do this even though not always necessary. One less thing to think about (not rebuilding them has sometimes caused me problems).

root # emerge xorg-server xorg-drivers

7. Rebuild NetworkManager if it is installed

I always do this even though not always necessary. One less thing to think about (not rebuilding it has sometimes caused me problems).

root # emerge networkmanager

8. If there is a new version of the Intel CPU microcode, generate it and copy it to the boot directory

For several years updates to the package sys-kernel/linux-firmware have not resulted in a change to the version of Intel CPU microcode for the legacy Intel Core i7-720QM CPU in my Compal NBLB2 laptop, as Intel no longer supports that version of CPU. Nevertheless it does no harm to repeat the procedure.

root # emerge sys-firmware/intel-microcode
root # rm /boot/microcode.cpio
root # iucode_tool -S --write-earlyfw=/boot/microcode.cpio /lib/firmware/intel-ucode/*
root # rm /boot/intel-uc.img

(The fourth command is to stop the grub-mkconfig command (see Step 9.2) adding intel-uc.img to the initrd line in the grub.cfg file.)

Note the USE flags for that I have set and cleared for sys-firmware/intel-microcode:

root # equery uses intel-microcode
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-firmware/intel-microcode-20210608_p20210830:
 U I
 - - hostonly    : only install ucode(s) supported by currently available (=online) processor(s) 
 - - initramfs   : install a small initramfs for use with CONFIG_MICROCODE_EARLY 
 + + split-ucode : install the split binary ucode files (used by the kernel directly) 
 - - vanilla     : install only microcode updates from Intel's official microcode tarball

9. Create a new grub.cfg file

9.1 First check the contents of /etc/default/grub to make sure it will be OK for the new version of the kernel

root # nano /etc/default/grub

Modify the contents of /etc/default/grub if necessary for the kernel version that has just been built.

9.2 Generate a new grub.cfg file

root # grub-mkconfig -o /boot/grub/grub.cfg

9.3 Check the new grub.cfg file includes the loading of the CPU microcode

root # nano /boot/grub/grub.cfg

The last line for each menu entry (i.e. the line before the closing curly bracket of the menu entry) should contain:

initrd /microcode.cpio /initramfs-<kernel version>-gentoo-x86_64.img

as shown in the example file excerpt below:

[...]
### BEGIN /etc/grub.d/10_linux ###
menuentry 'Gentoo GNU/Linux' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-9db2f668-a682-4d6f-abc5-ed6f6c515b95' {
load_video
set gfxpayload=1024x768
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos3'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos3 --hint-efi=hd0,msdos3 --hint-baremetal=ahci0,msdos3  597e8c88-8d50-443f-ae19-f510844f5d4e
else
search --no-floppy --fs-uuid --set=root 597e8c88-8d50-443f-ae19-f510844f5d4e
fi
echo	'Loading Linux 5.15.0-gentoo-x86_64 ...'
linux	/vmlinuz-5.15.0-gentoo-x86_64 root=/dev/sda6 ro BOOT_IMAGE=/kernel-genkernel-x86_64-5.15.0-gentoo root=/dev/ram0 ramdisk=8192 real_root=/dev/sda6 init=/linuxrc resume=swap:/dev/sda5 real_resume=/dev/sda5 intel_iommu=off net.ifnames=0 snd_hda_intel.power_save=0 radeon.modeset=1
echo	'Loading initial ramdisk ...'
initrd	/microcode.cpio /initramfs-5.15.0-gentoo-x86_64.img
}
submenu 'Advanced options for Gentoo GNU/Linux' $menuentry_id_option 'gnulinux-advanced-9db2f668-a682-4d6f-abc5-ed6f6c515b95' {
menuentry 'Gentoo GNU/Linux, with Linux 5.15.0-gentoo-x86_64' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.15.0-gentoo-x86_64-advanced-9db2f668-a682-4d6f-abc5-ed6f6c515b95' {
load_video
set gfxpayload=1024x768
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos3'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos3 --hint-efi=hd0,msdos3 --hint-baremetal=ahci0,msdos3  597e8c88-8d50-443f-ae19-f510844f5d4e
else
search --no-floppy --fs-uuid --set=root 597e8c88-8d50-443f-ae19-f510844f5d4e
fi
echo	'Loading Linux 5.15.0-gentoo-x86_64 ...'
linux	/vmlinuz-5.15.0-gentoo-x86_64 root=/dev/sda6 ro BOOT_IMAGE=/kernel-genkernel-x86_64-5.15.0-gentoo root=/dev/ram0 ramdisk=8192 real_root=/dev/sda6 init=/linuxrc resume=swap:/dev/sda5 real_resume=/dev/sda5 intel_iommu=off net.ifnames=0 snd_hda_intel.power_save=0 radeon.modeset=1
echo	'Loading initial ramdisk ...'
initrd	/microcode.cpio /initramfs-5.15.0-gentoo-x86_64.img
}
menuentry 'Gentoo GNU/Linux, with Linux 5.15.0-gentoo-x86_64 (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.15.0-gentoo-x86_64-recovery-9db2f668-a682-4d6f-abc5-ed6f6c515b95' {
load_video
set gfxpayload=1024x768
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos3'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos3 --hint-efi=hd0,msdos3 --hint-baremetal=ahci0,msdos3  597e8c88-8d50-443f-ae19-f510844f5d4e
else
search --no-floppy --fs-uuid --set=root 597e8c88-8d50-443f-ae19-f510844f5d4e
fi
echo	'Loading Linux 5.15.0-gentoo-x86_64 ...'
linux	/vmlinuz-5.15.0-gentoo-x86_64 root=/dev/sda6 ro single BOOT_IMAGE=/kernel-genkernel-x86_64-5.15.0-gentoo root=/dev/ram0 ramdisk=8192 real_root=/dev/sda6 init=/linuxrc resume=swap:/dev/sda5 real_resume=/dev/sda5 intel_iommu=off net.ifnames=0 snd_hda_intel.power_save=0 radeon.modeset=1
echo	'Loading initial ramdisk ...'
initrd	/microcode.cpio /initramfs-5.15.0-gentoo-x86_64.img
}
}

### END /etc/grub.d/10_linux ###
[...]

10. Reboot

11. Rebuild VirtualBox if it is installed

root # emerge virtualbox

12. Check the current version of the Intel CPU microcode

Either:

root # dmesg | grep microcode

or:

root # grep microcode /proc/cpuinfo

For example:

root # dmesg | grep microcode
[    0.000000] microcode: microcode updated early to revision 0xa, date = 2018-05-08
[    0.127937] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    1.558008] microcode: sig=0x106e5, pf=0x10, revision=0xa
[    1.559335] microcode: Microcode Update Driver: v2.2.
root # grep microcode /proc/cpuinfo
microcode       : 0xa
microcode       : 0xa
microcode       : 0xa
microcode       : 0xa
microcode       : 0xa
microcode       : 0xa
microcode       : 0xa
microcode       : 0xa

Note from the output of the dmesg command that this specific CPU model is susceptible to the MDS (Microarchitectural Data Sampling) vulnerability.

13. Edit /var/lib/portage/world and add (or change) the specific kernel sources package version

I do this in order to ensure the command ‘emerge --depclean‘ does not remove a specific kernel’s source code during a world update. I want Portage always to install the latest version of gentoo-sources but not to delete the version of gentoo-sources that corresponds to the kernel my installation is currently using.

For example, let’s say I have just replaced a kernel built from gentoo-sources:5.15.11 with a kernel built from gentoo-sources:5.15.12. My world file would initially contain the following:

[...]
sys-kernel/gentoo-sources
sys-kernel/gentoo-sources:5.15.11
[...]

If, following a successful reboot with kernel 5.15.12, I want to delete the files for kernel 5.15.11 in /boot/ (initramfs-5.15.11-gentoo-x86_64.img, System.map-5.15.11-gentoo-x86_64 and vmlinuz-5.15.11-gentoo-x86_64) and to edit the file /boot/grub/grub.cfg to remove the menu entries for kernel 5.15.11, I would change the world file’s contents to:

[...]
sys-kernel/gentoo-sources
sys-kernel/gentoo-sources:5.15.12
[...]

On the other hand, if, following a successful reboot, I want to keep the files for both kernel 5.15.11 and kernel 5.15.12, I would change the world file’s contents to:

[...]
sys-kernel/gentoo-sources
sys-kernel/gentoo-sources:5.15.11
sys-kernel/gentoo-sources:5.15.12
[...]

Advertisement

Work-around if movie subtitles restart after the final subtitle is displayed

If I’m watching movies in a language I don’t understand, I want subtitles. On my computers this is possible as long as there is a subtitles file with the name suffix .srt and the same prefix name as the .mp4 video file in the same directory. I usually prefer to view movies on my TV with a bigger screen, so I copy the movie to a HDD that is normally connected to my TV (a FINLUX model 43-FUD-8020). However, the built-in media player in the TV does not show the subtitles in the .srt file, even when it is in the same directory as the .mp4 file. Therefore I use the MKVToolNix utility mkvmerge to put the movie and subtitles into a Matroska multimedia container (.mkv file), and the TV’s media player can play these .mkv files and does display the subtitles. In fact, so can my laptops and desktop running Linux (I have not tried on a machine running Windows 10, but I assume Windows 10 would have no trouble either).

To install in Lubuntu 20.10:

user $ sudo apt install mkvtoolnix

To install in Gentoo Linux:

root # emerge mkvtoolnix

To create a Matroska file containing the movie plus subtitles:

user $ mkvmerge -o movie_with_subtitles.mkv movie_without_subtitles.mp4 subtitles.srt

Normally the last subtitle in a movie does not occur at the very end of the movie. For example, there could be action without dialogue at the end of the movie, and/or final credits without dialogue. The media players on my laptops and desktop running Linux display the last subtitle and play the rest of the movie in the Matroska container as expected. However, the media player in my FINLUX TV displays the last subtitle and then displays the subtitles from the beginning again, at breakneck speed. Annoying to say the least. As the problem does not occur on my laptops and desktop with the same .mkv file, I assume the problem lies with the media player in the TV.

At first I suspected that the .srt file was the cause, but it correctly uses UTF-8 encoding and the syntax of the contents is correct. Anyway, just to be sure I ran it through an online cleaner for .srt files and re-generated the .mkv file, but that made no difference on the TV. Since there is no problem playing the .mkv file on my computers, I can only assume the TV’s media player is indeed at fault. I cannot do anything about the TV’s media player, so I came up with an acceptable work-around: I added a dummy subtitle at the end of the .srt file that is set to be displayed at the very end of the movie. For example, let’s say the movie duration is two hours, 12 minutes and twenty-two seconds but the last subtitle is at 01:56:38,201:

188
01:56:38,201 --> 01:56:40,286
The end justifies the means.

I edited the file and added a dummy subtitle at the end:

188
01:56:38,201 --> 01:56:40,286
The end justifies the means.

189
02:12:19,001 --> 02:12:21,999
THE END.

I then re-generated the .mkv file using the mkvmerge command and, lo and behold, after the subtitle displayed between at 01:56:38,201 and 01:56:40,286 the TV no longer displays any more subtitles until the very end of the movie when it displays ‘THE END’ and the video ends. Actually, in reality the movie must be very slightly longer than 02:12:21,999 because, after displaying ‘THE END’, the first six subtitles in the subtitle file are displayed in rapid succession before the media player stops playing, but that is no big deal.

I searched the Web quite a lot and was unable to find any mention of this particular problem, so I am posting my work-around here just in case it helps someone else experiencing the same problem.

Gentoo Linux: Building/rebuilding a kernel and Intel CPU microcode in an installation without initramfs

In a 2016 post I explained how to update the Intel CPU microcode in a Gentoo Linux Stable Branch installation without an initramfs (I do not use sys-kernel/genkernel to build the kernel in the installation on my Clevo W230SS laptop). The behaviour of the tool sys-apps/iucode_tool for updating the Intel CPU microcode has changed since that post, hence this update.

Although not essential I normally perform the microcode upgrade procedure when I either rebuild or upgrade the Linux kernel, therefore I explain both procedures contiguously here.

These days the grub-mkconfig command edits the file /boot/grub/grub.cfg to add a line to the GRUB menu entries, to load the CPU microcode at boot, but nevertheless I prefer to follow a slightly different method that works reliably for me.

Below is the procedure I follow to build/rebuild the kernel and the Intel CPU microcode. Others may have a different approach, but this has always worked well for me, even if some of the steps are sometimes nugatory.

1. Mount the boot directory if it is on a separate partition

root # mount /dev/sda1 /boot

2. Check which kernel sources are installed and which of those sources is currently selected

root # eselect kernel list

3. Make a back-up of the current kernel configuration file

root # cp /usr/src/linux-`uname -r`/.config /home/fitzcarraldo/kernel-config-`uname -r`

4. Select the kernel sources I want to build

root # eselect kernel set <n>

5. Change to the currently selected kernel sources directory

root # cd /usr/src/linux

6. If wanting to build a new version of the kernel, create a template configuration file

N.B. Do NOT do this if rebuilding the kernel version that is currently in use.

root # cp /usr/src/linux-`uname -r`/.config /usr/src/linux/.config

7. Remove any existing object files

Definitely needed if the ‘make‘ command (see further on) returns an error message mentioning an old version of the compiler. It does no harm to perform this step in any case, so I always do it.

root # make clean

8. If building a new version of the kernel, create a new configuration file

N.B. Do NOT do this if rebuilding the kernel version that is currently in use.

root # make olddefconfig

The command ‘make olddefconfig‘ will edit the existing /usr/src/linux/.config file, keeping all the existing options in the file and setting any new options to their recommended (i.e. default) values.

9. Display a TUI menu of the kernel options in the .config file and make any desired changes

root # make menuconfig

I have configured the following kernel options relating to the early loading of the Intel CPU microcode (see later):

root # grep CONFIG_BLK_DEV_INITRD /usr/src/linux/.config
CONFIG_BLK_DEV_INITRD=y
root # grep CONFIG_MICROCODE /usr/src/linux/.config
CONFIG_MICROCODE=y
CONFIG_MICROCODE_INTEL=y
# CONFIG_MICROCODE_AMD is not set
# CONFIG_MICROCODE_OLD_INTERFACE is not set
root # grep CONFIG_INITRAMFS_SOURCE /usr/src/linux/.config
CONFIG_INITRAMFS_SOURCE=""

10. Build the kernel and modules

root # make && make modules_install
root # make install

11. Rebuild any third-party packages containing kernel modules

These could include packages such as nvidia-drivers, for example.

root # emerge @module-rebuild

In my case, currently the @module-rebuild set only comprises the following two packages:

root # cat /var/lib/module-rebuild/moduledb
a:1:app-emulation/virtualbox-modules-6.1.24
a:1:x11-drivers/nvidia-drivers-470.63.01

12. Rebuild the X Windows Server and X Windows drivers

I always do this even though not always necessary. One less thing to think about (not rebuilding them has sometimes caused me problems).

root # emerge xorg-server xorg-drivers

13. Rebuild NetworkManager if it is installed

I always do this even though not always necessary. One less thing to think about (not rebuilding it has sometimes caused me problems).

root # emerge networkmanager

14. If there is a new version of the Intel CPU microcode, generate it and copy it to the boot directory

Updates to the package sys-firmware/intel-microcode in the last couple of years have not resulted in a change to the version of Intel CPU microcode for the fourth-generation Intel Core i7-4810MQ CPU in my Clevo W230SS laptop, so I assume Intel no longer supports that version of CPU. Nevertheless it does no harm to repeat the procedure.

root # rm /boot/microcode.cpio
root # iucode_tool -S --write-earlyfw=/boot/microcode.cpio /lib/firmware/intel-ucode/*
root # rm /boot/intel-uc.img

(The third command is to stop the grub-mkconfig command (see later) adding intel-uc.img to the initrd line in the grub.cfg file.)

15. If a different version of the kernel has just been built, or if this is the first time upgrading the CPU microcode, create a new grub.cfg file

15.1 First check the contents of /etc/default/grub to make sure it will be OK for the new version of the kernel

root # nano /etc/default/grub

Modify the contents of /etc/default/grub if necessary for the kernel that has just been built.

15.2 Generate a new grub.cfg file

root # grub-mkconfig -o /boot/grub/grub.cfg

15.3 Check the new grub.cfg file includes the loading of the CPU microcode

root # nano /boot/grub/grub.cfg

The last line for each menu entry (i.e. the line before the closing curly bracket of the menu entry) should contain only ‘initrd /microcode.cpio‘, as shown in the example file excerpt below:

[...]
### BEGIN /etc/grub.d/10_linux ###
menuentry 'Gentoo GNU/Linux' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-525a90f1-8ad2-44a3-ade3-20f18a0a9595' {
load_video
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  f6ffc085-66fe-4bbe-b080-cec355749f85
else
search --no-floppy --fs-uuid --set=root f6ffc085-66fe-4bbe-b080-cec355749f85
fi
echo	'Loading Linux 5.10.61-gentoo ...'
linux	/vmlinuz-5.10.61-gentoo root=/dev/sda5 ro  locale=en_GB i965.modeset=1 rcutree.rcu_idle_gp_delay=1 acpi_enforce_resources=lax reboot=force raid=noautodetect resume=/dev/sda2
echo	'Loading initial ramdisk ...'
initrd	/microcode.cpio
}
submenu 'Advanced options for Gentoo GNU/Linux' $menuentry_id_option 'gnulinux-advanced-525a90f1-8ad2-44a3-ade3-20f18a0a9595' {
menuentry 'Gentoo GNU/Linux, with Linux 5.10.61-gentoo' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.61-gentoo-advanced-525a90f1-8ad2-44a3-ade3-20f18a0a9595' {
load_video
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  f6ffc085-66fe-4bbe-b080-cec355749f85
else
search --no-floppy --fs-uuid --set=root f6ffc085-66fe-4bbe-b080-cec355749f85
fi
echo	'Loading Linux 5.10.61-gentoo ...'
linux	/vmlinuz-5.10.61-gentoo root=/dev/sda5 ro  locale=en_GB i965.modeset=1 rcutree.rcu_idle_gp_delay=1 acpi_enforce_resources=lax reboot=force raid=noautodetect resume=/dev/sda2
echo	'Loading initial ramdisk ...'
initrd	/microcode.cpio
}
menuentry 'Gentoo GNU/Linux, with Linux 5.10.61-gentoo (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.61-gentoo-recovery-525a90f1-8ad2-44a3-ade3-20f18a0a9595' {
load_video
insmod gzio
insmod part_msdos
insmod ext2
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  f6ffc085-66fe-4bbe-b080-cec355749f85
else
search --no-floppy --fs-uuid --set=root f6ffc085-66fe-4bbe-b080-cec355749f85
fi
echo	'Loading Linux 5.10.61-gentoo ...'
linux	/vmlinuz-5.10.61-gentoo root=/dev/sda5 ro single
echo	'Loading initial ramdisk ...'
initrd	/microcode.cpio
}
}

### END /etc/grub.d/10_linux ###
[...]

16. Reboot

17. Rebuild VirtualBox if it is installed

root # emerge virtualbox

18. Check the current version of the Intel CPU microcode

Either:

root # dmesg | grep microcode

or:

root # grep microcode /proc/cpuinfo

For example:

root # dmesg | grep microcode
[    0.000000] microcode: microcode updated early to revision 0x28, date = 2019-11-12
[    0.335631] microcode: sig=0x306c3, pf=0x10, revision=0x28
[    0.335730] microcode: Microcode Update Driver: v2.2.
root # grep microcode /proc/cpuinfo
microcode       : 0x28
microcode       : 0x28
microcode       : 0x28
microcode       : 0x28
microcode       : 0x28
microcode       : 0x28
microcode       : 0x28
microcode       : 0x28

19. Edit /var/lib/portage/world and add (or change) the specific kernel sources package version

I do this in order to ensure the command ‘emerge --depclean‘ does not remove a specific kernel’s source code during a world update. I want Portage always to install the latest (stable) version of gentoo-sources but not to delete the version of gentoo-sources that corresponds to the kernel my installation is currently using.

For example, let’s say I have just replaced a kernel built from gentoo-sources:4.19.57 with a kernel built from gentoo-sources:4.19.66. My world file would initially contain the following:

[...]
sys-kernel/gentoo-sources
sys-kernel/gentoo-sources:4.19.57
[...]

If, following a successful reboot with kernel 4.19.66, I want to delete the files for kernel 4.19.17 in /boot/ (System.map-4.19.17-gentoo, config-4.19.17-gentoo and vmlinuz-4.19.17-gentoo) and to edit the file /boot/grub/grub.cfg to remove the menu entries for kernel 4.19.57, I would change the world file’s contents to:

[...]
sys-kernel/gentoo-sources
sys-kernel/gentoo-sources:4.19.66
[...]

On the other hand, if, following a successful reboot, I want to keep the files for both kernel 4.19.17 and kernel 4.19.66, I would change the world file’s contents to:

[...]
sys-kernel/gentoo-sources
sys-kernel/gentoo-sources:4.19.57
sys-kernel/gentoo-sources:4.19.66
[...]

Browsing a WebDAV share in Linux and Windows 10

In this post I explain how I configured my machines running two Linux distributions (Gentoo Linux and Lubuntu 20.10) and my Windows 10 test machine to enable me to browse a shared folder on my file server (running ownCloud, in my case) that uses the WebDAV protocol. I cover two options for configuring Linux to browse WebDAV shares. Further options exist in Linux, but the two methods I give here are fine for my purposes.

I installed ownCloud on my Linux server in a slightly different way to the method in the ownCloud installation manual, and my examples in this post use the URI https://fitzcarraldo.ddns.net/owncloud/remote.php/webdav rather than the usual https://fitzcarraldo.ddns.net/remote.php/webdav for ownCloud, so replace the URI in my examples with the appropriate URI in your case. The username of the user account on each client machine is ‘fitz’, and the ownCloud username (davusername) on the server is ‘bsf’. Obviously replace those with the usernames in your case.

PART 1 – LINUX

Unless I mention the distribution explicitly, the following steps apply to both Linux distributions. As my Gentoo Linux installations use KDE, the steps for Gentoo Linux assume the file manager is Dolphin. My Lubuntu installation uses the file manager PCManFM-Qt.

1. Install davfs2 if it is not already installed

Gentoo Linux:

root # emerge davfs2

That command installs three packages:

acct-group/davfs2
acct-user/davfs2
net-fs/davfs2

Lubuntu 20.10:

user $ sudo apt install davfs2

2. Lubuntu 20.10: Allow mounting by non-root users

user $ sudo dpkg-reconfigure davfs2

   Package configuration
   
    ┌──────────────────────────────────────────┤ Configuring davfs2 ├───────────────────────────────────────────┐
    │                                                                                                           │
    │ The file /sbin/mount.davfs must have the SUID bit set if you want to allow unprivileged (non-root) users  │
    │ to mount WebDAV resources.                                                                                │
    │                                                                                                           │
    │ If you do not choose this option, only root will be allowed to mount WebDAV resources. This can later be  │
    │ changed by running 'dpkg-reconfigure davfs2'.                                                             │
    │                                                                                                           │
    │ Should unprivileged users be allowed to mount WebDAV resources?                                           │
    │                                                                                                           │
    │                               <Yes>                                  <No>                                 │
    │                                                                                                           │
    └───────────────────────────────────────────────────────────────────────────────────────────────────────────┘

(Do not do anything in Gentoo Linux; the SUID bit should be set automatically.)

3. Check the SUID bit has been set (notice the ‘s’ in the file’s permissions)

Gentoo Linux:

user $ ls -la /sbin/mount.davfs
lrwxrwxrwx 1 root root 21 Sep 25 23:03 /sbin/mount.davfs -> /usr/sbin/mount.davfs
user $ ls -la /usr/sbin/mount.davfs
-rws--x--x 1 root root 130752 Sep 25 23:03 /usr/sbin/mount.davfs

If the SUID bit has not be set automatically, you can do it manually:

user $ sudo chmod u+s /usr/sbin/mount.davfs

Lubuntu 20.10:

user $ ls -la /sbin/mount.davfs
-rwsr-xr-x 1 root root 137464 Aug  8  2020 /sbin/mount.davfs

4. Add the user to the davfs2 group

user $ sudo usermod -aG davfs2 fitz

Logout and login again and check the user is a member of the group:

user $ groups | grep -q davfs2 && echo "OK"
OK

5. Leave the lines in the following files commented out (i.e. accept the defaults)

/etc/davfs2/davfs2.conf (system-wide)

~/.davfs2/davfs2.conf (user-specific)

6. Option 1 (simplest!) – Enter the URI in the file manager and bookmark it

6.1 Gentoo Linux with KDE

Enter the following URI on the Dolphin file manager’s address line and press Enter:

webdavs://fitzcarraldo.ddns.net/owncloud/remote.php/webdav

You will be prompted to enter the username and password for the WebDAV share.

Select ‘File’ > ‘Add to Places’ in Dolphin to bookmark the share. From then on, you can browse the share by clicking on the share in the Remote section in Dolphin’s Places pane. You can rename the bookmark if you wish (right-click and select ‘Edit…’).

Another way to do this in KDE is as follows:

1. click on ‘Network’ in the Places pane;
2. click on ‘Add Network Folder’ next to the address bar;
3. select ‘WebFolder (webdav)’ and click ‘Next’;
4. enter the fields as follows:
   • Name: webdav
   • User: bsf
   • Server: fitzcarraldo.ddns.net
   • Port: 443 (I use Port 443 but you may be using a different port)
   • Folder: owncloud/remote.php/webdav
5. select ‘Create an icon for this folder’ and ‘Use encryption’;
6. click ‘Save & Connect’;
7. right-click on the webdav icon in the main Dolphin pane and select ‘Add to Places’.

6.2 Lubuntu 20.10

Enter the following URI on the PCManFM-Qt file manager’s address line and press Enter:

davs://fitzcarraldo.ddns.net/owncloud/remote.php/webdav

You will be prompted to enter the username and password for the WebDAV share.

Select ‘Bookmarks’ > ‘Add to Bookmarks’ in PCManFM-Qt to bookmark the share. From then on, you can browse the share by clicking on the share in the Bookmarks section in PCManFM-Qt’s Lists pane. You can rename the bookmark if you wish (Bookmarks > Edit Bookmarks).

7. Option 2 – Assign a mountpoint at boot:

Add the following credentials line in the file ~/.davfs2/secrets:

https://fitzcarraldo.ddns.net/owncloud/remote.php/webdav <davusername> <davpassword>

and set the file permissions as follows:

user $ chmod 600 ~/.davfs2/secrets

Create a user directory onto which to mount the share:

user $ mkdir ~/webdav

Add a line in /etc/fstab to map the WebDAV share onto that directory at boot:

# <file system>                                            <mount point>       <type>  <options>        <dump>  <pass>
https://fitzcarraldo.ddns.net/owncloud/remote.php/webdav   /home/fitz/webdav   davfs   noauto,user,rw   0       0

The options ‘auto‘ and ‘_netdev‘ do not mount the WebDAV share automatically at boot in my installations; I am prompted to enter the davuser and davpassword manually early in the boot process if I include those options. To avoid the latter I use the ‘noauto‘ option and do not bother including the ‘_netdev‘ option. There are ways to mount a WebDAV share automatically at boot whether your installation uses systemd, OpenRC or other rc systems. Nevertheless I prefer the WebDAV share not to be mounted auomatically at boot, especially in the case of my laptops.

Reboot to check everything works.

Lubuntu 20.10:

The share will be listed as ‘webdav’ (unmounted) in the Devices section under Lists in PCManFM-Qt. You can click on the unmounted share to mount it, and click on the Unmount icon to unmount it. Everything works as expected.

Gentoo Linux with KDE:

The share is not listed in the Places pane in Dolphin but the share can be mounted manually from the command line as follows:

user $ mount ~/webdav
/sbin/mount.davfs: warning: the server does not support locks

(The ‘user‘ option in /etc/fstab allows the non-root user to mount the share.)

The main pane displaying the contents of ~/webdav/ will only be populated with the contents of the remote folder after the share is mounted.

The share is browsable in Dolphin. I can perform all file and folder operations in KDE apart from one thing: I cannot copy files to the server (neither from the local machine nor from the server); Dolphin displays messages such as ‘There is not enough space on the disk to write file:///home/fitz/testfile.txt’. I suspect the problem is with KDE, because I can copy files to and on the share by using the command line (for example the commands ‘cp ~/test1.txt ~/webdav/‘ and ‘cp ~/webdav/test2.txt ~/webdav/test3.txt‘ work fine). I have yet to find a solution to this issue, so I use Option 1 for Gentoo Linux running KDE, which works fine. To create a bookmark in Dolphin’s Places pane, browse the share and select ‘File’ > ‘Add to Places’.
 
PART 2 – WINDOWS 10

There is a Map Network Drive Wizard, but it is not as straightforward for WebDAV shares as it is with SMB shares. See the thread Cannot connect to webdav service for the type of behaviour I experienced, althought in my case I could rarely establish a connection using either ‘Map network drive’ or ‘Add a network location’, and the mapping was always lost if I logged out or rebooted, despite selecting ‘Reconnect at sign-in’. I then discovered several invalid URIs in Registry keys. Presumably these were left in the Registry after my various unsuccessful configuration attempts using the wizard. To finally succeed in mapping the ownCloud WebDAV shared folder I had to search for the string ‘fitzcarraldo.ddns.net’ in the Registry (see Steps 1 & 2 below for how to open the Registry) and delete any existing strings similar or identical to ‘https://fitzcarraldo.ddns.net/ownloud/remote.php/webdav‘, as they seemed to interfere with successful mapping of the network directory.

After making sure the Registry no longer contained any incorrect-looking WebDAV URIs for my ownCloud server, I used the following steps:

1. Right-click on Windows’ Start Menu icon on the left of the Task Bar and select ‘Run’.
2. Enter ‘regedit’ in the Open box and click ‘OK’.
3. Select Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters
4. If the value in BasicAuthLevel is not already 2, change it to 2.
5. In the ‘Type here to search’ box on the Task Bar, enter ‘Services’ and press Enter.
6. Click ‘Services App’.
7. Scroll down to ‘WebClient’ in the Services window.
8. Right-click ‘WebClient’ and select ‘Properties’.
9. If ‘Startup type’ is not already set to ‘Automatic’, change it to ‘Automatic’ and click ‘Apply’.
10. Launch File Explorer.
11. Right-click ‘This PC’ and select ‘Map network drive…’.
12. Select the drive letter (default is Z:).
13. In the Folder box enter \\fitzcarraldo.ddns.net@SSL\owncloud\remote.php\webdav and make sure only ‘Reconnect at sign-in’ is ticked.
14. Click ‘Finish’.
15. A network icon and the label ‘webdav (\\fitzcarraldo.ddns.net@SSL\owncloud\remote.php) (Z:)’ should appear under ‘My PC’. Clicking that icon displays the contents of the shared folder of my ownCloud account on my server.

The only Registry entries containing ‘fitzcarraldo.ddns.net’ found by ‘Edit’ > ‘Find…’ are now the following:

Computer\HKEY_CURRENT_USER\Network\Z
RemotePath     REG_SZ     \\fitzcarraldo.ddns.net@SSL\owncloud\remote.php\webdav

Computer\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Map Network Drive MRU
a     REG_SZ     \\fitzcarraldo.ddns.net@SSL\owncloud\remote.php\webdav

Computer\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\##fitzcarraldo.ddns.net@SSL#owncloud#remote.php#webdav
LabelFromReg     REG_SZ     webdav (\\fitzcarraldo.ddns.net@SSL\owncloud\remote.php)

Computer\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\PublishingWizard\AddNetworkPlace\AddNetPlace\LocationMRU
a     REG_SZ     https://fitzcarraldo.ddns.net/owncloud/remote.php/webdav

Computer\HKEY_USERS\S-1-5-21-4039722433-590489090-552845671-1001\Network\Z
RemotePath     REG_SZ     \\fitzcarraldo.ddns.net@SSL\owncloud\remote.php\webdav

Computer\HKEY_USERS\S-1-5-21-4039722433-590489090-552845671-1001\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Map Network Drive MRU
a     REG_SZ     \\fitzcarraldo.ddns.net@SSL\owncloud\remote.php\webdav

Computer\HKEY_USERS\S-1-5-21-4039722433-590489090-552845671-1001\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\##fitzcarraldo.ddns.net@SSL#owncloud#remote.php#webdav
LabelFromReg     REG_SZ     webdav (\\fitzcarraldo.ddns.net@SSL\owncloud\remote.php)

Computer\HKEY_USERS\S-1-5-21-4039722433-590489090-552845671-1001\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\PublishingWizard\AddNetworkPlace\AddNetPlace\LocationMRU
a     REG_SZ     https://fitzcarraldo.ddns.net/owncloud/remote.php/webdav

 
CONCLUSION

There you have it. I can browse my ownCloud user account folders on my server from my machines running Linux and from my test machine running Windows 10.

Installing and configuring davfs2 in Linux, and using Option 1 to browse a WebDAV share is very easy in both Gentoo Linux running KDE and in Lubuntu 20.10. Using Option 2 is also very easy in Lubuntu 20.10 but is not easy in Gentoo Linux running KDE, and I still need to find out if there is a better approach for Option 2 in Gentoo Linux running KDE.

I found Windows 10 the most problematic, despite the apparent simplicity of the ‘Map network drive’ and ‘Add a network location’ wizards. I discovered that, if I didn’t get the format of the URI correct the first time, Windows 10 would leave ‘cruft’ in the Registry that apparently prevented further mapping attempts from working properly and consistently.

Anyway, everything works the way I want and I hope this post is of some help to others wanting to browse a share using WebDAV, be that a folder in ownCloud, Nextcloud or any other network service requiring the WebDAV protocol.

Removing PipeWire in Gentoo Linux

PipeWire, all the rage these days, was originally developed for video but was later enhanced to support audio as well, and is now an alternative to PulseAudio and JACK. My laptop running Gentoo Stable (amd64) with the KDE Plasma Desktop had been working fine with PipeWire for some time. The pulseaudio and screencast USE flags were both declared in the file /etc/portage/make.conf. Both audio playback and recording worked fine until a recent upgrade of the packages in my world file, when neither worked any more. The Audio Volume loudspeaker icon (the applet kde-plasma/plasma-pa) on the KDE Plasma panel had a red line through it, and the KMix loudspeaker icon (the applet kde-apps/kmix) on the panel was greyed out. Although I cannot be sure, I suspect the problem started when the first version of PipeWire that supported audio was released. The output of the command ‘ps -ef | grep pulse‘ showed me that both PulseAudio and PipeWire were running. At the time I did not know that PulseAudio is not supposed to be running at the same time as PipeWire. Sometimes when I booted the laptop and logged in, the loudspeaker icons on the Panel would appear correctly and audio output would work properly, but usually this was not the case. This behaviour made me wonder if there was some sort of race condition between the two applications at startup.

Anyway, I stopped PulseAudio being launched automatically at startup. I did this by editing the file /etc/pulse/client.conf to add the line ‘autospawn = no‘ (a comment in the as-installed file indicates that the default value for autospawn is ‘yes‘). That did indeed stop PulseAudio from being launched automatically, and left only PipeWire running. The loudspeaker icons were then displayed correctly on the Panel when I logged in to the KDE Plasma Desktop, and audio output then worked. However, PipeWire did not detect the laptop’s built-in microphone, and no Recording channel was displayed by KMix and Audio Volume. The troubleshooting chapter of the Arch Linux Wiki article on PipeWire has a section suggesting a couple of fixes for this problem (Microphone is not detected by PipeWire) but, even so, I decided to ditch PipeWire and revert to PulseAudio. As much as I dislike PulseAudio (see some of my previous posts on the various problems I have experienced with it), these days it is more or less stable on this laptop and I do not have to mess around too much with audio settings.

A few KDE packages in Gentoo Linux depend on PipeWire (they require the screencast USE flag to be set). I therefore added the following two entries to a file in the directory /etc/portage/package.use/ in order to stop PipeWire being required:

>=sys-apps/xdg-desktop-portal-1.8.1 -screencast
>=kde-apps/krfb-20.12.3 -wayland

I was then able to use the usual command ‘emerge -uvDN @world‘, followed by the command ‘emerge --ask --depclean‘, to rebuild the affected packages and remove PipeWire. I also deleted the line ‘autospawn = no‘ that I had previously added to the file /etc/pulse/client.conf, so that PulseAudio would again be launched automatically at startup. Audio playback and recording are now back to normal. I will probably try PipeWire again in the future but, for the moment, I don’t need it. According to the Gentoo Linux Wiki article on PipeWire:

Warning
As of mid 2021, PipeWire is still in active development and not everything is fully integrated, tested, or implemented – though the project is moving along. While replacing existing audio solutions on Gentoo is possible, the experience is currently not guaranteed to be perfect or free of issues and bugs.

I will therefore wait until the concensus amongst Gentoo Linux users is that PipeWire is trouble-free before I try it again.

croc – another file transfer method

I have lost count of the number of times I have had to send a large file to someone at work, usually in a hurry. I’ve used Dropbox, ownCloud, Firefox Send (no longer available) etc. Transferring large files became a bit easier when e-mail service providers increased the size limit for attachments, but that is still not a solution for very large files. The xkcd cartoon FILE TRANSFER sums up the situation nicely.

I recently discovered the command line utility croc, which the author claims is a way to ‘easily and securely transfer stuff from one computer to another.’ I thought I’d give it a try, if only to have another tool to fall back on in an emergency. It does rely on both ends having croc installed, but hopefully that should not be a show-stopper as croc is available for Linux, Windows, macOS and BSD. To quote the author:

croc differs from a utility like scp because it doesn’t require any two computers to have enabled port-forwarding. Instead, croc will uses a relay – a temporary server setup locally (if both computers are on lan) or publicly (default is at croc4.schollz.com). Any two computers can connect to the relay, and after securing their channel with PAKE [password authenticated key exchange], they can transfer encrypted metadata and data through the relay. The relay works by first having the computers communicate the PAKE protocol via websockets, and then exchanging encrypted metadata, and then stapling the TCP connections directly so that they can transfer directly.

So, to use croc you will be dependent on the public relay provided by the author unless you set up your own relay (instructions are provided in the author’s original 2018 blog post introducing croc – see link above – and in various third-party articles about croc, such as ‘Securely Transfer Files and Folders Between Computers Using Croc‘ and ‘Transfer Files And Folders Between Computers With Croc‘).

Anyway, I installed croc in Lubuntu and Gentoo Linux from the author’s GitHub repository and indeed it is easy to use and works fine. The binary releases for the various OSs and Linux distributions can be found on the Releases page of the GitHub repository or via the OS package manager.

Lubuntu 20.10:

user $ wget https://github.com/schollz/croc/releases/download/v9.1.6/croc_9.1.6_Linux-64bit.deb
user $ sudo dpkg -i croc_9.1.6_Linux-64bit.deb

Gentoo Linux:

root # emerge net-misc/croc

(Note that croc ebuilds are not currently marked as Stable in the Gentoo Linux Portage tree, so you’ll have to unmask them by keyword if you are using the Stable branch.)

Termux:

I even installed croc in Termux on my Samsung Galaxy Note 20 Ultra 5G, and it works in Android too:

$ pkg install croc

Other OSs and other Linux distributions:

See the instructions in the README file online.

Using croc

Using croc is as simple as entering a command on one computer, informing (via e-mail, telephone, SMS, Signal or other social media) the person using the other computer of the command to use, and entering that command on the other computer. For example:

Sender

user $ croc send Documents/flight-times.ods
Sending 'flight-times.ods' (16.6 kB)
Code is: 8878-salary-courage-roger
On the other computer run

croc 8878-salary-courage-roger

Receiver

user $ croc 8878-salary-courage-roger
Accept 'flight-times.ods' (16.6 kB)? (Y/n) 

If the receiving user then enters ‘Y’, the sending user sees something similar to this:

user $ croc send Documents/flight-times.ods
Sending 'flight-times.ods' (16.6 kB)
Code is: 8878-salary-courage-roger
On the other computer run

croc 8878-salary-courage-roger

Sending (->192.168.1.74:60740)
 100% |████████████████████| (17/17 kB, 10.918 MB/s)
user $ 

and the receiving user sees something similar to this:

user $ croc 8878-salary-courage-roger
Accept 'flight-times.ods' (16.6 kB)? (Y/n) Y

Receiving (<-[::1]:39442)
 100% |████████████████████| (17/17 kB, 3.989 MB/s)
user $ 

The observant reader will notice that the above example shows a file being transferred on the same computer. When transferred between different computers the IP addresses of each computer will be displayed instead. I have used croc to transfer files between different computers on my home network (I would normally just use my NAS for this, though), between remote computers on the Internet, and between my computers and my phone via mobile broadband, and croc works in all cases.

I have not mentioned all croc’s features. I’ll leave you to read up on croc in more detail in the links I’ve given above. It looks like it might be a useful tool to have installed.

Using adb tools in Linux to remove bloatware from my Samsung Galaxy Note 20 Ultra

Samsung included a lot of bloatware on my Galaxy Note 20 Ultra 5G, and it is not possible to uninstall it using Play Store. However, it is possible to remove this stuff using adb tools. I got rid of the bloatware I don’t want very easily using the Linux version of the adb tools.

I have never had a Facebook account and never will, so I decided to remove all trace of it as follows:

1. Installed adb tools

In Lubuntu 20.10:

user $ sudo apt install android-tools-adb

In Gentoo Linux:

root # emerge dev-util/android-tools

2. Enabled ‘Developer Options’ on the phone

‘Settings’ > ‘About Phone’ > ‘Software Information’ and quickly tapped 7 times on ‘Build number’.

3. Enabled USB Debugging on the phone

‘Settings’ > ‘Developer options’, scrolled down and tapped on ‘USB debugging’.

4. Launched adb

user $ adb start-server
* daemon not running; starting now at tcp:5037
* daemon started successfully

5. Connected the phone to the computer using the USB cable

A few prompts on the phone asked whether or not I wanted to allow USB debugging. Tapped ‘Always allow from this computer’ and tapped ‘OK’.

6. Uninstalled Facebook

The packages I needed to uninstall were:

com.facebook.appmanager
com.facebook.katana
com.facebook.services
com.facebook.system

First I tried to uninstall with the ‘-k‘ option:

user $ adb uninstall -k --user 0 com.facebook.appmanager
The -k option uninstalls the application while retaining the data/cache.
At the moment, there is no way to remove the remaining data.
You will have to reinstall the application with the same signature, and fully uninstall it.
If you truly wish to continue, execute 'adb shell cmd package uninstall -k'.

See ‘Difference between pm clear and pm uninstall -k on Android‘

I have never been a member of Facebook and never will, so I dispensed with the ‘-k‘ option and entered the following commands:

user $ adb uninstall --user 0 com.facebook.appmanager
Success
user $ adb uninstall --user 0 com.facebook.katana
Success
user $ adb uninstall --user 0 com.facebook.services
Success
user $ adb uninstall --user 0 com.facebook.system
Success

I didn’t want the LinkedIn, Samsung Global Goals and Spotify apps either, so I uninstalled those too:

user $ adb uninstall --user 0 com.linkedin.android
Success
user $ adb uninstall --user 0 com.samsung.sree
Success
user $ adb uninstall --user 0 com.spotify.music
Success

7. Stopped the adb server on the computer

user $ adb kill-server

8. Unplugged the phone from the computer.

That’s it.

In order to disable the apps using this method, you will need to know the exact package name of the app you want to get rid of. For this, use Play Store and install App Inspector (there are several apps with this name in Play Store; I installed the app by Projectoria Ltd but the others look OK too). Launch App Inspector and you can find the package name under the name of the app. This starts with a ‘com‘ or ‘net‘ followed by words separated by dots.

For example, App Inspector shows the package name for LinkedIn as ‘com.linkedin.android‘.

Some useful links:

• How to install and setup ADB tools on Linux
• How To Remove Bloatware on Samsung Galaxy Note 20 and Improve Performance
• WARNING: Do *NOT* delete all the apps in this list
• Disable Bloatware on Your Galaxy Note 20 for Free Without Root
• How to Remove Samsung Bloatware without Root (ADB)

To get a list of all the packages installed on my phone:

user $ adb shell pm list packages

To get a list of system apps only:

user $ adb shell pm list packages -s

To get a list of only Samsung packages:

user $ adb shell pm list packages | grep samsung

To search for e.g. facebook packages:

user $ adb shell pm list packages | grep facebook

(Returns nothing now, as I already deleted all the Facebook packages. Yay!)

To search for other packages, e.g.:

user $ adb shell pm list packages | grep kids
package:com.samsung.android.kidsinstaller
package:com.sec.android.app.kidshome

Is Gentoo Linux an anachronism?

When I started visiting the Gentoo Linux discussion forums in 2007 there were at least three pages of posts daily, if not more. These days there is usually one page. I’m sure the number of Gentoo Linux users has dropped significantly since then. Interest in the distribution has certainly decreased since its heyday: Google Trends – gentoo linux.

I don’t think the drop in interest is limited to individuals either. Articles such as ‘Flying Circus Internet Operations GmbH – Migrating a Hosting Infrastructure from Gentoo to NixOS‘ lead me to suspect that some companies have switched to other distributions over the years. NASDAQ’s use of ‘a modified version of Gentoo Linux’ was publicised in 2011 (How Linux Mastered Wall Street) but I do not know if it still uses the distribution and, in any case, that is only a single significant entity. I personally have never come across another user (corporation or individual) of Gentoo Linux, although I do know several companies and individuals using distributions such as Ubuntu and Fedora.

Gentoo Linux is certainly not for everyone. In recent years the user base seems to have settled down to a smaller number of people, primarily consisting of enthusiasts who appreciate its advanced features and are prepared to put in the extra effort and time required to create and maintain a working installation. I’m sure it also still has a place in some specialised commercial applications, but I have my doubts its deployment comes anywhere near that of the major distributions such as Ubuntu, Red Hat, Fedora, etc. If I were only interested in using an OS that enabled me to perform typical personal and professional tasks, I wouldn’t be using Gentoo Linux. Some people touted Gentoo Linux’s configurability as giving it a speed advantage over binary distributions but, having correctly installed and used Gentoo Linux and various other distributions on the same hardware, I cannot say I noticed an improvement in performance.

I think one has to choose the right tool for the job. I wouldn’t dream of installing Gentoo Linux on any of my family’s machines or on older hardware. Personal experience doing the latter has taught me it is a waste of time (both installation itself and subsequent maintenance). I installed Lubuntu on my family’s desktop machine because it is a reliable, low-maintenance OS with automatic update notifications and painless, fast updates. On the other hand I installed Gentoo Linux on my laptops because I want to tinker with the OS, configure it exactly the way I want, be able to apply patches to source code easily, install multiple versions of the same application (‘slotted applications’), learn more about how the OS works, and experiment. You can do that too with a binary distribution, but with Gentoo Linux I feel I know a lot more about the kernel, OS internals, package management and package customisation than with a pre-canned binary distribution. It really is good for learning about Linux in more depth than a binary distribution.

My old Compal NBLB2 laptop with a first-generation Intel Core i7 CPU is eleven years old and I have never touched the package installation log file /var/log/emerge.log since installation in March 2010. Ten years after I first installed Gentoo Linux on it I ran the command ‘qlop -c -H‘ out of curiosity to see how much time had been spent building packages over its lifetime. The reported statistics were as follows:

total: 492 days, 5 hours, 47 minutes, 44 seconds for 67841 merges, 4295 unmerges, 446 syncs

That’s roughly 13% of its then 124 months ‘life’ spent compiling.

It has an Intel Core i7 720QM CPU (1.6 GHz but throttled to 933 MHz by Compal due to Compal’s PSU size, although I bought a higher Wattage PSU a year ago and it seems to run at 1.6 GHz since then). It has always had KDE installed, and numerous upgrades to KDE have kept it busy compiling. Each version of LibreOffice, qtwebengine, Firefox etc. has also taken a very long time to compile. Until I removed qtwebengine and the few packages dependent on it this year, even with jumbo-build enabled qtwebengine took more than a day to build. Admittedly I did have trouble some years ago with the HDD becoming almost full with temporary directories and files over a long period of time (/usr/tmp/portage/ contained a whopping 30GB of directories and files until I cleared it out), which also slowed things down, but that is no longer the case. Unfortunately that laptop has ~amd64 (Gentoo Linux Testing) installed rather than amd64 (Gentoo Linux Stable), so it’s not possible to install the binary package of LibreOffice due to dependency conflicts. As all the big packages take so long to compile on this particular laptop I ended up merging the firefox-bin package rather than the firefox source code package, and I use Microsoft Office 2007 running under WINE rather than LibreOffice.

My Clevo W230SS laptop (fourth-generation Intel Core i7-4810MQ CPU @ 2.80GHz) running Gentoo amd64 (Gentoo Linux Stable) with a few ~amd64 (testing) packages is six years old and I have not touched /var/log/emerge.log since installation in April 2015. Five years after I first installed Gentoo Linux on it I ran the command ‘qlop -c -H‘ to see how it compared to the older Compal NBLB2 laptop running Gentoo ~amd64 mentioned above. The reported statistics were as follows:

total: 53 days, 11 hours, 3 minutes, 31 seconds for 24494 merges, 1717 unmerges, 169 syncs

That’s roughly 3% of its then 64 months ‘life’ spent compiling. Nowhere near as bad as my older laptop, but still a lot of time spent compiling. The merge time for qtwebengine 5.14.2 was 4 hours 25 minutes with that fourth-generation Intel Core i7 CPU, but later versions of qtwebengine take even longer to build.

I personally would now only consider installing Gentoo Linux on a machine with at least 16 GB RAM and a CPU with at least four cores and a speed of circa 3 GHz or more. Additionally, although I have been a user of KDE in Gentoo Linux all these years, I would probably switch from KDE to a simpler, less resource-hungry and less feature-rich (some might say less ‘bloated’!) desktop environment such as LXQt in new installations of Gentoo Linux.

One thing that has improved a lot since I started using Gentoo Linux over a decade ago is the package manager Portage, at least in terms of dependency resolution and blockage handling. I used to have to do a lot more work to resolve problems during package upgrades; ‘merging world’ (upgrading installed packages) is generally a lot less troublesome than it used to be ten years ago. Portage is a lot slower than it used to be, but that’s because it does a lot more than it used to do. I used to have to use revdep-rebuild – a utility to resolve reverse dependencies and rebuild affected packages – frequently, but not any more. Building software from source code takes time, though, so plenty of RAM and a fast CPU are important for installing packages, however good the package manager itself.

Some people maintain that the reduction in posts in the Gentoo Linux Forums could just mean users have fewer problems these days compared to earlier years. However, I have my doubts that would account for the much larger number of pages of ‘posts from the last 24 hours’ in earlier years, nor for the big drop in Google Trends statistics since 2004. Posts from new users do appear from time to time in the forums, so I suspect there are simply not as many new users as a decade or more ago. There are also posts from long-time users when there are major changes such as an upgrade to a newer version of Python or a profile change.

Another argument against a drop in popularity is that many of the users in the high number of users online in a 24-hour period in earlier years were spambots. I used to be a moderator of the Sabayon Linux forums, so I’m well aware of the phenomenon and I had to ban quite a few spammers & spambots in my time. But I’m not buying for one moment that the majority or even a significant number of the 1850 users logged in to the Gentoo Forums on 30 December 2004 were spambots. I am aware of puerile, more-recent attempts by a few lone individuals to boost the distribution’s exposure, such as ‘Gentoo Linux Forums – I’ve just set up Gentoo on distrowatch as my homepage‘ but I doubt very much that has had any impact on uptake. Mind you, such antics are not confined to Gentoo Linux; I’ve seen similar posts in the forums of some other distributions.

I think former Gentoo Linux developer and Council member Donnie Berkholz got it about right in his 2013 article Ranking Linux distributions, and the decline of the traditional distros.

A discussion on Reddit in 2016 indicated that other Linux users have noticed a decline in use of the distribution: Why did Gentoo peak in popularity in 2005, then fade into obscurity?.

The decline in use of Gentoo Linux is not just due to lower uptake by new users; veteran users have also moved away due to its demands on time and effort: ‘Au Revoir, Gentoo – Sell Me A New Linux Distro‘. There are occasionally posts in the Gentoo Linux forums by previous users announcing that they have started using the distribution again, but I strongly suspect they are exceptions to the general trend.

Gentoo Linux is not as popular as it used to be, and there is no way of dressing it up any other way. However, Gentoo Linux can still be worthwhile for the Linux enthusiast and ‘power user’ who enjoys tinkering and learning more about Linux internals, and who does not mind the significant additional time required to maintain it, and the time, effort and extra energy consumption required to compile packages. But I would not recommend Gentoo Linux if you just want a Linux installation in order to perform typical desktop tasks such as browsing the Web, sending e-mails, word processing, working on spreadsheets and so on.

Hardware has become much more powerful since Gentoo Linux’s heyday, and drivers have improved significantly (I shudder to think of the time I spent years ago getting Linux to work with some devices), making the optimisation and lower-level tinkering that Gentoo Linux facilitates less of a necessity. Furthermore, binary distributions have improved noticeably over the years, becoming easier to install, more user-friendly, easier to maintain, more reliable and better-looking. The improvements in binary distributions have, in my opinion, also contributed to the drift away from Gentoo Linux.

Nevertheless, I believe Gentoo Linux will not disappear; it is rather unique and there will always be people who enjoy the challenge of developing it and/or using it rather than a binary distribution. Furthermore, the additional control Gentoo Linux offers those who are prepared to put in the extra time and effort to use it, plus its high degree of ‘customisability’, make it attractive to certain users or for certain specialist applications. Then there are those who simply prefer not to follow the mainstream and want to try something different. I certainly hope Gentoo Linux continues long into the future and manages to maintain its distinctiveness, including the ease in not using systemd if the user so choses. Using OpenRC – which has never caused me a problem in over a decade – instead of systemd has become increasingly difficult for many Gentoo Linux users because upstream software is increasingly being written specifically to use systemd and would require significant effort to patch (KDE Control Module Plasma Firewall being a recent example). Portage is an excellent, powerful package manager, as is the accompanying suite of tools, and I don’t think there is anything that can beat that (probably one of the reasons the developers of Google Chrome OS opted to use Portage). Now, if only someone could release a machine an average home user could afford that could compile source-code packages such as qtwebengine, LibreOffice and Firefox in, say, one minute, perhaps Gentoo Linux’s popularity would increase! 😉 Until Moore’s Law results in manufacturers of home computers catching up with the build requirements of Gentoo Linux, the distribution will definitely remain a niche player. Personally, that does not bother me, although I must admit I am finding the time and effort to maintain my installations rather irksome these days.

How to patch kde-plasma/plasma-firewall-5.21.2 for UFW in Gentoo Linux with OpenRC

Unfortunately plasma-firewall-5.21.2, a new Plasma frontend for firewalld and UFW, has been written only for Linux installations with systemd. However, I use OpenRC and syslog-ng in Gentoo Linux and wanted to try to get plasma-firewall to work on my laptop which uses UFW. I therefore set about patching plasma-firewall-5.21.2. I did not touch the firewalld part of plasma-firewall, as I do not use firewalld (and the plasma-firewall code for firewalld is more complicated). Below is what I did.

root # wget https://invent.kde.org/plasma/plasma-firewall/-/archive/Plasma/5.21/plasma-firewall-Plasma-5.21.tar.gz
root # tar -xzf plasma-firewall-Plasma-5.21.tar.gz
root # cp -pr plasma-firewall-Plasma-5.21 a
root # cp -pr plasma-firewall-Plasma-5.21 b
root # nano b/kcm/backends/ufw/ufwclient.cpp # Apply changes shown in Part 1 below.
root # nano b/kcm/backends/ufw/helper/helper.cpp # Apply changes shown in Part 2 below.
root # nano /usr/bin/print_ufw_messages # Create Bash script shown in Part 2 below.
root # chmod 755 /usr/bin/print_ufw_messages
root # nano b/kcm/backends/ufw/ufwlogmodel.cpp # Apply changes shown in Part 3 below.
root # diff -ruN a b > plasma-firewall-5.21.2-ufw.patch
root # mkdir -p /etc/portage/patches/kde-plasma/plasma-firewall-5.21.2
root # cp plasma-firewall-5.21.2-ufw.patch /etc/portage/patches/kde-plasma/plasma-firewall-5.21.2/
root # emerge -1v plasma-firewall
root # nano /etc/syslog-ng/syslog-ng.conf # Apply changes shown in Part 4 below.

You should now be able to use plasma-firewall for UFW in KDE Plasma’s ‘System Settings’ > ‘Firewall’ in the Network section, although I have not tried all the functions. Additionally, I believe there may be some outstanding bugs in the original 5.21.2 version of the Plasma module when using it with systemd.

Part 1

In /kcm/backends/ufw/ufwclient.cpp change:

bool UfwClient::isCurrentlyLoaded() const
{
QProcess process;
const QString name = "systemctl";
const QStringList args = {"status", "ufw"};

process.start(name, args);
process.waitForFinished();

// systemctl returns 0 for status if the app is loaded, and 3 otherwise.
qDebug() << "Ufw is loaded?" << (process.exitCode() == EXIT_SUCCESS);

return process.exitCode() == EXIT_SUCCESS;
}

to:

bool UfwClient::isCurrentlyLoaded() const
{
QProcess process;
const QString name = "rc-service";
const QStringList args = {"--exists", "ufw"};

process.start(name, args);
process.waitForFinished();

// "rc-service --exists" returns 0 if the app is loaded, and -1 otherwise.
qDebug() << "Ufw is loaded?" << (process.exitCode() == EXIT_SUCCESS);

return process.exitCode() == EXIT_SUCCESS;
}

Part 2

In /kcm/backends/ufw/helper/helper.cpp change:

QStringList getLogFromSystemd(const QString &lastLine)
{
QString program = "journalctl";
QStringList arguments {"-xb","-n", "100","-g", "UFW"};

QProcess myProcess;
myProcess.start(program, arguments);
myProcess.waitForFinished();

auto resultString = QString(myProcess.readAllStandardOutput());
auto resultList = resultString.split("\n");

// Example Line from Systemd:
// Dec 06 17:42:45 tomatoland kernel: [UFW BLOCK] IN=wlan0 OUT= MAC= SRC=192.168.50.181 DST=224.0.0.252 LEN=56 TOS=0x00
//     PREC=0x00 TTL=255 ID=52151 PROTO=UDP SPT=5355 DPT=5355 LEN=36
// We need to remove everything up to the space after ']'.

QStringList result;
for(const QString& line : resultList) {
if (!lastLine.isEmpty() && line == lastLine) {
result.clear();
continue;
}
result.append(line);
}
return result;
}

to:

QStringList getLogFromSystemd(const QString &lastLine)
{
QString program = "print_ufw_messages";
QStringList arguments {"UFW", "100"};

QProcess myProcess;
myProcess.start(program, arguments);
myProcess.waitForFinished();

auto resultString = QString(myProcess.readAllStandardOutput());
auto resultList = resultString.split("\n");

// Example line from /var/log/messages populated by sylog-ng:
// Mar  6 00:10:19 localhost kernel: [UFW BLOCK] IN=wlan0 OUT= MAC=00:12:5b:8a:83:6d:b7:2a:da:59:d4:10:09:00 SRC=192.168.1.27
//      DST=192.168.1.139 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=41659 DF PROTO=TCP SPT=445 DPT=52140 WINDOW=260 RES=0x00 ACK URGP=0
// We need to remove everything up to the space after ']'.

QStringList result;
for(const QString& line : resultList) {
if (!lastLine.isEmpty() && line == lastLine) {
result.clear();
continue;
}
result.append(line);
}
return result;
}

where the program print_ufw_messages is a user-created Bash script /usr/bin/print_ufw_messages (-rwxr-xr-x root.root) containing:

#!/bin/bash
awk '{if (/localhost syslog-ng/ && /syslog-ng starting up/ && !/COMMAND/) {chunk=""} else {chunk=chunk $0 RS}} END {printf "%s", chunk}' /var/log/messages | grep "$1" | head -n "$2" | grep -v print_ufw_messages

Part 3

During my investigations into how to modify the plasma-firewall-5.21.2 source code, I discovered a bug in the source code. In /kcm/backends/ufw/ufwlogmodel.cpp change:

for (const QString& key : {"IN", "SRC", "DST", "PROTO", "STP", "DPT"}) {

to:

for (const QString& key : {"IN", "SRC", "DST", "PROTO", "SPT", "DPT"}) {

i.e. “STP” needs to be changed to “SPT“.

Part 4

I am not sure if this makes a difference to plasma-firewall (which was coded assuming systemd-journald is installed), but the default date format for messages in /var/log/messages printed by syslog-ng has only one digit in the day of the month when it is less than the 10th day of the month. For example:

Mar  9 03:09:39 clevow230ss syslog-ng[23735]:  syslog-ng starting up; version='3.30.1'

However, systemd-journalctl always outputs two-digit days of the month, and I think (but am not certain) the following date format might be needed in order for the existing code in /kcm/backends/ufw/ufwlogmodel.cpp to parse the syslog-ng output correctly:

Mar 09 03:09:39 clevow230ss syslog-ng[23735]:  syslog-ng starting up; version='3.30.1'

Therefore edit /etc/syslog-ng/syslog-ng.conf and add a template:

template template_date_format {
template("${MONTH_ABBREV} ${DAY} ${HOUR}:${MIN}:${SEC} ${HOST} ${MSGHDR}${MSG}\n");
template_escape(no);
};

and change the line:

destination messages { file("/var/log/messages"); };

to:

destination messages { file("/var/log/messages" template(template_date_format)); };

Then restart syslog-ng:

root # rc-service syslog-ng restart

From now on the day of the month is always two digits (01, 02,…31) in /var/log/messages.

Removing qtwebengine from a Gentoo Linux installation

At the beginning of March I updated the world set in Gentoo Testing (~amd64) running the KDE suite (Plasma, Frameworks and Applications) on my secondary laptop, an eleven-year-old Compal NBLB2. It has a first-generation Core i7 CPU and the maximum amount of RAM that can be installed in that model (8 GB).

root # uname -a
Linux meshedgedx 5.0.11-gentoo #1 SMP Fri Jun 7 15:33:06 BST 2019 x86_64 Intel(R) Core(TM) i7 CPU Q 720 @ 1.60GHz GenuineIntel GNU/Linux

Gentoo Linux being a source-based distribution, updates to the largest packages take hours to build on older machines. Actually, some packages can take hours to build on newer machines too. On this older laptop I therefore merge the www-client/firefox-bin binary package instead of the www-client/firefox source-code package, and have installed Microsoft Office 2007 running in WINE instead of trying to merge the app-office/libreoffice source-code package (app-office/libreoffice-bin cannot be merged in this Testing installation because of incompatibility with the versions of installed dependencies, so it would only be a viable alternative binary package in a Stable installation).

Possibly the worst source-code package to build is dev-qt/qtwebengine. Nowadays it takes a ridiculous amount of time to build on this laptop, even with the jumbo-build USE flag set and MAKEOPTS="-j4" or even MAKEOPTS="-j1". The latest merge on the laptop took more than 14 hours:

root # genlop -t qtwebengine | tail -n 3
     Fri Mar  5 02:02:07 2021 >>> dev-qt/qtwebengine-5.15.2_p20210224
       merge time: 14 hours, 14 minutes and 7 seconds.

That is actually quite fast for that laptop; qtwebengine has sometimes taken two days to merge in the past.

What a waste of time and electricity, not to mention the unnecessary wear on the laptop (fan bearing; prolonged heat on components; etc.).

This one package is such a hassle to merge that it had me wondering if I should switch from Gentoo Linux to a binary distribution. Even on my six-year-old Compal W230SS laptop with a fourth-generation Core i7 CPU and 16 GB of RAM, qtwebengine takes circa five hours to merge. After several years putting up with this scourge of source-based Linux distributions on my secondary laptop, I had finally had enough and decided to excise the package, which did not look like an easy task with the full KDE suite installed. This is how I did it…

1. First I made sure the installation was up-to-date (see my earlier post ‘My system upgrade procedure for Gentoo Linux‘ for the steps I normally use to update all packages to their latest versions).

2. I ascertained which packages depended on qtwebengine:

root # equery depends qtwebengine
 * These packages depend on qtwebengine:
kde-apps/kaccounts-providers-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5)
kde-apps/kalgebra-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-apps/kdenlive-20.12.2 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
kde-apps/kimagemapeditor-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-apps/ktp-text-ui-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-apps/marble-20.12.2 (webengine ? >=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-apps/parley-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-plasma/kdeplasma-addons-5.21.1 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
kde-plasma/libksysguard-5.21.1 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
net-libs/signon-ui-0.15_p20171022-r1 (dev-qt/qtwebengine:5)
net-p2p/ktorrent-20.12.2 (rss ? >=dev-qt/qtwebengine-5.15.2:5)
                         (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
www-client/falkon-3.1.0-r1 (>=dev-qt/qtwebengine-5.12.3:5[widgets])

3. I disabled the USE flag ‘webengine‘ globally:

root # nano /etc/portage/make.conf # Add -webengine to the list of USE flags

4. I merged the world set in order to incorporate the USE flag change:

root # emerge -uvDN @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] kde-apps/marble-20.12.2:5/20.12::gentoo  USE="dbus geolocation kde nls pbf phonon -aprs -debug -designer -gps -handbook -shapefile -test -webengine*" 0 KiB
[ebuild   R    ] kde-apps/kdeedu-meta-20.12.2:5::gentoo  USE="-webengine*" 0 KiB
[ebuild   R    ] kde-apps/kdecore-meta-20.12.2:5::gentoo  USE="share thumbnail -handbook -webengine*" 0 KiB
[ebuild   R    ] net-p2p/ktorrent-20.12.2:5::gentoo  USE="bwscheduler downloadorder infowidget ipfilter kross logviewer magnetgenerator mediaplayer rss scanfolder shutdown stats upnp zeroconf -debug -handbook -test -webengine*" 0 KiB
[ebuild   R    ] kde-apps/kdenetwork-meta-20.12.2:5::gentoo  USE="bittorrent -dropbox -webengine*" 0 KiB
[ebuild   R    ] kde-apps/kdeutils-meta-20.12.2:5::gentoo  USE="cups rar -7zip -floppy -gpg -lrz -webengine*" 0 KiB

Total: 6 packages (6 reinstalls), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 6) kde-apps/marble-20.12.2::gentoo
>>> Emerging (2 of 6) kde-apps/kdecore-meta-20.12.2::gentoo
>>> Emerging (3 of 6) net-p2p/ktorrent-20.12.2::gentoo
>>> Emerging (4 of 6) kde-apps/kdeutils-meta-20.12.2::gentoo
>>> Installing (2 of 6) kde-apps/kdecore-meta-20.12.2::gentoo
>>> Installing (4 of 6) kde-apps/kdeutils-meta-20.12.2::gentoo
>>> Installing (3 of 6) net-p2p/ktorrent-20.12.2::gentoo
>>> Emerging (5 of 6) kde-apps/kdenetwork-meta-20.12.2::gentoo
>>> Installing (5 of 6) kde-apps/kdenetwork-meta-20.12.2::gentoo
>>> Installing (1 of 6) kde-apps/marble-20.12.2::gentoo
>>> Emerging (6 of 6) kde-apps/kdeedu-meta-20.12.2::gentoo
>>> Installing (6 of 6) kde-apps/kdeedu-meta-20.12.2::gentoo
>>> Jobs: 6 of 6 complete                           Load avg: 1.93, 3.62, 3.86
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.
 * After world updates, it is important to remove obsolete packages with
 * emerge --depclean. Refer to `man emerge` for more information.

5. I uninstalled packages that were no longer required by any other packages and also not required by me (I do not use the Falkon browser, Telepathy and KAlgebra, to give a few examples, and so did not mind various specific packages being removed):

root # emerge --ask --depclean

 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace `.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 * 
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence of this, it often becomes necessary to run 
 * `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:                                                                                                                                                                                                

 kde-apps/parley
    selected: 20.12.2 
   protected: none 
     omitted: none 

 www-client/falkon
    selected: 3.1.0-r1 
   protected: none 
     omitted: none 

 kde-apps/kimagemapeditor
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/plasma-telepathy-meta
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/kalgebra
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-kded-module
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-desktop-applets
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-accounts-kcm
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-send-file
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-approver
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-auth-handler
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-contact-runner
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-text-ui
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/signon-kwallet-extension
    selected: 20.12.2 
   protected: none 
     omitted: none 

 net-im/telepathy-connection-managers
    selected: 2-r2 
   protected: none 
     omitted: none 

 kde-apps/ktp-filetransfer-handler
    selected: 20.12.2 
   protected: none 
     omitted: none 

 kde-apps/ktp-contact-list
    selected: 20.12.2 
   protected: none 
     omitted: none 

 net-irc/telepathy-idle
    selected: 0.2.0-r3 
   protected: none 
     omitted: none 

 net-voip/telepathy-salut
    selected: 0.8.1-r3 
   protected: none 
     omitted: none 

 net-voip/telepathy-gabble
    selected: 0.18.4-r2 
   protected: none 
     omitted: none 

 kde-apps/ktp-common-internals
    selected: 20.12.2 
   protected: none 
     omitted: none 

 net-libs/telepathy-accounts-signon
    selected: 2.1 
   protected: none 
     omitted: none 

 net-libs/libnice
    selected: 0.1.15 
   protected: none 
     omitted: none 

 net-libs/telepathy-logger-qt
    selected: 17.09.0 
   protected: none 
     omitted: none 

 net-im/telepathy-logger
    selected: 0.8.2-r1 
   protected: none 
     omitted: none 

 net-libs/gupnp-igd
    selected: 0.2.5-r10 
   protected: none 
     omitted: none 

 net-libs/libsignon-glib
    selected: 2.1 
   protected: none 
     omitted: none 

 net-libs/telepathy-qt
    selected: 0.9.8 
   protected: none 
     omitted: none 

 net-libs/gupnp
    selected: 1.2.4 
   protected: none 
     omitted: none 

 net-libs/gssdp
    selected: 1.2.3 
   protected: none 
     omitted: none 

 net-libs/libsoup
    selected: 2.70.0 
   protected: none 
     omitted: none 

 net-libs/libpsl
    selected: 0.21.1 
   protected: none 
     omitted: none 

 net-libs/glib-networking
    selected: 2.66.0 
   protected: none 
     omitted: none 

 net-im/telepathy-mission-control
    selected: 5.16.5 
   protected: none 
     omitted: none 

 net-libs/telepathy-glib
    selected: 0.24.1-r1 
   protected: none 
     omitted: none 

All selected packages: =kde-apps/ktp-desktop-applets-20.12.2 =kde-apps/ktp-contact-runner-20.12.2 =kde-apps/ktp-contact-list-20.12.2 =net-libs/telepathy-accounts-signon-2.1 =net-libs/telepathy-glib-0.24.1-r1 =net-voip/telepathy-salut-0.8.1-r3 =kde-apps/ktp-text-ui-20.12.2 =net-libs/libsignon-glib-2.1 =net-im/telepathy-connection-managers-2-r2 =kde-apps/ktp-accounts-kcm-20.12.2 =kde-apps/kimagemapeditor-20.12.2 =kde-apps/ktp-common-internals-20.12.2 =kde-apps/parley-20.12.2 =net-libs/libnice-0.1.15 =net-libs/libsoup-2.70.0 =kde-apps/ktp-auth-handler-20.12.2 =net-libs/gssdp-1.2.3 =net-irc/telepathy-idle-0.2.0-r3 =net-libs/libpsl-0.21.1 =kde-apps/kalgebra-20.12.2 =net-libs/gupnp-igd-0.2.5-r10 =kde-apps/ktp-filetransfer-handler-20.12.2 =kde-apps/ktp-send-file-20.12.2 =net-libs/gupnp-1.2.4 =kde-apps/ktp-kded-module-20.12.2 =net-im/telepathy-mission-control-5.16.5 =kde-apps/plasma-telepathy-meta-20.12.2 =net-voip/telepathy-gabble-0.18.4-r2 =net-im/telepathy-logger-0.8.2-r1 =kde-apps/signon-kwallet-extension-20.12.2 =net-libs/telepathy-logger-qt-17.09.0 =net-libs/telepathy-qt-0.9.8 =net-libs/glib-networking-2.66.0 =kde-apps/ktp-approver-20.12.2 =www-client/falkon-3.1.0-r1

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] Yes 
>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 35) kde-apps/parley-20.12.2...
>>> Unmerging (2 of 35) www-client/falkon-3.1.0-r1...
>>> Unmerging (3 of 35) kde-apps/kimagemapeditor-20.12.2...
>>> Unmerging (4 of 35) kde-apps/plasma-telepathy-meta-20.12.2...
>>> Unmerging (5 of 35) kde-apps/kalgebra-20.12.2...
>>> Unmerging (6 of 35) kde-apps/ktp-kded-module-20.12.2...
>>> Unmerging (7 of 35) kde-apps/ktp-desktop-applets-20.12.2...
>>> Unmerging (8 of 35) kde-apps/ktp-accounts-kcm-20.12.2...
>>> Unmerging (9 of 35) kde-apps/ktp-send-file-20.12.2...
>>> Unmerging (10 of 35) kde-apps/ktp-approver-20.12.2...
>>> Unmerging (11 of 35) kde-apps/ktp-auth-handler-20.12.2...
>>> Unmerging (12 of 35) kde-apps/ktp-contact-runner-20.12.2...
>>> Unmerging (13 of 35) kde-apps/ktp-text-ui-20.12.2...
>>> Unmerging (14 of 35) kde-apps/signon-kwallet-extension-20.12.2...
>>> Unmerging (15 of 35) net-im/telepathy-connection-managers-2-r2...
>>> Unmerging (16 of 35) kde-apps/ktp-filetransfer-handler-20.12.2...
>>> Unmerging (17 of 35) kde-apps/ktp-contact-list-20.12.2...
>>> Unmerging (18 of 35) net-irc/telepathy-idle-0.2.0-r3...
>>> Unmerging (19 of 35) net-voip/telepathy-salut-0.8.1-r3...
>>> Unmerging (20 of 35) net-voip/telepathy-gabble-0.18.4-r2...
>>> Unmerging (21 of 35) kde-apps/ktp-common-internals-20.12.2...
>>> Unmerging (22 of 35) net-libs/telepathy-accounts-signon-2.1...
>>> Unmerging (23 of 35) net-libs/libnice-0.1.15...
>>> Unmerging (24 of 35) net-libs/telepathy-logger-qt-17.09.0...
>>> Unmerging (25 of 35) net-im/telepathy-logger-0.8.2-r1...
>>> Unmerging (26 of 35) net-libs/gupnp-igd-0.2.5-r10...
>>> Unmerging (27 of 35) net-libs/libsignon-glib-2.1...
>>> Unmerging (28 of 35) net-libs/telepathy-qt-0.9.8...
>>> Unmerging (29 of 35) net-libs/gupnp-1.2.4...
>>> Unmerging (30 of 35) net-libs/gssdp-1.2.3...
>>> Unmerging (31 of 35) net-libs/libsoup-2.70.0...
>>> Unmerging (32 of 35) net-libs/libpsl-0.21.1...
>>> Unmerging (33 of 35) net-libs/glib-networking-2.66.0...
>>> Unmerging (34 of 35) net-im/telepathy-mission-control-5.16.5...
>>> Unmerging (35 of 35) net-libs/telepathy-glib-0.24.1-r1...
Packages installed:   1651
Packages in world:    329
Packages in system:   43
Required packages:    1651
Number removed:       35

 * GNU info directory index is up-to-date.

Notice that the package qtwebengine had not been removed, so something still depended on it.

6. I checked if there were any packages still installed with a dependency on qtwebengine:

root # equery depends qtwebengine
 * These packages depend on qtwebengine:
kde-apps/kaccounts-providers-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5)
kde-apps/kdenlive-20.12.2 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
kde-apps/marble-20.12.2 (webengine ? >=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-plasma/kdeplasma-addons-5.21.1 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
kde-plasma/libksysguard-5.21.1 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
net-libs/signon-ui-0.15_p20171022-r1 (dev-qt/qtwebengine:5)
net-p2p/ktorrent-20.12.2 (rss ? >=dev-qt/qtwebengine-5.15.2:5)
                         (webengine ? >=dev-qt/qtwebengine-5.15.2:5)

As can be seen from the above output, the only remaining installed packages that ‘hard-depended’ on the ‘webengine‘ USE flag were kde-apps/kaccounts-providers-20.12.2 and net-libs/signon-ui-0.15_p20171022-r1.

Additionally, the package net-p2p/ktorrent-20.12.2 still depended on qtwebengine because the rss USE flag was enabled. So I added the line ‘net-p2p/ktorrent -rss‘ to the file /etc/portage/package.use/package.use and re-merged net-p2p/ktorrent. Actually, I re-merged the following packages just in case they needed to be rebuilt, although in retrospect I believe that was unnecessary:

     Fri Mar  5 05:37:26 2021 >>> kde-apps/kdecore-meta-20.12.2
     Fri Mar  5 05:37:55 2021 >>> kde-apps/kdeutils-meta-20.12.2
     Fri Mar  5 05:45:49 2021 >>> net-p2p/ktorrent-20.12.2
     Fri Mar  5 05:46:49 2021 >>> kde-apps/kdenetwork-meta-20.12.2
     Fri Mar  5 05:57:41 2021 >>> kde-apps/marble-20.12.2
     Fri Mar  5 05:58:15 2021 >>> kde-apps/kdeedu-meta-20.12.2

7. By now another day had dawned, so I checked if new versions of the ebuilds for any KDE packages had been uploaded to the Portage repositories:

root # emaint sync -a
root # eix-update && updatedb

8. I rebooted the laptop and checked which packages still depended on qtwebengine. It turned out that only the two packages with a hard-dependency on qtwebengine were still preventing me from removing it:

root # equery depends qtwebengine
 * These packages depend on qtwebengine:
kde-apps/kaccounts-providers-20.12.2 (>=dev-qt/qtwebengine-5.15.2:5)
net-libs/signon-ui-0.15_p20171022-r1 (dev-qt/qtwebengine:5)

9. I checked if any packages depended on those two packages:

root # equery depends kaccounts-providers
 * These packages depend on kaccounts-providers:
kde-misc/kio-gdrive-20.12.2 (>=kde-apps/kaccounts-providers-20.12.2:5)
# equery depends kio-gdrive
 * These packages depend on kio-gdrive:
kde-apps/kdenetwork-meta-20.12.2 (>=kde-misc/kio-gdrive-20.12.2:5)

root # equery depends signon-ui
 * These packages depend on signon-ui:
kde-apps/kaccounts-providers-20.12.2 (net-libs/signon-ui)

So kdenetwork-meta hard-depends on kio-gdrive, which does not make much sense, really, given that not all KDE users have a Google Drive account and those users therefore do not need the kio-gdrive package to be installed.

10. The contents of the kdenetwork-meta-20.12.3 ebuild look like this:

root # cat /usr/portage/kde-apps/kdenetwork-meta/kdenetwork-meta-20.12.3.ebuild
# Copyright 1999-2021 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI=7

DESCRIPTION="kdenetwork - merge this to pull in all kdenetwork-derived packages"
HOMEPAGE="https://kde.org/"

LICENSE="metapackage"
SLOT="5"
KEYWORDS="~amd64 ~arm64 ~ppc64 ~x86"
IUSE="+bittorrent dropbox +webengine"

RDEPEND="
        >=kde-apps/kdenetwork-filesharing-${PV}:${SLOT}
        >=kde-apps/kget-${PV}:${SLOT}
        >=kde-apps/kopete-${PV}:${SLOT}
        >=kde-apps/krdc-${PV}:${SLOT}
        >=kde-apps/krfb-${PV}:${SLOT}
        >=kde-apps/zeroconf-ioslave-${PV}:${SLOT}
        >=kde-misc/kdeconnect-${PV}:${SLOT}
        >=kde-misc/kio-gdrive-${PV}:${SLOT}
        >=net-irc/konversation-${PV}:${SLOT}
        bittorrent? (
                >=net-libs/libktorrent-${PV}:${SLOT}
                >=net-p2p/ktorrent-${PV}:${SLOT}
        )
        dropbox? ( >=kde-apps/dolphin-plugins-dropbox-${PV}:${SLOT} )
"

so I created an ebuild for kdenetwork-meta-20.12.3 in my local overlay with the dependency on kio-gdrive removed:

root # mkdir -p /usr/local/portage/kde-apps/kdenetwork-meta
root # cd /usr/local/portage/kde-apps/kdenetwork-meta
root # cp /usr/portage/kde-apps/kdenetwork-meta/kdenetwork-meta-20.12.3.ebuild .
root # nano kdenetwork-meta-20.12.3.ebuild # Delete the line containing ">=kde-misc/kio-gdrive-${PV}:${SLOT}"
root # ebuild kdenetwork-meta-20.12.3.ebuild manifest
>>> Creating Manifest for /usr/local/portage/kde-apps/kdenetwork-meta
root # # eix-update && updatedb

11. I re-merged the world set in order to update all KDE packages that now had a newer ebuild version:

root # emerge -uvDN @world

12. I rechecked the three packages that had depended on qtwebengine:

root # equery depends signon-ui
 * These packages depend on signon-ui:
kde-apps/kaccounts-providers-20.12.3 (net-libs/signon-ui)
root # equery depends kaccounts-providers
 * These packages depend on kaccounts-providers:
kde-misc/kio-gdrive-20.12.3 (kaccounts ? >=kde-apps/kaccounts-providers-20.08.3:5)
root # equery depends kio-gdrive
 * These packages depend on kio-gdrive:
root #

As can be seen above, my modified ebuild for kdenetwork-meta-20.12.3 had indeed removed the impediment to uninstalling kio-gdrive and therefore the impediment to uninstalling kaccount-providers and signon-ui.

13. I merged my modified version of kdenetwork-meta-20.12.3:

Up to this point kde-apps/kdenetwork-meta-20.12.3 had been merged from the main Portage tree:

root # eix -I kde-apps/kdenetwork-meta
[I] kde-apps/kdenetwork-meta
     Available versions:  (5) 20.08.3-r1 (~)20.12.3 (~)20.12.3[1]
       {+bittorrent dropbox +webengine}
     Installed versions:  20.12.3(5)(15:23:08 05/03/21)(bittorrent -dropbox -webengine)
     Homepage:            https://kde.org/
     Description:         kdenetwork - merge this to pull in all kdenetwork-derived packages

[1] "local_overlay" /usr/local/portage

I then merged the version from my local overlay:

root # emerge -1v kdenetwork-meta::local_overlay

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] kde-apps/kdenetwork-meta-20.12.3:5::local_overlay [20.12.3:5::gentoo] USE="bittorrent -dropbox -webengine" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) kde-apps/kdenetwork-meta-20.12.3::local_overlay
>>> Installing (1 of 1) kde-apps/kdenetwork-meta-20.12.3::local_overlay
>>> Jobs: 1 of 1 complete                           Load avg: 1.76, 0.88, 0.61
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.

root # eix -I kde-apps/kdenetwork-meta
[I] kde-apps/kdenetwork-meta
     Available versions:  (5) 20.08.3-r1 (~)20.12.3 (~)20.12.3[1]
       {+bittorrent dropbox +webengine}
     Installed versions:  20.12.3(5)[1](16:40:43 05/03/21)(bittorrent -dropbox -webengine)
     Homepage:            https://kde.org/
     Description:         kdenetwork - merge this to pull in all kdenetwork-derived packages

[1] "local_overlay" /usr/local/portage

14. I checked which packages still depended on qtwebengine:

root # equery depends qtwebengine
 * These packages depend on qtwebengine:
kde-apps/kaccounts-providers-20.12.3 (>=dev-qt/qtwebengine-5.15.2:5)
kde-apps/kdenlive-20.12.3 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
kde-apps/marble-20.12.3 (webengine ? >=dev-qt/qtwebengine-5.15.2:5[widgets])
kde-plasma/kdeplasma-addons-5.21.2 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
kde-plasma/libksysguard-5.21.2 (webengine ? >=dev-qt/qtwebengine-5.15.2:5)
net-libs/signon-ui-0.15_p20171022-r1 (dev-qt/qtwebengine:5)
net-p2p/ktorrent-20.12.3 (rss ? >=dev-qt/qtwebengine-5.15.2:5)
                         (webengine ? >=dev-qt/qtwebengine-5.15.2:5)

Eureka! kdenetwork-meta no longer depends on qtwebengine.

15. I was then able to remove qtwebengine and the remaining packages that hard-depend on it:

root # emerge --ask --depclean qtwebengine kaccounts-providers signon-ui kio-gdrive

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:                                                                                                                                                                                                

 kde-misc/kio-gdrive
    selected: 20.12.3 
   protected: none 
     omitted: none 

 kde-apps/kaccounts-providers
    selected: 20.12.3 
   protected: none 
     omitted: none 

 net-libs/signon-ui
    selected: 0.15_p20171022-r1 
   protected: none 
     omitted: none 

 dev-qt/qtwebengine
    selected: 5.15.2_p20210224 
   protected: none 
     omitted: none 

All selected packages: =dev-qt/qtwebengine-5.15.2_p20210224 =kde-apps/kaccounts-providers-20.12.3 =kde-misc/kio-gdrive-20.12.3 =net-libs/signon-ui-0.15_p20171022-r1

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] Yes
>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 4) kde-misc/kio-gdrive-20.12.3...
>>> Unmerging (2 of 4) kde-apps/kaccounts-providers-20.12.3...
>>> Unmerging (3 of 4) net-libs/signon-ui-0.15_p20171022-r1...
>>> Unmerging (4 of 4) dev-qt/qtwebengine-5.15.2_p20210224...
Packages installed:   1648
Packages in world:    329
Packages in system:   43
Required packages:    1648
Number removed:       4

 * GNU info directory index is up-to-date.

\o/ \o/ \o/ \o/ No more qtwebengine in Gentoo Linux Testing (~amd64) running KDE.

Of course this was only possible because I do not need the specific packages that had been uninstalled during this entire procedure. Other people may not be in the same position.

16. I added the following lines to the file /etc/portage/package.mask/package.mask so that the packages are not pulled in automatically when merging the world set in future:

dev-qt/qtwebengine
kde-apps/kdenetwork-meta::gentoo
kde-misc/kio-gdrive
kde-apps/kaccounts-providers
net-libs/signon-ui

17. In future I will have to modify new versions of the kdenetwork-meta ebuild and add them to my local overlay. Furthermore, if other packages become dependent on qtwebengine in future and I do not require them, I will have to repeat the above steps in order to remove them (if viable). I just hope I can keep the qtwebengine package from ever being installed again.