Firewall zones (profiles) in Linux, and how to switch them automatically if you use UFW

Firstly, a note on terminology: UFW (Uncomplicated Firewall) and its two GUI front-ends Gufw and UFW Frontends use the term ‘application profile’ to refer to a pre-configured set of rules specified in a file. Files containing UFW application profiles are placed in the directory /etc/ufw/applications.d/. An application profile for SMB, for example, enables the root user to use the UFW command ‘ufw allow Samba‘ (‘ufw allow CIFS‘ in Gentoo Linux) rather than having to enter UFW commands specifying the precise ports and network protocols that SMB uses. However, this blog post is not about UFW’s application profiles; it is about what Gufw calls ‘profiles’ and firewalld calls ‘zones’.

In essence a profile/zone is a collection of firewall policies and rules. Both Gufw and firewalld include the concept of a ‘zone’, although Gufw uses the term ‘profile’ rather than ‘zone’. UFW Frontends does not have the concept of a ‘zone’; rules entered via UFW Frontends apply to any network to which you connect your laptop. The ability to define different zones for different networks is handy. For example, you can have certain policies and rules when your laptop is connected to your home network, and different policies and rules when your laptop is connected to the network in a café, hotel, airport or other public place.

An attractive feature of firewalld when used in conjunction with NetworkManager and KDE Plasma is that it is possible to use the desktop environment’s network management module (‘System Settings’ > ‘Connections’) to specify a particular firewalld zone for a particular network connection. For example, let’s say you used firewalld to specify certain policies and rules for a zone you named ‘office’, and you then specified in the System Settings – Connections GUI that a connection named ‘ACM’ should use the zone ‘office’. Thereafter, whenever you connect your laptop to the network named ‘ACM’, firewalld will use the policies and rules you previously configured for the zone ‘office’.

Unlike firewalld, Gufw does not have the ability to switch profiles automatically according to which network the laptop is connected. You have to select manually the profile you wish to use. You would launch Gufw prior to connecting to, for example, your office’s network, select the profile ‘Office’ (or whatever you have named it), then connect your laptop to that network.

I think many people would be satisfied with the functionality currently provided by Gufw. I could use the Gufw GUI to create Gufw profiles with names such as ‘Home’, ‘HomeDave’, ‘Public’, ‘HQoffice’, ‘USoffice’, ‘PestanaRio’ and so on, and specify the different policies and rules I want for each profile. At home I would launch Gufw on my laptop and select the Home profile then connect to my home network; in the office at work I would launch Gufw on my laptop and select the HQoffice profile then connect to the office network; at my friend Dave’s house I would launch Gufw on my laptop and select the HomeDave profile then connect to the house network; and so on. Nevertheless I do see the attraction of automated zone switching, as provided by firewalld in conjunction with NetworkManager and KDE. It would be handy if my laptop could switch automatically to the Home profile when my laptop connected to the network at my home with the name ‘BTHub5-8EUQ’, automatically switch to the HQoffice profile when my laptop connected to the network named ‘HQ-Office2’ in the office, and so on.

I use UFW on my two laptops running Gentoo Linux. The package ufw-frontends is also installed but normally I use UFW directly via the command line. However I wanted to learn about zones/profiles while using UFW, and I also wanted to see if I could automate the switching of zones without resorting to installing firewalld. NetworkManager has the ability to launch ‘hook’ scripts when certain things happen — when a network connection changes, for example — and this seemed to me to be a way of switching profiles automatically.

I had not used Gufw before, so I decided to install it. A package is available in many Linux distributions but there is no ebuild for Gufw in Gentoo’s main Portage tree and I could not find an up-to-date ebuild for it in any Portage overlays. Therefore I created the ebuild for net-firewall/gufw-19.10.0 shown below. It probably needs improving, but it does install a working Gufw in Gentoo Linux.

# Copyright 1999-2019 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=7
PYTHON_COMPAT=( python3_{5,6,7} )
DISTUTILS_IN_SOURCE_BUILD=1

inherit distutils-r1

MY_PN="gui-ufw"
MY_PV="$(ver_cut 1-2)"

DESCRIPTION="GUI frontend for managing ufw."
HOMEPAGE="https://gufw.org/"
SRC_URI="https://launchpad.net/${MY_PN}/trunk/${MY_PV}/+download/${MY_PN}-${PV}.tar.gz"

LICENSE="GPL-3"
SLOT="0"
KEYWORDS="~amd64"
IUSE=""

DEPEND="dev-python/python-distutils-extra"
RDEPEND="net-firewall/ufw
	dev-python/netifaces
	dev-python/pygobject:3
	net-libs/webkit-gtk[introspection]
	sys-auth/elogind
	sys-auth/polkit
	x11-libs/gtk+:3[introspection]
	x11-themes/gnome-icon-theme-symbolic
"
S=${WORKDIR}/${MY_PN}-${PV}

pkg_postinst() {
	sed '/dist-packages/d' -i /usr/bin/gufw-pkexec
	sed -E '/\/share\//d' -i /usr/bin/gufw-pkexec
	local PYTHONVERSION="$(python -c 'import sys; print("{}.{}".format(sys.version_info.major, sys.version_info.minor))')"
	sed -E "s|python3\.[0-9]|python${PYTHONVERSION}|g" -i /usr/bin/gufw-pkexec
	sed -E 's|\/lib\/|\/lib64\/|g' -i /usr/bin/gufw-pkexec
}

How To Set Up a Firewall with GUFW on Linux‘ is a good tutorial on Gufw.

As I had not used Gufw previously, I had to play around with it to understand better its functional design. I found that if I configure rules directly via UFW on the command line without using Gufw, Gufw does not allow me to edit those rules (but does allow me to delete them) and those rules exist whichever Gufw profile is selected in the Gufw GUI. Gufw profiles are stored in files named ‘/etc/gufw/*.profile‘ (e.g. /etc/gufw/Home.profile) and these files will not include UFW rules entered via the command line. On the other hand, UFW rules created via the Gufw GUI apply solely to the currently-selected Gufw profile, which is what I would have expected. In other words, I can create a different set of policies and rules in each Gufw profile. Therefore I believe Gufw profiles (as distinct from UFW application profiles) are basically analogous to firewalld’s zones. It also appears to me that Gufw maintains configuration files specifying policies and rules independently of UFW, which Gufw then applies to UFW. In other words, if you are a Gufw user you should not use UFW directly to configure policies and rules, otherwise Gufw’s configuration files will not include what you did directly using UFW. To reiterate, use only Gufw or only UFW, not both.

The current Gufw profile’s name is listed in the file /etc/gufw/gufw.cfg. For example, I currently have the Home profile selected in the Gufw GUI, and the file gufw.cfg contains the following:

[GufwConfiguration]
profile = Home
windowwidth = 542
windowheight = 530
confirmdetelerule = yes

If I examine the contents of the file /etc/gufw/Home.profle I see that it contains the UFW policies and rules I specified for the Gufw Home profile:

[fwBasic]
status = enabled
incoming = deny
outgoing = allow
routed = disabled

[Rule0]
ufw_rule = 137,138/udp ALLOW IN 192.168.1.0/24
description = Samba
command = /usr/sbin/ufw allow in proto udp from 192.168.1.0/24 to any port 137,138
policy = allow
direction = in
protocol = 
from_ip = 192.168.1.0/24
from_port = 
to_ip = 
to_port = 137,138/udp
iface = 
routed = 
logging = 

[Rule1]
ufw_rule = 139,445/tcp ALLOW IN 192.168.1.0/24
description = Samba
command = /usr/sbin/ufw allow in proto tcp from 192.168.1.0/24 to any port 139,445
policy = allow
direction = in
protocol = 
from_ip = 192.168.1.0/24
from_port = 
to_ip = 
to_port = 139,445/tcp
iface = 
routed = 
logging =

I also notice that the other Gufw profiles can differ. For example, my Office.profile file contains the following:

[fwBasic]
status = enabled
incoming = deny
outgoing = allow
routed = allow

The profile name listed in gufw.cfg gets changed when the user changes the profile using the Gufw GUI. It appears to me that only at the point in time when the user selects a certain Gufw profile in the Gufw GUI does Gufw parse the applicable *.profile file and issue commands to UFW to implement the policies and rules specified in the *.profile file.

Initially I tried to automate the process of changing the Gufw profile by doing the following:

  1. I created a NetworkManager Dispatcher hook script to:

    1. detect when the laptop connects to a network;

    2. determine whether the network is at my home, at my workplace or in a public place (café, airport or wherever) by looking at the connection name;

    3. edit gufw.cfg to change the name of the Gufw profile according to the network connected.
  2. I configured KDE to launch Gufw automatically at login, hoping that would implement the Gufw profile specified in gufw.cfg.

When I connected the laptop to various networks, Gufw did indeed show the name of the profile selected by the NetworkManager Dispatcher hook script, but the associated Gufw profile’s rules had not been applied. They were only applied if I clicked on the ‘Profile’ pull-down menu in Gufw, selected a different Gufw profile, then re-selected the desired Gufw profile. Therefore driving Gufw from a NetworkManager Dispatcher hook script is not possible. This is a pity, as Gufw is an easy way to manage UFW from a GUI; it allows the user to create, delete and edit zones (Gufw profiles) and to select them manually. What Gufw doesn’t do is enable the user to associate those zones with connection names, nor trigger specific zone automatically based on the selected network connection. firewalld, on the other hand, does enable the user to do both those things.

As my attempt at automating the switching of zones in Gufw had failed, I decided to create a NetworkManager Dispatcher hook script to switch zones automatically by using UFW commands. Initially I though about creating a bespoke UFW application profile for each zone and allowing/denying those in the script, but it is actually easier to use the fundamental UFW commands in the script, especially as UFW commands are relatively easy to understand. Also, this approach means everything is in a single file, which facilitates configuration. I can simply edit the script in order to: a) add or delete a zone; b) change a zone’s name; c) change policies and rules for a zone; d) add or delete a connection; e) change the name of a connection; f) change the zone a connection uses. Granted, editing a script is not as user-friendly as using the firewalld GUI to configure a zone and then using KDE Plasma’s system settings module Connections to specify that zone for a specific connection, but my script is not particularly difficult to understand and edit. And by using such a script I can continue to use UFW rather than installing firewalld and having to learn how to use it.

My NetworkManager Dispatcher hook script /etc/NetworkManager/dispatcher.d/20_ufw-zones is listed below. In the main body of the script I define the zone I wish to use for each connection, and in the function select_zone I define the policies and rules I want each zone to use.

#!/bin/bash
INTERFACE=$1
STATUS=$2
WIRED=enp4s0f1
WIFI=wlp3s0

CT_helper_rule() {
    echo "# The following is needed to enable Samba commands to" >> /etc/ufw/before.rules
    echo "# work properly for broadcast NetBIOS name resolution" >> /etc/ufw/before.rules
    echo "#"  >> /etc/ufw/before.rules
    echo "# raw table rules" >> /etc/ufw/before.rules
    echo "*raw" >> /etc/ufw/before.rules
    echo ":OUTPUT ACCEPT [0:0]" >> /etc/ufw/before.rules
    echo "-F OUTPUT" >> /etc/ufw/before.rules
    echo "-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns" >> /etc/ufw/before.rules
    echo "COMMIT" >> /etc/ufw/before.rules
}
 
select_zone() {
    ufw --force reset
    ufw --force enable
    ZONE=$1
    case "$ZONE" in
    'Home')
        ufw default deny incoming
        ufw default allow outgoing
        #
        # Rules for SMB
        ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
        ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp
        CT_helper_rule
        #
        # Rules for KDEConnect
        ufw allow from 192.168.1.0/24 to any port 1714:1764 proto udp
        ufw allow from 192.168.1.0/24 to any port 1714:1764 proto tcp
    ;;
    'Office')
        ufw default deny incoming
        ufw default allow outgoing
    ;;
    'Public')
        ufw default reject incoming
        ufw default allow outgoing
    ;;
    'JohnsHouse')
        ufw default deny incoming
        ufw default allow outgoing
        #
        # Rules for SMB
        ufw allow from 192.168.42.0/24 to any port 137,138 proto udp
        ufw allow from 192.168.42.0/24 to any port 139,445 proto tcp
        CT_helper_rule
        #
        # Rules for KDEConnect
        ufw allow from 192.168.42.0/24 to any port 1714:1764 proto udp
        ufw allow from 192.168.42.0/24 to any port 1714:1764 proto tcp
    ;;
    esac
    ufw --force reload
    rm /etc/ufw/*.rules.20* # Delete backups of *.rules files ufw makes every time it is reset
    echo -n `date +"[%F %T %Z]"` >> /var/log/ufw-zones.log
    echo " Zone $ZONE selected for connection $ACTIVE on interface $INTERFACE." >> /var/log/ufw-zones.log
}
 
# Check if either the wired or wireless interface is up
if [ "$INTERFACE" = "$WIRED" -o "$INTERFACE" = "$WIFI" ] && [ "$STATUS" = "up" ]; then
 
    # Check if a single connection is active
    if [ `nmcli c | grep -v "\-\-" | grep -v "NAME.*UUID.*TYPE.*DEVICE" | wc -l` -eq 1 ]; then
 
        # Ascertain the name of the active connection
        ACTIVE=`nmcli c | grep -v "\-\-" | grep -v "NAME.*UUID.*TYPE.*DEVICE" | awk -F' ' '{print $1}'`
 
        case "$ACTIVE" in
 
        'eth0')
            ZONE="Home"
        ;;
        'POR1-wired')
            ZONE="Office"
        ;;
        'BTHub5-8EUQ')
            ZONE="Home"
        ;;
        'BTHub5-8EUQ-5GHz')
            ZONE="Home"
        ;;
        'John1')
            ZONE="JohnsHouse"
        ;;
        'GRAND MERCURE')
            ZONE="Public"
        ;;
        *)
            # If connection name is not in above list
            ZONE="Public"
        ;;

        esac

        select_zone $ZONE
        exit $?

    fi
fi

The log file that the script uses contains a chronological record of the connections made and the zones selected:

$ cat /var/log/ufw-zones.log 
[2019-09-30 20:13:52 BST] Zone Home selected for connection eth0 on interface enp4s0f1.
[2019-10-01 22:59:18 BST] Zone Home selected for connection BTHub5-8EUQ-5GHz on interface wlp3s0.
[2019-10-02 17:59:23 EDT] Zone Public selected for connection loganwifi on interface wlp3s0.
[2019-10-03 10:12:46 EDT] Zone Office selected for connection POR1-wired on interface enp4s0f1.
Advertisements

Paul Gideon Dann’s patchset for Poppler to enable Okular (Qt5) to use Cairo rather than Splash to render PDF files

If you view the same PDF file in Okular (KDE) and Evince (GNOME), you may notice that fonts and lines are rendered better in Evince. Both applications use Poppler to render text and graphics in PDF files, but Poppler uses a different rendering backend in the two applications. For Evince Poppler uses the Cairo library, whereas for Okular Poppler uses Splash, a backend inherited from Poppler’s predecessor Xpdf (still in development). Unfortunately for KDE users, Cairo often does a better job than Splash. However, independent software engineer Paul Gideon Dann came to the rescue by producing the patchset poppler-cairo-backend to modify Poppler in order to make it use the Cairo library instead of Splash when Poppler is used by Okular. To quote the README file for Paul’s patchset:

Purpose of this Patchset

Currently, the default backend for the Qt5 wrapper (used by Okular) is Splash. Unfortunately, Splash does not support subpixel rendering of fonts, so those of us using KDE are stuck with somewhat ugly-looking fonts. This patchset adds support for the Cairo backend to the Qt5 wrapper. It also forces subpixel rendering in the Cairo backend. The upshot of this is that we get beautiful fonts in Okular.

The README focuses on fonts, but in fact the rendering of lines in graphics in PDF files can also be improved by the application of the patchset.

Apparently the Poppler maintainer feels that the introduction of a dependency on Cairo to the Qt5 wrapper (even an optional dependency) in Poppler would be controversial, and he is not willing to merge the patchset. For Okular users who already have Cairo installed (e.g. for Firefox, Inkscape, Scribus and so on), and who are noticing inadequate rendering of some PDF files, Paul’s patchset is worth trying.

In Gentoo Linux, which is a source code-based distribution, it is very easy to apply the patchset. For example, I did the following to apply the patchset for Poppler 0.80.0 in a ~amd64 (Testing Branch) installation:

1. Created a package-specific and version-specific directory to hold the patchset:

root # mkdir -p /etc/portage/patches/app-text/poppler-0.80.0

2. Downloaded the patchset for Poppler 0.80.0 from the following Web page:

https://github.com/giddie/poppler-cairo-backend/tree/76e607bcf010d6d9b8df5cb0f851ef9c91d4caf2

3. Copied the patchset to the directory created in Step 1:

root # cp /home/fitzcarraldo/Downloads/*.patch /etc/portage/patches/app-text/poppler-0.80.0/
root # ls -1 /etc/portage/patches/app-text/poppler-0.80.0
0001-Cairo-backend-added-to-Qt5-wrapper.patch
0002-Setting-default-Qt5-backend-to-Cairo.patch
0003-Apply-subpixel-rendering-in-Cairo-Backend.patch

4. Checked first that the patchset could be applied successfully before actually using it:

root # cd /usr/portage/app-text/poppler
root # ebuild poppler-0.80.0.ebuild clean prepare
 * poppler-0.80.0.tar.xz BLAKE2B SHA512 size ;-) ...                                     [ ok ]
 * checking ebuild checksums ;-) ...                                                     [ ok ]
 * checking auxfile checksums ;-) ...                                                    [ ok ]
 * checking miscfile checksums ;-) ...                                                   [ ok ]
>>> Unpacking source...
>>> Unpacking poppler-0.80.0.tar.xz to /var/tmp/portage/app-text/poppler-0.80.0/work
>>> Source unpacked in /var/tmp/portage/app-text/poppler-0.80.0/work
>>> Preparing source in /var/tmp/portage/app-text/poppler-0.80.0/work/poppler-0.80.0 ...
 * Applying poppler-0.60.1-qt5-dependencies.patch ...                                    [ ok ]
 * Applying poppler-0.28.1-fix-multilib-configuration.patch ...                          [ ok ]
 * Applying poppler-0.78.0-respect-cflags.patch ...                                      [ ok ]
 * Applying poppler-0.61.0-respect-cflags.patch ...                                      [ ok ]
 * Applying poppler-0.57.0-disable-internal-jpx.patch ...                                [ ok ]
 * Applying 0001-Cairo-backend-added-to-Qt5-wrapper.patch ...                            [ ok ]
 * Applying 0002-Setting-default-Qt5-backend-to-Cairo.patch ...                          [ ok ]
 * Applying 0003-Apply-subpixel-rendering-in-Cairo-Backend.patch ...                     [ ok ]
 * User patches applied.
>>> Source prepared.

5. Re-merged Poppler to apply the patchset to the Poppler source code and rebuild the patched package:

root # emerge -1v poppler

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] app-text/poppler-0.80.0:0/90::gentoo  USE="cairo cjk cxx introspection jpeg jpeg2k lcms png qt5 tiff utils -curl -debug -doc -nss" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) app-text/poppler-0.80.0::gentoo
>>> Installing (1 of 1) app-text/poppler-0.80.0::gentoo
>>> Jobs: 1 of 1 complete                           Load avg: 1.06, 1.11, 0.95
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.

6. Re-merged Okular so that it uses the patched Poppler dependency:

root # emerge -1v okular

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] kde-apps/okular-19.08.1:5::gentoo  USE="chm crypt djvu image-backend pdf postscript tiff -debug -epub -handbook -markdown -mobi -mobile -plucker -share -speech -test" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) kde-apps/okular-19.08.1::gentoo
>>> Installing (1 of 1) kde-apps/okular-19.08.1::gentoo
>>> Jobs: 1 of 1 complete                           Load avg: 1.17, 1.13, 1.04
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.

My thanks go to Paul for taking the time to produce the patchset.

How to change the height of the Kickoff Application Launcher menu in KDE Plasma

The height of the KDE Plasma Kickoff Application Launcher menu is not user-configurable, which is odd in a Desktop Environment with a reputation for being highly user-configurable.

It turns out that the height and width of the pop-up menu are hard-coded in the ASCII file /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml:

root # grep -E "Layout.minimumHeight.*units.gridUnit" /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml
    Layout.minimumHeight: units.gridUnit * 34
root # grep -E "Layout.minimumWidth.*units.gridUnit" /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml
    Layout.minimumWidth: units.gridUnit * 26

Now, I was a bit fed up having to scroll up and down the launcher menu to see all fourteen entries in my Favourites list, so I decided to increase the height of the menu, which I did by editing /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml as root user:

root # nano /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml
root # grep -E "Layout.minimumHeight.*units.gridUnit" /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml
    Layout.minimumHeight: units.gridUnit * 44

The only downside to this is that the file will be overwritten when the package kde-plasma/plasma-desktop is upgraded.

The following command would allow me to make sure the file contains the height value of ’44’ that I want:

root # sed -i '/Layout.minimumHeight: units.gridUnit/ c\    Layout.minimumHeight: units.gridUnit * 44' /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml

Therefore, to automate the editing of the file in my Gentoo installations that use OpenRC I created a shell script /etc/local.d/50-set_Kickoff_height.start with the following contents:

#!/bin/bash
if [ -e /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml ]; then
    sed -i '/Layout.minimumHeight: units.gridUnit/ c\    Layout.minimumHeight: units.gridUnit * 44' /usr/share/plasma/plasmoids/org.kde.plasma.kickoff/contents/ui/FullRepresentation.qml
fi

The FullRepresentation.qml file will then be edited every time the machine boots, which is a tad inefficient but not a big overhead.

This is not a perfect solution because the menu will revert to its default height following an upgrade to the package kde-plasma/plasma-desktop until I reboot the machine, but it is good enough for me.

How to change the keymap (keyboard layout) used by the GRUB shell in Gentoo Linux

The default keymap in the GRUB shell is US English. Because Linux has not yet been booted, the GRUB keymap is not governed by the keymap for the Linux console specified in /etc/conf.d/keymaps in the case of OpenRC, or in /etc/vconsole.conf in the case of systemd. This can be inconvenient if your keyboard has a different layout and you need to use the GRUB Rescue Shell. Below I explain how I configured my Gentoo Linux installation to be able to use a different keyboard layout in the GRUB shell.

There are, however, certain limitations to the keymap in the GRUB shell. The official GRUB documentation states the following:

17.4 Input terminal

Firmware console on BIOS, IEEE1275 and ARC doesn’t allow you to enter non-ASCII characters. EFI specification allows for such but author is unaware of any actual implementations. Serial input is currently limited for latin1 (unlikely to change). Own keyboard implementations (at_keyboard and usb_keyboard) supports any key but work on one-char-per-keystroke. So no dead keys or advanced input method. Also there is no keymap change hotkey. In practice it makes difficult to enter any text using non-Latin alphabet. Moreover all current input consumers are limited to ASCII.

Note that the GRUB documentation states ‘ASCII’, not ‘Extended ASCII’. ASCII is limited to codes 000 to 127 (see the character table in e.g. http://www.asciitable.com/).

Some Linux distributions have the utility grub-kbdcomp to generate a GRUB keyboard layout file. grub-kbdcomp is simply a shell script that is a wrapper for the Debian ckbcomp utility and grub-mklayout. There is a Gentoo Portage ebuild for ckbcomp:

root # eix ckbcomp
[I] sys-apps/ckbcomp
     Available versions:  (~)1.164
     Homepage:            https://anonscm.debian.org/cgit/d-i/console-setup.git
     Description:         Compile an XKB keymap for loadkeys

However, I noticed that the latest version currently available in Debian is 1.191 (https://salsa.debian.org/installer-team/console-setup.git), so I created an ebuild ckbcomp-1.191.ebuild in a local overlay on one of my laptops running Gentoo Linux Stable Branch, and I installed ckbcomp-1.191.

# Copyright 1999-2019 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=6

DESCRIPTION="Compile an XKB keymap for loadkeys"
HOMEPAGE="https://salsa.debian.org/installer-team/console-setup.git"

if [[ ${PV} == 9999 ]]; then
        inherit git-r3
        EGIT_REPO_URI="https://salsa.debian.org/installer-team/console-setup.git"
else
        SRC_URI="https://salsa.debian.org/installer-team/console-setup/-/archive/${PV}/${P}.tar.gz -> ${P}.tar.gz"
        KEYWORDS="~amd64"
        S="${WORKDIR}"
fi

LICENSE="GPL-2"
SLOT="0"

DEPEND=""
RDEPEND="
        dev-lang/perl:*
        sys-apps/kbd
        x11-misc/xkeyboard-config"

src_compile() {
        :
}

src_install() {
        dobin console-setup-${PV}-*/Keyboard/ckbcomp
}

I have tried the above-mentioned ckbcomp command on my PC BIOS Core i7 laptop running Gentoo Stable with OpenRC and GRUB Version 2.02-r3:

root # eix -I grub
[I] sys-boot/grub
     Available versions:  (2) 2.02-r3(2/2.02-r3)^st **9999(2/9999)^st
       {debug device-mapper doc efiemu +fonts libzfs mount multislot nls sdl static test +themes truetype GRUB_PLATFORMS="coreboot efi-32 efi-64 emu ieee1275 loongson multiboot pc qemu qemu-mips uboot xen xen-32"}
     Installed versions:  2.02-r3(2/2.02-r3)^st(02:33:36 23/03/19)(fonts nls sdl themes truetype -debug -device-mapper -doc -efiemu -libzfs -mount -multislot -static -test GRUB_PLATFORMS="pc -coreboot -efi-32 -efi-64 -emu -ieee1275 -loongson -multiboot -qemu -qemu-mips -uboot -xen -xen-32")
     Homepage:            https://www.gnu.org/software/grub/
     Description:         GNU GRUB boot loader

I used the following steps:

1. Installed sys-apps/ckbcomp.

root # emerge ckbcomp
root # eix ckbcomp
[I] sys-apps/ckbcomp
     Available versions:  (~)1.164 (~)1.191[1]
     Installed versions:  1.191[1](22:09:15 20/04/19)
     Homepage:            https://salsa.debian.org/installer-team/console-setup.git
     Description:         Compile an XKB keymap for loadkeys

[1] "local_overlay" /usr/local/portage

2. Created a new sub-directory to store the GRUB keyboard layout files.

root # mkdir /boot/grub/layouts

3. Converted the X11 keymap to the GRUB keymap. The option for ckbcomp must exist in the directory /usr/share/X11/xkb/symbols/ for this to work.

root # ckbcomp gb extd | grub-mklayout -o /boot/grub/layouts/gb.gkb
Unknown keyboard scan identifier Meta_Tab
Unknown keyboard scan identifier Meta_Tab
Unknown keyboard scan code 0x54
Unknown keyboard scan code 0x65
Unknown keyboard scan code 0x7f

I used the following commands to generate a br.gkb (Brazilian Portuguese keymap) file and a us.gkb (US English keymap) as well, as it is possible to switch keyboard layouts from the GRUB command line using the keymap command, as I show further on:

root # ckbcomp br nodeadkeys | grub-mklayout -o /boot/grub/layouts/br.gkb
Unknown keyboard scan identifier Meta_Tab
Unknown keyboard scan identifier Meta_Tab
Unknown keyboard scan identifier KP_Comma
Unknown keyboard scan identifier KP_Comma
Unknown keyboard scan identifier KP_Comma
Unknown keyboard scan identifier KP_Comma
Unknown keyboard scan code 0x54
Unknown keyboard scan code 0x65
Unknown keyboard scan code 0x7f
root # ckbcomp us | grub-mklayout -o /boot/grub/layouts/us.gkb
Unknown keyboard scan identifier Meta_Tab
Unknown keyboard scan identifier Meta_Tab
Unknown keyboard scan code 0x54
Unknown keyboard scan code 0x65
Unknown keyboard scan code 0x7f

The resulting files can be seen in the directory /boot/grub/layouts/:

root # ls -la /boot/grub/layouts
total 11
drwxr-xr-x 2 root root 1024 Apr 21 21:20 .
drwxr-xr-x 7 root root 1024 Nov 26 00:01 ..
-rw-r--r-- 1 root root 2572 Apr 21 21:29 br.gkb
-rw-r--r-- 1 root root 2572 Apr 21 21:29 gb.gkb
-rw-r--r-- 1 root root 2572 Apr 21 21:30 us.gkb

4. Append ‘GRUB_TERMINAL_INPUT=at_keyboard‘ to /etc/default/grub.

root # grep GRUB_TERMINAL_INPUT /etc/default/grub
GRUB_TERMINAL_INPUT="at_keyboard"

5. Add ‘insmod‘ and ‘keymap‘ lines to /etc/grub.d/40_custom as shown below.

root # tail -n 2 /etc/grub.d/40_custom
insmod keylayouts
keymap $prefix/layouts/gb.gkb

6. Check what locales are available for the keymap.

root # locale --all-locales | grep -i gb
en_GB
en_GB.iso88591
en_GB.utf8

7. Add ‘locale=en_GB‘ to GRUB_CMDLINE_LINUX.

root # grep locale /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="locale=en_GB i915.modeset=1 rcutree.rcu_idle_gp_delay=1 acpi_enforce_resources=lax reboot=force raid=noautodetect resume=/dev/sda2"

8. Regenerate the grub.cfg file.

root # grub-mkconfig -o /boot/grub/grub.cfg
root # grep terminal_input /boot/grub/grub.cfg
terminal_input at_keyboard
root # grep gkb /boot/grub/grub.cfg
keymap $prefix/layouts/gb.gkb
root # grep layouts /boot/grub/grub.cfg
insmod keylayouts
keymap $prefix/layouts/gb.gkb

9. If the machine uses UEFI rather than PC BIOS, update the GRUB files in the EFI directory.

root # grub-install --efi-directory=/boot/efi

10. Reboot to check if the gb keymap has been loaded for the GRUB shell.

root # reboot

When I press ‘c‘ when the GRUB menu appears, I now see the following if I press each key on the second-to-last row of keys on the keyboard:

grub> \zxcvbnm,./

That corresponds to a British English keyboard layout. As I mentioned before, due to GRUB’s limitations only standard ASCII chars are possible, so it is not possible to type characters such as é and è, or symbols such as £ and etc. on the GRUB command line, whatever the keymap.

You can tell if the GRUB keylayouts module is loaded by entering the following command on the GRUB command line:

lsmod

Below is what I then see on the screen of a PC BIOS machine running up-to-date Gentoo Linux (Stable Branch) when I press ‘c‘ when the GRUB menu is displayed.


                                                GNU GRUB  version 2.02~beta3

   Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists possible device or file completions. ESC at any time exits.


grub> lsmod
Name    Ref Count       Dependencies
minicmd 1
gfxterm_background      1              bitmap,video,extcmd,gfxterm,bitmap_scale,video_colors
bitmap_scale    2               bitmap
video_colors    2
png     1               bitmap,bufio
bitmap  7
search  1               search_label,extcmd,search_fs_file,search_fs_uuid
search_label    2
search_fs_file  2
search_fs_uuid  2
at_keyboard     1               boot,keylayouts
keylayouts      3
gfxterm 3               video,font
all_video       1               video_cirrus,video_bochs,vga,vbe
video_cirrus    2               video_fb,pci,video
video_bochs     2               video_fb,pci,video
pci     6
vga     2               video_fb,video
vbe     2               video_fb,video
video_fb        12
font    5               video,bufio
video   24
loadenv 1               extcmd,disk
disk    2
test    1
normal  1               gettext,boot,extcmd,bufio,crypto,terminal
gzio    0
gettext 3
boot    4
extcmd  8
bufio   10
crypto  2
terminal        2
biosdisk        1
part_msdos      2
ext2    4               fshelp
fshelp  5
grub>

Shown below is what I see when I perform the following steps on the GRUB command line:

  1. press each key on the second-to-last line of keys on the keyboard and press Enter;
  2. check which GRUB terminal input module is loaded;
  3. change from the British English keyboard layout to the Brazilian Portuguese keyboard layout;
  4. press each key on the second-to-last line of keys on the keyboard and press Enter;
  5. switch back to the British English keyboard layout;
  6. press each key on the second-to-last line of keys on the keyboard and press Enter;
  7. switch to the US English keyboard layout;
  8. press each key on the second-to-last line of keys on the keyboard and press Enter;
  9. switch back to the British English keyboard layout.

In each case the output on the screen is correct for the keyboard layout selected:

grub> \zxcvbnm,./
error: can't find command `zxcvbnm,./'.
grub> terminal_input
Active input terminals:
at_keyboard
Available input terminals:
console serial_* serial
grub> keymap br
grub> \zxcvbnm,.;
error: can't find command `zxcvbnm,.;'.
grub> keymap gb
grub> \zxcvbnm,./
error: can't find command `zxcvbnm,./'.
grub> keymap us
grub> <zxcvbnm,./
error: syntax error.
error: Incorrect command.
error: syntax error.
grub> keymap gb
grub> 

There is one more caveat…

When the GRUB menu first appears at boot, the following lines are still displayed at the bottom of the GRUB menu:

   Use the ↑ and ↓ keys to select which entry is highlighted.
   Press enter to boot the selected OS, `e' to edit the commands before booting or `c' for a command-line.
The highlighted entry will be executed automatically in 4s.

However, the highlighted entry on the GRUB menu is no longer executed automatically and I have to press ENTER in order to get GRUB to boot Linux. That is not a big deal in my case.

Automatically clearing the /usr/tmp/portage directory in Gentoo Linux

Gentoo Linux has been in use for nine years on one of my old laptops. A couple of days ago I performed the usual rolling update of the installation, but the latest version of a large package that normally takes several hours to compile failed to compile due to a lack of disk space. Sure enough, the command ‘df -h‘ showed me that the root partition was full. After a little digging I discovered that the directory /usr/tmp/portage/ contained a whopping 30GB of directories and files.

Portage uses the directory /usr/tmp/portage/ as a temporary store for the package source code when merging a package. The temporary files are not deleted if a merge fails, but the emerge command should delete them on the next merge of that package. On the other hand the ebuild command does not delete the temporary files, although normally you only use the ebuild command if you are creating a manifest.

Anyway, in the nine years that Gentoo Linux has been installed on the laptop I had never bothered to check that /usr/tmp/portage/ was actually empty, and its contents had slowly increased. The cure to my immediate problem was simply to empty the directory:

root # rm -rf /usr/tmp/portage/*

I doubt the laptop would still be working by the time /usr/tmp/portage would become that full again, but the situation got me thinking: What if I were to create a script to delete the temporary directories and files in /usr/tmp/portage/ at shutdown?

Gentoo Linux on this laptop uses OpenRC so I simply created a file /etc/local.d/99delete_tmp_files_from_failed_merges.stop containing the following code:

#!/bin/bash
# If root partition is more than 90% full, delete any temporary directories and
# files that were left in /var/tmp/portage/ instead of being deleted.
#
# The root partition is on /dev/sda6 and the emerge command must not be running.
#
if [ `pgrep -c emerge` -eq 0 ] && [ `df | awk '/sda6/ {print $5}' | awk -F% '{print $1}'` -gt 90 ]; then
    rm -rf /usr/tmp/portage/*
fi

I made the script executable:

root # chmod +x /etc/local.d/99delete_tmp_files_from_failed_merges.stop

Now, if the root partition is more than 90% full when I shut down the laptop, the script will automatically empty that directory. One less thing to think about.

HEIC image files in Linux

I was at an event recently where the attendees were asked to upload their camera and smartphone photos and videos to a shared Google Drive folder. Some of the uploaded photo files have a .HEIC (High Efficiency Image Container) extension, which I had not come across before. I have since learnt that these HEIC files were produced by iPhones running iOS 11, encoded using the HEIF (High Efficiency Image File) format. Apparently the HEIF format is superior to the JPEG format in a number of ways (see the links at the end of my post, especially the image examples given by Nokia), although it is subject to patents and therefore I believe there are certain constraints to coding image files in HEIC format. Anyway, I’ll leave you to read the fine print. My interest was simply because I wanted to be able to download the above-mentioned photo files and view them all in the file managers and image-viewing applications in Linux and Android on my various devices.

Now, I can browse and view the above-mentioned shared HEIC images in Google Drive in the Firefox and Chrome browsers in Linux, although an ownCloud site viewed using the same browsers displays the HEIC files as grey icons that can only be downloaded, not opened and viewed in the browser. I also found that Cirrus, the Android app for ownCloud that I use on my Galaxy Note 8 phone, cannot display HEIC photos either.

I downloaded the HEIC files to a machine running Lubuntu 18.04 and to a machine running Gentoo Linux. The file manager PCManFM in Lubuntu 18.04 displays grey icons rather than thumbnails for these HEIC files, and KDE’s Dolphin 18.08.3 file manager in Gentoo Linux displays green image icons rather than thumbnails for them. As far as Linux image viewers go, in Lubuntu 18.04 I find that GPicView 0.2.5 and Geeqie 1.4 cannot display HEIC images, and in Gentoo Linux KDE I find that GQview 2.1.5-r1, Okular 18.08.3 and Gwenview 18.08.3 cannot display HEIC images. So I set about converting all the HEIC files to JPG files. I managed to do this but needed to use a range of tools, as illustrated by a couple of examples below for Lubuntu 18.04 and Gentoo Linux. This post might seem long-winded but perhaps may be of help to Linux users coming across .HEIC files for the first time.

From the .HEIC files I had downloaded I picked one at random to try and convert to a JPG file: IMG_3706.HEIC. Its EXIF data confirms it is an HEIC file:

user $ exiftool IMG_3706.HEIC | grep "File Type"
File Type                       : HEIC
File Type Extension             : heic
user $ exiftool IMG_3706.HEIC | grep "Camera Model"
Camera Model Name               : iPhone 7 Plus

Several of the files with the .HEIC suffix that I downloaded were not real HEIC files according to their EXIF data:

user $ exiftool IMG_9474.HEIC | grep "File Type"
File Type                       : JPEG
File Type Extension             : jpg
user $ exiftool IMG_9474.HEIC | grep "Camera Model"
Camera Model Name               : iPhone 8

Those files were apparently treated as JPEG files by the tools I mention below, so I have omitted the results for those ‘false’ HEIC files.

Lubuntu 18.04

1. I installed the libheif example tools:

user $ sudo apt install libheif-examples

2. I used the heif-info command to check the file:

user $ heif-info IMG_3706.HEIC 
image: 3024x4032 (id=49), primary
  thumbnail: 240x320
  alpha channel: no
  depth channel: no

3. I tried to convert the file using the heif-convert command:

user $ heif-convert IMG_3706.HEIC IMG_3706.jpg
File contains 1 images
Written to IMG_3706.jpg

4. Apparently Imagemagick >=7.0.7-22 compiled with --with-libheif is supposed to be able to convert HEIC files to JPG. Anyway, I tried to convert the file using the current version of Imagemagick in Lubuntu 18.04 (the current package version is 8:6.9.7.4+dfsg-16ubuntu6.4):

user $ convert IMG_3706.HEIC IMG_3706a.jpg
convert-im6.q16: no decode delegate for this image format `HEIC' @ error/constitute.c/ReadImage/504.
convert-im6.q16: no images defined `IMG_3706a.jpg' @ error/convert.c/ConvertImageCommand/3258.

5. Apparently the GIMP >=2.10.2 supports HEIF by using heif-gimp-plugin. Anyway, I tried to open the file with the current version of the GIMP in Lubuntu 18.04 (the current package version is 2.8.22-1). The GIMP launches and pops-up a window with the title ‘GIMP Message’ containing the following message and an ‘OK’ button:

GIMP Message
Opening /home/fitzcarraldo/IMG_3706.HEIC’ failed: Unknown file type

6. I used the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/) in a browser window. This can load the file IMG_3706.HEIC (‘Browse…’ button) and convert it (‘Save image…’ button) to IMG_3706.jpeg.

Gentoo Linux with KDE 5

1. I installed the libheif example tools implicitly by re-merging Imagemagick with USE="heif", which installs libheif.

root # cat /etc/portage/package.use/imagemagick 
media-gfx/imagemagick heif
root # emerge imagemagick

2. I used the heif-info command to check the file:

user $ heif-info IMG_3706.HEIC  
image: 3024x4032 (id=49), primary
  thumbnail: 240x320
  alpha channel: no
  depth channel: no

3. I tried to convert the file using the heif-convert command:

user $ heif-convert IMG_3706.HEIC IMG_3706.jpg
File contains 1 images
Written to IMG_3706.jpg

4. I tried to convert the file using Imagemagick >=7.0.7-22 compiled with --with-libheif (Imagemagick merged with USE="heif"):

user $ convert IMG_3706.HEIC IMG_3706a.jpg
user $

So Imagemagick 7.0.8.16 in Gentoo has no trouble with the file IMG_3706.HEIC.

5. I tried to open the file with the GIMP >=2.10.2, which supports HEIF using heif-gimp-plugin (GIMP >=2.10.6-r1 with USE="heif" in the case of Gentoo Linux)

First I re-merged the GIMP with the heif USE flag:

root # cat /etc/portage/package.accept_keywords/gimp
=media-gfx/gimp-2.10.8-r1 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/libmypaint-1.3.0 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-gfx/mypaint-brushes-1.3.0-r1 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/gegl-0.4.12 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/babl-0.1.60 ~amd64
root # cat /etc/portage/package.use/gimp
media-gfx/gimp heif
root # emerge -1vp gimp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ~] media-gfx/mypaint-brushes-1.3.0-r1:1.0::gentoo  2,390 KiB
[ebuild     U ~] media-libs/babl-0.1.60::gentoo [0.1.38::gentoo] USE="(-altivec)" CPU_FLAGS_X86="mmx sse sse2 sse3%* sse4_1 -f16c" 670 KiB
[ebuild  N     ] media-libs/gexiv2-0.10.8::gentoo  USE="-introspection -python -static-libs -test -vala" PYTHON_TARGETS="python2_7 python3_6 -python3_4 -python3_5" 620 KiB
[ebuild  NS   ~] media-libs/gegl-0.4.12:0.4::gentoo [0.2.0-r5:0::gentoo] USE="cairo ffmpeg introspection lcms openexr sdl svg tiff v4l -debug -jpeg2k -lensfun -libav -raw -test -umfpack -vala -webp" CPU_FLAGS_X86="mmx sse" 6,900 KiB
[ebuild  NS    ] media-libs/gegl-0.3.26:0.3::gentoo [0.2.0-r5:0::gentoo] USE="cairo ffmpeg introspection lcms openexr sdl svg tiff v4l -debug -jpeg2k -lensfun -raw -test -umfpack -vala -webp" CPU_FLAGS_X86="mmx sse" 6,378 KiB
[ebuild  N    ~] media-libs/libmypaint-1.3.0::gentoo  USE="gegl nls openmp -introspection" 428 KiB
[ebuild     U ~] media-gfx/gimp-2.10.8-r1:2::gentoo [2.8.22-r1:2::gentoo] USE="alsa heif%* mng openexr%* udev wmf -aalib (-altivec) (-aqua) -debug -doc -gnome -jpeg2k -postscript -python -smp -test -unwind% -vector-icons% -webp% -xpm (-bzip2%*) (-curl%) (-dbus%*) (-exif%*) (-jpeg%*) (-lcms%*) (-pdf%*) (-png%*) (-svg%*) (-tiff%*)" CPU_FLAGS_X86="mmx sse" PYTHON_TARGETS="python2_7" 31,206 KiB

Total: 7 packages (2 upgrades, 3 new, 2 in new slots), Size of downloads: 48,591 KiB

I then launched the GIMP and successfully opened the file IMG_3706.HEIC, and I was able to export it as IMG_3706.jpg.

6. As would be expected, the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/) behaves exactly the same in Gentoo Linux as it does in Lubuntu 18.04 (see earlier).

Summary

So there you have it; if the Linux file manager and/or image viewing applications you use cannot already handle HEIC files, the tools in Linux that I found may work are as follows:

  • heif-convert (from the package libheif-examples in Ubuntu/Lubuntu, or from from the package libheif in Gentoo).
  • Imagemagick (not every version).
  • The GIMP (not every version).
  • the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/).

I have not tried the copyright open-source code from Nokia (see link under Further Reading below), qt-heif-image-plugin and tifig (not in active development). If you have had success using another tool to convert HEIC files, please post a comment below for the benefit of other users, giving the name of the tool, the package name and version, and the Linux distribution (including release number, if not a rolling distribution).

Further reading

  1. Wikipedia – High Efficiency Image File Format
  2. Lifewire – What Are HEIF and HEIC, and Why Is Apple Using Them?
  3. Nokia – High Efficiency Image File Format (HEIF)
  4. libheif – a ISO/IEC 23008-12:2017 HEIF file format decoder and encoder
  5. askubuntu – Any app on Ubuntu to open HEIF (.heic, High Efficiency Image File Format) pictures?

A brief discussion about package installation times in Gentoo Linux

I thought that perhaps users of binary-based Linux distributions who are contemplating trying out the source-based distribution Gentoo Linux might be interested to know a bit about package installation times in contrast to binary distributions. I am not going to go into great detail here; this is just to give interested people a quick idea of possible package installation times in Gentoo Linux.

The package manager of a binary-based distribution such as Ubuntu downloads and installs ‘binary’ packages, i.e. packages containing pre-built executables. On the other hand, Gentoo’s package manager Portage downloads source-code packages and builds the binaries (executables) on your machine. Nevertheless, a small number of Portage packages contain binaries rather than source code, either because the source code could take many hours to build on older hardware or because the source code is simply not available in the public domain. An example of the first scenario is Firefox, which is available in Gentoo both as the source code package www-client/firefox and as the binary package www-client/firefox-bin so that the user can choose which to install (‘merge’, in Gentoo parlance). An example of the second scenario is TeamViewer, which is only available as the binary package net-misc/teamviewer because TeamViewer is closed-source software (i.e. the company that develops TeamViewer does not release its source code).

Clearly, installing an application from a package containing source code that has to be compiled and linked on your machine will take longer than installing the application from a package containing the executable binary that someone else has already built. The time to install from source-code packages of course depends on the precise package (some can take only a minute or so to install), the power of the machine on which the package is being installed, and various other factors.

I currently have two laptops running Gentoo Linux: one is over eight years old with an early Intel Core i7-720QM (throttled to 933 MHz because of the small PSU); the other is three and a half years old and has a newer Intel Core i7-4810MQ (2.8 GHz). Although the older laptop is slow by today’s standards, the newer laptop is still reasonably powerful. Even so, those few large source-code packages such as Firefox, Chromium and LibreOffice can take quite a long time to install, as illustrated below for Firefox by using the Gentoo utility genlop to find out the time it took to install:

clevow230ss /home/fitzcarraldo # genlop -t firefox | tail -n 3
     Wed Sep 26 19:53:47 2018 >>> www-client/firefox-62.0.2
       merge time: 1 hour, 4 minutes and 22 seconds.

On the other hand, installing TeamViewer took no time at all:

clevow230ss /home/fitzcarraldo # # genlop -t teamviewer | tail -n 3
     Mon Oct  1 21:39:29 2018 >>> net-misc/teamviewer-13.2.13582
       merge time: 16 seconds.

And youtube-dl also installed quickly:

clevow230ss /home/fitzcarraldo # genlop -t youtube-dl | tail -n 3
     Mon Oct  1 21:42:26 2018 >>> net-misc/youtube-dl-2018.09.26
       merge time: 53 seconds.

I also used to run Gentoo Linux on a 1998 Gateway Solo 9300 laptop (Intel Mobile Pentium III Coppermine 800 MHz). Neither its CPU nor its 288 MB RAM could cope with building large source code packages such as Firefox and LibreOffice, so on that machine I installed www-client/firefox-bin and app-office/libreoffice-bin instead.

Anyway, now let’s have a quick look at a typical package upgrade session, which I aim to perform at least once a week. On the above-mentioned newer laptop today I upgraded all installed packages that had newer versions available in the so-called Portage ‘tree’. This amounted to installing (‘merging’) the following fifteen packages:

  1. net-analyzer/traceroute-2.1.0
  2. media-libs/exiftool-11.03
  3. dev-libs/libtasn1-4.13
  4. x11-misc/shared-mime-info-1.10
  5. x11-libs/libxcb-1.13.1
  6. media-libs/x265-2.8
  7. dev-libs/redland-1.0.17-r1
  8. media-libs/mesa-18.1.9
  9. dev-libs/dbus-glib-0.110
  10. sys-power/upower-0.99.8
  11. app-text/libspectre-0.2.8
  12. app-crypt/pinentry-1.1.0-r2
  13. net-im/pidgin-2.13.0
  14. app-emulation/wine-vanilla-3.17
  15. media-video/handbrake-1.1.1

It took just under thirty-four minutes to install the above fifteen packages on my newer laptop. Below is what the upgrade process looked like in a terminal window. I have used the package manager’s long command options here, but normally I would use the single-letter shortcuts. As the focus of this post is on package installation time, I will not bother to show the command I used before this to synchronise (update) package information on the laptop with the remote package servers.

clevow230ss /home/fitzcarraldo # emerge --update --verbose --deep --newuse --with-bdeps=y --keep-going @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] net-analyzer/traceroute-2.1.0::gentoo [2.0.20::gentoo] USE="-static" 70 KiB
[ebuild     U  ] media-libs/exiftool-11.03::gentoo [10.95::gentoo] USE="-doc" 4,311 KiB
[ebuild   R    ] dev-libs/libtasn1-4.13:0/6::gentoo  USE="-doc -static-libs -test% -valgrind" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild     U  ] x11-misc/shared-mime-info-1.10::gentoo [1.9::gentoo] USE="-test" 603 KiB
[ebuild     U  ] x11-libs/libxcb-1.13.1:0/1.12::gentoo [1.13:0/1.12::gentoo] USE="xkb -doc (-selinux) -static-libs -test" ABI_X86="32 (64) (-x32)" 495 KiB
[ebuild  r  U  ] media-libs/x265-2.8:0/160::gentoo [2.6:0/146::gentoo] USE="10bit 12bit -numa -pic (-power8) -test" ABI_X86="(64) -32 (-x32)" 1,348 KiB
[ebuild     U  ] dev-libs/redland-1.0.17-r1::gentoo [1.0.16::gentoo] USE="berkdb -iodbc -mysql -odbc -postgres -sqlite -static-libs (-xml%*)" 1,584 KiB
[ebuild     U  ] media-libs/mesa-18.1.9::gentoo [18.1.6::gentoo] USE="classic dri3 egl gallium gbm llvm nptl wayland -bindist -d3d9 -debug -gles1 -gles2 -opencl -openmax -osmesa -pax_kernel -pic (-selinux) -unwind -vaapi -valgrind -vdpau -vulkan -xa -xvmc" ABI_X86="32 (64) (-x32)" VIDEO_CARDS="i915 i965 intel (-freedreno) (-imx) -nouveau -r100 -r200 -r300 -r600 -radeon -radeonsi (-vc4) -virgl (-vivante) -vmware" 10,885 KiB
[ebuild     U  ] dev-libs/dbus-glib-0.110::gentoo [0.108::gentoo] USE="-debug -static-libs -test" ABI_X86="(64) -32 (-x32)" 817 KiB
[ebuild     U  ] sys-power/upower-0.99.8:0/3::gentoo [0.99.5:0/3::gentoo] USE="introspection -doc -ios (-selinux)" 439 KiB
[ebuild     U  ] app-text/libspectre-0.2.8::gentoo [0.2.7::gentoo] USE="-debug -doc -static-libs" 412 KiB
[ebuild     U  ] app-crypt/pinentry-1.1.0-r2::gentoo [1.0.0-r2::gentoo] USE="gnome-keyring gtk ncurses qt5 -caps -emacs -fltk% -static" 457 KiB
[ebuild     U  ] net-im/pidgin-2.13.0:0/2::gentoo [2.12.0:0/2::gentoo] USE="dbus gnutls gstreamer gtk ncurses networkmanager nls spell xscreensaver zeroconf (-aqua) -debug -doc -eds -gadu -groupwise -idn -meanwhile -perl -pie -prediction -python -sasl -silc -tcl -tk -zephyr" PYTHON_TARGETS="python2_7" 8,784 KiB
[ebuild  NS   ~] app-emulation/wine-vanilla-3.17:3.17::gentoo [3.16:3.16::gentoo] USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner sdl ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gssapi -gstreamer -kerberos -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) -test -vkd3d -vulkan -xinerama" ABI_X86="32 64 (-x32)" 20,955 KiB
[ebuild  rR   ~] media-video/handbrake-1.1.1::gentoo  USE="fdk gstreamer gtk x265 -libav -libav-aac" 0 KiB

Total: 15 packages (12 upgrades, 1 in new slot, 2 reinstalls), Size of downloads: 51,156 KiB

The following packages are causing rebuilds:

  (media-libs/x265-2.8:0/160::gentoo, ebuild scheduled for merge) causes rebuilds for:
    (media-video/handbrake-1.1.1:0/0::gentoo, ebuild scheduled for merge)

>>> Verifying ebuild manifests
>>> Running pre-merge checks for app-emulation/wine-vanilla-3.17
>>> Emerging (1 of 15) net-analyzer/traceroute-2.1.0::gentoo
>>> Emerging (2 of 15) media-libs/exiftool-11.03::gentoo
>>> Emerging (3 of 15) dev-libs/libtasn1-4.13::gentoo
>>> Installing (1 of 15) net-analyzer/traceroute-2.1.0::gentoo
>>> Installing (2 of 15) media-libs/exiftool-11.03::gentoo
>>> Installing (3 of 15) dev-libs/libtasn1-4.13::gentoo
>>> Emerging (4 of 15) x11-misc/shared-mime-info-1.10::gentoo
>>> Installing (4 of 15) x11-misc/shared-mime-info-1.10::gentoo
>>> Emerging (5 of 15) x11-libs/libxcb-1.13.1::gentoo
>>> Installing (5 of 15) x11-libs/libxcb-1.13.1::gentoo
>>> Emerging (6 of 15) media-libs/x265-2.8::gentoo
>>> Emerging (7 of 15) dev-libs/redland-1.0.17-r1::gentoo
>>> Emerging (8 of 15) media-libs/mesa-18.1.9::gentoo
>>> Installing (7 of 15) dev-libs/redland-1.0.17-r1::gentoo
>>> Installing (6 of 15) media-libs/x265-2.8::gentoo
>>> Installing (8 of 15) media-libs/mesa-18.1.9::gentoo
>>> Emerging (9 of 15) dev-libs/dbus-glib-0.110::gentoo
>>> Installing (9 of 15) dev-libs/dbus-glib-0.110::gentoo
>>> Emerging (10 of 15) sys-power/upower-0.99.8::gentoo
>>> Emerging (11 of 15) app-text/libspectre-0.2.8::gentoo
>>> Emerging (12 of 15) app-crypt/pinentry-1.1.0-r2::gentoo
>>> Installing (10 of 15) sys-power/upower-0.99.8::gentoo
>>> Installing (11 of 15) app-text/libspectre-0.2.8::gentoo
>>> Installing (12 of 15) app-crypt/pinentry-1.1.0-r2::gentoo
>>> Emerging (13 of 15) net-im/pidgin-2.13.0::gentoo
>>> Emerging (14 of 15) app-emulation/wine-vanilla-3.17::gentoo
>>> Emerging (15 of 15) media-video/handbrake-1.1.1::gentoo
>>> Installing (15 of 15) media-video/handbrake-1.1.1::gentoo
>>> Installing (13 of 15) net-im/pidgin-2.13.0::gentoo
>>> Installing (14 of 15) app-emulation/wine-vanilla-3.17::gentoo
>>> Jobs: 15 of 15 complete                         Load avg: 2.94, 5.84, 6.01
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * Regenerating GNU info directory index...
 * Processed 129 info files.
 * After world updates, it is important to remove obsolete packages with
 * emerge --depclean. Refer to `man emerge` for more information.
 

The genlop tool in Gentoo can be used to check chronologically which packages were installed and removed. For example:

clevow230ss /home/fitzcarraldo # genlop -l | grep "Mon Oct  1"
     Mon Oct  1 18:16:19 2018 >>> net-analyzer/traceroute-2.1.0
     Mon Oct  1 18:16:27 2018 >>> media-libs/exiftool-11.03
     Mon Oct  1 18:16:49 2018 >>> dev-libs/libtasn1-4.13
     Mon Oct  1 18:17:02 2018 >>> x11-misc/shared-mime-info-1.10
     Mon Oct  1 18:17:49 2018 >>> x11-libs/libxcb-1.13.1
     Mon Oct  1 18:19:11 2018 >>> dev-libs/redland-1.0.17-r1
     Mon Oct  1 18:28:07 2018 >>> media-libs/x265-2.8
     Mon Oct  1 18:28:14 2018 >>> media-libs/mesa-18.1.9
     Mon Oct  1 18:28:30 2018 >>> dev-libs/dbus-glib-0.110
     Mon Oct  1 18:28:52 2018 >>> sys-power/upower-0.99.8
     Mon Oct  1 18:28:57 2018 >>> app-text/libspectre-0.2.8
     Mon Oct  1 18:29:03 2018 >>> app-crypt/pinentry-1.1.0-r2
     Mon Oct  1 18:30:02 2018 >>> media-video/handbrake-1.1.1
     Mon Oct  1 18:30:34 2018 >>> net-im/pidgin-2.13.0
     Mon Oct  1 18:47:45 2018 >>> app-emulation/wine-vanilla-3.17
clevow230ss /home/fitzcarraldo #

When packages are being upgraded, the package manager will inform you if a package has an update to the associated application’s configuration file. Even if the package manager does not indicate that any configuration files need to be updated, it does no harm to check anyway:

clevow230ss /home/fitzcarraldo # etc-update
Scanning Configuration files...
Exiting: Nothing left to do; exiting. :)

To remove packages that are no longer required (either because the package is no longer a dependency or because a newer version of a slotted* package has been installed), the ‘depclean‘ option is used. Below is what happened when I used that option to clean up after the above-mentioned fifteen packages were upgraded/re-installed.

* Some Gentoo packages are ‘slotted’, i.e. more than one version of the package can be installed and used concurrently. Examples include WINE, Python and kernel sources.

clevow230ss /home/fitzcarraldo # emerge --ask --depclean

 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace `.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 * 
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence of this, it often becomes necessary to run 
 * `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:                                                                                                                                           

 app-emulation/wine-vanilla
    selected: 3.16 
   protected: none 
     omitted: 3.17 

 dev-python/pyliblzma
    selected: 0.5.3-r1 
   protected: none 
     omitted: none 

 dev-python/twisted-web
    selected: 15.2.1 
   protected: none 
     omitted: none 

 dev-python/psutil
    selected: 5.4.3 
   protected: none 
     omitted: none 

 dev-python/twisted-core
    selected: 15.2.1 
   protected: none 
     omitted: none 

 dev-python/service_identity
    selected: 16.0.0 
   protected: none 
     omitted: none 

 dev-python/pyasn1-modules
    selected: 0.0.8 
   protected: none 
     omitted: none 

 dev-python/attrs
    selected: 16.3.0-r1 
   protected: none 
     omitted: none 

 dev-python/zope-interface
    selected: 4.4.3 
   protected: none 
     omitted: none 

 dev-python/namespace-zope
    selected: 1 
   protected: none 
     omitted: none 

All selected packages: =dev-python/pyliblzma-0.5.3-r1 =dev-python/namespace-zope-1 =dev-python/zope-interface-4.4.3 =dev-python/pyasn1-modules-0.0.8 =app-emulation/wine-vanilla-3.16 =dev-python/psutil-5.4.3 =dev-python/twisted-web-15.2.1 =dev-python/twisted-core-15.2.1 =dev-python/service_identity-16.0.0 =dev-python/attrs-16.3.0-r1

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] Yes
>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 10) app-emulation/wine-vanilla-3.16...
>>> Unmerging (2 of 10) dev-python/pyliblzma-0.5.3-r1...
>>> Unmerging (3 of 10) dev-python/twisted-web-15.2.1...
>>> Unmerging (4 of 10) dev-python/psutil-5.4.3...
>>> Unmerging (5 of 10) dev-python/twisted-core-15.2.1...
>>> Unmerging (6 of 10) dev-python/service_identity-16.0.0...
>>> Unmerging (7 of 10) dev-python/pyasn1-modules-0.0.8...
>>> Unmerging (8 of 10) dev-python/attrs-16.3.0-r1...
>>> Unmerging (9 of 10) dev-python/zope-interface-4.4.3...
>>> Unmerging (10 of 10) dev-python/namespace-zope-1...
Packages installed:   1714
Packages in world:    198
Packages in system:   43
Required packages:    1714
Number removed:       10

 * GNU info directory index is up-to-date.

As you can see above, the package manager found a number of installed packages that were no longer required, either because a new version of a slotted package had been installed (notice wine-vanilla) or because they are no longer dependencies. As I did not want to keep both wine-3.16 and wine-3.17 installed simultaneously, I simply replied ‘Yes’. By the way, the depclean option can also be used to remove (uninstall) older versions of an individual slotted package:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean wine-vanilla

It can also be used to remove an individual package if it is not a dependency. For example, the following command does not allow me to remove the installed package vlc because other installed packages depend on it:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean vlc

Calculating dependencies... done!
>>> No packages selected for removal by depclean
>>> To see reverse dependencies, use --verbose
Packages installed:   1714
Packages in world:    198
Packages in system:   43
Required packages:    1714
Number removed:       0

Whereas the following command would allow me to remove the installed package winetricks because no other installed package depends on it:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean winetricks

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:                                                                                                                                           

 app-emulation/winetricks
    selected: 20180815 
   protected: none 
     omitted: none 

All selected packages: =app-emulation/winetricks-20180815

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] No

Quitting.

Packages installed:   1714
Packages in world:    198
Packages in system:   43
Required packages:    1713
Number to remove:     1

Anyway, I hope this post has given potential adopters a rough idea of installation times in Gentoo Linux, even if it is far from thorough.

Installing Dropbox in Gentoo Linux following the recent restrictions introduced for Dropbox for Linux

In a 2013 post I explained how I installed Dropbox in Gentoo Linux running KDE 4. The Dropbox company has recently imposed some restrictions in the Linux client, so this is to explain what I did to get Dropbox working again in my two Gentoo Linux installations, both using the ext4 filesystem (unencrypted) and, these days, KDE Plasma 5.

Both my laptops running Gentoo Linux had a version of Dropbox installed via the Portage package manager: dropbox-45.3.88 in the case of the laptop running Gentoo amd64, and dropbox-48.3.56 in the case of the laptop running Gentoo ~amd64. Recently a Dropbox window popped up, warning me to upgrade Dropbox to the latest version within seven days otherwise the client would no longer be able to sync with the remote Dropbox server. I also received an e-mail from the Dropbox company titled ‘[Action required] We’re updating Linux system requirements‘ informing me that the only supported Linux distributions from now on would be Ubuntu 14.04 or higher and Fedora 21 or higher, and furthermore that the client will only work on an unencrypted ext4 filesystem. As both my Gentoo installations use unencrypted ext4, I was OK on that score, but I still had the problem that an up-to-date Dropbox ebuild is not available for Gentoo and the old Dropbox versions I was using no longer sync. However, I managed to install the latest version of Dropbox (currently 55.4.171) in Gentoo, and it works fine. The Dropbox client’s icon is on the KDE Plasma 5 Panel, and the local Dropbox directory is being sync’ed correctly. Below I explain what I did.

1. I selected ‘Quit Dropbox’ from the old Dropbox client’s menu, and the Dropbox icon disappeared from the Panel.

2. I removed the Dropbox daemon from the list of script files to be started at login (‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’).

3. I unmerged (uninstalled) the dropbox package:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean dropbox

4. I deleted the directories ~/.dropbox and ~/.dropbox-dist but kept the directory ~/Dropbox and its contents.

fitzcarraldo@clevow230ss ~ $ rm -rf ~/.dropbox ~/.dropbox-dist

5. I followed the instructions under ‘Dropbox Headless install via command line‘ on the Dropbox Website to re-install the latest version of the daemon and client:

fitzcarraldo@clevow230ss ~ $ cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf -

6. I configured KDE Plasma 5 to start ~/.dropbox-dist/dropboxd at login (‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’ > ‘Add Script…’).

7. I launched ~/.dropbox-dist/dropboxd manually from a Konsole window. The Dropbox client icon appeared on the Panel and I was prompted to login to my Dropbox account via a Web browser, as per the instructions on the Dropbox Website (see link in in Step 5):

If you’re running Dropbox on your server for the first time, you’ll be asked to copy and paste a link in a working browser to create a new account or add your server to an existing account. Once you do, your Dropbox folder will be created in your home directory.

8. I logged in to my Dropbox account via the Firefox browser. As soon as I had logged in via the browser, a message appeared in the browser window informing me that “Your computer was successfully linked to your account”, and the Dropbox client icon appeared on the Panel and showed that the contents of ~/Dropbox were being synchronised.

Everything seems to be working as before. The Dropbox icon on the Panel has the same menu items it had previously. ‘Preferences…’ shows the Dropbox version as v55.4.171. I have not ticked ‘Start Dropbox on system startup’ under Dropbox Preferences because I configured automatic startup using KDE Plasma 5 ‘System Settings’ as described in Step 6 above, and the Dropbox daemon is indeed started automatically when I login.

The Dropbox Website’s instructions (see link in Step 5) also include the following:

Download this Python script to control Dropbox from the command line. For easy access, put a symlink to the script anywhere in your PATH.

I did download that Python script and made it executable:

fitzcarraldo@clevow230ss ~/Dropbox $ chmod +x dropbox.py

However the Python 3.6 interpreter in my Gentoo Linux installations report a syntax error in the script when I run it, I assume because it was written for a different version of Python:

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py 
  File "./dropbox.py", line 233
    except OSError, e:
                  ^
SyntaxError: invalid syntax

Anyway, as the Dropbox client icon is on the KDE Plasma 5 Panel and I can control Dropbox from there, I see no need for the Python script.

9. My Gentoo installations have a Bash script ~/dbox.sh that I had created to be launched by a Desktop Configuration file ~/Desktop/Dropbox.desktop with a nice icon which I double-click on if I want to relaunch the Dropbox daemon (if I previously quit Dropbox from the client’s menu, for example). I had to modify ~/dbox.sh by replacing the command ‘dbus-launch dropbox start > /dev/null‘ with the command ‘/home/fitzcarraldo/.dropbox-dist/dropboxd‘ as shown below.

dbox.sh

#!/bin/bash
notify-send 'Launching Dropbox' 'Daemon will be (re)started in 20 seconds' --icon=dialog-information
sleep 20s
ps auxww | awk '$0~/dropbox/&&$0!~/awk/{print $2}' | xargs kill
/home/fitzcarraldo/.dropbox-dist/dropboxd

Dropbox.desktop

[Desktop Entry]
Comment[en_GB]=(re)launch Dropbox daemon
Comment=(re)launch Dropbox daemon
Exec=/home/fitzcarraldo/dbox.sh
GenericName[en_GB]=Dropbox
GenericName=Dropbox
Icon=kipi-dropbox
MimeType=
Name[en_GB]=Dropbox
Name=Dropbox
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

10. At the moment Dropbox is working fine again in my Gentoo installations. However, I noticed that Gentoo Linux user zsitvaij posted the following comment in a Gentoo Forums thread:

On every dropbox update, I have to remove ~/.dropbox-dist/dropbox-lnx./libdrm.so.2 to avoid having it crash on launch, works fine after until they update again.

I do not know if that will be necessary in my case, as I have not yet had to upgrade Dropbox from the Version 55.4.171 that I recently installed. When a new version of Dropbox becomes available I will update this post to confirm whether or not I had to do anything to keep Dropbox working.

Addendum (5 October 2019): With reference to my addendum of 31 August 2018, the Python script dropbox.py that can be downloaded from the Dropbox Web site has been updated and is now written in Python 3, so you can ignore my addendum of 31 August 2018.

Addendum (1 October 2018): With reference to my addendum of 2 September 2018, if you are using OpenRC it is possible to automate the deletion of the file ~/.dropbox-dist/dropbox-lnx.x86_64-/libdrm.so.2 by creating a Bash script /etc/local.d/40dropbox.start containing the following:

#!/bin/bash
if [ -e /home/fitzcarraldo/.dropbox-dist/dropbox-lnx.x86_64-*/libdrm.so.2 ]
then
    rm /home/fitzcarraldo/.dropbox-dist/dropbox-lnx.x86_64-*/libdrm.so.2
fi

Replace my username with your username, obviously. Of course the conditional test could be dispensed with and the script could just contain the shebang line and the rm line, which would still work even if the file does not exist, but it feels a bit tidier to only attempt to delete the file if it actually exists.

Addendum (2 September 2018): I have just installed Dropbox Version 56.4.94 in my Gentoo ~amd64 installation and I had to use the command shown below once in order to stop the daemon segfaulting when I entered the command ~/.dropbox-dist/dropboxd in a Konsole window:

fitzcarraldo@clevow230ss ~/Dropbox $ rm ~/.dropbox-dist/dropbox-lnx.x86_64-56.4.94/libdrm.so.2

Addendum (31 August 2018): The Python script dropbox.py that can be downloaded from the Dropbox Web site (see Step 8 above) is old, as can be seen in the comments in the header of the script:

# Dropbox frontend script
# This file is part of nautilus-dropbox 2015.10.28.

It is written in Python 2. Although I do not need to use it, I managed to get it to run in my Gentoo installations by replacing the shebang line ‘#!/usr/bin/python‘ with ‘#!/usr/bin/env python2‘. This works in my Gentoo installations because they have both Python 2.7 and Python 3.6 installed. When I now run dropbox.py I see the following:

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py 
Dropbox command-line interface

commands:

Note: use dropbox help  to view usage for a specific command.

 status       get current status of the dropboxd
 throttle     set bandwidth limits for Dropbox
 help         provide help
 stop         stop dropboxd
 running      return whether dropbox is running
 start        start dropboxd
 filestatus   get current sync status of one or more files
 ls           list directory contents with current sync status
 autostart    automatically start dropbox at login
 exclude      ignores/excludes a directory from syncing
 lansync      enables or disables LAN sync
 sharelink    get a shared link for a file in your dropbox
 proxy        set proxy settings for Dropbox

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py status
Up to date
fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py running
fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py filestatus ~/Dropbox/Getting\ Started.pdf 
/home/fitzcarraldo/Dropbox/Getting Started.pdf: up to date
fitzcarraldo@clevow230ss ~/Dropbox $

Notice that the command ./dropbox.py running does not return anything even though the daemon is definitely running, so I do not trust the script anyway.

Installing the Onboard on-screen keyboard in Gentoo Linux

Onboard on-screen keyboard with the Compact British English layout, Droid theme and international character selection enabled for the long-press action

Onboard on-screen keyboard configured with the British English layout, Droid theme and international character selection enabled for the long-press action



The most sophisticated and polished virtual keyboard I have seen so far in Linux is Onboard, the on-screen keyboard previously provided in Ubuntu prior to the switch to GNOME 3. The current version of Onboard is 1.4.1 and it can be installed and used in other Linux distributions and desktop environments. Thanks to Gentoo Linux user wjn an ebuild is available in the wjn-overlay overlay and can be installed from there via Portage. However, I prefer to use a local overlay so I copied the ebuild to my local overlay and installed it from there instead. Here is what I did to install app-accessibility/onboard in the Gentoo Stable installation running KDE Plasma 5 on my Clevo W230SS laptop:

1. I first made sure the preferred Python interpreter was selected (I should have done that when the Gentoo Linux developers recently replaced Python 3.5 with Python 3.6 in the default Python targets for Gentoo installations):

root # grep PYTHON /etc/portage/make.conf
PYTHON_TARGETS="python2_7 python3_6"
PYTHON_SINGLE_TARGET="python3_6"
root # eselect python list
Available Python interpreters, in order of preference:
  [1]   python2.7
  [2]   python3.5 (uninstalled)
  [3]   python3.4
  [4]   python3.6 (fallback)
root # eselect python set 4
root # eselect python list 
Available Python interpreters, in order of preference:
  [1]   python3.6
  [2]   python2.7
  [3]   python3.5 (uninstalled)
  [4]   python3.4

2. Then I copied the ebuild to my local overlay:

root # mkdir -p /usr/local/portage/app-accessibility/onboard/files
root # cd /usr/local/portage/app-accessibility/onboard/files/
root # wget https://bitbucket.org/wjn/wjn-overlay/raw/5d7fe162af7c0cde9b401a9a30fb3ab8b2b65e3d/app-accessibility/onboard/files/# onboard-1.4.1-remove-duplicated-docs.patch
root # cd ..
root # wget wget https://bitbucket.org/wjn/wjn-overlay/raw/5d7fe162af7c0cde9b401a9a30fb3ab8b2b65e3d/app-accessibility/onboard/onboard-1.4.1.ebuild
root # ebuild onboard-1.4.1.ebuild manifest

3. As I am using using Gentoo Stable I unmasked the ebuild by keyword:

root # nano /etc/portage/package.accept_keywords/onboard
root # cat /etc/portage/package.accept_keywords/onboard
=app-accessibility/onboard-1.4.1 **

4. Then I installed the package:

root # emerge onboard
root # eix onboard
[I] app-accessibility/onboard [1]
     Available versions:  (~)1.4.1^m {PYTHON_TARGETS="python3_5 python3_6"}
     Installed versions:  1.4.1^m(15:28:57 25/06/18)(PYTHON_TARGETS="python3_6 -python3_4 -python3_5")
     Homepage:            https://launchpad.net/onboard
     Description:         Onscreen keyboard for everybody who can't use a hardware keyboard

[1] "local_overlay" /usr/local/portage

Icons for Onboard and Onboard Settings were added to the KDE Application Launcher’s menu (Applications > Utilities) and they can be launched from there or by entering the command ‘onboard‘ in a Konsole window under the user’s account.

5. The only thing that did not work ‘out of the box’ in KDE Plasma 5.12.5 in Gentoo was selecting ‘Help’ from the pop-up menu displayed by clicking on the Onboard icon on the Plasma 5 Panel:

FileNotFoundError: [Errno 2] No such file or directory: ‘/usr/bin/yelp’: ‘/usr/bin/yelp’

This was simply because the package gnome-extra/yelp had not been installed in my KDE installation. Now, I could have just installed it separately:

root # emerge yelp

but I chose intead to edit the onboard ebuild to add yelp to the list of runtime dependencies:

RDEPEND="${COMMON_DEPEND}
        app-accessibility/at-spi2-core
        app-text/iso-codes
        gnome-extra/mousetweaks
        gnome-extra/yelp
        x11-libs/libxkbfile"

and I then re-installed the package, which then automatically installed yelp and its dependencies:

root # ebuild onboard-1.4.1.ebuild manifest
root # emerge onboard

Onboard is a nice utility, and I hope its developers continue to maintain and develop it even though Ubuntu now uses the GNOME 3 on-screen keyboard instead, as it can be used in other desktop environments and in other Linux distributions.

Trouble again with PulseAudio and Thunderbird sound notifications

In an earlier post I described how I fixed a scratchy-sounding sound file which the Thunderbird e-mail client plays when a new message arrives. Well, the problem started again recently, but this time the contents of /etc/pulse/daemon.conf looked OK to me. Furthermore, the sound file sounds fine when played using following commands:

aplay ~/Music/wav/E-mail_notifications/halmsg.wav
paplay ~/Music/wav/E-mail_notifications/halmsg.wav
mplayer ~/Music/wav/E-mail_notifications/halmsg.wav
cvlc ~/Music/wav/E-mail_notifications/halmsg.wav

Now, Thunderbird uses libcanberra to play sounds, so I began to wonder if the problem lay with libcanberra. As it happens, libcanberra is maintained by the same person who invented PulseAudio. However, I notice from the libcanberra Git repository that its source code has not been changed since 2012.

My Gentoo Linux installation had libcanberra installed with support for both ALSA and PulseAudio:

root # eix -I libcanberra
[I] media-libs/libcanberra
     Available versions:  0.30-r5 {alsa gnome gstreamer +gtk +gtk3 oss pulseaudio +sound tdb udev ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  0.30-r5(08:27:41 18/05/18)(alsa gtk gtk3 pulseaudio sound udev -gnome -gstreamer -oss -tdb ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://git.0pointer.net/libcanberra.git/
     Description:         Portable sound event library

So, even though my installation uses PulseAudio, I decided to try and re-install libcanberra without PulseAudio support, only ALSA support:

root # USE="-pulseaudio" emerge -1v libcanberra
root # eix -I libcanberra
[I] media-libs/libcanberra
     Available versions:  0.30-r5 {alsa gnome gstreamer +gtk +gtk3 oss pulseaudio +sound tdb udev ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  0.30-r5(15:47:14 26/05/18)(alsa gtk gtk3 sound udev -gnome -gstreamer -oss -pulseaudio -tdb ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://git.0pointer.net/libcanberra.git/
     Description:         Portable sound event library

Lo and behold, Thunderbird (libcanberra) plays the sound file correctly now. So I have added the following line to my file /etc/portage/package.use/thunderbird in order to make the change permanent:

media-libs/libcanberra -pulseaudio

PulseAudio 🙄