HEIC image files in Linux

I was at an event recently where the attendees were asked to upload their camera and smartphone photos and videos to a shared Google Drive folder. Some of the uploaded photo files have a .HEIC (High Efficiency Image Container) extension, which I had not come across before. I have since learnt that these HEIC files were produced by iPhones running iOS 11, encoded using the HEIF (High Efficiency Image File) format. Apparently the HEIF format is superior to the JPEG format in a number of ways (see the links at the end of my post, especially the image examples given by Nokia), although it is subject to patents and therefore I believe there are certain constraints to coding image files in HEIC format. Anyway, I’ll leave you to read the fine print. My interest was simply because I wanted to be able to download the above-mentioned photo files and view them all in the file managers and image-viewing applications in Linux and Android on my various devices.

Now, I can browse and view the above-mentioned shared HEIC images in Google Drive in the Firefox and Chrome browsers in Linux, although an ownCloud site viewed using the same browsers displays the HEIC files as grey icons that can only be downloaded, not opened and viewed in the browser. I also found that Cirrus, the Android app for ownCloud that I use on my Galaxy Note 8 phone, cannot display HEIC photos either.

I downloaded the HEIC files to a machine running Lubuntu 18.04 and to a machine running Gentoo Linux. The file manager PCManFM in Lubuntu 18.04 displays grey icons rather than thumbnails for these HEIC files, and KDE’s Dolphin 18.08.3 file manager in Gentoo Linux displays green image icons rather than thumbnails for them. As far as Linux image viewers go, in Lubuntu 18.04 I find that GPicView 0.2.5 and Geeqie 1.4 cannot display HEIC images, and in Gentoo Linux KDE I find that GQview 2.1.5-r1, Okular 18.08.3 and Gwenview 18.08.3 cannot display HEIC images. So I set about converting all the HEIC files to JPG files. I managed to do this but needed to use a range of tools, as illustrated by a couple of examples below for Lubuntu 18.04 and Gentoo Linux. This post might seem long-winded but perhaps may be of help to Linux users coming across .HEIC files for the first time.

From the .HEIC files I had downloaded I picked one at random to try and convert to a JPG file: IMG_3706.HEIC. Its EXIF data confirms it is an HEIC file:

user $ exiftool IMG_3706.HEIC | grep "File Type"
File Type                       : HEIC
File Type Extension             : heic
user $ exiftool IMG_3706.HEIC | grep "Camera Model"
Camera Model Name               : iPhone 7 Plus

Several of the files with the .HEIC suffix that I downloaded were not real HEIC files according to their EXIF data:

user $ exiftool IMG_9474.HEIC | grep "File Type"
File Type                       : JPEG
File Type Extension             : jpg
user $ exiftool IMG_9474.HEIC | grep "Camera Model"
Camera Model Name               : iPhone 8

Those files were apparently treated as JPEG files by the tools I mention below, so I have omitted the results for those ‘false’ HEIC files.

Lubuntu 18.04

1. I installed the libheif example tools:

user $ sudo apt install libheif-examples

2. I used the heif-info command to check the file:

user $ heif-info IMG_3706.HEIC 
image: 3024x4032 (id=49), primary
  thumbnail: 240x320
  alpha channel: no
  depth channel: no

3. I tried to convert the file using the heif-convert command:

user $ heif-convert IMG_3706.HEIC IMG_3706.jpg
File contains 1 images
Written to IMG_3706.jpg

4. Apparently Imagemagick >=7.0.7-22 compiled with --with-libheif is supposed to be able to convert HEIC files to JPG. Anyway, I tried to convert the file using the current version of Imagemagick in Lubuntu 18.04 (the current package version is 8:

user $ convert IMG_3706.HEIC IMG_3706a.jpg
convert-im6.q16: no decode delegate for this image format `HEIC' @ error/constitute.c/ReadImage/504.
convert-im6.q16: no images defined `IMG_3706a.jpg' @ error/convert.c/ConvertImageCommand/3258.

5. Apparently the GIMP >=2.10.2 supports HEIF by using heif-gimp-plugin. Anyway, I tried to open the file with the current version of the GIMP in Lubuntu 18.04 (the current package version is 2.8.22-1). The GIMP launches and pops-up a window with the title ‘GIMP Message’ containing the following message and an ‘OK’ button:

GIMP Message
Opening /home/fitzcarraldo/IMG_3706.HEIC’ failed: Unknown file type

6. I used the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/) in a browser window. This can load the file IMG_3706.HEIC (‘Browse…’ button) and convert it (‘Save image…’ button) to IMG_3706.jpeg.

Gentoo Linux with KDE 5

1. I installed the libheif example tools implicitly by re-merging Imagemagick with USE="heif", which installs libheif.

root # cat /etc/portage/package.use/imagemagick 
media-gfx/imagemagick heif
root # emerge imagemagick

2. I used the heif-info command to check the file:

user $ heif-info IMG_3706.HEIC  
image: 3024x4032 (id=49), primary
  thumbnail: 240x320
  alpha channel: no
  depth channel: no

3. I tried to convert the file using the heif-convert command:

user $ heif-convert IMG_3706.HEIC IMG_3706.jpg
File contains 1 images
Written to IMG_3706.jpg

4. I tried to convert the file using Imagemagick >=7.0.7-22 compiled with --with-libheif (Imagemagick merged with USE="heif"):

user $ convert IMG_3706.HEIC IMG_3706a.jpg
user $

So Imagemagick in Gentoo has no trouble with the file IMG_3706.HEIC.

5. I tried to open the file with the GIMP >=2.10.2, which supports HEIF using heif-gimp-plugin (GIMP >=2.10.6-r1 with USE="heif" in the case of Gentoo Linux)

First I re-merged the GIMP with the heif USE flag:

root # cat /etc/portage/package.accept_keywords/gimp
=media-gfx/gimp-2.10.8-r1 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/libmypaint-1.3.0 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-gfx/mypaint-brushes-1.3.0-r1 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/gegl-0.4.12 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/babl-0.1.60 ~amd64
root # cat /etc/portage/package.use/gimp
media-gfx/gimp heif
root # emerge -1vp gimp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ~] media-gfx/mypaint-brushes-1.3.0-r1:1.0::gentoo  2,390 KiB
[ebuild     U ~] media-libs/babl-0.1.60::gentoo [0.1.38::gentoo] USE="(-altivec)" CPU_FLAGS_X86="mmx sse sse2 sse3%* sse4_1 -f16c" 670 KiB
[ebuild  N     ] media-libs/gexiv2-0.10.8::gentoo  USE="-introspection -python -static-libs -test -vala" PYTHON_TARGETS="python2_7 python3_6 -python3_4 -python3_5" 620 KiB
[ebuild  NS   ~] media-libs/gegl-0.4.12:0.4::gentoo [0.2.0-r5:0::gentoo] USE="cairo ffmpeg introspection lcms openexr sdl svg tiff v4l -debug -jpeg2k -lensfun -libav -raw -test -umfpack -vala -webp" CPU_FLAGS_X86="mmx sse" 6,900 KiB
[ebuild  NS    ] media-libs/gegl-0.3.26:0.3::gentoo [0.2.0-r5:0::gentoo] USE="cairo ffmpeg introspection lcms openexr sdl svg tiff v4l -debug -jpeg2k -lensfun -raw -test -umfpack -vala -webp" CPU_FLAGS_X86="mmx sse" 6,378 KiB
[ebuild  N    ~] media-libs/libmypaint-1.3.0::gentoo  USE="gegl nls openmp -introspection" 428 KiB
[ebuild     U ~] media-gfx/gimp-2.10.8-r1:2::gentoo [2.8.22-r1:2::gentoo] USE="alsa heif%* mng openexr%* udev wmf -aalib (-altivec) (-aqua) -debug -doc -gnome -jpeg2k -postscript -python -smp -test -unwind% -vector-icons% -webp% -xpm (-bzip2%*) (-curl%) (-dbus%*) (-exif%*) (-jpeg%*) (-lcms%*) (-pdf%*) (-png%*) (-svg%*) (-tiff%*)" CPU_FLAGS_X86="mmx sse" PYTHON_TARGETS="python2_7" 31,206 KiB

Total: 7 packages (2 upgrades, 3 new, 2 in new slots), Size of downloads: 48,591 KiB

I then launched the GIMP and successfully opened the file IMG_3706.HEIC, and I was able to export it as IMG_3706.jpg.

6. As would be expected, the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/) behaves exactly the same in Gentoo Linux as it does in Lubuntu 18.04 (see earlier).


So there you have it; if the Linux file manager and/or image viewing applications you use cannot already handle HEIC files, the tools in Linux that I found may work are as follows:

  • heif-convert (from the package libheif-examples in Ubuntu/Lubuntu, or from from the package libheif in Gentoo).
  • Imagemagick (not every version).
  • The GIMP (not every version).
  • the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/).

I have not tried the copyright open-source code from Nokia (see link under Further Reading below), qt-heif-image-plugin and tifig (not in active development). If you have had success using another tool to convert HEIC files, please post a comment below for the benefit of other users, giving the name of the tool, the package name and version, and the Linux distribution (including release number, if not a rolling distribution).

Further reading

  1. Wikipedia – High Efficiency Image File Format
  2. Lifewire – What Are HEIF and HEIC, and Why Is Apple Using Them?
  3. Nokia – High Efficiency Image File Format (HEIF)
  4. libheif – a ISO/IEC 23008-12:2017 HEIF file format decoder and encoder
  5. askubuntu – Any app on Ubuntu to open HEIF (.heic, High Efficiency Image File Format) pictures?

How to display the times in various time zones from the LXDE Panel

I normally check the time in the time zone of family, friends and colleagues who live in various places around the World before I call or message them. In KDE Plasma on my laptop it is possible to configure the digital clock widget to display the times in a list of time zones of my choice when I hover the mouse pointer over the widget. However, my family’s PC has Lubuntu 18.04 installed, which uses LXDE, and the digital clock on the LXDE Panel does not have that ability. Therefore I installed the GUI utility gworldclock in Lubuntu 18.04:

$ sudo apt-get install gworldclock

I added it to the ‘Application Launch and Task Bar’ plugin on the LXDE Panel, and a clock icon is now displayed on the Panel.

gworldclock icon on Application Launch and Task Bar on LXDE Panel

gworldclock icon on Application Launch and Task Bar on LXDE Panel

When I click the clock icon, a window opens on the Desktop and displays the date and time at each of the World locations I configured in gworldclock (‘Options’ > ‘Add Timezone’). Excellent, and almost as convenient as the World time feature in the Digital Clock widget in KDE Plasma 5.

gworldclock window default size

gworldclock window default size

I have configured gworldclock to display a list of ten additional time zones when I click on the clock icon on the Application Launch and Task Bar on the Panel. However, the size of the pop-up gworldclock window was relatively small; only six of the eleven time zones were visible, so I had to use the window’s scroll bar in order to view some of the configured time zone entries. I therefore made some changes in LXDE in order to display a larger gworldclock window showing all eleven time zones. This is how I did it.

1. I installed wmctrl:

$ sudo apt-get install wmctrl

2. I created a hidden Bash script ~/.gworldclockfitzcarraldo.sh containing the following:

gworldclock &
sleep 0.5s
wmctrl -F gworldclock -r gworldclock -e 0,500,300,300,340

and made it executable:

$ chmod +x ~/.gworldclockfitzcarraldo.sh

See man wmctrl for the meaning of the options in the above-mentioned Bash script.

3. I created the Desktop Configuration File ~/.local/share/applications/gworldclockfitzcarraldo.desktop containing the following:

[Desktop Entry]
Comment=See the time in other timezones
GenericName=World Clock
Comment[fr]=Voir l'heure dans d'autres fuseaux horaires

4. I edited the file ~/.config/lxpanel/Lubuntu/panels/panel and added an entry for the new Desktop Configuration File to the end of list for the Application Launch and Task Bar, as shown in the following excerpt from the file:

Plugin {
  Config {
    Button {
    Button {
    Button {
    Button {
    Button {
    Button {
    Button {

Then I logged out and back in again. Now, when I click on the clock icon on the Panel, the gworldclock window opens at the location and size specified by the wmctrl command in the Bash script I created.

gworldclock window resized by the Bash script

gworldclock window resized by the Bash script

Configure a keyboard shortcut in Lubuntu 18.04 to take a screenshot of a screen region

As installed, Lubuntu 18.04 is configured so that the user can capture a screenshot of the whole screen by pressing the PrtScrn key, and a screenshot of the active window by pressing Alt+PrtScrn. However, no keyboard shortcut is configured to enable the user to capture a user-specified region of the screen.

Now, as it happens, the ‘-s‘ option of the scrot command allows a region of the screen to be captured and saved. The man page for scrot tells us:

-s, --select
Interactively select a window or rectangle with the mouse.

So here is how to configure a keyboard shortcut to do that in Lubuntu 18.04.

Open the file ~/.config/openbox/lubuntu-rc.xml in a text editor (either nano from the command line or LXTerminal from the GUI) and look for the following lines:

    <keybind key="Print">
      <action name="Execute">
        <command>lxsession-default screenshot</command>
    <keybind key="A-Print">
      <action name="Execute">
        <command>lxsession-default screenshot window</command>

Append the following lines to that group of lines:

    <keybind key="C-A-Print">
      <action name="Execute">
        <command>scrot -s</command>

The new group of lines should then look like this:

    <keybind key="Print">
      <action name="Execute">
        <command>lxsession-default screenshot</command>
    <keybind key="A-Print">
      <action name="Execute">
        <command>lxsession-default screenshot window</command>
    <keybind key="C-A-Print">
      <action name="Execute">
        <command>scrot -s</command>

Now logout and login again.

If you now press Ctrl+Alt+PrtScrn and use your mouse to select a rectangular region of the screen, a screenshot of that region will be saved automatically to your home directory. Simple.

You just have to remember:

PrtScrn — Captures the entire screen.
Alt+PrtScrn — Captures the active window.
Ctrl+Alt+PrtScrn — Captures the area of the screen you select with the mouse.

How to create missing favicons for your bookmarks in Firefox Quantum for Linux

Prior to the advent of Firefox Quantum, several add-ons were available that enabled you to replace the favicons of bookmarks in Firefox, or to create custom favicons for bookmarks lacking a favicon. One of my favourite such add-ons was Favicon Picker 3. However, Firefox Quantum currently lacks an add-on that would enable you to insert a favicon of your choice for bookmarks without one. This has annoyed me for some time, as my bookmarks menu in Firefox includes several Web sites without favicons. Today I found a 2013 post ‘How to change Firefox bookmark icons (favicons) without extensions or addons!‘ for Windows predating Firefox Quantum. I have applied a similar procedure for Firefox Quantum in Linux in order to add a custom favicon to each bookmark in my bookmarks menu that did not already have a favicon. The procedure I used is given below.

1. Launch Firefox and enter ‘about:support’ (without the quotes) in the address bar.

2. Click on ‘Profile Directory’ > ‘Open Directory’ (which, in my case, is the directory /home/fitzcarraldo/.mozilla/firefox/l7nt0jx2.default).

3. Create the sub-directory ‘chrome‘.

4. Open the new directory’s folder.

5. Create the file ‘userChrome.css‘ in the new directory.

6. Open the file userChrome.css with a text editor and paste the following CSS code into it:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");

.bookmark-item[label="<bookmark name>"] image {
    padding: 0 0 16px 16px !important;
    background:url(<base64 string>)!important; 

7. Make the following changes to the above code:

7.1 Replace ‘<bookmark name>’ with the name of the bookmark as it appears in the bookmarks menu. You must supply a name, even if you do not want a name to appear.

7.2 Replace ‘<base64 string>’ with the base64-encoded string of the desired favicon, which you can obtain by uploading the 16×16 pixel image file to a website such as https://www.base64-image.de/.


Below are the steps I took to create a favicon for the bookmark for Hotmail I have in Firefox:

1. I used a search engine to find an image of the Microsoft ‘flag’ logo. The image happened to be a 200×200 pixels PNG file. I downloaded it to my home directory and named it ‘Microsoft.png‘.

2. I loaded the image file into the GIMP, scaled it to 16×16 pixels and exported it as Microsoft.png to overwrite the original downloaded file.

3. I viewed the Web page https://www.base64-image.de/ in a Web browser and clicked on the button ‘OR CLICK HERE’ to upload my image file Microsoft.png to that site. Then I clicked on the button ‘</> show code’, selected and copied the text displayed underneath ‘For use as CSS background:’ that looked like this:


4. I created the directory ~/.mozilla/firefox/l7nt0jx2.default/chrome and the file userChrome.css containing the following:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");

.bookmark-item[label="Hotmail"] image {
    padding: 0 0 16px 16px !important;

5. I restarted Firefox, and my custom favicon is now visible for the Hotmail entry in the Bookmarks Menu. I then deleted the file ~/Microsoft.png as it is no longer required.

6. I also wanted to create a custom favicon for another bookmark I have, the Web site of a Brazilian newspaper Correio Popular. As I was unable to find an image that would be suitable for a favicon, I created one myself (16×16 pixels) using the GIMP, and exported it as the file ‘~/Correio Popular.png‘.

7. I followed the same procedure to encode the image file as a Base64 image, and I added another entry to the file ~/.mozilla/firefox/l7nt0jx2.default/chrome/userChrome.css, which now contains the following:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");

.bookmark-item[label="Hotmail"] image {
    padding: 0 0 16px 16px !important;

.bookmark-item[label="Correio Popular"] image {
    padding: 0 0 16px 16px !important;

8. For any other bookmarks that do not have a favicon, I will just repeat the procedure to add further entries to the file ~/.mozilla/firefox/l7nt0jx2.default/chrome/userChrome.css. It is not as easy as using an add-on, but, until someone creates one, the process described above at least allows me to fill those annoying favicon gaps in the Firefox bookmark menus on my machines.

A brief discussion about package installation times in Gentoo Linux

I thought that perhaps users of binary-based Linux distributions who are contemplating trying out the source-based distribution Gentoo Linux might be interested to know a bit about package installation times in contrast to binary distributions. I am not going to go into great detail here; this is just to give interested people a quick idea of possible package installation times in Gentoo Linux.

The package manager of a binary-based distribution such as Ubuntu downloads and installs ‘binary’ packages, i.e. packages containing pre-built executables. On the other hand, Gentoo’s package manager Portage downloads source-code packages and builds the binaries (executables) on your machine. Nevertheless, a small number of Portage packages contain binaries rather than source code, either because the source code could take many hours to build on older hardware or because the source code is simply not available in the public domain. An example of the first scenario is Firefox, which is available in Gentoo both as the source code package www-client/firefox and as the binary package www-client/firefox-bin so that the user can choose which to install (‘merge’, in Gentoo parlance). An example of the second scenario is TeamViewer, which is only available as the binary package net-misc/teamviewer because TeamViewer is closed-source software (i.e. the company that develops TeamViewer does not release its source code).

Clearly, installing an application from a package containing source code that has to be compiled and linked on your machine will take longer than installing the application from a package containing the executable binary that someone else has already built. The time to install from source-code packages of course depends on the precise package (some can take only a minute or so to install), the power of the machine on which the package is being installed, and various other factors.

I currently have two laptops running Gentoo Linux: one is over eight years old with an early Intel Core i7-720QM (throttled to 933 MHz because of the small PSU); the other is three and a half years old and has a newer Intel Core i7-4810MQ (2.8 GHz). Although the older laptop is slow by today’s standards, the newer laptop is still reasonably powerful. Even so, those few large source-code packages such as Firefox, Chromium and LibreOffice can take quite a long time to install, as illustrated below for Firefox by using the Gentoo utility genlop to find out the time it took to install:

clevow230ss /home/fitzcarraldo # genlop -t firefox | tail -n 3
     Wed Sep 26 19:53:47 2018 >>> www-client/firefox-62.0.2
       merge time: 1 hour, 4 minutes and 22 seconds.

On the other hand, installing TeamViewer took no time at all:

clevow230ss /home/fitzcarraldo # # genlop -t teamviewer | tail -n 3
     Mon Oct  1 21:39:29 2018 >>> net-misc/teamviewer-13.2.13582
       merge time: 16 seconds.

And youtube-dl also installed quickly:

clevow230ss /home/fitzcarraldo # genlop -t youtube-dl | tail -n 3
     Mon Oct  1 21:42:26 2018 >>> net-misc/youtube-dl-2018.09.26
       merge time: 53 seconds.

I also used to run Gentoo Linux on a 1998 Gateway Solo 9300 laptop (Intel Mobile Pentium III Coppermine 800 MHz). Neither its CPU nor its 288 MB RAM could cope with building large source code packages such as Firefox and LibreOffice, so on that machine I installed www-client/firefox-bin and app-office/libreoffice-bin instead.

Anyway, now let’s have a quick look at a typical package upgrade session, which I aim to perform at least once a week. On the above-mentioned newer laptop today I upgraded all installed packages that had newer versions available in the so-called Portage ‘tree’. This amounted to installing (‘merging’) the following fifteen packages:

  1. net-analyzer/traceroute-2.1.0
  2. media-libs/exiftool-11.03
  3. dev-libs/libtasn1-4.13
  4. x11-misc/shared-mime-info-1.10
  5. x11-libs/libxcb-1.13.1
  6. media-libs/x265-2.8
  7. dev-libs/redland-1.0.17-r1
  8. media-libs/mesa-18.1.9
  9. dev-libs/dbus-glib-0.110
  10. sys-power/upower-0.99.8
  11. app-text/libspectre-0.2.8
  12. app-crypt/pinentry-1.1.0-r2
  13. net-im/pidgin-2.13.0
  14. app-emulation/wine-vanilla-3.17
  15. media-video/handbrake-1.1.1

It took just under thirty-four minutes to install the above fifteen packages on my newer laptop. Below is what the upgrade process looked like in a terminal window. I have used the package manager’s long command options here, but normally I would use the single-letter shortcuts. As the focus of this post is on package installation time, I will not bother to show the command I used before this to synchronise (update) package information on the laptop with the remote package servers.

clevow230ss /home/fitzcarraldo # emerge --update --verbose --deep --newuse --with-bdeps=y --keep-going @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] net-analyzer/traceroute-2.1.0::gentoo [2.0.20::gentoo] USE="-static" 70 KiB
[ebuild     U  ] media-libs/exiftool-11.03::gentoo [10.95::gentoo] USE="-doc" 4,311 KiB
[ebuild   R    ] dev-libs/libtasn1-4.13:0/6::gentoo  USE="-doc -static-libs -test% -valgrind" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild     U  ] x11-misc/shared-mime-info-1.10::gentoo [1.9::gentoo] USE="-test" 603 KiB
[ebuild     U  ] x11-libs/libxcb-1.13.1:0/1.12::gentoo [1.13:0/1.12::gentoo] USE="xkb -doc (-selinux) -static-libs -test" ABI_X86="32 (64) (-x32)" 495 KiB
[ebuild  r  U  ] media-libs/x265-2.8:0/160::gentoo [2.6:0/146::gentoo] USE="10bit 12bit -numa -pic (-power8) -test" ABI_X86="(64) -32 (-x32)" 1,348 KiB
[ebuild     U  ] dev-libs/redland-1.0.17-r1::gentoo [1.0.16::gentoo] USE="berkdb -iodbc -mysql -odbc -postgres -sqlite -static-libs (-xml%*)" 1,584 KiB
[ebuild     U  ] media-libs/mesa-18.1.9::gentoo [18.1.6::gentoo] USE="classic dri3 egl gallium gbm llvm nptl wayland -bindist -d3d9 -debug -gles1 -gles2 -opencl -openmax -osmesa -pax_kernel -pic (-selinux) -unwind -vaapi -valgrind -vdpau -vulkan -xa -xvmc" ABI_X86="32 (64) (-x32)" VIDEO_CARDS="i915 i965 intel (-freedreno) (-imx) -nouveau -r100 -r200 -r300 -r600 -radeon -radeonsi (-vc4) -virgl (-vivante) -vmware" 10,885 KiB
[ebuild     U  ] dev-libs/dbus-glib-0.110::gentoo [0.108::gentoo] USE="-debug -static-libs -test" ABI_X86="(64) -32 (-x32)" 817 KiB
[ebuild     U  ] sys-power/upower-0.99.8:0/3::gentoo [0.99.5:0/3::gentoo] USE="introspection -doc -ios (-selinux)" 439 KiB
[ebuild     U  ] app-text/libspectre-0.2.8::gentoo [0.2.7::gentoo] USE="-debug -doc -static-libs" 412 KiB
[ebuild     U  ] app-crypt/pinentry-1.1.0-r2::gentoo [1.0.0-r2::gentoo] USE="gnome-keyring gtk ncurses qt5 -caps -emacs -fltk% -static" 457 KiB
[ebuild     U  ] net-im/pidgin-2.13.0:0/2::gentoo [2.12.0:0/2::gentoo] USE="dbus gnutls gstreamer gtk ncurses networkmanager nls spell xscreensaver zeroconf (-aqua) -debug -doc -eds -gadu -groupwise -idn -meanwhile -perl -pie -prediction -python -sasl -silc -tcl -tk -zephyr" PYTHON_TARGETS="python2_7" 8,784 KiB
[ebuild  NS   ~] app-emulation/wine-vanilla-3.17:3.17::gentoo [3.16:3.16::gentoo] USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner sdl ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gssapi -gstreamer -kerberos -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) -test -vkd3d -vulkan -xinerama" ABI_X86="32 64 (-x32)" 20,955 KiB
[ebuild  rR   ~] media-video/handbrake-1.1.1::gentoo  USE="fdk gstreamer gtk x265 -libav -libav-aac" 0 KiB

Total: 15 packages (12 upgrades, 1 in new slot, 2 reinstalls), Size of downloads: 51,156 KiB

The following packages are causing rebuilds:

  (media-libs/x265-2.8:0/160::gentoo, ebuild scheduled for merge) causes rebuilds for:
    (media-video/handbrake-1.1.1:0/0::gentoo, ebuild scheduled for merge)

>>> Verifying ebuild manifests
>>> Running pre-merge checks for app-emulation/wine-vanilla-3.17
>>> Emerging (1 of 15) net-analyzer/traceroute-2.1.0::gentoo
>>> Emerging (2 of 15) media-libs/exiftool-11.03::gentoo
>>> Emerging (3 of 15) dev-libs/libtasn1-4.13::gentoo
>>> Installing (1 of 15) net-analyzer/traceroute-2.1.0::gentoo
>>> Installing (2 of 15) media-libs/exiftool-11.03::gentoo
>>> Installing (3 of 15) dev-libs/libtasn1-4.13::gentoo
>>> Emerging (4 of 15) x11-misc/shared-mime-info-1.10::gentoo
>>> Installing (4 of 15) x11-misc/shared-mime-info-1.10::gentoo
>>> Emerging (5 of 15) x11-libs/libxcb-1.13.1::gentoo
>>> Installing (5 of 15) x11-libs/libxcb-1.13.1::gentoo
>>> Emerging (6 of 15) media-libs/x265-2.8::gentoo
>>> Emerging (7 of 15) dev-libs/redland-1.0.17-r1::gentoo
>>> Emerging (8 of 15) media-libs/mesa-18.1.9::gentoo
>>> Installing (7 of 15) dev-libs/redland-1.0.17-r1::gentoo
>>> Installing (6 of 15) media-libs/x265-2.8::gentoo
>>> Installing (8 of 15) media-libs/mesa-18.1.9::gentoo
>>> Emerging (9 of 15) dev-libs/dbus-glib-0.110::gentoo
>>> Installing (9 of 15) dev-libs/dbus-glib-0.110::gentoo
>>> Emerging (10 of 15) sys-power/upower-0.99.8::gentoo
>>> Emerging (11 of 15) app-text/libspectre-0.2.8::gentoo
>>> Emerging (12 of 15) app-crypt/pinentry-1.1.0-r2::gentoo
>>> Installing (10 of 15) sys-power/upower-0.99.8::gentoo
>>> Installing (11 of 15) app-text/libspectre-0.2.8::gentoo
>>> Installing (12 of 15) app-crypt/pinentry-1.1.0-r2::gentoo
>>> Emerging (13 of 15) net-im/pidgin-2.13.0::gentoo
>>> Emerging (14 of 15) app-emulation/wine-vanilla-3.17::gentoo
>>> Emerging (15 of 15) media-video/handbrake-1.1.1::gentoo
>>> Installing (15 of 15) media-video/handbrake-1.1.1::gentoo
>>> Installing (13 of 15) net-im/pidgin-2.13.0::gentoo
>>> Installing (14 of 15) app-emulation/wine-vanilla-3.17::gentoo
>>> Jobs: 15 of 15 complete                         Load avg: 2.94, 5.84, 6.01
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * Regenerating GNU info directory index...
 * Processed 129 info files.
 * After world updates, it is important to remove obsolete packages with
 * emerge --depclean. Refer to `man emerge` for more information.

The genlop tool in Gentoo can be used to check chronologically which packages were installed and removed. For example:

clevow230ss /home/fitzcarraldo # genlop -l | grep "Mon Oct  1"
     Mon Oct  1 18:16:19 2018 >>> net-analyzer/traceroute-2.1.0
     Mon Oct  1 18:16:27 2018 >>> media-libs/exiftool-11.03
     Mon Oct  1 18:16:49 2018 >>> dev-libs/libtasn1-4.13
     Mon Oct  1 18:17:02 2018 >>> x11-misc/shared-mime-info-1.10
     Mon Oct  1 18:17:49 2018 >>> x11-libs/libxcb-1.13.1
     Mon Oct  1 18:19:11 2018 >>> dev-libs/redland-1.0.17-r1
     Mon Oct  1 18:28:07 2018 >>> media-libs/x265-2.8
     Mon Oct  1 18:28:14 2018 >>> media-libs/mesa-18.1.9
     Mon Oct  1 18:28:30 2018 >>> dev-libs/dbus-glib-0.110
     Mon Oct  1 18:28:52 2018 >>> sys-power/upower-0.99.8
     Mon Oct  1 18:28:57 2018 >>> app-text/libspectre-0.2.8
     Mon Oct  1 18:29:03 2018 >>> app-crypt/pinentry-1.1.0-r2
     Mon Oct  1 18:30:02 2018 >>> media-video/handbrake-1.1.1
     Mon Oct  1 18:30:34 2018 >>> net-im/pidgin-2.13.0
     Mon Oct  1 18:47:45 2018 >>> app-emulation/wine-vanilla-3.17
clevow230ss /home/fitzcarraldo #

When packages are being upgraded, the package manager will inform you if a package has an update to the associated application’s configuration file. Even if the package manager does not indicate that any configuration files need to be updated, it does no harm to check anyway:

clevow230ss /home/fitzcarraldo # etc-update
Scanning Configuration files...
Exiting: Nothing left to do; exiting. :)

To remove packages that are no longer required (either because the package is no longer a dependency or because a newer version of a slotted* package has been installed), the ‘depclean‘ option is used. Below is what happened when I used that option to clean up after the above-mentioned fifteen packages were upgraded/re-installed.

* Some Gentoo packages are ‘slotted’, i.e. more than one version of the package can be installed and used concurrently. Examples include WINE, Python and kernel sources.

clevow230ss /home/fitzcarraldo # emerge --ask --depclean

 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace `.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence of this, it often becomes necessary to run 
 * `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:                                                                                                                                           

    selected: 3.16 
   protected: none 
     omitted: 3.17 

    selected: 0.5.3-r1 
   protected: none 
     omitted: none 

    selected: 15.2.1 
   protected: none 
     omitted: none 

    selected: 5.4.3 
   protected: none 
     omitted: none 

    selected: 15.2.1 
   protected: none 
     omitted: none 

    selected: 16.0.0 
   protected: none 
     omitted: none 

    selected: 0.0.8 
   protected: none 
     omitted: none 

    selected: 16.3.0-r1 
   protected: none 
     omitted: none 

    selected: 4.4.3 
   protected: none 
     omitted: none 

    selected: 1 
   protected: none 
     omitted: none 

All selected packages: =dev-python/pyliblzma-0.5.3-r1 =dev-python/namespace-zope-1 =dev-python/zope-interface-4.4.3 =dev-python/pyasn1-modules-0.0.8 =app-emulation/wine-vanilla-3.16 =dev-python/psutil-5.4.3 =dev-python/twisted-web-15.2.1 =dev-python/twisted-core-15.2.1 =dev-python/service_identity-16.0.0 =dev-python/attrs-16.3.0-r1

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] Yes
>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 10) app-emulation/wine-vanilla-3.16...
>>> Unmerging (2 of 10) dev-python/pyliblzma-0.5.3-r1...
>>> Unmerging (3 of 10) dev-python/twisted-web-15.2.1...
>>> Unmerging (4 of 10) dev-python/psutil-5.4.3...
>>> Unmerging (5 of 10) dev-python/twisted-core-15.2.1...
>>> Unmerging (6 of 10) dev-python/service_identity-16.0.0...
>>> Unmerging (7 of 10) dev-python/pyasn1-modules-0.0.8...
>>> Unmerging (8 of 10) dev-python/attrs-16.3.0-r1...
>>> Unmerging (9 of 10) dev-python/zope-interface-4.4.3...
>>> Unmerging (10 of 10) dev-python/namespace-zope-1...
Packages installed:   1714
Packages in world:    198
Packages in system:   43
Required packages:    1714
Number removed:       10

 * GNU info directory index is up-to-date.

As you can see above, the package manager found a number of installed packages that were no longer required, either because a new version of a slotted package had been installed (notice wine-vanilla) or because they are no longer dependencies. As I did not want to keep both wine-3.16 and wine-3.17 installed simultaneously, I simply replied ‘Yes’. By the way, the depclean option can also be used to remove (uninstall) older versions of an individual slotted package:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean wine-vanilla

It can also be used to remove an individual package if it is not a dependency. For example, the following command does not allow me to remove the installed package vlc because other installed packages depend on it:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean vlc

Calculating dependencies... done!
>>> No packages selected for removal by depclean
>>> To see reverse dependencies, use --verbose
Packages installed:   1714
Packages in world:    198
Packages in system:   43
Required packages:    1714
Number removed:       0

Whereas the following command would allow me to remove the installed package winetricks because no other installed package depends on it:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean winetricks

Calculating dependencies... done!
>>> Calculating removal order...

>>> These are the packages that would be unmerged:                                                                                                                                           

    selected: 20180815 
   protected: none 
     omitted: none 

All selected packages: =app-emulation/winetricks-20180815

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

Would you like to unmerge these packages? [Yes/No] No


Packages installed:   1714
Packages in world:    198
Packages in system:   43
Required packages:    1713
Number to remove:     1

Anyway, I hope this post has given potential adopters a rough idea of installation times in Gentoo Linux, even if it is far from thorough.

Automatic backup of users’ files on a NAS device to an external USB HDD

One of my Linux machines is a 4-bay server that performs various roles, one of which is as NAS (network-attached storage) for family and visitors’ devices connected to my home network. I had configured each pair of HDDs in a RAID 1 array in order to provide some internal redundancy, but I was nervous about not having an external backup for users’ shares. Therefore I recently purchased a 6TB external USB 3.0 HDD (Western Digital Elements Desktop WDBWLG0060HBK-EESN) to connect permanently to one of the server’s USB 3.0 ports for backup purposes. I created a Bash script ~/backup_to_usbhdd.sh to perform the backup, plus a cron job to launch it automatically at 05:01 daily:

user $ sudo crontab -e
user $ sudo crontab -l | grep -v ^# | grep backup
01 05 * * * sudo /home/fitzcarraldo/backup_to_usbhdd.sh

The use of ‘sudo‘ in the crontab command may appear superfluous because the cron job was created for the root user (i.e. by using ‘sudo crontab -e‘ rather than ‘crontab -e‘). However, this is done to make cron use the root user’s environment rather than the minimal set of environment variables cron would otherwise use [1].

# This script backs up to an external USB HDD (NTFS) labelled "Elements" the contents
# of the directories /nas/shares/ on my server.
# It can be launched from the server either manually using sudo or as a root-user cron
# job (Use 'sudo crontab -e' to configure the job).
# Clean up if the backup did not complete last time:
umount /media/usbhdd 2>/dev/null
rm -rf /media/usbhdd/*
# Unmount the external USB HDD if mounted by udisks2 with the logged-in username in the path:
umount /media/*/Elements 2>/dev/null
# Find out the USB HDD device:
DEVICE=$( blkid | grep "Elements" | cut -d ":" -f1 )
# Create a suitable mount point if it does not already exist, and mount the device on it:
mkdir /media/usbhdd 2>/dev/null
mount -t ntfs-3g $DEVICE /media/usbhdd 2>/dev/null
sleep 10s
# Create the backup directories on the USB HDD if they do not already exist:
mkdir -p /media/usbhdd/nas 2>/dev/null
# Backup recursively the directories and add a time-stamped summary to the log file:
echo "********** Backing up nas shares directory **********" >> /home/fitzcarraldo/backup_to_usbhdd.log
date >> /home/fitzcarraldo/backup_to_usbhdd.log
# Need to use rsync rather than cp, so that can rate-limit the copying to the USB HDD:
rsync --recursive --times --perms --links --protect-args --bwlimit=22500 /nas/shares /media/usbhdd/nas/ 2>> /home/fitzcarraldo/backup_to_usbhdd.log
# No --delete option is used, so that any backed-up files deleted on the server are not deleted from the USB HDD.
echo "Copying completed" >> /home/fitzcarraldo/backup_to_usbhdd.log
date >> /home/fitzcarraldo/backup_to_usbhdd.log
df -h | grep Filesystem >> /home/fitzcarraldo/backup_to_usbhdd.log
df -h | grep usbhdd >> /home/fitzcarraldo/backup_to_usbhdd.log
echo "********** Backup completed **********" >> /home/fitzcarraldo/backup_to_usbhdd.log
cp /home/fitzcarraldo/backup_to_usbhdd.log /media/usbhdd/
# Unmount the USB HDD:
umount /media/usbhdd
exit 0

The initial version of the above script used ‘cp‘ rather than ‘rsync‘, which worked fine when I launched the script manually:

user $ sudo ./backup_to_usbhdd.sh

However, the script always failed when launched as a cron job. In this case the command ‘df -h‘ showed the root directory on the server was ‘100% used’ (full). Also, the mount point directory /media/usbhdd/ had not been unmounted. The log file had twenty or so lines similar to the following, indicating the script had failed due to the root filesystem becoming full:

cp: failed to extend ‘/media/usbhdd/nas/user1/Videos/20130822_101433.mp4’: No space left on device

Apparently data was being read from the server’s HDD into the RAM buffer/cache faster than it could be written to the external HDD. The bottleneck in this case is not USB 3.0, but the USB HDD itself. The specifications for the USB HDD do not mention drive write speed, but a quick search of the Web indicated that an external USB HDD might have a write speed of around 25 to 30 MBps (Megabytes per second). I do not know why the problem happened only when the script was launched as a cron job, but I clearly needed to throttle the rate of writing to the external HDD. Unfortunately the ‘cp‘ command does not have such an option, but the ‘rsync‘ command does:

--bwlimit=RATE          limit socket I/O bandwidth

where RATE is in KiB if no units are specified. I opted to use a rate of 22500 KiB to be safe, and it is not too far below the aforementioned 25 MBps. Indeed, using this limit the script runs to completion successfully when launched by cron:

user $ cat backup_to_usbhdd.log
********** Backing up nas shares directory **********
Thu Sep 13 05:01:26 BST 2018
Copying completed
Thu Sep 13 11:41:31 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  386G  5.1T   7% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Fri Sep 14 05:01:26 BST 2018
Copying completed
Fri Sep 14 05:20:08 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  403G  5.1T   8% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Sat Sep 15 05:01:26 BST 2018
Copying completed
Sat Sep 15 05:04:58 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  404G  5.1T   8% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Sun Sep 16 05:01:26 BST 2018
Copying completed
Sun Sep 16 05:15:14 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  416G  5.1T   8% /media/usbhdd
********** Backup completed **********
********** Backing up nas shares directory **********
Mon Sep 17 05:01:26 BST 2018
Copying completed
Mon Sep 17 05:04:15 BST 2018
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdf1       5.5T  416G  5.1T   8% /media/usbhdd
********** Backup completed **********

Notice that the first job listed in the log file took much longer than subsequent jobs. This was because rsync had to copy every file to the external USB HDD. In subsequent runs it only had to copy new files and files that had changed since they were last copied.

The disk in the external USB HDD spins down after 10 minutes of inactivity and the drive goes into Power Saver Mode. Its LED blinks to indicate the drive is in this mode. Therefore the cron job only spins up and down the external HDD once per day.

1. Why does root cron job script need ‘sudo’ to run properly?

Installing Dropbox in Gentoo Linux following the recent restrictions introduced for Dropbox for Linux

In a 2013 post I explained how I installed Dropbox in Gentoo Linux running KDE 4. The Dropbox company has recently imposed some restrictions in the Linux client, so this is to explain what I did to get Dropbox working again in my two Gentoo Linux installations, both using the ext4 filesystem (unencrypted) and, these days, KDE Plasma 5.

Both my laptops running Gentoo Linux had a version of Dropbox installed via the Portage package manager: dropbox-45.3.88 in the case of the laptop running Gentoo amd64, and dropbox-48.3.56 in the case of the laptop running Gentoo ~amd64. Recently a Dropbox window popped up, warning me to upgrade Dropbox to the latest version within seven days otherwise the client would no longer be able to sync with the remote Dropbox server. I also received an e-mail from the Dropbox company titled ‘[Action required] We’re updating Linux system requirements‘ informing me that the only supported Linux distributions from now on would be Ubuntu 14.04 or higher and Fedora 21 or higher, and furthermore that the client will only work on an unencrypted ext4 filesystem. As both my Gentoo installations use unencrypted ext4, I was OK on that score, but I still had the problem that an up-to-date Dropbox ebuild is not available for Gentoo and the old Dropbox versions I was using no longer sync. However, I managed to install the latest version of Dropbox (currently 55.4.171) in Gentoo, and it works fine. The Dropbox client’s icon is on the KDE Plasma 5 Panel, and the local Dropbox directory is being sync’ed correctly. Below I explain what I did.

1. I selected ‘Quit Dropbox’ from the old Dropbox client’s menu, and the Dropbox icon disappeared from the Panel.

2. I removed the Dropbox daemon from the list of script files to be started at login (‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’).

3. I unmerged (uninstalled) the dropbox package:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean dropbox

4. I deleted the directories ~/.dropbox and ~/.dropbox-dist but kept the directory ~/Dropbox and its contents.

fitzcarraldo@clevow230ss ~ $ rm -rf ~/.dropbox ~/.dropbox-dist

5. I followed the instructions under ‘Dropbox Headless install via command line‘ on the Dropbox Website to re-install the latest version of the daemon and client:

fitzcarraldo@clevow230ss ~ $ cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf -

6. I configured KDE Plasma 5 to start ~/.dropbox-dist/dropboxd at login (‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’ > ‘Add Script…’).

7. I launched ~/.dropbox-dist/dropboxd manually from a Konsole window. The Dropbox client icon appeared on the Panel and I was prompted to login to my Dropbox account via a Web browser, as per the instructions on the Dropbox Website (see link in in Step 5):

If you’re running Dropbox on your server for the first time, you’ll be asked to copy and paste a link in a working browser to create a new account or add your server to an existing account. Once you do, your Dropbox folder will be created in your home directory.

8. I logged in to my Dropbox account via the Firefox browser. As soon as I had logged in via the browser, a message appeared in the browser window informing me that “Your computer was successfully linked to your account”, and the Dropbox client icon appeared on the Panel and showed that the contents of ~/Dropbox were being synchronised.

Everything seems to be working as before. The Dropbox icon on the Panel has the same menu items it had previously. ‘Preferences…’ shows the Dropbox version as v55.4.171. I have not ticked ‘Start Dropbox on system startup’ under Dropbox Preferences because I configured automatic startup using KDE Plasma 5 ‘System Settings’ as described in Step 6 above, and the Dropbox daemon is indeed started automatically when I login.

The Dropbox Website’s instructions (see link in Step 5) also include the following:

Download this Python script to control Dropbox from the command line. For easy access, put a symlink to the script anywhere in your PATH.

I did download that Python script and made it executable:

fitzcarraldo@clevow230ss ~/Dropbox $ chmod +x dropbox.py

However the Python 3.6 interpreter in my Gentoo Linux installations report a syntax error in the script when I run it, I assume because it was written for a different version of Python:

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py 
  File "./dropbox.py", line 233
    except OSError, e:
SyntaxError: invalid syntax

Anyway, as the Dropbox client icon is on the KDE Plasma 5 Panel and I can control Dropbox from there, I see no need for the Python script.

9. My Gentoo installations have a Bash script ~/dbox.sh that I had created to be launched by a Desktop Configuration file ~/Desktop/Dropbox.desktop with a nice icon which I double-click on if I want to relaunch the Dropbox daemon (if I previously quit Dropbox from the client’s menu, for example). I had to modify ~/dbox.sh by replacing the command ‘dbus-launch dropbox start > /dev/null‘ with the command ‘/home/fitzcarraldo/.dropbox-dist/dropboxd‘ as shown below.


notify-send 'Launching Dropbox' 'Daemon will be (re)started in 20 seconds' --icon=dialog-information
sleep 20s
ps auxww | awk '$0~/dropbox/&&$0!~/awk/{print $2}' | xargs kill


[Desktop Entry]
Comment[en_GB]=(re)launch Dropbox daemon
Comment=(re)launch Dropbox daemon

10. At the moment Dropbox is working fine again in my Gentoo installations. However, I noticed that Gentoo Linux user zsitvaij posted the following comment in a Gentoo Forums thread:

On every dropbox update, I have to remove ~/.dropbox-dist/dropbox-lnx./libdrm.so.2 to avoid having it crash on launch, works fine after until they update again.

I do not know if that will be necessary in my case, as I have not yet had to upgrade Dropbox from the Version 55.4.171 that I recently installed. When a new version of Dropbox becomes available I will update this post to confirm whether or not I had to do anything to keep Dropbox working.

Addendum (1 October 2018): With reference to my addendum of 2 September 2018, if you are using OpenRC it is possible to automate the deletion of the file ~/.dropbox-dist/dropbox-lnx.x86_64-/libdrm.so.2 by creating a Bash script /etc/local.d/40dropbox.start containing the following:

if [ -e /home/fitzcarraldo/.dropbox-dist/dropbox-lnx.x86_64-*/libdrm.so.2 ]
    rm /home/fitzcarraldo/.dropbox-dist/dropbox-lnx.x86_64-*/libdrm.so.2

Replace my username with your username, obviously. Of course the conditional test could be dispensed with and the script could just contain the shebang line and the rm line, which would still work even if the file does not exist, but it feels a bit tidier to only attempt to delete the file if it actually exists.

Addendum (2 September 2018):I have just installed Dropbox Version 56.4.94 in my Gentoo ~amd64 installation and I had to use the command shown below once in order to stop the daemon segfaulting when I entered the command ~/.dropbox-dist/dropboxd in a Konsole window:

fitzcarraldo@clevow230ss ~/Dropbox $ rm ~/.dropbox-dist/dropbox-lnx.x86_64-56.4.94/libdrm.so.2

Addendum (31 August 2018): The Python script dropbox.py that can be downloaded from the Dropbox Web site (see Step 8 above) is old, as can be seen in the comments in the header of the script:

# Dropbox frontend script
# This file is part of nautilus-dropbox 2015.10.28.

It is written in Python 2. Although I do not need to use it, I managed to get it to run in my Gentoo installations by replacing the shebang line ‘#!/usr/bin/python‘ with ‘#!/usr/bin/env python2‘. This works in my Gentoo installations because they have both Python 2.7 and Python 3.6 installed. When I now run dropbox.py I see the following:

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py 
Dropbox command-line interface


Note: use dropbox help  to view usage for a specific command.

 status       get current status of the dropboxd
 throttle     set bandwidth limits for Dropbox
 help         provide help
 stop         stop dropboxd
 running      return whether dropbox is running
 start        start dropboxd
 filestatus   get current sync status of one or more files
 ls           list directory contents with current sync status
 autostart    automatically start dropbox at login
 exclude      ignores/excludes a directory from syncing
 lansync      enables or disables LAN sync
 sharelink    get a shared link for a file in your dropbox
 proxy        set proxy settings for Dropbox

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py status
Up to date
fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py running
fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py filestatus ~/Dropbox/Getting\ Started.pdf 
/home/fitzcarraldo/Dropbox/Getting Started.pdf: up to date
fitzcarraldo@clevow230ss ~/Dropbox $

Notice that the command ./dropbox.py running does not return anything even though the daemon is definitely running, so I do not trust the script anyway.

How to move a mouse pointer automatically in Linux to simulate user activity

My various Linux installations all have Suspend to RAM enabled with a specified timeout. Sometimes I want to override the timeout; for example if I have left something running in a terminal window or I have left the package manager in a virtual machine upgrading the guest installation. I could of course launch the system’s power manager GUI and temporarily disable Suspend to RAM or increase the timeout, but I prefer to use a shell script, launched by double-clicking on a Desktop icon, to move the mouse pointer automatically to fool the OS into believing someone is using the machine. There are various ways of doing this, but the method I prefer is given below.

1. Create a Bash script ‘/home/fitzcarraldo/keep_mouse_moving.sh‘ containing the following:

# Script to keep mouse pointer moving so that, for example, Suspend to RAM timeout does not occur.
# The mouse pointer will move around its current position on the screen, i.e. around any position
# on the screen where you place the pointer. However, if you prefer it to move around the centre
# of the screen then change mousemove_relative to mousemove in the xdotool command below.
# Set LENGTH to 0 if you do not want the mouse pointer to actually move.
# Set LENGTH to 1 if you want the mouse pointer to move just a tiny fraction.
# Set LENGTH to e.g. 100 if you want to see more easily the mouse pointer move.
# Set DELAY to the desired number of seconds between each move of the mouse pointer.
while true
    for ANGLE in 0 90 180 270
	xdotool mousemove_relative --polar $ANGLE $LENGTH
        sleep $DELAY

Do not forget to make the script executable.

As you can see in the above script, it is possible to control how much, if at all, the mouse pointer actually moves on the screen. While the script is running you are not precluded from moving the mouse manually as well.

2. Create a Desktop Configuration File ‘/home/fitzcarraldo/Desktop/keep_mouse_moving.desktop‘ containing the following:

[Desktop Entry]
Comment[en_GB]=Keep mouse moving automatically
Comment=Keep mouse moving automatically
Exec=xterm -iconic -e "bash -c /home/fitzcarraldo/keep_mouse_moving.sh"
GenericName[en_GB]=Keep mouse moving automatically
GenericName=Keep mouse moving automatically

(In installations that use KDE or GNOME I replace the ‘LXDE‘ above with ‘KDE‘ or ‘GNOME‘, as appropriate.)

3. Use the distribution’s package manager to install xterm and xdotool if they are not already installed.

4. Whenever I want to fool the OS into thinking a user is moving the mouse, I double-click on the mouse icon on the Desktop and the Bash script is launched in a minimised xterm window, which I can see on the Panel. I can then leave the installation knowing that it will not suspend to RAM and that the screensaver will not kick in. When I want to stop the mouse pointer moving automatically, I simply click on the xterm bar on the Panel to open the xterm window, and click on Close (×) on the window’s title bar to terminate xterm and the shell script.

Note that the X Windows Toolkit option -iconic may not work in some Desktop Environments (GNOME, for example), in which case you can minimise the xterm window manually if you want, or use another terminal emulator.

By the way, if you use GNOME and it is currently configured not to display icons on the Desktop, you can change this by using the following command:

user $ gsettings set org.gnome.desktop.background show-desktop-icons true

Of course you are not obliged to have the .desktop file on the Desktop; it could be in any directory.

Configuring Lubuntu 18.04 to enable hibernation using a swap file

In an earlier post about Lubuntu 18.04 I stated that hibernation is precluded because the Lubuntu Installer installs the OS with a swap file instead of a swap partition. In fact, even with a swap file it is possible to configure Lubuntu so that hibernation is possible. This is how I did it.

1.  This PC has 4 GiB RAM but the Lubuntu Installer had created a 2 GiB swap file named /swapfile, so I increased the size of the swap file to 4 GiB to ensure it was large enough to store the memory image:

user $ sudo swapoff -a
user $ sudo dd if=/dev/zero of=/swapfile bs=1024 count=4M
user $ sudo chmod 600 /swapfile
user $ sudo mkswap /swapfile
user $ sudo swapoff -a
user $ sudo swapon /swapfile
user $ cat /proc/meminfo | grep -i memtotal
MemTotal:        3924108 kB
user $ ls -la /swapfile
-rw------- 1 root root 4294967296 Jul 10 18:25 /swapfile

Note that you can check the status of the swap file before and after the above steps by using either of the following commands:

user $ swapon -s
user $ free -m

2.  The Lubuntu Installer had previously configured /etc/fstab for the swap file, so I left that as it is:

user $ grep swapfile /etc/fstab
/swapfile                                 none            swap    sw              0       0

3.  I checked on which device the root partition with the file /swapfile is located (sda2 in my case) and found out its UUID (ignore the PARTUUID):

user $ sudo blkid
[sudo] password for fitzcarraldo:
/dev/sda1: UUID="3602-BD57" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="72b3693e-b81f-7299-84fb-bf3781bef43d"
/dev/sda2: UUID="afe17116-26fa-4169-b2d9-fb6ac8afc63c" TYPE="ext4" PARTUUID="738fed17-293d-832f-c7a4-e83471fe8ca6"

4.  I found the resume_offset for the file /swapfile, which is 16746496 in my case (look for the first value in the two columns under ‘physical_offset‘):

user $ sudo filefrag -v /swapfile
Filesystem type is: ef53
File size of /swapfile is 4294967296 (1048576 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..   30719:   16746496..  16777215:  30720:            
   1:    30720..   63487:   16809984..  16842751:  32768:   16777216:
   2:    63488..   96255:   16842752..  16875519:  32768:            
   3:    96256..  126975:   16875520..  16906239:  30720:            
   4:   126976..  129023:   16908288..  16910335:   2048:   16906240:
   5:   129024..  161791:   16912384..  16945151:  32768:   16910336:
   6:   161792..  194559:   16945152..  16977919:  32768:            
   7:   194560..  227327:   16977920..  17010687:  32768:            
   8:   227328..  249855:   17010688..  17033215:  22528:            
   9:   249856..  282623:   17035264..  17068031:  32768:   17033216:
  10:   282624..  315391:   17068032..  17100799:  32768:            
  11:   315392..  319487:   17100800..  17104895:   4096:            
  12:   319488..  321535:   17121280..  17123327:   2048:   17104896:
  13:   321536..  325631:   17129472..  17133567:   4096:   17123328:
  14:   325632..  327679:   17137664..  17139711:   2048:   17133568:
  15:   327680..  329727:   17143808..  17145855:   2048:   17139712:
  16:   329728..  331775:   17154048..  17156095:   2048:   17145856:
  17:   331776..  339967:   17162240..  17170431:   8192:   17156096:
  18:   339968..  344063:   24485888..  24489983:   4096:   17170432:
  19:   344064..  346111:   32665600..  32667647:   2048:   24489984:
  20:   346112..  348159:   32677888..  32679935:   2048:   32667648:
  21:   348160..  350207:   33261568..  33263615:   2048:   32679936:
  22:   350208..  352255:   33363968..  33366015:   2048:   33263616:
  23:   352256..  354303:   33853440..  33855487:   2048:   33366016:
  24:   354304..  356351:   34000896..  34002943:   2048:   33855488:
  25:   356352..  389119:   34027520..  34060287:  32768:   34002944:
  26:   389120..  391167:   34060288..  34062335:   2048:            
  27:   391168..  393215:   34134016..  34136063:   2048:   34062336:
  28:   393216..  395263:   34158592..  34160639:   2048:   34136064:
  29:   395264..  428031:   34189312..  34222079:  32768:   34160640:
  30:   428032..  452607:   34222080..  34246655:  24576:            
  31:   452608..  485375:   34248704..  34281471:  32768:   34246656:
  32:   485376..  518143:   34281472..  34314239:  32768:            
  33:   518144..  550911:   34314240..  34347007:  32768:            
  34:   550912..  583679:   34347008..  34379775:  32768:            
  35:   583680..  616447:   34379776..  34412543:  32768:            
  36:   616448..  643071:   34412544..  34439167:  26624:            
  37:   643072..  645119:   34445312..  34447359:   2048:   34439168:
  38:   645120..  649215:   34457600..  34461695:   4096:   34447360:
  39:   649216..  659455:   34463744..  34473983:  10240:   34461696:
  40:   659456..  688127:   34476032..  34504703:  28672:   34473984:
  41:   688128..  690175:   34506752..  34508799:   2048:   34504704:
  42:   690176..  692223:   34510848..  34512895:   2048:   34508800:
  43:   692224..  724991:   34514944..  34547711:  32768:   34512896:
  44:   724992..  757759:   34549760..  34582527:  32768:   34547712:
  45:   757760..  778239:   34582528..  34603007:  20480:            
  46:   778240..  786431:   34637824..  34646015:   8192:   34603008:
  47:   786432..  819199:   34648064..  34680831:  32768:   34646016:
  48:   819200..  843775:   34680832..  34705407:  24576:            
  49:   843776..  845823:   34707456..  34709503:   2048:   34705408:
  50:   845824..  849919:   34713600..  34717695:   4096:   34709504:
  51:   849920..  854015:   34729984..  34734079:   4096:   34717696:
  52:   854016..  886783:   34744320..  34777087:  32768:   34734080:
  53:   886784..  919551:   34777088..  34809855:  32768:            
  54:   919552..  950271:   34809856..  34840575:  30720:            
  55:   950272..  983039:   34842624..  34875391:  32768:   34840576:
  56:   983040.. 1015807:   34875392..  34908159:  32768:            
  57:  1015808.. 1048575:   34908160..  34940927:  32768:             last,eof
/swapfile: 38 extents found

Note that you can also find the resume_offset by installing the package uswsusp and using the command swap-offset on the swap file:

user $ sudo apt install uswsusp
user $ sudo swap-offset /swapfile
resume offset = 16746496

5.  I updated the file /boot/grub/grub.cfg using the information found in Steps 3 & 4 as follows:

5.1  I added ‘resume=UUID=afe17116-26fa-4169-b2d9-fb6ac8afc63c resume_offset=16746496 resumedelay=15‘ to the parameters in the variable GRUB_CMDLINE_LINUX_DEFAULT in the file /etc/default/grub (Your existing parameters could be different to mine; that is not a problem):

user $ sudo nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_backlight=vendor acpi_osi='!Windows 2013' acpi_osi='!Windows 2012' resume=UUID=afe17116-26fa-4169-b2d9-fb6ac8afc63c resume_offset=16746496 resumedelay=15"

Note that the optional ‘resumedelay=15‘ specifies the delay (in seconds) to pause before attempting to read the resume files. I added this to try to allow enough time for the filesystem containing the swap file to become Read-Write.

5.2  I regenerated /boot/grub/grub.cfg by using the following command:

user $ sudo update-grub

6.  I edited the file /etc/initramfs-tools/conf.d/resume using the information found in Steps 3 & 4, and regenerated the initramfs files for the kernel images in the /boot directory:

user $ sudo nano /etc/initramfs-tools/conf.d/resume
RESUME=UUID=afe17116-26fa-4169-b2d9-fb6ac8afc63c resume_offset=16746496
# Resume from /swapfile
user $ sudo update-initramfs -u -k all

7.  I edited the Polkit rules files to permit hibernation (Create the files if they do not already exist):

7.1  For Polkit version 0.106 and higher

user $ sudo nano /etc/polkit-1/rules.d/85-suspend.rules
polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.login1.suspend" ||
        action.id == "org.freedesktop.login1.suspend-multiple-sessions" ||
        action.id == "org.freedesktop.login1.hibernate" ||
        action.id == "org.freedesktop.login1.hibernate-multiple-sessions")
        return polkit.Result.YES;

7.2  For Polkit versions below 0.106

user $ sudo nano /var/lib/polkit-1/localauthority/50-local.d/50-enable-suspend-on-lockscreen.pkla
[Allow hibernation and suspending with lock screen]

It does not do any harm to create both the above-mentioned rules files, whatever the version of Polkit that happens to be installed.

8.  I rebooted, logged in, launched a few GUI applications and then clicked on the Lubuntu menu icon on the Panel and selected ‘Logout’ > ‘Hibernate’, which did put the machine into hibernation. I then pressed the PC’s power push-button to resume from disk, entered my password on the lock screen and the Desktop appeared exactly as it was prior to hibernation. All good.

Getting the lock screen to work reliably when resuming from suspension in a single-seat, multi-user Lubuntu 18.04 installation

In an earlier post I described my attempt at getting the lock screen to work reliably in the single-seat, multi-user Lubuntu 17.10 installation on my family’s desktop PC. Although the modifications described in that post seemed to improve matters somewhat, users were still not always able to login from the LightDM greeter screen after resuming from Suspend to RAM in the following situation:

  1. User_A logs in to User_A’s account but does not log out after using the account.
  2. User_B clicks on ‘Logout’ > ‘Switch User’ to log in to User_B’s account but does not log out.
  3. User_A clicks on ‘Logout’ > ‘Switch User’ to get back to User_A’s account.
  4. User_A allows his/her session to timeout and suspend to RAM.
  5. User_B presses a key on the keyboard to resume from suspension, and the LightDM lock screen is displayed.
  6. User_B enters his/her password and then clicks on ‘Unlock’, but the LightDM lock screen remains on display and nobody can log in any more, although the keys on the lock screen are still clickable.

When this occurs, the only way users can access their Desktop is to click on the Power icon in the top right corner of the lock screen and select ‘Restart…’.

The Software Updater in Lubuntu 17.10 recently offered me the choice of upgrading to Lubuntu 18.04, which I accepted. The upgrade was performed and the only hitch that resulted was an incorrect initramfs, which was simple enough to fix (see my post Lubuntu 18.04 ‘Gave up waiting for suspend/resume device’). However, the above-mentioned problem of unlocking after resuming from suspension still occurred in Lubuntu 18.04. Below are the changes I made since the modifications described in my post Getting the lock screen to work properly when resuming from Suspend-to-RAM with multiple sessions in Lubuntu 17.10 (the other changes in that post remain), which seem to have cured the problem.

Change to Item 2 in my earlier post

I reverted the Exec line in /etc/xdg/autostart/light-locker.desktop back to how it was originally following installation of Lubuntu:

user $ grep Exec /etc/xdg/autostart/light-locker.desktop

Change to Item 3 in my earlier post

I deleted the file /lib/systemd/system-sleep/hang-fix that I had previously created:

user $ sudo rm /lib/systemd/system-sleep/hang-fix

Change to Item 7 in my earlier post

The Xfce Power Manager ‘Security’ tab for each user now has ‘Lock screen when system is going for sleep’ ticked:

Light Locker

  • Automatically lock the session: Never
  • Delay locking after screensaver for: ‘1 Seconds’ is greyed out
  • ‘Lock screen when system is going for sleep’ is ticked

The full Xfce Power Manager settings for each user (see the file ~/.config/xfce4/xfconf/xfce-perchannel-xml/xfce4-power-manager.xml in each user’s home directory) are now configured as follows:

<?xml version="1.0" encoding="UTF-8"?>

<channel name="xfce4-power-manager" version="1.0">
  <property name="xfce4-power-manager" type="empty">
    <property name="power-button-action" type="empty"/>
    <property name="show-tray-icon" type="empty"/>
    <property name="brightness-switch-restore-on-exit" type="int" value="1"/>
    <property name="brightness-switch" type="int" value="0"/>
    <property name="presentation-mode" type="bool" value="false"/>
    <property name="inactivity-on-ac" type="uint" value="30"/>
    <property name="blank-on-ac" type="int" value="10"/>
    <property name="dpms-on-ac-sleep" type="uint" value="0"/>
    <property name="dpms-on-ac-off" type="uint" value="0"/>
    <property name="brightness-on-ac" type="uint" value="9"/>
    <property name="lock-screen-suspend-hibernate" type="bool" value="true"/>
    <property name="logind-handle-lid-switch" type="bool" value="false"/>
    <property name="dpms-enabled" type="bool" value="false"/>
    <property name="general-notification" type="bool" value="true"/>

Additional modifications

In another of my posts (Prevent Lubuntu 17.10 from leaving an external HDD mounted incorrectly for other users) I explained the modifications I made in Lubuntu 17.10 for a single-seat, multi-user installation to work properly with a permanently connected external USB HDD. However, I recently noticed the following problems resulting from those modifications:

A. The following error message in the LightDM log file /var/log/lightdm/lightdm.log:

[SeatDefaults] is now called [Seat:*], please update this configuration

So I changed the contents of the file /etc/lightdm/lightdm.conf.d/10_lubuntu.conf from:




B. The following error message in the LightDM log file/var/log/lightdm/lightdm.log when the USB external HDD happened to not be mounted at the time:

DEBUG: Launching process 8569: /etc/lightdm/lightdm.conf.d/unmount_FREECOM_HDD.sh
DEBUG: Process 8569 terminated with signal 11

So I changed the contents of my Bash script /etc/lightdm/lightdm.conf.d/unmount_FREECOM_HDD.sh from:

udisksctl unmount --block-device /dev/disk/by-uuid/C6576A087368B015


STATUS=`mount | grep $(readlink -f /dev/disk/by-uuid/C6576A087368B015 )`
if [[ ! -z $STATUS ]]; then
    udisksctl unmount --block-device /dev/disk/by-uuid/C6576A087368B015
exit 0


It is early days, but so far the login problem for other users after resuming from suspension has not reoccurred since I made the latest changes. I am not sure if the modifications described in my post ‘Prevent Lubuntu 17.10 from leaving an external HDD mounted incorrectly for other users‘ contributed to (or caused) the login problem, or whether one or more of systemd-logind, LightDM, LightDM GTK+ Greeter, Light Locker and Xfce Power Manager are to blame (since they have to work holistically to provide the required functionality). It is frustrating not knowing the root cause of the problem, but at least my family no longer has to worry about being able to log in if a family member has not logged out and left the PC to suspend.