My system upgrade procedure for Gentoo Linux

Gentoo Linux is a so-called ‘rolling-release’ distribution, and each Gentoo Linux user has their own preferred sequence of steps for keeping their installation up-to-date. Below is the general procedure I use for system maintenance of my Gentoo installations, which I perform approximately weekly.

1. Update the ebuilds on the machine (see Gentoo Wiki – Project:Portage/Sync)

root # emaint sync -a

If I were using the deprecated Portage sync method I would instead have used the following commands:

root # emerge --sync # Update the ebuilds from the main Portage tree
root # layman -S # Update the ebuilds from 3rd-party overlays

2. Upgrade the Portage package manager if the console output from Step 1 included a message telling me to upgrade portage

root # emerge -1v portage

3. As I use the eix and the mlocate utilities, update their data files

root # eix-update && updatedb

4. Check if there are any News items I have not read yet

root # eselect news list

5. Read new News items and make necessary changes, if any

root # eselect news read <n>

6. Perform a dry run for the upgrade of any packages in the World file that have new versions

root # emerge -uvpDN --with-bdeps=y @world

7. If no problems were flagged in Step 6, go to Step 9

8. Sort out any problem(s) flagged in Step 6 then go back to Step 6

9. Launch the upgrade of those packages in the World file that have new versions

root # emerge -uvDN --with-bdeps=y --keep-going @world

My decision on whether or not to include the option ‘--keep-going‘ will depend on the precise circumstances.

10. If Step 9 ran to completion successfully, go to Step 14

11. If Step 9 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go to Step 12, otherwise fix the problem(s)* and go back to Step 9

*Sometimes I find that one or more packages do not merge successfully during Step 9 but do merge successfully simply by repeating Step 9.

12. Resume the upgrade process

root # emerge --resume --skipfirst

13. If Step 12 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go back to Step 12, otherwise fix the problem(s) and go back to Step 9

14. Upgrade any packages that are still built against old versions of libraries if the console output from Step 9 or Step 12 includes a message telling me to do that

root # emerge @preserved-rebuild

15. If any problems remain, fix them and go back to Step 14

16. Scan libraries and binaries for missing shared library dependencies and re-merge any broken binaries and shared libraries

root # revdep-rebuild -i

Actually, I cannot recall the last time ‘revdep-rebuild‘ was needed, as Portage has improved so much over the years.

17. Remove outdated and unneeded packages

root # emerge --ask --depclean

18. Merge any configuration files

root # etc-update

I always check the differences between the listed existing and new configuration files before going ahead, and may edit the new configuration file if I deem it necessary.

19. As I use the mlocate utility I make sure its index file is bang up to date

root # updatedb

20. Optionally, I clear out any old source-code and binary packages

root # eclean-dist --deep

21. If I remember to do it, I check if there are any installed obsolete packages and then remove them

root # eix-test-obsolete

22. I make sure no temporary work files have been left around by any failed merges

root # rm -rf /usr/tmp/portage/*

Actually, I created a script in directory /etc/local.d/ to do this automatically when HDD free space gets low (see my blog post ‘Automatically clearing the /usr/tmp/portage directory in Gentoo Linux‘).

23. I wait for at least 24 hours (usually about a week) and then go to Step 1

Notes

Actually, I have added the option ‘--with-bdeps=y‘ to EMERGE_DEFAULT_OPS in the file /etc/portage/make.conf so I do not need to type that option every time.

One of my laptops has an older Core i7 CPU and I initially added ‘--jobs=8 --load-average=8‘ to EMERGE_DEFAULT_OPTS in order to merge packages in parallel, which speeds up upgrading. However, I found this slowed interactive use of the machine and therefore I changed the options to ‘--jobs=6 --load-average=6‘ which works a bit better on that machine.

In order to prevent the number of log files increasing indefinitely, I added ‘clean-logs‘ to FEATURES in the file /etc/portage/make.conf so that log files older than seven days are deleted automatically.

Automatically clearing the /usr/tmp/portage directory in Gentoo Linux

Gentoo Linux has been in use for nine years on one of my old laptops. A couple of days ago I performed the usual rolling update of the installation, but the latest version of a large package that normally takes several hours to compile failed to compile due to a lack of disk space. Sure enough, the command ‘df -h‘ showed me that the root partition was full. After a little digging I discovered that the directory /usr/tmp/portage/ contained a whopping 30GB of directories and files.

Portage uses the directory /usr/tmp/portage/ as a temporary store for the package source code when merging a package. The temporary files are not deleted if a merge fails, but the emerge command should delete them on the next merge of that package. On the other hand the ebuild command does not delete the temporary files, although normally you only use the ebuild command if you are creating a manifest.

Anyway, in the nine years that Gentoo Linux has been installed on the laptop I had never bothered to check that /usr/tmp/portage/ was actually empty, and its contents had slowly increased. The cure to my immediate problem was simply to empty the directory:

root # rm -rf /usr/tmp/portage/*

I doubt the laptop would still be working by the time /usr/tmp/portage would become that full again, but the situation got me thinking: What if I were to create a script to delete the temporary directories and files in /usr/tmp/portage/ at shutdown?

Gentoo Linux on this laptop uses OpenRC so I simply created a file /etc/local.d/99delete_tmp_files_from_failed_merges.stop containing the following code:

#!/bin/bash
# If root partition is more than 90% full, delete any temporary directories and
# files that were left in /var/tmp/portage/ instead of being deleted.
#
# The root partition is on /dev/sda6 and the emerge command must not be running.
#
if [ `pgrep -c emerge` -eq 0 ] && [ `df | awk '/sda6/ {print $5}' | awk -F% '{print $1}'` -gt 90 ]; then
    rm -rf /usr/tmp/portage/*
fi

I made the script executable:

root # chmod +x /etc/local.d/99delete_tmp_files_from_failed_merges.stop

Now, if the root partition is more than 90% full when I shut down the laptop, the script will automatically empty that directory. One less thing to think about.

Elogviewer: A handy GUI for viewing Portage elog messages in Gentoo Linux

When merging (installing) packages in Gentoo, ebuilds often output console messages with information or warnings from the writer of the ebuild, usually at the end of the installation process. However, these ‘elog’ messages will not be displayed if you have configured the environment variable EMERGE_DEFAULT_OPTS so as to merge packages quietly or in parallel. Even if you did not configure EMERGE_DEFAULT_OPTS that way, it is easy to miss these messages as they scroll up and off screen if several packages are merging, one after the other.

The Gentoo package manager Portage has a logging facility that, if enabled, will log these elog messages to files so that you can review them afterwards. You can enable this facility by editing the file /etc/portage/make.conf and adding the environment variable PORTAGE_ELOG_SYSTEM="save" and the environment variable PORTAGE_ELOG_CLASSES with one or more logging classes. Here are the relevant lines from the file /etc/portage/make.conf on my laptop:

PORT_LOGDIR="/var/log/portage"
PORTAGE_ELOG_SYSTEM="save"
PORTAGE_ELOG_CLASSES="info warn error log qa"

For example, after merging the package www-misc/bluegriffon-bin-1.8 at 06:01:00 on 14 April 2016, I am able to examine the contents of the log file for that specific job:

$ cat /var/log/portage/elog/www-misc:bluegriffon-bin-1.8:20160414-060100.log
INFO: setup
Package: www-misc/bluegriffon-bin-1.8
Repository: local_overlay
USE: abi_x86_64 amd64 elibc_glibc kernel_linux userland_GNU
FEATURES: preserve-libs sandbox userpriv usersandbox
LOG: install
If you use BlueGriffon in KDE, use System Settings > Common Appearance and Behaviour > Application Appearance > GTK
and select any GTK theme other than Oyxgen, otherwise BlueGriffon will crash when you click on any pull-down menu.
QA: other
QA Notice: Pre-stripped files found:
/opt/bluegriffon/libreplace_jemalloc.so
/opt/bluegriffon/libnssdbm3.so
/opt/bluegriffon/libnss3.so
/opt/bluegriffon/bluegriffon-bin
/opt/bluegriffon/libnssutil3.so
/opt/bluegriffon/gmp-clearkey/0.1/libclearkey.so
/opt/bluegriffon/libsmime3.so
/opt/bluegriffon/libplc4.so
/opt/bluegriffon/libnssckbi.so
/opt/bluegriffon/plugin-container
/opt/bluegriffon/libsoftokn3.so
/opt/bluegriffon/libssl3.so
/opt/bluegriffon/libnspr4.so
/opt/bluegriffon/libxul.so
/opt/bluegriffon/libfreebl3.so
/opt/bluegriffon/components/libmozgnome.so
/opt/bluegriffon/components/libdbusservice.so
/opt/bluegriffon/libplds4.so
/opt/bluegriffon/libmozsqlite3.so
/opt/bluegriffon/bluegriffon

Of particular interest is the elog message:

If you use BlueGriffon in KDE, use System Settings > Common Appearance and Behaviour > Application Appearance > GTK
and select any GTK theme other than Oyxgen, otherwise BlueGriffon will crash when you click on any pull-down menu.

Clearly, some of the elog messages are important and must not be missed. After reading such messages, users can take appropriate action.

To facilitate reading Portage elog files, there is a GUI utility called Elogviewer which is easy to install and use:

# emerge elogviewer

$ elogviewer --help
usage: elogviewer [-h] [-p ELOGPATH] [--log {DEBUG,INFO,WARNING,ERROR}]

Elogviewer should help you not to miss important information. You need to
enable the elog feature by setting at least one of PORTAGE_ELOG_CLASSES="info
warn error log qa" and PORTAGE_ELOG_SYSTEM="save" in /etc/make.conf. You need
to add yourself to the portage group to use elogviewer without privileges.
Read /etc/make.conf.example for more information.

optional arguments:
  -h, --help            show this help message and exit
  -p ELOGPATH, --elogpath ELOGPATH
                        path to the elog directory
  --log {DEBUG,INFO,WARNING,ERROR}
                        set logging level

I happen to have configured EMERGE_DEFAULT_OPTS="--jobs=8 --load-average=8" (i.e. perform installation jobs in parallel using all eight cores of the CPU) in my /etc/portage/make.conf file, so I don’t see elog messages on screen, and therefore Elogviewer comes in handy. The output in the Konsole window looks like the following when I merge a package:

# emerge -v bluegriffon-bin

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ~] www-misc/bluegriffon-bin-1.8::local_overlay  0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) www-misc/bluegriffon-bin-1.8::local_overlay
>>> Installing (1 of 1) www-misc/bluegriffon-bin-1.8::local_overlay
>>> Jobs: 1 of 1 complete                           Load avg: 0.11, 0.07, 0.13
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.

Notice that the important elog message regarding switching the GTK theme in KDE that is included in the log file was not displayed on the console during installation of the package, because of my setting for EMERGE_DEFAULT_OPTS.

If I then launch Elogviewer, either from the command line or using the KDE launcher, a window pops up as shown below. I can then view a list of recently merged packages and click on each to read the elog output easily. Whether installing only one package or many packages in one session, this makes life easier.

Elogviewer window

Sabayon Linux developers split the Portage sabayon overlay into two new overlays

If you are a Gentoo Linux user who added the sabayon overlay, or if you are a Sabayon Linux user who already uses Portage, note that the developers of Sabayon Linux have just split the overlay into two overlays. One of the overlays (sabayon-distro) contains ebuilds that are specific to the Sabayon Linux distribution and unlikely to be of interest to users of other distributions that use the Portage package manager. The other overlay (sabayon) contains ebuilds that could be of interest to Portage users of other distributions. For example, the package app-misc/sabayon-version will only be of relevance to users of Sabayon Linux, so you’ll only find it in the sabayon-distro overlay, not the sabayon overlay:

# eix sabayon-version
* app-misc/sabayon-version [1]
Available versions: (~)5-r5 (~)7-r1
Homepage: http://www.sabayon.org/
Description: Sabayon System Release virtual package

[1] "sabayon-distro" /var/lib/layman/sabayon-distro

All you need to do in order to cater for this change is the following as root user:

layman -d sabayon
layman -S
layman -a sabayon
layman -a sabayon-distro
eix-update

You only need to add the sabayon-distro overlay if you are a user of Sabayon Linux or want to install any of the distribution-specific ebuilds from it. Of course omit the eix-update command if you do not have the excellent eix utility installed.

From then onwards you can just continue as normal using the usual Portage commands as root user in order to synchronise the main Portage tree and synchronise all the overlays added on your machine:

emerge --sync
layman -S
eix-update

or, if you have an asterisk on its own line in the file /etc/eix-sync.conf then you can replace the above three commands with the following single command:

eix-sync