A brief discussion about package installation times in Gentoo Linux
October 1, 2018 Leave a comment
I thought that perhaps users of binary-based Linux distributions who are contemplating trying out the source-based distribution Gentoo Linux might be interested to know a bit about package installation times in contrast to binary distributions. I am not going to go into great detail here; this is just to give interested people a quick idea of possible package installation times in Gentoo Linux.
The package manager of a binary-based distribution such as Ubuntu downloads and installs ‘binary’ packages, i.e. packages containing pre-built executables. On the other hand, Gentoo’s package manager Portage downloads source-code packages and builds the binaries (executables) on your machine. Nevertheless, a small number of Portage packages contain binaries rather than source code, either because the source code could take many hours to build on older hardware or because the source code is simply not available in the public domain. An example of the first scenario is Firefox, which is available in Gentoo both as the source code package www-client/firefox and as the binary package www-client/firefox-bin so that the user can choose which to install (‘merge’, in Gentoo parlance). An example of the second scenario is TeamViewer, which is only available as the binary package net-misc/teamviewer because TeamViewer is closed-source software (i.e. the company that develops TeamViewer does not release its source code).
Clearly, installing an application from a package containing source code that has to be compiled and linked on your machine will take longer than installing the application from a package containing the executable binary that someone else has already built. The time to install from source-code packages of course depends on the precise package (some can take only a minute or so to install), the power of the machine on which the package is being installed, and various other factors.
I currently have two laptops running Gentoo Linux: one is over eight years old with an early Intel Core i7-720QM (throttled to 933 MHz because of the small PSU); the other is three and a half years old and has a newer Intel Core i7-4810MQ (2.8 GHz). Although the older laptop is slow by today’s standards, the newer laptop is still reasonably powerful. Even so, those few large source-code packages such as Firefox, Chromium and LibreOffice can take quite a long time to install, as illustrated below for Firefox by using the Gentoo utility genlop to find out the time it took to install:
clevow230ss /home/fitzcarraldo # genlop -t firefox | tail -n 3
Wed Sep 26 19:53:47 2018 >>> www-client/firefox-62.0.2
merge time: 1 hour, 4 minutes and 22 seconds.
On the other hand, installing TeamViewer took no time at all:
clevow230ss /home/fitzcarraldo # # genlop -t teamviewer | tail -n 3
Mon Oct 1 21:39:29 2018 >>> net-misc/teamviewer-13.2.13582
merge time: 16 seconds.
And youtube-dl also installed quickly:
clevow230ss /home/fitzcarraldo # genlop -t youtube-dl | tail -n 3
Mon Oct 1 21:42:26 2018 >>> net-misc/youtube-dl-2018.09.26
merge time: 53 seconds.
I also used to run Gentoo Linux on a 1998 Gateway Solo 9300 laptop (Intel Mobile Pentium III Coppermine 800 MHz). Neither its CPU nor its 288 MB RAM could cope with building large source code packages such as Firefox and LibreOffice, so on that machine I installed www-client/firefox-bin and app-office/libreoffice-bin instead.
Anyway, now let’s have a quick look at a typical package upgrade session, which I aim to perform at least once a week. On the above-mentioned newer laptop today I upgraded all installed packages that had newer versions available in the so-called Portage ‘tree’. This amounted to installing (‘merging’) the following fifteen packages:
- net-analyzer/traceroute-2.1.0
- media-libs/exiftool-11.03
- dev-libs/libtasn1-4.13
- x11-misc/shared-mime-info-1.10
- x11-libs/libxcb-1.13.1
- media-libs/x265-2.8
- dev-libs/redland-1.0.17-r1
- media-libs/mesa-18.1.9
- dev-libs/dbus-glib-0.110
- sys-power/upower-0.99.8
- app-text/libspectre-0.2.8
- app-crypt/pinentry-1.1.0-r2
- net-im/pidgin-2.13.0
- app-emulation/wine-vanilla-3.17
- media-video/handbrake-1.1.1
It took just under thirty-four minutes to install the above fifteen packages on my newer laptop. Below is what the upgrade process looked like in a terminal window. I have used the package manager’s long command options here, but normally I would use the single-letter shortcuts. As the focus of this post is on package installation time, I will not bother to show the command I used before this to synchronise (update) package information on the laptop with the remote package servers.
clevow230ss /home/fitzcarraldo # emerge --update --verbose --deep --newuse --with-bdeps=y --keep-going @world
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild U ] net-analyzer/traceroute-2.1.0::gentoo [2.0.20::gentoo] USE="-static" 70 KiB
[ebuild U ] media-libs/exiftool-11.03::gentoo [10.95::gentoo] USE="-doc" 4,311 KiB
[ebuild R ] dev-libs/libtasn1-4.13:0/6::gentoo USE="-doc -static-libs -test% -valgrind" ABI_X86="32 (64) (-x32)" 0 KiB
[ebuild U ] x11-misc/shared-mime-info-1.10::gentoo [1.9::gentoo] USE="-test" 603 KiB
[ebuild U ] x11-libs/libxcb-1.13.1:0/1.12::gentoo [1.13:0/1.12::gentoo] USE="xkb -doc (-selinux) -static-libs -test" ABI_X86="32 (64) (-x32)" 495 KiB
[ebuild r U ] media-libs/x265-2.8:0/160::gentoo [2.6:0/146::gentoo] USE="10bit 12bit -numa -pic (-power8) -test" ABI_X86="(64) -32 (-x32)" 1,348 KiB
[ebuild U ] dev-libs/redland-1.0.17-r1::gentoo [1.0.16::gentoo] USE="berkdb -iodbc -mysql -odbc -postgres -sqlite -static-libs (-xml%*)" 1,584 KiB
[ebuild U ] media-libs/mesa-18.1.9::gentoo [18.1.6::gentoo] USE="classic dri3 egl gallium gbm llvm nptl wayland -bindist -d3d9 -debug -gles1 -gles2 -opencl -openmax -osmesa -pax_kernel -pic (-selinux) -unwind -vaapi -valgrind -vdpau -vulkan -xa -xvmc" ABI_X86="32 (64) (-x32)" VIDEO_CARDS="i915 i965 intel (-freedreno) (-imx) -nouveau -r100 -r200 -r300 -r600 -radeon -radeonsi (-vc4) -virgl (-vivante) -vmware" 10,885 KiB
[ebuild U ] dev-libs/dbus-glib-0.110::gentoo [0.108::gentoo] USE="-debug -static-libs -test" ABI_X86="(64) -32 (-x32)" 817 KiB
[ebuild U ] sys-power/upower-0.99.8:0/3::gentoo [0.99.5:0/3::gentoo] USE="introspection -doc -ios (-selinux)" 439 KiB
[ebuild U ] app-text/libspectre-0.2.8::gentoo [0.2.7::gentoo] USE="-debug -doc -static-libs" 412 KiB
[ebuild U ] app-crypt/pinentry-1.1.0-r2::gentoo [1.0.0-r2::gentoo] USE="gnome-keyring gtk ncurses qt5 -caps -emacs -fltk% -static" 457 KiB
[ebuild U ] net-im/pidgin-2.13.0:0/2::gentoo [2.12.0:0/2::gentoo] USE="dbus gnutls gstreamer gtk ncurses networkmanager nls spell xscreensaver zeroconf (-aqua) -debug -doc -eds -gadu -groupwise -idn -meanwhile -perl -pie -prediction -python -sasl -silc -tcl -tk -zephyr" PYTHON_TARGETS="python2_7" 8,784 KiB
[ebuild NS ~] app-emulation/wine-vanilla-3.17:3.17::gentoo [3.16:3.16::gentoo] USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner sdl ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gssapi -gstreamer -kerberos -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) -test -vkd3d -vulkan -xinerama" ABI_X86="32 64 (-x32)" 20,955 KiB
[ebuild rR ~] media-video/handbrake-1.1.1::gentoo USE="fdk gstreamer gtk x265 -libav -libav-aac" 0 KiB
Total: 15 packages (12 upgrades, 1 in new slot, 2 reinstalls), Size of downloads: 51,156 KiB
The following packages are causing rebuilds:
(media-libs/x265-2.8:0/160::gentoo, ebuild scheduled for merge) causes rebuilds for:
(media-video/handbrake-1.1.1:0/0::gentoo, ebuild scheduled for merge)
>>> Verifying ebuild manifests
>>> Running pre-merge checks for app-emulation/wine-vanilla-3.17
>>> Emerging (1 of 15) net-analyzer/traceroute-2.1.0::gentoo
>>> Emerging (2 of 15) media-libs/exiftool-11.03::gentoo
>>> Emerging (3 of 15) dev-libs/libtasn1-4.13::gentoo
>>> Installing (1 of 15) net-analyzer/traceroute-2.1.0::gentoo
>>> Installing (2 of 15) media-libs/exiftool-11.03::gentoo
>>> Installing (3 of 15) dev-libs/libtasn1-4.13::gentoo
>>> Emerging (4 of 15) x11-misc/shared-mime-info-1.10::gentoo
>>> Installing (4 of 15) x11-misc/shared-mime-info-1.10::gentoo
>>> Emerging (5 of 15) x11-libs/libxcb-1.13.1::gentoo
>>> Installing (5 of 15) x11-libs/libxcb-1.13.1::gentoo
>>> Emerging (6 of 15) media-libs/x265-2.8::gentoo
>>> Emerging (7 of 15) dev-libs/redland-1.0.17-r1::gentoo
>>> Emerging (8 of 15) media-libs/mesa-18.1.9::gentoo
>>> Installing (7 of 15) dev-libs/redland-1.0.17-r1::gentoo
>>> Installing (6 of 15) media-libs/x265-2.8::gentoo
>>> Installing (8 of 15) media-libs/mesa-18.1.9::gentoo
>>> Emerging (9 of 15) dev-libs/dbus-glib-0.110::gentoo
>>> Installing (9 of 15) dev-libs/dbus-glib-0.110::gentoo
>>> Emerging (10 of 15) sys-power/upower-0.99.8::gentoo
>>> Emerging (11 of 15) app-text/libspectre-0.2.8::gentoo
>>> Emerging (12 of 15) app-crypt/pinentry-1.1.0-r2::gentoo
>>> Installing (10 of 15) sys-power/upower-0.99.8::gentoo
>>> Installing (11 of 15) app-text/libspectre-0.2.8::gentoo
>>> Installing (12 of 15) app-crypt/pinentry-1.1.0-r2::gentoo
>>> Emerging (13 of 15) net-im/pidgin-2.13.0::gentoo
>>> Emerging (14 of 15) app-emulation/wine-vanilla-3.17::gentoo
>>> Emerging (15 of 15) media-video/handbrake-1.1.1::gentoo
>>> Installing (15 of 15) media-video/handbrake-1.1.1::gentoo
>>> Installing (13 of 15) net-im/pidgin-2.13.0::gentoo
>>> Installing (14 of 15) app-emulation/wine-vanilla-3.17::gentoo
>>> Jobs: 15 of 15 complete Load avg: 2.94, 5.84, 6.01
>>> Auto-cleaning packages...
>>> No outdated packages were found on your system.
* Regenerating GNU info directory index...
* Processed 129 info files.
* After world updates, it is important to remove obsolete packages with
* emerge --depclean. Refer to `man emerge` for more information.
The genlop tool in Gentoo can be used to check chronologically which packages were installed and removed. For example:
clevow230ss /home/fitzcarraldo # genlop -l | grep "Mon Oct 1"
Mon Oct 1 18:16:19 2018 >>> net-analyzer/traceroute-2.1.0
Mon Oct 1 18:16:27 2018 >>> media-libs/exiftool-11.03
Mon Oct 1 18:16:49 2018 >>> dev-libs/libtasn1-4.13
Mon Oct 1 18:17:02 2018 >>> x11-misc/shared-mime-info-1.10
Mon Oct 1 18:17:49 2018 >>> x11-libs/libxcb-1.13.1
Mon Oct 1 18:19:11 2018 >>> dev-libs/redland-1.0.17-r1
Mon Oct 1 18:28:07 2018 >>> media-libs/x265-2.8
Mon Oct 1 18:28:14 2018 >>> media-libs/mesa-18.1.9
Mon Oct 1 18:28:30 2018 >>> dev-libs/dbus-glib-0.110
Mon Oct 1 18:28:52 2018 >>> sys-power/upower-0.99.8
Mon Oct 1 18:28:57 2018 >>> app-text/libspectre-0.2.8
Mon Oct 1 18:29:03 2018 >>> app-crypt/pinentry-1.1.0-r2
Mon Oct 1 18:30:02 2018 >>> media-video/handbrake-1.1.1
Mon Oct 1 18:30:34 2018 >>> net-im/pidgin-2.13.0
Mon Oct 1 18:47:45 2018 >>> app-emulation/wine-vanilla-3.17
clevow230ss /home/fitzcarraldo #
When packages are being upgraded, the package manager will inform you if a package has an update to the associated application’s configuration file. Even if the package manager does not indicate that any configuration files need to be updated, it does no harm to check anyway:
clevow230ss /home/fitzcarraldo # etc-update
Scanning Configuration files...
Exiting: Nothing left to do; exiting. :)
To remove packages that are no longer required (either because the package is no longer a dependency or because a newer version of a slotted* package has been installed), the ‘depclean‘ option is used. Below is what happened when I used that option to clean up after the above-mentioned fifteen packages were upgraded/re-installed.
* Some Gentoo packages are ‘slotted’, i.e. more than one version of the package can be installed and used concurrently. Examples include WINE, Python and kernel sources.
clevow230ss /home/fitzcarraldo # emerge --ask --depclean
* Always study the list of packages to be cleaned for any obvious
* mistakes. Packages that are part of the world set will always
* be kept. They can be manually added to this set with
* `emerge --noreplace `. Packages that are listed in
* package.provided (see portage(5)) will be removed by
* depclean, even if they are part of the world set.
*
* As a safety measure, depclean will not remove any packages
* unless *all* required dependencies have been resolved. As a
* consequence of this, it often becomes necessary to run
* `emerge --update --newuse --deep @world` prior to depclean.
Calculating dependencies... done!
>>> Calculating removal order...
>>> These are the packages that would be unmerged:
app-emulation/wine-vanilla
selected: 3.16
protected: none
omitted: 3.17
dev-python/pyliblzma
selected: 0.5.3-r1
protected: none
omitted: none
dev-python/twisted-web
selected: 15.2.1
protected: none
omitted: none
dev-python/psutil
selected: 5.4.3
protected: none
omitted: none
dev-python/twisted-core
selected: 15.2.1
protected: none
omitted: none
dev-python/service_identity
selected: 16.0.0
protected: none
omitted: none
dev-python/pyasn1-modules
selected: 0.0.8
protected: none
omitted: none
dev-python/attrs
selected: 16.3.0-r1
protected: none
omitted: none
dev-python/zope-interface
selected: 4.4.3
protected: none
omitted: none
dev-python/namespace-zope
selected: 1
protected: none
omitted: none
All selected packages: =dev-python/pyliblzma-0.5.3-r1 =dev-python/namespace-zope-1 =dev-python/zope-interface-4.4.3 =dev-python/pyasn1-modules-0.0.8 =app-emulation/wine-vanilla-3.16 =dev-python/psutil-5.4.3 =dev-python/twisted-web-15.2.1 =dev-python/twisted-core-15.2.1 =dev-python/service_identity-16.0.0 =dev-python/attrs-16.3.0-r1
>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.
Would you like to unmerge these packages? [Yes/No] Yes
>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 10) app-emulation/wine-vanilla-3.16...
>>> Unmerging (2 of 10) dev-python/pyliblzma-0.5.3-r1...
>>> Unmerging (3 of 10) dev-python/twisted-web-15.2.1...
>>> Unmerging (4 of 10) dev-python/psutil-5.4.3...
>>> Unmerging (5 of 10) dev-python/twisted-core-15.2.1...
>>> Unmerging (6 of 10) dev-python/service_identity-16.0.0...
>>> Unmerging (7 of 10) dev-python/pyasn1-modules-0.0.8...
>>> Unmerging (8 of 10) dev-python/attrs-16.3.0-r1...
>>> Unmerging (9 of 10) dev-python/zope-interface-4.4.3...
>>> Unmerging (10 of 10) dev-python/namespace-zope-1...
Packages installed: 1714
Packages in world: 198
Packages in system: 43
Required packages: 1714
Number removed: 10
* GNU info directory index is up-to-date.
As you can see above, the package manager found a number of installed packages that were no longer required, either because a new version of a slotted package had been installed (notice wine-vanilla) or because they are no longer dependencies. As I did not want to keep both wine-3.16 and wine-3.17 installed simultaneously, I simply replied ‘Yes’. By the way, the depclean option can also be used to remove (uninstall) older versions of an individual slotted package:
clevow230ss /home/fitzcarraldo # emerge --ask --depclean wine-vanilla
It can also be used to remove an individual package if it is not a dependency. For example, the following command does not allow me to remove the installed package vlc because other installed packages depend on it:
clevow230ss /home/fitzcarraldo # emerge --ask --depclean vlc
Calculating dependencies... done!
>>> No packages selected for removal by depclean
>>> To see reverse dependencies, use --verbose
Packages installed: 1714
Packages in world: 198
Packages in system: 43
Required packages: 1714
Number removed: 0
Whereas the following command would allow me to remove the installed package winetricks because no other installed package depends on it:
clevow230ss /home/fitzcarraldo # emerge --ask --depclean winetricks
Calculating dependencies... done!
>>> Calculating removal order...
>>> These are the packages that would be unmerged:
app-emulation/winetricks
selected: 20180815
protected: none
omitted: none
All selected packages: =app-emulation/winetricks-20180815
>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.
Would you like to unmerge these packages? [Yes/No] No
Quitting.
Packages installed: 1714
Packages in world: 198
Packages in system: 43
Required packages: 1713
Number to remove: 1
Anyway, I hope this post has given potential adopters a rough idea of installation times in Gentoo Linux, even if it is far from thorough.
