My thoughts on systemd

When I first read about systemd a few years ago, the main ‘marketing point’ seemed to be that Linux boots faster using it rather than SysVinit — see, for example, On to better booting. When I read that article back in September 2012, a paragraph caught my eye:

It looks like the crowd lining up behind systemd is growing fast. With Fedora, OpenSUSE, Mandiva, Arch, Debian, Gentoo, Frugalware — Did I forget anyone? And RedHat will probably be joining with the release of RedHat7. Ubuntu seems to be sticking with Upstart.

As a Gentoo user I was surprised to read that the distribution was ‘lining up behind systemd’ — systemd was not even mentioned in the Gentoo Handbook — so I started a thread in the Gentoo Forums enquiring whether there were plans to change the default init system from OpenRC to systemd: Gentoo ‘lining up behind systemd’?.

Revisiting that ITWorld article two years later, I notice the paragraph was subsequently changed, as it now reads:

It looks like the crowd lining up behind systemd is growing fast. With Fedora, OpenSUSE, Mandiva, Arch, Debian, Frugalware — Did I forget anyone? And RedHat will probably be joining with the release of RedHat7. Gentoo has its own next generation rc system called OpenRC, which competes with both systemd and upstart, but provides its users with the option of installing systemd. Ubuntu seems to be sticking with Upstart.

OpenRC works fine for me; I know how to use it and I do not particularly want to have to learn how to use another system when what I use now works perfectly well for my purposes. By no means do I begrudge others the use of systemd, as long as my freedom of choice is not curtailed. Each to his own, as the saying goes. But events over the past couple of years have left me wondering about my freedom to choose in the future, and indeed about the future form of the operating system itself.

Faster startup?

As I mentioned above, the main marketing point when I first read about systemd was its boot time. When I tried systemd in May 2013 on a 2009 Acer Aspire 5536-643G25Mn laptop (dual-core CPU; 3 GB RAM) — no match for today’s state-of-the-art hardware — the times to boot from the GRUB 2 menu to the DM login screen for a fully-updated 64-bit Sabayon Linux Xfce installation were as follows:

OpenRC with rc_parallel=”NO” 33 seconds
OpenRC with rc_parallel=”YES” 31 seconds
systemd 29 seconds

I posted the above timings in the Sabayon Linux Forums and the developer of the distribution responded:

speed improvements heavily depend on the CPU and I/O subsystem. In particular, systemd is able to leverage multi-core CPUs much better (because it spawns services in parallel). Spawning services in parallel also puts more pressure on the I/O subsystem, which means that if you have faster I/O (like with SSDs), the speed boost compared to OpenRC becomes quite relevant.

Indeed a few users of newer machines with 8 GB or more RAM and/or SSD drives wrote in the Sabayon Linux Forums that their boot times had almost been halved (e.g. from circa 40 seconds to circa 21 seconds). However, I expect the boot times for OpenRC and systemd would not be dramatically different from each other on any of my existing machines and, in any case, I’m not bothered about saving a few seconds in boot time on my machines (and I don’t reboot often either).

Nowadays one hears less mention of faster boot times from systemd proponents. For example, the author of the December 2011 article Here We Go Again, Another Linux Init: Intro to systemd mentioned ‘Faster Startups’ early on, whereas in a September 2014 article Understanding and Using Systemd, the same author states:

For whatever reason, it seems that the proponents of SysVInit replacements are obsessed with boot times. My systemd systems, like CentOS 7, don’t boot up all that much faster than the others. It’s not something I particularly care about in any case, since most boot speed measurements only measure reaching the login prompt, and not how long it takes for the system to completely start and be usable.

Erosion of choice?

A couple of Gentoo developers commented in the above-mentioned 2012 Gentoo Forums thread that adoption of systemd as the default init system in Gentoo would be extremely unlikely. Several posters pointed out that alternative init systems exist and Gentoo users could simply install an alternative if desired. However, I started to wonder about the long-term availability of choice when I began to see blog posts such as Red Hat Flag and, indeed, when I myself was affected by systemd’s subsumption of udev (see e.g. udev is doomed) earlier this year and, more recently, when a Gentoo developer had to create the package upower-pm-utils because systemd subsumed hibernate & suspend support from upower (UPower 0.99.0 was released to ~arch as of 2014-06-01).

Whatever one’s views on systemd, the impact on non-systemd users when systemd developers change something that previously worked fine is frustrating. Gentoo developers have been doing a good job working around such nugatory subsumptions, but it’s extra work for them and extra effort for the Gentoo user who doesn’t use systemd. I began to wonder how long this state of affairs could continue.

Another layer of abstraction?

Although my main laptop has Gentoo installed with OpenRC, on a couple of other machines I tinker with Sabayon Linux, which uses systemd. Sabayon Linux is pre-compiled Gentoo with an overlay and its own binary package manager, so there is some synergy in using the two distributions. Also, it is clear from several threads requesting help in the Gentoo Forums that a number of Gentoo users are migrating to systemd. In answering or just reading forum posts involving systemd, and in writing blog posts of my own explaining how to use systemd for certain tasks, in some cases it feels to me as if systemd sometimes adds a layer of abstraction that is not necessarily helpful. Two examples that spring to mind are given below.

  1. The Gentoo Forums thread [SOLVED]I don’t understand transient hostnames in which a Gentoo user using systemd was trying to stop the hostname being changed when he connected to some networks:

    I’m running a systemd based system for the first time and I’m having an issue with my hostname. Basically, when I connect to (some) networks the network assigns me a transient hostname. This transient hostname seems to completely break everything with my desktop manager (KDM). I have to log out, and then log in with the new transient hostname (which is presumably assigned shortly after I log in and network manager connects to the network). Is this expected? I have set my hostname as described in the handbook and the systemd wiki page. How do I make my hostname *never* change based on dhcp crap?

    After I pointed out that it is possible to force a persistent hostname by specifying the hostname in NetworkManager.conf if NetworkManager is being used with either systemd or OpenRC, he wrote:

    My hostname is now written in 3 places (/etc/hosts, /etc/hostname, and /etc/NetworkManager/NetworkManager.conf). Oh, and I have some random executable (hostnamectl) sitting on my system doing a seemingly poor job of ‘controlling’ my hostname.

    Out of curiosity I then experimented with hostnamectl and compared the outcome with the outcome of setting the hostname in an installation that does not use systemd, and the results differed, so I’m not sure of the scope of hostnamectl.

  2. Time (Arch Wiki)

    The hardware clock can be queried and set with the timedatectl command. To change the hardware clock time standard to localtime, use:

    # timedatectl set-local-rtc 1

    If you want to revert to the hardware clock being in UTC, do:

    # timedatectl set-local-rtc 0

    These will generate /etc/adjtime automatically and update the RTC accordingly; no further configuration is required.

    During kernel startup, at the point when the RTC driver is loaded, the system clock may be set from the hardware clock. Whether this occurs depends on the hardware platform, the version of the kernel and kernel build options. If this does occur, at this point in the boot sequence, the hardware clock time is assumed to be UTC and the value of /sys/class/rtc/rtcN/hctosys (N=0,1,2,..) will be set to 1.

    Later, the system clock is set again from the hardware clock by systemd, dependent on values in /etc/adjtime. Hence, having the hardware clock using localtime may cause some unexpected behavior during the boot sequence; e.g system time going backwards, which is always a bad idea (there is a lot more to it). To avoid it systemd (>215) will only synchronize back, if the hardware clock is set to UTC and keep the kernel uninformed about the local timezone. As a consequence timestamps on a FAT filesystem touched by the Linux system will be in UTC.

    Note: The use of timedatectl requires an active dbus. Therefore, it may not be possible to use this command under a chroot (such as during installation). In these cases, you can revert back to the hwclock command.

Swimming against the tide?

There have been a number of dissenting voices on the Internet. Some argue against systemd on technical grounds, others on philosophical grounds. Here are are few examples:

One of the few distributions bucking the trend is Void Linux, which has replaced systemd with runit as the default init system, with systemd becoming an option.

Slackware and Gentoo are two distributions that do not use systemd by default, although it is actually reasonably straightforward to switch to systemd in Gentoo. It seems one or two people are moving to Gentoo, or contemplating it, because systemd isn’t the default in Gentoo (see, for examples, Re: OT: Open letter to the Linux World and IgnorantGuru’s Hiatus).

Several threads in the Gentoo Forums contain prolonged discussions on systemd, and it is clear that some Gentoo users have strong reservations about systemd. As I write this, a poll in the Gentoo Forums shows only 13 per cent of respondents support the idea of Gentoo adopting systemd as the default init system. I suspect that users of binary-based distributions find this lack of enthusiasm odd, but I believe it is because Gentoo is a source-based distribution and its users are usually more familiar with Linux internals and more comfortable, more adept and more interested in getting ‘under the hood’ than are users of binary-based distributions. Gentoo is not quite Linux From Scratch, but Gentoo users build their installations almost from the ground up (Gentoo does not even have an installer application, for example). I believe that anything interfering with the current way of doing things, or adding a level of abstraction, is not of interest to many Gentoo users. On the other hand, I think users of binary-based distributions who just want to get something done in the quickest way possible are less likely to object.

The beginnings of a new OS?

systemd is not just an init system, it has grown into much more than that and continues to add functionality (referred to by some as ‘feature creep’). Below is the current list of functionality provided in systemd, and the list is growing.

init system, journal logging, login management, device management, temporary and volatile file management, binary format registration, backlight save/restore, rfkill save/restore, bootchart, read-ahead, encrypted storage setup, EFI/GPT partition discovery, virtual machine/container registration, minimal container management, hostname management, locale management, time management, random seed management, sysctl variable management, console management, network management, time synchronisation

I think more of the seasoned GNU/Linux users are waking up to the impact of what Lennart Poettering (the chief architect of systemd) and his fellow systemd ‘cabal’ (his word) members are doing. They are re-designing GNU/Linux from the ground up, and I don’t think it would be an exaggeration to call the end result ‘systemd/Linux’, a de facto new OS. Personally I’m not interested in using that, but I think the vast majority of existing users or newcomers to ‘Linux’ — to use the popular term for the OS — simply don’t care. As long as it works out-of-the-box, that’s all that matters to most people.

And as far as distribution developers are concerned, the status quo says it all: most have already switched their distributions to systemd or have committed to doing so in the future. I suspect it is because they see systemd as being easier to maintain, thus making their lives easier, and they also see systemd as being easier for their desktop users (and with a boot time closer to Windows 8/8.1 Fast Startup). Ultimately the systemd ‘cabal’ will effectively make the different package managers obsolete too, as the latest plan Revisiting How We Put Together Linux Systems shows.

In my opinion, the end result of the radical systemd redesign of the OS could be greater take-up of the OS on the desktop, but the distinction between distributions will lessen and many will cease to exist, as there will be little point choosing one over another. The greater commonality and the larger user base of systemd/Linux will attract more crackers than at present. In some respects the de facto new OS will be much more like Windows or Mac OS than the current GNU/Linux.

Widening the net?

In a Chromium OS discussion group thread earlier this year — The future of initsystem in Chromium OS — a systemd proponent (a Fedora developer?) offered to help the Chrome OS developers switch from Upstart to systemd, but the Chrome OS developers saw no reason to do so without proof of tangible benefits. I believe there is too much revenue at stake to risk switching an important component of the OS of a commercial product (Chromebooks) on a whim. With Chromebooks looking increasingly like the main threat to Microsoft’s hegemony on the desktop, I’m sure Google has no interest in jeopardising that without a very good reason.

The future?

The future seemed clear to me once Mark Shuttleworth announced Ubuntu would drop Upstart and adopt systemd (he made the announcement after Debian developers decided to adopt systemd). Given the number of Ubuntu users (I’ve seen figures in the tens of millions bandied about), I believe Ubuntu will be the making or breaking of systemd. In my opinion, as was the case with PulseAudio (another Lennart Poettering project) there will be a couple of choppy years with plenty of posts in the Ubuntu Forums seeking help, more bugs will be shaken out, and I believe the vast majority of Linux users will end up as accepting users of systemd. In some cases systemd does make configuration a little easier for a newcomer (see, for example, my post Configuring the Linux clock). Personally I find systemd no easier to use than what I use now; in some cases I find systemd more complicated than what I use now (e.g. OpenRC’s local.d). My concerns regarding abstraction, subsumption, loss of choice and the morphing of the OS into something closer to the Windows approach remain, although I can see that many people, including newcomers to Linux, would actually welcome those things. Ironically, although I used to get frustrated with the large number of Linux distributions and their differences, the implications of greater commonality now worry me from a security (predominantly cybercrime) point of view. I am satisfied and comfortable with the existing GNU/Linux tools, configuration files, modus operandi and OpenRC (which works with no problems whatsoever in my case), so I have no desire to migrate to systemd on my main machine running Gentoo Linux. I will also continue to tinker with Sabayon Linux, and thus with systemd, but it will be several years hence before this jury returns a verdict on systemd.

Configuring the Linux clock

A problem often reported by users in Linux forums is that the time displayed by the system clock is incorrect. Typically the problem is that it displays UTC instead of local time. The most common cause is incorrect configuration, although dual booting with Windows will also change the Linux system clock’s time unless either a) Linux is configured to use local time for the hardware clock, or b) Windows is configured to use UTC* and Windows’ time synchronisation with an Internet time server is disabled.

* A change to the Windows Registry is required in order to enable Windows to use UTC for hardware clock time, but even then you may still face problems (see e.g. Force Windows 8 to use UTC when dealing with BIOS clock).

Some terms you need to know: ‘hardware clock’, ‘RTC’ (real-time clock), ‘CMOS clock’ and ‘BIOS clock’ are terms commonly used for the same thing. The hardware clock is non-volatile. ‘system clock’ is the software clock the operating system uses. It is volatile (it is in the kernel). The system clock is normally set from the hardware clock when the operating system boots, taking into account various configuration options. I recommend that you read the manual for the hwclock command, as it explains in detail about clocks in Linux:

# man hwclock

To configure your time zone and clocks correctly, follow the steps below.

If your installation uses the OpenRC init system

  1. Set the time zone:

    1. Specify your time zone in the file /etc/timezone. If you are in Spain, for example, the contents of that file should be ‘Europe/Madrid‘ (without the quotes).

      # nano /etc/timezone

      You can find out which time zones are available by looking in the directory /usr/share/zoneinfo/ and its sub-directories.

    2. Provide the information for your time zone. If you are in Spain, for example, you would enter the following command:

      # cp /usr/share/zoneinfo/Europe/Madrid /etc/localtime

      Note: Some people recommend making /etc/localtime a symlink rather than creating it by copying a file from /usr/share/zoneinfo/, however I recommend copying. You should definitely not use a symlink if /usr is on a different partition (see Ref. 1).

  2. Set the system clock’s time to your local time now (I’ll assume it is 23:54:30 on 21 September 2014 when you press Enter):

    # date 092123542014.30

  3. Specify whether the hardware clock time is UTC:

    # hwclock --utc

    or local time:

    # hwclock --localtime

    It is preferable for the hardware clock time to be in UTC. However, if you dual boot with Windows it is recommended for the hardware clock time to be in local time rather than UTC unless both the following are true: a) the Windows Registry has been configured for hardware clock time in UTC; b) Windows’ time synchronisation with an Internet time server is disabled.

  4. Set the hardware clock’s time by specifying your local time (I’ll assume it is 23:54:50 on 21 September 2014 when you press Enter):

    # rm /etc/adjtime
    # hwclock --set --date="2014-09-21 23:54:50"

    Note: You must specify local time in the above command, even if you intend the hardware clock time to be UTC.

    Note that the third line in /etc/adjtime should be ‘UTC’ if the hardware clock time is in UTC, or ‘LOCAL’ if the hardware clock time is in local time. For example:

    # cat /etc/adjtime
    0.000000 1412038530 0.000000
    1412038530
    UTC

  5. Tell OpenRC whether the hardware clock uses UTC or local time, and to update the hardware clock time from the system clock time at every shutdown (taking into account any time zone difference between the two clocks):

    # nano /etc/conf.d/hwclock

    1. Make clock="UTC" (or clock="local" if you dual boot with Windows and the Windows Registry is not configured to use UTC for hardware clock time).

    2. Make clock_systohc="YES" to adjust the hardware clock time at shutdown based on the software clock time (taking time zone into account). However, you don’t need to bother doing this if your kernel configuration has CONFIG_RTC_SYSTOHC=y and your installation is configured to synchronise the system clock with a NTP time server (see my post Synchronise your Gentoo Linux clock with an Internet time server). You can check if the hardware clock time is being updated from the software clock time (taking time zone into account) by looking at the kernel messages displayed when in verbose mode during shutdown to see if the following message is displayed:

      Setting hardware clock using the system clock [UTC] ...

    3. Make clock_hctosys="YES" to adjust the system clock time at start up based on the hardware clock time (taking time zone into account). However, you don’t need to bother doing this if your kernel configuration has CONFIG_RTC_HCTOSYS=y. You can check by looking at the kernel messages displayed when in verbose mode during boot to see if the following message is displayed:

      Setting system clock using the hardware clock [UTC] ...

  6. Reboot and check everything is working as expected:

    # date
    Tue 30 Sep 01:55:55 BST 2014
    # hwclock
    Tue 30 Sep 2014 01:55:59 BST -0.031877 seconds

    The hwclock command always shows local time, even if you keep your hardware clock in UTC.

  7. Optionally (but I recommend it), configure your installation to synchronise the system clock with an Internet NTP server (see my post Synchronise your Gentoo Linux clock with an Internet time server).

If your installation uses systemd

  1. Set the time zone:

    1. Find out the available time zones:

      # timedatectl list-timezones

    2. Specify your time zone (I’ll assume you are in Spain in this example):

      # timedatectl set-timezone Europe/Madrid

  2. Set the system clock time to your local time (I’ll assume it is 23:54:30 on 21 September 2014 when you press Enter):

    # timedatectl set-time "2014-09-21 23:54:30"

  3. Inform systemd that the hardware clock uses UTC, and set the current UTC in the hardware clock now:

    # timedatectl set-local-rtc 0

    Note that, by default, systemd assumes the hardware clock uses UTC. Although systemd can be configured to assume the hardware clock uses local time, for technical reasons it is not recommended (see man timedatectl and Arch Wiki article Time for an explanation of why this is not advisable).

    Note that the third line in /etc/adjtime should be ‘UTC’ if the hardware clock time is in UTC, or ‘LOCAL’ if the hardware clock time is in local time. For example:

    # cat /etc/adjtime
    0.000000 1412038530 0.000000
    1412038530
    UTC

  4. Reboot and check everything is working as expected:

    $ timedatectl

  5. Optionally (but I recommend it), configure your installation to synchronise the system clock with an Internet NTP server (see my post Synchronising the clock using NTP in Sabayon Linux).

References

[1] Gentoo Forums – how exactly to set timezone in /etc/localtime ?[solved]

Synchronising the clock using NTP in Sabayon Linux

In my previous post I explained how to install Sabayon Linux via the command line from the Sabayon Linux ‘SpinBase’ ISO.

If the system clock in your current installation is not being synchronised with an external time server over the Internet, the simplest way to achieve this is to install the net-misc/ntp package and configure systemd as shown below.

Note that I use net-misc/ntp rather than net-misc/chrony because the latter does not work in my Sabayon Linux installation; after I have configured the Chrony daemon, the command ‘systemctl status chronyd‘ returns the error message ‘Can’t initialise from real time clock, driver not loaded’. Of course, if Chrony happens to be installed and is working fine, you don’t need to do anything at all. But if Chrony is not working correctly and you want to try the ntp daemon instead, before performing the steps below you’ll first need to disable Chrony as explained under ‘Caveat’ at the end of this post.

  1. Install the ntp package:

    # equo install ntp

  2. Enable the ntp daemon so that it starts at boot:

    # systemctl enable ntpd
    # timedatectl set-ntp true

  3. Start the ntp daemon running now:

    # systemctl start ntpd

  4. Check whether the daemon is running:

    # systemctl status ntpd

Below I show the console output for the complete sequence from Step 2 to Step 4 with the command ‘timedatectl status‘ between each step so that you can see the effect. Note that I had previously set the hardware clock time (which, by default, systemd assumes to be UTC) to be the same time as local time. Since BST and UTC do not coincide, clearly both clocks should not both contain 10:06, so watch what happens below once the NTP daemon is launched.

sabayon fitzcarraldo # timedatectl status
      Local time: Mon 2014-09-22 10:06:06 BST
  Universal time: Mon 2014-09-22 09:06:06 UTC
        RTC time: Mon 2014-09-22 10:06:07
        Timezone: Europe/London (BST, +0100)
     NTP enabled: no
NTP synchronized: no
 RTC in local TZ: no
      DST active: yes
 Last DST change: DST began at
                  Sun 2014-03-30 00:59:59 GMT
                  Sun 2014-03-30 02:00:00 BST
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2014-10-26 01:59:59 BST
                  Sun 2014-10-26 01:00:00 GMT
sabayon fitzcarraldo # systemctl enable ntpd
ln -s '/usr/lib/systemd/system/ntpd.service' '/etc/systemd/system/multi-user.target.wants/ntpd.service'
sabayon fitzcarraldo # timedatectl status
      Local time: Mon 2014-09-22 10:06:39 BST
  Universal time: Mon 2014-09-22 09:06:39 UTC
        RTC time: Mon 2014-09-22 10:06:39
        Timezone: Europe/London (BST, +0100)
     NTP enabled: no
NTP synchronized: no
 RTC in local TZ: no
      DST active: yes
 Last DST change: DST began at
                  Sun 2014-03-30 00:59:59 GMT
                  Sun 2014-03-30 02:00:00 BST
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2014-10-26 01:59:59 BST
                  Sun 2014-10-26 01:00:00 GMT
sabayon fitzcarraldo # timedatectl set-ntp true
sabayon fitzcarraldo # timedatectl status
      Local time: Mon 2014-09-22 10:06:57 BST
  Universal time: Mon 2014-09-22 09:06:57 UTC
        RTC time: Mon 2014-09-22 10:06:58
        Timezone: Europe/London (BST, +0100)
     NTP enabled: yes
NTP synchronized: no
 RTC in local TZ: no
      DST active: yes
 Last DST change: DST began at
                  Sun 2014-03-30 00:59:59 GMT
                  Sun 2014-03-30 02:00:00 BST
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2014-10-26 01:59:59 BST
                  Sun 2014-10-26 01:00:00 GMT
sabayon fitzcarraldo # systemctl start ntpd
sabayon fitzcarraldo # timedatectl status
      Local time: Mon 2014-09-22 10:07:13 BST
  Universal time: Mon 2014-09-22 09:07:13 UTC
        RTC time: Mon 2014-09-22 09:07:13
        Timezone: Europe/London (BST, +0100)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: yes
 Last DST change: DST began at
                  Sun 2014-03-30 00:59:59 GMT
                  Sun 2014-03-30 02:00:00 BST
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2014-10-26 01:59:59 BST
                  Sun 2014-10-26 01:00:00 GMT
sabayon fitzcarraldo # systemctl status ntpd
ntpd.service - Network Time Service
   Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled)
   Active: active (running) since Mon 2014-09-22 10:07:11 BST; 23s ago
 Main PID: 2420 (ntpd)
   CGroup: /system.slice/ntpd.service
           └─2420 /usr/sbin/ntpd -g -n

Sep 22 10:07:11 sabayon ntpd[2420]: Listen normally on 5 eth0 fe80::a00:27ff:fe86:21f3 UDP 123
Sep 22 10:07:11 sabayon ntpd[2420]: peers refreshed
Sep 22 10:07:11 sabayon ntpd[2420]: Listening on routing socket on fd #22 for interface updates
Sep 22 10:07:13 sabayon ntpd[2420]: Deferring DNS for 0.gentoo.pool.ntp.org 1
Sep 22 10:07:14 sabayon ntpd[2420]: Deferring DNS for 1.gentoo.pool.ntp.org 1
Sep 22 10:07:15 sabayon ntpd[2420]: Deferring DNS for 2.gentoo.pool.ntp.org 1
Sep 22 10:07:16 sabayon ntpd[2420]: Deferring DNS for 3.gentoo.pool.ntp.org 1
Sep 22 10:07:18 sabayon ntpd_intres[2422]: DNS 0.gentoo.pool.ntp.org -> 85.236.36.4
Sep 22 10:07:18 sabayon ntpd_intres[2422]: DNS 1.gentoo.pool.ntp.org -> 78.46.197.35
Sep 22 10:07:18 sabayon ntpd_intres[2422]: DNS 2.gentoo.pool.ntp.org -> 91.234.160.19
sabayon fitzcarraldo #

Notice that the NTP daemon synchronised the RTC (hardware clock) time to UTC (I had previously set the hardware clock time to be the same as local time to show what would happen when the NTP deamon was launched). That is correct: notice ‘RTC in local TZ: no‘ in the output above, meaning that systemd by default assumes the time in the RTC is UTC.

By the way, in case you’re wondering, the NTP daemon of course synchronises the time in the system clock too. You do not need to worry: the system clock uses the local time of the time zone you previously configured (see my previous post), so the Linux date command and the clock of the Desktop Environment will show you local time by default.

Caveat
Note that I am using the ntp daemon instead of the Chrony daemon or the OpenNTPD daemon. If Chrony happens to be installed and active in your Sabayon Linux installation, before performing any of the above steps you’ll first need to do the following:

  1. Uninstall the Chrony package:

    # equo remove chrony

  2. Disable the Chrony daemon so that it does not start at boot:

    # systemctl disable chronyd
    # timedatectl set-ntp false

  3. Stop the Chrony daemon running now:

    # systemctl stop chronyd

  4. Check whether the daemon is still running:

    # systemctl status chronyd

Change systemd’s binary logging to text logging in Sabayon Linux

systemd logs system messages in a binary file which the developers call a ‘journal’. However, if you would prefer system messages to be logged in a text file, here is how to do it for Sabayon Linux. The steps will probably be similar in other distributions.

1. See what log files currently exist in your installation:

# ls /var/log
anaconda btmp cups faillog kdm.log mysql portage sandbox tallylog Xorg.0.log
boot.log chrony entropy journal lastlog news samba spice-vdagentd wtmp Xorg.0.log.old

Notice that the traditional text log file /var/log/messages is not present. The systemd binary log files are in a sub-directory of /var/log/journal/:

# ls /var/log/journal
5d387e19d28644c5b24f06f8bc6e9b9c
# ls /var/log/journal/5d387e19d28644c5b24f06f8bc6e9b9c
system.journal user-1000.journal

2. Edit the file /etc/systemd/journald.conf and change #Storage=auto to Storage=none

# nano /etc/systemd/journald.conf

3. Install the system logger syslog-ng:

# equo install syslog-ng

If it’s not already installed, install the logrotate package:

# equo install logrotate

4. Enable the service so that it starts the next time you boot:

# systemctl enable syslog-ng.service
ln -s '/usr/lib/systemd/system/syslog-ng.service' '/etc/systemd/system/syslog.service'
ln -s '/usr/lib/systemd/system/syslog-ng.service' '/etc/systemd/system/multi-user.target.wants/syslog-ng.service'

5. Start the service now:

# systemctl start syslog-ng.service

6. Check if the text log file /var/log/messages has been created:

# ls /var/log
anaconda btmp cups faillog kdm.log messages news samba spice-vdagentd wtmp Xorg.0.log.old
boot.log chrony entropy journal lastlog mysql portage sandbox tallylog Xorg.0.log

If you now reboot you’ll see that the text file /var/log/messages is being used to log system messages, and you can view it easily using commands such as cat, more, less, nano or whatever, and search through it using grep etc. Note that the systemd journal service is still running; it is just not logging to a binary file any more and is forwarding system messages to syslog-ng.

How to install Sabayon Linux via the command line using the ‘text installer’

If you have trouble installing Sabayon Linux from one of the ISOs that include KDE, GNOME or Xfce, you could try installing from the ‘SpinBase’ ISO or ‘Minimal’ ISO instead and add a desktop environment later.

And sometimes the Sabayon Linux installer is unable to install the distribution correctly for the GPU or APU in your machine, so installing the distribution to use the X.Org VESA video driver then later trying another video driver can also be a way to get you to a working installation.

Whichever ‘spin’ of Sabayon Linux I try to install, I find my success varies considerably with the release of ISO. So, if you end up with a blank screen when you reboot after installing the distribution, try installing it again with a different ISO, such as a Monthly release instead of a Daily release. If none of the current ISOs result in a working installation, you may have to wait a few days for a new Daily release to be uploaded. Despite the name, the Daily ISOs are not necessarily released daily.

I recommend you practice installation in VirtualBox before trying with an actual HDD. You can do this on a PC running VirtualBox under Windows, Mac OS or Linux. N.B. If the console goes black at any time in VirtualBox, press Enter to refresh the contents.

The steps below worked for me with a Sabayon Linux ISO dated 14 September 2014 in VirtualBox running under Windows 8.1 on a machine with Intel HD Graphics APU. In the steps below I configure the installation for the time zone in which São Paulo is situated and specify British English and Brazilian Portuguese as the languages to be used. Obviously you should choose instead the time zone and language that you want in your installation.

1. Download the Daily SpinBase ISO (Sabayon_Linux_DAILY_x86_SpinBase.iso) from one of the ISO repositories (see the Download link on the distribution’s Web site).

2. Create a LiveDVD from the ISO (see my post Help for Windows users: How to create a Linux LiveCD, LiveDVD or LivePenDrive from an ISO file if you don’t know how to do that).

3. Boot the LiveDVD.

4. You have two choices here, really. Try (a) first and, if the LiveDVD will not continue to load the OS or it will but you end up with an installation that does not work, then re-boot the LiveDVD and perform (b) instead.

Either:

a) Press ‘Enter‘ at the Sabayon Linux boot menu, to start the OS from the LiveDVD.

or:

b) Press F5 at the Sabayon Linux boot menu, then Esc to close the pop-up list of boot parameters. Then delete the “---” at the end of the list of kernel boot parameters, delete the “vga=791“, add “xdriver=vesa” to the end of the list and press ‘Enter‘ to start the OS from the LiveDVD. Note that, unlike the other Sabayon Linux ISO releases, the SpinBase ISO does not include video drivers for X Windows. However, by booting the LiveDVD with xdriver=vesa the installation on the HDD will include that parameter in /etc/default/sabayon-grub and /boot/grub/grub.cfg.

In either case, Sabayon Linux on the LiveDVD should start in text mode and you should see a root user prompt:

sabayon ~ #

5. Enter the following command and follow the prompts to enter the relevant information to install Sabayon to your HDD:

# installer

Select 2) 'Use text mode' then work your way through steps 1 to 5, answering the installer’s prompts:

1) Timezone settings
2) Set root password
3) Create user
4) Network settings
5) Install Destination

When you select Step 5 (Install Destination) you will be presented with three ‘Autopartitioning Options’ (Replace Existing Linux System(s); Use All Space; Use Free Space). When you make your choice from those three options, you will then be presented with the options for partitioning:

Partition Scheme Options

[ ] 1) Standard Partition

[x] 2) LVM

[ ] 3) BTRFS

Select a partition scheme configuration.

Please make your choice from above ['q' to quit | 'c' to continue | 'r' to refresh]:

You may find that the installer crashes if you have selected either the default LVM partition scheme or the BTRFS partition scheme. If it does crash, run the installer again but select Option 1 instead (Standard Partition), which will create standard ext4 partitions. Actually, I prefer this option.

Quite a few Python-related error messages may be displayed towards the end of the installation process. I just ignore them.

6. Once the installer has completed installation, press Enter to quit, and you will see a root user prompt again.

7. Enter the following command to reboot (If you installed using VirtualBox, make sure you remove the ISO from the virtual CD/DVD drive, or you will just reboot from the ISO instead of the virtual HDD):

systemctl reboot

If you still have the LiveDVD in the drive, select ‘Boot from first hard drive’ in the LiveDVD’s boot menu.

8. You will see a log-in prompt:

sabayon login:

Log-in as user ‘root’ with the root user’s password you specified during installation.

9. List the available time zones:

# timedatectl list-timezones

(Press the Space Bar to page through the list, and the Q key to exit the list.)

10. Specify the timezone you want:

# timedatectl set-timezone America/Sao_Paulo

11. List the current locale:

# localectl list-locales
en_US
en_US.iso88591
en_US.utf8

12. If you want to change the current locale or add a locale, edit the locale.gen file:

# nano /etc/locale.gen

I wanted to have just British English and Brazilian Portuguese, so I made the file contain only the following:

en_GB.UTF-8 UTF-8
en_GB ISO-8859-1
pt_BR.UTF-8 UTF-8
pt_BR ISO-8859-1

If I only wanted one language (I’ll use Brazilian Portuguese as an example), I would have made it contain the following instead:

pt_BR.UTF-8 UTF-8
pt_BR ISO-8859-1

13. Generate the locale(s) you want:

# locale-gen
* Generating 4 locales (this might take a while) with 1 jobs
* (1/4) Generating en_GB.UTF-8 ... [ ok ]
* (2/4) Generating en_GB.ISO-8859-1 ... [ ok ]
* (3/4) Generating pt_BR.UTF-8 ... [ ok ]
* (4/4) Generating pt_BR.ISO-8859-1 ... [ ok ]
* Generation complete

14. List the locales you have configured, just to be sure:

# localectl list-locales
en_GB
en_GB.iso88591
en_GB.utf8
pt_BR
pt_BR.iso88591
pt_BR.utf8

15. Set the language you want to use:

# localectl set-locale LANG=pt_BR.UTF-8

16. List the console keymaps available:

# localectl list-keymaps

17. Chose the console keymap you wish to use:

# localectl set-keymap br-abnt2

N.B. The above systemd command does not change the console keymap specified in /boot/grub/grub.cfg, which remains as “vconsole.keymap=us“. You will have to fix that later by editing /etc/default/sabayon-grub and running grub2-mkconfig -o /boot/grub/grub.cfg, as I show further on.

18. List the X11 keymaps available:

# localectl list-x11-keymap-layouts

19. Chose the X11 keymap you wish to use:

# localectl set-x11-keymap br

20. Update the environment variables and profile to adopt what you specified:

# env-update && source /etc/profile

21. Reboot (Remove the LiveDVD if still have not already done so):

# systemctl reboot

22. Log-in as the root user.

23. Fix the console keymap specified in grub.cfg:

# nano /etc/default/sabayon-grub

and replace “vconsole.keymap=us” with “vconsole.keymap=br-abnt2“.

# grub2-mkconfig -o /boot/grub/grub.cfg

24. Make sure the Entropy package database in your installation is up to date:

# equo update

25. Roll (upgrade) all installed packages to their latest version:

# equo upgrade

If you are prompted regarding any package licences, just accept them.

26. Update any superseded configuration files:

# equo conf update

27. Check if there are any missing/incorrect dependencies:

# equo deptest

28. Check if there are any missing/incorrect libraries:

# equo libtest

29. Install the X.Org VESA video driver:

# equo install xf86-video-vesa

This is needed because the VESA video driver does not get installed even if you specify “xdriver=vesa” initially in the kernel boot parameter list when booting a LiveDVD using SpinBase.

30. Install the desktop environment of your choice (I’ll choose KDE):

# equo install kde-meta

31. If you are installing KDE, also install the KDE language pack(s) for the locale(s) configured earlier:

# equo install kde-l10n-en_GB kde-l10n-pt_BR

32. Install the Sabayon Linux theme for KDE and KDM (I find this package is not installed automatically, and stops KDM from working if it is not present):

# equo install sabayon-artwork-kde

33. Reboot:

# systemctl reboot

34. Log-in as the root user and, if you installed KDE, enable KDM as the desktop manager (log-in screen):

# systemctl enable kdm

35. Then reboot:

# systemctl reboot

The installation on your HDD should now boot to the KDM log-in screen. You should be able to enter your user name and password, and the installation should launch the Desktop Environment and display the Desktop. From here you will be able to open a Konsole/Terminal window and install other packages. If you want to change the video driver from the X.Org VESA driver to a driver for an AMD, Intel or NVIDIA GPU/APU, you will need to install the relevant driver package and edit /etc/X11/xorg.conf and /etc/default/sabayon-grub accordingly (and regenerate grub.cfg). Post in the Sabayon Linux Forums to ask for help if you need it. Good luck!

Where have my Konqueror favicons gone?

I upgraded to KDE 4.13.0 recently only to find that Konqueror no longer displayed some of the favicons, neither in the Bookmarks menu nor in the URL address bar. It seems this is a known KDE bug first reported in 2007 (Bug 153049 – Konqueror from KDE4 doesn’t load some favicons) although apparently it does not affect many users, which is why it still has not been fixed, I suppose.

In 2010 a KDE user reported in the KDE Community Forums thread Konqueror favicons again the steps he used to fix the problem in his installation, but he did not give the precise file names and paths of the files he deleted. In any case, I did not fancy deleting any sockets.

I tried various things, such as exporting and reimporting bookmarks in Konqueror, but was unable to get the missing favicons to display again. In the end I accepted I would have to lose all my bookmarks and decided to reinstall Konqueror. However, not all files are removed when a package is uninstalled, so I made sure everything was gone as follows:

1. Uninstall Konqueror

# emerge -C konqueror

2. Delete left-over directories and files relating to Konqueror

# rm -rf /home/fitzcarraldo/.kde4/share/apps/konqueror/
# rm /home/fitzcarraldo/.kde4/share/config/konq*

3. Log out of KDE and switch to a VT (virtual terminal, a.k.a. console)

# rm /var/tmp/kdecache-fitzcarraldo/favicons/*
# rm /var/tmp/kdecache-fitzcarraldo/icon-cache.kcache

4. Log in to KDE again and re-install Konqueror

# emerge -1v konqueror

5. Launch Konqueror and bookmark all your favourite Web sites.

That will get favicons working again in Konqueror, but what a hassle. KDE developers, please fix this old bug (no. 153049)!

Using KWrite to find and replace a character with a CRLF (Carriage Return/Line Feed)

Occasionally I need to edit a long string and replace the space character with a CRLF and some text. Even though I was sure the KDE editor KWrite could do that, I had never bothered to find out how. Today I finally bit the bullet. It’s not difficult, of course. To show you how it is done, I’ll give an actual example…

I wanted to edit in KWrite the single line of text shown below. (Not that it’s relevant to the subject of this post, but the line was a command to the Gentoo package manager to install a long list of packages, and I wanted to split it into separate commands in order to install each package individually.)

emerge -vD1 --backtrack=30 media-video/dvdrip:0 dev-vcs/git:0 dev-vcs/subversion:0 net-print/foomatic-db-engine:0 app-antivirus/clamtk:0 dev-perl/XML-SAX:0 dev-perl/X11-Protocol:0 dev-perl/Goo-Canvas:0 dev-perl/Readonly:0 dev-perl/File-Find-Rule:0 dev-perl/Net-SSLeay:0 dev-perl/XML-LibXML:0 dev-perl/HTTP-Message:0 dev-perl/Digest-SHA1:0 dev-perl/XML-XPath:0 dev-perl/File-Which:0 dev-perl/Authen-SASL:0 dev-perl/glib-perl:0 dev-perl/prefork:0 dev-perl/IO-Socket-SSL:0 dev-perl/Exception-Class:0 dev-perl/Proc-Simple:0 dev-perl/WWW-Mechanize:0 dev-perl/gnome2-canvas:0 dev-perl/gnome2-vfs-perl:0 dev-perl/IO-String:0 dev-perl/HTML-Tagset:0 dev-perl/Carp-Clan:0 dev-perl/Pod-Spell:0 dev-perl/Sane:0 dev-perl/TermReadKey:0 dev-perl/HTTP-Date:0 dev-perl/Encode-Locale:0 dev-perl/Event-RPC:0 dev-perl/File-HomeDir:0 dev-perl/Bit-Vector:0 dev-perl/gnome2-wnck:0 dev-perl/File-Copy-Recursive:0 dev-perl/Text-Unidecode:0 dev-perl/Unicode-EastAsianWidth:0 dev-perl/extutils-pkgconfig:0 dev-perl/Clone:0 dev-perl/Event-ExecFlow:0 dev-perl/B-Keywords:0 dev-perl/PDF-API2:0 dev-perl/HTTP-Negotiate:0 dev-perl/HTML-Form:0 dev-perl/extutils-depends:0 dev-perl/PlRPC:0 dev-perl/libwww-perl:0 dev-perl/gtk2-perl:0 dev-perl/File-MimeInfo:0 dev-perl/Font-TTF:0 dev-perl/libintl-perl:0 dev-perl/List-MoreUtils:0 dev-perl/Log-Log4perl:0 dev-perl/XML-DOM:0 dev-perl/HTML-Parser:0 dev-perl/Try-Tiny:0 dev-perl/XML-Twig:0 dev-perl/Gtk2-Ex-Simple-List:0 dev-perl/LWP-MediaTypes:0 dev-perl/LWP-Protocol-https:0 dev-perl/XML-Simple:0 dev-perl/Pango:0 dev-perl/set-scalar:0 dev-perl/Gtk2-Unique:0 dev-perl/Params-Util:0 dev-perl/Net-Daemon:0 dev-perl/GSSAPI:0 dev-perl/XML-NamespaceSupport:0 dev-perl/PPI:0 dev-perl/Proc-ProcessTable:0 dev-perl/String-Format:0 dev-perl/Date-Calc:0 dev-perl/XML-Parser:0 dev-perl/Email-Address:0 dev-perl/Class-Data-Inheritable:0 dev-perl/Email-Simple:0 dev-perl/JSON:0 dev-perl/gnome2-perl:0 dev-perl/XML-SAX-Base:0 dev-perl/Net-SMTP-SSL:0 dev-perl/Gtk2-ImageView:0 dev-perl/IO-HTML:0 dev-perl/WWW-RobotRules:0 dev-perl/Digest-HMAC:0 dev-perl/HTTP-Cookies:0 dev-perl/DBI:0 dev-perl/URI:0 dev-perl/Text-Iconv:0 dev-perl/gtk2-ex-formfactory:0 dev-perl/Email-Date-Format:0 dev-perl/libxml-perl:0 dev-perl/XML-SAX-Writer:0 dev-perl/XML-Filter-BufferText:0 dev-perl/Number-Compare:0 dev-perl/XML-RegExp:0 dev-perl/Email-LocalDelivery:0 dev-perl/config-general:0 dev-perl/HTTP-Daemon:0 dev-perl/File-Listing:0 dev-perl/Devel-StackTrace:0 dev-perl/Set-IntSpan:0 dev-perl/Cairo:0 dev-perl/Email-FolderType:0 dev-perl/XML-Handler-YAWriter:0 dev-perl/Archive-Zip:0 dev-perl/Net-DBus:0 dev-perl/DBD-mysql:0 dev-perl/AnyEvent:0 dev-perl/perltidy:0 dev-perl/Locale-gettext:0 dev-perl/Sort-Naturally:0 dev-perl/Net-HTTP:0 dev-perl/Perl-Critic:0 media-gfx/gscan2pdf:0 media-libs/exiftool:0 perl-core/CPAN-Meta-Requirements:0 virtual/perl-CPAN-Meta-Requirements:0 perl-core/IPC-Cmd:0 virtual/perl-IPC-Cmd:0 perl-core/Storable:0 virtual/perl-Storable:0 perl-core/File-Spec:0 virtual/perl-File-Spec:0 perl-core/CPAN-Meta:0 virtual/perl-CPAN-Meta:0 perl-core/Getopt-Long:0 virtual/perl-Getopt-Long:0 perl-core/Locale-Maketext-Simple:0 virtual/perl-Locale-Maketext-Simple:0 perl-core/ExtUtils-Manifest:0 virtual/perl-ExtUtils-Manifest:0 perl-core/Pod-Simple:0 virtual/perl-Pod-Simple:0 perl-core/CPAN-Meta-YAML:0 virtual/perl-CPAN-Meta-YAML:0 perl-core/Encode:0 virtual/perl-Encode:0 perl-core/Compress-Raw-Bzip2:0 virtual/perl-Compress-Raw-Bzip2:0 perl-core/Module-Load:0 virtual/perl-Module-Load:0 perl-core/Archive-Tar:0 virtual/perl-Archive-Tar:0 perl-core/Scalar-List-Utils:0 virtual/perl-Scalar-List-Utils:0 perl-core/ExtUtils-CBuilder:0 virtual/perl-ExtUtils-CBuilder:0 perl-core/Parse-CPAN-Meta:0 virtual/perl-Parse-CPAN-Meta:0 perl-core/version:0 virtual/perl-version:0 perl-core/Digest-SHA:0 virtual/perl-Digest-SHA:0 perl-core/Module-Load-Conditional:0 virtual/perl-Module-Load-Conditional:0 perl-core/Compress-Raw-Zlib:0 virtual/perl-Compress-Raw-Zlib:0 perl-core/ExtUtils-Install:0 virtual/perl-ExtUtils-Install:0 perl-core/IO:0 virtual/perl-IO:0 perl-core/Time-Local:0 virtual/perl-Time-Local:0 perl-core/Module-CoreList:0 virtual/perl-Module-CoreList:0 perl-core/Digest-MD5:0 virtual/perl-Digest-MD5:0 perl-core/JSON-PP:0 virtual/perl-JSON-PP:0 perl-core/ExtUtils-ParseXS:0 virtual/perl-ExtUtils-ParseXS:0 perl-core/File-Temp:0 virtual/perl-File-Temp:0 perl-core/Params-Check:0 virtual/perl-Params-Check:0 perl-core/Module-Metadata:0 virtual/perl-Module-Metadata:0 perl-core/Sys-Syslog:0 virtual/perl-Sys-Syslog:0 perl-core/IO-Compress:0 virtual/perl-IO-Compress:0 perl-core/Test-Harness:0 virtual/perl-Test-Harness:0

With the above line of text in a KWrite window, I did the following:

1. I selected Edit > Replace… from the KWrite menu.

2. At the bottom of the KWrite window, I changed the Mode from ‘Plain text’ to ‘Regular expression’.

3. I clicked in the ‘Find’ box and pressed the Space bar to enter a space character.

4. I clicked in the ‘Replace’ box and entered the following text (note that there is ‘\n’ at the beginning and a space at the end):

\nemerge -vD1 --backtrack=30 

The ‘\n‘ represents a CRLF (Carriage Return plus Line Feed).

5. I ticked ‘Selection only’.

6. With the mouse I selected the text in which I wanted to make the replacement, i.e. I selected from (and including) the space following the first package (media-video/dvdrip:0) all the way to the end of the line.

7. I clicked on ‘Replace All’.

The result looked like this:

emerge -vD1 --backtrack=30 media-video/dvdrip:0
emerge -vD1 --backtrack=30 dev-vcs/git:0
emerge -vD1 --backtrack=30 dev-vcs/subversion:0
emerge -vD1 --backtrack=30 net-print/foomatic-db-engine:0
emerge -vD1 --backtrack=30 app-antivirus/clamtk:0
emerge -vD1 --backtrack=30 dev-perl/XML-SAX:0
emerge -vD1 --backtrack=30 dev-perl/X11-Protocol:0
emerge -vD1 --backtrack=30 dev-perl/Goo-Canvas:0
emerge -vD1 --backtrack=30 dev-perl/Readonly:0
emerge -vD1 --backtrack=30 dev-perl/File-Find-Rule:0
emerge -vD1 --backtrack=30 dev-perl/Net-SSLeay:0
emerge -vD1 --backtrack=30 dev-perl/XML-LibXML:0
emerge -vD1 --backtrack=30 dev-perl/HTTP-Message:0
emerge -vD1 --backtrack=30 dev-perl/Digest-SHA1:0
emerge -vD1 --backtrack=30 dev-perl/XML-XPath:0
emerge -vD1 --backtrack=30 dev-perl/File-Which:0
emerge -vD1 --backtrack=30 dev-perl/Authen-SASL:0
emerge -vD1 --backtrack=30 dev-perl/glib-perl:0
emerge -vD1 --backtrack=30 dev-perl/prefork:0
emerge -vD1 --backtrack=30 dev-perl/IO-Socket-SSL:0
emerge -vD1 --backtrack=30 dev-perl/Exception-Class:0
emerge -vD1 --backtrack=30 dev-perl/Proc-Simple:0
emerge -vD1 --backtrack=30 dev-perl/WWW-Mechanize:0
emerge -vD1 --backtrack=30 dev-perl/gnome2-canvas:0
emerge -vD1 --backtrack=30 dev-perl/gnome2-vfs-perl:0
emerge -vD1 --backtrack=30 dev-perl/IO-String:0
emerge -vD1 --backtrack=30 dev-perl/HTML-Tagset:0
emerge -vD1 --backtrack=30 dev-perl/Carp-Clan:0
emerge -vD1 --backtrack=30 dev-perl/Pod-Spell:0
emerge -vD1 --backtrack=30 dev-perl/Sane:0
emerge -vD1 --backtrack=30 dev-perl/TermReadKey:0
emerge -vD1 --backtrack=30 dev-perl/HTTP-Date:0
emerge -vD1 --backtrack=30 dev-perl/Encode-Locale:0
emerge -vD1 --backtrack=30 dev-perl/Event-RPC:0
emerge -vD1 --backtrack=30 dev-perl/File-HomeDir:0
emerge -vD1 --backtrack=30 dev-perl/Bit-Vector:0
emerge -vD1 --backtrack=30 dev-perl/gnome2-wnck:0
emerge -vD1 --backtrack=30 dev-perl/File-Copy-Recursive:0
emerge -vD1 --backtrack=30 dev-perl/Text-Unidecode:0
emerge -vD1 --backtrack=30 dev-perl/Unicode-EastAsianWidth:0
emerge -vD1 --backtrack=30 dev-perl/extutils-pkgconfig:0
emerge -vD1 --backtrack=30 dev-perl/Clone:0
emerge -vD1 --backtrack=30 dev-perl/Event-ExecFlow:0
emerge -vD1 --backtrack=30 dev-perl/B-Keywords:0
emerge -vD1 --backtrack=30 dev-perl/PDF-API2:0
emerge -vD1 --backtrack=30 dev-perl/HTTP-Negotiate:0
emerge -vD1 --backtrack=30 dev-perl/HTML-Form:0
emerge -vD1 --backtrack=30 dev-perl/extutils-depends:0
emerge -vD1 --backtrack=30 dev-perl/PlRPC:0
emerge -vD1 --backtrack=30 dev-perl/libwww-perl:0
emerge -vD1 --backtrack=30 dev-perl/gtk2-perl:0
emerge -vD1 --backtrack=30 dev-perl/File-MimeInfo:0
emerge -vD1 --backtrack=30 dev-perl/Font-TTF:0
emerge -vD1 --backtrack=30 dev-perl/libintl-perl:0
emerge -vD1 --backtrack=30 dev-perl/List-MoreUtils:0
emerge -vD1 --backtrack=30 dev-perl/Log-Log4perl:0
emerge -vD1 --backtrack=30 dev-perl/XML-DOM:0
emerge -vD1 --backtrack=30 dev-perl/HTML-Parser:0
emerge -vD1 --backtrack=30 dev-perl/Try-Tiny:0
emerge -vD1 --backtrack=30 dev-perl/XML-Twig:0
emerge -vD1 --backtrack=30 dev-perl/Gtk2-Ex-Simple-List:0
emerge -vD1 --backtrack=30 dev-perl/LWP-MediaTypes:0
emerge -vD1 --backtrack=30 dev-perl/LWP-Protocol-https:0
emerge -vD1 --backtrack=30 dev-perl/XML-Simple:0
emerge -vD1 --backtrack=30 dev-perl/Pango:0
emerge -vD1 --backtrack=30 dev-perl/set-scalar:0
emerge -vD1 --backtrack=30 dev-perl/Gtk2-Unique:0
emerge -vD1 --backtrack=30 dev-perl/Params-Util:0
emerge -vD1 --backtrack=30 dev-perl/Net-Daemon:0
emerge -vD1 --backtrack=30 dev-perl/GSSAPI:0
emerge -vD1 --backtrack=30 dev-perl/XML-NamespaceSupport:0
emerge -vD1 --backtrack=30 dev-perl/PPI:0
emerge -vD1 --backtrack=30 dev-perl/Proc-ProcessTable:0
emerge -vD1 --backtrack=30 dev-perl/String-Format:0
emerge -vD1 --backtrack=30 dev-perl/Date-Calc:0
emerge -vD1 --backtrack=30 dev-perl/XML-Parser:0
emerge -vD1 --backtrack=30 dev-perl/Email-Address:0
emerge -vD1 --backtrack=30 dev-perl/Class-Data-Inheritable:0
emerge -vD1 --backtrack=30 dev-perl/Email-Simple:0
emerge -vD1 --backtrack=30 dev-perl/JSON:0
emerge -vD1 --backtrack=30 dev-perl/gnome2-perl:0
emerge -vD1 --backtrack=30 dev-perl/XML-SAX-Base:0
emerge -vD1 --backtrack=30 dev-perl/Net-SMTP-SSL:0
emerge -vD1 --backtrack=30 dev-perl/Gtk2-ImageView:0
emerge -vD1 --backtrack=30 dev-perl/IO-HTML:0
emerge -vD1 --backtrack=30 dev-perl/WWW-RobotRules:0
emerge -vD1 --backtrack=30 dev-perl/Digest-HMAC:0
emerge -vD1 --backtrack=30 dev-perl/HTTP-Cookies:0
emerge -vD1 --backtrack=30 dev-perl/DBI:0
emerge -vD1 --backtrack=30 dev-perl/URI:0
emerge -vD1 --backtrack=30 dev-perl/Text-Iconv:0
emerge -vD1 --backtrack=30 dev-perl/gtk2-ex-formfactory:0
emerge -vD1 --backtrack=30 dev-perl/Email-Date-Format:0
emerge -vD1 --backtrack=30 dev-perl/libxml-perl:0
emerge -vD1 --backtrack=30 dev-perl/XML-SAX-Writer:0
emerge -vD1 --backtrack=30 dev-perl/XML-Filter-BufferText:0
emerge -vD1 --backtrack=30 dev-perl/Number-Compare:0
emerge -vD1 --backtrack=30 dev-perl/XML-RegExp:0
emerge -vD1 --backtrack=30 dev-perl/Email-LocalDelivery:0
emerge -vD1 --backtrack=30 dev-perl/config-general:0
emerge -vD1 --backtrack=30 dev-perl/HTTP-Daemon:0
emerge -vD1 --backtrack=30 dev-perl/File-Listing:0
emerge -vD1 --backtrack=30 dev-perl/Devel-StackTrace:0
emerge -vD1 --backtrack=30 dev-perl/Set-IntSpan:0
emerge -vD1 --backtrack=30 dev-perl/Cairo:0
emerge -vD1 --backtrack=30 dev-perl/Email-FolderType:0
emerge -vD1 --backtrack=30 dev-perl/XML-Handler-YAWriter:0
emerge -vD1 --backtrack=30 dev-perl/Archive-Zip:0
emerge -vD1 --backtrack=30 dev-perl/Net-DBus:0
emerge -vD1 --backtrack=30 dev-perl/DBD-mysql:0
emerge -vD1 --backtrack=30 dev-perl/AnyEvent:0
emerge -vD1 --backtrack=30 dev-perl/perltidy:0
emerge -vD1 --backtrack=30 dev-perl/Locale-gettext:0
emerge -vD1 --backtrack=30 dev-perl/Sort-Naturally:0
emerge -vD1 --backtrack=30 dev-perl/Net-HTTP:0
emerge -vD1 --backtrack=30 dev-perl/Perl-Critic:0
emerge -vD1 --backtrack=30 media-gfx/gscan2pdf:0
emerge -vD1 --backtrack=30 media-libs/exiftool:0
emerge -vD1 --backtrack=30 perl-core/CPAN-Meta-Requirements:0
emerge -vD1 --backtrack=30 virtual/perl-CPAN-Meta-Requirements:0
emerge -vD1 --backtrack=30 perl-core/IPC-Cmd:0
emerge -vD1 --backtrack=30 virtual/perl-IPC-Cmd:0
emerge -vD1 --backtrack=30 perl-core/Storable:0
emerge -vD1 --backtrack=30 virtual/perl-Storable:0
emerge -vD1 --backtrack=30 perl-core/File-Spec:0
emerge -vD1 --backtrack=30 virtual/perl-File-Spec:0
emerge -vD1 --backtrack=30 perl-core/CPAN-Meta:0
emerge -vD1 --backtrack=30 virtual/perl-CPAN-Meta:0
emerge -vD1 --backtrack=30 perl-core/Getopt-Long:0
emerge -vD1 --backtrack=30 virtual/perl-Getopt-Long:0
emerge -vD1 --backtrack=30 perl-core/Locale-Maketext-Simple:0
emerge -vD1 --backtrack=30 virtual/perl-Locale-Maketext-Simple:0
emerge -vD1 --backtrack=30 perl-core/ExtUtils-Manifest:0
emerge -vD1 --backtrack=30 virtual/perl-ExtUtils-Manifest:0
emerge -vD1 --backtrack=30 perl-core/Pod-Simple:0
emerge -vD1 --backtrack=30 virtual/perl-Pod-Simple:0
emerge -vD1 --backtrack=30 perl-core/CPAN-Meta-YAML:0
emerge -vD1 --backtrack=30 virtual/perl-CPAN-Meta-YAML:0
emerge -vD1 --backtrack=30 perl-core/Encode:0
emerge -vD1 --backtrack=30 virtual/perl-Encode:0
emerge -vD1 --backtrack=30 perl-core/Compress-Raw-Bzip2:0
emerge -vD1 --backtrack=30 virtual/perl-Compress-Raw-Bzip2:0
emerge -vD1 --backtrack=30 perl-core/Module-Load:0
emerge -vD1 --backtrack=30 virtual/perl-Module-Load:0
emerge -vD1 --backtrack=30 perl-core/Archive-Tar:0
emerge -vD1 --backtrack=30 virtual/perl-Archive-Tar:0
emerge -vD1 --backtrack=30 perl-core/Scalar-List-Utils:0
emerge -vD1 --backtrack=30 virtual/perl-Scalar-List-Utils:0
emerge -vD1 --backtrack=30 perl-core/ExtUtils-CBuilder:0
emerge -vD1 --backtrack=30 virtual/perl-ExtUtils-CBuilder:0
emerge -vD1 --backtrack=30 perl-core/Parse-CPAN-Meta:0
emerge -vD1 --backtrack=30 virtual/perl-Parse-CPAN-Meta:0
emerge -vD1 --backtrack=30 perl-core/version:0
emerge -vD1 --backtrack=30 virtual/perl-version:0
emerge -vD1 --backtrack=30 perl-core/Digest-SHA:0
emerge -vD1 --backtrack=30 virtual/perl-Digest-SHA:0
emerge -vD1 --backtrack=30 perl-core/Module-Load-Conditional:0
emerge -vD1 --backtrack=30 virtual/perl-Module-Load-Conditional:0
emerge -vD1 --backtrack=30 perl-core/Compress-Raw-Zlib:0
emerge -vD1 --backtrack=30 virtual/perl-Compress-Raw-Zlib:0
emerge -vD1 --backtrack=30 perl-core/ExtUtils-Install:0
emerge -vD1 --backtrack=30 virtual/perl-ExtUtils-Install:0
emerge -vD1 --backtrack=30 perl-core/IO:0
emerge -vD1 --backtrack=30 virtual/perl-IO:0
emerge -vD1 --backtrack=30 perl-core/Time-Local:0
emerge -vD1 --backtrack=30 virtual/perl-Time-Local:0
emerge -vD1 --backtrack=30 perl-core/Module-CoreList:0
emerge -vD1 --backtrack=30 virtual/perl-Module-CoreList:0
emerge -vD1 --backtrack=30 perl-core/Digest-MD5:0
emerge -vD1 --backtrack=30 virtual/perl-Digest-MD5:0
emerge -vD1 --backtrack=30 perl-core/JSON-PP:0
emerge -vD1 --backtrack=30 virtual/perl-JSON-PP:0
emerge -vD1 --backtrack=30 perl-core/ExtUtils-ParseXS:0
emerge -vD1 --backtrack=30 virtual/perl-ExtUtils-ParseXS:0
emerge -vD1 --backtrack=30 perl-core/File-Temp:0
emerge -vD1 --backtrack=30 virtual/perl-File-Temp:0
emerge -vD1 --backtrack=30 perl-core/Params-Check:0
emerge -vD1 --backtrack=30 virtual/perl-Params-Check:0
emerge -vD1 --backtrack=30 perl-core/Module-Metadata:0
emerge -vD1 --backtrack=30 virtual/perl-Module-Metadata:0
emerge -vD1 --backtrack=30 perl-core/Sys-Syslog:0
emerge -vD1 --backtrack=30 virtual/perl-Sys-Syslog:0
emerge -vD1 --backtrack=30 perl-core/IO-Compress:0
emerge -vD1 --backtrack=30 virtual/perl-IO-Compress:0
emerge -vD1 --backtrack=30 perl-core/Test-Harness:0
emerge -vD1 --backtrack=30 virtual/perl-Test-Harness:0

That’s all there is to it. :-)

Installing the Windows version of Google Earth in WINE

Some Gentoo Linux users have reported that, although the native Linux release of Google Earth crashes, they can run the Windows version successfully under WINE. However, those users have also reported that the Windows installer for Google Earth did not work under WINE and so they copied the C:\Program Files\Google\Google Earth\ directory from a Windows PC to the virtual C:\ drive in their .wine directory (it would be ‘Program Files (x86)‘ in a 64-bit Windows installation, as Google Earth is a 32-bit application).

Now, if you download the Windows Google Earth installer from the Google Web site, what you get is a file GoogleEarthWin.exe that is 534.6 KiB in size (the size may vary depending on the release). However, you can instead download the Offline Installer using the following URL:

http://dl.google.com/earth/client/advanced/current/GoogleEarthWin.exe

and then you get a file GoogleEarthWin.exe that is 24.3 MiB in size (the size will vary depending on the release), which does run in WINE and does install the Windows version of Google Earth in WINE.

So, you might like to try that if you cannot run Google Earth in Linux but you have WINE installed. However, note that you will be wasting your time if the native Linux version of Google Earth crashes because of its incompatibility with the closed-source ATI or NVIDIA video driver. For example, Google Earth 7.1.2.2041 for Linux crashes on my main laptop using the 14.3_beta version of ati-drivers (AMD ATI Catalyst driver, a.k.a. FGLRX).

Anyway, if you want to install the Windows release of Google Earth under WINE here’s how to do it in a Konsole/Terminal window:

$ cd
$ export WINEPREFIX=$HOME/.wine-googleearth
$ export WINEARCH="win32"
$ winecfg
$ cd ./.wine-googleearth/drive_c/
$ wget http://dl.google.com/earth/client/advanced/current/GoogleEarthWin.exe
$ wine GoogleEarthWin.exe

And, to run it later:

$ env WINEPREFIX="/home/fitzcarraldo/.wine-googleearth" WINEARCH="win32" wine C:\\windows\\command\\start.exe /Unix /home/fitzcarraldo/.wine-googleearth/dosdevices/c:/users/fitzcarraldo/Start\ Menu/Programs/Google\ Earth/Google\ Earth.lnk

(Of course replace “fitzcarraldo” with your user name.)

But, as I wrote above, if the native Linux version of Google Earth crashes due to its incompatibility with the closed-source video driver (ATI or NVIDIA), it is highly unlikely the native Windows version will work under WINE.

Bypassing a corporate Web filter when using the command line

or ‘How to bypass a corporate Web filter and download YouTube videos via the command line’

One of the offices where I work uses a Web filter to block access to certain sites, such as YouTube. However, sometimes it is necessary to view blocked Web sites for work purposes. For example, these days a lot of companies or individuals post product reviews on YouTube that are useful for work purposes. In such cases I have used Tor to access the blocked sites in a Web browser such as Firefox, Chrome, Konqueror etc. See my post How to install and use Tor for anonymous browsing or to access country-restricted content from another country for details of how to set up and use Tor with a Web browser.

But sometimes I need to access blocked Web sites from the command line. For example, today I needed to download a YouTube video for work purposes, and I wanted to use youtube-dl to do it. The solution was simple…

First I launched vidalia and polipo as explained in the above-mentioned post on Tor, then I launched another Konsole/Terminal window and entered the commands shown below:

$ # First find out what resolutions are available for the video I want to download:
$ youtube-dl -F https://www.youtube.com/watch?v=T3Rr4CUoTSQ
Setting language
T3Rr4CUoTSQ: Downloading webpage
T3Rr4CUoTSQ: Downloading video info webpage
T3Rr4CUoTSQ: Extracting video information
[info] Available formats for T3Rr4CUoTSQ:
format code extension resolution note
140 m4a audio only DASH audio , audio@128k (worst)
160 mp4 192p DASH video
133 mp4 240p DASH video
134 mp4 360p DASH video
135 mp4 480p DASH video
136 mp4 720p DASH video
17 3gp 176x144
36 3gp 320x240
5 flv 400x240
43 webm 640x360
18 mp4 640x360
22 mp4 1280x720 (best)
$ # Now try to download the video at the resolution I want:
$ youtube-dl -f 22 -o Clevo_W230ST_overview.flv https://www.youtube.com/watch?v=T3Rr4CUoTSQ
Setting language
T3Rr4CUoTSQ: Downloading webpage
T3Rr4CUoTSQ: Downloading video info webpage
T3Rr4CUoTSQ: Extracting video information
ERROR: unable to download video data: HTTP Error 403: Forbidden

As you can see above, the corporate Web filter blocked youtube-dl from downloading the video.

So I informed the shell session about the local HTTP proxy (polipo) running on my laptop, by assigning and exporting the environment variable http_proxy using the following syntax:

export http_proxy=http://server-ip:port/

which in my case meant the following (refer to my article on Tor):

$ export http_proxy=http://127.0.0.1:8123/

and then I was able to download the video from YouTube despite the corporate Web filter:

$ youtube-dl -f 22 -o Clevo_W230ST_overview.flv https://www.youtube.com/watch?v=T3Rr4CUoTSQ
Setting language
T3Rr4CUoTSQ: Downloading webpage
T3Rr4CUoTSQ: Downloading video info webpage
T3Rr4CUoTSQ: Extracting video information
[download] Destination: Clevo_W230ST_overview.flv
[download] 100% of 100.23MiB in 05:50
$

Useful Reference: How To Use Proxy Server To Access Internet at Shell Prompt With http_proxy Variable

Fixing a problem with received video in Skype when using the AMD Catalyst (FGLRX) driver in Linux

Some users of Skype for Linux have reported that the bottom half of the received video image is corrupted in installations that use the closed-source video driver for ATI GPUs (the AMD Catalyst proprietary Linux driver, also known as the ‘FGLRX’ driver). One user described the lower half of the video image as “covered in small coloured squares like a chequer board”.

From what I have read in a few forums, it seems the problem does not occur when the open-source Radeon driver is used. My own experience corroborates that: I use the Radeon driver on one of my laptops, and received video in Skype is fine.

My main laptop has an AMD ATI Mobility Radeon HD 5650 GPU and I am using the Catalyst driver under Gentoo Linux. In this case there was a problem with received video in most Skype sessions. Either of the following effects usually occurred:

Snapshot 1 - Extract of received video image in Skype, showing an example of the corrupted image

Snapshot 1 - Extract of received video image in Skype, showing an example of the corrupted image

Snapshot 2 - Extract of received video image in Skype, showing another example of the corrupted image

Snapshot 2 - Extract of received video image in Skype, showing another example of the corrupted image

As shown in Snapshot 1, the lower half of the received video image was covered in a grid of thin green lines with areas tinged with purple, blue or green, whereas there was no grid of lines in the upper half of the image but some areas were tinged with red or blue.

As shown in Snapshot 2, the lower half of the received video image was covered in a grid of thin red lines, with a purple tinge in some areas, whereas there was no grid of lines in the upper half of the image, which looked reasonable but had some red-, green- or blue-tinged areas.

In all cases Skype’s thumbnail of my Webcam’s video image looked fine, and the person on the other end of the call said the video image received from me looked fine too.

Because of a bug in a previous version of the Catalyst driver a few years ago — see my blog posts Playing QuickTime videos in Firefox and Chromium + XVideo bug in AMD Catalyst 11.11 and 11.12 driver and AMD Catalyst for Linux driver 12.2 fixes the XVideo bug that crashed X.Org Server 1.11.x — I happen to know that Sykpe uses X11 overlays with the XVideo extension (xv), rather than using the OpenGL renderer (gl) or X11 with the SHM extension (x11). This made me wonder whether the use of XVideo with the Catalyst driver was causing the current problem. Unlike media players such as MPlayer and VLC, it is not possible to configure Skype to use gl or x11 instead of xv, so I thought it would not be possible to test whether the use of gl or x11 instead of xv would make a difference. Until, that is, I came upon a ‘trick’ posted by openSUSE user queequeg in 2009 during the period when an earlier version of the Catalyst driver had the aforementioned bug:

Skype Video Workaround for ATI

Anybody trying to make a video call with Skype and ATI fglrx drivers has had problems due to Skype using the “xv” video mode with the driver can’t handle. For anyone interested that is affected by this, there is a workaround:

1. Run the xvinfo command and look at the number of xv sessions available. Some cards have only 1, some have as many as 4. This is the number of xv occurances that the card can do at one time.
2. “Use up” all these xv sessions by opening videos in your favorite video player making sure to use xv for the video output. The videos can then be paused.
3. Once this (or they) are open, skype can be started and will default to X11 video and work properly with video calls.

I know this is a goofy way to get around this issue, but until fglrx can handle xv or skype allows an option to choose X11 for video render, I don’t know of any other way to do it.

(From what I hear, the 11.1 fglrx drivers can handle xv, but I haven’t confirmed this.)

So I tried his work-around. I had to launch four media players in order to use all available XVideo sessions. Lo and behold, when I launched Skype and made a video call the received video image was perfect. So it appeared that the Catalyst driver is not able to handle well the XVideo output from Skype. However, playing and pausing four videos every time I want to make a video call in Skype would hardly be practical, would it? And that is not the only downside: when I maximised a Firefox window during the Skype video call, my laptop spontaneously rebooted (I assume the X.Org server crashed).

I did also wonder whether just disabling compositing would solve the problem, so I disabled KWin Desktop Effects, but that didn’t make any difference.

I had also read in several forums that enabling or disabling the TexturedVideo and/or VideoOverlay options in the xorg.conf file have an effect on the video image produced by the Catalyst driver, but I could not find a post mentioning the use of either of those options to fix the specific problem I was seeing. So I decided not to pursue the xorg.conf route.

In my searches of the Web I came across a post somewhere that mentioned using GTK+ UVC Viewer (guvcview) to adjust video properties and improve video in Skype. I thought guvcview was only for adjusting the video image from a Webcam connected to my machine, i.e. adjusting the outgoing video image, and would not have any effect on received video. Nevertheless, I decided to install and launch guvcview to see if I could adjust both incoming and outgoing video properties. To my surprise, guvcview appeared to have fixed the problem with the received video. These are the steps I followed:

  1. I launched Skype and started a video call. The received video image had a grid of thin red lines and purple/green/blue tinting (similar to Snapshot 2).
  2. I Installed guvcview using the package manager.
  3. I launched guvcview in a Konsole (terminal) window. After guvcview created the file /home/fitzcarraldo/.config/guvcview/video0 and checked various video and audio settings it exited because my Webcam was being used by Skype (‘libv4l2: error setting pixformat: Device or resource busy‘).
  4. I clicked on the Webcam icon in the Skype call window, to turn my Webcam off.
  5. I launched guvcview again. The lower half of the received video image in Skype changed from a grid of thin red lines to a continuous green-coloured band, and the upper half of the image now looked reasonable but still had some red- or blue-tinged areas (see Snapshot 3 below).
  6. Snapshot 3 - Extract of received video image in Skype after I launched guvcview again

    Snapshot 3 - Extract of received video image in Skype after I launched guvcview again

  7. On the ‘Image Controls’ tab in the ‘GUVCViewer Controls’ window I changed the video frequency from 60 Hz to 50 Hz then back to 60 Hz again. I was just tinkering, and I believe this had no bearing on the outcome.
  8. I clicked on the ‘Quit’ button in the guvcview window to terminate the application.
  9. I clicked on the Webcam icon in the Skype call window to turn on again the Webcam, and the received Skype video image changed to a perfect image (see Snapshot 4 below).
  10. Snapshot 4 - Extract of received video image in Skype after I turned on again my Webcam in Skype

    Snapshot 4 - Extract of received video image in Skype after I turned on again my Webcam in Skype

It appears that guvcview had an effect on the received video image in Skype, although, if it did, I do not understand how. To check if the fix was permanent I ended the Skype video call, signed out of Skype and quit the application, rebooted and made a new Skype video call. The received video image in Skype was again perfect. I even deleted the guvcview configuration file and repeated this check, just in case the configuration file was somehow being used even though I had not launched guvcview, but the received video in yet another Skype video call was still perfect. I also clicked on the Webcam icon in the Skype call window several times during each call in order to turn my Webcam off and on several times; the received video image of the other person remained perfect.

So there you have it: when using an AMD ATI GPU and the Catalyst driver, it seems that guvcview can be used — at least in my case — to eliminate the type of image corruption in received Skype video shown in Snapshots 1 and 2. So, if you are also using the AMD Catalyst for Linux driver and are experiencing a similar problem, try guvcview. It might just do the trick.

Follow

Get every new post delivered to your Inbox.

Join 52 other followers