Gentoo Linux installations without initramfs: Updating Intel CPU microcode

This post explains how I configured the Gentoo Linux (amd64) installation on my Clevo W230SS laptop to make it update the Intel CPU microcode using the kernel Microcode Early Update driver.

Back in April I installed Gentoo Linux Stable and built the kernel using the default manual configuration steps given in the Gentoo Handbook, and my installation therefore did not use an initramfs. The contents of the /boot directory were as listed below:

# ls -1 /boot
System.map-3.18.11-gentoo
config-3.18.11-gentoo
grub
lost+found
vmlinuz-3.18.11-gentoo

Contrast the above with the boot directory of the Gentoo Linux Testing (~amd64) installation on my Compal NBLB2 laptop, which does use an initramfs because I built the kernel using the optional genkernel utility:

# ls -1 /boot
System.map-genkernel-x86_64-3.18.11-gentoo
boot
grub
initramfs-genkernel-x86_64-3.18.11-gentoo
kernel-genkernel-x86_64-3.18.11-gentoo
lost+found

Since I wrote my earlier post on udating CPU microcode, the use of a kernel module and initscript to update CPU microcode has been dropped (in Gentoo Linux, at least). This is because updating CPU microcode relatively late in the boot process may cause problems if some processes have started before the update has taken place. The only safe way to update CPU microcode is to use the kernel’s built-in Microcode Early Update driver (METHOD 2 in my earlier post).

Now, new versions of the relevant Gentoo Linux ebuilds (microcode-ctl-1.28-r1, microcode-data-20150121-r1 and iucode_tool-1.3) have recently been released (see Gentoo Bug Report No. 528712). If a Gentoo Linux installation does not use an initramfs, the microcode-data ebuild now includes an ‘initramfs‘ USE flag which you can set in order to create a minimal initramfs to load the CPU microcode at boot. Below I explain in detail how I configured my installation to update the CPU microcode early during boot.

Before doing anything at all, I checked the microcode version for my CPU (Intel Core i7-4810MQ @ 2.80GHz):

# grep microcode /proc/cpuinfo
microcode   : 0x12
microcode   : 0x12
microcode   : 0x12
microcode   : 0x12
microcode   : 0x12
microcode   : 0x12
microcode   : 0x12
microcode   : 0x12
# dmesg | grep microcode
[    0.262544] microcode: CPU0 sig=0x306c3, pf=0x10, revision=0x12
[    0.262641] microcode: CPU1 sig=0x306c3, pf=0x10, revision=0x12
[    0.262740] microcode: CPU2 sig=0x306c3, pf=0x10, revision=0x12
[    0.262841] microcode: CPU3 sig=0x306c3, pf=0x10, revision=0x12
[    0.262939] microcode: CPU4 sig=0x306c3, pf=0x10, revision=0x12
[    0.263038] microcode: CPU5 sig=0x306c3, pf=0x10, revision=0x12
[    0.263138] microcode: CPU6 sig=0x306c3, pf=0x10, revision=0x12
[    0.263236] microcode: CPU7 sig=0x306c3, pf=0x10, revision=0x12
[    0.263367] microcode: Microcode Update Driver: v2.00 , Peter Oruba

Fortunately I did not need to rebuild the kernel as I had already configured it back in April when I installed Gentoo, with a view to implementing early microcode updating at some point:

# grep CONFIG_BLK_DEV_INITRD /usr/src/linux/.config
CONFIG_BLK_DEV_INITRD=y
# grep CONFIG_MICROCODE /usr/src/linux/.config
CONFIG_MICROCODE=y
CONFIG_MICROCODE_INTEL=y
# CONFIG_MICROCODE_AMD is not set
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_MICROCODE_INTEL_EARLY=y
# CONFIG_MICROCODE_AMD_EARLY is not set
CONFIG_MICROCODE_EARLY=y
# grep CONFIG_INITRAMFS_SOURCE /usr/src/linux/.config
CONFIG_INITRAMFS_SOURCE=""

Then I set the initramfs USE flag and unmasked the ~amd64 ebuilds created by Gentoo developer SpanKY (microcode-ctl-1.28-r1.ebuild, microcode-data-20150121-r1.ebuild and iucode_tool-1.3.ebuild) as discussed in the above-mentioned Gentoo bug report:

# cat /etc/portage/package.use/microcode-data
sys-apps/microcode-data initramfs
# cat /etc/portage/package.accept_keywords/microcode-data
sys-apps/microcode-data ~amd64
sys-apps/iucode_tool ~amd64
# cat /etc/portage/package.accept_keywords/microcode-ctl
sys-apps/microcode-ctl ~amd64

Then I merged the packages:

# emerge --ask microcode-ctl microcode-data

This resulted in the following files:

# ls -la /lib/firmware/microcode.cpio
-rw-r--r-- 1 root root 663552 Aug 26 15:06 /lib/firmware/microcode.cpio
# ls -la /lib/firmware/intel-ucode
total 768
drwxr-xr-x  2 root root  4096 Aug 26 15:06 .
drwxr-xr-x 62 root root 16384 Aug 26 15:06 ..
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-03-02
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-05-00
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-05-01
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-05-02
-rw-r--r--  1 root root  8192 Aug 26 15:06 06-05-03
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-06-00
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-06-05
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-06-0a
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-06-0d
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-07-01
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-07-02
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-07-03
-rw-r--r--  1 root root 10240 Aug 26 15:06 06-08-01
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-08-03
-rw-r--r--  1 root root 10240 Aug 26 15:06 06-08-06
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-08-0a
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-09-05
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-0a-00
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-0a-01
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-0b-01
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-0b-04
-rw-r--r--  1 root root  2048 Aug 26 15:06 06-0d-06
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-0e-08
-rw-r--r--  1 root root  8192 Aug 26 15:06 06-0e-0c
-rw-r--r--  1 root root  8192 Aug 26 15:06 06-0f-02
-rw-r--r--  1 root root 12288 Aug 26 15:06 06-0f-06
-rw-r--r--  1 root root  8192 Aug 26 15:06 06-0f-07
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-0f-0a
-rw-r--r--  1 root root 28672 Aug 26 15:06 06-0f-0b
-rw-r--r--  1 root root 12288 Aug 26 15:06 06-0f-0d
-rw-r--r--  1 root root 12288 Aug 26 15:06 06-16-01
-rw-r--r--  1 root root 20480 Aug 26 15:06 06-17-06
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-17-07
-rw-r--r--  1 root root 24576 Aug 26 15:06 06-17-0a
-rw-r--r--  1 root root 14336 Aug 26 15:06 06-1a-04
-rw-r--r--  1 root root 10240 Aug 26 15:06 06-1a-05
-rw-r--r--  1 root root 15360 Aug 26 15:06 06-1c-02
-rw-r--r--  1 root root 20480 Aug 26 15:06 06-1c-0a
-rw-r--r--  1 root root  4096 Aug 26 15:06 06-1d-01
-rw-r--r--  1 root root  6144 Aug 26 15:06 06-1e-04
-rw-r--r--  1 root root  7168 Aug 26 15:06 06-1e-05
-rw-r--r--  1 root root  8192 Aug 26 15:06 06-25-02
-rw-r--r--  1 root root  3072 Aug 26 15:06 06-25-05
-rw-r--r--  1 root root 10240 Aug 26 15:06 06-26-01
-rw-r--r--  1 root root 10240 Aug 26 15:06 06-2a-07
-rw-r--r--  1 root root 16384 Aug 26 15:06 06-2d-06
-rw-r--r--  1 root root 17408 Aug 26 15:06 06-2d-07
-rw-r--r--  1 root root 13312 Aug 26 15:06 06-2f-02
-rw-r--r--  1 root root 12288 Aug 26 15:06 06-3a-09
-rw-r--r--  1 root root 21504 Aug 26 15:06 06-3c-03
-rw-r--r--  1 root root 14336 Aug 26 15:06 06-3d-04
-rw-r--r--  1 root root 13312 Aug 26 15:06 06-3e-04
-rw-r--r--  1 root root 11264 Aug 26 15:06 06-3e-06
-rw-r--r--  1 root root 15360 Aug 26 15:06 06-3e-07
-rw-r--r--  1 root root 28672 Aug 26 15:06 06-3f-02
-rw-r--r--  1 root root 20480 Aug 26 15:06 06-45-01
-rw-r--r--  1 root root 23552 Aug 26 15:06 06-46-01
-rw-r--r--  1 root root  4096 Aug 26 15:06 0f-00-07
-rw-r--r--  1 root root  6144 Aug 26 15:06 0f-00-0a
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-01-02
-rw-r--r--  1 root root  6144 Aug 26 15:06 0f-02-04
-rw-r--r--  1 root root  8192 Aug 26 15:06 0f-02-05
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-02-06
-rw-r--r--  1 root root  6144 Aug 26 15:06 0f-02-07
-rw-r--r--  1 root root  6144 Aug 26 15:06 0f-02-09
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-03-02
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-03-03
-rw-r--r--  1 root root  7168 Aug 26 15:06 0f-03-04
-rw-r--r--  1 root root 10240 Aug 26 15:06 0f-04-01
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-04-03
-rw-r--r--  1 root root  3072 Aug 26 15:06 0f-04-04
-rw-r--r--  1 root root  3072 Aug 26 15:06 0f-04-07
-rw-r--r--  1 root root  9216 Aug 26 15:06 0f-04-08
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-04-09
-rw-r--r--  1 root root  4096 Aug 26 15:06 0f-04-0a
-rw-r--r--  1 root root  3072 Aug 26 15:06 0f-06-02
-rw-r--r--  1 root root  6144 Aug 26 15:06 0f-06-04
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-06-05
-rw-r--r--  1 root root  2048 Aug 26 15:06 0f-06-08

(By the way, the file 06-3c-03 is the microcode file for my particular CPU: Family 06h, Model 03Ch, Stepping 03h, as determined from the CPU World Web site.)

Then I copied to the boot directory the cpio file created when I merged the packages:

# cp /lib/firmware/microcode.cpio /boot/

Then I added an initrd line to /boot/grub/grub.cfg as shown below:

# grep -B 15 -A 1 initrd /boot/grub/grub.cfg

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Gentoo GNU/Linux' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-525a90f1-8ad2-44a3-ade3-20f18a0a9595' {
        load_video
        insmod gzio
        insmod part_msdos
        insmod ext2
        set root='hd0,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  f6ffc085-66fe-4bbe-b080-cec355749f85
        else
          search --no-floppy --fs-uuid --set=root f6ffc085-66fe-4bbe-b080-cec355749f85
        fi
        echo    'Loading Linux 3.18.11-gentoo ...'
        linux   /vmlinuz-3.18.11-gentoo root=/dev/sda5 ro  drm_kms_helper.edid_firmware=edid/1920x1080_Clevo_W230SS.bin i915.modeset=1 rcutree.rcu_idle_gp_delay=1
        initrd /microcode.cpio
}

After rebooting:

# grep microcode /proc/cpuinfo
microcode       : 0x1c
microcode       : 0x1c
microcode       : 0x1c
microcode       : 0x1c
microcode       : 0x1c
microcode       : 0x1c
microcode       : 0x1c
microcode       : 0x1c
# dmesg | grep microcode
[    0.000000] CPU0 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.049968] CPU1 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.065070] CPU2 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.080101] CPU3 microcode updated early to revision 0x1c, date = 2014-07-03
[    0.268736] microcode: CPU0 sig=0x306c3, pf=0x10, revision=0x1c
[    0.268838] microcode: CPU1 sig=0x306c3, pf=0x10, revision=0x1c
[    0.268935] microcode: CPU2 sig=0x306c3, pf=0x10, revision=0x1c
[    0.269034] microcode: CPU3 sig=0x306c3, pf=0x10, revision=0x1c
[    0.269131] microcode: CPU4 sig=0x306c3, pf=0x10, revision=0x1c
[    0.269230] microcode: CPU5 sig=0x306c3, pf=0x10, revision=0x1c
[    0.269329] microcode: CPU6 sig=0x306c3, pf=0x10, revision=0x1c
[    0.269448] microcode: CPU7 sig=0x306c3, pf=0x10, revision=0x1c
[    0.269570] microcode: Microcode Update Driver: v2.00 , Peter Oruba

As you can see above, the microcode early update kernel driver updates the microcode version to 0x1c at boot (cf. 0x12 originally). If you have a different model of CPU to mine, of course the version in your case could be different.

Note that the line ‘initrd /lib/firmware/microcode.cpio‘ in the file grub.cfg does not work for me. It has to be ‘initrd /microcode.cpio‘ and the file microcode.cpio has to be in the /boot directory:

# ls /boot
System.map-3.18.11-gentoo  config-3.18.11-gentoo  grub  lost+found  microcode.cpio  vmlinuz-3.18.11-gentoo

Note also that the precise steps specified in Comment #41 of Gentoo Bug Report No. 528712 do not work for me and others (see Gentoo Forums thread microcode-data-20150121-r1 with USE=initramfs: kernel panic). If I build the kernel with CONFIG_INITRAMFS_SOURCE="/lib/firmware/microcode.cpio", delete the line ‘initrd /microcode‘ from grub.cfg, delete the file /boot/microcode.cpio and reboot, the kernel panics, and the last two console lines are as follows:

[    3.142079] drm_kms_helper: panic occurred, switching back to text console
[    3.142879] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,5)

Unless Gentoo developer SpanKY explains how he managed to get the procedure he gave in Comment #41 of Gentoo Bug Report No. 528712 to work, it seems the normal behaviour when applying the steps he gave would be for the kernel to panic. Even Gentoo developer Daniel Pielmeier experienced a kernel panic when he applied SpanKY‘s steps (Comment #54 of Gentoo Bug Report No. 528712).

As the procedure I gave above works fine, I therefore reverted to using CONFIG_INITRAMFS_SOURCE="", copying /lib/firmware/microcode.cpio to /boot/microcode.cpio again and re-adding the line ‘initrd /microcode.cpio‘ in the file grub.cfg. However, if you did not have an initramfs and were able to get the precise procedure given in Comment #41 of Gentoo Bug Report #528712 to work, please post a comment here.

‘Waiting for 192.168.1.254…’ (Why I could not access a home hub’s management page)

I had not been able to access the Manager of the BT Home Hub 3 on my home network to view and configure the hub’s settings. All the network’s users could access the Internet, and I could ping the hub, but trying to access the BT Home Hub Manager from a Web browser resulted in the message ‘Waiting for 192.168.1.254…’. The same thing happened whatever the PC, OS, browser and method of connection (wired or wireless). Sometimes, after about ten minutes or so, an incomplete Manager page would appear, but usually the browser would just display ‘Waiting for 192.168.1.254…’ forever.

I should point out that my Ethernet wired connections use Powerline adapters (HomePlug) connected to the mains wiring of my semi-detached house.

Actually, I did find a temporary work-around to enable me to access the Home Hub Manager. If I switched off then on the power supply to the Home Hub I could access the Manager for a short period (the time varied, but typically was less than half an hour). Then I would be back in the same position of seeing ‘Waiting for 192.168.1.254…’ in a browser window if I tried later to access the Manager. Although I do not need to access the Home Hub Manager often, it was still a nuisance to have to cycle the power to the hub every time I needed to access the Manager.

Searching the Web, it seems this is quite a common problem and can occur irrespective of the manufacturer of the hub (or router) and its IP address. In some cases users have fixed the problem by upgrading the hub’s firmware or by performing a ‘factory reset’ of the hub, but some users never found a solution.

In my case, the BT Home Hub 3 has the latest available version of firmware installed. Not only did I check that via the Web, I also checked the firmware version of another BT Home Hub 3 in the house of someone I know who lives in another town. The curious thing was that he has no trouble accessing the BT Home Hub Manager (also IP address 192.168.1.254).

So I decided to perform a ‘factory reset’ of the Home Hub, but that made no difference.

Then, after many hours searching the Web, I found a thread about a similar problem with a different model of hub: Can’t access BT HomeHub 4? But I’m online ok?. A post by user troublegum in that thread made me sit up:

I still reckon it’s the homeplugs. Regardless of whether your PC is connected to it or not, If one of them is connected to your neighbour’s as well as your router, then it’s going to put 2 DHCP servers on your network.

Disconnect the homeplug from the router, renew your DHCP lease if necessary and try again.

Even before finding that thread I had wondered if the problem was somehow linked to my use of Powerline (HomePlug) adapters.

It seems that, if one PC on a home network is connected to the Home Hub via a Powerline adapter AND a neighbour also happens to be using Powerline adapters AND his single-phase mains house wiring is somehow linked to yours (which is unusual, as adjacent houses are normally connected to a different mains phase), there is the possibility that none of your PCs will be able to access the Home Hub Manager (even if they are connected directly to the Home Hub by Ethernet cable or Wi-Fi rather than via a Powerline adapter).

I have been using Powerline (HomePlug) adapters successfully for about nine years. In late December 2012 I changed from HomePlug 1.0 adapters (14 Mbps) to HomePlug AV adapters (200 Mbps). HomePlug 1.0 adapters and HomePlug AV adapters can operate concurrently over the same mains wiring but can only communicate with adapters of the same standard. The problem of not being able to access the Home Hub Manager started two or three years ago, so I assume that either my neighbour began using Powerline adapters at that time or, coincidentally, I changed to the same standard and manufacturer of Powerline adapter he uses.

Powerline adapters each have a non-volatile encryption key, intended to enable separate Powerline networks to co-exist on the same mains wiring by using a different encryption key for each network.

Since the end of December 2012 I have been using NETGEAR XAVB1301 200 Mbps Powerline adapters but had not bothered to change the encryption key in them (they all come configured with the factory default encryption key ‘HomePlugAV’). If my neighbour happens to be using Powerline adapters with the same default encryption key, and a hub with the same IP address as mine, we would both have two DHCP servers on the same network.

So I changed the encryption key on each of the four Powerline adapters I use:

  • Ethernet connection from the BT Home Hub to a mains socket in the Lounge.
  • Ethernet connection from a PC to a mains socket in the Lounge.
  • Ethernet connection from a laptop to a mains socket in my upstairs office.
  • Ethernet connection from a laptop to a mains socket in a bedroom.

It is supposed to be easy to set the encryption key in the model of Powerline adapter I use. You have to press a button on one adapter for 2 seconds, then a button on the next adapter for 2 seconds, and so on. You have to do them all within 2 minutes. The adapters only generate an encryption key once, so if you want to repeat the process you first have to press a recessed Factory Reset button on all the adapters.

However, despite following to the letter the instructions in the NETGEAR manual, I could not get all four adapters to connect to the network. So I downloaded the NETGEAR Powerline Universal Utility, installed it on the PC running Windows 10 in my lounge, connected the Ethernet port of that PC to one of the Powerline adapters and plugged it into a mains wall socket, plugged the other three Powerline adapters into a multi-socket mains adapter and plugged that into a mains wall socket in the lounge, launched the Powerline Universal Utility and I allocated all four adapters the same encryption key. Each adapter has its own MAC address, serial number and ‘Device Password’ (PWD) printed on it, and the NETGEAR utility program required me to enter the relevant PWD for each MAC address. Then I entered an encryption key (any string of characters of my choice) and clicked a button to set the adapters to use that encryption key. As that encryption key is different to the default key used by my neighbour, the two networks can now coexist without interfering with each other.

NETGEAR Powerline Utility showing my four Powerline adapters

NETGEAR Powerline Utility showing my four Powerline adapters.

The use of the NETGEAR Powerline utility program is explained in NETGEAR’s ‘How To’ Setting network encryption key on Powerline Adapters using the Config utility.

Problem finally solved! I can now access the Home Hub Manager without any trouble. And, as a bonus, Internet access seems a little quicker.

Installing the DraftSight Free 2D CAD application in Gentoo Linux

I occasionally have to view AutoCAD files and had been using a 2009 version of VariCAD Viewer for Linux, installed from an RPM package using the rpm command in Gentoo Linux. It was the only version that I could get to work correctly in Gentoo Linux. Versions from more recent years would either not install at all or would install but not run correctly. And the 2009 version of VariCAD Viewer could not open more-recent AutoCAD files. So I had been looking for an alternative for quite a while.

Recently I found out about DraftSight, which is produced by Dassault Systèmes SolidWorks Corp. and is available for Windows, Mac OS and Linux. Apart from the commercial versions DraftSight Professional and DraftSight Enterprise, there is also a free version, DraftSight Free, which is billed as ‘straightforward 2D CAD software for students, hobbyists and other individuals’. I thought this would suit my purposes, as it can read and write DWG and DXF files (see the features page on the DraftSight Web site).

The Gentoo Linux betagarden overlay has ebuilds for releases of DraftSight Free. The package is a proprietary binary package, the Gentoo ebuild is named media-gfx/draftsight-bin and the current version in the betagarden overlay is 1.6.1_beta.

So I added the betagarden overlay using Layman, downloaded the file draftSight.deb (DraftSight 2015 beta) from the DraftSight Web site and copied it to /usr/portage/distfiles/draftsight-1.6.1_beta.deb as specified in the ebuild, and issued the usual emerge command to merge the package. However the ebuild would not install the package whatever I tried: Portage gave an error message that the .deb file could not be downloaded (despite it already being in the distfiles directory). In the end I copied the ebuild and its files sub-directory to /usr/local/portage/media-gfx/draftsight-bin/ in my local overlay, disconnected from the network and merged the package:

# layman -a betagarden
# rm /usr/portage/distfiles/draftsight*
# cp /home/fitzcarraldo/Downloads/draftSight.deb /usr/portage/distfiles/draftsight-1.6.1_beta.deb
# mkdir -p /usr/local/portage/media-gfx/draftsight-bin/files
# cp /var/lib/layman/betagarden/media-gfx/draftsight-bin/draftsight-bin-1.6.1_beta.ebuild /usr/local/portage/media-gfx/draftsight-bin/
# cp /var/lib/layman/betagarden/media-gfx/draftsight-bin/files/* /usr/local/portage/media-gfx/draftsight-bin/files/
# layman -d betagarden
# cd /usr/local/portage/media-gfx/draftsight-bin/
# ebuild draftsight-bin-1.6.1_beta.ebuild manifest
# emerge --ask draftsight-bin

This worked, and I can now launch DraftSight from the KDE launcher’s menu (the only additional thing I did was to specify an icon myself using the KDE Menu Editor, as the DraftSight entry in the KDE launcher menu was icon-less) or by issuing the command ‘draftsight‘ from the command line.

DraftSight is a big advance on using an old version of VariCAD Viewer, and I have finally found a decent 2D CAD application to use in Linux.

The problem of scanning using USB multi-function printers in Linux (success at last)

After my investigations described in a couple of earlier posts ([1], [2]), I finally got Gentoo Linux to scan reliably via the USB connection to my Canon MP560 MFP (multi-function peripheral), a single USB device with three interfaces: scanner (Interface 0); printer (Interface 1); mass storage (Interface 2). Well, ninety-nine per cent reliably, if that isn’t a contradiction in terms. The formula for success? I had to do all the following:

  1. Stop the kernel from binding the usb-storage driver to the device.

  2. Create a UDEV rule to modify the ACL (access control list) of the device to give both the ‘lp‘ and ‘scanner‘ groups Read-Write access to the device.

  3. Create a UDEV rule to: a) change the owner of the device from ‘root‘ to my user account (fitzcarraldo); b) change the device’s group to ‘scanner‘ instead of ‘lp‘; c) give all users Read-Write access to the device (see my note about this at the end of this post).

  4. Disable the SANE backend from accessing the MP560 via the network.

Let’s look in more detail at each of these …

Stop the kernel from binding the usb-storage driver to the device

Originally I had built the usb-storage driver into the kernel (CONFIG_USB_STORAGE=y). During the course of my investigations into this problem I rebuilt the kernel with CONFIG_USB_STORAGE=m, i.e. I rebuilt the driver as an external module, although it should not matter either way.

The vendor ID (04a9) and product ID (173e) of the MP560 can be found using the lsusb command:

# lsusb | grep -i canon
Bus 001 Device 007: ID 04a9:173e Canon, Inc. MP560

If the usb-storage driver is built as an external module, the kernel configuration file will include the following:

# grep CONFIG_USB_STORAGE= /usr/src/linux/.config
CONFIG_USB_STORAGE=m

In this case, it is possible to stop the driver binding to the USB storage interface in the MP560 by creating a ‘quirk’ in the file /etc/modprobe.d/usb-storage.conf as shown below:

# cat /etc/modprobe.d/usb-storage.conf
options usb-storage quirks=04a9:173e:i

If the usb-storage driver is built into the kernel, the kernel configuration file will include the following:

# grep CONFIG_USB_STORAGE= /usr/src/linux/.config
CONFIG_USB_STORAGE=y

In this case, it is possible to stop the driver binding to the USB storage interface in the MP560 by adding a ‘quirk’ (usb-storage.quirks=<idVendor>:<idProduct>:i) to the kernel boot line as shown below (ignore the rest of the parameters shown below, as yours are likely to be different):

# grep usb-storage /boot/grub/grub.cfg
linux /vmlinuz-3.18.11-gentoo root=/dev/sda5 ro drm_kms_helper.edid_firmware=edid/1920x1080_Clevo_W230SS.bin i915.modeset=1 rcutree.rcu_idle_gp_delay=1 usb-storage.quirks=04a9:173e:i

If you edit the file /boot/grub/grub.cfg directly, the change will be lost if you rebuild or upgrade in future. The ‘proper’ way to add the quirk would be to add it to the list of kernel boot parameters in GRUB_CMDLINE_LINUX_DEFAULT in the file /etc/default/grub and regenerate grub.cfg using the command ‘grub2-mkconfig -o /boot/grub/grub.cfg‘.

Create a UDEV rules file to modify the device’s ACL

I created a UDEV rules file /etc/udev/rules.d/95-libsane.rules containing the following:

# ACL settings for Canon PIXMA MP560 printer MFP
ATTR{idVendor}=="04a9", ATTR{idProduct}=="173e", GOTO="canon"
GOTO="canon_end"
 
LABEL="canon"
RUN+="/bin/setfacl -m g:scanner:rw -m g:lp:rw $env{DEVNAME}"
TEST=="/var/run/ConsoleKit/database", \
        RUN+="udev-acl --action=$env{ACTION} --device=$env{DEVNAME}"
 
LABEL="canon_end"

Create a UDEV rules file to change the owner, group and permissions of the device

The file 41-libsane.rules in my installation is the only UDEV rules file in the directory /lib/udev/rules.d/ that explicitly mentions the MP560:

# grep 173e /lib/udev/rules.d/*
/lib/udev/rules.d/41-libsane.rules:ATTRS{idVendor}=="04a9", ATTRS{idProduct}=="173e", MODE="0664", GROUP="scanner", ENV{libsane_matched}="yes"

As the USB device was always in the ‘lp‘ group (see the output of the ‘ls -la‘ command [1]), the above UDEV rule was clearly being overridden by another rule, but which one in the long list of rules files created automatically when I installed the OS and various packages (see below)?

# ls -F1 /lib/udev/rules.d/
10-ft-rockey.rules
10-virtualbox.rules
40-gentoo.rules
40-usb-media-players.rules
41-libsane.rules
42-usb-hid-pm.rules
50-udev-default.rules
56-hpmud.rules
60-block.rules
60-cdrom_id.rules
60-drm.rules
60-evdev.rules
60-persistent-alsa.rules
60-persistent-input.rules
60-persistent-storage-tape.rules
60-persistent-storage.rules
60-persistent-v4l.rules
60-serial.rules
61-accelerometer.rules
64-btrfs.rules
69-cd-sensors.rules
70-libgphoto2.rules
70-mouse.rules
70-printers.rules
70-touchpad.rules
70-udev-acl.rules
75-net-description.rules
75-probe_mtd.rules
77-mm-cinterion-port-types.rules
77-mm-ericsson-mbm.rules
77-mm-huawei-net-port-types.rules
77-mm-longcheer-port-types.rules
77-mm-mtk-port-types.rules
77-mm-nokia-port-types.rules
77-mm-pcmcia-device-blacklist.rules
77-mm-platform-serial-whitelist.rules
77-mm-simtech-port-types.rules
77-mm-telit-port-types.rules
77-mm-usb-device-blacklist.rules
77-mm-usb-serial-adapters-greylist.rules
77-mm-x22x-port-types.rules
77-mm-zte-port-types.rules
77-nm-olpc-mesh.rules
78-sound-card.rules
80-drivers.rules
80-mm-candidate.rules
80-net-name-slot.rules
80-udisks2.rules
85-regulatory.rules
90-alsa-restore.rules
90-libgpod.rules
90-network.rules
90-pulseaudio.rules
95-cd-devices.rules
95-upower-battery-recall-dell.rules
95-upower-battery-recall-fujitsu.rules
95-upower-battery-recall-gateway.rules
95-upower-battery-recall-ibm.rules
95-upower-battery-recall-lenovo.rules
95-upower-battery-recall-toshiba.rules
95-upower-csr.rules
95-upower-hid.rules
95-upower-wup.rules
97-hid2hci.rules
99-fuse.rules
99-ntfs3g.rules
99-nvidia.rules

Anyway, I created a UDEV rules file /etc/udev/rules.d/99-canon-mp560.rules to change the owner of the MP560 USB device from ‘root‘ to me (i.e. user name ‘fitzcarraldo‘), to put the device in the ‘scanner‘ group instead of the ‘lp‘ group, and to give all users Read-Write permission for the device:

ATTRS{idVendor}=="04a9", ATTRS{idProduct}=="173e", MODE="0666", OWNER="fitzcarraldo", GROUP="scanner", ENV{libsane_matched}="yes"

So I now have two user-created UDEV rules files:

# ls /etc/udev/rules.d/
95-libsane.rules 99-canon-mp560.rules

Note that user-created UDEV rules files should be put in the directory /etc/udev/rules.d/, not the directory /lib/udev/rules.d/ which is reserved for UDEV rules installed when you install the OS and by packages you install later. Unfortunately, there is no consistency in UDEV rules file names and their contents between Linux distributions, which makes debugging UDEV rules even more difficult. Higher numbered rules files override lower-numbered rules files.

I then switched off the MP560 and switched it on again. The two UDEV rules files I created had worked as intended. The ownership and group of the device had changed, and so had the permissions and ACL:

# ls -la /dev/bus/usb/001
total 0
drwxr-xr-x  2 root         root       200 Jul 29 10:10 .
drwxr-xr-x  4 root         root        80 Jul 29 09:56 ..
crw-rw-r--  1 root         usb     189, 0 Jul 29 11:25 001
crw-rw-r--  1 root         usb     189, 1 Jul 29 09:56 002
crw-rw-r--  1 root         usb     189, 2 Jul 29 11:25 003
crw-rw-r--  1 root         usb     189, 3 Jul 29 09:56 004
crw-rw-r--  1 root         usb     189, 4 Jul 29 11:25 005
crw-rw-r--  1 root         usb     189, 5 Jul 29 11:25 006
crw-rw-r--  1 root         usb     189, 7 Jul 29 11:25 008
crw-rw-rw-+ 1 fitzcarraldo scanner 189, 8 Jul 29 11:27 009

# getfacl /dev/bus/usb/001/009
getfacl: Removing leading '/' from absolute path names
# file: dev/bus/usb/001/009
# owner: fitzcarraldo
# group: scanner
user::rw-
user:fitzcarraldo:rw-
group::rw-
group:lp:rw-
group:scanner:rw-
mask::rw-
other::rw-

Disable the SANE backend from accessing the MP560 via the network

Although scanning via the USB connection sometimes still works if I have also specified a network connection in the file /etc/sane.d/pixma.conf, apparently the two interfaces should not normally be enabled simultaneously. See the post [sane-devel] Pixma 530 series – tested and not working by SANE developer Rolf Bensch:

Please disconnect the scanner from network and try again. The backend
has problems if both network and USB are connected at once.

Therefore, to try using SANE to access the MP560 via USB, I disabled network access to the MP560 by commenting out the URI in the relevant SANE backend configuration file, which in the case of the MP560 is the file /etc/sane.d/pixma.conf. I commented out the line ‘bjnp://192.168.1.78‘ which I had previously inserted to enable me to scan via my home network because the USB connection was unreliable.

Trying to scan

I then checked if the USB-connected scanner is detected by SANE when running as root user and when running under my user account:

# scanimage -L
device `pixma:04A9173E_1653C4' is a CANON Canon PIXMA MP560 multi-function peripheral
# sane-find-scanner -q
found USB scanner (vendor=0x04a9 [Canon], product=0x173e [MP560 series]) at libusb:001:010
# exit
exit
$ scanimage -L
device `pixma:04A9173E_1653C4' is a CANON Canon PIXMA MP560 multi-function peripheral
$ sane-find-scanner -q
found USB scanner (vendor=0x04a9 [Canon], product=0x173e [MP560 series]) at libusb:001:010

The test using the scanimage command worked:

$ scanimage -d pixma:04A9173E_1653C4 -T
scanimage: scanning image of size 638x877 pixels at 24 bits/pixel
scanimage: acquiring RGB frame, 8 bits/sample
scanimage: reading one scanline, 1914 bytes...  PASS
scanimage: reading one byte...          PASS
scanimage: stepped read, 2 bytes...     PASS
scanimage: stepped read, 4 bytes...     PASS
scanimage: stepped read, 8 bytes...     PASS
scanimage: stepped read, 16 bytes...    PASS
scanimage: stepped read, 32 bytes...    PASS
scanimage: stepped read, 64 bytes...    PASS
scanimage: stepped read, 128 bytes...   PASS
scanimage: stepped read, 256 bytes...   PASS
scanimage: stepped read, 512 bytes...   PASS
scanimage: stepped read, 1024 bytes...  PASS
scanimage: stepped read, 2048 bytes...  PASS
scanimage: stepped read, 2047 bytes...  PASS
scanimage: stepped read, 1023 bytes...  PASS
scanimage: stepped read, 511 bytes...   PASS
scanimage: stepped read, 255 bytes...   PASS
scanimage: stepped read, 127 bytes...   PASS
scanimage: stepped read, 63 bytes...    PASS
scanimage: stepped read, 31 bytes...    PASS
scanimage: stepped read, 15 bytes...    PASS
scanimage: stepped read, 7 bytes...     PASS
scanimage: stepped read, 3 bytes...     PASS

So I tried to scan an A4 page from the command line:

$ scanimage -d pixma:04A9173E_1653C4 -l 0 -t 0 -x 215 -y 297 --resolution 150 --mode Color | convert - scanned-page.png

That worked, so I launched XSane, one of the GUI scanner applications:

$ xsane

That also worked.

Sometimes either scanimage or a GUI scanner application (XSane, gscan2pdf, Simple Scan or whatever) hangs, but, more often than not, scanning now works via the USB connection. However, if either scanimage or a GUI scanner application hangs, I kill them, run them again and then they usually work.

Importantly, printing still works, even though I have changed the USB device group from ‘lp‘ to ‘scanner‘.

So I have satisfied my curiosity and managed to get the MP560 to scan when connected to my laptop via USB, even though it already worked fine when using a network connection instead.

A note on the version of SANE

For information, I currently have Version 1.0.25_pre20150725 (daily snapshot from 25 July 2015) of the package sane-backends installed:

$ eix -I sane-backends
[I] media-gfx/sane-backends
     Available versions:  1.0.24-r5 (~)1.0.25_pre20150628 (~)1.0.25_pre20150725[1] {avahi doc gphoto2 ipv6 snmp systemd threads usb v4l xinetd ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32" SANE_BACKENDS="+abaton +agfafocus +apple +artec +artec_eplus48u +as6e +avision +bh +canon +canon630u +canon_dr -canon_pp +cardscan +coolscan +coolscan2 +coolscan3 +dc210 +dc240 +dc25 +dell1600n_net +dmc +epjitsu +epson +epson2 +fujitsu +genesys +gt68xx +hp +hp3500 +hp3900 +hp4200 +hp5400 +hp5590 +hpljm1005 -hpsj5s +hs2p +ibm +kodak +kodakaio +kvs1025 +kvs20xx kvs40xx +leo +lexmark +ma1509 +magicolor +matsushita +microtek +microtek2 +mustek -mustek_pp +mustek_usb mustek_usb2 +nec +net +niash +p5 +pie +pixma +plustek +plustek_pp -pnm +qcam +ricoh +rts8891 +s9036 +sceptre +sharp +sm3600 +sm3840 +snapscan +sp15c +st400 +stv680 +tamarack +teco1 +teco2 +teco3 +test +u12 +umax +umax1220u +umax_pp +xerox_mfp"}
     Installed versions:  1.0.25_pre20150725[1](01:39:24 25/07/15)(avahi gphoto2 ipv6 usb v4l -doc -snmp -systemd -threads -xinetd ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32" SANE_BACKENDS="abaton agfafocus apple artec artec_eplus48u as6e avision bh canon canon630u canon_dr cardscan coolscan coolscan2 coolscan3 dc210 dc240 dc25 dell1600n_net dmc epjitsu epson epson2 fujitsu genesys gt68xx hp hp3500 hp3900 hp4200 hp5400 hp5590 hpljm1005 hs2p ibm kodak kodakaio kvs1025 kvs20xx leo lexmark ma1509 magicolor matsushita microtek microtek2 mustek mustek_usb nec net niash p5 pie pixma plustek plustek_pp qcam ricoh rts8891 s9036 sceptre sharp sm3600 sm3840 snapscan sp15c st400 stv680 tamarack teco1 teco2 teco3 test u12 umax umax1220u umax_pp xerox_mfp -canon_pp -hpsj5s -kvs40xx -mustek_pp -mustek_usb2 -pnm")
     Homepage:            http://www.sane-project.org/
     Description:         Scanner Access Now Easy - Backends

[1] "local_overlay" /usr/local/portage

However, the same symptoms occurred with versions 1.0.24-r5, 1.0.25_pre20150628 and 1.0.25_pre20150725 of the sane-backends package, so I believe the problem was not version-specific.

A note about scanning in Linux
An early developer’s Web site for the SANE pixma backend has the following warning:

If you grant a group a full permission to access the scanner (by using udev, hotplug, resmgr etc.), every members in the group will have full access rights not only for the scanner part but also for the other parts of the device i.e. printer, memory-card reader and fax. These users can bypass your system’s security policy and do everything with the device intentionally and unintentionally.

It is all very well to say that, but it is the only way I could get the backend to work reliably (almost). And, judging by the number of people who posted on the Web that they had to do the same with other Linux distributions, other manufacturers and models of MFPs, this is a common requirement in order to get Linux to work with MFP scanners. It’s not impressive when people have to resort to this to get something to work.

Note that the success of the approach described in this post is dependent on the contents and order of all the UDEV rules files. Just because the rules files I have posted above work in my case does not guarantee they will work in your case too. It depends what rules files your installation already has, what are their precise contents and in what order they are. In my opinion this is not a good concept.

I stand by my comments in the Conclusion of my first post regarding this problem. To have to jump through so many hoops to be able to scan using Linux on a computer connected to a MFP via USB is ridiculous in my opinion. The fact that it works reliably via a network connection does not alter that. I have been ‘scratching an itch’ as I enjoy tinkering, but I feel sorry for users who just need something that works without having to mess around for hours or days.

Background

  1. The problem of scanning using USB multi-function printers in Linux
  2. The problem of scanning using USB multi-function printers in Linux (continued)

How to group the entries in the right-click menu of a KDE file manager

This is easier to explain by giving an example. I will use PeaZip as my example, but the principle applies whatever the menu entries.

Immediately after installing PeaZip Portable on my main laptop running Gentoo Linux by using the steps in one of my earlier posts, the resulting right-click menu in KDE Dolphin looks like the screenshot below. Notice that the PeaZip entries in the Actions sub-menu are not listed contiguously.

Actions sub-menu of right-click menu in KDE

But in KDE it is relatively easy to group menu entries tidily in a sub-menu, as shown in the screenshot below.

Sub-menu of Actions sub-menu for right-click menu in KDE

The peazip*.desktop files for the five PeaZip actions shown in the above screenshots are located in the directory ~/.kde4/share/kde4/services/ServiceMenus/ and each file includes the line ‘X-KDE-Submenu='.

All I had to do was edit each of the five peazip*.desktop files and change that line in each file to ‘X-KDE-Submenu=PeaZip‘.

That’s it!

The problem of scanning using USB multi-function printers in Linux (continued)

My previous post discussed the inability of the Linux installation on my main laptop to use the scanner of my Canon PIXMA MP560 MFP (multi-function peripheral) when they are connected via USB. Linux sees the MP560 as a single USB device with multiple interfaces: Interface 0 (scanner), Interface 1 (printer) and Interface 2 (USB mass storage). The basic problem seems to be that the usb-storage driver — whether built into the kernel or as an external module — claims the USB device and blocks access by userspace’s libusb, which is what the SANE backend (‘pixma‘, in the case of the MP560) uses to access the USB device. However, scanning works fine when using a network connection instead of the USB connection.

Attempts such as unbinding the usb-storage driver from the USB device — either manually or using a UDEV rule — in order to try and allow libusb/SANE to access the scanner, did not solve the problem. Although I can scan by connecting to the MP560 via my home network, the inability of the OS to scan via the USB interface piqued my interest, so I continued my investigations and below I discuss my latest findings.

Disabling kernel auto-binding

I learned from the interesting blog post Controlling USB device access on Linux how to stop the kernel automatically binding a driver to a USB device in the first place, as distinct from unbinding a driver after the kernel has bound it to a device. I thought this approach could be used to stop the usb-storage driver from binding to the MP560 and thus allow libusb/SANE to access Interface 0. The method using sysfs to stop the kernel binding the driver is to disable kernel USB auto-probing before launching a scanner application, by using the following command:

# echo 0 > /sys/bus/usb/drivers_autoprobe

This did indeed stop the kernel from automatically binding USB drivers to USB devices, but unfortunately libusb/SANE would still not access the MP560 scanner via USB. When drivers_autoprobe contains zero, applications such as XSane can use the network-connected MP560 as before, but still hang if the USB-connected MP560 is selected.

Unloading the usb-storage module

According to various blog and forum posts, some people have been able to use MFP scanners via USB after manually unloading the usb-storage module. I decided to try this. As I had built the kernel with the usb-storage driver internally rather than an external module, I first had to rebuild the kernel with CONFIG_USB_STORAGE=m instead of CONFIG_USB_STORAGE=y. Then I unloaded manually the usb_storage module using the command ‘rmmod usb_storage‘ and launched a scanner application, but the application still hung.

Disabling the usb-storage driver using a ‘quirk’

Then I had a thought: Perhaps there is a kernel ‘quirk’ to disable the usb-storage driver from binding to a specific USB device. Indeed there is (see Kernel Parameters or /usr/src/linux-<version>/Documentation/kernel-parameters.txt).

If the usb-storage driver is built into the kernel (CONFIG_USB_STORAGE=y), I would need to add the following kernel boot parameter to the kernel boot line:

usb-storage.quirks=04a9:173e:i

’04a9′ is the Vendor ID and ‘173e’ is the Product ID of the MP560. The ‘i‘ stands for ‘ignore’.

If the usb-storage driver is built as a kernel module (CONFIG_USB_STORAGE=m), I would need to create a file with a name such as /etc/modprobe.d/usb-storage.conf containing the following line:

options usb-storage quirks=04a9:173e:i

So I set up the quirk. Upon rebooting my laptop, the dmesg command reported the following:

usb 1-1.2.2: new high-speed USB device number 8 using ehci-pci
usb 1-1.2.2: New USB device found, idVendor=04a9, idProduct=173e
usb 1-1.2.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-1.2.2: Product: MP560 series
usb 1-1.2.2: Manufacturer: Canon
usb 1-1.2.2: SerialNumber: 1653C4
usb-storage 1-1.2.2:1.2: USB Mass Storage device detected
usb-storage 1-1.2.2:1.2: device ignored

Notice the final line. The quirk does indeed cause the usb-storage driver to ignore the MP560.

The lsusb command still shows the device, which is a good sign:

# lsusb
Bus 002 Device 005: ID 0bc2:3300 Seagate RSS LLC
Bus 002 Device 004: ID 0411:01d9 BUFFALO INC. (formerly MelCo., Inc.)
Bus 002 Device 003: ID 05e3:0606 Genesys Logic, Inc. USB 2.0 Hub / D-Link DUB-H4 USB 2.0 Hub
Bus 002 Device 002: ID 8087:8000 Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 006: ID 5986:055c Acer, Inc
Bus 001 Device 004: ID 8087:07dc Intel Corp.
Bus 001 Device 007: ID 0603:00f2 Novatek Microelectronics Corp. Keyboard (Labtec Ultra Flat Keyboard)
Bus 001 Device 008: ID 04a9:173e Canon, Inc. MP560
Bus 001 Device 005: ID 045e:00d1 Microsoft Corp. Optical Mouse with Tilt Wheel
Bus 001 Device 003: ID 05e3:0606 Genesys Logic, Inc. USB 2.0 Hub / D-Link DUB-H4 USB 2.0 Hub
Bus 001 Device 002: ID 8087:8008 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

# lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
        |__ Port 2: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 1: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 480M
            |__ Port 3: Dev 5, If 0, Class=Mass Storage, Driver=usb-storage, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M
        |__ Port 2: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 1: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
            |__ Port 2: Dev 8, If 0, Class=Vendor Specific Class, Driver=usbfs, 480M
            |__ Port 2: Dev 8, If 1, Class=Printer, Driver=, 480M
            |__ Port 2: Dev 8, If 2, Class=Mass Storage, Driver=, 480M
            |__ Port 4: Dev 7, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
            |__ Port 4: Dev 7, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
        |__ Port 3: Dev 4, If 0, Class=Wireless, Driver=btusb, 12M
        |__ Port 3: Dev 4, If 1, Class=Wireless, Driver=btusb, 12M
        |__ Port 4: Dev 6, If 0, Class=Video, Driver=uvcvideo, 480M
        |__ Port 4: Dev 6, If 1, Class=Video, Driver=uvcvideo, 480M

The output of the lsusb -v -d 04a9:173e command applicable to the MP560 is as follows:

Bus 001 Device 008: ID 04a9:173e Canon, Inc. MP560
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x04a9 Canon, Inc.
  idProduct          0x173e MP560
  bcdDevice            0.04
  iManufacturer           1 Canon
  iProduct                2 MP560 series
  iSerial                 3 1653C4
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           85
    bNumInterfaces          3
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower                2mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           3
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol    255 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x07  EP 7 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x88  EP 8 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x89  EP 9 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval              11
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         7 Printer
      bInterfaceSubClass      1 Printer
      bInterfaceProtocol      2 Bidirectional
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x84  EP 4 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x05  EP 5 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      1
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0001
  Self Powered

The USB device created at boot was as follows:

# ls -la /dev/bus/usb/001/008
crw-rw-r--+ 1 root lp 189, 7 Jul 24 11:01 /dev/bus/usb/001/008

Whether run by the root user or under my user account, the SANE tools detected the MP560 scanner network and USB connections:

$ scanimage -L
device `pixma:MP560_192.168.1.78' is a CANON Canon PIXMA MP560 multi-function peripheral
device `pixma:04A9173E' is a CANON Canon PIXMA MP560 multi-function peripheral

$ sane-find-scanner

  # sane-find-scanner will now attempt to detect your scanner. If the
  # result is different from what you expected, first make sure your
  # scanner is powered up and properly connected to your computer.

  # No SCSI scanners found. If you expected something different, make sure that
  # you have loaded a kernel SCSI driver for your SCSI adapter.

found USB scanner (vendor=0x04a9 [Canon], product=0x173e [MP560 series]) at libusb:001:008
  # Your USB scanner was (probably) detected. It may or may not be supported by
  # SANE. Try scanimage -L and read the backend's manpage.

  # Not checking for parallel port scanners.

  # Most Scanners connected to the parallel port or other proprietary ports
  # can't be detected by this program.

But scanner applications still hung! This time the message displayed by the dmesg command was:

usb 1-1.2.2: usbfs: interface 0 claimed by usbfs while 'scanimage' sets config #1

I unplugged the USB cable and plugged it in again. The output of the dmesg command was:

usb 1-1.2.2: USB disconnect, device number 8
usb 1-1.2.2: new high-speed USB device number 9 using ehci-pci
usb 1-1.2.2: New USB device found, idVendor=04a9, idProduct=173e
usb 1-1.2.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-1.2.2: Product: MP560 series
usb 1-1.2.2: Manufacturer: Canon
usb 1-1.2.2: SerialNumber: 1653C4
usb-storage 1-1.2.2:1.2: USB Mass Storage device detected
usb-storage 1-1.2.2:1.2: device ignored

I then launched XSane several times and each time I was able to select either the MP560 via my home network (pixma:MP560_192.168.1.78) or via the USB cable (pixma:04A9173E_1653C4) and scan successfully.

By the way, it is possible to scan via the command line instead of using a GUI scanner application. For example, tjhe following command will scan an A4 page via the MP560’s USB connection (see man scanimage for details of the options):

$ scanimage -d pixma:04A9173E_1653C4 -l 0 -t 0 -x 215 -y 297 --resolution 150 --mode Color | convert - scanned-page.png

So the problem is solved, right? Wrong! For whatever reason, scanning via the USB interface is not guaranteed to work every time I boot the laptop. Sometimes scanner applications (and the scanimage command) hang if I select the USB interface; at other times they don’t hang and scanning works fine. If scanning using the USB device is not working, if I unplug and re-insert the USB cable a few times it becomes possible to scan using the USB interface. And once scanning via the USB interface does work, from then onwards it works consistently during that session.

I had been using Version 1.0.24-r5 of the sane-backends package in Gentoo Linux, which is the latest stable version according to the Gentoo package manager. To check if that particular version of the SANE pixma backend might be the cause of the problem I installed the latest version available in the Gentoo package manager (currently 1.0.5_pre20150625). However it made no discernable difference.

To sum up, in my case the most successful approach so far has been to use a kernel ‘quirk’ to force the usb-storage driver to ignore the device and not bind to it. However, this approach does not result in libusb/SANE being able to access the scanner during every boot session, but, if access is successful, it remains successful during the current session. Even if access via the USB interface does not work initially, sometimes I can get it to work by unplugging and re-inserting the USB cable.

UPDATE (July 30, 2015): I have now managed to get scanning to work with almost 100% reliability when connected to the MP560 via USB — see my latest post for details.

The problem of scanning using USB multi-function printers in Linux

Whilst searching the Web I have found many other people experiencing this problem, which I believe happens as a consequence of the way Linux is designed. Basically, if you have a multi-function peripheral that you connect to your computer via USB, the chances are that either scanning or printing will not work. The Web is littered with blog posts, forum threads, mailing list posts and bug reports regarding this phenomenon. There are numerous suggested fixes, usually entailing the addition or modification of udev rules. Sometimes the suggested fixes work; often they don’t. Linux can be a major hassle where peripherals are concerned.

I have a Canon Pixma MP560 MFP (multi-function printer/peripheral), which is a printer, scanner and copier. It also has memory card slots and a USB pen drive socket, to be able to store images directly from the scanner without the need to connect a computer. The MP560 provides two alternative means of connection to a computer: USB and Wi-Fi. I normally connect it via USB to my Clevo W230SS laptop running Gentoo Linux (amd64), but I can also connect it to the laptop via my home Wi-Fi network. Either connection method allows me to print, but I can scan only via the wireless connection.

The dmesg command displays the following messages after I plug a USB cable from the MP560 into my laptop:

usb 1-1.2.2: new high-speed USB device number 8 using ehci-pci
usb 1-1.2.2: New USB device found, idVendor=04a9, idProduct=173e
usb 1-1.2.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-1.2.2: Product: MP560 series
usb 1-1.2.2: Manufacturer: Canon
usb 1-1.2.2: SerialNumber: 1653C4
usb-storage 1-1.2.2:1.2: USB Mass Storage device detected
scsi host6: usb-storage 1-1.2.2:1.2
scsi 6:0:0:0: Direct-Access     Canon    MP560 series     0104 PQ: 0 ANSI: 2
sd 6:0:0:0: Attached scsi generic sg1 type 0
sd 6:0:0:0: [sdb] Attached SCSI removable disk

Notice that the device also has some internal storage.

The lsusb command shows the device:

$ lsusb
Bus 002 Device 002: ID 8087:8000 Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 006: ID 5986:055c Acer, Inc
Bus 001 Device 004: ID 8087:07dc Intel Corp.
Bus 001 Device 007: ID 0603:00f2 Novatek Microelectronics Corp. Keyboard (Labtec Ultra Flat Keyboard)
Bus 001 Device 009: ID 04a9:173e Canon, Inc. MP560
Bus 001 Device 005: ID 045e:00d1 Microsoft Corp. Optical Mouse with Tilt Wheel
Bus 001 Device 003: ID 05e3:0606 Genesys Logic, Inc. USB 2.0 Hub / D-Link DUB-H4 USB 2.0 Hub
Bus 001 Device 002: ID 8087:8008 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

My Linux user account is in the lp and scanner groups:

$ groups
disk lp wheel floppy uucp cron audio cdrom video games cdrw usb users plugdev crontab wireshark vboxusers scanner fitzcarraldo

The device is in the lp group and I can print without any trouble:

$ ls -la /dev/bus/usb/001/009
crw-rw-r--+ 1 root lp 189, 8 Jun 19 02:55 /dev/bus/usb/001/009

The usblp driver, a common cause of problems with USB printers, does not exist because I purposely did not set CONFIG_USB_PRINTER when I built the kernel, specifically to avoid such problems:

$ grep CONFIG_USB_PRINTER /usr/src/linux/.config
# CONFIG_USB_PRINTER is not set

so, obviously, the usblp driver is neither built into the kernel nor compiled as a module:

$ lsmod | grep usb
btusb                  22292  0
bluetooth             281605  27 bnep,hidp,btusb,rfcomm

I have the latest stable versions of the udev-related packages installed (as you can see below, I use eudev instead of udev):

$ eix -I udev
[I] dev-python/pyudev
     Available versions:  0.16.1-r1 {pygobject pyqt4 pyside test PYTHON_TARGETS="python2_7 python3_3 python3_4"}
     Installed versions:  0.16.1-r1(03:32:17 18/04/15)(pyqt4 -pygobject -pyside -test PYTHON_TARGETS="python2_7 python3_3 -python3_4")
     Homepage:            http://pyudev.readthedocs.org https://github.com/pyudev/pyudev
     Description:         Python binding to libudev

[I] sys-fs/eudev
     Available versions:  *1.3 *1.5.3-r1 1.9-r2 1.10-r2 ~2.1.1 ~3.0 3.1.2 ~3.1.2-r10 **9999 {doc gudev (+)hwdb introspection (+)keymap (+)kmod +modutils +openrc +rule-generator selinux static-libs test ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  3.1.2(19:44:12 16/07/15)(gudev hwdb introspection kmod -doc -selinux -static-libs -test ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            https://github.com/gentoo/eudev
     Description:         Linux dynamic and persistent device naming support (aka userspace devfs)

[I] sys-fs/udev-init-scripts
     Available versions:  27^t [M]~29^t ~30^t **9999^t
     Installed versions:  27^t(05:29:06 09/04/15)
     Homepage:            http://www.gentoo.org
     Description:         udev startup scripts for openrc

[I] virtual/libgudev
     Available versions:  215-r1(0/0) 215-r2(0/0) 215-r3(0/0) ~230(0/0) {introspection static-libs systemd ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  215-r3(10:18:16 11/06/15)(introspection -static-libs -systemd ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="64 -32 -x32")
     Description:         Virtual for libgudev providers

[I] virtual/libudev
     Available versions:  215-r1(0/1) {static-libs systemd ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  215-r1(17:51:42 18/04/15)(-static-libs -systemd ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Description:         Virtual for libudev providers

[I] virtual/udev
     Available versions:  215 ~217 {systemd}
     Installed versions:  215(05:28:55 09/04/15)(-systemd)
     Description:         Virtual to select between different udev daemon providers

Found 6 matches.

I have the latest stable SANE-related packages installed:

$ eix -I sane
[I] dev-perl/Sane
     Available versions:  (~)0.50.0-r1 {test}
     Installed versions:  0.50.0-r1(17:39:10 23/06/15)(-test)
     Homepage:            http://search.cpan.org/dist/Sane/
     Description:         The Sane module allows you to access SANE-compatible scanners in a Perl

[I] kde-apps/ksaneplugin
     Available versions:  (4) 4.14.3(4/4.14)
       {aqua debug}
     Installed versions:  4.14.3(4)(14:13:01 17/04/15)(-aqua -debug)
     Homepage:            http://www.kde.org/
     Description:         SANE Plugin for KDE

[I] kde-apps/libksane
     Available versions:  (4) 4.14.3(4/4.14)
       {aqua debug}
     Installed versions:  4.14.3(4)(13:55:44 17/04/15)(-aqua -debug)
     Homepage:            http://www.kde.org/
     Description:         SANE Library interface for KDE

[I] media-gfx/sane-backends
     Available versions:  1.0.24-r5 ~1.0.25_pre20150628 {avahi doc gphoto2 ipv6 snmp systemd threads usb v4l xinetd ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32" SANE_BACKENDS="+abaton +agfafocus +apple +artec +artec_eplus48u +as6e +avision +bh +canon +canon630u +canon_dr -canon_pp +cardscan +coolscan +coolscan2 +coolscan3 +dc210 +dc240 +dc25 +dell1600n_net +dmc +epjitsu +epson +epson2 +fujitsu +genesys +gt68xx +hp +hp3500 +hp3900 +hp4200 +hp5400 +hp5590 +hpljm1005 -hpsj5s +hs2p +ibm +kodak +kodakaio +kvs1025 +kvs20xx kvs40xx +leo +lexmark +ma1509 +magicolor +matsushita +microtek +microtek2 +mustek -mustek_pp +mustek_usb mustek_usb2 +nec +net +niash +p5 +pie +pixma +plustek +plustek_pp -pnm +qcam +ricoh +rts8891 +s9036 +sceptre +sharp +sm3600 +sm3840 +snapscan +sp15c +st400 +stv680 +tamarack +teco1 +teco2 +teco3 +test +u12 +umax +umax1220u +umax_pp +xerox_mfp"}
     Installed versions:  1.0.24-r5(17:55:11 18/04/15)(avahi gphoto2 ipv6 usb v4l -doc -snmp -systemd -threads -xinetd ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32" SANE_BACKENDS="abaton agfafocus apple artec artec_eplus48u as6e avision bh canon canon630u canon_dr cardscan coolscan coolscan2 coolscan3 dc210 dc240 dc25 dell1600n_net dmc epjitsu epson epson2 fujitsu genesys gt68xx hp hp3500 hp3900 hp4200 hp5400 hp5590 hpljm1005 hs2p ibm kodak kodakaio kvs1025 kvs20xx leo lexmark ma1509 magicolor matsushita microtek microtek2 mustek mustek_usb nec net niash p5 pie pixma plustek plustek_pp qcam ricoh rts8891 s9036 sceptre sharp sm3600 sm3840 snapscan sp15c st400 stv680 tamarack teco1 teco2 teco3 test u12 umax umax1220u umax_pp xerox_mfp -canon_pp -hpsj5s -kvs40xx -mustek_pp -mustek_usb2 -pnm")
     Homepage:            http://www.sane-project.org/
     Description:         Scanner Access Now Easy - Backends

[I] media-gfx/sane-frontends
     Available versions:  1.0.14 {gimp}
     Installed versions:  1.0.14(19:47:50 18/06/15)(gimp)
     Homepage:            http://www.sane-project.org
     Description:         Scanner Access Now Easy

[I] media-gfx/xsane
     Available versions:  0.999-r1 {gimp jpeg lcms nls ocr png tiff}
     Installed versions:  0.999-r1(12:03:12 19/04/15)(gimp jpeg lcms nls png tiff -ocr)
     Homepage:            http://www.xsane.org/
     Description:         graphical scanning frontend

Found 6 matches.

The following SANE backends were installed when I installed SANE:

$ ls -F1 /etc/sane.d/ | columns -c 3 -w 20
abaton.conf         agfafocus.conf      apple.conf
artec.conf          artec_eplus48u.conf avision.conf
bh.conf             canon.conf          canon630u.conf
canon_dr.conf       cardscan.conf       coolscan.conf
coolscan2.conf      coolscan3.conf      dc210.conf
dc240.conf          dc25.conf           dell1600n_net.conf
dll.conf            dll.d/              dmc.conf
epjitsu.conf        epson.conf          epson2.conf
fujitsu.conf        genesys.conf        gphoto2.conf
gt68xx.conf         hp.conf             hp3900.conf
hp4200.conf         hp5400.conf         hs2p.conf
ibm.conf            kodak.conf          kodakaio.conf
leo.conf            lexmark.conf        ma1509.conf
magicolor.conf      matsushita.conf     microtek.conf
microtek2.conf      mustek.conf         mustek_usb.conf
nec.conf            net.conf            p5.conf
pie.conf            pixma.conf          plustek.conf
plustek_pp.conf     qcam.conf           ricoh.conf
rts8891.conf        s9036.conf          saned.conf
sceptre.conf        sharp.conf          sm3840.conf
smfp.conf           snapscan.conf       sp15c.conf
st400.conf          stv680.conf         tamarack.conf
teco1.conf          teco2.conf          teco3.conf
test.conf           u12.conf            umax.conf
umax1220u.conf      umax_pp.conf        v4l.conf
xerox_mfp.conf

I checked all the SANE backend configuration files, and none of them mention the MP560 (I was not expecting them to mention it, as, according to man sane-pixma, the pixma backend detects Canon PIXMA USB devices automatically):

$ grep -rnwi /etc/sane.d/ -e "173e"
$

The SANE file dll.conf does list the pixma backend configuration file which is only for declaring networked Canon PIXMA devices, as the pixma backend detects Canon USB devices automatically):

$ grep pixma /etc/sane.d/dll.conf
pixma

$ cat /etc/sane.d/pixma.conf
# pixma.conf configuration for the sane pixma backend
#
# define URI's of scanners (one per line)
# This is only used for network scanners.
# normally scanners will be detected by sending a broadcast
# if this does not work under your OS, or if the scanners
# are on a different subnet, configure your scanners URI here
#
# method must be bjnp
# port number can normally be left out, port 8612 is used as default
# Example:
# bjnp://myscanner.my.domain:8612
# bjnp://printer-1.pheasant.org
#

I don’t think it is even used these days, but the file /etc/hotplug/usb/libsane.usermap already includes the device:

# Canon PIXMA MP560
libusbscanner 0x0003 0x04a9 0x173e 0x0000 0x0000 0x00 0x00 0x00 0x00 0x00 0x00 0x00000000

The file /lib64/udev/rules.d/41-libsane.rules already includes the device:

# Canon PIXMA MP560
ATTRS{idVendor}=="04a9", ATTRS{idProduct}=="173e", MODE="0664", GROUP="scanner", ENV{libsane_matched}="yes"

However, scanning from Linux via USB is a troubled affair. Very occasionally the scanner application (XSane, Simple Scan, gscan2pdf, or whatever) will launch and the scanner works, but usually the scanner application just hangs and the dmesg command shows one or more error messages such as the following:

usb 1-1.2.2: usbfs: interface 2 claimed by usb-storage while 'xsane' sets config #1

usb 1-1.2.2: usbfs: interface 2 claimed by usb-storage while 'gscan2pdf' sets config #1

I have USB mass storage support enabled in the kernel so that my laptop can access external USB mass storage devices:

$ grep CONFIG_USB_STORAGE= /usr/src/linux/.config
CONFIG_USB_STORAGE=y

As the three interfaces (printer, scanner and internal storage) are all accessed as a single USB device (/dev/bus/usb/001/009, or whatever), there appears to be a conflict between the kernel claiming the MFP’s internal mass storage and SANE trying to access the scanner. I do not want to disable the kernel’s USB mass storage support (not even by rebuilding the kernel with CONFIG_USB_STORAGE=m and temporarily unloading manually the usb-storage module, as some people suggest), as I would not be able to access my external USB mass storage devices.

The scanimage and sane-find-scanner utilities detect the scanner correctly via USB:

$ scanimage -L
device `v4l:/dev/video0' is a Noname BisonCam, NB Pro virtual device
device `pixma:04A9173E_1653C4' is a CANON Canon PIXMA MP560 multi-function peripheral

(The BisonCam is the Webcam built into my laptop.)

$ sane-find-scanner

  # sane-find-scanner will now attempt to detect your scanner. If the
  # result is different from what you expected, first make sure your
  # scanner is powered up and properly connected to your computer.

  # No SCSI scanners found. If you expected something different, make sure that
  # you have loaded a kernel SCSI driver for your SCSI adapter.

found USB scanner (vendor=0x04a9 [Canon], product=0x173e [MP560 series]) at libusb:001:009
  # Your USB scanner was (probably) detected. It may or may not be supported by
  # SANE. Try scanimage -L and read the backend's manpage.

  # Not checking for parallel port scanners.

  # Most Scanners connected to the parallel port or other proprietary ports
  # can't be detected by this program.

  # You may want to run this program as root to find all devices. Once you
  # found the scanner devices, be sure to adjust access permissions as
  # necessary.

The scanimage test commands do not indicate any problem:

$ scanimage -d test -T
scanimage: scanning image of size 157x196 pixels at 8 bits/pixel
scanimage: acquiring gray frame, 8 bits/sample
scanimage: reading one scanline, 157 bytes...   PASS
scanimage: reading one byte...          PASS
scanimage: stepped read, 2 bytes...     PASS
scanimage: stepped read, 4 bytes...     PASS
scanimage: stepped read, 8 bytes...     PASS
scanimage: stepped read, 16 bytes...    PASS
scanimage: stepped read, 32 bytes...    PASS
scanimage: stepped read, 64 bytes...    PASS
scanimage: stepped read, 128 bytes...   PASS
scanimage: stepped read, 256 bytes...   PASS
scanimage: stepped read, 255 bytes...   PASS
scanimage: stepped read, 127 bytes...   PASS
scanimage: stepped read, 63 bytes...    PASS
scanimage: stepped read, 31 bytes...    PASS
scanimage: stepped read, 15 bytes...    PASS
scanimage: stepped read, 7 bytes...     PASS
scanimage: stepped read, 3 bytes...     PASS

$ scanimage -d pixma -T
scanimage: scanning image of size 638x877 pixels at 24 bits/pixel
scanimage: acquiring RGB frame, 8 bits/sample
scanimage: reading one scanline, 1914 bytes...  PASS
scanimage: reading one byte...          PASS
scanimage: stepped read, 2 bytes...     PASS
scanimage: stepped read, 4 bytes...     PASS
scanimage: stepped read, 8 bytes...     PASS
scanimage: stepped read, 16 bytes...    PASS
scanimage: stepped read, 32 bytes...    PASS
scanimage: stepped read, 64 bytes...    PASS
scanimage: stepped read, 128 bytes...   PASS
scanimage: stepped read, 256 bytes...   PASS
scanimage: stepped read, 512 bytes...   PASS
scanimage: stepped read, 1024 bytes...  PASS
scanimage: stepped read, 2048 bytes...  PASS
scanimage: stepped read, 2047 bytes...  PASS
scanimage: stepped read, 1023 bytes...  PASS
scanimage: stepped read, 511 bytes...   PASS
scanimage: stepped read, 255 bytes...   PASS
scanimage: stepped read, 127 bytes...   PASS
scanimage: stepped read, 63 bytes...    PASS
scanimage: stepped read, 31 bytes...    PASS
scanimage: stepped read, 15 bytes...    PASS
scanimage: stepped read, 7 bytes...     PASS
scanimage: stepped read, 3 bytes...     PASS

As mentioned earlier, the udev/eudev rules file appears to contain a valid rule for the scanner:

# Canon PIXMA MP560
ATTRS{idVendor}=="04a9", ATTRS{idProduct}=="173e", MODE="0664", GROUP="scanner", ENV{libsane_matched}="yes"

Notice the rule assigns the device’s group to be ‘scanner‘, not ‘lp‘, although changing the group assignment to ‘lp‘ in the rule makes no discernable difference to the scanning problem in my case. As the device’s group is actually ‘lp‘, I assume another rule overrides this rule, but I have no idea which rule it is and in which rules file it occurs.

Occasionally I am lucky and scanning does work, but, more often than not, scanner applications cannot access the scanner and just hang.

A work-around if the peripheral also has a network interface

I configured my home network router always to use the IP address 192.168.1.78 for the MP560 and I added the following line to the file /etc/sane.d/pixma.conf in order to be able to scan via Wi-Fi instead of USB (I have the package cups-bjnp installed for printing purposes, but I am not sure if that package is also needed by SANE in this case):

bjnp://192.168.1.78

With the above line added to /etc/sane.d/pixma.conf the scanimage utility detects the scanner correctly via both the Wi-Fi connection and the USB interface:

$ scanimage -L
device `v4l:/dev/video0' is a Noname BisonCam, NB Pro virtual device
device `pixma:MP560_192.168.1.78' is a CANON Canon PIXMA MP560 multi-function peripheral
device `pixma:04A9173E_1653C4' is a CANON Canon PIXMA MP560 multi-function peripheral

So, as I usually cannot get scanning to work via USB, I can scan via Wi-Fi instead. It’s a work-around, not a solution, but at least by doing that I can scan reliably as well as print. Of course you can’t use such a work-around if you have a model of MFP that only provides a USB connection.

Failed experiments

Experiment 1

I also tried unbinding the usb-storage driver from the usb device:

# ls /sys/bus/usb/drivers
btusb hub usb usb-storage usbfs usbhid uvcvideo
# ls /sys/bus/usb/drivers/btusb/
1-1.3:1.0 1-1.3:1.1 bind module new_id remove_id uevent unbind
# ls /sys/bus/usb/drivers/hub
1-0:1.0 1-1.2:1.0 1-1:1.0 2-0:1.0 2-1:1.0 bind module new_id remove_id uevent unbind
# ls /sys/bus/usb/drivers/usb
1-1 1-1.2 1-1.2.1 1-1.2.2 1-1.2.4 1-1.3 1-1.4 2-1 bind uevent unbind usb1 usb2
# ls /sys/bus/usb/drivers/usb-storage/
1-1.2.2:1.2 bind module new_id remove_id uevent unbind
# ls /sys/bus/usb/drivers/usbfs
bind module new_id remove_id uevent unbind
# ls /sys/bus/usb/drivers/usbhid
1-1.2.1:1.0 1-1.2.4:1.0 1-1.2.4:1.1 bind module new_id remove_id uevent unbind
# ls /sys/bus/usb/drivers/uvcvideo
1-1.4:1.0 1-1.4:1.1 bind module new_id remove_id uevent unbind
# tree /sys/bus/usb/drivers/usb-storage/
/sys/bus/usb/drivers/usb-storage/
├── 1-1.2.2:1.2 -> ../../../../devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/1-1.2.2/1-1.2.2:1.2
├── bind
├── module -> ../../../../module/usb_storage
├── new_id
├── remove_id
├── uevent
└── unbind

2 directories, 5 files
# echo -n "1-1.2.2:1.2" > /sys/bus/usb/drivers/usb-storage/unbind
# tree /sys/bus/usb/drivers/usb-storage/
/sys/bus/usb/drivers/usb-storage/
├── bind
├── module -> ../../../../module/usb_storage
├── new_id
├── remove_id
├── uevent
└── unbind

1 directory, 5 files
# tree /sys/bus/usb/devices/1-1.2.2:1.2
/sys/bus/usb/devices/1-1.2.2:1.2
├── bAlternateSetting
├── bInterfaceClass
├── bInterfaceNumber
├── bInterfaceProtocol
├── bInterfaceSubClass
├── bNumEndpoints
├── ep_05
│   ├── bEndpointAddress
│   ├── bInterval
│   ├── bLength
│   ├── bmAttributes
│   ├── direction
│   ├── interval
│   ├── power
│   ├── type
│   ├── uevent
│   └── wMaxPacketSize
├── ep_84
│   ├── bEndpointAddress
│   ├── bInterval
│   ├── bLength
│   ├── bmAttributes
│   ├── direction
│   ├── interval
│   ├── power
│   ├── type
│   ├── uevent
│   └── wMaxPacketSize
├── modalias
├── power
├── subsystem -> ../../../../../../../../bus/usb
├── supports_autosuspend
└── uevent

6 directories, 27 files

After unbinding the usb-storage driver from the USB device, no more messages such as ‘usbfs: interface 2 claimed by usb-storage while 'xsane' sets config #1‘ are added to /var/log/messages when I launch a scanner application and select the USB device, but the scanner application still hangs (e.g. the XSane windows do not appear until I switch off the MP560 or unplug the USB cable). This happens irrespective of whether the USB device permissions are:

# ls -la /dev/bus/usb/001/009
crw-rw-r--+ 1 root lp 189, 8 Jul 20 20:35 /dev/bus/usb/001/009

or:

# ls -la /dev/bus/usb/001/009
crw-rw-r--+ 1 fitzcarraldo lp 189, 8 Jul 20 20:35 /dev/bus/usb/001/009

or:

# ls -la /dev/bus/usb/001/009
crw-rw-r--+ 1 fitzcarraldo scanner 189, 8 Jul 20 20:51 /dev/bus/usb/001/009

To reiterate, neither unbinding the usb-storage driver from the USB device first nor changing the owner and group of the USB device solves the problem. So possibly the problem is not caused by udev or by a USB driver, but by the SANE software.

Experiment 2

I found a thread in the SANE developers’ mailing list about the same problem with a different model of MFP: Help needed diagnosing strange failure to scan with Samsung SCX-4500W. The same user posted the same query in the Arch Linux forums: [SOLVED] Samsung SCX4500W MFP can’t get scanner to work. Like my Canon PIXMA MP560, that MFP has more than one interface in a single USB device. Notice in the Arch Linux thread that the user never found a solution and finally adopted the same work-around I use, namely to scan via a network interface rather than via the USB interface.

Theodore Kilgore gives a good explanation in that SANE developers’ mailing list thread of why the problem occurs in Linux, and Thorsten Müller’s post suggests a udev rule that sets the permissions for the MFP so that SANE, CUPS and logged-in users can all access the USB device. All I had to do was replace references to Samsung (such as the vendor ID and product ID) with those for my Canon PIXMA MP560. I left everything else in my installation exactly the way it was (for example I did not touch /lib64/udev/rules.d/41-libsane.rules and /lib64/udev/rules.d/70-printers.rules). The only thing I did was to add a udev rules file /etc/udev/rules.d/99-canon-mp560.rules with the following contents:

# ACL settings for Canon PIXMA MP560 printer MFP
ATTR{idVendor}=="04a9", ATTR{idProduct}=="173e", GOTO="canon"
GOTO="canon_end"

LABEL="canon"
RUN+="/bin/setfacl -m g:scanner:rw -m g:lp:rw $env{DEVNAME}"
TEST=="/var/run/ConsoleKit/database", \
        RUN+="udev-acl --action=$env{ACTION} --device=$env{DEVNAME}"

LABEL="canon_end"

Both /bin/setfacl and /var/run/ConsoleKit/database exist in my installation, and, as shown earlier, my user account is a member of both the lp and scanner groups. After adding the above file, I rebooted and it appears the rule works as intended:

# getfacl /dev/bus/usb/001/009
getfacl: Removing leading '/' from absolute path names
# file: dev/bus/usb/001/009
# owner: root
# group: lp
user::rw-
user:fitzcarraldo:rw-
group::rw-
group:lp:rw-
group:scanner:rw-
mask::rw-
other::r--

However, although I was still able to print I still could not scan. Mind you, the line beginning ‘TEST==‘ in the file /etc/udev/rules.d/99-canon-mp560.rules is also in the file /lib64/udev/rules.d/70-udev-acl.rules, but I’m no expert in udev rules, which are frankly beyond the ability of most users.

Experiment 3

I found a blog post about the same problem but with a Canon MP210 MFP: Canon multifunction printer: getting the printer drivers and scanner to work in debian/ubuntu/mint. This post suggested creating a udev rule file /etc/udev/rules.d/40-scanner-permissions.rules containing the following:

# usb scanner
SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", MODE:="0666"
SUBSYSTEM=="usb_device",MODE:="0666"

but, although the rule appears to work as intended, it is still not possible to scan in my case. I then changed the file name to /etc/udev/rules.d/95-scanner-permissions.rules, but that made no difference in my case.

Experiment 4

I also tried changing MODE="0664" to MODE="0666" in the following line in the file /lib64/udev/rules.d/50-udev-default.rules but that made no difference either:

SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", MODE="0664"

Experiment 5

I am able to change the group of the USB device to ‘scanner‘ instead of ‘lp‘ by adding a rules file /lib64/udev/rules.d/99-canon.rules (apparently 41-libsane.rules does not work as intended because it is overruled by a higher-numbered file, but I have no idea which one, as 41-libsane.rules is the only one explicitly mentioning the MP560) containing the following:

ATTRS{idVendor}=="04a9", ATTRS{idProduct}=="173e", MODE="0664", GROUP="scanner", ENV{libsane_matched}="yes"

But scanning still does not work, whether or not I unbind the usb-storage driver from the USB device first and whether or not I change the owner and group of the USB device to root:lp, root:scanner, fitzcarraldo:lp or fitzcarraldo:scanner.

Conclusion

Although my installation can print via USB, scanning via USB does not work, whether or not the usb-storage driver is bound to the USB device and whoever is the owner and group of the USB device. So it could be that the problem is not caused by udev or by the kernel (a USB driver), but by the SANE software itself. However, I think it is more likely to be some sort of contention between kernel and userspace, despite unbinding the usb-storage driver from the device. Although I’m not excluding the possibility of a bug in SANE, irrespective of that it seems to me that the way GNU/Linux handles multiple interfaces of a single USB device is flawed. The udev design results in a morass of confusing and potentially conflicting rules in umpteen files in different directories, and the apparently uncoordinated independent demands on the same USB interface by the kernel and userspace are another potential source of problems. There is no standardisation in the udev rules files and their contents between the different Linux distributions, making investigation and solution of such problems even more complicated. Having to resort to work-arounds such as scanning via a network interface or scanning to a USB pen drive or memory card is a poor way of circumventing the problem, and is of no help anyway if a multi-function peripheral does not have a network interface, pen drive socket or memory card slot.

UPDATE (July 24, 2015): I have tried some other approaches, and had more success using a ‘quirk’ for the usb-storage driver — see my next post for details.

Addendum: How to stop scanning applications detecting a Webcam

By the way, to stop XSane and other scanning applications detecting my Webcam via the Video for Linux SANE backend, I commented out all the device lines in the file /etc/sane.d/v4l.conf like so:

$ cat
# In order to use the v4linux backend you have to give the device
# You can enable multiple lines if
# you really have multible v4l devices.
#
#/dev/bttv0
#/dev/video0
#/dev/video1
#/dev/video2
#/dev/video3

Making the X Windows cursor theme the same for KDM and KDE

For a long time it irritated me that the X Windows cursor theme on the KDM log-in screen differed to the X Windows cursor theme on the KDE Desktop. The former was usually the old-fashioned core X Windows cursor theme (or perhaps the ‘KDE Classic’ theme, I’m not sure which), whereas the latter is the theme I selected via ‘System Settings’ > ‘Workspace Appearance’ > ‘Cursor Theme’. To confuse me further, when I upgraded X Windows recently the X Windows cursor theme on the KDM log-in screen was Adwaita when I next booted my laptop, but susequently reverted to the classic cursor theme.

Anyway, I had to do the following in order to make the KDM cursor theme the same as the KDE cursor theme:

1. Create a directory /usr/share/icons/default if it does not already exist (it did not in my case):

# mkdir /usr/share/icons/default

2. Check which X Windows cursor themes are currently installed:

# ls /usr/share/icons
Adwaita HighContrast Humanity KDE_Classic Oxygen_Black Oxygen_Blue Oxygen_White Oxygen_Yellow Oxygen_Zion default gnome hicolor locolor mono nuvola oxygen ubuntu-mono-dark ubuntu-mono-light

I also find the three X Windows cursor themes ‘handhelds’, ‘redglass’ and ‘whiteglass’, installed when I installed the package x11-themes/xcursor-themes using the Portage package manager, in a different directory:

# ls /usr/share/cursors/xorg-x11/
Adwaita handhelds redglass whiteglass

The ‘Adwaita’ cursor theme was already in /usr/share/cursors/xorg-x11/ before I installed the package x11-themes/xcursor-themes, and also in the directory /usr/share/icons/ but I do not know why only that specific cursor theme is in both directories.

3. Create a file /usr/share/icons/default/index.theme with the following contents (I opted to use the Oxygen_White cursor theme, but you can choose whichever you want from the list of installed cursor themes):

[Icon Theme]
Name = Oxygen_White
Comment = Default icon theme
Inherits = Oxygen_White

4. Make sure ‘System Settings’ > ‘Worskspace Appearance’ > ‘Cursor Theme’ has the theme selected that you want for the KDE Desktop (I opted to use the Oxygen_White cursor theme).

For example, if I had wanted the cursor theme to be Adwaita, I would have selected Adwaita in KDE using ‘System Settings’ > ‘Worskspace Appearance’ > ‘Cursor Theme’ and then I would have edited /usr/share/icons/default/index.theme to contain the following:

[Icon Theme]
Name = Adwaita
Comment = Default icon theme
Inherits = Adwaita

Easy when you know how.

According to the Arch Linux Wiki, for user-specific configuration you should create or edit the file ~/.icons/default/index.theme, whereas for system-wide configuration you should create or edit the file /usr/share/icons/default/index.theme but the latter file is owned by libXcursor and user changes to it will be overwritten on update. However, in Gentoo Linux it would be possible to get around that by creating a script file in the directory /etc/local.d/ to revert the file change. For example, I could make the file /usr/share/icons/default/index.theme contain the following:

[Icon Theme]
Name = Oxygen_White
Comment = Default icon theme
Inherits = Oxygen_White

Then copy that file to somewhere safe that will not be overwritten:

# cp /usr/share/icons/default/index.theme /home/fitzcarraldo/

Then create a script file named, say, 80-xcursor.start in /etc/local.d/ with the following contents:

#!/bin/bash
# Make sure X windows cursor theme on the KDM screen is the one I want:
cp /home/fitzcarraldo/index.theme /usr/share/icons/default/index.theme

and make the script file executable:

# chmod +x /etc/local.d/80-xcursor.start

Then, if something does overwrite or delete /usr/share/icons/default/index.theme in future, the script in /etc/local.d/ will restore it before the KDM log-in screen appears, so you would always see the cursor theme specified in /home/fitzcarraldo/index.theme.

Switching between Intel and NVIDIA graphics processors on a laptop with NVIDIA Optimus hardware running Gentoo Linux

I have a Clevo W230SS laptop with an Intel HD 4600 IGP and an NVIDIA GeForce GTX 860M GPU, running KDE in Gentoo Linux. I could have installed Bumblebee in order to use the NVIDIA GPU with selected applications, but I decided instead to switch manually between the graphics processors. To achieve this I created two Desktop Configuration Files with nice icons on my Desktop, to launch two simple Bash scripts which configure my installation for the desired graphics processor, which will then be used when I next login to KDE. All the necessary files are listed below for Gentoo Linux. You would need to modify them if you are using a different distribution.

The Bash script ~/intel.sh contains the following:

#!/bin/bash
echo
echo "Your installation is currently configured to use the following graphics processor:"
echo
GPU=`eselect opengl list | grep \* | awk '{ print $2 }'`
if [ "$GPU" = "xorg-x11" ]; then
  echo "Intel HD 4600 Integrated Graphics Processor"
  echo
  echo "You do not need to do anything. Please close this window."
elif [ "$GPU" = "nvidia" ]; then
  echo "NVIDIA GeForce GTX 860M"
  echo
  echo "This script will configure your installation to use the"
  echo "Intel HD 4600 Integrated Graphics Controller all the time."
  echo
  echo "Enter your own password."
  echo
  sudo eselect opengl set xorg-x11
  sudo cp /home/fitzcarraldo/Xsetup.intel /usr/share/config/kdm/Xsetup
  sudo cp /home/fitzcarraldo/xorg.conf.intel /etc/X11/xorg.conf
  echo
  echo "Now you should logout to restart X Windows."
fi

The Bash script ~/nvidia.sh contains the following:

#!/bin/bash
echo
echo "Your installation is currently configured to use the following graphics processor:"
echo
GPU=`eselect opengl list | grep \* | awk '{ print $2 }'`
if [ "$GPU" = "nvidia" ]; then
  echo "NVIDIA GeForce GTX 860M"
  echo
  echo "You do not need to do anything. Please close this window."
elif [ "$GPU" = "xorg-x11" ]; then
  echo "Intel HD 4600 Integrated Graphics Processor"
  echo
  echo "This script will configure your installation to use the NVIDIA GeForce GTX 860M GPU all the time."
  echo
  echo "Enter your own password."
  echo
  sudo eselect opengl set nvidia
  sudo cp /home/fitzcarraldo/Xsetup.nvidia /usr/share/config/kdm/Xsetup
  sudo cp /home/fitzcarraldo/xorg.conf.nvidia /etc/X11/xorg.conf
  echo
  echo "Now you should logout to restart X Windows."
fi

I created two files, xorg.conf.intel and xorg.conf.nvidia, in my home directory. The corresponding Bash script copies the applicable file to the directory /etc/X11/ to create an xorg.conf file with the correct contents for the graphics processor.

The file ~/xorg.conf.intel contains the following:

Section "Device" 
   Identifier  "Intel Graphics" 
   Driver      "intel" 
   Option      "AccelMethod" "sna" 
   Option      "TearFree" "true" 
EndSection

The file ~/xorg.conf.nvidia contains the following:

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "nvidia" 0 0
    Inactive       "intel"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "nvidia"
    Driver         "nvidia"
    BusID          "PCI:1:0:0"
EndSection

Section "Device"
    Identifier     "intel"
    Driver         "modesetting"
    BusID          "PCI:0:2:0"
EndSection

Section "Screen"
    Identifier     "nvidia"
    Device         "nvidia"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "UseDisplayDevice" "none"
    SubSection     "Display"
        Depth       24
#        Modes      "nvidia-auto-select"
        Virtual     1920 1080
    EndSubSection
EndSection

Section "Screen"
    Identifier     "intel"
    Device         "intel"
    Monitor        "Monitor0"
EndSection

I created two files, Xsetup.intel and Xsetup.nvidia, in my home directory. The corresponding Bash script copies the applicable file to the directory /usr/share/config/kdm/ to create an Xsetup file with the correct contents for the graphics processor.

The file ~/Xsetup.intel is listed below. As you can see, everything is commented out in the file.

#! /bin/sh
# Xsetup - run as root before the login dialog appears

#xconsole -geometry 480x130-0-0 -notify -verbose -fn fixed -exitOnFail -file /dev/xconsole &

The file Xsetup.nvidia is listed below. As you can see, it contains two xrandr commands.

#! /bin/sh
# Xsetup - run as root before the login dialog appears

#xconsole -geometry 480x130-0-0 -notify -verbose -fn fixed -exitOnFail -file /dev/xconsole &
xrandr --setprovideroutputsource modesetting NVIDIA-0
xrandr --auto

I downloaded nice icons from the Web for the Desktop Configuration Files and put them in the directory ~/Pictures/Icons/.

The file ~/Desktop/Select\ Intel\ HD\ Graphics.desktop is listed below.

[Desktop Entry]
Comment[en_GB]=Run a script to configure your installation to use Intel Integrated Graphics when you restart X Windows
Comment=Run a script to configure your installation to use Intel Integrated Graphics when you restart X Windows
Exec=/home/fitzcarraldo/intel.sh
GenericName[en_GB]=Configure your installation to use Intel HD Graphics
GenericName=Configure your installation to use Intel HD Graphics
Icon=/home/fitzcarraldo/Pictures/Icons/intel_hd_graphics_icon.png
MimeType=
Name[en_GB]=Intel HD Graphics
Name=Intel HD Graphics
Path=
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

The file ~/Desktop/Select\ NVIDIA\ GPU.desktop is listed below.

[Desktop Entry]
Comment[en_GB]=Run a script to configure your installation to use the NVIDIA GeForce GTX 860M GPU when you restart X Windows
Comment=Run a script to configure your installation to use the NVIDIA GeForce GTX 860M GPU when you restart X Windows
Exec=/home/fitzcarraldo/nvidia.sh
GenericName[en_GB]=Configure your installation to use the NVIDIA GeForce GTX 860M GPU
GenericName=Configure your installation to use the NVIDIA GeForce GTX 860M GPU
Icon=/home/fitzcarraldo/Pictures/Icons/nvidia_icon.png
MimeType=
Name[en_GB]=NVIDIA GPU
Name=NVIDIA GPU
Path=
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

Don’t forget to make all the Bash script files and .desktop files executable. All I have to do is double-click on either icon on the Desktop and a Konsole window pops up and tells me what to do. Next time I log in, the graphics processor I selected will be active.

My thanks go to Gentoo Linux user arthanis for showing me the way in the Gentoo Forums thread [SOLVED] Optimus and Nvidia.

Computer memories

Please forgive the double entendre in the title but, for my one-hundredth post, I thought I would reminisce about the calculating and computing equipment I have used over the years, and the amazing changes there have been in my lifetime.

My late father became interested in computers in the 1960s for use in his work, and I recall him showing me magnetic core memory and explaining how it worked. His use of programmable calculators fascinated and inspired me in my teens. He had the vision and kindness to give me two early models of ‘pocket calculator’ (the HP-45 and the programmable TI-59) and an Apple II+ which he bought second-hand and brought half-way around the world to me, along with lots of early commercial software on 5.25″ floppy disks and a bundle of magazines full of Applesoft BASIC programs. These all helped me greatly during my studies and early professional life, and I will always be extremely grateful.

Anyway, in roughly chronological order, as best I can remember them …

A Multo 113 mechanical pinwheel calculator at home (my father’s, which he bought in the 1950s and which was still in use at home well into my teens).

The Olivetti Programma 101 (my father’s, for work). This was a desktop programmable calculator weighing nearly 30 kg! Actually I never used it myself, but I mention it here for interest as it was, I believe, the World’s first desktop programmable calculator. Mind you, there was not much space left on a desk for anything else when this was on it!

A slide rule (I still have my CONCISE circular slide rule) at high school. When I sat my university entrance exams we were allowed to use logarithm tables and slide rules but not one of the recently-invented ‘pocket calculators’.

An HP-45 scientific calculator in my final year at high school. Anyone remember Reverse Polish Notation?

A TI-59 programmable calculator (it had an attachable base station with a thermal printer) at university. An excellent piece of equipment that could store and retrieve programs on magnetic cards, and also had ROM modules with pre-canned programs (‘solid state software library’), although the magnetic card reader was rather unreliable. I remember the August 1978 issue of Personal Computer World magazine had an article on flowcharting using the game of NIM as an example, and I was engrossed for a couple of weeks in my summer holiday shoehorning it into the available memory of the TI-59, a real exercise in program optimisation. But possibly my most satisfying achievement was programming it during an exam to solve a problem by using the Newton-Raphson method while my fellow students resorted to recursion tables on paper.

The GEORGE 3 operating system at university, running on an ICL 1906. I initially used punched cards, but quickly moved to a 110-baud Teletype Model 33 terminal (known as a ‘TTY’ for short) with those cylindrical keys with long travel that you had to press hard and went “kerrrrchunkkk”! I also used 300-baud DEC dot-matrix printing terminals (the LA36 model, if I remember correctly).

I can’t remember the name of the OS on the CDC 7600 supercomputer at university, but I ran a lot of large FORTRAN 66 and FORTRAN 77 jobs, delivered to it as batch jobs via the ICL 1906.

Commodore BASIC on a Commodore PET 8k at work.

The RT-11 and RSX-11M operating systems on a DEC PDP-11 at work, on which I programmed in BASIC.

Applesoft BASIC, Apple DOS 3.2 & 3.3 on an Apple II+ (48k RAM, later expanded to 64k!) and, later, on an Apple //e. I programmed in Applesoft BASIC, 6502 assembler and Apple Pascal, and some years later even built an A-to-D converter card and a 240 VAC TRIAC switch card for the Apple II+ in order to control the central heating in my house.

I had VisiCalc (5.25″ floppy disk), the very first spreadsheet application, invented by two guys named Dan Bricklin and Bob Frankston. VisiCalc was the father of all spreadsheets: Lotus 1-2-3, MultiPlan, Quattro Pro and Excel. I actually used VisiCalc at work. I also had the first generation of SubLOGIC’s Flight Simulator for the Apple II, the predecessor of Microsoft Flight Simulator for the PC. It used line vector graphics, redrawn at a grindingly slow rate by today’s standards, but it was good fun.

CP/M on an Apple //e with a Microsoft Z-80 SoftCard and also the less popular Digital Research Z80 card. I programmed in Digital Research CBASIC and used various applications such as WordStar and MultiPlan.

ProDOS on an Apple //c with a whopping 128k RAM. This was my father’s machine for work, bought on my recommendation, and I used to help him sometimes with programming in BASIC.

Sharp PC-1500 pocket computer. This was another of my father’s work tools, also bought on my recommendation.

MP/M on a RAIR Black Box at work. I did some 8080 assembly language programming on it but mainly acted as a system admin (several VT100-style terminals around the office were connected to the RAIR Black Box). Also some Z80 assembly language programming on a microboard for work.

OSC245 on Ferranti Argus computers at work, programming in the real-time language CORAL 66.

VMS on a DEC VAX at work, programming (badly) in Pascal. VAX/VMS was an excellent OS.

MSDOS 3.2 on an Amstrad PC1512, for BBS (bulletin board system) access, one or two games (the first Leisure Suit Larry was fun), WordStar and WordPerfect. When WordPerfect 5.1 for DOS was released I thought it was the height of sophistication.

RISC OS 2 on an Acorn Archimedes A3000. What a brilliant microcomputer and OS. I just had to have one as soon as I saw the 3D vector graphics lander demo of the game Zarch. I also ran the Acorn PC Emulator software so that I could use MSDOS to run applications such as WordPerfect 5.1 and Quattro Pro for DOS, an excellent spreadsheet that introduced tabbed worksheets earlier than Microsoft Excel.

MSDOS 4.x, 5.x and 6.x on an IBM PC and various clones such as the ACT Apricot at work.

INTERACTIVE Unix (Intel PC, a computer manufactured by Intel having an 80386 CPU) at work, programming shell scripts, SQL procedures in Sybase SQL Server, and IEC 1131-3 (now known as 61131-3) languages.

IBM OS/2 1.3 and 2.0 on DEC PCs at work. A really nice OS, and streets ahead of the contemporary Windows 3.1.

MSDOS 3.22 on an HP 95LX palmtop. Even its RS-232 interface was useful for hardware interface testing at work, for BBS access using my Pace MicroLin fx modem, and for printing to my HP Deskjet 310 portable printer. It had built-in Lotus 1-2-3 in ROM.

Windows 3.11 on a PC clone at work, mainly for wordprocessing using Word.

Windows 95 on a tower PC and laptops. An excellent OS for its time, and streets ahead of its predecessor Windows 3.11.

Windows Me on a laptop and my family’s tower PC. This OS had a bad reputation but I never had any trouble with it on my Gateway Solo 9300 laptop and the tower PC, both of which were heavily used.

Windows NT on a desktop PC at work.

Windows 2000 on a desktop PC at work.

Windows XP on a laptop and my family’s tower PC.

At this point Linux came into the picture in my case and slowly became my OS of choice, Ubuntu 6.06 in October 2006 being replaced in March 2007 by Sabayon Linux 3.2 as my main distribution, with PCLinuxOS 2007 as a set-and-forget choice for old laptops and PCs. In 2010 I switched from Sabayon Linux to Gentoo Linux for my main distribution, although I still use Sabayon Linux on old laptops and an HTPC.

Windows Vista on my family’s tower PC (Intel Pentium Dual Core E2160 CPU). After a few years the performance of Vista became so slow, despite periodic deframenting, that I ditched Vista.

Windows 7, dual-booting with Gentoo Linux on a laptop. I hardly ever booted Windows 7, and it was only left on the laptop in case I had a work application that could not run using WINE in Linux. I dislike Windows 7, although I found it faster and more robust than Vista, which I also disliked. Mind you, the UI of Windows 8.1 on my family’s current tower PC (Intel Pentium Dual Core G2030 3.00 GHz CPU) is even worse; what an awful OS.

Other microcomputers I’ve used through the years: Grundy NewBrain (my father’s, bought for work purposes on my recommendation) at home, and a BBC Micro at work, programming in BASIC.

I remember my first modem was a 300-baud and 1200/75-baud (remember Prestel, anyone?) Minor Miracles WS-2000, and I programmed my Apple II+ to pulse-dial it so that I could access various BBSs and British Telecom’s Telecom Gold, an ASCII electronic mail service and means of sending telexes (which I sent several times). In later years I progressed to a British Telecom 9600-baud modem for BBS use with my Acorn Archimedes A3000, and a Pace MicroLin fx pocket modem for use with my HP 95LX and Acorn Archimedes.

So, in my lifetime, calculating tools for home use have changed from a mechanical pinwheel calculator to the 2.8 GHz Core i7 laptop with 16 GB RAM running Gentoo Linux that I’m typing this on now. Amazing.

Follow

Get every new post delivered to your Inbox.

Join 61 other followers