Replacing the KDE Plasma widget ‘Thermal Monitor’ with ‘Kargos’ in Gentoo Linux

The KDE Plasma widget Thermal Monitor has not been working correctly in my Gentoo Linux installations for quite some time. I notice Thermal Monitor’s repository has not been updated for a couple of years, despite several new versions of KDE Plasma having been released. Perhaps that is the reason.

On my laptop running the Stable Branch of Gentoo Linux, Thermal Monitor displays the GPU and HDD temperatures automatically but CPU temperatures were only displayed if I right-clicked on the widget and selected ‘Reload Temperature Sources’. I managed to get the widget to display the CPU temperatures automatically by editing the file ~/.local/share/plasma/plasmoids/org.kde.thermalMonitor/contents/ui/main.qml and commenting out a line as shown in the file excerpt below:

[...]
        onSourceAdded: {

            if (source.indexOf(lmSensorsStart) === 0 || source.indexOf(acpiStart) === 0) {
/*
 *                systemmonitorAvailableSources.push(source)
 */
                var staIndex = systemmonitorSourcesToAdd.indexOf(source)
                if (staIndex > -1) {
                    addToSourcesOfDatasource(systemmonitorDS, source)
                    systemmonitorSourcesToAdd.splice(staIndex, 1)
                }

            }

        }
[...]

The above modification is suggested in a comment to Issue #53 in the widget’s repository.

However, the above-mentioned edit does not fix Thermal Monitor on my laptop running the Testing Branch of Gentoo Linux, and Thermal Monitor no longer displays the GPU temperature either. Actually, the CPU’s four core temperatures and the GPU temperature are no longer listed in the Thermal Monitor configuration window, only a single CPU temperature. Not surprisingly, none of the suggested changes to the file ~/.local/share/plasma/plasmoids/org.kde.thermalMonitor/contents/ui/main.qml that I found in Web searches made a difference. However, while researching the problem I came across a Manjaro Forums post by user bogdancovaciu about the Kargos Plasma widget, a KDE Plasma port of GNOME Argos and OSX BitBar. Kargos enables you to create a Plasma widget that runs your own script, which can be written in any language, providing its output adheres to a specified format. I also found a repository named k-argos-plugins containing further example scripts for Kargos. As none of the solutions suggested for Thermal Monitor in that Manjaro thread worked for me, I decided to try the Kargos widget instead. It works a treat.

kargos widget on KDE Plasma Panel

kargos widget on KDE Plasma Panel of my Compal NBLB2 laptop

Below I explain what I did to install and configure the Kargos widget on my KDE Panel in Gentoo Linux (see screenshot). The packages lm-sensors and hddtemp were already installed in my case, but if they had not been, I would have needed to install and configure them, so I have included those steps below.

1. Install and configure lm-sensors

root # emerge lm-sensors
root # rc-update add lm_sensors default
root # sensors-detect

In my case sensors-detect created the file /etc/modules-load.d/lm_sensors.conf containing only the following:

# Generated by sensors-detect on Sun Oct 27 03:07:08 2019
coretemp

2. Start lm-sensors now, rather than rebooting

root # /etc/init.d/lm_sensors start

3. I wanted to use the nc command in my shell script for Kargos, so I installed its package

root # emerge netcat

4. Install and configure hddtemp

root # emerge hddtemp
root # rc-update add hddtemp default

Specify in the config file /etc/conf.d/hddtemp which drives to check:

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

# the hddtemp executable
HDDTEMP_EXEC=/usr/sbin/hddtemp

# various options to pass to the daemon
HDDTEMP_OPTS="--listen=127.0.0.1"

# a list of drives to check
HDDTEMP_DRIVES="/dev/sda"

5. Start hddtemp now, rather than rebooting

root # /etc/init.d/hddtemp start

6. Install Kargos

On the KDE Plasma Desktop, click on the ‘Desktop’ menu icon (the three horizontal lines in the top right corner of the Desktop) and select: ‘Unlock Widgets’ > ‘Add Widgets…’ > ‘Get New Widgets…’ > ‘Download New Plasma Widgets’. Search for, and install, ‘kargos’ widget.

7. Create the Bash script ~/temperatures.3s.sh containing the following:

#!/bin/bash
temp=$(sensors | grep -oP 'Core.*?\+\K[0-9.]+')
temp0=$(sensors | grep 'Core 0' | cut -c '16-17')
temp1=$(sensors | grep 'Core 1' | cut -c '16-17')
temp2=$(sensors | grep 'Core 2' | cut -c '16-17')
temp3=$(sensors | grep 'Core 3' | cut -c '16-17')
hdd_temp=$(nc localhost 7634 | cut -c '33-34')
gpu_temp=$(sensors | grep -A 2 'radeon' | grep 'temp1' | cut -c '16-17')
echo "<br><font size='1'>CPU1&nbsp;&nbsp;CPU2&nbsp;&nbsp;CPU3&nbsp;&nbsp;CPU4&nbsp;&nbsp;GPU&nbsp;&nbsp;HDD</font><br>${temp0%%.*}°&nbsp;&nbsp;${temp1%%.*}°&nbsp;&nbsp;${temp2%%.*}°&nbsp;&nbsp;${temp3%%.*}°&nbsp;${gpu_temp}°&nbsp;${hdd_temp}°| font=Hack-Regular size=10"
# Uncomment the lines below if you want to be able to click on the kargos widget and display a pop-up TOP
#echo "---"
#TOP_OUTPUT=$(top -b -n 1 | head -n 20 | awk 1 ORS="\\\\n")
#echo "$TOP_OUTPUT | font=monospace iconName=htop"

The script above is specifically for the temperature sensors in my Clevo NBLB2 laptop. To find out which temperatures are available, and which characters to extract, use the following command:

root # sensors

Don’t forget to make the script executable:

user $ chmod +x ~/temperatures.3s.sh

Note that the ‘.3s‘ in the script name is optional but, if included, will override the kargos configuration (see further on) and run the script every 3 seconds. I could have specified another frequency, such as ‘.5s‘ or whatever.

8. Add the kargos widget to the KDE Panel.

9. Right-click on the kargos widget on the KDE Panel and select ‘Configure kargos…’.

10. Configure the kargos widget

In the first box in the configuration window, enter the full path of the script:

/home/fitzcarraldo/temperatures.3s.sh

In the second box leave ‘Interval in seconds’ as ‘1‘. This is overridden anyway if the script filename includes the interval.

In the third box leave ‘Rotation delay in seconds’ as ‘6‘.

On the KDE Plasma Desktop, click on the Desktop menu icon (three horizontal lines) and select: ‘Lock Widgets’.

11. Depending on the font configuration for the KDE Desktop, it may be necessary to edit the Bash script ~/temperatures.3s.sh to change the font name or size, the number of non-breaking spaces between the names displayed on the top line, and the number of non-breaking spaces between the temperature values displayed on the bottom line.

How to enable a Windows application in WINE to access a Samba share on a NAS (continued)

In a 2016 post ‘How to enable a Windows application in WINE to access a Samba share on a NAS‘ I explained how to mount in Linux a networked SMB shared folder so that a Windows application running via WINE could access the folder as Drive Y: in order to open and save files in it. In that blog post I also listed a couple of Bash scripts to facilitate the mounting and unmounting of the SMB share for the WINEPREFIX used for the Windows application (~/.wine-pdfxve6 in the example I gave for PDF-XChange Editor, Version 6). However, as I have several Windows applications running via WINE on my machines, and I have used a different WINEPREFIX for each of them, I wanted to be able to mount the SMB share for whichever of those applications I happen to be using at the time. Therefore I modified the original Bash scripts as shown below. The Desktop Configuration files (.desktop files) to launch the scripts are essentially the same as in my earlier blog post; I have just removed the references to the specific Windows application. The four modified files are listed below. Obviously change the username, SMB share name and SMB server name to suit your own situation.

1. Bash script ~/mount_bsfnas1_brianfolder_share.sh

#!/bin/bash
mount_share () {
    echo
    echo "Enter your Linux account password below..."
    echo
    sudo ln -s /media/bsfnas1/brianfolder ~/$PREFIX/dosdevices/y:
    sudo mount.cifs //bsfnas1/brianfolder/ -o user=brianfolder,pass=enricocaruso,uid=$(id -u),gid=$(id -g) ~/$PREFIX/dosdevices/y:
}
echo
echo "This will mount the Samba share folder brianfolder on the bsfnas1 machine."
echo
echo
echo "== Select which WINEPREFIX you wish to use =="
echo
ls ~/.wine-* | grep .wine | awk -F'/' '{print NR " " substr($4, 1, length($4)-1)}'
NUMPREFIXES=$(ls ~/.wine-* | grep .wine | wc -l)
echo
read -p "Enter number (q to abort) and press ENTER: " CHOICE
if [ "$CHOICE" != "q" ] && [ "$CHOICE" -gt 0 ] && [ "$CHOICE" -le $NUMPREFIXES ]; then
    PREFIX=$(ls ~/.wine-* | grep .wine | awk -F'/' '{print NR " " substr($4, 1, length($4)-1)}' | grep "$CHOICE " | awk -F' ' '{print $2}')
    echo
    if [ ! -e ~/$PREFIX/dosdevices/y: ]; then
        mount_share
    else
        echo -n "~/$PREFIX/dosdevices/y: already exists. Is it OK to proceed anyway (y/n)? "
        read ANSWER
        if [ $ANSWER = "y" ]; then
            rm ~/$PREFIX/dosdevices/y:
            mount_share
        fi
    fi
    echo
fi
if grep -q "/media/bsfnas1/brianfolder" /proc/mounts; then
    echo "Samba share //bsfnas1/brianfolder is mounted for WINEPREFIX ~/$PREFIX ."
else
    echo "Samba share //bsfnas1/brianfolder is not mounted."
fi
echo
echo "You may now close this window."
read ANSWER
exit

2. Bash script ~/umount_bsfnas1_brianfolder_share.sh

#!/bin/bash
echo
echo "This will unmount the Samba share folder brianfolder on the bsfnas1 machine."
echo
echo "Enter your Linux account password below..."
echo
sudo umount ~/.wine-*/dosdevices/y: 2>/dev/null
echo
if grep -q "/media/bsfnas1/brianfolder" /proc/mounts; then
  echo "Samba share //bsfnas1/brianfolder is mounted."
else
  echo "Samba share //bsfnas1/brianfolder is not mounted."
fi
echo
echo "You may now close this window."
exit

3. Desktop Configuration file ~/Desktop/mount_bsfnas1_brianfolder_share.desktop

[Desktop Entry]
Comment[en_GB]=Mount bsfnas1 brianfolder share for current WINEPREFIX
Comment=Mount bsfnas1 brianfolder share for current WINEPREFIX
Exec=sh /home/fitzcarraldo/mount_bsfnas1_brianfolder_share.sh
GenericName[en_GB]=Mount bsfnas1 brianfolder share for current WINEPREFIX
GenericName=Mount bsfnas1 brianfolder share for current WINEPREFIX
Icon=media-mount
MimeType=
Name[en_GB]=mount_bsfnas1_brianfolder_share
Name=mount_bsfnas1_brianfolder_share
Path=
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

4. Desktop Configuration file ~/Desktop/umount_bsfnas1_brianfolder_share.desktop

[Desktop Entry]
Comment[en_GB]=Unmount bsfnas1 brianfolder share for current WINEPREFIX
Comment=Unmount bsfnas1 brianfolder share for current WINEPREFIX
Exec=sh /home/fitzcarraldo/umount_bsfnas1_brianfolder_share.sh
GenericName[en_GB]=Unmount bsfnas1 brianfolder share for current WINEPREFIX
GenericName=Unmount bsfnas1 brianfolder share for current WINEPREFIX
Icon=media-eject
MimeType=
Name[en_GB]=umount_bsfnas1_brianfolder_share
Name=umount_bsfnas1_brianfolder_share
Path=
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

Now when I double-click on the icon to mount the SMB share for a Windows application running via WINE, a terminal window pops up displaying the WINEPREFIXs currently installed on my machine:


This will mount the Samba share folder brianfolder on the bsfnas1 machine.


== Select which WINEPREFIX you wish to use ==

1 .wine-3dimviewer
2 .wine-myphoneexplorer
3 .wine-nbtscan
4 .wine-pdfxve6
5 .wine-PortableApps
6 .wine-radiant
7 .wine-symmetry
8 .wine-visio
9 .wine-xnviewmp

Enter number (q to abort) and press ENTER: 

Let’s say I want to use the Windows application XnViewMP. I would enter ‘9’ and press ‘Enter’. The rest of the interaction should be obvious:


This will mount the Samba share folder brianfolder on the bsfnas1 machine.


== Select which WINEPREFIX you wish to use ==

1 .wine-3dimviewer
2 .wine-myphoneexplorer
3 .wine-nbtscan
4 .wine-pdfxve6
5 .wine-PortableApps
6 .wine-radiant
7 .wine-symmetry
8 .wine-visio
9 .wine-xnviewmp

Enter number (q to abort) and press ENTER: 9

~/.wine-xnviewmp/dosdevices/y: already exists. Is it OK to proceed anyway (y/n)? y

Enter your Linux account password below...

[sudo] password for fitzcarraldo: 

Samba share //bsfnas1/brianfolder is mounted for WINEPREFIX ~/.wine-xnviewmp .

You may now close this window.

Henceforth the Windows application XnViewMP will be able to access the Y: drive which is actually the SMB share //bsfnas1/brianfolder.

Once I have finished using the application, I just double-click on the the icon to unmount the SMB share, and a terminal window pops up displaying the following:


This will unmount the Samba share folder brianfolder on the bsfnas1 machine.

Enter your Linux account password below...

[sudo] password for fitzcarraldo: 

Samba share //bsfnas1/brianfolder is not mounted.

You may now close this window.

Once I have entered my Linux password for the local machine, the script will unmount the SMB share and the terminal window will close automatically if you have configured the Desktop Configuration file by right-clicking on the icon and unticking ‘Do not close when command exits’ in KDE, ‘Keep terminal window open after command execution’ in LXDE, or similar in other desktop environments.

Note: If you use Microsoft Office via WINE, you also might be interested in a comment on my earlier blog post about a Microsoft Office problem in saving files to a remote SMB share.

Creating a RAID of USB pendrives in Linux

USB hub and USB pendrives used as RAID10 with my laptop

USB hub and pendrives used as RAID10 with my laptop.

If you’re not familiar with the RAID (Redundant Array of Inexpensive Disks) concept and the different types of array, the article ‘RAID 0, RAID 1, RAID 5, RAID 10 Explained with Diagrams‘ gives a quick summary (and links to another article ‘RAID 2, RAID 3, RAID 4, RAID 6 Explained with Diagram‘). Another helpful article is ‘RAID Levels Explained‘.

A few years ago I came across a YouTube video by a Mac user, titled ‘Use a bunch of USB Flash drives in a RAID array‘. Purely out of interest he had experimented with creating RAIDs using USB pendrives (also known as ‘USB flash drives’ or ‘USB memory sticks’). The creation of a RAID using USB pendrives for his Apple Macs was very easy, and, since then, I had wanted to try this using one of my laptops running Linux, just to satisfy my curiosity. I have previously created software RAIDs in a Linux server using internal 3.5-inch HDDs, for the root, home and swap partitions, and for file storage partitions for a Cloud server and NAS. However, I had never created a RAID using external USB drives. This week I happened to have a spare four-port USB 3.0 hub and four old 4GB USB 2.0 pendrives, so I finally got the chance to create a RAID with USB pendrives (see photo). I decided to use my main laptop, which has Gentoo Linux with OpenRC, elogind, eudev and KDE installed. That installation does not have an initramfs so I did not need to rebuild an initramfs to assemble the RAID. Anyway, early assembly of a RAID by an initramfs would only be needed if the RAID were being used to hold the directories required by the OS (the root partition, for example). As my RAID would be pluggable external storage, I wanted to mount it manually rather than adding it to /etc/fstab to be mounted automatically at boot. As I had not used a RAID on this laptop before, I had not enabled the RAID drivers in the kernel configuration, so I needed to do that and rebuild the kernel. I opted to make the RAID drivers kernel modules rather than built into the kernel, so that I could load only the relevant module for whichever type of RAID I wished to create.

I had to decide which filesystem to use in the RAID. I have always used ext4 in my RAIDs using HDDs. However, F2FS is an interesting filesystem developed by Samsung for devices using flash memory, such as SD cards, USB pendrives and SSDs. So I decided to format the pendrives to use F2FS, and create an F2FS RAID. As I had not used F2FS previously on this laptop, I had not enabled the F2FS driver in the kernel configuration, so I enabled the F2FS driver in the kernel at the same time as I enabled the RAID drivers. As with the RAID drivers, I opted to make the F2FS driver a kernel module rather than built into the kernel, so that I could load it and unload it whenever I wanted.

Not only did it turn out to be easy to create a RAID using USB pendrives, I found that the Linux RAID module gets loaded automatically when I connect the USB hub. Furthermore the RAID is recognised by KDE and listed under ‘Places’ in the Dolphin file manager’s windows, which I can click on to mount and unmount the RAID. So I did not even need to configure the OS to load the RAID module at boot (the OS does not load the module automatically at boot if the hub is not connected).

DigitalOcean produced a good tutorial on creating RAIDs in Ubuntu: ‘How To Create RAID Arrays with mdadm on Ubuntu 16.04‘. The procedure is essentially the same in Gentoo Linux, the only differences being the path of the mdadm.conf file and the method of updating an initramfs (which I did not need to do anyway in this particular installation).

As I had four spare USB pendrives and a four-port hub, I decided to create a RAID10 array. Below is a summary of the steps I took.

1. I rebuilt the kernel in order to build the RAID and F2FS modules. The relevant kernel configuration parameters I set are shown below:

root # grep RAID /usr/src/linux/.config | grep -v "#"
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=m
CONFIG_ASYNC_RAID6_RECOV=m
CONFIG_RAID6_PQ=m
root # grep F2FS /usr/src/linux/.config | grep -v "#"
CONFIG_F2FS_FS=m
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
root # uname -a
Linux clevow230ss 4.19.72-gentoo #2 SMP Tue Oct 15 01:36:57 BST 2019 x86_64 Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz GenuineIntel GNU/Linux

2. I installed the mdadm tool:

root # eix -I mdadm
[I] sys-fs/mdadm
     Available versions:  4.1^t {static}
     Installed versions:  4.1^t(01:52:17 15/10/19)(-static)
     Homepage:            https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/
     Description:         Tool for running RAID systems - replacement for the raidtools

3. I installed the F2FS tools:

root # eix -I f2fs
[I] sys-fs/f2fs-tools
     Available versions:  1.10.0(0/4) 1.11.0-r1(0/5) 1.12.0-r1(0/6) ~1.13.0(0/6) {selinux}
     Installed versions:  1.12.0-r1(0/6)(02:05:17 15/10/19)(-selinux)
     Homepage:            https://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs-tools.git/about/
     Description:         Tools for Flash-Friendly File System (F2FS)

4. I rebooted the laptop.

5. The f2fs module was not loaded automatically, therefore I loaded it manually and edited /etc/conf.d/modules to add the module name so that it would be loaded automatically in future:

root # modprobe f2fs
root # lsmod | grep f2fs
f2fs                  466944  0
root # nano /etc/conf.d/modules
root # grep ^modules /etc/conf.d/modules
modules="fuse bnep rfcomm hidp uvcvideo cifs mmc_block snd-seq-midi iptable_raw xt_CT uinput f2fs"

6. I plugged the four USB pendrives into the USB hub, and connected the hub to the laptop.

7. I launched GParted, deleted the existing partition on each pendrive (three had been formatted as FAT32, one as exFAT), reformatted them individually as F2FS and gave them each a label (USBPD01 to USBPD04). I could have done all that from the command line but it is easier using GParted, and I like an easy life.

Note that the mdadm USE flag in Gentoo Linux needed to be set when GParted was merged, so GParted would need to be re-merged with USE="mdadm" if that is not the case. Furthermore, GParted will only include F2FS in the list of available filesystems if either the F2FS module is loaded or the F2FS driver has been built into the kernel.

8. I ascertained the names of the USB pendrives:

root # lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
NAME     SIZE FSTYPE TYPE MOUNTPOINT
sda    698.7G        disk
├─sda1   128M ext2   part
├─sda2    16G swap   part [SWAP]
├─sda5   128G ext4   part /
├─sda6   256G ext4   part /home
└─sda7 298.5G ntfs   part /media/NTFS
sdb      3.8G        disk
└─sdb1   3.8G f2fs   part
sdc      3.8G        disk
└─sdc1   3.8G f2fs   part
sdd      3.8G        disk
└─sdd1   3.8G f2fs   part
sde      3.8G        disk
└─sde1   3.8G f2fs   part

As you can see above, the four USB pendrives are sdb to sde.

9. I loaded the raid10 module:

root # modprobe raid10
root # lsmod | grep raid
raid10                 57344  1

10. I created the RAID10 array:

root # mdadm --create --verbose /dev/md0 --level=10 --raid-devices=4 /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: layout defaults to n2
mdadm: layout defaults to n2
mdadm: chunk size defaults to 512K
mdadm: partition table exists on /dev/sdb
mdadm: partition table exists on /dev/sdb but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/sdc
mdadm: partition table exists on /dev/sdc but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/sdd
mdadm: partition table exists on /dev/sdd but will be lost or
       meaningless after creating array
mdadm: partition table exists on /dev/sde
mdadm: partition table exists on /dev/sde but will be lost or
       meaningless after creating array
mdadm: size set to 3913728K
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

It takes a while for the RAID to be created, so I checked progress periodically as follows:

root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [>....................]  resync =  2.8% (222272/7827456) finish=23.8min speed=5308K/sec
      
unused devices: <none>
root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [========>............]  resync = 44.0% (3449856/7827456) finish=12.9min speed=5637K/sec
      
unused devices: <none>
root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [==============>......]  resync = 74.0% (5797760/7827456) finish=5.9min speed=5698K/sec
      
unused devices: <none>
root # cat /proc/mdstat
Personalities : [raid10]
md0 : active raid10 sde[3] sdd[2] sdc[1] sdb[0]
      7827456 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      
unused devices: <none>

11. I formatted the RAID:

root # sudo mkfs.f2fs -f /dev/md0

        F2FS-tools: mkfs.f2fs Ver: 1.12.0 (2018-11-12)

Info: Disable heap-based policy
Info: Debug level = 0
Info: Trim is enabled
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 15654912 (7644 MB)
Info: zone aligned segment0 blkaddr: 512
Info: format version with
  "Linux version 4.19.72-gentoo (root@clevow230ss) (gcc version 8.3.0 (Gentoo 8.3.0-r1 p1.1)) #2 SMP Tue Oct 15 01:36:57 BST 2019"
Info: [/dev/md0] Discarding device
Info: This device doesn't support BLKSECDISCARD
Info: This device doesn't support BLKDISCARD
Info: Overprovision ratio = 2.300%
Info: Overprovision segments = 179 (GC reserved = 94)
Info: format successful

The option ‘-f‘ forces mkfs to overwrite any existing filesystem. (I believe the same option is ‘-F‘ in Ubuntu, rather than ‘-f‘.)

12. I created a mount point so I could mount the RAID from the command line if I wanted:

root # mkdir -p /mnt/md0

13. I mounted the RAID from the command line and checked its size. In the case of RAID10 I would expect the size to be double the size of one of the formatted USB pendrives, i.e. approximtely 2 x 3.8GB = 7.6GB):

root # mount /dev/md0 /mnt/md0
root # df -h -x devtmpfs -x tmpfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       126G   36G   84G  31% /
/dev/sda6       252G  137G  103G  57% /home
/dev/sda7       299G  257G   43G  86% /media/NTFS
/dev/md0        7.5G  419M  7.1G   6% /mnt/md0
root # blkid | grep -v sda
/dev/md0: UUID="d565c117-37e0-48eb-b635-a2fe70b83272" TYPE="f2fs"
/dev/sdb: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="45a488a0-5126-0b95-0c28-eb1f743f77c7" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdc: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="ef7de228-cf4d-c6bf-c74a-462a0e27f8bd" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdd: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="b5dd5c41-3ab2-fa38-bd28-0b965883775c" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sde: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="16149e7e-5a96-ece6-65ba-25721bcee49f" LABEL="clevow230ss:0" TYPE="linux_raid_member"

So /dev/md0 looked correct.

14. I checked that nothing was already configured in mdadm.conf and added the array’s details to it:

root # grep -v "#" /etc/mdadm.conf
root # mdadm --detail --scan | sudo tee -a /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=clevow230ss:0 UUID=d1288120:a1614809:3e89bb5f:967df69b
root # grep -v "#" /etc/mdadm.conf
ARRAY /dev/md0 metadata=1.2 name=clevow230ss:0 UUID=d1288120:a1614809:3e89bb5f:967df69b

15. As the RAID will have only a partition for file storage, and as the RAID array will not always be connected to the laptop, it does not need to be assembled automatically early during boot, so there is no need to add mdadm.conf to an initramfs (which this laptop does not have anyway) and no need to specify /dev/md0 in /etc/fstab to be mounted at boot.

16. I left the USB hub connected to the laptop and rebooted.

17. I checked that the modules were loaded at boot:

root # lsmod | grep raid
raid10                 57344  1
root # lsmod | grep f2fs
f2fs                  466944  0

18. I checked that the RAID had been assembled correctly at boot:

root # blkid | grep -v sda
/dev/sdb: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="45a488a0-5126-0b95-0c28-eb1f743f77c7" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdc: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="ef7de228-cf4d-c6bf-c74a-462a0e27f8bd" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/sdd: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="b5dd5c41-3ab2-fa38-bd28-0b965883775c" LABEL="clevow230ss:0" TYPE="linux_raid_member"
/dev/md0: UUID="d565c117-37e0-48eb-b635-a2fe70b83272" TYPE="f2fs"
/dev/sde: UUID="d1288120-a161-4809-3e89-bb5f967df69b" UUID_SUB="16149e7e-5a96-ece6-65ba-25721bcee49f" LABEL="clevow230ss:0" TYPE="linux_raid_member"

19. I rebooted a few times with and without the USB hub connected. The module raid10 only gets loaded if the USB hub is connected. If I reboot without the hub connected, raid10 is no longer loaded automatically at boot. If I plug in the hub after the laptop has booted, raid10 gets loaded and the RAID array is recognised by the OS.

20. I mounted the RAID from the command line and copied a file to it as root user:

root # mount /dev/md0 /mnt/md0
root # ls -la /mnt/md0
total 8
drwxr-xr-x 2 root root 4096 Oct 15 07:40 .
drwxr-xr-x 7 root root 4096 Oct 15 07:42 ..
root # cp ./Paper_sheet_sizes.png /mnt/md0
root # ls -la /mnt/md0
total 268
drwxr-xr-x 2 root root   4096 Oct 15 08:07 .
drwxr-xr-x 7 root root   4096 Oct 15 07:42 ..
-rw-r--r-- 1 root root 265760 Oct 15 08:07 Paper_sheet_sizes.png
root # umount /dev/md0
root # ls -la /mnt/md0
total 8
drwxr-xr-x 2 root root 4096 Oct 15 07:42 .
drwxr-xr-x 7 root root 4096 Oct 15 07:42 ..

However, /mnt/md0/ is owned by the root user, so user fitzcarraldo cannot copy files into it. Therefore I changed the ownership:

root # mount /dev/md0 /mnt/md0
root # ls -la /mnt/
total 28
drwxr-xr-x  7 root root 4096 Oct 15 07:42 .
drwxr-xr-x 22 root root 4096 Oct  6 08:31 ..
-rw-r--r--  1 root root    0 Apr  9  2015 .keep
drwxr-xr-x  2 root root 4096 Apr 19  2015 cdrom
drwxr-xr-x  2 root root 4096 Jan 16  2017 floppy
drwxr-xr-x  2 root root 4096 Oct 15 08:07 md0
drwxr-xr-x  2 root root 4096 Apr 17  2015 pendrive
drwxr-xr-x  2 root root 4096 Mar 18  2016 usbstick
root # chown fitzcarraldo:fitzcarraldo /mnt/md0
root # ls -la /mnt/
total 28
drwxr-xr-x  7 root         root         4096 Oct 15 07:42 .
drwxr-xr-x 22 root         root         4096 Oct  6 08:31 ..
-rw-r--r--  1 root         root            0 Apr  9  2015 .keep
drwxr-xr-x  2 root         root         4096 Apr 19  2015 cdrom
drwxr-xr-x  2 root         root         4096 Jan 16  2017 floppy
drwxr-xr-x  2 fitzcarraldo fitzcarraldo 4096 Oct 15 08:07 md0
drwxr-xr-x  2 root         root         4096 Apr 17  2015 pendrive
drwxr-xr-x  2 root         root         4096 Mar 18  2016 usbstick
root # umount /dev/md0

21. ‘Places’ in Dolphin shows /mnt/md0 as ‘7.5 GiB Hard Drive’.

22. I can still mount the RAID from the command line:

root # mount /dev/md0 /mnt/md0
root # df -h /dev/md0
Filesystem      Size  Used Avail Use% Mounted on
/dev/md0        7.5G  420M  7.1G   6% /mnt/md0
root # umount /dev/md0

23. If I want to use the RAID in KDE I must use Dolphin to mount it, not mount it from the command line. To do this I click on the RAID ‘7.5 GiB Hard Drive’ listed under ‘Places’, and a window pop-ups prompting me to enter the root user’s password.

If I mount /dev/md0 via Dolphin instead of via the command line, KDE mounts it on a different directory:

root # df -h /run/media/fitzcarraldo/d565c117-37e0-48eb-b635-a2fe70b83272/
Filesystem      Size  Used Avail Use% Mounted on
/dev/md0        7.5G  420M  7.1G   6% /run/media/fitzcarraldo/d565c117-37e0-48eb-b635-a2fe70b83272

If I want to unmount it, I right-click on the RAID in ‘Places’ and select ‘Unmount’ in the right-click menu. Once it has been unmounted, I can unplug the hub from the laptop. If I plug the hub back into the laptop, the RAID is detected and can be mounted as usual.

So, it works! A USB hub and pendrives are a handy way to:

  • experiment with creating the various types of RAID;
  • compare the capacity of the RAID with the capacity of the USB pendrives used;
  • measure the time to write and read a large file to/from the RAID and compare those times with the time to write and read the same file to/from a single USB pendrive of the same model.

Firewall zones (profiles) in Linux, and how to switch them automatically if you use UFW

Firstly, a note on terminology: UFW (Uncomplicated Firewall) and its two GUI front-ends Gufw and UFW Frontends use the term ‘application profile’ to refer to a pre-configured set of rules specified in a file. Files containing UFW application profiles are placed in the directory /etc/ufw/applications.d/. An application profile for SMB, for example, enables the root user to use the UFW command ‘ufw allow Samba‘ (‘ufw allow CIFS‘ in Gentoo Linux) rather than having to enter UFW commands specifying the precise ports and network protocols that SMB uses. However, this blog post is not about UFW’s application profiles; it is about what Gufw calls ‘profiles’ and firewalld calls ‘zones’.

In essence a profile/zone is a collection of firewall policies and rules. Both Gufw and firewalld include the concept of a ‘zone’, although Gufw uses the term ‘profile’ rather than ‘zone’. UFW Frontends does not have the concept of a ‘zone’; rules entered via UFW Frontends apply to any network to which you connect your laptop. The ability to define different zones for different networks is handy. For example, you can have certain policies and rules when your laptop is connected to your home network, and different policies and rules when your laptop is connected to the network in a café, hotel, airport or other public place.

An attractive feature of firewalld when used in conjunction with NetworkManager and KDE Plasma is that it is possible to use the desktop environment’s network management module (‘System Settings’ > ‘Connections’) to specify a particular firewalld zone for a particular network connection. For example, let’s say you used firewalld to specify certain policies and rules for a zone you named ‘office’, and you then specified in the System Settings – Connections GUI that a connection named ‘ACM’ should use the zone ‘office’. Thereafter, whenever you connect your laptop to the network named ‘ACM’, firewalld will use the policies and rules you previously configured for the zone ‘office’.

Unlike firewalld, Gufw does not have the ability to switch profiles automatically according to which network the laptop is connected. You have to select manually the profile you wish to use. You would launch Gufw prior to connecting to, for example, your office’s network, select the profile ‘Office’ (or whatever you have named it), then connect your laptop to that network.

I think many people would be satisfied with the functionality currently provided by Gufw. I could use the Gufw GUI to create Gufw profiles with names such as ‘Home’, ‘HomeDave’, ‘Public’, ‘HQoffice’, ‘USoffice’, ‘PestanaRio’ and so on, and specify the different policies and rules I want for each profile. At home I would launch Gufw on my laptop and select the Home profile then connect to my home network; in the office at work I would launch Gufw on my laptop and select the HQoffice profile then connect to the office network; at my friend Dave’s house I would launch Gufw on my laptop and select the HomeDave profile then connect to the house network; and so on. Nevertheless I do see the attraction of automated zone switching, as provided by firewalld in conjunction with NetworkManager and KDE. It would be handy if my laptop could switch automatically to the Home profile when my laptop connected to the network at my home with the name ‘BTHub5-8EUQ’, automatically switch to the HQoffice profile when my laptop connected to the network named ‘HQ-Office2’ in the office, and so on.

I use UFW on my two laptops running Gentoo Linux. The package ufw-frontends is also installed but normally I use UFW directly via the command line. However I wanted to learn about zones/profiles while using UFW, and I also wanted to see if I could automate the switching of zones without resorting to installing firewalld. NetworkManager has the ability to launch ‘hook’ scripts when certain things happen — when a network connection changes, for example — and this seemed to me to be a way of switching profiles automatically.

I had not used Gufw before, so I decided to install it. A package is available in many Linux distributions but there is no ebuild for Gufw in Gentoo’s main Portage tree and I could not find an up-to-date ebuild for it in any Portage overlays. Therefore I created the ebuild for net-firewall/gufw-19.10.0 shown below. It probably needs improving, but it does install a working Gufw in Gentoo Linux.

# Copyright 1999-2019 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=7
PYTHON_COMPAT=( python3_{5,6,7} )
DISTUTILS_IN_SOURCE_BUILD=1

inherit distutils-r1

MY_PN="gui-ufw"
MY_PV="$(ver_cut 1-2)"

DESCRIPTION="GUI frontend for managing ufw."
HOMEPAGE="https://gufw.org/"
SRC_URI="https://launchpad.net/${MY_PN}/trunk/${MY_PV}/+download/${MY_PN}-${PV}.tar.gz"

LICENSE="GPL-3"
SLOT="0"
KEYWORDS="~amd64"
IUSE=""

DEPEND="dev-python/python-distutils-extra"
RDEPEND="net-firewall/ufw
	dev-python/netifaces
	dev-python/pygobject:3
	net-libs/webkit-gtk[introspection]
	sys-auth/elogind
	sys-auth/polkit
	x11-libs/gtk+:3[introspection]
	x11-themes/gnome-icon-theme-symbolic
"
S=${WORKDIR}/${MY_PN}-${PV}

pkg_postinst() {
	sed '/dist-packages/d' -i /usr/bin/gufw-pkexec
	sed -E '/\/share\//d' -i /usr/bin/gufw-pkexec
	local PYTHONVERSION="$(python -c 'import sys; print("{}.{}".format(sys.version_info.major, sys.version_info.minor))')"
	sed -E "s|python3\.[0-9]|python${PYTHONVERSION}|g" -i /usr/bin/gufw-pkexec
	sed -E 's|\/lib\/|\/lib64\/|g' -i /usr/bin/gufw-pkexec
}

How To Set Up a Firewall with GUFW on Linux‘ is a good tutorial on Gufw.

As I had not used Gufw previously, I had to play around with it to understand better its functional design. I found that if I configure rules directly via UFW on the command line without using Gufw, Gufw does not allow me to edit those rules (but does allow me to delete them) and those rules exist whichever Gufw profile is selected in the Gufw GUI. Gufw profiles are stored in files named ‘/etc/gufw/*.profile‘ (e.g. /etc/gufw/Home.profile) and these files will not include UFW rules entered via the command line. On the other hand, UFW rules created via the Gufw GUI apply solely to the currently-selected Gufw profile, which is what I would have expected. In other words, I can create a different set of policies and rules in each Gufw profile. Therefore I believe Gufw profiles (as distinct from UFW application profiles) are basically analogous to firewalld’s zones. It also appears to me that Gufw maintains configuration files specifying policies and rules independently of UFW, which Gufw then applies to UFW. In other words, if you are a Gufw user you should not use UFW directly to configure policies and rules, otherwise Gufw’s configuration files will not include what you did directly using UFW. To reiterate, use only Gufw or only UFW, not both.

The current Gufw profile’s name is listed in the file /etc/gufw/gufw.cfg. For example, I currently have the Home profile selected in the Gufw GUI, and the file gufw.cfg contains the following:

[GufwConfiguration]
profile = Home
windowwidth = 542
windowheight = 530
confirmdetelerule = yes

If I examine the contents of the file /etc/gufw/Home.profle I see that it contains the UFW policies and rules I specified for the Gufw Home profile:

[fwBasic]
status = enabled
incoming = deny
outgoing = allow
routed = disabled

[Rule0]
ufw_rule = 137,138/udp ALLOW IN 192.168.1.0/24
description = Samba
command = /usr/sbin/ufw allow in proto udp from 192.168.1.0/24 to any port 137,138
policy = allow
direction = in
protocol = 
from_ip = 192.168.1.0/24
from_port = 
to_ip = 
to_port = 137,138/udp
iface = 
routed = 
logging = 

[Rule1]
ufw_rule = 139,445/tcp ALLOW IN 192.168.1.0/24
description = Samba
command = /usr/sbin/ufw allow in proto tcp from 192.168.1.0/24 to any port 139,445
policy = allow
direction = in
protocol = 
from_ip = 192.168.1.0/24
from_port = 
to_ip = 
to_port = 139,445/tcp
iface = 
routed = 
logging =

I also notice that the other Gufw profiles can differ. For example, my Office.profile file contains the following:

[fwBasic]
status = enabled
incoming = deny
outgoing = allow
routed = allow

The profile name listed in gufw.cfg gets changed when the user changes the profile using the Gufw GUI. It appears to me that only at the point in time when the user selects a certain Gufw profile in the Gufw GUI does Gufw parse the applicable *.profile file and issue commands to UFW to implement the policies and rules specified in the *.profile file.

Initially I tried to automate the process of changing the Gufw profile by doing the following:

  1. I created a NetworkManager Dispatcher hook script to:

    1. detect when the laptop connects to a network;

    2. determine whether the network is at my home, at my workplace or in a public place (café, airport or wherever) by looking at the connection name;

    3. edit gufw.cfg to change the name of the Gufw profile according to the network connected.
  2. I configured KDE to launch Gufw automatically at login, hoping that would implement the Gufw profile specified in gufw.cfg.

When I connected the laptop to various networks, Gufw did indeed show the name of the profile selected by the NetworkManager Dispatcher hook script, but the associated Gufw profile’s rules had not been applied. They were only applied if I clicked on the ‘Profile’ pull-down menu in Gufw, selected a different Gufw profile, then re-selected the desired Gufw profile. Therefore driving Gufw from a NetworkManager Dispatcher hook script is not possible. This is a pity, as Gufw is an easy way to manage UFW from a GUI; it allows the user to create, delete and edit zones (Gufw profiles) and to select them manually. What Gufw doesn’t do is enable the user to associate those zones with connection names, nor trigger specific zone automatically based on the selected network connection. firewalld, on the other hand, does enable the user to do both those things.

As my attempt at automating the switching of zones in Gufw had failed, I decided to create a NetworkManager Dispatcher hook script to switch zones automatically by using UFW commands. Initially I though about creating a bespoke UFW application profile for each zone and allowing/denying those in the script, but it is actually easier to use the fundamental UFW commands in the script, especially as UFW commands are relatively easy to understand. Also, this approach means everything is in a single file, which facilitates configuration. I can simply edit the script in order to: a) add or delete a zone; b) change a zone’s name; c) change policies and rules for a zone; d) add or delete a connection; e) change the name of a connection; f) change the zone a connection uses. Granted, editing a script is not as user-friendly as using the firewalld GUI to configure a zone and then using KDE Plasma’s system settings module Connections to specify that zone for a specific connection, but my script is not particularly difficult to understand and edit. And by using such a script I can continue to use UFW rather than installing firewalld and having to learn how to use it.

My NetworkManager Dispatcher hook script /etc/NetworkManager/dispatcher.d/20_ufw-zones is listed below. In the main body of the script I define the zone I wish to use for each connection, and in the function select_zone I define the policies and rules I want each zone to use.

#!/bin/bash
INTERFACE=$1
STATUS=$2
WIRED=enp4s0f1
WIFI=wlp3s0

CT_helper_rule() {
    echo "# The following is needed to enable Samba commands to" >> /etc/ufw/before.rules
    echo "# work properly for broadcast NetBIOS name resolution" >> /etc/ufw/before.rules
    echo "#"  >> /etc/ufw/before.rules
    echo "# raw table rules" >> /etc/ufw/before.rules
    echo "*raw" >> /etc/ufw/before.rules
    echo ":OUTPUT ACCEPT [0:0]" >> /etc/ufw/before.rules
    echo "-F OUTPUT" >> /etc/ufw/before.rules
    echo "-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns" >> /etc/ufw/before.rules
    echo "COMMIT" >> /etc/ufw/before.rules
}
 
select_zone() {
    ufw --force reset
    ufw --force enable
    ZONE=$1
    case "$ZONE" in
    'Home')
        ufw default deny incoming
        ufw default allow outgoing
        #
        # Rules for SMB
        ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
        ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp
        CT_helper_rule
        #
        # Rules for KDEConnect
        ufw allow from 192.168.1.0/24 to any port 1714:1764 proto udp
        ufw allow from 192.168.1.0/24 to any port 1714:1764 proto tcp
    ;;
    'Office')
        ufw default deny incoming
        ufw default allow outgoing
    ;;
    'Public')
        ufw default reject incoming
        ufw default allow outgoing
    ;;
    'JohnsHouse')
        ufw default deny incoming
        ufw default allow outgoing
        #
        # Rules for SMB
        ufw allow from 192.168.42.0/24 to any port 137,138 proto udp
        ufw allow from 192.168.42.0/24 to any port 139,445 proto tcp
        CT_helper_rule
        #
        # Rules for KDEConnect
        ufw allow from 192.168.42.0/24 to any port 1714:1764 proto udp
        ufw allow from 192.168.42.0/24 to any port 1714:1764 proto tcp
    ;;
    esac
    ufw --force reload
    rm /etc/ufw/*.rules.20* # Delete backups of *.rules files ufw makes every time it is reset
    echo -n `date +"[%F %T %Z]"` >> /var/log/ufw-zones.log
    echo " Zone $ZONE selected for connection $ACTIVE on interface $INTERFACE." >> /var/log/ufw-zones.log
}
 
# Check if either the wired or wireless interface is up
if [ "$INTERFACE" = "$WIRED" -o "$INTERFACE" = "$WIFI" ] && [ "$STATUS" = "up" ]; then
 
    # Check if a single connection is active
    if [ `nmcli c | grep -v "\-\-" | grep -v "NAME.*UUID.*TYPE.*DEVICE" | wc -l` -eq 1 ]; then
 
        # Ascertain the name of the active connection
        ACTIVE=`nmcli c | grep -v "\-\-" | grep -v "NAME.*UUID.*TYPE.*DEVICE" | awk -F' ' '{print $1}'`
 
        case "$ACTIVE" in
 
        'eth0')
            ZONE="Home"
        ;;
        'POR1-wired')
            ZONE="Office"
        ;;
        'BTHub5-8EUQ')
            ZONE="Home"
        ;;
        'BTHub5-8EUQ-5GHz')
            ZONE="Home"
        ;;
        'John1')
            ZONE="JohnsHouse"
        ;;
        'GRAND MERCURE')
            ZONE="Public"
        ;;
        *)
            # If connection name is not in above list
            ZONE="Public"
        ;;

        esac

        select_zone $ZONE
        exit $?

    fi
fi

The log file that the script uses contains a chronological record of the connections made and the zones selected:

$ cat /var/log/ufw-zones.log 
[2019-09-30 20:13:52 BST] Zone Home selected for connection eth0 on interface enp4s0f1.
[2019-10-01 22:59:18 BST] Zone Home selected for connection BTHub5-8EUQ-5GHz on interface wlp3s0.
[2019-10-02 17:59:23 EDT] Zone Public selected for connection loganwifi on interface wlp3s0.
[2019-10-03 10:12:46 EDT] Zone Office selected for connection POR1-wired on interface enp4s0f1.

Gentoo Linux: A work-around to be able to Resume from Suspend to RAM when using the NVIDIA closed-source driver

My Clevo W230SS laptop has NVIDIA Optimus graphics hardware (NVIDIA GPU plus Intel IGP). I do not use Bumblebee, preferring to switch between the Intel video driver and the NVIDIA closed-source driver myself (see Switching between Intel and NVIDIA graphics processors on a laptop with NVIDIA Optimus hardware running Gentoo Linux). The laptop can suspend to RAM and resume perfectly when using the Intel video driver (but see Stopping my laptop spontaneously resuming immediately after Suspend to RAM, which is applicable whatever the GPU or IGP).

In order to be able to resume properly from Suspend-to-RAM when using the NVIDIA driver, the laptop needs to disable compositing before suspending, then re-enable compositing after resuming. For how I achieve that, see under Problem 2 in the third link above. If this is not done, the graphics on the Desktop are corrupted after resuming.

However, recently when using the NVIDIA driver and KDE Plasma 5 (I am currently using nvidia-drivers-387.22 and plasma-meta-5.11.5), when resuming from suspension the monitor would briefly display the LightDM wallpaper (I use different wallpapers for the display manager and the lock screen, so I know it was not the KDE lock screen) followed by a blank screen with a mouse pointer (which I could move normally). More recently, in between displaying the display manager’s wallpaper and the blank screen, the monitor would briefly display an earlier image of the Desktop just before the laptop suspended.

Now, I could simply leave the laptop configured to use the Intel driver. However, sometimes I need to use a CAD application and the performance is better when using the NVIDIA GPU.

There are umpteen posts on the Web about this problem, and the root cause seems to be the closed-source NVIDIA driver. I have seen the KDE lock screen mentioned in some posts as the culprit, so I disabled the lock screen (‘System Settings’ > ‘Desktop Behaviour’ > ‘Screen Locking’) but that did not solve the problem.

I put up with this for several weeks in the hope that the next release of the NVIDIA driver would fix the problem. If I suspended to RAM while the laptop was using the NVIDIA driver, I was able to resume and get to a working Desktop – albeit without the open windows and applications that had been running before suspending – by pressing Ctrl+Alt+F1 to get to TTY1, logging in as the root user and entering the command ‘/etc/init.d/xdm restart‘. However, the final straw was in a meeting a couple of weeks ago when I wanted to resume the laptop and show a worksheet to someone. The laptop monitor of course displayed a blank screen with a mouse pointer, and it took me a couple of minutes to restart the display manager, login to KDE Plasma 5 and open the spreadsheet again. So this week I decided to look into the problem to see if I could at least find a work-around that would enable the laptop to resume without needing to restart X Windows and login to Plasma 5 each time.

I created a Bash script in /etc/pm/sleep.d/ to unload the NVIDIA modules before suspending to RAM and to re-load them when resuming, but that did not solve the problem either.

I switched the rendering background from OpenGL 2.0 to OpenGL 3.1 (‘System Settings’ > ‘Display and Monitor’ > ‘Compositor’), but that did not work either. I switched the rendering backend to XRender, and that did enable the laptop to resume from suspend successfully with the NVIDIA driver, but I do not want to use that work-around. Firstly, with software rendering there is a performance hit, and, secondly, there was no KDE Desktop Cube when using XRender instead of OpenGL. I use the Desktop Cube when working, as I often have a lot of windows open on each virtual desktop (cube side), and I find it easier to use the cube than a flat UI.

Eventually I found that, after resuming, if I pressed Ctrl+Alt+F1 to get to a virtual console, logged into my user account, entered the command ‘DISPLAY=:0 /usr/bin/kwin_x11 --resume‘ and then pressed Ctrl+Alt+F7 to get back to TTY7, my Desktop would appear on TTY7. Even so, I noticed on TTY1 that the following error messages were displayed when I ran that command:

kwin_core: OpenGL 2 compositing setup failed
kwin_core: Failed to initialize compositing, compositing disabled

Anyway, the Plasma 5 Desktop was displayed on TTY7, and with the windows that were open when I suspended the laptop, so restarting KWin would at least be a viable work-around until NVIDIA fix their video driver.

I incoporated the command in my script /etc/pm/sleep.d/02-toggle-compositing like so:

#!/bin/sh
#
# Turn off compositing on hibernate or suspend
# Turn on compositing on thaw or resume

username=fitzcarraldo
userhome=/home/$username
export XAUTHORITY="$userhome/.Xauthority"
export DISPLAY=":0"

case "$1" in
     suspend|hibernate)
          su $username -c "qdbus org.kde.KWin /Compositor suspend" &
     ;;
     resume|thaw)
          su $username -c "qdbus org.kde.KWin /Compositor resume" &
          su $username -c "/usr/bin/kwin_x11 --replace" &
     ;;
     *)
          exit $NA
     ;;
esac

It is an ugly hack, but at least now the laptop can resume properly from Suspend-to-RAM while the NVIDIA driver is being used.

Perhaps Linus Torvalds was correct. I will try to avoid NVIDIA hardware when I replace my current laptop.

Prevent Linux firewalls interfering with Samba commands in a home network that uses broadcast NetBIOS name resolution

Or “How come devices in a home network can browse SMB shares but Linux Samba commands and Windows nbtstat commands do not work properly?”

Introduction

In a previous post I explained how it is possible to browse SMB shares when using broadcast NetBIOS name resolution in a home network consisting of machines running Linux, Windows and other operating systems. Browsing SMB/Samba shares will work as expected, but Samba commands such as ‘smbtree‘, ‘smbclient‘ and ‘nmblookup‘ will not work properly if the Linux machines use a firewall that has not been configured for broadcast NetBIOS name resolution. This post is to explain how to do that.

If broadcast NetBIOS name resolution is being used and none of the Linux machines has a firewall enabled, or if their firewalls have been correctly configured, the output of e.g. the ‘smbtree‘ command on one of those machines would look something like the example below.

anne@akhanaten:~$ smbtree
Enter anne's password: 
HOME
        \\AKHANATEN                     Samba 4.3.11-Ubuntu
                \\AKHANATEN\IPC$                IPC Service (Samba 4.3.11-Ubuntu)
                \\AKHANATEN\guest               guest account
                \\AKHANATEN\matthew             matthew share
                \\AKHANATEN\marilla             marilla share
                \\AKHANATEN\anne                anne share
        \\TUTANKHAMUN                   Samba 4.5.10
                \\TUTANKHAMUN\Samsung_Xpress_C460FW     Samsung Xpress C460FW
                \\TUTANKHAMUN\Canon_MP560_Printer       Canon PIXMA MP560
                \\TUTANKHAMUN\Canon_MP510_Printer       Canon PIXMA MP510
                \\TUTANKHAMUN\Virtual_PDF_Printer       Virtual PDF Printer
                \\TUTANKHAMUN\IPC$              IPC Service (Samba 4.2.11)
                \\TUTANKHAMUN\Public
                \\TUTANKHAMUN\anne-share
                \\TUTANKHAMUN\print$
                \\TUTANKHAMUN\netlogon          Network Logon Service
        \\BTHUB5                        BT Home Hub 5.0A File Server
                \\BTHUB5\IPC$                   IPC Service (BT Home Hub 5.0A File Server)
        \\THUTMOSEIII                   Windows 10 computer

If Linux firewalls have not been correctly configured, the output would be missing some information about other machines in the network. For example, compare the output above with the output below from the same network, this time with the Linux firewalls configured using typical rules for Samba specified in Web articles, blog posts and forums.

anne@akhanaten:~$ smbtree
Enter anne's password: 
HOME
        \\AKHANATEN                     Samba 4.3.11-Ubuntu
                \\AKHANATEN\IPC$                IPC Service (Samba 4.3.11-Ubuntu)
                \\AKHANATEN\guest               guest account
                \\AKHANATEN\matthew             matthew share
                \\AKHANATEN\marilla             marilla share
                \\AKHANATEN\anne                anne share
        \\TUTANKHAMUN                   Samba 4.5.10
        \\BTHUB5                        BT Home Hub 5.0A File Server
        \\THUTMOSEIII                   Windows 10 computer

To avoid this problem you need to add a further Linux firewall rule to the set of rules usually used for Samba. Below I first list the usual firewall rules for Samba, then I give the additional rule necessary if using broadcast NetBIOS name resolution. In each case I give the applicable rules for a pure IPTABLES firewall and for UFW (Uncomplicated Firewall). The rules listed here assume the IP address range of the home network is 192.168.1.0/24, so change the range to suit the specific network.

Firewall rules typically specified for machines using Samba

IPTABLES

The rules listed below assume the machine uses interface eth0, so change the interface to suit the specific machine.

# NetBIOS Name Service (name resolution)
iptables -A INPUT -i eth0 -p udp --dport 137 -s 192.168.1.0/24 -j ACCEPT

# NetBIOS Datagram Service (BROWSER service)
iptables -A INPUT -i eth0 -p udp --dport 138 -s 192.168.1.0/24 -j ACCEPT

# NetBIOS Session Service (data transfer legacy SMB/NetBIOS/TCP)
iptables -A INPUT -i eth0 -p tcp --dport 139 -s 192.168.1.0/24 -j ACCEPT

# Microsoft Directory Service (data transfer SMB/TCP)
iptables -A INPUT -i eth0 -p tcp --dport 445 -s 192.168.1.0/24 -j ACCEPT

UFW

In some Linux distributions the ufw application allows a single command to add Samba support, such as:

user $ sudo ufw allow Samba

or

user $ sudo ufw allow CIFS

These ‘application profiles’ are specified in files in the directory /etc/ufw/applications.d/, so you could add application profiles or modify existing ones if you wish. In one of my installations the file /etc/ufw/applications.d/ufw-fileserver includes the following application profile for Samba, for example:

[CIFS]
title=SMB/CIFS server
description=SMB/CIFS server
ports=137,138/udp|139,445/tcp

If such an application profile does not exist in your installation, typical Samba rules can be added in UFW using the following two commands:

user $ sudo ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
user $ sudo ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp

The correct addition of the rules can be checked using the following command:

user $ sudo ufw status verbose
Password:
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
137,138/udp (CIFS)         ALLOW IN    192.168.1.0/24
139,445/tcp (CIFS)         ALLOW IN    192.168.1.0/24

The extra rule required when using broadcast NetBIOS name resolution

The reason why an extra rule is required when using broadcast NetBIOS name resolution is because UFW (which is based on IPTABLES) is ‘stateful’, as is a purely IPTABLES firewall (unless explicitly configured not to be stateful). The firewall does not consider packets it receives in response to its broadcast to be ESTABLISHED or RELATED, and therefore drops those packets. So, despite the IPTABLES and UFW rules listed above including a rule to accept incoming UDP packets on Port 137, any UDP packets received on Port 137 that do not constitute a one-to-one, two-way communication flow are dropped by the firewall. The extra rule below overrules this and makes the firewall accept packets coming from other devices’ Port 137 in response to broadcast NetBIOS Name Service packets. To do this, the extra rule uses a CT (Connection Tracking) helper named ‘netbios-ns‘ (obviously meaning ‘NetBIOS Name Service’). In order to use this rule the kernel must have been configured to use the IPTABLES ‘raw‘ table and to use CT (see the section ‘Kernel configuration’ further on).

IPTABLES

# All NetBIOS clients must have the netbios-ns helper enabled for broadcast name resolution to work
iptables -t raw -A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns

By the way, in addition to flushing the usual tables, flush the ‘raw‘ table too when you restart the firewall:

iptables -t raw -F OUTPUT

UFW

Add the following lines to the end of the file /etc/ufw/before.rules

# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

Note that the output of the command ‘ufw status verbose‘ will not include the above rule. This is not a bug.

Kernel configuration

If you are using a binary-based distribution such as Ubuntu Linux, the kernel will probably have been configured to include the needed modules (CONFIG_IP_NF_RAW=m, CONFIG_IP6_NF_RAW=m and CONFIG_NETFILTER_XT_TARGET_CT=m), and the installation configured to load the modules automatically. However, if you are using a source-based distribution such as Gentoo Linux make sure the kernel configuration includes these three options before you build the kernel, and also add the module names ‘iptable_raw‘ and ‘xt_CT‘ to the module list in the file /etc/conf.d/modules as shown in the example below, so that the modules are loaded at boot:

modules="r8169 nvidia agpgart fuse bnep rfcomm hidp uvcvideo cifs mmc_block rtsx_pci snd-seq-midi vboxdrv vboxnetadp vboxnetflt iptable_raw xt_CT"

You can use the following two commands to check if the two modules are loaded:

user $ sudo lsmod | grep iptable_raw
user $ sudo lsmod | grep xt_CT

How to check the additional rule is active

You can use the command below whether you are using pure IPTABLES or UFW.

user $ sudo iptables -nvL -t raw
Password: 
Chain PREROUTING (policy ACCEPT 2613 packets, 1115K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 2773 packets, 475K bytes)
 pkts bytes target     prot opt in     out     source               destination         
   16  1248 CT         udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:137 CT helper netbios-ns

The packet and byte counts will increase whenever you use a Samba command.

Bibliography

  1. The netfilter.org "iptables" project
  2. Iptables Tutorial
  3. Introduction to IPTables
  4. Gentoo Wiki : iptables
  5. Arch Linux Wiki : Samba : "Browsing" network fails with "Failed to retrieve share list from server"
  6. Ubuntu : Manpage : ufw-framework
  7. Gentoo Wiki : UFW

xdotool comes to the rescue

In a previous post I explained how I implemented a method for adding my current location and the local time to my e-mail signature wherever I happen to be in the World, irrespective of the time on the laptop’s hardware clock and system clock. In that post I described how I created a keyboard shortcut using the Linux application AutoKey. Unfortunately AutoKey has not been updated for several years and no longer works properly in KDE Plasma 5 on my laptops. Therefore I decided to replace it with a KDE keyboard shortcut, and this is to explain how I did it.

First create a custom shortcut in KDE:

  1. ‘System Settings’ > ‘Shortcuts’ > ‘Custom Shortcuts’
  2. ‘Edit’ > ‘New’ > ‘Global Shortcut’ > ‘Command/URL’, and name the New Action ‘Insert current time’
  3. On the Comment pane for ‘Insert current time’, add the comment ‘Insert current time at specified location’ (without the quotes)
  4. On the Trigger pane, configure the shortcut to be Ctrl+Alt+Space
  5. On the Action pane, enter the Command/URL as ‘/home/fitzcarraldo/timezone_signature_GeoNames.sh‘ (without the quotes)
  6. Click ‘Apply’

Next modify the Bash script timezone_signature_GeoNames.sh so that it contains the following (obviously change the username and path to suit):

#!/bin/bash

place=$(kdialog --title "Current Location" --inputbox "Enter your location:")

placetime=$(perl /home/fitzcarraldo/now1.pl $place)

# xdotool does not output a space in a string, so we have to extract each field from the string
# and print each field individually, separated by a space character.

city=$(echo $placetime | awk -F "|" '{print $1}')
country=$(echo $placetime | awk -F "|" '{print $2}' | sed 's/[)(]//g')
region=$(echo $placetime | awk -F "|" '{print $4}')

datetime=$(/usr/bin/zdump $region | awk -F " " '{print $2" "$3" "$4" "$5" "$6" "$7}')
dayofweek=$(echo $datetime | awk -F " " '{print $1}')
month=$(echo $datetime | awk -F " " '{print $2}')
day=$(echo $datetime | awk -F " " '{print $3}')
time=$(echo $datetime | awk -F " " '{print $4}')
year=$(echo $datetime | awk -F " " '{print $5}')
timezone=$(echo $datetime | awk -F " " '{print $6}')

activewindow=$(xdotool getactivewindow)

xdotool type --window $activewindow "Sent from:"
for oneword in $city; do
    xdotool key --window $activewindow space
    sleep 0.1s
    xdotool type --window $activewindow --delay 100 $oneword
done
xdotool key --window $activewindow comma
for oneword in $country; do
    xdotool key --window $activewindow space
    sleep 0.1s
    xdotool type --window $activewindow --delay 100 $oneword
done
xdotool key --window $activewindow Return
xdotool type --window $activewindow "Local time now: "
xdotool type --window $activewindow $dayofweek
xdotool type --window $activewindow " "
xdotool type --window $activewindow $month
xdotool type --window $activewindow " "
xdotool type --window $activewindow $day
xdotool type --window $activewindow " "
xdotool type --window $activewindow $time
xdotool type --window $activewindow " "
xdotool type --window $activewindow $year
xdotool type --window $activewindow " "
if [ ${timezone:0:1} = "-" ]; then
    timezone="UTC-"${timezone#*-}
elif [ ${timezone:0:1} = "+" ]; then
    timezone="UTC+"${timezone#*+}
fi
xdotool type --window $activewindow $timezone
xdotool type --window $activewindow " "
xdotool key --window $activewindow Return
xdotool key --window $activewindow Return
echo

The Perl script now1.pl is listed in my my earlier post. Notice that the script timezone_signature_GeoNames.sh in my earlier post was much simpler. This was because the AutoKey shortcut took care of sending the text to the currently active window. Without AutoKey, I now had to do this myself in the script timezone_signature_GeoNames.sh, and the command xdotool came to the rescue. The developer explains what xdotool does as follows:

This tool lets you simulate keyboard input and mouse activity, move and resize windows, etc. It does this using X11’s XTEST extension and other Xlib functions.

Additionally, you can search for windows and move, resize, hide, and modify window properties like the title. If your window manager supports it, you can use xdotool to switch desktops, move windows between desktops, and change the number of desktops.

So I installed xdotool via the Gentoo package manager:

# emerge xdotool
# eix xdotool
[I] x11-misc/xdotool
     Available versions:  3.20150503.1-r1^t ~3.20160805.1^t {examples}
     Installed versions:  3.20150503.1-r1^t(22:51:30 02/04/17)(-examples)
     Homepage:            http://www.semicomplete.com/projects/xdotool/
     Description:         Simulate keyboard input and mouse activity, move and resize windows

Anyway, my Bash script using xdotool works a treat with Thunderbird (and KWrite, LibreOffice Writer, etc.). I used to experience a problem with certain characters, for example a colon was printed as a semi-colon (see the xdotool bug report xdotool writes the wrong case #121), but that no longer happens in my current KDE Plasma 5 installation:

Sent from: Galeão International Airport, Brazil
Local time now: Thu Jul 6 15:11:40 2017 UTC-03

What a useful tool xdotool is!

Stuttering audio in Linux: PulseAudio strikes again

I unmasked PulseAudio 10.0 back in January 2017 and installed it in my Gentoo Stable amd64 installation, and everything worked fine… until a couple of days ago, when the audio in streaming YouTube videos started to stutter every so often. It sounded rather like a scratched LP jumping. At first I thought the problem lay with Firefox, but the stuttering audio also occurred in Chrome. Then I wondered if my Internet connection was to blame; perhaps the ISP’s service had deteriorated. But a Windows 10 machine on my home network didn’t suffer from the problem, so that seemed to rule out the Internet connection. I tested the broadband throughput, and it was circa 32 Mbps, actually a little higher than the last time I tested it last year.

Now, Gentoo is a rolling distribution and I update my laptops regularly, but I couldn’t think what had been upgraded in the last couple of months that could be causing the problem. Although PulseAudio had not been upgraded since January, I began to wonder if PulseAudio could be involved, as my audio woes in the past have usually been due to PulseAudio.

I have always had PulseAudio installed with USE=”-realtime”:

user $ eix -I pulseaudio
[I] media-sound/pulseaudio
     Available versions:  10.0 {+X +alsa +alsa-plugin +asyncns bluetooth +caps dbus doc equalizer +gdbm +glib gnome gtk ipv6 jack libressl libsamplerate lirc native-headset neon ofono-headset +orc oss qt4 realtime selinux sox ssl system-wide systemd tcpd test +udev +webrtc-aec zeroconf ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  10.0(16:07:53 19/04/17)(X alsa alsa-plugin asyncns bluetooth caps dbus gdbm glib gnome gtk ipv6 jack orc qt4 ssl tcpd udev webrtc-aec zeroconf -doc -equalizer -libressl -libsamplerate -lirc -native-headset -neon -ofono-headset -oss -realtime -selinux -sox -system-wide -systemd -test ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://www.pulseaudio.org/
     Description:         A networked sound server with an advanced plugin system

but I wondered if PulseAudio’s real-time scheduling was somehow the cause of the problem, so I edited /etc/pulse/daemon.pa and added ‘realtime-scheduling = no‘ (I assume the default is ‘yes‘, as it was commented as such in the file):

; realtime-scheduling = yes
realtime-scheduling = no

Problem solved. PulseAudio is indeed a demon. 😡

Using the ClamAV daemon to scan files placed in my Downloads directory in Gentoo Linux

In a previous post I explained how to automatically detect files placed in my Downloads directory in Linux and scan them for viruses. The method I described in that post used clamscan, the command-line anti-virus scanner of ClamAV. Now, in addition ClamAV has a daemon (a program that runs continuously in the background), clamdscan, that you can enable. So I decided to switch to using clamdscan, as its response to downloaded files is much faster because the process waiting for new files to appear in ~/Downloads/ does not have to load clamscan from disk each time a new file arrives. Anyway, if you want to monitor a download directory in Gentoo Linux (running OpenRC) by using the ClamAV daemon — which will also download virus signature database updates automatically — then the procedure to set this up is given below.

1. Install clamav if it is not installed already:

root # emerge clamav

2. Add the service to the default runlevel:

root # rc-update add clamd default

The daemon will be launched automatically next time the computer boots.

3. The first download of the virus database has to be done manually:

root # freshclam

4. Start the daemon now:

root # rc-service clamd start

5. Create the Bash script ~/monitorDownloadsGUI with the following contents:

#!/bin/bash

DIR=$HOME/Downloads

# Get rid of old log file, if any
rm $HOME/virus-scan.log 2> /dev/null

IFS=$(echo -en "\n\b")

# Optionally, you can use shopt to avoid creating two processes due to the pipe
shopt -s lastpipe
inotifywait --quiet --monitor --event close_write,moved_to --recursive --format '%w%f' $DIR | while read FILE
# Added '--recursive' so that a directory copied into $DIR also triggers clamscan/clamdscan, although downloads
# from the Web would just be files, not directories.
do
     # Have to check file length is nonzero otherwise commands may be repeated
     if [ -s $FILE ]; then
          # Replace 'date >' with 'date >>' if you want to keep log file entries for previous scans.
          date > $HOME/virus-scan.log
          clamdscan --move=$HOME/virus-quarantine $FILE >> $HOME/virus-scan.log
          kdialog --title "Virus scan of $FILE" --msgbox "$(cat $HOME/virus-scan.log)"
     fi
done

Make it executable:

user $ chmod +x ~/monitorDownloadsGUI

6. Create the directory ~/virus-quarantine/ to store infected files pending investigation/deletion:

user $ mkdir ~/virus-quarantine

7. Install kdialog if it is not already installed:

root # emerge kdialog

8. Use ‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’ to add the script ~/monitorDownloadsGUI to the list of script files that are automatically started each time you log in to KDE.

9. Log out then back in again, and you should see that everything is running as expected:

user $ rc-status | grep clam
 clamd                                                             [  started  ]

user $ ps -ef | grep clam | grep -v grep
clamav    1920     1  0 01:48 ?        00:00:00 /usr/sbin/clamd
clamav    1929     1  0 01:48 ?        00:00:00 /usr/bin/freshclam -d

user $ ps -ef | grep GUI | grep -v grep
fitzcarraldo      9143  8971  0 13:56 ?        00:00:00 /bin/bash /home/fitzcarraldo/.config/autostart-scripts/monitorDownloadsGUI.sh

10. To test, surf to http://www.eicar.org/ and download one of the EICAR test files into your ~/Downloads/ directory. You should see a pop-up KDialog window with a message similar to the following:

Virus scan of /home/fitzcarraldo/Downloads/eicarcom2.zip — KDialog

Mon 27 Feb 14:05:26 GMT 2017
/home/fitzcarraldo/Downloads/eicarcom2.zip: Eicar-Test-Signature FOUND
/home/fitzcarraldo/Downloads/eicarcom2.zip: moved to ‘/home/fitzcarraldo/virus-quarantine/eicarcom2.zip’

———– SCAN SUMMARY ———–
Infected files: 1
Time: 0.001 sec (0 m 0 s)

Note that the above-mentioned pop-up window may be preceded by one or more pop-up windows with an error message. I’m using the Chrome browser at the moment, but you may get a similar message if you are using another browser. Here is an example:

Virus scan of /home/fitzcarraldo/Downloads/.com.google.Chrome.Uh3oGm — KDialog ?

Mon 27 Feb 14:16:30 GMT 2017
/home/fitzcarraldo/Downloads/.com.google.Chrome.Uh3oGm: Access denied. ERROR

———– SCAN SUMMARY ———–
Infected files: 0
Total errors: 1
Time: 0.000 sec (0 m 0 s)

Read the error message and click ‘OK’, as this is not an actual problem; it is inotifywait detecting temporary files in the ~/Downloads/ directory during the download process. With larger files sometimes several such messages are displayed, presumably because the file being downloaded is being opened and closed more than once during the downloading process. This issue does not occur if you copy or move a file into ~/Downloads/ from another directory in your installation; try it and see for yourself. Then you only get the one pop-up window with the scan result for the file you put in ~/Downloads/.

Also have a look in ~/virus-quarantine/ and you will see the EICAR test file in that directory. You can delete it if you want (it is not infected with a real virus, so does no harm).

In future be sure to read the messages in the pop-up windows before clicking ‘OK’, as they will inform you that an infected file has been moved to the quarantine directory.

That’s all there is to it. Very simple, and quite handy if you want to check quickly that files you download don’t have a malware payload. Just make sure you download all files into ~/Downloads/ or they will not be checked automatically. Also, if you are given e.g. a USB pen drive with a file on it, you can copy the file to ~/Downloads/ if you want it to be scanned for malware.

Using an external USB 3.5-inch floppy disk drive in Linux

Back in 2004 I needed to get some files off my old 3.5″ floppy disks, so I bought an external USB floppy disk drive to use with a laptop running Windows XP. The label on the drive gives the manufacturer and model as ‘SmartDisk: FDUSB-TM2, Mitsumi Model #: D353FUE’.

Anyway, today I wanted to throw out some 720KB DD (Double Density) and 1440KB HD (High Density) 3.5″ floppy disks but first needed to check their contents and wipe them. So I dug out the SmartDisk USB drive to see if it would work with the current Gentoo Linux installation on my newest laptop. I was pleased to discover that it does, and below are some notes on how to use it in case anyone else needs to use one of these devices.

Once plugged in to a USB port on my laptop, the lsusb command shows the device has been recognised:

Bus 001 Device 013: ID 03ee:6901 Mitsumi SmartDisk FDD

Note that the Linux floppy driver is not needed for USB floppy disk drives:

root # grep -i CONFIG_BLK_DEV_FD /usr/src/linux/.config
# CONFIG_BLK_DEV_FD is not set

A Linux utility named ufiformat is used to low-level format floppy disks in USB floppy disk drives. A Gentoo Linux ebuild for Version 0.9.9 of ufiformat is listed below, and it can be used in a local overlay under the category sys-fs:

# Copyright 1999-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=5

DESCRIPTION="USB Floppy Disk formatting tool"
HOMEPAGE="http://www.geocities.jp/tedi_world/format_usbfdd_e.html"
SRC_URI="http://www.geocities.jp/tedi_world/${P}.tar.gz"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""

RDEPEND="sys-fs/e2fsprogs"
DEPEND=${RDEPEND}

UPDATE (2 November 2019): As of March 31, 2019, Yahoo! Geocities service has been discontinued, so the ufiformat repository has been moved to GitHub. I have updated the link in the paragraph above accordingly. Therefore, in the ebuild the HOMEPAGE and SRC_URI variables need to be changed – see the update at the end of this blog post.

The ufiformat utility is straightforward to use:

root # ufiformat --help
Usage: ufiformat [OPTION]... [DEVICE]
Format a floppy disk in a USB floppy disk DEVICE.

  -f, --format [SIZE]  specify format capacity SIZE in KB
                       without -f option, the format of the current media will be used
  -V, --verify         verify the medium after formatting
  -F, --force          do not perform any safety checks
  -i, --inquire        show device information, instead of performing format
                       without DEVICE argument, list USB floppy disk devices
  -v, --verbose        show detailed output
  -q, --quiet          suppress minor output
  -h, --help           show this message

To find the device name, use the blkid command before plugging in the USB cable and again after plugging in the USB cable. The extra device listed the second time will be the floppy disk drive. For example, in my case the new line at the end of the blkid output indicated the drive was /dev/sdd:

/dev/sdd: SEC_TYPE="msdos" UUID="BBBA-37AF" TYPE="vfat"

The fdisk command will confirm that the device is the floppy drive:

root # fdisk -l /dev/sdd
Disk /dev/sdd: 720 KiB, 737280 bytes, 1440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Note that /dev/sdd will not be listed in the output of blkid if there is no disk in the floppy drive, although the ls command will list /dev/sdd while the drive is connected to the computer.

Notice that there are no devices /dev/fd0, /dev/fd1, /dev/fd2 and /dev/fd3, etc. This does not matter.

root # ls /dev
audio1           dri      i2c-10   kmem          mapper              nvidiactl  sda1       sequencer2  tty0   tty2   tty30  tty41  tty52  tty63    usbmon3     vcs2    vcsa12
autofs           dsp1     i2c-11   kmsg          mcelog              nvram      sda2       sg0         tty1   tty20  tty31  tty42  tty53  tty7     usbmon4     vcs3    vcsa2
block            fb0      i2c-2    log           mem                 pktcdvd    sda3       sg1         tty10  tty21  tty32  tty43  tty54  tty8     v4l         vcs4    vcsa3
bsg              fd       i2c-3    loop-control  memory_bandwidth    port       sda5       sg2         tty11  tty22  tty33  tty44  tty55  tty9     vboxdrv     vcs5    vcsa4
bus              full     i2c-4    loop0         mixer               ptmx       sda6       sg3         tty12  tty23  tty34  tty45  tty56  ttyS0    vboxdrvu    vcs6    vcsa5
char             fuse     i2c-5    loop1         mixer1              pts        sda7       shm         tty13  tty24  tty35  tty46  tty57  ttyS1    vboxnetctl  vcs7    vcsa6
console          hidraw0  i2c-6    loop2         mqueue              random     sdb        snapshot    tty14  tty25  tty36  tty47  tty58  ttyS2    vboxusb     vcs8    vcsa7
core             hidraw1  i2c-7    loop3         network_latency     rfkill     sdb1       snd         tty15  tty26  tty37  tty48  tty59  ttyS3    vcs         vcs9    vcsa8
cpu              hidraw2  i2c-8    loop4         network_throughput  root       sdc        stderr      tty16  tty27  tty38  tty49  tty6   urandom  vcs1        vcsa    vcsa9
cpu_dma_latency  hpet     i2c-9    loop5         null                rtc        sdc1       stdin       tty17  tty28  tty39  tty5   tty60  usbmon0  vcs10       vcsa1   vga_arbiter
cuse             i2c-0    initctl  loop6         nvidia-modeset      rtc0       sdd        stdout      tty18  tty29  tty4   tty50  tty61  usbmon1  vcs11       vcsa10  video0
disk             i2c-1    input    loop7         nvidia0             sda        sequencer  tty         tty19  tty3   tty40  tty51  tty62  usbmon2  vcs12       vcsa11  zero
root # ls -la /dev/fd
lrwxrwxrwx 1 root root 13 Jan 17 02:13 /dev/fd -> /proc/self/fd
root # ls -la /proc/self/fd
total 0
dr-x------ 2 root root  0 Jan 17 04:26 .
dr-xr-xr-x 8 root root  0 Jan 17 04:26 ..
lrwx------ 1 root root 64 Jan 17 04:26 0 -> /dev/pts/1
lrwx------ 1 root root 64 Jan 17 04:26 1 -> /dev/pts/1
lrwx------ 1 root root 64 Jan 17 04:26 2 -> /dev/pts/1
lr-x------ 1 root root 64 Jan 17 04:26 3 -> /proc/17669/fd

To format an HD floppy disk with the FAT file system, I did the following:

root # ufiformat -f 1440 /dev/sdd
geometry: track=80, head=2, sector=18, block=512
done                                   
root # /usr/sbin/mkfs.vfat /dev/sdd
mkfs.fat 4.0 (2016-05-06)
attribute "partition" not found
root # ufiformat -i /dev/sdd
vendor:  MITSUMI
product: USB FDD
write protect: off
media type: 2HD
status      block size   kb
formatted    2880  512 1440
formattable  2880  512 1440
formattable  1232 1024 1232
formattable  2400  512 1200

To format a DD floppy disk with the FAT file system, I did the following:

root # ufiformat -f 720 /dev/sdd
geometry: track=80, head=2, sector=9, block=512
done                                   
root # /usr/sbin/mkfs.vfat /dev/sdd
mkfs.fat 4.0 (2016-05-06)
attribute "partition" not found
root # ufiformat -i /dev/sdd
vendor:  MITSUMI
product: USB FDD
write protect: off
media type: 2DD
status      block size   kb
formatted    1440  512  720
formattable  1440  512  720

I use the KDE Desktop Environment. The Device Notifier widget in the System Tray shows the drive and — once a formatted floppy disk is in the drive — it is possible to use the Device Notifier to mount, open and unmount the floppy disk. However, it is also possible to use the command line:

root # mkdir /mnt/floppy
root # mount /dev/sdd /mnt/floppy
root # ls /mnt/floppy
root # cp /test.txt /mnt/floppy/
root # ls /mnt/floppy
test.txt
root # umount /dev/sdd

Earlier in this post I showed examples of formatting floppy disks using the FAT file system, but it is of course possible to format them using other file systems, such as:

root # mkfs.ext2 /dev/sdd
mke2fs 1.43.3 (04-Sep-2016)
/dev/sdd contains a vfat file system
Proceed anyway? (y,n) y
Creating filesystem with 720 1k blocks and 96 inodes

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

Anyway, I was able to check the contents of the floppies and wipe them before disposing of them. It’s good to know that some old technologies can still be used when needs be. I won’t be throwing out the old floppy disk drive just yet.

UPDATE (2 November 2019): As of March 31, 2019, Yahoo! Geocities service has been discontinued, so the ufiformat repository has been moved to GitHub. Therefore, in the ebuild the HOMEPAGE and SRC_URI variables need to be changed from:

HOMEPAGE="http://www.geocities.jp/tedi_world/format_usbfdd_e.html"
SRC_URI="http://www.geocities.jp/tedi_world/${P}.tar.gz"

to:

HOMEPAGE="https://github.com/tedigh/ufiformat/"
SRC_URI="https://github.com/tedigh/ufiformat/archive/v0.9.9.zip"

or:

HOMEPAGE="https://github.com/tedigh/ufiformat/"
SRC_URI="https://github.com/tedigh/ufiformat/archive/v{$PV}.zip"