Installing the Onboard on-screen keyboard in Gentoo Linux

Onboard on-screen keyboard configured with the British English layout, Droid theme and international character selection enabled for the long-press action

The most sophisticated and polished virtual keyboard I have seen so far in Linux is Onboard, the on-screen keyboard previously provided in Ubuntu prior to the switch to GNOME 3. The current version of Onboard is 1.4.1 and it can be installed and used in other Linux distributions and desktop environments. Thanks to Gentoo Linux user wjn an ebuild is available in the wjn-overlay overlay and can be installed from there via Portage. However, I prefer to use a local overlay so I copied the ebuild to my local overlay and installed it from there instead. Here is what I did to install app-accessibility/onboard in the Gentoo Stable installation running KDE Plasma 5 on my Clevo W230SS laptop:

1. I first made sure the preferred Python interpreter was selected (I should have done that when the Gentoo Linux developers recently replaced Python 3.5 with Python 3.6 in the default Python targets for Gentoo installations):

root # grep PYTHON /etc/portage/make.conf
PYTHON_TARGETS="python2_7 python3_6"
PYTHON_SINGLE_TARGET="python3_6"
root # eselect python list
Available Python interpreters, in order of preference:
  [1]   python2.7
  [2]   python3.5 (uninstalled)
  [3]   python3.4
  [4]   python3.6 (fallback)
root # eselect python set 4
root # eselect python list 
Available Python interpreters, in order of preference:
  [1]   python3.6
  [2]   python2.7
  [3]   python3.5 (uninstalled)
  [4]   python3.4

2. Then I copied the ebuild to my local overlay:

root # mkdir -p /usr/local/portage/app-accessibility/onboard/files
root # cd /usr/local/portage/app-accessibility/onboard/files/
root # wget https://bitbucket.org/wjn/wjn-overlay/raw/5d7fe162af7c0cde9b401a9a30fb3ab8b2b65e3d/app-accessibility/onboard/files/# onboard-1.4.1-remove-duplicated-docs.patch
root # cd ..
root # wget wget https://bitbucket.org/wjn/wjn-overlay/raw/5d7fe162af7c0cde9b401a9a30fb3ab8b2b65e3d/app-accessibility/onboard/onboard-1.4.1.ebuild
root # ebuild onboard-1.4.1.ebuild manifest

3. As I am using using Gentoo Stable I unmasked the ebuild by keyword:

root # nano /etc/portage/package.accept_keywords/onboard
root # cat /etc/portage/package.accept_keywords/onboard
=app-accessibility/onboard-1.4.1 **

4. Then I installed the package:

root # emerge onboard
root # eix onboard
[I] app-accessibility/onboard [1]
     Available versions:  (~)1.4.1^m {PYTHON_TARGETS="python3_5 python3_6"}
     Installed versions:  1.4.1^m(15:28:57 25/06/18)(PYTHON_TARGETS="python3_6 -python3_4 -python3_5")
     Homepage:            https://launchpad.net/onboard
     Description:         Onscreen keyboard for everybody who can't use a hardware keyboard

[1] "local_overlay" /usr/local/portage

Icons for Onboard and Onboard Settings were added to the KDE Application Launcher’s menu (Applications > Utilities) and they can be launched from there or by entering the command ‘onboard‘ in a Konsole window under the user’s account.

5. The only thing that did not work ‘out of the box’ in KDE Plasma 5.12.5 in Gentoo was selecting ‘Help’ from the pop-up menu displayed by clicking on the Onboard icon on the Plasma 5 Panel:

FileNotFoundError: [Errno 2] No such file or directory: ‘/usr/bin/yelp’: ‘/usr/bin/yelp’

This was simply because the package gnome-extra/yelp had not been installed in my KDE installation. Now, I could have just installed it separately:

root # emerge yelp

but I chose intead to edit the onboard ebuild to add yelp to the list of runtime dependencies:

RDEPEND="${COMMON_DEPEND}
        app-accessibility/at-spi2-core
        app-text/iso-codes
        gnome-extra/mousetweaks
        gnome-extra/yelp
        x11-libs/libxkbfile"

and I then re-installed the package, which then automatically installed yelp and its dependencies:

root # ebuild onboard-1.4.1.ebuild manifest
root # emerge onboard

Onboard is a nice utility, and I hope its developers continue to maintain and develop it even though Ubuntu now uses the GNOME 3 on-screen keyboard instead, as it can be used in other desktop environments and in other Linux distributions.

Advertisements

Trouble again with PulseAudio and Thunderbird sound notifications

In an earlier post I described how I fixed a scratchy-sounding sound file which the Thunderbird e-mail client plays when a new message arrives. Well, the problem started again recently, but this time the contents of /etc/pulse/daemon.conf looked OK to me. Furthermore, the sound file sounds fine when played using following commands:

aplay ~/Music/wav/E-mail_notifications/halmsg.wav
paplay ~/Music/wav/E-mail_notifications/halmsg.wav
mplayer ~/Music/wav/E-mail_notifications/halmsg.wav
cvlc ~/Music/wav/E-mail_notifications/halmsg.wav

Now, Thunderbird uses libcanberra to play sounds, so I began to wonder if the problem lay with libcanberra. As it happens, libcanberra is maintained by the same person who invented PulseAudio. However, I notice from the libcanberra Git repository that its source code has not been changed since 2012.

My Gentoo Linux installation had libcanberra installed with support for both ALSA and PulseAudio:

root # eix -I libcanberra
[I] media-libs/libcanberra
     Available versions:  0.30-r5 {alsa gnome gstreamer +gtk +gtk3 oss pulseaudio +sound tdb udev ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  0.30-r5(08:27:41 18/05/18)(alsa gtk gtk3 pulseaudio sound udev -gnome -gstreamer -oss -tdb ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://git.0pointer.net/libcanberra.git/
     Description:         Portable sound event library

So, even though my installation uses PulseAudio, I decided to try and re-install libcanberra without PulseAudio support, only ALSA support:

root # USE="-pulseaudio" emerge -1v libcanberra
root # eix -I libcanberra
[I] media-libs/libcanberra
     Available versions:  0.30-r5 {alsa gnome gstreamer +gtk +gtk3 oss pulseaudio +sound tdb udev ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  0.30-r5(15:47:14 26/05/18)(alsa gtk gtk3 sound udev -gnome -gstreamer -oss -pulseaudio -tdb ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://git.0pointer.net/libcanberra.git/
     Description:         Portable sound event library

Lo and behold, Thunderbird (libcanberra) plays the sound file correctly now. So I have added the following line to my file /etc/portage/package.use/thunderbird in order to make the change permanent:

media-libs/libcanberra -pulseaudio

PulseAudio 🙄

Gentoo Linux: A work-around to be able to Resume from Suspend to RAM when using the NVIDIA closed-source driver

My Clevo W230SS laptop has NVIDIA Optimus graphics hardware (NVIDIA GPU plus Intel IGP). I do not use Bumblebee, preferring to switch between the Intel video driver and the NVIDIA closed-source driver myself (see Switching between Intel and NVIDIA graphics processors on a laptop with NVIDIA Optimus hardware running Gentoo Linux). The laptop can suspend to RAM and resume perfectly when using the Intel video driver (but see Stopping my laptop spontaneously resuming immediately after Suspend to RAM, which is applicable whatever the GPU or IGP).

In order to be able to resume properly from Suspend-to-RAM when using the NVIDIA driver, the laptop needs to disable compositing before suspending, then re-enable compositing after resuming. For how I achieve that, see under Problem 2 in the third link above. If this is not done, the graphics on the Desktop are corrupted after resuming.

However, recently when using the NVIDIA driver and KDE Plasma 5 (I am currently using nvidia-drivers-387.22 and plasma-meta-5.11.5), when resuming from suspension the monitor would briefly display the LightDM wallpaper (I use different wallpapers for the desktop manager and the lock screen, so I know it was not the KDE lock screen) followed by a blank screen with a mouse pointer (which I could move normally). More recently, in between displaying the desktop manager’s wallpaper and the blank screen, the monitor would briefly display an earlier image of the Desktop just before the laptop suspended.

Now, I could simply leave the laptop configured to use the Intel driver. However, sometimes I need to use a CAD application and the performance is better when using the NVIDIA GPU.

There are umpteen posts on the Web about this problem, and the root cause seems to be the closed-source NVIDIA driver. I have seen the KDE lock screen mentioned in some posts as the culprit, so I disabled the lock screen (‘System Settings’ > ‘Desktop Behaviour’ > ‘Screen Locking’) but that did not solve the problem.

I put up with this for several weeks in the hope that the next release of the NVIDIA driver would fix the problem. If I suspended to RAM while the laptop was using the NVIDIA driver, I was able to resume and get to a working Desktop – albeit without the open windows and applications that had been running before suspending – by pressing Ctrl+Alt+F1 to get to TTY1, logging in as the root user and entering the command ‘/etc/init.d/xdm restart‘. However, the final straw was in a meeting a couple of weeks ago when I wanted to resume the laptop and show a worksheet to someone. The laptop monitor of course displayed a blank screen with a mouse pointer, and it took me a couple of minutes to restart the desktop manager, login to KDE Plasma 5 and open the spreadsheet again. So this week I decided to look into the problem to see if I could at least find a work-around that would enable the laptop to resume without needing to restart X Windows and login to Plasma 5 each time.

I created a Bash script in /etc/pm/sleep.d/ to unload the NVIDIA modules before suspending to RAM and to re-load them when resuming, but that did not solve the problem either.

I switched the rendering background from OpenGL 2.0 to OpenGL 3.1 (‘System Settings’ > ‘Display and Monitor’ > ‘Compositor’), but that did not work either. I switched the rendering backend to XRender, and that did enable the laptop to resume from suspend successfully with the NVIDIA driver, but I do not want to use that work-around. Firstly, with software rendering there is a performance hit, and, secondly, there was no KDE Desktop Cube when using XRender instead of OpenGL. I use the Desktop Cube when working, as I often have a lot of windows open on each virtual desktop (cube side), and I find it easier to use the cube than a flat UI.

Eventually I found that, after resuming, if I pressed Ctrl+Alt+F1 to get to a virtual console, logged into my user account, entered the command ‘DISPLAY=:0 /usr/bin/kwin_x11 --resume‘ and then pressed Ctrl+Alt+F7 to get back to TTY7, my Desktop would appear on TTY7. Even so, I noticed on TTY1 that the following error messages were displayed when I ran that command:

kwin_core: OpenGL 2 compositing setup failed
kwin_core: Failed to initialize compositing, compositing disabled

Anyway, the Plasma 5 Desktop was displayed on TTY7, and with the windows that were open when I suspended the laptop, so restarting KWin would at least be a viable work-around until NVIDIA fix their video driver.

I incoporated the command in my script /etc/pm/sleep.d/02-toggle-compositing like so:

#!/bin/sh
#
# Turn off compositing on hibernate or suspend
# Turn on compositing on thaw or resume

username=fitzcarraldo
userhome=/home/$username
export XAUTHORITY="$userhome/.Xauthority"
export DISPLAY=":0"

case "$1" in
     suspend|hibernate)
          su $username -c "qdbus org.kde.KWin /Compositor suspend" &
     ;;
     resume|thaw)
          su $username -c "qdbus org.kde.KWin /Compositor resume" &
          su $username -c "/usr/bin/kwin_x11 --replace" &
     ;;
     *)
          exit $NA
     ;;
esac

It is an ugly hack, but at least now the laptop can resume properly from Suspend-to-RAM while the NVIDIA driver is being used.

Perhaps Linus Torvalds was correct. I will try to avoid NVIDIA hardware when I replace my current laptop.

Running a shell script at shutdown only (not at reboot) – a comparison between OpenRC and systemd

Gentoo Linux on my laptops uses OpenRC with SysVinit, whereas Lubuntu 17.10 on my family’s PC uses systemd. I have had to configure both Linux distributions to run a backup job at shutdown, so I thought it would be interesting to summarise the two approaches.

OpenRC

Create a Bash script /etc/local.d/10-run_on_shutdown.stop with the following contents:

#!/bin/bash
if [ `who -r | awk '{print $2}'` = "0" ]; then
    ########################################################################
    # Put Bash commands here to be executed on shutdown but not on reboot. #
    # For example, backup home directories to an external USB HDD.         #
    ########################################################################
fi

From now on the script will run to completion when you shutdown the machine, but not when you reboot it.

systemd

1. Create a Bash script /usr/local/sbin/run_on_shutdown.sh with the following contents:

#!/bin/bash
REBOOT=$( systemctl list-jobs | egrep -q 'reboot.target.*start' && echo "rebooting" || echo "not_rebooting" )
if [ $REBOOT = "not_rebooting" ]; then
    ########################################################################
    # Put Bash commands here to be executed on shutdown but not on reboot. #
    # For example, backup home directories to an external USB HDD.         #
    ########################################################################
fi

2. Create a unit file /etc/systemd/system/run_on_shutdown.service with the following contents:

[Unit]
Description=Run a Bash script at shutdown
DefaultDependencies=no
Before=shutdown.target halt.target
# If your script requires any mounted directories, add them below: 
RequiresMountsFor=/home

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/run_on_shutdown.sh

[Install]
WantedBy=halt.target shutdown.target

I have assumed the Bash script is to backup /home, so change the directory list in ‘RequiresMountsFor=‘ if the script necessitates that other directories are still mounted – see ‘man systemd.unit‘ for details.

3. Enable the unit file and start the service as follows:

user $ sudo systemctl enable run_on_shutdown.service
user $ sudo reboot

From now on, the script will run to completion when you shutdown the machine, but not when you reboot it.

Notes on the systemd solution:

There are plenty of posts on the Web suggesting how to run a script at shutdown in installations that use systemd, but the approaches either do not work in my case or they do not discriminate between shutting down and rebooting. For example, if you are wondering why I do not use ‘Conflicts=reboot.target‘ in the unit file, I found it does not prevent the Bash script from being launched when the system is rebooted (see the entry for ‘Conflicts=‘ in ‘man systemd.unit‘ for the functionality). If you are wondering why I do not simply place a script in /lib/systemd/system-shutdown/, it is because scripts in that directory are launched late in the shutdown process, after file systems have been mounted read-only. And if you are wondering why the Bash script tests if the system is rebooting (as distinct from shutting down) rather than the unit file being configured to do that, it is because the systemd shutdown target is active in any type of system termination, be it rebooting or halting, and it therefore does not help in discriminating between the termination types (see ‘man systemd.special‘). I tried several approaches in the unit file to see if it could be made to launch a Bash script when shutting down but not when rebooting, but the only solution that works for me in the Lubuntu 17.10 installation on my family’s PC uses the two files listed above working in conjunction with each other.

Furthermore, of all the unit files I found on the Web that actually make systemd launch a Bash script when a user initiates shutdown, only one of them prevents systemd from shutting down the machine before a time-consuming script has run to completion (for example a script to backup to an external HDD).

Some background reading

1. Easy Systemd Startup and Shutdown Scripts for Ubuntu 16.04 and Mint 18
2. Stack Exchange Super User : How do I run a script before everything else on shutdown with systemd?
3. Stack Overflow : how can a systemd controlled service distinguish between shutdown and reboot?
4. Unix & Linux Stack Exchange : Systemd : How to execute script at shutdown only (not at reboot)

Moving to the slotted WINE package system in Gentoo Linux

Earlier this year the Gentoo Linux developers responsible for maintaining the package app-emulation/wine decided to split it and slot it so that different versions of WINE can be installed and co-exist simultaneously:

clevow230ss fitzcarraldo # eselect news read 34
2017-04-10-split-and-slotted-wine
  Title                     app-emulation/wine split and slotting
  Author                    NP-Hardass 
  Posted                    2017-04-10
  Revision                  1

Starting with Wine 2.0, Wine in Gentoo is transitioning away from its
traditional packaging and toward a new, split and slotted, Wine.

As many Wine users know, there are often regressions or an application
works better on one version of wine than another.  Going forward, 
packaging in Gentoo will allow simultaneous installation of multiple
versions of Wine.

Additionally, to expedite vanilla releases as well as permit multiple
configurations for each Wine installation, the major patchsets have
been split out into separate packages.

Going forward, app-emulation/wine will transition to:
app-emulation/wine-vanilla: upstream Wine with no external patchsets
             (like if the old packaging forced USE="-staging -d3d9")
app-emulation/wine-staging: Wine with Wine-Staging's patchset
             (like if the old packaging forced USE="+staging -d3d9")
app-emulation/wine-d3d9: Wine with Ixit's Gallium Nine patchset
             (like if the old packaging forced USE="-staging +d3d9")
app-emulation/wine-any: Wine with any of the patchsets or flags
             (exactly like the old packaging regarding USE flags)

wine-any exists to allow the user to build any combination that they'd
like (like the old packaging).  This means the user could use wine-any
to use both Wine-Staging and Gallium Nine.  Alternatively, the user
could use wine-any to try out another configuration from other
packages.  For example, the user could build wine-vanilla without
PulseAudio, and could build wine-any with PulseAudio.  The sky is the
limit on how a user may choose to use app-emulation/wine-any.

Users may opt for any specific package, or may emerge virtual/wine,
which is provided for dependency resolution.
Maintainers: Please note, app-emulation/wine will be dropped, so
please use virtual/wine going forward.

Users may call each version specifically, or may call a symlink based
on their installed patchset, for example wine-2.1, wine-staging-2.2,
or wine-d3d9.

Symlinks for wine are managed with app-eselect/eselect-wine.
# eselect wine set wine-vanilla-2.0
/usr/bin/wine -> /usr/bin/wine-vanilla-2.0
# eselect wine set --staging wine-staging-2.4
/usr/bin/wine-staging -> /usr/bin/wine-staging-2.4

Earlier this year the Gentoo Linux Forums thread ‘wine: questions on recent changes‘ discussed the new system and how to use it. Several users, myself included, posted questions in that thread asking how to go about the change. I received differing advice and remained uncertain about what to do. At the time, the slotted packages for the new system had not yet been unmasked, so I left my installation as it was and decided to put off making the change for as long as possible. Today when I synchronised my installation with the Portage tree there was a Gentoo Linux news item ‘2017-11-21 Old Wine versions moving to wine-overlay’. So the time had come for me to make the switch from app-emulation/wine-2.3 to the new slotted WINE package system. Here is what I did…

Previous situation

I had been using WINE Staging:

clevow230ss fitzcarraldo # eix -I wine
[?] app-emulation/wine
     Available versions:  [M]2.0^t [M](~)2.1^t [M](~)2.2^t [M](~)2.3^t [M]**9999^t {+X +alsa capi cups custom-cflags d3d9 dos +fontconfig +gecko gphoto2 gsm gstreamer +jpeg +lcms ldap +mono mp3 ncurses netapi nls odbc openal opencl +opengl osmesa oss pcap +perl pipelight +png prelink pulseaudio +realtime +run-exes s3tc samba scanner selinux +ssl staging test themes +threads +truetype udev +udisks v4l vaapi +xcomposite xinerama +xml ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="(+)32 (+)64 x32" ELIBC="glibc" KERNEL="FreeBSD" LINGUAS="ar bg ca cs da de el en en_US eo es fa fi fr he hi hr hu it ja ko lt ml nb_NO nl or pa pl pt_BR pt_PT rm ro ru sk sl sr_RS@cyrillic sr_RS@latin sv te th tr uk wa zh_CN zh_TW"}
     Installed versions:  2.3^t(19:16:31 20/05/17)(X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl staging threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -d3d9 -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba -selinux -test -themes -vaapi -xinerama ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32" ELIBC="glibc" KERNEL="-FreeBSD" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW")
     Homepage:            https://www.winehq.org/
     Description:         Free implementation of Windows(tm) on Unix

[I] app-emulation/winetricks
     Available versions:  20170823^t **99999999^t {gtk kde rar}
     Installed versions:  20170823^t(02:50:56 06/09/17)(gtk kde -rar)
     Homepage:            https://github.com/Winetricks/winetricks https://wiki.winehq.org/Winetricks
     Description:         Easy way to install DLLs needed to work around problems in Wine

Found 2 matches

The file /etc/portage/package.accept_keywords/wine in my installation included the following line:

app-emulation/wine ~amd64

The file /etc/portage/package.use/wine in my installation included the following line:

app-emulation/wine -ldap -pipelight staging abi_x86_32

New situation

The file /etc/portage/package.accept_keywords/wine in my installation now includes the lines listed below. The reason why I had to include app-emulation/wine-vanilla is explained further on.

virtual/wine ~amd64
app-emulation/wine-staging ~amd64
app-emulation/wine-vanilla ~amd64

The file /etc/portage/package.use/wine in my installation now includes the lines listed below. The reason why I had to include app-emulation/wine-vanilla is explained further on.

virtual/wine -d3d9 staging abi_x86_32
app-emulation/wine-staging X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl staging threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -d3d9 -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba -selinux -test -themes -vaapi -xinerama abi_x86_32
app-emulation/wine-vanilla X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl staging threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -d3d9 -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba -selinux -test -themes -vaapi -xinerama abi_x86_32

(Note that e.g. the pipelight USE flag does not actually exist in the package app-emulation/wine-vanilla, but it does no harm to include “-pipelight” in the line for app-emulation/wine-vanilla in the file package.use.)

After adding the above-mentioned lines to the two files, I then had to uninstall the masked package app-emulation/wine-2.3 and the package app-emulation/winetricks-20170823 that depended on it:

clevow230ss fitzcarraldo # emerge -aC app-emulation/wine app-emulation/winetricks

Then I had to install the package virtual/wine, which pulled in the package app-emulation/wine-staging as I wanted, but also pulled in the package app-emulation/vanilla:

clevow230ss fitzcarraldo # emerge -a virtual/wine

Then I had to select wine-staging:

clevow230ss fitzcarraldo # eselect wine list
clevow230ss fitzcarraldo # eselect wine set

Finally, I had to install winetricks, which would now use the selected slotted package:

clevow230ss fitzcarraldo # emerge winetricks

How I arrived at the contents of package.accept_keywords and package.use

Note that, before I added the line for app-emulation/wine-vanilla to /etc/portage/package.use/wine and /etc/portage/package.accept_keywords/wine, this is what Portage wanted to install:

clevow230ss fitzcarraldo # emerge -p virtual/wine

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-emulation/wine-gecko-2.47-r1  ABI_X86="32 (64)" 
[ebuild  N     ] app-emulation/wine-desktop-common-20150204 
[ebuild  N     ] app-eselect/eselect-wine-1.2.2 
[ebuild   R    ] net-nds/openldap-2.4.44  ABI_X86="32*" 
[ebuild  N     ] app-emulation/wine-vanilla-2.0.2  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms ldap mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) {-test} -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] app-emulation/wine-staging-2.19  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl (staging) threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba (-selinux) {-test} -themes -vaapi -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] virtual/wine-0-r6  USE="staging -d3d9" ABI_X86="32 64" 

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by app-emulation/wine-vanilla-2.0.2::gentoo[ldap]
# required by virtual/wine-0-r6::gentoo
# required by virtual/wine (argument)
>=net-nds/openldap-2.4.44 abi_x86_32
# required by app-emulation/wine-vanilla-2.0.2::gentoo[gecko]
# required by virtual/wine-0-r6::gentoo
# required by virtual/wine (argument)
>=app-emulation/wine-gecko-2.47-r1 abi_x86_32

!!! The following installed packages are masked:
- dev-qt/qtwebkit-4.8.7::gentoo (masked by: package.mask)
/usr/portage/profiles/package.mask:
# Andreas Sturmlechner  (16 Nov 2017)
# Qt4WebKit is ancient and full of security holes.
# Masked for removal in 30 days. Bug #620684

For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

clevow230ss fitzcarraldo #

After I added a line for app-emulation/wine-vanilla to /etc/portage/package.use/wine and /etc/portage/package.accept_keywords/wine, this is what Portage wanted to do:

clevow230ss fitzcarraldo # emerge -p virtual/wine                        

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-emulation/wine-gecko-2.47-r1  ABI_X86="32 (64)" 
[ebuild  N     ] app-eselect/eselect-wine-1.2.2 
[ebuild  N     ] app-emulation/wine-desktop-common-20150204 
[ebuild  N    ~] app-emulation/wine-vanilla-2.20  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -kerberos -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) {-test} -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] app-emulation/wine-staging-2.19  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl (staging) threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba (-selinux) {-test} -themes -vaapi -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] virtual/wine-0-r6  USE="staging -d3d9" ABI_X86="32 64" 

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by app-emulation/wine-vanilla-2.20::gentoo[gecko]
# required by virtual/wine-0-r6::gentoo
# required by virtual/wine (argument)
>=app-emulation/wine-gecko-2.47-r1 abi_x86_32
clevow230ss fitzcarraldo #

So I added the above-mentioned USE change for app-emulation/wine-gecko to /etc/portage/package.use/wine, and then Portage wanted to do the following:

clevow230ss fitzcarraldo # emerge -p virtual/wine

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-emulation/wine-gecko-2.47-r1  ABI_X86="32 (64)" 
[ebuild  N     ] app-eselect/eselect-wine-1.2.2 
[ebuild  N     ] app-emulation/wine-desktop-common-20150204 
[ebuild  N    ~] app-emulation/wine-vanilla-2.20  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -kerberos -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) {-test} -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] app-emulation/wine-staging-2.19  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl (staging) threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba (-selinux) {-test} -themes -vaapi -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] virtual/wine-0-r6  USE="staging -d3d9" ABI_X86="32 64" 
clevow230ss fitzcarraldo #

I was happy with that, so I went ahead without the --pretend option:

clevow230ss fitzcarraldo # emerge -a virtual/wine

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] app-emulation/wine-gecko-2.47-r1  ABI_X86="32 (64)" 
[ebuild  N     ] app-eselect/eselect-wine-1.2.2 
[ebuild  N     ] app-emulation/wine-desktop-common-20150204 
[ebuild  N    ~] app-emulation/wine-vanilla-2.20  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -kerberos -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -prelink -samba (-selinux) {-test} -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] app-emulation/wine-staging-2.19  USE="X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl (staging) threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba (-selinux) {-test} -themes -vaapi -xinerama" ABI_X86="32 64 (-x32)" LINGUAS="en pt_BR -ar -bg -ca -cs -da -de -el -en_US -eo -es -fa -fi -fr -he -hi -hr -hu -it -ja -ko -lt -ml -nb_NO -nl -or -pa -pl -pt_PT -rm -ro -ru -sk -sl -sr_RS@cyrillic -sr_RS@latin -sv -te -th -tr -uk -wa -zh_CN -zh_TW" 
[ebuild  N    ~] virtual/wine-0-r6  USE="staging -d3d9" ABI_X86="32 64" 

Would you like to merge these packages? [Yes/No] Yes
>>> Verifying ebuild manifests
>>> Running pre-merge checks for app-emulation/wine-vanilla-2.20
>>> Running pre-merge checks for app-emulation/wine-staging-2.19
>>> Emerging (1 of 6) app-emulation/wine-gecko-2.47-r1::gentoo
>>> Emerging (2 of 6) app-eselect/eselect-wine-1.2.2::gentoo
>>> Installing (1 of 6) app-emulation/wine-gecko-2.47-r1::gentoo
>>> Installing (2 of 6) app-eselect/eselect-wine-1.2.2::gentoo
>>> Emerging (3 of 6) app-emulation/wine-desktop-common-20150204::gentoo
>>> Installing (3 of 6) app-emulation/wine-desktop-common-20150204::gentoo
>>> Emerging (4 of 6) app-emulation/wine-vanilla-2.20::gentoo
>>> Emerging (5 of 6) app-emulation/wine-staging-2.19::gentoo
>>> Installing (4 of 6) app-emulation/wine-vanilla-2.20::gentoo
>>> Installing (5 of 6) app-emulation/wine-staging-2.19::gentoo
>>> Emerging (6 of 6) virtual/wine-0-r6::gentoo
>>> Installing (6 of 6) virtual/wine-0-r6::gentoo
>>> Recording virtual/wine in "world" favorites file...
>>> Jobs: 6 of 6 complete                           Load avg: 6.80, 7.34, 7.06
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.
clevow230ss fitzcarraldo # eselect wine list
Available wine versions:
  [1]   wine-staging-2.19
  [2]   wine-vanilla-2.20 *
clevow230ss fitzcarraldo # eselect wine set 1
clevow230ss fitzcarraldo # eselect wine list
Available wine versions:
  [1]   wine-staging-2.19 *
  [2]   wine-vanilla-2.20
clevow230ss fitzcarraldo # emerge app-emulation/winetricks
Calculating dependencies... done!
>>> Verifying ebuild manifests
>>> Emerging (1 of 1) app-emulation/winetricks-20170823::gentoo
>>> Installing (1 of 1) app-emulation/winetricks-20170823::gentoo
>>> Recording app-emulation/winetricks in "world" favorites file...
>>> Jobs: 1 of 1 complete                           Load avg: 0.51, 2.78, 5.09
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.
clevow230ss fitzcarraldo #

Summary

So, in summary, this is what I did in order to install and use app-emulation/wine-staging:

1. Add the following lines to /etc/portage/package.accept_keywords/wine:

virtual/wine ~amd64
app-emulation/wine-staging ~amd64
app-emulation/wine-vanilla ~amd64

2. Add the following lines to /etc/portage/package.use/wine:

virtual/wine -d3d9 staging abi_x86_32
app-emulation/wine-staging X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl staging threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -d3d9 -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba -selinux -test -themes -vaapi -xinerama abi_x86_32
app-emulation/wine-vanilla X alsa cups fontconfig gecko gphoto2 gsm jpeg lcms mp3 ncurses nls openal opengl perl png pulseaudio realtime run-exes scanner ssl staging threads truetype udev udisks v4l xcomposite xml -capi -custom-cflags -d3d9 -dos -gstreamer -ldap -mono -netapi -odbc -opencl -osmesa -oss -pcap -pipelight -prelink -s3tc -samba -selinux -test -themes -vaapi -xinerama abi_x86_32
# required by app-emulation/wine-vanilla-2.20::gentoo[gecko]
# required by virtual/wine-0-r6::gentoo
>=app-emulation/wine-gecko-2.47-r1 abi_x86_32

3. Uninstall the old, un-slotted WINE packages:

root # emerge -aC app-emulation/wine app-emulation/winetricks

4. Merge the new virtual WINE package:

root # emerge -a virtual/wine

5. Select the slotted WINE package I wish to use (WINE Staging):

root # eselect wine list
root # eselect wine set wine-staging-2.19

6. Merge the winetricks package, which will now recognise the slotted WINE package selected:

root # emerge app-emulation/winetricks

After completing the above process, I was still able to launch as before the various Microsoft Windows applications installed on my laptop under WINE.

UPDATE (December 1, 2017): As explained by user Chiltoo in a new post in the Gentoo Linux Forums thread mentioned earlier in the above post, a bug in Portage Version 2.3.13-r1 results in app-emulation/wine-vanilla being installed unnecessarily, as borne out in my post above.

The work-around he proposed does indeed work for me. Below are the commands I have just used for the work-around (I have left the new entries I made in package.accept_keywords and package.use in those files):

clevow230ss fitzcarraldo # emerge -aC app-emulation/wine-vanilla app-emulation/wine-staging virtual/wine app-emulation/wine-desktop-common app-emulation/wine-gecko app-eselect/eselect-wine app-emulation/winetricks
clevow230ss fitzcarraldo # emerge -1 app-emulation/wine-staging && emerge virtual/wine && emerge winetricks

The outcome of the above two commands in my case is:

clevow230ss fitzcarraldo # eix --installed --compact wine
[I] app-emulation/wine-desktop-common (20150204@01/12/17): Various desktop menu items and icons for wine
[I] app-emulation/wine-gecko (2.47-r1(2.47)@01/12/17): A Mozilla Gecko based version of Internet Explorer for Wine
[I] app-emulation/wine-staging (2.19(2.19)@01/12/17): Free implementation of Windows(tm) on Unix, with Wine-Staging patchset
[I] app-emulation/winetricks (20170823@01/12/17): Easy way to install DLLs needed to work around problems in Wine
[I] app-eselect/eselect-wine (1.2.2@01/12/17): Manage active wine version
[I] virtual/wine (0-r6@01/12/17): Virtual for Wine that supports multiple variants and slotting
Found 6 matches
clevow230ss fitzcarraldo # eselect wine list
Available wine versions:
  [1]   wine-staging-2.19 *
clevow230ss fitzcarraldo #

The superfluous app-emulation/wine-vanilla is no longer installed, which is a cleaner outcome.

Prevent Linux firewalls interfering with Samba commands in a home network that uses broadcast NetBIOS name resolution

Or “How come devices in a home network can browse SMB shares but Linux Samba commands and Windows nbtstat commands do not work properly?”

Introduction

In a previous post I explained how it is possible to browse SMB shares when using broadcast NetBIOS name resolution in a home network consisting of machines running Linux, Windows and other operating systems. Browsing SMB/Samba shares will work as expected, but Samba commands such as ‘smbtree‘, ‘smbclient‘ and ‘nmblookup‘ will not work properly if the Linux machines use a firewall that has not been configured for broadcast NetBIOS name resolution. This post is to explain how to do that.

If broadcast NetBIOS name resolution is being used and none of the Linux machines has a firewall enabled, or if their firewalls have been correctly configured, the output of e.g. the ‘smbtree‘ command on one of those machines would look something like the example below.

anne@akhanaten:~$ smbtree
Enter anne's password: 
HOME
        \\AKHANATEN                     Samba 4.3.11-Ubuntu
                \\AKHANATEN\IPC$                IPC Service (Samba 4.3.11-Ubuntu)
                \\AKHANATEN\guest               guest account
                \\AKHANATEN\matthew             matthew share
                \\AKHANATEN\marilla             marilla share
                \\AKHANATEN\anne                anne share
        \\TUTANKHAMUN                   Samba 4.5.10
                \\TUTANKHAMUN\Samsung_Xpress_C460FW     Samsung Xpress C460FW
                \\TUTANKHAMUN\Canon_MP560_Printer       Canon PIXMA MP560
                \\TUTANKHAMUN\Canon_MP510_Printer       Canon PIXMA MP510
                \\TUTANKHAMUN\Virtual_PDF_Printer       Virtual PDF Printer
                \\TUTANKHAMUN\IPC$              IPC Service (Samba 4.2.11)
                \\TUTANKHAMUN\Public
                \\TUTANKHAMUN\anne-share
                \\TUTANKHAMUN\print$
                \\TUTANKHAMUN\netlogon          Network Logon Service
        \\BTHUB5                        BT Home Hub 5.0A File Server
                \\BTHUB5\IPC$                   IPC Service (BT Home Hub 5.0A File Server)
        \\THUTMOSEIII                   Windows 10 computer

If Linux firewalls have not been correctly configured, the output would be missing some information about other machines in the network. For example, compare the output above with the output below from the same network, this time with the Linux firewalls configured using typical rules for Samba specified in Web articles, blog posts and forums.

anne@akhanaten:~$ smbtree
Enter anne's password: 
HOME
        \\AKHANATEN                     Samba 4.3.11-Ubuntu
                \\AKHANATEN\IPC$                IPC Service (Samba 4.3.11-Ubuntu)
                \\AKHANATEN\guest               guest account
                \\AKHANATEN\matthew             matthew share
                \\AKHANATEN\marilla             marilla share
                \\AKHANATEN\anne                anne share
        \\TUTANKHAMUN                   Samba 4.5.10
        \\BTHUB5                        BT Home Hub 5.0A File Server
        \\THUTMOSEIII                   Windows 10 computer

To avoid this problem you need to add a further Linux firewall rule to the set of rules usually used for Samba. Below I first list the usual firewall rules for Samba, then I give the additional rule necessary if using broadcast NetBIOS name resolution. In each case I give the applicable rules for a pure IPTABLES firewall and for UFW (Uncomplicated Firewall). The rules listed here assume the IP address range of the home network is 192.168.1.0/24, so change the range to suit the specific network.

Firewall rules typically specified for machines using Samba

IPTABLES

The rules listed below assume the machine uses interface eth0, so change the interface to suit the specific machine.

# NetBIOS Name Service (name resolution)
iptables -A INPUT -i eth0 -p udp --dport 137 -s 192.168.1.0/24 -j ACCEPT

# NetBIOS Datagram Service (BROWSER service)
iptables -A INPUT -i eth0 -p udp --dport 138 -s 192.168.1.0/24 -j ACCEPT

# NetBIOS Session Service (data transfer legacy SMB/NetBIOS/TCP)
iptables -A INPUT -i eth0 -p tcp --dport 139 -s 192.168.1.0/24 -j ACCEPT

# Microsoft Directory Service (data transfer SMB/TCP)
iptables -A INPUT -i eth0 -p tcp --dport 445 -s 192.168.1.0/24 -j ACCEPT

UFW

In some Linux distributions the ufw application allows a single command to add Samba support, such as:

user $ sudo ufw allow Samba

or

user $ sudo ufw allow CIFS

These ‘application profiles’ are specified in files in the directory /etc/ufw/applications.d/, so you could add application profiles or modify existing ones if you wish. In one of my installations the file /etc/ufw/applications.d/ufw-fileserver includes the following application profile for Samba, for example:

[CIFS]
title=SMB/CIFS server
description=SMB/CIFS server
ports=137,138/udp|139,445/tcp

If such an application profile does not exist in your installation, typical Samba rules can be added in UFW using the following two commands:

user $ sudo ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
user $ sudo ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp

The correct addition of the rules can be checked using the following command:

user $ sudo ufw status verbose
Password:
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
137,138/udp (CIFS)         ALLOW IN    192.168.1.0/24
139,445/tcp (CIFS)         ALLOW IN    192.168.1.0/24

The extra rule required when using broadcast NetBIOS name resolution

The reason why an extra rule is required when using broadcast NetBIOS name resolution is because UFW (which is based on IPTABLES) is ‘stateful’, as is a purely IPTABLES firewall (unless explicitly configured not to be stateful). The firewall does not consider packets it receives in response to its broadcast to be ESTABLISHED or RELATED, and therefore drops those packets. So, despite the IPTABLES and UFW rules listed above including a rule to accept incoming UDP packets on Port 137, any UDP packets received on Port 137 that do not constitute a one-to-one, two-way communication flow are dropped by the firewall. The extra rule below overrules this and makes the firewall accept packets coming from other devices’ Port 137 in response to broadcast NetBIOS Name Service packets. To do this, the extra rule uses a CT (Connection Tracking) helper named ‘netbios-ns‘ (obviously meaning ‘NetBIOS Name Service’). In order to use this rule the kernel must have been configured to use the IPTABLES ‘raw‘ table and to use CT (see the section ‘Kernel configuration’ further on).

IPTABLES

# All NetBIOS clients must have the netbios-ns helper enabled for broadcast name resolution to work
iptables -t raw -A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns

By the way, in addition to flushing the usual tables, flush the ‘raw‘ table too when you restart the firewall:

iptables -t raw -F OUTPUT

UFW

Add the following lines to the end of the file /etc/ufw/before.rules

# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

Note that the output of the command ‘ufw status verbose‘ will not include the above rule. This is not a bug.

Kernel configuration

If you are using a binary-based distribution such as Ubuntu Linux, the kernel will probably have been configured to include the needed modules (CONFIG_IP_NF_RAW=m, CONFIG_IP6_NF_RAW=m and CONFIG_NETFILTER_XT_TARGET_CT=m), and the installation configured to load the modules automatically. However, if you are using a source-based distribution such as Gentoo Linux make sure the kernel configuration includes these three options before you build the kernel, and also add the module names ‘iptable_raw‘ and ‘xt_CT‘ to the module list in the file /etc/conf.d/modules as shown in the example below, so that the modules are loaded at boot:

modules="r8169 nvidia agpgart fuse bnep rfcomm hidp uvcvideo cifs mmc_block rtsx_pci snd-seq-midi vboxdrv vboxnetadp vboxnetflt iptable_raw xt_CT"

You can use the following two commands to check if the two modules are loaded:

user $ sudo lsmod | grep iptable_raw
user $ sudo lsmod | grep xt_CT

How to check the additional rule is active

You can use the command below whether you are using pure IPTABLES or UFW.

user $ sudo iptables -nvL -t raw
Password: 
Chain PREROUTING (policy ACCEPT 2613 packets, 1115K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 2773 packets, 475K bytes)
 pkts bytes target     prot opt in     out     source               destination         
   16  1248 CT         udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:137 CT helper netbios-ns

The packet and byte counts will increase whenever you use a Samba command.

Bibliography

1. The netfilter.org "iptables" project
2. Iptables Tutorial
3. Introduction to IPTables
4. Gentoo Wiki : iptables
5. Arch Linux Wiki : Samba : "Browsing" network fails with "Failed to retrieve share list from server"
6. Ubuntu : Manpage : ufw-framework
7. Gentoo Wiki : UFW

xdotool comes to the rescue

In a previous post I explained how I implemented a method for adding my current location and the local time to my e-mail signature wherever I happen to be in the World, irrespective of the time on the laptop’s hardware clock and system clock. In that post I described how I created a keyboard shortcut using the Linux application AutoKey. Unfortunately AutoKey has not been updated for several years and no longer works properly in KDE Plasma 5 on my laptops. Therefore I decided to replace it with a KDE keyboard shortcut, and this is to explain how I did it.

First create a custom shortcut in KDE:

1. ‘System Settings’ > ‘Shortcuts’ > ‘Custom Shortcuts’
2. ‘Edit’ > ‘New’ > ‘Global Shortcut’ > ‘Command/URL’, and name the New Action ‘Insert current time’
3. On the Comment pane for ‘Insert current time’, add the comment ‘Insert current time at specified location’ (without the quotes)
4. On the Trigger pane, configure the shortcut to be Ctrl+Alt+Space
5. On the Action pane, enter the Command/URL as ‘/home/fitzcarraldo/timezone_signature_GeoNames.sh‘ (without the quotes)
6. Click ‘Apply’

Next modify the Bash script timezone_signature_GeoNames.sh so that it contains the following (obviously change the username and path to suit):

#!/bin/bash

place=$(kdialog --title "Current Location" --inputbox "Enter your location:")

placetime=$(perl /home/fitzcarraldo/now1.pl $place)

# xdotool does not output a space in a string, so we have to extract each field from the string
# and print each field individually, separated by a space character.

city=$(echo $placetime | awk -F "|" '{print $1}')
country=$(echo $placetime | awk -F "|" '{print $2}' | sed 's/[)(]//g')
region=$(echo $placetime | awk -F "|" '{print $4}')

datetime=$(/usr/bin/zdump $region | awk -F " " '{print $2" "$3" "$4" "$5" "$6" "$7}')
dayofweek=$(echo $datetime | awk -F " " '{print $1}')
month=$(echo $datetime | awk -F " " '{print $2}')
day=$(echo $datetime | awk -F " " '{print $3}')
time=$(echo $datetime | awk -F " " '{print $4}')
year=$(echo $datetime | awk -F " " '{print $5}')
timezone=$(echo $datetime | awk -F " " '{print $6}')

activewindow=$(xdotool getactivewindow)

xdotool type --window $activewindow "Sent from:"
for oneword in $city; do
    xdotool key --window $activewindow space
    sleep 0.1s
    xdotool type --window $activewindow --delay 100 $oneword
done
xdotool key --window $activewindow comma
for oneword in $country; do
    xdotool key --window $activewindow space
    sleep 0.1s
    xdotool type --window $activewindow --delay 100 $oneword
done
xdotool key --window $activewindow Return
xdotool type --window $activewindow "Local time now: "
xdotool type --window $activewindow $dayofweek
xdotool type --window $activewindow " "
xdotool type --window $activewindow $month
xdotool type --window $activewindow " "
xdotool type --window $activewindow $day
xdotool type --window $activewindow " "
xdotool type --window $activewindow $time
xdotool type --window $activewindow " "
xdotool type --window $activewindow $year
xdotool type --window $activewindow " "
if [ ${timezone:0:1} = "-" ]; then
    timezone="UTC-"${timezone#*-}
elif [ ${timezone:0:1} = "+" ]; then
    timezone="UTC+"${timezone#*+}
fi
xdotool type --window $activewindow $timezone
xdotool type --window $activewindow " "
xdotool key --window $activewindow Return
xdotool key --window $activewindow Return
echo

The Perl script now1.pl is listed in my my earlier post. Notice that the script timezone_signature_GeoNames.sh in my earlier post was much simpler. This was because the AutoKey shortcut took care of sending the text to the currently active window. Without AutoKey, I now had to do this myself in the script timezone_signature_GeoNames.sh, and the command xdotool came to the rescue. The developer explains what xdotool does as follows:

This tool lets you simulate keyboard input and mouse activity, move and resize windows, etc. It does this using X11’s XTEST extension and other Xlib functions.

Additionally, you can search for windows and move, resize, hide, and modify window properties like the title. If your window manager supports it, you can use xdotool to switch desktops, move windows between desktops, and change the number of desktops.

So I installed xdotool via the Gentoo package manager:

# emerge xdotool
# eix xdotool
[I] x11-misc/xdotool
     Available versions:  3.20150503.1-r1^t ~3.20160805.1^t {examples}
     Installed versions:  3.20150503.1-r1^t(22:51:30 02/04/17)(-examples)
     Homepage:            http://www.semicomplete.com/projects/xdotool/
     Description:         Simulate keyboard input and mouse activity, move and resize windows

Anyway, my Bash script using xdotool works a treat with Thunderbird (and KWrite, LibreOffice Writer, etc.). I used to experience a problem with certain characters, for example a colon was printed as a semi-colon (see the xdotool bug report xdotool writes the wrong case #121), but that no longer happens in my current KDE Plasma 5 installation:

Sent from: Galeão International Airport, Brazil
Local time now: Thu Jul 6 15:11:40 2017 UTC-03

What a useful tool xdotool is!

Stuttering audio in Linux: PulseAudio strikes again

I unmasked PulseAudio 10.0 back in January 2017 and installed it in my Gentoo Stable amd64 installation, and everything worked fine… until a couple of days ago, when the audio in streaming YouTube videos started to stutter every so often. It sounded rather like a scratched LP jumping. At first I thought the problem lay with Firefox, but the stuttering audio also occurred in Chrome. Then I wondered if my Internet connection was to blame; perhaps the ISP’s service had deteriorated. But a Windows 10 machine on my home network didn’t suffer from the problem, so that seemed to rule out the Internet connection. I tested the broadband throughput, and it was circa 32 Mbps, actually a little higher than the last time I tested it last year.

Now, Gentoo is a rolling distribution and I update my laptops regularly, but I couldn’t think what had been upgraded in the last couple of months that could be causing the problem. Although PulseAudio had not been upgraded since January, I began to wonder if PulseAudio could be involved, as my audio woes in the past have usually been due to PulseAudio.

I have always had PulseAudio installed with USE=”-realtime”:

user $ eix -I pulseaudio
[I] media-sound/pulseaudio
     Available versions:  10.0 {+X +alsa +alsa-plugin +asyncns bluetooth +caps dbus doc equalizer +gdbm +glib gnome gtk ipv6 jack libressl libsamplerate lirc native-headset neon ofono-headset +orc oss qt4 realtime selinux sox ssl system-wide systemd tcpd test +udev +webrtc-aec zeroconf ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  10.0(16:07:53 19/04/17)(X alsa alsa-plugin asyncns bluetooth caps dbus gdbm glib gnome gtk ipv6 jack orc qt4 ssl tcpd udev webrtc-aec zeroconf -doc -equalizer -libressl -libsamplerate -lirc -native-headset -neon -ofono-headset -oss -realtime -selinux -sox -system-wide -systemd -test ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://www.pulseaudio.org/
     Description:         A networked sound server with an advanced plugin system

but I wondered if PulseAudio’s real-time scheduling was somehow the cause of the problem, so I edited /etc/pulse/daemon.pa and added ‘realtime-scheduling = no‘ (I assume the default is ‘yes‘, as it was commented as such in the file):

; realtime-scheduling = yes
realtime-scheduling = no

Problem solved. PulseAudio is indeed a demon. 😡

Using the ClamAV daemon to scan files placed in my Downloads directory in Gentoo Linux

In a previous post I explained how to automatically detect files placed in my Downloads directory in Linux and scan them for viruses. The method I described in that post used clamscan, the command-line anti-virus scanner of ClamAV. Now, in addition ClamAV has a daemon (a program that runs continuously in the background), clamdscan, that you can enable. So I decided to switch to using clamdscan, as its response to downloaded files is much faster because the process waiting for new files to appear in ~/Downloads/ does not have to load clamscan from disk each time a new file arrives. Anyway, if you want to monitor a download directory in Gentoo Linux (running OpenRC) by using the ClamAV daemon — which will also download virus signature database updates automatically — then the procedure to set this up is given below.

1. Install clamav if it is not installed already:

root # emerge clamav

2. Add the service to the default runlevel:

root # rc-update add clamd default

The daemon will be launched automatically next time the computer boots.

3. The first download of the virus database has to be done manually:

root # freshclam

4. Start the daemon now:

root # rc-service clamd start

5. Create the Bash script ~/monitorDownloadsGUI with the following contents:

#!/bin/bash

DIR=$HOME/Downloads

# Get rid of old log file, if any
rm $HOME/virus-scan.log 2> /dev/null

IFS=$(echo -en "\n\b")

# Optionally, you can use shopt to avoid creating two processes due to the pipe
shopt -s lastpipe
inotifywait --quiet --monitor --event close_write,moved_to --recursive --format '%w%f' $DIR | while read FILE
# Added '--recursive' so that a directory copied into $DIR also triggers clamscan/clamdscan, although downloads
# from the Web would just be files, not directories.
do
     # Have to check file length is nonzero otherwise commands may be repeated
     if [ -s $FILE ]; then
          # Replace 'date >' with 'date >>' if you want to keep log file entries for previous scans.
          date > $HOME/virus-scan.log
          clamdscan --move=$HOME/virus-quarantine $FILE >> $HOME/virus-scan.log
          kdialog --title "Virus scan of $FILE" --msgbox "$(cat $HOME/virus-scan.log)"
     fi
done

Make it executable:

user $ chmod +x ~/monitorDownloadsGUI

6. Create the directory ~/virus-quarantine/ to store infected files pending investigation/deletion:

user $ mkdir ~/virus-quarantine

7. Install kdialog if it is not already installed:

root # emerge kdialog

8. Use ‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’ to add the script ~/monitorDownloadsGUI to the list of script files that are automatically started each time you log in to KDE.

9. Log out then back in again, and you should see that everything is running as expected:

user $ rc-status | grep clam
 clamd                                                             [  started  ]

user $ ps -ef | grep clam | grep -v grep
clamav    1920     1  0 01:48 ?        00:00:00 /usr/sbin/clamd
clamav    1929     1  0 01:48 ?        00:00:00 /usr/bin/freshclam -d

user $ ps -ef | grep GUI | grep -v grep
fitzcarraldo      9143  8971  0 13:56 ?        00:00:00 /bin/bash /home/fitzcarraldo/.config/autostart-scripts/monitorDownloadsGUI.sh

10. To test, surf to http://www.eicar.org/85-0-Download.html and download one of the EICAR test files into your ~/Downloads/ directory. You should see a pop-up KDialog window with a message similar to the following:

Virus scan of /home/fitzcarraldo/Downloads/eicarcom2.zip — KDialog

Mon 27 Feb 14:05:26 GMT 2017
/home/fitzcarraldo/Downloads/eicarcom2.zip: Eicar-Test-Signature FOUND
/home/fitzcarraldo/Downloads/eicarcom2.zip: moved to ‘/home/fitzcarraldo/virus-quarantine/eicarcom2.zip’

———– SCAN SUMMARY ———–
Infected files: 1
Time: 0.001 sec (0 m 0 s)

Note that the above-mentioned pop-up window may be preceded by one or more pop-up windows with an error message. I’m using the Chrome browser at the moment, but you may get a similar message if you are using another browser. Here is an example:

Virus scan of /home/fitzcarraldo/Downloads/.com.google.Chrome.Uh3oGm — KDialog ?

Mon 27 Feb 14:16:30 GMT 2017
/home/fitzcarraldo/Downloads/.com.google.Chrome.Uh3oGm: Access denied. ERROR

———– SCAN SUMMARY ———–
Infected files: 0
Total errors: 1
Time: 0.000 sec (0 m 0 s)

Read the error message and click ‘OK’, as this is not an actual problem; it is inotifywait detecting temporary files in the ~/Downloads/ directory during the download process. With larger files sometimes several such messages are displayed, presumably because the file being downloaded is being opened and closed more than once during the downloading process. This issue does not occur if you copy or move a file into ~/Downloads/ from another directory in your installation; try it and see for yourself. Then you only get the one pop-up window with the scan result for the file you put in ~/Downloads/.

Also have a look in ~/virus-quarantine/ and you will see the EICAR test file in that directory. You can delete it if you want (it is not infected with a real virus, so does no harm).

In future be sure to read the messages in the pop-up windows before clicking ‘OK’, as they will inform you that an infected file has been moved to the quarantine directory.

That’s all there is to it. Very simple, and quite handy if you want to check quickly that files you download don’t have a malware payload. Just make sure you download all files into ~/Downloads/ or they will not be checked automatically. Also, if you are given e.g. a USB pen drive with a file on it, you can copy the file to ~/Downloads/ if you want it to be scanned for malware.

Using an external USB 3.5-inch floppy disk drive in Linux

Back in 2004 I needed to get some files off my old 3.5″ floppy disks, so I bought an external USB floppy disk drive to use with a laptop running Windows XP. The label on the drive gives the manufacturer and model as ‘SmartDisk: FDUSB-TM2, Mitsumi Model #: D353FUE’.

Anyway, today I wanted to throw out some 720KB DD (Double Density) and 1440KB HD (High Density) 3.5″ floppy disks but first needed to check their contents and wipe them. So I dug out the SmartDisk USB drive to see if it would work with the current Gentoo Linux installation on my newest laptop. I was pleased to discover that it does, and below are some notes on how to use it in case anyone else needs to use one of these devices.

Once plugged in to a USB port on my laptop, the lsusb command shows the device has been recognised:

Bus 001 Device 013: ID 03ee:6901 Mitsumi SmartDisk FDD

Note that the Linux floppy driver is not needed for USB floppy disk drives:

root # grep -i CONFIG_BLK_DEV_FD /usr/src/linux/.config
# CONFIG_BLK_DEV_FD is not set

A Linux utility named ufiformat is used to low-level format floppy disks in USB floppy disk drives. A Gentoo Linux ebuild for Version 0.9.9 of ufiformat is listed below, and it can be used in a local overlay under the category sys-fs:

# Copyright 1999-2014 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=5

DESCRIPTION="USB Floppy Disk formatting tool"
HOMEPAGE="http://www.geocities.jp/tedi_world/format_usbfdd_e.html"
SRC_URI="http://www.geocities.jp/tedi_world/${P}.tar.gz"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""

RDEPEND="sys-fs/e2fsprogs"
DEPEND=${RDEPEND}

The ufiformat utility is straightforward to use:

root # ufiformat --help
Usage: ufiformat [OPTION]... [DEVICE]
Format a floppy disk in a USB floppy disk DEVICE.

  -f, --format [SIZE]  specify format capacity SIZE in KB
                       without -f option, the format of the current media will be used
  -V, --verify         verify the medium after formatting
  -F, --force          do not perform any safety checks
  -i, --inquire        show device information, instead of performing format
                       without DEVICE argument, list USB floppy disk devices
  -v, --verbose        show detailed output
  -q, --quiet          suppress minor output
  -h, --help           show this message

To find the device name, use the blkid command before plugging in the USB cable and again after plugging in the USB cable. The extra device listed the second time will be the floppy disk drive. For example, in my case the new line at the end of the blkid output indicated the drive was /dev/sdd:

/dev/sdd: SEC_TYPE="msdos" UUID="BBBA-37AF" TYPE="vfat"

The fdisk command will confirm that the device is the floppy drive:

root # fdisk -l /dev/sdd
Disk /dev/sdd: 720 KiB, 737280 bytes, 1440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Note that /dev/sdd will not be listed in the output of blkid if there is no disk in the floppy drive, although the ls command will list /dev/sdd while the drive is connected to the computer.

Notice that there are no devices /dev/fd0, /dev/fd1, /dev/fd2 and /dev/fd3, etc. This does not matter.

root # ls /dev
audio1           dri      i2c-10   kmem          mapper              nvidiactl  sda1       sequencer2  tty0   tty2   tty30  tty41  tty52  tty63    usbmon3     vcs2    vcsa12
autofs           dsp1     i2c-11   kmsg          mcelog              nvram      sda2       sg0         tty1   tty20  tty31  tty42  tty53  tty7     usbmon4     vcs3    vcsa2
block            fb0      i2c-2    log           mem                 pktcdvd    sda3       sg1         tty10  tty21  tty32  tty43  tty54  tty8     v4l         vcs4    vcsa3
bsg              fd       i2c-3    loop-control  memory_bandwidth    port       sda5       sg2         tty11  tty22  tty33  tty44  tty55  tty9     vboxdrv     vcs5    vcsa4
bus              full     i2c-4    loop0         mixer               ptmx       sda6       sg3         tty12  tty23  tty34  tty45  tty56  ttyS0    vboxdrvu    vcs6    vcsa5
char             fuse     i2c-5    loop1         mixer1              pts        sda7       shm         tty13  tty24  tty35  tty46  tty57  ttyS1    vboxnetctl  vcs7    vcsa6
console          hidraw0  i2c-6    loop2         mqueue              random     sdb        snapshot    tty14  tty25  tty36  tty47  tty58  ttyS2    vboxusb     vcs8    vcsa7
core             hidraw1  i2c-7    loop3         network_latency     rfkill     sdb1       snd         tty15  tty26  tty37  tty48  tty59  ttyS3    vcs         vcs9    vcsa8
cpu              hidraw2  i2c-8    loop4         network_throughput  root       sdc        stderr      tty16  tty27  tty38  tty49  tty6   urandom  vcs1        vcsa    vcsa9
cpu_dma_latency  hpet     i2c-9    loop5         null                rtc        sdc1       stdin       tty17  tty28  tty39  tty5   tty60  usbmon0  vcs10       vcsa1   vga_arbiter
cuse             i2c-0    initctl  loop6         nvidia-modeset      rtc0       sdd        stdout      tty18  tty29  tty4   tty50  tty61  usbmon1  vcs11       vcsa10  video0
disk             i2c-1    input    loop7         nvidia0             sda        sequencer  tty         tty19  tty3   tty40  tty51  tty62  usbmon2  vcs12       vcsa11  zero
root # ls -la /dev/fd
lrwxrwxrwx 1 root root 13 Jan 17 02:13 /dev/fd -> /proc/self/fd
root # ls -la /proc/self/fd
total 0
dr-x------ 2 root root  0 Jan 17 04:26 .
dr-xr-xr-x 8 root root  0 Jan 17 04:26 ..
lrwx------ 1 root root 64 Jan 17 04:26 0 -> /dev/pts/1
lrwx------ 1 root root 64 Jan 17 04:26 1 -> /dev/pts/1
lrwx------ 1 root root 64 Jan 17 04:26 2 -> /dev/pts/1
lr-x------ 1 root root 64 Jan 17 04:26 3 -> /proc/17669/fd

To format an HD floppy disk with the FAT file system, I did the following:

root # ufiformat -f 1440 /dev/sdd
geometry: track=80, head=2, sector=18, block=512
done                                   
root # /usr/sbin/mkfs.vfat /dev/sdd
mkfs.fat 4.0 (2016-05-06)
attribute "partition" not found
root # ufiformat -i /dev/sdd
vendor:  MITSUMI
product: USB FDD
write protect: off
media type: 2HD
status      block size   kb
formatted    2880  512 1440
formattable  2880  512 1440
formattable  1232 1024 1232
formattable  2400  512 1200

To format a DD floppy disk with the FAT file system, I did the following:

root # ufiformat -f 720 /dev/sdd
geometry: track=80, head=2, sector=9, block=512
done                                   
root # /usr/sbin/mkfs.vfat /dev/sdd
mkfs.fat 4.0 (2016-05-06)
attribute "partition" not found
root # ufiformat -i /dev/sdd
vendor:  MITSUMI
product: USB FDD
write protect: off
media type: 2DD
status      block size   kb
formatted    1440  512  720
formattable  1440  512  720

I use the KDE Desktop Environment. The Device Notifier widget in the System Tray shows the drive and — once a formatted floppy disk is in the drive — it is possible to use the Device Notifier to mount, open and unmount the floppy disk. However, it is also possible to use the command line:

root # mkdir /mnt/floppy
root # mount /dev/sdd /mnt/floppy
root # ls /mnt/floppy
root # cp /test.txt /mnt/floppy/
root # ls /mnt/floppy
test.txt
root # umount /dev/sdd

Earlier in this post I showed examples of formatting floppy disks using the FAT file system, but it is of course possible to format them using other file systems, such as:

root # mkfs.ext2 /dev/sdd
mke2fs 1.43.3 (04-Sep-2016)
/dev/sdd contains a vfat file system
Proceed anyway? (y,n) y
Creating filesystem with 720 1k blocks and 96 inodes

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

Anyway, I was able to check the contents of the floppies and wipe them before disposing of them. It’s good to know that some old technologies can still be used when needs be. I won’t be throwing out the old floppy disk drive just yet.