A Linux command-line utility to discover and list WSD-enabled computers and printers on a home network

In an earlier post I covered the installation and use of wsdd, a WS-Discovery (WSD) daemon that can run on Linux machines and enable machines running Microsoft Windows 10 to discover Linux machines in File Explorer now that Windows 10 has dropped Computer Browser, NetBIOS and SMBv1. All my Linux machines in my home network have wsdd running alongside NetBIOS broadcast name resolution, SMBv2 (used by my Android phone) and SMBv3 (used by my Linux machines). If any visitors to my house happen to bring a laptop running Windows 10, they will be able to discover my SMB shares in File Explorer, which I have always been able to do in Linux and in earlier Windows releases that supported NetBIOS and Computer Browser.

As I pointed out in a comment to another of my earlier posts, a downside of not using the (insecure) SMBv1 protocol is that the Samba utility smbtree incorrectly returns nothing if you enter the command smbtree when using SMBv2 or SMBv3. As all the Linux machines in my home network are running the wsdd daemon in addition to NetBIOS, SMBv2 and SMBv3 — and any visitors’ laptops could be running Windows 10 — it would be nice to have a command-line utility that would discover all machines. Well, here is a stab at such a utility, written by a close relative of mine as a learning exercise in WSD and Python, and is provided here as-is without any warranty or support. It consists of the following five files:

wsd-discover.sh

#!/bin/bash

function del-tmp-files() {
   if ls /tmp/wsd-*.txt 1> /dev/null 2>&1; then
      rm /tmp/wsd-*
   fi
return 0
}

# Delete pre-existing temporary work files.

del-tmp-files

# Get the V5 UUID of this machine
UUID=$(python3 $HOME/discover/wsd-gen-uuid.py)

# Send a multicast probe to all WSD capable devices and store the XML output in wsd-probe1.txt
echo
echo "Please wait.....sending multicast discovery probe and waiting 2 seconds for responses"
echo
python3 $HOME/discover/wsd-mcast-probe.py > /tmp/wsd-probe1.txt

# Iterate through the XML until the UUID to IPv4 mappings are obtained in wsd-probe9.txt
more /tmp/wsd-probe1.txt | grep Computer | awk -F "<wsa:Address>" '{print $2}' > /tmp/wsd-probe2.txt

sort -u /tmp/wsd-probe2.txt > /tmp/wsd-probe3.txt

more /tmp/wsd-probe3.txt | awk -F "uuid:" '{print $2}' > /tmp/wsd-probe4.txt

more /tmp/wsd-probe4.txt | awk -F "</wsa:Address>" '{print $1,"******",$2}' > /tmp/wsd-probe5.txt

more /tmp/wsd-probe5.txt | awk -F "from" '{print $1,"******",$2}' > /tmp/wsd-probe6.txt

more /tmp/wsd-probe6.txt | awk -F "******" '{print $1 $3}' > /tmp/wsd-probe7.txt

more /tmp/wsd-probe7.txt | awk -F "\\\('" '{print $1 $2}' > /tmp/wsd-probe8.txt

more /tmp/wsd-probe8.txt | awk -F "'" '{print $1}' > /tmp/wsd-probe9.txt

# Read the UUID to IPv4 mappings until end of file and send XML requests to each WSD host

while read RECORD; do

	URN=$(echo $RECORD | cut -d" " -f1)
	IPA=$(echo $RECORD | cut -d" " -f2)

	# Generate the HTTP/XML request file from the template
	cat $HOME/discover/wsd-template.xml | sed 's/XXXXXXXXXX/'$URN'/g' > /tmp/wsd-request.txt
	cat /tmp/wsd-request.txt | sed -i 's/YYYYYYYYYY/'$UUID'/g' /tmp/wsd-request.txt

	# Send the XML/SOAP request to the target machine
	curl -s -A wsd --header "Accept-Encoding: identity" --header "Connection: Close" \
	--header "Content-Type: application/soap+xml" --header "User-Agent: wsd" \
	--data @/tmp/wsd-request.txt http://$IPA:5357/$URN > /tmp/wsd-response-$IPA.txt

	# Extract, format and display the information returned
	echo
	echo "Device IP : $IPA"
	echo "==========================="
	echo -n "Name         :";cat /tmp/wsd-response-$IPA.txt | awk -F "FriendlyName" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Manufacturer :";cat /tmp/wsd-response-$IPA.txt | awk -F "Manufacturer" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Model        :";cat /tmp/wsd-response-$IPA.txt | awk -F "ModelName" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Category     :";cat /tmp/wsd-response-$IPA.txt | awk -F "DeviceCategory" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "URN          :";cat /tmp/wsd-response-$IPA.txt | awk -F "Address" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Type         :";cat /tmp/wsd-response-$IPA.txt | awk -F "Types" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Workgroup    :";cat /tmp/wsd-response-$IPA.txt | awk -F "<pub:Computer" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo

done < /tmp/wsd-probe9.txt

# This next bit is just a bit of fluff to display printers. The formatting is inconsistent because every printer
# has a different web page. Printer manufacturers are listed in the file $HOME/discover/printers.txt. If the printer
# is not in this file it won't be found in the HTTP information

# Check whether the original multicast response contains any printer information
cat /tmp/wsd-probe1.txt | grep -q -A2 Print

if [[ $? -eq 0 ]]; then # A printer of some sort has been found

   # Get the line that contains 'Print' and the two lines after it (one of which contains the printer IP and URL)
   more /tmp/wsd-probe1.txt | grep -A2 Print > /tmp/wsd-probe10.txt

   # Remove any duplicate entries
   sort -u /tmp/wsd-probe10.txt > /tmp/wsd-probe11.txt

   # Isolate the printer IP and URL information
   cat /tmp/wsd-probe11.txt | awk -F"XAddrs>" '{print $2}' | awk -F"/wsd" '{print $1}' > /tmp/wsd-probe12.txt

   # Remove blank lines to clean up the file
   sed '/^$/d' /tmp/wsd-probe12.txt > /tmp/wsd-probe13.txt

   # Read each line of the file containing the printer URLs and contact the printers in turn
   while read RECORD; do

	echo "Printers"
	echo "==========================="
	URL=$RECORD
	# Try to get the printer's HTML page
        curl -s $URL/index.html > /tmp/wsd-printer.txt
	if [[ $? -ne 0 ]]; then
           echo "Couldn't get HTML info from $URL"
	else
	   # Read each line of the printers.txt file and try to get the Make and Model from the HTML
           while read PRT; do
		 grep -q $PRT /tmp/wsd-printer.txt
		 if [[ $? -eq 0 ]]; then # Printer in the list is contained in the returned HTML
		    # Extract the Make and the following word hoping it's the Model
		    TYP=$(grep $PRT /tmp/wsd-printer.txt | awk -v a=$PRT '{for(i=1;i<=NF;i++) if ($i==a) print $i,$(i+1)}')
		    echo "URL   : $URL"
                    echo "Make  : $TYP"
		 fi
	   done < $HOME/discover/printers.txt
	fi

   done < /tmp/wsd-probe13.txt

fi

echo

#
# Delete the latest temporary work files.
#
del-tmp-files

wsd-gen-uuid.py

import uuid
import socket

hostName = (socket.gethostname())

# nameSpaces = [uuid.NAMESPACE_DNS, uuid.NAMESPACE_URL, uuid.NAMESPACE_OID, uuid.NAMESPACE_X500]
nameSpaces = [uuid.NAMESPACE_DNS]

for namespace in nameSpaces:

    print (uuid.uuid5(namespace, hostName))

wsd-mcast-probe.py

import socket
import struct
import sys
import uuid

# Create a V1 UUID for the MessageID based on the host address and current time
# The MessageID must be unique but it isn't necessary to have anything other than a V1 UUID
uuid1 = uuid.uuid1()
myuuid = str(uuid1)
print ("Generating UUID for MessageID")
print(myuuid)

# The string 'message' is a template WSD probe that is multicast to group 239.255.255.250 port 3702
# The template should not change unless there is a major change to the WSD specifications
# Escape double quotation marks within the message string (but not the outer double quotation marks)
message = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:pnpx=\"http://schemas.microsoft.com/windows/pnpx/2005/10\" xmlns:pub=\"http://schemas.microsoft.com/windows/pub/2005/07\" xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:wsa=\"http://schemas.xmlsoap.org/ws/2004/08/addressing\" xmlns:wsd=\"http://schemas.xmlsoap.org/ws/2005/04/discovery\" xmlns:wsdp=\"http://schemas.xmlsoap.org/ws/2006/02/devprof\" xmlns:wsx=\"http://schemas.xmlsoap.org/ws/2004/09/mex\"><soap:Header><wsa:To>urn:schemas-xmlsoap-org:ws:2005:04:discovery</wsa:To><wsa:Action>http://schemas.xmlsoap.org/ws/2005/04/discovery/Probe</wsa:Action><wsa:MessageID>urn:uuid:" + myuuid + "</wsa:MessageID></soap:Header><soap:Body><wsd:Probe><wsd:Types>wsdp:Device</wsd:Types></wsd:Probe></soap:Body></soap:Envelope>"

# Convert the message to a UTF-8 byte string
bytstr = message.encode('utf-8')

# Define a variable for the multicast group and multicast destination port
multicast_group = ('239.255.255.250', 3702)
multicast_address = '239.255.255.250'

# Cheeky way to get the Internet facing Ethernet IP address for use further down
# Create a socket, pretend to use it to connect to an Internet service. Nothing is actually sent
# but the IP address of the Internet facing interface is returned 
def get_ip_address():
    sock1 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock1.connect(("8.8.8.8", 80))
    return sock1.getsockname()[0]

# Create datagram socket 1 for multicasts and allow the IP address and port to be reused in case something
# else is using them e.g. the WSD service
sock1 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)

IPADDR = (get_ip_address())

# Set the multicasts TTL to 1 so they stay on the local segment
sock1.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 1)

# Set a timeout so the socket stops listening if no data is received within the timeout
# This prevents it from locking up
sock1.settimeout(2.0)

# Bind the socket to the IP and port that we wish to use as the source IP and port of datagrams we transmit
# AND the destination IP and port of datagrams that we receive
sock1.bind ((IPADDR, 3702))

# Join the 239.255.255.250 multicast group. This isn't necessary if this script is being run on a machine
# that is also running the wsdd daemon. Joining the multicast group allows the script to be run on any
# machine regardless

mreq = struct.pack("4sl", socket.inet_aton(multicast_address), socket.INADDR_ANY)
sock1.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

try:

    # Send the WSD probe (bytstr) to the multicast group and port
    # print ('\nsending "%s"' % bytstr)
    sent = sock1.sendto(bytstr, multicast_group)

    # Listen for up to 4096 byte responses from all responders to the multicast message
    while True:
       print ('\nwaiting to receive responses')
       try:
           data, addr = sock1.recvfrom(4096)

           # We could use the format below to split 'addr' into its component IP and port fields but is isn't necessary
           # data, (ip, port) = sock1.recvfrom (4096)

       except:
           # This exception only occurs if no data is received on socket for the timeout period
           print ('\ntimed out, no more responses socket1')
           break
       else:
           # This is the response data that the bash script writes out to the wsd-probe1.txt file
           print ('\nreceived %s from %s' % (data.decode('utf-8'), addr))

finally:
    print ('\nsocket closed\n')

wsd-template.xml

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope
      xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/10"
      xmlns:pub="http://schemas.microsoft.com/windows/pub/2005/07"
      xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
      xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"
      xmlns:wsd="http://schemas.xmlsoap.org/ws/2005/04/discovery"
      xmlns:wsdp="http://schemas.xmlsoap.org/ws/2006/02/devprof"
      xmlns:wsx="http://schemas.xmlsoap.org/ws/2004/09/mex">
      <soap:Header>
            <wsa:To>urn:uuid:XXXXXXXXXX</wsa:To>
            <wsa:Action>http://schemas.xmlsoap.org/ws/2004/09/transfer/Get</wsa:Action>
            <wsa:MessageID>urn:uuid:fe11d044-bc13-11ea-b98c-2c56dc778d37</wsa:MessageID>
            <wsa:ReplyTo>
                 <wsa:Address>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:Address>
            </wsa:ReplyTo>
            <wsa:From>
                 <wsa:Address>urn:uuid:YYYYYYYYYY</wsa:Address>
            </wsa:From>
      </soap:Header>
      <soap:Body />
</soap:Envelope>

printers.txt

Brother
Canon
Epson
HP
Kodak
Lexmark

How to install

user $ mkdir $HOME/discover

Use a text editor to create the five files listed above in the directory $HOME/discover/.

Make the Bash script and the Python scripts executable:

user $ chmod u+x $HOME/discover/*.sh $HOME/discover/*.py

How to use

user $ $HOME/discover/wsd-discover.sh

The script will list discovered devices (see the caveat in the Description section below). For example:

user $ $HOME/discover/wsd-discover.sh

Please wait.....sending multicast discovery probe and waiting 2 seconds for responses


Device IP : 192.168.1.121
===========================
Name         :WSD Device tutankhamun
Manufacturer :wsdd
Model        :wsdd
Category     :Computers
URN          :urn:uuid:ff03f853-8a45-5ad9-b75b-fe4f632c8c5b
Type         :pub:Computer
Workgroup    :TUTANKHAMUN/Workgroup:HOME


Device IP : 192.168.1.10
===========================
Name         :WSD Device akhanaten
Manufacturer :wsdd
Model        :wsdd
Category     :Computers
URN          :urn:uuid:ad8fedfb-a22c-5551-92b4-653aae69f379
Type         :pub:Computer
Workgroup    :AKHANATEN/Workgroup:HOME


Device IP : 192.168.1.74
===========================
Name         :WSD Device thutmoseiii
Manufacturer :wsdd
Model        :wsdd
Category     :Computers
URN          :urn:uuid:9bf49ac3-e58d-57a4-87ea-7c0d5ef02234
Type         :pub:Computer
Workgroup    :THUTMOSEIII/Workgroup:HOME

Printers
===========================
URL   : http://192.168.1.78:80
Make  : Canon MP560


The example output above was for a network of three Linux machines running the wsdd daemon, connected via Ethernet, plus a printer connected via Wi-Fi.

Description

The scripts are non-intrusive and discover WSD-enabled devices in multicast group 239.255.255.250 port 3702, namely a) Windows 10 and b) other Linux machines running the WSD daemon wsdd or other WSD software. It runs over Ethernet and Wi-Fi. The script joins the multicast group (with a reusable socket) and sends out a WSD Probe. The responses contain the UUID-to-IP address mappings of the devices it discovers. Each discovered device is then contacted individually on its IP address TCP port 5357 to retrieve basic information.

If you run the script on Linux with the WSD Daemon (wsdd) also running (see earlier post), the script discovers itself as well as other devices. If you run the script on a machine that is not running the WSD Daemon it still discovers other devices, but not itself.

The script also discovers any WSD-enabled printers that listen for multicasts on UPnP / SSDP group 239.255.255.250 but don’t care about what UDP port is being used. If a WSD-enabled printer is detected, the script attempts to retrieve the make and model of the printer using HTTP. To detect different printer makes, add the manufacturer e.g. Canon, Epson, Lexmark etc. to the file ‘printers.txt‘. The script reports on the printer make and tries to extract the model type. It may not always format the output 100% accurately.

The main thing to bear in mind is that the scripts do not maintain state i.e. a single discovery probe is transmitted. Multicast is fundamentally unreliable and only devices that respond are reported. If the probe is lost or an end device doesn’t respond, for whatever reason, it doesn’t get reported. You can run the script a few times to ensure that it picks up as many of the devices as it possibly can.

Reconfiguring the time zone, locales and keymaps in Sabayon Linux

This is an example of how to reconfigure the time zone, locales and keymaps in a Sabayon Linux installation from the command line. Sabayon Linux uses systemd, therefore much of this example should also be applicable in other Linux distributions that use systemd, and will certainly be applicable in Gentoo Linux installations that use systemd rather than OpenRC.

You can check the currently selected keymaps (console and X Windows) and locale using the ‘localectl status‘ command. For example:

user $ localectl status
   System Locale: LANG=en_GB.UTF-8
       VC Keymap: uk
      X11 Layout: gb
       X11 Model: pc105

Let’s say I had previously configured my installation to use only the en_GB and en_US locales but I now want to add Swiss Italian. The steps would be as shown below. I will assume the system being configured is in Switzerland and therefore I will also reconfigure the time zone accordingly, but that is not essential.

Check if the desired time zone exists:

root # timedatectl list-timezones | grep Zurich
Europe/Zurich

Set the desired time zone:

root # timedatectl set-timezone Europe/Zurich

Check if the Swiss Italian locale (it_CH) has already been added:

root # localectl list-locales
C.utf8
en_GB
en_GB.iso88591
en_GB.utf8
en_US
en_US.iso88591
en_US.utf8

If the desired locale is not present, add it:

root # nano /etc/locale.gen
root # grep -v "^#\|^$" /etc/locale.gen
C.UTF-8 UTF-8
en_GB.UTF-8 UTF-8
en_GB ISO-8859-1
en_US.UTF-8 UTF-8
en_US ISO-8859-1
it_CH.UTF-8 UTF-8
it_CH ISO-8859-1

Generate the locales:

root # locale-gen
 * Generating 7 locales (this might take a while) with 1 jobs
 *  (1/7) Generating C.UTF-8 ...                                          [ ok ]
 *  (2/7) Generating en_GB.ISO-8859-1 ...                                 [ ok ]
 *  (3/7) Generating en_GB.UTF-8 ...                                      [ ok ]
 *  (4/7) Generating en_US.ISO-8859-1 ...                                 [ ok ]
 *  (5/7) Generating en_US.UTF-8 ...                                      [ ok ]
 *  (6/7) Generating it_CH.ISO-8859-1 ...                                 [ ok ]
 *  (7/7) Generating it_CH.UTF-8 ...                                      [ ok ]
 * Generation complete
 * Adding locales to archive ...                                          [ ok ]

Check the locales have been added:

root # localectl list-locales
C.utf8
en_GB
en_GB.iso88591
en_GB.utf8
en_US
en_US.iso88591
en_US.utf8
it_CH
it_CH.iso88591
it_CH.utf8

Set the desired locale:

root # localectl set-locale LANG=it_CH.UTF-8

Check which Italian console keymaps are available:

root # localectl list-keymaps | grep it
it
it-ibm
it2
mac-it

But let’s say I want to use a Swiss German keymap (sg) for the console instead of an Italian keymap. Check if a console keymap for Swiss German exists:

root # localectl list-keymaps | grep sg
sg
sg-latin1
sg-latin1-lk450

By the way, Debian, Ubuntu and its derivatives store console keymaps differently to some distributions, and the command ‘localectl list-keymaps‘ in Debian, Ubuntu and its derivatives will return an error message (the command ‘localectl set-keymap’ will still work though):

user $ localectl list-keymaps
Failed to read list of keymaps: No such file or directory

Set the console keymap to Swiss German:

root # localectl set-keymap sg

Let’s say I want to use a Swiss keymap in X Windows. Check if it exists:

root # localectl list-x11-keymap-layouts | grep sg
root # localectl list-x11-keymap-layouts | grep ch
ch

Set the X Windows keymap to Swiss:

root # localectl set-x11-keymap ch

Update the environment variables and profile:

root # env-update && source /etc/profile
>>> Regenerating /etc/ld.so.cache...

Edit /etc/default/grub and change (or add, if none exists) the console keymap entry in GRUB_CMDLINE_LINUX_DEFAULT to be vconsole.keymap=sg, and also rd.vconsole.keymap=sg (‘rd‘ stands for ‘RAM disk’) because Sabayon Linux uses an initramfs:

root # nano /etc/default/grub

Regenerate grub.cfg:

root # grub-mkconfig -o /boot/grub/grub.cfg
Generazione file di configurazione GRUB...
Trovato sfondo: /boot/grub/default-splash.png
Trovata immagine linux: /boot/kernel-genkernel-x86_64-5.4.0-sabayon
Trovata immagine initrd: /boot/initramfs-genkernel-x86_64-5.4.0-sabayon
fatto

Reboot to check if everything is working:

root # systemctl reboot

Check that the list of locales is as expected:

root # eselect locale list
Available targets for the LANG variable:
  [1]   C
  [2]   C.utf8
  [3]   en_GB
  [4]   en_GB.iso88591
  [5]   en_GB.utf8
  [6]   en_US
  [7]   en_US.iso88591
  [8]   en_US.utf8
  [9]   it_CH
  [10]  it_CH.iso88591
  [11]  it_CH.utf8
  [12]  POSIX
  [13]  it_CH.UTF-8 *
  [ ]   (free form)

Check if the current configuration is as expected:

root # localectl status
   System Locale: LANG=it_CH.UTF-8
       VC Keymap: sg
      X11 Layout: ch

If the Desktop Environment is KDE, check the file ~/.config/locale-plasmarc to see if the LANG variable is set to the locale just configured.

root # cat /home/fitzcarraldo/.config/plasma-localerc
[Formats]
LANG=en_GB.UTF-8

If it is not, delete the file:

root # rm /home/fitzcarraldo/.config/plasma-localerc

then logout, login again and re-check the file:

root # cat /home/fitzcarraldo/.config/plasma-localerc
[Formats]
LANG=it_CH.UTF-8

The system should now be ready for use with the new time zone, locale and keymaps.

user $ date
gio 2 lug 2020, 15:55:21, CEST
user $ localectl status
   System Locale: LANG=it_CH.UTF-8
       VC Keymap: sg
      X11 Layout: ch

Using a mixture of locale variables

It is not mandatory for all the locale variables to be for the same locale. For example, suppose I want to use the currency and number formats of one of the other locales I added. That is not so outlandish: I could be a Swiss national whose mother tongue is Swiss Italian, working in the Swiss branch of a British company and I want the currency format and number format on my work computer to be British, but everything else to be Swiss. To achieve this I can additionally do the following:

root # localectl set-locale LC_MONETARY=en_GB.UTF-8
root # localectl set-locale LC_NUMERIC=en_GB.UTF-8
root # env-update && source /etc/profile

Check that the main locale and keymaps remain as they were but that the two locale variables have been changed to the British locale:

root # localectl status
   System Locale: LANG=it_CH.UTF-8
                  LC_NUMERIC=en_GB.UTF-8
                  LC_MONETARY=en_GB.UTF-8
       VC Keymap: sg
      X11 Layout: ch
root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC=en_GB.UTF-8
LC_TIME="it_CH.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=

You can see above that only $LC_NUMERIC and $LC_MONETARY have changed, as I wanted. As I did not change the time zone, the command I used earlier to set the time zone to Europe/Zurich is still in force:

root # date
gio 2 lug 2020, 16:12:18, CEST

By the way, if you try to change one of the variables from en_GB.UTF-8 back to it_CH.UTF-8, the change does not show in the output of the locale command. For example, let’s say you want to change LC_NUMERIC back to it_CH.UTF-8:

root # localectl status
   System Locale: LANG=it_CH.UTF-8
                  LC_NUMERIC=en_GB.UTF-8
                  LC_MONETARY=en_GB.UTF-8
       VC Keymap: sg
      X11 Layout: ch
root # localectl set-locale LC_NUMERIC=it_CH.UTF-8
root # cat /etc/locale.conf   
LANG=it_CH.UTF-8
LC_MONETARY=en_GB.UTF-8
root # cat /etc/env.d/02locale
LANG=it_CH.UTF-8
LC_MONETARY=en_GB.UTF-8
root # localectl status
   System Locale: LANG=it_CH.UTF-8
                  LC_MONETARY=en_GB.UTF-8
       VC Keymap: sg
      X11 Layout: ch
root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC=en_GB.UTF-8  <-- Notice it didn't change
LC_TIME="it_CH.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=
root # env-update && source /etc/profile
>>> Regenerating /etc/ld.so.cache...
root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC=en_GB.UTF-8 <-- Notice it still didn't change
LC_TIME="en_GB.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=

This is one of the reasons I’m not keen on the layer of abstraction added by systemd. The way to get LC_NUMERIC back to it_CH.UTF-8 is to change all the locale variables to en_GB.UTF-8 then back to it_CH.UTF-8:

root # localectl set-locale LANG=en_GB.UTF-8
root # env-update && source /etc/profile
root # localectl set-locale LANG=it_CH.UTF-8
root # env-update && source /etc/profile
root # reboot

After rebooting, the change will have been applied:

root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC="it_CH.UTF-8"
LC_TIME="it_CH.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY="it_CH.UTF-8"
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=
root # localectl status
   System Locale: LANG=it_CH.UTF-8
       VC Keymap: sh
      X11 Layout: ch

Notice that everything has changed back to it_CH.UTF-8, including $LC_MONETARY, so you’d have to repeat the command ‘localectl set-locale LC_MONETARY=en_GB.UTF-8‘ if you wanted that to still be the British format.

If you use KDE, also check the contents of the file ~/.config/plasma-localerc to make sure it contains the correct locale:

user $ cat ~/.config/plasma-localerc
[Formats]
LANG=it_CH.UTF-8

Optionally you could edit that file to add desired settings. For example:

[Formats]
LANG=it_CH.UTF-8
LC_CTYPE=it_CH.UTF-8
LC_NUMERIC=en_GB.UTF-8
LC_TIME=it_CH.UTF-8
LC_COLLATE=it_CH.UTF-8
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES=it_CH.UTF-8
LC_PAPER=it_CH.UTF-8
LC_NAME=it_CH.UTF-8
LC_ADDRESS=it_CH.UTF-8
LC_TELEPHONE=it_CH.UTF-8
LC_MEASUREMENT=it_CH.UTF-8
LC_IDENTIFICATION=it_CH.UTF-8
useDetailed=true

Alternatively, delete that file then logout and login again to make KDE Plasma pick up the values of the variables from the existing configuration. KDE Plasma will recreate the file.

Quick work-around for bug in KDE Plasma 5.19.1 (Restart and Shut Down from system menu result in Log Out instead)

See the recent post on 17 June 2020 by reddit and Arch Linux user SnowGigs re KDE Plasma 5.19.1: Cannot shutdown or restart from system menu. Actually I experience the same problem with Plasma 5.19.0 in Gentoo Linux.

Until you can install a newer version of Plasma which fixes the bug, here is a quick and easy work-around that is perfectly usable:

Install kde-apps/kdialog and add the following Desktop Configuration file Shutdown.desktop to your Desktop directory — also drag it to the Panel if you wish — and make it executable:

[Desktop Entry]
Comment[en_GB]=Shutdown system
Comment=Shutdown system
Exec=kdialog --title "Shutdown system" --warningcontinuecancel "Are you sure you want to shutdown?" && loginctl poweroff
GenericName[en_GB]=Shutdown
GenericName=Shutdown
Icon=system-shutdown
MimeType=
Name[en_GB]=Shutdown
Name=Shutdown
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

(Obviously replace ‘fitzcarraldo‘ with your own username.)

user $ chown 744 ~/Desktop/Shutdown.desktop

I have assumed your installation has elogind installed, hence the command ‘loginctl poweroff‘ in the above-mentioned file. If systemd is installed instead of elogind, use the command ‘systemctl poweroff‘ instead.

When you click/double-click on the ‘Shutdown’ icon, a window will pop-up asking you ‘Are you sure you want to shutdown?’ with ‘Continue’ and ‘Cancel’ buttons. If you click ‘Continue’ then the system will execute the ‘loginctl poweroff‘ command; if you click on ‘Cancel’ then it won’t. Simple as that.

If you want, you can also create a file Reboot.desktop to reboot the system:

[Desktop Entry]
Comment[en_GB]=Reboot system
Comment=Reboot system
Exec=kdialog --title "Reboot system" --warningcontinuecancel "Are you sure you want to reboot?" && loginctl reboot
GenericName[en_GB]=Reboot
GenericName=Reboot
Icon=system-reboot
MimeType=
Name[en_GB]=Reboot
Name=Reboot
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

Similarly, if systemd is installed instead of elogind, use the command ‘systemctl reboot‘ instead.

Using WS-Discovery to enable Windows 10 to browse SMB shares in my home network of Linux computers

I have not used Windows 10 for more than two years now (see ‘Bye bye Windows 10, and good riddance‘ regarding my failed attempts to upgrade Windows 10 Version 1607 to 1703 and 1709). Nevertheless I am aware that, since Version 1709, Windows 10 no longer has SMBv1 and Computer Browser service installed by default. Computer Browser service used NetBIOS and SMBv1 to provide what Microsoft named ‘My Network Places‘ or ‘Network Neighborhood’. Thus Microsoft has dropped the concepts of network ‘workgroups’, ‘master browsers’, NetBIOS, NetBIOS broadcasts, WINS and so on. SMB has not been dropped, though; Versions 2 and 3 of the SMB protocol are now used, albeit using a different mechanism for device discovery.

Although they perform different jobs, Microsoft bundled the Computer Browser service software with the SMBv1 software. Microsoft could have provided them separately, but it made some sense to bundle them together in the early days of Windows networking. Thus, as SMBv1 is not installed by default in Windows 10 Version 1709 and later versions, neither is Computer Browser service. To put it another way, if you install SMBv1 in Windows 10 you automatically install Computer Browser service as well. None of that interested me since I stopped using Windows 10 after Version 1607. Since then my home network has comprised a server, desktop and laptops running various Linux distributions with Samba and using broadcast NetBIOS for name resolution. Of course I know that NetBIOS — especially broadcast NetBIOS for name resolution — is ancient networking technology, but it works well for my home networking needs. All my machines can browse each other’s SMB shares and create/copy/move/delete remote files and folders. The File Manager + app on my phone running Android 9 can also browse SMB shares on the Linux machines and create/copy/move/delete remote files and folders.

Two of my blog posts from 2016 and 2017 explain how I set up my home network for file sharing. One of the machines in the network had Windows 10 1607 installed, but that was replaced with Lubuntu in 2018.

SMBv1 is an inherently insecure protocol, so, after I dropped Windows, I reconfigured Samba on my Linux machines to use only SMBv3, which works fine. Subsequently I found that Android 9 on my Samsung Galaxy Note 8 phone apparently does not support SMBv3, only SMBv1 and SMBv2, so I reconfigured Samba on my Linux machines to allow SMBv2 as well as SMBv3. In other words, the Linux machines use SMBv3 with each other but SMBv2 with the phone (see my comments in the Comments section of my 2016 post ‘A correct method of configuring Samba for browsing SMB shares in a home network‘).

Anyway, I happen to have an evaluation copy of Windows 10 Enterprise Version 1709 installed in a VirtualBox VM (virtual machine) on one of my Linux laptops and, purely to satisfy my curiosity, I decided to try to get Windows 10 Version 1709 to browse and access SMB shares on the Linux machines in my home network, and vice versa, without having to dispense with broadcast NetBIOS name resolution for the Linux machines and without having to install SMBv1 (and Computer Browser service) in Windows 10.

When I first booted Windows 10 Enterprise 1709, SMB shares on my Linux machines were not displayed in File Explorer, and Windows 10 could not find them if I entered the UNC (Universal Naming Convention) address ‘\\hostname\foldername‘ or ‘\\IPaddress\foldername‘ (e.g. ‘\\AKHANATEN\anne‘ or ‘\\192.168.1.70\anne‘) in File Explorer’s address bar. My Web searches indicated that many people cannot see SMB shares in File Explorer either but can access a share by entering the UNC address in the File Explorer address bar. Apparently the advice from Microsoft these days is to use ‘Map a Network Drive…’ in File Explorer. Therefore, given that I wanted to be able to browse SMB shares in ‘File Explorer’ > ‘Network’, I clearly had some work to do. My goal for Windows 10 was twofold: to be able to view my remote SMB shares in Windows 10 File Explorer automatically and to be able to access (copy/move/delete/open) my remote SMB shares in Windows 10 File Explorer. Of course I also wanted to be able to browse and access SMB shares on the Windows 10 machine from the Linux machines.

Now, Windows 10 comes with Web Services Dynamic Discovery (WS-Discovery) installed. This enables SMB hosts running WS-Discovery software to be found by clients running WS-Discovery software. I believe Version 20.04 of the KDE Applications package kio-extras will support SMB host discovery using WS-Discovery, but that version is not available in the Stable Branch of Gentoo Linux installed on my main laptop, nor in Lubuntu 18.04 which is installed on my family’s desktop machine. So I thought I would have a look at what is currently available for those two distributions. I was particularly interested to see if I could find an implementation of WS-Discovery for Linux that would run in parallel with broadcast NetBIOS name resolution currently installed on the Linux machines in my home network, as broadcast NetBIOS name resolution works fine with SMBv2 and SMBv3 for Linux and Android devices in a home network (my Samsung Galaxy Note 8 phone can browse the SMB shares on any of the Linux machines in my home network).

Thanks are due to Steffen Christgau for creating a daemon that can be used in Linux installations to enable Windows 10 to discover SMB shares on Linux machines via WS-Discovery: wsdd – A Web Service Discovery host daemon. The README file for wsdd states:

wsdd implements a Web Service Discovery host daemon. This enables (Samba) hosts, like your local NAS device, to be found by Web Service Discovery Clients like Windows.

It also implements the client side of the discovery protocol which allows to search for Windows machines and other devices implementing WSD. This mode of operation is called discovery mode.

wsdd only depends on Python 3 and can be installed in many Linux distributions. If no wsdd package exists for a specific distribution, it can simply be run from the command line or from a Bash script. The following blog post by Ralph Mönchmeyer explains how to use wsdd (although not a complete solution for my specific case): Samba 4, shares, wsdd and Windows 10 – how to list Linux Samba servers in the Win 10 Explorer.

Below I list the steps I took to enable me to browse SMB shares in an evaluation copy of Windows 10 Enterprise Version 1709 running in a VM on one of my Linux laptops. I don’t have access to the latest version of Windows 10 (2004), but hopefully some or most of the following will still be applicable.

Step 1. Disable firewalls temporarily

I disabled the firewall in the Linux machine and in the Windows 10 machine so that the firewalls could be ruled out if there were any problems getting share browsing to work. Once all the steps were completed I re-enabled the firewalls.

Step 2. Specify the workgroup in Windows 10

Select ‘Control Panel’ > ‘System and Security’ > ‘System’ and, under ‘Computer name, domain, and workgroup settings’, if necessary click ‘Change settings’ to rename the workgroup. The default workgroup name was ‘WORKGROUP‘ so I renamed it to ‘HOME‘, my current network’s workgroup.

Step 3. Ensure the correct SMB protocol in Windows 10

SMBv1 (and Computer Browser service) are disabled by default in Windows 10 Version 1709 and later (see ‘SMBv1 is not installed by default in Windows 10 version 1709, Windows Server version 1709 and later versions‘) but I nevertheless made sure that SMBv1 is disabled and that SMBv2 and SMBv3 are installed (see ‘How to detect, enable and disable SMBv1, SMBv2, and SMBv3 in Windows‘). I did the following in PowerShell (Run as administrator):

PS C:\WINDOWS\system32> Disable-WindowsOptionalFeature -Online -FeatureName SMB1Protocol
PS C:\WINDOWS\system32> Set-SmbServerConfiguration -EnableSMB2Protocol $true

Step 4. Disable NetBIOS-over-TCP/IP in Windows 10

Select ‘Settings’ > ‘Network & Internet’ > ‘Ethernet’ > ‘Change adapter options’.

Right-click ‘Ethernet’, click ‘Properties’, select ‘Internet Protocol Version 4 (TCP/IPv4)’ and click ‘Properties’. Click ‘Advanced’. Click on the WINS tab (even though my network does not use WINS), select ‘Disable NetBIOS over TCP/IP’ and click ‘OK’, ‘OK’ and ‘Close’.

Step 5. Configure ‘Function Discovery’ in Windows 10

See the article ‘SMBv1 is not installed by default in Windows 10 version 1709, Windows Server version 1709 and later versions | Microsoft Docs‘, in particular the following:

Explorer Network Browsing

The Computer Browser service relies on the SMBv1 protocol to populate the Windows Explorer Network node (also known as “Network Neighborhood”). This legacy protocol is long deprecated, doesn’t route, and has limited security. Because the service cannot function without SMBv1, it is removed at the same time.

However, if you still have to use the Explorer Network in home and small business workgroup environments to locate Windows-based computers, you can follow these steps on your Windows-based computers that no longer use SMBv1:

  1. Start the “Function Discovery Provider Host” and “Function Discovery Resource Publication” services, and then set them to Automatic (Delayed Start).
  2. When you open Explorer Network, enable network discovery when you are prompted.

All Windows devices within that subnet that have these settings will now appear in Network for browsing. This uses the WS-DISCOVERY protocol. Contact your other vendors and manufacturers if their devices still don’t appear in this browse list after the Windows devices appear. It is possible they have this protocol disabled or that they support only SMBv1.

Press Windows Key+R, enter ‘services.msc‘ (without the quotes) and click ‘OK’.

Change the ‘Startup type’ of ‘Functions Discovery Provider Host’ to ‘Automatic (Delayed Start)’.

Change the ‘Startup type’ of ‘Function Discovery Resource Publication’ to ‘Automatic (Delayed Start)’.

Step 6. Configure the sharing options in Windows 10

Select ‘Settings’ > ‘Network & Internet’ > ‘Sharing options’ and configure the options as follows:

Private (current profile)
  1. Network discovery
    • ‘Turn on network discovery’ is selected.
    • ‘Turn on automatic setup of network connected devices.’ is ticked.
  2. File and printer sharing
    • ‘Turn on file and printer sharing’ is selected.
  3. HomeGroup connections
    • ‘Allow Windows to manage homegroup connections (recommended)’ is selected.
Guest or Public
  1. Network discovery
    • ‘Turn on network discovery’ is selected.
  2. File and printer sharing
    • ‘Turn on file and printer sharing’ is selected.
All Networks
  1. Public folder sharing
    • ‘Turn on sharing so anyone with network access can read and write files in the Public folders’ is selected.
  2. Media streaming
    • Nothing is selected.
  3. File sharing connections
    • ‘Use 128-bit encryption to help protect file sharing connections (recommended)’ is selected.
  4. Password protected sharing
    • ‘Turn off password protected sharing’ is selected.

Step 7. Install WS-Discovery daemon on the Linux machines

Gentoo Linux
In Gentoo I simply installed the package net-misc/wsdd from the guru overlay:

root # eix -I wsdd
[I] net-misc/wsdd [1]
     Available versions:  (~)0.5 (~)0.6 {samba PYTHON_TARGETS="python3_6 python3_7 python3_8"}
     Installed versions:  0.6(00:39:07 07/06/20)(-samba PYTHON_TARGETS="python3_7 -python3_6 -python3_8")
     Homepage:            https://github.com/christgau/wsdd
     Description:         A Web Service Discovery host daemon.

[1] "guru" /var/lib/layman/guru

and, as I use OpenRC in Gentoo, I configured /etc/conf.d/wsdd.conf as follows:

# /etc/conf.d/wsdd

# Override the default user/group under which wsdd runs.
# Must follow the user[:group] notation.
#WSDD_USER="daemon:daemon"

# Specify alternative log file location.
#WSDD_LOG_FILE="/var/log/wsdd.log"

# Disable automatic detection of the workgroup from samba configuration.
#WSDD_WORKGROUP="MYGROUP"
WSDD_WORKGROUP="HOME"

# Additional options for the daemon, e.g. to listen on interface eth0 only.
# Refer to wsdd(1) for details.
#WSDD_OPTS="-i eth0"
WSDD_OPTS="--shortlog --interface enp4s0f1 --interface wlp3s0 --hostname tutankhamun --discovery"

Specifying the interfaces and hostname are optional, but wsdd seemed to work better when I specified them explicitly. You can ascertain the interfaces by using the command ‘ip address‘ or the deprecated command ‘ifconfig‘.

I added the service to the default runlevel so that it is started automatically when I boot the machine, and then I started it:

root # rc-update add wsdd default
root # rc-service wsdd start

Lubuntu 18.04

In Lubuntu 18.04 (which uses systemd) wsdd can be installed either manually or from a package:

a) Manual installation

user $ wget https://github.com/christgau/wsdd/archive/master.zip
user $ unzip master.zip
user $ sudo cp wsdd-master/src/wsdd.py /usr/bin/wsdd
user $ sudo cp wsdd-master/etc/systemd/wsdd.service /etc/systemd/system/

Edit the systemd service file /etc/systemd/system/wsdd.service to add desired options to the ExecStart command and to change the group from ‘nobody‘ to ‘nogroup‘:

...
ExecStart=/usr/bin/wsdd --workgroup HOME --shortlog --interface eno1 --interface wlp2s0 --hostname thutmoseiii --discovery
...
User=nobody
Group=nogroup
...

You can check whether the user and group exist in your installation as follows:

user $ grep ^nobody /etc/passwd
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
user $ grep ^nobody /etc/group
user $ grep ^nogroup /etc/group
nogroup:x:65534:

Actually, I prefer to specify ‘daemon‘ for the user and group in the wsdd.service file (which is also what the Gentoo Linux ebuild uses and what the .deb package uses):

...
ExecStart=/usr/bin/wsdd --workgroup HOME --shortlog --interface eno1 --interface wlp2s0 --hostname thutmoseiii --discovery
...
User=daemon
Group=daemon
...

You can check that this user and group also exist:

user $ grep ^daemon /etc/passwd
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
user $ grep ^daemon /etc/group
daemon:x:1:

(I tried both nobody:nogroup and daemon:daemon, and there was no apparent difference in behaviour.)

Enable the service so that it starts automatically when the machine is booted, and also start it now:

user $ sudo systemctl enable wsdd
user $ sudo systemctl start wsdd

b) Installing from a package

Here is a link to a .deb package for wsdd Version 0.6.0:

https://pkg.ltec.ch/public/pool/main/w/wsdd/

The resulting installation differs slightly from the manual procedure; the package creates a configuration file /etc/wsdd.conf and you declare the wsdd options in that file instead:

# command line parameters for wsdd (consult man page)
WSDD_PARAMS=""

The package also installs a systemd service file /lib/systemd/system/wsdd.service containing the following:

[Unit]
Description=Web Services Dynamic Discovery host daemon
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
EnvironmentFile=/etc/wsdd.conf
ExecStart=/usr/bin/wsdd $WSDD_PARAMS
User=daemon
Group=daemon

[Install]
WantedBy=multi-user.target

The package installs the Python 3 executable wsdd in the directory /usr/bin/. It’s a very straightforward package.

Step 8. Configure Samba to make Windows 10 prompt for username and password

When you click on a network share in Windows 10’s File Explorer, Windows 10 uses the Windows 10 username and password to try to access the SMB share on the remote machine (see ‘Samba share does not ask for credentials from Windows Client‘). This will obviously not work unless the usernames/passwords on both machines match. To make Windows 10 prompt the user to enter the remote username and password, edit the file /etc/samba/smb.conf on each Linux machine and comment out the line ‘map to guest = bad user‘ (see the smb.conf files listed in my 2016 article ‘A correct method of configuring Samba for browsing SMB shares in a home network‘).

Step 9. Enable guest access in Windows 10

If I enter a SMB share’s UNC address in File Explorer’s address bar, or if I double-click on the remote machine’s icon in File Explorer (after WS-Discovery has made the SMB share visible in File Explorer), Windows 10 displays the following error message:

Network Error

Windows cannot access \\hostname

Check the spelling of the name. Otherwise, there might be a problem with your network. To try to identify and resolve network problems, click Diagnose.

Error Code: 0x80070035
The network path was not found.

This has nothing to do with the fact that SMBv1 is disabled in Windows 10. It happens because Windows 10 1709 and onwards have guest logins disabled:

To enable guest logins I edited the Windows 10 Registry and changed the following key from zero to one:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters]

“AllowInsecureGuestAuth”=dword:1

Step 10. Configure the Windows 10 firewall

Select ‘Windows Defender Security Center’ > ‘Firewall & network protection’.

Click ‘Allow an app through the firewall’.

If not already ticked, select ‘Private’ and ‘Public’ for ‘Network Discovery’ and for ‘File and Printer Sharing’.

Step 11. Configure the Linux firewall

This is where things get more complicated. According to the README for wsdd:

Firewall Setup

Both incoming and outgoing multicast traffic on port 3702 must be allowed. For IPv4, the multicast address is 239.255.255.250, for IPv6 the link local SSDP multicast address (ff02::c) is used.

Incoming TCP traffic (and related outgoing traffic) on port 5357 must be allowed.

My laptops and desktop use UFW, and below I explain how I configured UFW to satisfy the above requirements.

Firstly, as my firewall is configured to deny incoming traffic and allow outgoing traffic by default, I enabled UFW and added the following DNS rules to UFW’s main rules (the following two commands add rules for both IPv4 and IPv6):

user $ sudo ufw allow 53/tcp
user $ sudo ufw allow 53/udp

Note that, in order for the multicast rule I use to work, xt_pkttype must either have been built into the kernel or built as a kernel module and have been loaded:

user $ lsmod | grep pkttype
xt_pkttype             16384  2
x_tables               40960  17 ip6table_filter,xt_conntrack,iptable_filter,xt_LOG,xt_multiport,xt_tcpudp,xt_addrtype,ip6t_rt,ip6_tables,ipt_REJECT,xt_CT,xt_pkttype,iptable_raw,ip_tables,xt_limit,xt_hl,ip6t_REJECT

To load the module automatically at boot, in Gentoo Linux I added ‘xt_pkttype‘ to the list of modules in the file /etc/conf.d/modules, and in Lubuntu 18.04 I added ‘xt_pkttype‘ to the list of modules in the file /etc/modules-load.d/modules.conf.

Also note that my firewall had previously already been configured for NetBIOS and SMB by using the following commands:

user $ # Rules for SMB
user $ # IPv4:
user $ sudo ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
user $ sudo ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp
user $ # IPv6:
user $ # (NetBIOS is undefined for IPv6 but I believe SMB uses Port 445 in IPv6)
user $ sudo ufw allow from ff80::/10 to any port 445 proto tcp

IPv4

The end of the file /etc/ufw/before.rules previously looked like this:

...
# allow MULTICAST mDNS for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 224.0.0.251 --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 239.255.255.250 --dport 1900 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

I inserted seven lines as shown below:

...
# allow MULTICAST mDNS for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 224.0.0.251 --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 239.255.255.250 --dport 1900 -j ACCEPT

# allow MULTICAST WS-Discovery for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -m pkttype --pkt-type multicast -j ACCEPT
-A ufw-before-input -p udp -s 192.168.1.0/24 --dport 3702 -j ACCEPT
-A ufw-before-input -p udp -s 192.168.1.0/24 --sport 3702 -j ACCEPT
-A ufw-before-input -p tcp -s 192.168.1.0/24 --dport 5357 -j ACCEPT
-A ufw-before-input -p tcp -s 192.168.1.0/24 --sport 5357 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

Actually the two IPv4 rules shown above for mDNS and UPnP that were already in the file /etc/ufw/before.rules have become redundant because the first of the five new rules I added encompasses them. It does no harm to leave those two rules in the file, though.

IPv6

The end of the file /etc/ufw/before6.rules previously looked like this:

...
# allow MULTICAST mDNS for service discovery
-A ufw6-before-input -p udp -d ff02::fb --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery
-A ufw6-before-input -p udp -d ff02::f --dport 1900 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT

I inserted six lines as shown below:

...
# allow MULTICAST mDNS for service discovery
-A ufw6-before-input -p udp -d ff02::fb --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery
-A ufw6-before-input -p udp -d ff02::f --dport 1900 -j ACCEPT

# allow MULTICAST WS-Discovery for service discovery
-A ufw6-before-input -m pkttype --pkt-type multicast -j ACCEPT
-A ufw6-before-input -p udp -s fe80::/10 --dport 3702 -j ACCEPT
-A ufw6-before-input -p udp -s fe80::/10 --sport 3702 -j ACCEPT
-A ufw6-before-input -p tcp -s fe80::/10 --dport 5357 -j ACCEPT
-A ufw6-before-input -p tcp -s fe80::/10 --sport 5357 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT

Actually the two IPv6 rules shown above for mDNS and UPnP that were already in the file /etc/ufw/before6.rules have become redundant because the first of the five new rules I added encompasses them. It does no harm to leave those two rules in the file, though.

Because the Linux machines in my network still use broadcast NetBIOS for name resolution I left all the NetBIOS rules in UFW as they were, including the extra lines I previously added to /etc/ufw/before.rules (see the raw table rule at the end of /etc/ufw/before.rules listed above and my blog post ‘Prevent Linux firewalls interfering with Samba commands in a home network that uses broadcast NetBIOS name resolution‘).

Actually, as my laptops change firewall zones automatically (see my post ‘Firewall zones (profiles) in Linux, and how to switch them automatically if you use UFW‘), on my laptops I added the new rules to the zone for my home network specified in my NetworkManager Dispatcher hook script /etc/NetworkManager/dispatcher.d/20_ufw-zones.

After reloading UFW, the UFW status on my machines now looks like this (I’ve excluded rules unrelated to this topic):

user $ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
137,138/udp                ALLOW IN    192.168.1.0/24
139,445/tcp                ALLOW IN    192.168.1.0/24
53/tcp                     ALLOW IN    Anywhere
53/udp                     ALLOW IN    Anywhere
445/tcp                    ALLOW IN    ff80::/10
53/tcp (v6)                ALLOW IN    Anywhere (v6)
53/udp (v6)                ALLOW IN    Anywhere (v6)

Note that UFW does not display rules declared in /etc/ufw/{before,before6}.rules

Step 12. Re-enable the Windows 10 firewall

Select ‘Settings’ > ‘Network & Internet’ > ‘Windows Firewall’.

Step 13. Check that wsdd is working as expected

To check that wsdd is actually detecting other machines running WS-Discovery, you can stop the daemon running and instead launch wsdd manually in a terminal window with verbose logging enabled.

For example, on my laptop running Gentoo Linux I did the following:

user $ sudo rc-service wsdd stop
user $ wsdd --workgroup HOME --verbose --interface enp4s0f1 --interface wlp3s0 --hostname tutankhamun --discovery

And on my family’s desktop running Lubuntu 18.04 I did the following:

user $ sudo systemctl stop wsdd
user $ wsdd --workgroup HOME --verbose --interface eno1 --interface wlp2s0 --hostname thutmoseiii --discovery

Check the output in the terminal window includes a discovered line for each machine running Windows 10 and for each Linux machine running wsdd. For example:

...
2020-06-16 00:31:09,331:wsdd INFO(pid 17574): discovered MSWIN10PC in Workgroup:HOME on 192.168.1.111%eno1
...
2020-06-16 00:31:10,013:wsdd INFO(pid 17574): discovered MSWIN10PC in Workgroup:HOME on [fe80::fc7e:7068:8c2c:e664]%eno1
...

After pressing Ctrl+C to stop wsdd running in the terminal, you can restart the daemon:

Gentoo Linux

user $ sudo rc-service wsdd start

Lubuntu 18.04

user $ sudo systemctl start wsdd

With wsdd running on the Linux machines they become visible in File Explorer on Windows 10 machines connected to the network. However, the converse is not necessarily true, as explained further on.

As I had previously configured Samba on my Linux machines to use broadcast NetBIOS to resolve names, Samba on the Linux machines fails to resolve the hostnames of the Windows 10 machines because Windows 10 no longer supports NetBIOS name resolution (neither broadcast nor WINS). I confirmed this by using the smbclient command in a terminal window:

user $ sudo smbclient //MSEDGEWIN10/TestSMBShare1 --debuglevel=10
...
added interface eno1 ip=192.168.1.111 bcast=192.168.1.255 netmask=255.255.255.0
Netbios name list:-
my_netbios_names[0]="THUTMOSEIII"
Client started (version 4.7.6-Ubuntu).
Opening cache file at /var/cache/samba/gencache.tdb
Opening cache file at /var/run/samba/gencache_notrans.tdb
sitename_fetch: No stored sitename for realm ''
internal_resolve_name: looking up MSEDGEWIN10#20 (sitename (null))
no entry for MSEDGEWIN10#20 found.
name_resolve_bcast: Attempting broadcast lookup for name MSEDGEWIN10
Connection to MSEDGEWIN10 failed (Error NT_STATUS_UNSUCCESSFUL)

However, in Gentoo Linux (Stable Branch, KDE Plasma 5.18.5, KDE Applications 19.12.3) on my main laptop I can enter ‘smb://hostname/sharename‘ (e.g. smb://msedgewin10/Users/Public) in the Dolphin file manager’s address bar and browse the contents of the SMB share on the Window 10 machine. I assume this is because Avahi on the Linux machine performs name resolution anyway even though the broadcast NetBIOS lookup has failed. Although Lubuntu 18.04 also has the Avahi daemon running, it does not resolve the hostname when I enter ‘smb://hostname/sharename‘ in PCManFM’s address bar; I have to enter ‘smb://IPaddress/sharename‘ (e.g. smb://192.168.1.64/Users/Public) to be able to browse the contents of the Windows 10 shared folder.

Conclusion

wsdd running on Linux machines enables Windows 10 to view networked Linux machines in File Explorer and browse SMBv2 and SMBv3 shares residing on Linux machines. It does not guarantee I will be able to view Windows 10 machines in Linux file managers automatically, though. But I can access Windows 10 machines by entering ‘smb://IPaddress/sharename‘ in the Linux file manager’s address bar, or, depending on what has been installed in the Linux installation and how it has been configured, by entering ‘smb://hostname/sharename‘.

To access a Linux SMB shared folder (as declared in that machine’s smb.conf file) in Windows 10 File Explorer, either I double-click on the Linux machine’s icon in the Network view or I enter the UNC address (e.g. \\tutankhamun\Users\Public) in the address bar. I can then access the files and sub-folders.

To browse a Windows 10 SMB shared folder and files in KDE Dolphin in Gentoo Linux current Stable Branch on my main laptop, I enter the UNC address (e.g. smb://msedgewin10/Users/Public) or click on the location I previously bookmarked under ‘Places’ in the left pane of the Dolphin window. I can then access the files and sub-folders. To browse a Windows 10 SMB shared folder and files in LXDE PCManFM in Lubuntu 18.04, I enter the UNC address with an IP address instead of a hostname (e.g. smb://192.168.1.64/Users/Public). I can then access the files and sub-folders. I am going to have to do some more digging to try to find out why KDE Dolphin in Gentoo Linux on my main laptop (kio-extras installed from Gentoo ebuild kio-extras-19.12.3-r2) can access Windows 10 by hostname but PCManFM in Lubuntu 18.04 cannot.

To enable machines running Window 10 to browse SMB shares on my other Linux machines I would need to perform the same Linux-related steps in each of those installations. My server firewall uses IPTABLES directly, rather than UFW, so the syntax of the additional firewall rules would be different.

Addendum, 16 June 2020: I suspected the problem browsing the Windows 10 SMB shares from Lubuntu 18.04 is due to PCManFM, so I installed a different file manager: SpaceFM (Version 1.0.5 for GTK2) and its associated utility udevil (Version 0.4.4). SpaceFM allows me to enter UNC addresses such as ‘smb://mswin10pc/Users/Public‘ without any problems. So, problem solved in Lubuntu 18.04 now as well.

How to send a message to running X Windows sessions in a multi-user Linux system

The ‘wall‘ command can be used to broadcast a message from a TTY console to other logged-in TTY console users in a multi-user Linux system. The Linux command ‘notify-send‘ can be used to send a message (a.k.a. notification) within an X Windows session, and Desktop Environments such as KDE and GNOME use notify-send to display pop-up notifications to the user. However, apparently no program exists in Linux to broadcast a message to other running X Windows sessions; that sounds like the sort of thing systemd developers would implement. A few years ago Unix & Linux Stack Exchange user Andy posted a Bash script notify-send-all to do just that (see Show a notification across all running X displays). For example, if you wanted to send a message to all the users of a multi-seat, multi-user Linux system who are currently logged-in to Desktop Environments, you could enter the following command to run the script in your home directory:

$ sudo ./notify-send-all -t 50000 "Warning" "Don't forget the staff meeting at 15:00 today."

Below is a slightly modified version of Andy‘s script that works for me in Lubuntu 18.04:

#!/bin/bash
PATH=/usr/bin:/bin
who|grep -E "\(:[0-9](\.[0-9])*\)"|awk '{print $1$5}'|sort -u > /tmp/xusers
while read XUSER; do
    NAME=(${XUSER/(/ })
    DISPLAY=${NAME[1]/)/}
    DBUS_ADDRESS=unix:path=/run/user/$(id -u ${NAME[0]})/bus
    sudo -u ${NAME[0]} DISPLAY=${DISPLAY} \
                       DBUS_SESSION_BUS_ADDRESS=${DBUS_ADDRESS} \
                       PATH=${PATH} \
                       notify-send "$@"
done < /tmp/xusers

Here is my tidied-up version:

#!/bin/bash
who | awk '{print $1, $NF}' | tr -d "()" | sort -u |
while read XUSER DISPNUM; do
    sudo -u $XUSER DISPLAY=$DISPNUM \
                   DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u $XUSER)/bus \
                   notify-send "$@"
done

In Gentoo Linux DBUS_SESSION_BUS_ADDRESS needs to be found differently, and the following version of Andy‘s script works for me in that Linux distribution:

#!/bin/bash
PATH=/usr/bin:/bin
export $(dbus-launch)
who|grep -E "\(:[0-9](\.[0-9])*\)"|awk '{print $1$5}'|sort -u > /tmp/xusers
while read XUSER; do
    NAME=(${XUSER/(/ })
    DISPLAY=${NAME[1]/)/}
    sudo -u ${NAME[0]} DISPLAY=${DISPLAY} \
                       ${DBUS_SESSION_BUS_ADDRESS} \
                       PATH=${PATH} \
                       notify-send "$@"
done < /tmp/xusers

And here is my tidied-up version:

#!/bin/bash
export $(dbus-launch)
who | awk '{print $1, $NF}' | tr -d "()" | sort -u |
while read XUSER DISPNUM; do
    sudo -u $XUSER DISPLAY=$DISPNUM \
                   $BUS_SESSION_BUS_ADDRESS \
                   notify-send "$@"
done

notify-send-all will be of academic interest to users of single-user systems, but it’s nice to know such a thing is possible relatively easily in Linux.

Jitsi Meet, my favourite video conferencing platform (and a way to share audio when using it in Linux)

During the current COVID-19 lockdown I have been using video conferencing platforms a lot for family virtual meet-ups, quizzes and multi-player games by Jackbox Games. Zoom seems to be the most popular video conference platform at the moment, although several articles in the media have pointed out some of its security limitations (see, e.g., ‘‘Zoom is malware’: why experts worry about the video conferencing platform‘). Although many people like Zoom, my favourite video conferencing platform is Jitsi Meet.

For an excellent third-party video introduction to Jitsi Meet, watch the video: ‘Using Jitsi: A free, no-registration video conferencing site‘. WIRED Magazine’s recent article on Jitsi Meet is also worth reading: ‘Want to Ditch Zoom? Jitsi Offers an Open-Source Alternative‘.

The reasons I prefer Jitsi Meet to Zoom include the following:

  1. no subscriptions are required to use all the features of Jitsi Meet;
  2. unlike Zoom, Jitsi Meet does not require you to sign up;
  3. unlike Zoom, Jitsi Meet does not require the installation of an application — it runs in Google Chrome or Firefox;
  4. unlike the free version of Zoom, Jitsi Meet does not impose a time limit on the length of the meeting;
  5. unlike the free version of Zoom, Jitsi Meet does not have a limit on the number of meeting attendees;
  6. Jitsi Meet provides end-to-end encryption for one-to-one video calls*;
  7. I find the performance of Jitsi Meet better than Zoom, which seems to be corroborated in basic benchmarking by Jitsi Meet’s developers (‘WebRTC vs. Zoom – A Simple Congestion Test‘);
  8. I find image quality better in Jitsi Meet;
  9. I find Jitsi Meet on a desktop/laptop more intuitive and easier to use than Zoom;
  10. if I share audio in Zoom for Linux, the audio is very distorted**;
  11. I find the UI of the Jitsi Meet app for Android easy to use (the app can be installed via Google Play);
  12. Jitsi Meet is open-source, so anyone can inspect the source code;
  13. if I wanted to, I could download the Jitsi software to my own server and set up a Jitsi Meet server to handle meetings instead of using the Cloud server provided by 8×8, Inc. (the company that develops the Jitsi Meetings software).

* Neither platform currently provides end-to-end encryption for group meetings, although the developers of Jitsi Meet are apparently working on implementing end-to-end encryption for group meetings using a new feature of Google Chrome called ‘Insertable Streams’.

** There is a work-around for this problem in Zoom for Linux; see my answer to the Unix & Linux Stack Exchange question ‘Play audio output as input to Zoom’. In the case of Jitsi Meet in Linux, PulseAudio Volume Control can be used to share audio, as I explain further down.

Jitsi Meet requires no installation; it runs in a browser window. Either Google Chrome or Firefox can be used, although I find it runs better in Google Chrome. Actually, an Ubuntu 16.04 user told me that Firefox hangs when he tries to join a Jitsi Meet meeting, but Jitsi Meet works fine in Firefox in my two Gentoo Linux installations and in my family’s Lubuntu 18.04 installation. When using Google Chrome, to be able to share your screen you need to install the Google Chrome extension ‘Jitsi Meetings’ by meet.ji.si in the Google Chrome Web Store.

One of my family here at home has a laptop running Windows 10. Google Chrome, but not Firefox, displays a ‘Share audio’ tick box when the ‘Share your screen’ icon is clicked (see ‘Jitsi Meet features update, April 2020‘). The ‘Share audio’ feature is needed when, for example, you are casting via the Internet to remote players a multi-user game running on your machine. During the current COVID-19 lockdown we have been having fun playing Jackbox Games Party Pack 6 this way with family and friends in different locations (see ‘(My Solution) Best method for Virtual Couch Multiplayer‘). Each household connects a laptop to their TV via HDMI and joins the Jitsi Meet meeting. The Jackbox Games games are cast via Jitsi Meet from the laptop at my house, and the group of players in each household can view and hear the game on their TV and participate using their mobile phones as per the Jackbox Games paradigm.

Jitsi Meet provides a ‘Share audio’ function in Windows only, but I found a work-around to to be able to share any application’s audio in Linux if I ever want to use my Linux machines to cast games by Jackbox Games or other suppliers. For once, I have found PulseAudio useful! I use PulseAudio Volume Control to redirect the audio output from the desired application (be it a game, music player, video player or whatever) to the microphone input. The precise way to do this depends on the audio hardware your machine has, but an example is given in the blog post: ‘Redirect Audio Out to Mic In (Linux)‘.

My family’s desktop machine running Lubuntu 18.04 uses a Webcam with an integral microphone connected via USB, and external powered speakers connected to the machine’s Line Out green-coloured 3.5 mm jack socket. The contents of the tabs in PulseAudio Volume Control when no applications that produce audio are running are shown in the following screenshots:

PulseAudio Volume Control - Configuration

PulseAudio Volume Control - Playback

PulseAudio Volume Control - Recording

PulseAudio Volume Control - Output Devices

PulseAudio Volume Control - Input Devices

I make sure ‘All Streams’ is selected on the ‘Playback’ and ‘Recording’ tabs, ‘All Output Devices’ is selected on the ‘Output Devices’ tab, and ‘All Input Devices’ is selected on the ‘Input Devices’ tab.

Let us say I have launched Audacious to play some music and I want to cast that music to members of a Jitsi Meet meeting. When I am using Jitsi Meet for a meeting in Google Chrome, the contents of the PulseAudio Volume Control tabs on this machine are as follows:

PulseAudio Volume Control - Playback

PulseAudio Volume Control - Recording

PulseAudio Volume Control - Output Devices

PulseAudio Volume Control - Input Devices

To redirect the audio from e.g. Audacious to the meeting members, I select (click on the square button with the green disc and white tick) ‘Monitor of Built-in Audio Analogue Stereo’ on the ‘Input Devices’ tab, and on the ‘Recording’ tab I click on Chrome input: RecordStream from ‘Camera Analogue Mono’ and select Chrome input: RecordStream from ‘Monitor of Built-in Audio Analogue Stereo’, as shown below.

PulseAudio Volume Control - Recording

PulseAudio Volume Control - Input Devices

With the above settings in PulseAudio volume control, all the members of the meeting will be able to hear clearly the audio from Audacious. To switch back to my microphone to speak, I simply click on Chrome input: RecordStream from ‘Monitor of Built-in Audio Analogue Stereo’ on the ‘Recording’ tab and select Chrome input: RecordStream from ‘Camera Analogue Mono’ again.

Using Bash scripts in Linux to delete the history, cookies and cache files of Firefox, Google Chrome and Thunderbird

The browsing data stored by Firefox, Google Chrome and Thunderbird can be deleted using the respective application’s GUI. But you can also do that using a Bash script, which could be useful if you want to delete unnecessary/unwanted files before e.g. backing up your home directory, or if you want a quick and easy way to clear-out browsing data. In this post I list the scripts and Desktop Configuration files I have created in Gentoo Linux and in Lubuntu 18.04 to remove browsing data.

In the case of Thunderbird, I am not sure if it is safe to delete Thunderbird’s cache files so my script only deletes cookies. Anyway, that could be added later if it transpires there is no harm in deleting Thunderbird’s cache files.

I am using the following 64-bit versions of the two browsers and e-mail client:

  • Mozilla Firefox 74.0
  • Mozilla Thunderbird 68.5.0 in Gentoo Linux only
  • Google Chrome 80.0.3987.132 in Gentoo Linux
  • Google Chrome 67.0.3396.99 in Lubuntu 18.04

I have not tested my scripts with other versions of Firefox, Chrome and Thunderbird, nor in other installations, so please do check carefully the directory paths and commands in the script against the directory paths in your installation before selecting ‘[D]elete‘ in the running script.

In Lubuntu 18.04 I had to install sqlite3 first:

$ sudo apt install sqlite3

In Gentoo Linux it was already installed:

$ eix -I sqlite
[I] dev-db/sqlite
     Available versions:  (3) 3.29.0^t 3.30.1^t 3.31.1^t
       {debug doc icu +readline secure-delete static-libs tcl test tools ABI_MIPS="n32 n64 o32" ABI_RISCV="lp64 lp64d" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  3.31.1(3)^t(19:53:28 13/03/20)(icu readline secure-delete -debug -doc -static-libs -tcl -test -tools ABI_MIPS="-n32 -n64 -o32" ABI_RISCV="-lp64 -lp64d" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            https://sqlite.org/
     Description:         SQL database engine

Firefox and Thunderbird

I created the Bash script Firefox_or_Thunderbird_-_Clear_data.sh listed below. The user can select only Firefox or only Thunderbird, or both, and the script enables the user to choose whether to just view the current situation or to delete the data. The script checks if the applications are running and will not do anything if they are. In fact, the script offers the user the option to terminate the applications if they happen to be running. The script is still usable if either Firefox or Thunderbird are not installed. The same script can be used in Gentoo and in Lubuntu 18.04, and I believe it would also work in Ubuntu but have not tested it with that distribution.

#!/bin/bash
#
# If Delete is selected for Firefox, this script deletes the entire history, cookies, site data and cache.
# If Delete is selected for Thunderbird, this script deletes the cookies and leaves the cache intact.
#
echo
echo "WARNING:"
echo "The Firefox Browser must not be running if you are going to list or delete its data files."
echo "The Thunderbird e-mail client must not be running if you are going to list or delete its data files."
echo
FIREFOX=$( ls $HOME/.mozilla/firefox 2>/dev/null | grep .default )
THUNDERBIRD=$( ls $HOME/.thunderbird 2>/dev/null | grep .default )
while true
do
   echo -n "[F]irefox, [T]hunderbird, [B]oth or [E]xit: "
   read -n1 PROMPT2
   echo
   case $PROMPT2 in
      [fF]* ) CHOICE2="F"; break;;
      [tT]* ) CHOICE2="T"; break;;
      [bB]* ) CHOICE2="B"; break;;
      [eE]* ) exit;;
      * )     echo "Invalid entry.";;
   esac
done
FRUNNING="N"
pgrep -u $USER firefox > /dev/null
if [[ $? -eq 0 ]]; then
   FRUNNING="Y"
   while true
   do
      echo
      echo -n "The Firefox browser is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT0
      echo
      case $PROMPT0 in
         [yY]* ) CHOICE0="Y"; break;;
         [nN]* ) CHOICE0="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE0 == "Y" ]]; then
      FPID=$( pgrep -u $USER firefox )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      FRUNNING="N"
   fi
fi
TRUNNING="N"
pgrep -u $USER thunderbird > /dev/null
if [[ $? -eq 0 ]]; then
   TRUNNING="Y"
   while true
   do
      echo -n "The Thunderbird e-mail client is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT1
      echo
      case $PROMPT1 in
         [yY]* ) CHOICE1="Y"; break;;
         [nN]* ) CHOICE1="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE1 == "Y" ]]; then
      FPID=$( pgrep -u $USER thunderbird )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      TRUNNING="N"
   fi
fi
ABORT="N"
if [[ $FRUNNING == "Y" ]]; then
   if [[ $CHOICE2 == "F" ]] || [[ $CHOICE2 == "B" ]]; then
      echo
      echo "Please quit Firefox then re-run this script."
      echo
      ABORT="Y"
   fi
fi
if [[ $TRUNNING == "Y" ]]; then
   if [[ $CHOICE2 == "T" ]] || [[ $CHOICE2 == "B" ]]; then
      echo
      echo "Please quit Thunderbird then re-run this script."
      echo
      ABORT="Y"
   fi
fi
if [[ $ABORT == "N" ]]; then
   while true
   do
      echo
      echo -n "[D]elete, [L]ist or [E]xit: "
      read -n1 PROMPT3
      echo
      case $PROMPT3 in
         [dD]* ) CHOICE3="D"; break;;
         [lL]* ) CHOICE3="L"; break;;
         [eE]* ) exit;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE3 == "D" ]]; then
      if [[ $CHOICE2 == "F" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$FIREFOX" ]; then
            echo "Firefox directory does not exist"
         else
            echo
            echo "Deleting Firefox History..........."
            echo "==================================="
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/places.sqlite "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;"
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/places.sqlite "delete from moz_historyvisits;"
            echo
            echo "Deleting Firefox Cookies........"
            echo "================================"
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;"
            echo
            echo "Deleting Firefox Site Data........"
            echo "=================================="
            ls ${HOME}/.mozilla/firefox/${FIREFOX}/storage/default/ | grep http
            find ${HOME}/.mozilla/firefox/${FIREFOX}/storage/default -name "http*" -type d -exec rm -r "{}" \; -prune
            echo
            echo "Deleting Firefox Cache..........."
            echo "================================="
            NUM=$( ls -1 ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries | wc -l )
            SIZ=$( du -sbh ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo "Files: $NUM Size: $SIZ"
            find ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries -type f -delete 2>/dev/null
            echo
         fi
      fi
      if [[ $CHOICE2 == "T" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$THUNDERBIRD" ]; then
            echo "Thunderbird directory does not exist"
         else
            echo "Deleting Thunderbird Cookies........"
            echo "===================================="
            sqlite3 ${HOME}/.thunderbird/${THUNDERBIRD}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;"
         fi
      fi
   fi
   if [[ $CHOICE3 == "L" ]]; then
      if [[ $CHOICE2 == "F" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$FIREFOX" ]; then
            echo "Firefox directory does not exist"
         else
            echo
            echo "Current Firefox History........."
            echo "================================"
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/places.sqlite "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | more
            echo
            echo "Current Firefox Cookies........"
            echo "==============================="
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | more
            echo
            echo "Current Firefox Site Data........"
            echo "================================="
            ls ${HOME}/.mozilla/firefox/${FIREFOX}/storage/default/ | grep http | more
            echo
            echo "Current Firefox Cache..........."
            echo "================================="
            NUM=$( ls -1 ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries | wc -l )
            SIZ=$( du -sbh ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo "Files: $NUM Size: $SIZ"
            echo
         fi
      fi
      if [[ $CHOICE2 == "T" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$THUNDERBIRD" ]; then
            echo "Thunderbird directory does not exist"
         else
            echo "Current Thunderbird Cookies......"
            echo "================================="
            sqlite3 ${HOME}/.thunderbird/${THUNDERBIRD}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | more
         fi
      fi
   fi
fi
printf "\n"
read -rep $'\n Press ENTER to exit ' PROMPT4

Listed below is the Desktop Configuration file Firefox_or_Thunderbird_-_Clear_data.desktop for KDE in Gentoo Linux that I created in the ~/Desktop/ directory. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/Mozilla.png, although of course the Desktop Configuration file can be modified if the icon file were to be stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Firefox or Thunderbird caches and cookies
Comment=Clear Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/Firefox_or_Thunderbird_-_Clear_data.sh
GenericName[en_GB]=Clear Firefox or Thunderbird caches and cookies
GenericName=Clear Firefox or Thunderbird caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Mozilla.png
MimeType=
Name[en_GB]=Firefox_or_Thunderbird_-_Clear_data
Name=Firefox_or_Thunderbird_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

The equivalent Desktop Configuration file I created for Lubuntu 18.04 is very similar:

[Desktop Entry]
Comment[en_GB]=Clear Firefox or Thunderbird caches and cookies
Comment=Clear Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/Firefox_or_Thunderbird_-_Clear_data.sh
GenericName[en_GB]=Clear Firefox or Thunderbird caches and cookies
GenericName=Clear Firefox or Thunderbird caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Mozilla.png
MimeType=
Name[en_GB]=Firefox_or_Thunderbird_-_Clear_data
Name=Firefox_or_Thunderbird_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-LXDE-SubstituteUID=false
X-LXDE-Username=fitzcarraldo

Google Chrome

I created the following Bash script Google-Chrome_-_Clear_data.sh for the Google Chrome browser in Gentoo Linux. Different scripts have to be used in Gentoo Linux and in Lubuntu 18.04 because the paths are different.

#!/bin/bash
#
# If Delete is selected, this script deletes the entire cache, the code cache,
# the Visited Links file and the Top Sites file.
#
echo
echo "WARNING:"
echo "The Google Chrome browser must not be running if you are going to list or delete its data files."
echo
while true
do
   echo -n "[D]elete, [L]ist or [E]xit: "
   read -n1 PROMPT1
   echo
   case $PROMPT1 in
      [dD]* ) CHOICE1="D"; break;;
      [lL]* ) CHOICE1="L"; break;;
      [eE]* ) exit;;
      * )     echo "Invalid entry.";;
   esac
done
CRUNNING="N"
pgrep -u $USER chrome > /dev/null
if [[ $? -eq 0 ]]; then
   CRUNNING="Y"
   while true
   do
      echo
      echo -n "The Google Chrome browser is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT0
      echo
      case $PROMPT0 in
         [yY]* ) CHOICE0="Y"; break;;
         [nN]* ) CHOICE0="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE0 == "Y" ]]; then
      FPID=$( pgrep -u $USER chrome )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      CRUNNING="N"
   fi
fi
if [[ $CRUNNING == "Y" ]]; then
   echo
   echo "Please quit Google Chrome then re-run this script."
   echo
else
   if [[ $CHOICE1 == "D" ]]; then
      echo
      echo "Deleting URLs..........."
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls; delete from urls;"
      echo
      echo "Deleting Cookies........"
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies; delete from cookies;"
      if [[ -d ${HOME}/.cache/google-chrome/Profile\ 2/Cache ]]; then
         echo
         echo "Deleting Chrome Cache......."
         echo "====================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Profile\ 2/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Profile\ 2/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
         rm -r ${HOME}/.cache/google-chrome/Profile\ 2/Cache
      fi
      if [[ -d ${HOME}/.cache/google-chrome/Profile\ 2/Code\ Cache ]]; then
         echo
         echo "Deleting Code Cache......"
         echo "========================="
         rm -r ${HOME}/.cache/google-chrome/Profile\ 2/Code\ Cache
      fi
      if [[ -f ${HOME}/.config/google-chrome/Profile\ 2/Visited\ Links ]]; then
         echo
         echo "Deleting Visited Links......"
         echo "============================"
         rm ${HOME}/.config/google-chrome/Profile\ 2/Visited\ Links
      fi
      if [[ -f ${HOME}/.config/google-chrome/Profile\ 2/Top\ Sites ]]; then
         echo
         echo "Deleting Top Sites......"
         echo "========================"
         rm ${HOME}/.config/google-chrome/Profile\ 2/Top\ Sites
      fi
   fi
   if [[ $CHOICE1 == "L" ]]; then
      echo
      echo "Current URLs..........."
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls;" | more
      echo
      echo "Current Cookies........"
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies;" | more
      if [[ -d ${HOME}/.cache/google-chrome/Profile\ 2/Cache ]]; then
         echo
         echo "Current Chrome Cache............."
         echo "================================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Profile\ 2/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Profile\ 2/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
      fi
   fi
fi
printf "\n"
read -rep $'\n Press ENTER to exit ' PROMPT2

And below is the Desktop Configuration file Google-Chrome_-_Clear_data.desktop for Gentoo Linux. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/Google-Chrome.png, although of course the Desktop Configuration file can be adjusted if the icon file were stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Google Chrome cache and cookies
Comment=Clear Google Chrome cache and cookies
Exec=/home/fitzcarraldo/Google-Chrome_-_Clear_data.sh
GenericName[en_GB]=Clear Google Chrome cache and cookies
GenericName=Clear Google Chrome cache and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Google-Chrome.png
MimeType=
Name[en_GB]=Google-Chrome_-_Clear_data
Name=Google-Chrome_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=

Below is the version of the script Google-Chrome_-_Clear_data.sh for Lubuntu 18.04:

#!/bin/bash
#
# If Delete is selected, this script deletes the entire cache, the code cache,
# the Visited Links file and the Top Sites file.
#
echo
echo "WARNING:"
echo "The Google Chrome browser must not be running if you are going to list or delete its data files."
echo
while true
do
   echo -n "[D]elete, [L]ist or [E]xit: "
   read -n1 PROMPT1
   echo
   case $PROMPT1 in
      [dD]* ) CHOICE1="D"; break;;
      [lL]* ) CHOICE1="L"; break;;
      [eE]* ) exit;;
      * )     echo "Invalid entry.";;
   esac
done
CRUNNING="N"
pgrep -u $USER chrome > /dev/null
if [[ $? -eq 0 ]]; then
   CRUNNING="Y"
   while true
   do
      echo
      echo -n "The Google Chrome browser is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT0
      echo
      case $PROMPT0 in
         [yY]* ) CHOICE0="Y"; break;;
         [nN]* ) CHOICE0="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE0 == "Y" ]]; then
      FPID=$( pgrep -u $USER chrome )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      CRUNNING="N"
   fi
fi
if [[ $CRUNNING == "Y" ]]; then
   echo
   echo "Please quit Google Chrome then re-run this script."
   echo
else
   if [[ $CHOICE1 == "D" ]]; then
      echo
      echo "Deleting URLs..........."
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Default/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls; delete from urls;"
      echo
      echo "Deleting Cookies........"
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Default/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies; delete from cookies;"
      if [[ -d ${HOME}/.cache/google-chrome/Default/Cache ]]; then
         echo
         echo "Deleting Chrome Cache......."
         echo "====================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Default/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Default/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
         rm -r ${HOME}/.cache/google-chrome/Default/Cache
      fi
      if [[ -d ${HOME}/.cache/google-chrome/Default/Code\ Cache ]]; then
         echo
         echo "Deleting Code Cache......"
         echo "========================="
         rm -r ${HOME}/.cache/google-chrome/Default/Code\ Cache
      fi
      if [[ -f ${HOME}/.config/google-chrome/Default/Visited\ Links ]]; then
         echo
         echo "Deleting Visited Links......"
         echo "============================"
         rm ${HOME}/.config/google-chrome/Default/Visited\ Links
      fi
      if [[ -f ${HOME}/.config/google-chrome/Default/Top\ Sites ]]; then
         echo
         echo "Deleting Top Sites......"
         echo "========================"
         rm ${HOME}/.config/google-chrome/Default/Top\ Sites
      fi
   fi
   if [[ $CHOICE1 == "L" ]]; then
      echo
      echo "Current URLs..........."
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Default/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls;" | more
      echo
      echo "Current Cookies........"
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Default/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies;" | more
      if [[ -d ${HOME}/.cache/google-chrome/Default/Cache ]]; then
         echo
         echo "Current Chrome Cache............."
         echo "================================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Default/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Default/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
      fi
   fi
fi
printf "\n"
read -rep $'\n Press ENTER to exit ' PROMPT2

And below is the Desktop Configuration file Google-Chrome_-_Clear_data.desktop for Lubuntu 18.04. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/Google-Chrome.png, although of course the Desktop Configuration file can be adjusted if the icon file were stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Google Chrome cache and cookies
Comment=Clear Google Chrome cache and cookies
Exec=/home/fitzcarraldo/Google-Chrome_-_Clear_data.sh
GenericName[en_GB]=Clear Google Chrome cache and cookies
GenericName=Clear Google Chrome cache and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Google-Chrome.png
MimeType=
Name[en_GB]=Google-Chrome_-_Clear_data
Name=Google-Chrome_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-LXDE-SubstituteUID=false
X-LXDE-Username=fitzcarraldo

Below are a few examples of the output when I launch the scripts.

Here is the output of the script that deals with Firefox and/or Thunderbird when I only list the current situation:


WARNING:
The Firefox Browser must not be running if you are going to list or delete its data files.
The Thunderbird e-mail client must not be running if you are going to list or delete its data files.
 
[F]irefox, [T]hunderbird, [B]oth or [E]xit: b 
 
The Firefox browser is running. Do you wish to close it now? [Y/N]: y 
The Thunderbird e-mail client is running. Do you wish to close it now? [Y/N]: y 
 
[D]elete, [L]ist or [E]xit: l 
 
Current Firefox History.........
================================
2020-03-19 17:27:24|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:27:20|https://www.youtube.com/
2020-03-19 17:27:30|https://www.theguardian.com/uk
2020-03-19 17:27:38|http://www.jb.com.br/
2020-03-19 17:27:38|https://www.jb.com.br/
 
Current Firefox Cookies........
===============================
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:22|accounts.google.com
2020-03-19 17:27:23|.doubleclick.net
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:24|.google.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:25|.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:30|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:39|.denakop.com
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:46|www.jb.com.br
2020-03-19 17:27:46|.www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
 
Current Firefox Site Data........
=================================
https+++www.google.com
https+++www.theguardian.com
https+++www.youtube.com
 
Current Firefox Cache...........
=================================
Files: 383 Size: 15M
 
Current Thunderbird Cookies......
=================================


 Press ENTER to exit

Here is the output of the script that deals with Firefox and/or Thunderbird when I delete only the Firefox data:

 
WARNING:
The Firefox Browser must not be running if you are going to list or delete its data files.
The Thunderbird e-mail client must not be running if you are going to list or delete its data files.
 
[F]irefox, [T]hunderbird, [B]oth or [E]xit: f 
 
[D]elete, [L]ist or [E]xit: d 
 
Deleting Firefox History...........
===================================
2020-03-19 17:27:24|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:27:20|https://www.youtube.com/
2020-03-19 17:27:30|https://www.theguardian.com/uk
2020-03-19 17:27:38|http://www.jb.com.br/
2020-03-19 17:27:38|https://www.jb.com.br/
 
Deleting Firefox Cookies........
================================
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:22|accounts.google.com
2020-03-19 17:27:23|.doubleclick.net
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:24|.google.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:25|.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:30|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:39|.denakop.com
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:46|www.jb.com.br
2020-03-19 17:27:46|.www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
 
Deleting Firefox Site Data........
==================================
https+++www.google.com
https+++www.theguardian.com
https+++www.youtube.com
 
Deleting Firefox Cache...........
=================================
Files: 383 Size: 15M
 


 Press ENTER to exit

Here is the output of the script that deals with Google Chrome when I just list the current situation:


WARNING:
The Google Chrome browser must not be running if you are going to list or delete its data files.
 
[D]elete, [L]ist or [E]xit: l 
 
Current URLs...........
=======================
2020-03-19 17:30:41|https://duckduckgo.com/
2020-03-19 17:30:44|https://www.youtube.com/
2020-03-19 17:30:49|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:30:57|http://www.folha.uol.com.br/
2020-03-19 17:30:57|https://www.folha.uol.com.br/
 
Current Cookies........
=======================
2020-03-19 17:30:49|.accuweather.com
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.bt.uol.com.br
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:30:57|.accuweather.com
2020-03-19 17:30:45|accounts.google.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:46|.doubleclick.net
2020-03-19 17:31:08|player.mais.uol.com.br
2020-03-19 17:30:50|.google.com
2020-03-19 17:31:11|.dna.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:58|.www.accuweather.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:31:05|.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:30:58|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:58|www.accuweather.com
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:30:49|www.accuweather.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:08|.uol.com.br
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:30:58|.uol.com.br
2020-03-19 17:30:58|.navdmp.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|ivccf.ivcbrasil.org.br
2020-03-19 17:31:10|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:04|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:05|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:04|.prg.smartadserver.com
2020-03-19 17:31:22|www.folha.uol.com.br
2020-03-19 17:31:20|.uol.com.br
2020-03-19 17:31:10|.uol.com.br
 
Current Chrome Cache.............
=================================
Files: 317 Size: 8.6M


 Press ENTER to exit

Here is the output of the script that deals with Google Chrome when I delete the browser data:


WARNING:
The Google Chrome browser must not be running if you are going to list or delete its data files.
 
[D]elete, [L]ist or [E]xit: d 
 
Deleting URLs...........
========================
2020-03-19 17:30:41|https://duckduckgo.com/
2020-03-19 17:30:44|https://www.youtube.com/
2020-03-19 17:30:49|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:30:57|http://www.folha.uol.com.br/
2020-03-19 17:30:57|https://www.folha.uol.com.br/
 
Deleting Cookies........
========================
2020-03-19 17:30:49|.accuweather.com
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.bt.uol.com.br
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:30:57|.accuweather.com
2020-03-19 17:30:45|accounts.google.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:46|.doubleclick.net
2020-03-19 17:31:08|player.mais.uol.com.br
2020-03-19 17:30:50|.google.com
2020-03-19 17:31:11|.dna.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:58|.www.accuweather.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:31:05|.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:30:58|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:58|www.accuweather.com
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:30:49|www.accuweather.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:08|.uol.com.br
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:30:58|.uol.com.br
2020-03-19 17:30:58|.navdmp.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|ivccf.ivcbrasil.org.br
2020-03-19 17:31:10|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:04|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:05|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:04|.prg.smartadserver.com
2020-03-19 17:31:22|www.folha.uol.com.br
2020-03-19 17:31:20|.uol.com.br
2020-03-19 17:31:10|.uol.com.br
 
Deleting Chrome Cache.......
=====================
Files: 317 Size: 8.6M
 
Deleting Code Cache......
=========================
 
Deleting Visited Links......
============================
 
Deleting Top Sites......
========================


 Press ENTER to exit

Update March 23, 2020: For those of you who would prefer a single script that can delete the browsing history, cookies and cache files of Google Chrome and/or Firefox and/or Thunderbird, I have now created the script clear_browser_data.sh shown below with appropriate Desktop Configuration files for Gentoo Linux and Lubuntu 18.04. In addition I have now added the ability to list and delete the history and cache files of Thunderbird, as I have checked that deleting those is not detremental to Thunderbird. Furthermore, this new script finds the directories itself rather than having them partially hard-coded in the script, so I am using the same script in both Gentoo and Lubuntu 18.04. That said, I have not tested the script with other versions of Chrome, Firefox and Thunderbird, nor with other Linux distributions, so do not select [D]elete until you have checked that the directories evaluated by the script match the directories in your installation.

#!/bin/bash
#
########################################################################
#
# Bash script to enable the user to list and/or delete browser data of:
# Google Chrome browser
# Mozilla Firefox browser
# Mozilla Thunderbird e-mail client
#
########################################################################
#
# Check for existence of the applications' directories
FIREFOX=$( ls $HOME/.mozilla/firefox 2>/dev/null | grep profiles.ini )
THUNDERBIRD=$( ls $HOME/.thunderbird  2>/dev/null | grep profiles.ini )
CHROME=$( ls -d $HOME/.config/google-chrome 2>/dev/null )
if [ -z "$CHROME" ]; then
   echo
   echo "Chrome directory does not exist"
else
   # Chrome directory locations
   #
   CLOCALST=$( find $HOME/.config/google-chrome -type f -name "Local State" )
   CCURPROF=$( cat "$CLOCALST" | awk -F "," '{for (I=1;I<=NF;I++) if ($I~/"info_cache"/) {print $(I)};}' | awk -F "info_cache" '{print $2}' | awk -F '"' '{print $3}' )   
   CHISTORY=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "History" )
   CCOOKIES=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "Cookies" )
   CVISITED=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "Visited Links" )
   CTOPSITS=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "Top Sites" )
   CCACHEDF=$( find $HOME/.cache/google-chrome/"$CCURPROF" -type d -name "Cache" )
   CCODECAC=$( find $HOME/.cache/google-chrome/"$CCURPROF" -maxdepth 2 -type d -name "Code Cache" )
   echo; echo "Chrome is using profile: $CCURPROF"
   
fi
if [ -z "$FIREFOX" ]; then
   echo
   echo "Firefox directory does not exist"
else
   # Firefox file locations. First get the current Profile Name
   # The profiles are listed in 'profiles.ini'.
   # If profiles.ini does not contain a line 'Locked=1' then the current profile
   # is followed by the line 'Default=1'. If profiles.ini does contain 'Locked=1'
   # then the current profile is followed by the line 'Locked=1'
   #
   FPROFILE=$( find $HOME/.mozilla/firefox/ -type f -name "profiles.ini" )
   if grep -q "Locked=1$" "$FPROFILE"; then
      FCURPROF=$( grep -B 1 "Locked=1$" "$FPROFILE" | grep -v Locked )
   else
      FCURPROF=$( grep -B 1 "Default=1$" "$FPROFILE" | grep -v Default )
   fi
   FCURPROF=$( echo "$FCURPROF" | cut -d"=" -f2 )
   FHISTORY=$( find $HOME/.mozilla/firefox/"$FCURPROF" -type f -name "places.sqlite" )
   FCOOKIES=$( find $HOME/.mozilla/firefox/"$FCURPROF" -type f -name "cookies.sqlite" )
   FSITEDAT=$( find $HOME/.mozilla/firefox/"$FCURPROF" -type d -name "default" )
   FCACHEDF=$( find $HOME/.cache/mozilla/firefox/"$FCURPROF" -type d -name "entries" )
   echo; echo "Firefox is using profile: $FCURPROF"
fi
if [ -z "$THUNDERBIRD" ]; then
   echo
   echo "Thunderbird directory does not exist"
else
   # Thunderbird file locations. First get the current Profile Name
   # The profiles are listed in 'profiles.ini'.
   # If profiles.ini does not contain the line 'Locked=1' then the current profile
   # is followed by the line 'Default=1'. If profiles.ini does contain 'Locked=1'
   # then the current profile is followed by the line 'Locked=1'
   #
   TPROFILE=$( find $HOME/.thunderbird/ -type f -name "profiles.ini" )
   if grep -q "Locked=1$" "$TPROFILE"; then
      TCURPROF=$( grep -B 1 "Locked=1$" "$TPROFILE" | grep -v Locked )
   else
      TCURPROF=$( grep -B 1 "Default=1$" "$TPROFILE" | grep -v Default )
   fi
   TCURPROF=$( echo "$TCURPROF" | cut -d"=" -f2 )
   THISTORY=$( find $HOME/.thunderbird/"$TCURPROF" -type f -name "places.sqlite" )
   TCOOKIES=$( find $HOME/.thunderbird/"$TCURPROF" -type f -name "cookies.sqlite" )
   TCACHEDF=$( find $HOME/.cache/thunderbird/"$TCURPROF" -type d -name "entries" )
   echo; echo "Thunderbird is using profile: $TCURPROF"
fi
#
MENU=""
while [[ $MENU != "E" ]] && [[ $MENU != "e" ]]; do
   echo
   echo -n "[C]hrome, [F]irefox, T[hunderbird or [E]xit: "
   read -n1 MENU
   echo
   case $MENU in
      [Cc] ) ;;
      [Ff] ) ;;
      [Tt] ) ;;
      [Ee] ) ;;
      * ) echo; echo " Enter 'C/c', 'F/f', 'T/t' or 'E/e'"
   esac
   #########
   # Chrome
   #########
   while [[ $MENU == "C" ]] || [[ $MENU == "c" ]]; do
      pgrep -u $USER chrome > /dev/null
      if [[ $? -eq 0 ]]; then
         echo
         echo -n "Chrome Browser is open, do you wish to close it now? [Y/N]: "
         read -n1 YN
         echo
         if [[ $YN == "Y" ]] || [[ $YN == "y" ]]; then
            CPID=$( pgrep -u $USER chrome )
            CPID=$( echo $CPID | cut -d" " -f1 )
            kill -1 $CPID
         else
            echo
            echo " Cannot list or delete browser URLs if the browser is open"
            break
         fi
      fi
      if [ -z "$CHROME" ]; then
         echo
         echo "Chrome directory does not exist"
         break
      else
         CPROMPT=""
         echo
         echo -n "[D]elete, [L]ist or [E]xit: "
         read -n1 CPROMPT
         echo
         case $CPROMPT in
            [Dd] ) echo; echo " Chrome is using profile: $CCURPROF";;
            [Ll] ) echo; echo " Chrome is using profile: $CCURPROF";;
            [Ee] ) ;;
            * ) echo; echo " Enter 'D/d', 'L/l' or 'E/e'";;
         esac
         if [[ $CPROMPT == "E" ]] || [[ $CPROMPT == "e" ]]; then MENU=""; fi
         if [[ $CPROMPT == "D" ]] || [[ $CPROMPT == "d" ]]; then
            echo
            echo " Deleting Chrome URLs............."
            echo " ================================="
            sqlite3 "$CHISTORY" "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls; delete from urls;" | sed 's/^/ /'
            echo
            echo " Deleting Chrome Cookies.........."
            echo " ================================="
            sqlite3 "$CCOOKIES" "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies; delete from cookies;" | sed 's/^/ /'
            if [[ -d "$CCACHEDF" ]]; then
               echo
               echo " Deleting Chrome Cache............"
               echo " ================================="
               NUM=$( ls -1 "$CCACHEDF" | wc -l )
               SIZ=$( du -sbh "$CCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               rm -r "$CCACHEDF"
            fi
            if [[ -d "$CCODECAC" ]]; then
               echo
               echo " Deleting Chrome Code Cache......."
               echo " ================================="
               NUM=$( ls -1 "$CCODECAC" | wc -l )
               SIZ=$( du -sbh "$CCODECAC" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               rm -r "$CCODECAC"
            fi
            if [[ -f "$CVISITED" ]]; then
               echo
               echo " Deleting Visited Links..........."
               echo " ================================="
               rm "$CVISITED"
            fi
            if [[ -f "$CTOPSITS" ]]; then
               echo
               echo " Deleting Chrome Top Sites........"
               echo " ================================="
               rm "$CTOPSITS"
            fi
         fi
         if [[ $CPROMPT == "L" ]] || [[ $CPROMPT == "l" ]]; then
            echo
            echo " Current Chrome URLs.............."
            echo " ================================="
            sqlite3 "$CHISTORY" "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls;" | sed 's/^/ /' | more
            echo
            echo " Current Chrome Cookies..........."
            echo " ================================="
            sqlite3 "$CCOOKIES" "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies;" | sed 's/^/ /' |more
            if [[ -d "$CCACHEDF" ]]; then
               echo
               echo " Current Chrome Cache............."
               echo " ================================="
               NUM=$( ls -1 "$CCACHEDF" | wc -l )
               SIZ=$( du -sbh "$CCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
            fi
            if [[ -d "$CCODECAC" ]]; then
               echo
               echo " Current Chrome Code Cache............."
               echo " ======================================"
               NUM=$( ls -1 "$CCODECAC" | wc -l )
               SIZ=$( du -sbh "$CCODECAC" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
            fi
         fi
         CPROMPT=""
      fi
   done
   ##########
   # Firefox
   ##########
   while [[ $MENU == "F" ]] || [[ $MENU == "f" ]]; do
      pgrep -u $USER firefox > /dev/null
      if [[ $? -eq 0 ]]; then
         echo
         echo -n "Firefox Browser is open, do you wish to close it now? [Y/N]: "
         read -n1 YN
         echo
         if [[ $YN == "Y" ]] || [[ $YN == "y" ]]; then
            FPID=$( pgrep -u $USER firefox )
            FPID=$( echo $FPID | cut -d" " -f1 )
            kill -1 $FPID
         else
            echo
            echo " Cannot list or delete browser URLs if the browser is open"
            break
         fi
      fi
      if [ -z "$FIREFOX" ]; then
         echo
         echo "Firefox directory does not exist"
         break
      else
         FPROMPT=""
         echo
         echo -n "[D]elete, [L]ist or [E]xit: "
         read -n1 FPROMPT
         echo
         case $FPROMPT in
            [Dd] ) echo; echo " Firefox is using profile: $FCURPROF";;
            [Ll] ) echo; echo " Firefox is using profile: $FCURPROF";;
            [Ee] ) ;;
            * ) echo; echo " Enter 'D/d', 'L/l' or 'E/e'";;
         esac
         if [[ $FPROMPT == "E" ]] || [[ $FPROMPT == "e" ]]; then MENU=""; fi
         if [[ $FPROMPT == "D" ]] || [[ $FPROMPT == "d" ]]; then
            echo
            echo " Deleting Firefox History........."
            echo " ================================="
            sqlite3 "$FHISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /'
            sqlite3 "$FHISTORY" "delete from moz_historyvisits;"
            echo
            echo " Deleting Firefox Cookies........."
            echo " ================================="
            sqlite3 "$FCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;" | sed 's/^/ /'
            echo
            echo " Deleting Firefox Site Data......."
            echo " ================================="
            ls "$FSITEDAT" | grep http | sed 's/^/ /'
            find "$FSITEDAT" -name "http*" -type d -exec rm -r "{}" \; -prune
            echo
            echo " Deleting Firefox Cache..........."
            echo " ================================="
            if [[ $( ls -A "$FCACHEDF" ) ]]; then # Directory not empty
               NUM=$( ls -1 "$FCACHEDF" | wc -l )
               SIZ=$( du -sbh "$FCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               find "$FCACHEDF" -type f -delete
            fi
         fi
         if [[ $FPROMPT == "L" ]] || [[ $FPROMPT == "l" ]]; then
            echo
            echo " Current Firefox History.........."
            echo " ================================="
            sqlite3 "$FHISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /' | more
            echo
            echo " Current Firefox Cookies.........."
            echo " ================================="
            sqlite3 "$FCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | sed 's/^/ /' | more
            echo
            echo " Current Firefox Site Data........"
            echo " ================================="
            ls "$FSITEDAT" | grep http | sed 's/^/ /' | more
            echo
            echo " Current Firefox Cache..........."
            echo " ================================"
            NUM=$( ls -1 "$FCACHEDF" | wc -l )
            SIZ=$( du -sbh "$FCACHEDF" )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo " Files: $NUM Size: $SIZ"
         fi
         FPROMPT=""
      fi
   done
   ##############
   # Thunderbird
   ##############
   while [[ $MENU == "T" ]] || [[ $MENU == "t" ]]; do
      pgrep -u $USER thunderbird > /dev/null
      if [[ $? -eq 0 ]]; then
         echo
         echo -n "Thunderbird e-mail client is open, do you wish to close it now? [Y/N]: "
         read -n1 YN
         echo
         if [[ $YN == "Y" ]] || [[ $YN == "y" ]]; then
            TPID=$( pgrep -u $USER thunderbird )
            TPID=$( echo $TPID | cut -d" " -f1 )
            kill -1 $TPID
         else
            echo
            echo " Cannot list or delete Thunderbird URLs if the e-mail client is open"
            break
         fi
      fi
      if [ -z "$THUNDERBIRD" ]; then
         echo
         echo "Thunderbird directory does not exist"
         break
      else
         TPROMPT=""
         echo
         echo -n "[D]elete, [L]ist or [E]xit: "
         read -n1 TPROMPT
         echo
         case $TPROMPT in
            [Dd] ) echo; echo " Thunderbird is using profile: $TCURPROF";;
            [Ll] ) echo; echo " Thunderbird is using profile: $TCURPROF";;
            [Ee] ) ;;
            * ) echo; echo " Enter 'D/d', 'L/l' or 'E/e'";;
         esac
         if [[ $TPROMPT == "E" ]] || [[ $TPROMPT == "e" ]]; then MENU=""; fi
         if [[ $TPROMPT == "D" ]] || [[ $TPROMPT == "d" ]]; then
            echo
            echo " Deleting Thunderbird History........."
            echo " ====================================="
            sqlite3 "$THISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /'
            sqlite3 "$THISTORY" "delete from moz_historyvisits;"
            echo
            echo " Deleting Thunderbird Cookies........."
            echo " ====================================="
            sqlite3 "$TCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;" | sed 's/^/ /'
            echo
            echo " Deleting Thunderbird Cache..........."
            echo " ====================================="
            if [[ $( ls -A "$TCACHEDF" ) ]]; then # Directory not empty
               NUM=$( ls -1 "$TCACHEDF" | wc -l )
               SIZ=$( du -sbh "$TCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               find "$TCACHEDF" -type f -delete
            fi
         fi
         if [[ $TPROMPT == "L" ]] || [[ $TPROMPT == "l" ]]; then
            echo
            echo " Current Thunderbird History.........."
            echo " ====================================="
            sqlite3 "$THISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /' | more
            echo
            echo " Current Thunderbird Cookies.........."
            echo " ====================================="
            sqlite3 "$TCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | sed 's/^/ /' | more
            echo
            echo " Current Thunderbird Cache..........."
            echo " ===================================="
            NUM=$( ls -1 "$TCACHEDF" | wc -l )
            SIZ=$( du -sbh "$TCACHEDF"  )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo " Files: $NUM Size: $SIZ"
         fi
         TPROMPT=""
      fi
   done
done
echo

Listed below is the Desktop Configuration file clear_browser_data.desktop for KDE in Gentoo Linux that I created in the ~/Desktop/ directory. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/broom.png, although of course the Desktop Configuration file can be modified if the icon file were to be stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Chrome, Firefox or Thunderbird caches and cookies
Comment=Clear Chrome, Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/clear_browser_data.sh
GenericName[en_GB]=Clear browser caches and cookies
GenericName=Clear browser caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/broom.png
MimeType=
Name[en_GB]=clear_browser_data
Name=clear_browser_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

Listed below is the Desktop Configuration file clear_browser_data.desktop for LXDE in Lubuntu 18.04 that I created in the ~/Desktop/ directory. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/broom.png, although of course the Desktop Configuration file can be modified if the icon file were to be stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Chrome, Firefox or Thunderbird caches and cookies
Comment=Clear Chrome, Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/clear_browser_data.sh
GenericName[en_GB]=Clear browser caches and cookies
GenericName=Clear browser caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/broom.png
MimeType=
Name[en_GB]=clear_browser_data
Name=clear_browser_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-LXDE-SubstituteUID=false
X-LXDE-Username=fitzcarraldo

Update April 5, 2020: I have modified Lines 24, 37 (a comment), 40 and 55 in the above script. The original Line 24 did not select the Chrome profile name correctly in another installation, the original Line 40 did not select the Firefox profile name correctly in another installation, and the search string for the Thunderbird profile was not tight enough in the original Line 55. The above script now works in three different installations.

Update April 25, 2020: Arrgghh! I installed Thunderbird in Lubuntu 18.04 and discovered that clear_browser_data.sh did not find the Thunderbird cached files to delete. Unlike Thunderbird in Gentoo Linux on my main laptop, the file profiles.ini for Thunderbird (but not Firefox) in my Lubuntu 18.04 installation contains a line ‘Locked=1‘, and the actual profile used by Thunderbird in Lubuntu 18.04 is the profile specified in the line before ‘Locked=1‘, not the profile specified in the line before ‘Default=1‘. Someone I know uses Ubuntu 16.04 and he tells me Firefox in his installation is like this too. So, for reasons I don’t know, profiles.ini contains ‘Locked=1‘ in some Firefox or Thunderbird installations but not in others. Anyway, I have now modified the script to cater for both cases. It checks if the profiles.ini files for Firefox and Thunderbird contain ‘Locked=1‘ and, if they do, finds the currently-used profile from the line before ‘Locked=1‘ instead of the line before ‘Default=1‘. The line numbers mentioned in my previous update will be different now.

My system upgrade procedure for Gentoo Linux

Gentoo Linux is a so-called ‘rolling-release’ distribution, and each Gentoo Linux user has their own preferred sequence of steps for keeping their installation up-to-date. Below is the general procedure I use for system maintenance of my Gentoo installations, which I perform approximately weekly.

1. Update the ebuilds on the machine (see Gentoo Wiki – Project:Portage/Sync)

root # emaint sync -a

If I were using the deprecated Portage sync method I would instead have used the following commands:

root # emerge --sync # Update the ebuilds from the main Portage tree
root # layman -S # Update the ebuilds from 3rd-party overlays

2. Upgrade the Portage package manager if the console output from Step 1 included a message telling me to upgrade portage

root # emerge -1v portage

3. As I use the eix and the mlocate utilities, update their data files

root # eix-update && updatedb

4. Check if there are any News items I have not read yet

root # eselect news list

5. Read new News items and make necessary changes, if any

root # eselect news read <n>

6. Perform a dry run for the upgrade of any packages in the World file that have new versions

root # emerge -uvpDN --with-bdeps=y @world

7. If no problems were flagged in Step 6, go to Step 9

8. Sort out any problem(s) flagged in Step 6 then go back to Step 6

9. Launch the upgrade of those packages in the World file that have new versions

root # emerge -uvDN --with-bdeps=y --keep-going @world

My decision on whether or not to include the option ‘--keep-going‘ will depend on the precise circumstances.

10. If Step 9 ran to completion successfully, go to Step 14

11. If Step 9 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go to Step 12, otherwise fix the problem(s)* and go back to Step 9

*Sometimes I find that one or more packages do not merge successfully during Step 9 but do merge successfully simply by repeating Step 9.

12. Resume the upgrade process

root # emerge --resume --skipfirst

13. If Step 12 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go back to Step 12, otherwise fix the problem(s) and go back to Step 9

14. Upgrade any packages that are still built against old versions of libraries if the console output from Step 9 or Step 12 includes a message telling me to do that

root # emerge @preserved-rebuild

15. If any problems remain, fix them and go back to Step 14

16. Scan libraries and binaries for missing shared library dependencies and re-merge any broken binaries and shared libraries

root # revdep-rebuild -i

Actually, I cannot recall the last time ‘revdep-rebuild‘ was needed, as Portage has improved so much over the years.

17. Remove outdated and unneeded packages

root # emerge --ask --depclean

18. Merge any configuration files

root # etc-update

I always check the differences between the listed existing and new configuration files before going ahead, and may edit the new configuration file if I deem it necessary.

19. As I use the mlocate utility I make sure its index file is bang up to date

root # updatedb

20. Optionally, I clear out any old source-code and binary packages

root # eclean-dist --deep

21. If I remember to do it, I check if there are any installed obsolete packages and then remove them

root # eix-test-obsolete

22. I make sure no temporary work files have been left around by any failed merges

root # rm -rf /usr/tmp/portage/*

Actually, I created a script in directory /etc/local.d/ to do this automatically when HDD free space gets low (see my blog post ‘Automatically clearing the /usr/tmp/portage directory in Gentoo Linux‘).

23. I wait for at least 24 hours (usually about a week) and then go to Step 1

Notes

Actually, I have added the option ‘--with-bdeps=y‘ to EMERGE_DEFAULT_OPS in the file /etc/portage/make.conf so I do not need to type that option every time.

One of my laptops has an older Core i7 CPU and I initially added ‘--jobs=8 --load-average=8‘ to EMERGE_DEFAULT_OPTS in order to merge packages in parallel, which speeds up upgrading. However, I found this slowed interactive use of the machine and therefore I changed the options to ‘--jobs=6 --load-average=6‘ which works a bit better on that machine.

In order to prevent the number of log files increasing indefinitely, I added ‘clean-logs‘ to FEATURES in the file /etc/portage/make.conf so that log files older than seven days are deleted automatically.

Porteus Linux: A portable Linux with a difference

Xfce spin of Live Linux distribution Porteus Linux - As-installed desktop on a 1280x1024 monitor.

Xfce spin of Live Linux distribution Porteus Linux - As-installed desktop on a 1280x1024 monitor.

I’m writing this in Porteus Linux v5.0rc1 for x86_64, a Live Linux distribution booted from a USB pendrive. It is fast, good-looking and has a good range of applications and utilities. I stumbled upon Porteus recently while looking for a compact Live Linux distribution to install on a couple of spare SD cards. It seemed ideal, as it is a portable distribution designed for USB pendrives and CDs, and optionally can be configured to be persistent between reboots and shutdowns. Porteus is based on Slackware, although I gather the developers might switch to Arch Linux at some undefined future date. Spins of Porteus with various Desktop Environments are available, and I settled on Xfce after trying a couple of the others.

Although my original objective was to install a portable Linux distribution on SD cards, I only managed to install Porteus on an SD card by using YUMI Multiboot USB Creator for Windows, which I run using WINE in Linux, rather than in Windows. The reason Porteus boots from an SD card when installed by YUMI is because YUMI installs its own boot manager on the SD card and chainloads the OS. Actually, if an SD card or USB pendrive has sufficient capacity, YUMI can install several OSs on a single SD card or single USB pendrive and you can choose from the YUMI bootloader menu which OS to boot.

Anyway, Porteus is interesting because, optionally, it can be configured quite easily to be persistent. I.e. if you want it to, Porteus can save new files, applications you install, browser bookmarks, edited configuration files and so on between reboots/shutdowns. However, I was unable to get persistence working with Porteus installed by YUMI on an SD card, but persistence works perfectly when I install Porteus on USB pendrives, which is the medium Porteus is really designed to be installed on.

I happened to have a couple of spare USB 2.0 pendrives (2GB and 32GB), and I have installed Porteus on both. I opted to configure both to have persistence. There are several ways of making Porteus persistent. The first method is to create a so-called ‘save file’ on the FAT32-formatted pendrive. The second method is to create a second partition on the pendrive, formatted with a Linux filesystem (ext2, ext4, Btrfs, XFS, etc.). Another method is to use ‘Magic Folders’, but I won’t go into that here. I decided to use the first method on the 2GB FAT32-formatted pendrive, and the second method on the 32GB pendrive, which I repartitioned with a 1GB FAT32 partition and the remaining space as an ext4 partition with journalling disabled. Both methods work well. Furthermore, both pendrives boot on a desktop with UEFI firmware and on a laptop with PC BIOS firmware. Neither of the pendrives has its FAT32 partition type set as ef00.

Both pendrives initially had an msdos partition table and a single FAT32 partition. Whether you are installing Porteus from Windows or Linux, it is not mandatory to use UNetbootin, Rufus, YUMI, UUI, dd or any of the other usual methods of installing ISOs on USB pendrives. The unpacked ISO contains the shell script Porteus-installer-for-Linux.com for Linux and the program Porteus-installer-for-Windows.exe for Windows. Instructions for installing to a USB pendrive from either Windows or Linux are given on the Porteus Web site.

I downloaded an ISO file from one of the Porteus repository mirrors listed on the Porteus Web site and used the Linux command line.

Installation on a USB pendrive with a single FAT32 partition

I used a UEFI desktop machine running Lubuntu 18.04, but any machine (either UEFI or BIOS firmware) and Linux distribution would suffice. For my unbranded 2GB pendrive with an msdos partition table and single FAT32 partition, I did the following to install Porteus:

$ sudo blkid # Find which device is the pendrive (sdb in my case)
$ sudo mkdir /mnt/iso
$ sudo mount /home/fitzcarraldo/Downloads/Porteus-XFCE-v5.0rc1-x86_64.iso /mnt/iso
$ sudo mkdir /mnt/pendrive
$ sudo mount /dev/sdb1 /mnt/pendrive
$ sudo cp -a /mnt/iso/* /mnt/pendrive/
$ cd /mnt/pendrive/boot/
$ sudo ./Porteus-installer-for-Linux.com

To enable persistence I needed to edit two configuration files and create a ‘save file’ as follows:

1. Edit /mnt/sdb1/boot/syslinux/porteus.cfg

$ sudo nano /mnt/sdb1/boot/syslinux/porteus.cfg

Change ‘APPEND changes=/porteus‘ to ‘APPEND changes=EXIT:/porteus/porteussave.dat‘:

LABEL GRAPHICAL
MENU LABEL Graphics mode
KERNEL /boot/syslinux/vmlinuz
INITRD /boot/syslinux/initrd.xz
APPEND changes=EXIT:/porteus/porteussave.dat
TEXT HELP
    Run Porteus the best way we can.
    Try to autoconfigure graphics
    card and use the maximum
    allowed resolution
ENDTEXT

Note: The ‘EXIT:‘ makes Porteus save changes when you shutdown or reboot Porteus. If I understand the Porteus tutorials and forum posts correctly, without the ‘EXIT:‘ Porteus would save changes in real time. However, this did not happen in my case, so I had no choice but to add the ‘EXIT:‘ in order to save changes.

2. Edit /mnt/sdb1/porteus/porteus-v5.0-x86_64.cfg

$ sudo nano /mnt/sdb1/porteus/porteus-v5.0-x86_64.cfg

Add the following lines:

changes=/porteus/porteussave.dat
timezone=Europe/London
kmap=gb,us,br

Obviously change the timezone according to your location. You can specify up to three keyboard layouts of your choice. I chose British, US and Brazilian keyboard layouts.

Check you edited the file correctly:

$ grep -v ^# /mnt/sdb1/porteus/porteus-v5.0-x86_64.cfg | grep -v ^$
changes=/porteus/porteussave.dat
timezone=Europe/London
kmap=gb,us,br

3. Create the ‘save file’ when booted into Porteus

The GUI utility ‘Porteus save file manager’ is used to create the file used to save any changes you make in the Live environment. I chose the name porteussave.dat but you can use any name you want, suffixed with .dat. It is mandatory to use such a file if the filesystem is FAT32 or NTFS. Use ‘Applications’ > ‘System’ > ‘Porteus save file manager’ to create a new save file /mnt/sdb1/porteus/porteussave.dat.

With persistence enabled, all my files, browser bookmarks, browsing history and browser configurations remain whenever I boot Porteus. As I explain further down, the configuration changes to ALSA and PulseAudio that I made in order to get Skype working properly persist across reboots.

Porteus ‘modules’

In addition to the configuration for persistence of changes using a ‘save file’ or separate partition, Porteus uses what it calls ‘modules’, pre-packaged binaries with the suffix ‘.xzm‘ that contain either a Desktop Environment or an application. For example, I wanted to install Skype in Porteus and make it persistent, so I downloaded a Slackware package in the Live environment, installed it in the Live environment (right-click and select ‘Install package’), then converted the package to a Porteus module (right-click and select ‘txz2xzm’) and copied the module to the dedicated directory for such modules:

guest@porteus:~$ ls /mnt/sdb1/porteus/modules
firefox-70.0.1-x86_64-en-GB-1.xzm* skypeforlinux-8.18.0.6-x86_64-1_slonly.xzm*

Actually, the Porteus mirrors have some modules already available (‘bundles’), and there is a GUI utility to download and activate them. Alternatively you can download one of these modules yourself from one of the Porteus mirrors and activate it manually by double-clicking on it. To make it persistent you then copy it to the above-mentioned directory /mnt/sdb1/porteus/modules/. There is also a dedicated GUI utility to install a Web browser of your choice and activate the browser module. As you can see in the terminal output copied above, I opted to install Firefox and make it persistent.

The base OS and Desktop Environment are also Porteus modules:

guest@porteus:~$ ls /mnt/sdb1/porteus/base
000-kernel.xzm* 001-core.xzm* 002-xorg.xzm* 003-xfce.xzm*

As I wanted to try the other Desktop Environments, I downloaded the Porteus modules for those and put them in a directory that exists for optional modules:

guest@porteus:~$ ls /mnt/sdb1/porteus/optional
003-cinnamon.xzm* 003-lxde.xzm* 003-mate.xzm*
003-kde.xzm* 003-lxqt.xzm* 003-openbox.xzm*

I can then replace the module /mnt/sdb1/porteus/base/003-xfce.xzm with, for example, 003-kde.xzm to make Porteus use KDE instead of Xfce. Actually, a configuration file can be edited to load a desired Desktop Environment module and inhibit loading the base Desktop Environment module, but I have not tried that method yet.

I downloaded the Porteus modules 07-printing-x86_64-2019-11-12.xzm and 07-printing-lxqt-xfce-x86_64-2019-08-15.xzm from a link given in a Porteus Forums post, copied them to the directory /mnt/sdb1/porteus/modules/ then activated them by double-clicking on each. I was then able to configure CUPS in a browser window (http://localhost:631/admin) and get my old Canon PIXMA MP510 to print using the Gutenprint driver that was already installed without me having to install the Gutenprint printer drivers package. The two modules also enable both XSane and Document Scanner to use the Canon PIXMA MP510’s scanner.

Another ‘bundle’ module I downloaded is onlyoffice-5.0rc1-alldesktops.xzm, an open-source office suite produced by the Latvian company Ascensio System SIA. I had not heard of OnlyOffice before, but it works nicely and has text, spreadsheet and presentation editors with features similar to Microsoft Office (Word, Excel and PowerPoint). I have only tried it very briefly so far and was able to open a Word .docx document, but not an Excel .xlsx spreadsheet, but I still need to evaluate it thoroughly. It not only allows you to create and edit local files, but also to access files in the Cloud. I was able to access my remote ownCloud server documents, for example.

Installation on a USB pendrive using a second partition for persistence

Below I cover in detail the installation and configuration of Porteus on my 32GB USB pendrive.

I used a UEFI desktop machine running Lubuntu 18.04, but any machine (either UEFI or BIOS firmware) and Linux distribution would suffice.

As I wanted to install the Porteus Xfce spin on the pendrive, I downloaded the file Porteus-XFCE-v5.0rc1-x86_64.iso from from one of the Porteus Linux mirrors.

I inserted my Kingston Data Traveller 2.0 32GB USB pendrive into one of the USB ports on the front of the running desktop machine.

Note that I could have used a GUI utility such as GParted to partition and format the pendrive, but I decided to use the command line to do that part.

I opened a terminal window and typed the commands shown below.

1. Find out which device is the USB pendrive

$ sudo blkid # Find out which device the USB pendrive is. It should be sdb if no other drives are connected.
/dev/sda1: UUID="2905-DB96" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="36e3693c-b81f-4797-88fb-de3710bff86e"
/dev/sda2: LABEL="ROOT" UUID="dce73116-10fa-4169-b2d9-fb6ac8ffb83b" TYPE="ext4" PARTUUID="738fed12-239c-486d-b6e1-d90143f43ea7"
/dev/sdb1: LABEL="KINGSTON" UUID="A516-23A5" TYPE="vfat" PARTUUID="6cd1a8de-01"

Notice that, in my case, the pendrive is /dev/sdb.

2. Create a new partition table and two partitions

$ sudo fdisk /dev/sdb

Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): d
No partition is defined yet!

Command (m for help): o

Created a new DOS disklabel with disk identifier 0x8e8bace5.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 
Partition number (1-4, default 1): 
First sector (2048-60978815, default 2048): 
Last sector, +sectors or +size{K,M,G,T,P} (2048-60978815, default 60978815): +1G

Created a new partition 1 of type 'Linux' and of size 1 GiB.
Partition #1 contains a vfat signature.

Do you want to remove the signature? [Y]es/[N]o: Y

The signature will be removed by a write command.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): 
Partition number (2-4, default 2): 
First sector (2099200-60978815, default 2099200): 
Last sector, +sectors or +size{K,M,G,T,P} (2099200-60978815, default 60978815): 

Created a new partition 2 of type 'Linux' and of size 28.1 GiB.

Command (m for help): t
Partition number (1,2, default 2): 1
Hex code (type L to list all codes): b

Changed type of partition 'Linux' to 'W95 FAT32'.

Command (m for help): t
Partition number (1,2, default 2): 
Hex code (type L to list all codes): 83

Changed type of partition 'Linux' to 'Linux'.

Command (m for help): a
Partition number (1,2, default 2): 1

The bootable flag on partition 1 is enabled now.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Synching disks.

3. Double-check that the partitions have been created correctly

$ sudo fdisk /dev/sdb # Just to check partitions have been created as required.

Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p
Disk /dev/sdb: 29.1 GiB, 31221153792 bytes, 60978816 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x8e8bace5

Device     Boot   Start      End  Sectors  Size Id Type
/dev/sdb1  *       2048  2099199  2097152    1G  b W95 FAT32
/dev/sdb2       2099200 60978815 58879616 28.1G 83 Linux

Command (m for help): q

4. Format the partitions

$ sudo mkfs.fat -F 32 /dev/sdb1
mkfs.fat 4.1 (2017-01-24)
$ sudo mkfs.ext4 -O ^has_journal /dev/sdb2 # I opted to disable journaling.
mke2fs 1.44.1 (24-Mar-2018)
Creating filesystem with 7359952 4k blocks and 1843200 inodes
Filesystem UUID: 4b837147-bca3-4e31-a9f1-77da19682f77
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done   

5. Install Porteus

$ sudo mkdir /mnt/iso
$ sudo mkdir /mnt/sdb1
$ sudo mount /home/fitzcarraldo/Downloads/Porteus-XFCE-v5.0rc1-x86_64.iso /mnt/iso
mount: /mnt/iso: WARNING: device write-protected, mounted read-only.
$ sudo mount /dev/sdb1 /mnt/sdb1
$ sudo cp -a /mnt/iso/* /mnt/sdb1/
$ cd /mnt/sdb1/boot
$ sudo ./Porteus-installer-for-Linux.com
Verifying archive integrity... All good.
Uncompressing Porteus Installer......

                             _.====.._
                           ,:._       ~-_
                               '\        ~-_
                                 \        \.
                               ,/           ~-_
                      -..__..-''   PORTEUS   ~~--..__

==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--

Installing Porteus to /dev/sdb1
WARNING: Make sure this is the right partition before proceeding.

Type 'ok' to continue or press Ctrl+c to exit.
ok
Flushing filesystem buffers...

Using extlinux bootloader.

Installation finished successfully.
You may reboot your PC now and start using Porteus.
Please check the /boot/docs folder for additional information about
the installation process, Porteus requirements and booting parameters.
In case of making tweaks to the bootloader config,
please edit: /mnt/sdb1/boot/syslinux/porteus.cfg file.

Press Enter to exit.

6. Configure Porteus to be persistent across reboots/shutdowns

$ sudo nano /mnt/sdb1/porteus/porteus-v5.0-x86_64.cfg

Add the following lines:

from=/mnt/sdb1/porteus
changes=/mnt/sdb2/changes
timezone=Europe/London
kmap=gb,us,br

Change the timezone according to your location. You can specify up to three keyboard layouts of your choice. I chose British, US and Brazilian keyboard layouts.

Check you have made the edits correctly:

$ grep -v ^# /mnt/sdb1/porteus/porteus-v5.0-x86_64.cfg | grep -v ^$
from=/mnt/sdb1/porteus
changes=/mnt/sdb2/changes
timezone=Europe/London
kmap=gb,us,br

$ sudo nano /mnt/sdb1/boot/syslinux/porteus.cfg

Change ‘APPEND changes=/porteus‘ to ‘APPEND changes=EXIT:/mnt/sdb2‘:

LABEL GRAPHICAL
MENU LABEL Graphics mode
KERNEL /boot/syslinux/vmlinuz
INITRD /boot/syslinux/initrd.xz
APPEND changes=EXIT:/mnt/sdb2
TEXT HELP
    Run Porteus the best way we can.
    Try to autoconfigure graphics
    card and use the maximum
    allowed resolution
ENDTEXT

Note: The ‘EXIT:‘ makes Porteus save changes when you shutdown or reboot Porteus. If I understand the Porteus tutorials and forum posts correctly, without the ‘EXIT:‘ Porteus would save changes in real time. However, this did not happen in my case, so I had no choice but to add the ‘EXIT:‘ in order to save changes.

7. Unmount the ISO and pendrive

$ cd
$ sudo umount /mnt/iso
$ sudo umount /mnt/sdb1

The configuration of ‘changes=‘ in /mnt/sdb1/boot/syslinux/porteus.cfg means that, when you reboot or shutdown from the Live session, Porteus will save any and all changes during a session. And that means every change: new files; edited files; browser bookmarks, browser history; desktop environment configuration; and so on. However, you can define precisely what is persistent by editing the file /etc/changes-exit.conf:

guest@porteus:~$ cat /etc/changes-exit.conf 
# Folders listed in this config file will be saved during reboot and shutdown when 'changes=EXIT:' cheatcode is used.
# Folders starting with '!' are omitted. This is useful if you want to save whole folder except for particular subfolder(s).
# An example is inclued in default config below: Porteus will save whole /var folder except for /var/run and /var/tmp subfolders.
# Other example: "!/home/guest/.mozilla/firefox/c3pp43bg.default/Cache" will skip saving of Firefox caches from guest account.
# Thanks to Rava for suggesting implementation of '!' exceptions.

/bin
/etc
/home
/lib
/lib64
/opt
/root
/sbin
/usr
/var
!/var/run
!/var/tmp

Note: In the case of a pendrive using a ‘save file’ for persistence, relative paths are specified for ‘changes=‘, therefore the pendrive will be able to boot even if several drives are connected to the machine (i.e. it will not matter if the pendrive is device sdb, sdc, sdd or whatever). However, in the case of a pendrive using a second partition for persistence, absolute paths are specified for ‘changes=‘ and ‘from=‘, therefore the device letter may be different if more than just drive sda and the pendrive are connected to the machine. Therefore you may need to edit the two .cfg files to change the device from sdb to sdc or whatever in the path specified for ‘changes=‘ and ‘from=‘. Whenever you boot the pendrive a message is displayed indicating whether or not the changes partition has been found. If it has not, simply edit the two .cfg files from the Live environment and change the paths accordingly (use the command ‘sudo blkid‘ to find out which device is the pendrive now), then reboot.

8. Reboot

You may have to press F12 at boot (or whatever key your machine requires you to press in order to display a Boot Menu) and select the USB pendrive to boot from. The desktop machine I am using has UEFI firmware (notice the partitions on /dev/sda in Step 1 above, and, furthermore, /sys/firmware/efi/ exists when Lubuntu is running) and the USB pendrive boots fine. My laptops use PC BIOS and the USB pendrive boots fine on those too.

You will find that Porteus automatically creates the directory /changes on the partition of the USB pendrive with the Linux filesystem.

Installing Skype for Linux and fixing distorted sound in Skype

You can install Skype in Porteus. Download the Slackware package skypeforlinux-8.18.0.6-x86_64-1_slonly.txz, right-click on the package and select ‘Install package’. Don’t forget to also convert it to a Porteus module (right-click on the package and select ‘txz2xzm’) and copy skypeforlinux-8.18.0.6-x86_64-1_slonly.xzm to /mnt/sdb1/porteus/modules/ so that it persists.

You may find that sound in Skype is distorted/scratchy. The problem is due to PulseAudio. You can fix this as follows:

$ env PULSE_LATENCY_MSEC=90 /usr/bin/skypeforlinux

Experiement with the value if ‘90‘ does not work. If that improves the sound in Skype, edit the file /home/guest/.config/autostart/skypeforlinux.desktop (if it exists) and change the Exec line as follows:

Exec=env PULSE_LATENCY_MSEC=90 /usr/bin/skypeforlinux

and edit (as root user) the file /usr/share/applications/skypeforlinux.desktop and change the Exec line as follows:

Exec=env PULSE_LATENCY_MSEC=90 /usr/bin/skypeforlinux %U

Also edit the file /etc/pulse/daemon.conf as root user and insert the line ‘realtime-scheduling = no‘:

[...]
; realtime-scheduling = yes
; realtime-priority = 5
realtime-scheduling = no
[...]

Also, run alsamixer from the command line:

$ alsamixer -c 0 # Press F6 to select your soundcard if necessary

Adjust the volume levels in alsamixer and unmute any muted channels so that you can both hear the caller and your own recorded voice when you make a test call in Skype, then save the alsamixer settings to a file:

$ /usr/sbin/alsactl --file /home/guest/.config/asound.state store

Edit the file /etc/rc.d/rc.local as root user and add the following line to load the ALSA settings when Porteus next boots:

alsactl --file /home/guest/.config/asound.state restore

Summary

Xfce spin of Live Linux distribution Porteus Linux.

Xfce spin of Live Linux distribution Porteus Linux.

I like Porteus. The upsides are:

  • Easy and quick to install on USB pendrives.
  • Can be installed without using an ISO installer (UNetbootin, Rufus, YUMI, UUI or whatever).
  • Boots fast.
  • Fast performance.
  • Polished GUI (I have tried Openbox, Xfce and KDE so far, and settled on Xfce).
  • A good range of applications are already installed out-of-the-box.
  • The Porteus module concept is easy and fast to use.
  • Persistence works well.
  • There are a number of Porteus modules available to download (‘bundles’).
  • Converting Slackware packages to Porteus modules is easy. I right-click on the downloaded Slackware package and select ‘txz2xzm’ from the drop-down menu.
  • Switching to a different Desktop Environment is easy. Modules for all the DEs are available to download from Porteus repository mirrors such as http://ftp.nluug.nl/os/Linux/distr/porteus/x86_64/Porteus-v5.0/.

The downsides I have come across so far are:

  • The Unified Slackware Package Manager (USM) does not work. When I click on ‘Download’ to download a package, a window pops up with the message ‘Fatal error LIBS.TXT’, even though I have updated USM and all the package databases. Therefore I use a Web browser to download the relevant Slackware package (a file ending in .txz), right-click on the package and select ‘Install package’ from the drop-down menu.
  • Slackware does not have the latest versions of some packages (signal-desktop is just one example).
  • The first time I installed Porteus, when I clicked on ‘Browse Networks’ in the Thunar file manager I could browse SMB shares on a server connected to my home network (without me having edited the installed /etc/samba/smb.conf). However, in subsequent re-installations of Porteus, when I clicked on ‘Browse Networks’ a window popped up displaying the following message:

    Failed to open "/ on ".
    Message recipient disconnected from message bus without replying.

    I edited smb.conf to use the parameters for my network (see my blog post ‘A correct method of configuring Samba for browsing SMB shares in a home network‘), but that made no difference. I don’t know yet how to fix this, although on my 32GB pendrive ‘Browse Networks’ is now working again for some reason. I had installed LXDE’s PCManFM in Xfce to see if ‘Browse Networks’ would work in that, but it didn’t so I uninstalled it. Then I noticed a new console message ‘cp: can't stat‘ during boot because Porteus could not find a PCManFM file, so I deleted the /changes directory on the Linux partition to get rid of that message (Porteus creates a new /changes directory automatically, although of course you need to redo whatever was lost). It could be a coincidence, but the next time I booted Porteus, ‘Browse Network’ in Thunar worked (with the as-installed smb.conf) and continues to work.

There is quite a bit more to Porteus than I have covered in this post; for example I have not covered ‘Magic Folders’. You can find out more by reading the Porteus online documentation and forums, as well as the documention installed on USB pendrives in the directories /mnt/sdb1/ and /mnt/sdb1/boot/docs/. Although the development team is small, I am impressed with what they have implemented. Porteus will be my portable Linux distribution of choice from now on, and I look forward to learning more about it and using it in the field.

Installing and using NeXT OPENSTEP in VirtualBox for Linux


Introduction and some history

My first micro computer was an Apple II+, which I used extensively both for work and leisure. In fact I liked it so much that I bought a //e when Apple Computer, Inc. released that model. I was not tempted by the Apple /// or Lisa when they were released, although I did quite fancy the IIGS but could not justify buying one. The //c was a nice portable, and a family member bought one on my recommendation. I was not at all tempted by the first Macintosh and subsequent models using the so-called Classic Mac OS, but I drooled when Steve Jobs founded NeXT, Inc. in 1985 and launched the magnesium-cased NeXT workstations: the cube-shaped NeXT Computer (Motorola 68030 CPU) in 1989 and in 1990 the second generation (Motorola 68040 CPU) NeXTcube and the NeXTstation (commonly referred to as ‘the slab’) running the NEXTSTEP operating system. The hardware build quality and aesthetic were fabulous, and the machines and NEXTSTEP were way ahead of their time. NEXTSTEP, which was built around Unix and therefore fully multi-tasking, looked amazing when compared to the competition and its performance was superior. Drooling was all I could do, though, because the price of any NeXT machine was totally out of my league.

OPENSTEP 4.2 Desktop in a VirtualBox VM

OPENSTEP 4.2 Desktop in a VirtualBox VM.

By the way, Tim Berners-Lee invented HTTP, HTML and the first HTML browser using NEXTSTEP on a NeXTcube at CERN: see The Science Museum, London – The World Wide Web: A global information space.

Following Apple’s acquisition in 1997 of NeXT, which by then was only a software company (NeXT Software, Inc.), Apple developed Mac OS X based on OPENSTEP (the successor to NEXTSTEP). Even today some of the features in macOS are the same as in NEXTSTEP and OPENSTEP: NeXTSTEP vs Mac OS X – System Demo and Comparison. The final release of NEXTSTEP was NEXTSTEP 3.3, succeeded by OPENSTEP, the final release of which was OPENSTEP 4.2. OPENSTEP was effectively NEXTSTEP 4.

So, even though the NeXT company only sold around 50,000 machines during its relatively short existence as a manufacturer between 1988 and 1993, its impact on modern computing has been significant. Below are a few links to interesting videos about the company and some of its products. You’ll find plenty more videos about NeXT on YouTube.

You can still find the occasional second-hand NeXT computer on eBay, but they are either incomplete or very expensive. As I write this there is a complete and pristine-looking NeXTcube system, including (non-working) NeXT laser printer, in Portugal listed on eBay at US$35,000 plus US$750 shipping! So I will never get to play with a real NeXT computer. But, thanks to VirtualBox, I can at least install the i386 release of OPENSTEP 4.2 in a VM (virtual machine) to try it out for fun. I decided to install the OS and the type of applications I would typically use (assuming I could find packages on the Web, that is). I wanted to find out how usable the OS was, how good the applications were, and whether I could access Unix easily from the GUI. As NeXT hardware and software are obsolete I had to spend a lot of time searching the Web for applications that would actually install and work. Some applications work in both NEXTSTEP and OPENSTEP, but plenty of applications have different packages for the two versions of the OS, which made my searches more complicated. Some OPENSTEP packages are so-called ‘fat binaries’ containing executables for some or all the different CPU types that OPENSTEP supported, and I found a few such packages on the Web. I wanted to install and try to use at least a Web browser, a word processor, a spreadsheet, an mp3 player and a video player. I also wanted to see if I could access files on a server on my home network using Samba.

There are quite a few tutorials and videos available on the Web explaining how to install OPENSTEP in a VM, but I did not find any on installing applications in OPENSTEP. Also, many of the OS installation tutorials I found are incomplete, for example not covering either audio or networking. I am not going to give a step-by-step explanation here of how I installed the OS and the applications, but I will explain what I installed, how I rated it, and any other information I found interesting or useful. Hopefully the tips I provide will be of some help if you fancy installing the OS and any applications yourself. I should also mention that you will have an advantage if you are a Unix and/or Linux user and are au fait with using the command line. OPENSTEP 4.2 provides the C Shell (csh). I did come across a package for the Bourne Again Shell (bash), but have not tried to install it. Sometimes I had to resort to the Unix command line to change ownership or permissions of a file and to move applications to folders owned by the root user. The pwd, cd, ls, su, cp, mv, chmod and chown commands came in handy a few times. By the way, unlike Linux the ls -la command does not display the group to which a file belongs, only its owner; you need to use the command ls -lag to show both. Also, the chown command accepts the notation owner.group but not owner:group when changing attributes.

Installation of OPENSTEP/Mach 4.2 for Intel i386 in VirtualBox

‘Mach’ refers to the Mach kernel, a microkernel developed at Carnegie Mellon University. OPENSTEP was available for Motorola 68k, Intel i386 and Sun SPARC CPUs. VirtualBox supports both 32-bit and 64-bit Intel CPUs, so the 32-bit OS can be installed in a VirtualBox 32-bit VM. NEXTSTEP also supported Hewlett-Packard’s PA-RISC CPU, but NeXT dropped support for that CPU in OPENSTEP.

Regarding the spelling of the two OSs, apparently the APIs are spelt ‘NeXTStep’ and ‘OpenStep’, and the OSs are spelt ‘NEXTSTEP’ and ‘OPENSTEP’. Confusing, or what? It’s no wonder these are used interchangeably all over the Web.

I found a reasonable tutorial on the installation of OPENSTEP 4.2, including links to download the image files of the CDROM and floppy disks required. Unlike many tutorials on the Web, it also explains how to get network access working, and I was able to ping other nodes on my home network and the Internet once I had completed the tutorial: ‘Installing NextStep OS (OPENSTEP) in VirtualBox‘. There were only one or two minor differences between the tutorial and what I saw on screen, and installation in VirtualBox for Linux was essentially painless. One of the packages that has to be installed (OS42MachUserPatch4.pkg) includes a Y2K patch for the OS. The tutorial tells you to use the command line to install that package, and I followed the instructions in the tutorial but, having now learned how to install packages via the OPENSTEP GUI by selecting a package and then ‘Services’ > ‘Open Sesame’ > ‘Open As Root’ > ‘Login’ to launch the Installer, I could have used only the GUI instead of the command line to install OS42MachUserPatch4.pkg (which I have checked). No matter, though, because using the OPENSTEP command line in Terminal.app is a good learning exercise. The tutorial does not mention some other things I had to configure in VirtualBox. To get audio working I had to select ‘SoundBlaster 16’ for the Audio Controller, install a driver in OPENSTEP and reboot the VM (see details further on), and under ‘Network’ in VirtualBox Manager I had to select ‘Bridged Adapter, PCnet-PCI II (Am79C970A)’ with ‘Promiscuous Model: Allow All’. I also enabled ‘Serial Ports’ and disabled ‘USB Controller’ (USB had not yet been invented back then!).

The OS installer installs US English support and offers the option of installing support for any of five other languages too: Swedish, Spanish, Italian, German and French. I unticked all those and completed the installation. Later I decided it might be useful to have support for those additional languages, and I found it very easy to install them retrospectively: I simply loaded the OPENSTEP-Install-4.2.iso file into the VM’s ‘optical drive’, browsed the CDROM’s contents, selected Upgrader.app and then ‘Workspace’ > ‘File’ > ‘Open as Folder’. I found the language packages (SwedishEssentials.pkg etc.) in the folder ‘NextCD’ > ‘Packages’. I could then select each language package and use ‘Services’ > ‘Open Sesame’ and so on to install it, as explained earlier.

To get sound working in OPENSTEP running in VirtualBox the procedure given in a 2009 tutorial ‘Installation of OPENSTEP 4.2 in VMware 3.0 and VirtualBox‘ miraculously still worked for me:

Audio: Alejandro Diaz Infante (aka astroboy) managed to make the OPENSTEP Sound Blaster driver work under VMWare and VirtualBox.
The solution: use the drivers created by University of Glasgow (Thanks, developer(s) of them, wherever you are, for drivers you never imagined would be so useful in the future).

  1. Download SBSoundMidi.I.b.tar.gz and SBMixer.I.tar.gz
  2. Install SBSoundMidi driver for either Vibra16Cpnp or AWE32pnp. Both work great! (I use the default irq and io, but the second DMA I put it on 7, ’cause it was the detected one when used VMWare to test Windoze. Anyway, I didn’t detect any failure when using the second DMA in its default of 5, so I guess it could be up to you. In VirtualBox I didn’t change any default setting, just select the driver “SoundBlaster 16” in VirtualBox audio setting before installing.
  3. Install SBMixer to have better control of your sound card.

That’s it. Put those audio CD’s and multimedia apps back!

After copying SBSoundMidi.I.b.tar.gz to OPENSTEP I double-clicked on it to unpack it, and then double-clicked on SBSoundMidi.config to install the SoundBlaster 16 drivers. I then navigated to ‘openstep’ > ‘NextAdmin’ > ‘Configure.app’, selected the loudspeaker icon and specified the driver ‘SBSoundMidi driver for SoundBlaster AWE32 PnP (v3.38)‘.

SBMixer works, and OPENSTEP’s Sound Inspector can play .snd files without having to install additional software, although I found that some .snd files would not play completely. TheNeXTSong.snd (16-bit Linear format) which I downloaded from one of the OPENSTEP software repositories on the Web (see links at the end of this post) plays perfectly (and is amusing), but the shorter Welcome-to-the-NeXT-world.snd (8-bit muLaw format) stalls. I did manage to install a couple of audio players (see further down).

The only minor problem that occurs every time you login if the floppy disk drive is empty is a pop-up window with the message ‘The floppy disk is unreadable’. You can just click on ‘Eject’ but, to stop this happening, you can change the boot order in VirtualBox Manager and load one of the OPENSTEP floppy disk image files in the VM’s floppy disk drive (‘Settings…’ > ‘Storage’ > ‘Floppy Drive’ in VirtualBox Manager). Actually, I copied Driver_Floppy.img to Work_Floppy.img, loaded the latter in the VM’s floppy disk drive and I changed the Boot Order from ‘Floppy’|’Optical’|’Hard Disk’ to ‘Hard Disk’|’Optical’|’Floppy’ (‘Settings…’ > ‘System’ > ‘Motherboard’ > ‘Boot Order’ in the VirtualBox Manager). Furthermore, although not essential, I selected Work_Floppy in File Viewer, then in the Workspace menu I selected ‘Disk’ > ‘Initialize…’ and initialised (formatted) the floppy disk. Its icon disappears momentarily from File Viewer, then reappears after it has been formatted.

The command ifconfig on my VM host computer running Lubuntu 18.04 tells me that the IP address of the host machine is 192.168.1.74 (I had previous configured my router to always assign this address to this machine), the netmask is 255.255.255.0 and the broadcast IP address is 192.168.1.255. My router’s Management page in a Web browser has the DHCP network range configured as 192.168.1.64 – 192.168.1.253, so I decided the OPENSTEP VM would have a static IP address of 192.168.1.63. The router’s Management page also told me that the ISP’s Primary DNS IP address is 81.139.57.100 and the Secondary DNS IP address is 81.139.56.100. Therefore, in accordance with the OPENSTEP installation tutorial I followed, I edited the file /etc/hostconfig in OPENSTEP to have the following shell variables:

# /etc/hostconfig
#
# This file sets up shell variables used by the various rc scripts to
# configure the host.  Edit this file instead of rc.boot.
#
# Warning:  This is sourced by /bin/sh.  Make sure there are no spaces
#           on either side of the "=".
#
# There are some special keywords used by rc boot and the programs it
# calls:
#
#       -AUTOMATIC-     Configure automatically
#       -YES-           Turn a feature on
#       -NO-            Leave a feature off or do not configure
#
HOSTNAME=openstep
INETADDR=192.168.1.63
ROUTER=192.168.1.254
IPNETMASK=255.255.255.0
IPBROADCAST=192.168.1.255
YPDOMAIN=-NO-
NETMASTER=-NO-
TIME=-AUTOMATIC-

I also created the file /etc/resolv.conf as specified in the tutorial, containing the following two lines with the ISP’s nameserver IP addresses I found from my router:

nameserver 81.139.57.100
nameserver 81.139.56.100

It was not specified in the tutorial, but to get NFS working later I found it was necessary to edit the file /etc/hosts to comment out the list of IP addresses and to add the hostname I had chosen (openstep) for the OPENSTEP VM plus the IP address (192.168.1.74) and hostname (aspirexc600) of the VM host machine running Lubuntu 18.04:

#
# NOTE: This file is never consulted if NetInfo or Yellow Pages is running.
#
#
# To do anything on the network, you need to assign an address to your
# machine.  This default host table will get you started.  "myhost"
# can be used for the first machine on the network, and client[1-8]
# can be used for subsequent machines.  You must make sure that no two
# machines have the same address.  If you need to add more machines
# just keep adding entries.  Each digit in the four digit number must
# be between 1 and 254 inclusive.
#
#192.42.172.1	myhost
#192.42.172.2	client1
#192.42.172.3	client2
#192.42.172.4	client3
#192.42.172.5	client4
#192.42.172.6	client5
#192.42.172.7	client6
#192.42.172.8	client7
#192.42.172.9	client8
#
# This is the reserved address for the loopback interface.  Don't muck
# with it.
#
127.0.0.1       localhost       openstep
192.168.1.74    aspirexc600

While setting up networking in the VM I also temporarily disabled the firewall in the VM host to make sure the VM host was not interfering in any way with the network connection of the VM, then enabled it again once I was happy it was not causing any problems. Later, when I configured the VM host as an NFS server and the VM as an NFS client, I had to create the appropriate rules for NFS in the VM host’s firewall (see further down).

You will see NetInfo mentioned in the OPENSTEP networking apps. You should ignore NetInfo unless you are going to network a cluster of machines running NEXTSTEP/OPENSTEP, as it is an obsolete NeXT networking system configuration database and we don’t want to use it.

Installation of utilities and applications

After installing the OS neither the ‘me’ account nor the root account are password protected. You can use the OS like this if you wish, but I set up a password for the ‘me’ account by navigating to ‘openstep’ > ‘NextApps’ > Preferences.app and clicking on the padlock icon. Then I logged out and logged in to the root account and did the same to set up a password for the root user. If you want to save a bit of time during installation of applications, you could do this after installing all the packages.

OPENSTEP comes with quite a few utilities, such as Terminal.app, TextEdit.app, Draw.app, Sound.app (possibly useful if your host computer has a microphone socket and you enabled audio input in VirtualBox Manager), PhotoAlbum.app, CDPlayer.app, Webster.app (yes, a full dictionary), Librarian.app, PrintManager.app, Grab.app (to grab snapshots of all or parts of the screen and save them to .tiff files), Preview.app (an image file viewer), Mail.app, and others. You can try these and they are reasonably intuitive so I won’t dwell on them here, instead concentrating on how I installed third-party apps and utilities.

I had to trawl the Web to find packages and applications suitable for OPENSTEP/Mach 4.2 for i386. I find the filenames of the files stored on these Web sites confusing. I think.s‘ in the filename of a compressed file means it contains source code, and ‘.b‘ means it contains binary code, i.e. executable. However, some filenames have ‘.bs‘ but only contain source code, so I could be wrong. Also, I’m not sure what the letters ‘N‘, ‘I‘, ‘H‘ and ‘S‘ represent in these filenames; NeXT (Motorola 68k), Intel, Hewlett-Packard PA-RISC and SPARC, presumably? Some OPENSTEP packages are called ‘fat binaries’ as they contain binaries for several or all the supported CPU types, thus enabling the package to be installed in OPENSTEP on different hardware. So my guess about the letters in the filenames could be correct.

Without a Web browser in OPENSTEP, the easiest way to copy files to the OPENSTEP VM initially is to use the Linux mkisofs command to create an ISO file and then to load it into the VM’s optical drive. For example, let’s say I want to copy the file OpenUp-1.01.tar to the VM, I would type the following on the host machine:

$ mkdir ~/ToCopy
$ cp ~/Downloads/OpenUp-1.01.tar ~/ToCopy
$ mkisofs -o ToCopy.iso ~/ToCopy

I then use the VirtualBox Manager GUI (‘Settings’ > ‘Storage’ > ‘Choose Virtual Optical Disk File…’) to insert the ToCopy.iso file into the VM’s optical drive. OPENSTEP mounts the ‘CDROM’ automatically and it becomes visible in the OPENSTEP File Viewer window. When I click on the CDROM icon a window opens and I see it contains the file openup_1.tar which I can then drag to the Shelf or to another folder directly.

Packages for installation using the OPENSTEP Installer have a ‘.pkg‘ suffix (e.g. ParaSheet.pkg) and are actually a folder, not a file. Applications have a ‘.app‘ suffix (e.g. ParaSheet.app) and are also a folder, not a file. Some of the compressed files I found for OPENSTEP on the Web are tarballs of OPENSTEP packages (e.g. OpenWrite.2.1.8.NIHS.b.tar.gz contains OpenWrite.pkg), others are tarballs of OPENSTEP applications (e.g. mpap.1.0.m.I.b.tar.gz contains mpap.app) which require unpacking but no installation, just copying to a folder. The mkisofs command truncates filenames to the Short Filename format (a.k.a. DOS 8.3 format), so if I had any uncompressed .pkg files, .app files and indeed any other files (.pdf, .mp3 or whatever) to transfer to the VM, I compressed them first as .tar files before creating the .iso file. Even though the .tar filename is truncated to DOS 8.3 by mkisofs, the filenames of the packed files are not.

Installing a package in OPENSTEP 4.2.

a) Installing a package in OPENSTEP 4.2.

Installing a package in OPENSTEP 4.2.

b) Installing a package in OPENSTEP 4.2.

Once you get the hang of installing packages in OPENSTEP, it is actually simple. For example, to install the package ParaSheet.pkg, I drag the .tar file from the CDROM to the Shelf, and from there to the folder /me. I double-click on the .tar file which opens a window showing the ParaSheet.pkg inside. I drag that to the /me folder. Then I select the package, and select ‘Workspace’ > ‘Open Sesame’ > ‘Open As Root’ > ‘Login’ and the Installer GUI opens. I then click on ‘Set…’ to specify the folder into which I want to install the application (e.g. /LocalApps/Office, as I had created the Office folder beforehand using Terminal.app) and then ‘Install’, and the Installer takes care of the rest.

In the case of applications that are not packaged and are just .app folders, I do not need to use the Installer, I just copy the .app folder to the folder I wish (/LocalApps/, /me/LocalApps/ or just /me/).

I found that, as-installed, OPENSTEP 4.2 can unpack .tar files from the GUI but does not have a GUI app for unpacking .tar.gz files, so the first thing I did was to install the OpenUp utility: OpenUp-1.01.m.NI.b.tgz which can be found at http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Apps/Compression_Utilities/ and works very well. Of course, I could have instead unpacked .tar.gz files in the host machine first and copied the .tar files to OPENSTEP using the mkisofs method I explained above, which the OPENSTEP GUI can unpack when I double-click on the .tar file. But OpenUp is well worth installing. After I had installed OpenUp and the OmniWeb browser in OPENSTEP, I was also able to download .tar.gz files directly in OPENSTEP from the various file repositories on the Web (see links at the end of this post) and unpack them in OPENSTEP.

By the way, see the links at the end of this post for user documentation. The OPENSTEP GUI is intuitive but I didn’t realise I could rename files from the GUI by clicking on the filename below the icon to get a cursor and typing directly (just like macOS), and I also didn’t know that I could use the ‘shelf’ at the top of the File Viewer as a temporary place to put copies of files to copy files between folders as an alternative to opening another File Viewer window. I also wondered how to select multiple files in a window when they are not adjacent, since using the mouse to select the group of files is not feasible in that case. It turns out the you hold down the Shift key and click on each file you want to select, which is analogous to holding down the Ctrl key and clicking on each file in Linux. I also found that I can copy a file between two File Viewer windows by clicking on it and holding down the Alt Gr key then dragging across to the other window.

Installation of a Web browser

This is where things start to get trickier. Bear in mind that NEXTSTEP and OPENSTEP were created in the 1980s and 1990s when the Web was in its infancy. As I mentioned earlier, the first Web browser was written on a NeXTcube at CERN, and that machine was the first Web server in existence. The best Web browser I could find for the platform is OmniWeb 3.1 for OPENSTEP. Before installing it, you need to install Omni Frameworks 1998G2. Also, the browser does not support HTTPS, Javascript and Flash out of the box and you have to install plugins. Unfortunately the plugins for these are very flaky, so you are severely limited in which sites and pages you can browse. Note that Netscape Communications created HTTPS in 1994, Netscape Communications and Sun Microsystems released JavaScript in December 1995, and Macromedia released Flash in November 1996. I don’t know if the OmniWeb plugins for HTTPS, JavaScript and Flash for OPENSTEP that I found are the latest or best versions for this version of OmniWeb, but they are what I could find online. JavaScript in Web pages results in a lot of pop-up error messages and made opening pages even less likely to be successful, so in the OmniWeb menu I navigated to ‘Info’ > ‘User Preferences…’ > ‘JavaScript’ and unticked ‘Display panel for errors’. I also navigated to ‘Info’ > ‘Administrator Preferences’ > ‘HTTPS – SSL’ and ticked ‘Enable TLSv1’, which seemed to enable a few HTTPS Web pages to load, at least partially.

You have to install OpenSSL before installing the HTTPS plugin for OmniWeb. I installed the package OpenSSL.0.9.5a.m.NIS.b.tar.gz which I downloaded from http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Apps/Internet/WWW/Web%20Browsers/Omniweb/Plugins/. Then I installed the package HTTPS.1.09b.m.NIS.b.tar.gz from the same site, which installs the file (folder) HTTPS.plugin, which needs to be in the folder /LocalLibrary/Plugins/ (‘NEXTSTEP’ > ‘LocalLibrary’ > ‘Plugins’).

Then I downloaded and installed the two packages JavaScript-OWPlugin-1999-07-20-OSM-NIS.tar.gz (installs JavaScript.plugin) and Flash-OWPlugin-19990621-OSM-NIS.tar (installs Flash.plugin) which also need to be in the folder /LocalLibrary/Plugins/ (‘NEXTSTEP’ > ‘LocalLibrary’ > ‘Plugins’ in the File Viewer). I found these two packages via a BetaArchive post [offer] OmniGroup software (NeXTSTEP, OpenStep & Rhapsody), which has a link to a .rar file at http://www.mediafire.com/file/wzyon54l4dt/OmniGroup.rar/file.

Unfortunately, even with the HTTPS and JavaScript plugins installed, almost all Web pages fail to load in OmniWeb, one exception being https://www.google.com. Old HTTP Web sites do load providing they are simple, but any JavaScript seems to cause a problem.

Installation of a PDF file reader

The best PDF file reader I could find for the platform is OmniPDF 3 for OPENSTEP. If you have not already installed Omni Frameworks, you first need to install Omni Frameworks 1998G2.

Installation of an image viewer

The best (supposedly) image file viewer I could find for the platform is OmniImage 4.0 for OPENSTEP. If you have not already installed Omni Frameworks, you first need to install Omni Frameworks 1998G2. However, according to the file /OmniImage.pkg/OmniImage.info it is a beta release and, in addition to Omni Frameworks, requires ‘Omni Plugins’:

Title OmniImage 4.0 beta for OPENSTEP/Mach 4.2
Version 4.0 beta 4 (1-Oct-1998)
Description This package contains a beta version of OmniImage. This beta release only supports viewing of images, not saving them. This release will not run unless the the Omni Frameworks (version 1998G2) are installed, and will not be fully functional (e.g., images may not be rendered) unless the Omni PlugIns (version 3.0 beta 8) are also installed. This software requires OPENSTEP/Mach 4.2.

I found the file OmniPlugIns-3.0b8-OSM-NIS.pkg.tar.gz in the BetaArchive post mentioned earlier in this post. I downloaded the tarball, created an ISO file containing it, loaded the ISO file in the VM CDROM drive, unpacked the tarball to /me/OmniPlugIns.pkg and installed the package using the OPENSTEP GUI Installer using the procedure explained earlier in this post. The Omni PlugIns were installed in the folder /LocalLibrary/PlugIns/ and I then found that OmniImage can open JPG files, even a 3456×2304 pixel JPG file with the following properties (as reported by the file command in Linux):

JPEG image data, JFIF standard 1.01, resolution (DPI), density 300x300, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=4, manufacturer=Canon, model=Canon EOS 600D], baseline, precision 8, 3456x2304, frames 3

Installation of wordprocessor and spreadsheet apps

OpenWrite and ParaSheet in use

OpenWrite and ParaSheet in use.

I created the folder /LocalApps/Office/ and installed OpenWrite from OpenWrite.2.1.8.NIHS.b.tar.gz which I downloaded from Index of /OpenStep/Soft/misc/NEXTTOYOU/97.1-Fruehjahr/APPSTOYOU. If you have not already installed it, before installing these apps you need to install Omni Frameworks 1998G2.

In the folder /LocalApps/Office/ I also installed ParaSheet from ParaSheet-1.7.pkg.tar.gz which I downloaded from Index of /NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design/ParaSheet. If you have not already installed Omni Frameworks, before installing these apps you need to install Omni Frameworks 1998G2.

The first time you launch OpenWrite and ParaSheet you will be notified that you cannot use the application until you enter a licence key. Exit the application and use ‘Open Sesame’ (see earlier) to launch the application as root user, and then you well be able to enter the licence. You will find a list of licences for these packages on the Web page Index of /NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design.

Installation of audio players

mpap and MMP audio players in action

mpap and MMP audio players in action.

The only audio players I could find that actually worked (partially) in OPENSTEP are mpap 1.0 (download mpap.1.0.m.I.b.tar.gz) and MMP 2 (download mmp2.I.b.tar.gz). mpap can play some, but not all, of the mp3 files I have, whereas I could not get MMP to play mp3 files at all, although it can play .snd files. MMP can also play MIDI files, but I had to download the Timidity patches instruments.tar.gz (not so easy to find!) and follow the instructions in the MMP Info Panel in order to install the instruments patch file. It works fine! mpap cannot play an mp3 file which the files command in Linux tells me is an ‘Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v2.5, 32 kbps, 11.025 kHz, Stereo’ but it can play an mp3 file which is an ‘Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v1, 192 kbps, 44.1 kHz, Stereo’. mpap has a basic playlist feature, but it is not as sophisticated as any of the modern audio players.

Installation of video players

MPLAY and Movie players in action

MPLAY and Movie players in action.

This is where OPENSTEP is severely lacking in comparison to any modern OS; apparently we’re talking 5.5 or 6 frames per second and e.g. 288×224 pixels on NeXT hardware, and no sound. I only managed to find a couple of basic video players, both at Index of /OpenStep/Soft/video/apps: MPlay 3.0 (MPlay.app unpacked from MPlay.3.0.NIHS.b.tar.gz) and Movie 3.0 (Movie3.0 folder unpacked from Movie.3.0.NIHS.bs.tar.gz). MPlay is only designed to play MPEG (.mpg and .mpeg) files, which I found it can do for the old, tiny MPEG files I downloaded from Web repositories of NEXTSTEP/OPENSTEP files. I found that Movie can also only play MPEG files, despite the app’s README file stating it can play (without sound) MPEG, TIFF sequences, ‘QuickTime and other formats’. Movie comes with a couple of demo videos (no audio), the largest of which is hula_full.mpg in the mpeg1video format, consisting of 39 frames of 352×240 pixels, with a desired frame rate of 8 fps which actually plays at between 8 and 9 frames per second in OPENSTEP in the VM, i.e. it plays for around 4 to 5 seconds. In a video player in Linux on my desktop machine it plays for just over 2 seconds at 15 frames per second. These videos and players may have been state-of-the-art in the 1980s and early 1990s, but they certainly are not now!

I could not find an app package to play .avi files. The page I linked to above has a source-code tarball named VideoStreamV1.OSrc.tar.gz for an app named VideoStream, the README of which claims the app can play .avi files, but I have not found an executable package. Anyway, the README file states it cannot play videos with sound, so obviously I didn’t bother trying to install it.

Games

I am not particularly interested in computer games, but a few are installed by default with the OS: Chess.app, Billiards.app and BoinkOut.app (a clone of Breakout). More games for OPENSTEP can be found on the Web (for example at Index of /OpenStep/Soft/). The computer game Doom was originally developed in NEXTSTEP on NeXT computers, and a version for OPENSTEP can be downloaded from the Web, although I have not tried it.

File sharing

NEXTSTEP/OPENSTEP was designed to use NFS (Network File System). However I don’t use NFS in my home network; I use SMB and have a dedicated Linux SMB server which works well with all SMB clients (Linux, Windows and Android) on my home network. Unsurprisingly I could only find early versions of Samba packages for NEXTSTEP and OPENSTEP. I also came across ramba, a Unix clone of Samba later renamed to Sharity-Light. I downloaded them both and briefly tried to get OPENSTEP to connect to my network Samba server. I was unsuccessful, which does not surprise me as the version of Samba for NEXTSTEP/OPENSTEP I found is Version 2.0.7.1 from May 2000, and the obsolete version of rumba I found is Version 0.4 from February 1997. In NEXTSTEP/OPENSTEP the Samba configuration file smb.conf is located in the directory /usr/samba/lib/ rather than /etc/samba/. I did not spend much time trying to get Samba/Rumba working as I assume there would be incompatibility between the early SMB protocol used by Samba V2.0.7.1 / Rumba V0.4 with Samba V4.* running in the Linux SMB server on my network. Perhaps I could have made it work, but I decided to try to make the VM’s host computer (192.168.1.74) a NFS server to see if I could get the VM (192.168.1.63) to access it as a NFS client. The Web page OpenStep on Microsoft Windows PC Emulators states the following, which indicates that NFS works:

Device: Network
OpenStep Configuration: AMD PCnet-32 PCI Ethernet Adapter
VirtualBox Configuration: Bridged Adapter, PCnet-PCI II, Promiscuous Mode All
Observations: This works fine. Using SimpleNetworkStarter I was able to give OpenStep an IP address on my subnet, using my real router and real DNS servers. This allowed OpenStep to be ‘seen’ on the subnet. Standard networking facilities such as FTP and NFS work. It may help to run the a command such as the following from the VirtualBox installation directory, where “OpenStep” is whatever you name the virtual machine and “192.168.1.0” depends on your local subnet:

VBoxManage modifyvm OpenStep --natnet1 "192.168.1.0/24"

As I had named the VM ‘OPENSTEP4.2’ in VirtualBox Manager, I used the following command:

$ VBoxManage modifyvm OPENSTEP4.2 --natnet1 "192.168.1.0/24"

However I doubt this made any difference, because I had set the VM’s network adapter to ‘Bridged Adapter’ in the VirtualBox Manager, not ‘NAT’. I had to select ‘Bridged Adapter’ because I could not get the VM to connect to the network otherwise.

I also made sure the adapter in the VirtualBox Manager is set to ‘PCnet-PCI II (Am79C970A)’ and Promiscuous Mode is set to ‘Allow All’.

In addition to the network configuration notes in the OPENSTEP installation tutorial I mentioned earlier, for information only see the old tutorial ‘NeXTStep/OpenStep Ethernet-Based Network Configuration For Cable Modems, DSL, LANs, Etc…‘.

Anyway, below is what I did to get NFS working. The crucial thing to note is that OPENSTEP 4.2 uses NFSv2. I spent many hours unsuccessfully trying to get NFS working between the NFS server (a machine with IP address 192.168.1.74) and the NFS client (a VM with IP address 192.168.63) until I realised this. The NFS server is running Lubuntu 18.04, which uses NFSv4 by default. Therefore I had to configure the NFS server to use NFSv2 as well. Not only that, but I had to configure NFSv2 to use static ports, because the ports can change randomly in NFSv2 which would stop NFS working if there is a firewall enabled on the host machine.

In the NFS server (Lubuntu 18.04 running on a desktop machine)

N.B. My NFS server is running in Lubuntu 18.04 on a machine with an IP address of 192.168.1.74, and my NFS client is running in OPENSTEP 4.2 on a VM with IP address of 192.168.1.63. Change the IP addresses below to suit your situation.

1. Install the NFS server software

$ sudo apt-get update
$ sudo apt-get install nfs-kernel-server

2. Create a mountpoint for the NFS shared directory

$ sudo mkdir /var/nfs
$ sudo chown nobody:nogroup /var/nfs
$ sudo chmod 777 /var/nfs

3. Configure the NFS export

$ sudo nano /etc/exports

3.1 Choose which of the following types of share you want to have

3.1.1 Less secure:

/home/fitzcarraldo/nfsshare 192.168.1.63(rw,sync,no_root_squash,no_subtree_check)

If ‘no_root_squash‘ is used, remote root users are able to change any file on the shared file system and leave trojaned applications for other users to inadvertently execute.

3.1.2 More secure:

/var/nfs 192.168.1.63(rw,sync,no_subtree_check)

3.2 Update the current table of exports for the NFS server

$ sudo exportfs -a

You can check the current table settings:

$ sudo exportfs -s
/home/fitzcarraldo/nfsshare  192.168.1.63(rw,wdelay,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
/var/nfs  192.168.1.63(rw,wdelay,root_squash,no_subtree_check,sec=sys,rw,secure,root_squash,no_all_squash)

If you wanted to clear the table (unexport the shared directories) you would do:

$ sudo exportfs -u 192.168.1.63:/home/fitzcarraldo/nfsshare
$ sudo exportfs -u 192.168.1.63:/var/nfs
$ sudo exportfs -s
$

4. Load the NFSv2 kernel module

If lockd is built as a module (which it is in Lubuntu 18.04), create file /etc/modprobe.d/nfsv2.conf containing the following:

options lockd.nlm_udpport=4001 lockd.nlm_tcpport=4001
$ sudo modprobe nfsv2

If you want to make that permanent so it happens automatically when booting/rebooting add ‘nfsv2‘ (without the quotes) to the file /etc/modules-load.d/modules.conf (which in Lubuntu 18.04 is symlinked to /etc/modules).

5. Configure the NFS server

See ‘How can I make the nfs server support protocol version 2 in Ubuntu 17.10?‘.

Edit /etc/default/nfs-kernel-server to include NFSv2 and to specify static ports:

$ sudo nano /etc/default/nfs-kernel-server
# Number of servers to start up
RPCNFSDCOUNT=8

# Runtime priority of server (see nice(1))
RPCNFSDPRIORITY=0

# Options for rpc.mountd.
# If you have a port-based firewall, you might want to set up
# a fixed port here using the --port option. For more information, 
# see rpc.mountd(8) or http://wiki.debian.org/SecuringNFS
# To disable NFSv4 on the server, specify '--no-nfs-version 4' here
RPCMOUNTDOPTS="--manage-gids -p 32767"
# -p 32767 above added by Fitzcarraldo

# Do you want to start the svcgssd daemon? It is only required for Kerberos
# exports. Valid alternatives are "yes" and "no"; the default is "no".
NEED_SVCGSSD=""

# Options for rpc.svcgssd.
RPCSVCGSSDOPTS=""

# All options below this comment were added by Fitzcarraldo
#
# Options to pass to rpc.statd
# ex. RPCSTATDOPTS="-p 32765 -o 32766"
RPCSTATDOPTS="-p 32765 -o 32766"
#
# Options to pass to rpc.rquotad
# ex. RPCRQUOTADOPTS="-p 32764"
RPCRQUOTADOPTS="-p 32764"
#
RPCNFSDOPTS="--nfs-version 2,3,4 --debug --syslog"
#
# To confirm above mods are in effect after service restart use
#    cat /run/sysconfig/nfs-utils
#  or 
#    service nfs-kernel-server status
#

Edit /etc/default/nfs-common to specify static ports for rpc-statd:

# If you do not set values for the NEED_ options, they will be attempted
# autodetected; this should be sufficient for most people. Valid alternatives
# for the NEED_ options are "yes" and "no".


# Options for rpc.statd.
#   Should rpc.statd listen on a specific port? This is especially useful
#   when you have a port-based firewall. To use a fixed port, set this
#   this variable to a statd argument like: "--port 4000 --outgoing-port 4001".
#   For more information, see rpc.statd(8) or http://wiki.debian.org/SecuringNFS
STATDOPTS="-o 32766 -p 32765"
# -o 32766 -p 32765 above were added by Fitzcarraldo

# Do you want to start the gssd daemon? It is required for Kerberos mounts.
NEED_GSSD=

(I had to edit /etc/default/nfs-common to specify the ports for rpc-statd in STATDOPTS because specifying the ports in RPCSTATDOPTS in /etc/default/nfs-kernel-server did not make the status ports static.)

Edit /etc/sysctl.conf to add a static port mapping for lockd:

$ sudo nano /etc/sysctl.conf
[...]
# All lines below added by Fitzcarraldo
# TCP Port for lock manager
fs.nfs.nlm_tcpport = 4001
# UDP Port for lock manager
fs.nfs.nlm_udpport = 4001

Modify the lockd kernel parameters now during runtime rather than having to reboot:

$ sudo sysctl -p

Note that it is necessary to specify static ports in the configuration files so that tight rules can be added to the firewall in the NFS server.

6. Start the NFS server

Either the sysvinit way, which still works in Lubuntu 18.04:

$ sudo service nfs-kernel-server start

or the systemd way, which also works in Lubuntu 18.04:

sudo systemctl start nfs-kernel-server

If you want, you could enable the service so it starts automatically after the system is rebooted:

$ sudo systemctl enable nfs-kernel-server

7. Start the NSM (Network Status Monitor) daemon

Either the sysvinit way, which still works in Lubuntu 18.04:

$ sudo service rpc-statd start

or the systemd way, which also works in Lubuntu 18.04:

$ sudo systemctl start rpc-statd

If you want, you could enable the service so it starts automatically after the system is rebooted:

$ sudo systemctl enable rpc-statd

8. Check that NFSv2 is running and the ports are the ones specified in the config files

$ rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100005    1   udp  32767  mountd
    100005    1   tcp  32767  mountd
    100005    2   udp  32767  mountd
    100005    2   tcp  32767  mountd
    100005    3   udp  32767  mountd
    100005    3   tcp  32767  mountd
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100227    2   tcp   2049
    100227    3   tcp   2049
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100227    2   udp   2049
    100227    3   udp   2049
    100021    1   udp   4001  nlockmgr
    100021    3   udp   4001  nlockmgr
    100021    4   udp   4001  nlockmgr
    100021    1   tcp   4001  nlockmgr
    100021    3   tcp   4001  nlockmgr
    100021    4   tcp   4001  nlockmgr
    100024    1   udp  32765  status
    100024    1   tcp  32765  status

9. Configure the firewall in Lubuntu 18.04

I used Gufw (LXDE Menu > ‘Preferences’ > ‘Firewall Configuration’) to add the following two UFW rules:

111,2049,4001,32765:32768/udp ALLOW IN 192.168.1.0/24
111,2049,4001,32765:32768/tcp ALLOW IN 192.168.1.0/24

The above rules permit NFSv2 to function consistently because I had configured the NFS ports to be static. If I had not done that the firewall would sometimes stop NFS from working because NFSv2 ports change randomly otherwise.

In OPENSTEP running in the VM

10. Make sure basic networking has been configured

I navigated to ‘openstep’ > ‘NextAdmin’ > ‘SimpleNetworkStartup.app’ and did the following:

  • Unticked ‘Maintain the master copy of network administrative data.’
  • Selected ‘Use the network, but don’t share administrative data.’
  • Entered the Hostname ‘openstep‘ (no quotes) and IP address 192.168.1.63.
  • Clicked on ‘Network Options…’. In the window that opened I did the following:
    • Made sure router IP is set to 192.168.1.254
    • Made sure NIS Domain Name is set to ‘None’
    • Made sure Netmask is set to 255.255.255.0
    • Made sure Broadcast Address is set to 192.168.1.255
    • ‘Limit access to local NetInfo data to the local network’ is unticked.
    • Clicked on ‘Set’.
  • Clicked on ‘Configure’.

11. Create the shared NFS director[y,ies]

N.B. I could probably have created the directory /mnt/nfs/nfsshare and/or /mnt/nfs/var/nfs (whichever you chose to create — see 3.1 above) using ‘openstep’ > ‘NextAdmin’ > ‘NFSManager.app’ instead of using the command line, but I opened a Terminal window in OPENSTEP and did the following:

openstep> su
openstep:1# mkdir /mnt
openstep:2# mkdir /mnt/nfs
openstep:3# mkdir /mnt/nfs/nfsshare
openstep:4# mkdir /mnt/nfs/var
openstep:5# mkdir /mnt/nfs/var/nfs

12. Mount the NFS share(s)

openstep:6# mount 192.168.1.74:/home/fitzcarraldo/nfsshare /mnt/nfs/nfsshare
openstep:7# mount 192.168.1.74:/var/nfs /mnt/nfs/var/nfs

Use the df command to check they are mounted correctly:

openstep:8# df

13. Test the shared director[y,ies]

In Lubuntu on the machine with hostname ‘aspirexc600‘, copy a file into /var/nfs/ (or /home/fitzcarraldo/nfsshare/). You should see it appear in /mnt/nfs/var/nfs/ (or /mnt/nfs/nfsshare/) in OPENSTEP in the VM with hostname ‘openstep‘.

In OPENSTEP on the VM with hostname ‘openstep‘, copy a file into /mnt/nfs/var/nfs/ (not /mnt/nfs/nfsshare/, as that will not be allowed). You should see it appear in /var/nfs/ in Linux in the machine with hostname ‘aspirexc600‘.

In Lubuntu on the machine with hostname ‘aspirexc600‘, delete the file in /var/nfs/ and you should see it removed from /mnt/nfs/var/nfs/ in OPENSTEP on the VM with hostname ‘openstep‘.

In Lubuntu on the machine with hostname ‘aspirexc600‘, delete the file in /home/fitzcarraldo/nfsshare/ and you should see it removed from /mnt/nfs/nfsshare/ in OPENSTEP on the VM with hostname ‘openstep‘.

14. If you later want to unmount the NFS shared folder(s)

openstep:9# umount /mnt/nfs/nfsshare
openstep:10# umount /mnt/nfs/var/nfs

15. If you want OPENSTEP to mount the NFS shared folder(s) automatically when it boots

I was unable to get OPENSTEP to mount NFS shared folders automatically at boot by adding the appropriate lines in /etc/fstab, but OPENSTEP does mount them automatically if I add the mount commands to /etc/rc.local like so:

#!/bin/sh -u
#
# This script is for augmenting the standard system startup commands. It is 
# executed automatically by the system during boot up. 
#
# Copyright (C) 1993 by NeXT Computer, Inc.  All rights reserved.
#
# In its released form, this script does nothing. You may customize
# it as you wish.
#

fbshow -B -I "Starting local services" -z 92

# Read in configuration information
. /etc/hostconfig

# (echo -n 'local daemons:')                                    >/dev/console
#
# Run your own commands here
mount 192.168.1.74:/var/nfs /mnt/nfs/var/nfs
mount 192.168.1.74:/home/fitzcarraldo/nfsshare /mnt/nfs/nfsshare
#
# (echo '.')                                                    >/dev/console

File sharing: Summary

So, I managed to get NFS working, albeit not using OPENSTEP’s NFSManager.app tool. Had I known more about OPENSTEP networking I probably could have used the OPENSTEP GUI utilities to configure NFS, but at least I have proved it is possible to copy files to and from an NFS server (which happens to be the host machine of the VM) running Lubuntu 18.04 and the VM running OPENSTEP 4.2. Mind you, NFSv2 is old. NFSv4 would be the protocol to use had OPENSTEP supported it. Also, bear in mind that NFSv2 cannot encrypt the connection, so it is not secure. Another reason to have a good firewall enabled in the VirtualBox host machine and in my router too.

Conclusions

I have had fun installing and tinkering with OPENSTEP and its applications over the last few days. Getting file sharing to work was by far the most difficult part, but I got there in the end once I had discovered OPENSTEP only supports NFSv2. It is a pity OPENSTEP and the applications for it have not been developed for many years and are all obsolete. If development of OPENSTEP drivers, networking software, productivity applications and multimedia applications had continued, the OS itself would still have been perfectly usable on modern hardware, albeit not as straightforward to use as any of the main Desktop Environments in Linux. But the OS still feels quite modern; it was definitely ahead of its time. Tinkering with OPENSTEP 4.2 has given me a new respect for Steve Jobs, for the talented hardware and software engineers in the NeXT company, and indeed for Mac OS X and macOS. The choice of Unix for NEXTSTEP/OPENSTEP was truely inspired.

In this blog post I have not covered the sophisticated development tools for NEXTSTEP/OPENSTEP, which were also way ahead of their time. I’ll leave you to read the articles, documents and videos available on the Web about the development tools.

Please comment below if you notice any errors or omissions in this post, or if you know a better way of doing something in OPENSTEP, or you know of newer versions of the OPENSTEP software than the versions I have mentioned. I’d also be interested to hear from anyone who has a NeXT machine and/or is still using one; let me know what you have and how you’re using it.

Useful links

These are just a few of the many Web pages and sites I browsed when installing OPENSTEP 4.2 and looking for applications and ways to get various things to work.

Documentation

Software repositories

Sometimes differences between NEXTSTEP and OPENSTEP may mean a NETSCAPE application cannot be installed in OPENSTEP or, if it can, may not work. Furthermore, be aware that different revisions of the same application/utility exist online, so you need to try and find the latest revisions.