That UPS you bought for your home server may not be as useful as you think

Some years ago I decided to install a server at home for use as a NAS (network-attached storage) in my home network, and for an Internet-facing server. I live in a place where blackouts are very infrequent (perhaps a couple per year), but occasionally the mains drops out for only a second or two. I suspect these very short dropouts occur when substation switchgear operates, but have no way of being sure. Anyway, with a server running 24/7 I obviously wanted protection against any loss of the mains supply.

I ended up buying a 700VA APC Back-UPS ES-BE700G-UK, which has four mains sockets that are battery-backed and also have surge protection, plus another four mains sockets that have surge protection but are not battery-backed. Additionally, it has two RJ45 sockets to provide pass-through filtering for an Ethernet connection. It also has a USB port for connection to the server so that it can transmit unsolicited status information to the server (including requesting the server to shutdown) and can also be interrogated by the server using the apcaccess command. The APC UPS daemon works with this model of UPS, and was relatively easy to set up. APC, formerly American Power Conversion Corporation, is a subsidiary of European company Schneider Electric. My UPS was manufactured in The Philippines.

I have three devices plugged into the battery-backed sockets on the APC UPS: the server, an external 6TB USB HDD connected to the server for automated daily backups by the server, and a 5-port Ethernet switch. The battery in the UPS would provide between 15 and 20 minutes of power when the mains fails, although I have configured the UPS to trigger the server to shutdown when 30 per cent of the battery power remains, as battery life is reduced considerably if its power is allowed to drain completely. In case you’re wondering why my router is not also plugged into the UPS, due to the position of the broadband provider’s socket the router is in a different room and I have therefore had to connect it to a different UPS, an iLEPO multi-functional DC UPS (the ECO PLUS 412P, which is tiny but can keep the router powered for several hours). Obviously the router needs to be connected to a UPS, otherwise the server would not be able to send me e-mails when there is a mains power cut. Being able to receive UPS status e-mails from the server is important to me when I am away from home on work trips.

So I thought I had covered all bases, and, indeed, the UPS proved useful on several occasions. I would quite often be on a work trip and receive an e-mail from the server informing me that mains power to the UPS had been lost, then another e-mail soon after informing me that mains power to the UPS had returned. Only once did the power cut last longer than the battery capacity, and the server was shutdown automatically.

Now, the life of the 12-volt lead-acid battery in the APC UPS is supposed to last approximately three to five years. The life will depend on how many times the battery is discharged and ambient temperature.

While I was away from home on a long work trip, suddenly I could no longer connect to my server and I had not received an e-mail from the server informing me of any problem. Luckily it was near the end of my trip so I was not too inconvenienced. When I arrived home I found that the UPS was sounding an alarm and was not supplying power to the server even though there was mains supply to the UPS. It transpired that the UPS battery had suddenly died without warning and could no longer hold a charge, and this had happened while there was mains supply to the UPS, i.e. there had not been a power cut while I was away. Fortunately there was no loss of data on the server; I was able to run fsck during boot-up.

This failure was annoying on two counts. Firstly, the battery was only about thirteen months old (the manufacturing date stamped on the UPS box was only two or three months before the date I purchased the UPS). Secondly, I certainly did not expect the UPS to stop supplying power to the server while there was mains supply to the UPS. The APC white paper on UPS topologies, ‘The Different Types of UPS Systems‘, does not make this behaviour clear.

It turns out that the type of UPS topology (‘Standby’ — see the model’s Technical Specifications) in this model of UPS does not continue to provide power to connected equipment when the UPS battery either fails or is disconnected for whatever reason when there is mains power supply to the UPS. A quick search of the Web showed me that I was not alone in discovering this ‘feature’: an APC Back-UPS 350 owner posted ‘UPS Battery Backup — useless when the battery dies?‘.

From what I have read, the so-called ‘Line Interactive’ UPS topology does not suffer this shortcoming, so, the next time I have to buy a UPS for a piece of equipment that requires power 24/7, I will buy a line-interactive UPS rather than a standby UPS. But, before purchasing, I will be sure to ask the manufacturer what the precise model would do if its battery fails or is disconnected while there is mains supply to the UPS. The APC line-interactive UPSs are more expensive than the APC Back-UPS models; now I know why. It’s a pity I was not aware of the shortcoming of the Back-UPS models, as I would have spent more and bought a UPS that continues to work when the battery dies while there is mains supply. I would also hope the UPS would issue an alarm if the battery has failed or is disconnected while there is mains supply. Be sure to ask the manufacturer all these questions if you cannot tolerate a sudden loss of power to your equipment if the battery dies while there is mains supply.

Anyway, after checking that the dead battery was indeed useless I replaced it with a new Yuasa battery that has lasted nearly three years now. I will be replacing it shortly as a precaution, even though it has not had to be used much at all since I installed it. I have not replaced the APC Back-UPS model but I will be replacing the battery at least every three years just to be cautious, and of course taking the old batteries to my local waste disposal centre to be recycled properly. By the way, it is possible to purchase a 12-volt battery manufactured by one of the reputable battery manufacturers such as Yuasa with the same specification as the APC battery, for a significantly lower price than APC charges for replacement batteries (which I suspect are badged by APC in any case).

Re-enabling OpenGL compositing automatically after it crashes KWin at login to KDE Plasma

One of my laptops has NVIDIA Optimus hardware and runs Gentoo Linux with the closed-source NVIDIA driver. Almost every time I logged-in to KDE Plasma for the first time after booting, OpenGL and compositing would be disabled (see screenshot below), and the usual methods of toggling compositing on/off would not work.

KDE Plasma - System Settings - Compositor Settings for Desktop Effects

KDE Plasma - System Settings - Compositor Settings for Desktop Effects.

I had to perform the following ritual in order to get ‘wobbly windows’ working again:

  1. select ‘System Settings’ > ‘Display and Monitor’ > ‘Compositor’
  2. click on ‘Re-enable OpenGL detection’
  3. deselect ‘Enable compositor on startup’
  4. click ‘Apply’
  5. select ‘Enable compositor on startup’
  6. click ‘Apply’

After having to perform this tedious process almost every time I logged in to KDE Plasma following boot-up, I finally decided to find an automated method of re-enabling OpenGL detection and compositing. I discovered that, when the problem occurred, the value of the variable OpenGLIsUnsafe in the file ~/.config/kwinrc had become ‘false‘. To get things working again I created the Bash script ~/restart_compositing.sh listed below. The script reverts the value of the variable OpenGLIsUnsafe to ‘true‘, reverts the value of the variable Enabled in the same section to ‘true‘ if it happens to be ‘false‘, and restarts KWin. Simple as that.

#!/bin/bash
#
# OpenGL compositing usually crashes KWin when I login, and compositing is then disabled.
# I have to select 'System Settings' > 'Display and Monitor' > 'Compositor' and perform
# the following steps to get compositing to work in the session:
#
# 1. click 'Re-enable OpenGL detection'
# 2. deselect 'Enable compositor on startup'
# 3. click 'Apply'
# 4. select  'Enable compositor on startup'
# 5. click 'Apply
#
# This script enables me to avoid having to perform the above manual procedure.
# This script is configured to run automatically at Plasma Startup - see:
# 'System Settings' > 'Startup and Shutdown' > 'Autostart'
#
edit_kwinrc () {
                # Extract the [Compositing] section from kwinrc
                awk '/\[Compositing\]/,/^$/' $HOME/.config/kwinrc > /tmp/kwinrc-extract
                # Remove the header in the extracted section
                sed -i '/\[Compositing\]/d' /tmp/kwinrc-extract
                # Remove the empty line at the end of the extracted section
                sed -i '/^$/d' /tmp/kwinrc-extract
                # Change the state configured for next login
                if [ $1 == "disablecompositing" ]; then
                    sed -i 's/Enabled=true/Enabled=false/g' /tmp/kwinrc-extract
                elif [ $1 == "enablecompositing" ]; then
                    sed -i 's/Enabled=false/Enabled=true/g' /tmp/kwinrc-extract
                elif [ $1 == "openglunsafe" ]; then
                    sed -i 's/OpenGLIsUnsafe=false/OpenGLIsUnsafe=true/g' /tmp/kwinrc-extract
                elif [ $1 == "openglsafe" ]; then
                    sed -i 's/OpenGLIsUnsafe=true/OpenGLIsUnsafe=false/g' /tmp/kwinrc-extract
                fi
                # Replace the [Compositing] section in kwinrc
                awk 'BEGIN {p=1} /^\[Compositing\]/ {print;system("cat /tmp/kwinrc-extract");p=0} /^$/ {p=1} p' $HOME/.config/kwinrc > /tmp/kwinrc
                cp /tmp/kwinrc $HOME/.config/kwinrc
}
#
# Avoid backing up an incorrectly-edited file
if [ ! -f $HOME/.config/kwinrc.bak ]; then
    cp $HOME/.config/kwinrc $HOME/.config/kwinrc.bak
fi
#
sleep 120s # This delay works for my specific laptop but might need to be adjusted on other machines.
if $( grep -q "OpenGLIsUnsafe=true" $HOME/.config/kwinrc ); then
    edit_kwinrc openglsafe
    edit_kwinrc enablecompositing # Just in case it was disabled as well.
    kwin_x11 --replace & > /dev/null 2>&1
fi
exit 0

I then selected ‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’, clicked on ‘Add Script…’ and specified that /home/fitzcarraldo/restart_compositing.sh has to be run at ‘Startup’ (of Plasma). Problem solved.

Updating the Powerline adapters in my home network

I have blogged previously about a couple of problems with using Powerline adapters in my home network:

As my NETGEAR XAV1301 (200 Mbps) Powerline adapters bought in 2012 apparently do not fully support IPv6, and as my NETGEAR XAV5221 (500 Mbps) adapters bought in 2016 are no longer manufactured either, I decided to invest in some new Powerline adapters that would guarantee IPv6 support. My Web searches did not confirm that the current models of NETGEAR Powerline adapters support IPv6, so I decided to try TP-Link Powerline adapters because the TP-Link Web site states that all current TP-Link Powerline adapters support IPv6. I wanted Powerline adapters for five devices (router, smart TV and three computers), plus the ability to use a mains plug on at least two of those (i.e. so-called ‘pass-through’ adapters). I also wanted to avoid buying different models, in order to minimise the possibility of any problems. TP-Link have a range of 600 Mbps adapters under the name ‘AV600’, so I plumped for two TP-PL4010 adapters (single Ethernet port per adapter), one TP-PL4010P adapter (single Ethernet port and one mains pass-through socket) and one TP-PL4020P (two Ethernet ports and one mains pass-through socket). These all use the Qualcomm Atheros QCA7420 Powerline chipset (which happens to be the same chipset used in my old NETGEAR XAV5221 500 Mbps adapters).

Like NETGEAR, TP-Link does not have a Powerline utility program for Linux, so I had to install TP-Link’s tpPLC utility program in Windows 10 running in a VM (virtual machine) in order to configure the four TP-Link adapters and set the ‘Powerline network name’ to avoid crosstalk with my neighbour’s Powerline adapters that use the factory default network name (‘HomePlugAV’).

Anyway, I got everything set up and working, but soon noticed that there were quite frequent dropouts of the connection to my router and the Internet. Some dropouts did occur when I was using the old NETGEAR Powerline adapters, but I was surprised to find that the performance of the new TP-Link adapters was much worse. The dropouts typically lasted a minute or two. This was annoying, to say the least.

I started searching the Web, and ‘TP-Link’ and ‘dropout’ occur together a lot. I had already disabled Power Saving Mode in the adapters, so knew that was not the cause. I happen to know someone who also uses TP-Link adapters, and he mentioned that he also experienced frequent dropouts. In addition to turning off Power Saving Mode, he had implemented a shell script on his machines to ping an Internet site periodically to try and keep the connection from dropping out, but this did not appear to make any difference. I wrote the script below to try the same thing, and it did not cure the dropouts either:

#!/bin/bash
#
# Script to try to keep the Powerline adapter connected to this machine
# from dropping the connection to the router
#
FIRSTPASS=1
PREVIOUS=2
while true
do
    ping -W 2 -c 1 8.8.8.8 >>/dev/null 2>&1
    STATUS=$?
    if [ $PREVIOUS -ne 0 ] && [ $STATUS -eq 0 ]; then
        logger "Ping successful: connection to Internet is up."
#        echo "Ping successful: connection to Internet is up."
    elif [ $PREVIOUS -eq 0 ] && [ $STATUS -ne 0 ]; then
        logger "Ping unsuccessful: connection to Internet may be down."
#        echo "Ping unsuccessful: connection to Internet may be down."
    elif [ $FIRSTPASS -eq 1 ] && [ $STATUS -ne 0 ]; then
        logger "Ping unsuccessful: connection to Internet may be down."
#        echo "Ping unsuccessful: connection to Internet may be down."
    fi
    PREVIOUS=$STATUS
    FIRSTPASS=0
    sleep 10
done

In my Web searches I came across a a thread in the TP-Link SOHO Community forums with a URL for a new version of firmware for TP-Link Powerline adapters that use the Qualcomm Atheros QCA7420 chipset. I learned from the TP-Link forums that the firmware in NVM (Non-Volatile Memory) depends on the chipset manufacturer’s chipset, not on the Powerline manufacturer’s adapter model, whereas the adapter’s PIB (Parameter Information Block) does change depending on the model (including the country). So I started searching online for a PIB file for the three models of TP-Link adapter that I am using, but I could not find them. However, the Linux open-plc-tools command ‘plctool‘ enabled me to read the PIB from each adapter and store it as a file:

user $ sudo plctool -i eth0 -p TL-PA4010P.pib <MAC address printed on the adapter>
user $ sudo plctool -i eth0 -p TL-PA4010_TV.pib <MAC address printed on the adapter>
user $ sudo plctool -i eth0 -p TL-PA4010_HOME-HUB.pib <MAC address printed on the adapter>
user $ sudo plctool -i eth0 -p TL-PA4020P.pib <MAC address printed on the adapter>

The Ethernet interface in the computer I used is named ‘eth0′, so change it accordingly. You can give any name to the PIB files.

It is also easy to find out the adapters’ MAC addresses and current firmware by using another open-plc-tools command:

user $ plcstat -t -i eth0

The TP-Link tpPLC utility for Windows also shows the firmware version. I was surprised to see that the firmware version was different in the three models I had just bought:

  • TL-PA4010P firmware version: 1.4.0.20-00_401115_191120_901
  • TL-PA4010 firmware version: 1.3.1.2141-00_401013_171025_901
  • TL-PA4020P firmware version: 1.4.0.20-00_402114_191120_901

The command to update the firmware in an adapter using the NVM file I downloaded from the URL in the above-mentioned TP-Link Community forum thread and the PIB file read from the relevant adapter, is as follows:

user $ sudo plctool -i <interface> -P <PIB file> -N <NVM file> -R <MAC address of adapter>

For example:

user $ sudo plctool -i eth0 -P TL-PA4010P.pib -N FW-QCA7420-1.5.0.0026-02-CS-20200114.nvm -R 15:B3:D2:D8:5F:BA

I am fortunate in that the three models of TP-Link Powerline adapter I bought all use the Qualcomm Atheros QCA7420 chipset, so I could use the same NVM file for all four adapters that I bought. I only needed to repeat the command with a different PIB file for each adapter model. The plcstat command can be used to check that the firmware version is different from the factory original version:

user $ plcstat -t -i eth0

Actually, the tpPLC utility in Windows 10 also has the ability to upload an NVM file and a PIB file to an adapter, so, as I have tpPLC installed in a VM, I can use that instead to update firmware in my TP-Link Powerline adapters.

And what difference did upgrading the firmware in my new TP-Link adapters make? A big difference. There are no more dropouts; the connection is now stable and I no longer get interruptions while browsing the Internet. It’s a pity that TP-Link does not supply every chipset’s latest firmware file and every model’s PIB file on their support Web site so that users can update their Powerline adapters.

A Linux command-line utility to discover and list WSD-enabled computers and printers on a home network

In an earlier post I covered the installation and use of wsdd, a WS-Discovery (WSD) daemon that can run on Linux machines and enable machines running Microsoft Windows 10 to discover Linux machines in File Explorer now that Windows 10 has dropped Computer Browser, NetBIOS and SMBv1. All my Linux machines in my home network have wsdd running alongside NetBIOS broadcast name resolution, SMBv2 (used by my Android phone) and SMBv3 (used by my Linux machines). If any visitors to my house happen to bring a laptop running Windows 10, they will be able to discover my SMB shares in File Explorer, which I have always been able to do in Linux and in earlier Windows releases that supported NetBIOS and Computer Browser.

As I pointed out in a comment to another of my earlier posts, a downside of not using the (insecure) SMBv1 protocol is that the Samba utility smbtree incorrectly returns nothing if you enter the command smbtree when using SMBv2 or SMBv3. As all the Linux machines in my home network are running the wsdd daemon in addition to NetBIOS, SMBv2 and SMBv3 — and any visitors’ laptops could be running Windows 10 — it would be nice to have a command-line utility that would discover all machines. Well, here is a stab at such a utility, written by a close relative of mine as a learning exercise in WSD and Python, and is provided here as-is without any warranty or support. It consists of the following five files:

wsd-discover.sh

#!/bin/bash

function del-tmp-files() {
   if ls /tmp/wsd-*.txt 1> /dev/null 2>&1; then
      rm /tmp/wsd-*
   fi
return 0
}

# Delete pre-existing temporary work files.

del-tmp-files

# Get the V5 UUID of this machine
UUID=$(python3 $HOME/discover/wsd-gen-uuid.py)

# Send a multicast probe to all WSD capable devices and store the XML output in wsd-probe1.txt
echo
echo "Please wait.....sending multicast discovery probe and waiting 2 seconds for responses"
echo
python3 $HOME/discover/wsd-mcast-probe.py > /tmp/wsd-probe1.txt

# Iterate through the XML until the UUID to IPv4 mappings are obtained in wsd-probe9.txt
more /tmp/wsd-probe1.txt | grep Computer | awk -F "<wsa:Address>" '{print $2}' > /tmp/wsd-probe2.txt

sort -u /tmp/wsd-probe2.txt > /tmp/wsd-probe3.txt

more /tmp/wsd-probe3.txt | awk -F "uuid:" '{print $2}' > /tmp/wsd-probe4.txt

more /tmp/wsd-probe4.txt | awk -F "</wsa:Address>" '{print $1,"******",$2}' > /tmp/wsd-probe5.txt

more /tmp/wsd-probe5.txt | awk -F "from" '{print $1,"******",$2}' > /tmp/wsd-probe6.txt

more /tmp/wsd-probe6.txt | awk -F "******" '{print $1 $3}' > /tmp/wsd-probe7.txt

more /tmp/wsd-probe7.txt | awk -F "\\\('" '{print $1 $2}' > /tmp/wsd-probe8.txt

more /tmp/wsd-probe8.txt | awk -F "'" '{print $1}' > /tmp/wsd-probe9.txt

# Read the UUID to IPv4 mappings until end of file and send XML requests to each WSD host

while read RECORD; do

	URN=$(echo $RECORD | cut -d" " -f1)
	IPA=$(echo $RECORD | cut -d" " -f2)

	# Generate the HTTP/XML request file from the template
	cat $HOME/discover/wsd-template.xml | sed 's/XXXXXXXXXX/'$URN'/g' > /tmp/wsd-request.txt
	cat /tmp/wsd-request.txt | sed -i 's/YYYYYYYYYY/'$UUID'/g' /tmp/wsd-request.txt

	# Send the XML/SOAP request to the target machine
	curl -s -A wsd --header "Accept-Encoding: identity" --header "Connection: Close" \
	--header "Content-Type: application/soap+xml" --header "User-Agent: wsd" \
	--data @/tmp/wsd-request.txt http://$IPA:5357/$URN > /tmp/wsd-response-$IPA.txt

	# Extract, format and display the information returned
	echo
	echo "Device IP : $IPA"
	echo "==========================="
	echo -n "Name         :";cat /tmp/wsd-response-$IPA.txt | awk -F "FriendlyName" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Manufacturer :";cat /tmp/wsd-response-$IPA.txt | awk -F "Manufacturer" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Model        :";cat /tmp/wsd-response-$IPA.txt | awk -F "ModelName" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Category     :";cat /tmp/wsd-response-$IPA.txt | awk -F "DeviceCategory" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "URN          :";cat /tmp/wsd-response-$IPA.txt | awk -F "Address" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Type         :";cat /tmp/wsd-response-$IPA.txt | awk -F "Types" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo -n "Workgroup    :";cat /tmp/wsd-response-$IPA.txt | awk -F "<pub:Computer" '{print $2}' | awk -F "<" '{print $1}' | cut -d">" -f2
	echo

done < /tmp/wsd-probe9.txt

# This next bit is just a bit of fluff to display printers. The formatting is inconsistent because every printer
# has a different web page. Printer manufacturers are listed in the file $HOME/discover/printers.txt. If the printer
# is not in this file it won't be found in the HTTP information

# Check whether the original multicast response contains any printer information
cat /tmp/wsd-probe1.txt | grep -q -A2 Print

if [[ $? -eq 0 ]]; then # A printer of some sort has been found

   # Get the line that contains 'Print' and the two lines after it (one of which contains the printer IP and URL)
   more /tmp/wsd-probe1.txt | grep -A2 Print > /tmp/wsd-probe10.txt

   # Remove any duplicate entries
   sort -u /tmp/wsd-probe10.txt > /tmp/wsd-probe11.txt

   # Isolate the printer IP and URL information
   cat /tmp/wsd-probe11.txt | awk -F"XAddrs>" '{print $2}' | awk -F"/wsd" '{print $1}' > /tmp/wsd-probe12.txt

   # Remove blank lines to clean up the file
   sed '/^$/d' /tmp/wsd-probe12.txt > /tmp/wsd-probe13.txt

   # Read each line of the file containing the printer URLs and contact the printers in turn
   while read RECORD; do

	echo "Printers"
	echo "==========================="
	URL=$RECORD
	# Try to get the printer's HTML page
        curl -s $URL/index.html > /tmp/wsd-printer.txt
	if [[ $? -ne 0 ]]; then
           echo "Couldn't get HTML info from $URL"
	else
	   # Read each line of the printers.txt file and try to get the Make and Model from the HTML
           while read PRT; do
		 grep -q $PRT /tmp/wsd-printer.txt
		 if [[ $? -eq 0 ]]; then # Printer in the list is contained in the returned HTML
		    # Extract the Make and the following word hoping it's the Model
		    TYP=$(grep $PRT /tmp/wsd-printer.txt | awk -v a=$PRT '{for(i=1;i<=NF;i++) if ($i==a) print $i,$(i+1)}')
		    echo "URL   : $URL"
                    echo "Make  : $TYP"
		 fi
	   done < $HOME/discover/printers.txt
	fi

   done < /tmp/wsd-probe13.txt

fi

echo

#
# Delete the latest temporary work files.
#
del-tmp-files

wsd-gen-uuid.py

import uuid
import socket

hostName = (socket.gethostname())

# nameSpaces = [uuid.NAMESPACE_DNS, uuid.NAMESPACE_URL, uuid.NAMESPACE_OID, uuid.NAMESPACE_X500]
nameSpaces = [uuid.NAMESPACE_DNS]

for namespace in nameSpaces:

    print (uuid.uuid5(namespace, hostName))

wsd-mcast-probe.py

import socket
import struct
import sys
import uuid

# Create a V1 UUID for the MessageID based on the host address and current time
# The MessageID must be unique but it isn't necessary to have anything other than a V1 UUID
uuid1 = uuid.uuid1()
myuuid = str(uuid1)
print ("Generating UUID for MessageID")
print(myuuid)

# The string 'message' is a template WSD probe that is multicast to group 239.255.255.250 port 3702
# The template should not change unless there is a major change to the WSD specifications
# Escape double quotation marks within the message string (but not the outer double quotation marks)
message = "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:pnpx=\"http://schemas.microsoft.com/windows/pnpx/2005/10\" xmlns:pub=\"http://schemas.microsoft.com/windows/pub/2005/07\" xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:wsa=\"http://schemas.xmlsoap.org/ws/2004/08/addressing\" xmlns:wsd=\"http://schemas.xmlsoap.org/ws/2005/04/discovery\" xmlns:wsdp=\"http://schemas.xmlsoap.org/ws/2006/02/devprof\" xmlns:wsx=\"http://schemas.xmlsoap.org/ws/2004/09/mex\"><soap:Header><wsa:To>urn:schemas-xmlsoap-org:ws:2005:04:discovery</wsa:To><wsa:Action>http://schemas.xmlsoap.org/ws/2005/04/discovery/Probe</wsa:Action><wsa:MessageID>urn:uuid:" + myuuid + "</wsa:MessageID></soap:Header><soap:Body><wsd:Probe><wsd:Types>wsdp:Device</wsd:Types></wsd:Probe></soap:Body></soap:Envelope>"

# Convert the message to a UTF-8 byte string
bytstr = message.encode('utf-8')

# Define a variable for the multicast group and multicast destination port
multicast_group = ('239.255.255.250', 3702)
multicast_address = '239.255.255.250'

# Cheeky way to get the Internet facing Ethernet IP address for use further down
# Create a socket, pretend to use it to connect to an Internet service. Nothing is actually sent
# but the IP address of the Internet facing interface is returned 
def get_ip_address():
    sock1 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock1.connect(("8.8.8.8", 80))
    return sock1.getsockname()[0]

# Create datagram socket 1 for multicasts and allow the IP address and port to be reused in case something
# else is using them e.g. the WSD service
sock1 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)

IPADDR = (get_ip_address())

# Set the multicasts TTL to 1 so they stay on the local segment
sock1.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 1)

# Set a timeout so the socket stops listening if no data is received within the timeout
# This prevents it from locking up
sock1.settimeout(2.0)

# Bind the socket to the IP and port that we wish to use as the source IP and port of datagrams we transmit
# AND the destination IP and port of datagrams that we receive
sock1.bind ((IPADDR, 3702))

# Join the 239.255.255.250 multicast group. This isn't necessary if this script is being run on a machine
# that is also running the wsdd daemon. Joining the multicast group allows the script to be run on any
# machine regardless

mreq = struct.pack("4sl", socket.inet_aton(multicast_address), socket.INADDR_ANY)
sock1.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

try:

    # Send the WSD probe (bytstr) to the multicast group and port
    # print ('\nsending "%s"' % bytstr)
    sent = sock1.sendto(bytstr, multicast_group)

    # Listen for up to 4096 byte responses from all responders to the multicast message
    while True:
       print ('\nwaiting to receive responses')
       try:
           data, addr = sock1.recvfrom(4096)

           # We could use the format below to split 'addr' into its component IP and port fields but is isn't necessary
           # data, (ip, port) = sock1.recvfrom (4096)

       except:
           # This exception only occurs if no data is received on socket for the timeout period
           print ('\ntimed out, no more responses socket1')
           break
       else:
           # This is the response data that the bash script writes out to the wsd-probe1.txt file
           print ('\nreceived %s from %s' % (data.decode('utf-8'), addr))

finally:
    print ('\nsocket closed\n')

wsd-template.xml

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope
      xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/10"
      xmlns:pub="http://schemas.microsoft.com/windows/pub/2005/07"
      xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
      xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"
      xmlns:wsd="http://schemas.xmlsoap.org/ws/2005/04/discovery"
      xmlns:wsdp="http://schemas.xmlsoap.org/ws/2006/02/devprof"
      xmlns:wsx="http://schemas.xmlsoap.org/ws/2004/09/mex">
      <soap:Header>
            <wsa:To>urn:uuid:XXXXXXXXXX</wsa:To>
            <wsa:Action>http://schemas.xmlsoap.org/ws/2004/09/transfer/Get</wsa:Action>
            <wsa:MessageID>urn:uuid:fe11d044-bc13-11ea-b98c-2c56dc778d37</wsa:MessageID>
            <wsa:ReplyTo>
                 <wsa:Address>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:Address>
            </wsa:ReplyTo>
            <wsa:From>
                 <wsa:Address>urn:uuid:YYYYYYYYYY</wsa:Address>
            </wsa:From>
      </soap:Header>
      <soap:Body />
</soap:Envelope>

printers.txt

Brother
Canon
Epson
HP
Kodak
Lexmark

How to install

user $ mkdir $HOME/discover

Use a text editor to create the five files listed above in the directory $HOME/discover/.

Make the Bash script and the Python scripts executable:

user $ chmod u+x $HOME/discover/*.sh $HOME/discover/*.py

How to use

user $ $HOME/discover/wsd-discover.sh

The script will list discovered devices (see the caveat in the Description section below). For example:

user $ $HOME/discover/wsd-discover.sh

Please wait.....sending multicast discovery probe and waiting 2 seconds for responses


Device IP : 192.168.1.121
===========================
Name         :WSD Device tutankhamun
Manufacturer :wsdd
Model        :wsdd
Category     :Computers
URN          :urn:uuid:ff03f853-8a45-5ad9-b75b-fe4f632c8c5b
Type         :pub:Computer
Workgroup    :TUTANKHAMUN/Workgroup:HOME


Device IP : 192.168.1.10
===========================
Name         :WSD Device akhanaten
Manufacturer :wsdd
Model        :wsdd
Category     :Computers
URN          :urn:uuid:ad8fedfb-a22c-5551-92b4-653aae69f379
Type         :pub:Computer
Workgroup    :AKHANATEN/Workgroup:HOME


Device IP : 192.168.1.74
===========================
Name         :WSD Device thutmoseiii
Manufacturer :wsdd
Model        :wsdd
Category     :Computers
URN          :urn:uuid:9bf49ac3-e58d-57a4-87ea-7c0d5ef02234
Type         :pub:Computer
Workgroup    :THUTMOSEIII/Workgroup:HOME

Printers
===========================
URL   : http://192.168.1.78:80
Make  : Canon MP560


The example output above was for a network of three Linux machines running the wsdd daemon, connected via Ethernet, plus a printer connected via Wi-Fi.

Description

The scripts are non-intrusive and discover WSD-enabled devices in multicast group 239.255.255.250 port 3702, namely a) Windows 10 and b) other Linux machines running the WSD daemon wsdd or other WSD software. It runs over Ethernet and Wi-Fi. The script joins the multicast group (with a reusable socket) and sends out a WSD Probe. The responses contain the UUID-to-IP address mappings of the devices it discovers. Each discovered device is then contacted individually on its IP address TCP port 5357 to retrieve basic information.

If you run the script on Linux with the WSD Daemon (wsdd) also running (see earlier post), the script discovers itself as well as other devices. If you run the script on a machine that is not running the WSD Daemon it still discovers other devices, but not itself.

The script also discovers any WSD-enabled printers that listen for multicasts on UPnP / SSDP group 239.255.255.250 but don’t care about what UDP port is being used. If a WSD-enabled printer is detected, the script attempts to retrieve the make and model of the printer using HTTP. To detect different printer makes, add the manufacturer e.g. Canon, Epson, Lexmark etc. to the file ‘printers.txt‘. The script reports on the printer make and tries to extract the model type. It may not always format the output 100% accurately.

The main thing to bear in mind is that the scripts do not maintain state i.e. a single discovery probe is transmitted. Multicast is fundamentally unreliable and only devices that respond are reported. If the probe is lost or an end device doesn’t respond, for whatever reason, it doesn’t get reported. You can run the script a few times to ensure that it picks up as many of the devices as it possibly can.

Reconfiguring the time zone, locales and keymaps in Sabayon Linux

This is an example of how to reconfigure the time zone, locales and keymaps in a Sabayon Linux installation from the command line. Sabayon Linux uses systemd, therefore much of this example should also be applicable in other Linux distributions that use systemd, and will certainly be applicable in Gentoo Linux installations that use systemd rather than OpenRC.

You can check the currently selected keymaps (console and X Windows) and locale using the ‘localectl status‘ command. For example:

user $ localectl status
   System Locale: LANG=en_GB.UTF-8
       VC Keymap: uk
      X11 Layout: gb
       X11 Model: pc105

Let’s say I had previously configured my installation to use only the en_GB and en_US locales but I now want to add Swiss Italian. The steps would be as shown below. I will assume the system being configured is in Switzerland and therefore I will also reconfigure the time zone accordingly, but that is not essential.

Check if the desired time zone exists:

root # timedatectl list-timezones | grep Zurich
Europe/Zurich

Set the desired time zone:

root # timedatectl set-timezone Europe/Zurich

Check if the Swiss Italian locale (it_CH) has already been added:

root # localectl list-locales
C.utf8
en_GB
en_GB.iso88591
en_GB.utf8
en_US
en_US.iso88591
en_US.utf8

If the desired locale is not present, add it:

root # nano /etc/locale.gen
root # grep -v "^#\|^$" /etc/locale.gen
C.UTF-8 UTF-8
en_GB.UTF-8 UTF-8
en_GB ISO-8859-1
en_US.UTF-8 UTF-8
en_US ISO-8859-1
it_CH.UTF-8 UTF-8
it_CH ISO-8859-1

Generate the locales:

root # locale-gen
 * Generating 7 locales (this might take a while) with 1 jobs
 *  (1/7) Generating C.UTF-8 ...                                          [ ok ]
 *  (2/7) Generating en_GB.ISO-8859-1 ...                                 [ ok ]
 *  (3/7) Generating en_GB.UTF-8 ...                                      [ ok ]
 *  (4/7) Generating en_US.ISO-8859-1 ...                                 [ ok ]
 *  (5/7) Generating en_US.UTF-8 ...                                      [ ok ]
 *  (6/7) Generating it_CH.ISO-8859-1 ...                                 [ ok ]
 *  (7/7) Generating it_CH.UTF-8 ...                                      [ ok ]
 * Generation complete
 * Adding locales to archive ...                                          [ ok ]

Check the locales have been added:

root # localectl list-locales
C.utf8
en_GB
en_GB.iso88591
en_GB.utf8
en_US
en_US.iso88591
en_US.utf8
it_CH
it_CH.iso88591
it_CH.utf8

Set the desired locale:

root # localectl set-locale LANG=it_CH.UTF-8

Check which Italian console keymaps are available:

root # localectl list-keymaps | grep it
it
it-ibm
it2
mac-it

But let’s say I want to use a Swiss German keymap (sg) for the console instead of an Italian keymap. Check if a console keymap for Swiss German exists:

root # localectl list-keymaps | grep sg
sg
sg-latin1
sg-latin1-lk450

By the way, Debian, Ubuntu and its derivatives store console keymaps differently to some distributions, and the command ‘localectl list-keymaps‘ in Debian, Ubuntu and its derivatives will return an error message (the command ‘localectl set-keymap’ will still work though):

user $ localectl list-keymaps
Failed to read list of keymaps: No such file or directory

Set the console keymap to Swiss German:

root # localectl set-keymap sg

Let’s say I want to use a Swiss keymap in X Windows. Check if it exists:

root # localectl list-x11-keymap-layouts | grep sg
root # localectl list-x11-keymap-layouts | grep ch
ch

Set the X Windows keymap to Swiss:

root # localectl set-x11-keymap ch

Update the environment variables and profile:

root # env-update && source /etc/profile
>>> Regenerating /etc/ld.so.cache...

Edit /etc/default/grub and change (or add, if none exists) the console keymap entry in GRUB_CMDLINE_LINUX_DEFAULT to be vconsole.keymap=sg, and also rd.vconsole.keymap=sg (‘rd‘ stands for ‘RAM disk’) because Sabayon Linux uses an initramfs:

root # nano /etc/default/grub

Regenerate grub.cfg:

root # grub-mkconfig -o /boot/grub/grub.cfg
Generazione file di configurazione GRUB...
Trovato sfondo: /boot/grub/default-splash.png
Trovata immagine linux: /boot/kernel-genkernel-x86_64-5.4.0-sabayon
Trovata immagine initrd: /boot/initramfs-genkernel-x86_64-5.4.0-sabayon
fatto

Reboot to check if everything is working:

root # systemctl reboot

Check that the list of locales is as expected:

root # eselect locale list
Available targets for the LANG variable:
  [1]   C
  [2]   C.utf8
  [3]   en_GB
  [4]   en_GB.iso88591
  [5]   en_GB.utf8
  [6]   en_US
  [7]   en_US.iso88591
  [8]   en_US.utf8
  [9]   it_CH
  [10]  it_CH.iso88591
  [11]  it_CH.utf8
  [12]  POSIX
  [13]  it_CH.UTF-8 *
  [ ]   (free form)

Check if the current configuration is as expected:

root # localectl status
   System Locale: LANG=it_CH.UTF-8
       VC Keymap: sg
      X11 Layout: ch

If the Desktop Environment is KDE, check the file ~/.config/locale-plasmarc to see if the LANG variable is set to the locale just configured.

root # cat /home/fitzcarraldo/.config/plasma-localerc
[Formats]
LANG=en_GB.UTF-8

If it is not, delete the file:

root # rm /home/fitzcarraldo/.config/plasma-localerc

then logout, login again and re-check the file:

root # cat /home/fitzcarraldo/.config/plasma-localerc
[Formats]
LANG=it_CH.UTF-8

The system should now be ready for use with the new time zone, locale and keymaps.

user $ date
gio 2 lug 2020, 15:55:21, CEST
user $ localectl status
   System Locale: LANG=it_CH.UTF-8
       VC Keymap: sg
      X11 Layout: ch

Using a mixture of locale variables

It is not mandatory for all the locale variables to be for the same locale. For example, suppose I want to use the currency and number formats of one of the other locales I added. That is not so outlandish: I could be a Swiss national whose mother tongue is Swiss Italian, working in the Swiss branch of a British company and I want the currency format and number format on my work computer to be British, but everything else to be Swiss. To achieve this I can additionally do the following:

root # localectl set-locale LC_MONETARY=en_GB.UTF-8
root # localectl set-locale LC_NUMERIC=en_GB.UTF-8
root # env-update && source /etc/profile

Check that the main locale and keymaps remain as they were but that the two locale variables have been changed to the British locale:

root # localectl status
   System Locale: LANG=it_CH.UTF-8
                  LC_NUMERIC=en_GB.UTF-8
                  LC_MONETARY=en_GB.UTF-8
       VC Keymap: sg
      X11 Layout: ch
root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC=en_GB.UTF-8
LC_TIME="it_CH.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=

You can see above that only $LC_NUMERIC and $LC_MONETARY have changed, as I wanted. As I did not change the time zone, the command I used earlier to set the time zone to Europe/Zurich is still in force:

root # date
gio 2 lug 2020, 16:12:18, CEST

By the way, if you try to change one of the variables from en_GB.UTF-8 back to it_CH.UTF-8, the change does not show in the output of the locale command. For example, let’s say you want to change LC_NUMERIC back to it_CH.UTF-8:

root # localectl status
   System Locale: LANG=it_CH.UTF-8
                  LC_NUMERIC=en_GB.UTF-8
                  LC_MONETARY=en_GB.UTF-8
       VC Keymap: sg
      X11 Layout: ch
root # localectl set-locale LC_NUMERIC=it_CH.UTF-8
root # cat /etc/locale.conf   
LANG=it_CH.UTF-8
LC_MONETARY=en_GB.UTF-8
root # cat /etc/env.d/02locale
LANG=it_CH.UTF-8
LC_MONETARY=en_GB.UTF-8
root # localectl status
   System Locale: LANG=it_CH.UTF-8
                  LC_MONETARY=en_GB.UTF-8
       VC Keymap: sg
      X11 Layout: ch
root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC=en_GB.UTF-8  <-- Notice it didn't change
LC_TIME="it_CH.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=
root # env-update && source /etc/profile
>>> Regenerating /etc/ld.so.cache...
root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC=en_GB.UTF-8 <-- Notice it still didn't change
LC_TIME="en_GB.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=

This is one of the reasons I’m not keen on the layer of abstraction added by systemd. The way to get LC_NUMERIC back to it_CH.UTF-8 is to change all the locale variables to en_GB.UTF-8 then back to it_CH.UTF-8:

root # localectl set-locale LANG=en_GB.UTF-8
root # env-update && source /etc/profile
root # localectl set-locale LANG=it_CH.UTF-8
root # env-update && source /etc/profile
root # reboot

After rebooting, the change will have been applied:

root # locale
LANG=it_CH.UTF-8
LC_CTYPE="it_CH.UTF-8"
LC_NUMERIC="it_CH.UTF-8"
LC_TIME="it_CH.UTF-8"
LC_COLLATE="it_CH.UTF-8"
LC_MONETARY="it_CH.UTF-8"
LC_MESSAGES="it_CH.UTF-8"
LC_PAPER="it_CH.UTF-8"
LC_NAME="it_CH.UTF-8"
LC_ADDRESS="it_CH.UTF-8"
LC_TELEPHONE="it_CH.UTF-8"
LC_MEASUREMENT="it_CH.UTF-8"
LC_IDENTIFICATION="it_CH.UTF-8"
LC_ALL=
root # localectl status
   System Locale: LANG=it_CH.UTF-8
       VC Keymap: sh
      X11 Layout: ch

Notice that everything has changed back to it_CH.UTF-8, including $LC_MONETARY, so you’d have to repeat the command ‘localectl set-locale LC_MONETARY=en_GB.UTF-8‘ if you wanted that to still be the British format.

If you use KDE, also check the contents of the file ~/.config/plasma-localerc to make sure it contains the correct locale:

user $ cat ~/.config/plasma-localerc
[Formats]
LANG=it_CH.UTF-8

Optionally you could edit that file to add desired settings. For example:

[Formats]
LANG=it_CH.UTF-8
LC_CTYPE=it_CH.UTF-8
LC_NUMERIC=en_GB.UTF-8
LC_TIME=it_CH.UTF-8
LC_COLLATE=it_CH.UTF-8
LC_MONETARY=en_GB.UTF-8
LC_MESSAGES=it_CH.UTF-8
LC_PAPER=it_CH.UTF-8
LC_NAME=it_CH.UTF-8
LC_ADDRESS=it_CH.UTF-8
LC_TELEPHONE=it_CH.UTF-8
LC_MEASUREMENT=it_CH.UTF-8
LC_IDENTIFICATION=it_CH.UTF-8
useDetailed=true

Alternatively, delete that file then logout and login again to make KDE Plasma pick up the values of the variables from the existing configuration. KDE Plasma will recreate the file.

Quick work-around for bug in KDE Plasma 5.19.1 (Restart and Shut Down from system menu result in Log Out instead)

See the recent post on 17 June 2020 by reddit and Arch Linux user SnowGigs re KDE Plasma 5.19.1: Cannot shutdown or restart from system menu. Actually I experience the same problem with Plasma 5.19.0 in Gentoo Linux.

Until you can install a newer version of Plasma which fixes the bug, here is a quick and easy work-around that is perfectly usable:

Install kde-apps/kdialog and add the following Desktop Configuration file Shutdown.desktop to your Desktop directory — also drag it to the Panel if you wish — and make it executable:

[Desktop Entry]
Comment[en_GB]=Shutdown system
Comment=Shutdown system
Exec=kdialog --title "Shutdown system" --warningcontinuecancel "Are you sure you want to shutdown?" && loginctl poweroff
GenericName[en_GB]=Shutdown
GenericName=Shutdown
Icon=system-shutdown
MimeType=
Name[en_GB]=Shutdown
Name=Shutdown
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

(Obviously replace ‘fitzcarraldo‘ with your own username.)

user $ chown 744 ~/Desktop/Shutdown.desktop

I have assumed your installation has elogind installed, hence the command ‘loginctl poweroff‘ in the above-mentioned file. If systemd is installed instead of elogind, use the command ‘systemctl poweroff‘ instead.

When you click/double-click on the ‘Shutdown’ icon, a window will pop-up asking you ‘Are you sure you want to shutdown?’ with ‘Continue’ and ‘Cancel’ buttons. If you click ‘Continue’ then the system will execute the ‘loginctl poweroff‘ command; if you click on ‘Cancel’ then it won’t. Simple as that.

If you want, you can also create a file Reboot.desktop to reboot the system:

[Desktop Entry]
Comment[en_GB]=Reboot system
Comment=Reboot system
Exec=kdialog --title "Reboot system" --warningcontinuecancel "Are you sure you want to reboot?" && loginctl reboot
GenericName[en_GB]=Reboot
GenericName=Reboot
Icon=system-reboot
MimeType=
Name[en_GB]=Reboot
Name=Reboot
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

Similarly, if systemd is installed instead of elogind, use the command ‘systemctl reboot‘ instead.

Using WS-Discovery to enable Windows 10 to browse SMB shares in my home network of Linux computers

I have not used Windows 10 for more than two years now (see ‘Bye bye Windows 10, and good riddance‘ regarding my failed attempts to upgrade Windows 10 Version 1607 to 1703 and 1709). Nevertheless I am aware that, since Version 1709, Windows 10 no longer has SMBv1 and Computer Browser service installed by default. Computer Browser service used NetBIOS and SMBv1 to provide what Microsoft named ‘My Network Places‘ or ‘Network Neighborhood’. Thus Microsoft has dropped the concepts of network ‘workgroups’, ‘master browsers’, NetBIOS, NetBIOS broadcasts, WINS and so on. SMB has not been dropped, though; Versions 2 and 3 of the SMB protocol are now used, albeit using a different mechanism for device discovery.

Although they perform different jobs, Microsoft bundled the Computer Browser service software with the SMBv1 software. Microsoft could have provided them separately, but it made some sense to bundle them together in the early days of Windows networking. Thus, as SMBv1 is not installed by default in Windows 10 Version 1709 and later versions, neither is Computer Browser service. To put it another way, if you install SMBv1 in Windows 10 you automatically install Computer Browser service as well. None of that interested me since I stopped using Windows 10 after Version 1607. Since then my home network has comprised a server, desktop and laptops running various Linux distributions with Samba and using broadcast NetBIOS for name resolution. Of course I know that NetBIOS — especially broadcast NetBIOS for name resolution — is ancient networking technology, but it works well for my home networking needs. All my machines can browse each other’s SMB shares and create/copy/move/delete remote files and folders. The File Manager + app on my phone running Android 9 can also browse SMB shares on the Linux machines and create/copy/move/delete remote files and folders.

Two of my blog posts from 2016 and 2017 explain how I set up my home network for file sharing. One of the machines in the network had Windows 10 1607 installed, but that was replaced with Lubuntu in 2018.

SMBv1 is an inherently insecure protocol, so, after I dropped Windows, I reconfigured Samba on my Linux machines to use only SMBv3, which works fine. Subsequently I found that Android 9 on my Samsung Galaxy Note 8 phone apparently does not support SMBv3, only SMBv1 and SMBv2, so I reconfigured Samba on my Linux machines to allow SMBv2 as well as SMBv3. In other words, the Linux machines use SMBv3 with each other but SMBv2 with the phone (see my comments in the Comments section of my 2016 post ‘A correct method of configuring Samba for browsing SMB shares in a home network‘).

Anyway, I happen to have an evaluation copy of Windows 10 Enterprise Version 1709 installed in a VirtualBox VM (virtual machine) on one of my Linux laptops and, purely to satisfy my curiosity, I decided to try to get Windows 10 Version 1709 to browse and access SMB shares on the Linux machines in my home network, and vice versa, without having to dispense with broadcast NetBIOS name resolution for the Linux machines and without having to install SMBv1 (and Computer Browser service) in Windows 10.

When I first booted Windows 10 Enterprise 1709, SMB shares on my Linux machines were not displayed in File Explorer, and Windows 10 could not find them if I entered the UNC (Universal Naming Convention) address ‘\\hostname\foldername‘ or ‘\\IPaddress\foldername‘ (e.g. ‘\\AKHANATEN\anne‘ or ‘\\192.168.1.70\anne‘) in File Explorer’s address bar. My Web searches indicated that many people cannot see SMB shares in File Explorer either but can access a share by entering the UNC address in the File Explorer address bar. Apparently the advice from Microsoft these days is to use ‘Map a Network Drive…’ in File Explorer. Therefore, given that I wanted to be able to browse SMB shares in ‘File Explorer’ > ‘Network’, I clearly had some work to do. My goal for Windows 10 was twofold: to be able to view my remote SMB shares in Windows 10 File Explorer automatically and to be able to access (copy/move/delete/open) my remote SMB shares in Windows 10 File Explorer. Of course I also wanted to be able to browse and access SMB shares on the Windows 10 machine from the Linux machines.

Now, Windows 10 comes with Web Services Dynamic Discovery (WS-Discovery) installed. This enables SMB hosts running WS-Discovery software to be found by clients running WS-Discovery software. I believe Version 20.04 of the KDE Applications package kio-extras will support SMB host discovery using WS-Discovery, but that version is not available in the Stable Branch of Gentoo Linux installed on my main laptop, nor in Lubuntu 18.04 which is installed on my family’s desktop machine. So I thought I would have a look at what is currently available for those two distributions. I was particularly interested to see if I could find an implementation of WS-Discovery for Linux that would run in parallel with broadcast NetBIOS name resolution currently installed on the Linux machines in my home network, as broadcast NetBIOS name resolution works fine with SMBv2 and SMBv3 for Linux and Android devices in a home network (my Samsung Galaxy Note 8 phone can browse the SMB shares on any of the Linux machines in my home network).

Thanks are due to Steffen Christgau for creating a daemon that can be used in Linux installations to enable Windows 10 to discover SMB shares on Linux machines via WS-Discovery: wsdd – A Web Service Discovery host daemon. The README file for wsdd states:

wsdd implements a Web Service Discovery host daemon. This enables (Samba) hosts, like your local NAS device, to be found by Web Service Discovery Clients like Windows.

It also implements the client side of the discovery protocol which allows to search for Windows machines and other devices implementing WSD. This mode of operation is called discovery mode.

wsdd only depends on Python 3 and can be installed in many Linux distributions. If no wsdd package exists for a specific distribution, it can simply be run from the command line or from a Bash script. The following blog post by Ralph Mönchmeyer explains how to use wsdd (although not a complete solution for my specific case): Samba 4, shares, wsdd and Windows 10 – how to list Linux Samba servers in the Win 10 Explorer.

Below I list the steps I took to enable me to browse SMB shares in an evaluation copy of Windows 10 Enterprise Version 1709 running in a VM on one of my Linux laptops. I don’t have access to the latest version of Windows 10 (2004), but hopefully some or most of the following will still be applicable.

Step 1. Disable firewalls temporarily

I disabled the firewall in the Linux machine and in the Windows 10 machine so that the firewalls could be ruled out if there were any problems getting share browsing to work. Once all the steps were completed I re-enabled the firewalls.

Step 2. Specify the workgroup in Windows 10

Select ‘Control Panel’ > ‘System and Security’ > ‘System’ and, under ‘Computer name, domain, and workgroup settings’, if necessary click ‘Change settings’ to rename the workgroup. The default workgroup name was ‘WORKGROUP‘ so I renamed it to ‘HOME‘, my current network’s workgroup.

Step 3. Ensure the correct SMB protocol in Windows 10

SMBv1 (and Computer Browser service) are disabled by default in Windows 10 Version 1709 and later (see ‘SMBv1 is not installed by default in Windows 10 version 1709, Windows Server version 1709 and later versions‘) but I nevertheless made sure that SMBv1 is disabled and that SMBv2 and SMBv3 are installed (see ‘How to detect, enable and disable SMBv1, SMBv2, and SMBv3 in Windows‘). I did the following in PowerShell (Run as administrator):

PS C:\WINDOWS\system32> Disable-WindowsOptionalFeature -Online -FeatureName SMB1Protocol
PS C:\WINDOWS\system32> Set-SmbServerConfiguration -EnableSMB2Protocol $true

Step 4. Disable NetBIOS-over-TCP/IP in Windows 10

Select ‘Settings’ > ‘Network & Internet’ > ‘Ethernet’ > ‘Change adapter options’.

Right-click ‘Ethernet’, click ‘Properties’, select ‘Internet Protocol Version 4 (TCP/IPv4)’ and click ‘Properties’. Click ‘Advanced’. Click on the WINS tab (even though my network does not use WINS), select ‘Disable NetBIOS over TCP/IP’ and click ‘OK’, ‘OK’ and ‘Close’.

Step 5. Configure ‘Function Discovery’ in Windows 10

See the article ‘SMBv1 is not installed by default in Windows 10 version 1709, Windows Server version 1709 and later versions | Microsoft Docs‘, in particular the following:

Explorer Network Browsing

The Computer Browser service relies on the SMBv1 protocol to populate the Windows Explorer Network node (also known as “Network Neighborhood”). This legacy protocol is long deprecated, doesn’t route, and has limited security. Because the service cannot function without SMBv1, it is removed at the same time.

However, if you still have to use the Explorer Network in home and small business workgroup environments to locate Windows-based computers, you can follow these steps on your Windows-based computers that no longer use SMBv1:

  1. Start the “Function Discovery Provider Host” and “Function Discovery Resource Publication” services, and then set them to Automatic (Delayed Start).
  2. When you open Explorer Network, enable network discovery when you are prompted.

All Windows devices within that subnet that have these settings will now appear in Network for browsing. This uses the WS-DISCOVERY protocol. Contact your other vendors and manufacturers if their devices still don’t appear in this browse list after the Windows devices appear. It is possible they have this protocol disabled or that they support only SMBv1.

Press Windows Key+R, enter ‘services.msc‘ (without the quotes) and click ‘OK’.

Change the ‘Startup type’ of ‘Functions Discovery Provider Host’ to ‘Automatic (Delayed Start)’.

Change the ‘Startup type’ of ‘Function Discovery Resource Publication’ to ‘Automatic (Delayed Start)’.

Step 6. Configure the sharing options in Windows 10

Select ‘Settings’ > ‘Network & Internet’ > ‘Sharing options’ and configure the options as follows:

Private (current profile)
  1. Network discovery
    • ‘Turn on network discovery’ is selected.
    • ‘Turn on automatic setup of network connected devices.’ is ticked.
  2. File and printer sharing
    • ‘Turn on file and printer sharing’ is selected.
  3. HomeGroup connections
    • ‘Allow Windows to manage homegroup connections (recommended)’ is selected.
Guest or Public
  1. Network discovery
    • ‘Turn on network discovery’ is selected.
  2. File and printer sharing
    • ‘Turn on file and printer sharing’ is selected.
All Networks
  1. Public folder sharing
    • ‘Turn on sharing so anyone with network access can read and write files in the Public folders’ is selected.
  2. Media streaming
    • Nothing is selected.
  3. File sharing connections
    • ‘Use 128-bit encryption to help protect file sharing connections (recommended)’ is selected.
  4. Password protected sharing
    • ‘Turn off password protected sharing’ is selected.

Step 7. Install WS-Discovery daemon on the Linux machines

Gentoo Linux
In Gentoo I simply installed the package net-misc/wsdd from the guru overlay:

root # eix -I wsdd
[I] net-misc/wsdd [1]
     Available versions:  (~)0.5 (~)0.6 {samba PYTHON_TARGETS="python3_6 python3_7 python3_8"}
     Installed versions:  0.6(00:39:07 07/06/20)(-samba PYTHON_TARGETS="python3_7 -python3_6 -python3_8")
     Homepage:            https://github.com/christgau/wsdd
     Description:         A Web Service Discovery host daemon.

[1] "guru" /var/lib/layman/guru

and, as I use OpenRC in Gentoo, I configured /etc/conf.d/wsdd.conf as follows:

# /etc/conf.d/wsdd

# Override the default user/group under which wsdd runs.
# Must follow the user[:group] notation.
#WSDD_USER="daemon:daemon"

# Specify alternative log file location.
#WSDD_LOG_FILE="/var/log/wsdd.log"

# Disable automatic detection of the workgroup from samba configuration.
#WSDD_WORKGROUP="MYGROUP"
WSDD_WORKGROUP="HOME"

# Additional options for the daemon, e.g. to listen on interface eth0 only.
# Refer to wsdd(1) for details.
#WSDD_OPTS="-i eth0"
WSDD_OPTS="--shortlog --interface enp4s0f1 --interface wlp3s0 --hostname tutankhamun --discovery"

Specifying the interfaces and hostname are optional, but wsdd seemed to work better when I specified them explicitly. You can ascertain the interfaces by using the command ‘ip address‘ or the deprecated command ‘ifconfig‘.

I added the service to the default runlevel so that it is started automatically when I boot the machine, and then I started it:

root # rc-update add wsdd default
root # rc-service wsdd start

Lubuntu 18.04

In Lubuntu 18.04 (which uses systemd) wsdd can be installed either manually or from a package:

a) Manual installation

user $ wget https://github.com/christgau/wsdd/archive/master.zip
user $ unzip master.zip
user $ sudo cp wsdd-master/src/wsdd.py /usr/bin/wsdd
user $ sudo cp wsdd-master/etc/systemd/wsdd.service /etc/systemd/system/

Edit the systemd service file /etc/systemd/system/wsdd.service to add desired options to the ExecStart command and to change the group from ‘nobody‘ to ‘nogroup‘:

...
ExecStart=/usr/bin/wsdd --workgroup HOME --shortlog --interface eno1 --interface wlp2s0 --hostname thutmoseiii --discovery
...
User=nobody
Group=nogroup
...

You can check whether the user and group exist in your installation as follows:

user $ grep ^nobody /etc/passwd
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
user $ grep ^nobody /etc/group
user $ grep ^nogroup /etc/group
nogroup:x:65534:

Actually, I prefer to specify ‘daemon‘ for the user and group in the wsdd.service file (which is also what the Gentoo Linux ebuild uses and what the .deb package uses):

...
ExecStart=/usr/bin/wsdd --workgroup HOME --shortlog --interface eno1 --interface wlp2s0 --hostname thutmoseiii --discovery
...
User=daemon
Group=daemon
...

You can check that this user and group also exist:

user $ grep ^daemon /etc/passwd
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
user $ grep ^daemon /etc/group
daemon:x:1:

(I tried both nobody:nogroup and daemon:daemon, and there was no apparent difference in behaviour.)

Enable the service so that it starts automatically when the machine is booted, and also start it now:

user $ sudo systemctl enable wsdd
user $ sudo systemctl start wsdd

b) Installing from a package

Here is a link to a .deb package for wsdd Version 0.6.0:

https://pkg.ltec.ch/public/pool/main/w/wsdd/

The resulting installation differs slightly from the manual procedure; the package creates a configuration file /etc/wsdd.conf and you declare the wsdd options in that file instead:

# command line parameters for wsdd (consult man page)
WSDD_PARAMS=""

The package also installs a systemd service file /lib/systemd/system/wsdd.service containing the following:

[Unit]
Description=Web Services Dynamic Discovery host daemon
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
EnvironmentFile=/etc/wsdd.conf
ExecStart=/usr/bin/wsdd $WSDD_PARAMS
User=daemon
Group=daemon

[Install]
WantedBy=multi-user.target

The package installs the Python 3 executable wsdd in the directory /usr/bin/. It’s a very straightforward package.

Step 8. Configure Samba to make Windows 10 prompt for username and password

When you click on a network share in Windows 10’s File Explorer, Windows 10 uses the Windows 10 username and password to try to access the SMB share on the remote machine (see ‘Samba share does not ask for credentials from Windows Client‘). This will obviously not work unless the usernames/passwords on both machines match. To make Windows 10 prompt the user to enter the remote username and password, edit the file /etc/samba/smb.conf on each Linux machine and comment out the line ‘map to guest = bad user‘ (see the smb.conf files listed in my 2016 article ‘A correct method of configuring Samba for browsing SMB shares in a home network‘).

Step 9. Enable guest access in Windows 10

If I enter a SMB share’s UNC address in File Explorer’s address bar, or if I double-click on the remote machine’s icon in File Explorer (after WS-Discovery has made the SMB share visible in File Explorer), Windows 10 displays the following error message:

Network Error

Windows cannot access \\hostname

Check the spelling of the name. Otherwise, there might be a problem with your network. To try to identify and resolve network problems, click Diagnose.

Error Code: 0x80070035
The network path was not found.

This has nothing to do with the fact that SMBv1 is disabled in Windows 10. It happens because Windows 10 1709 and onwards have guest logins disabled:

To enable guest logins I edited the Windows 10 Registry and changed the following key from zero to one:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters]

“AllowInsecureGuestAuth”=dword:1

Step 10. Configure the Windows 10 firewall

Select ‘Windows Defender Security Center’ > ‘Firewall & network protection’.

Click ‘Allow an app through the firewall’.

If not already ticked, select ‘Private’ and ‘Public’ for ‘Network Discovery’ and for ‘File and Printer Sharing’.

Step 11. Configure the Linux firewall

This is where things get more complicated. According to the README for wsdd:

Firewall Setup

Both incoming and outgoing multicast traffic on port 3702 must be allowed. For IPv4, the multicast address is 239.255.255.250, for IPv6 the link local SSDP multicast address (ff02::c) is used.

Incoming TCP traffic (and related outgoing traffic) on port 5357 must be allowed.

My laptops and desktop use UFW, and below I explain how I configured UFW to satisfy the above requirements.

Firstly, as my firewall is configured to deny incoming traffic and allow outgoing traffic by default, I enabled UFW and added the following DNS rules to UFW’s main rules (the following two commands add rules for both IPv4 and IPv6):

user $ sudo ufw allow 53/tcp
user $ sudo ufw allow 53/udp

Note that, in order for the multicast rule I use to work, xt_pkttype must either have been built into the kernel or built as a kernel module and have been loaded:

user $ lsmod | grep pkttype
xt_pkttype             16384  2
x_tables               40960  17 ip6table_filter,xt_conntrack,iptable_filter,xt_LOG,xt_multiport,xt_tcpudp,xt_addrtype,ip6t_rt,ip6_tables,ipt_REJECT,xt_CT,xt_pkttype,iptable_raw,ip_tables,xt_limit,xt_hl,ip6t_REJECT

To load the module automatically at boot, in Gentoo Linux I added ‘xt_pkttype‘ to the list of modules in the file /etc/conf.d/modules, and in Lubuntu 18.04 I added ‘xt_pkttype‘ to the list of modules in the file /etc/modules-load.d/modules.conf.

Also note that my firewall had previously already been configured for NetBIOS and SMB by using the following commands:

user $ # Rules for SMB
user $ # IPv4:
user $ sudo ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
user $ sudo ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp
user $ # IPv6:
user $ # (NetBIOS is undefined for IPv6 but I believe SMB uses Port 445 in IPv6)
user $ sudo ufw allow from ff80::/10 to any port 445 proto tcp

IPv4

The end of the file /etc/ufw/before.rules previously looked like this:

...
# allow MULTICAST mDNS for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 224.0.0.251 --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 239.255.255.250 --dport 1900 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

I inserted seven lines as shown below:

...
# allow MULTICAST mDNS for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 224.0.0.251 --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 239.255.255.250 --dport 1900 -j ACCEPT

# allow MULTICAST WS-Discovery for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -m pkttype --pkt-type multicast -j ACCEPT
-A ufw-before-input -p udp -s 192.168.1.0/24 --dport 3702 -j ACCEPT
-A ufw-before-input -p udp -s 192.168.1.0/24 --sport 3702 -j ACCEPT
-A ufw-before-input -p tcp -s 192.168.1.0/24 --dport 5357 -j ACCEPT
-A ufw-before-input -p tcp -s 192.168.1.0/24 --sport 5357 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

Actually the two IPv4 rules shown above for mDNS and UPnP that were already in the file /etc/ufw/before.rules have become redundant because the first of the five new rules I added encompasses them. It does no harm to leave those two rules in the file, though.

IPv6

The end of the file /etc/ufw/before6.rules previously looked like this:

...
# allow MULTICAST mDNS for service discovery
-A ufw6-before-input -p udp -d ff02::fb --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery
-A ufw6-before-input -p udp -d ff02::f --dport 1900 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT

I inserted six lines as shown below:

...
# allow MULTICAST mDNS for service discovery
-A ufw6-before-input -p udp -d ff02::fb --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery
-A ufw6-before-input -p udp -d ff02::f --dport 1900 -j ACCEPT

# allow MULTICAST WS-Discovery for service discovery
-A ufw6-before-input -m pkttype --pkt-type multicast -j ACCEPT
-A ufw6-before-input -p udp -s fe80::/10 --dport 3702 -j ACCEPT
-A ufw6-before-input -p udp -s fe80::/10 --sport 3702 -j ACCEPT
-A ufw6-before-input -p tcp -s fe80::/10 --dport 5357 -j ACCEPT
-A ufw6-before-input -p tcp -s fe80::/10 --sport 5357 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT

Actually the two IPv6 rules shown above for mDNS and UPnP that were already in the file /etc/ufw/before6.rules have become redundant because the first of the five new rules I added encompasses them. It does no harm to leave those two rules in the file, though.

Because the Linux machines in my network still use broadcast NetBIOS for name resolution I left all the NetBIOS rules in UFW as they were, including the extra lines I previously added to /etc/ufw/before.rules (see the raw table rule at the end of /etc/ufw/before.rules listed above and my blog post ‘Prevent Linux firewalls interfering with Samba commands in a home network that uses broadcast NetBIOS name resolution‘).

Actually, as my laptops change firewall zones automatically (see my post ‘Firewall zones (profiles) in Linux, and how to switch them automatically if you use UFW‘), on my laptops I added the new rules to the zone for my home network specified in my NetworkManager Dispatcher hook script /etc/NetworkManager/dispatcher.d/20_ufw-zones.

After reloading UFW, the UFW status on my machines now looks like this (I’ve excluded rules unrelated to this topic):

user $ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
137,138/udp                ALLOW IN    192.168.1.0/24
139,445/tcp                ALLOW IN    192.168.1.0/24
53/tcp                     ALLOW IN    Anywhere
53/udp                     ALLOW IN    Anywhere
445/tcp                    ALLOW IN    ff80::/10
53/tcp (v6)                ALLOW IN    Anywhere (v6)
53/udp (v6)                ALLOW IN    Anywhere (v6)

Note that UFW does not display rules declared in /etc/ufw/{before,before6}.rules

Step 12. Re-enable the Windows 10 firewall

Select ‘Settings’ > ‘Network & Internet’ > ‘Windows Firewall’.

Step 13. Check that wsdd is working as expected

To check that wsdd is actually detecting other machines running WS-Discovery, you can stop the daemon running and instead launch wsdd manually in a terminal window with verbose logging enabled.

For example, on my laptop running Gentoo Linux I did the following:

user $ sudo rc-service wsdd stop
user $ wsdd --workgroup HOME --verbose --interface enp4s0f1 --interface wlp3s0 --hostname tutankhamun --discovery

And on my family’s desktop running Lubuntu 18.04 I did the following:

user $ sudo systemctl stop wsdd
user $ wsdd --workgroup HOME --verbose --interface eno1 --interface wlp2s0 --hostname thutmoseiii --discovery

Check the output in the terminal window includes a discovered line for each machine running Windows 10 and for each Linux machine running wsdd. For example:

...
2020-06-16 00:31:09,331:wsdd INFO(pid 17574): discovered MSWIN10PC in Workgroup:HOME on 192.168.1.111%eno1
...
2020-06-16 00:31:10,013:wsdd INFO(pid 17574): discovered MSWIN10PC in Workgroup:HOME on [fe80::fc7e:7068:8c2c:e664]%eno1
...

After pressing Ctrl+C to stop wsdd running in the terminal, you can restart the daemon:

Gentoo Linux

user $ sudo rc-service wsdd start

Lubuntu 18.04

user $ sudo systemctl start wsdd

With wsdd running on the Linux machines they become visible in File Explorer on Windows 10 machines connected to the network. However, the converse is not necessarily true, as explained further on.

As I had previously configured Samba on my Linux machines to use broadcast NetBIOS to resolve names, Samba on the Linux machines fails to resolve the hostnames of the Windows 10 machines because Windows 10 no longer supports NetBIOS name resolution (neither broadcast nor WINS). I confirmed this by using the smbclient command in a terminal window:

user $ sudo smbclient //MSEDGEWIN10/TestSMBShare1 --debuglevel=10
...
added interface eno1 ip=192.168.1.111 bcast=192.168.1.255 netmask=255.255.255.0
Netbios name list:-
my_netbios_names[0]="THUTMOSEIII"
Client started (version 4.7.6-Ubuntu).
Opening cache file at /var/cache/samba/gencache.tdb
Opening cache file at /var/run/samba/gencache_notrans.tdb
sitename_fetch: No stored sitename for realm ''
internal_resolve_name: looking up MSEDGEWIN10#20 (sitename (null))
no entry for MSEDGEWIN10#20 found.
name_resolve_bcast: Attempting broadcast lookup for name MSEDGEWIN10
Connection to MSEDGEWIN10 failed (Error NT_STATUS_UNSUCCESSFUL)

However, in Gentoo Linux (Stable Branch, KDE Plasma 5.18.5, KDE Applications 19.12.3) on my main laptop I can enter ‘smb://hostname/sharename‘ (e.g. smb://msedgewin10/Users/Public) in the Dolphin file manager’s address bar and browse the contents of the SMB share on the Window 10 machine. I assume this is because Avahi on the Linux machine performs name resolution anyway even though the broadcast NetBIOS lookup has failed. Although Lubuntu 18.04 also has the Avahi daemon running, it does not resolve the hostname when I enter ‘smb://hostname/sharename‘ in PCManFM’s address bar; I have to enter ‘smb://IPaddress/sharename‘ (e.g. smb://192.168.1.64/Users/Public) to be able to browse the contents of the Windows 10 shared folder.

Conclusion

wsdd running on Linux machines enables Windows 10 to view networked Linux machines in File Explorer and browse SMBv2 and SMBv3 shares residing on Linux machines. It does not guarantee I will be able to view Windows 10 machines in Linux file managers automatically, though. But I can access Windows 10 machines by entering ‘smb://IPaddress/sharename‘ in the Linux file manager’s address bar, or, depending on what has been installed in the Linux installation and how it has been configured, by entering ‘smb://hostname/sharename‘.

To access a Linux SMB shared folder (as declared in that machine’s smb.conf file) in Windows 10 File Explorer, either I double-click on the Linux machine’s icon in the Network view or I enter the UNC address (e.g. \\tutankhamun\Users\Public) in the address bar. I can then access the files and sub-folders.

To browse a Windows 10 SMB shared folder and files in KDE Dolphin in Gentoo Linux current Stable Branch on my main laptop, I enter the UNC address (e.g. smb://msedgewin10/Users/Public) or click on the location I previously bookmarked under ‘Places’ in the left pane of the Dolphin window. I can then access the files and sub-folders. To browse a Windows 10 SMB shared folder and files in LXDE PCManFM in Lubuntu 18.04, I enter the UNC address with an IP address instead of a hostname (e.g. smb://192.168.1.64/Users/Public). I can then access the files and sub-folders. I am going to have to do some more digging to try to find out why KDE Dolphin in Gentoo Linux on my main laptop (kio-extras installed from Gentoo ebuild kio-extras-19.12.3-r2) can access Windows 10 by hostname but PCManFM in Lubuntu 18.04 cannot.

To enable machines running Window 10 to browse SMB shares on my other Linux machines I would need to perform the same Linux-related steps in each of those installations. My server firewall uses IPTABLES directly, rather than UFW, so the syntax of the additional firewall rules would be different.

Addendum, 16 June 2020: I suspected the problem browsing the Windows 10 SMB shares from Lubuntu 18.04 is due to PCManFM, so I installed a different file manager: SpaceFM (Version 1.0.5 for GTK2) and its associated utility udevil (Version 0.4.4). SpaceFM allows me to enter UNC addresses such as ‘smb://mswin10pc/Users/Public‘ without any problems. So, problem solved in Lubuntu 18.04 now as well.

How to send a message to running X Windows sessions in a multi-user Linux system

The ‘wall‘ command can be used to broadcast a message from a TTY console to other logged-in TTY console users in a multi-user Linux system. The Linux command ‘notify-send‘ can be used to send a message (a.k.a. notification) within an X Windows session, and Desktop Environments such as KDE and GNOME use notify-send to display pop-up notifications to the user. However, apparently no program exists in Linux to broadcast a message to other running X Windows sessions; that sounds like the sort of thing systemd developers would implement. A few years ago Unix & Linux Stack Exchange user Andy posted a Bash script notify-send-all to do just that (see Show a notification across all running X displays). For example, if you wanted to send a message to all the users of a multi-seat, multi-user Linux system who are currently logged-in to Desktop Environments, you could enter the following command to run the script in your home directory:

$ sudo ./notify-send-all -t 50000 "Warning" "Don't forget the staff meeting at 15:00 today."

Below is a slightly modified version of Andy‘s script that works for me in Lubuntu 18.04:

#!/bin/bash
PATH=/usr/bin:/bin
who|grep -E "\(:[0-9](\.[0-9])*\)"|awk '{print $1$5}'|sort -u > /tmp/xusers
while read XUSER; do
    NAME=(${XUSER/(/ })
    DISPLAY=${NAME[1]/)/}
    DBUS_ADDRESS=unix:path=/run/user/$(id -u ${NAME[0]})/bus
    sudo -u ${NAME[0]} DISPLAY=${DISPLAY} \
                       DBUS_SESSION_BUS_ADDRESS=${DBUS_ADDRESS} \
                       PATH=${PATH} \
                       notify-send "$@"
done < /tmp/xusers

Here is my tidied-up version:

#!/bin/bash
who | awk '{print $1, $NF}' | tr -d "()" | sort -u |
while read XUSER DISPNUM; do
    sudo -u $XUSER DISPLAY=$DISPNUM \
                   DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u $XUSER)/bus \
                   notify-send "$@"
done

In Gentoo Linux DBUS_SESSION_BUS_ADDRESS needs to be found differently, and the following version of Andy‘s script works for me in that Linux distribution:

#!/bin/bash
PATH=/usr/bin:/bin
export $(dbus-launch)
who|grep -E "\(:[0-9](\.[0-9])*\)"|awk '{print $1$5}'|sort -u > /tmp/xusers
while read XUSER; do
    NAME=(${XUSER/(/ })
    DISPLAY=${NAME[1]/)/}
    sudo -u ${NAME[0]} DISPLAY=${DISPLAY} \
                       ${DBUS_SESSION_BUS_ADDRESS} \
                       PATH=${PATH} \
                       notify-send "$@"
done < /tmp/xusers

And here is my tidied-up version:

#!/bin/bash
export $(dbus-launch)
who | awk '{print $1, $NF}' | tr -d "()" | sort -u |
while read XUSER DISPNUM; do
    sudo -u $XUSER DISPLAY=$DISPNUM \
                   $BUS_SESSION_BUS_ADDRESS \
                   notify-send "$@"
done

notify-send-all will be of academic interest to users of single-user systems, but it’s nice to know such a thing is possible relatively easily in Linux.

Powerline adapters and IPv6

My home network includes a number of devices connected via Powerline (HomePlug) adapters. Back in 2015 I blogged about ‘crosstalk’ between my and my neighbour’s home networks, both of which use Powerline adapters (see my post ‘Waiting for 192.168.1.254…’ (Why I could not access a home hub’s management page)), which I was able to resolve by changing the encryption key so that it is different to the default key used by my neighbour. Since then the Powerline adapters have worked well. However, an unrelated network problem recently highlighted another problem with my Powerline adapters…

In November last year there was an external fault with the broadband service to my house, so I had to contact my ISP (the company BT) to fix the problem. BT does not use highly-skilled field personnel to diagnose broadband problems; they tend to use a ‘shotgun’ approach to problem solving. Their first attempt was to replace my router, a BT Home Hub 5, which I knew was actually working perfectly. I was not going to argue, though, because they replaced the router with the newest model, a BT Smart Hub 2. Unlike the Home Hub 5, the Smart Hub 2 fully supports IPv6. BT’s broadband network has supported IPv6 for several years (see ISPreview – UPDATE All BT Broadband Lines Now Support IPv6 Internet Addresses) so I was expecting the computers on my home network to be assigned IPv6 addresses, but ‘ifconfig‘ and ‘ip address‘ showed they were not being assigned IPv6 addresses when connected via the Powerline adapters, only when connected to the Smart Hub 2 via Wi-Fi.

All my computers have IPv6 enabled:

$ sudo sysctl -a | grep disable_ipv6
[sudo] password for fitzcarraldo: 
sysctl: net.ipv6.conf.all.disable_ipv6 = 0
reading key "net.ipv6.conf.all.stable_secret"
sysctl: reading key "net.ipv6.conf.default.stable_secret"
net.ipv6.conf.default.disable_ipv6 = 0
sysctl: reading key "net.ipv6.conf.eno1.stable_secret"
net.ipv6.conf.eno1.disable_ipv6 = 0
sysctl: reading key "net.ipv6.conf.lo.stable_secret"
net.ipv6.conf.lo.disable_ipv6 = 0
sysctl: net.ipv6.conf.wlp2s0.disable_ipv6 = 0
reading key "net.ipv6.conf.wlp2s0.stable_secret"
$ test -f /proc/net/if_inet6 && echo "IPv6 supported" || echo "IPv6 not supported"
IPv6 supported

The fact that the computers on the home network were allocated an IPv6 address when connected to the Smart Hub 2 via Wi-Fi, and that WhatIsMyIPAddress.com confirmed the BT broadband public network was also allocating an IPv6 address, made me suspect the problem of no IPv6 via the wired network was due to the Powerline adapters.

As more machines were added to my home network over the years, I had to buy more Powerline adapters. In 2014 I bought some NETGEAR XAVB5221 (500 Mbps) Powerline adapters to supplement the superseded model NETGEAR XAVB1301 (200 Mbps) Powerline adapters I bought in 2012. Powerline adapters conforming to the HomePlug AV standard work together, so these had no problem communicating. A schematic diagram of my home network is shown below. To keep things simple, only some of the devices are shown. As you can in the diagram, a NETGEAR XAVB1301 adapter was used to connect the BT Smart Hub 2 to the network; some of the computers were connected via NETGEAR XAVB5221 adapters, and others via NETGEAR XAVB1301 adapters.

Simplified schematic diagram of my original home network

I could find no mention of IPv6 for its Powerline adapters in NETGEAR’s documentation and on the NETGEAR Web site. The NETGEAR user manual for the XAV1301 is dated ‘September 2011’ and it lists, under SPECIFICATIONS, compliance with IEEE 802.3 and IEEE 802.3u. The data sheet (no user manual available) for the XAVB5221 is dated ‘2014’ and it lists, under SPECIFICATIONS, compliance with IEEE 1901 and IEEE 802.3.

The Wikipedia page for IEEE 1901-2010 mentions IPv6, so support for IPv6 is relevant to the protocol:

“An IETF RFC Draft address the higher layers of the protocol, namely the specifics of passing IPv6 packets over the PHY and MAC layers of PLC [power-line communication] systems like IEEE 1901.”

I think the following draft Internet Engineering Task Force (IETF) document must be the latest version of the IETF Draft mentioned on the above-mentioned Wikipedia page for IEEE 1901:

Transmission of IPv6 Packets over PLC Networks

Anyway, all this lead me to wonder if the NETGEAR XAVB1301 does not fully comply with IEEE 1901 and does not support IPv6. So I decided to try connecting the BT Smart Hub 2 to the network via a NETGEAR XAVB5221 adapter instead of the older model XAVB1301, as shown in the schematic diagram below.

Simplified schematic diagram of my latest home network

What I then found was that any computer connected to the network via a NETGEAR XAVB5221 adapter was assigned an IPv6 address in addition to an IPv4 address, and WhatIsMyIPAddress.com showed public IPv6 and IPv4 addresses in a Web browser on the device. However, any computer connected to the network via a NETGEAR XAVB1301 adapter was assigned an IPv6 address in addition to an IPv4 address but WhatIsMyIPAddress.com displayed ‘IPv6 not detected’ in a Web browser. So it transpired that NETGEAR XAVB5221 adapters can handle IPv6 but the older XAVB1301 model cannot.

Although not essential, I toyed with the idea of replacing the older NETGEAR XAVB1301 adapters with XAVB5221 adapters, but that model is no longer on sale. The latest available Powerline adapter model from NETGEAR for wired networking is the PL1000 (1000 Mbps). However, its documentation does not mention IPv6 or IEEE 1901, and the following question on the Amazon UK Web site about IPv6 support for the PL1000, and NETGEAR’s answer on 5 May 2020 makes it clear that the PL1000 does not support IPv6:

Question: Does this model support ipv6? netgear xav1301 adapters only support ipv4. my router & pcs support ipv6 but can’t use ipv6 with my xav1301 adapters.

Answer: Thank you for your interest in the NETGEAR PL1000.

The PL1000 supports IPv4.

If you have any questions, you can also check out our NETGEAR Community at any time.

Best regards,
NETGEAR Amazon UK

Unlike NETGEAR, the TP-Link Web site makes it clear that all TP-Link Powerline adapters currently on sale support IPv6:

Most frequently asked questions about TP-Link powerline devices – Part3: Other questions about Powerline Device

Q3.12: Can TP-Link Powerline devices transfer IPv6 packets?

A: Yes, all the on sale TP-Link powerline devices can transfer IPv6 packets. Kindly note this is supported by default and does not require any configuration, our powerline products do not have setting entries for IPv6 either.

I also asked someone I know who uses TP-Link Powerline adapters and a BT Smart Hub 2, and he confirmed that the TP-Link adapters can handle IPv6.

Therefore, the bottom line is: if you want to use Powerline adapters and IPv6, avoid buying NETGEAR Powerline adapters and look at other manufacturers’ adapters instead. I have only investigated TP-Link’s adapters, which do support IPv6. A number of other companies also manufacture Powerline adapters, but you would need to check if they support IPv6; if necessary contact the manufacturer to be sure.

Jitsi Meet, my favourite video conferencing platform (and a way to share audio when using it in Linux)

During the current COVID-19 lockdown I have been using video conferencing platforms a lot for family virtual meet-ups, quizzes and multi-player games by Jackbox Games. Zoom seems to be the most popular video conference platform at the moment, although several articles in the media have pointed out some of its security limitations (see, e.g., ‘‘Zoom is malware’: why experts worry about the video conferencing platform‘). Although many people like Zoom, my favourite video conferencing platform is Jitsi Meet.

For an excellent third-party video introduction to Jitsi Meet, watch the video: ‘Using Jitsi: A free, no-registration video conferencing site‘. WIRED Magazine’s recent article on Jitsi Meet is also worth reading: ‘Want to Ditch Zoom? Jitsi Offers an Open-Source Alternative‘.

The reasons I prefer Jitsi Meet to Zoom include the following:

  1. no subscriptions are required to use all the features of Jitsi Meet;
  2. unlike Zoom, Jitsi Meet does not require you to sign up;
  3. unlike Zoom, Jitsi Meet does not require the installation of an application — it runs in Google Chrome or Firefox;
  4. unlike the free version of Zoom, Jitsi Meet does not impose a time limit on the length of the meeting;
  5. unlike the free version of Zoom, Jitsi Meet does not have a limit on the number of meeting attendees;
  6. Jitsi Meet provides end-to-end encryption for one-to-one video calls*;
  7. I find the performance of Jitsi Meet better than Zoom, which seems to be corroborated in basic benchmarking by Jitsi Meet’s developers (‘WebRTC vs. Zoom – A Simple Congestion Test‘);
  8. I find image quality better in Jitsi Meet;
  9. I find Jitsi Meet on a desktop/laptop more intuitive and easier to use than Zoom;
  10. if I share audio in Zoom for Linux, the audio is very distorted**;
  11. I find the UI of the Jitsi Meet app for Android easy to use (the app can be installed via Google Play);
  12. Jitsi Meet is open-source, so anyone can inspect the source code;
  13. if I wanted to, I could download the Jitsi software to my own server and set up a Jitsi Meet server to handle meetings instead of using the Cloud server provided by 8×8, Inc. (the company that develops the Jitsi Meetings software).

* Neither platform currently provides end-to-end encryption for group meetings, although the developers of Jitsi Meet are apparently working on implementing end-to-end encryption for group meetings using a new feature of Google Chrome called ‘Insertable Streams’.

** There is a work-around for this problem in Zoom for Linux; see my answer to the Unix & Linux Stack Exchange question ‘Play audio output as input to Zoom’. In the case of Jitsi Meet in Linux, PulseAudio Volume Control can be used to share audio, as I explain further down.

Jitsi Meet requires no installation; it runs in a browser window. Either Google Chrome or Firefox can be used, although I find it runs better in Google Chrome. Actually, an Ubuntu 16.04 user told me that Firefox hangs when he tries to join a Jitsi Meet meeting, but Jitsi Meet works fine in Firefox in my two Gentoo Linux installations and in my family’s Lubuntu 18.04 installation. When using Google Chrome, to be able to share your screen you need to install the Google Chrome extension ‘Jitsi Meetings’ by meet.ji.si in the Google Chrome Web Store.

One of my family here at home has a laptop running Windows 10. Google Chrome, but not Firefox, displays a ‘Share audio’ tick box when the ‘Share your screen’ icon is clicked (see ‘Jitsi Meet features update, April 2020‘). The ‘Share audio’ feature is needed when, for example, you are casting via the Internet to remote players a multi-user game running on your machine. During the current COVID-19 lockdown we have been having fun playing Jackbox Games Party Pack 6 this way with family and friends in different locations (see ‘(My Solution) Best method for Virtual Couch Multiplayer‘). Each household connects a laptop to their TV via HDMI and joins the Jitsi Meet meeting. The Jackbox Games games are cast via Jitsi Meet from the laptop at my house, and the group of players in each household can view and hear the game on their TV and participate using their mobile phones as per the Jackbox Games paradigm.

Jitsi Meet provides a ‘Share audio’ function in Windows only, but I found a work-around to to be able to share any application’s audio in Linux if I ever want to use my Linux machines to cast games by Jackbox Games or other suppliers. For once, I have found PulseAudio useful! I use PulseAudio Volume Control to redirect the audio output from the desired application (be it a game, music player, video player or whatever) to the microphone input. The precise way to do this depends on the audio hardware your machine has, but an example is given in the blog post: ‘Redirect Audio Out to Mic In (Linux)‘.

My family’s desktop machine running Lubuntu 18.04 uses a Webcam with an integral microphone connected via USB, and external powered speakers connected to the machine’s Line Out green-coloured 3.5 mm jack socket. The contents of the tabs in PulseAudio Volume Control when no applications that produce audio are running are shown in the following screenshots:

PulseAudio Volume Control - Configuration

PulseAudio Volume Control - Playback

PulseAudio Volume Control - Recording

PulseAudio Volume Control - Output Devices

PulseAudio Volume Control - Input Devices

I make sure ‘All Streams’ is selected on the ‘Playback’ and ‘Recording’ tabs, ‘All Output Devices’ is selected on the ‘Output Devices’ tab, and ‘All Input Devices’ is selected on the ‘Input Devices’ tab.

Let us say I have launched Audacious to play some music and I want to cast that music to members of a Jitsi Meet meeting. When I am using Jitsi Meet for a meeting in Google Chrome, the contents of the PulseAudio Volume Control tabs on this machine are as follows:

PulseAudio Volume Control - Playback

PulseAudio Volume Control - Recording

PulseAudio Volume Control - Output Devices

PulseAudio Volume Control - Input Devices

To redirect the audio from e.g. Audacious to the meeting members, I select (click on the square button with the green disc and white tick) ‘Monitor of Built-in Audio Analogue Stereo’ on the ‘Input Devices’ tab, and on the ‘Recording’ tab I click on Chrome input: RecordStream from ‘Camera Analogue Mono’ and select Chrome input: RecordStream from ‘Monitor of Built-in Audio Analogue Stereo’, as shown below.

PulseAudio Volume Control - Recording

PulseAudio Volume Control - Input Devices

With the above settings in PulseAudio volume control, all the members of the meeting will be able to hear clearly the audio from Audacious. To switch back to my microphone to speak, I simply click on Chrome input: RecordStream from ‘Monitor of Built-in Audio Analogue Stereo’ on the ‘Recording’ tab and select Chrome input: RecordStream from ‘Camera Analogue Mono’ again.