Firewall zones (profiles) in Linux, and how to switch them automatically if you use UFW

Firstly, a note on terminology: UFW (Uncomplicated Firewall) and its two GUI front-ends Gufw and UFW Frontends use the term ‘application profile’ to refer to a pre-configured set of rules specified in a file. Files containing UFW application profiles are placed in the directory /etc/ufw/applications.d/. An application profile for SMB, for example, enables the root user to use the UFW command ‘ufw allow Samba‘ (‘ufw allow CIFS‘ in Gentoo Linux) rather than having to enter UFW commands specifying the precise ports and network protocols that SMB uses. However, this blog post is not about UFW’s application profiles; it is about what Gufw calls ‘profiles’ and firewalld calls ‘zones’.

In essence a profile/zone is a collection of firewall policies and rules. Both Gufw and firewalld include the concept of a ‘zone’, although Gufw uses the term ‘profile’ rather than ‘zone’. UFW Frontends does not have the concept of a ‘zone’; rules entered via UFW Frontends apply to any network to which you connect your laptop. The ability to define different zones for different networks is handy. For example, you can have certain policies and rules when your laptop is connected to your home network, and different policies and rules when your laptop is connected to the network in a café, hotel, airport or other public place.

An attractive feature of firewalld when used in conjunction with NetworkManager and KDE Plasma is that it is possible to use the desktop environment’s network management module (‘System Settings’ > ‘Connections’) to specify a particular firewalld zone for a particular network connection. For example, let’s say you used firewalld to specify certain policies and rules for a zone you named ‘office’, and you then specified in the System Settings – Connections GUI that a connection named ‘ACM’ should use the zone ‘office’. Thereafter, whenever you connect your laptop to the network named ‘ACM’, firewalld will use the policies and rules you previously configured for the zone ‘office’.

Unlike firewalld, Gufw does not have the ability to switch profiles automatically according to which network the laptop is connected. You have to select manually the profile you wish to use. You would launch Gufw prior to connecting to, for example, your office’s network, select the profile ‘Office’ (or whatever you have named it), then connect your laptop to that network.

I think many people would be satisfied with the functionality currently provided by Gufw. I could use the Gufw GUI to create Gufw profiles with names such as ‘Home’, ‘HomeDave’, ‘Public’, ‘HQoffice’, ‘USoffice’, ‘PestanaRio’ and so on, and specify the different policies and rules I want for each profile. At home I would launch Gufw on my laptop and select the Home profile then connect to my home network; in the office at work I would launch Gufw on my laptop and select the HQoffice profile then connect to the office network; at my friend Dave’s house I would launch Gufw on my laptop and select the HomeDave profile then connect to the house network; and so on. Nevertheless I do see the attraction of automated zone switching, as provided by firewalld in conjunction with NetworkManager and KDE. It would be handy if my laptop could switch automatically to the Home profile when my laptop connected to the network at my home with the name ‘BTHub5-8EUQ’, automatically switch to the HQoffice profile when my laptop connected to the network named ‘HQ-Office2’ in the office, and so on.

I use UFW on my two laptops running Gentoo Linux. The package ufw-frontends is also installed but normally I use UFW directly via the command line. However I wanted to learn about zones/profiles while using UFW, and I also wanted to see if I could automate the switching of zones without resorting to installing firewalld. NetworkManager has the ability to launch ‘hook’ scripts when certain things happen — when a network connection changes, for example — and this seemed to me to be a way of switching profiles automatically.

I had not used Gufw before, so I decided to install it. A package is available in many Linux distributions but there is no ebuild for Gufw in Gentoo’s main Portage tree and I could not find an up-to-date ebuild for it in any Portage overlays. Therefore I created the ebuild for net-firewall/gufw-19.10.0 shown below. It probably needs improving, but it does install a working Gufw in Gentoo Linux.

# Copyright 1999-2019 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=7
PYTHON_COMPAT=( python3_{5,6,7} )
DISTUTILS_IN_SOURCE_BUILD=1

inherit distutils-r1

MY_PN="gui-ufw"
MY_PV="$(ver_cut 1-2)"

DESCRIPTION="GUI frontend for managing ufw."
HOMEPAGE="https://gufw.org/"
SRC_URI="https://launchpad.net/${MY_PN}/trunk/${MY_PV}/+download/${MY_PN}-${PV}.tar.gz"

LICENSE="GPL-3"
SLOT="0"
KEYWORDS="~amd64"
IUSE=""

DEPEND="dev-python/python-distutils-extra"
RDEPEND="net-firewall/ufw
	dev-python/netifaces
	dev-python/pygobject:3
	net-libs/webkit-gtk[introspection]
	sys-auth/elogind
	sys-auth/polkit
	x11-libs/gtk+:3[introspection]
	x11-themes/gnome-icon-theme-symbolic
"
S=${WORKDIR}/${MY_PN}-${PV}

pkg_postinst() {
	sed '/dist-packages/d' -i /usr/bin/gufw-pkexec
	sed -E '/\/share\//d' -i /usr/bin/gufw-pkexec
	local PYTHONVERSION="$(python -c 'import sys; print("{}.{}".format(sys.version_info.major, sys.version_info.minor))')"
	sed -E "s|python3\.[0-9]|python${PYTHONVERSION}|g" -i /usr/bin/gufw-pkexec
	sed -E 's|\/lib\/|\/lib64\/|g' -i /usr/bin/gufw-pkexec
}

How To Set Up a Firewall with GUFW on Linux‘ is a good tutorial on Gufw.

As I had not used Gufw previously, I had to play around with it to understand better its functional design. I found that if I configure rules directly via UFW on the command line without using Gufw, Gufw does not allow me to edit those rules (but does allow me to delete them) and those rules exist whichever Gufw profile is selected in the Gufw GUI. Gufw profiles are stored in files named ‘/etc/gufw/*.profile‘ (e.g. /etc/gufw/Home.profile) and these files will not include UFW rules entered via the command line. On the other hand, UFW rules created via the Gufw GUI apply solely to the currently-selected Gufw profile, which is what I would have expected. In other words, I can create a different set of policies and rules in each Gufw profile. Therefore I believe Gufw profiles (as distinct from UFW application profiles) are basically analogous to firewalld’s zones. It also appears to me that Gufw maintains configuration files specifying policies and rules independently of UFW, which Gufw then applies to UFW. In other words, if you are a Gufw user you should not use UFW directly to configure policies and rules, otherwise Gufw’s configuration files will not include what you did directly using UFW. To reiterate, use only Gufw or only UFW, not both.

The current Gufw profile’s name is listed in the file /etc/gufw/gufw.cfg. For example, I currently have the Home profile selected in the Gufw GUI, and the file gufw.cfg contains the following:

[GufwConfiguration]
profile = Home
windowwidth = 542
windowheight = 530
confirmdetelerule = yes

If I examine the contents of the file /etc/gufw/Home.profle I see that it contains the UFW policies and rules I specified for the Gufw Home profile:

[fwBasic]
status = enabled
incoming = deny
outgoing = allow
routed = disabled

[Rule0]
ufw_rule = 137,138/udp ALLOW IN 192.168.1.0/24
description = Samba
command = /usr/sbin/ufw allow in proto udp from 192.168.1.0/24 to any port 137,138
policy = allow
direction = in
protocol = 
from_ip = 192.168.1.0/24
from_port = 
to_ip = 
to_port = 137,138/udp
iface = 
routed = 
logging = 

[Rule1]
ufw_rule = 139,445/tcp ALLOW IN 192.168.1.0/24
description = Samba
command = /usr/sbin/ufw allow in proto tcp from 192.168.1.0/24 to any port 139,445
policy = allow
direction = in
protocol = 
from_ip = 192.168.1.0/24
from_port = 
to_ip = 
to_port = 139,445/tcp
iface = 
routed = 
logging =

I also notice that the other Gufw profiles can differ. For example, my Office.profile file contains the following:

[fwBasic]
status = enabled
incoming = deny
outgoing = allow
routed = allow

The profile name listed in gufw.cfg gets changed when the user changes the profile using the Gufw GUI. It appears to me that only at the point in time when the user selects a certain Gufw profile in the Gufw GUI does Gufw parse the applicable *.profile file and issue commands to UFW to implement the policies and rules specified in the *.profile file.

Initially I tried to automate the process of changing the Gufw profile by doing the following:

  1. I created a NetworkManager Dispatcher hook script to:

    1. detect when the laptop connects to a network;

    2. determine whether the network is at my home, at my workplace or in a public place (café, airport or wherever) by looking at the connection name;

    3. edit gufw.cfg to change the name of the Gufw profile according to the network connected.
  2. I configured KDE to launch Gufw automatically at login, hoping that would implement the Gufw profile specified in gufw.cfg.

When I connected the laptop to various networks, Gufw did indeed show the name of the profile selected by the NetworkManager Dispatcher hook script, but the associated Gufw profile’s rules had not been applied. They were only applied if I clicked on the ‘Profile’ pull-down menu in Gufw, selected a different Gufw profile, then re-selected the desired Gufw profile. Therefore driving Gufw from a NetworkManager Dispatcher hook script is not possible. This is a pity, as Gufw is an easy way to manage UFW from a GUI; it allows the user to create, delete and edit zones (Gufw profiles) and to select them manually. What Gufw doesn’t do is enable the user to associate those zones with connection names, nor trigger specific zone automatically based on the selected network connection. firewalld, on the other hand, does enable the user to do both those things.

As my attempt at automating the switching of zones in Gufw had failed, I decided to create a NetworkManager Dispatcher hook script to switch zones automatically by using UFW commands. Initially I though about creating a bespoke UFW application profile for each zone and allowing/denying those in the script, but it is actually easier to use the fundamental UFW commands in the script, especially as UFW commands are relatively easy to understand. Also, this approach means everything is in a single file, which facilitates configuration. I can simply edit the script in order to: a) add or delete a zone; b) change a zone’s name; c) change policies and rules for a zone; d) add or delete a connection; e) change the name of a connection; f) change the zone a connection uses. Granted, editing a script is not as user-friendly as using the firewalld GUI to configure a zone and then using KDE Plasma’s system settings module Connections to specify that zone for a specific connection, but my script is not particularly difficult to understand and edit. And by using such a script I can continue to use UFW rather than installing firewalld and having to learn how to use it.

My NetworkManager Dispatcher hook script /etc/NetworkManager/dispatcher.d/20_ufw-zones is listed below. In the main body of the script I define the zone I wish to use for each connection, and in the function select_zone I define the policies and rules I want each zone to use.

#!/bin/bash
INTERFACE=$1
STATUS=$2
WIRED=enp4s0f1
WIFI=wlp3s0

CT_helper_rule() {
    echo "# The following is needed to enable Samba commands to" >> /etc/ufw/before.rules
    echo "# work properly for broadcast NetBIOS name resolution" >> /etc/ufw/before.rules
    echo "#"  >> /etc/ufw/before.rules
    echo "# raw table rules" >> /etc/ufw/before.rules
    echo "*raw" >> /etc/ufw/before.rules
    echo ":OUTPUT ACCEPT [0:0]" >> /etc/ufw/before.rules
    echo "-F OUTPUT" >> /etc/ufw/before.rules
    echo "-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns" >> /etc/ufw/before.rules
    echo "COMMIT" >> /etc/ufw/before.rules
}
 
select_zone() {
    ufw --force reset
    ufw --force enable
    ZONE=$1
    case "$ZONE" in
    'Home')
        ufw default deny incoming
        ufw default allow outgoing
        #
        # Rules for SMB
        ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
        ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp
        CT_helper_rule
        #
        # Rules for KDEConnect
        ufw allow from 192.168.1.0/24 to any port 1714:1764 proto udp
        ufw allow from 192.168.1.0/24 to any port 1714:1764 proto tcp
    ;;
    'Office')
        ufw default deny incoming
        ufw default allow outgoing
    ;;
    'Public')
        ufw default reject incoming
        ufw default allow outgoing
    ;;
    'JohnsHouse')
        ufw default deny incoming
        ufw default allow outgoing
        #
        # Rules for SMB
        ufw allow from 192.168.42.0/24 to any port 137,138 proto udp
        ufw allow from 192.168.42.0/24 to any port 139,445 proto tcp
        CT_helper_rule
        #
        # Rules for KDEConnect
        ufw allow from 192.168.42.0/24 to any port 1714:1764 proto udp
        ufw allow from 192.168.42.0/24 to any port 1714:1764 proto tcp
    ;;
    esac
    ufw --force reload
    rm /etc/ufw/*.rules.20* # Delete backups of *.rules files ufw makes every time it is reset
    echo -n `date +"[%F %T %Z]"` >> /var/log/ufw-zones.log
    echo " Zone $ZONE selected for connection $ACTIVE on interface $INTERFACE." >> /var/log/ufw-zones.log
}
 
# Check if either the wired or wireless interface is up
if [ "$INTERFACE" = "$WIRED" -o "$INTERFACE" = "$WIFI" ] && [ "$STATUS" = "up" ]; then
 
    # Check if a single connection is active
    if [ `nmcli c | grep -v "\-\-" | grep -v "NAME.*UUID.*TYPE.*DEVICE" | wc -l` -eq 1 ]; then
 
        # Ascertain the name of the active connection
        ACTIVE=`nmcli c | grep -v "\-\-" | grep -v "NAME.*UUID.*TYPE.*DEVICE" | awk -F' ' '{print $1}'`
 
        case "$ACTIVE" in
 
        'eth0')
            ZONE="Home"
        ;;
        'POR1-wired')
            ZONE="Office"
        ;;
        'BTHub5-8EUQ')
            ZONE="Home"
        ;;
        'BTHub5-8EUQ-5GHz')
            ZONE="Home"
        ;;
        'John1')
            ZONE="JohnsHouse"
        ;;
        'GRAND MERCURE')
            ZONE="Public"
        ;;
        *)
            # If connection name is not in above list
            ZONE="Public"
        ;;

        esac

        select_zone $ZONE
        exit $?

    fi
fi

The log file that the script uses contains a chronological record of the connections made and the zones selected:

$ cat /var/log/ufw-zones.log 
[2019-09-30 20:13:52 BST] Zone Home selected for connection eth0 on interface enp4s0f1.
[2019-10-01 22:59:18 BST] Zone Home selected for connection BTHub5-8EUQ-5GHz on interface wlp3s0.
[2019-10-02 17:59:23 EDT] Zone Public selected for connection loganwifi on interface wlp3s0.
[2019-10-03 10:12:46 EDT] Zone Office selected for connection POR1-wired on interface enp4s0f1.

Prevent Linux firewalls interfering with Samba commands in a home network that uses broadcast NetBIOS name resolution

Or “How come devices in a home network can browse SMB shares but Linux Samba commands and Windows nbtstat commands do not work properly?”

Introduction

In a previous post I explained how it is possible to browse SMB shares when using broadcast NetBIOS name resolution in a home network consisting of machines running Linux, Windows and other operating systems. Browsing SMB/Samba shares will work as expected, but Samba commands such as ‘smbtree‘, ‘smbclient‘ and ‘nmblookup‘ will not work properly if the Linux machines use a firewall that has not been configured for broadcast NetBIOS name resolution. This post is to explain how to do that.

If broadcast NetBIOS name resolution is being used and none of the Linux machines has a firewall enabled, or if their firewalls have been correctly configured, the output of e.g. the ‘smbtree‘ command on one of those machines would look something like the example below.

anne@akhanaten:~$ smbtree
Enter anne's password: 
HOME
        \\AKHANATEN                     Samba 4.3.11-Ubuntu
                \\AKHANATEN\IPC$                IPC Service (Samba 4.3.11-Ubuntu)
                \\AKHANATEN\guest               guest account
                \\AKHANATEN\matthew             matthew share
                \\AKHANATEN\marilla             marilla share
                \\AKHANATEN\anne                anne share
        \\TUTANKHAMUN                   Samba 4.5.10
                \\TUTANKHAMUN\Samsung_Xpress_C460FW     Samsung Xpress C460FW
                \\TUTANKHAMUN\Canon_MP560_Printer       Canon PIXMA MP560
                \\TUTANKHAMUN\Canon_MP510_Printer       Canon PIXMA MP510
                \\TUTANKHAMUN\Virtual_PDF_Printer       Virtual PDF Printer
                \\TUTANKHAMUN\IPC$              IPC Service (Samba 4.2.11)
                \\TUTANKHAMUN\Public
                \\TUTANKHAMUN\anne-share
                \\TUTANKHAMUN\print$
                \\TUTANKHAMUN\netlogon          Network Logon Service
        \\BTHUB5                        BT Home Hub 5.0A File Server
                \\BTHUB5\IPC$                   IPC Service (BT Home Hub 5.0A File Server)
        \\THUTMOSEIII                   Windows 10 computer

If Linux firewalls have not been correctly configured, the output would be missing some information about other machines in the network. For example, compare the output above with the output below from the same network, this time with the Linux firewalls configured using typical rules for Samba specified in Web articles, blog posts and forums.

anne@akhanaten:~$ smbtree
Enter anne's password: 
HOME
        \\AKHANATEN                     Samba 4.3.11-Ubuntu
                \\AKHANATEN\IPC$                IPC Service (Samba 4.3.11-Ubuntu)
                \\AKHANATEN\guest               guest account
                \\AKHANATEN\matthew             matthew share
                \\AKHANATEN\marilla             marilla share
                \\AKHANATEN\anne                anne share
        \\TUTANKHAMUN                   Samba 4.5.10
        \\BTHUB5                        BT Home Hub 5.0A File Server
        \\THUTMOSEIII                   Windows 10 computer

To avoid this problem you need to add a further Linux firewall rule to the set of rules usually used for Samba. Below I first list the usual firewall rules for Samba, then I give the additional rule necessary if using broadcast NetBIOS name resolution. In each case I give the applicable rules for a pure IPTABLES firewall and for UFW (Uncomplicated Firewall). The rules listed here assume the IP address range of the home network is 192.168.1.0/24, so change the range to suit the specific network.

Firewall rules typically specified for machines using Samba

IPTABLES

The rules listed below assume the machine uses interface eth0, so change the interface to suit the specific machine.

# NetBIOS Name Service (name resolution)
iptables -A INPUT -i eth0 -p udp --dport 137 -s 192.168.1.0/24 -j ACCEPT

# NetBIOS Datagram Service (BROWSER service)
iptables -A INPUT -i eth0 -p udp --dport 138 -s 192.168.1.0/24 -j ACCEPT

# NetBIOS Session Service (data transfer legacy SMB/NetBIOS/TCP)
iptables -A INPUT -i eth0 -p tcp --dport 139 -s 192.168.1.0/24 -j ACCEPT

# Microsoft Directory Service (data transfer SMB/TCP)
iptables -A INPUT -i eth0 -p tcp --dport 445 -s 192.168.1.0/24 -j ACCEPT

UFW

In some Linux distributions the ufw application allows a single command to add Samba support, such as:

user $ sudo ufw allow Samba

or

user $ sudo ufw allow CIFS

These ‘application profiles’ are specified in files in the directory /etc/ufw/applications.d/, so you could add application profiles or modify existing ones if you wish. In one of my installations the file /etc/ufw/applications.d/ufw-fileserver includes the following application profile for Samba, for example:

[CIFS]
title=SMB/CIFS server
description=SMB/CIFS server
ports=137,138/udp|139,445/tcp

If such an application profile does not exist in your installation, typical Samba rules can be added in UFW using the following two commands:

user $ sudo ufw allow from 192.168.1.0/24 to any port 137,138 proto udp
user $ sudo ufw allow from 192.168.1.0/24 to any port 139,445 proto tcp

The correct addition of the rules can be checked using the following command:

user $ sudo ufw status verbose
Password:
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
137,138/udp (CIFS)         ALLOW IN    192.168.1.0/24
139,445/tcp (CIFS)         ALLOW IN    192.168.1.0/24

The extra rule required when using broadcast NetBIOS name resolution

The reason why an extra rule is required when using broadcast NetBIOS name resolution is because UFW (which is based on IPTABLES) is ‘stateful’, as is a purely IPTABLES firewall (unless explicitly configured not to be stateful). The firewall does not consider packets it receives in response to its broadcast to be ESTABLISHED or RELATED, and therefore drops those packets. So, despite the IPTABLES and UFW rules listed above including a rule to accept incoming UDP packets on Port 137, any UDP packets received on Port 137 that do not constitute a one-to-one, two-way communication flow are dropped by the firewall. The extra rule below overrules this and makes the firewall accept packets coming from other devices’ Port 137 in response to broadcast NetBIOS Name Service packets. To do this, the extra rule uses a CT (Connection Tracking) helper named ‘netbios-ns‘ (obviously meaning ‘NetBIOS Name Service’). In order to use this rule the kernel must have been configured to use the IPTABLES ‘raw‘ table and to use CT (see the section ‘Kernel configuration’ further on).

IPTABLES

# All NetBIOS clients must have the netbios-ns helper enabled for broadcast name resolution to work
iptables -t raw -A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns

By the way, in addition to flushing the usual tables, flush the ‘raw‘ table too when you restart the firewall:

iptables -t raw -F OUTPUT

UFW

Add the following lines to the end of the file /etc/ufw/before.rules

# The following is needed to enable Samba commands to
# work properly for broadcast NetBIOS name resolution
#
# raw table rules
*raw
:OUTPUT ACCEPT [0:0]
-F OUTPUT
-A OUTPUT -p udp -m udp --dport 137 -j CT --helper netbios-ns
COMMIT

Note that the output of the command ‘ufw status verbose‘ will not include the above rule. This is not a bug.

Kernel configuration

If you are using a binary-based distribution such as Ubuntu Linux, the kernel will probably have been configured to include the needed modules (CONFIG_IP_NF_RAW=m, CONFIG_IP6_NF_RAW=m and CONFIG_NETFILTER_XT_TARGET_CT=m), and the installation configured to load the modules automatically. However, if you are using a source-based distribution such as Gentoo Linux make sure the kernel configuration includes these three options before you build the kernel, and also add the module names ‘iptable_raw‘ and ‘xt_CT‘ to the module list in the file /etc/conf.d/modules as shown in the example below, so that the modules are loaded at boot:

modules="r8169 nvidia agpgart fuse bnep rfcomm hidp uvcvideo cifs mmc_block rtsx_pci snd-seq-midi vboxdrv vboxnetadp vboxnetflt iptable_raw xt_CT"

You can use the following two commands to check if the two modules are loaded:

user $ sudo lsmod | grep iptable_raw
user $ sudo lsmod | grep xt_CT

How to check the additional rule is active

You can use the command below whether you are using pure IPTABLES or UFW.

user $ sudo iptables -nvL -t raw
Password: 
Chain PREROUTING (policy ACCEPT 2613 packets, 1115K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 2773 packets, 475K bytes)
 pkts bytes target     prot opt in     out     source               destination         
   16  1248 CT         udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:137 CT helper netbios-ns

The packet and byte counts will increase whenever you use a Samba command.

Bibliography

  1. The netfilter.org "iptables" project
  2. Iptables Tutorial
  3. Introduction to IPTables
  4. Gentoo Wiki : iptables
  5. Arch Linux Wiki : Samba : "Browsing" network fails with "Failed to retrieve share list from server"
  6. Ubuntu : Manpage : ufw-framework
  7. Gentoo Wiki : UFW

Editing from a Linux PC the phone book (contacts list) in an Android phone

A mobile phone stores contacts in the phone itself and/or on the SIM. Over the years, the phone book on my SIM became cluttered with duplicate entries, inconsistently-named contacts, and so on. I decided recently to tidy up the phone book, and looked for a way to do it from Linux.

Now, the last time I tidied up my phone book was back in 2003 when my SIM was in a Sony Ericsson T68i. These days I’m using the same SIM in an Android smartphone: the HTC Desire. Back then I used a shareware application called Mobile Navigator on a laptop running Windows XP. So, naturally, I thought it would be possible to do something similar from a PC running Linux.

An indirect method of editing a phone book in an Android phone is recommended in many Web forums. The advice is to synchronise the phone’s phone book with GMail on your PC, edit the contacts in GMail on the PC and then the contacts will be updated in the phone when the phone next synchronises with GMail. That is all very well, but I don’t want to import my phone contacts into GMail and thus have them stored in ‘The Cloud’, plus I don’t see why I should be forced to do that just to make it easier for me to tidy a phone book in my phone. So I set about searching for an application to do the job.

My search turned up three applications that appeared to enable a user to edit the phone book (phone and SIM):

1. Gammu (CLI) and Wammu (GUI front-end), FOSS with versions for both Linux and Windows.

2. MOBILedit!, a closed-source commercial product for Windows.

3. MyPhoneExplorer (Version 1.8.1), closed-source ‘donationware’ for Windows, recommended in various forums and that has been used with WINE in Linux. Not only does MyPhoneExplorer allow you to edit the phone book, it also provides a host of other features such as: viewing the phone’s call logs, messages, hardware status, files and applications; sending messages via the PC; etc. It’s a nifty application.

Web browsing told me that Gammu/Wammu does not yet work with the HTC Desire (see the bug report Don’t get any connection to HTC Desire via bluetooth in Wammu/Gammu). This is a pity, as Wammu and Gammu look well-designed, and Wammu has a professional feel to it.

The price for MOBILedit! is not extortionate but I wanted to find a cost-free solution if possible, and, more importantly, I could find no evidence on the Web of MOBILedit! having been used with WINE in Linux to edit the phone books in the HTC Desire.

The MyPhoneExplorer forums include a thread on using it with WINE, so MyPhoneExplorer looked like my only option (although it is a pity it is not FOSS). I installed MyPhoneExplorer with its own WINEPREFIX but, despite following the instructions in the MyPhoneExplorer forums, I was unable to get MyPhoneExplorer to connect to the HTC Desire via a USB cable in Linux. This may be because, despite what I have read on the android developers Web site (see Using Hardware Devices and OEM USB Drivers), as is the case with Windows it is necessary in Linux to install ADB USB drivers (the ADB drivers can be installed in Linux by installing the Android SDK). However, I was able to connect MyPhoneExplorer to my HTC Desire via WiFi at home (but not in a hotel, as connection depends on the router and internal network IP address space), and via Bluetooth (anywhere).

How to install MyPhoneExplorer in WINE

1. Use the Android Market to install MyPhoneExplorer Client on the phone.

2. Surf over to the MyPhoneExplorer Home Page and download MyPhoneExplorer_Setup_1.8.1.exe to ~/Desktop/

3. Open a Konsole/Terminal window and perform all the following steps in the same window.

4. Install MyPhoneExplorer on your PC under WINE:

$ cd
$ export WINEPREFIX=$HOME/.wine-myphoneexplorer
$ export WINEARCH="win32"
$ winecfg
$ wget "http://download.microsoft.com/download/5/a/d/5ad868a0-8ecd-4bb0-a882-fe53eb7ef348/VB6.0-KB290887-X86.exe"
$ wine VB6.0-KB290887-X86.exe
$ cd ./.wine-myphoneexplorer/drive_c/
$ cp $HOME/Desktop/MyPhoneExplorer_Setup_1.8.1.exe .
$ mv ${WINEPREFIX}/drive_c/windows/system32/oleaut32.dll ${WINEPREFIX}/drive_c/windows/system32/oleaut32-alt.dll
$ mv ${WINEPREFIX}/drive_c/windows/system32/olepro32.dll ${WINEPREFIX}/drive_c/windows/system32/olepro32-alt.dll
$ echo -e REGEDIT4\\n\\n[HKEY_CURRENT_USER\\Software\\Wine\\DllOverrides]\\n\"asycfilt\"=\"native\"\\n\"comcat\"=\"native\"\\n\"msvbvm60\"=\"native\"\\n\"oleaut32\"=\"native\"\\n\"oleaut32\"=\"native\"\\n\"olepro32\"=\"native\"\\n\"stdole2.tlb\"=\"native\"\\n[HKEY_CURRENT_USER\\Software\\MyPhoneExplorer]\\n\"Language\"=\"English.lng\" > temp.reg
$ wine regedit temp.reg
$ wget http://www.kegel.com/wine/winetricks
$ chmod +x winetricks
$ ./winetricks # Install Visual Basic 6 SP6 (I had to do this as well as wine VB6.0-KB290887-X86.exe)
$ ./winetricks # Install msxml3
$ wine MyPhoneExplorer_Setup_1.8.1.exe # Install MyPhoneExplorer

(Make sure you select the WINEPREFIX first when using winetricks)

5. Edit your Desktop Environment’s menu to use the following command for launching MyPhoneExplorer:

env WINEPREFIX="/home/fitzcaraldo/.wine-myphoneexplorer" WINEARCH="win32" wine C:\\windows\\command\\start.exe /Unix /home/fitzcarraldo/.wine-myphoneexplorer/dosdevices/c:/users/Public/Start\ Menu/Programs/MyPhoneExplorer/MyPhoneExplorer.lnk

(Of course replace “fitzcarraldo” with your user name.)

How to connect via Bluetooth

1. Look up the phone’s Bluetooth MAC address in the phone itself, write it down and keep it to hand. (I use the MAC address 11:22:33:AA:BB:C1 in the examples below.)

2. Enable Blutooth on your PC and phone, make them both discoverable, and pair them.

3. Launch MyPhoneExplorer Client on the phone and wait a few seconds for it to load and run.

4. Now check which channel is used by MyPhoneExplorer Client (In the example below it is Channel 17):

$ sudo sdptool browse
Inquiring ...
Browsing 11:22:33:AA:BB:C1 ...
Service RecHandle: 0x10000
Service Class ID List:
"PnP Information" (0x1200)

Service Name: Headset Gateway
Service RecHandle: 0x10001
Service Class ID List:
"Headset Audio Gateway" (0x1112)
"Generic Audio" (0x1203)
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 1
Profile Descriptor List:
"Headset" (0x1108)
Version: 0x0100

Service Name: Handsfree Gateway
Service RecHandle: 0x10002
Service Class ID List:
"Handsfree Audio Gateway" (0x111f)
"Generic Audio" (0x1203)
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 2
Profile Descriptor List:
"Handsfree" (0x111e)
Version: 0x0105

Service Name: Object Push
Service RecHandle: 0x10003
Service Class ID List:
"OBEX Object Push" (0x1105)
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 3
"OBEX" (0x0008)
Profile Descriptor List:
"OBEX Object Push" (0x1105)
Version: 0x0100

Service RecHandle: 0x10004
Service Class ID List:
"AV Remote Target" (0x110c)
Protocol Descriptor List:
"L2CAP" (0x0100)
PSM: 23
"AVCTP" (0x0017)
uint16: 0x100
Profile Descriptor List:
"AV Remote" (0x110e)
Version: 0x0100

Service Name: BRCM Advanced Audio
Service RecHandle: 0x10005
Service Class ID List:
"Audio Source" (0x110a)
Protocol Descriptor List:
"L2CAP" (0x0100)
PSM: 25
"AVDTP" (0x0019)
uint16: 0x102
Profile Descriptor List:
"Advanced Audio" (0x110d)
Version: 0x0102

Service Name: Phonebook Access PSE
Service RecHandle: 0x10006
Service Class ID List:
"Phonebook Access - PSE" (0x112f)
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 4
"OBEX" (0x0008)
Profile Descriptor List:
"Phonebook Access" (0x1130)
Version: 0x0100

Service Name: OBEX File Transfer
Service RecHandle: 0x10007
Service Class ID List:
"OBEX File Transfer" (0x1106)
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 5
"OBEX" (0x0008)
Profile Descriptor List:
"OBEX File Transfer" (0x1106)
Version: 0x0100

Service Name: MyPhoneExplorer
Service RecHandle: 0x10008
Service Class ID List:
UUID 128: 00001101-0000-1000-7000-00304b7f34de
Protocol Descriptor List:
"L2CAP" (0x0100)
"RFCOMM" (0x0003)
Channel: 17

5. Bind a virtual serial device to the phone (You can look up the phone’s Bluetooth MAC address in the phone itself to double check):

$ sudo rfcomm bind 1 11:22:33:AA:BB:C1 17

N.B. If you have been tinkering already with binding, and the virtual device /dev/rfcomm1 is therefore already bound, you will get an error message saying “Can't create device: Address already in use“. You can release the virtual device by using the command:

$ sudo rfcomm release /dev/rfcomm1

6. Create a symbolic link for the virtual serial link:

$ export WINEPREFIX=$HOME/.wine-myphoneexplorer
$ sudo ln -is /dev/rfcomm1 ${WINEPREFIX}/dosdevices/com2
$ sudo chmod 777 ${WINEPREFIX}/dosdevices/com2 # you may not need to do this, as the permissions may already be 777.

7. Launch MyPhoneExplorer (e.g. in KDE this would be: Kickoff > Applications > Wine> Programs > MyPhoneExplorer > MyPhoneExplorer), then press F2 (or select File > Settings).

8. Select ‘Connection’ in the left pane and then select ‘Phone type’ to be ‘SonyEricsson phone with modeminterface’. Tick ‘Stable connection (some features are deactivated)’.

9. Select ‘Connection’ in the left pane and then select ‘Phone type’ to be ‘Phone with Google Android-OS’.

10. Select ‘Connection’ in the left pane and then select ‘Connect via…’ to be ‘Bluetooth’ and enter “COM2” (upper case, and without the quotes) and click on ‘OK’.

11. Press F1 (or select File > Connect) and your PC should be able to connect to the phone and synchronise with your phone’s phone book (Extras > Start Multi-sync), thus enabling you to edit the contents and re-synchronise to update the phone book in the phone.

Script to make subsequent connections via Bluetooth easier

Note that the channel used by MyPhoneExplorer Client will change the next time you want to connect, so there is no point editing /etc/bluetooth/rfcomm.conf (the Linux configuration file in which you would normally specify the channel). Therefore, to make life a little easier, I wrote a simple Bash script to automate as much as possible the process of reconnecting via Bluetooth in future. If you want to do the same, use your favourite text editor and save the following script with the file name myphoneexplorer.sh:

#!/bin/bash
echo -n "Please switch on Bluetooth via System Tray icon, make Bluetooth discoverable then press ENTER: "
read RESPONSE
echo -n "Please enable Bluetooth on the phone, make it discoverable then press ENTER: "
read RESPONSE
export WINEPREFIX="/home/fitzcarraldo/.wine-myphoneexplorer"
export WINEARCH="win32"
echo "Checking current rfcomm status..."
sudo rfcomm -a
echo "Trying to release rfcomm1 in case it is in use (no message means rfcomm1 is released and is now OK to use,"
echo "and the message Cannot release device: No such device also means rfcomm1 is OK to use)..."
sudo rfcomm release /dev/rfcomm1
echo -n "Now please launch MyPhoneExplorer Client on the phone, wait a few seconds then press ENTER: "
read RESPONSE
echo "Find channel of MyPhoneExplorer Client:"
sudo sdptool browse
echo -n "Enter the channel number: "
read RESPONSE
sudo rfcomm bind 1 11:22:33:AA:BB:C1 $RESPONSE
echo "Checking current rfcomm status..."
sudo rfcomm -a
echo "If all is OK then please launch MyPhoneExplorer from Kickoff > Applications > Wine > Programs > MyPhoneExplorer"
echo "and press F1 to connect to the phone."

(Of course replace “fitzcarraldo” with your user name, and replace “11:22:33:AA:BB:C1” with the Bluetooth MAC address of your phone.)

Then make the script executable:

$ chmod +x myphoneexplorer.sh

The next time you want to use MyPhoneExplorer, open a Konsole/Terminal, execute the script:

$ sh myphoneexplorer.sh

and follow the instructions it displays in the Konsole/Terminal window.

How to connect via WiFi

Connecting MyPhoneExplorer to the phone via WiFi is easier than via Bluetooth. Note that WiFi connection will not work with a public WiFi network, just a home WiFi network.

1. Enable WiFi on your PC and phone.

2. Launch MyPhoneExplorer Client on the phone and wait a few seconds for it to load and run.

3. Launch MyPhoneExplorer (e.g. in KDE this would be: Kickoff > Applications > Wine> Programs > MyPhoneExplorer > MyPhoneExplorer), then press F2 (or select File > Settings).

4. Select ‘Connection’ in the left pane and then select ‘Phone type’ to be ‘SonyEricsson phone with modeminterface’. Tick ‘Stable connection (some features are deactivated)’.

5. Select ‘Connection’ in the left pane and then select ‘Phone type’ to be ‘Phone with Google Android-OS’.

6. Select ‘Connection’ in the left pane and then select ‘Connect via…’ to be ‘WiFi’ and click on ‘OK’.

7. Press F1 (or select File > Connect) and your PC should be able to connect to the phone and synchronise with your phone’s phone book (Extras > Start Multi-sync), thus enabling you to edit the contents and re-synchronise to update the phone book in the phone.

Useful references

The MyPhoneExplorer forums are oriented to Windows as it is a Windows application, but there is a thread for Linux users: HowTo: Use MyPhoneExplorer under GNU/Linux (English Version)

The thread for Windows users of MyPhoneExplorer may also be of some use: Howto and FAQ: Use Android Phones with MyPhoneExplorer

Limitations in WINE

The only limitation I have found using MyPhoneExplorer 1.8.1 in WINE is that it crashes if I try to cut and paste when editing a phone book entry. Additionally, some people have reported that the buttons on the button bar do not work when you are connected to the phone; they usually do work for me (apparently it depends on whether you have installed msxml3) but sometimes don’t. In any case, if they don’t then you can use the pull-down menus from the menu bar instead.

Other than that, MyPhoneExplorer works well in WINE and enables me to edit the phone book in my Android phone without involving Big Brother! It’s a nice application.

Firewall rules if using WiFi

If you’re using MyPhoneExplorer to connect with the phone via WiFi instead of Bluetooth, and your PC has a firewall enabled, according to a post on 7 June 2011 in the MyPhoneExplorer forums you have to configure the firewall with the following rules:

TCP port 80 (outgoing) – Check for update
TCP port 5210 (outgoing) – Communication with client
UDP port 5211 (outgoing) – Communication with client
UDP port 5212 (inbound) – Broadcast from the client

Below I list the rules I implemented in Uncomplicated Firewall:

# ufw status
Status: active

To              Action          From

--              ------          ----
Anywhere        ALLOW           5212/udp

80/tcp          ALLOW OUT       Anywhere
5210/tcp        ALLOW OUT       Anywhere
5211/udp        ALLOW OUT       Anywhere

Alternatively you could just disable the firewall on your PC while you’re using MyPhoneExplorer with your phone, which I do anyway when I’m at home as my home network is protected by the firewall in my router.

EDIT (October 25, 2012): Now there is another way of editing an Android phone’s contacts list from Linux: AirDroid. And you don’t need to mess around with WINE to use it. For details see my post AirDroid, a handy Android app for managing your phone from Linux.

Why can’t I access a specific Web site?

Is there one specific Web site that you can never access? That always causes your Web browser to stall when you click on a link to that site or enter the site’s URL in the browser’s address bar? Perhaps it’s an Internet banking site? You don’t have trouble with other sites, just this one particular site or perhaps just a couple of sites?

The chances are that this problem is caused by what is called a ‘black hole’ router somewhere between you and that site. For the technical explanation of a black hole router, see e.g. the article Black Hole (networking).

I experienced this precise problem a couple of days ago with the popular Web site IMDb. The old Gateway Solo 9300 laptop I was using has no built-in networking hardware so I use a couple of CardBus cards for wireless and wired network access. The Linksys WPC54G (EU) v7.1 wireless notebook adapter works fine but, irrespective of the Web browser, I could not access the IMDb Web site; the browser always stalled when trying to load the site. I discovered I also had the same problem with my bank’s Web site.

I could ping other sites by IP address and by domain name, but pinging the IMDb site never received a reply (and it still doesn’t, even after the fix explained below which allows browsing of the site).

I also experienced this problem several years ago under Windows XP when trying to access the Amazon UK Web site, using an MTU that I had incorrectly set to a value smaller than possible. The problem this time was due to a correctly-set MTU. In both cases, the two MTUs are smaller than a router somewhere en route to those sites wanted to handle.

BACKGROUND

Bear with me while I explain how the problem arose, as it’s relevant (and useful) information.

I had just installed Sabayon Linux on the laptop and got the wireless network connection up, and the first thing I then did was to launch Firefox. With an MTU of 1500 for wlan0, Firefox would only load Google (and that, only intermittently). All other sites would stall. So I used the following command iteratively in order to find the MTU for this hardware, which is 1464 (the maximum packet size in bytes that does not fragment + 28 bytes):

# ping -M do -s packet_size_in_bytes www.cisco.com

e.g.

# ping -M do -s 1472 www.cisco.com

I just varied the packet size in the ping command until “Frag needed and DF set” was not displayed.

I can reduce the MTU to 1464 as follows:

# ifconfig wlan0 mtu 1464

I could make this permanent by adding an entry (mtu_wlan0=”1464″) to the file /etc/conf.d/net, but I chose instead to make it permanent by adding the above ifconfig command to the file /etc/conf.d/local. This (correct) MTU works fine for all sites I have visited so far except for two, which just stall the browser: IMDb and my bank’s Web site.

I also have a Belkin F5D5010 CardBus Network Card (wired Ethernet) for the laptop. It works fine with an MTU of 1500, and there is no problem accessing the IMDb Web site in Firefox. So, apparently, the MTU is the problem when using the wireless network connection. Now let’s look at the solution…

THE SOLUTION

Well, in the case of Windows it is possible to edit the Registry to solve the problem of network ‘black holes’. But what do you do in the case of Linux? It turns out that the kernel /proc file system provides an easy way to enable and disable TCP MTU Probing by changing a value in the ‘file’ /proc/sys/net/ipv4/tcp_mtu_probing. A value of 0 = disabled; 1 = enabled when a black hole router is detected; 2 = always enabled.

This is what my laptop had in that ‘file’:

# cat /proc/sys/net/ipv4/tcp_mtu_probing
0

So all I needed to do was:

# echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

and, bingo, a browser can access IMDb and my bank’s Web site without a problem. To make it permanent, I just added the above command to the file /etc/conf.d/local so that it is executed every time I boot the laptop. Here is what my file /etc/conf.d/local looks like now:

# Here is where you can put anything you need to start
# that there is not an init script for.

local_start() {
# This is a good place to load any misc programs
# on startup (use &>/dev/null to hide output)

# START OF MY ADDITIONS
# The Linksys CardBus card does not work with a MTU greater than 1464:
ifconfig wlan0 mtu 1464
# Enable TCP MTU Probing in order to deal with black hole routers:
echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing
# END OF MY ADDITIONS

# We should always return 0
return 0
}

local_stop() {
# This is a good place to unload any misc.
# programs you started above.

# We should always return 0
return 0
}

UPDATE (May 13, 2011): OpenRC 0.8.0 and later in Gentoo and Sabayon Linux no longer use the file /etc/conf.d/local but instead require the commands to be in files in the directory /etc/local.d/. So I created a file /etc/local.d/01network.start containing the following lines:

# The Linksys CardBus card does not work with MTU greater than 1464:
ifconfig wlan0 mtu 1464
# Enable TCP MTU probing to deal with black hole routers:
echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

and made it executable:

# chmod +x /etc/local.d/01network.start

UPDATE (April 11, 2012): If “echo 2” does not solve the problem, try “echo 1” instead. The possible values are:

0   Do not perform PLPMTUD (Packetization Layer Path MTU Discovery)
1   Perform PLPMTUD only after detecting a ‘blackhole’.
2   Always perform PLPMTUD.