Investigating an OpenVPN connection problem in Gentoo Linux and Lubuntu 22.04 but not in Android 13

Back in 2020 I successfully used OpenVPN to access a remote server from my laptop and from my smartphone. Fast forward to 2023 and I needed to use OpenVPN again to access the same remote server. I could again access the remote server when connecting from my smartphone, now running Android 13, using the OpenVPN client app. However, when I tried to use OpenVPN from my laptop running Gentoo Linux, and from my desktop machine running Lubuntu 22.04, VPN connection attempts always timed out. The OpenVPN client configuration file (referred to generically as ‘client.ovpn‘ in some articles, named ‘bsf1.ovpn‘ in my case) and the OpenVPN password were both correct, so I did not understand why a connection could not be established from the laptop and desktop machine to the remote server, especially as there was not a problem connecting from my smartphone running Android 13.

I use KDE in Gentoo Linux, and Lubuntu 22.04 uses LXQt. In both those Desktop Environments I used the relevant GUI front-end to NetworkManager to import the client.ovpn file and to try to connect to the VPN server. As these connection attempts had failed, I launched the OpenVPN client from the command line in Gentoo Linux to see if there were any error messages, and the output is shown below. (For privacy reasons I have annonimised my public IP address, domain, e-mail address, country, location and organisation in all the command line output shown in this post.)

root # openvpn --config bsf1.ovpn
2023-09-30 01:07:02 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
2023-09-30 01:07:02 DEPRECATED OPTION: --cipher set to 'AES-128-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305). OpenVPN ignores --cipher for cipher negotiations. 
2023-09-30 01:07:02 OpenVPN 2.6.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
2023-09-30 01:07:02 library versions: OpenSSL 3.0.10 1 Aug 2023, LZO 2.10
2023-09-30 01:07:02 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
Enter Private Key Password:
2023-09-30 01:08:57 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2023-09-30 01:08:58 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:08:58 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:08:58 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:08:58 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:08:58 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:08:58 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:08:58 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2023-09-30 01:08:58 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=fc150984 3a9d3b4f
2023-09-30 01:08:58 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:08:58 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:08:58 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:08:58 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:08:58 TLS Error: TLS handshake failed
2023-09-30 01:08:58 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:08:58 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:08:58 Restart pause, 1 second(s)
2023-09-30 01:08:59 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:08:59 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:08:59 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:08:59 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:08:59 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:08:59 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:08:59 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:08:59 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=00c193e4 f375c3df
2023-09-30 01:08:59 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:08:59 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:08:59 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:08:59 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:08:59 TLS Error: TLS handshake failed
2023-09-30 01:08:59 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:08:59 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:08:59 Restart pause, 1 second(s)
2023-09-30 01:09:00 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:00 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:00 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:00 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:00 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:00 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:00 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:00 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=415fddcd dc5faf65
2023-09-30 01:09:01 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:01 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:01 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:01 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:01 TLS Error: TLS handshake failed
2023-09-30 01:09:01 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:01 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:01 Restart pause, 1 second(s)
2023-09-30 01:09:02 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:02 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:02 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:02 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:02 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:02 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:02 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:02 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=1458e510 821d0c37
2023-09-30 01:09:02 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:02 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:02 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:02 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:02 TLS Error: TLS handshake failed
2023-09-30 01:09:02 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:02 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:02 Restart pause, 1 second(s)
2023-09-30 01:09:03 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:03 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:03 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:03 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:03 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:03 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:03 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:03 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=9ab3df76 86d3e81d
2023-09-30 01:09:03 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:03 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:03 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:03 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:03 TLS Error: TLS handshake failed
2023-09-30 01:09:03 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:03 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:03 Restart pause, 2 second(s)
2023-09-30 01:09:05 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:05 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:05 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:05 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:05 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:05 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:05 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:05 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=9f21e4f5 a4d9fecb
2023-09-30 01:09:05 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:05 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:05 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:05 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:05 TLS Error: TLS handshake failed
2023-09-30 01:09:05 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:05 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:05 Restart pause, 4 second(s)
2023-09-30 01:09:09 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:10 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:10 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:10 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:10 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:10 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:10 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:10 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=bb6d7166 52844953
2023-09-30 01:09:10 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:10 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:10 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:10 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:10 TLS Error: TLS handshake failed
2023-09-30 01:09:10 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:10 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:10 Restart pause, 8 second(s)
2023-09-30 01:09:18 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:18 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:18 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:18 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:19 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:19 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:19 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:19 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=099e3b11 8ddc87c1
2023-09-30 01:09:19 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:19 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:19 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:19 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:19 TLS Error: TLS handshake failed
2023-09-30 01:09:19 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:19 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:19 Restart pause, 16 second(s)
2023-09-30 01:09:35 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-09-30 01:09:35 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:35 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 01:09:35 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:35 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 01:09:35 TCPv4_CLIENT link local: (not bound)
2023-09-30 01:09:35 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 01:09:35 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=38ffac21 50e59cc4
2023-09-30 01:09:35 TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled. This can be caused by mismatched tls-version-min and tls-version-max options on client and server. If your OpenVPN client is between v2.3.6 and v2.3.2 try adding tls-version-min 1.0 to the client configuration to use TLS 1.0+ instead of TLS 1.0 only
2023-09-30 01:09:35 OpenSSL: error:0A000102:SSL routines::unsupported protocol
2023-09-30 01:09:35 TLS_ERROR: BIO read tls_read_plaintext error
2023-09-30 01:09:35 TLS Error: TLS object -> incoming plaintext read error
2023-09-30 01:09:35 TLS Error: TLS handshake failed
2023-09-30 01:09:35 Fatal TLS error (check_tls_errors_co), restarting
2023-09-30 01:09:35 SIGUSR1[soft,tls-error] received, process restarting
2023-09-30 01:09:35 Restart pause, 32 second(s)
^C

The error message ‘TLS error: Unsupported protocol. This typically indicates that client and server have no common TLS version enabled.‘ made me suspect that the version of the OpenSSL package (which also provides TLS) in Gentoo Linux on my laptop is incompatible with the version of the OpenSSL package on the remote server.

The command line output in Lubuntu 22.04 on the desktop machine also showed there was a problem with TLS when I tried to launch the VPN client:

user $ sudo openvpn --config bsf1.ovpn
[sudo] password for fitzcarraldo: 
2023-10-03 14:00:43 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
2023-10-03 14:00:43 DEPRECATED OPTION: --cipher set to 'AES-128-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM). Future OpenVPN version will ignore --cipher for cipher negotiations. Add 'AES-128-CBC' to --data-ciphers or change --cipher 'AES-128-CBC' to --data-ciphers-fallback 'AES-128-CBC' to silence this warning.
2023-10-03 14:00:43 OpenVPN 2.5.5 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 14 2022
2023-10-03 14:00:43 library versions: OpenSSL 3.0.2 15 Mar 2022, LZO 2.10
2023-10-03 14:00:43 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
🔐 Enter Private Key Password: ************            
2023-10-03 14:00:48 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2023-10-03 14:00:48 Outgoing Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:00:48 Incoming Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:00:48 RESOLVE: Cannot resolve host address: bsfitzgerald.ddns.net:1194 (Temporary failure in name resolution)
2023-10-03 14:00:48 RESOLVE: Cannot resolve host address: bsfitzgerald.ddns.net:1194 (Temporary failure in name resolution)
2023-10-03 14:00:48 Could not determine IPv4/IPv6 protocol
2023-10-03 14:00:48 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2023-10-03 14:00:48 SIGUSR1[soft,init_instance] received, process restarting
2023-10-03 14:00:48 Restart pause, 5 second(s)
2023-10-03 14:00:53 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-10-03 14:00:53 Outgoing Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:00:53 Incoming Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:00:55 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-10-03 14:00:55 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-10-03 14:00:55 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194 [nonblock]
2023-10-03 14:00:55 TCP connection established with [AF_INET]111.222.333.444:1194
2023-10-03 14:00:55 TCP_CLIENT link local: (not bound)
2023-10-03 14:00:55 TCP_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-10-03 14:00:56 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=c3ec9433 c4139af0
2023-10-03 14:00:57 VERIFY OK: depth=1, C=PE, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-10-03 14:00:57 VERIFY OK: depth=0, C=PE, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-10-03 14:00:58 OpenSSL: error:0A0C0103:SSL routines::internal error
2023-10-03 14:00:58 TLS_ERROR: BIO read tls_read_plaintext error
2023-10-03 14:00:58 TLS Error: TLS object -> incoming plaintext read error
2023-10-03 14:00:58 TLS Error: TLS handshake failed
2023-10-03 14:00:58 Fatal TLS error (check_tls_errors_co), restarting
2023-10-03 14:00:58 SIGUSR1[soft,tls-error] received, process restarting
2023-10-03 14:00:58 Restart pause, 5 second(s)
2023-10-03 14:01:03 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-10-03 14:01:03 Outgoing Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:01:03 Incoming Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:01:03 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-10-03 14:01:03 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-10-03 14:01:03 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194 [nonblock]
2023-10-03 14:01:03 TCP connection established with [AF_INET]111.222.333.444:1194
2023-10-03 14:01:03 TCP_CLIENT link local: (not bound)
2023-10-03 14:01:03 TCP_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-10-03 14:01:03 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=486e45aa bbf5d0b0
2023-10-03 14:01:05 VERIFY OK: depth=1, C=GB, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-10-03 14:01:05 VERIFY OK: depth=0, C=GB, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-10-03 14:01:05 OpenSSL: error:0A0C0103:SSL routines::internal error
2023-10-03 14:01:05 TLS_ERROR: BIO read tls_read_plaintext error
2023-10-03 14:01:05 TLS Error: TLS object -> incoming plaintext read error
2023-10-03 14:01:05 TLS Error: TLS handshake failed
2023-10-03 14:01:05 Fatal TLS error (check_tls_errors_co), restarting
2023-10-03 14:01:05 SIGUSR1[soft,tls-error] received, process restarting
2023-10-03 14:01:05 Restart pause, 5 second(s)
2023-10-03 14:01:10 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
2023-10-03 14:01:10 Outgoing Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:01:10 Incoming Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
2023-10-03 14:01:10 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-10-03 14:01:10 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-10-03 14:01:10 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194 [nonblock]
2023-10-03 14:01:10 TCP connection established with [AF_INET]111.222.333.444:1194
2023-10-03 14:01:10 TCP_CLIENT link local: (not bound)
2023-10-03 14:01:10 TCP_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-10-03 14:01:11 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=2523066e a7169e23
2023-10-03 14:01:12 VERIFY OK: depth=1, C=GB, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-10-03 14:01:12 VERIFY OK: depth=0, C=GB, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-10-03 14:01:13 OpenSSL: error:0A0C0103:SSL routines::internal error
2023-10-03 14:01:13 TLS_ERROR: BIO read tls_read_plaintext error
2023-10-03 14:01:13 TLS Error: TLS object -> incoming plaintext read error
2023-10-03 14:01:13 TLS Error: TLS handshake failed
2023-10-03 14:01:13 Fatal TLS error (check_tls_errors_co), restarting
2023-10-03 14:01:13 SIGUSR1[soft,tls-error] received, process restarting
2023-10-03 14:01:13 Restart pause, 5 second(s)
^C

The version of OpenSSL currently running on the remote server is as follows:

user $ openssl version
OpenSSL 1.0.2l  25 May 2017

whereas the version of OpenSSL currently running in Gentoo Linux on the laptop is:

user $ openssl version
OpenSSL 3.0.10 1 Aug 2023 (Library: OpenSSL 3.0.10 1 Aug 2023)

and the version of OpenSSL currently running in Lubuntu 22.04 on the desktop machine is:

user $ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)

The last time I successfully used the OpenVPN client on the laptop was in 2020, when the version of OpenSSL on the laptop was 1.1.1g, so my suspicion grew that the root cause of the problem is a version difference in OpenSSL between the client machine and the remote server.

I searched the Web and found OpenVPN bug report #348: ‘2.6 client cannot connect to old 2.2 and 2.1 server‘. Therefore I tried adding the following three lines to the client.ovpn file (bsf1.ovpn) in Gentoo Linux:

tls-cert-profile insecure
providers legacy default
compat-mode 1.1.1

Now the laptop can connect to the remote VPN server via the command line in a terminal window, as shown in the output below:

root # openvpn --config bsf1.ovpn
2023-09-30 03:13:26 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
2023-09-30 03:13:26 OpenVPN 2.6.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
2023-09-30 03:13:26 library versions: OpenSSL 3.0.10 1 Aug 2023, LZO 2.10
2023-09-30 03:13:26 WARNING: No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mitm for more info.
Enter Private Key Password:
2023-09-30 03:13:33 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2023-09-30 03:13:33 TCP/UDP: Preserving recently used remote address: [AF_INET]111.222.333.444:1194
2023-09-30 03:13:33 Socket Buffers: R=[131072->131072] S=[16384->16384]
2023-09-30 03:13:33 Attempting to establish TCP connection with [AF_INET]111.222.333.444:1194
2023-09-30 03:13:33 TCP connection established with [AF_INET]111.222.333.444:1194
2023-09-30 03:13:33 TCPv4_CLIENT link local: (not bound)
2023-09-30 03:13:33 TCPv4_CLIENT link remote: [AF_INET]111.222.333.444:1194
2023-09-30 03:13:33 NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
2023-09-30 03:13:33 TLS: Initial packet from [AF_INET]111.222.333.444:1194, sid=4103e161 8493e221
2023-09-30 03:13:34 VERIFY OK: depth=1, C=PE, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-09-30 03:13:34 VERIFY OK: depth=0, C=PE, ST=Loreto, L=Iquitos, O=Work, OU=vpn, CN=server, name=server, emailAddress=brian@bsfitzgerald.com
2023-09-30 03:13:35 Control Channel: TLSv1, cipher SSLv3 DHE-RSA-AES256-SHA, peer certificate: 2048 bit RSA, signature: RSA-SHA256
2023-09-30 03:13:35 [server] Peer Connection Initiated with [AF_INET]111.222.333.444:1194
2023-09-30 03:13:35 TLS: move_session: dest=TM_ACTIVE src=TM_INITIAL reinit_src=1
2023-09-30 03:13:35 TLS: tls_multi_process: initial untrusted session promoted to trusted
2023-09-30 03:13:36 SENT CONTROL [server]: 'PUSH_REQUEST' (status=1)
2023-09-30 03:13:36 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-options DNS 8.8.8.8,route-gateway 10.10.1.1,topology subnet,ping 10,ping-restart 120,ifconfig 10.10.1.4 255.255.255.0'
2023-09-30 03:13:36 Options error: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:2: dhcp-options (2.6.4)
2023-09-30 03:13:36 OPTIONS IMPORT: --ifconfig/up options modified
2023-09-30 03:13:36 OPTIONS IMPORT: route options modified
2023-09-30 03:13:36 OPTIONS IMPORT: route-related options modified
2023-09-30 03:13:36 Using peer cipher 'AES-128-CBC'
2023-09-30 03:13:36 net_route_v4_best_gw query: dst 0.0.0.0
2023-09-30 03:13:36 net_route_v4_best_gw result: via 192.168.96.245 dev wlp3s0
2023-09-30 03:13:36 ROUTE_GATEWAY 192.168.96.245/255.255.255.0 IFACE=wlp3s0 HWADDR=d8:fc:93:c5:f4:6d
2023-09-30 03:13:36 TUN/TAP device tun0 opened
2023-09-30 03:13:36 net_iface_mtu_set: mtu 1500 for tun0
2023-09-30 03:13:36 net_iface_up: set tun0 up
2023-09-30 03:13:36 net_addr_v4_add: 10.10.1.4/24 dev tun0
2023-09-30 03:13:36 net_route_v4_add: 111.222.333.444/32 via 192.168.96.245 dev [NULL] table 0 metric -1
2023-09-30 03:13:36 net_route_v4_add: 0.0.0.0/1 via 10.10.1.1 dev [NULL] table 0 metric -1
2023-09-30 03:13:36 net_route_v4_add: 128.0.0.0/1 via 10.10.1.1 dev [NULL] table 0 metric -1
2023-09-30 03:13:36 UID set to nobody
2023-09-30 03:13:36 GID set to nogroup
2023-09-30 03:13:36 Capabilities retained: CAP_NET_ADMIN
2023-09-30 03:13:36 Initialization Sequence Completed
2023-09-30 03:13:36 Data Channel: cipher 'AES-128-CBC', auth 'SHA256', compression: 'lzo'
2023-09-30 03:13:36 Timers: ping 10, ping-restart 12
[...]

and I can access the SMB shares on the remote server by entering ‘smb://192.168.0.10‘ in the Dolphin file manager’s window in KDE on the laptop.

However, if I terminate the openvpn client on the command line and then import the modified bsf1.ovpn file into NetworkManager using plasma-nm (the KDE GUI front-end to NetworkManager), connection attempts to the VPN still time out. The created NetworkManager connection file /etc/NetworkManager/system-connections/bsf1.nmconnection does not contain the settings in the additional three lines that I added to the client.ovpn file, as apparently they are meaningless to NetworkManager. When I use the plasma-nm GUI to import the client.ovpn file I assume plasma-nm or NetworkManager simply ignore the additional three lines when they parse the configuration file.

In Lubuntu 22.04 I edited bsf1.ovpn to add the three lines:

tls-cert-profile insecure
providers legacy default
compat-mode 1.1.1

and launched the openvpn client from the command line, but OpenVPN objected to the compat-mode line:

user $ sudo openvpn --config wjh1.ovpn
Options error: Unrecognized option or missing or extra parameter(s) in bsf1.ovpn:93: compat-mode (2.5.5)
Use --help for more information.

For the openvpn client command to execute successfully in Lubuntu 22.04 I had to omit the line ‘compat-mode 1.1.1‘, i.e. the two manually-added lines to bsf1.ovpn in Lubuntu 22.04 are:

tls-cert-profile insecure
providers legacy default

In my situation (i.e. the client is using OpenSSL 3.x.x in Lubuntu 22.04 and the remote server is using OpenSSL 1.x.x) the VPN connection still times out when attempting to connect using the GUI front-end (nm-tray or nm-applet) to NetworkManager in LXQt, so the work-around is still to launch the OpenVPN client from the command line as already explained:

user $ openvpn --conf bsf1.ovpn

and I can access the SMB shares on the remote server by entering ‘smb://192.168.0.10‘ in the PCManFM-Qt file manager’s window in LXQt on the desktop machine.

The remaining question I had was why my Samsung Galaxy Note 20 Ultra 5G running Android 13 can connect to the remote server via OpenVPN. I suspected that Android 13 uses an earlier version of OpenSSL that is compatible with the version on the remote server. A little research online showed me that Android 13 (and a couple of previous versions of Android) uses a Google fork of OpenSSL called BoringSSL. So, to find the version of OpenSSL/BoringSSL in Android 13, I cloned the BoringSSL repository on my laptop:

user $ git clone https://android.googlesource.com/platform/external/boringssl

and then I searched for the environment variable OPENSSL_VERSION_TEXT:

user $ grep -r OPENSSL_VERSION_TEXT boringssl/*
boringssl/include/openssl/crypto.h:// OPENSSL_VERSION_TEXT contains a string the identifies the version of
boringssl/include/openssl/crypto.h:#define OPENSSL_VERSION_TEXT "OpenSSL 1.1.1 (compatible; BoringSSL)"
boringssl/src/include/openssl/crypto.h:// OPENSSL_VERSION_TEXT contains a string the identifies the version of
boringssl/src/include/openssl/crypto.h:#define OPENSSL_VERSION_TEXT "OpenSSL 1.1.1 (compatible; BoringSSL)"
boringssl/src/crypto/crypto_test.cc:// Test that OPENSSL_VERSION_NUMBER and OPENSSL_VERSION_TEXT are consistent.
boringssl/src/crypto/crypto_test.cc:// Node.js parses the version out of OPENSSL_VERSION_TEXT instead of using
boringssl/src/crypto/crypto_test.cc:            std::string(OPENSSL_VERSION_TEXT).substr(0, strlen(expected)));

As I had surmised, OpenSSL (well, the fork BoringSSL) in Android 13 is Version 1.1.1. That is why the original bsf1.ovpn still works with the OpenVPN client on my phone to access the OpenVPN server running on the remote server.

By the way, in order to use OpenVPN in Lubuntu 22.04, you need to install the following three Ubuntu packages:

openvpn
network-manager-openvpn
network-manager-openvpn-gnome (despite Lubuntu using LXQt rather than GNOME)

The NetworkManager GUI front-end nm-tray does not support password asking, so you need to use the NetworkManager GUI front-end nm-applet to connect to the VPN (assuming both client and server have compatible versions of OpenSSL, that is). You can either type ‘nm-applet‘ on the command line or you can switch to using nm-applet instead of nm-tray by deselecting nm-tray in LXQt ‘Application Menu’ > ‘Preferences’ > ‘LXQt Settings’ > ‘Session Settings’ > ‘Autostart’ and adding nm-applet to the Autostart list. See Lubuntu Manual Appendix G Advanced Networking for further details. As already mentioned, in my situation the client is using OpenSSL 3.x.x and the server is using OpenSSL 1.x.x so the VPN connection still times out, therefore the work-around is still to launch openvpn from the command line:

user $ openvpn --conf bsf1.ovpn

To summarise my situation:

On my smartphone running Android 13 I simply use the OpenVPN client app for Android with the original client.ovpn configuration file (named ‘bsf1.ovpn‘ in my case) in order to connect to the VPN server on the remote server running OpenSSL 1.0.2l.

In Gentoo Linux currently running OpenSSL 3.0.10, I have to edit the original client.ovpn configuration file and add three lines, then launch the openvpn client from the command line in a terminal window in order to connect to the VPN server on the remote server running OpenSSL 1.0.2l.

In Lubuntu 22.04 currently running OpenSSL 3.0.2, I have to edit the original client.ovpn configuration file and add two lines, then launch the openvpn client from the command line in a terminal window in order to connect to the VPN server on the remote server running OpenSSL 1.0.2l.

The bottom line is: To avoid problems, check you have the same version of OpenSSL installed in the client and server devices when using OpenVPN.

‘IP configuration was unavailable’: a laptop cannot connect wirelessly to a router

I recently switched my ISP from BT to Virgin Media because the speed and reliability of the broadband connection were low. A Virgin Media Hub 3 was supplied as part of the package, and the TV, laptops (Gentoo Linux, Windows 10 and macOS), desktops (Lubuntu and Windows 10), tablets and phones (Android and iOS) could connect to the Hub 3 without any trouble. A few weeks later Virgin Media offered to upgrade the hub to a Hub 4. I don’t look a gift horse in the mouth, so I accepted the offer. The Hub 4 does indeed improve on the already excellent broadband speeds I was getting with the Hub 3. On the downside the Hub 4’s configuration software has a couple of bugs, but I was able to live with them.

In addition to the above-mentioned hub configuration bugs, one of my laptops (a Compal NBLB2 with Intel Wireless WiFi Link 5300 AGN adapter) running Linux could not connect to the hub via Wi-Fi, even though it had no trouble connecting to the Hub 3. All other devices so far can connect to the Hub 4, so I was scratching my head. The laptop has no trouble connecting to the Hub 4 via Ethernet cable.

The hub’s 5G and 2.4G Wi-Fi bands originally had the same SSID (I’ll call it ‘VM1234567‘ here). I decided to rename the two bands ‘VM1234567_5G‘ and ‘VM1234567_2.4G‘ respectively, via the hub’s Settings in a Web browser. Very occasionally the laptop could connect to either SSID, but usually it could not connect and the following notification would pop up:

Wireless interface (wlan0)
IP configuration was unavailable

I did various things to try to get the laptop to connect, such as:

• changing Wi-Fi channel selection in the hub from Auto to Manual and specifying different channels myself;
• specifying the BSSID in the Desktop Environment’s GUI front-end to NetworkManager;
• explicitly restricting the connection to the specific (and only) Wi-Fi interface (‘wlan0‘, in my case) in the DE’s GUI front-end to NetworkManager;
• disabling IPv6 (Virgin Media does not use IPv6) in the DE’s GUI front-end to NetworkManager;
• disabling the UFW firewall.

None of the above enabled the laptop to connect to the hub via Wi-Fi.

I installed the GUI Wi-Fi scanner LinSSID on my other Linux machines so I could check which 2.4G and 5G Wi-Fi channels were being used by the hub and by my neighbours’ hubs/routers. Note that LinSSID requires the utility iw to be installed and CONFIG_CFG80211_WEXT to be set in the kernel. The NetworkManager command ‘nmcli dev wifi list‘ can also be used to check which channels are being used. The channels selected automatically by the hub looked reasonable to me, and the different channels I selected manually did not improve the situation.

Now, coincidentally that laptop can dual-boot Windows 7, so I booted Windows 7 to see if it could connect to the hub via Wi-Fi. However, Windows 7 had the same Wi-Fi connectivity problem as Linux. The Network and Sharing Centre displayed the error message ‘The default gateway is not available’ and allowed me to run the so-called Troubleshooter, which fixed the problem in Windows 7. The laptop could then connect to the hub and to the Internet via the 5G Wi-Fi band (the hub’s DHCP server allocated IP address 192.168.0.145 to the laptop). So it appeared the lack of a specified default gateway was the problem in both OSs. This surprised me because I had never had to specify a default gateway on my machines, and still do not have to on the other machines. Anyway, I booted back into Linux and did the following:

STEP 1 (on the Compal laptop)

Connected to the hub via an Ethernet cable.

Opened the Hub 4 Settings page (192.168.0.1) in a Web browser.

Selected ‘Advanced settings’ > ‘DHCP’

Added the MAC address of the laptop’s Wi-Fi adapter and the IP address 192.168.0.145 to the Reserved list.

STEP 2 (on the Compal laptop)

Selected ‘System Settings’ > ‘Network’ | ‘Connections’

Selected Wi-Fi connection VM1234567_5G

Entered the following on the ‘IPv4’ tab:

Method: Manual
DNS Servers: 194.168.4.100,194.168.8.100
Search Domains: cable.virginm.net (The laptop connects without this entry, so I’m not sure if it makes any difference.)

Clicked ‘+ Add’ and added the gateway details as follows:

Address
192.168.0.145

Netmask
255.255.255.0

Gateway
192.168.0.1

Ticked ‘IPv4 is required for this connection’.

Set the following on the ‘Wi-Fi’ tab (this is optional):

BSSID: <hub’s MAC address corresponding to the SSID>
Restrict to device: wlan0 (<MAC address of the laptop’s Wi-Fi adapter>)

The BSSID can be found either by using LinSSID on a machine that can access the Wi-Fi network or by using the command ‘nmcli dev wifi list‘ in a terminal window. The MAC address of the laptop’s Wi-Fi adapter can be found using the commands ‘ip link‘ or ‘ifconfig‘.

Clicked on the down arrow in the ‘Restrict to device:’ box and selected the device (wlan0, in my case).

STEP 3 (on the Compal laptop)

Selected ‘System Settings’ > ‘Network’ | ‘Connections’

Selected Wi-Fi connection VM1234567_2.4G

Performed the same configuration steps as for VM1234567_5G except that the SSID V1234567_2.4G has a different BSSID (found using LinSSID or nmcli) to the SSID V1234567_5G.

The laptop’s 5G W-Fi connection now works very well with the Hub 4. The 2.4G connection can be slow (even when the signal is at 100%) and sometimes stalls, so I’m not sure I have fixed that connection completely, or even if it is fixable in this case. I still do not know why the problem occurs with the Hub 4 but not the Hub 3, and why it only happens with one specific machine. Anyway, the 5G connection now works fine, so I’m happy.

croc – another file transfer method

I have lost count of the number of times I have had to send a large file to someone at work, usually in a hurry. I’ve used Dropbox, ownCloud, Firefox Send (no longer available) etc. Transferring large files became a bit easier when e-mail service providers increased the size limit for attachments, but that is still not a solution for very large files. The xkcd cartoon FILE TRANSFER sums up the situation nicely.

I recently discovered the command line utility croc, which the author claims is a way to ‘easily and securely transfer stuff from one computer to another.’ I thought I’d give it a try, if only to have another tool to fall back on in an emergency. It does rely on both ends having croc installed, but hopefully that should not be a show-stopper as croc is available for Linux, Windows, macOS and BSD. To quote the author:

croc differs from a utility like scp because it doesn’t require any two computers to have enabled port-forwarding. Instead, croc will uses a relay – a temporary server setup locally (if both computers are on lan) or publicly (default is at croc4.schollz.com). Any two computers can connect to the relay, and after securing their channel with PAKE [password authenticated key exchange], they can transfer encrypted metadata and data through the relay. The relay works by first having the computers communicate the PAKE protocol via websockets, and then exchanging encrypted metadata, and then stapling the TCP connections directly so that they can transfer directly.

So, to use croc you will be dependent on the public relay provided by the author unless you set up your own relay (instructions are provided in the author’s original 2018 blog post introducing croc – see link above – and in various third-party articles about croc, such as ‘Securely Transfer Files and Folders Between Computers Using Croc‘ and ‘Transfer Files And Folders Between Computers With Croc‘).

Anyway, I installed croc in Lubuntu and Gentoo Linux from the author’s GitHub repository and indeed it is easy to use and works fine. The binary releases for the various OSs and Linux distributions can be found on the Releases page of the GitHub repository or via the OS package manager.

Lubuntu 20.10:

user $ wget https://github.com/schollz/croc/releases/download/v9.1.6/croc_9.1.6_Linux-64bit.deb
user $ sudo dpkg -i croc_9.1.6_Linux-64bit.deb

Gentoo Linux:

root # emerge net-misc/croc

(Note that croc ebuilds are not currently marked as Stable in the Gentoo Linux Portage tree, so you’ll have to unmask them by keyword if you are using the Stable branch.)

Termux:

I even installed croc in Termux on my Samsung Galaxy Note 20 Ultra 5G, and it works in Android too:

$ pkg install croc

Other OSs and other Linux distributions:

See the instructions in the README file online.

Using croc

Using croc is as simple as entering a command on one computer, informing (via e-mail, telephone, SMS, Signal or other social media) the person using the other computer of the command to use, and entering that command on the other computer. For example:

Sender

user $ croc send Documents/flight-times.ods
Sending 'flight-times.ods' (16.6 kB)
Code is: 8878-salary-courage-roger
On the other computer run

croc 8878-salary-courage-roger

Receiver

user $ croc 8878-salary-courage-roger
Accept 'flight-times.ods' (16.6 kB)? (Y/n) 

If the receiving user then enters ‘Y’, the sending user sees something similar to this:

user $ croc send Documents/flight-times.ods
Sending 'flight-times.ods' (16.6 kB)
Code is: 8878-salary-courage-roger
On the other computer run

croc 8878-salary-courage-roger

Sending (->192.168.1.74:60740)
 100% |████████████████████| (17/17 kB, 10.918 MB/s)
user $ 

and the receiving user sees something similar to this:

user $ croc 8878-salary-courage-roger
Accept 'flight-times.ods' (16.6 kB)? (Y/n) Y

Receiving (<-[::1]:39442)
 100% |████████████████████| (17/17 kB, 3.989 MB/s)
user $ 

The observant reader will notice that the above example shows a file being transferred on the same computer. When transferred between different computers the IP addresses of each computer will be displayed instead. I have used croc to transfer files between different computers on my home network (I would normally just use my NAS for this, though), between remote computers on the Internet, and between my computers and my phone via mobile broadband, and croc works in all cases.

I have not mentioned all croc’s features. I’ll leave you to read up on croc in more detail in the links I’ve given above. It looks like it might be a useful tool to have installed.

Deleted e-mails in an Office 365 Outlook account keep reappearing in the Thunderbird e-mail client

Whatever OS and e-mail client you use, if you search the Web you’ll find plenty of posts about deleted e-mails that reappear after you empty the ‘Trash’/’Deleted Items’/’Recycle bin’ folder. This problem seems to occur mostly with e-mail accounts that use the IMAP (Internet Message Access Protocol) or the Microsoft EWS (Exchange Web Services) protocols.

The obvious thing to check first in the e-mail client is if it has been configured to actually delete the e-mails on the server when you empty the ‘Deleted Items’ folder. Furthermore, internal files called ‘folder index files’ (.msf) in Thunderbird can sometimes become damaged, and these damaged files can also result in deleted e-mails reappearing. There is a ‘Repair Folder’ option in Thunderbird that sometimes fixes this problem. If that does not work, deleting the relevant folder’s .msf file and allowing Thunderbird to rebuild it sometimes fixes the problem. Anyway, I tried all the suggested approaches and more, as well as completely removing the account in Thunderbird (including ticking ‘Remove message data’ and manually deleting any remaining files for that account that remained in the Thunderbird directory). But nothing worked. However, I eventually cracked the problem as explained below.

Let’s just recap my situation:

1. Of the many e-mails in a corporate Office 365 account that I had deleted over the last few months and emptied from the account’s ‘Deleted Items’ folder, five of them would keep reappearing in that account’s ‘Deleted Items’ folder in Thunderbird. This happened if I deleted the e-mails individually from the ‘Deleted Items’ folder and if I right-clicked on the folder and selected ‘Empty Deleted’.
2. Whenever I logged in to the Office 365 Outlook account via a Web browser, the ‘Deleted Items’ folder was empty and it showed ‘Recover items deleted from this folder (0 items)’, i.e. no deleted messages existed.
3. The Samsung e-mail client on my Samsung Galaxy Note 8 mobile phone showed the ‘Recycle bin’ for the same account was empty.

All the settings for the account in Thunderbird were correct. Completely removing the account from Thunderbird then adding the account again did not solve the problem, meaning that the problem could not be due to Thunderbird or the ExQuilla add-on Thunderbird uses to enable it to access the Office 365 Outlook account using Microsoft EWS. Even though my Samsung mobile phone’s e-mail client showed the Office 365 Outlook account’s ‘Recycle bin’ was empty, I selected ‘Email settings’ in the Samsung e-mail client, selected the account, scrolled down to ‘Empty Recycle bin’ and tapped it. The following message was displayed:

Empty Recycle bin?

This will permanently delete the items in the Recycle bin.

Cancel            Delete

I tapped ‘Delete’ and the Samsung e-mail client displayed ‘Success’. The five rogue e-mails then disappeared from the ‘Deleted Items’ folder in Thunderbird. The next time I logged in to the Office 365 Outlook account via a Web browser, the ‘Deleted Items’ folder was still empty but it displayed ‘Recover items deleted from this folder (5 items)’. I used the usual Office 365 Outlook procedure to recover the five e-mails and delete them permanently, resulting in ‘Recover items deleted from this folder (0 items)’ being displayed again in Office 365 Outlook in the browser window.

So, there you have it, the problem had nothing to do with Thunderbird or ExQuilla. If you access an Office 365 Outlook e-mail account via an e-mail client on a desktop or laptop and also via an e-mail client on a mobile phone, and you find that e-mails you deleted and emptied from the ‘Deleted Items’/’Trash’/’Recycle bin’ folder in the e-mail client on the desktop/laptop keep reappearing in that folder, try deleting them on all your devices, including your phone, even if the e-mails are not shown in the Office 365 Outlook account in a Web browser nor in the Office 365 Outlook account in a phone’s e-mail client.

Updating the Powerline adapters in my home network

I have blogged previously about a couple of problems with using Powerline adapters in my home network:

• ‘Waiting for 192.168.1.254…’ (Why I could not access a home hub’s management page)
• Powerline adapters and IPv6

As my NETGEAR XAV1301 (200 Mbps) Powerline adapters bought in 2012 apparently do not fully support IPv6, and as my NETGEAR XAV5221 (500 Mbps) adapters bought in 2016 are no longer manufactured either, I decided to invest in some new Powerline adapters that would guarantee IPv6 support. My Web searches did not confirm that the current models of NETGEAR Powerline adapters support IPv6, so I decided to try TP-Link Powerline adapters because the TP-Link Web site states that all current TP-Link Powerline adapters support IPv6. I wanted Powerline adapters for five devices (router, smart TV and three computers), plus the ability to use a mains plug on at least two of those (i.e. so-called ‘pass-through’ adapters). I also wanted to avoid buying different models, in order to minimise the possibility of any problems. TP-Link have a range of 600 Mbps adapters under the name ‘AV600’, so I plumped for two TP-PL4010 adapters (single Ethernet port per adapter), one TP-PL4010P adapter (single Ethernet port and one mains pass-through socket) and one TP-PL4020P (two Ethernet ports and one mains pass-through socket). These all use the Qualcomm Atheros QCA7420 Powerline chipset (which happens to be the same chipset used in my old NETGEAR XAV5221 500 Mbps adapters).

Like NETGEAR, TP-Link does not have a Powerline utility program for Linux, so I had to install TP-Link’s tpPLC utility program in Windows 10 running in a VM (virtual machine) in order to configure the four TP-Link adapters and set the ‘Powerline network name’ to avoid crosstalk with my neighbour’s Powerline adapters that use the factory default network name (‘HomePlugAV’).

Anyway, I got everything set up and working, but soon noticed that there were quite frequent dropouts of the connection to my router and the Internet. Some dropouts did occur when I was using the old NETGEAR Powerline adapters, but I was surprised to find that the performance of the new TP-Link adapters was much worse. The dropouts typically lasted a minute or two. This was annoying, to say the least.

I started searching the Web, and ‘TP-Link’ and ‘dropout’ occur together a lot. I had already disabled Power Saving Mode in the adapters, so knew that was not the cause. I happen to know someone who also uses TP-Link adapters, and he mentioned that he also experienced frequent dropouts. In addition to turning off Power Saving Mode, he had implemented a shell script on his machines to ping an Internet site periodically to try and keep the connection from dropping out, but this did not appear to make any difference. I wrote the script below to try the same thing, and it did not cure the dropouts either:

#!/bin/bash
#
# Script to try to keep the Powerline adapter connected to this machine
# from dropping the connection to the router
#
FIRSTPASS=1
PREVIOUS=2
while true
do
    ping -W 2 -c 1 8.8.8.8 >>/dev/null 2>&1
    STATUS=$?
    if [ $PREVIOUS -ne 0 ] && [ $STATUS -eq 0 ]; then
        logger "Ping successful: connection to Internet is up."
#        echo "Ping successful: connection to Internet is up."
    elif [ $PREVIOUS -eq 0 ] && [ $STATUS -ne 0 ]; then
        logger "Ping unsuccessful: connection to Internet may be down."
#        echo "Ping unsuccessful: connection to Internet may be down."
    elif [ $FIRSTPASS -eq 1 ] && [ $STATUS -ne 0 ]; then
        logger "Ping unsuccessful: connection to Internet may be down."
#        echo "Ping unsuccessful: connection to Internet may be down."
    fi
    PREVIOUS=$STATUS
    FIRSTPASS=0
    sleep 10
done

In my Web searches I came across a a thread in the TP-Link SOHO Community forums with a URL for a new version of firmware for TP-Link Powerline adapters that use the Qualcomm Atheros QCA7420 chipset. I learned from the TP-Link forums that the firmware in NVM (Non-Volatile Memory) depends on the chipset manufacturer’s chipset, not on the Powerline manufacturer’s adapter model, whereas the adapter’s PIB (Parameter Information Block) does change depending on the model (including the country). So I started searching online for a PIB file for the three models of TP-Link adapter that I am using, but I could not find them. However, the Linux open-plc-tools command ‘plctool‘ enabled me to read the PIB from each adapter and store it as a file:

user $ sudo plctool -i eth0 -p TL-PA4010P.pib <MAC address printed on the adapter>
user $ sudo plctool -i eth0 -p TL-PA4010_TV.pib <MAC address printed on the adapter>
user $ sudo plctool -i eth0 -p TL-PA4010_HOME-HUB.pib <MAC address printed on the adapter>
user $ sudo plctool -i eth0 -p TL-PA4020P.pib <MAC address printed on the adapter>

The Ethernet interface in the computer I used is named ‘eth0′, so change it accordingly. You can give any name to the PIB files.

It is also easy to find out the adapters’ MAC addresses and current firmware by using another open-plc-tools command:

user $ plcstat -t -i eth0

The TP-Link tpPLC utility for Windows also shows the firmware version. I was surprised to see that the firmware version was different in the three models I had just bought:

• TL-PA4010P firmware version: 1.4.0.20-00_401115_191120_901
• TL-PA4010 firmware version: 1.3.1.2141-00_401013_171025_901
• TL-PA4020P firmware version: 1.4.0.20-00_402114_191120_901

The command to update the firmware in an adapter using the NVM file I downloaded from the URL in the above-mentioned TP-Link Community forum thread and the PIB file read from the relevant adapter, is as follows:

user $ sudo plctool -i <interface> -P <PIB file> -N <NVM file> -R <MAC address of adapter>

For example:

user $ sudo plctool -i eth0 -P TL-PA4010P.pib -N FW-QCA7420-1.5.0.0026-02-CS-20200114.nvm -R 15:B3:D2:D8:5F:BA

I am fortunate in that the three models of TP-Link Powerline adapter I bought all use the Qualcomm Atheros QCA7420 chipset, so I could use the same NVM file for all four adapters that I bought. I only needed to repeat the command with a different PIB file for each adapter model. The plcstat command can be used to check that the firmware version is different from the factory original version:

user $ plcstat -t -i eth0

Actually, the tpPLC utility in Windows 10 also has the ability to upload an NVM file and a PIB file to an adapter, so, as I have tpPLC installed in a VM, I can use that instead to update firmware in my TP-Link Powerline adapters.

And what difference did upgrading the firmware in my new TP-Link adapters make? A big difference. There are no more dropouts; the connection is now stable and I no longer get interruptions while browsing the Internet. It’s a pity that TP-Link does not supply every chipset’s latest firmware file and every model’s PIB file on their support Web site so that users can update their Powerline adapters.

Powerline adapters and IPv6

My home network includes a number of devices connected via Powerline (HomePlug) adapters. Back in 2015 I blogged about ‘crosstalk’ between my and my neighbour’s home networks, both of which use Powerline adapters (see my post ‘Waiting for 192.168.1.254…’ (Why I could not access a home hub’s management page)), which I was able to resolve by changing the encryption key so that it is different to the default key used by my neighbour. Since then the Powerline adapters have worked well. However, an unrelated network problem recently highlighted another problem with my Powerline adapters…

In November last year there was an external fault with the broadband service to my house, so I had to contact my ISP (the company BT) to fix the problem. BT does not use highly-skilled field personnel to diagnose broadband problems; they tend to use a ‘shotgun’ approach to problem solving. Their first attempt was to replace my router, a BT Home Hub 5, which I knew was actually working perfectly. I was not going to argue, though, because they replaced the router with the newest model, a BT Smart Hub 2. Unlike the Home Hub 5, the Smart Hub 2 fully supports IPv6. BT’s broadband network has supported IPv6 for several years (see ISPreview – UPDATE All BT Broadband Lines Now Support IPv6 Internet Addresses) so I was expecting the computers on my home network to be assigned IPv6 addresses, but ‘ifconfig‘ and ‘ip address‘ showed they were not being assigned IPv6 addresses when connected via the Powerline adapters, only when connected to the Smart Hub 2 via Wi-Fi.

All my computers have IPv6 enabled:

$ sudo sysctl -a | grep disable_ipv6
[sudo] password for fitzcarraldo: 
sysctl: net.ipv6.conf.all.disable_ipv6 = 0
reading key "net.ipv6.conf.all.stable_secret"
sysctl: reading key "net.ipv6.conf.default.stable_secret"
net.ipv6.conf.default.disable_ipv6 = 0
sysctl: reading key "net.ipv6.conf.eno1.stable_secret"
net.ipv6.conf.eno1.disable_ipv6 = 0
sysctl: reading key "net.ipv6.conf.lo.stable_secret"
net.ipv6.conf.lo.disable_ipv6 = 0
sysctl: net.ipv6.conf.wlp2s0.disable_ipv6 = 0
reading key "net.ipv6.conf.wlp2s0.stable_secret"

$ test -f /proc/net/if_inet6 && echo "IPv6 supported" || echo "IPv6 not supported"
IPv6 supported

The fact that the computers on the home network were allocated an IPv6 address when connected to the Smart Hub 2 via Wi-Fi, and that WhatIsMyIPAddress.com confirmed the BT broadband public network was also allocating an IPv6 address, made me suspect the problem of no IPv6 via the wired network was due to the Powerline adapters.

As more machines were added to my home network over the years, I had to buy more Powerline adapters. In 2014 I bought some NETGEAR XAVB5221 (500 Mbps) Powerline adapters to supplement the superseded model NETGEAR XAVB1301 (200 Mbps) Powerline adapters I bought in 2012. Powerline adapters conforming to the HomePlug AV standard work together, so these had no problem communicating. A schematic diagram of my home network is shown below. To keep things simple, only some of the devices are shown. As you can in the diagram, a NETGEAR XAVB1301 adapter was used to connect the BT Smart Hub 2 to the network; some of the computers were connected via NETGEAR XAVB5221 adapters, and others via NETGEAR XAVB1301 adapters.

I could find no mention of IPv6 for its Powerline adapters in NETGEAR’s documentation and on the NETGEAR Web site. The NETGEAR user manual for the XAV1301 is dated ‘September 2011’ and it lists, under SPECIFICATIONS, compliance with IEEE 802.3 and IEEE 802.3u. The data sheet (no user manual available) for the XAVB5221 is dated ‘2014’ and it lists, under SPECIFICATIONS, compliance with IEEE 1901 and IEEE 802.3.

The Wikipedia page for IEEE 1901-2010 mentions IPv6, so support for IPv6 is relevant to the protocol:

“An IETF RFC Draft address the higher layers of the protocol, namely the specifics of passing IPv6 packets over the PHY and MAC layers of PLC [power-line communication] systems like IEEE 1901.”

I think the following draft Internet Engineering Task Force (IETF) document must be the latest version of the IETF Draft mentioned on the above-mentioned Wikipedia page for IEEE 1901:

Transmission of IPv6 Packets over PLC Networks

Anyway, all this lead me to wonder if the NETGEAR XAVB1301 does not fully comply with IEEE 1901 and does not support IPv6. So I decided to try connecting the BT Smart Hub 2 to the network via a NETGEAR XAVB5221 adapter instead of the older model XAVB1301, as shown in the schematic diagram below.

What I then found was that any computer connected to the network via a NETGEAR XAVB5221 adapter was assigned an IPv6 address in addition to an IPv4 address, and WhatIsMyIPAddress.com showed public IPv6 and IPv4 addresses in a Web browser on the device. However, any computer connected to the network via a NETGEAR XAVB1301 adapter was assigned an IPv6 address in addition to an IPv4 address but WhatIsMyIPAddress.com displayed ‘IPv6 not detected’ in a Web browser. So it transpired that NETGEAR XAVB5221 adapters can handle IPv6 but the older XAVB1301 model cannot.

Although not essential, I toyed with the idea of replacing the older NETGEAR XAVB1301 adapters with XAVB5221 adapters, but that model is no longer on sale. The latest available Powerline adapter model from NETGEAR for wired networking is the PL1000 (1000 Mbps). However, its documentation does not mention IPv6 or IEEE 1901, and the following question on the Amazon UK Web site about IPv6 support for the PL1000, and NETGEAR’s answer on 5 May 2020 makes it clear that the PL1000 does not support IPv6:

Question: Does this model support ipv6? netgear xav1301 adapters only support ipv4. my router & pcs support ipv6 but can’t use ipv6 with my xav1301 adapters.

Answer: Thank you for your interest in the NETGEAR PL1000.

The PL1000 supports IPv4.

If you have any questions, you can also check out our NETGEAR Community at any time.

Best regards,
NETGEAR Amazon UK

Unlike NETGEAR, the TP-Link Web site makes it clear that all TP-Link Powerline adapters currently on sale support IPv6:

Most frequently asked questions about TP-Link powerline devices – Part3: Other questions about Powerline Device

Q3.12: Can TP-Link Powerline devices transfer IPv6 packets?

A: Yes, all the on sale TP-Link powerline devices can transfer IPv6 packets. Kindly note this is supported by default and does not require any configuration, our powerline products do not have setting entries for IPv6 either.

I also asked someone I know who uses TP-Link Powerline adapters and a BT Smart Hub 2, and he confirmed that the TP-Link adapters can handle IPv6.

Therefore, the bottom line is: if you want to use Powerline adapters and IPv6, avoid buying NETGEAR Powerline adapters and look at other manufacturers’ adapters instead. I have only investigated TP-Link’s adapters, which do support IPv6. A number of other companies also manufacture Powerline adapters, but you would need to check if they support IPv6; if necessary contact the manufacturer to be sure.

Jitsi Meet, my favourite video conferencing platform (and a way to share audio when using it in Linux)

During the current COVID-19 lockdown I have been using video conferencing platforms a lot for family virtual meet-ups, quizzes and multi-player games by Jackbox Games. Zoom seems to be the most popular video conference platform at the moment, although several articles in the media have pointed out some of its security limitations (see, e.g., ‘‘Zoom is malware’: why experts worry about the video conferencing platform‘). Although many people like Zoom, my favourite video conferencing platform is Jitsi Meet.

For an excellent third-party video introduction to Jitsi Meet, watch the video: ‘Using Jitsi: A free, no-registration video conferencing site‘. WIRED Magazine’s recent article on Jitsi Meet is also worth reading: ‘Want to Ditch Zoom? Jitsi Offers an Open-Source Alternative‘.

The reasons I prefer Jitsi Meet to Zoom include the following:

1. no subscriptions are required to use all the features of Jitsi Meet;
2. unlike Zoom, Jitsi Meet does not require you to sign up;
3. unlike Zoom, Jitsi Meet does not require the installation of an application — it runs in Google Chrome or Firefox;
4. unlike the free version of Zoom, Jitsi Meet does not impose a time limit on the length of the meeting;
5. unlike the free version of Zoom, Jitsi Meet does not have a limit on the number of meeting attendees;
6. Jitsi Meet provides end-to-end encryption for one-to-one video calls*;
7. I find the performance of Jitsi Meet better than Zoom, which seems to be corroborated in basic benchmarking by Jitsi Meet’s developers (‘WebRTC vs. Zoom – A Simple Congestion Test‘);
8. I find image quality better in Jitsi Meet;
9. I find Jitsi Meet on a desktop/laptop more intuitive and easier to use than Zoom;
10. if I share audio in Zoom for Linux, the audio is very distorted**;
11. I find the UI of the Jitsi Meet app for Android easy to use (the app can be installed via Google Play);
12. Jitsi Meet is open-source, so anyone can inspect the source code;
13. if I wanted to, I could download the Jitsi software to my own server and set up a Jitsi Meet server to handle meetings instead of using the Cloud server provided by 8×8, Inc. (the company that develops the Jitsi Meetings software).

* Neither platform currently provides end-to-end encryption for group meetings, although the developers of Jitsi Meet are apparently working on implementing end-to-end encryption for group meetings using a new feature of Google Chrome called ‘Insertable Streams’.

** There is a work-around for this problem in Zoom for Linux; see my answer to the Unix & Linux Stack Exchange question ‘Play audio output as input to Zoom’. In the case of Jitsi Meet in Linux, PulseAudio Volume Control can be used to share audio, as I explain further down.

Jitsi Meet requires no installation; it runs in a browser window. Either Google Chrome or Firefox can be used, although I find it runs better in Google Chrome. Actually, an Ubuntu 16.04 user told me that Firefox hangs when he tries to join a Jitsi Meet meeting, but Jitsi Meet works fine in Firefox in my two Gentoo Linux installations and in my family’s Lubuntu 18.04 installation. When using Google Chrome, to be able to share your screen you need to install the Google Chrome extension ‘Jitsi Meetings’ by meet.ji.si in the Google Chrome Web Store.

One of my family here at home has a laptop running Windows 10. Google Chrome, but not Firefox, displays a ‘Share audio’ tick box when the ‘Share your screen’ icon is clicked (see ‘Jitsi Meet features update, April 2020‘). The ‘Share audio’ feature is needed when, for example, you are casting via the Internet to remote players a multi-user game running on your machine. During the current COVID-19 lockdown we have been having fun playing Jackbox Games Party Pack 6 this way with family and friends in different locations (see ‘(My Solution) Best method for Virtual Couch Multiplayer‘). Each household connects a laptop to their TV via HDMI and joins the Jitsi Meet meeting. The Jackbox Games games are cast via Jitsi Meet from the laptop at my house, and the group of players in each household can view and hear the game on their TV and participate using their mobile phones as per the Jackbox Games paradigm.

Jitsi Meet provides a ‘Share audio’ function in Windows only, but I found a work-around to to be able to share any application’s audio in Linux if I ever want to use my Linux machines to cast games by Jackbox Games or other suppliers. For once, I have found PulseAudio useful! I use PulseAudio Volume Control to redirect the audio output from the desired application (be it a game, music player, video player or whatever) to the microphone input. The precise way to do this depends on the audio hardware your machine has, but an example is given in the blog post: ‘Redirect Audio Out to Mic In (Linux)‘.

My family’s desktop machine running Lubuntu 18.04 uses a Webcam with an integral microphone connected via USB, and external powered speakers connected to the machine’s Line Out green-coloured 3.5 mm jack socket. The contents of the tabs in PulseAudio Volume Control when no applications that produce audio are running are shown in the following screenshots:

I make sure ‘All Streams’ is selected on the ‘Playback’ and ‘Recording’ tabs, ‘All Output Devices’ is selected on the ‘Output Devices’ tab, and ‘All Input Devices’ is selected on the ‘Input Devices’ tab.

Let us say I have launched Audacious to play some music and I want to cast that music to members of a Jitsi Meet meeting. When I am using Jitsi Meet for a meeting in Google Chrome, the contents of the PulseAudio Volume Control tabs on this machine are as follows:

To redirect the audio from e.g. Audacious to the meeting members, I select (click on the square button with the green disc and white tick) ‘Monitor of Built-in Audio Analogue Stereo’ on the ‘Input Devices’ tab, and on the ‘Recording’ tab I click on Chrome input: RecordStream from ‘Camera Analogue Mono’ and select Chrome input: RecordStream from ‘Monitor of Built-in Audio Analogue Stereo’, as shown below.

With the above settings in PulseAudio volume control, all the members of the meeting will be able to hear clearly the audio from Audacious. To switch back to my microphone to speak, I simply click on Chrome input: RecordStream from ‘Monitor of Built-in Audio Analogue Stereo’ on the ‘Recording’ tab and select Chrome input: RecordStream from ‘Camera Analogue Mono’ again.

Using Bash scripts in Linux to delete the history, cookies and cache files of Firefox, Google Chrome and Thunderbird

The browsing data stored by Firefox, Google Chrome and Thunderbird can be deleted using the respective application’s GUI. But you can also do that using a Bash script, which could be useful if you want to delete unnecessary/unwanted files before e.g. backing up your home directory, or if you want a quick and easy way to clear-out browsing data. In this post I list the scripts and Desktop Configuration files I have created in Gentoo Linux and in Lubuntu 18.04 to remove browsing data.

In the case of Thunderbird, I am not sure if it is safe to delete Thunderbird’s cache files so my script only deletes cookies. Anyway, that could be added later if it transpires there is no harm in deleting Thunderbird’s cache files.

I am using the following 64-bit versions of the two browsers and e-mail client:

• Mozilla Firefox 74.0
• Mozilla Thunderbird 68.5.0 in Gentoo Linux only
• Google Chrome 80.0.3987.132 in Gentoo Linux
• Google Chrome 67.0.3396.99 in Lubuntu 18.04

I have not tested my scripts with other versions of Firefox, Chrome and Thunderbird, nor in other installations, so please do check carefully the directory paths and commands in the script against the directory paths in your installation before selecting ‘[D]elete‘ in the running script.

In Lubuntu 18.04 I had to install sqlite3 first:

$ sudo apt install sqlite3

In Gentoo Linux it was already installed:

$ eix -I sqlite
[I] dev-db/sqlite
     Available versions:  (3) 3.29.0^t 3.30.1^t 3.31.1^t
       {debug doc icu +readline secure-delete static-libs tcl test tools ABI_MIPS="n32 n64 o32" ABI_RISCV="lp64 lp64d" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  3.31.1(3)^t(19:53:28 13/03/20)(icu readline secure-delete -debug -doc -static-libs -tcl -test -tools ABI_MIPS="-n32 -n64 -o32" ABI_RISCV="-lp64 -lp64d" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            https://sqlite.org/
     Description:         SQL database engine

Firefox and Thunderbird

I created the Bash script Firefox_or_Thunderbird_-_Clear_data.sh listed below. The user can select only Firefox or only Thunderbird, or both, and the script enables the user to choose whether to just view the current situation or to delete the data. The script checks if the applications are running and will not do anything if they are. In fact, the script offers the user the option to terminate the applications if they happen to be running. The script is still usable if either Firefox or Thunderbird are not installed. The same script can be used in Gentoo and in Lubuntu 18.04, and I believe it would also work in Ubuntu but have not tested it with that distribution.

#!/bin/bash
#
# If Delete is selected for Firefox, this script deletes the entire history, cookies, site data and cache.
# If Delete is selected for Thunderbird, this script deletes the cookies and leaves the cache intact.
#
echo
echo "WARNING:"
echo "The Firefox Browser must not be running if you are going to list or delete its data files."
echo "The Thunderbird e-mail client must not be running if you are going to list or delete its data files."
echo
FIREFOX=$( ls $HOME/.mozilla/firefox 2>/dev/null | grep .default )
THUNDERBIRD=$( ls $HOME/.thunderbird 2>/dev/null | grep .default )
while true
do
   echo -n "[F]irefox, [T]hunderbird, [B]oth or [E]xit: "
   read -n1 PROMPT2
   echo
   case $PROMPT2 in
      [fF]* ) CHOICE2="F"; break;;
      [tT]* ) CHOICE2="T"; break;;
      [bB]* ) CHOICE2="B"; break;;
      [eE]* ) exit;;
      * )     echo "Invalid entry.";;
   esac
done
FRUNNING="N"
pgrep -u $USER firefox > /dev/null
if [[ $? -eq 0 ]]; then
   FRUNNING="Y"
   while true
   do
      echo
      echo -n "The Firefox browser is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT0
      echo
      case $PROMPT0 in
         [yY]* ) CHOICE0="Y"; break;;
         [nN]* ) CHOICE0="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE0 == "Y" ]]; then
      FPID=$( pgrep -u $USER firefox )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      FRUNNING="N"
   fi
fi
TRUNNING="N"
pgrep -u $USER thunderbird > /dev/null
if [[ $? -eq 0 ]]; then
   TRUNNING="Y"
   while true
   do
      echo -n "The Thunderbird e-mail client is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT1
      echo
      case $PROMPT1 in
         [yY]* ) CHOICE1="Y"; break;;
         [nN]* ) CHOICE1="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE1 == "Y" ]]; then
      FPID=$( pgrep -u $USER thunderbird )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      TRUNNING="N"
   fi
fi
ABORT="N"
if [[ $FRUNNING == "Y" ]]; then
   if [[ $CHOICE2 == "F" ]] || [[ $CHOICE2 == "B" ]]; then
      echo
      echo "Please quit Firefox then re-run this script."
      echo
      ABORT="Y"
   fi
fi
if [[ $TRUNNING == "Y" ]]; then
   if [[ $CHOICE2 == "T" ]] || [[ $CHOICE2 == "B" ]]; then
      echo
      echo "Please quit Thunderbird then re-run this script."
      echo
      ABORT="Y"
   fi
fi
if [[ $ABORT == "N" ]]; then
   while true
   do
      echo
      echo -n "[D]elete, [L]ist or [E]xit: "
      read -n1 PROMPT3
      echo
      case $PROMPT3 in
         [dD]* ) CHOICE3="D"; break;;
         [lL]* ) CHOICE3="L"; break;;
         [eE]* ) exit;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE3 == "D" ]]; then
      if [[ $CHOICE2 == "F" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$FIREFOX" ]; then
            echo "Firefox directory does not exist"
         else
            echo
            echo "Deleting Firefox History..........."
            echo "==================================="
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/places.sqlite "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;"
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/places.sqlite "delete from moz_historyvisits;"
            echo
            echo "Deleting Firefox Cookies........"
            echo "================================"
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;"
            echo
            echo "Deleting Firefox Site Data........"
            echo "=================================="
            ls ${HOME}/.mozilla/firefox/${FIREFOX}/storage/default/ | grep http
            find ${HOME}/.mozilla/firefox/${FIREFOX}/storage/default -name "http*" -type d -exec rm -r "{}" \; -prune
            echo
            echo "Deleting Firefox Cache..........."
            echo "================================="
            NUM=$( ls -1 ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries | wc -l )
            SIZ=$( du -sbh ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo "Files: $NUM Size: $SIZ"
            find ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries -type f -delete 2>/dev/null
            echo
         fi
      fi
      if [[ $CHOICE2 == "T" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$THUNDERBIRD" ]; then
            echo "Thunderbird directory does not exist"
         else
            echo "Deleting Thunderbird Cookies........"
            echo "===================================="
            sqlite3 ${HOME}/.thunderbird/${THUNDERBIRD}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;"
         fi
      fi
   fi
   if [[ $CHOICE3 == "L" ]]; then
      if [[ $CHOICE2 == "F" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$FIREFOX" ]; then
            echo "Firefox directory does not exist"
         else
            echo
            echo "Current Firefox History........."
            echo "================================"
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/places.sqlite "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | more
            echo
            echo "Current Firefox Cookies........"
            echo "==============================="
            sqlite3 ${HOME}/.mozilla/firefox/${FIREFOX}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | more
            echo
            echo "Current Firefox Site Data........"
            echo "================================="
            ls ${HOME}/.mozilla/firefox/${FIREFOX}/storage/default/ | grep http | more
            echo
            echo "Current Firefox Cache..........."
            echo "================================="
            NUM=$( ls -1 ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries | wc -l )
            SIZ=$( du -sbh ${HOME}/.cache/mozilla/firefox/${FIREFOX}/cache2/entries )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo "Files: $NUM Size: $SIZ"
            echo
         fi
      fi
      if [[ $CHOICE2 == "T" ]] || [[ $CHOICE2 == "B" ]]; then
         if [ -z "$THUNDERBIRD" ]; then
            echo "Thunderbird directory does not exist"
         else
            echo "Current Thunderbird Cookies......"
            echo "================================="
            sqlite3 ${HOME}/.thunderbird/${THUNDERBIRD}/cookies.sqlite "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | more
         fi
      fi
   fi
fi
printf "\n"
read -rep $'\n Press ENTER to exit ' PROMPT4

Listed below is the Desktop Configuration file Firefox_or_Thunderbird_-_Clear_data.desktop for KDE in Gentoo Linux that I created in the ~/Desktop/ directory. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/Mozilla.png, although of course the Desktop Configuration file can be modified if the icon file were to be stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Firefox or Thunderbird caches and cookies
Comment=Clear Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/Firefox_or_Thunderbird_-_Clear_data.sh
GenericName[en_GB]=Clear Firefox or Thunderbird caches and cookies
GenericName=Clear Firefox or Thunderbird caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Mozilla.png
MimeType=
Name[en_GB]=Firefox_or_Thunderbird_-_Clear_data
Name=Firefox_or_Thunderbird_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

The equivalent Desktop Configuration file I created for Lubuntu 18.04 is very similar:

[Desktop Entry]
Comment[en_GB]=Clear Firefox or Thunderbird caches and cookies
Comment=Clear Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/Firefox_or_Thunderbird_-_Clear_data.sh
GenericName[en_GB]=Clear Firefox or Thunderbird caches and cookies
GenericName=Clear Firefox or Thunderbird caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Mozilla.png
MimeType=
Name[en_GB]=Firefox_or_Thunderbird_-_Clear_data
Name=Firefox_or_Thunderbird_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-LXDE-SubstituteUID=false
X-LXDE-Username=fitzcarraldo

Google Chrome

I created the following Bash script Google-Chrome_-_Clear_data.sh for the Google Chrome browser in Gentoo Linux. Different scripts have to be used in Gentoo Linux and in Lubuntu 18.04 because the paths are different.

#!/bin/bash
#
# If Delete is selected, this script deletes the entire cache, the code cache,
# the Visited Links file and the Top Sites file.
#
echo
echo "WARNING:"
echo "The Google Chrome browser must not be running if you are going to list or delete its data files."
echo
while true
do
   echo -n "[D]elete, [L]ist or [E]xit: "
   read -n1 PROMPT1
   echo
   case $PROMPT1 in
      [dD]* ) CHOICE1="D"; break;;
      [lL]* ) CHOICE1="L"; break;;
      [eE]* ) exit;;
      * )     echo "Invalid entry.";;
   esac
done
CRUNNING="N"
pgrep -u $USER chrome > /dev/null
if [[ $? -eq 0 ]]; then
   CRUNNING="Y"
   while true
   do
      echo
      echo -n "The Google Chrome browser is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT0
      echo
      case $PROMPT0 in
         [yY]* ) CHOICE0="Y"; break;;
         [nN]* ) CHOICE0="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE0 == "Y" ]]; then
      FPID=$( pgrep -u $USER chrome )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      CRUNNING="N"
   fi
fi
if [[ $CRUNNING == "Y" ]]; then
   echo
   echo "Please quit Google Chrome then re-run this script."
   echo
else
   if [[ $CHOICE1 == "D" ]]; then
      echo
      echo "Deleting URLs..........."
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls; delete from urls;"
      echo
      echo "Deleting Cookies........"
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies; delete from cookies;"
      if [[ -d ${HOME}/.cache/google-chrome/Profile\ 2/Cache ]]; then
         echo
         echo "Deleting Chrome Cache......."
         echo "====================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Profile\ 2/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Profile\ 2/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
         rm -r ${HOME}/.cache/google-chrome/Profile\ 2/Cache
      fi
      if [[ -d ${HOME}/.cache/google-chrome/Profile\ 2/Code\ Cache ]]; then
         echo
         echo "Deleting Code Cache......"
         echo "========================="
         rm -r ${HOME}/.cache/google-chrome/Profile\ 2/Code\ Cache
      fi
      if [[ -f ${HOME}/.config/google-chrome/Profile\ 2/Visited\ Links ]]; then
         echo
         echo "Deleting Visited Links......"
         echo "============================"
         rm ${HOME}/.config/google-chrome/Profile\ 2/Visited\ Links
      fi
      if [[ -f ${HOME}/.config/google-chrome/Profile\ 2/Top\ Sites ]]; then
         echo
         echo "Deleting Top Sites......"
         echo "========================"
         rm ${HOME}/.config/google-chrome/Profile\ 2/Top\ Sites
      fi
   fi
   if [[ $CHOICE1 == "L" ]]; then
      echo
      echo "Current URLs..........."
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls;" | more
      echo
      echo "Current Cookies........"
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Profile\ 2/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies;" | more
      if [[ -d ${HOME}/.cache/google-chrome/Profile\ 2/Cache ]]; then
         echo
         echo "Current Chrome Cache............."
         echo "================================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Profile\ 2/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Profile\ 2/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
      fi
   fi
fi
printf "\n"
read -rep $'\n Press ENTER to exit ' PROMPT2

And below is the Desktop Configuration file Google-Chrome_-_Clear_data.desktop for Gentoo Linux. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/Google-Chrome.png, although of course the Desktop Configuration file can be adjusted if the icon file were stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Google Chrome cache and cookies
Comment=Clear Google Chrome cache and cookies
Exec=/home/fitzcarraldo/Google-Chrome_-_Clear_data.sh
GenericName[en_GB]=Clear Google Chrome cache and cookies
GenericName=Clear Google Chrome cache and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Google-Chrome.png
MimeType=
Name[en_GB]=Google-Chrome_-_Clear_data
Name=Google-Chrome_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=

Below is the version of the script Google-Chrome_-_Clear_data.sh for Lubuntu 18.04:

#!/bin/bash
#
# If Delete is selected, this script deletes the entire cache, the code cache,
# the Visited Links file and the Top Sites file.
#
echo
echo "WARNING:"
echo "The Google Chrome browser must not be running if you are going to list or delete its data files."
echo
while true
do
   echo -n "[D]elete, [L]ist or [E]xit: "
   read -n1 PROMPT1
   echo
   case $PROMPT1 in
      [dD]* ) CHOICE1="D"; break;;
      [lL]* ) CHOICE1="L"; break;;
      [eE]* ) exit;;
      * )     echo "Invalid entry.";;
   esac
done
CRUNNING="N"
pgrep -u $USER chrome > /dev/null
if [[ $? -eq 0 ]]; then
   CRUNNING="Y"
   while true
   do
      echo
      echo -n "The Google Chrome browser is running. Do you wish to close it now? [Y/N]: "
      read -n1 PROMPT0
      echo
      case $PROMPT0 in
         [yY]* ) CHOICE0="Y"; break;;
         [nN]* ) CHOICE0="N"; break;;
         * )     echo "Invalid entry.";;
      esac
   done
   if [[ $CHOICE0 == "Y" ]]; then
      FPID=$( pgrep -u $USER chrome )
      FPID=$( echo $FPID | cut -d" " -f1 )
      kill -1 $FPID
      CRUNNING="N"
   fi
fi
if [[ $CRUNNING == "Y" ]]; then
   echo
   echo "Please quit Google Chrome then re-run this script."
   echo
else
   if [[ $CHOICE1 == "D" ]]; then
      echo
      echo "Deleting URLs..........."
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Default/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls; delete from urls;"
      echo
      echo "Deleting Cookies........"
      echo "========================"
      sqlite3 ${HOME}/.config/google-chrome/Default/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies; delete from cookies;"
      if [[ -d ${HOME}/.cache/google-chrome/Default/Cache ]]; then
         echo
         echo "Deleting Chrome Cache......."
         echo "====================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Default/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Default/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
         rm -r ${HOME}/.cache/google-chrome/Default/Cache
      fi
      if [[ -d ${HOME}/.cache/google-chrome/Default/Code\ Cache ]]; then
         echo
         echo "Deleting Code Cache......"
         echo "========================="
         rm -r ${HOME}/.cache/google-chrome/Default/Code\ Cache
      fi
      if [[ -f ${HOME}/.config/google-chrome/Default/Visited\ Links ]]; then
         echo
         echo "Deleting Visited Links......"
         echo "============================"
         rm ${HOME}/.config/google-chrome/Default/Visited\ Links
      fi
      if [[ -f ${HOME}/.config/google-chrome/Default/Top\ Sites ]]; then
         echo
         echo "Deleting Top Sites......"
         echo "========================"
         rm ${HOME}/.config/google-chrome/Default/Top\ Sites
      fi
   fi
   if [[ $CHOICE1 == "L" ]]; then
      echo
      echo "Current URLs..........."
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Default/History "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls;" | more
      echo
      echo "Current Cookies........"
      echo "======================="
      sqlite3 ${HOME}/.config/google-chrome/Default/Cookies "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies;" | more
      if [[ -d ${HOME}/.cache/google-chrome/Default/Cache ]]; then
         echo
         echo "Current Chrome Cache............."
         echo "================================="
         NUM=$( ls -1 ${HOME}/.cache/google-chrome/Default/Cache | wc -l )
         SIZ=$( du -sbh ${HOME}/.cache/google-chrome/Default/Cache )
         SIZ=$( echo $SIZ | cut -d" " -f1 )
         echo "Files: $NUM Size: $SIZ"
      fi
   fi
fi
printf "\n"
read -rep $'\n Press ENTER to exit ' PROMPT2

And below is the Desktop Configuration file Google-Chrome_-_Clear_data.desktop for Lubuntu 18.04. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/Google-Chrome.png, although of course the Desktop Configuration file can be adjusted if the icon file were stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Google Chrome cache and cookies
Comment=Clear Google Chrome cache and cookies
Exec=/home/fitzcarraldo/Google-Chrome_-_Clear_data.sh
GenericName[en_GB]=Clear Google Chrome cache and cookies
GenericName=Clear Google Chrome cache and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/Google-Chrome.png
MimeType=
Name[en_GB]=Google-Chrome_-_Clear_data
Name=Google-Chrome_-_Clear_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-LXDE-SubstituteUID=false
X-LXDE-Username=fitzcarraldo

Below are a few examples of the output when I launch the scripts.

Here is the output of the script that deals with Firefox and/or Thunderbird when I only list the current situation:

WARNING:
The Firefox Browser must not be running if you are going to list or delete its data files.
The Thunderbird e-mail client must not be running if you are going to list or delete its data files.
 
[F]irefox, [T]hunderbird, [B]oth or [E]xit: b 
 
The Firefox browser is running. Do you wish to close it now? [Y/N]: y 
The Thunderbird e-mail client is running. Do you wish to close it now? [Y/N]: y 
 
[D]elete, [L]ist or [E]xit: l 
 
Current Firefox History.........
================================
2020-03-19 17:27:24|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:27:20|https://www.youtube.com/
2020-03-19 17:27:30|https://www.theguardian.com/uk
2020-03-19 17:27:38|http://www.jb.com.br/
2020-03-19 17:27:38|https://www.jb.com.br/
 
Current Firefox Cookies........
===============================
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:22|accounts.google.com
2020-03-19 17:27:23|.doubleclick.net
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:24|.google.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:25|.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:30|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:39|.denakop.com
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:46|www.jb.com.br
2020-03-19 17:27:46|.www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
 
Current Firefox Site Data........
=================================
https+++www.google.com
https+++www.theguardian.com
https+++www.youtube.com
 
Current Firefox Cache...........
=================================
Files: 383 Size: 15M
 
Current Thunderbird Cookies......
=================================


 Press ENTER to exit

Here is the output of the script that deals with Firefox and/or Thunderbird when I delete only the Firefox data:

 
WARNING:
The Firefox Browser must not be running if you are going to list or delete its data files.
The Thunderbird e-mail client must not be running if you are going to list or delete its data files.
 
[F]irefox, [T]hunderbird, [B]oth or [E]xit: f 
 
[D]elete, [L]ist or [E]xit: d 
 
Deleting Firefox History...........
===================================
2020-03-19 17:27:24|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:27:20|https://www.youtube.com/
2020-03-19 17:27:30|https://www.theguardian.com/uk
2020-03-19 17:27:38|http://www.jb.com.br/
2020-03-19 17:27:38|https://www.jb.com.br/
 
Deleting Firefox Cookies........
================================
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:20|.youtube.com
2020-03-19 17:27:22|accounts.google.com
2020-03-19 17:27:23|.doubleclick.net
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:24|.google.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:24|www.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:25|.accuweather.com
2020-03-19 17:27:24|.accuweather.com
2020-03-19 17:27:25|www.accuweather.com
2020-03-19 17:27:30|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:36|.theguardian.com
2020-03-19 17:27:39|.denakop.com
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:43|.t.tailtarget.com
2020-03-19 17:27:44|www.jb.com.br
2020-03-19 17:27:44|.tt-10969-0.seg.t.tailtarget.com
2020-03-19 17:27:44|.t.tailtarget.com
2020-03-19 17:27:46|www.jb.com.br
2020-03-19 17:27:46|.www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
2020-03-19 17:27:43|www.jb.com.br
 
Deleting Firefox Site Data........
==================================
https+++www.google.com
https+++www.theguardian.com
https+++www.youtube.com
 
Deleting Firefox Cache...........
=================================
Files: 383 Size: 15M
 


 Press ENTER to exit

Here is the output of the script that deals with Google Chrome when I just list the current situation:

WARNING:
The Google Chrome browser must not be running if you are going to list or delete its data files.
 
[D]elete, [L]ist or [E]xit: l 
 
Current URLs...........
=======================
2020-03-19 17:30:41|https://duckduckgo.com/
2020-03-19 17:30:44|https://www.youtube.com/
2020-03-19 17:30:49|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:30:57|http://www.folha.uol.com.br/
2020-03-19 17:30:57|https://www.folha.uol.com.br/
 
Current Cookies........
=======================
2020-03-19 17:30:49|.accuweather.com
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.bt.uol.com.br
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:30:57|.accuweather.com
2020-03-19 17:30:45|accounts.google.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:46|.doubleclick.net
2020-03-19 17:31:08|player.mais.uol.com.br
2020-03-19 17:30:50|.google.com
2020-03-19 17:31:11|.dna.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:58|.www.accuweather.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:31:05|.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:30:58|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:58|www.accuweather.com
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:30:49|www.accuweather.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:08|.uol.com.br
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:30:58|.uol.com.br
2020-03-19 17:30:58|.navdmp.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|ivccf.ivcbrasil.org.br
2020-03-19 17:31:10|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:04|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:05|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:04|.prg.smartadserver.com
2020-03-19 17:31:22|www.folha.uol.com.br
2020-03-19 17:31:20|.uol.com.br
2020-03-19 17:31:10|.uol.com.br
 
Current Chrome Cache.............
=================================
Files: 317 Size: 8.6M


 Press ENTER to exit

Here is the output of the script that deals with Google Chrome when I delete the browser data:

WARNING:
The Google Chrome browser must not be running if you are going to list or delete its data files.
 
[D]elete, [L]ist or [E]xit: d 
 
Deleting URLs...........
========================
2020-03-19 17:30:41|https://duckduckgo.com/
2020-03-19 17:30:44|https://www.youtube.com/
2020-03-19 17:30:49|https://www.accuweather.com/en/gb/united-kingdom-weather
2020-03-19 17:30:57|http://www.folha.uol.com.br/
2020-03-19 17:30:57|https://www.folha.uol.com.br/
 
Deleting Cookies........
========================
2020-03-19 17:30:49|.accuweather.com
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.bt.uol.com.br
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:30:57|.accuweather.com
2020-03-19 17:30:45|accounts.google.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:46|.doubleclick.net
2020-03-19 17:31:08|player.mais.uol.com.br
2020-03-19 17:30:50|.google.com
2020-03-19 17:31:11|.dna.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:58|.www.accuweather.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:50|.scorecardresearch.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:30:44|.youtube.com
2020-03-19 17:31:05|.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:30:58|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:51|www.accuweather.com
2020-03-19 17:31:10|www.folha.uol.com.br
2020-03-19 17:30:58|www.accuweather.com
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:58|.accuweather.com
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:30:50|.accuweather.com
2020-03-19 17:31:01|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:31:11|.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|paywall.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:30:49|www.accuweather.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:08|.uol.com.br
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:06|player.mais.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:30:58|.uol.com.br
2020-03-19 17:30:58|.navdmp.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:30:51|.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:06|ivccf.ivcbrasil.org.br
2020-03-19 17:31:10|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:04|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:05|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:04|www.folha.uol.com.br
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:05|.tt-10162-1.seg.t.tailtarget.com
2020-03-19 17:31:05|.tt-12340-4.seg.t.tailtarget.com
2020-03-19 17:31:05|.t.tailtarget.com
2020-03-19 17:31:00|.t.tailtarget.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:30:57|www.accuweather.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.rubiconproject.com
2020-03-19 17:31:04|.smartadserver.com
2020-03-19 17:30:50|www.accuweather.com
2020-03-19 17:31:04|.prg.smartadserver.com
2020-03-19 17:31:22|www.folha.uol.com.br
2020-03-19 17:31:20|.uol.com.br
2020-03-19 17:31:10|.uol.com.br
 
Deleting Chrome Cache.......
=====================
Files: 317 Size: 8.6M
 
Deleting Code Cache......
=========================
 
Deleting Visited Links......
============================
 
Deleting Top Sites......
========================


 Press ENTER to exit

Update March 23, 2020: For those of you who would prefer a single script that can delete the browsing history, cookies and cache files of Google Chrome and/or Firefox and/or Thunderbird, I have now created the script clear_browser_data.sh shown below with appropriate Desktop Configuration files for Gentoo Linux and Lubuntu 18.04. In addition I have now added the ability to list and delete the history and cache files of Thunderbird, as I have checked that deleting those is not detremental to Thunderbird. Furthermore, this new script finds the directories itself rather than having them partially hard-coded in the script, so I am using the same script in both Gentoo and Lubuntu 18.04. That said, I have not tested the script with other versions of Chrome, Firefox and Thunderbird, nor with other Linux distributions, so do not select [D]elete until you have checked that the directories evaluated by the script match the directories in your installation.

#!/bin/bash
#
########################################################################
#
# Bash script to enable the user to list and/or delete browser data of:
# Google Chrome browser
# Mozilla Firefox browser
# Mozilla Thunderbird e-mail client
#
########################################################################
#
# Check for existence of the applications' directories
FIREFOX=$( ls $HOME/.mozilla/firefox 2>/dev/null | grep profiles.ini )
THUNDERBIRD=$( ls $HOME/.thunderbird  2>/dev/null | grep profiles.ini )
CHROME=$( ls -d $HOME/.config/google-chrome 2>/dev/null )
if [ -z "$CHROME" ]; then
   echo
   echo "Chrome directory does not exist"
else
   # Chrome directory locations
   #
   CLOCALST=$( find $HOME/.config/google-chrome -type f -name "Local State" )
   CCURPROF=$( cat "$CLOCALST" | awk -F "," '{for (I=1;I<=NF;I++) if ($I~/"info_cache"/) {print $(I)};}' | awk -F "info_cache" '{print $2}' | awk -F '"' '{print $3}' )   
   CHISTORY=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "History" )
   CCOOKIES=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "Cookies" )
   CVISITED=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "Visited Links" )
   CTOPSITS=$( find $HOME/.config/google-chrome/"$CCURPROF" -type f -name "Top Sites" )
   CCACHEDF=$( find $HOME/.cache/google-chrome/"$CCURPROF" -type d -name "Cache" )
   CCODECAC=$( find $HOME/.cache/google-chrome/"$CCURPROF" -maxdepth 2 -type d -name "Code Cache" )
   echo; echo "Chrome is using profile: $CCURPROF"
   
fi
if [ -z "$FIREFOX" ]; then
   echo
   echo "Firefox directory does not exist"
else
   # Firefox file locations. First get the current Profile Name
   # The profiles are listed in 'profiles.ini'.
   # If profiles.ini does not contain a line 'Locked=1' then the current profile
   # is followed by the line 'Default=1'. If profiles.ini does contain 'Locked=1'
   # then the current profile is followed by the line 'Locked=1'
   #
   FPROFILE=$( find $HOME/.mozilla/firefox/ -type f -name "profiles.ini" )
   if grep -q "Locked=1$" "$FPROFILE"; then
      FCURPROF=$( grep -B 1 "Locked=1$" "$FPROFILE" | grep -v Locked )
   else
      FCURPROF=$( grep -B 1 "Default=1$" "$FPROFILE" | grep -v Default )
   fi
   FCURPROF=$( echo "$FCURPROF" | cut -d"=" -f2 )
   FHISTORY=$( find $HOME/.mozilla/firefox/"$FCURPROF" -type f -name "places.sqlite" )
   FCOOKIES=$( find $HOME/.mozilla/firefox/"$FCURPROF" -type f -name "cookies.sqlite" )
   FSITEDAT=$( find $HOME/.mozilla/firefox/"$FCURPROF" -type d -name "default" )
   FCACHEDF=$( find $HOME/.cache/mozilla/firefox/"$FCURPROF" -type d -name "entries" )
   echo; echo "Firefox is using profile: $FCURPROF"
fi
if [ -z "$THUNDERBIRD" ]; then
   echo
   echo "Thunderbird directory does not exist"
else
   # Thunderbird file locations. First get the current Profile Name
   # The profiles are listed in 'profiles.ini'.
   # If profiles.ini does not contain the line 'Locked=1' then the current profile
   # is followed by the line 'Default=1'. If profiles.ini does contain 'Locked=1'
   # then the current profile is followed by the line 'Locked=1'
   #
   TPROFILE=$( find $HOME/.thunderbird/ -type f -name "profiles.ini" )
   if grep -q "Locked=1$" "$TPROFILE"; then
      TCURPROF=$( grep -B 1 "Locked=1$" "$TPROFILE" | grep -v Locked )
   else
      TCURPROF=$( grep -B 1 "Default=1$" "$TPROFILE" | grep -v Default )
   fi
   TCURPROF=$( echo "$TCURPROF" | cut -d"=" -f2 )
   THISTORY=$( find $HOME/.thunderbird/"$TCURPROF" -type f -name "places.sqlite" )
   TCOOKIES=$( find $HOME/.thunderbird/"$TCURPROF" -type f -name "cookies.sqlite" )
   TCACHEDF=$( find $HOME/.cache/thunderbird/"$TCURPROF" -type d -name "entries" )
   echo; echo "Thunderbird is using profile: $TCURPROF"
fi
#
MENU=""
while [[ $MENU != "E" ]] && [[ $MENU != "e" ]]; do
   echo
   echo -n "[C]hrome, [F]irefox, T[hunderbird or [E]xit: "
   read -n1 MENU
   echo
   case $MENU in
      [Cc] ) ;;
      [Ff] ) ;;
      [Tt] ) ;;
      [Ee] ) ;;
      * ) echo; echo " Enter 'C/c', 'F/f', 'T/t' or 'E/e'"
   esac
   #########
   # Chrome
   #########
   while [[ $MENU == "C" ]] || [[ $MENU == "c" ]]; do
      pgrep -u $USER chrome > /dev/null
      if [[ $? -eq 0 ]]; then
         echo
         echo -n "Chrome Browser is open, do you wish to close it now? [Y/N]: "
         read -n1 YN
         echo
         if [[ $YN == "Y" ]] || [[ $YN == "y" ]]; then
            CPID=$( pgrep -u $USER chrome )
            CPID=$( echo $CPID | cut -d" " -f1 )
            kill -1 $CPID
         else
            echo
            echo " Cannot list or delete browser URLs if the browser is open"
            break
         fi
      fi
      if [ -z "$CHROME" ]; then
         echo
         echo "Chrome directory does not exist"
         break
      else
         CPROMPT=""
         echo
         echo -n "[D]elete, [L]ist or [E]xit: "
         read -n1 CPROMPT
         echo
         case $CPROMPT in
            [Dd] ) echo; echo " Chrome is using profile: $CCURPROF";;
            [Ll] ) echo; echo " Chrome is using profile: $CCURPROF";;
            [Ee] ) ;;
            * ) echo; echo " Enter 'D/d', 'L/l' or 'E/e'";;
         esac
         if [[ $CPROMPT == "E" ]] || [[ $CPROMPT == "e" ]]; then MENU=""; fi
         if [[ $CPROMPT == "D" ]] || [[ $CPROMPT == "d" ]]; then
            echo
            echo " Deleting Chrome URLs............."
            echo " ================================="
            sqlite3 "$CHISTORY" "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls; delete from urls;" | sed 's/^/ /'
            echo
            echo " Deleting Chrome Cookies.........."
            echo " ================================="
            sqlite3 "$CCOOKIES" "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies; delete from cookies;" | sed 's/^/ /'
            if [[ -d "$CCACHEDF" ]]; then
               echo
               echo " Deleting Chrome Cache............"
               echo " ================================="
               NUM=$( ls -1 "$CCACHEDF" | wc -l )
               SIZ=$( du -sbh "$CCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               rm -r "$CCACHEDF"
            fi
            if [[ -d "$CCODECAC" ]]; then
               echo
               echo " Deleting Chrome Code Cache......."
               echo " ================================="
               NUM=$( ls -1 "$CCODECAC" | wc -l )
               SIZ=$( du -sbh "$CCODECAC" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               rm -r "$CCODECAC"
            fi
            if [[ -f "$CVISITED" ]]; then
               echo
               echo " Deleting Visited Links..........."
               echo " ================================="
               rm "$CVISITED"
            fi
            if [[ -f "$CTOPSITS" ]]; then
               echo
               echo " Deleting Chrome Top Sites........"
               echo " ================================="
               rm "$CTOPSITS"
            fi
         fi
         if [[ $CPROMPT == "L" ]] || [[ $CPROMPT == "l" ]]; then
            echo
            echo " Current Chrome URLs.............."
            echo " ================================="
            sqlite3 "$CHISTORY" "select datetime(last_visit_time/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),url from urls;" | sed 's/^/ /' | more
            echo
            echo " Current Chrome Cookies..........."
            echo " ================================="
            sqlite3 "$CCOOKIES" "select datetime(creation_utc/1000000 + (strftime('%s', '1601-01-01')),'unixepoch'),host_key from cookies;" | sed 's/^/ /' |more
            if [[ -d "$CCACHEDF" ]]; then
               echo
               echo " Current Chrome Cache............."
               echo " ================================="
               NUM=$( ls -1 "$CCACHEDF" | wc -l )
               SIZ=$( du -sbh "$CCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
            fi
            if [[ -d "$CCODECAC" ]]; then
               echo
               echo " Current Chrome Code Cache............."
               echo " ======================================"
               NUM=$( ls -1 "$CCODECAC" | wc -l )
               SIZ=$( du -sbh "$CCODECAC" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
            fi
         fi
         CPROMPT=""
      fi
   done
   ##########
   # Firefox
   ##########
   while [[ $MENU == "F" ]] || [[ $MENU == "f" ]]; do
      pgrep -u $USER firefox > /dev/null
      if [[ $? -eq 0 ]]; then
         echo
         echo -n "Firefox Browser is open, do you wish to close it now? [Y/N]: "
         read -n1 YN
         echo
         if [[ $YN == "Y" ]] || [[ $YN == "y" ]]; then
            FPID=$( pgrep -u $USER firefox )
            FPID=$( echo $FPID | cut -d" " -f1 )
            kill -1 $FPID
         else
            echo
            echo " Cannot list or delete browser URLs if the browser is open"
            break
         fi
      fi
      if [ -z "$FIREFOX" ]; then
         echo
         echo "Firefox directory does not exist"
         break
      else
         FPROMPT=""
         echo
         echo -n "[D]elete, [L]ist or [E]xit: "
         read -n1 FPROMPT
         echo
         case $FPROMPT in
            [Dd] ) echo; echo " Firefox is using profile: $FCURPROF";;
            [Ll] ) echo; echo " Firefox is using profile: $FCURPROF";;
            [Ee] ) ;;
            * ) echo; echo " Enter 'D/d', 'L/l' or 'E/e'";;
         esac
         if [[ $FPROMPT == "E" ]] || [[ $FPROMPT == "e" ]]; then MENU=""; fi
         if [[ $FPROMPT == "D" ]] || [[ $FPROMPT == "d" ]]; then
            echo
            echo " Deleting Firefox History........."
            echo " ================================="
            sqlite3 "$FHISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /'
            sqlite3 "$FHISTORY" "delete from moz_historyvisits;"
            echo
            echo " Deleting Firefox Cookies........."
            echo " ================================="
            sqlite3 "$FCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;" | sed 's/^/ /'
            echo
            echo " Deleting Firefox Site Data......."
            echo " ================================="
            ls "$FSITEDAT" | grep http | sed 's/^/ /'
            find "$FSITEDAT" -name "http*" -type d -exec rm -r "{}" \; -prune
            echo
            echo " Deleting Firefox Cache..........."
            echo " ================================="
            if [[ $( ls -A "$FCACHEDF" ) ]]; then # Directory not empty
               NUM=$( ls -1 "$FCACHEDF" | wc -l )
               SIZ=$( du -sbh "$FCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               find "$FCACHEDF" -type f -delete
            fi
         fi
         if [[ $FPROMPT == "L" ]] || [[ $FPROMPT == "l" ]]; then
            echo
            echo " Current Firefox History.........."
            echo " ================================="
            sqlite3 "$FHISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /' | more
            echo
            echo " Current Firefox Cookies.........."
            echo " ================================="
            sqlite3 "$FCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | sed 's/^/ /' | more
            echo
            echo " Current Firefox Site Data........"
            echo " ================================="
            ls "$FSITEDAT" | grep http | sed 's/^/ /' | more
            echo
            echo " Current Firefox Cache..........."
            echo " ================================"
            NUM=$( ls -1 "$FCACHEDF" | wc -l )
            SIZ=$( du -sbh "$FCACHEDF" )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo " Files: $NUM Size: $SIZ"
         fi
         FPROMPT=""
      fi
   done
   ##############
   # Thunderbird
   ##############
   while [[ $MENU == "T" ]] || [[ $MENU == "t" ]]; do
      pgrep -u $USER thunderbird > /dev/null
      if [[ $? -eq 0 ]]; then
         echo
         echo -n "Thunderbird e-mail client is open, do you wish to close it now? [Y/N]: "
         read -n1 YN
         echo
         if [[ $YN == "Y" ]] || [[ $YN == "y" ]]; then
            TPID=$( pgrep -u $USER thunderbird )
            TPID=$( echo $TPID | cut -d" " -f1 )
            kill -1 $TPID
         else
            echo
            echo " Cannot list or delete Thunderbird URLs if the e-mail client is open"
            break
         fi
      fi
      if [ -z "$THUNDERBIRD" ]; then
         echo
         echo "Thunderbird directory does not exist"
         break
      else
         TPROMPT=""
         echo
         echo -n "[D]elete, [L]ist or [E]xit: "
         read -n1 TPROMPT
         echo
         case $TPROMPT in
            [Dd] ) echo; echo " Thunderbird is using profile: $TCURPROF";;
            [Ll] ) echo; echo " Thunderbird is using profile: $TCURPROF";;
            [Ee] ) ;;
            * ) echo; echo " Enter 'D/d', 'L/l' or 'E/e'";;
         esac
         if [[ $TPROMPT == "E" ]] || [[ $TPROMPT == "e" ]]; then MENU=""; fi
         if [[ $TPROMPT == "D" ]] || [[ $TPROMPT == "d" ]]; then
            echo
            echo " Deleting Thunderbird History........."
            echo " ====================================="
            sqlite3 "$THISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /'
            sqlite3 "$THISTORY" "delete from moz_historyvisits;"
            echo
            echo " Deleting Thunderbird Cookies........."
            echo " ====================================="
            sqlite3 "$TCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies; delete from moz_cookies;" | sed 's/^/ /'
            echo
            echo " Deleting Thunderbird Cache..........."
            echo " ====================================="
            if [[ $( ls -A "$TCACHEDF" ) ]]; then # Directory not empty
               NUM=$( ls -1 "$TCACHEDF" | wc -l )
               SIZ=$( du -sbh "$TCACHEDF" )
               SIZ=$( echo $SIZ | cut -d" " -f1 )
               echo " Files: $NUM Size: $SIZ"
               find "$TCACHEDF" -type f -delete
            fi
         fi
         if [[ $TPROMPT == "L" ]] || [[ $TPROMPT == "l" ]]; then
            echo
            echo " Current Thunderbird History.........."
            echo " ====================================="
            sqlite3 "$THISTORY" "SELECT datetime(moz_historyvisits.visit_date/1000000,'unixepoch'), moz_places.url FROM moz_places, moz_historyvisits WHERE moz_places.id = moz_historyvisits.place_id;" | sed 's/^/ /' | more
            echo
            echo " Current Thunderbird Cookies.........."
            echo " ====================================="
            sqlite3 "$TCOOKIES" "select datetime(creationTime/1000000,'unixepoch'),host from moz_cookies;" | sed 's/^/ /' | more
            echo
            echo " Current Thunderbird Cache..........."
            echo " ===================================="
            NUM=$( ls -1 "$TCACHEDF" | wc -l )
            SIZ=$( du -sbh "$TCACHEDF"  )
            SIZ=$( echo $SIZ | cut -d" " -f1 )
            echo " Files: $NUM Size: $SIZ"
         fi
         TPROMPT=""
      fi
   done
done
echo

Listed below is the Desktop Configuration file clear_browser_data.desktop for KDE in Gentoo Linux that I created in the ~/Desktop/ directory. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/broom.png, although of course the Desktop Configuration file can be modified if the icon file were to be stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Chrome, Firefox or Thunderbird caches and cookies
Comment=Clear Chrome, Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/clear_browser_data.sh
GenericName[en_GB]=Clear browser caches and cookies
GenericName=Clear browser caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/broom.png
MimeType=
Name[en_GB]=clear_browser_data
Name=clear_browser_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

Listed below is the Desktop Configuration file clear_browser_data.desktop for LXDE in Lubuntu 18.04 that I created in the ~/Desktop/ directory. I downloaded a nice PNG icon from the Web, which I saved as ~/Pictures/Icons/broom.png, although of course the Desktop Configuration file can be modified if the icon file were to be stored elsewhere. Obviously change the username accordingly.

[Desktop Entry]
Comment[en_GB]=Clear Chrome, Firefox or Thunderbird caches and cookies
Comment=Clear Chrome, Firefox or Thunderbird caches and cookies
Exec=/home/fitzcarraldo/clear_browser_data.sh
GenericName[en_GB]=Clear browser caches and cookies
GenericName=Clear browser caches and cookies
Icon=/home/fitzcarraldo/Pictures/Icons/broom.png
MimeType=
Name[en_GB]=clear_browser_data
Name=clear_browser_data
Path=/home/fitzcarraldo
StartupNotify=true
Terminal=true
TerminalOptions=\s--noclose
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-LXDE-SubstituteUID=false
X-LXDE-Username=fitzcarraldo

Update April 5, 2020: I have modified Lines 24, 37 (a comment), 40 and 55 in the above script. The original Line 24 did not select the Chrome profile name correctly in another installation, the original Line 40 did not select the Firefox profile name correctly in another installation, and the search string for the Thunderbird profile was not tight enough in the original Line 55. The above script now works in three different installations.

Update April 25, 2020: Arrgghh! I installed Thunderbird in Lubuntu 18.04 and discovered that clear_browser_data.sh did not find the Thunderbird cached files to delete. Unlike Thunderbird in Gentoo Linux on my main laptop, the file profiles.ini for Thunderbird (but not Firefox) in my Lubuntu 18.04 installation contains a line ‘Locked=1‘, and the actual profile used by Thunderbird in Lubuntu 18.04 is the profile specified in the line before ‘Locked=1‘, not the profile specified in the line before ‘Default=1‘. Someone I know uses Ubuntu 16.04 and he tells me Firefox in his installation is like this too. So, for reasons I don’t know, profiles.ini contains ‘Locked=1‘ in some Firefox or Thunderbird installations but not in others. Anyway, I have now modified the script to cater for both cases. It checks if the profiles.ini files for Firefox and Thunderbird contain ‘Locked=1‘ and, if they do, finds the currently-used profile from the line before ‘Locked=1‘ instead of the line before ‘Default=1‘. The line numbers mentioned in my previous update will be different now.

Installing and using NeXT OPENSTEP in VirtualBox for Linux

Introduction and some history

My first micro computer was an Apple II+, which I used extensively both for work and leisure. In fact I liked it so much that I bought a //e when Apple Computer, Inc. released that model. I was not tempted by the Apple /// or Lisa when they were released, although I did quite fancy the IIGS but could not justify buying one. The //c was a nice portable, and a family member bought one on my recommendation. I was not at all tempted by the first Macintosh and subsequent models using the so-called Classic Mac OS, but I drooled when Steve Jobs founded NeXT, Inc. in 1985 and launched the magnesium-cased NeXT workstations: the cube-shaped NeXT Computer (Motorola 68030 CPU) in 1989 and in 1990 the second generation (Motorola 68040 CPU) NeXTcube and the NeXTstation (commonly referred to as ‘the slab’) running the NEXTSTEP operating system. The hardware build quality and aesthetic were fabulous, and the machines and NEXTSTEP were way ahead of their time. NEXTSTEP, which was built around Unix and therefore fully multi-tasking, looked amazing when compared to the competition and its performance was superior. Drooling was all I could do, though, because the price of any NeXT machine was totally out of my league.

OPENSTEP 4.2 Desktop in a VirtualBox VM.

By the way, Tim Berners-Lee invented HTTP, HTML and the first HTML browser using NEXTSTEP on a NeXTcube at CERN: see The Science Museum, London – The World Wide Web: A global information space.

Following Apple’s acquisition in 1997 of NeXT, which by then was only a software company (NeXT Software, Inc.), Apple developed Mac OS X based on OPENSTEP (the successor to NEXTSTEP). Even today some of the features in macOS are the same as in NEXTSTEP and OPENSTEP: NeXTSTEP vs Mac OS X – System Demo and Comparison. The final release of NEXTSTEP was NEXTSTEP 3.3, succeeded by OPENSTEP, the final release of which was OPENSTEP 4.2. OPENSTEP was effectively NEXTSTEP 4.

So, even though the NeXT company only sold around 50,000 machines during its relatively short existence as a manufacturer between 1988 and 1993, its impact on modern computing has been significant. Below are a few links to interesting videos about the company and some of its products. You’ll find plenty more videos about NeXT on YouTube.

• History of NeXT
• Why NeXT Wasn’t A Failure For Steve Jobs
• Complete NeXT Computer / NeXTcube setup & demo
• Tech : NeXT Cube Demo
• NeXT Station | When Steve Jobs Left Apple – Part 1
• NeXT Station | When Steve Jobs Left Apple – Part 2
• NeXT NeXTstation Turbo Color Computer from 1992 running NEXTSTEP 3.3

You can still find the occasional second-hand NeXT computer on eBay, but they are either incomplete or very expensive. As I write this there is a complete and pristine-looking NeXTcube system, including (non-working) NeXT laser printer, in Portugal listed on eBay at US$35,000 plus US$750 shipping! So I will never get to play with a real NeXT computer. But, thanks to VirtualBox, I can at least install the i386 release of OPENSTEP 4.2 in a VM (virtual machine) to try it out for fun. I decided to install the OS and the type of applications I would typically use (assuming I could find packages on the Web, that is). I wanted to find out how usable the OS was, how good the applications were, and whether I could access Unix easily from the GUI. As NeXT hardware and software are obsolete I had to spend a lot of time searching the Web for applications that would actually install and work. Some applications work in both NEXTSTEP and OPENSTEP, but plenty of applications have different packages for the two versions of the OS, which made my searches more complicated. Some OPENSTEP packages are so-called ‘fat binaries’ containing executables for some or all the different CPU types that OPENSTEP supported, and I found a few such packages on the Web. I wanted to install and try to use at least a Web browser, a word processor, a spreadsheet, an mp3 player and a video player. I also wanted to see if I could access files on a server on my home network using Samba.

There are quite a few tutorials and videos available on the Web explaining how to install OPENSTEP in a VM, but I did not find any on installing applications in OPENSTEP. Also, many of the OS installation tutorials I found are incomplete, for example not covering either audio or networking. I am not going to give a step-by-step explanation here of how I installed the OS and the applications, but I will explain what I installed, how I rated it, and any other information I found interesting or useful. Hopefully the tips I provide will be of some help if you fancy installing the OS and any applications yourself. I should also mention that you will have an advantage if you are a Unix and/or Linux user and are au fait with using the command line. OPENSTEP 4.2 provides the C Shell (csh). I did come across a package for the Bourne Again Shell (bash), but have not tried to install it. Sometimes I had to resort to the Unix command line to change ownership or permissions of a file and to move applications to folders owned by the root user. The pwd, cd, ls, su, cp, mv, chmod and chown commands came in handy a few times. By the way, unlike Linux the ls -la command does not display the group to which a file belongs, only its owner; you need to use the command ls -lag to show both. Also, the chown command accepts the notation owner.group but not owner:group when changing attributes.

Installation of OPENSTEP/Mach 4.2 for Intel i386 in VirtualBox

‘Mach’ refers to the Mach kernel, a microkernel developed at Carnegie Mellon University. OPENSTEP was available for Motorola 68k, Intel i386 and Sun SPARC CPUs. VirtualBox supports both 32-bit and 64-bit Intel CPUs, so the 32-bit OS can be installed in a VirtualBox 32-bit VM. NEXTSTEP also supported Hewlett-Packard’s PA-RISC CPU, but NeXT dropped support for that CPU in OPENSTEP.

Regarding the spelling of the two OSs, apparently the APIs are spelt ‘NeXTStep’ and ‘OpenStep’, and the OSs are spelt ‘NEXTSTEP’ and ‘OPENSTEP’. Confusing, or what? It’s no wonder these are used interchangeably all over the Web.

I found a reasonable tutorial on the installation of OPENSTEP 4.2, including links to download the image files of the CDROM and floppy disks required. Unlike many tutorials on the Web, it also explains how to get network access working, and I was able to ping other nodes on my home network and the Internet once I had completed the tutorial: ‘Installing NextStep OS (OPENSTEP) in VirtualBox‘. There were only one or two minor differences between the tutorial and what I saw on screen, and installation in VirtualBox for Linux was essentially painless. One of the packages that has to be installed (OS42MachUserPatch4.pkg) includes a Y2K patch for the OS. The tutorial tells you to use the command line to install that package, and I followed the instructions in the tutorial but, having now learned how to install packages via the OPENSTEP GUI by selecting a package and then ‘Services’ > ‘Open Sesame’ > ‘Open As Root’ > ‘Login’ to launch the Installer, I could have used only the GUI instead of the command line to install OS42MachUserPatch4.pkg (which I have checked). No matter, though, because using the OPENSTEP command line in Terminal.app is a good learning exercise. The tutorial does not mention some other things I had to configure in VirtualBox. To get audio working I had to select ‘SoundBlaster 16’ for the Audio Controller, install a driver in OPENSTEP and reboot the VM (see details further on), and under ‘Network’ in VirtualBox Manager I had to select ‘Bridged Adapter, PCnet-PCI II (Am79C970A)’ with ‘Promiscuous Model: Allow All’. I also enabled ‘Serial Ports’ and disabled ‘USB Controller’ (USB had not yet been invented back then!).

The OS installer installs US English support and offers the option of installing support for any of five other languages too: Swedish, Spanish, Italian, German and French. I unticked all those and completed the installation. Later I decided it might be useful to have support for those additional languages, and I found it very easy to install them retrospectively: I simply loaded the OPENSTEP-Install-4.2.iso file into the VM’s ‘optical drive’, browsed the CDROM’s contents, selected Upgrader.app and then ‘Workspace’ > ‘File’ > ‘Open as Folder’. I found the language packages (SwedishEssentials.pkg etc.) in the folder ‘NextCD’ > ‘Packages’. I could then select each language package and use ‘Services’ > ‘Open Sesame’ and so on to install it, as explained earlier.

To get sound working in OPENSTEP running in VirtualBox the procedure given in a 2009 tutorial ‘Installation of OPENSTEP 4.2 in VMware 3.0 and VirtualBox‘ miraculously still worked for me:

Audio: Alejandro Diaz Infante (aka astroboy) managed to make the OPENSTEP Sound Blaster driver work under VMWare and VirtualBox.
The solution: use the drivers created by University of Glasgow (Thanks, developer(s) of them, wherever you are, for drivers you never imagined would be so useful in the future).

1. Download SBSoundMidi.I.b.tar.gz and SBMixer.I.tar.gz
2. Install SBSoundMidi driver for either Vibra16Cpnp or AWE32pnp. Both work great! (I use the default irq and io, but the second DMA I put it on 7, ’cause it was the detected one when used VMWare to test Windoze. Anyway, I didn’t detect any failure when using the second DMA in its default of 5, so I guess it could be up to you. In VirtualBox I didn’t change any default setting, just select the driver “SoundBlaster 16” in VirtualBox audio setting before installing.
3. Install SBMixer to have better control of your sound card.

That’s it. Put those audio CD’s and multimedia apps back!

After copying SBSoundMidi.I.b.tar.gz to OPENSTEP I double-clicked on it to unpack it, and then double-clicked on SBSoundMidi.config to install the SoundBlaster 16 drivers. I then navigated to ‘openstep’ > ‘NextAdmin’ > ‘Configure.app’, selected the loudspeaker icon and specified the driver ‘SBSoundMidi driver for SoundBlaster AWE32 PnP (v3.38)‘.

SBMixer works, and OPENSTEP’s Sound Inspector can play .snd files without having to install additional software, although I found that some .snd files would not play completely. TheNeXTSong.snd (16-bit Linear format) which I downloaded from one of the OPENSTEP software repositories on the Web (see links at the end of this post) plays perfectly (and is amusing), but the shorter Welcome-to-the-NeXT-world.snd (8-bit muLaw format) stalls. I did manage to install a couple of audio players (see further down).

The only minor problem that occurs every time you login if the floppy disk drive is empty is a pop-up window with the message ‘The floppy disk is unreadable’. You can just click on ‘Eject’ but, to stop this happening, you can change the boot order in VirtualBox Manager and load one of the OPENSTEP floppy disk image files in the VM’s floppy disk drive (‘Settings…’ > ‘Storage’ > ‘Floppy Drive’ in VirtualBox Manager). Actually, I copied Driver_Floppy.img to Work_Floppy.img, loaded the latter in the VM’s floppy disk drive and I changed the Boot Order from ‘Floppy’|’Optical’|’Hard Disk’ to ‘Hard Disk’|’Optical’|’Floppy’ (‘Settings…’ > ‘System’ > ‘Motherboard’ > ‘Boot Order’ in the VirtualBox Manager). Furthermore, although not essential, I selected Work_Floppy in File Viewer, then in the Workspace menu I selected ‘Disk’ > ‘Initialize…’ and initialised (formatted) the floppy disk. Its icon disappears momentarily from File Viewer, then reappears after it has been formatted.

The command ifconfig on my VM host computer running Lubuntu 18.04 tells me that the IP address of the host machine is 192.168.1.74 (I had previous configured my router to always assign this address to this machine), the netmask is 255.255.255.0 and the broadcast IP address is 192.168.1.255. My router’s Management page in a Web browser has the DHCP network range configured as 192.168.1.64 – 192.168.1.253, so I decided the OPENSTEP VM would have a static IP address of 192.168.1.63. The router’s Management page also told me that the ISP’s Primary DNS IP address is 81.139.57.100 and the Secondary DNS IP address is 81.139.56.100. Therefore, in accordance with the OPENSTEP installation tutorial I followed, I edited the file /etc/hostconfig in OPENSTEP to have the following shell variables:

# /etc/hostconfig
#
# This file sets up shell variables used by the various rc scripts to
# configure the host.  Edit this file instead of rc.boot.
#
# Warning:  This is sourced by /bin/sh.  Make sure there are no spaces
#           on either side of the "=".
#
# There are some special keywords used by rc boot and the programs it
# calls:
#
#       -AUTOMATIC-     Configure automatically
#       -YES-           Turn a feature on
#       -NO-            Leave a feature off or do not configure
#
HOSTNAME=openstep
INETADDR=192.168.1.63
ROUTER=192.168.1.254
IPNETMASK=255.255.255.0
IPBROADCAST=192.168.1.255
YPDOMAIN=-NO-
NETMASTER=-NO-
TIME=-AUTOMATIC-

I also created the file /etc/resolv.conf as specified in the tutorial, containing the following two lines with the ISP’s nameserver IP addresses I found from my router:

nameserver 81.139.57.100
nameserver 81.139.56.100

It was not specified in the tutorial, but to get NFS working later I found it was necessary to edit the file /etc/hosts to comment out the list of IP addresses and to add the hostname I had chosen (openstep) for the OPENSTEP VM plus the IP address (192.168.1.74) and hostname (aspirexc600) of the VM host machine running Lubuntu 18.04:

#
# NOTE: This file is never consulted if NetInfo or Yellow Pages is running.
#
#
# To do anything on the network, you need to assign an address to your
# machine.  This default host table will get you started.  "myhost"
# can be used for the first machine on the network, and client[1-8]
# can be used for subsequent machines.  You must make sure that no two
# machines have the same address.  If you need to add more machines
# just keep adding entries.  Each digit in the four digit number must
# be between 1 and 254 inclusive.
#
#192.42.172.1	myhost
#192.42.172.2	client1
#192.42.172.3	client2
#192.42.172.4	client3
#192.42.172.5	client4
#192.42.172.6	client5
#192.42.172.7	client6
#192.42.172.8	client7
#192.42.172.9	client8
#
# This is the reserved address for the loopback interface.  Don't muck
# with it.
#
127.0.0.1       localhost       openstep
192.168.1.74    aspirexc600

While setting up networking in the VM I also temporarily disabled the firewall in the VM host to make sure the VM host was not interfering in any way with the network connection of the VM, then enabled it again once I was happy it was not causing any problems. Later, when I configured the VM host as an NFS server and the VM as an NFS client, I had to create the appropriate rules for NFS in the VM host’s firewall (see further down).

You will see NetInfo mentioned in the OPENSTEP networking apps. You should ignore NetInfo unless you are going to network a cluster of machines running NEXTSTEP/OPENSTEP, as it is an obsolete NeXT networking system configuration database and we don’t want to use it.

Installation of utilities and applications

After installing the OS neither the ‘me’ account nor the root account are password protected. You can use the OS like this if you wish, but I set up a password for the ‘me’ account by navigating to ‘openstep’ > ‘NextApps’ > Preferences.app and clicking on the padlock icon. Then I logged out and logged in to the root account and did the same to set up a password for the root user. If you want to save a bit of time during installation of applications, you could do this after installing all the packages.

OPENSTEP comes with quite a few utilities, such as Terminal.app, TextEdit.app, Draw.app, Sound.app (possibly useful if your host computer has a microphone socket and you enabled audio input in VirtualBox Manager), PhotoAlbum.app, CDPlayer.app, Webster.app (yes, a full dictionary), Librarian.app, PrintManager.app, Grab.app (to grab snapshots of all or parts of the screen and save them to .tiff files), Preview.app (an image file viewer), Mail.app, and others. You can try these and they are reasonably intuitive so I won’t dwell on them here, instead concentrating on how I installed third-party apps and utilities.

I had to trawl the Web to find packages and applications suitable for OPENSTEP/Mach 4.2 for i386. I find the filenames of the files stored on these Web sites confusing. I think ‘.s‘ in the filename of a compressed file means it contains source code, and ‘.b‘ means it contains binary code, i.e. executable. However, some filenames have ‘.bs‘ but only contain source code, so I could be wrong. Also, I’m not sure what the letters ‘N‘, ‘I‘, ‘H‘ and ‘S‘ represent in these filenames; NeXT (Motorola 68k), Intel, Hewlett-Packard PA-RISC and SPARC, presumably? Some OPENSTEP packages are called ‘fat binaries’ as they contain binaries for several or all the supported CPU types, thus enabling the package to be installed in OPENSTEP on different hardware. So my guess about the letters in the filenames could be correct.

Without a Web browser in OPENSTEP, the easiest way to copy files to the OPENSTEP VM initially is to use the Linux mkisofs command to create an ISO file and then to load it into the VM’s optical drive. For example, let’s say I want to copy the file OpenUp-1.01.tar to the VM, I would type the following on the host machine:

$ mkdir ~/ToCopy
$ cp ~/Downloads/OpenUp-1.01.tar ~/ToCopy
$ mkisofs -o ToCopy.iso ~/ToCopy

I then use the VirtualBox Manager GUI (‘Settings’ > ‘Storage’ > ‘Choose Virtual Optical Disk File…’) to insert the ToCopy.iso file into the VM’s optical drive. OPENSTEP mounts the ‘CDROM’ automatically and it becomes visible in the OPENSTEP File Viewer window. When I click on the CDROM icon a window opens and I see it contains the file openup_1.tar which I can then drag to the Shelf or to another folder directly.

Packages for installation using the OPENSTEP Installer have a ‘.pkg‘ suffix (e.g. ParaSheet.pkg) and are actually a folder, not a file. Applications have a ‘.app‘ suffix (e.g. ParaSheet.app) and are also a folder, not a file. Some of the compressed files I found for OPENSTEP on the Web are tarballs of OPENSTEP packages (e.g. OpenWrite.2.1.8.NIHS.b.tar.gz contains OpenWrite.pkg), others are tarballs of OPENSTEP applications (e.g. mpap.1.0.m.I.b.tar.gz contains mpap.app) which require unpacking but no installation, just copying to a folder. The mkisofs command truncates filenames to the Short Filename format (a.k.a. DOS 8.3 format), so if I had any uncompressed .pkg files, .app files and indeed any other files (.pdf, .mp3 or whatever) to transfer to the VM, I compressed them first as .tar files before creating the .iso file. Even though the .tar filename is truncated to DOS 8.3 by mkisofs, the filenames of the packed files are not.

a) Installing a package in OPENSTEP 4.2.

b) Installing a package in OPENSTEP 4.2.

Once you get the hang of installing packages in OPENSTEP, it is actually simple. For example, to install the package ParaSheet.pkg, I drag the .tar file from the CDROM to the Shelf, and from there to the folder /me. I double-click on the .tar file which opens a window showing the ParaSheet.pkg inside. I drag that to the /me folder. Then I select the package, and select ‘Workspace’ > ‘Open Sesame’ > ‘Open As Root’ > ‘Login’ and the Installer GUI opens. I then click on ‘Set…’ to specify the folder into which I want to install the application (e.g. /LocalApps/Office, as I had created the Office folder beforehand using Terminal.app) and then ‘Install’, and the Installer takes care of the rest.

In the case of applications that are not packaged and are just .app folders, I do not need to use the Installer, I just copy the .app folder to the folder I wish (/LocalApps/, /me/LocalApps/ or just /me/).

I found that, as-installed, OPENSTEP 4.2 can unpack .tar files from the GUI but does not have a GUI app for unpacking .tar.gz files, so the first thing I did was to install the OpenUp utility: OpenUp-1.01.m.NI.b.tgz which can be found at http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Apps/Compression_Utilities/ and works very well. Of course, I could have instead unpacked .tar.gz files in the host machine first and copied the .tar files to OPENSTEP using the mkisofs method I explained above, which the OPENSTEP GUI can unpack when I double-click on the .tar file. But OpenUp is well worth installing. After I had installed OpenUp and the OmniWeb browser in OPENSTEP, I was also able to download .tar.gz files directly in OPENSTEP from the various file repositories on the Web (see links at the end of this post) and unpack them in OPENSTEP.

By the way, see the links at the end of this post for user documentation. The OPENSTEP GUI is intuitive but I didn’t realise I could rename files from the GUI by clicking on the filename below the icon to get a cursor and typing directly (just like macOS), and I also didn’t know that I could use the ‘shelf’ at the top of the File Viewer as a temporary place to put copies of files to copy files between folders as an alternative to opening another File Viewer window. I also wondered how to select multiple files in a window when they are not adjacent, since using the mouse to select the group of files is not feasible in that case. It turns out the you hold down the Shift key and click on each file you want to select, which is analogous to holding down the Ctrl key and clicking on each file in Linux. I also found that I can copy a file between two File Viewer windows by clicking on it and holding down the Alt Gr key then dragging across to the other window.

Installation of a Web browser

This is where things start to get trickier. Bear in mind that NEXTSTEP and OPENSTEP were created in the 1980s and 1990s when the Web was in its infancy. As I mentioned earlier, the first Web browser was written on a NeXTcube at CERN, and that machine was the first Web server in existence. The best Web browser I could find for the platform is OmniWeb 3.1 for OPENSTEP. Before installing it, you need to install Omni Frameworks 1998G2. Also, the browser does not support HTTPS, Javascript and Flash out of the box and you have to install plugins. Unfortunately the plugins for these are very flaky, so you are severely limited in which sites and pages you can browse. Note that Netscape Communications created HTTPS in 1994, Netscape Communications and Sun Microsystems released JavaScript in December 1995, and Macromedia released Flash in November 1996. I don’t know if the OmniWeb plugins for HTTPS, JavaScript and Flash for OPENSTEP that I found are the latest or best versions for this version of OmniWeb, but they are what I could find online. JavaScript in Web pages results in a lot of pop-up error messages and made opening pages even less likely to be successful, so in the OmniWeb menu I navigated to ‘Info’ > ‘User Preferences…’ > ‘JavaScript’ and unticked ‘Display panel for errors’. I also navigated to ‘Info’ > ‘Administrator Preferences’ > ‘HTTPS – SSL’ and ticked ‘Enable TLSv1’, which seemed to enable a few HTTPS Web pages to load, at least partially.

You have to install OpenSSL before installing the HTTPS plugin for OmniWeb. I installed the package OpenSSL.0.9.5a.m.NIS.b.tar.gz which I downloaded from http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Apps/Internet/WWW/Web%20Browsers/Omniweb/Plugins/. Then I installed the package HTTPS.1.09b.m.NIS.b.tar.gz from the same site, which installs the file (folder) HTTPS.plugin, which needs to be in the folder /LocalLibrary/Plugins/ (‘NEXTSTEP’ > ‘LocalLibrary’ > ‘Plugins’).

Then I downloaded and installed the two packages JavaScript-OWPlugin-1999-07-20-OSM-NIS.tar.gz (installs JavaScript.plugin) and Flash-OWPlugin-19990621-OSM-NIS.tar (installs Flash.plugin) which also need to be in the folder /LocalLibrary/Plugins/ (‘NEXTSTEP’ > ‘LocalLibrary’ > ‘Plugins’ in the File Viewer). I found these two packages via a BetaArchive post [offer] OmniGroup software (NeXTSTEP, OpenStep & Rhapsody), which has a link to a .rar file at http://www.mediafire.com/file/wzyon54l4dt/OmniGroup.rar/file.

Unfortunately, even with the HTTPS and JavaScript plugins installed, almost all Web pages fail to load in OmniWeb, one exception being https://www.google.com. Old HTTP Web sites do load providing they are simple, but any JavaScript seems to cause a problem.

Installation of a PDF file reader

The best PDF file reader I could find for the platform is OmniPDF 3 for OPENSTEP. If you have not already installed Omni Frameworks, you first need to install Omni Frameworks 1998G2.

Installation of an image viewer

The best (supposedly) image file viewer I could find for the platform is OmniImage 4.0 for OPENSTEP. If you have not already installed Omni Frameworks, you first need to install Omni Frameworks 1998G2. However, according to the file /OmniImage.pkg/OmniImage.info it is a beta release and, in addition to Omni Frameworks, requires ‘Omni Plugins’:

Title OmniImage 4.0 beta for OPENSTEP/Mach 4.2
Version 4.0 beta 4 (1-Oct-1998)
Description This package contains a beta version of OmniImage. This beta release only supports viewing of images, not saving them. This release will not run unless the the Omni Frameworks (version 1998G2) are installed, and will not be fully functional (e.g., images may not be rendered) unless the Omni PlugIns (version 3.0 beta 8) are also installed. This software requires OPENSTEP/Mach 4.2.

I found the file OmniPlugIns-3.0b8-OSM-NIS.pkg.tar.gz in the BetaArchive post mentioned earlier in this post. I downloaded the tarball, created an ISO file containing it, loaded the ISO file in the VM CDROM drive, unpacked the tarball to /me/OmniPlugIns.pkg and installed the package using the OPENSTEP GUI Installer using the procedure explained earlier in this post. The Omni PlugIns were installed in the folder /LocalLibrary/PlugIns/ and I then found that OmniImage can open JPG files, even a 3456×2304 pixel JPG file with the following properties (as reported by the file command in Linux):

JPEG image data, JFIF standard 1.01, resolution (DPI), density 300x300, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=4, manufacturer=Canon, model=Canon EOS 600D], baseline, precision 8, 3456x2304, frames 3

Installation of wordprocessor and spreadsheet apps

OpenWrite and ParaSheet in use.

I created the folder /LocalApps/Office/ and installed OpenWrite from OpenWrite.2.1.8.NIHS.b.tar.gz which I downloaded from Index of /OpenStep/Soft/misc/NEXTTOYOU/97.1-Fruehjahr/APPSTOYOU. If you have not already installed it, before installing these apps you need to install Omni Frameworks 1998G2.

In the folder /LocalApps/Office/ I also installed ParaSheet from ParaSheet-1.7.pkg.tar.gz which I downloaded from Index of /NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design/ParaSheet. If you have not already installed Omni Frameworks, before installing these apps you need to install Omni Frameworks 1998G2.

The first time you launch OpenWrite and ParaSheet you will be notified that you cannot use the application until you enter a licence key. Exit the application and use ‘Open Sesame’ (see earlier) to launch the application as root user, and then you well be able to enter the licence. You will find a list of licences for these packages on the Web page Index of /NeXTfiles/Software/NEXTSTEP/Apps/Lighthouse_Design.

Installation of audio players

mpap and MMP audio players in action.

The only audio players I could find that actually worked (partially) in OPENSTEP are mpap 1.0 (download mpap.1.0.m.I.b.tar.gz) and MMP 2 (download mmp2.I.b.tar.gz). mpap can play some, but not all, of the mp3 files I have, whereas I could not get MMP to play mp3 files at all, although it can play .snd files. MMP can also play MIDI files, but I had to download the Timidity patches instruments.tar.gz (not so easy to find!) and follow the instructions in the MMP Info Panel in order to install the instruments patch file. It works fine! mpap cannot play an mp3 file which the files command in Linux tells me is an ‘Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v2.5, 32 kbps, 11.025 kHz, Stereo’ but it can play an mp3 file which is an ‘Audio file with ID3 version 2.4.0, contains:MPEG ADTS, layer III, v1, 192 kbps, 44.1 kHz, Stereo’. mpap has a basic playlist feature, but it is not as sophisticated as any of the modern audio players.

Installation of video players

MPLAY and Movie players in action.

This is where OPENSTEP is severely lacking in comparison to any modern OS; apparently we’re talking 5.5 or 6 frames per second and e.g. 288×224 pixels on NeXT hardware, and no sound. I only managed to find a couple of basic video players, both at Index of /OpenStep/Soft/video/apps: MPlay 3.0 (MPlay.app unpacked from MPlay.3.0.NIHS.b.tar.gz) and Movie 3.0 (Movie3.0 folder unpacked from Movie.3.0.NIHS.bs.tar.gz). MPlay is only designed to play MPEG (.mpg and .mpeg) files, which I found it can do for the old, tiny MPEG files I downloaded from Web repositories of NEXTSTEP/OPENSTEP files. I found that Movie can also only play MPEG files, despite the app’s README file stating it can play (without sound) MPEG, TIFF sequences, ‘QuickTime and other formats’. Movie comes with a couple of demo videos (no audio), the largest of which is hula_full.mpg in the mpeg1video format, consisting of 39 frames of 352×240 pixels, with a desired frame rate of 8 fps which actually plays at between 8 and 9 frames per second in OPENSTEP in the VM, i.e. it plays for around 4 to 5 seconds. In a video player in Linux on my desktop machine it plays for just over 2 seconds at 15 frames per second. These videos and players may have been state-of-the-art in the 1980s and early 1990s, but they certainly are not now!

I could not find an app package to play .avi files. The page I linked to above has a source-code tarball named VideoStreamV1.OSrc.tar.gz for an app named VideoStream, the README of which claims the app can play .avi files, but I have not found an executable package. Anyway, the README file states it cannot play videos with sound, so obviously I didn’t bother trying to install it.

Games

I am not particularly interested in computer games, but a few are installed by default with the OS: Chess.app, Billiards.app and BoinkOut.app (a clone of Breakout). More games for OPENSTEP can be found on the Web (for example at Index of /OpenStep/Soft/). The computer game Doom was originally developed in NEXTSTEP on NeXT computers, and a version for OPENSTEP can be downloaded from the Web, although I have not tried it.

File sharing

NEXTSTEP/OPENSTEP was designed to use NFS (Network File System). However I don’t use NFS in my home network; I use SMB and have a dedicated Linux SMB server which works well with all SMB clients (Linux, Windows and Android) on my home network. Unsurprisingly I could only find early versions of Samba packages for NEXTSTEP and OPENSTEP. I also came across ramba, a Unix clone of Samba later renamed to Sharity-Light. I downloaded them both and briefly tried to get OPENSTEP to connect to my network Samba server. I was unsuccessful, which does not surprise me as the version of Samba for NEXTSTEP/OPENSTEP I found is Version 2.0.7.1 from May 2000, and the obsolete version of rumba I found is Version 0.4 from February 1997. In NEXTSTEP/OPENSTEP the Samba configuration file smb.conf is located in the directory /usr/samba/lib/ rather than /etc/samba/. I did not spend much time trying to get Samba/Rumba working as I assume there would be incompatibility between the early SMB protocol used by Samba V2.0.7.1 / Rumba V0.4 with Samba V4.* running in the Linux SMB server on my network. Perhaps I could have made it work, but I decided to try to make the VM’s host computer (192.168.1.74) a NFS server to see if I could get the VM (192.168.1.63) to access it as a NFS client. The Web page OpenStep on Microsoft Windows PC Emulators states the following, which indicates that NFS works:

Device: Network
OpenStep Configuration: AMD PCnet-32 PCI Ethernet Adapter
VirtualBox Configuration: Bridged Adapter, PCnet-PCI II, Promiscuous Mode All
Observations: This works fine. Using SimpleNetworkStarter I was able to give OpenStep an IP address on my subnet, using my real router and real DNS servers. This allowed OpenStep to be ‘seen’ on the subnet. Standard networking facilities such as FTP and NFS work. It may help to run the a command such as the following from the VirtualBox installation directory, where “OpenStep” is whatever you name the virtual machine and “192.168.1.0” depends on your local subnet:

VBoxManage modifyvm OpenStep --natnet1 "192.168.1.0/24"

As I had named the VM ‘OPENSTEP4.2’ in VirtualBox Manager, I used the following command:

$ VBoxManage modifyvm OPENSTEP4.2 --natnet1 "192.168.1.0/24"

However I doubt this made any difference, because I had set the VM’s network adapter to ‘Bridged Adapter’ in the VirtualBox Manager, not ‘NAT’. I had to select ‘Bridged Adapter’ because I could not get the VM to connect to the network otherwise.

I also made sure the adapter in the VirtualBox Manager is set to ‘PCnet-PCI II (Am79C970A)’ and Promiscuous Mode is set to ‘Allow All’.

In addition to the network configuration notes in the OPENSTEP installation tutorial I mentioned earlier, for information only see the old tutorial ‘NeXTStep/OpenStep Ethernet-Based Network Configuration For Cable Modems, DSL, LANs, Etc…‘.

Anyway, below is what I did to get NFS working. The crucial thing to note is that OPENSTEP 4.2 uses NFSv2. I spent many hours unsuccessfully trying to get NFS working between the NFS server (a machine with IP address 192.168.1.74) and the NFS client (a VM with IP address 192.168.63) until I realised this. The NFS server is running Lubuntu 18.04, which uses NFSv4 by default. Therefore I had to configure the NFS server to use NFSv2 as well. Not only that, but I had to configure NFSv2 to use static ports, because the ports can change randomly in NFSv2 which would stop NFS working if there is a firewall enabled on the host machine.

In the NFS server (Lubuntu 18.04 running on a desktop machine)

N.B. My NFS server is running in Lubuntu 18.04 on a machine with an IP address of 192.168.1.74, and my NFS client is running in OPENSTEP 4.2 on a VM with IP address of 192.168.1.63. Change the IP addresses below to suit your situation.

1. Install the NFS server software

$ sudo apt-get update
$ sudo apt-get install nfs-kernel-server

2. Create a mountpoint for the NFS shared directory

$ sudo mkdir /var/nfs
$ sudo chown nobody:nogroup /var/nfs
$ sudo chmod 777 /var/nfs

3. Configure the NFS export

$ sudo nano /etc/exports

3.1 Choose which of the following types of share you want to have

3.1.1 Less secure:

/home/fitzcarraldo/nfsshare 192.168.1.63(rw,sync,no_root_squash,no_subtree_check)

If ‘no_root_squash‘ is used, remote root users are able to change any file on the shared file system and leave trojaned applications for other users to inadvertently execute.

3.1.2 More secure:

/var/nfs 192.168.1.63(rw,sync,no_subtree_check)

3.2 Update the current table of exports for the NFS server

$ sudo exportfs -a

You can check the current table settings:

$ sudo exportfs -s
/home/fitzcarraldo/nfsshare  192.168.1.63(rw,wdelay,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
/var/nfs  192.168.1.63(rw,wdelay,root_squash,no_subtree_check,sec=sys,rw,secure,root_squash,no_all_squash)

If you wanted to clear the table (unexport the shared directories) you would do:

$ sudo exportfs -u 192.168.1.63:/home/fitzcarraldo/nfsshare
$ sudo exportfs -u 192.168.1.63:/var/nfs
$ sudo exportfs -s
$

4. Load the NFSv2 kernel module

If lockd is built as a module (which it is in Lubuntu 18.04), create file /etc/modprobe.d/nfsv2.conf containing the following:

options lockd.nlm_udpport=4001 lockd.nlm_tcpport=4001

$ sudo modprobe nfsv2

If you want to make that permanent so it happens automatically when booting/rebooting add ‘nfsv2‘ (without the quotes) to the file /etc/modules-load.d/modules.conf (which in Lubuntu 18.04 is symlinked to /etc/modules).

5. Configure the NFS server

See ‘How can I make the nfs server support protocol version 2 in Ubuntu 17.10?‘.

Edit /etc/default/nfs-kernel-server to include NFSv2 and to specify static ports:

$ sudo nano /etc/default/nfs-kernel-server

# Number of servers to start up
RPCNFSDCOUNT=8

# Runtime priority of server (see nice(1))
RPCNFSDPRIORITY=0

# Options for rpc.mountd.
# If you have a port-based firewall, you might want to set up
# a fixed port here using the --port option. For more information, 
# see rpc.mountd(8) or http://wiki.debian.org/SecuringNFS
# To disable NFSv4 on the server, specify '--no-nfs-version 4' here
RPCMOUNTDOPTS="--manage-gids -p 32767"
# -p 32767 above added by Fitzcarraldo

# Do you want to start the svcgssd daemon? It is only required for Kerberos
# exports. Valid alternatives are "yes" and "no"; the default is "no".
NEED_SVCGSSD=""

# Options for rpc.svcgssd.
RPCSVCGSSDOPTS=""

# All options below this comment were added by Fitzcarraldo
#
# Options to pass to rpc.statd
# ex. RPCSTATDOPTS="-p 32765 -o 32766"
RPCSTATDOPTS="-p 32765 -o 32766"
#
# Options to pass to rpc.rquotad
# ex. RPCRQUOTADOPTS="-p 32764"
RPCRQUOTADOPTS="-p 32764"
#
RPCNFSDOPTS="--nfs-version 2,3,4 --debug --syslog"
#
# To confirm above mods are in effect after service restart use
#    cat /run/sysconfig/nfs-utils
#  or 
#    service nfs-kernel-server status
#

Edit /etc/default/nfs-common to specify static ports for rpc-statd:

# If you do not set values for the NEED_ options, they will be attempted
# autodetected; this should be sufficient for most people. Valid alternatives
# for the NEED_ options are "yes" and "no".


# Options for rpc.statd.
#   Should rpc.statd listen on a specific port? This is especially useful
#   when you have a port-based firewall. To use a fixed port, set this
#   this variable to a statd argument like: "--port 4000 --outgoing-port 4001".
#   For more information, see rpc.statd(8) or http://wiki.debian.org/SecuringNFS
STATDOPTS="-o 32766 -p 32765"
# -o 32766 -p 32765 above were added by Fitzcarraldo

# Do you want to start the gssd daemon? It is required for Kerberos mounts.
NEED_GSSD=

(I had to edit /etc/default/nfs-common to specify the ports for rpc-statd in STATDOPTS because specifying the ports in RPCSTATDOPTS in /etc/default/nfs-kernel-server did not make the status ports static.)

Edit /etc/sysctl.conf to add a static port mapping for lockd:

$ sudo nano /etc/sysctl.conf

[...]
# All lines below added by Fitzcarraldo
# TCP Port for lock manager
fs.nfs.nlm_tcpport = 4001
# UDP Port for lock manager
fs.nfs.nlm_udpport = 4001

Modify the lockd kernel parameters now during runtime rather than having to reboot:

$ sudo sysctl -p

Note that it is necessary to specify static ports in the configuration files so that tight rules can be added to the firewall in the NFS server.

6. Start the NFS server

Either the sysvinit way, which still works in Lubuntu 18.04:

$ sudo service nfs-kernel-server start

or the systemd way, which also works in Lubuntu 18.04:

sudo systemctl start nfs-kernel-server

If you want, you could enable the service so it starts automatically after the system is rebooted:

$ sudo systemctl enable nfs-kernel-server

7. Start the NSM (Network Status Monitor) daemon

Either the sysvinit way, which still works in Lubuntu 18.04:

$ sudo service rpc-statd start

or the systemd way, which also works in Lubuntu 18.04:

$ sudo systemctl start rpc-statd

If you want, you could enable the service so it starts automatically after the system is rebooted:

$ sudo systemctl enable rpc-statd

8. Check that NFSv2 is running and the ports are the ones specified in the config files

$ rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100005    1   udp  32767  mountd
    100005    1   tcp  32767  mountd
    100005    2   udp  32767  mountd
    100005    2   tcp  32767  mountd
    100005    3   udp  32767  mountd
    100005    3   tcp  32767  mountd
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100227    2   tcp   2049
    100227    3   tcp   2049
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100227    2   udp   2049
    100227    3   udp   2049
    100021    1   udp   4001  nlockmgr
    100021    3   udp   4001  nlockmgr
    100021    4   udp   4001  nlockmgr
    100021    1   tcp   4001  nlockmgr
    100021    3   tcp   4001  nlockmgr
    100021    4   tcp   4001  nlockmgr
    100024    1   udp  32765  status
    100024    1   tcp  32765  status

9. Configure the firewall in Lubuntu 18.04

I used Gufw (LXDE Menu > ‘Preferences’ > ‘Firewall Configuration’) to add the following two UFW rules:

111,2049,4001,32765:32768/udp ALLOW IN 192.168.1.0/24
111,2049,4001,32765:32768/tcp ALLOW IN 192.168.1.0/24

The above rules permit NFSv2 to function consistently because I had configured the NFS ports to be static. If I had not done that the firewall would sometimes stop NFS from working because NFSv2 ports change randomly otherwise.

In OPENSTEP running in the VM

10. Make sure basic networking has been configured

I navigated to ‘openstep’ > ‘NextAdmin’ > ‘SimpleNetworkStartup.app’ and did the following:

• Unticked ‘Maintain the master copy of network administrative data.’
• Selected ‘Use the network, but don’t share administrative data.’
• Entered the Hostname ‘openstep‘ (no quotes) and IP address 192.168.1.63.
• Clicked on ‘Network Options…’. In the window that opened I did the following:
  • Made sure router IP is set to 192.168.1.254
  • Made sure NIS Domain Name is set to ‘None’
  • Made sure Netmask is set to 255.255.255.0
  • Made sure Broadcast Address is set to 192.168.1.255
  • ‘Limit access to local NetInfo data to the local network’ is unticked.
  • Clicked on ‘Set’.
• Clicked on ‘Configure’.

11. Create the shared NFS director[y,ies]

N.B. I could probably have created the directory /mnt/nfs/nfsshare and/or /mnt/nfs/var/nfs (whichever you chose to create — see 3.1 above) using ‘openstep’ > ‘NextAdmin’ > ‘NFSManager.app’ instead of using the command line, but I opened a Terminal window in OPENSTEP and did the following:

openstep> su
openstep:1# mkdir /mnt
openstep:2# mkdir /mnt/nfs
openstep:3# mkdir /mnt/nfs/nfsshare
openstep:4# mkdir /mnt/nfs/var
openstep:5# mkdir /mnt/nfs/var/nfs

12. Mount the NFS share(s)

openstep:6# mount 192.168.1.74:/home/fitzcarraldo/nfsshare /mnt/nfs/nfsshare
openstep:7# mount 192.168.1.74:/var/nfs /mnt/nfs/var/nfs

Use the df command to check they are mounted correctly:

openstep:8# df

13. Test the shared director[y,ies]

In Lubuntu on the machine with hostname ‘aspirexc600‘, copy a file into /var/nfs/ (or /home/fitzcarraldo/nfsshare/). You should see it appear in /mnt/nfs/var/nfs/ (or /mnt/nfs/nfsshare/) in OPENSTEP in the VM with hostname ‘openstep‘.

In OPENSTEP on the VM with hostname ‘openstep‘, copy a file into /mnt/nfs/var/nfs/ (not /mnt/nfs/nfsshare/, as that will not be allowed). You should see it appear in /var/nfs/ in Linux in the machine with hostname ‘aspirexc600‘.

In Lubuntu on the machine with hostname ‘aspirexc600‘, delete the file in /var/nfs/ and you should see it removed from /mnt/nfs/var/nfs/ in OPENSTEP on the VM with hostname ‘openstep‘.

In Lubuntu on the machine with hostname ‘aspirexc600‘, delete the file in /home/fitzcarraldo/nfsshare/ and you should see it removed from /mnt/nfs/nfsshare/ in OPENSTEP on the VM with hostname ‘openstep‘.

14. If you later want to unmount the NFS shared folder(s)

openstep:9# umount /mnt/nfs/nfsshare
openstep:10# umount /mnt/nfs/var/nfs

15. If you want OPENSTEP to mount the NFS shared folder(s) automatically when it boots

I was unable to get OPENSTEP to mount NFS shared folders automatically at boot by adding the appropriate lines in /etc/fstab, but OPENSTEP does mount them automatically if I add the mount commands to /etc/rc.local like so:

#!/bin/sh -u
#
# This script is for augmenting the standard system startup commands. It is 
# executed automatically by the system during boot up. 
#
# Copyright (C) 1993 by NeXT Computer, Inc.  All rights reserved.
#
# In its released form, this script does nothing. You may customize
# it as you wish.
#

fbshow -B -I "Starting local services" -z 92

# Read in configuration information
. /etc/hostconfig

# (echo -n 'local daemons:')                                    >/dev/console
#
# Run your own commands here
mount 192.168.1.74:/var/nfs /mnt/nfs/var/nfs
mount 192.168.1.74:/home/fitzcarraldo/nfsshare /mnt/nfs/nfsshare
#
# (echo '.')                                                    >/dev/console

File sharing: Summary

So, I managed to get NFS working, albeit not using OPENSTEP’s NFSManager.app tool. Had I known more about OPENSTEP networking I probably could have used the OPENSTEP GUI utilities to configure NFS, but at least I have proved it is possible to copy files to and from an NFS server (which happens to be the host machine of the VM) running Lubuntu 18.04 and the VM running OPENSTEP 4.2. Mind you, NFSv2 is old. NFSv4 would be the protocol to use had OPENSTEP supported it. Also, bear in mind that NFSv2 cannot encrypt the connection, so it is not secure. Another reason to have a good firewall enabled in the VirtualBox host machine and in my router too.

Conclusions

I have had fun installing and tinkering with OPENSTEP and its applications over the last few days. Getting file sharing to work was by far the most difficult part, but I got there in the end once I had discovered OPENSTEP only supports NFSv2. It is a pity OPENSTEP and the applications for it have not been developed for many years and are all obsolete. If development of OPENSTEP drivers, networking software, productivity applications and multimedia applications had continued, the OS itself would still have been perfectly usable on modern hardware, albeit not as straightforward to use as any of the main Desktop Environments in Linux. But the OS still feels quite modern; it was definitely ahead of its time. Tinkering with OPENSTEP 4.2 has given me a new respect for Steve Jobs, for the talented hardware and software engineers in the NeXT company, and indeed for Mac OS X and macOS. The choice of Unix for NEXTSTEP/OPENSTEP was truely inspired.

In this blog post I have not covered the sophisticated development tools for NEXTSTEP/OPENSTEP, which were also way ahead of their time. I’ll leave you to read the articles, documents and videos available on the Web about the development tools.

Please comment below if you notice any errors or omissions in this post, or if you know a better way of doing something in OPENSTEP, or you know of newer versions of the OPENSTEP software than the versions I have mentioned. I’d also be interested to hear from anyone who has a NeXT machine and/or is still using one; let me know what you have and how you’re using it.

Useful links

These are just a few of the many Web pages and sites I browsed when installing OPENSTEP 4.2 and looking for applications and ways to get various things to work.

• The Best of NeXT
• Shaw’s NEXTSTEP & OPENSTEP Resource Page
• The NEXTSTEP/OpenStep FAQ
• NeXT-FAQ.txt
• Commonly Asked (by Mac Users) Questions About NextStep/OpenStep Systems (I am not a Mac user and have very little knowledge about Macs and macOS, but this might be of help to Mac users.)

Documentation

• NEXTSTEP User’s Guide
• Quick Start to Using the OPENSTEP Desktop
• Using the OPENSTEP Desktop
• NEXTSTEP 3.3 Network and System Administration Manual online

Software repositories

Sometimes differences between NEXTSTEP and OPENSTEP may mean a NETSCAPE application cannot be installed in OPENSTEP or, if it can, may not work. Furthermore, be aware that different revisions of the same application/utility exist online, so you need to try and find the latest revisions.

• Omni Group apps for NEXTSTEP and OPENSTEP
• Index of /NeXTfiles/Software/OPENSTEP
• Index of /OpenStep

Installing Dropbox in Gentoo Linux following the recent restrictions introduced for Dropbox for Linux

In a 2013 post I explained how I installed Dropbox in Gentoo Linux running KDE 4. The Dropbox company has recently imposed some restrictions in the Linux client, so this is to explain what I did to get Dropbox working again in my two Gentoo Linux installations, both using the ext4 filesystem (unencrypted) and, these days, KDE Plasma 5.

Both my laptops running Gentoo Linux had a version of Dropbox installed via the Portage package manager: dropbox-45.3.88 in the case of the laptop running Gentoo amd64, and dropbox-48.3.56 in the case of the laptop running Gentoo ~amd64. Recently a Dropbox window popped up, warning me to upgrade Dropbox to the latest version within seven days otherwise the client would no longer be able to sync with the remote Dropbox server. I also received an e-mail from the Dropbox company titled ‘[Action required] We’re updating Linux system requirements‘ informing me that the only supported Linux distributions from now on would be Ubuntu 14.04 or higher and Fedora 21 or higher, and furthermore that the client will only work on an unencrypted ext4 filesystem. As both my Gentoo installations use unencrypted ext4, I was OK on that score, but I still had the problem that an up-to-date Dropbox ebuild is not available for Gentoo and the old Dropbox versions I was using no longer sync. However, I managed to install the latest version of Dropbox (currently 55.4.171) in Gentoo, and it works fine. The Dropbox client’s icon is on the KDE Plasma 5 Panel, and the local Dropbox directory is being sync’ed correctly. Below I explain what I did.

1. I selected ‘Quit Dropbox’ from the old Dropbox client’s menu, and the Dropbox icon disappeared from the Panel.

2. I removed the Dropbox daemon from the list of script files to be started at login (‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’).

3. I unmerged (uninstalled) the dropbox package:

clevow230ss /home/fitzcarraldo # emerge --ask --depclean dropbox

4. I deleted the directories ~/.dropbox and ~/.dropbox-dist but kept the directory ~/Dropbox and its contents.

fitzcarraldo@clevow230ss ~ $ rm -rf ~/.dropbox ~/.dropbox-dist

5. I followed the instructions under ‘Dropbox Headless install via command line‘ on the Dropbox Website to re-install the latest version of the daemon and client:

fitzcarraldo@clevow230ss ~ $ cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf -

6. I configured KDE Plasma 5 to start ~/.dropbox-dist/dropboxd at login (‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’ > ‘Add Script…’).

7. I launched ~/.dropbox-dist/dropboxd manually from a Konsole window. The Dropbox client icon appeared on the Panel and I was prompted to login to my Dropbox account via a Web browser, as per the instructions on the Dropbox Website (see link in in Step 5):

If you’re running Dropbox on your server for the first time, you’ll be asked to copy and paste a link in a working browser to create a new account or add your server to an existing account. Once you do, your Dropbox folder will be created in your home directory.

8. I logged in to my Dropbox account via the Firefox browser. As soon as I had logged in via the browser, a message appeared in the browser window informing me that “Your computer was successfully linked to your account”, and the Dropbox client icon appeared on the Panel and showed that the contents of ~/Dropbox were being synchronised.

Everything seems to be working as before. The Dropbox icon on the Panel has the same menu items it had previously. ‘Preferences…’ shows the Dropbox version as v55.4.171. I have not ticked ‘Start Dropbox on system startup’ under Dropbox Preferences because I configured automatic startup using KDE Plasma 5 ‘System Settings’ as described in Step 6 above, and the Dropbox daemon is indeed started automatically when I login.

The Dropbox Website’s instructions (see link in Step 5) also include the following:

Download this Python script to control Dropbox from the command line. For easy access, put a symlink to the script anywhere in your PATH.

I did download that Python script and made it executable:

fitzcarraldo@clevow230ss ~/Dropbox $ chmod +x dropbox.py

However the Python 3.6 interpreter in my Gentoo Linux installations report a syntax error in the script when I run it, I assume because it was written for a different version of Python:

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py 
  File "./dropbox.py", line 233
    except OSError, e:
                  ^
SyntaxError: invalid syntax

Anyway, as the Dropbox client icon is on the KDE Plasma 5 Panel and I can control Dropbox from there, I see no need for the Python script.

9. My Gentoo installations have a Bash script ~/dbox.sh that I had created to be launched by a Desktop Configuration file ~/Desktop/Dropbox.desktop with a nice icon which I double-click on if I want to relaunch the Dropbox daemon (if I previously quit Dropbox from the client’s menu, for example). I had to modify ~/dbox.sh by replacing the command ‘dbus-launch dropbox start > /dev/null‘ with the command ‘/home/fitzcarraldo/.dropbox-dist/dropboxd‘ as shown below.

dbox.sh

#!/bin/bash
notify-send 'Launching Dropbox' 'Daemon will be (re)started in 20 seconds' --icon=dialog-information
sleep 20s
ps auxww | awk '$0~/dropbox/&&$0!~/awk/{print $2}' | xargs kill
/home/fitzcarraldo/.dropbox-dist/dropboxd

Dropbox.desktop

[Desktop Entry]
Comment[en_GB]=(re)launch Dropbox daemon
Comment=(re)launch Dropbox daemon
Exec=/home/fitzcarraldo/dbox.sh
GenericName[en_GB]=Dropbox
GenericName=Dropbox
Icon=kipi-dropbox
MimeType=
Name[en_GB]=Dropbox
Name=Dropbox
Path=
StartupNotify=true
Terminal=false
TerminalOptions=
Type=Application
X-DBUS-ServiceName=
X-DBUS-StartupType=none
X-KDE-SubstituteUID=false
X-KDE-Username=fitzcarraldo

10. At the moment Dropbox is working fine again in my Gentoo installations. However, I noticed that Gentoo Linux user zsitvaij posted the following comment in a Gentoo Forums thread:

On every dropbox update, I have to remove ~/.dropbox-dist/dropbox-lnx./libdrm.so.2 to avoid having it crash on launch, works fine after until they update again.

I do not know if that will be necessary in my case, as I have not yet had to upgrade Dropbox from the Version 55.4.171 that I recently installed. When a new version of Dropbox becomes available I will update this post to confirm whether or not I had to do anything to keep Dropbox working.

Addendum (5 October 2019): With reference to my addendum of 31 August 2018, the Python script dropbox.py that can be downloaded from the Dropbox Web site has been updated and is now written in Python 3, so you can ignore my addendum of 31 August 2018.

Addendum (1 October 2018): With reference to my addendum of 2 September 2018, if you are using OpenRC it is possible to automate the deletion of the file ~/.dropbox-dist/dropbox-lnx.x86_64-/libdrm.so.2 by creating a Bash script /etc/local.d/40dropbox.start containing the following:

#!/bin/bash
if [ -e /home/fitzcarraldo/.dropbox-dist/dropbox-lnx.x86_64-*/libdrm.so.2 ]
then
    rm /home/fitzcarraldo/.dropbox-dist/dropbox-lnx.x86_64-*/libdrm.so.2
fi

Replace my username with your username, obviously. Of course the conditional test could be dispensed with and the script could just contain the shebang line and the rm line, which would still work even if the file does not exist, but it feels a bit tidier to only attempt to delete the file if it actually exists.

Addendum (2 September 2018): I have just installed Dropbox Version 56.4.94 in my Gentoo ~amd64 installation and I had to use the command shown below once in order to stop the daemon segfaulting when I entered the command ~/.dropbox-dist/dropboxd in a Konsole window:

fitzcarraldo@clevow230ss ~/Dropbox $ rm ~/.dropbox-dist/dropbox-lnx.x86_64-56.4.94/libdrm.so.2

Addendum (31 August 2018): The Python script dropbox.py that can be downloaded from the Dropbox Web site (see Step 8 above) is old, as can be seen in the comments in the header of the script:

# Dropbox frontend script
# This file is part of nautilus-dropbox 2015.10.28.

It is written in Python 2. Although I do not need to use it, I managed to get it to run in my Gentoo installations by replacing the shebang line ‘#!/usr/bin/python‘ with ‘#!/usr/bin/env python2‘. This works in my Gentoo installations because they have both Python 2.7 and Python 3.6 installed. When I now run dropbox.py I see the following:

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py 
Dropbox command-line interface

commands:

Note: use dropbox help  to view usage for a specific command.

 status       get current status of the dropboxd
 throttle     set bandwidth limits for Dropbox
 help         provide help
 stop         stop dropboxd
 running      return whether dropbox is running
 start        start dropboxd
 filestatus   get current sync status of one or more files
 ls           list directory contents with current sync status
 autostart    automatically start dropbox at login
 exclude      ignores/excludes a directory from syncing
 lansync      enables or disables LAN sync
 sharelink    get a shared link for a file in your dropbox
 proxy        set proxy settings for Dropbox

fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py status
Up to date
fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py running
fitzcarraldo@clevow230ss ~/Dropbox $ ./dropbox.py filestatus ~/Dropbox/Getting\ Started.pdf 
/home/fitzcarraldo/Dropbox/Getting Started.pdf: up to date
fitzcarraldo@clevow230ss ~/Dropbox $

Notice that the command ./dropbox.py running does not return anything even though the daemon is definitely running, so I do not trust the script anyway.