‘Server not found’ by browser at launch

I haven’t had any significant Linux problems or new requirements in the last few months, hence no new posts here. My last real problem was back in June 2013 when I rolled my Gentoo installation to latest using Portage and found that, whenever I launched Firefox, it displayed the ‘Server not found’ page and I had to click ‘Try Again’, and then Firefox displayed the expected Web site. From then onwards, Firefox would work as expected until I exited the application. Thunderbird was also unable to access e-mail servers on the first attempt after it was launched. The same thing happened in Sabayon Linux when I rolled to latest using Entropy a couple of days later. Anyway, here is how I fixed the problem in both distributions.

First I used Wireshark to see what was going on, and it transpired that Gentoo (and Sabayon) was sending an IPv4 request followed quickly by an IPv6 request, but the reply to the IPv6 request was being received first and was a ‘server not found’ message since my ISP does not support IPv6 and my router apparently does not handle IPv6 requests correctly. Gentoo (and Sabayon) then used an IPv4 address when I clicked ‘Try Again’ in the browser window, and thereafter Firefox always dispayed the expected Web sites.

I should point out that IPv6 is enabled in the kernels I use and I’ve never before had to disable IPv6 in Firefox (or system-wide) on the affected laptops. So why the change in functionality, I wonder?

With Wireshark capturing packets, when I launched Firefox I was seeing a server failure message indicating “AAAA” (IPv6) instead of “A” (IPv4). To stop this happening I could have chosen any one of the three following solutions:

1. I could have used about:config in Firefox (and Config Editor in Thunderbird) to change the value of network.dns.disableIPv6 to true instead of false.

2. I could have disabled IPv6 system-wide by editing /etc/modprobe.d/aliases.conf and uncommenting the line “alias net-pf-10 off“.

3. I could have forced the getaddrinfo() function in glibc to make the IPv4 and IPv6 requests sequentially rather than in parallel.

Just for the fun of it I chose the third option on a couple of my laptops, and, as they use NetworkManager, this is how I did it:

fitzcarraldo@aspire5536 ~ $ su
Password:
aspire5536 fitzcarraldo # cat /etc/resolv.conf
# Generated by resolvconf
domain home
nameserver 192.168.1.254
aspire5536 fitzcarraldo # cd /etc/NetworkManager/dispatcher.d/
aspire5536 dispatcher.d # touch 06-dhclientoptions
aspire5536 dispatcher.d # nano 06-dhclientoptions
aspire5536 dispatcher.d # cat 06-dhclientoptions
#!/bin/bash
echo "options single-request" >> /etc/resolv.conf
aspire5536 dispatcher.d # chmod +x /etc/NetworkManager/dispatcher.d/06-dhclientoptions
aspire5536 dispatcher.d # # Now I disconnect then reconnect to the network
aspire5536 dispatcher.d # cat /etc/resolv.conf
# Generated by resolvconf
domain home
nameserver 192.168.1.254
options single-request
aspire5536 dispatcher.d #

As you can see above, I added a two-line Bash script 06-dhclientoptions in the directory /etc/NetworkManager/dispatcher.d/ that appends the line “options single-request” (without the quotes) to the contents of the file /etc/resolv.conf. The addition of the line “options single-request” in resolve.conf causes the getaddrinfo() function in glibc to make the IPv4 and IPv6 requests sequentially rather than in parallel. With this change, Firefox and Thunderbird no longer have a problem accessing the Internet the first time they are launched.

From “man 5 resolv.conf” under “options”:

single-request (since glibc 2.10)
sets RES_SNGLKUP in _res.options. By default, glibc performs IPv4 and IPv6 lookups in parallel since version 2.9. Some appliance DNS servers cannot handle these queries properly and make the requests time out. This option disables the behavior and makes glibc perform the IPv6 and IPv4 requests sequentially (at the cost of some slowdown of the resolving process).

single-request-reopen (since glibc 2.9)
The resolver uses the same socket for the A and AAAA requests. Some hardware mistakenly sends back only one reply. When that happens the client system will sit and wait for the second reply. Turning this option on changes this behavior so that if two requests from the same port are not handled correctly it will close the socket and open a new one before sending the second request.

I had to use NetworkManagerDispatcher to add the line “options single-request” to /etc/resolv.conf because NetworkManager overwrites /etc/resolv.conf if you edit it manually.

UPDATE (February 4, 2014): As I have recently seen the line “options single-request” occurring more than once in the file /etc/resolv.conf I now recommend /etc/NetworkManager/dispatcher.d/06-dhclientoptions consists of the following:

#!/bin/bash
if grep -q "options single-request" /etc/resolv.conf; then
    exit
else
    echo "options single-request" >> /etc/resolv.conf
fi

Synchronise your Gentoo Linux clock with an Internet time server

There are a number of ways to synchronise Gentoo Linux with a time server on the Internet. Here I look at a few alternatives.

ntp-client

ntp-client and the NTP daemon ntpd are installed when you install the package net-misc/ntp. Although I have read on some Web sites that /etc/init.d/ntp-client should be added to the default runlevel in order to read the time from an NTP server (once-only, during start-up), this in fact does not work because usually the network connection is not up by the time the ntp-client initscript runs. Bear in mind that ntp-client does not run continuously; it syncs once with an external time server if there is a network connection, and that’s it.

NetworkManager Dispatcher

If you are using NetworkManager, an elegant solution is to use NetworkManagerDispatcher to restart ntp-client in order to resync your system clock every time a network connection comes up. This is my favoured solution for laptops; see further on for how to configure your machine to do this.

cronjob

Another way would be to create a cronjob to run periodically the ‘/etc/init.d/ntp-client restart‘ command or the ‘ntpd -q‘ command (the -q option means “set the time and quit”).

Wait a while after start up

A ‘quick-and-dirty’ method, which I have used sometimes to synchronise a laptop’s system clock every time it boots, would be to delay running ntp-client until the network is up by putting e.g. the command below in a file 10_ntp-client.start in the directory /etc/local.d/ (10 seconds is usually enough time for a wired or wireless connection to my home network to be established):

#!/bin/bash
sleep 10s && /etc/init.d/ntp-client restart

Don’t forget to make it executable:

# chmod 744 /etc/local.d/10_ntp-client.start

NTP daemon

Regarding the NTP daemon, it is possible to configure this from the command line, rather than via a Desktop Environment GUI, to run at start-up and continue running to adjust your system clock. The command:

# rc-update add ntpd default

will add the daemon’s initscript to the default runlevel so that it is launched automatically at the next startup, and the command:

# /etc/init.d/ntpd start

will start the daemon running right now.

Note that, by default, the NTP daemon won’t correct, all in one go, a time difference between your system clock and the remote NTP server if that difference is above a certain size. However, if you want to override the default behaviour, i.e. allow the NTP daemon to make a large first adjustment to the system clock, you can set the environment variable NTPD_OPTS in the file /etc/conf.d/ntpd as follows:

NTPD_OPTS="-g"
# The -g option enables ntpd to make large adjustments.

This would mean that you would not need to run ntp-client before ntpd. However, if you run ntp-client automatically — either once after start-up or periodically — then that would be good enough for the typical Desktop user, and could be an alternative to having a continuously-running NTP daemon. Nothing stops you doing both if you want, of course.

Updating the hardware clock

If you make clock_systohc="YES" in the file /etc/conf.d/hwclock then the time in the system clock will be written to the BIOS (CMOS) clock (a.k.a. hardware clock) when you shut down your PC.

How to configure NetworkManager Dispatcher to synchronise the system clock only when a network connection is made

If you’re using a machine that is permanently connected to a network, running the NTP daemon makes sense. But what if you have a machine that is not always connected to a network when it is powered up? I have a laptop and I don’t want the NTP daemon running all the time. But I would like my laptop to synchronise with an external time server once after start up when I connect to the Internet. NetworkManager has a handy tool called NetworkManager Dispatcher for doing just this.

If you have installed NetworkManager, you’ll find there is an initscript /usr/portage/net-misc/networkmanager/files/NetworkManagerDispatcher. Copy it to the directory /etc/init.d/ and give it the necessary restrictive permissions:

# cp /usr/portage/net-misc/networkmanager/files/NetworkManagerDispatcher /etc/init.d/
# chmod 744 NetworkManagerDispatcher

Then create a shell script called e.g. 99_ntp-client in the directory /etc/NetworkManager/dispatcher.d/ to be run by NetworkManagerDispatcher when a network connection is established, containing the following code:

#!/bin/bash

INTERFACE=$1 # The interface which is brought up or down
STATUS=$2 # The new state of the interface

case "$STATUS" in
    'up') # $INTERFACE is up
        echo "System time before starting ntp-client:" > /home/fitzcarraldo/ntp-client.txt
        date >> /home/fitzcarraldo/ntp-client.txt
        echo "Starting ntp-client:" >> /home/fitzcarraldo/ntp-client.txt
        rc-config restart ntp-client &>> /home/fitzcarraldo/ntp-client.txt
        echo "System time after starting ntp-client:" >> /home/fitzcarraldo/ntp-client.txt
        date >> /home/fitzcarraldo/ntp-client.txt
        ;;
    'down') # $INTERFACE is down
        # Check for active interface and down if no one active
        if [ ! `nm-tool|grep State|cut -f2 -d' '` = "connected" ]; then
                echo "Stopping ntp-client at:" > /home/fitzcarraldo/ntp-client.txt
                date >> /home/fitzcarraldo/ntp-client.txt
                rc-config stop ntp-client &>> /home/fitzcarraldo/ntp-client.txt
        fi
        ;;
esac

Make the root user the owner of the script, and only allow the root user to write to it and execute it:

# cd /etc/NetworkManager/dispatcher.d/
# chown root:root 99_ntp-client
# chmod 744 99_ntp-client

Then add NetworkManagerDispatcher to the default runlevel so that it will be launched every time you boot your machine:

# rc-update add NetworkManagerDispatcher default

As the package net-misc/ntp installs both /etc/init.d/ntpd and /etc/init.d/net-client, users could optionally add the NTP daemon ntpd to the default runlevel too if desired, which would provide continuous, incremental adjustments to the system clock once net-client has done its one-shot adjustment each time a network comes up:

# rc-update add ntpd default

But users who don’t leave their PCs on for days on end — or who use laptops — can ignore the above step and just stick with the NetworkManagerDispatcher and net-client solution, whereas users who leave their machines on for days or weeks on end can also use the NTP daemon to keep the system clock in sync in between the times when ntp-client has synchronised.

Don’t forget to delete ntp-client from the start-up level if you are using NetworkManagerDispatcher to run it:

# rc-update del ntp-client

Notice that the script /etc/NetworkManager/dispatcher.d/99_ntp-client logs some information in a text file ntp-client.txt in my home directory which I can check. Here is an example of what ntp-client.txt contains after I select a network (or it is selected automatically) following start up of my laptop:

System time before starting ntp-client:
Sun Jun 3 19:24:08 BST 2012
Starting ntp-client:
Restarting init script
* Setting clock via the NTP client 'ntpd' ...ntpd: time slew +0.067178s
[ ok ]
System time after starting ntp-client:
Sun Jun 3 19:24:17 BST 2012

As you can see above, the ntpd command was executed once by NetworkManagerDispatcher and made a small adjustment to the system time on my laptop.

Replacing ntpdate with ntpd in ntp-client

Just for the fun of it, I changed /etc/conf.d/ntp-client to use the command ntpd instead of ntpdate, even though the ntpdate command works fine. Anyway, here’s my /etc/conf.d/ntp-client file these days:

NTPCLIENT_CMD="ntpd"
NTPCLIENT_OPTS="-g -q"

I have added the -g option so that the ntpd command can make large adjustments to the system time if it is way off the actual time. This is useful at the beginning and end of Daylight Saving Time, or if you dual boot with Windows. Here is an example of the former when I powered up my laptop the morning after the clocks changed from BST to GMT at the end of Summer 2010:

$ cat /home/fitzcarraldo/ntp-client.txt
System time before starting ntp-client:
Sun Oct 31 09:37:23 GMT 2010
Starting ntp-client:
Starting init script
* Setting clock via the NTP client 'ntpd'...ntpd: time set -3600.122381s
[ ok ]
System time after starting ntp-client:
Sun Oct 31 08:37:30 GMT 2010

You can specify the NTP server or NTP server pool in the file /etc/ntp.conf, but the default server pool already specified in that file should work. Note again that, when ntpd is run with the -q option, it synchronises the system clock once and terminates, i.e. it is not running as a daemon.

A fast and easy-to-use scanner of IP addresses and ports

This week I’m working in offices with a local network that uses static IP addressing. Believe it or not, it seems the IT department does not keep a list of configured IP addresses, and sometimes inadvertently configure devices to have the same IP address.

A couple of days ago I was printing quite happily to a networked printer from my laptop in the morning but, when I got back from lunch, found that I couldn’t print any more. It transpired that someone had allocated the IP address of my laptop to a new plotter in my absence. What to do if the organisation has no central list of configured IP addresses? There are too many devices on the local network to check them manually, and I do not have access to them anyway.

So I installed Angry IP Scanner (a.k.a. ipscan), an easy-to-use scanner of IP addresses and ports, and use it to find an IP address that is currently unused on the network.

I wanted to install ipscan by using Gentoo’s package manager Portage. There is no ebuild for it in the Portage main tree, but a quick search of the Web told me that there is an ebuild in the floppym overlay (http://gpo.zugaina.org/Overlays/floppym/net-analyzer/ipscan). Rather than add the floppym overlay, I decided to install the package using a local overlay:

1. I downloaded the ebuild:

wget http://gpo.zugaina.org/AJAX/Ebuild/2251131/View --output-document=/home/fitzcarraldo/Desktop/ipscan-3.0_beta4.ebuild

The ebuild contents are as follows:

# Copyright 1999-2011 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

EAPI=3

inherit java-pkg-2

DESCRIPTION="Angry IP - The fast and friendly network scanner"
HOMEPAGE="http://www.angryip.org"

MY_PV=${PV/_/-}
SRC_BASE="mirror://sourceforge/${PN}/${PN}${PV:0:1}-binary/${MY_PV}"
SRC_URI="amd64? ( ${SRC_BASE}/${PN}-linux64-${MY_PV}.jar )
x86? ( ${SRC_BASE}/${PN}-linux-${MY_PV}.jar )"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~amd64 ~x86"
IUSE=""

DEPEND=""
RDEPEND=">=virtual/jre-1.6.0"

INSTALL_DIR=/opt/AngryIP

S=${WORKDIR}

src_unpack() {
:
}

src_install() {
local jarname
use amd64 && jarname="${PN}-linux64-${MY_PV}.jar"
use x86 && jarname="${PN}-linux-${MY_PV}.jar"
java-pkg_jarinto "${INSTALL_DIR}"
java-pkg_newjar "${DISTDIR}/${jarname}"
java-pkg_dolauncher
}

2. I synchronised the Portage main tree’s ebuilds on my PC with those in the repositories:

# emerge --sync

3. I made sure that /etc/make.conf has the following lines at the end of the file:

PORTDIR_OVERLAY="${PORTDIR_OVERLAY} /usr/local/portage/"
ACCEPT_LICENSE="*"

4. I created the required directories in the local overlay:

# mkdir -p /usr/local/portage/net-analyzer/ipscan

5. I made sure Portage would not nag me about a missing name for my local overlay:

# mkdir /usr/local/portage/profiles
# echo "local_overlay" > /usr/local/portage/profiles/repo_name

6. I copied the ipscan ebuild into the relevant directory in the local overlay:

# cd /usr/local/portage/net-analyzer/ipscan
# cp /home/fitzcarraldo/Desktop/ipscan-3.0_beta4.ebuild .

7. I generated the manifest for the package:

# ebuild ipscan-3.0_beta4.ebuild manifest

8. I merged the package:

# emerge -1v ipscan

9. I downloaded a 64×64 pixel PNG image of the Angry IP Scanner logo from http://www.angryip.org/icon.png. Then I right-clicked on Kickoff, selected Menu Editor and added an entry for Angry IP Scanner under Applications | System, although I can also launch ipscan from the command line if I want:

$ ipscan

To find an unused IP address on the local network which I can allocate to my laptop whilst I’m working in this office, I launch ipscan and enter the start and end IP addresses of the range I wish to check, and click on Start. The application quickly searches through the IP addresses in that range and lists which are in use and which are not. It’s as simple as that. I then use Network Management to edit the connection and enter the static IP address.

Note that Angry IP Scanner will not detect a live IP address if the device is behind a properly-configured firewall. See the Angry IP Scanner Web site’s FAQ and Documentation pages for further details.

Why can’t I access a specific Web site?

Is there one specific Web site that you can never access? That always causes your Web browser to stall when you click on a link to that site or enter the site’s URL in the browser’s address bar? Perhaps it’s an Internet banking site? You don’t have trouble with other sites, just this one particular site or perhaps just a couple of sites?

The chances are that this problem is caused by what is called a ‘black hole’ router somewhere between you and that site. For the technical explanation of a black hole router, see e.g. the article Black Hole (networking).

I experienced this precise problem a couple of days ago with the popular Web site IMDb. The old Gateway Solo 9300 laptop I was using has no built-in networking hardware so I use a couple of CardBus cards for wireless and wired network access. The Linksys WPC54G (EU) v7.1 wireless notebook adapter works fine but, irrespective of the Web browser, I could not access the IMDb Web site; the browser always stalled when trying to load the site. I discovered I also had the same problem with my bank’s Web site.

I could ping other sites by IP address and by domain name, but pinging the IMDb site never received a reply (and it still doesn’t, even after the fix explained below which allows browsing of the site).

I also experienced this problem several years ago under Windows XP when trying to access the Amazon UK Web site, using an MTU that I had incorrectly set to a value smaller than possible. The problem this time was due to a correctly-set MTU. In both cases, the two MTUs are smaller than a router somewhere en route to those sites wanted to handle.

BACKGROUND

Bear with me while I explain how the problem arose, as it’s relevant (and useful) information.

I had just installed Sabayon Linux on the laptop and got the wireless network connection up, and the first thing I then did was to launch Firefox. With an MTU of 1500 for wlan0, Firefox would only load Google (and that, only intermittently). All other sites would stall. So I used the following command iteratively in order to find the MTU for this hardware, which is 1464 (the maximum packet size in bytes that does not fragment + 28 bytes):

# ping -M do -s packet_size_in_bytes http://www.cisco.com

e.g.

# ping -M do -s 1472 http://www.cisco.com

I just varied the packet size in the ping command until “Frag needed and DF set” was not displayed.

I can reduce the MTU to 1464 as follows:

# ifconfig wlan0 mtu 1464

I could make this permanent by adding an entry (mtu_wlan0=”1464″) to the file /etc/conf.d/net, but I chose instead to make it permanent by adding the above ifconfig command to the file /etc/conf.d/local. This (correct) MTU works fine for all sites I have visited so far except for two, which just stall the browser: IMDb and my bank’s Web site.

I also have a Belkin F5D5010 CardBus Network Card (wired Ethernet) for the laptop. It works fine with an MTU of 1500, and there is no problem accessing the IMDb Web site in Firefox. So, apparently, the MTU is the problem when using the wireless network connection. Now let’s look at the solution…

THE SOLUTION

Well, in the case of Windows it is possible to edit the Registry to solve the problem of network ‘black holes’. But what do you do in the case of Linux? It turns out that the kernel /proc file system provides an easy way to enable and disable TCP MTU Probing by changing a value in the ‘file’ /proc/sys/net/ipv4/tcp_mtu_probing. A value of 0 = disabled; 1 = enabled when a black hole router is detected; 2 = always enabled.

This is what my laptop had in that ‘file':

# cat /proc/sys/net/ipv4/tcp_mtu_probing
0

So all I needed to do was:

# echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

and, bingo, a browser can access IMDb and my bank’s Web site without a problem. To make it permanent, I just added the above command to the file /etc/conf.d/local so that it is executed every time I boot the laptop. Here is what my file /etc/conf.d/local looks like now:

# Here is where you can put anything you need to start
# that there is not an init script for.

local_start() {
# This is a good place to load any misc programs
# on startup (use &>/dev/null to hide output)

# START OF MY ADDITIONS
# The Linksys CardBus card does not work with a MTU greater than 1464:
ifconfig wlan0 mtu 1464
# Enable TCP MTU Probing in order to deal with black hole routers:
echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing
# END OF MY ADDITIONS

# We should always return 0
return 0
}

local_stop() {
# This is a good place to unload any misc.
# programs you started above.

# We should always return 0
return 0
}

UPDATE (May 13, 2011): OpenRC 0.8.0 and later in Gentoo and Sabayon Linux no longer use the file /etc/conf.d/local but instead require the commands to be in files in the directory /etc/local.d/. So I created a file /etc/local.d/01network.start containing the following lines:

# The Linksys CardBus card does not work with MTU greater than 1464:
ifconfig wlan0 mtu 1464
# Enable TCP MTU probing to deal with black hole routers:
echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

and made it executable:

# chmod +x /etc/local.d/01network.start

UPDATE (April 11, 2012): If “echo 2″ does not solve the problem, try “echo 1″ instead. The possible values are:

0 Do not perform PLPMTUD (Packetization Layer Path MTU Discovery)
1 Perform PLPMTUD only after detecting a ‘blackhole’.
2 Always perform PLPMTUD.

Follow

Get every new post delivered to your Inbox.

Join 51 other followers