Cannot browse some Web sites: yet another cause

Having spent hours yesterday trying to figure out why I could no longer browse a few specific Web sites from my old Gateway Solo 9300 laptop running Linux, whereas all the other Linux and Windows laptops that use my wireless network can browse any Web site, I thought I’d post the solution as Google did not turn up this particular cause.

There are several possible reasons why one may not be able to browse some Web sites: IPv6; the MTU; a black hole router en route; the settings in the various ip_* and tcp_* files in /proc/sys/net/ipv4/, and so on. So, unfortunately I had many avenues to pursue.

My old Gateway Solo 9300 laptop had enabled me to browse the Web reliably via my home wireless network until recently. Suddenly I could neither browse the Sabayon Linux Web site nor access the Entropy package manager’s repository. I could also no longer browse some other Web sites, although most Web sites were still accessible without trouble.

I tried many different things without success, and then finally remembered that I had reconfigured my router a couple of weeks ago because one of my family had reset it whilst I was away from home. Now, when this model of router (BT Home Hub, Version 1.0) is reset it defaults to WEP encryption for wireless networking. However, I normally have it set to use WPA-PSK encryption. So, when I got home I accessed the router’s Web interface via Ethernet from a Desktop PC and changed wireless networking encryption to “WPA-PSK Version: WPA+WPA2″. In the past I had set it to “WPA-PSK Version: WPA”, and thought that setting “WPA+WPA2″ instead of “WPA” this time would not cause any problems. Well, it didn’t on all the other laptops using my network, but my old Gateway Solo 9300 with Linksys CardBus card (‘Linksys WPC54G (EU) v7.1 wireless notebook adapter card’) was different.

As soon as I wondered if the encryption setting was the cause of my problem, I browsed to the BT Home Hub configuration page and changed wireless networking encryption from “WPA-PSK Version: WPA+WPA2″ to “WPA-PSK Version: WPA”. Bingo, my Gateway Solo 9300 running Sabayon Linux E17 Edition and Firefox 4 could browse all Web sites again. Mind you, I also need to have the ipv6 module disabled (I had previously uncommented the line alias net-pf-10 off in the file /etc/modprobe.d/aliases.conf and used the command update-modules) as well as the modifications mentioned in an earlier post (Why can’t I access a specific Web site?). But all those other measures do not solve the problem if the router is configured for “WPA-PSK Version: WPA+WPA2″.

The router’s wireless security option “WPA-PSK Version: WPA” is intended for a network that would only be used by clients configured to use WPA encryption; the router’s option “WPA-PSK Version: WPA2″ is intended for a network that would only be used by clients configured to use WPA2 encryption; the router’s option “WPA-PSK Version: WPA+WPA2″ is intended for a network that would be used by clients configured to use either WPA or WPA2 encryption. So I wonder why there was a problem, as the laptop and CardBus card are configured for “WPA & WPA2 Personal” in nm-applet, and the card supports WPA (TKIP) and WPA2 (CCMP a.k.a. AES).

Why can’t I access a specific Web site?

Is there one specific Web site that you can never access? That always causes your Web browser to stall when you click on a link to that site or enter the site’s URL in the browser’s address bar? Perhaps it’s an Internet banking site? You don’t have trouble with other sites, just this one particular site or perhaps just a couple of sites?

The chances are that this problem is caused by what is called a ‘black hole’ router somewhere between you and that site. For the technical explanation of a black hole router, see e.g. the article Black Hole (networking).

I experienced this precise problem a couple of days ago with the popular Web site IMDb. The old Gateway Solo 9300 laptop I was using has no built-in networking hardware so I use a couple of CardBus cards for wireless and wired network access. The Linksys WPC54G (EU) v7.1 wireless notebook adapter works fine but, irrespective of the Web browser, I could not access the IMDb Web site; the browser always stalled when trying to load the site. I discovered I also had the same problem with my bank’s Web site.

I could ping other sites by IP address and by domain name, but pinging the IMDb site never received a reply (and it still doesn’t, even after the fix explained below which allows browsing of the site).

I also experienced this problem several years ago under Windows XP when trying to access the Amazon UK Web site, using an MTU that I had incorrectly set to a value smaller than possible. The problem this time was due to a correctly-set MTU. In both cases, the two MTUs are smaller than a router somewhere en route to those sites wanted to handle.


Bear with me while I explain how the problem arose, as it’s relevant (and useful) information.

I had just installed Sabayon Linux on the laptop and got the wireless network connection up, and the first thing I then did was to launch Firefox. With an MTU of 1500 for wlan0, Firefox would only load Google (and that, only intermittently). All other sites would stall. So I used the following command iteratively in order to find the MTU for this hardware, which is 1464 (the maximum packet size in bytes that does not fragment + 28 bytes):

# ping -M do -s packet_size_in_bytes


# ping -M do -s 1472

I just varied the packet size in the ping command until “Frag needed and DF set” was not displayed.

I can reduce the MTU to 1464 as follows:

# ifconfig wlan0 mtu 1464

I could make this permanent by adding an entry (mtu_wlan0=”1464″) to the file /etc/conf.d/net, but I chose instead to make it permanent by adding the above ifconfig command to the file /etc/conf.d/local. This (correct) MTU works fine for all sites I have visited so far except for two, which just stall the browser: IMDb and my bank’s Web site.

I also have a Belkin F5D5010 CardBus Network Card (wired Ethernet) for the laptop. It works fine with an MTU of 1500, and there is no problem accessing the IMDb Web site in Firefox. So, apparently, the MTU is the problem when using the wireless network connection. Now let’s look at the solution…


Well, in the case of Windows it is possible to edit the Registry to solve the problem of network ‘black holes’. But what do you do in the case of Linux? It turns out that the kernel /proc file system provides an easy way to enable and disable TCP MTU Probing by changing a value in the ‘file’ /proc/sys/net/ipv4/tcp_mtu_probing. A value of 0 = disabled; 1 = enabled when a black hole router is detected; 2 = always enabled.

This is what my laptop had in that ‘file':

# cat /proc/sys/net/ipv4/tcp_mtu_probing

So all I needed to do was:

# echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

and, bingo, a browser can access IMDb and my bank’s Web site without a problem. To make it permanent, I just added the above command to the file /etc/conf.d/local so that it is executed every time I boot the laptop. Here is what my file /etc/conf.d/local looks like now:

# Here is where you can put anything you need to start
# that there is not an init script for.

local_start() {
# This is a good place to load any misc programs
# on startup (use &>/dev/null to hide output)

# The Linksys CardBus card does not work with a MTU greater than 1464:
ifconfig wlan0 mtu 1464
# Enable TCP MTU Probing in order to deal with black hole routers:
echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

# We should always return 0
return 0

local_stop() {
# This is a good place to unload any misc.
# programs you started above.

# We should always return 0
return 0

UPDATE (May 13, 2011): OpenRC 0.8.0 and later in Gentoo and Sabayon Linux no longer use the file /etc/conf.d/local but instead require the commands to be in files in the directory /etc/local.d/. So I created a file /etc/local.d/01network.start containing the following lines:

# The Linksys CardBus card does not work with MTU greater than 1464:
ifconfig wlan0 mtu 1464
# Enable TCP MTU probing to deal with black hole routers:
echo 2 > /proc/sys/net/ipv4/tcp_mtu_probing

and made it executable:

# chmod +x /etc/local.d/01network.start

UPDATE (April 11, 2012): If “echo 2″ does not solve the problem, try “echo 1″ instead. The possible values are:

0   Do not perform PLPMTUD (Packetization Layer Path MTU Discovery)
1   Perform PLPMTUD only after detecting a ‘blackhole’.
2   Always perform PLPMTUD.


Get every new post delivered to your Inbox.

Join 51 other followers