HEIC image files in Linux

I was at an event recently where the attendees were asked to upload their camera and smartphone photos and videos to a shared Google Drive folder. Some of the uploaded photo files have a .HEIC (High Efficiency Image Container) extension, which I had not come across before. I have since learnt that these HEIC files were produced by iPhones running iOS 11, encoded using the HEIF (High Efficiency Image File) format. Apparently the HEIF format is superior to the JPEG format in a number of ways (see the links at the end of my post, especially the image examples given by Nokia), although it is subject to patents and therefore I believe there are certain constraints to coding image files in HEIC format. Anyway, I’ll leave you to read the fine print. My interest was simply because I wanted to be able to download the above-mentioned photo files and view them all in the file managers and image-viewing applications in Linux and Android on my various devices.

Now, I can browse and view the above-mentioned shared HEIC images in Google Drive in the Firefox and Chrome browsers in Linux, although an ownCloud site viewed using the same browsers displays the HEIC files as grey icons that can only be downloaded, not opened and viewed in the browser. I also found that Cirrus, the Android app for ownCloud that I use on my Galaxy Note 8 phone, cannot display HEIC photos either.

I downloaded the HEIC files to a machine running Lubuntu 18.04 and to a machine running Gentoo Linux. The file manager PCManFM in Lubuntu 18.04 displays grey icons rather than thumbnails for these HEIC files, and KDE’s Dolphin 18.08.3 file manager in Gentoo Linux displays green image icons rather than thumbnails for them. As far as Linux image viewers go, in Lubuntu 18.04 I find that GPicView 0.2.5 and Geeqie 1.4 cannot display HEIC images, and in Gentoo Linux KDE I find that GQview 2.1.5-r1, Okular 18.08.3 and Gwenview 18.08.3 cannot display HEIC images. So I set about converting all the HEIC files to JPG files. I managed to do this but needed to use a range of tools, as illustrated by a couple of examples below for Lubuntu 18.04 and Gentoo Linux. This post might seem long-winded but perhaps may be of help to Linux users coming across .HEIC files for the first time.

From the .HEIC files I had downloaded I picked one at random to try and convert to a JPG file: IMG_3706.HEIC. Its EXIF data confirms it is an HEIC file:

user $ exiftool IMG_3706.HEIC | grep "File Type"
File Type                       : HEIC
File Type Extension             : heic
user $ exiftool IMG_3706.HEIC | grep "Camera Model"
Camera Model Name               : iPhone 7 Plus

Several of the files with the .HEIC suffix that I downloaded were not real HEIC files according to their EXIF data:

user $ exiftool IMG_9474.HEIC | grep "File Type"
File Type                       : JPEG
File Type Extension             : jpg
user $ exiftool IMG_9474.HEIC | grep "Camera Model"
Camera Model Name               : iPhone 8

Those files were apparently treated as JPEG files by the tools I mention below, so I have omitted the results for those ‘false’ HEIC files.

Lubuntu 18.04

1. I installed the libheif example tools:

user $ sudo apt install libheif-examples

2. I used the heif-info command to check the file:

user $ heif-info IMG_3706.HEIC 
image: 3024x4032 (id=49), primary
  thumbnail: 240x320
  alpha channel: no
  depth channel: no

3. I tried to convert the file using the heif-convert command:

user $ heif-convert IMG_3706.HEIC IMG_3706.jpg
File contains 1 images
Written to IMG_3706.jpg

4. Apparently Imagemagick >=7.0.7-22 compiled with --with-libheif is supposed to be able to convert HEIC files to JPG. Anyway, I tried to convert the file using the current version of Imagemagick in Lubuntu 18.04 (the current package version is 8:6.9.7.4+dfsg-16ubuntu6.4):

user $ convert IMG_3706.HEIC IMG_3706a.jpg
convert-im6.q16: no decode delegate for this image format `HEIC' @ error/constitute.c/ReadImage/504.
convert-im6.q16: no images defined `IMG_3706a.jpg' @ error/convert.c/ConvertImageCommand/3258.

5. Apparently the GIMP >=2.10.2 supports HEIF by using heif-gimp-plugin. Anyway, I tried to open the file with the current version of the GIMP in Lubuntu 18.04 (the current package version is 2.8.22-1). The GIMP launches and pops-up a window with the title ‘GIMP Message’ containing the following message and an ‘OK’ button:

GIMP Message
Opening /home/fitzcarraldo/IMG_3706.HEIC’ failed: Unknown file type

6. I used the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/) in a browser window. This can load the file IMG_3706.HEIC (‘Browse…’ button) and convert it (‘Save image…’ button) to IMG_3706.jpeg.

Gentoo Linux with KDE 5

1. I installed the libheif example tools implicitly by re-merging Imagemagick with USE="heif", which installs libheif.

root # cat /etc/portage/package.use/imagemagick 
media-gfx/imagemagick heif
root # emerge imagemagick

2. I used the heif-info command to check the file:

user $ heif-info IMG_3706.HEIC  
image: 3024x4032 (id=49), primary
  thumbnail: 240x320
  alpha channel: no
  depth channel: no

3. I tried to convert the file using the heif-convert command:

user $ heif-convert IMG_3706.HEIC IMG_3706.jpg
File contains 1 images
Written to IMG_3706.jpg

4. I tried to convert the file using Imagemagick >=7.0.7-22 compiled with --with-libheif (Imagemagick merged with USE="heif"):

user $ convert IMG_3706.HEIC IMG_3706a.jpg
user $

So Imagemagick 7.0.8.16 in Gentoo has no trouble with the file IMG_3706.HEIC.

5. I tried to open the file with the GIMP >=2.10.2, which supports HEIF using heif-gimp-plugin (GIMP >=2.10.6-r1 with USE="heif" in the case of Gentoo Linux)

First I re-merged the GIMP with the heif USE flag:

root # cat /etc/portage/package.accept_keywords/gimp
=media-gfx/gimp-2.10.8-r1 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/libmypaint-1.3.0 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-gfx/mypaint-brushes-1.3.0-r1 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/gegl-0.4.12 ~amd64
# required by media-gfx/gimp-2.10.8-r1::gentoo
=media-libs/babl-0.1.60 ~amd64
root # cat /etc/portage/package.use/gimp
media-gfx/gimp heif
root # emerge -1vp gimp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ~] media-gfx/mypaint-brushes-1.3.0-r1:1.0::gentoo  2,390 KiB
[ebuild     U ~] media-libs/babl-0.1.60::gentoo [0.1.38::gentoo] USE="(-altivec)" CPU_FLAGS_X86="mmx sse sse2 sse3%* sse4_1 -f16c" 670 KiB
[ebuild  N     ] media-libs/gexiv2-0.10.8::gentoo  USE="-introspection -python -static-libs -test -vala" PYTHON_TARGETS="python2_7 python3_6 -python3_4 -python3_5" 620 KiB
[ebuild  NS   ~] media-libs/gegl-0.4.12:0.4::gentoo [0.2.0-r5:0::gentoo] USE="cairo ffmpeg introspection lcms openexr sdl svg tiff v4l -debug -jpeg2k -lensfun -libav -raw -test -umfpack -vala -webp" CPU_FLAGS_X86="mmx sse" 6,900 KiB
[ebuild  NS    ] media-libs/gegl-0.3.26:0.3::gentoo [0.2.0-r5:0::gentoo] USE="cairo ffmpeg introspection lcms openexr sdl svg tiff v4l -debug -jpeg2k -lensfun -raw -test -umfpack -vala -webp" CPU_FLAGS_X86="mmx sse" 6,378 KiB
[ebuild  N    ~] media-libs/libmypaint-1.3.0::gentoo  USE="gegl nls openmp -introspection" 428 KiB
[ebuild     U ~] media-gfx/gimp-2.10.8-r1:2::gentoo [2.8.22-r1:2::gentoo] USE="alsa heif%* mng openexr%* udev wmf -aalib (-altivec) (-aqua) -debug -doc -gnome -jpeg2k -postscript -python -smp -test -unwind% -vector-icons% -webp% -xpm (-bzip2%*) (-curl%) (-dbus%*) (-exif%*) (-jpeg%*) (-lcms%*) (-pdf%*) (-png%*) (-svg%*) (-tiff%*)" CPU_FLAGS_X86="mmx sse" PYTHON_TARGETS="python2_7" 31,206 KiB

Total: 7 packages (2 upgrades, 3 new, 2 in new slots), Size of downloads: 48,591 KiB

I then launched the GIMP and successfully opened the file IMG_3706.HEIC, and I was able to export it as IMG_3706.jpg.

6. As would be expected, the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/) behaves exactly the same in Gentoo Linux as it does in Lubuntu 18.04 (see earlier).

Summary

So there you have it; if the Linux file manager and/or image viewing applications you use cannot already handle HEIC files, the tools in Linux that I found may work are as follows:

  • heif-convert (from the package libheif-examples in Ubuntu/Lubuntu, or from from the package libheif in Gentoo).
  • Imagemagick (not every version).
  • The GIMP (not every version).
  • the online tool ‘libheif decoder demo’ (https://strukturag.github.io/libheif/).

I have not tried the copyright open-source code from Nokia (see link under Further Reading below), qt-heif-image-plugin and tifig (not in active development). If you have had success using another tool to convert HEIC files, please post a comment below for the benefit of other users, giving the name of the tool, the package name and version, and the Linux distribution (including release number, if not a rolling distribution).

Further reading

  1. Wikipedia – High Efficiency Image File Format
  2. Lifewire – What Are HEIF and HEIC, and Why Is Apple Using Them?
  3. Nokia – High Efficiency Image File Format (HEIF)
  4. libheif – a ISO/IEC 23008-12:2017 HEIF file format decoder and encoder
  5. askubuntu – Any app on Ubuntu to open HEIF (.heic, High Efficiency Image File Format) pictures?
Advertisements

How to create missing favicons for your bookmarks in Firefox Quantum for Linux

Prior to the advent of Firefox Quantum, several add-ons were available that enabled you to replace the favicons of bookmarks in Firefox, or to create custom favicons for bookmarks lacking a favicon. One of my favourite such add-ons was Favicon Picker 3. However, Firefox Quantum currently lacks an add-on that would enable you to insert a favicon of your choice for bookmarks without one. This has annoyed me for some time, as my bookmarks menu in Firefox includes several Web sites without favicons. Today I found a 2013 post ‘How to change Firefox bookmark icons (favicons) without extensions or addons!‘ for Windows predating Firefox Quantum. I have applied a similar procedure for Firefox Quantum in Linux in order to add a custom favicon to each bookmark in my bookmarks menu that did not already have a favicon. The procedure I used is given below.

1. Launch Firefox and enter ‘about:support’ (without the quotes) in the address bar.

2. Click on ‘Profile Directory’ > ‘Open Directory’ (which, in my case, is the directory /home/fitzcarraldo/.mozilla/firefox/l7nt0jx2.default).

3. Create the sub-directory ‘chrome‘.

4. Open the new directory’s folder.

5. Create the file ‘userChrome.css‘ in the new directory.

6. Open the file userChrome.css with a text editor and paste the following CSS code into it:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");

.bookmark-item[label="<bookmark name>"] image {
    width:0!important;
    height:0!important;
    padding: 0 0 16px 16px !important;
    background:url(<base64 string>)!important; 
}

7. Make the following changes to the above code:

7.1 Replace ‘<bookmark name>’ with the name of the bookmark as it appears in the bookmarks menu. You must supply a name, even if you do not want a name to appear.

7.2 Replace ‘<base64 string>’ with the base64-encoded string of the desired favicon, which you can obtain by uploading the 16×16 pixel image file to a website such as https://www.base64-image.de/.

Example

Below are the steps I took to create a favicon for the bookmark for Hotmail I have in Firefox:

1. I used a search engine to find an image of the Microsoft ‘flag’ logo. The image happened to be a 200×200 pixels PNG file. I downloaded it to my home directory and named it ‘Microsoft.png‘.

2. I loaded the image file into the GIMP, scaled it to 16×16 pixels and exported it as Microsoft.png to overwrite the original downloaded file.

3. I viewed the Web page https://www.base64-image.de/ in a Web browser and clicked on the button ‘OR CLICK HERE’ to upload my image file Microsoft.png to that site. Then I clicked on the button ‘</> show code’, selected and copied the text displayed underneath ‘For use as CSS background:’ that looked like this:

url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAOVBMVEXxURv7vAmAzCgAre/////xTxf8wB8csfD7wCXyXjDyWyuHzzaJ0Dohs/DyWSj0Zz2Q00b8xDMxt/HxkgKEAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH4goMEQsUc/RzwQAAAB5JREFUGNNjYIAAViYoYKCPACsUwAWYoYARCgZIAADfYwHMYVta3QAAAABJRU5ErkJggg==')

4. I created the directory ~/.mozilla/firefox/l7nt0jx2.default/chrome and the file userChrome.css containing the following:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");

.bookmark-item[label="Hotmail"] image {
    width:0!important;
    height:0!important;
    padding: 0 0 16px 16px !important;
    background:url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAOVBMVEXxURv7vAmAzCgAre/////xTxf8wB8csfD7wCXyXjDyWyuHzzaJ0Dohs/DyWSj0Zz2Q00b8xDMxt/HxkgKEAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH4goMEQsUc/RzwQAAAB5JREFUGNNjYIAAViYoYKCPACsUwAWYoYARCgZIAADfYwHMYVta3QAAAABJRU5ErkJggg==')!important; 
}

5. I restarted Firefox, and my custom favicon is now visible for the Hotmail entry in the Bookmarks Menu. I then deleted the file ~/Microsoft.png as it is no longer required.

6. I also wanted to create a custom favicon for another bookmark I have, the Web site of a Brazilian newspaper Correio Popular. As I was unable to find an image that would be suitable for a favicon, I created one myself (16×16 pixels) using the GIMP, and exported it as the file ‘~/Correio Popular.png‘.

7. I followed the same procedure to encode the image file as a Base64 image, and I added another entry to the file ~/.mozilla/firefox/l7nt0jx2.default/chrome/userChrome.css, which now contains the following:

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");

.bookmark-item[label="Hotmail"] image {
    width:0!important;
    height:0!important;
    padding: 0 0 16px 16px !important;
    background:url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAOVBMVEXxURv7vAmAzCgAre/////xTxf8wB8csfD7wCXyXjDyWyuHzzaJ0Dohs/DyWSj0Zz2Q00b8xDMxt/HxkgKEAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH4goMEQsUc/RzwQAAAB5JREFUGNNjYIAAViYoYKCPACsUwAWYoYARCgZIAADfYwHMYVta3QAAAABJRU5ErkJggg==')!important; 
}

.bookmark-item[label="Correio Popular"] image {
    width:0!important;
    height:0!important;
    padding: 0 0 16px 16px !important;
    background:url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAIAAACQkWg2AAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH4goMFzEqk7HMoQAAAQdJREFUKM/NkrGKg0AURd+EwSkMBPwLSxlI5dR+haSxFE0T7KxtghCwtfNH7FLZpkkXgvMDophwtzArYUMWUizsad7j8i7cC48BoE9Y0If8vYFPo+u6MAxXq9VyubxcLoyxsiz3+31VVUqp6/WqlIqiiIgIAADO+el0wje73W5a5oP1el0UBQACUBSF67p4ou97ALfbbTakaer7PoAFER2PR8dxnoMKIX5Er+taSvnoMI4j5/xdy+12ez6fpZRxHD86ZFnmed7vkWYIwDAMRKS1ntUkSQDc7/dXAyciwzC01kEQ2LZtmmbbtk3TENHhcBBC5Hm+2Wwsy5oSsne/BIAx9qqz//d8XzCj1kv+0nVJAAAAAElFTkSuQmCC')!important; 
}

8. For any other bookmarks that do not have a favicon, I will just repeat the procedure to add further entries to the file ~/.mozilla/firefox/l7nt0jx2.default/chrome/userChrome.css. It is not as easy as using an add-on, but, until someone creates one, the process described above at least allows me to fill those annoying favicon gaps in the Firefox bookmark menus on my machines.

Syncing browser bookmarks between browsers and machines in Linux

I use several computers and various browsers (predominantly Firefox, Chrome and Chromium) and was fed up with bookmarking a site on one machine and later not finding it on another machine. For quite some time I had therefore been looking for a simple way of synchronising browser bookmarks across all my machines and browsers, and I finally found one. Below I explain what I did.

I wanted to avoid storing my bookmarks on a third-party company’s server, so that ruled out tools such as Xmarks, EverSync, Google Bookmarks and the like. I wanted the bookmark database to reside on one of my own servers that is already accessible securely via the Internet. Apparently Xmarks optionally does enable you to use your own server providing you use only Firefox, but I use various browsers (Firefox is the default browser on my main laptop whereas Chrome is the default browser on my backup laptop, for example). Furthermore, I prefer to use open-source solutions whenever possible.

Although I was looking for a GUI solution, it turns out that the command-line bookmark manager Buku does a good job in a drop-down terminal such as Yakuake, Guake or Tilda. Buku is quite powerful, yet simple to use. It is certainly practical to use in a drop-down terminal (I’m currently using it with Yakuake in KDE, and with Tilda in LXDE). Not only can you click on links to open pages in the default browser, you can also easily configure your desktop environment to use a keyboard shortcut to bookmark directly from the browser window (see the instructions in the Buku Wiki for details).

Of course, if you only want to use Buku as a local bookmark manager on a machine, you can just install it and use it solely on that machine.

It is not difficult to set up a centralised Buku database that is then synchronised with any machine on which Buku is installed. If you do not have your own Cloud server (ownCloud or Nextcloud, for example), you could use Dropbox instead. The instructions are given in the Buku Wiki. Basically, I did the following to configure several machines to use Buku via the Cloud:

1. Use each browser’s bookmark manager to export the bookmarks to a file.

2. Install Buku on each machine (see ‘Installation‘ on the package’s GitHub repository page if your Linux distribution’s package manager does not offer Buku).

3. Launch Buku once on each machine to create the local database:

$ buku -p
DB file is being created at /home/fitzcarraldo/.local/share/buku/bookmarks.db.
You should encrypt it.
[ERROR] 0 records

4. On one machine, move the Buku database file (~/.local/share/buku/bookmarks.db) to a folder on the machine that is already being synced with the Cloud, then set up a symlink to it. For example:

fitzcarraldo@clevow230ss ~ $ ls -la ~/.local/share/buku/bookmarks.db
lrwxrwxrwx 1 fitzcarraldo fitzcarraldo 51 Mar 21 13:17 /home/fitzcarraldo/.local/share/buku/bookmarks.db -> /media/NTFS/Windows/ownCloud/Bookmarks/bookmarks.db

5. Allow the Cloud client on the other machines to download the bookmarks.db file into their local Cloud sync folder, then delete the local Buku database on each machine (~/.local/share/buku/bookmarks.db) and create a symlink to the Cloud-synchronised database file. For example, in addition to the symlink shown above on the machine clevow230ss, I have the following symlinks on two other machines:

fitzcarraldo@aspirexc600:~$ ls -la ~/.local/share/buku/bookmarks.db
lrwxrwxrwx 1 fitzcarraldo fitzcarraldo 42 Mar 21 16:05 /home/fitzcarraldo/.local/share/buku/bookmarks.db -> /home/fitzcarraldo/ownCloud/Bookmarks/bookmarks.db
fitzcarraldo@meshedgedx ~ $ ls -la /home/fitzcarraldo/.local/share/buku/bookmarks.db
lrwxrwxrwx 1 fitzcarraldo users 42 Mar 26 19:15 /home/fitzcarraldo/.local/share/buku/bookmarks.db -> /home/fitzcarraldo/ownCloud/Bookmarks/bookmarks.db

6. Use Buku on each machine to import the browser bookmark files that you created in Step 1. See the Buku documentation for the command. You can find documentation and a demo video on the above-mentioned GitHub page. The commands ‘man buku‘ and ‘buku --help‘ also list the commands. The man(ual) page also contains several examples to help you.

7. Use Buku as normal on each machine. You will be able to search the synchronised database, add bookmarks and edit them (title, URL, comment and tags), delete bookmarks, print bookmarks, click on links to view the pages in the default browser, and so on.

Looking through a flat list of bookmarks in a terminal window to find something is not as fast as in a GUI but, overall, Buku is a decent bookmark manager and its options are easy to learn and use. Buku’s comprehensive search options of course help to find bookmarks, but it is still not quite as ergonomic as a GUI bookmark manager in my opinion. The ability to have multiple tags in Buku does help, as you can search for either any or all tags. In a browser’s bookmark manager I would copy the same bookmark into different folders if the Web page covers multiple topics.

In summary, Buku is a viable bookmark manager and I like it. It is extremely easy to configure for use with a Cloud server, and I have set it up to synchronise bookmarks on all my machines. I have already imported into Buku the 1,300+ bookmarks from the various browsers on my machines, and deleted the bookmarks in those browsers, so I am using Buku in earnest. I just kept a few of the most-used bookmarks on the browser’s Bookmarks Toolbar, but I’m using Buku on my machines for all the other bookmarks.

If I do have to use a third-party machine running Windows or Linux without Buku installed, I would not be able to access my bookmarks from my Cloud server. To partially get around that, I created a cron job for my user account on each of my machines to periodically run Buku and print the bookmarks to a text file synced on my Cloud server. That way I can at least search through the text file remotely via the Cloud’s Web browser interface (or via WebDAV or via OpenVPN) if I cannot find the Web page I want in a search engine on the third-party machine.

fitzcarraldo@clevow230ss ~ $ crontab -l | grep -v \#
6,26,46 * * * * rm /media/NTFS/Windows/ownCloud/Bookmarks/*.txt; sleep 30s && /usr/bin/buku -p --nc > /media/NTFS/Windows/ownCloud/Bookmarks/Buku_bookmarks_backup.txt
fitzcarraldo@aspirexc600:~$ crontab -l | grep -v \#
1,21,41 * * * * rm /home/fitzcarraldo/ownCloud/Bookmarks/*.txt; sleep 30s && /usr/local/bin/buku -p --nc > /home/fitzcarraldo/ownCloud/Bookmarks/Buku_bookmarks_backup.txt
fitzcarraldo@meshedgedx ~ $ crontab -l | grep -v \#
11,31,51 * * * * rm /home/fitzcarraldo/ownCloud/Bookmarks/*.txt; sleep 30s && /usr/bin/buku -p --nc > /home/fitzcarraldo/ownCloud/Bookmarks/Buku_bookmarks_backup.txt

Below is a small taste of searching the bookmark database using Buku on any of my machines. Output is colour-coded (user-configurable), and links are clickable in a terminal window. You can search for any keyword(s), all keywords, sub-strings, just a tag or tags, regular expression matches, and so on. You can make titles immutable (read-only) if you want, or allow Buku to update them with the title from the Web site page. There is even a command that will check and list broken links. I will leave you to study the Buku documentation.

fitzcarraldo@aspirexc600:~$ buku -S Brazil samba
1. Kaká e Mário Monteiro são os novos carnavalescos da Imperatriz Leopoldinense [159]
   > http://www.sidneyrezende.com/editoria/carnaval
   +  Notícias sobre Carnaval 2016, escolas de samba, desfiles do Grupo Especial, Série A, ensaios técnicos, enredos, carnavalescos, bateria, mestre-sala, porta-bandeira, samba. Mangueira, Unidos da Tijuca, Vila Isabel, Beija-Flor, Grande Rio, Imperatriz, Mocidade, Portela, Salgueiro, União da Ilha, Viradouro, São Clemente, Porto da Pedra, Império da Tijuca, Império Serrano, Estácio de Sá, Caprichosos de Pilares, Tradição, Cubango, Em Cima da Hora, Inocentes de Belford Roxo, Alegria da Zona Sul, Unidos de Padre Miguel, Unidos de Bangu, Renascer de Jacarepaguá, Acadêmicos da Rocinha, Acadêmicos de Santa Cruz, Paraíso de Tuiuti, União de Jacarepaguá, União do Parque Curicica.
        
   # brazil,carnaval

2. Samba do Tuiuti 2018  Versão Acústica - YouTube [1270]
   > https://www.youtube.com/watch?v=yUxfwAzHOeY
   # brazil,carnaval,music,samba,video

buku (? for help) q

In this post I have only scratched the surface of what Buku can do. For example, a simple Buku command will encrypt (AES256) the bookmark database so you can prevent others viewing your bookmarks after you have finished searching the database, should you decide to store the database on a third-party Cloud server such as Dropbox. The search and editing tools are comprehensive yet straightforward, and you will quickly learn how to use them. I take my hat off to its developer, Arun Prakash Jana from Bangalore, India. He and the other contributors to Buku have done a great job, and I recommend you give Buku a try.

Stuttering audio in Linux: PulseAudio strikes again

I unmasked PulseAudio 10.0 back in January 2017 and installed it in my Gentoo Stable amd64 installation, and everything worked fine… until a couple of days ago, when the audio in streaming YouTube videos started to stutter every so often. It sounded rather like a scratched LP jumping. At first I thought the problem lay with Firefox, but the stuttering audio also occurred in Chrome. Then I wondered if my Internet connection was to blame; perhaps the ISP’s service had deteriorated. But a Windows 10 machine on my home network didn’t suffer from the problem, so that seemed to rule out the Internet connection. I tested the broadband throughput, and it was circa 32 Mbps, actually a little higher than the last time I tested it last year.

Now, Gentoo is a rolling distribution and I update my laptops regularly, but I couldn’t think what had been upgraded in the last couple of months that could be causing the problem. Although PulseAudio had not been upgraded since January, I began to wonder if PulseAudio could be involved, as my audio woes in the past have usually been due to PulseAudio.

I have always had PulseAudio installed with USE=”-realtime”:

user $ eix -I pulseaudio
[I] media-sound/pulseaudio
     Available versions:  10.0 {+X +alsa +alsa-plugin +asyncns bluetooth +caps dbus doc equalizer +gdbm +glib gnome gtk ipv6 jack libressl libsamplerate lirc native-headset neon ofono-headset +orc oss qt4 realtime selinux sox ssl system-wide systemd tcpd test +udev +webrtc-aec zeroconf ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32"}
     Installed versions:  10.0(16:07:53 19/04/17)(X alsa alsa-plugin asyncns bluetooth caps dbus gdbm glib gnome gtk ipv6 jack orc qt4 ssl tcpd udev webrtc-aec zeroconf -doc -equalizer -libressl -libsamplerate -lirc -native-headset -neon -ofono-headset -oss -realtime -selinux -sox -system-wide -systemd -test ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
     Homepage:            http://www.pulseaudio.org/
     Description:         A networked sound server with an advanced plugin system

but I wondered if PulseAudio’s real-time scheduling was somehow the cause of the problem, so I edited /etc/pulse/daemon.pa and added ‘realtime-scheduling = no‘ (I assume the default is ‘yes‘, as it was commented as such in the file):

; realtime-scheduling = yes
realtime-scheduling = no

Problem solved. PulseAudio is indeed a demon. 😡

Using the ClamAV daemon to scan files placed in my Downloads directory in Gentoo Linux

In a previous post I explained how to automatically detect files placed in my Downloads directory in Linux and scan them for viruses. The method I described in that post used clamscan, the command-line anti-virus scanner of ClamAV. Now, in addition ClamAV has a daemon (a program that runs continuously in the background), clamdscan, that you can enable. So I decided to switch to using clamdscan, as its response to downloaded files is much faster because the process waiting for new files to appear in ~/Downloads/ does not have to load clamscan from disk each time a new file arrives. Anyway, if you want to monitor a download directory in Gentoo Linux (running OpenRC) by using the ClamAV daemon — which will also download virus signature database updates automatically — then the procedure to set this up is given below.

1. Install clamav if it is not installed already:

root # emerge clamav

2. Add the service to the default runlevel:

root # rc-update add clamd default

The daemon will be launched automatically next time the computer boots.

3. The first download of the virus database has to be done manually:

root # freshclam

4. Start the daemon now:

root # rc-service clamd start

5. Create the Bash script ~/monitorDownloadsGUI with the following contents:

#!/bin/bash

DIR=$HOME/Downloads

# Get rid of old log file, if any
rm $HOME/virus-scan.log 2> /dev/null

IFS=$(echo -en "\n\b")

# Optionally, you can use shopt to avoid creating two processes due to the pipe
shopt -s lastpipe
inotifywait --quiet --monitor --event close_write,moved_to --recursive --format '%w%f' $DIR | while read FILE
# Added '--recursive' so that a directory copied into $DIR also triggers clamscan/clamdscan, although downloads
# from the Web would just be files, not directories.
do
     # Have to check file length is nonzero otherwise commands may be repeated
     if [ -s $FILE ]; then
          # Replace 'date >' with 'date >>' if you want to keep log file entries for previous scans.
          date > $HOME/virus-scan.log
          clamdscan --move=$HOME/virus-quarantine $FILE >> $HOME/virus-scan.log
          kdialog --title "Virus scan of $FILE" --msgbox "$(cat $HOME/virus-scan.log)"
     fi
done

Make it executable:

user $ chmod +x ~/monitorDownloadsGUI

6. Create the directory ~/virus-quarantine/ to store infected files pending investigation/deletion:

user $ mkdir ~/virus-quarantine

7. Install kdialog if it is not already installed:

root # emerge kdialog

8. Use ‘System Settings’ > ‘Startup and Shutdown’ > ‘Autostart’ to add the script ~/monitorDownloadsGUI to the list of script files that are automatically started each time you log in to KDE.

9. Log out then back in again, and you should see that everything is running as expected:

user $ rc-status | grep clam
 clamd                                                             [  started  ]

user $ ps -ef | grep clam | grep -v grep
clamav    1920     1  0 01:48 ?        00:00:00 /usr/sbin/clamd
clamav    1929     1  0 01:48 ?        00:00:00 /usr/bin/freshclam -d

user $ ps -ef | grep GUI | grep -v grep
fitzcarraldo      9143  8971  0 13:56 ?        00:00:00 /bin/bash /home/fitzcarraldo/.config/autostart-scripts/monitorDownloadsGUI.sh

10. To test, surf to http://www.eicar.org/85-0-Download.html and download one of the EICAR test files into your ~/Downloads/ directory. You should see a pop-up KDialog window with a message similar to the following:

Virus scan of /home/fitzcarraldo/Downloads/eicarcom2.zip — KDialog

Mon 27 Feb 14:05:26 GMT 2017
/home/fitzcarraldo/Downloads/eicarcom2.zip: Eicar-Test-Signature FOUND
/home/fitzcarraldo/Downloads/eicarcom2.zip: moved to ‘/home/fitzcarraldo/virus-quarantine/eicarcom2.zip’

———– SCAN SUMMARY ———–
Infected files: 1
Time: 0.001 sec (0 m 0 s)

Note that the above-mentioned pop-up window may be preceded by one or more pop-up windows with an error message. I’m using the Chrome browser at the moment, but you may get a similar message if you are using another browser. Here is an example:

Virus scan of /home/fitzcarraldo/Downloads/.com.google.Chrome.Uh3oGm — KDialog ?

Mon 27 Feb 14:16:30 GMT 2017
/home/fitzcarraldo/Downloads/.com.google.Chrome.Uh3oGm: Access denied. ERROR

———– SCAN SUMMARY ———–
Infected files: 0
Total errors: 1
Time: 0.000 sec (0 m 0 s)

Read the error message and click ‘OK’, as this is not an actual problem; it is inotifywait detecting temporary files in the ~/Downloads/ directory during the download process. With larger files sometimes several such messages are displayed, presumably because the file being downloaded is being opened and closed more than once during the downloading process. This issue does not occur if you copy or move a file into ~/Downloads/ from another directory in your installation; try it and see for yourself. Then you only get the one pop-up window with the scan result for the file you put in ~/Downloads/.

Also have a look in ~/virus-quarantine/ and you will see the EICAR test file in that directory. You can delete it if you want (it is not infected with a real virus, so does no harm).

In future be sure to read the messages in the pop-up windows before clicking ‘OK’, as they will inform you that an infected file has been moved to the quarantine directory.

That’s all there is to it. Very simple, and quite handy if you want to check quickly that files you download don’t have a malware payload. Just make sure you download all files into ~/Downloads/ or they will not be checked automatically. Also, if you are given e.g. a USB pen drive with a file on it, you can copy the file to ~/Downloads/ if you want it to be scanned for malware.

After updating Firefox for Linux, the folder icons in the bookmarks menu disappeared

Only a short post this time, but this problem has been annoying me for a few weeks. I use KDE 4.14 (kde-meta-4.14.3-r1) in Gentoo Linux on my main laptop, and recently upgraded Firefox to Version 46.0. The folder icons in Firefox’s bookmarks menu were no longer visible, although favicons were still visible in the bookmarks menu. The 2011 mozillaZine Forums thread After updating- ff4 bookmark folder icons disappeared [Linux] steered me in the right direction: I checked KDE 4’s ‘System Settings’ > ‘Application Appearance’ > ‘GTK’ and found that ‘Show icons in GTK menus’ was not ticked. I ticked this and clicked on ‘Apply’, and the problem was solved.

Automatically detecting files placed in my Downloads directory in Gentoo Linux and scanning them for viruses

I have been using Linux for almost a decade and have never been unduly concerned about viruses on my machines running Linux. However, I do receive files from people who use Windows and Mac OS, and some of those files might contain Windows or Mac OS viruses, so, as a matter of courtesy and assistance to others, it would make some sense to scan those files before passing them on. Furthermore, as I use some Windows applications under WINE, it would also make sense to scan received files for Windows viruses if I am going to use those files with a Windows application running under WINE.

External files could get into my Gentoo Linux installations via pen drives, memory cards, optical discs, e-mails, my Dropbox directory and downloads from Web sites. In this post I am going to concentrate on the last of these. All the various e-mail account providers I use already scan e-mails for viruses on their e-mail servers before I even download e-mail into the e-mail client on my laptop (standard practice these days), so e-mail is not a particular worry.

I have had ClamAV and its GUI, ClamTk, installed for a long time. Whilst ClamTk can be used to schedule a daily update of virus signatures and a daily scan of one’s home directory by ClamAV, I normally run ClamTk and ClamAV ad hoc. However, I can see some benefit in launching ClamAV automatically when I download a file from the Internet, so I decided to do the following …

Automatically scan a file downloaded via a Web browser

I use Firefox to browse the Web, and had configured it to download files to the directory /home/fitzcarraldo/Downloads/. I decided to monitor automatically the Downloads directory for the addition of any file. As I use the ext4 file system, the method I opted to use is inotify, specifically the inotifywait command which is available once you install the package sys-fs/inotify-tools.

It is surprisingly easy to create a shell script to detect files downloaded into a directory. The following script, running continuously in a terminal, would detect any files created in my /home/fitzcarraldo/Downloads directory, scan the new files with ClamAV and display a report in the terminal window:

#!/bin/bash

echo
DIR=$HOME/Downloads

inotifywait -q -m -e create --format '%w%f' $DIR | while read FILE
do
     date
     echo "File $FILE has been detected. Scanning it for viruses now ..."
     clamscan $FILE
     echo
done

A usable script would need to be a bit more sophisticated than the one shown above, because an existing file in the directory could be overwritten by one with the same name, or opened and amended. Furthermore, the script above would need a permanently open terminal window. Therefore I created a script to run in the background and use a GUI dialogue tool to pop up a window with the virus scanner’s report when the script detects a new or changed file in the Downloads directory. As this laptop has KDE 4 installed I opted to use KDialog to display the pop-up window, but I could instead have used Zenity. The final script is shown below.

#!/bin/bash

DIR=$HOME/Downloads

# Get rid of old log file
rm $HOME/virus-scan.log 2> /dev/null

inotifywait -q -m -e close_write,moved_to --format '%w%f' $DIR | while read FILE
do
     # Have to check file length is nonzero otherwise commands may be repeated
     if [ -s $FILE ]; then
          date > $HOME/virus-scan.log
          clamscan $FILE >> $HOME/virus-scan.log
          kdialog --title "Virus scan of $FILE" --msgbox "$(cat $HOME/virus-scan.log)"
     fi
done

Now when I download a file in Firefox, a window pops up, displaying a message similar to the following:

Virus scan of /home/fitzcarraldo/Downloads/eicar_com.zip – KDialog

Fri 19 Feb 23:42:02 GMT 2016
/home/fitzcarraldo/Downloads/eicar_com.zip: Eicar-Test-Signature FOUND

———– SCAN SUMMARY ———–
Known viruses: 4259980
Engine version: 0.98.7
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 4.595 sec (0 m 4 s)

Notice in the above message that ClamAV detected a virus in a file eicar_com.zip that I downloaded from the European Expert Group for IT Security Web site (originally ‘European Institute for Computer Antivirus Research’). In fact the executable eicar.com does not contain a real virus; it was designed to contain a known signature that virus scanner creators and users can use in checking anti-virus software. You can find out more about the virus test files on the EICAR Web site.

Of course, if I use applications other than Firefox to download files, I need to make sure they download the files into the applicable directory so that the script can detect and scan the files:

fitzcarraldo@clevow230ss ~ $ cd Downloads/
fitzcarraldo@clevow230ss ~/Downloads $ youtube-dl -o Carnavalito.mp4 -f 18 https://www.youtube.com/watch?v=ZDUL3w7zFD4
ZDUL3w7zFD4: Downloading webpage
ZDUL3w7zFD4: Downloading video info webpage
ZDUL3w7zFD4: Extracting video information
ZDUL3w7zFD4: Downloading MPD manifest
[download] Destination: Carnavalito.mp4
[download] 100% of 16.61MiB in 00:05

So, now I have a shell script that pops up a window informing me whether or not any file I put in $HOME/Downloads/ contains a virus. But I would like the script to be launched automatically when I login to the Desktop Environment. Therefore, as I use KDE 4, I selected ‘System Settings’ > ‘Startup and Shutdown’ and, in the ‘Autostart’ pane, clicked on ‘Add Script…’ and entered the path to my shell script (I left ‘create as symlink’ ticked). Now, every time I use KDE, any file placed (automatically or manually) into $HOME/Downloads/ is scanned for viruses automatically and a window pops up giving the result.

As my laptop is not always connected to the Internet, I prefer to update the ClamAV virus signatures database manually, which I do either using the ClamTk GUI or via the command line using the freshclam command:

fitzcarraldo@clevow230ss ~ $ su
Password:
clevow230ss fitzcarraldo # freshclam
ClamAV update process started at Sat Feb 20 10:51:01 2016
WARNING: Your ClamAV installation is OUTDATED!
WARNING: Local version: 0.98.7 Recommended version: 0.99
DON'T PANIC! Read http://www.clamav.net/support/faq
main.cvd is up to date (version: 55, sigs: 2424225, f-level: 60, builder: neo)
Downloading daily-21375.cdiff [100%]
Downloading daily-21376.cdiff [100%]
Downloading daily-21377.cdiff [100%]
Downloading daily-21378.cdiff [100%]
Downloading daily-21379.cdiff [100%]
Downloading daily-21380.cdiff [100%]
Downloading daily-21381.cdiff [100%]
Downloading daily-21382.cdiff [100%]
Downloading daily-21383.cdiff [100%]
Downloading daily-21384.cdiff [100%]
Downloading daily-21385.cdiff [100%]
Downloading daily-21386.cdiff [100%]
Downloading daily-21387.cdiff [100%]
Downloading daily-21388.cdiff [100%]
Downloading daily-21389.cdiff [100%]
Downloading daily-21390.cdiff [100%]
Downloading daily-21391.cdiff [100%]
daily.cld updated (version: 21391, sigs: 1850214, f-level: 63, builder: neo)
bytecode.cld is up to date (version: 271, sigs: 47, f-level: 63, builder: anvilleg)
Database updated (4274486 signatures) from db.UK.clamav.net (IP: 129.67.1.218)
WARNING: Clamd was NOT notified: Can't connect to clamd through /var/run/clamav/clamd.sock: No such file or directory

Not able to access the CUPS Printer Manager in Firefox when, or after, using a proxy server

Just a quick tip if you are trying to use the CUPS Printer Manager in a Firefox browser window (http://localhost:631/) to add a printer, manage a printer or view print jobs but the brower displays a message such as ‘Connecting to 10.48.80.49…’ and eventually times out and displays the following message:

 Network Error (tcp_error)

A communication error occurred: “Operation timed out”
The Web Server may be down, too busy, or experiencing other problems preventing it from responding to requests. You may wish to try again at a later time.

For assistance, contact your network support team.

or the following message:

Unable to connect

Firefox can’t establish a connection to the server at 10.48.80.49.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

    Try Again

This sometimes happens to me when my laptop is connected to a corporate network and the browser is, or recently was, configured to use a proxy server. Anyway, in Firefox I select ‘Open menu’ > ‘Preferences’ > ‘Network’ > ‘Settings…’ and make sure ‘No proxy’ is selected, then I select ‘Open menu’ > ‘Preferences’ and click on the ‘Clear Now’ button for ‘Cached Web Content’ (I have to click twice to clear the cache, for some reason) and the ‘Clear Now’ button for ‘Offline Web Content and User Data’. After doing this, I can access all the CUPS Printer Manager pages in the browser again.

Preventing a DNS Leak and WebRTC Leak when using Tor in Linux

Background

I have added to my 2011 Tor post a note on how to avoid a DNS Leak and WebRTC Leak, but am repeating it here in a new post, along with a Bash script that can be used to toggle the relevant Firefox user preferences before and after using Firefox with Tor, which makes the process easier.

The original eleven steps I gave in my above-mentioned post will not prevent the so-called DNS Leak problem. If your Web browser is not configured correctly it will still use your ISP’s DNS servers instead of the DNS servers favoured by Tor, in which case your ISP will know which sites you are accessing. See What is a DNS leak? for details. Reference 1 at the end of this post is a link to an article about DNS leakage, and Reference 2 is a link to an article on the Tor Browser, a browser designed to help avoid DNS leakage.

Furthermore, now that WebRTC is incorporated in some browsers, a ‘WebRTC Leak‘ is also possible if you have not configured your browser correctly.

Using the Tor Browser

Instead of performing Steps 1 to 11 in my original Tor post, download the Tor Browser, unpack it (no installation is required) and use that browser. Reference 3 below is a link to the download page, and Reference 4 below is a link to the instructions on how to unpack the tarball and launch the browser.

If you want even more security, you could instead download the ISO for the Tails Linux distribution, burn a LiveDVD or LivePenDrive — see my post Help for Windows users: How to create a Linux LiveCD, LiveDVD or LivePenDrive from an ISO file if you don’t know how to do that — and launch the browser from a Live Environment.

Using Tor with Firefox

However, if you still want to use the method I gave in my original Tor post then you could try all the additional steps given below to stop DNS leakage and WebRTC leakage.

  1. Use the OpenDNS servers instead of your ISP’s DNS servers. That will not help, though, if your ISP is using a Transparent DNS Proxy.
  2. Make the following changes to the preferences in Firefox (enter about:config in the Firefox address bar):
    Preference Name                       Status   Type     Value
    network.dns.disableIPv6               default  boolean  false  Change to true
    network.dns.disablePrefetch           default  boolean  false  Change to true
    network.proxy.socks_remote_dns        default  boolean  false  Change to true
    browser.safebrowsing.enabled          default  boolean  true   Change to false
    browser.safebrowsing.malware.enabled  default  boolean  true   Change to false
    media.peerconnection.enabled          default  boolean  true   Change to false
    

    (When you have finished using Tor, set media.peerconnection.enabled back to true if you want to use WebRTC. If you also want Firefox to warn you of phishing Web sites and Web sites that download malware, also set browser.safebrowsing.enabled and browser.safebrowsing.enabled back to true after you have finished using Tor.)

    You may be wondering why I disable IPv6 DNS requests. It is because some IPv6-capable DNS servers may return an IPv4 address when an IPv6 address is requested. I disable the two ‘safe browsing’ preferences because, if enabled, they cause Firefox to compare visited URLs against a remotely-stored blacklist or submit URLs to a third party to determine whether a site is legitimate, and I don’t want the possibility of Firefox contacting other sites outside Tor or trying to find an IP address for a URL. The PeerConnection preference relates to WebRTC, and I disable that to stop Firefox contacting STUN servers (see Reference 5 below).

  3. Test if there is still leakage by visiting the DNS leak test Web site and clicking on the Standard test button, and visiting the IP/DNS Detect site.

Furthermore, do not forget to use a Private Browsing window in Firefox.

Automate the editing of Firefox user preferences

Using about:config to change the user preferences in Firefox is laborious, so I created a Bash script edit_firefox.sh to toggle the relevant user preferences:

#!/bin/bash
# Script to change Firefox user preferences rather than
# using about:config from within Firefox.
# Make sure you only run this script when Firefox is not running.
#
FILE="/home/fitzcarraldo/.mozilla/firefox/fm8q09x0.default/prefs.js"
#
#
STATE=$(grep media.peerconnection.enabled $FILE | cut -c 43- | cut -d')' -f1)
if ! grep -q media.peerconnection.enabled $FILE ; then
  echo 'user_pref("media.peerconnection.enabled", false);' >> $FILE
  echo 'Added media.peerconnection.enabled false (secure) to prefs.js'
elif [ $STATE = "true" ]; then
     sed -i s/^.*media.peerconnection.enabled.*$/'user_pref("media.peerconnection.enabled", false);'/ $FILE
     echo 'media.peerconnection.enabled changed to false (secure) in prefs.js'
  else
     sed -i s/^.*media.peerconnection.enabled.*$/'user_pref("media.peerconnection.enabled", true);'/ $FILE
     echo 'media.peerconnection.enabled changed to true (not secure) in prefs.js'
fi
#
STATE=$(grep browser.safebrowsing.malware.enabled $FILE | cut -c 51- | cut -d')' -f1)
if ! grep -q browser.safebrowsing.malware.enabled $FILE ; then
  echo 'user_pref("browser.safebrowsing.malware.enabled", false);' >> $FILE
  echo 'Added browser.safebrowsing.malware.enabled false (secure) to prefs.js'
elif [ $STATE = "true" ]; then
     sed -i s/^.*browser.safebrowsing.malware.enabled.*$/'user_pref("browser.safebrowsing.malware.enabled", false);'/ $FILE
     echo 'browser.safebrowsing.malware.enabled changed to false (secure) in prefs.js'
  else
     sed -i s/^.*browser.safebrowsing.malware.enabled.*$/'user_pref("browser.safebrowsing.malware.enabled", true);'/ $FILE
     echo 'browser.safebrowsing.malware.enabled changed to true (not secure) in prefs.js'
fi
#
STATE=$(grep browser.safebrowsing.enabled $FILE | cut -c 43- | cut -d')' -f1)
if ! grep -q browser.safebrowsing.enabled $FILE ; then
  echo 'user_pref("browser.safebrowsing.enabled", false);' >> $FILE
  echo 'Added browser.safebrowsing.enabled false (secure) to prefs.js'
elif [ $STATE = "true" ]; then
     sed -i s/^.*browser.safebrowsing.enabled.*$/'user_pref("browser.safebrowsing.enabled", false);'/ $FILE
     echo 'browser.safebrowsing.enabled changed to false (secure) in prefs.js'
  else
     sed -i s/^.*browser.safebrowsing.enabled.*$/'user_pref("browser.safebrowsing.enabled", true);'/ $FILE
     echo 'browser.safebrowsing.enabled changed to true (not secure) in prefs.js'
fi
#
STATE=$(grep network.proxy.socks_remote_dns $FILE | cut -c 45- | cut -d')' -f1)
if ! grep -q network.proxy.socks_remote_dns $FILE ; then
  echo 'user_pref("network.proxy.socks_remote_dns", true);' >> $FILE
  echo 'Added network.proxy.socks_remote_dns true (secure) to prefs.js'
elif [ $STATE = "true" ]; then
     sed -i s/^.*network.proxy.socks_remote_dns.*$/'user_pref("network.proxy.socks_remote_dns", false);'/ $FILE
     echo 'network.proxy.socks_remote_dns changed to false (not secure) in prefs.js'
  else
     sed -i s/^.*network.proxy.socks_remote_dns.*$/'user_pref("network.proxy.socks_remote_dns", true);'/ $FILE
     echo 'network.proxy.socks_remote_dns changed to true (secure) in prefs.js'
fi
#
STATE=$(grep network.dns.disablePrefetch $FILE | cut -c 42- | cut -d')' -f1)
if ! grep -q network.dns.disablePrefetch $FILE ; then
  echo 'user_pref("network.dns.disablePrefetch", true);' >> $FILE
  echo 'Added network.dns.disablePrefetch true (secure) to prefs.js'
elif [ $STATE = "true" ]; then
     sed -i s/^.*network.dns.disablePrefetch.*$/'user_pref("network.dns.disablePrefetch", false);'/ $FILE
     echo 'network.dns.disablePrefetch changed to false (not secure) in prefs.js'
  else
     sed -i s/^.*network.dns.disablePrefetch.*$/'user_pref("network.dns.disablePrefetch", true);'/ $FILE
     echo 'network.dns.disablePrefetch changed to true (secure) in prefs.js'
fi
#
STATE=$(grep network.dns.disableIPv6 $FILE | cut -c 38- | cut -d')' -f1)
if ! grep -q network.dns.disableIPv6 $FILE ; then
  echo 'user_pref("network.dns.disableIPv6", true);' >> $FILE
  echo 'Added network.dns.disableIPv6 true (secure) to prefs.js'
elif [ $STATE = "true" ]; then
     sed -i s/^.*network.dns.disableIPv6.*$/'user_pref("network.dns.disableIPv6", false);'/ $FILE
     echo 'network.dns.disableIPv6 changed to false (not secure) in prefs.js'
  else
     sed -i s/^.*network.dns.disableIPv6.*$/'user_pref("network.dns.disableIPv6", true);'/ $FILE
     echo 'network.dns.disableIPv6 changed to true (secure) in prefs.js'
fi

You will need to change the path to the Firefox prefs.js file in the sixth line of the script, to suit your installation. If you have the utility mlocate installed you can find the file easily by using the command:

$ locate prefs.js | grep firefox

You will also need to make the script executable:

$ chmod +x edit_firefox.sh

You can see below how the script works:

$ ./edit_firefox.sh
media.peerconnection.enabled changed to false (secure) in prefs.js
browser.safebrowsing.malware.enabled changed to false (secure) in prefs.js
browser.safebrowsing.enabled changed to false (secure) in prefs.js
network.proxy.socks_remote_dns changed to true (secure) in prefs.js
network.dns.disablePrefetch changed to true (secure) in prefs.js
network.dns.disableIPv6 changed to true (secure) in prefs.js
$ ./edit_firefox.sh
media.peerconnection.enabled changed to true (not secure) in prefs.js
browser.safebrowsing.malware.enabled changed to true (not secure) in prefs.js
browser.safebrowsing.enabled changed to true (not secure) in prefs.js
network.proxy.socks_remote_dns changed to false (not secure) in prefs.js
network.dns.disablePrefetch changed to false (not secure) in prefs.js
network.dns.disableIPv6 changed to false (not secure) in prefs.js
$

Procedure to use Tor

So, if I am not using the Tor Browser, in summary I do the following (refer to my 2011 Tor post for the details):

  1. Launch Polipo from a Konsole window.
  2. Launch Vidalia from a Konsole window.
  3. Launch edit_firefox.sh to make sure the relevant user preferences are set securely.
  4. Launch Firefox and change the network settings to enable use of Polipo and Vidalia.
  5. Launch a Firefox Private Browsing window and close the original window.
  6. Visit TorCheck at Xenobite.eu, What Is My IP Address?, DNS leak test and IP/DNS Detect to be sure I am using Tor and that there is no DNS leak or WebRTC leak.

The router provided by my ISP does not allow me to change its DNS server settings. Using the router’s Web browser interface I was able to view the IP addresses of the DNS servers the router uses (Whois Lookup is a good place to check to whom an IP address belongs), and they are indeed owned by the ISP. However, the leak test Web sites I mention above show me that there is no DNS leakage to the ISP’s DNS servers when I have performed all the steps above.

When I have finished using Tor, I do the following:

  1. Exit Firefox.
  2. Stop Tor from the Vidalia GUI, exit Vidalia and end the Konsole session.
  3. Stop Polipo and end the Konsole session.
  4. Launch edit_firefox.sh to set the relevant user preferences back to their original settings.
  5. Launch Firefox and change the network settings back to the original settings.

References

1. Preventing Tor DNS Leaks
2. Tor new advice (February 2014)
3. Download Tor Browser
4. Linux Instructions for Tor Browser
5. New Browser Based Flaw Leaks VPN Users’ IP Addresses

Make Firefox for Linux use Dolphin to ‘Open Containing Folder’

When I click on the Download Manager icon on the tool bar, Firefox for Linux 32.0 opens a small pane listing downloads in progress, if any, and a link ‘Show All Downloads’. If I click on the link, Firefox pops up a window listing all the files downloaded via the browser, each with a folder icon beside it. Hovering the mouse pointer over the folder icon displays a tooltip ‘Open Containing Folder’. For as long as I can remember with Firefox for Linux, clicking that folder icon resulted in the Audacious music player launching and playing an MP4 file that happens to be in my ~/Downloads/ directory!

Firstly, I have no idea why Firefox was launching a media player instead of opening the directory. Secondly, I have no idea why Firefox wanted to open that specific file rather than any of the other files in the directory. Thirdly, I have no idea why it was launching Audacious, because Audacious is not even specified as the default media player for MP4 files in KDE’s ‘System Settings’ > ‘File Associations’.

This has annoyed me for a long time, but only today did I resolve to fix it, although it was not so easy to find a working solution by searching the Web. It seems it is a common problem with Firefox in Linux, and I found threads in various forums recommending the creation of a set of preferences by using about:config. Some of those threads state that one of those preferences should specify Konqueror; other threads state that one of the preferences should specify a shell script. In the end I discovered a post in an openSUSE Forums thread from 2012 Re: How use Dolphin to “open containing folder” from firefox downloads? providing a broken link to a thread at a different Web site and, fortunately, quoting the solution which worked for me, which is to create a file named ~/.local/share/applications/defaults.list containing the following:

[Default Applications]
x-directory/normal=kde4-dolphin.desktop;kde4-kfmclient_dir.desktop;

Now when I click on the ‘Open Containing Folder’ icon in Firefox, Dolphin launches and displays the contents of ~/Downloads/ just as I would have expected from the beginning.