Thunderbird’s method of enabling anti-virus software to scan incoming e-mail messages is explained in the mozillaZine article ‘Download each e-mail to a separate file before adding to Inbox‘ and in Mozilla bug report no. 116443 (the bug report that resulted in the functionality being implemented). It is my contention that the design is deficient and is actually not a solution. In this post I explain why I believe this to be the case. Although here I will discuss Thunderbird in Linux, I believe the deficiency applies to Thunderbird in all OSs.
By default, Thunderbird inserts new incoming e-mail messages into an Inbox file. However, it is possible to configure Thunderbird to first create a temporary file containing each individual e-mail message in the /tmp directory, to enable external anti-virus software to scan each message before Thunderbird inserts it into the Inbox file. This approach only works for POP3 e-mail. The developers’ rationale for implementing this approach was to avoid the possibility of anti-virus software deleting or quarantining an entire Inbox.
In summary, if you want to scan incoming e-mails on your machine without running the risk of losing the entire Thunderbird Inbox, you must:
- configure Thunderbird so it creates temporary files
/tmp/newmsg* (each file contains a single e-mail message containing ASCII characters);
- configure the anti-virus software not to scan the directory containing the Inbox;
- configure the anti-virus software to scan the
/tmp directory.
Nevertheless, it seems Thunderbird developers would prefer you to disable local scanning of e-mail messages entirely: ‘mozillaZine – Email scanning – pros and cons‘.
Nowadays e-mail servers scan e-mail messages before you even download them. Some e-mail servers even send you an automated e-mail to inform you about an infected incoming e-mail or about an infected outgoing e-mail rejected by a receiving e-mail server. So local scanning of e-mail messages is far less important. Furthermore, I am not sure if the anti-virus software I use (ClamAV) is capable of detecting viruses in e-mail attachments encoded as ASCII characters. Anyway, purely out of curiosity I decided to investigate whether it would be possible to scan Thunderbird’s temporary files reliably.
To configure Thunderbird to create the temporary message files, it is necessary to select ‘Edit’ > ‘Preferences’ > ‘Security’ > ‘Antivirus’ and tick ‘Allow anti-virus clients to quarantine individual incoming messages’ (which sets mailnews.downloadToTempFile to true). Once that option has been selected, Thunderbird creates a temporary file /tmp/newmsg per message, which exists for a very brief (and inconstant) time before Thunderbird deletes it. When downloading several e-mail messages in very rapid succession, Thunderbird creates temporary files with different names (‘newmsg‘, ‘newmsg-1‘, ‘newmsg-2‘, ‘newmsg-3‘ and so on) to avoid overwriting messages, but usually one file named newmsg is sufficient to cater for the message download rate, as Thunderbird only keeps the temporary files for a very short time until it moves the message to the Inbox.
The problem with this approach is that Thunderbird does not provide any handshake mechanism to inform external anti-virus software that it has finished writing to a temporary file and that the file is available for scanning, nor does Thunderbird provide any handshake mechanism for external anti-virus software to inform Thunderbird when the scan of each temporary message file has finished (i.e. to tell Thunderbird that it can go ahead and delete the temporary file). In other words, the Thunderbird ‘solution’ is not a solution at all. In fact, I have found empirically that, if the anti-virus software is not fast enough, it can scan an incomplete temporary message file (i.e. the evaluation of the e-mail message would not be thorough and hence would be invalid). The Bash script below, for example, is sometimes able to scan an entire Thunderbird temporary file but at other times only manages to capture part of the file (it appears Thunderbird opens and closes the temporary file more than once) before Thunderbird deletes it:
#!/bin/bash
# This script only works with Thunderbird.
# This script only works for POP3 e-mail.
#
# You must configure Thunderbird to create temporary files /tmp/newmsg*.
# To do that, set Edit > Preferences > Security > Antivirus and tick
# 'Allow anti-virus clients to quarantine individual incoming messages'
# which sets mailnews.downloadToTempFile to true.
WORK=$HOME/clamtmp
mkdir $WORK 2> /dev/null
rm $WORK/* 2> /dev/null
counter=1
# Watch for newmsg* file(s) created by Thunderbird in /tmp
inotifywait -q -m -e close_write --format '%f' /tmp | while read FILE
do
if [ "${FILE:0:6}" = "newmsg" ] && [ -s /tmp/$FILE ]; then
TMPFILE=${counter}$FILE
cp -p /tmp/$FILE $WORK/$TMPFILE
# Do not let clamscan write temporary files to /tmp as inotifywait will detect them!
clamscan --tempdir=$WORK $WORK/$TMPFILE
counter=$((counter+1))
fi
done
Below is an example of an incomplete newmsg file that the above script copied to the directory $WORK when Thunderbird downloaded an e-mail message:
From - Sun Feb 21 09:40:58 2016
X-Account-Key: account8
X-UIDL: AAAAAKlA+Ah2LghLoJE4Le5Z5U0BAI04UNOj2gdNjwPO57yvmrIAATVHik4AAA==
X-Mozilla-Status: 0000
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:
I began to wonder if a valid scan would be possible if the script were to lock the temporary file (e.g. using the chattr +i command) until it has completed copying it. The use of the chattr command in the script means it has to be executed by the root user. When I first did that with a modified version of the script using KDialog to display the result of ClamAV’s scan, the following error message was displayed and the script aborted:
kdialog(xxxxxx)/kdeui (kdelibs): Session bus not found
To circumvent this problem try the following command (with Linux and bash)
export $(dbus-launch)
I therefore added the line ‘export $(dbus-launch)‘ to the script as follows:
#!/bin/bash
# This script must be launched by the root user.
export XAUTHORITY="/home/fitzcarraldo/.Xauthority"
export DISPLAY=":0"
export $(dbus-launch)
WORK=$HOME/clamtmp
mkdir $WORK 2> /dev/null
rm $WORK/* 2> /dev/null
inotifywait -q -m -e modify --format '%f' /tmp | while read FILE
do
# If file name begins with "newmsg" then copy it to work directory and scan it.
if [ "${FILE:0:6}" = "newmsg" ] && [ -f /tmp/$FILE ]; then
chattr +i /tmp/$FILE # Stop Thunderbird opening/deleting file.
cp -p /tmp/$FILE $WORK/
chattr -i /tmp/$FILE # Allow Thunderbird to open/delete file.
clamscan --tempdir=$WORK $WORK/$FILE >> $WORK/$FILE.log
kdialog --msgbox "$(cat $WORK/$FILE.log)"
fi
done
However, when the above script was running, Thunderbird displayed a pop-up window if it attempted to copy the temporary file to the Inbox before the script released the lock on the file:
There was an error copying the downloaded message from the temporary download file. Perhaps it contained a virus, or you are low on disk space.
From:
Subject: Test to see what happens if script locks newmsg file
Do you want to skip this message?
I clicked ‘No’ and the e-mail message in the file /tmp/newmsg was deleted without being copied to the Inbox, and the message was not deleted from the e-mail server. Subsequent attempts to re-download the message resulted in the same behaviour if the script had not finished processing the message before Thunderbird tried to move the message to the Inbox. Had I clicked ‘Yes’ I assume Thunderbird would simply have deleted the message on the mail server.
I did not bother looking into it further, but presumably the chattr command triggers inotifywait, in which case the script could cycle several times for the same file.
An approach that would probably work would be for Thunderbird to provide some sort of interlock so that it waits to delete newmsg* files until an anti-virus application gives the go-ahead.
An alternative approach would be for Thunderbird not to delete a temporary file after it writes the message to the Inbox and just leave it in the /tmp directory without overwriting it. An anti-virus application would quarantine infected temporary files and leave uninfected temporary files in /tmp, and therefore the anti-virus application would have to be written so that it deletes the temporary file once it has finished scanning it.
The second approach mentioned above would not be as good as the first approach for the following reasons:
- It would not stop Thunderbird adding a message to the Inbox (which would mean the user would have to delete a message manually from the Thunderbird Inbox if the virus scanner reported a message as infected).
- Thunderbird would have to use a different file name to any existing temporary files (at present it reuses ‘
newmsg‘ if a file of that name does not already exist).
- The user would have to ensure the temporary files do not accumulate ad infinitum. In my case, the contents of the
/tmp directory are deleted each time I reboot, but, in theory, a partition could become full if a user never switched off a machine and received a lot of e-mails.
Regarding the second reason listed above, Thunderbird already names the temporary files ‘newmsg‘, ‘newmsg-1‘, ‘newmsg-2‘ etc., so perhaps the existing Thunderbird code would automatically use a different file name if a file with the same name were still present, rather than overwriting it. If e.g. files newmsg and newmsg-2 happened to exist, I would hope Thunderbird would name the next temporary file ‘newmsg-1‘ or ‘newmsg-3‘.
I wondered if it would be possible to catch up with Thunderbird by just copying the temporary message files from the /tmp directory to another directory (see the script below), and then processing them afterwards with another script. However, even if a script just copies the temporary files to another directory without running ClamAV or KDialog, I found it is still not fast enough to catch all temporary files before Thunderbird deletes them. If Thunderbird downloads a single message and no others are waiting on the server(s) to be downloaded, it seems a script can copy the temporary messages successfully. However, if there are several messages waiting to be downloaded from the e-mail server(s) and Thunderbird downloads them in rapid succession, Thunderbird deletes some of the temporary messages before the script can copy them fully.
#!/bin/bash
WORK=$HOME/clamtmp
# Create work directory if it does not already exist
mkdir $WORK 2> /dev/null
# Delete old working files if they exist
rm $WORK/* 2> /dev/null
counter=1
inotifywait -q -m -e close_write --format '%f' /tmp | while read FILE
do
if [ "${FILE:0:6}" = "newmsg" ] && [ -s /tmp/$FILE ]; then
TMPFILE=${counter}$FILE
cp -p /tmp/$FILE $WORK/$TMPFILE
counter=$((counter+1))
fi
done
Consider the following six messages copied to $HOME/clamtmp by the above script (the script adds the first character to the name of the copied file):
-rw------- 1 fitzcarraldo fitzcarraldo 3829 Feb 23 02:24 1newmsg
-rw------- 1 fitzcarraldo fitzcarraldo 3107 Feb 23 02:25 2newmsg
-rw------- 1 fitzcarraldo fitzcarraldo 1158576 Feb 23 02:26 3newmsg
-rw------- 1 fitzcarraldo fitzcarraldo 237 Feb 23 02:28 4newmsg
-rw------- 1 fitzcarraldo fitzcarraldo 2106 Feb 23 02:28 5newmsg-1
-rw------- 1 fitzcarraldo fitzcarraldo 3107 Feb 23 02:28 6newmsg-2
The first three messages were each the sole message on all three e-mail servers accessed, and the copied files newmsg -> 1newmsg, newmsg -> 2newmsg and newmsg -> 3newmsg were all complete messages. However, the last three messages were on three e-mail servers simultaneously waiting to be downloaded, and when I clicked on ‘Get All New Messages’ in Thunderbird, the copied files newmsg -> 4newmsg, newmsg-1 -> 5newmsg-1 and newmsg-2 -> 6newmsg-2 were downloaded by Thunderbird in rapid succession. The copy 4newmsg was incomplete, the copy 5newmsg-1 was complete and the copy 6newmsg-2 was complete. So, even with a faster script, there is no guarantee that a script can catch all the temporary message files. Therefore, as I mentioned earlier, the only way to guarantee that temporary message files are properly scanned would be to modify Thunderbird to provide either a handshake (e.g. a file lock or inter-application flag) or to leave each temporary message file on /tmp and give it a unique file name.
The downside with both the above-mentioned approaches would be that the anti-virus software developer would need to know about the method, and write the software to perform the appropriate actions. If the first approach were adopted, the anti-virus software would need to signal to Thunderbird that it had completed scanning the file (e.g. by releasing a file lock or by an inter-application message). If the second approach were adopted, the anti-virus software would need to delete the message file from /tmp once it had completed scanning the file. The second approach would be easier for a simple Bash script to use, and, had the Thunderbird source code not been so complicated, I would have had a go at patching it to leave temporary message files in the /tmp directory after Thunderbird copies their contents to the Inbox file. But, as e-mail servers already do a good job of scanning messages before Thunderbird downloads them, I will not spend more time on this. Some e-mail servers even send an e-mail to the user informing them about an infected e-mail (see examples below), so it is not worth bothering.
Example 1
Automated e-mail server message to john@smith.com warning him that the e-mail with an infected attachment he sent to dave@hotmail.com was blocked by the receiving e-mail server.
Subject: Mail delivery failed: returning message to sender
Date: Wed, 17 Feb 2016 11:30:10 +0100
From: Mail Delivery System
To: john@smith.com
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of
its recipients. This is a permanent error. The following address
failed:
Reason:
virus/suspect content found
— The header of the original message is following. —
Example 2
Automated e-mail server message to john@smith.com warning him that an e-mail with an infected attachment sent to him by dave@hotmail.com was blocked by the receiving e-mail server.
Subject: VIRUS SUSPECTED: “Dave (Hotmail)”
Date: Wed, 17 Feb 2016 11:46:07 +0100 (CET)
From: Mail Delivery System
To: john@smith.com
A virus was detected in the following e-mail!
Mail details:
From: “Dave (Hotmail)”
TO: John Smith
Date: Wed, 17 Feb 2016 10:45:55 +0000
Subject: EICAR test file attachment
The concerned e-mail has been handled according to your Virus Protection Settings.
Sincerely
Your E-mail Service Provider Team
[ This is an automatically generated email, do not reply to this sender. You may find more
information in the online help of your client. ]
Other articles of interest: mozillaZine – Antivirus software.
Fitzcarraldo's Blog 