Deleted e-mails in an Office 365 Outlook account keep reappearing in the Thunderbird e-mail client

Whatever OS and e-mail client you use, if you search the Web you’ll find plenty of posts about deleted e-mails that reappear after you empty the ‘Trash’/’Deleted Items’/’Recycle bin’ folder. This problem seems to occur mostly with e-mail accounts that use the IMAP (Internet Message Access Protocol) or the Microsoft EWS (Exchange Web Services) protocols.

The obvious thing to check first in the e-mail client is if it has been configured to actually delete the e-mails on the server when you empty the ‘Deleted Items’ folder. Furthermore, internal files called ‘folder index files’ (.msf) in Thunderbird can sometimes become damaged, and these damaged files can also result in deleted e-mails reappearing. There is a ‘Repair Folder’ option in Thunderbird that sometimes fixes this problem. If that does not work, deleting the relevant folder’s .msf file and allowing Thunderbird to rebuild it sometimes fixes the problem. Anyway, I tried all the suggested approaches and more, as well as completely removing the account in Thunderbird (including ticking ‘Remove message data’ and manually deleting any remaining files for that account that remained in the Thunderbird directory). But nothing worked. However, I eventually cracked the problem as explained below.

Let’s just recap my situation:

  1. Of the many e-mails in a corporate Office 365 account that I had deleted over the last few months and emptied from the account’s ‘Deleted Items’ folder, five of them would keep reappearing in that account’s ‘Deleted Items’ folder in Thunderbird. This happened if I deleted the e-mails individually from the ‘Deleted Items’ folder and if I right-clicked on the folder and selected ‘Empty Deleted’.
  2. Whenever I logged in to the Office 365 Outlook account via a Web browser, the ‘Deleted Items’ folder was empty and it showed ‘Recover items deleted from this folder (0 items)’, i.e. no deleted messages existed.
  3. The Samsung e-mail client on my Samsung Galaxy Note 8 mobile phone showed the ‘Recycle bin’ for the same account was empty.

All the settings for the account in Thunderbird were correct. Completely removing the account from Thunderbird then adding the account again did not solve the problem, meaning that the problem could not be due to Thunderbird or the ExQuilla add-on Thunderbird uses to enable it to access the Office 365 Outlook account using Microsoft EWS. Even though my Samsung mobile phone’s e-mail client showed the Office 365 Outlook account’s ‘Recycle bin’ was empty, I selected ‘Email settings’ in the Samsung e-mail client, selected the account, scrolled down to ‘Empty Recycle bin’ and tapped it. The following message was displayed:

Empty Recycle bin?

This will permanently delete the items in the Recycle bin.

Cancel            Delete

I tapped ‘Delete’ and the Samsung e-mail client displayed ‘Success’. The five rogue e-mails then disappeared from the ‘Deleted Items’ folder in Thunderbird. The next time I logged in to the Office 365 Outlook account via a Web browser, the ‘Deleted Items’ folder was still empty but it displayed ‘Recover items deleted from this folder (5 items)’. I used the usual Office 365 Outlook procedure to recover the five e-mails and delete them permanently, resulting in ‘Recover items deleted from this folder (0 items)’ being displayed again in Office 365 Outlook in the browser window.

So, there you have it, the problem had nothing to do with Thunderbird or ExQuilla. If you access an Office 365 Outlook e-mail account via an e-mail client on a desktop or laptop and also via an e-mail client on a mobile phone, and you find that e-mails you deleted and emptied from the ‘Deleted Items’/’Trash’/’Recycle bin’ folder in the e-mail client on the desktop/laptop keep reappearing in that folder, try deleting them on all your devices, including your phone, even if the e-mails are not shown in the Office 365 Outlook account in a Web browser nor in the Office 365 Outlook account in a phone’s e-mail client.

Why does Thunderbird add ‘\A0’ and other strange-looking strings in e-mails I send?

I use Linux and have used the Thunderbird e-mail client since 2008. I used to use DavMail to enable Thunderbird to access various company Microsoft Exchange WebMail accounts but, several years ago, DavMail would no longer work with a particular Microsoft Exchange account so I switched to the Thunderbird add-on ExQuilla, for which I pay an annual licence fee. I do not know if the more recent versions of DavMail would work with this particular account but ExQuilla got me out of a hole so I stuck with it. Recently this particular corporation decided to stop using an in-house Microsoft Exchange server and switched to Microsoft 365.

Recently people receiving my e-mails sent using this particular account told me there were strange strings of characters in e-mails of mine that quote other e-mails. The most frequent occurrence is the three-character string ‘\A0’, although other strings are sometimes present too. The following e-mail extract illustrates the effect:

Hi Claudia,

I have had a look at your draft and agree with your assessment. Let’s sit down together and prepare a list of possible remedial measures.


On 07/10/2020 13:02, Claudia wrote:
> Hi,
> \A0
> Could you please have a look at the draft I have attached.
> \A0
> There are several main issues requiring attention. The operation was basically run by one person \2013 (John) during the tests, which led to several issues.
> \A0
> He does not have the time to do everything by himself.\A0 The other staff who had assisted him during earlier tests were not present.

Notice various occurrences of ‘\A0’ and an occurrence of ‘\2013’.

I searched the Web to see if other Thunderbird users had come across this problem, and found several reports of similar problems, although not identical. The most promising page I found was in the Mozilla support forums for Thunderbird: Why do my sent messages magically add “�” at the end of my sentences?. However, none of the various fixes suggested in that thread worked in my case. My Thunderbird installation was configured to use ‘Unicode (UTF-8)’ text encoding for Outgoing Mail and Western (ISO-8859-1) for Incoming Mail ( ‘Edit’ > ‘Preferences’ > ‘General’ > ‘Language & Appearance’ > ‘Advanced…’ > ‘Text Encoding’). I changed the text encoding for incoming mail to ‘Unicode (UTF-8)’ but that made no difference. I ticked ‘When possible, use the default text encoding in replies’ but that also made no difference. Anyway, I left the settings like that and hoped an update to Thunderbird would fix the issue.

I was not sure if the problem started with an upgrade to Thunderbird, or whether the switch to Microsoft 365 was the cause. I suspect Microsoft 365 is the culprit because the problem does not occur when I use other e-mail accounts. Anyway, it is annoying and I have still not found a fix for it. One of the replies in the above-mentioned Thunderbird support thread is not identical to what I’m seeing, but it looks to be essentially the same problem:

Jorg K
2/4/18, 6:17 AM

There is NO bug in Thunderbird. Sadly some US ISP’s like AT&T and Bellsouth have started *corrupting* their customers’ e-mail.

If the customer sends in windows-1252 and includes for example special punctuation characters or a non-break space xA0, the ISP doesn’t correctly interpret the the message as windows-1252 but as UTF-8. In UTF-8, xA0 is not valid and gets replaced by the so-called replacement character, � (0xEF 0xBF 0xBD).

Since the e-mail is still windows-1252 encoded, the recipient’s client displays �.


Affected users should complain heavily to their mail providers. As a workaround, they need to send all messages as UTF-8.

This seems to be a possible explanation of what I am experiencing, but it is impractical for me to check what text encoding all my contacts are using, or get them to switch to UTF-8 if they are not already using it in their e-mails. I noticed that there is actually a space in what look like blank lines in the e-mails I quote, and, if I delete that space, the ‘\A0’ no longer appears on those lines when I view the contents of e-mails in the Sent Items mailbox. I think that the space is, in fact, a non-breaking space (xA0), which is apparently invalid in UTF-8 and gets displayed as ‘\A0’ by Thunderbird (I’m currently using Version 78.4.2).

Trying to find and delete all the non-breaking spaces and other non-UTF-8 characters in a quoted e-mail is impractical. However, I found a somewhat cumbersome work-around to the problem of non-breaking spaces (and, I think, other non UTF-8 characters). When I click on the ‘Reply’ button in Thunderbird and a window pops up for me to compose my reply which includes a quoted e-mail or e-mails, I use Ctrl-C to copy all the contents of the window, then Ctrl-V to paste it back into the window. This seems to get rid of the character strings representing non-UTF-8 characters. It does add some extra blank lines in the quoted e-mail(s) in the window in which I am composing my e-mail, but those extra blank lines are normally not present when viewing the e-mail after it has been sent.

This work-around is not ideal as it relies on me remembering to do it when composing an e-mail in which I am quoting a previous e-mail or e-mails. But at least it gets rid of the multiple additional occurrences of ‘\A0’ (non-breaking space). It’s a pity there is no mechanism in Thunderbird to filter out non-UTF-8 characters such as a non-breaking space when quoting other e-mails. Even if Jorg K in the above-mentioned thread is correct and the cause of the problem does not lie in Thunderbird, I would rather Thunderbird act differently if the user has configured it to send e-mails using UTF-8 text encoding, and filter out non-UTF-8 characters rather than including strings of gobbledygook in the e-mail.

Thunderbird’s defective method of enabling anti-virus software to scan incoming POP3 e-mail messages

Thunderbird’s method of enabling anti-virus software to scan incoming e-mail messages is explained in the mozillaZine article ‘Download each e-mail to a separate file before adding to Inbox‘ and in Mozilla bug report no. 116443 (the bug report that resulted in the functionality being implemented). It is my contention that the design is deficient and is actually not a solution. In this post I explain why I believe this to be the case. Although here I will discuss Thunderbird in Linux, I believe the deficiency applies to Thunderbird in all OSs.

By default, Thunderbird inserts new incoming e-mail messages into an Inbox file. However, it is possible to configure Thunderbird to first create a temporary file containing each individual e-mail message in the /tmp directory, to enable external anti-virus software to scan each message before Thunderbird inserts it into the Inbox file. This approach only works for POP3 e-mail. The developers’ rationale for implementing this approach was to avoid the possibility of anti-virus software deleting or quarantining an entire Inbox.

In summary, if you want to scan incoming e-mails on your machine without running the risk of losing the entire Thunderbird Inbox, you must:

  1. configure Thunderbird so it creates temporary files /tmp/newmsg* (each file contains a single e-mail message containing ASCII characters);
  2. configure the anti-virus software not to scan the directory containing the Inbox;
  3. configure the anti-virus software to scan the /tmp directory.

Nevertheless, it seems Thunderbird developers would prefer you to disable local scanning of e-mail messages entirely: ‘mozillaZine – Email scanning – pros and cons‘.

Nowadays e-mail servers scan e-mail messages before you even download them. Some e-mail servers even send you an automated e-mail to inform you about an infected incoming e-mail or about an infected outgoing e-mail rejected by a receiving e-mail server. So local scanning of e-mail messages is far less important. Furthermore, I am not sure if the anti-virus software I use (ClamAV) is capable of detecting viruses in e-mail attachments encoded as ASCII characters. Anyway, purely out of curiosity I decided to investigate whether it would be possible to scan Thunderbird’s temporary files reliably.

To configure Thunderbird to create the temporary message files, it is necessary to select ‘Edit’ > ‘Preferences’ > ‘Security’ > ‘Antivirus’ and tick ‘Allow anti-virus clients to quarantine individual incoming messages’ (which sets mailnews.downloadToTempFile to true). Once that option has been selected, Thunderbird creates a temporary file /tmp/newmsg per message, which exists for a very brief (and inconstant) time before Thunderbird deletes it. When downloading several e-mail messages in very rapid succession, Thunderbird creates temporary files with different names (‘newmsg‘, ‘newmsg-1‘, ‘newmsg-2‘, ‘newmsg-3‘ and so on) to avoid overwriting messages, but usually one file named newmsg is sufficient to cater for the message download rate, as Thunderbird only keeps the temporary files for a very short time until it moves the message to the Inbox.

The problem with this approach is that Thunderbird does not provide any handshake mechanism to inform external anti-virus software that it has finished writing to a temporary file and that the file is available for scanning, nor does Thunderbird provide any handshake mechanism for external anti-virus software to inform Thunderbird when the scan of each temporary message file has finished (i.e. to tell Thunderbird that it can go ahead and delete the temporary file). In other words, the Thunderbird ‘solution’ is not a solution at all. In fact, I have found empirically that, if the anti-virus software is not fast enough, it can scan an incomplete temporary message file (i.e. the evaluation of the e-mail message would not be thorough and hence would be invalid). The Bash script below, for example, is sometimes able to scan an entire Thunderbird temporary file but at other times only manages to capture part of the file (it appears Thunderbird opens and closes the temporary file more than once) before Thunderbird deletes it:


# This script only works with Thunderbird.
# This script only works for POP3 e-mail.
# You must configure Thunderbird to create temporary files /tmp/newmsg*.
# To do that, set Edit > Preferences > Security > Antivirus and tick
# 'Allow anti-virus clients to quarantine individual incoming messages'
# which sets mailnews.downloadToTempFile to true.

mkdir $WORK 2> /dev/null
rm $WORK/* 2> /dev/null


# Watch for newmsg* file(s) created by Thunderbird in /tmp
inotifywait -q -m -e close_write --format '%f' /tmp | while read FILE
     if [ "${FILE:0:6}" = "newmsg" ] && [ -s /tmp/$FILE ]; then
          cp -p /tmp/$FILE $WORK/$TMPFILE
          # Do not let clamscan write temporary files to /tmp as inotifywait will detect them!
          clamscan --tempdir=$WORK $WORK/$TMPFILE

Below is an example of an incomplete newmsg file that the above script copied to the directory $WORK when Thunderbird downloaded an e-mail message:

From - Sun Feb 21 09:40:58 2016
X-Account-Key: account8
X-Mozilla-Status: 0000
X-Mozilla-Status2: 00000000

I began to wonder if a valid scan would be possible if the script were to lock the temporary file (e.g. using the chattr +i command) until it has completed copying it. The use of the chattr command in the script means it has to be executed by the root user. When I first did that with a modified version of the script using KDialog to display the result of ClamAV’s scan, the following error message was displayed and the script aborted:

kdialog(xxxxxx)/kdeui (kdelibs): Session bus not found
To circumvent this problem try the following command (with Linux and bash)
export $(dbus-launch)

I therefore added the line ‘export $(dbus-launch)‘ to the script as follows:


# This script must be launched by the root user.
export XAUTHORITY="/home/fitzcarraldo/.Xauthority"
export DISPLAY=":0"
export $(dbus-launch)

mkdir $WORK 2> /dev/null
rm $WORK/* 2> /dev/null

inotifywait -q -m -e modify --format '%f' /tmp | while read FILE
     # If file name begins with "newmsg" then copy it to work directory and scan it.
     if [ "${FILE:0:6}" = "newmsg" ] && [ -f /tmp/$FILE ]; then
          chattr +i /tmp/$FILE # Stop Thunderbird opening/deleting file.
          cp -p /tmp/$FILE $WORK/
          chattr -i /tmp/$FILE # Allow Thunderbird to open/delete file.
          clamscan --tempdir=$WORK $WORK/$FILE >> $WORK/$FILE.log
          kdialog --msgbox "$(cat $WORK/$FILE.log)"

However, when the above script was running, Thunderbird displayed a pop-up window if it attempted to copy the temporary file to the Inbox before the script released the lock on the file:

There was an error copying the downloaded message from the temporary download file. Perhaps it contained a virus, or you are low on disk space.
Subject: Test to see what happens if script locks newmsg file
Do you want to skip this message?

I clicked ‘No’ and the e-mail message in the file /tmp/newmsg was deleted without being copied to the Inbox, and the message was not deleted from the e-mail server. Subsequent attempts to re-download the message resulted in the same behaviour if the script had not finished processing the message before Thunderbird tried to move the message to the Inbox. Had I clicked ‘Yes’ I assume Thunderbird would simply have deleted the message on the mail server.

I did not bother looking into it further, but presumably the chattr command triggers inotifywait, in which case the script could cycle several times for the same file.

An approach that would probably work would be for Thunderbird to provide some sort of interlock so that it waits to delete newmsg* files until an anti-virus application gives the go-ahead.

An alternative approach would be for Thunderbird not to delete a temporary file after it writes the message to the Inbox and just leave it in the /tmp directory without overwriting it. An anti-virus application would quarantine infected temporary files and leave uninfected temporary files in /tmp, and therefore the anti-virus application would have to be written so that it deletes the temporary file once it has finished scanning it.

The second approach mentioned above would not be as good as the first approach for the following reasons:

  1. It would not stop Thunderbird adding a message to the Inbox (which would mean the user would have to delete a message manually from the Thunderbird Inbox if the virus scanner reported a message as infected).
  2. Thunderbird would have to use a different file name to any existing temporary files (at present it reuses ‘newmsg‘ if a file of that name does not already exist).
  3. The user would have to ensure the temporary files do not accumulate ad infinitum. In my case, the contents of the /tmp directory are deleted each time I reboot, but, in theory, a partition could become full if a user never switched off a machine and received a lot of e-mails.

Regarding the second reason listed above, Thunderbird already names the temporary files ‘newmsg‘, ‘newmsg-1‘, ‘newmsg-2‘ etc., so perhaps the existing Thunderbird code would automatically use a different file name if a file with the same name were still present, rather than overwriting it. If e.g. files newmsg and newmsg-2 happened to exist, I would hope Thunderbird would name the next temporary file ‘newmsg-1‘ or ‘newmsg-3‘.

I wondered if it would be possible to catch up with Thunderbird by just copying the temporary message files from the /tmp directory to another directory (see the script below), and then processing them afterwards with another script. However, even if a script just copies the temporary files to another directory without running ClamAV or KDialog, I found it is still not fast enough to catch all temporary files before Thunderbird deletes them. If Thunderbird downloads a single message and no others are waiting on the server(s) to be downloaded, it seems a script can copy the temporary messages successfully. However, if there are several messages waiting to be downloaded from the e-mail server(s) and Thunderbird downloads them in rapid succession, Thunderbird deletes some of the temporary messages before the script can copy them fully.


# Create work directory if it does not already exist
mkdir $WORK 2> /dev/null
# Delete old working files if they exist
rm $WORK/* 2> /dev/null


inotifywait -q -m -e close_write --format '%f' /tmp | while read FILE
     if [ "${FILE:0:6}" = "newmsg" ] && [ -s /tmp/$FILE ]; then
          cp -p /tmp/$FILE $WORK/$TMPFILE

Consider the following six messages copied to $HOME/clamtmp by the above script (the script adds the first character to the name of the copied file):

-rw-------   1 fitzcarraldo fitzcarraldo    3829 Feb 23 02:24 1newmsg
-rw-------   1 fitzcarraldo fitzcarraldo    3107 Feb 23 02:25 2newmsg
-rw-------   1 fitzcarraldo fitzcarraldo 1158576 Feb 23 02:26 3newmsg
-rw-------   1 fitzcarraldo fitzcarraldo     237 Feb 23 02:28 4newmsg
-rw-------   1 fitzcarraldo fitzcarraldo    2106 Feb 23 02:28 5newmsg-1
-rw-------   1 fitzcarraldo fitzcarraldo    3107 Feb 23 02:28 6newmsg-2

The first three messages were each the sole message on all three e-mail servers accessed, and the copied files newmsg -> 1newmsg, newmsg -> 2newmsg and newmsg -> 3newmsg were all complete messages. However, the last three messages were on three e-mail servers simultaneously waiting to be downloaded, and when I clicked on ‘Get All New Messages’ in Thunderbird, the copied files newmsg -> 4newmsg, newmsg-1 -> 5newmsg-1 and newmsg-2 -> 6newmsg-2 were downloaded by Thunderbird in rapid succession. The copy 4newmsg was incomplete, the copy 5newmsg-1 was complete and the copy 6newmsg-2 was complete. So, even with a faster script, there is no guarantee that a script can catch all the temporary message files. Therefore, as I mentioned earlier, the only way to guarantee that temporary message files are properly scanned would be to modify Thunderbird to provide either a handshake (e.g. a file lock or inter-application flag) or to leave each temporary message file on /tmp and give it a unique file name.

The downside with both the above-mentioned approaches would be that the anti-virus software developer would need to know about the method, and write the software to perform the appropriate actions. If the first approach were adopted, the anti-virus software would need to signal to Thunderbird that it had completed scanning the file (e.g. by releasing a file lock or by an inter-application message). If the second approach were adopted, the anti-virus software would need to delete the message file from /tmp once it had completed scanning the file. The second approach would be easier for a simple Bash script to use, and, had the Thunderbird source code not been so complicated, I would have had a go at patching it to leave temporary message files in the /tmp directory after Thunderbird copies their contents to the Inbox file. But, as e-mail servers already do a good job of scanning messages before Thunderbird downloads them, I will not spend more time on this. Some e-mail servers even send an e-mail to the user informing them about an infected e-mail (see examples below), so it is not worth bothering.

Example 1
Automated e-mail server message to warning him that the e-mail with an infected attachment he sent to was blocked by the receiving e-mail server.

Subject: Mail delivery failed: returning message to sender
Date: Wed, 17 Feb 2016 11:30:10 +0100
From: Mail Delivery System

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of
its recipients. This is a permanent error. The following address

virus/suspect content found

— The header of the original message is following. —

Example 2
Automated e-mail server message to warning him that an e-mail with an infected attachment sent to him by was blocked by the receiving e-mail server.

Subject: VIRUS SUSPECTED: “Dave (Hotmail)”
Date: Wed, 17 Feb 2016 11:46:07 +0100 (CET)
From: Mail Delivery System

A virus was detected in the following e-mail!

Mail details:

From: “Dave (Hotmail)”
TO: John Smith
Date: Wed, 17 Feb 2016 10:45:55 +0000
Subject: EICAR test file attachment

The concerned e-mail has been handled according to your Virus Protection Settings.

Your E-mail Service Provider Team

[ This is an automatically generated email, do not reply to this sender. You may find more
information in the online help of your client. ]

Other articles of interest: mozillaZine – Antivirus software.

How to specify the e-mail account Thunderbird uses to reply to event invitations

Usually applications with a GUI are intuitive to use and not too difficult to configure. But sometimes I end up banging my head against a brick wall. Such was the case when I wanted to change the default e-mail account that the Thunderbird e-mail client uses to send an ‘Email Notification’ in reply to a meeting invitation.

I have several e-mail accounts and use Thunderbird with the Lightning calendar extension. When my e-mail accounts receive a meeting invitiation e-mail, the row of buttons ‘Accept’, ‘Tentative’ and ‘Decline’ is displayed at the top of the invitation window. The problem is that, whichever of my e-mail accounts receives an event invitation e-mail, clicking on the aforementioned buttons always results in Thunderbird sending the reply (‘Event Notification Email’) from a fixed e-mail account. To give a hypothetical example, let’s say I had the e-mail accounts,, and Whichever of those accounts receives an event invitation e-mail, when I click on ‘Accept’ the reply is always sent from the account

It turns out that the e-mail account Thunderbird uses to send event notification replies is tied to the calendar, not to the e-mail account which received the invitation (see Mozilla Bugzilla Bug No. 589081 – Wrong outgoing server for meet confirmation in Lighting plugin for Thunderbird). However, it is not obvious how to specify the default e-mail account to be used by the calendar, so here is how to do it in Thunderbird 31.2.0 and Lightning 3.3.2:

  1. There are two icons at the top right of the Thunderbird window in my case: a calendar icon and a clipboard icon. Hovering the mouse pointer over those two icons displays the tooltips ‘Switch to the calendar tab’ and ‘Switch to the tasks tab’ respectively. Click on the calendar icon to display the calendar tab.
  2. The calendar tab should be displayed. Click on ‘Edit’ and select ‘Calendar Properties…’ from the drop-down menu. A window titled ‘Edit Calendar’ should pop up.
  3. In the ‘Edit Calendar’ window, select the default e-mail account from the drop-down menu of e-mail accounts, make sure ‘Read Only’ is not ticked, and click ‘OK’.

That’s all there is to it. It is a pity the Thunderbird UI does not make it at all obvious how to do it.