Email migration from Eudora (or Thunderbird) to GMail

Update November 17 2012: 

I started this post back on July 20 – four months ago – and have only finished the migration now. That’s because I found a lot of unexpected problems along the way.

The rest of this post is my new, improved, and corrected step-by-step guide on how to migrate email from Eudora (up to version 7) or Thunderbird to GMail, based on hard-won experience.

BACKGROUND

I’ve been using Eudora since 1995 (17 years). Before that I used Unix mail (Sun; early 1990s), CompuServe (back to 1981), and something called “The Source” (1979; I was ‘TCA818’). Disk space was expensive back then so I didn’t save a lot of email.

But since 1995 I’ve kept everything – disk drives got bigger faster than my mail archive. So I had about 14 GBytes of email in Eudora.

Eudora has been abandoned by Qualcomm since 2006 and is getting old; there’s a new open source version based on Thunderbird (“Eudora OSE”), which sucks.

As our company grows the job of administering Eudora users (plus myself) was getting too big for me, so I decided to outsource it to Google Apps for Business. Which means GMail, and migrating the old email.

ABBREVIATIONS

In the following, I’ll use these abbreviations:

  • E7 – Eudora 7 (or any earlier version of Eudora)
  • OSE – Eudora Open Source Edition (used only for migration)
  • Tbird – Mozilla Thunderbird
  • Outlook – Microsoft Outlook 2003 or later
  • OE – Outlook Express
THINGS THAT WON’T WORK

To save you time, here are some things that won’t work:

  1. Syncing E7 to GMail using IMAP
  2. Converting E7 to E-OSE, then sync E-OSE with GMail using IMAP
  3. Converting E7 to E-OSE, then sync Tbird with GMail using IMAP
  4. Converting E7 to Outlook Express, that to Outlook, then Outlook to GMail
  5. IMAPSize
  6. Importing E7 data with Outlook, then sycning to GMail
  7. Importing Tbird data with Outlook, then sycning to GMail
  8. Importing E7 data with Thunderbird
  9. Syncing Outlook with GMail via IMAP (use this instead; it works)

Trust me, don’t bother.

If you really care or don’t believe me, in the Appendix at the very end of this posting I’ve put some of my notes on why some of those don’t work.

(I haven’t the patience to describe all the problems…comments in the Python files give some more details.)

STUFF YOU WILL NEED

To do the migration, you’ll need the following:

These are only for the migration. Once it’s done you can throw away all of them.

All of those are free except for Outlook, but you probably know someone who has a copy you can borrow for the migration. Alternatively, you could look on eBay for an older version cheap. (I used Outlook 2003, so you don’t need a recent one to do this.)

(You need real Outlook, not “Outlook Express”.)

If installing Python and running my Python scripts is too complex for you, this method won’t work for you, and I don’t have any suggestions – I haven’t found a simpler way that works. (Not for lack of trying.)

STEP 1 – GET GMAIL WORKING

Before you do anything else, you need to have your GMail account working, and new mail needs to arrive at GMail (not Eudora or Thunderbird). Test it by sending yourself an email.

If you’re using free GMail (@gmail.com), you you can skip the rest of this step.

To get mail to arrive at GMail using your own domain name:

  • CNAME records need to be alerted at your domain server (usually the web hosting site – wherever the A record points), not at the domain registrar. If you’re using cPanel, this is most easily done using the “Advanced DNS Zone Editor”.
  • The same goes for MX records – these need to be changed at the host the A record points to, not at the domain registrar.

Our domain registrar is 1and1.com, but the DNS A records point at our server at hostgator.com, where the web page is hosted. This was confusing because our domain registrar also offers web hosting, so they have an interface to change your CNAME (and I think also MX) records. But that only has an effect if the domain registrar is also the web host.

http://www.mxtoolbox.com is a really useful tool to check your DNS records.

Another problem I had was that email was getting to GMail just fine from everywhere except internal company addresses; those emails kept arriving at the old mail server (at Fastmail.fm – highly recommended if you only need email).

It turns out that during the transition to GMail some users were still sending internal email via Fastmail’s SMTP server (not yet Google’s), and Fastmail still thought it was supposed to handle incoming mail for our domain. So when it got an email directed internally, it wasn’t even looking at the MX records (and therefore not picking up the change on where to deliver the mail). So,

  • Remove your domain from the list of domains handled by your old mail provider

Once I did that (under “Virtual Domains” at Fastmail), internal-origin mail started getting delivered to GMail.

STEP 2 – SETTING UP GMAIL

In each user’s individual GMail account, I’ve setup:

Settings>POP/IMAP Enable IMAP
Settings>Labels Check “Show in IMAP” for both “All mail” and “Drafts” (per Google)
Settings>General Set up signature
Conversation View: Off (more like Eudora)
Button Labels: Text
Settings>Themes High contrast (helps distinguish panel areas)
Settings>Web Clips Turn off

Except for IMAP (which we’ll need), these things are a matter of taste.

I wanted to use another domain as an alias for the company domain, for testing purposes. Google supports that, but wants you to put a cookie in the root of your web page to verify that you own the domain. Unfortunately, the test domain I wanted to use was redirected to our main web page (that domain is a mis-spelling of our correct domain), and Google didn’t like that.

They offer another method to verify ownership, by adding a TXT record with the cookie to the domain’s DNS entry. At first I tried adding a TXT record to “mailtest.myTestDomain.com”, and that didn’t work.

It turns out that you can have multiple TXT records for a single domain (who knew?) – Google was looking for the cookie in the TXT records for “myTestDomain.com” (not any sub-domain). Once I did that, Google was happy and I could use the test alias.

Like the CNAME and MX records, the TXT records need to be modified at your domain server (in my case, at Hostgator), not the registrar.

STEP 3 – PREPARATION FOR THE MIGRATION

If you’re converting from Thunderbird instead of Eudora, skip forward to Step 6 below (you already have a Thunderbird data folder).

Before you start the migration, make sure you’ve done the following:

  1. Verify new email arrives at GMail (see above)
  2. Reduce the total number of folders in Eudora (or Thunderbird) to (much) less than 500.
  3. Eudora 7: Special>Empty Trash
  4. Eudora 7: Special>Compact Mailboxes
  5. Exit Eudora 7. Do not run it again (this will mess things up).
  6. Make sure you have at least 3x as much free disk space as occupied by the Eudora 7 user folders.
  7. Backup your Eudora data folder (in case something goes wrong and you need to start over, or if you ever want to use Eudora to look at your old email again).

Re #2, the 500 folders includes all sub-sub-sub-folders. After a lot of discussions with GMail support people at Google, I found that GMail simply can’t handle large numbers of labels (each folder gets converted to a GMail label). Their stated limit is 500 labels.

If you go past 500 labels, things will seem to work, but then break in hard to predict ways. If you get “oops…error #400” from GMail, this is the problem – too many labels. There’s no workaround; just avoid too many labels in GMail.

Re #5 and #7, the migration process will mangle your Eudora 7 users folders; running Eudora 7 again will un-mangle them and break things. So backup your user folder (#7). After the migration is complete, you can restore it and then use Eudora to access your old email. (It’s fine to run Eudora again after you’ve restored the user folders.)

If you have more than one Eudora user folder on your computer, make sure the one you want to migrate is the last one you open with Eudora. When you start Eudora OSE, it will try to migrate the last Eudora 7 mail account you opened (I’m guessing this is stored somewhere in the registry, but I haven’t found where).

STEP 4 – RUN FixEudora7Mail.py SCRIPT

Make sure you’ve done steps 1 to 7 (above) to prepare.

  1. Start up Python
  2. Change the Python working directory to the Eudora user data folder.

You can do this with:

import os 
os.chdir('path to Eudora user data folder')
os.getcwd() # just to verify that it worked
  1. Run the Python script “FixEudora7Mail.py“.

Just FYI, this will read descmap.pce (mapping of file names to mailbox names), then for each mailbox:

  • Make a binary copy of mailbox
  • Read each message in the order it appears in the .TOC index file
  • Rewrite the “From ???@???” line with a time derived from the UTC time in the .TOC file
  • Delete the original mailbox, and replace it with the revised one using a “safe” name acceptable to both Tbird and Windows
STEP 5 – MIGRATE EUDORA 7 DATA TO EUDORA OSE

Do not run E7 any more on this folder. If you do, E7 will break some of the corrections applied in the previous step. (If you need to run E7 later, restore your backup first.)

In this step we’re going to convert E7 data to Thunderbird format using Eudora OSE. (E-OSE and Tbird use the same data files.) Thunderbird stores its data here:

Vista or Win7:

<user path>\AppData\Roaming\Thunderbird

WinXP:

<user path>\Application Data\Thunderbird

Linux and OS X probably store it somewhere else – find out where.

Now,

  1. Navigate to the Thunderbird folder (just mentioned).
  2. Delete it (…\Thunderbird) if it already exists from previous use of Thunderbird. (Or rename it if you want to keep it.)
  3. Turn off your antivirus software – this is really important, otherwise you’ll get “Unable to save your message as draft. Error writing temporary file.” errors.
  4. Start Eudora OSE. When it starts and doesn’t find a Thunderbird profile, it will offer to migrate the Eudora 7 mail. Let it. (“Import from Eudora”, then Next.)
  5. Wait. OSE will import everything from the last-used Eudora 7 user data folder into the Thunderbird folder. The import is amazingly slow. If you have a lot of messages, allow several hours.
  6. When it’s done importing, hit ‘Cancel’ (to the “System Integration” box), and exit OSE.
  7. Turn your antivirus software back on.

If you don’t run out of disk space and didn’t have an antivirus program running, the import should complete without any error messages.

We’re not going to transfer email directly from OSE to Google because OSE isn’t very good at that; instead we’ll use Tbird, which is better and faster at it. But first we have to fix some things in the next steps.

STEP 6 – FIX CONVERTED EMAIL

Now we’re going to fix a lot of mistakes OSE made in the conversion. (If you’re starting from Tbird instead of Eudora, this step is harmless and may fix some problems.)

  1. Be sure your antivirus software is back on.
  2. Start up Python again
  3. Change the Python working directory to the Thunderbird “Local Folders” folder.

This is a sub-folder buried deep within the main Thunderbird data folder mentioned earlier. It’s here:

Vista or Win7:

<user path>\AppData\Roaming\Thunderbird\Profiles\<unique key>\Mail\Local Folders

WinXP:

<user path>\Application Data\Thunderbird\Profiles\<unique key>\Mail\Local Folders

As before, you can change the Python working directory like this:

import os 
os.chdir('<user path>\AppData\Roaming\Thunderbird\Profiles\<unique key>\Mail\Local Folders')
os.getcwd() # just to verify that it worked
  1. Run the Python script “FixTbirdMail.py“.

Just FYI, for each mailbox, this will:

  • Remove the .msf file (Tbird index file; we will let Tbird rebuild this)
  • Open a new output mailbox (“.new”)
  • Fix HTML entities split across lines (example: “&micro;”)
  • Replace all ‘&amp;’ with ‘&’
  • Identify mailboxes with attachments too large for GMail
  • Delete the original mailbox, replace it with the fixed one.
STEP 7 – SET UP AUTOMATIC MAILBOX RECONFIGURATION

If you had a lot of mailboxes in Eudora (too many for GMail) and you want to mechanically reduce the number (instead of doing it all by hand), you can use this step to do it.

If that isn’t a problem for you, you can skip this step.

The next (and final) Python script will parse the entire ‘Local Folders’ tree looking for files with any of the three following names:

  • CONCATENATE.ME
  • MOVEUP.ME
  • EXECUTE.ME

If it finds ‘CONCATENATE.ME’, it will concatenate all the mailboxes in that folder (the folder where the file was found) into a single mailbox, with the name of the folder where the file was found. This is a quick way to merge a bunch of mailboxes.

If it finds ‘MOVEUP.ME’, it will move up the entire folder (where the file was found) by one level in the folder tree.

If it finds ‘EXECUTE.ME’, it will execute the Python commands in the file, in the context of the folder where the file was found. This is a way to customize changes of your own to the folder structure (you’ll have to know a little Python). Here’s an example of an EXECUTE.ME file I used:

src = ‘3GPP’
dest = ‘Old stuff.sbd\Organizations.sbd\3GPP’
shutil.move(os.path.join(path, src), dest)

src = ‘ETSI TIPHON’
dest = ‘Old stuff.sbd\Organizations.sbd\ETSI TIPHON’
shutil.move(os.path.join(path, src), dest)

src = ‘GII’
dest = ‘Old stuff.sbd\Organizations.sbd\GII’
shutil.move(os.path.join(path, src), dest)

src = ‘Misc’
dest = ‘Old stuff.sbd\Organizations.sbd\Misc’
shutil.move(os.path.join(path, src), dest)

The contents of the CONCATENATE.ME and MOVEUP.ME files don’t matter – they can be empty files; all the script cares about is whether the file names exist.

So place the files where you want them (if you care).

STEP 8 – MERGE AND SPLIT MAILBOXES

This step has two purposes:

  • It will process the CONCATENATE.ME, MOVEUP.ME and EXECUTE.ME files as just mentioned. If they don’t exist, that’s OK.
  • It will split any mailboxes that are > 2 GBytes in size into a number of smaller mailboxes. This prevents Thunderbird from choking on excessively-large mailboxes (you can re-combine them in GMail later if you want).

Any mailboxes that get split will have the old name plus a suffix to distinguish it. For example, if the mailbox called “Customers” was split into 3 parts, you’ll have:

Customers
Customers_2
Customers_3

So,

  1. Start up Python again
  2. Make sure the Python working directory is again at the Thunderbird “Local Folders” folder.
  3. Run the Python script “MoveMergeSplitTBirdMailboxes.py“.
  4. Save (cut and paste) the output from the script – it will identify which mailboxes contained messages that are too large for GMail (> 25 MBytes). You’ll need this info in the next step.

Again, just FYI, for each mailbox, this will:

  • For each path, excecute any Python commands in “EXECUTE.ME” file, then delete the “EXECUTE.ME” file.
  • For each path, if file “MOVEUP.ME” exists, delete “MOVEUP.ME” then move entire folder up one level.
  • Reparse the new directory structure.
  • For each mailbox, if “CONCATENATE.ME” exists in the folder, concatenates all mailboxes there into a single mailbox.
  • For each mailbox, split the file into smaller chunks if it’s > 2 GB in size (avoid Tbird 4 GB size limit).
  • For each folder, remove any “CONCATENATE.ME” files.
  • Remove empty files and folders.
STEP 9 – START THUNDERBIRD AND REMOVE MESSAGES TOO BIG FOR GOOGLE

Now we’re ready to run Thunderbird and look at the mail.

  1. Start TBird.
  2. Click “Cancel’ to the inital “System Integration” box.
  3. It will ask you for the password to your old email – don’t give it. Hit Cancel.
  4. Your mailboxes should be visible under “Local Folders”.
  5. Expand all the mailbox folders in Thunderbird. You have to open each one to get Thunderbird to index the folder and count the messages in it. Click on the top most folder (to show the mailboxes), then hit the down arrow on the keyboard to go to the next one. Repeat for all folders. Wait a while for Thunderbird to index everything.
  6. Refer to the output from the FixTbirdMail.py script – this should tell you which mailboxes have messages with attachments that are too big for GMail. (Look for mailboxes with more than zero messages that are “tooBig”.) With luck you won’t have any at all.
  7. If you have some, open up each mailbox with over-large messages. Right-click on the “Subject” heading for the messages and turn on the “Size” column. Then sort by size to find the messages that are over 25 MBytes in size. For each one, either delete the whole message or remove enough attachments to get it under 25 MBytes. (If you didn’t save the output from FixTbirdMail.py, you’ll have to check every mailbox.)

Note that the output from FixTbirdMail.py tells you which input file (before splitting) had “tooBig” messages; it doesn’t identify which of the new (split-up) mailboxes have the large messages.

For example, if you see:

Read: Dave Write: Dave.new 2,952 output messages.
Write: Dave_2.new 1,798 output messages.
Write: Dave_3.new 1,720 output messages.
Write: Dave_4.new 205 output messages. 6,675 messages 7 tooBig.

This indicates that the input mailbox “Dave” had 7 messages that were “tooBig” for GMail; you need to check all four output files to find them.

It’s normal that some of your mailbox names may have been slightly altered – some characters that either Windows or Thunderbird can’t handle in mailbox names were automatically changed to ‘_’; the most common one is a slash. For now, leave these alone – you can change them back if you like once the mail is at GMail.

Now your Thunderbird email should be in good shape, and ready to transfer to Google.

STEP 10 – SETUP THUNDERBIRD TO TALK TO GMAIL VIA IMAP

We’re going to transfer the mail from Thunderbird to GMail using the IMAP protocol.

To configure Thunderbird to do that, in Thunderbird do:

  1. Tools>Account Settings>Account Actions (button): Delete any imported email accounts
  2. Tools>Account Settings>Local Folders>Junk Settings: Disable
  3. Tools>Account Settings>Account Actions>Add mail account… Then fill in:
  • Your name
  • Your complete email address (to be used with GMail)
  • Password (for GMail)
  • Click CONTINUE
  • Immediately click MANUAL CONFIG (don’t let it auto-configure)
  • Set Incoming: IMAP imap.gmail.com 993 SSL/TLS Autodetect
  • Set Outgoing: SMTP smtp.gmail.com 587 STARTTLS Autodetect
  • Set Username: Your complete email address (including domain)
  • Click Done.
  1. Tools>Account Settings>GMail account>Server Settings: JUST MARK IT AS DELETED
  2. Tools>Account Settings>Copies & Folders:
  • Clear “PLACE A COPY IN”
  • Keep message drafts in “OTHER”
  1. Tools>Account Settings>GMail account>Junk Settings: DISABLE
  2. Click OK

At this point Thunderbird will log into Google and synchronize with your GMail account.

There will be two trees of mailboxes – “Local Folders” (stored on your computer) at the top, and below that, your GMail account (with your email address). This has your synchronized GMail. If you click on “Inbox”, you should see all your mailboxes at GMail.

FYI, Google’s instructions on this are here and here, but they don’t cover recent versions of Thunderbird, which has changed a lot (I used Thunderbird version 14).

STEP 11 – TRANSFER MAIL TO GMAIL

Finally, transfer the mail from Thunderbird to GMail. This is done by dragging the mailboxes from “Local Folders” in Thunderbird into the GMail account.

If you drag a whole folder, Tbird will copy the folder to GMail.

If you drag messages within a folder, Tbird will move the messages to GMail (deleting them from Tbird).

Do it any way you want. Each transfer requires a lot of time – GMail will only accept uploaded mail at a fairly slow rate. In the lower left corner, Tbird will show the status of the transfer (how many emails have been copied so far).

Also, GMail has bandwidth limits on how much email it will accept via IMAP – see here. At this writing, the limit is supposedly 500 MBytes/day (but it practice it seems to accept more). If you try to upload too much, at some point Google will just stop the transfer. You’ll get a (brief; easy to miss) message from Tbird that it “can’t append message”, and the transfer will stop. Try again later.

There’s no harm in re-uploading the same messages multiple times – GMail will detect that they’re duplicates; you’ll only have one copy of each message.

My suggestion is to copy the folders one at a time, and after each one is copied, check the number of messages in Thunderbird (or Eudora) against the number in GMail. Once it checks OK, rename the moved folder in Thunderbird to indicate it has been moved OK.

You can quickly see how many GMail messages have a given label – go into Settings>Labels on GMail and there’s a count next to each label. (Be sure to turn off “conversation view” to count messages instead of conversations.)

Don’t be too concerned if there are a few less messages in GMail than in Eudora 7 – GMail is very good about noticing and collapsing duplicate messages that crept into the Eudora 7 mailboxes over time. (Of course if there are more than a few missing, investigate.)

TROUBLESHOOTING

If there are missing messages after the transfer, it’s probably because the transfer of messages from Tbird to GMail stopped before the whole mailbox was sent. This can be because:

  • GMail saw a message that was too large (> 25 MBytes).
  • You’ve exceeded Google’s bandwidth limit for the day.
  • GMail saw a malformed message that it “doesn’t like”.

Start by checking for over-large messages in the mailbox.

If that’s not the problem, consider if you’ve exceeded the bandwidth limit. If so, wait a day and try again.

If you still can’t transfer a mailbox completely, probably the mailbox has a malformed message that GMail doesn’t like. Messages with missing From: or To: fields are likely culprits. If you suspect a message is causing a problem, try moving just that message by itself. If you get an error from Tbird, that’s the problem – you’ll have to remove it from the mailbox.

If you can’t figure out which message(s) are causing the problem, you’ll have to find it by divide-and-conquer:

  1. Create a new mailbox called “name – part 2”.
  2. Move half the messages from the problem mailbox into the new mailbox
  3. Try transferring each of the 2 mailboxes separately.

Probably one of them will transfer OK, and the other won’t. The one that doesn’t has the problem email. If neither transfers, both mailboxes may have problem emails.

In either case, take the problem mailbox and do 1, 2, 3, above again (split it 2, try to transfer each half). Repeat until you’ve narrowed it down to the problem messages.

I had to do this on several mailboxes, but unless you have huge, old, mail archives, you probably won’t run into it.

BACKING UP YOUR GMAIL DATA

If you don’t 100% trust Google to keep all the correspondence you’ve accumulated over a lifetime safe, and want to back it up, here are some ways:

I’ve only tried the first so far (Outlook syncs very nicely with GMail using Google’s tool), but I plan to find a more automated method.

I do not recommend using Thunderbird/IMAP for backup – the IMAP sync is too flaky. Outlook, using Google Apps Sync for Microsoft Outlook®, syncs much more reliably.

Appendix

The rest of this post provides some random useful information I discovered along the way.

ABOUT FILE FORMATS – GENERAL

Since Eudora OSE is a variant of Thunderbird, it stores email in the same format Thunderbird does, which is Unix mbox format (plus or minus some minor variations).

The end of the message header and start of the message body is marked by a blank line.

Eudora 7 uses almost the same format, except that it extracts all email attachments and saves them in binary in its “attach” folder. Since the Unix mbox format only stores text, with attachments in-line, the import to OSE results in folders that are larger (by about 1/3) than the Eudora 7 data was, because all the attachments get MIME-encoded as text.

As a general aside, RFC2822 and RFC822 are useful references for mailbox file formats (but not to be fully trusted – there are lots of not-quite-conformant email clients out there).

ABOUT FILE FORMATS – EUDORA 7

Eudora doesn’t store the time/date messages were sent in the usual way (in the “Date:” field). Instead, it’s usually in the .TOC file (Table Of Contents) that’s unique to Eudora 7. Unfortunately, none of the migration tools (other than mine) read the .TOC file, so instead the wrong date gets picked up. Often the date is close enough to being correct that it isn’t obviously wrong, other times (depending on the history of the file and any past synchronization problems with the .TOC file,) it can be off by years.

The Python scripts read the .TOC file and fix this.

Some other things I learned along the way regarding Eudora 7 mail files:

  1. Messages in Eudora 7 format mailbox files start with the string “From ???@??? ” in the first characters of the line. This is followed by a date which is not trustworthy. My best guess is that this date is when the message was downloaded by Eudora from the server (not when it was sent). Usually; sometimes it’s just crazy.
  2. As mentioned above, attachments are not stored in the mail file. Instead, they’re stored as binary files in the “attach” folder.
  3. The best (only) reference for the format of the Eudora 7 .TOC file I know of is here: http://users.starpower.net/ksimler/eudora/toc.html (email me if that link dies; I have a copy of it).
  4. Message dates can be stored in three places in Eudora 7:
  • In the “From ???@???” field (“From”)
  • In the “Date:” field (“Date”)
  • In the .TOC file, in 2 places (binary and text)

Sometimes a date is present in all 3 places, sometimes less. If any of the 3 differ, the “From” is the least reliable (it seems to be the date/time when the email was downloaded by Eudora, not when it was sent.)

Re the .TOC dates, the binary one is in seconds from 1970-01-01 00:00:00 UTC (standard Unix format), the text one is in local time with truncated seconds. The two always match.

Read/unread status is stored in the .TOC file.

ABOUT FILE FORMATS – EUDORA OSE AND THUNDERBIRD

If you look in the Thunderbird mail folder (given above), you’ll see a single file with no file extension for each mailbox – for example “pending” is the pending mailbox. There is also “pending.msf”.

The file without the extension (“pending”) is the Unix mbox format mail spool with everything in it, including attachments. The “.msf” file is the index, and entirely redundant – OSE and/or Thunderbird will rebuild it if/when it needs to (takes a long time).

Mailbox names that use invalid filename characters are stored in “panacea.dat”.

Read/unread status is stored in the message header (in the mbox file) in fields X-Mozilla-Status and X-Mozilla-Status2. (Details are here.)

WHY IMPORTING EUDORA 7 EMAIL DIRECTLY TO OUTLOOK DOESN’T WORK

It seems that Outlook 2003 really can’t import data from Eudora, even tho it claims it can.

All my attempts end in:

The mail folder could not be opened. If another application is using this file, please close and try again.

Supposedly this is caused by the mail not being in the folder where Outlook expects it; I tried copying it to “C:\Program Files (x86)\Qualcomm\Eudora” (the old XP-era default location) and also “…\Users\dave\AppData\Roaming\Eudora” (the new default location); that didn’t help.

To be fair to Outlook, it only claims to be able to import from Eudora 4 or earlier, although I’m pretty sure the file formats didn’t change since then. I even tried to install an old copy of Eudora 4, but Win7 doesn’t support 16 bit executables anymore, so that didn’t work. I suppose I could have tried it on XP, but I’m not very confident it would have helped.

Besides, this site claims that even when it works, it doesn’t work very well (supposedly attachments get messed up).

WHY IMAPSIZE DOESN’T WORK

Google’s official advice is that the only really robust method of migration to GMail is from Outlook “.PST” files.

I found a few commercial programs that claim to migrate Eudora mail to Outlook. I’m not against paying for something, but given all the complications I found with migrating to GMail, I’m very skeptical that these actually work as promised, and I didn’t feel like paying for something that won’t work.

IMAPSize is a free program that converts Unix mbox files to Outlook Express (“OE”). Outlook (not OE) claims to be able to import from OE.

But OE can’t handle files larger than 2 GBytes. If all your mail files are smaller than that (including attachments), it might work for you. Otherwise, not.

Also, IMAPSize doesn’t process the X-Mozilla-Status and X-Mozilla-Status2 headers used by Thunderbird to keep track of read/unread status, so that gets lost.

30 thoughts on “Email migration from Eudora (or Thunderbird) to GMail

  1. “The same goes for MX records – these need to be changed at the host the A record points to, not at the domain registrar.”

    Thanks for this, it helped me very much.

  2. Thank you very much for this helpful and easy to follow post. I was able to move all my emails from Eudora to Gmail even though I don’t know much about computer 🙂

  3. Hi. First of all, I’d like to say that our planet needs more people like you. Thank you!

    Question: you say “Syncing E7 to GMail using IMAP” won’t work. Are you saying it won’t work as part of a process to permanently move from Eudora to Gmail, or are you making a broader statement that someone wishing to continue to use E7, IMAPed with gmail, is doomed to failure? I ask because I have a similar Eudora history as you, yet am on the verge of trying to imap with gmail, yet intend to continue to use E7 as my main email client.

    Thank you again!

  4. That’s one of the nicest things anyone has said to me, thanks.

    Re IMAP, I don’t recall the details any more, but it just doesn’t work. E7’s implementation of IMAP is too old and buggy to work well with GMail.

    Go ahead and try it if you want – you’ll see what I mean quickly. (Just backup your E7 files first…)

    Best of luck!

    –Dave

  5. File “FixEudora7Mail.py”, line 17, in
    from common import *
    ImportError: No module named common

    What did I do wrong?

  6. That line is trying to read the “common.py” file.

    Probably your Python working directory isn’t the same as the directory where you have the 4 .py files. Try changing it (“cd path”).

  7. Trying to run this from the eudora folder (“C:\\python FixEudora7Mail.py”) and I get the message:

    File “FixEudora7Mail.py”, line 363, in
    main()
    file “”FixEudora7Mail.py”, line 382, in main
    and (string.find(path, ‘attach’)) == -1 \
    AttributeError: ‘module’ object has no attribute ‘find’

    Thanks for the help!

  8. Oh, should have said: Python 3.3.1 and I get the error from the python environment also. Thanks!

  9. I don’t really know, but I haven’t tried it in Python 3 – I’m still on 2.7. You might try that.

  10. OK, uninstalled 3 and installed 2.7. Now i get the following:

    Traceback (most recent call last):
    File “C:\eudora\FixEudora7Mail.py”, line 363, in
    main()
    File “C:\eudora\FixEudora7Mail.py”, line 291, in main
    displayName = map[name.lower() + “.mbx”] # lookup display name for this file
    UnboundLocalError: local variable ‘map’ referenced before assignment

    Thanks again for the quick responce!

  11. Hm. Line 265 of that file should read “map = {}”, which should prevent that error.

    Is it possible you inadvertently changed the source file? Try getting a fresh copy from the blog site.

  12. Thank you so much for not only telling us what worked, but what DIDN’T. I appreciate the time you spent (however unsuccessfully) on the alternate methods. I used to teach people to use Eudora on Mac System…4? 5? in 1994 and on Windows 3.1 as well. Loads of fun. I think CompuSlave, err, CompuServe was my second email system. The first was a BBS. Fidonet, perhaps?

  13. Thank you so much ! I am not done yet, but think these instructions will save my bacon, aka my emails from history gone back in time.

    One note on theSTEP 5 MIGRATE EUDORA 7 DATA TO EUDORA OSE
    For Mac OSX, the download file is WRONG. The bz2 file must be RENAMED to dmg, see discussion here;
    https://wiki.mozilla.org/Talk:Eudora_Releases

  14. Also, the TOC format apparently varies by version and by platform. The python scripts don’t work on my Mac files, which use the same latest 2006 version of Eudora. It’s Eudora v6 on Mac, v7 on Win.

  15. I’m getting ready to try this. I’m using PC not MAC. I’ve seen comments that it may not work on a MAC. I tried to do this myself before I found your article and ran into many of the same issues you’ve described. What you’ve laid out here is by for the most comprehensive information on this kind of migration available. Using hindsight, is there anything you’d change about these instructions?

  16. Hi, I have the same problem as Patirck McKane. I had the same issue, solved installing python 2.7, but now I have the same “UnboundLocalError: local variable ‘map’ referenced before assignment” error. Did you have any hint?

    Thank you in advance,
    Paolo

  17. I read through this and got a bit lost, but it seems to be concerned only with migrating e-mail. I need to migrate only the Eudora Address Book to either GMail or Mac Mail. Have you posted any suggestions for that?

  18. I am not quite sure from your description how to run the python scripts after running the “import os” and “os.chdir” commands. What command do I run next to invoke the .py script?
    Anyway I finessed my lack of knowledge by embedding the “import os” and “os.chdir” commands in the “FixEudora7Mail.py” script and then executing “python.exe FixEudora7Mail.py”. I then received the same error as reported above (“UnboundLocalError: local variable ‘map’ referenced before assignment”).

    Any help appreciated,
    Mike

  19. Dave, thanks again for all this great info. I’m getting an error when running the Python script. I’m using v2.7.6 FYI. It runs through a few of my mailboxes but hits a snag pretty quickly and I didn’t see any references to this kind of error in your write-up or in the comments. Here’s the output:

    Parsing E:\Eudora Data Files
    ACF 463 messages 150 dup From 39 dup Date 0 noDate 0 multiDates, 0 463 mismatched Date,From vs TOC
    Billing 19,095 messages 12,406 dup From 4,688 dup Date 284 noDate 0 multiDates, 23 19,095 mismatched Date,From vs TOC
    Dimitris 412 messages 12 dup From 0 dup Date 234 noDate 0 multiDates, 0 412 mismatched Date,From vs TOC
    InBox1996 436 messages 0 dup From 15 dup Date 4 noDate 0 multiDates, 10 436 mismatched Date,From vs TOC

    Traceback (most recent call last):
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 363, in
    main()
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 300, in main
    = parseEudoraMailbox(path, name, fixedName)
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 129, in parseEudoraMailbox
    dateStamp = readDate(line)
    File “E:\Eudora Data Files\common.py”, line 119, in readDate
    return time.mktime(time.strptime(line[6:31], “%a, %d %b %Y %H:%M:%S”)) # nominal syntax from RFC4155
    OverflowError: mktime argument out of range

    Looks like a date issue. But I am unsure how to get past it. If you can provide any advice it would be appreciated, thank you so much again.

  20. I just wanted to add that I tried to run the script again without changing anything else and I got this output, with the similar error as others have reported:

    Parsing E:\Eudora Data Files

    Traceback (most recent call last):
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 363, in
    main()
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 291, in main
    displayName = map[name.lower() + “.mbx”] # lookup display name for this file
    UnboundLocalError: local variable ‘map’ referenced before assignment

  21. Sorry for all the posts, but I figured my thinking-out-loud might help somebody else out there. Ok, so I got this “UnboundLocalError: local variable ‘map’ referenced before assignment” error. I confirmed that map = {} existed on line 265 as you suggested but I still get the error. So, I added another map = {} just above line 291 and this made the error go away.

    Unfortunately when I ran the script again I’m back to time / date issues, here’s the output:

    Parsing E:\Eudora Data Files
    ACF 463 messages 39 dup From 39 dup Date 0 noDate 0 multiDates, 0 463 mismatched Date,From vs TOC
    Billing 19,095 messages 4,959 dup From 4,688 dup Date 284 noDate 0 multiDates, 23 19,095 mismatched Date,From vs TOC
    Dimitris 412 messages 0 dup From 0 dup Date 234 noDate 0 multiDates, 0 412 mismatched Date,From vs TOC

    Traceback (most recent call last):
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 364, in
    main()
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 301, in main
    = parseEudoraMailbox(path, name, fixedName)
    File “E:\Eudora Data Files\FixEudora7Mail.py”, line 104, in parseEudoraMailbox
    fromStamp = time.mktime(time.strptime(line[:-1], “From ???@??? %a %b %d %H:%M:%S %Y”))
    OverflowError: mktime argument out of range

    This is slightly different than the first time / date issue I posted. The script isn’t making it as far as before, but it does get through a few of my mailboxes so it isn’t as if it doesn’t work at all. I assume there is a completely corrupted date stamp on one of my emails and the script doesn’t know what to do with it.

    By the way, what exact version of Python 2.7 did you use? I’m trying 2.7.6.

    Once again, if you can provide any help it would be appreciated.

  22. Ok, so…I’ve been continuing to work on this. I have all the email in Tbird now and had it index each mailbox. I noticed in my email boxes that were Outboxes (I broke all my In and Out mailboxes up into years) several emails were now incorrectly dated 02/07/2101 for some reason. Always that date.

    I don’t know if it is enough mail to really worry about, maybe 100 messages total.

    Is there a way to hand edit a message to change the date?

    Also, on my old XP machine, to get Python to accept the location of the Tbird datafiles I had to put an “r” in the command, like this (example):

    os.chdir(r’C:\Documents and Settings\Administrator\Application Data\Thunderbird\Profiles\twx96lgk.default\Mail\Local Folders’)

    Without that r before the path it was chocking on the period in the “unique key” section of the path. Just an FYI.

    I did complete the project but am unsure about the new timestamps that were placed and I had about 400 or so emails that ended up with a date of 02/07/2101 for some reason. But I was moving thousands of message, so all in all I feel ok about it. Thank again Dave!

  23. Hi CM

    How did you manage to overcome the “OverflowError: mktime argument out of range”?

    I am also getting this.

    I also put map = {} just above line 291

    Thnx

  24. I don’t know enough about Python to make this work. I can’t get the FixEudora script to run. Do I enter it’s location in the Python command line? I double clicked on the script file and chose open which did nothing. What is the exact command I need to enter? How do I save or print the outputs? I hole someone can help me. The rest of this page I understand.

    Thank You

  25. Why does Eudora sometimes forget to ask me for my password?
    This happens both when sending and receiving email. It forgets to ask me for my ISP password.

    Thanks,
    Gary

  26. Really great sollution! The last 4 weeks I was busy trying to get my old backup Eudora mail to Gmail. No success. My mails kept on becomming corrupt in the process. Finally ended up on this page. I did the transfer to Gmail with success. I think some messages dissapeard in the process. When the mailboxes were reindexed I saw some numbers going down a bit. But the good thing is that I’ve got the main part.

    I also got the message “OverflowError: mktime argument out of range”, but after a retry it was OK. No idea what went wrong.

    I just want to mention a method, which came closest to a solution, but wasn’t mentioned here as something you don’t want to try. It was a combination of a tool called EUDRESCU (for fixing Eurdora mbx files for Thunderbird import) and EMPTI (for solving lost attachments of outgoing mail). But somehow the indexing kept on getting corrupt. So don’t try it and use the info on this page.

    O yes, one other thing, the link to Eudora OSE didn’t work anymore. Found it somewhere else via http://eudoraose.soft32.com and had to select a mirror site, because I wouln’t start the download automatically.

    Thanks for this great info…

  27. I moved from Eudora to gmail many months ago but never removed OSE as a client which I use to send invoices directly from my accounting program.
    Now (Nov 2015) Eudora won’t work due to gmail changes.
    Is there a way to sendmail directly from my accounting program through gmail?

  28. I have Eudora 7 checking a Gmail account using IMAP. I copied messages from another personality in Eudora to my Gmail inbox. HTML messages are displayed as plain text in Gmail instead of rendering the message in HTML. This is why we cannot just use Eudora to upload messages to Gmail using IMAP. Gmail will not display the uploaded HTML messages correctly.

Leave a Reply

Your email address will not be published. Required fields are marked *