Rather than copy and paste all of yesterday’s article to this one, please read that article and come back to this one. The only difference will be in step 3, which is the step in which we’re actually pulling email to the local machine.

New Step 3

The previous method used POP to download email. This step will use a different protocol called IMAP. Here’s the info you’ll need for enabling IMAP on your gateway account. Once you’ve got IMAP enabled, you need to download the Gmail Backup program, available for Windows or Linux. Once you have your client of choice, jump to the appropriate section below for instructions on setting up the client.

Generic Setup

For the most part, this software can be used uniformly across multiple platforms. If you want to use the GUI version in Linux, you need to install the WxPython packages (python-wxgtk2.8 in Ubuntu). You can also run it from the command line in any platform. With command line access comes the ability to script and schedule updates! The command line version may also work on OS X, but the GUI version doesn’t (yet).

GUI Version

I’ll just walk through this once, since it’s the same for both platforms. After installing (or extracting) Gmail Backup, launch the executable. If you’re on Linux, you want to run the gmail-backup-gui.sh file. You’ll be presented with a window like this one:

Fill in the first three text boxes in the GUI. If you only want to grab email for a date range, you can specify that in the Since Date and Before Date fields. This will allow you to grab email from only a certain timeframe, which makes it easier to archive your email by month.

Once you’ve got your archive, you can put it in your Dropbox or another safe location.

Linux – Command Line

I want to spend more time on this part because, let’s face it, I’m lazy and I like having things done for me. When run from the command line, the Linux variant of Gmail Backup can be be extremely powerful. You can easily set up Cron jobs to run the backup at a defined interval and use command line arguments to drill down to exactly what you want and where you want to place the backup.

1. Install Dropbox (optional)

If you plan on using Dropbox to store your backups, you can find Linux Dropbox packages here. When you’re run Dropbox and have defined a folder to use, you can proceed.

2. Tweak Command

In the folder you extracted from the Gmail Backup archive, you’ll find gmail-backup.sh. There are a number of command line arguments you can pass to this bash script. Whichever method you choose, you should consider appending 2>&1 /dev/null at the end because the command will generate a considerable amount of output. I only recommend doing this AFTER you’ve run it successfully so you know that it’s doing what its supposed to.

a. Cumulative Backup

For a cumulative backup (i.e. everything), simply use the following.

./gmail-backup.sh backup /path/to/backup/dir gateway@gmail.com password

b. Incremental Backup

Here’s a shot of what the file structure should look like on your disk (this is all in my home directory).

Incremental backups are more useful and easier to organize. Here’s the syntax for that.

./gmail-backup.sh backup /path/to/backup/dir/subdir gateway@gmail.com password 20081001 20081031

The last two arguments specify a start and end date for archiving. This example will archive everything from October 1, 2008, to October 31, 2008. But how do you calculate those dates in Bash, you ask?

You can use the –date parameter of the date command to get past and future dates. For instance, if you want to run a backup of your email for the last week (not including today, obviously), you can do something like this:

./gmail-backup.sh backup /path/to/backup/dir/subdir gateway@gmail.com password `date +%Y%m%d --date="7 days ago"` `date +%Y%m%d --date="1 day ago"`

I realize that’s pretty ugly, but it does the trick. If you’re ambitious enough to use a wrapper to get full control over directories, timeframes, etc., you can use something similar to this.

3. Insert Into Cron

This is the easy part – all you have to do is decide on a frequency and then make sure your $sd variable in the bash script matches that. If you run it every week, $sd should go back 7 days; if you run it every month, it should go back 1 month, etc. The command to run the script every Sunday at midnight:

0 0 * * 7 cd /home/mark/email_backup; ./gmail-backup.sh;

Restoring Archives

It’s important that I tell you that you can also restore backups made this way. However, I feel that it’s outside of the scope of this article. For info on that, check out the documentation page of Gmail Backup’s website.

Related Posts

• How To Use Gmail as an Email Backup Service
• Google Alerts, RSS Style
• Creating a Fluid-Based Google Reader App
• Laconica, XMPP, SMS, Oh My!
• Noise Filtering on Google Search Results

1. Create a Gateway Email Account

I call this a gateway account because all your archived email will pass through this account. If you’ve already got one, make sure you can dedicate it to this sole purpose. If not, head over to Gmail and sign up for an account. Once you’ve got the account set up, we can start forwarding email to it.

2. Gathering Email

There are two ways to accomplish this and both methods are outlined assuming you’re getting email from another Gmail account (Google Apps for Your Domain works this way too). Method A involves forwarding a copy of your email to your gateway account; Method B involves grabbing email using POP3 from your gateway account.

a. Forwarding Email

This is pretty straight forward. Go into each of the Gmail accounts you wish to forward email from and click on the Settings link in the upper right, then click on the Forwarding and POP/IMAP tab. Check the radio button next to “Forward a copy” and then enter the address for your gateway account. You’ll probably want to select keep Gmail’s copy in the inbox, especially if you intend on using the address. When you’re finished, click Save Changes.

b. Popping Email

Gmail has directions on enabling POP for your archive accounts, which is just step one. The beauty of this method is that you can set this up for any account that supports POP, which is just about every email service on the planet. Even if you’re not using Gmail, you can employ similar techniques to get this working.

Once you’ve enabled POP for your account, you can start pulling email from up to five accounts into your gateway account. Again, Google has directions on enabling Mail Fetcher, so check those out.

3. Setting Up the Email Client

There are a ton of applications you can use to do this, but I recommend using Mozilla Thunderbird (it’s free, fast, and just good). The basic idea is to set up Thunderbird to grab copies of email from your gateway account – and subsequently all other accounts funneled into it – and store them locally. Google already has good directions on how to do this, so I won’t reinvent the wheel.

Some tips for setting up your POP3 client are to make sure you enable POP for all mail and keep Gmail’s copy in the inbox (just in case). If you’re funneling high volumes of email into your gateway, you’ll want to keep an eye on your gateway’s disk quota so you don’t go over the limit. You could also tell Gmail to remove the server’s copy once you download it. After you do that, you can proceed with setting up Thunderbird.

FYI – You can also do this with Apple Mail 3.0, Outlook Express, Outlook 2003, Outlook 2007, or Windows Mail.

4. Safely Storing the Archive

The final step is to make sure you’re storing the archive somewhere safe. If you trust your computer, you can just leave it as is. But, the whole point of this article is having a backup, so I recommend putting a copy of the email archive somewhere safe, such as Dropbox. Any cloud storage solution will work and most offer around 2GB of storage, which should be plenty. You can also use an old-fashioned CD or thumb drive if you so desire.

To grab the mail archive from Thunderbird, you need to find your profile folder. This folder is generally a hidden folder, so you’ll need to unhide folders temporarily if you’ve got them hidden. The x characters below will be a random string, so don’t actually look for xxxxxxxx.default.

• In XP, go to C:\Documents and Settings\[User]\Application Data\Thunderbird\Profiles\xxxxxxxx.default\
• In Vista, go to C:\Users\[User]\AppData\Local\Thunderbird\Profiles\xxxxxxxx.default\
• In Leopard, go to ~/Library/Thunderbird/Profiles/xxxxxxxx.default/
• In Ubuntu, go to ~/.thunderbird/xxxxxxxx.default/

Your safest bet is to copy the entire profile folder to ensure you get everything. Once you’ve got that, throw it into your Dropbox in a folder named email_backup or something similarly indicative of its contents. Keep in mind the folders you’ll put into Dropbox are cumulative, meaning they’ll contain all the email that Thunderbird knows about.

5. Automate

Well, this isn’t really automation in the true spirit, but you can leave your Thunderbird running in the background grabbing email every 5 or so minutes. Then, you can set up automatic exports in Linux or OS X to copy your backup into your Dropbox. I don’t have advice on how to do that in anything other than Linux, so I’ll skip this step. Just be aware that it’s possible.

All your email are belong to you…

That’s it, 5 simple steps to backing up your email. Look for an article soon on importing email to Thunderbird from this backup. I’ll also be covering a similar solution using Gmail Backup which will use a lot of the same techniques, but will omit the use of Thunderbird.

Related Posts

• Backup Your Gmail With… Gmail Backup
• Google Alerts, RSS Style
• Creating a Fluid-Based Google Reader App
• Laconica, XMPP, SMS, Oh My!
• Noise Filtering on Google Search Results

SMS

Populate carriers table

If you look in the db folder, you’ll find a file called carrier.sql. If you run that against the laconica database you created for the install, all the providers in the file (58 in all) will be available to select in the SMS tab of your instance. I actually reduce the number to 16 because most of them are international services.

How SMS works is the Laconica server will fire off an email to your carrier’s gateway, which then should be delivered as an SMS. Not all carriers play well with this (I’m looking at you, US Cellular) so your mileage may vary in terms of results. I’ve already got a confirmation that AT&T works.

If carriers are missing from the table and you know the carrier’s gateway, it’s really easy to add them to the options. Simply create a new record in the sms_carrier table and fill in the name and email pattern.

Try it out

Head to the Settings -> SMS tab of your instance, enter your phone number, select your carrier, and hit the Add button. You should receive an SMS with a confirmation code, which you’ll have to enter in the new box that appears. Once you do that, your phone should be prime for receiving SMS to your phone.

SMS is one-way

What I mean by that is you can only receive notifications through SMS – you can’t post updates to the service (yet). This simplifies matters because admins won’t have to worry about setting up all the SMS inbound boxes and all that. For now, this will work.

XMPP

This is where things get ugly. If you’ve already got a Jabber/XMPP server, you’re golden. If not… get one.

Configuration

If you already have an XMPP server, you can do this now. Otherwise, install one and come back to this.

Head back to the trusty config.php file and scroll down to the XMPP section and you’ll find a plethora of options regarding XMPP. If you have a server, fill in the appropriate values and you should be good to go.

Choosing a Server

You can probably use whatever server you want to for this, provided it does the job. I chose to use Openfire, a bloated Java server. Frankly, I had no reasons behind the choice other than it worked after some configuration. All directions henceforth will be specific to Openfire.

Installation

Before I jump into the juicy details, bookmark the Openfire Documentation page. It will become your best friend.

sudo apt-get install sun-java6-bin sun-java6-jre sun-java6-jdk
cd /opt
wget http://www.igniterealtime.org/downloadServlet?filename=openfire/openfire_3_6_0a.tar.gz
tar -xzvf downloadServlet?filename=openfire/openfire_3_6_0a.tar.gz

The next step is to set up the database. I’m going to assume you’re using MySQL, but you can use any of the following: DB2, HSQLDB, MySQL, Oracle, PostgreSQL, SQLServer, or SyBase. I’ll also assume you’ve created a database to hold Openfire info (database named openfire).

cd /opt/openfire/resources/database
mysql -u [db_user] -p [db_pass] openfire < openfire_mysql.sql

Start Server

Now it’s time to kick this bad boy off and see if it works:

cd /opt/openfire
./bin/openfire start

If it complains of not being able to find a JVM, try doing this:

sudo update-alternatives --config java

and select a different JVM. The JVM installed from above is at /usr/lib/jvm/java-6-sun/jre/bin/java so that’s what I selected. I have no idea if that impacts the rest of the system at all.
OK, so now your server should be running. Time to set it up.

Setup

If you were able to run the server without incident, point your browser to port 9090 on the server you installed Openfire on. Follow the on-screen prompts. I screwed the pooch and didn’t take screenshots of the process.

Create an update user

Go into the Users/Groups page in your Openfire Admin Console (OAC) and create a new user and give it a password. I named my user update, but the name is arbitrary.

Update the Laconica config

If you’re just installing an XMPP server, you need to make sure to tell Laconica about your new Openfire user. Here’s what my XMPP config looks like:

$config['xmpp']['enabled'] = true;
$config['xmpp']['host'] = 'bockenstedt.net';
$config['xmpp']['server'] = 'bockenstedt.net';
$config['xmpp']['port'] = 5222;
$config['xmpp']['user'] = 'update';
$config['xmpp']['encryption'] = false;
$config['xmpp']['resource'] = 'xmpphp';
$config['xmpp']['password'] = '';
$config['xmpp']['public'][] = 'imbuddy@bockenstedt.net';

Simply change the above fields where appropriate and Laconica should be informed about your XMPP buddy. I dont’ know what the last line is for, so if anyone can clue me in, please do via a comment or email. If you look now, your update user should be offline in the OAC.

Unleash the daemons

This is the step that tripped me up. In the scripts directory there are a number of handlers and daemons that need to be running in order for everything to work properly. There’s also a handy little shell script that executes all the necessary daemons. You can run all the daemons at once by typing:

./startdaemons.sh

For some reason, I got errors when the script tried to open the files so I just started the daemons manually.

php xmppdaemon.php
php jabberqueuehandler.php
php publicqueuehandler.php
php xmppconfirmhandler.php
php smsqueuehandler.php
php ombqueuehandler.php

That should do it for getting the daemons running. If you look in your OAC again, your update user should be online.

Open ports

The final step is to open a few ports to outside access so users can communicate with your XMPP server. You need to open 5222, 5223, and 5296 for traffic to flow properly. If you’re going to need OAC access outside of your firewall, you have to open port 9090.

Caveats

Using the above methods, I was able to connect my Gmail account with the XMPP server and send updates successfully. However, another user who is using an afilliate Google account (ie john@doe.com signed up through Gmail) was unable to connect to the server.

Related Posts

• Backup Your Gmail With… Gmail Backup
• How To Use Gmail as an Email Backup Service
• How To Install Laconica