mark.bockenstedt.net » Google

Backup Your Gmail With… Gmail Backup

Mark — Wed, 05 Nov 2008 19:20:34 +0000

Yesterday’s article was written giving instructions on how to back up all your email accounts using some pretty simple techniques. In the end, Thunderbird was the agent that we used to grab all the email from our gateway account. Today, I’d like to show you how we can accomplish the same thing with a different service called Gmail Backup.

Rather than copy and paste all of yesterday’s article to this one, please read that article and come back to this one. The only difference will be in step 3, which is the step in which we’re actually pulling email to the local machine.

New Step 3

The previous method used POP to download email. This step will use a different protocol called IMAP. Here’s the info you’ll need for enabling IMAP on your gateway account. Once you’ve got IMAP enabled, you need to download the Gmail Backup program, available for Windows or Linux. Once you have your client of choice, jump to the appropriate section below for instructions on setting up the client.

Generic Setup

For the most part, this software can be used uniformly across multiple platforms. If you want to use the GUI version in Linux, you need to install the WxPython packages (python-wxgtk2.8 in Ubuntu). You can also run it from the command line in any platform. With command line access comes the ability to script and schedule updates! The command line version may also work on OS X, but the GUI version doesn’t (yet).

GUI Version

I’ll just walk through this once, since it’s the same for both platforms. After installing (or extracting) Gmail Backup, launch the executable. If you’re on Linux, you want to run the gmail-backup-gui.sh file. You’ll be presented with a window like this one:

Fill in the first three text boxes in the GUI. If you only want to grab email for a date range, you can specify that in the Since Date and Before Date fields. This will allow you to grab email from only a certain timeframe, which makes it easier to archive your email by month.

Once you’ve got your archive, you can put it in your Dropbox or another safe location.

Linux – Command Line

I want to spend more time on this part because, let’s face it, I’m lazy and I like having things done for me. When run from the command line, the Linux variant of Gmail Backup can be be extremely powerful. You can easily set up Cron jobs to run the backup at a defined interval and use command line arguments to drill down to exactly what you want and where you want to place the backup.

1. Install Dropbox (optional)

If you plan on using Dropbox to store your backups, you can find Linux Dropbox packages here. When you’re run Dropbox and have defined a folder to use, you can proceed.

2. Tweak Command

In the folder you extracted from the Gmail Backup archive, you’ll find gmail-backup.sh. There are a number of command line arguments you can pass to this bash script. Whichever method you choose, you should consider appending 2>&1 /dev/null at the end because the command will generate a considerable amount of output. I only recommend doing this AFTER you’ve run it successfully so you know that it’s doing what its supposed to.

a. Cumulative Backup

For a cumulative backup (i.e. everything), simply use the following.

./gmail-backup.sh backup /path/to/backup/dir gateway@gmail.com password

b. Incremental Backup

Here’s a shot of what the file structure should look like on your disk (this is all in my home directory).

Incremental backups are more useful and easier to organize. Here’s the syntax for that.

./gmail-backup.sh backup /path/to/backup/dir/subdir gateway@gmail.com password 20081001 20081031

The last two arguments specify a start and end date for archiving. This example will archive everything from October 1, 2008, to October 31, 2008. But how do you calculate those dates in Bash, you ask?

You can use the –date parameter of the date command to get past and future dates. For instance, if you want to run a backup of your email for the last week (not including today, obviously), you can do something like this:

./gmail-backup.sh backup /path/to/backup/dir/subdir gateway@gmail.com password `date +%Y%m%d --date="7 days ago"` `date +%Y%m%d --date="1 day ago"`

I realize that’s pretty ugly, but it does the trick. If you’re ambitious enough to use a wrapper to get full control over directories, timeframes, etc., you can use something similar to this.

3. Insert Into Cron

This is the easy part – all you have to do is decide on a frequency and then make sure your $sd variable in the bash script matches that. If you run it every week, $sd should go back 7 days; if you run it every month, it should go back 1 month, etc. The command to run the script every Sunday at midnight:

0 0 * * 7 cd /home/mark/email_backup; ./gmail-backup.sh;

Restoring Archives

It’s important that I tell you that you can also restore backups made this way. However, I feel that it’s outside of the scope of this article. For info on that, check out the documentation page of Gmail Backup’s website.

How To Use Gmail as an Email Backup Service

Mark — Tue, 04 Nov 2008 20:05:47 +0000

With recent horror stories of Google users getting locked out of their accounts floating around the Internet, it makes you reconsider putting so much trust in Google. A lot of people have all their eggs in the Google basket, but they could too be locked out at any given time. Gmail could also go down or lose some of your email when you need it most. Here’s a method for archiving all your email to a local machine using a Gmail account and a POP3 email client.

1. Create a Gateway Email Account

I call this a gateway account because all your archived email will pass through this account. If you’ve already got one, make sure you can dedicate it to this sole purpose. If not, head over to Gmail and sign up for an account. Once you’ve got the account set up, we can start forwarding email to it.

2. Gathering Email

There are two ways to accomplish this and both methods are outlined assuming you’re getting email from another Gmail account (Google Apps for Your Domain works this way too). Method A involves forwarding a copy of your email to your gateway account; Method B involves grabbing email using POP3 from your gateway account.

a. Forwarding Email

This is pretty straight forward. Go into each of the Gmail accounts you wish to forward email from and click on the Settings link in the upper right, then click on the Forwarding and POP/IMAP tab. Check the radio button next to “Forward a copy” and then enter the address for your gateway account. You’ll probably want to select keep Gmail’s copy in the inbox, especially if you intend on using the address. When you’re finished, click Save Changes.

b. Popping Email

Gmail has directions on enabling POP for your archive accounts, which is just step one. The beauty of this method is that you can set this up for any account that supports POP, which is just about every email service on the planet. Even if you’re not using Gmail, you can employ similar techniques to get this working.

Once you’ve enabled POP for your account, you can start pulling email from up to five accounts into your gateway account. Again, Google has directions on enabling Mail Fetcher, so check those out.

3. Setting Up the Email Client

There are a ton of applications you can use to do this, but I recommend using Mozilla Thunderbird (it’s free, fast, and just good). The basic idea is to set up Thunderbird to grab copies of email from your gateway account – and subsequently all other accounts funneled into it – and store them locally. Google already has good directions on how to do this, so I won’t reinvent the wheel.

Some tips for setting up your POP3 client are to make sure you enable POP for all mail and keep Gmail’s copy in the inbox (just in case). If you’re funneling high volumes of email into your gateway, you’ll want to keep an eye on your gateway’s disk quota so you don’t go over the limit. You could also tell Gmail to remove the server’s copy once you download it. After you do that, you can proceed with setting up Thunderbird.

FYI – You can also do this with Apple Mail 3.0, Outlook Express, Outlook 2003, Outlook 2007, or Windows Mail.

4. Safely Storing the Archive

The final step is to make sure you’re storing the archive somewhere safe. If you trust your computer, you can just leave it as is. But, the whole point of this article is having a backup, so I recommend putting a copy of the email archive somewhere safe, such as Dropbox. Any cloud storage solution will work and most offer around 2GB of storage, which should be plenty. You can also use an old-fashioned CD or thumb drive if you so desire.

To grab the mail archive from Thunderbird, you need to find your profile folder. This folder is generally a hidden folder, so you’ll need to unhide folders temporarily if you’ve got them hidden. The x characters below will be a random string, so don’t actually look for xxxxxxxx.default.

In XP, go to C:\Documents and Settings\[User]\Application Data\Thunderbird\Profiles\xxxxxxxx.default\
In Vista, go to C:\Users\[User]\AppData\Local\Thunderbird\Profiles\xxxxxxxx.default\
In Leopard, go to ~/Library/Thunderbird/Profiles/xxxxxxxx.default/
In Ubuntu, go to ~/.thunderbird/xxxxxxxx.default/

Your safest bet is to copy the entire profile folder to ensure you get everything. Once you’ve got that, throw it into your Dropbox in a folder named email_backup or something similarly indicative of its contents. Keep in mind the folders you’ll put into Dropbox are cumulative, meaning they’ll contain all the email that Thunderbird knows about.

5. Automate

Well, this isn’t really automation in the true spirit, but you can leave your Thunderbird running in the background grabbing email every 5 or so minutes. Then, you can set up automatic exports in Linux or OS X to copy your backup into your Dropbox. I don’t have advice on how to do that in anything other than Linux, so I’ll skip this step. Just be aware that it’s possible.

All your email are belong to you…

That’s it, 5 simple steps to backing up your email. Look for an article soon on importing email to Thunderbird from this backup. I’ll also be covering a similar solution using Gmail Backup which will use a lot of the same techniques, but will omit the use of Thunderbird.

Google Alerts, RSS Style

Mark — Fri, 31 Oct 2008 13:55:54 +0000

Yesterday Google allowed users to subscribe to their alerts via an RSS feed instead of email. I prefer this option as my RSS reader is open almost all day and I like to know about things as they happen. Sure, that was always available for the email notice too, but RSS is so much easier.

To get an alert through RSS, head over to google.com/alerts and set up (or change) an alert’s “deliver to” menu to feed.

The frequency menu will go away because RSS is delivered as it happens. Here’s what you should see if you se it up correctly.

You’ll be given an RSS link, which you can copy and paste into your reader of choice (or add it straight to Google Reader). After I heard about this yesterday, my alerts went from three to about 10. If you’re a blogger not already using Alerts, you really should set it up. It only takes a few minutes and it’ll let you know whenever your keywords are mentioned elsewhere. Alerts you should set up include your name, your blog’s name, your Twitter handle, etc.

As always, ReadWriteWeb has a fantastic article on using these new feeds with Yahoo! Pipes to filter out duplicates and do other fancy things, so check that out.

Creating a Fluid-Based Google Reader App

Mark — Tue, 23 Sep 2008 14:44:55 +0000

For Mac OS X: I’ve been looking for a solution to feed my addiction to Google Reader without having to go the the website every time. There’s an AIR application called ReadAIR, but I didn’t like it very much. I really like the usability of the Reader interface, so I would be hard-pressed to use anything else.

I found a solution over on FlingMedia that allowed me to use Fluid to create an SSB (Site Specific Browser) for Google Reader. The end result is that I have a dedicated “application” that does Reader and only Reader. I decided to install the theme as described in the article and am pleased with the end result.

There were a few modifications I made to the CSS for the Fluid app. For everything else, you can get directions from the article on FlingMedia. If you want to use my CSS modifications, you can get them here. Go ahead and modify them to your needs if you so desire. Since Fluid also supports user scripts, you can also use JavaScript to customize the page if you so desire. If you want to get really fancy, you can even add Growl notifications and Dock badge updates to tell you when new items arrive.

Screenshot

Noise Filtering on Google Search Results

Mark — Sun, 24 Aug 2008 16:00:22 +0000

Firefox with Greasemonkey: If you’ve ever searched for your website content on Google (and chances are you have), you’ll agree with me that it can be difficult to find your material amongst the results. Sometimes I search for my posts and see where they rank in specific queries to Google. Usually it takes me a while to find it because I’m impatient and skip over the link accidentally.

Off to trusty ol’ Greasemonkey we go for a quick script to run through each search result on the page and see if it points to my site (or has my site’s URL in it). If it does, it gets highlighted with a pretty orange box to make it stand out.

I also wanted to know if my first and last name were mentioned in any of the descriptions on the current page. These results get highlighted with a red box.

Some quick, easy noise filtration on Google’s results makes life much easier. Here’s the code for the script (feel free to use and modify it as you see fit).

// ==UserScript==
// @name           My Google Results
// @namespace      google.com
// @include        http://www.google.com/search?*
// ==/UserScript==
 
// Enter the name of the URL you wish to find - for multiple sites, separate urls by the vertical bar character |
// You don't need to enter the full url, just the domain is fine. (eg google.com)
var find_site = /mark.bockenstedt.net/i;
 
// If you wish to regex your full name also, enter it here
// Otherwise, comment out the next two lines
var first_name = /mark/i;
var last_name = /bockenstedt/i;
 
// Get a list of all li elements
var lis = document.getElementsByTagName("li");
for(var i = 0; i &lt; lis.length; i++)
{
	// the li we're working with
	var li = lis[i];
 
	// for search results, the element has a class of g
	if(li.className = "g")
	{
		// get the anchor and the result text
		var a = li.getElementsByTagName("a")[0];
		var d = li.getElementsByTagName("div")[0];
 
		// regex match the anchor href
		var result = a.href.match(find_site);
		if(result != null)
		{
			li.style.border = "2px solid orange";
			li.style.width = "600px";
		}
		// regex match the name
		else if(first_name &amp;&amp; last_name)
		{
			if(d.innerHTML.match(first_name) &amp;&amp; d.innerHTML.match(last_name))
			{
				li.style.border = "2px solid red";
				li.style.width = "600px";
			}
		}
	}
}

Optimize Google Reader for Wide Screens

Mark — Thu, 24 Jul 2008 16:35:56 +0000

Firefox only

I recently heard (via Lifehacker) of this GreaseMonkey script that will optimize your Google Reader window to fill 100% of the available horizontal space. This works not only for wide screens, but for standard screens as well. In my experience, Reader doesn’t maximize its horizontal space by default, but this script modifies the style for each article to fill that space.

To use this script, you must have the GreaseMonkey add-on installed. Download the script here and add it to GreaseMonkey. Then enjoy the pleasure of reduced vertical scrolling.

Website Stat Trackers

Mark — Fri, 18 Jul 2008 21:17:14 +0000

Everyone who runs a website should be keeping track of the number of visitors they have. There are many tools that allows people to do this. WordPress, for example, has a built-in tool that gives you information such as page views, referrers, top posts, clicks (links clicked on your blog), and incoming links. Google Analytics also has a tool that you embed into the body of your pages that sends a request over to their trackers. Google Analytics is much more in-depth than WordPress is, and tracks things such as visitor trending, visitor loyalty, page views, visitors, unique visitors, absolute unique visitors, where hits came from, and much more. It also provides a look into the client’s machine, providing information on which browser they used, what speed their connection was, what OS they used, what version of Flash they have, etc.

It’s somewhat scary to see the amount of information that Google Analytics provides. For the hardcore web admins, it’s great to know exactly what your readers are using so you can customize your site to what the readers use. All in all, it provides web admins with all the information they need to know about their site and the people viewing it.

Since I have two stat trackers running, it’s interesting to compare the numbers between the two. For instance, WordPress reports my site had 934 page views on July 17. Google reports only 790 page views. The same is to be said with top post views; WordPress reports 694 hits on this article, while Google only reports 660. So what gives? Why are the numbers so different?

I assumed whatever stat tracker I installed on my site would be accurate. For as long as this site has been up, I’ve taken Google’s word as bond. Now I’ve been introduced to a level of uncertainty that I can’t explain. I will admit that for the most part, I’m micro-managing. After all, they’re just numbers, right? Wrong. I’ve seen the statistics and can tell which types of articles are more popular, and I try to write posts about that so people will come back and enjoy the site more. Stats are a driving force not just in blogging, but in the world as a whole.

So now I’m just confused. I feel a little like Luke Skywalker who’s trying to be convinced by Darth Vader to join the Dark Side (Google Analytics). Obi-Wan is telling me to trust my instincts and use The Force (WordPress Stats). OK, maybe I’m not as cool as Luke, nor is Vader really trying to woo me. And Obi-Wan, he’s just there because my imagination runs wild. Maybe I should seek the aid of Yoda, the stat tracker that is wise beyond its years and knows the truth. So, it’s off to Dagobah I go to find just that.

mark.bockenstedt.net » Google

Backup Your Gmail With… Gmail Backup

Linux – Command Line

Related Posts

How To Use Gmail as an Email Backup Service

Related Posts

Google Alerts, RSS Style

Related Posts

Creating a Fluid-Based Google Reader App

Related Posts

Noise Filtering on Google Search Results

Related Posts

Optimize Google Reader for Wide Screens

Related Posts

Website Stat Trackers

Related Posts