I recently installed Lessn and then Lessn More to take control of my own short links. Lessn is a very simple URL shortener that is hosted on my own domain.

I registered upton.in to serve my short links. I chose the “.in” TLD because it’s short and common. I do wish there was an “.on” TLD so I could register upt.on, but I like the connotation of “in”. As I said on Twitter the other day, it can mean “Internet, inside, come in, infinity, interesting” among other things.

Lessn also integrates nicely with Mint, which I recently started using. I plan to write more on that shortly.

Santa brought me The Legend of Zelda: Skyward Sword for Christmas (thanks Mom!). I’ve enjoyed playing it, just as I’ve enjoyed playing video games for nearly my entire life.

In recent years, however, I’ve abandoned or not fully played all of my video games. I attempted to maintain a Google Docs spreadsheet to keep track of my unplayed games, but it quickly became more neglected than the games themselves.

I recently found out about Shaun Inman’s Unplayed, and it seemed perfect. It’s like a much simpler, lightweight Shelfari for video games. You can now see which games I’m playing (or not playing) at my Unplayed page. For now, I’m going back to Skyloft!

The first Christopher Nolan movie I saw was the incredible Memento. I’ve watched it many times since, and I always manage to catch something new. It’s a great movie the first and tenth time you watch it. It’s well-written – with some help from his brother Jonathan, who has helped out on other films since – and the attention to detail is incredible.

I think Nolan caught the attention of most of the movie-going public for his reboot of the Batman franchise. Batman Begins is one of the best comic book translations, outdone only by its sequel, The Dark Knight. Nolan wisely decided to take breaks between Batman films to make other films that he was interested in. He managed to pull this off because of his own production company, Syncopy Films, and his success as a director. As a result we’ve got two great films so far: The Prestige and Inception.

I love that Nolan has focused on other projects that he’s passionate about to avoid creating anything “stale” or boring. I think there’s a larger lesson there that most people can take away. Sometimes you can make something better by not giving it attention.

If Nolan’s “interstitial” movies keep improving at the same rate, we’re in for a treat in 2014.

I recently started using Byword on OS X and Elements on iOS for my writing in general. Both apps are Markdown editors that, in conjunction with Dropbox, let me write from nearly anywhere.

I write my blog posts in Markdown, but the format is useful for almost any kind of writing. Any application that can read plain text can read Markdown for those times that you need to keep it simple, or you can convert to HTML for when you want or need to make it a bit more presentable.

I love using Byword in its fullscreen and paragraph focus view. It’s a great way to minimize distractions with enough context to let you properly proofread your writing. The Markdown to HTML preview is a keyboard shortcut away, and saving to a folder in Dropbox means that I can access my files anywhere.

Elements requires a Dropbox account to work, so I specified the same folder that I save documents from Byword into. Et voilà, I can edit or add to a blog post or any other writing on my iPhone or iPad. Elements has a great Markdown preview, too, and it’s distraction-free (from other apps) by nature of being an iOS app.

I use SuperDuper! to make daily backups to my local hard drive. Remote backup is important, but it’s always nice to have an easily-accessible way to recover from a hard drive disaster, especially when you need to fix the situation quickly.

Your Backup Device

Choosing the right hard drive to use for your local backup isn’t a difficult choice, but it is an important one. Sometimes you can find great deals on good brands (Western Digital or Seagate, to name a couple brands that I have used and liked) on Newegg. The requirements for a hard drive really depend on how you want to use it.

If you want a hard drive dedicated to backup, any drive with a relatively quick speed (I would go with 7200 RPM or higher) and a size that’s at least a bit bigger than the drive you’re backing up will be a great option.

If you want to use a hard drive for other things – to store your media or back up another drive – you’ll obviously need something bigger. I have a 1 TB drive that I have split into two volumes. One volume is a bit bigger than my MacBook Pro’s hard drive (128 GB) and I use it to hold my backups; the other takes up the rest of the space and is used to store all of my music, movies, and photos.

Solid state drives are another option, but I’ve found that the extra cost isn’t worth the speed benefits. I would absolutely recommend a solid state drive to use in your main computer on a daily basis – I have a 128 GB solid state drive in my MacBook Pro and it’s noticeably faster than the mechanical hard drives I’ve used in MacBooks past. However, when it comes to something like backups, where you’re mostly writing to it in a situation that isn’t time-sensitive (e.g. at night), I think mechanical drives are the best option because you can get much more storage space for much less money.

Software That Helps You

I mentioned earlier that I use SuperDuper!, but there are many other backup applications out there. Some of them may fit your specific needs better. The key feature is to have an automated and bootable copy of your hard drive ready to go at a moment’s notice.

I admittedly have not given Time Machine a fair shot, but that’s because I don’t have a use case for its main draw. “Going back in time” to an earlier version of a file is not something I need to do on a regular basis. And now with Lion’s Versions feature, I don’t need a whole backup just to keep recent versions of my files around.

Another option is Carbon Copy Cloner. It makes bootable backups and can be scheduled to back up on a daily basis. It’s free to try; donations are requested if you continue to use the product.

I love SuperDuper! because it’s so simple. The interface tells you in plain English what is going to happen. I set up my nightly backups one time a couple years ago and the only time I ever see the app again is when I am traveling without my hard drive. When a backup fails, the application stays open to tell you what went wrong. That rarely happens though: usually an incremental backup occurs at 3am that takes less than ten minutes to complete. When it’s done, my data is safe and sound in two places.

The successful backups are bootable, meaning that you can just choose the backup volume as the boot volume when you hold Option after you turn your computer on and restore from there. If you just want to restore certain files instead of everything, the drive is right there in Finder for you to navigate as usual.

Backup Is Not a Daunting Task

Almost none of my “non-technical” friends (i.e. friends who do other things in their lives besides fiddle with computers) back up their computers. And yet I know that each one of those friends would be devastated if their hard drive crashed and they lost their music or photos. Local backups are the simplest way to avoid this, and it’s so simple to set up:

• Buy a hard drive (7200 RPM and at least as big as the drive you want to back up) and format it using OS X’s built-in Disk Utility.
• Set up a backup application to do daily backups of your entire disk.

That’s it. Back up your data. It’s so simple and so useful.

The Stop Online Piracy Act (explanation via Kottke on what’s wrong with SOPA) is bad for the internet in general. GoDaddy supports SOPA, and this has alarmed many of its customers, including this Redditor with fifty one domains and myself.

Again via Kottke, I came across this guide to moving from GoDaddy to NameCheap (Google cache). NameCheap has officially come out against SOPA, and their site and experience has been nothing but top notch since I started the transfer process.

I’ve also heard good things about Hover; by all means shop around for a registrar that fits your needs and doesn’t blatantly support SOPA.

My transfer process to NameCheap is nearly complete, and I couldn’t be happier to leave GoDaddy. Don’t forget that December 29th is Move Your Domain Day!

A Musical Year

Music has been a huge part of my life for a long time now. I started playing the violin in the fourth grade. I played the saxophone for six years, including for four years in my high school marching band. I played in a punk rock band with my best friends from high school for four years. The next and latest chapter of my musical life began when I moved to Austin, TX in June of 2010.

They call Austin the “Live Music Capital of the World”. To many, that sounds like exaggeration and a desperate grasp at something to be proud of. To those who have experienced it, it is almost an understatement. Whether you’ve been punched in the gut in the pit at Emo’s or just seen that gnarly rocker chick band Guilty Pleasures while walking past Thirsty Nickel on a Saturday night, there’s no denying that music is in the blood of Austin. I love it.

2011 was a musical year for me. According to Last.fm, I went to 46 musical events, including SXSW, ACL, and FFFFest. And that doesn’t count the numerous times I caught local acts on the weekends. Some might have thought of 2011 as a lackluster year for music, but I disagree. There were some fantastic releases by some great bands; old favorites thanked their fans with fresh material and new (but sometimes only new to me) acts dug their way into my ears and heart.

A New Medium

There is something else that affected my music listening habits in 2011: Rdio. I signed up in July and immediately started listening to tons of new music. In November, I listened to at least one new album each day as part of Month of New Music (see my #monm playlist.) Rdio tore down any remaining barriers that hindered listening to any music, anywhere, quickly. I chose Rdio over Spotify for a few reasons: Rdio has better social features, better recommendations, a similarly-sized catalogue, better apps on both the desktop and mobile, and a better blog filled with interviews with musicians, reviews, and recommendations.

According to iTunes, I have seventy one albums that were released in 2011. This is lower than any other year that I have been adding music to iTunes, but that is explained by my prolific use of Rdio. I have purchased a few albums after streaming them, but Rdio is almost like having a second library. I briefly looked for statistics on Rdio listening habits, but I couldn’t find any. Believe me when I say I listen to a ton of music every day while working.

I made a list of nineteen albums that were my favorites of the year. I’ve pared it down to ten that I want to talk about, but I’ll mention the others at the end. All in all, it was a great year of music for me.

Running From a Gamble by Company of Thieves

Genevieve Schatz can sing. She has a huge voice, especially for such a small, unassuming person. She’s backed by a band that can simultaneously rock out and complement her vocals. This is a poppy album, but it’s filled with soul and rock. The first song I heard from Running from a Gamble was the incredible “Death of Communication.” From a simple beginning, it quickly builds into a spunky anthem. The outro to that song leaves you wanting more, and more is what you get on Gamble. Every song has character, with lots of soul and even blues influences. See this band live and stand in awe.

Barton Hollow by The Civil Wars

If you had asked me five years ago if I would ever pick a folk or country-influenced album to put on my list of favorites from the past year, I would have laughed. Then I heard Barton Hollow.

This is a bit of a silly and meaningless (to everyone but me) story, but I’ll recount it anyway. I was at a friend’s house for a party over the summer. I had traveled halfway across the country to go, so I was staying over and was the only one left the next day. I have a tradition to trade music with this friend whenever we meet up, so he gave me a number of albums and told me to check one or two of them out while he went to take care of some errands. I just picked Barton Hollow by chance. I put it on and laid down on the bed and just stared at the ceiling and listened to the entire album. It’s so calming and soothing, and it was perfect for that moment.

I’ve since listened to this album many times. Both members of this band can sing incredibly well, and their song-writing and story-telling skills are second to none. The vocals are certainly the focal point of The Civil Wars’ music, but the guitar picking and occasional percussion are nothing to scoff at. This is beautiful music.

The King of Limbs by Radiohead

I’ll never understand why The King of Limbs was poorly received by both critics and fans alike. I tend not to compare Radiohead releases to each other — Kid A is one of my favorite albums, but it doesn’t mean I like it any better than OK Computer or Hail to the Thief, it just means that I enjoy it for what it is and how it makes me feel in comparison to other albums that are not made my crazy Englishmen.

Radiohead is definitely a band all about feel. Limbs seems to have more of an electronic feel to it. There are little “glitches” on the tracks, the percussion is so precise that it’s almost robotic, and synths are prevalent. Radiohead has always had hints of this kind of thing since early on, but it’s out in the open here.

The band released a set of remixes of the album throughout the year, and they actually serve to show how great the album is: each one of these songs can and is interpreted in a number of ways.

I love the muted guitar on “Morning Mr Magpie” and the haunting lyrics of the aptly-named “Give Up The Ghost”. If nothing else, Limbs has made me look forward to where Radiohead takes us next.

Weightless by Animals as Leaders

Two eight-string guitarists and one insane drummer — it’s such an unusual lineup, and Animals as Leaders is an unusual band. Started as a bit of a side or solo project by lead guitarist Tosin Abasi, his debut album featured just one man and programmed drums, but it laid the groundwork for a technical metal act like no other. After recruiting some bandmates, Animals went on tour and proceeded to melt the faces of anyone who got close.

Weightless takes the concepts explored on the debut — polyrhythms, syncopated percussion, quickly sweeping arpeggios — and multiplies everything by ten. Thousand. It’s heavier, more chaotic, and tighter than ever.

This is another band to see live. They somehow still have stage presence despite the lack of a proper frontman and vocals, and Abasi seems to take genuine pleasure in blowing minds with his music. Percussionist Navene Koperweis manages to keep up with both the twisted programmed beats of old and with his own work on Weightless. It is certainly a feat to play something that someone else wrote with a computer; it’s something else entirely to continue to write new material in the same style.

Burst Apart by The Antlers

My immediate reaction to “I Don’t Want Love” was “this is a lighter direction for The Antlers than Hospice.” Peter Silberman’s lyrics weren’t much (or even any) happier, but the music was lighter, more melodic, and a little poppier. There’s more rock influence throughout this record than on Hospice. In fact, there’s more, period. Every song is meatier. While some may say that the sparseness of Hospice was one of its key elements, the full-bodied, well-crafted songs of Burst Apart are even more essential to bringing The Antlers to life.

The main contributors to this full sound are Michael Lerner’s focused percussion and the addition of layered guitar work, provided live by Timothy Mislock. The riffs in Parentheses are a definite stand-out. Both on the stage and in the studio, one can see the passion behind the music. I’ve been lucky enough to catch The Antlers live a number of times, and each show is better than the last, with more energy than they could capture on the record. I’ve seen them on the intimate Parish stage, and I’ve seen them on the giant AMD stage at ACL. No matter the setting, these guys play their hearts out.

Father, Son, Holy Ghost by Girls

Once Father, Son, Holy Ghost gets going (which is right away), Girls start rocking. There’s a bit of Spoon influence on the opening track, and a bit of Wilco throughout, but Girls manage to carve out their own rock niche here. “Die” really puts the pedal to the metal, and crowd-favorite “Vomit” has an incredibly catchy melody.

I admittedly know very little about Girls. I saw that they were recommended to me on Rdio, and I couldn’t stop listening to the album. I saw them live at Fun Fun Fun Fest, and they were amazing, but the record is where they really shine. It’s the sort of rock that doesn’t really stand out from the crowd too much until you realize that you’ve stopped doing whatever else it was you were doing while listening and your undivided attention is given to the music.

Little Hell by City and Colour

With the addition of percussion and backup guitars to nearly every song, Dallas Green has fleshed out his solo project to a full-fledged folk act. His lyrics are still tragic and romantic, but now they have more meat behind them.

I must have listened to Bring Me Your Love two hundred times. I haven’t listened to Little Hell that much just yet, but I know I will. I can’t exactly pinpoint why, but this music is inherently re-playable. It never gets old. I usually chalk it up to Green’s incredible story-telling ability and pleasant accompaniment, but no matter how you try to explain it, this is great stuff.

Prepare to be saddened, but also prepare to leave what you’re doing and enter Green’s lyrical world.

No Devolución by Thursday

Thursday turned a furious corner with 2006’s A City by the Light Divided and hasn’t looked back. No Devolución is the ultimate manifestation of the post rock-influenced dynamic that was only hinted at on their previous two efforts. There is an almost monstrous quality to some of the tracks on No Devolución: “Past and Future Ruins” is chaotic and grandiose all at once. The percussion through the entire album is forceful: the drums drive and the bass pulses. Ghastly yells are next to quiet, almost croaked vocals.

Geoff Rickly’s lyrics have always been somewhat enigmatic, and that is no different here. There are moments of poignancy — “I lost my wedding ring / down the kitchen sink” — but there are still some puzzlers — “There’s a thousand black cars / driving around in my blood stream.”

No Devolución is Thursday’s magnum opus in a literal sense of the phrase. It is great, and it is the sum of all the passion that these guys have for music; a true work of the heart. It is sad that Thursday has decided step down off of the stage, but we can all be thankful for what they left us with.

Garden Window by O’Brother

Huge. Everything about Garden Window is full, monolithic, and epic. I first heard O’Brother live in August at Emo’s inside. Even there on that puny stage, they had a wall-of-sound presence that couldn’t be ignored. I saw them a couple of months later at the new Emo’s East, and they were even tighter and bigger sounding. Needless to say, I was excited about the release of their LP, Garden Window.

Every song on the album is meaningful — there is no filler. There are some lengthy songs here — crowd favorite “Poison!” clocks in at nearly eight minutes, and “Cleanse Me” is fourteen minutes long — but Garden Window never drags. Quite the opposite: each song is filled with so much building tension and dynamic that it feels like you’re going to explode with each listen. It’s hard to put O’Brother into a single category or genre, but it’s very heavy at its heaviest and it’s always unrelenting. This is post-rock for those who like their coffee strong and black.

I highly recommend seeing O’Brother live, in case that wasn’t obvious. They do a great job of capturing their sound on record, but it’s one thing to rock out to Garden Window with your headphones on at your desk and another thing entirely to hear the bass drone through a song with a cello bow while eye-bulging screams are heard over a wall of guitars and drums.

Major/Minor by Thrice

Thrice has delivered great musical experiences for over ten years. They have taken a curvy road to the release of Major/Minor, but it’s been an enjoyable journey every step of the way.

Major/Minor is the most natural and expected follow-up to any album in Thrice’s discography. It builds on Beggars’ grooviness but adds a lot more punch and poignancy. Every riff is carefully crafted, every beat brings you into the song. Every song is killer.

Thrice could not have ended their musical career on a higher note. The band could never put out any material that disappointed me, but I think this surpasses even harsh critics’ expectations. Let’s face it though: who’s a harsh critic of Thrice?

The first song I heard was the lead single and album opener, “Yellow Belly”. Its bluesy riff motif is echoed throughout the track, and right from the get-go we can feel that this is Thrice at their best. Other standout tracks include “Call It In The Air”, “Words In The Water”, and “Disarmed”, but there isn’t any filler here.

I was heartbroken to hear that Thrice was taking a break shortly after ending the tour that supported the release of Major/Minor. Thankfully I was able to see them live one last time. I know that every member of the band reached that decision after much contention. I’ve heard rumors that brothers Riley and Ed Breckenridge are working on something together, Dustin Kensrue remains the music worship leader at his church in Orange County, and I know that Teppei Teranishi cannot stay away from making music from long. I’m interested in hearing the direction that each of these guys go after being part of such a great band for so long.

Like I said earlier, I picked out nineteen total albums this year. Here are the “honorable mentions,” so to speak, with a little bit about what I loved about them.

Mastodon - The Hunter

Less “out-there” than Crack the Skye, it has some of the most poppy songs Mastodon has ever written, but all the face-melting is still there.

Man Man - Life Fantastic

Incredible live act; the song writing just keeps getting better (and more sincere).

Neon Indian - Era Extraña

I only just found out about this band. It’s great music for chilling out or working to.

Fleet Foxes - Helplessness Blues

Vocal heaven. There are so many instruments and layers going on here. This is gorgeous, sleepy music.

Mister Heavenly - Out of Love

“Doom wop” is such a hilariously accurate term, and I can’t wait to hear more from these guys if they keep it going.

August Burns Red - Leveler

These guys keep getting better and better. I don’t know how, but they outdid Constellations here. “Internal Cannon” is a standout track with its salsa breakdown — that’s not a typo.

Florence + The Machine - Ceremonials

A lot more serious this time around, but more heart and passion. I’d love to see this act live.

Childish Gambino - Camp

A lot of people dismiss Donald Glover as another actor-wannabe-rapper, but he’s got the chops. So many great beats and awesome, meaningful lyrics here.

Cloudkicker - Let Yourself Be Huge

It took a lot for me not to include this on my list of ten favorites. The closing track is amazing. The whole album is such a departure from other Cloudkicker material, and I’m curious to see where it goes next. Highly recommended.

A lot of my picks centered around bands that I have also seen live this year. There’s something visceral about watching a band make music right in front of you. Whether you’re hearing a new act live for the first time or you’re enjoying live renditions of some of your old favorites, there’s something to be said for experiencing that beauty up close and personal.

I hope you enjoyed my picks. Please let me know what you think and what your favorite music of 2011 is.

Hi there. Remember me? It’s been a while, I know. Sorry I have not kept in touch. While I would be lying if I promised that I would never go silent for that long again, I have, for mulitple reasons, renewed my interest in writing.

I do not have many topics lined up, but I have the desire to write. Stay tuned for a bit of a year-end cliché and we’ll take it from there.

/+

/@

/f

I remember reading about Tarsnap a couple of years ago, back when it was only an idea. I wasn’t too convinced about using a service that was in beta to back up my data, but I recently rediscovered that it had graduated to a full-blown product and signed up immediately.

Tarsnap is an encrypted backup tool based on archives. I’m not going to go into any details about the implementation, but you can read about the cryptography, the security, or anything else about the overall design of the tool on the Tarsnap site. Basically, it creates archives (hence the “tar” part of the name), encrypts them, and stores them on Amazon S3. The “snap” part of the name refers to the idea that backups are done in “snapshots,” which means that backups are incremental and duplicate data can be shared between archives.

After you sign up for a Tarsnap account, put at least $5 (via Paypal) into your account, and generate a key, you can begin backing up your data. You can read more about getting started and using tarsnap in general, but I really want to talk about automated backups with Tarsnap.

A Simple Wrapper

I found a blog post by Jonathan Street that detailed his automated backups, and that served as inspiration for my system. I wrote a little bash script to wrap tarsnap for my purposes:

1
2
3
4

#! /bin/bash
echo `date +%F\ %T`: Beginning back up of $2
/usr/local/bin/tarsnap -c -f $1-`date +%F` $2
echo `date +%F\ %T`: Completed back up of $2

Calling tarsnap-backup.sh tells tarsnap to create an archive of the specified directory with the given name and the current date. I was in business.

Generating a new key

An aside: Jonathan Street’s blog post mentioned creating a new key that only had permission to read and write archives. I initially did the same thing, but for reasons I’ll get into later, I wanted the ability to delete backups, too. Generating a new key was extremely easy:

1

$ tarsnap-keymgmt --outkeyfile /root/tarsnap-rw.key -r -w /root/tarsnap.key

This creates a new key in /root/ called tarsnap-rw.key that only has read and write permission.

Automation

newsyslog

The simple wrapper script above was great, but if I was going to automate it, I needed those echo statements to go to a more permanent log file. If I was going to do daily backups of directories, I needed some sort of log management. After searching around a bit, it became clear that newsyslog was the way to go on OS X. Looking at the file in /etc/newsyslog.conf was enough to give me the basic file structure, but the man pages go into a lot of detail.

I made a configuration called user.conf in /etc/newsyslog.d/ and put my tarsnap logs inside. I decided to use a distinct log for each automated backup I do, as opposed to a single tarsnap log. I still haven’t decided if this is the right way to go, but I do like being able to quickly see the result of the last backup. My user.conf looks like the following.

/var/log/tarsnap-backup-code.log                        640     5       1000    *       Z
/var/log/tarsnap-backup-documents.log                   640     5       1000    *       Z

This configuration tells newsyslog to gzip, roll to a new log once the current log exceeds 1MB in size, and keep at most five old logs.

cron

With log rotation in place, I could create a cron job.

0 4 * * * /usr/local/bin/tarsnap-backup code ~/code > /var/log/tarsnap-backup-code.log

This crontab schedules backups for my code directory at 4am daily and my Documents directory at 5am daily. I used sudo crontabe -e to create this because both tarsnap and my log file’s permissions require root privileges. This would have sufficed, but there was a nagging thought in the back of my head: I knew that launchd is used in place of cron in OS X, and I thought this would give me a good opportunity to dive into even more options that launchd has to offer.

launchd

Since I wanted these backups to run whenever possible, I decided to put my launchd backup configurations in /Library/LaunchDameons instead of /Library/LaunchAgents. LaunchDaemons are able to run without a logged-in user; this is exactly what I wanted. The launchd configuration for my code backup looks like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.thomasupton.backup-daily-code</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/tarsnap-backup</string>
        <string>code</string>
        <string>/Users/thomas/code</string>
    </array>
    <key>GroupName</key>
    <string>wheel</string>
    <key>UserName</key>
    <string>root</string>
    <key>Nice</key>
    <integer>1</integer>
    <key>StandardErrorPath</key>
    <string>/var/log/tarsnap-backup-code.log</string>
    <key>StandardOutPath</key>
    <string>/var/log/tarsnap-backup-code.log</string>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>5</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
</dict>
</plist>

The ProgramArguments section is exactly how I called the backup script from cron. The UserName and GroupName keys are important: they tell launchd to run the backup script as root, which, as I mentioned before, is necessary for using tarsnap and for appending to the log file. The StandardErrorPath and StandardOutPath keys tell launchd to redirect output to the proper log file. The StartCalendarInterval tells launchd to run this script at 5am daily.

After registering the configuration via launchctl load /Library/LaunchDaemons/com.thomasupton.backup-daily-documents.plist, my automated backup system was in place.

Backup Management

Since Tarsnap backs up data with the notion of “snapshots” and keeps track of blocks of data (and not archive data), keeping multiple archives of the same data doesn’t make much sense. However, running a daily backup by creating a new archive would mean that many archives would build up fast. I decided that keeping at most three previous backups of the same data would suffice. I wanted to automate this, too. This is the reason I decided not to use a read-write-only key.

I added the following lines to my tarsnap-backup.sh script.

1
2
3
4

# Remove the backup from three days previous, if there is one
echo `date +%F\ %T`: Removing backup of $2 from `date -v-3d +%F`
/usr/local/bin/tarsnap -d -f $1-`date -v-3d +%F`
echo `date +%F\ %T`: Completed removing backup of $2 from `date -v-3d +%F`

The key to this is the date in the archive name passed to tarsnap -d. date -v lets you add a value to the date output, so -v-3d outputs the date from three days previous. Now, every scheduled backup attempts to delete the archive from three days ago in addition to creating a backup for the current day. Of course, if a backup is missed, this can lead to an accumulation of old archives. This is where the log files come in handy: I can just inspect the logs every couple of days to see what successfully ran and manually prune the archive list if necessary.

Large Backups

I said “if a backup is missed,” but I didn’t mention why that might occur. The answer becomes apparent when you start talking about backing up large amounts of data. My ~/Documents folder was over 12GB, and with my terrible upload speeds, that would mean that it would take a long, long time to upload everything. Even though I was able to prune the contents of ~/Documents down to 6.5GB, I still needed more than an hour to back it up. tarsnap doesn’t perform more than one archive transaction at once, so if the documents archive was still running when the code archive process began, tarsnap would cancel the latter and continue with the former, hence a backup is missed. This is also another reason that I decided to keep separate log files for each backup job. The log lines for an in-progress job aren’t interspersed with a failed attempt to start another backup job.

The documents backup was still too large to have been done by the morning, and I didn’t really want to sacrifice my network connection just for the sake of a backup. Fortunately, tarsnap supports archive truncation. According to the man pages, tarsnap responds to the SIGQUIT interrupt by truncating the archive and appending “.part” to the archive name. When my large backup job was still running, all I had to do was send the SIGQUIT signal with kill -3 (alternatively, you could send ^Q if you use tarsnap from a console and not from a scheduled job) and tarsnap would effectively “pause” the backup. The next time that same data is archived, tarsnap will recognize it and only upload new data. This works even with a different archive name, thanks to snapshots and block data.

Restoring Backups

Tarsnap is a great service, but truly for those who know what they are doing. It took me far longer than I would like to admit to come up with a process for all of this, but it was worth it. Of course, creating backups is only one part of a complete system. The other, more important part, is restoration. Since tarsnap is built on tar and libarchive, this is incredibly simple. tarsnap -x extracts archives, and tarsnap -r writes a tar stream to stdout, which can be used to create a local tar.

If you like the idea of easy, encrypted backups, tarsnap is a great service. It’s cheap, secure, and reliable, plus it’s fun and easy to use if you’re comfortable with UNIX-style archiving tools.