Automated Backup to HP Cloud Object Storage, Code Included

(cross posted from the HPCloud Blog. With 75% more typos!)

One of the most basic problems with systems that need to persist data, is making sure that you can recover those systems in the case of a critical error. I’ve used and written backup systems for more time then I’d like to admit (for example). With the advent of cloud storage systems such as S3, moving your data offsite has become much easier, and much easier to recover data from your offsite storage system.

Back when I got started in this industry, a tape backup would take hours. And then you’d have to drive it to your safety deposit box and store it. When you needed to recover data, it was a drive and then hours to restore it.

The next iteration was removable hard drives. These were quicker to back up to and restore from, however the offsite portion was still onerous. That’s why I developed SyncScript – caching a local copy of the backup mades sense when most of the restore request were for things deleted in the last 24 – 48 hours.

However with the advent of Cloud Storage and higher speed internet pipes you can keep offsite backups, and get them back reasonably quickly. Since I work with OpenStack now days, that’s the hammer that makes sense to use.

Here is what I did to get a dead simple backup from a HP Cloud instance to HP’s Object Storage.

Continue reading “Automated Backup to HP Cloud Object Storage, Code Included”

Borked Filesystems

In a recent blog post, Brian Krebs mentions the following:

I also urge users to segment their systems so that important data files are on a separate chunk of hard drive space than the Windows operating system, which tends to make restoring backups a far simpler affair.

This is fantastic advice, and should be followed by every operating system rather then just Windows. I’ve espoused this method for years, and have actually gotten some grief about it.

I think this should into the “duh” category, but so many people opt for the easy and less safe monolithic partioning scheme (I’m looking at you, @mystic). If your OS implodes, and you have to re-install, it’s much easier to just wipe and reinstall the O/S partition and not worry about the user data in the /home partition. This does not in any manner replace the needs for backups – your /home partition could just as easily implode.

The event that really pressed this home for me was watching an old DOS 5 machine unravel when it’s FAT table got really confused. We ended up having to format from floppy, and reinstall everything. Total loss.