With SoftLayer Object Storage Python Client set up and the SoftLayer Object Storage Backup script — slbackup.py — in hand, I had the tools I needed to build a solid backup infrastructure easily. On Linux.org, I contributed an article about how to perform MySQL backups with those resources, so the database piece is handled, but I also need to back up my web files, so I whipped up another quick bash script to run:

#!/bin/bash
 
# The path the backups will be dumped to
DUMP_DIR="/home/backups/"
 
# Path to the web files to be backed up
BACKUP_PATH="/var/www/sites /"
 
# Back up folder name (mmddyyyy)
BACKUP_DIR="`date +%m%d%Y`"
 
# Backup File Name
DUMP_FILE="`date +%m_%d_%Y_%H_%M_%S`_site_files"
 
# SL container name
CONTAINER="site_backups"
 
# Create backup dir if doesn't exist
if [ ! -d $DUMP_DIR$BACKUP_DIR ]; then
        mkdir -p $DUMP_DIR$BACKUP_DIR
fi
 
tar -zcvpf $DUMP_DIR$BACKUP_DIR/$DUMP_FILE.tar.gz $BACKUP_PATH
 
# Make sure the archive exists
if [ -f $DUMP_DIR$BACKUP_DIR/$DUMP_FILE.tar.gz ]; then
        /root/slbackup.py -s $DUMP_DIR$BACKUP_DIR/ -o "$CONTAINER" -r 30
 
        # Remove the backup stored locally
        rm -rf $DUMP_DIR$BACKUP_DIR
 
        # Success
        exit 0
else
        echo "$DUMP_DIR$BACKUP_DIR/$DUMP_FILE.tar.gz does not exist."
        exit 1
fi

It’s not the prettiest bash script, but it gets the job done. By tweaking a few variables, you can easily generate backups for any important directory of files and push them to your SoftLayer Object Storage account. If you want to change the retention time of your backups to be longer or shorter, you can change the 30 after the –r in the line below to the number of days you want to keep each backup:

/root/slbackup.py -s $DUMP_DIR$BACKUP_DIR/ -o "$CONTAINER" -r 30

I created a script for each website on my server, and I set a CRON (crontab –e) entry to run each one on Sundays staggered by 5 minutes:

5 1 * * 0  /root/bin/cron/CRON-site1.com_web_files > /dev/null
10 1 * * 0  /root/bin/cron/CRON-site2.com_web_files > /dev/null
15 1 * * 0  /root/bin/cron/CRON-site3.com_web_files > /dev/null

If you’re looking for an easy way to automate and solidify your backups, this little bit of code could make life easier on you. Had I taken the few minutes to put this script together prior to the attack I experienced at my previous host, I wouldn’t have lost any of my data. It’s easy to get lulled into “backup apathy” when you don’t need your backups, but just because nothing *has* happened to your data doesn’t mean nothing *can* happen to your data.

Take it from me … Be over-prepared and save yourself a lot of trouble.

-Ronald

Over the past few years, we’ve built a great reputation at OSCON as not only a great hosting provider, but also as the operator of one of the best booths on the expo hall floor. As usual, the switchballs were crowd pleasers, and we sponsored the show’s Massage Booth, so we had great traffic through our booth all conference. When attendees left our booth, they were considerably more relaxed, they had the coolest swag at the show, and they had a better understanding of where SoftLayer fits in the open source space.

In addition to the conversations on the expo hall floor, we got to share a little expertise in a conference session. Senior Software Architect Harold Hannon presented an engaging educational session about how we implemented elasticsearch, Apache-based code that allows for scalable search for all kinds of documents in near real-time. At the moment, SoftLayer uses elasticsearch internally for hardware and ticketing, and we hope to extend this feature-rich scalable searching to our customers in an upcoming release of the customer portal. Because SoftLayer has built a great reputation for executing scalability well, Harold ended up presenting to a packed house (which you can see in the last few pictures of the slide show above).

SoftLayer’s significant investment in open source platforms like OpenStack Swift Object Storage and CloudStack-based Private Clouds wound up being a big topic of discussion throughout the conference. Harold’s elasticsearch presentation was a great conversation bridge to talk about the incredible search-and-retrieve functionality we implemented in our Object Storage service, and we were able to share and demonstrate how that functionality helps our customers manage large quantities of static data in cloud environments in an automated way.

The open source community has matured significantly over the past few years, and it’s exciting to see that evolution. We aren’t just talking about the incredibly popular open source operating systems like CentOS, Debian, Fedora, FreeBSD and Ubuntu that customers can get on a dedicated or cloud server … We’re talking about game-changing, innovative platforms that are redefining how the Internet works.

We want to thank the OSCON team for another phenomenal show, and if you attended the show but didn’t get a switchball from us, I’m sure you’ll have another chance at OSCON 2013. If you don’t think you can wait that long, come find us at one of our other upcoming events!

-Summer

Based on OpenStack Object Storage (codenamed Swift) — open-source software that allows the creation of redundant, scalable object storage on clusters of standardized servers — SoftLayer Object Storage provides customers with new opportunities to leverage cost-effective cloud-based storage and to simultaneously realize significant capex-related cost savings.

OpenStack has been phenomenally successful thanks to a global software community comprised of developers and other technologists that has built and tweaked a standards-based, massively scalable open-source platform for public and private cloud computing. The simple goal of the OpenStack project is to deliver code that enables any organization to create and offer feature-rich cloud computing services from industry-standard hardware. The overarching OpenStack technology consists of several interrelated project components: One for compute, one for an image service, one for object storage, and a few more projects in development.

SoftLayer Object Storage
Like the OpenStack Swift system on which it is based, SoftLayer Object Storage is not a file system or real-time data-storage system, rather it’s a long-term storage system for a more permanent type of static data that can be retrieved, leveraged and updated when necessary. Typical applications for this type of storage can involve virtual machine images, photo storage, email storage and backup archiving.

One of the primary benefits of Object Storage is the role that it can play in automating and streamlining data storage in cloud computing environments. SoftLayer Object Storage offers rich metadata features and search capability that can be leveraged to automate the way unstructured data gets accessed. In this way, SoftLayer Object Storage will provide organizations with new capabilities for improving overall data management and storage efficiency.

File Storage v. Object Storage
To better understand the difference between file storage and object storage, let’s look at how file storage and object storage differ when it comes to metadata and search for a simple photo image. When a digital camera or camera-enabled phone snaps a photo, it embeds a series of metadata values in the image. If you save the image in a standard image file format, you can search for it by standard file properties like name, date and size. If you save the same image with additional metadata as an object, you can set object metadata values for the image (after reading them from the image file). This detail provides granular search capability based on the metadata keys and values, in addition to the standard object properties. Here is a sample comparison of an image’s metadata value in both systems:

Using the rich metadata and search capability enabled by object storage, you would be able to search for all images with a dimension of 2240×1680 or a resolution of 72×72 in a quick/automated fashion. The object storage system “understands” more about what is being stored because it is able to differentiate files based on characteristics that you define.

What Makes SoftLayer Object Storage Different?
SoftLayer Object Storage features several unique features and ways for SoftLayer customers to upload, access and manage data:

• Search — Quickly access information through user-defined metadata key-value pairs, file name or unique identifier
• CDN — Serve your content globally over our high-performance content delivery network
• Private Network — Free, secure private network traffic between all data centers and storage cluster nodes
• API — Access to a full-feature OpenStack-compatible API with additional support for CDN and search integration
• Portal — Web application integrated into the SoftLayer portal
• Mobile — iPhone and Android mobile apps, with Windows Phone app coming soon
• Language Bindings — Feature-complete bindings for Java, PHP, Python and Ruby*

*Language bindings, documentation, and guides are available on SLDN.

We think SoftLayer Object Storage will be attractive to a broad range of current and prospective customers, from web-centric businesses dependent on file sharing and content distribution to legal/medical/financial-services companies which possess large volumes of data that must be stored securely while remaining readily accessible.

SoftLayer Object Storage significantly extends our cloud-services portfolio while substantially enriching the storage capabilities that we bring to our customers. What are you waiting for? Go order yourself some object storage @ $0.12/GB!

-Marc

The success of open-source projects like Linux, Ruby on Rails, and Android reveals the strength and diversity of having developers around the world contributing and providing feedback on code. The community becomes more than the sum of its parts, driving innovation and constant improvement. The case has been made for open source in and of itself, but a debate still rages over the developing case for businesses contributing to open source. Why would a business dedicate resources to the development of something it can’t sell?

The answer is simple and straightforward: Contributing to open source fosters a community that can inspire, create and fuel the innovation a business needs to keep providing its customers with even better products. It makes sense … Having hundreds of developers with different skills and perspectives working on a project can push that project further faster. The end result is a product that benefits the open-source community and the business world. The destiny of the community or the product cannot be defined by a single vendor or business; it’s the democratization of technology.

Open-Source Cloud Platforms
Today, there are several open-source cloud platforms vying for industry dominance. SoftLayer has always been a big proponent and supporter of open source, and we’ve been involved with the OpenStack project from the beginning. In fact, we just announced SoftLayer Object Storage, an offering based on OpenStack Object Storage (code-named Swift). We’ll provide code and support for Swift in hopes that it continues to grow and improve. The basic idea behind Swift Object Storage is to create redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. I could go on and on about object storage, but I know Marc Jones has a blog specifically about SoftLayer Object Storage being published tomorrow, and I don’t want to steal too much of his thunder.

We have to acknowledge and embrace the heterogeneous nature of IT industry. Just as you might use multiple operating systems and hypervisors, we’re plan on working with a variety of open-source cloud platforms. Right now, we’re looking into supporting initiatives like Eucalyptus, and we have our ear to the street to listen to what our customers are asking for. Our overarching goal is to provide our customers with much-needed technologies that are advancing the hosting industry, and one of the best ways to get to that end is to serve the needs of the open-source community.

As I write this blog post, I can’t help but think of it in terms of a the Lord of Rings reference: “One ring to rule them all.” The idea that “one ring” is all we need to focus on as a hosting provider just doesn’t work when it comes to the open-source community … It all comes down to enabling choice and flexibility. We’ll keep investing in innovation wherever we can, and we’ll let the market decide which ring will rule where.

What open-source projects are you working on now? How can SoftLayer get involved?

-Matt