executive-blog

June 24, 2008

A Little Philosophical Thought: The SoftLayer Family Tree

Somewhat picking up on the theme of the blog, “Here’s to you, that nerdy Sysadmin”, this is a sort of, “Here’s to you, our family tree of customers.”

I find it very interesting how everyone in the world has customers. Our customer’s, customer’s, customer’s, customer’s…and so on…depend on us here at SoftLayer. It is a sort of bottomless pit. Does it begin anywhere…or end anywhere. Does anyone NOT have customers?

For example:

Our customers have customers, who have customers, who have customer, who have customers, etc. We are the customer of INTEL, AMD, Seagate, etc. They are the customers of those that provide the material to make the hardware that they manufacture. The natural resources used to make the hardware are purchased from someone. That someone purchased the rights to those resources from someone, who purchased the rights from someone, who purchased the rights from someone, etc. I suppose somewhere upstream someone’s country went to war and took the rights to those resources from someone else. And, I suppose if you go far enough back, no one had “rights” to those resources before the land was “claimed”. As recent as 2005, the race was on to claim the land of the Arctic Circle. I suppose you could say that all customers began with a gift of land containing resources from God. And, SoftLayer gives free stuff to customers all the time…so…I guess God gave the land containing the resources to people…His customers!

Anyway, this necessity of life that we have termed “customers”, has been, and will always be, I suppose, the most important aspect of life in terms of survival. Without customers, basic necessities like food cannot be purchased. In other words, without customers, you cannot be a customer. And, if you are not a customer, you must, therefore, be dead. So, in terms on everyone being dependent on being a customer and having customers, we all depend on each other like a family of customers. And, we must take care of our family.

We know that our survival, here at SoftLayer, depends on our customers (our family) and that their survival depends on us. We take this responsibility very seriously and work very hard to provide for our family the way that we would like to be provided for.

In conclusion, I feel that we, here at SoftLayer, do a pretty good job of taking care of our “family”, and in turn, our family of customers do a great job of taking care of us. As we continue to grow together, our success will benefit each other for years and years to come.

*If you do not understand any of this, just write it off to the insane ramblings of a tired CSA at the end of a long, challenging, and yet satisfying day at Softlayer working for his family.

-David

Categories: 
June 23, 2008

Writing a Blog is Hard

A blog is a strong commitment. I mean, if you just set up a web page, it's obvious that you're going to update occasionally, maybe once or twice a month, add a new page or two. I've seen viable websites that haven't been updated since before Y2K. But it's OK, because it's a website. Not so with a blog! If you set up a blog engine*, you're not allowed to make just one or two updates a month. You have to keep the pump primed with awesome content.

Most of this just has to do with the design of a blog engine. For one, each post is stamped with a date. The assumption seems to be that if you have a date on something, the date is important. The date on a carton of milk tells you when to drink it by, so maybe the date on a blog post is also some kind of content expiration date? The assumption seems to be that dates on text mean that you want people to notice the date and act upon it. Not to mention that most blog engines have some kind of calendar that points to listings of posts by date. If you have a calendar widget that only has one or two days highlighted, the assumption is you don't care about your blog or something.

Again, it's not any kind of failure, it's just what's expected. Blogs all look similar: they have a home page of posts, which link to full post pages. Pages are tagged for quick taxonomy identification and grouping. Blogs also generally allow comments, upping the conversational angle. So you have to keep your blog pumping content. I've discovered three different schemes of blog content generation:

The Panic Morning News: The Panic Morning News is a strategy where a blogger panics, struggling to create content every day. What you end up with is some content which is well written, and some content that seems to be filler, designed to put something up to fill this day's update.

The Anything Goes Times: These are the blogs where you find incredibly boring posts about accounting suddenly appearing in between exciting posts. I'm not saying that accounting is boring, per se, more that suddenly discovering a post about accounting sandwiched between a post about video games and exploding cars sticks out. Of course, these blogs generally are a kind of string of consciousness blogs, where the blogs are more of a “What am I thinking now” type blog.

The Who Cares Star-Telegram: These are the trailblazers who don't care that you think they're lame for having only one or two posts a month. Their posts are well written, and it becomes obvious that, to them, a blog is more a Content Management Engine* than a two-way communication medium.

But this isn't just a blogging phenomenon… it happens with anything that updates daily. Comic strips and books, websites, news feeds. And filler content usually follows some kind of pattern. For comics, a comic/cartoon character is usually put into a silly situation for a day. Batman has a birthday party thrown by Joker and the Penguin, or Naruto goes on a tangent about ramen noodles for a whole episode. Blogs and Webcomics tend to have their own special type of ‘filler,' usually they have a whole update talking about how difficult it is to write blogs and/or webcomics.

-Zoey

* NOTE: If you want the convenience of a blog without the expectations of daily updates, look at making a wiki or use a Content Management System like Drupal.

Categories: 
June 20, 2008

I Always Have a Backup Plan

It was the day of the big secret meeting. All my vice presidents were there except for the unix system administrator. He was a strange man, always wearing that robe, with the long beard and long hair. He considered himself some sort of wizard, and after the conflict last month when we decided to switch all our servers over to SoftLayer, I really didn’t want him involved in the meeting I called today. You see, I called it so I could announce my plan to switch our servers over to Windows. My goal was really to get rid of him; he’s the only one who ever managed to thwart my plans.

Just as I finished that thought, he burst through the door, trailing a long ribbon of old-fashioned printer paper behind him. “How dare you have a systems meeting without me!” he intoned, dropping his stack of papers on the conference table in front of me. A quick glance at the stack tells me that he has printed out operating statistics for every version of Unix and every version of Windows going back to 1985. I didn’t have time for this. Luckily, I always have a back up plan.

Turning away slightly, I quickly activated a program on my Blackberry. You see, yesterday I had written a few custom programs that utilize the SoftLayer API to control a variety of our services. Within moments, a confirmation had appeared on my screen. All of our web traffic had been redirected from our load balanced main servers to our tertiary backup server. In the middle of the work day, that means it was only a matter of minutes before our bandwidth would be exceeded on that server. I allowed the sysadmin to begin his presentation, confident that he would barely get past the 8086 before disaster stuck.

I was right! Within minutes, an email arrived notifying us that we were nearing the bandwidth cap on the hostname last_resort. Panicked, the sysadmin left the meeting. Quickly I summarized my plans to the other VPs, we all voted unanimously for Windows, and I retreated to my office. Shortly after sitting behind my desk, my door burst open. Framed in the light from the hallway, his long shadow washing over me, stood the sysadmin, slowly twirling his staff. “Do you think you can stop me with a simple change to our load balancer? I was configuring load balancers when you were still on dial-up! Now, you will listen, AOL user, and you will see why Unix is your only choice!” Of course, I had a backup plan for just such a situation.

I dove out the window next to my desk, landing nimbly next to my secretary’s bright pink LeBaron. I had made copies of all her keys months ago in order to utilize her unique vehicle for any necessary escapes. I quickly tapped out a text message to Michael in SoftLayer sales. We have a standing agreement that when he receives a message from me containing only the word DAWT, he is to send the best sale at his disposal to my sysadmin. As I drove past the front door of the building I saw him running toward the car. He pulled out his Blackberry in mid-stride and suddenly stopped dead. “Free double RAM AND double hard drives!? IMPOSSIBLE!” he screamed, and I managed to swerve around him and escape. As I drove away, I thought about my secretary. When she first started here, I had convinced her that if her car were ever stolen, the best plan of action would be to change the building security policies so that only my badge could open the doors. I hoped I didn’t need to make use of that plan, but the sysadmin has proved a worthy adversary.

Unbelievable! Even with my masterful backup plan, he was still following me. I saw his battered VW Bus merge into traffic behind me, his vulture-like shadow looming behind the wheel. I sped up until we were both racing down the road, weaving in and out of the other vehicles. Finally we passed a police car, and my next plan sprang into action. I knew that standard procedure was to radio in the vehicles you were pursuing, and I knew my friend Joe was on duty today. Joe knew that if he ever received a radio call about a business man in a pink LeBaron being chased down the highway by a wizard in a VW Bus, he was to call off the police and park a fire truck at a certain intersection. You see, I had hired an actor to pretend to be a corporate Psychiatrist, and learned that the Sysadmin had an irrational fear of fire trucks. Why? Because it always pays to have a backup plan.

I angled toward the intersection and managed to squeeze past the truck just as it pulled up to block the street. I heard the squeal of tires as the sysadmin slammed on his breaks and reversed wildly behind me. Now that I was free, however, I couldn’t return to the office. Luckily I was prepared for just such an eventuality. As I drove to my next location, I quickly used my Blackberry to shut down one of our production web servers. I knew that it would be 20 minutes before the monitoring system would officially declare the server “down,” so I had time.

I made it to my secret office above the video arcade not long after. Before leaving the car I collected the grappling hook and rope from a secret compartment in the door, then went inside. I walked in to the darkened room and immediately noticed something was wrong. My security system wasn’t beeping! The door slammed behind me and the sysadmin boomed out “NO PLAN CAN DEFEAT ME, MORTAL!”

“I’m ALWAYS prepared!” I shot back, and quickly glanced at my watch. It had been 19 minutes and 45 seconds since I shut down my server, the timing was perfect! The sysadmin walked toward me, twirling that staff. Just as he was about to reach me, his blackberry beeped. Pausing to check, he let out a stream of curses and then lunged at me, but I had already rappelled down the side of the building and made my escape.

As soon as I reached the car, my Blackberry alerted me that the server I shut down was back up. How!? The sysadmin must have his own API programs! I cringed as I activated my final backup plan: a program that constantly shut down all our servers. Let’s see him handle that! I took the direct route back to the office, past the still-idling fire truck. I threw Joe a wave, knowing that I’d owe him a big favor for this, and rocketed back to the office. I knew that he would be right behind me, but hopefully with all our servers offline he won’t beat me to my destination. Also, once I made it into the building, the security system wouldn’t allow anyone in behind me. I would be safe!

I raced into the building, looking frantically around for the sysadmin, but he was nowhere to be seen. Finally! I had defeated him! I walked calmly to my office and opened the door, only to see HIM, climbing in through my window. I had forgotten to close it when I escaped this morning! I quickly opened the secret panel in the wall next to the door and put my finger on the red button.

“WAIT!” cried the sysadmin. “We need to put our differences behind us. Our plans have almost destroyed our servers!”

“What do you mean?” I demanded. “They’re fine!”

“No, they’re not,” he said in a sad voice. “You see, I always have a backup plan, and I knew that eventually someone would attempt to power off our machines, so I wrote a script to constantly turn the machines on!”

“B-but…” I stammered, “but I wrote a script to constantly turn them OFF”

“I know” he said, “and the constant power cycling has corrupted our data base. We need to set aside this silly feud and fix it.”

“Don’t worry, dear end user” I proudly proclaimed, “I always have a backup-“

It was right then I realized that in all my planning, I had never actually created any backups.

-Daniel

Categories: 
June 18, 2008

Planning for Data Center Disasters Doesn’t Have to Cost a Lot of $$

One of the hot topics over the past couple of weeks in our growing industry has been how to minimize downtime should your (or your host’s) data center experience catastrophic failure leading to outages that could span multiple days.

Some will think that it is the host’s responsibility to essentially maintain a spare data center into which they can migrate customers in case of catastrophe. The reason we don’t do this is simple economics. To maintain this type of redundancy, we’d need to charge you at least double our current rates. Because costs begin jumping exponentially instead of linearly as extensive redundancy is added, we’d likely need to charge you more than double our current rates. You know what? Nobody would buy at that point. It would be above the “reservation price” of the market. Go check your old Econ 101 notes for more details.

Given this economic reality, we at SoftLayer provide the infrastructure and tools for you to recover quickly from a catastrophe with minimal cost and downtime. But, every customer must determine which tools to use and build a plan that suits the needs of the business.

One way to do this is to maintain a hot-synched copy of your server at a second of our three geographically diverse locations. Should catastrophe happen to the location of your server, you will stay up and have no downtime. Many of you do this already, even keeping servers at multiple hosts. According to our customer surveys, 61% of our customers use multiple providers for exactly that reason – to minimize business risk.

Now I know what you’re thinking – “why should I maintain double redundancy and double my costs if you won’t do it?” Believe me, I understand this - I realize that your profit margins may not be able to handle a doubling of your costs. That is why SoftLayer provides the infrastructure and tools to provide an affordable alternative to running double infrastructure in multiple locations in case of catastrophe.

SoftLayer’s eVault offering can be a great cost effective alternative to the cost of placing servers in multiple locations. Justin Scott has already blogged about the rich backup features of eVault and how his backup data is in Seattle while his server is in Dallas, so I won’t continue to restate what he has already said. I will add that eVault is available in each of our data centers, so no matter where your server is at SoftLayer, you can work with your sales rep to have your eVault backups in a different location. Thus, for prices that are WAY lower than an extra server (eVault starts at $20/month), you can keep near real-time backups of your server data off site. And because the data transfer between locations happens on SoftLayer’s private network, your data is secure and the transfer doesn’t count toward your bandwidth allotment.

So let’s say your server is in our new Washington DC data center and your eVault backups are kept in one of our Dallas data centers. A terrorist group decides to bomb data centers in the Washington DC area in an attempt to cripple US government infrastructure and our facility is affected and won’t be back up for several days. At this point, you can order a server in Dallas, and once it is provisioned in an hour or so, you restore the eVault backup of your choice, wait on DNS to propagate based on TTL, and you’re rolling again.

Granted, you do experience some downtime with this recovery strategy. But the tradeoff is that you are up and running smoothly after the brief downtime at a cost for this contingency that begins at only $20 per month. And when you factor in your SLA credit on the destroyed server, this offsets the cost of ordering a new server, so the cost of your eVault is the only cost of this recovery plan.

This is much less than doubling your costs with offsite servers to almost guarantee no downtime. The reason that I throw in the word “almost” is that if an asteroid storm takes out all of our locations and your other providers’ locations, you will experience downtime. Significant downtime.

-Gary

June 16, 2008

More RAM!

More RAM. DDR2 must be going out of style, because Microcenter is selling a gig for $12.99. This time I don't make it in time before they run out, and I settle for sour grapes: my home pc can't use all the addressable memory, anyway. 4 Gigabytes. The maximum addressable memory for a 32-bit motherboard / OS. It used to be such a big deal to me- maxing out the 4 slots on my Dell, but not anymore. Why? Because now I work for SoftLayer. When you work with motherboards with up to 32 slots, 1 or 2 gig each; 4 measly slots just seems sad. I start nosing around for a video card that will fit the last expansion slot on my pc. No luck. I end up going home empty handed from Microcenter (outrageous!) and ordering the pci-e x1 video card from Newegg.

So, the hardware that customers can order at SoftLayer is impressive enough to jade the geek tech-lust of any home technician. And everything fits so nice and clean. Working on SoftLayer servers has really spoiled me for home pc's. Open up the case on your home pc and what do you see? Fabulous shiny bits? No. Cables. Cables in the computer. Cables behind the computer. Cables everywhere! You get the nifty zips and loops from Radio Shack, spend 2 hours zipping and looping, and as you proudly call your wife over to take a look she says, "Can't it all be wireless?"

The truth is neat cables take time, and SoftLayer engineers spend every spare minute making neat, organized, color-coordinated cables running to the servers. Cat 5e. Fiber. Special cables. Cables we can't talk about. All very neat and aesthetic. If Mr. Crosby ever takes you for a walk through the dc (datacenter), it looks effortless and lovely. But it took hours. Hundreds of hours. Just on cabling (I think SoftLayer might have stock in zip ties). You can be sure your SoftLayer server is not lost in a sea of Cat5 and power cables. It has been gracefully bound to its slot, the formidable innards pumping away at your command, your data streaming straight and true from switch to switch into the Internet beyond...

-Philip

Categories: 
June 14, 2008

In Memory of Dawn

Dawn was the best friend I’ve ever had, except for my little sister. Just yesterday I got home only to find out that Dawn had died silently in the night. No amount of resuscitation could bring her back. Needless to say, I was quite sad.

Dawn was my computer.*

The funny part of it all was just how much of my time involves a computer. I watch TV and Movies on my computer, I play games on my computer, I do my banking on my computer, I pay all my bills on my computer, I schedule my non-computer time on my computer, I use my computer as a jukebox.

In other words, I was completely lost. What made it worse, however, was that I had had yesterday scheduled to pay my bills. But where was my list of bills?

If you guessed “Dawn had all your bills”, then you are right.

What about paper bills? I’ve got the Internet and a computer! So, in most cases I’ve canceled paper bills. All paper bills I get are shredded forthwith. So I had no paper backup of bills.

Well, I made do. I kicked my roommate off his computer (a technique involving making annoying noises while he tries to concentrate playing Call of Duty 4) and used it to pay what bills I could remember. I kept track of the bills I was paying by entering them into a Google Document.

That’s when it hit me! Why wasn’t my bill spreadsheet on Google Documents? Along with my bill list? Along with all the other documents I work on every day? Cloud Computing For The Win! As soon as I get my next computer up and running (and I figure out a new naming algorithm) I’m going to put all my vital files on Google Docs. This ties in well with Justin Scott’s post; the key to not having your data disappear during a disaster is to have a backup copy. You want backups out there, far away from your potential point of failure. (I did have backups… but they’re all on CDs that I didn’t want to have to sort through to find just one file. And had the disaster been, say, a flood, I would have had no backups.)

Google Docs is a great example of Cloud Computing: Putting both the program and the file being worked on “in the cloud.” Having built internal applications for a few people, I would make the same recommendation: Since many business apps are moving to PHP anyway (thanks for the reminder, Daniel!), you might as well move the application AND the data out of the building and onto a secure server. And as Mr. Scott** mentioned, SoftLayer ALREADY has geographic diversity as well as a private network that will allow you to link your application and data servers together in real time through all datacenters… for free. Along with the added bonus of being able to access your application from any computer… should yours meet up with Misty, May, and Dawn at the Great Datacenter in the Sky.

-Zoey

* I had a system of naming my computers after the female protagonists from the Pokemon series. Dawn, however, is the last of that series…

** I’ve decided that since Justin is an Engineer, calling him Mr. Scott is funny.

June 12, 2008

Culture Shock?

After separating from the military about a year ago, I was sure I was in for a bit of a culture shock. As you may very well know, the military is very different from life in what we liked to call the “real world”. 24 hour duties, life on ship, the awful food, I was ready for a slight change of pace. Soon thereafter I stumbled upon a golden opportunity here at SoftLayer.

Little did I realize at the time that I would be on board with one of the fastest growing and most innovative companies in the industry. Looking back, I can see that one of the ingredients for our explosive growth is the culture that SL has… one as idiosyncratic as the military itself. Coincidently, the culture shock I prepared myself for seemed to be more of a shift in verbiage.

Allow to me to submit a few specifics:

  • Much like the Marine Corps loved to add “MC” before every single acronym, SoftLayer does much the same… (SLiki, SLales – you get the point).
  • There is a mindset here that the mission comes first. In this case, that mission is to provide our customers with the best possible hardware, support, and applications to accomplish THEIR mission.
  • The pride SL employees have of being part of the team is infectious, much like a close knit unit in the military. You can see this best when things go awry – fellow SL’ers rush to back up those in need. If you’ve read through the InnerLayer prior to reading my musings, you know that the team here takes great pride in being a part of SL’s success.
  • Much like the military, you’ve got your large mix of people, from all walks of life, each with something to add to the team. Those experiences conglomerate to further enhance our ability to innovate.

Needless to say, the things that made successful units and deployments while I was in, I now realize are the ingredients to a successful team anywhere, no matter whether you use MC or SL. I guess that culture shock I had prepared myself for wasn’t as bad or painful as I thought it would be. I’ve traded my rifle for a scan gun, and my camouflage uniforms for those SL Tees.

-Matthew

Categories: 
June 10, 2008

Have to Be Technical to Outsource? Nah!

I know what you're thinking; here I go again talking about why you HAVE to outsource your datacenter and infrastructure. Not this time. In previous blogs I have talked about doctors and baseball players, etc and the ways they outsource. It hit me that everyone most likely does some sort of outsourcing every day and it is just part of our evolving society.

Think about how fast things changed from Y2K until now. So much technology; the way we listen to music, HDTV, BluRay, Hydrogen, Electric, and Hybrid automobiles, the green movement. Change is everywhere. If you look back to 1900 or even the 1950's changes were much slower and they weren't nearly as life-changing. A change to someone's daily routine back then wasn't a life changing stressful event like it can be now.

So how do we cope with all these fast changes today? Some form of outsourcing. Think about it, even going out to eat is outsourcing. Some of my single friends have never even used the kitchen equipment in their homes. They outsource their kitchens! Most use the fridge to keep the adult beverages cold but that is to drink with the Papa John's pizza they have delivered.

Ok, so now for the real reason this blog fell out of my head. I was talking to my mom yesterday and she was talking about quilting. She is a quilting nut. My whole house is buried in quilts. My wife loves them. I am pretty sure my brother's house has quite a few as well. We have a wedding quilt with pics of the wedding and signatures and quotes from people that were at the wedding. We have a "charm" quilt with 1020 or so different fabrics in it including fabric from my favorite shirt (at that time). We have too many quilts to name and I don't know what about half of them are called really. They all have "quilty" names.

So for those of you who don't know a whole lot about quilting (and I don't either) there are machine quilted quilts and there are hand quilted quilts. Machine quilted quilts are faster to finish but aren't worth as much money and can stand the test of time better. Hand quilted quilts take much longer to finish and can't stand up to as much punishment but are worth more money in the long run.

So, I have always wondered how my mom can bust out all these quilts so fast. The ones that are machine quilted always made sense. She can make them faster and they are tougher. But she gives us several that are handmade and I could never figure out where she got all this free time. I mean she is retired and sleeps till like 11am! Anyway, to make a good quilt you have to go search for all the right fabric, sew the top all together, then the bottom, and then quilt it all up nice and pretty - like toilet paper really. Batting on the inside makes it fluffy.

It takes time to do all those things and she just keeps on making them at blazing speed. So I finally figured it out yesterday when on the phone with her she mentioned she dropped 3 quilts off to be quilted. It didn't even faze me at first and then it hit me, she is cheating! I said, "You did what with the quilts?" So I learned yesterday that some of my quilts I thought my mom hand quilted, were finished by people I don't even know! I guess outsourcing is even to the point where grannies can do it to trick their grandkids! Where will it all stop!

In this fast moving age, it probably won't so don't miss the outsourcing train; it can save you some time - time you can spend with your family making quilts if you choose.

-Skinman

P.S. I wonder if I set the record for using the letter "Q" the most times in a single blog?

Categories: 
June 6, 2008

VFB For Seattle

So it's been about 4 months now since Seattle went live. We have approximately 2000 servers active already! (That’s more than the last DC I worked at has and they’ve been selling servers in Seattle since August of 2005). Server room 1 has lots of cool servers with lots of blinky lights and we’ve been working hard on deploying Server room two around the clock to keep up with the demands of sales around here.

With the “Go-Live Team” back in Dallas loading up the truck to head out to Chantilly, everything is running smoothly up in the great northwest. We no longer have to hear about {insert random person from the Dallas Go-Live team} complain about not having a What-A-Burger for dinner. That’s fine because we have world famous Hot Dogs right up on the corner at Matt’s Famous Chilidogs to cure the hunger that strikes us in the middle of our shift.

With Server room two going live this last week we had another Seattle truck day (this was the first truck day that we completed with 100% staff from Seattle). We didn’t need those experts from Dallas to baby us through. No Brad Lewis’s to answer questions when we have them. :-)

I must say that everything this last week went great and to continue a new SoftLayer tradition, everyone in Seattle deserves a VFB!

So here is MY Seattle VFB to everyone!

- III!

Something cool is the guys from Operations decided to drop by late last Friday. They were more than pleased and impressed with everyone and the performance that we showed them with our preparation of SR02 this last week.

We’ve set sail guys, and are doing a great job, and we like being noticed by the guys down in Dallas and from what our customers have to say. We will keep up the great work.

It really rocks! I must say that we have it good at SL, but that must be because we got the best C.E.O. ever! (Wonder’s if I’ll see a extra $50 now.. hey Little Jones got it right!?! )

- III! to the guys in Seattle..

-Bill

June 4, 2008

Wait … Back up. I Missed Something!

I’ve been around computers all my life (OK, since 1977 but that’s almost all my life) and was lucky to get my first computer in 1983.

Over the summer of 1984, I was deeply embroiled in (up to that point) the largest programming project of my life, coding Z80 ASM on my trusty CP/M computer when I encountered the most dreaded of all BDOS errors, “BDOS ERROR ON B: BAD SECTOR”

In its most mild form, this cryptic message simply means “copy this data to another disk before this one fails.” However, in this specific instance, it represented the most severe case… “this disk is toast, kaputt, finito, your data is GONE!!!”

Via the School of Hard Knocks, I learned the value of keeping proper backups that day.
If you’ve been in this game for longer than about 10 milliseconds, it’s probable that you’ve experienced data loss in one form or another. Over the years, I’ve seen just about every kind of data loss imaginable, from the 1980’s accountant who tacked her data floppy to the filing cabinet with a magnet so she wouldn’t misplace it-- all the way to enterprise/mainframe class SAN equipment that pulverizes terabytes of critical data in less than a heartbeat due to operator error on the part of a contractor.

I’ve consulted with thousands of individuals and companies about their backup implementations and strategies, and am no longer surprised by administrators who believe they have a foolproof backup utilizing a secondary hard disk in their systems. I have witnessed disk controller failures which corrupt the contents of all attached disk drives, operator error and/or forgetfulness that leave gaping holes in so-called backup strategies and other random disasters. On the other side of the coin, I have personally experienced tragic media failure from “traditional backups” utilizing removable media such as tapes and/or CD/DVD/etc.

Your data is your life. I’ve waited up until this point to mention this, because it should be painfully obvious to every administrator, but in my experience the mentality is along the lines of “My data exists, therefore it is safe.” What happens when your data ceases to exist, and you become aware of the flaws in your backup plan? I’ll tell you – you go bankrupt, you go out of business, you get sued, you lose your job, you go homeless, and so-on. Sure, maybe those things won’t happen to you, but is your livelihood worth the gamble?

“But Justin… my data is safe because it’s stored on a RAID mirror!” I disagree. Your data is AVAILABLE, your data is FAULT TOLERANT, but it is not SAFE. RAID controllers fail. Disaster happens. Disgruntled or improperly trained personnel type ‘rm –rf /’ or accidentally select the wrong physical device when working with the Disk Manager in Windows. Mistakes happen. The unforeseeable, unavoidable, unthinkable happens.

Safe data is geographically diverse data. Safe data is up-to-date data. Safe data is readily retrievable data. Safe data is more than a single point-in-time instance.

Unsafe data is “all your eggs in one basket.” Unsafe data is “I’ll get around to doing that backup tomorrow.” Unsafe data is “I stored the backups at my house which is also underwater now.” Unsafe data is “I only have yesterday’s backup and last week’s backup, and this data disappeared two days ago.”

SoftLayer’s customers are privileged to have the option to build a truly safe data backup strategy by employing the Evault option on StorageLayer. This solution provides instantaneous off-site backups and efficiently utilizes tight compression and block-level delta technologies, is fully automated, has an extremely flexible retention policy system permitting multiple tiers of recovery points-in-time, is always online via our very sophisticated private network for speedy recovery, and most importantly—is incredibly economical for the value it provides. To really pour on the industry-speak acronym soup, it gives the customer the tools for their BCP to provide a DR scenario with the fastest RTO with the best RPO that any CAB would approve because of its obvious TCR (Total Cost of Recovery). Ok, so I made that last one up… but if you don’t recover from data loss, what does it cost you?

On my personal server, I utilize this offering to protect more than 22 GB of data. It backs up my entire server daily, keeping no less than seven daily copies representing at least one week of data. It backs up my databases hourly, keeping no less than 72 hourly copies representing at least three days of data. It does all this seamlessly, in the background, and emails me when it is successful or if there is an issue.

Most importantly, it keeps my data safe in Seattle, while my server is located in Dallas. Alternatively, if my server were located in Seattle, I could choose for my data to be stored in Dallas or our new Washington DC facility. Here’s the kicker, though. It provides me the ability to have this level of protection, with all the bells and whistles mentioned above, without overstepping the boundary of my 10 GB service. That’s right, I have 72 copies of my database and 7 copies of my server, of which the original data totals in excess of 22 GB, stored within 10 GB on the backup server.

That’s more than sufficient for my needs, but I could retain weekly data or monthly data without significant increase in storage requirements, due to the nature of my dataset.
This service costs a mere $20/mo, or $240/yr. How much would you expect to pay to be able to sleep at night, knowing your data is safe?

Are you missing something? Wait … Backup!

-Justin

Pages

Subscribe to executive-blog