executive-blog

May 1, 2009

What A Cluster

When you think about all the things that have to go right all the time where all the time is millions of times per second for a user to get your content it can be a little... daunting. The software, the network, the hardware all have to work for this bit of magic we call the Internet to actually occur.

There are points of failure all over the place. Take a server for example: hard drives can fail, power supplies can fail, the OS could fail. The people running servers can fail.. maybe you try something new and it has unforeseen consequences. This is simply the way of things.

Mitigation comes in many forms. If your content is mostly images you could use something like a content delivery network to move your content into the "cloud" so that failure in one area might not take out everything. On the server itself you can do things like redundant power supplies and RAID arrays. Proper testing and staging of changes can help minimize the occurrence of software bugs and configuration errors impacting your production setup.

Even if nothing fails there will come a time when you have to shut down a service or reboot an entire server. Patches can't always update files that are in use, for example. One way to work around this problem is to have multiple servers working together in a server cluster. Clustering can be done in various ways, using Unix machines, Windows machines and even a combination of operating systems.

Since I've recently setup a Windows 2008 cluster that is we're going to discuss. First we need to discuss some terms. A node is a member of a cluster. Nodes are used to host resources, which are things that a cluster provides. When a node in a cluster fails another node takes over the job of offering that resource to the network. This can be done because resources (files, IPs, etc) are stored on the network using shared storage, which is typically a set of SAN drives to which multiple machines can connect.

Windows clusters come in a couple of conceptual forms. Active/Passive clusters have the resources hosted on one node and have another node just sitting idle waiting for the first to fail. Active/Active clusters on the other hand host some resources on each node. This puts each node to work. The key with clusters is that you need to size the nodes such that your workloads can still function even if there is node failure.

Ok, so you have multiple machines, a SAN between them, some IPs and something you wish to serve up in a highly available manner. How does this work? Once you create the cluster you then go about defining resources. In the case of the cluster I set up my resource was a file share. I wanted these files to be available on the network even if I had to reboot one of the servers. The resource was actually combination of an IP address that could be answered by either machine and the iSCSI drive mount which contained the actual files.

Once the resource was established it was hosted on NodeA. When I rebooted NodeA though the resource was automatically failed over to NodeB so that the total interruption in service was only a couple of seconds. NodeB took possession of the IP address and the iSCSI mount automatically once it determined that NodeA had gone away.

File serving is a really basic example but you can clustering with much more complicated things like the Microsoft Exchange e-mail server, Internet Information Server, Virtual Machines and even network services like DHCP/DNS/WINs.

Clusters are not the end of service failures. The shared storage can fail, the network can fail, the software configuration or the humans could fail. With a proper technical staff implementing and maintaining them, however, clusters can be a useful tool in the quest for high availability.

April 29, 2009

Musician's Mind

One thing I have noticed about my SoftLayer family is the number of musicians here. I spent nine years as a musician, working the bar/festival circuits all over the midwest. When I arrived at SoftLayer, people jokingly asked if I was joining the SoftLayer band due to my previous experience. Just looking at the Operations Management Team, most of us have written/performed and many continue to. The more I thought about it, I think that this is a good thing.

There have been a number of scientific research projects about the academic performance of children and teens who are involved in a music program at school. The mixed left/right brain activity of music lends itself to problem solving and critical thinking as well as creativity which are required by many jobs. A musician's mind is capable of working complex geometric patterns into physical movements in coordination with muscle memory. Once more advanced levels are reached, a musician is capable of not only composing music, but also improvisation. These aspects are similar to many necessary thought processes used in the office.

I'd like to think that these mental processes allow me to think better on my feet, deal with change, and have a global view of the projects that I am involved with. Maybe that's why our Inventory, Hardware, & 2/5 Datacenter Managers are musicians. Not only that, but our Director of Operations is an avid musician. If you count the actual system admins who are musical, then you would be adding another 3 people.

So if all the research is correct, this may have something to do with the quality of operations here at SoftLayer. I wonder if being a musician had anything to do with my interview and its result. I know that it would catch my attention knowing that an applicant had experience with music or any other analytical/creative endeavor, especially if it has been shown to improve overall performance or intelligence.

Maybe I should write a SoftLayer song - an anthem to our Datacenter or a love song about the management network and IPMI?

April 27, 2009

Into the Cloud

You'll see the word "cloud" bouncing around quite a bit in IT nowadays. If you have been following The Inner Layer you'll have seen it a few times here as well. A cloud service is just something that is hosted on the Internet. Typically in a cloud scenario you are not actually doing the hosting but rather using hosted resources someone else is providing. Usually you'll hear it in terms of computing and storage.

This is going to be a brief article on a cloud storage product we are doing here at SoftLayer called CloudLayer™ Storage.

CloudLayer™ Storage is a WebDAV based system which uses client software on the local machine in order to redirect filesystem operations to the storage repository here at SoftLayer. In Windows you end up with a drive letter; on a Unix system you end up with a mount point. In both cases when you create folders and save files on those locations the actions actually happen on our storage repository here. Because the files are located with us you are able to access them wherever you are. Through the CloudLayer™ web UI you're able to also set up files for sharing with others so even if where you are never changes there is still value to using the cloud storage.

Even using a cloud storage system you must maintain proper backups. Hardware, software and human error all happens. Tied in with that concept of "errors happen" ... if you have files on CloudLayer™ that you need for a presentation, download them ahead of time. You don't want to be caught without files simply because the Internet connection at your hotel decided to take a nap the morning of your event.

Now what of security? Well, the connection to the CloudLayer™ endpoint here at SoftLayer is done via an encrypted session so people cannot snoop on your transmissions. This does mean you need to allow 443/tcp outbound communications via your firewall but since that is the normal HTTPS port I'd imagine you already have it open. Within CloudLayer™ you can control with whom you share your files.

Since CloudLayer™ is a filesystem redirected over the Internet the performance you get will be dependent on your local connection speed. Its best to treat CloudLayer™ Storage as simply a file repository. If you find you need some kind of off-machine storage for running applications on your server you could look into our iSCSI product.

So, dear readers, go forth and have a cloudy day.

April 25, 2009

Facebook and Geezers

Last week, more rumors about the valuation of Facebook were flowing. So, is Facebook the real deal? Or will it go the way of the CB radio “social networking” experiment in the 1970’s?

Last weekend, I attended an event that indicates that Facebook has more staying power than those old CB radios. It was a quasi high school reunion. Since a lot of graduates of Brownwood High School (my alma mater) wind up in the Dallas-Ft. Worth area, a 2-3 hour drive away from Brownwood, we had a get-together in Grapevine, TX for Brownwood High grads living in the area.

At the event, the oldest grad I bumped into was from the Class of ’81 and the youngest I saw was from the Class of ’90. Yes, there’s a “19” in front of those graduation years, making the age range of people I saw between ages 37 and 46 years of age. I won’t disclose where I fit in that group, but in the world of Facebook, we’re all pretty much “geezers” I imagine.

I wish I had counted the number of times I heard Facebook mentioned at the party on Saturday night. Many times people told of who they had found on Facebook that couldn’t make it to the party. Some of the comments I overheard went like this:

“I saw those pictures of your kids on Facebook. Man they’ve grown!”

Q: “So, is that crawfish boil you posted on Facebook an annual event?”
A: “Yeah, it got kinda wild this year.”

“You said in your Facebook status a while back that your daughter got hurt. How’s she doing now?”

You get the drift, I’m sure. Most everybody there in this age range was active on Facebook and was already connected to several in attendance on Facebook. Since the event, I’ve received friend requests from folks I saw, and I’ve also sent out a few friend requests.

After we all made it home early Sunday morning (hey we’re not THAT old – at least we think we’re not), the Facebook fun continued. My email account pinged all day letting me know I’d been tagged in a photo here, someone commented on a photo there, etc. Yes the cameras were out Saturday night, and the contents of those cameras got uploaded, tagged, and commented upon all day Sunday. In fact, I was tagged in one photo that had the caption “Brownwood High School geezers from class of __.”

As far as Facebook goes, I’ll bet stories like this occur all over the country by the thousands. Provided that Facebook keeps its financial house in order, they’re here for the long haul I think.

So, what’s the connection to SoftLayer here? Easy. We have a lot of customers who provide apps on Facebook. The infrastructure for those apps is hosted at SoftLayer. Consequently, we’re big cheerleaders for Facebook and the apps that run upon it. Go go go!

April 23, 2009

Dress To Impress

I’ve recently discovered the TV show Mad Men, which is well into its second or third season now, and is just an awesomely good time. If you haven’t seen it or heard about it, the basic premise is this:

“Late 1950’s advertising agents in NYC drink, smoke and fool around too much.“

That’s about it. There are a lot of layers on top of that, but the heart of the show is a Lucky Strike and a whiskey neat. The main character is Don Draper, the coolest cat who ever cooled. He wakes up with perfectly combed hair, knows all the right things to say and owns more suits than Men’s Warehouse.

The scene (or line, actually) that got me writing was one where Don Draper is hanging out with some beatniks listening to Miles Davis and ends up getting into an argument with one of them about “Conformity VS Rebellion”. Don represents the suit wearing, 9-to-5’ers and the beatnik represents the free-spirited bohemian lifestyle. An unrelated incident brings the police to the building and the beatniks are all scared to go outside because of them. Eventually Don gets tired of arguing (I can’t imagine he’s tired of the Miles Davis. Everyone loves Miles Davis), and decides to leave. The following exchange takes place:

Beatnik: Hey man, you can’t go out there.. there are cops outside.

Don Draper: “….You can’t”

And then he leaves. Of course, the cop outside gives him no hassle at all, and the audience yells “Oh Snap!” Well, I did anyway.

This illustrates a personal philosophy of mine that I’m happy to say we use in a lot of aspects at SoftLayer, and that is “Dress to impress”. Whether you want to think about it or not, your wardrobe is going to tell people a lot about you before you ever get a chance to. This goes for a lot of other areas as well, not just your pants and jacket.

If your desk is cluttered up with papers, I’m going to assume you’ve got too much work. Or you are sloppy.

If your passenger seat is full of stuff, I’m going to assume that you never have passengers, which means you probably live alone.

Conversely if your car is uncluttered in any way I’m going to assume you never drive it.

The same goes for your web page, your order forms, your forums. All of these things sway the needle of the “Good/Bad” scale one way or another, and most of the time it’s really, really subtle.

At SoftLayer we spend a LOT of time on attention to detail. Our datacenter is so clean you can eat off the floor, and it doesn’t happen by accident. When people walk into the room I want them to say “wow!” and I’m happy to say that it happens every time.

What does your look say about you?

-Jeaves

Categories: 
April 22, 2009

The Tao of the Slayer

In the ever-changing world of IT, there are few times when a technician gets to relax. There are always new issues, new products/services, and long hours of investigation. However, once in awhile you find a moment of Zen in all the commotion: Rack Prep.

Recently, I had assigned myself to Rack Prep to allow my teammates to focus on their other duties. During this time, I was able to complete a large portion of the rack assembly process and release myself from the direct stresses of the IT environment in a busy NOC (network operations center).

The preparation of new racks in the datacenter is an arduous (and sometimes monotonous) task, but gives a technician time to reflect on his accomplishments and direction for his career. There are no distractions, other than the occasional dropped cage nut or screw. This allows the free mind to ponder the inner workings of itself and the body it inhabits.

I thought about the first time I had installed a rack rail. I had only been working in IT for a few months and was assigned to the task due to my lack of knowledge on the other portions of the project. I learned a lot that summer about architecture of hardware, networks, and even business.

I had time to think about how I had arrived at one of the fastest-growing host providers in the world. All the different places I had worked. I remembered the people who shared information – technical or otherwise – which had furthered my ability to solve issues – in servers and myself.

I remembered the managers and supervisors that I looked up to and hope to emulate in my current position in management. I was trying to remember all the wisdom which had been passed to me, leading me to reevaluate my approach and initiatives.

In short, the Rack Prep allowed me to reflect on all the things in my life. I was able to forget the current project while mindlessly pushing in cage nuts and look at my career from a wider perspective. Luckily, I can say that I am proud of how far I have come. Now, I have to install the cables which require much more thought. I better leave the Zen and continue with the task at hand.

.IIIi

April 14, 2009

EVA, Cloud Computing, and the Capex vs. Opex Debate

So far in 2009, there’s been a fair amount of discussion pro and con regarding the financial benefits (or lack thereof) of cloud computing. It’s very reminiscent of the whole “do-it-yourself” or “outsource it” debate. Blog posts like this and articles like this are samples of the recent debate.

One thing I have not yet seen or heard discussed regarding cloud computing is the concept of EVA, or Economic Value Added. Let me add at this point that EVA is a registered service mark of EVA Dimensions LLC and of Stern Stewart & Co. It is the concept of economic income instead of accounting income. SoftLayer subscribes to software from EVA Dimensions LLC. Get more info here.

For you to buy into the premise of this post, you’ll have to be sold on EVA as a valuable metric. Bottom line, EVA cleans up the distortions of GAAP and aligns all areas of the business so that more EVA is always better than less EVA. Most other metrics when pushed to extremes can actually harm a business, but not EVA. Yes, even bottom line GAAP net income when pushed to an extreme can harm a business. (How that can happen is fodder for another blog post.) Several books have been written about EVA and its benefits, so that’s too much to write about in this post. This is a good summary link, and for more info you can Google it on your own. And if you do Google it on your own, be warned that you may have to wade through links regarding Eva Longoria and/or Eva Mendes .

Part of the Cloud computing debate revolves around “capex vs. opex.” Specifically, this involves paying for IT infrastructure yourself using capital expenditures (“capex”) or employing Cloud computing and buying IT infrastructure with operating expenditures (“opex”). Geva Perry recently said, “There is no reason to think that there is a financial benefit to making an OpEx expense vs. CapEx expense. Period.” I disagree. When you look at this in terms of EVA, whether you use capex or opex can make a big difference in creating value for your business.

Let’s look at the effect of switching capex to opex on EVA. Coca-Cola is a company that employs EVA. Years ago, they decided to ship their beverage concentrate in single-use cardboard containers instead of reusable stainless steel. This made GAAP measures worse – profit and profit margins actually went down. But EVA went up by making the move from capex to opex. How can this be? Grab something caffeinated and check out some numbers here if you dare.

OK, that’s all fine. But how would shifting IT spending from capex to opex affect EVA? Glad you asked. Last summer, I modeled some full-fledged financials to illustrate financial benefits of outsourcing IT vs. doing it yourself. I’ve taken those and added the EVA calcs to them. Take another swig of caffeine and check them out here and here.

Assuming that EVA is a worthwhile metric (and I think it is), moving capex to opex is possibly a very good financial decision. Any questions? As always, your mileage may vary. Model carefully!

April 8, 2009

More “SLingo”: SLanket vs. Snuggie

I am not ashamed to say that I own both the Slanket and the Snuggie. I got the Snuggie as a gift last year, and I purchased the Slanket for myself a couple of months ago (only partly because of the cool SL name). The Slanket turned out to be a WAY better product. As I sat on the couch with my Slanket and my kitten Linux, I started drawing these hilarious and uncanny parallels between the Slanket and my company, SoftLayer. Once I got started, I just couldn’t stop:

  1. The Slanket is bigger, and thicker. The fabric of the Slanket is far more robust, and doesn’t start “pilling” immediately. It’s designed for constant and vigorous use, and will last me a very long time before wearing out.
  2. There are more buying options with the Slanket – they come in different sizes and many more colors (including a shade that’s *almost* SoftLayer Red), so there’s a Slanket for every need.
  3. The Slanket has a better name – Sleeves + Blanket = Slanket. I’m not likely to talk about my “Snuggie” to my peers, because … well, I’m not a 4-year old.
  4. While the Slanket is a bit more expensive, it is a far superior product. In the Blankets-With-Sleeves industry, you seem to get what you pay for.
  5. I used the Contact Us form on the Slanket website to talk to their SLales team, and got great, personalized service, and a discount! Btw, the owner’s email address is also available online, which shows that he is involved in the day to day operations, and wants to hear from you.

Does any of this sound familiar? These are the same things people love about SoftLayer. I am a very proud SLanket customer. A++ would buy again.

SLanket

April 6, 2009

Solid State Drives – In House Performance Stats

I love working at SoftLayer. I get to play with the newest hardware before anyone else. Intel, Adaptec, Supermicro… The list goes on. If they are going to release something new, we get to play with it first. I also like progression. Speed, size, performance, reliability; I like new products and technologies that make big jumps in these areas. I am always looking to push components and complete systems to the limits.

But alas, Thomas Norris stole my thunder! Check out his article “SSD: A Peek into the Future” for the complete skinny on the SSD’s we use. I seem to be a bit to concise for a nice long blog anyways. But not to worry, I’ve got some nifty numbers that will blow the jam out of your toes!

Solid State Drives (SSD) represent a large jump in drive performance. Not to mention smaller physical size, lower power consumption, and lower heat emissions. The majority of drive activity is random read/write. SSD drives have drastically improved in this area compared to mechanical drives. This results in a drastic overall performance increase for SSD drives.

This is a comparison of the Intel 32GB X25-E Extreme drive vs. other drives we carry. Note the massive jump in the random read/write speed of the SSD drive.

No more waiting on physical R/W heads to move around. How archaic!

Chart

Please note that no performance utility should be used to definitively judge a component or system. In the end, only real time usage is the final judge. But performance tests can give you a good idea of how a component or system compares to others.

Single drive performance increases directly translate into big improvements for RAID configurations as well. I have compared two of our fastest SATA and SAS four drive RAID 10 setups to a four drive SSD RAID 10 using an Adaptec 5405 Controller.

Chart

The Adaptec 5405 RAID controller certainly plays a part in the performance increase, on top on the simple speed doubling due to 2 drives being read simultaneously. (See my future blog on the basics or RAID levels, or check Wikipedia) .

Propeller heads read on:

The numbers indicate a multiplied increase if you take the base drive speed (Cheetah – 11.7mbps / X25-E – 64.8mbps) and double it (the theoretical increase a RAID 10 would give): 23.4mbps and 129.6mbps respectively. Actually performance tests show 27.3mbps and 208.1mbps. That means the Cheetahs are getting a 15% performance boost on random read/write and the X25-E a whopping 37% due to the RAID card. Hooray for math!

Once again, this is all performance tests and a bit of math speculation. The only real measure of performance, IMO, is how it performs the job you need it to do.

April 2, 2009

We Need New Small Businesses

It is often said that small business is the backbone of our economy. According to the U.S. Small Business Administration, small business employs half of all private sector employees. Over the past decade, small business has produced between 60 and 80 percent of net new jobs. We need small businesses to prosper and lead us out of the economic mess in which we find ourselves.

I track growth in domain names every week. I think it indicates how quickly new small businesses are being formed. After all, what business can you think of today (large or small) that does not have some sort of web site? I can’t think of any. One of the things on any small business start up checklist today is the web site. Hence, most all of them register a domain name.

So what’s been happening with growth in domain names? Lately, it’s not too pretty.

Chart

With all the talk lately about stimulating the economy, one of the best ways to do this would be to encourage the formation of new businesses.

Some would argue that we need to fix the credit market mess to help banks be able to lend to small business startups. This couldn’t be further from the truth. How many small businesses do you know that started with a commercial loan from a bank? I cynically say that banks do not want to loan to businesses until the business can survive without need of a bank, and that was true even before the credit crisis. This was certainly true in SoftLayer’s case – when the founders were preparing for launch in late 2005, there wasn’t a bank anywhere that would touch the SoftLayer business plan. What I’m saying is that the credit crisis isn’t that much of a barrier to small business startups. Passionate entrepreneurs will find a way to get going.

But all the passion to start one’s own business doesn’t go very far in the face of the real barriers to starting a business. One of the real barriers that an entrepreneur must overcome is tax issues. Do they begin as a sole proprietor? A partnership? An LLC? An “S” Corp? Should they incorporate? All of them have different tax implications. All of them have to deal with either income taxes at the personal level or corporate level. Some have to deal with self-employment taxes. Others must deal with 941 taxes. Then there are state and local tax issues, such as the margin tax if you’re in Texas. And don’t forget sales taxes and property taxes either.

One of the strategies that allowed the Internet to cement itself in our society during the 1990’s was this: just let it develop without taxing it. Without that burden, the Internet took off like wildfire.

Ergo, if we’d like a bunch of new small businesses to get going, let’s ease up on the tax burden on new startups. This would cost the government hardly any money at all. Think about it – businesses that don’t yet exist do not pay any taxes. Workers that are not yet employed do not pay any taxes. Currently unemployed workers do not pay income taxes, except for a pittance on unemployment benefits. So allowing new businesses to form and employ workers and transact business “tax-free” for a defined start-up period would produce an EXPLOSION of small business startups.

How long should this tax free period be? Per the SBA, if a new business survives 4 years, they have a great shot at surviving long term. So why not give all new business startups a tax holiday for four years as they establish themselves? Can you imagine how big the tax base would grow as these healthy, strong 4-year- old businesses begin paying taxes?

It seems that the biggest issue facing our new President and his administration is how to pay for all the things they’d like to do. Let me suggest that expanding the tax base is the best way to grow government revenues, as opposed to increasing the rates on the current tax base. Allowing a flood of new businesses to take root and grow our tax base may be the best way to fund our growing public budgets.

Naturally, SoftLayer would be more than happy to assist these new businesses with our enterprise class data center outsourcing services so that the new businesses focus on their business plan – not their IT overhead.

Pages

Subscribe to executive-blog