Posts Tagged 'Power'

October 27, 2010

Oh No CoLo, Go Go Godzilla (Apologies to Blue Oyster Cult)

A traditional Co-location has certain advantages and for some customers it makes a great deal of sense. At least it does at first blush. Take a look:

  • Colo is cheaper than doing it yourself as physical infrastructure costs are shared across a number of customers.
  • The hardware is yours, not the co-location company’s. This means you can scale in the manner you please versus what suits the business model of the co-location company. The potential downside is that this assumes you were smart enough to pre-buy the space to grow into…
  • The software is yours, too. You are not limited to the management suite provided by the co-location company. Use what you wish.
  • Colo centers are usually more robust than a typical business environment. They deploy more physical security in an environment that is designed to properly manage power (multiple generators on-site for example) and the risks associated with fire and other natural disasters.
  • Upgrade paths are determined by you versus the hosting provider.

But what about the cost side of the equation? What does that look like? It goes without saying that it is (usually) cheaper to use a provider like SoftLayer to host your gear, but by how much? We have built a relatively simple model to get at some of these answers.

Assumptions:

  • A mix of 75 small servers (Xeon 5503, 2 GB RAM, 250 GB SATA) and 75 large servers (Xeon 5520, 3 GB RAM, 250 GB SATA)
  • Colo pricing was based on $100 per U per month, or $2,500 per 40U rack per month cost. Colo capex assumed the same base configuration but at current market prices.
  • We assumed a $199 price point for SoftLayer’s small servers and $359 for large servers
  • Bandwidth consumption of 2500 GB per server per month (this is about 50% of what we see in house). A price of $50 per Mbps was used.
  • A refresh schedule of 50% at 36 months, 25% at 48 months and 25% at 60 months

So what do the numbers tell us? Well, I think it paints a pretty compelling picture for SoftLayer. The 60 month Total Cash Outlay (TCO) for Colocation is 131% of the SoftLayer cost.

Total Cash Outlay

  Collocation Softlayer
Initial Capital Expenditure (Cash Outlay) $341,700 $0
Monthly Recurring Charges $64,778 $60,450
60 Month TCO $4,740,917 $3,627,000

In addition to the total cash outlay, we can add in a bunch of additional “hassle costs” – the hassle of driving to the DC in the middle of the night for an emergency, the hassle of doing your own software patching, setting up your own monitoring, waiting on hardware delivery (and you are not going to be first in line given your volumes are likely to be low compared to SoftLayer), the hassle of booking assets to the balance sheet, depreciation entries, salvage accounting entries, actual equipment disposal, downtime while you perform upgrades – ugh, the list is almost endless.

The argument for a SoftLayer solution is pretty strong based on the numbers alone. And I think that they ought to be persuasive enough for most to rethink a colocation decision. That said colocation decisions are not made from a cost perspective alone.

For example:

  • Issues around data integrity and security often drive companies to adopt a corporate philosophy that dictates co-location (or an on premise solution) over an outsourced solution. There is a deemed corporate need to have data / applications running over their own iron. Indeed, for many, colocation represents a significant and progressive decision.
  • Many companies have infrastructure in place and a decision will not be made to veer from the current solution until a technology refresh is in order. Never mind that fact that a transition to an outsourced solution (and this is the case when lots of things are outsourced, not just infrastructure) can generate significant internal anxiety.

Many outsourcing adoption models seem to show a similar trend. To a degree much of this becomes a market evolution consideration.

  1. Adoption is very slow to start. Companies do not understand the new model and as a result do not trust vendor promises of cost savings and service delivery. To be fair to customers, service delivery for many solutions is poor at the beginning and cost savings often disappear as a result.
  2. The vendor population responds to initial concerns regarding service delivery and perceptions around cost savings. Innovation drives significant improvements from a product and service delivery perspective. The solution now seems more viable and adoption picks up.
  3. For some services (payroll is a good example), the cost savings of outsourcing the solution are realized across the marketplace with excellent service delivery and support being commonplace. We are close to mass market adoption, but some companies will opt to keep things in house regardless.

So where are we on the evolutionary curve? That is a difficult question to answer as there are numerous things to consider dependent upon where you want to look.

For most SMBs, outsourcing functions like HR/Payroll or their IT infrastructure is a no brainer – capital is not as readily available and existing staff is likely overburdened making sure everything else works. At the end of the day, the desire is to focus on running their business, not the technology that enables it. The decision is relatively easy to make.

As we go further up the food chain, the decision matrix gets infinitely more complex driven by an increase in geographic reach (local – national – international), an increase in the complexity of requirements, an increase in the number (and complexity) of systems being used and typically large IT organization that can be a terrific driving (or drowning?) force in the organization. The end result is that decisions to outsource anything are not easy to reach. Outsourcing occurs in pockets and SoftLayer certainly sees some of this where enterprise customers use us for a few things versus everything.

At the end of the day, the hosting market will continue to be multifaceted. All businesses are not alike and different needs (real or otherwise) will drive different business decisions. While I believe colocation will remain a viable solution, I believe that it will be less important in the future. The advantages presented by companies like SoftLayer only get more powerful over time, and we are going to be ready.

-Mike

December 24, 2009

The Power of Christmas

The Power of Christmas

Putting up Christmas lights this year was a serious beating. I kept blowing breakers due to the amount of lights I put up in response to my wife’s request for ‘more lights!’ It seems like every year things get bigger and bigger (like most things in America – trucks, combo meals, taxes, and the deficit). The problem is there is only so much power in convenient areas of my house and those locations don’t have enough power to run my lights because they are shared with things inside the house. My front porch outlet ties in with my garage outlets so every time we open up the garage door, the breaker blows and the Christmas lights on the front of the house go out. I got tired of resetting breakers and I ended up running 2x 20amp 110v dedicated feeds to my roof and to the front yard.

As I was putting the lights up, I found myself doing power calculations in my head. I multiplied the amount of lights I put up by the watts each bulb consumes to get the total watts. Then I took the total watts and put it into this conversion tool (http://www.mhi-inc.com/Converter/watt_calculator.htm) to calculate what they use in a Kilowatt hour. I have timers setup to turn on the lights from 6pm to 11pm (CST) so that is 5 hours a day. I plan to run them from December 8th through January 3rd which is 27 days totaling 135 hours of run time. Take the Kilowatt hour the lights generate times the hours of operation and you get the total Kilowatt hours used for the holiday season. I was then curious how much this was going to cost me (I am a cheap bastard) so I took out my electric bill (TXU, yes I am paying too much) and took what they charge me for a Kilowatt hour and got the dollar figure it costs to run the lights. I was surprised it is not as much as I thought considering how much light my house now generates. It lights up the neighborhood like the Griswold’s house in Christmas Vacation <http://www.imdb.com/title/tt0097958/> . I would not be surprised if you can now see my house from the space shuttle.

I don’t envy Softlayer’s operation guys because they do these types of power calculations (albeit on a much grander scale) on a daily basis. They have to figure out what types of servers with different components (CPU, drives, memory, raid cards) can go into a single rack to insure that power strips are not blown. Some people don’t understand that you can’t just fill a rack up with 44 1U (or 22 2U) servers and turn them on. You have to carefully plan down to the watt how many of each type of server can go into a rack without overloading circuits. You also have to take into account customer upgrades and make sure there is enough headroom for power spikes upon booting. The math involved in my yearly Christmas light escapade made my head hurt; I can’t imagine what Robert and Brad go though. Hats go off to them. My head would have exploded by now….

Here is the math (rounded):

15 ½ stands of C9 Christmas lights each with 25 bulbs = 385 bulbs
385 7 watt bulbs = 2695 watts
2695 watts = 2.695 Kilowatt hours (from http://www.mhi-inc.com/Converter/watt_calculator.htm)
2.695 Kilowatt hours multiplied by (5 hours a day for 27 days = 135) = 364 total Kilowatt hours
364 total Kilowatt hours times $0.12 = $44

So lighting my house for one month actually uses significantly more electricity than running a server in a SoftLayer data center for the same period of time.

September 22, 2008

Yikes!

For starters, let me say that if you were affected by IKE my thoughts are with you as you try to cleanup and get life back to as close to normal as possible. I can’t even fathom what it would be like to be away from home and work and not know what lies ahead or even be allowed back to see the damage.

A few friends of mine live in the area surrounding Houston and I have heard from them that they still don’t have power and gas is hard to come by. It sure makes me understand what I take for granted every day. Even with short power outages at my house due to spring storms I find myself opening the microwave and then realizing that my burrito isn’t going to be very warm when I think it is done.

This particular blog came to me when I emailed one of my friends in far north Houston at his work email address. He works for a company that is also located in north Houston. After no response I decided to call him to get an update but ended up leaving a message. Before he called me back that evening my email to his work bounced. When I did talk to him I asked him a simple question, “are your company’s servers sitting in a closet onsite at your office?” He responded with what I already knew, “Yep!” Now, he is a national account manager and he has been meeting at his boss’s house this week to go over what they will need to do once things are back to normal. I asked him if it would help them if they could still receive emails from their customers and if they could post updates on their website if it was still up and running. He of course said, “Yep!”

So here is my plea to all you savvy IT gurus out in the world. Outsource it! Then you don’t take the brunt force of the storm when everything is down. Your servers, whether or not you are still receiving email, and whether your website is still up and running will probably and should probably be the last things on your mind when a disaster like IKE strikes. Leave that part to us.

-Skinman

February 4, 2008

Nnet Strikes Back

I'm not going to tell you my name for two reasons: First, I don't want a million tickets assigned to me asking if I'm crazy. Second, if I am crazy, I don't want anyone knowing it's me.

I'm not a writer myself, so I asked Shawn to write this up for me. He's a programmer, and more important a Trekkie, so he's likely to understand (and more important, believe) this story. Besides, he's written a few humorous, slightly preposterous posts for this blog, and that's very, very important.

Unlucky as I am, I was the first person to notice something strange going on. I'm a datacenter tech for the company (but I'm not going to tell you WHICH datacenter), and my job... well, I'm the power guy. I make rounds in the datacenter, checking breakers and power panels, keep an eye on voltages in the portal, that kind of thing. No power issues at the datacenter? That's because of me. So, I'm perusing the tickets and keeping an eye on things, like I should.

As I was answering a particularly interesting ticket, I received an IM from a datacenter engineer I hadn't met yet. That's not surprising; we're growing like crazy here, and I don't always get the "Welcome a new employee" email before I find myself working with the guy or gal. I finished my ticket and opened up the IM window. It was from "Nnet," and the contents caused me to leap out of my seat:

"The power strips on the new racks (205, 206, 207) are drawing too much current; it will pop the breakers in 52 minutes, 12 seconds."

I had just CHECKED those racks. I walked down to the server room, muttering about some whippersnapper of a new engineer playing a trick on me. I was going on vacation in a week, and I did NOT want any power issues; I was training another engineer to take the console while I was gone, and if anything happened during testing I would surely be called in. Anyway, I walked into the server room and checked the gauges on the power panel.

And they were drawing almost a full five amps too much. If we had turned on the third rack, the whole aisle would have gone down. That wouldn't have been too bad; no servers were hooked up. This is exactly why we test the power before we put servers in.

I and the rack crew worked for about an hour rewiring the racks, starting from the third rack. Sure enough, about 52 minutes later, rack 205 shut down. Mentally thanking "Nnet" for finding this (and more importantly, not tinkering with it before letting me know!), we got the racks wired more efficiently (they're supposed to be on separate breakers, but the electrician labeled the wires wrong), reset the breakers, and had absolutely no issues for the rest of the day.

I got back and thanked "Nnet" for finding that issue. The next day, I got to thinking about how "Nnet" had saved my vacation (I would have spent all week tracing wires to figure out what had happened), and I wanted to invite him or her to lunch. So I IMmed "Nnet" with an invitation. An hour went by with no response, but it's not too strange to have a datacenter tech away from their desk for a couple hours. So I sent an email to Nnet.

The email bounced back.

-"Mystery Author"
Maybe HR hadn't set up the email yet? So I called them up to see what was up with Nnet's email address.

That's when HR told me that nobody with the last name "Net" had been hired (I thought "Net" was a strange name for a tech, but it's not the strangest last name I've ever heard). I called the networking department to ask how I could receive a company IM from somebody who doesn't work here? They researched it and couldn't find any incoming links through our firewalls or any of the internal logs. Stranger yet, the Jabber server indeed DOES have an account for "Nnet", but the engineer who runs the server swears that he never set that up.

We were discussing this back and forth when one of the developers walked by, overhearing our conversation. He laughed, and when we asked why, he told us that he was reading a book about the human brain, and that the brain is made up of million of millions of neurons all interconnected with each other; that these interconnected neurons work together to create intelligence.

Could that be true? Absolutely not. It's preposterous. Sure, we've got tens of thousands of computers around here, dual cores and quad cores running various operating systems and applications, all connected by an incredibly fast private network...

...could it be?

The engineers are all completely sure that one of the datacenter techs must be playing a joke, and they're currently tracking it down. But I'm not too convinced. "Nnet" knew which power strips were having trouble in a room keycarded to open only for me and a hand full of other techs. And they all swear they didn't send it.

That's when I talked to Shawn. He told me that there's a lot of technically minded people out there who read fantastic science fiction stories and come up with solutions... even knowing that the tech is impossible, they can find a way to solve the problem. So we hatched up this idea to write out a fantastic blog post, an interesting narrative of my predicament.
Then we'd post it to the blog and watch for any discussion on the customer forums. Our customers are really smart, and they like solving problems. Maybe somebody out there has an idea of how we can figure out what's going on around here.

So here's the story. A completely fantastic modern day science fiction story about a sentient datacenter.

Preposterous!

...any ideas?

February 1, 2008

I Outsourced It

Have you ever wanted to tell your CIO that? His response might be, "you outsourced what?? You respond, "it!" With a perplexed look he asks again, "You outsourced what, it?" Again you respond with, "All of it." His reaction at that point could go either way. In most CIO type heads today, they can't grasp the savings associated with outsourcing and even the ones that DO understand would then have to go to the CEOs office and inform him or her that all of the company's valuable data will now be housed in a safe and secure facility off-site on dedicated servers... or "Hosted IT" even. Stop reading and go tell your CEO that right now. I'll wait...go ahead.

Ok, I see that you are back, are you still employed? We are hiring if you need a new job resumes@softlayer.com

Ok, really, how do you think that conversation would go? I have had that same conversation with ex-bosses and owners of small and medium sized businesses in the past and most of the time they don't go very well. Granted they were a few years ago so hopefully times are changing.

I have been told a few times, "no, I don't want to pay $300 per month for a server we don't own and put my data on it! That is ridiculous, just go buy me a new $3,500 server and we will put it in our local Datacenter, Server Room, Broom Closet, Bathroom, Office Managers office..." well you get my drift.

"But Sir, with this outsourced server we could easily have off-site backups, more processing power, some cool redundancy and it will not annoy everyone in the office with the loud fans and heat generation. And when we have a power outage in the office and everyone goes home for the day, they will be able to work from home because the server will still be online. Oh yeah, and our company website and email will still be functioning as well."

"Are you insane? Those challenges are so easy to overcome. We will simply add a small air conditioner to the broom closet and buy a big UPS system that will keep the server alive in the event of another power outage, and we can hire a service to come by every morning to pick up tapes and deliver them to an offsite bunker. Instead of a single connection to the internet we can buy two and have redundant connections also."

"Sir, I am no accountant, but by the time you pay someone to keep up with the depreciation of a new server, buy and install a small A/C unit and UPS unit, pay for a 2nd internet connection that will sit idle and pay a service to DRIVE here daily I really think the outsourced server would be cheaper. Not to mention in the event of data loss we could get the data restored to the server much quicker than waiting on a service to physically bring it to us." An interesting note here is, I don't care what kind of offsite data bunker you have, the Monster in Cloverfield IS going to destroy it so think multiple copies of data in multiple cities!

"Well I have made my decision; we will not be outsourcing my very valuable data - Hackers might get it, it is more secure here, so leave my office. Before you go could you please try to get my printer working again, and I am getting this annoying pop-up about spyware and it seems that my ITunes files have lost their license and I used to have a folder called Docs on my desktop with everyone's salary in it that is missing and my PDA will not sync...(zzzzzzz) -- OUTSOURCE IT!

-Skinman

January 17, 2008

Whatever Happened to CEOs - Chief *Electricity* Officers

The power grids that we enjoy today did not magically appear as power generation developed during the Industrial Revolution. In 1902, according to the US Census, the country had 3,600 central systems and over 50,000 isolated power plants in large homes, hotels, and other commercial establishments. Thus, it’s a pretty safe bet that companies employed a fair amount of people who were tasked with “keeping the lights on.” For our purposes, we’ll call them the Chief Electricity Officers (CEOs).

This was the era before electricity was a utility. As we know, the power grid eventually encompassed the whole country and provided all needed electricity on tap. Once companies found that it was far more economical to plug into the grid than to generate their own power, the poor CEOs had to find other things to do in their organizations. Of course, industry regulation played a part here also, but the basic economics worked – with the grid in place, it was cheaper to buy power than do it yourself.

I see several parallels in this present Information Age. Many companies have Chief Information Officers and/or Chief Technology Officers. Part of what these folks are tasked with is IT infrastructure, i.e., acquiring the computing and networking gear required by the business and operating it in a redundantly powered and cooled data center. In most companies, IT is not the core competency of the business, yet they lay out a lot of capital expenditures and employ a lot of people for an overhead operation – much like what the old CEOs did with independent power generation.

SoftLayer and companies like us parallel the rollout of the power grid which began about a hundred years ago. In the coming years, companies will realize that the time, people, and capital expenditures required to locate data center space, find redundant power, set up backup power, install redundant HVAC systems, expend capital to acquire routers/switches/servers/storage systems/load balancers/firewalls/operating system software, and hire the people to run it all and upgrade everything every few years will be far too great a cost compared to “plugging in” to a provider such as SoftLayer. With Softlayer, IT infrastructure is our core competency. Companies need only give us a shout to instantly have IT infrastructure provided at far better economics than doing it themselves. They’re essentially “plugging in” to IT as a basic utility to be used to perform their core competency – just like they plug into the power receptacle to use electricity to help perform their core competency.

Hey, when Sun Microsystems says they’ll be out of data center operations by 2015, that raises our eyebrows around here. Dan Golding of Tier 1 research concurs that by 2015 “enterprise data centers will be in decline.” Once the business leaders of companies grasp the economics of halting their independent data center operations in favor of plugging in to utility providers, the CIOs and CTOs will have to do what the old CEOs did – find other ways to add value to their companies.

-Gary

October 11, 2007

The Three P's are Changing

The three P's in the hosting world have always been Ping, Power and Pipe. Salespeople regurgitated them relentlessly and operations personnel just shortened them to the P's because we talked about them all the time. The three P's of hosting have changed in the recent years and those not aware of the changing landscape are doomed for failure. I propose a new three P standard (described below).

1) Power -- I list this one first because it is by far the most important. Power is the single greatest limiting factor to technology. If you don't understand the importance of power on future technology, you should exit the industry now. If you are not concerned with power, don't meter power and not fixated with power, you will be in serious trouble in the next 12 to 24 months. The entire industry has shifted to being "green" and large scale datacenter operators are so focused on power utilization, they are building and designing systems completely based on power usage and/or location. It's one of the most critical operating costs and must be understood to maximize long term success and profitability. Here at SoftLayer, we are obsessed with power utilization and efficiency and focus on mitigating power and heat (byproduct of power) to a bare minimum. We know the power usage of every server and network device located in the datacenter and track it real time. We are continuously seeking new low power technologies, engaged in industry consortiums looking for new alternatives, and actively planning our power needs through the end of 2010.

2) Packets -- Five years ago, the internet backbones were full of big fat packets that were easily passed by backbone and edge routers without issue. In the recent years, small packet technologies have greatly reduced the size of the average packet transversing the internet. For those of n00bs out there, smaller packets reduce the overall throughput of the routers processing the packets. The smaller the packets, the greater the reduction in horsepower of those routers. The fast rise in gaming, VOIP and other small packet intense applications has cut the average packet size in half in the last two years and I would expect that to occur again the next two years. Packet size can take the aggregate throughput of a router from several hundred gigs at large packet sizes to potentially single digits of gigabit throughput due to the processing required. Here at SoftLayer, we have installed and upgraded to the fastest routing technologies by Cisco to ensure the greatest network performance, but there are many legacy carrier, broadband, and enterprise routers out there that have limited capacity due to changing packet size. Hosting providers that were built on eBay surplus network equipment from the late 90's will soon begin to implode.

3) IP's (IP Addresses) -- Ok…not really a "P" but I take a little creative leeway here. IPv4 addresses are disappearing faster than norm's plate at the Hungry Heifer. ARIN has publically announced the need to shift to IPv6 and numerous articles have outlined the D-Day for IPv4 space. Most experts agree, its coming fast and that it will occur sometime in 2010 at the current pace (that's about two years for those counting). IPv6 brings enough IP space for an infinite number of users along with improved security features and several other operational efficiencies that will make it very popular. The problem lies between getting from IPv4 to IPv6. We are caught in this "chicken and egg" scenario where we can't leave one without the other being completely reliable. Although I think we will get to IPv6 without too much of a headache, I do think the IPv4 space will become extinct prior to a full scale transition and there will be a time where the cost of IPv4 IP's will skyrocket because of supply/demand. This should be at the top of your list as a hosting provider because additional IP space typically means new customer and/or expansion of existing customers. If you don't have a conservation plan for IPv4, migration plan for IPv6, and transition plan between the two – you may already be too late. Here at SoftLayer, we have been planning for over a year and 2008 will include a rollout of IPv6 to all those customers who seek to run dual stacks and will include incentives to customers who are able to shift to IPv6 completely.

The Three P's will likely change again in a few years as the industry continues to evolve and we find a way to solve the current challenges facing the industry. For now, focus and plan on these three and you should have a long successful existence.

-@lavosby

June 12, 2007

Being Green

For so many years growing up, I heard the "Sam I Am" / "Green Eggs and Ham" comments when being introduced to other kids. At this point, you would think I would hate the color green. On the contrary - being green is good.

One of the biggest costs in a datacenter is power, and if you're involved in datacenter operations you get to experience first hand the challenges of juggling power, cooling and floor space availability. If you use less power, your electrical costs go down and your cooling costs go down and there is a ripple affect across the entire facility. In an effort to reach that goal, we do everything we can to hone down the power requirements of our servers. We start by using 240v circuits to the rack. Doing so eliminates the need to step down to 110v which is much more efficient and it helps eliminate harmonic feedback in the circuit. Add to that “less heat” which means less wear and tear on the servers and that is a good first step.

Once you get power to the server, it helps to spec your servers properly. A properly sized power supply can save more than 25 Watts per server. When you multiply that by just 1,000 servers, that's a cool 25kW of power savings. When you multiply that by the number of servers in our facilities? Well, it's certainly worth the exercise of making sure we are ordering the proper equipment.

Aside from server equipment and datacenter power, SoftLayer has recently joined the Green Grid (more info). We are looking to use that association to join the likes of AMD, Intel, Dell, HP, IBM, Microsoft and many more to help reduce overall power consumption by datacenters. There are many lessons yet to be learned by IT companies to help reach that goal.

Being green is not confined to datacenter facilities. On SoftLayer Truck Day, we receive hundreds of cardboard boxes. Rather than just throwing those all away, we work with a local vendor to make sure the cardboard and packaging materials inside get recycled. Each server comes with various parts that are not needed (it's cheaper for the vendor to just ship the servers with all misc parts than it is to strip specific parts from specific orders). It would be easiest to just deposit all of those unneeded parts into a dumpster, but being green means doing more than just whatever is easiest. We sort spare power cords and recycle those for the copper. We sort screws and sell them to a local vendor (and use the money to buy Monster). Any spare part that we have not found a specific destination for, gets donated to a group that sells the parts and makes donations to charities.

Being green not only makes good financial sense, but it also makes good ecological sense. And – it keeps us stocked with Monster.

-SamF

Subscribe to power