Cabling a SoftLayer Server Rack

May 6, 2011

A few weeks ago, SamF posted "Before They Were SoftLayer Data Centers," a virtual scrapbook from the San Jose data center construction process, and based on the surge of traffic we saw to the post, our customers loved it. It's incredible to see an open warehouse-looking space transformed into an enterprise data center environment, and there's more amazingness where that came from.

In addition to the pre-"Truck Day" pictures we posted on the blog and in the San Jose DC Construction album on Flickr, we trained a video camera on a row in the data center to capture the cabling process.

What's so interesting about plugging in cables?

Consider the fact that each of the network switches we use in a rack has at least 48 ports. Now consider that each rack has two public network switches, two private network switches and one out-of-band management network switch that need to be connected to every SoftLayer server in the rack. That's 240 pre-measured network cables that need to be labeled and routed to specific heights in each rack ... without getting tangled and knotted up (see: behind your TV or under your computer desk).

The cabling process is so precise that if a single cable is out of place, the zip-tie on an entire bundle will be cut, and the process is started from scratch. The process is time-consuming, but the results speak for themselves:

SoftLayer Server Rack

Without further ado, here's the SJ data center team in action. The video is playing at 20x normal speed, and given the amount of time it takes to complete the cabling process for each rack, we enlisted the help of Spongebob SquarePants in our use of the "Two Hours Later" cut:

Impressed? Amazed?

Just wait until you see the time-lapse from Truck Day.

-Kevin

Comments

May 6th, 2011 at 12:48pm

That was awesome! I'm so glad that I don't have to worry about that stuff because I know Softlayer has it all taken care of. I can just order up some equipment and know its going to work hard for me.

May 6th, 2011 at 12:52pm

The ending makes the video worth it.... Play on!

:D

May 6th, 2011 at 6:47pm

Nice to see the redundancy in the cabling. It is very cool to see how organized those are.

And the ending definitely makes the video worth watching :)

May 7th, 2011 at 1:36am

What song was he singing/playing at the very end?

May 7th, 2011 at 9:26am

That was a fantastic video, just shows you how teamwork will prevail as those switches were racked up and cabled just awaiting on the servers, which was done in no time at all.

Very impressed, very professional approach with some good humour thrown in for good measure. Spongebob I think?

May 8th, 2011 at 10:41am

Makes me embarrassed to look at the rat's nest of cables under my desk right now. I can't even keep the 8 or 10 cables under there organized.

Curious though, what do the different colors mean? I assume that they're significant. Maybe internal vs external network?

May 9th, 2011 at 8:34am

Austin, I'm not sure what song he's playing at the end. He lip-syncs a few of the words, so if someone can figure it out, I'll send them a big package of SoftLayer swag.

Darren, the "2 Hours Later" clip was indeed from Spongebob ... It'd probably be best for all parties involved to not admit how quickly that clip came to mind. :-)

Ryan, the red cables carry public network traffic, the blue cables carry private network traffic and the green cables carry out-of-band management network traffic. A couple eagle-eyes on Twitter noticed some gray cables in the mix, and those are used in our cloud infrastructure to connect the cloud servers to the cloud storage.

May 10th, 2011 at 4:20pm

this is nice!
thanks a lot for the video!
I hope I'll see the contiuing of it: mounting servers and arranging other stuff.

May 10th, 2011 at 5:22pm

Every server in each rack has 4 NICs and a IPMI device in it? Or are you using 1U twin servers?

May 11th, 2011 at 10:05am

Each new server has a public connection, a redundant public connection, a private connection, a redundant private connection, and the IPMI connection.

May 12th, 2011 at 3:56pm

Mad props on a neat job.

However, if y'all anticipate having to replace any cabling, you might wanna think about velcro straps instead for the next job--so much easier to remove and replace than zip ties.

Otherwise, mad props.

May 16th, 2011 at 8:39am

@me, That's a good observation. In practices, zip ties are less expensive and tend to hold more securely/tightly. Because the cabling doesn't need to be changed very often, we don't need the ties to be easily removable.

January 9th, 2012 at 8:26am

how on earth do you keep up on top of the network management?

five 3750's to service 40 odd servers is massive.

January 20th, 2012 at 4:26am

Managing a network is really an art. Imagine dealing with all those cables and still making your data center neat like that. AMazing!

April 27th, 2012 at 2:49pm

Do you use cables with the same lenght?
what do you do with cables for servers near the switches? These will have a cables too long, you cut it?

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Comments

May 6th, 2011 at 12:48pm

That was awesome! I'm so glad that I don't have to worry about that stuff because I know Softlayer has it all taken care of. I can just order up some equipment and know its going to work hard for me.

May 6th, 2011 at 12:52pm

The ending makes the video worth it.... Play on!

:D

May 6th, 2011 at 6:47pm

Nice to see the redundancy in the cabling. It is very cool to see how organized those are.

And the ending definitely makes the video worth watching :)

May 7th, 2011 at 1:36am

What song was he singing/playing at the very end?

May 7th, 2011 at 9:26am

That was a fantastic video, just shows you how teamwork will prevail as those switches were racked up and cabled just awaiting on the servers, which was done in no time at all.

Very impressed, very professional approach with some good humour thrown in for good measure. Spongebob I think?

May 8th, 2011 at 10:41am

Makes me embarrassed to look at the rat's nest of cables under my desk right now. I can't even keep the 8 or 10 cables under there organized.

Curious though, what do the different colors mean? I assume that they're significant. Maybe internal vs external network?

May 9th, 2011 at 8:34am

Austin, I'm not sure what song he's playing at the end. He lip-syncs a few of the words, so if someone can figure it out, I'll send them a big package of SoftLayer swag.

Darren, the "2 Hours Later" clip was indeed from Spongebob ... It'd probably be best for all parties involved to not admit how quickly that clip came to mind. :-)

Ryan, the red cables carry public network traffic, the blue cables carry private network traffic and the green cables carry out-of-band management network traffic. A couple eagle-eyes on Twitter noticed some gray cables in the mix, and those are used in our cloud infrastructure to connect the cloud servers to the cloud storage.

May 10th, 2011 at 4:20pm

this is nice!
thanks a lot for the video!
I hope I'll see the contiuing of it: mounting servers and arranging other stuff.

May 10th, 2011 at 5:22pm

Every server in each rack has 4 NICs and a IPMI device in it? Or are you using 1U twin servers?

May 11th, 2011 at 10:05am

Each new server has a public connection, a redundant public connection, a private connection, a redundant private connection, and the IPMI connection.

May 12th, 2011 at 3:56pm

Mad props on a neat job.

However, if y'all anticipate having to replace any cabling, you might wanna think about velcro straps instead for the next job--so much easier to remove and replace than zip ties.

Otherwise, mad props.

May 16th, 2011 at 8:39am

@me, That's a good observation. In practices, zip ties are less expensive and tend to hold more securely/tightly. Because the cabling doesn't need to be changed very often, we don't need the ties to be easily removable.

January 9th, 2012 at 8:26am

how on earth do you keep up on top of the network management?

five 3750's to service 40 odd servers is massive.

January 20th, 2012 at 4:26am

Managing a network is really an art. Imagine dealing with all those cables and still making your data center neat like that. AMazing!

April 27th, 2012 at 2:49pm

Do you use cables with the same lenght?
what do you do with cables for servers near the switches? These will have a cables too long, you cut it?

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.