Posts Tagged 'Data Center'

June 14, 2012

My First Week as a SLayer in San Jose

As I write this post, I'm finishing my first week as an employee with SoftLayer. It might seem premature, but I think it's safe to say that it's the best job I've ever had. My friend Marcos gave me a great reference to get my foot in the door at SoftLayer in San Jose (SJC01) as a Server Build Technician (SBT), and I owe him a LOT for that help. Because first impressions are usually pretty significant, I thought I'd take a few minutes share my short experience with the company to provide a bit of perspective to anyone interested in "what it's really like" to work at SoftLayer.

To give you the best picture of what it's like to work at SoftLayer, I have to start with the other SLayers I've met. So far, my coworkers and supervisors have been easy to get along with, and they clearly know their stuff. SoftLayer's "Challenging, but not Overwhelming" motto isn't just for show ... I've got a long way to go to catch up with my peers when it comes to knowledge about the data center, but everyone around me has been so supportive that it doesn't feel too intimidating. The work environment is very casual, and while the tasks at hand are all serious, my coworkers are always telling jokes and fostering a friendly and welcoming work environment.

The second aspect of the job I should focus on is the day-to-day responsibilities I'm learning how to perform. In the data center, we're responsible for building and performing hardware maintenance on all of our customer servers, and a lot of our customer interaction is done via tickets. When a ticket is added to our data center queue, it's pretty wild to see an SBT claim it quickly and immediately spring into action. If a customer orders a new server in our facility, and that server configuration isn't readily available, we get notified, and we have to move quickly to make a hardware change so the server can get provisioned in under four hours. That's been my favorite part of the job so far.

I've always enjoyed putting computers together, so being able to do it on such a large scale (and having the chance to do it a few times per day) is a thrill for me. Even though I've built more than my share of computers in my lifetime, I still find myself learning a lot from the processes and procedures Softlayer has in place. It's pretty cool to see the inventory of high-power server hardware we have in our spare parts room, too.

Being new to a job usually involves a span of time where you feel like a "new guy," but that hasn't been the case at SoftLayer. The crew here at SJC01 has made me feel at home quickly, and they've been patient and helpful when I've had any questions. In fact, as I'm thinking about it, I can't say anything negative about my experience so far with Softlayer.

I'm excited about integrating into the team, and given how much my coworkers hang out during lunch, breaks and after work, I'm sure that'll happen quickly. I want to put on a big office potluck where I can bring down my barbecue grill and cook for them some afternoon ... And given SoftLayer's love of BBQ, I'd imagine that would be a big hit.

Man, all this talk of food is making me hungry.

-Jonathan

May 30, 2012

What Does Automation Look Like?

Innovation. Automation. Innovation. Automation. Innovation. Automation. That's been our heartbeat since SoftLayer was born on May 5, 2005. The "Innovation" piece is usually the most visible component of that heartbeat while "Automation" usually hangs out behind the scenes (enabling the "Innovation"). When we launch a new product line like Object Storage, add new functionality to the SoftLayer API, announce a partnership with a service provider like RightScale, or simply receive and rack the latest and greatest server hardware from our vendors, our automated platform allows us to do it quickly and seamlessly. Because our platform is built to do exactly what it's supposed to without any manual intervention, it's easily overlooked.

But what if we wanted to show what automation actually looks like?

It seems like a silly question to ask. If our automated platform is powered by software built by the SoftLayer development team, there's no easy way to show what that automation looks like ... At least not directly. While the bits and bytes aren't easily visible, the operational results of automation are exceptionally photogenic. Let's take a look at a few examples of what automation enables to get an indirect view of what it actually looks like.

Example: A New Server Order

A customer orders a dedicated server. That customer wants a specific hardware configuration with a specific suite of software in a specific data center, and it needs to be delivered within four hours. What does that usually look like from an operations perspective?

SoftLayer Server Rack

If you want to watch those blinking lights for two or three hours, you'll have effectively watched a new server get provisioned at SoftLayer. When an order comes in, the automated provisioning system will find a server matching the order's hardware requirements in the requested data center facility, and the software will be installed before it is handed over to the the customer.

Example: Server Reboot or Operating System Reload

A customer needs to reboot a server or install a new operating system. Whether they want a soft reboot, a hard reboot with a full power cycle or a blank operating system install, the scene in the data center will look eerily familiar:

SoftLayer Server Rack

Gone are the days of server build technicians wheeling a terminal over to every server that needs work done. From thousands of miles away, a customer can remotely "unplug" his or her server via the rack's power strip, initiate a soft reboot or reinstall an operating system. But what if they want even more accessibility?

Example: What's on the Screen?

When remotely rebooting or power cycling a server isn't enough, a customer might want someone in the data center to wheel over to their server in the rack to look at any of the messages that can only be read with a monitor attached. This would generally happen behind the server, but for the sake of this example, we'll just watch the data center technician pass in front of the servers to get to the back:

SoftLayer Server Rack

Yeah, you probably could have seen that one coming.

Because KVM over IP is included on every server, physical carts carrying "keyboard, video and mouse" are few and far between. By automating customers' access to their server and providing as much virtual access as we possibly can, we're able to "get out of the way" of our technical users and only step in to help when that help is needed.

I could go on and on with examples of cloud computing upgrades and downgrades, provisioning a firewall or adding a load balancers, but I'll practice a little restraint. If you want the full effect, you can scroll up and watch the blinking lights a little while longer.

Automation looks like what you don't see. No humanoid robots or needlessly complex machines (that I know of) ... Just a data center humming along with some beautiful flashing server lights.

-Duke

P.S. If you want to be able to remotely bask in the glow of some blinking server lights, bookmark the larger-sized SoftLayer Rack animated gif ... You could even title the bookmark, "Check on the Servers."

May 4, 2012

From "Computer Guy" to SoftLayer Server Build Technician

As I sat down to brainstorm ideas for this blog, I began to think about where I was when I started a few years ago and where I am now. When I was hired, I knew next to nothing about the inner-workings of data center IT. I was just your average computer nerd, or "the computer guy" as I became known around the house and to my friends. I had plenty of experience with hardware, but I had no clue just how deep the IT rabbit hole went ... I jumped in anyway.

Before I give you an example of one of the challenges I had to tackle early on, I should back up and explain a pretty important observation I had about SoftLayer: Despite how cheesy it may sound, SoftLayer is a family. If you are willing to learn and have a good work ethic, SoftLayer will to take you under its wing, and the sky is the limit. I was willing to learn, and I'd like to think I have a good work ethic, so I took on a pretty ambitious task: Learn Linux.

As a Server Build Technician — the physical "hands and eyes" in a data center — you can't get by without an intimate knowledge of Linux. As it turns out, trying to learn everything there is to know about Linux is sort of like saying "Get to the end of the Internet." Even after a few years of working with Linux, I still learn new things almost daily, and I'm sure that I'll continue to learn as long as I'm surrounded by Linux servers and other brilliant technicians who can share their Linux expertise. I could probably write a whole series of blog posts about all of the crazy things I've seen Linux servers do, but I'll focus on this "intro" blog first. Since starting with SoftLayer, my tenuous grasp of Linux was strengthened and eventually validated by my Linux+ certification!

That's only one little example of the kind of environment SoftLayer creates, and I could share dozens more.

When SLayers are treated like individuals rather than "employees," the culture is different. Managers and supervisors LISTEN to your problems/frustrations and are quick to offer their help and advice. I can feel comfortable to express personal issues with anyone in management, and I've had a handful of heartfelt talks with higher-ups that I would never dreamed of having at previous jobs. As a result, I'm excited when I walk into work because I feel like I get to hang out and work with friends for eight hours every day.

My coworkers and I can joke around one minute, and the next minute, we can have a serious and thoughtful conversation about how we could improve our processes or serve customers better better. Not only does that experience make for a comfortable working environment, it also creates a net of trust among coworkers. You know without a doubt that you can rely on your coworkers for anything.

I know it sounds like I'm stretching the truth (and the blog word count), but to be honest, there isn't enough room on this page to describe exactly how awesome I think the people at SoftLayer are. I've made many, many friends and roughly zero enemies. That's a pretty good ratio if you ask me. If you are even a TINY bit interested in IT, there's no better place to get your career started (or continued) than SoftLayer. There are positions for every skill set and level, and it doesn't stop there ... You aren't locked into one position or department if you find yourself more passionately drawn to another area of the business. SoftLayer encourages you to branch out and explore your career options, and if you want to move up, you're encouraged and supported by management to put forth the effort.

TL;DR If you're interested in getting into anything IT related, SoftLayer has a place for you, and as a very happy employee, I'd highly recommend taking advantage of that opportunity.

-Broc

April 16, 2012

TechStars Cloud @ SoftLayer - DAL05 Data Center Tour

Last week was HUGE for the inaugural class of companies in the TechStars Cloud accelerator in San Antonio. The program's three-month term concluded with "Demo Day" on Wednesday where all of the participating companies presented to more than 300 venture capitalists and investors, and given our relationship with TechStars, SoftLayer was well represented ... We were even honored to present a few of the companies we've been working with over the past few months. All of the 20-hour days, mentor sessions and elevator pitches culminated in one pitch, and while I can't talk much about the specifics, I can assure you that the event was a huge success when it came to connecting the teams to (very interested) investors.

Demo Day wasn't the end of the fun, though. After the post-pitch celebrations (and a much-needed night of sleep), the teams had one more item on their agenda for the week: A visit to SoftLayer.

On Thursday, the teams piled into a bus and made their way from San Antonio to Dallas where we could continue the celebration of their successful completion of the program ... And so many of the teams could see the actual hardware powering their businesses. After a nice little soiree on Thursday evening at the House of Blues in Dallas, we put the teams up in a hotel near our Alpha headquarters promised them an informative, interesting and fun Friday.

After a few hours of sleep, the teams were recharged on Friday morning and ready to experience some SoftLayer goodness so...

TechStars Cloud Data Center Tour

They loaded up the bus and took a 10-minute ride to our corporate headquarters.

TechStars Cloud Data Center Tour

Given our security and compliance processes, each visitor checked in at our front desk, and they were divided into smaller groups to take a quick data center tour.

I could tell that going on a data center tour wasn't the most exciting prospect for a few of the visitors, but I asked them to forget everything they thought they knew about data centers ... This is SoftLayer. Yes, that's pretty bold, but when each team walked out of SR01.DAL05, I could see in their eyes that they agreed.

TechStars Cloud Data Center Tour

The tour started innocently enough at a window looking into Server Room 01 (the first data center pod we built in DAL05). In the picture above, Joshua Daley, our DAL05 site manager, is explaining how all of SoftLayer's facilities are built identically to enable us to better manage the customer experience and our operational practices in any facility around the world. After a few notes about security and restrictions on what can/can't be done in the server room the group was led through the first set of secured doors between the facility's lobby and the data center floor.

TechStars Cloud Data Center Tour

From the next hallway, the tour group observed the generators and air conditioning units keeping DAL05 online 24x7. Josh explained the ways we safeguard the facility with n+1 redundancy and regular maintenance and load testing, and the group was led through two more stages of secured doors ... the first with badge access, the second requiring fingerprint authentication. When they made it through, they were officially in SR01.DAL05.

TechStars Cloud Data Center Tour

Josh explained how our data center CRAC units work, how each server row is powered and how we measure and optimize the server room environment. While that aspect of the data center could seem like "blocking and tackling," he talked about our continued quest to improve power efficiency as he shared a few of the innovative approaches we've been testing, and it was clear that the tour understood it to be easier than, "Plug in server. Turn on air conditioner."

TechStars Cloud Data Center Tour

The teams got a chance to get up close and personal (No Touching!) with a server rack, and they learned about our unique network-with-a-network topology that features public, private and out-of-band management functionality. Many "oohs" and "ahhhs" were expressed.

TechStars Cloud Data Center Tour

The tour wrapped up outside of the data center facility in front of the Alpha HQ's Network Operations Center. From here, the TechStars could see how our network team observes and responds to any network-related events, and they could ask questions about anything they saw during the tour (without having to shout over the air conditioning hum).

When the final tour concluded, the full group reconvened in one of our conference rooms. They'd seen the result of our hard work, and we wanted them to know where all that hard work started. Because SoftLayer was started in a Dallas living room a few short years ago, we knew our story would be interesting, inspirational and informative, and we wanted to provide as much guidance as possible to help these soon-to-grow businesses prepare for their own success. After a brief Q&A period, a few of the TechStars Cloud participants (and some of their Dallas-based Tech Wildcatters cousins) presented a little about their businesses and how they've grown and evolved through the TechStars program, and we got to ask our own questions to help them define their business moving forward.

After the presentations at the office, we knew we couldn't just load the bus up to send the teams back to San Antonio ... We had to bid them farewell SoftLayer style. We scheduled a quick detour to SpeedZone Dallas where a few hours of unlimited eats, drinks, games and go-kart races were waiting for them.

We couldn't have had a better time with the participating teams, and we're looking forward to seeing the amazing things they'll continue doing in the near future. If you want to see even more data center coverage from Friday, be sure to check out "TechStars Cloud Visits SoftLayer" on Flickr!

-@PaulFord

April 9, 2012

Scaling SoftLayer

SoftLayer is in the business of helping businesses scale. You need 1,000 cloud computing instances? We'll make sure our system can get them online in 10 minutes. You need to spin up some beefy dedicated servers loaded with dual 8-core Intel Xeon E5-2670 processors and high-capacity SSDs for a new application's I/O-intensive database? We'll get it online anywhere in the world in under four hours. Everywhere you look, you'll see examples of how we help our customers scale, but what you don't hear much about is how our operations team scales our infrastructure to ensure we can accommodate all of our customers' growth.

When we launch a new data center, there's usually a lot of fanfare. When AMS01 and SNG01 came online, we talked about the thousands of servers that are online and ready. We meet huge demand for servers on a daily basis, and that presents us with a challenge: What happens when the inventory of available servers starts dwindling?

Truck Day.

Truck Day not limited to a single day of the year (or even a single day in a given month) ... It's what we call any date our operations team sets for delivery and installation of new hardware. We communicate to all of our teams about the next Truck Day in each location so SLayers from every department can join the operations team in unboxing and preparing servers/racks for installation. The operations team gets more hands to speed up the unloading process, and every employee has an opportunity to get first-hand experience in how our data centers operate.

If you want a refresher course about what happens on a Truck Day, you can reference Sam Fleitman's "Truck Day Operations" blog, and if you want a peek into what it looks like, you can watch Truck Day at SR02.DAL05. I don't mean to make this post all about Truck Day, but Truck Day is instrumental in demonstrating the way SoftLayer scales our own infrastructure.

Let's say we install 1,000 servers to officially launch a new pod. Because each pod has slots for 5,000 servers, we have space/capacity for 3,000-4,000 more servers in the server room, so as soon as more server hardware becomes available, we'll order it and start preparing for our next Truck Day to supplement the pod's inventory. You'd be surprised how quickly 1,000 servers can be ordered, and because it's not very easy to overnight a pallet of servers, we have to take into account lead time and shipping speeds ... To accommodate our customers' growth, we have to stay one step ahead in our own growth.

This morning in a meeting, I saw a pretty phenomenal bullet that got me thinking about this topic:

Truck Day — 4/3 (All Sites): 2,673 Servers

In nine different data center facilities around the world, more than 2,500 servers were delivered, unboxed, racked and brought online. Last week. In one day.

Now I know the operations team wasn't looking for any kind of recognition ... They were just reporting that everything went as planned. Given the fact that an accomplishment like that is "just another day at SoftLayer" for those guys, they definitely deserve recognition for the amazing work they do. We host some of the most popular platforms, games and applications on the Internet, and the DC-Ops team plays a huge role in scaling SoftLayer so our customers can scale themselves.

-@gkdog

January 16, 2012

What I've Learned About Leading

What does it take to be a good leader? What kind of leader do you want to follow? Throughout my life, I've gravitated toward leadership positions. Even when I was young, I tended to take charge of a group to achieve a goal or accomplish a mission ... though most of the "missions" in my younger days happened to be some sort of mischief. Having participated in the Boy Scouts and JROTC, I joined the Marine Corps where I served for seven years, and throughout my life, I've been fortunate to have more than my fair share of incredible mentors.

When my service in the Marine Corps concluded, I "enlisted" at SoftLayer as a data center technician. My primary responsibilities included building severs to order and installing software for our customers in four hours or less, and it was all pretty foreign to me. I had a lot to learn about the technical side of operating a data center, but based on my impression of the company, I was confident that I'd be in good hands.

Because I always find myself asking for more challenges and additional responsibility, I transitioned into a Customer Systems Administrator role. The CSA position required a lot more learning (at a breakneck pace), and in addition to the technical aspect of the job, I found myself learning just as much about the 'soft skills' required to provide the great customer service. Equipped with that knowledge (and a bit more experience), I pursued a leadership role on the team, and I made it my goal to use what I'd learned in the data center and in support to lead my team. I'd be lying if I said it wasn't a challenge, but I've never backed down from one before.

I don't mean to make this post all about me ... my goal in sharing a little of my background is to give a little context for what I've learned about leadership. It goes without saying that I've been fortunate, both in the Marine Corps as well as with SoftLayer, to have some of the most intelligent, talented and sometimes downright enigmatic leaders. I've managed to pick up advice, tips and tricks for handling all of the curveballs that are thrown on a daily basis, and over my years as a leader, I've developed a few philosophies (an amalgam of some of the key points I absorbed from all of those who have led me in the past) that I try to abide by daily:

Lead by Example.

I make it a point to never ask someone to do something that I've never done or something that I'd never be willing to do. This is the keen avoidance of the "do as I say, not as I do" mentality. Failing to do this hands-down one of the easiest ways to lose your team.

Employ Your Team to Their Abilities. Empower Them to do Their Best Work.

It's very important to know your team's strengths and weaknesses and use those to make everyone better. Using the strengths of one team member, I can push another outside of his or her comfort zone to improve his or her overall skill set. I've also seen amazing results from providing freedom for my teams to make decisions. Not only does that freedom build trust, it also gives some real "ownership" to every person, and with that sense of ownership, each team member does better work. I've been a little surprised to notice this empowerment coming from the coffee juggernaut Starbucks: Starbucks baristas and store employees are encouraged to make real-time decisions in the interest of taking care of their customers.1

Keep Learning.

Leading a team requires that you stay on top of what they're doing. Always ask questions. Continue to research so you can be a resource for your team. Find opportunities to learn and take advantage of every one of them.

Most Importantly: Learn how to Have Fun.
I see this cliché often, so when you see, "Have Fun," I wouldn't be surprised if you just rolled your eyes. I'd be lying if I said that things were great all of the time ... Realistically in any business, there's going to be a time or two when the *&#@ is going to hit the fan. It's important to find opportunities to cut loose and relax a bit. Cooking steaks for the group during a long overnight shift, grabbing a bite after work or a providing a happy hour once in a while builds a great deal of cohesion outside the office walls. Providing a relaxed environment does not prevent your team from doing stressful work ... It actually builds camaraderie, and it will help the team get through those tough times. The brutal honesty is that if people aren't enjoying where they're at, they'll look elsewhere – leaders have to help foster an environment that enables success.

At the end of the day, these tips may not work for everyone. There are a plethora of studies out there pertaining to the different leadership styles, the different types of leaders and how they influence teams. What's important is that leaders need to be intentional about bettering their teams (and ultimately bettering their businesses).

Along the lines of continuous education, I'd love to hear the leadership philosophies you've learned in your experience as a leader. Leave a comment on this article to share what you think has made you successful.

-Matthew

1This comes from both observation and talking to current and past employees. I've never worked for Starbucks, so I can't cite a specific company policy to back this up, but that evident organic culture is probably worth more than a million company policies that would try to create that culture.

December 9, 2011

Earn Your Bars

In less than six years, SoftLayer has grown pretty drastically. We started as a small company with ten people crammed into a living room, brainstorming how to build one innovative data center in Dallas. Now we have more than six hundred employees managing thirteen data centers on three different continents. It's insane to see how far we've come when you read those two sentences, and as I think back, I remember the sacrifices employees have made to help our business get where it is today.

In the early days, we were taking out loans and tapping our bank accounts to buy servers. When customers started asking for more features and functionality in the portal, developers coded non-stop to make it happen. A lot of those sacrifices aren't very obvious from the outside, but we wouldn't be where we are today without them. One of the biggest sacrifices SLayers make is when we need to build new data centers to accommodate customer demand ... A "Go Live Crew" of employees moves away from their friends and family to those facilities to make sure the new SoftLayer data center meets our high expectations.

In the military, a soldier will "earn his/her stripes" by doing something that shows that he or she deserves a particular rank or position. The more stripes on the sleeve of your uniform, the higher your rank. As you've probably gathered from pictures and videos around the office, SoftLayer employees don't wear uniforms, but SLayers love to wear SoftLayer swag, and this "mechanic" shirt has been one of the most popular sellers in our company store:

Earn Your Bars Shirts

We wanted to recognize the employees that have given weeks (and sometimes months) of their time to join a Go Live Crew for a data center build-out, so we took that popular shirt and added a little flair. Following the "earn your stripes" idea, these employees have "earned their bars" for each data center they help build.

Earn Your Bars Shirts

Every employee who was on a Go Live Crew in Seattle, Washington, D.C., San Jose, Singapore or Amsterdam will get shirts with location-specific graphics to recognize their contribution, and their most recent shirt will have the "bars" you see in the picture above.

As a bit of added recognition, here are the shirt recipients for each data center location:

Earn Your Bars Shirts
Seattle Go Live Crew
John E., Edmund G., Robert G., Joe H., Brad L., Charles P., Joshua R., William S., Zane W.
Earn Your Bars Shirts
Washington, D.C. Go Live Crew
Troy D., John E., Reed F., Edmund G., Robert G., Brad L., Charles P., Joshua R., Zane W.
Earn Your Bars Shirts
San Jose Go Live Crew
Kalin D., John E., Chris F., Hector F., Edmund G., Robert G., Tim L., Russ M., Edward R., Brent R., Brandon S., Joshua Z.
Earn Your Bars Shirts
Singapore Go Live Crew
Chris F., Joshua F.. Ryan G., Robert G., Hao H., Tim L., Russ M., Todd M., Kyle S., Eric V.
Earn Your Bars Shirts
Amsterdam Go Live Crew
Raul A., Brian C., Elijah F., Hector F., Edmund G., Robert G., Sydney M., Stephen M., Michael P., Goran P., Mark Q., Edward R., Jason R., Brandon S., Sopheara S., Joshua Z.

And if you happened to compare the names between all five teams, you'll notice that Robert Guerra was on every crew. You know what that means?

Earn Your Bars Shirts

He has a brand new wardrobe.

CBNO.

-@lavosby

December 2, 2011

Global Network: The Proof is in the Traceroute

You've probably heard a lot about SoftLayer's global expansion into Asia and Europe, and while the idea of geographically diversifying is impressive in itself, one of the most significant implications of our international expansion is what it's done for the SoftLayer Network.

As George explained in "Globalization and Hosting: The World Wide Web is Flat," our strategic objective is to get a network point of presence within 40ms of all of our users and our users' users to provide the best network stability and performance possible anywhere on the planet. The reasoning is simple: The sooner a user gets on on our network, the quicker we can efficiently route them through our points of presence to a server in one of our data centers.

The cynics in the audience are probably yawning and shrugging that idea off as marketing mumbo jumbo, so I thought it would be good to demonstrate how the network expansion immediately and measurably improved our customers' network experience from Asia to the United States. Just look at the traceroutes.

As you're probably aware, a traceroute shows the "hops" or routers along the network path from an origin IP to a destination IP. When we were building out the Singapore data center (before the network points of presence were turned up in Asia), I ran a traceroute from Singapore to SoftLayer.com, and immediately after the launch of the data center, I ran another one:

Pre-Launch Traceroute to SoftLayer.com from Singapore

traceroute to softlayer.com (66.228.118.53), 64 hops max, 52 byte packets
 1  10.151.60.1 (10.151.60.1)  1.884 ms  1.089 ms  1.569 ms
 2  10.151.50.11 (10.151.50.11)  2.006 ms  1.669 ms  1.753 ms
 3  119.75.13.65 (119.75.13.65)  3.380 ms  3.388 ms  4.344 ms
 4  58.185.229.69 (58.185.229.69)  3.684 ms  3.348 ms  3.919 ms
 5  165.21.255.37 (165.21.255.37)  9.002 ms  3.516 ms  4.228 ms
 6  165.21.12.4 (165.21.12.4)  3.716 ms  3.965 ms  5.663 ms
 7  203.208.190.21 (203.208.190.21)  4.442 ms  4.117 ms  4.967 ms
 8  203.208.153.241 (203.208.153.241)  6.807 ms  55.288 ms  56.211 ms
 9  so-2-0-3-0.laxow-cr1.ix.singtel.com (203.208.149.238)  187.953 ms  188.447 ms  187.809 ms
10  ge-4-0-0-0.laxow-dr2.ix.singtel.com (203.208.149.34)  184.143 ms
    ge-4-1-1-0.sngc3-dr1.ix.singtel.com (203.208.149.138)  189.510 ms
    ge-4-0-0-0.laxow-dr2.ix.singtel.com (203.208.149.34)  289.039 ms
11  203.208.171.98 (203.208.171.98)  187.645 ms  188.700 ms  187.912 ms
12  te1-6.bbr01.cs01.lax01.networklayer.com (66.109.11.42)  186.482 ms  188.265 ms  187.021 ms
13  ae7.bbr01.cs01.lax01.networklayer.com (173.192.18.166)  188.569 ms  191.100 ms  188.736 ms
14  po5.bbr01.eq01.dal01.networklayer.com (173.192.18.140)  381.645 ms  410.052 ms  420.311 ms
15  ae0.dar01.sr01.dal01.networklayer.com (173.192.18.211)  415.379 ms  415.902 ms  418.339 ms
16  po1.slr01.sr01.dal01.networklayer.com (66.228.118.138)  417.426 ms  417.301 ms
    po2.slr01.sr01.dal01.networklayer.com (66.228.118.142)  416.692 ms
17  * * *

Post-Launch Traceroute to SoftLayer.com from Singapore

traceroute to softlayer.com (66.228.118.53), 64 hops max, 52 byte packets
 1  192.168.206.1 (192.168.206.1)  2.850 ms  1.409 ms  1.206 ms
 2  174.133.118.65-static.reverse.networklayer.com (174.133.118.65)  1.550 ms  1.680 ms  1.394 ms
 3  ae4.dar01.sr03.sng01.networklayer.com (174.133.118.136)  1.812 ms  1.341 ms  1.734 ms
 4  ae9.bbr01.eq01.sng02.networklayer.com (50.97.18.198)  35.550 ms  1.999 ms  2.124 ms
 5  50.97.18.169-static.reverse.softlayer.com (50.97.18.169)  174.726 ms  175.484 ms  175.491 ms
 6  po5.bbr01.eq01.dal01.networklayer.com (173.192.18.140)  203.821 ms  203.749 ms  205.803 ms
 7  ae0.dar01.sr01.dal01.networklayer.com (173.192.18.253)  306.755 ms
    ae0.dar01.sr01.dal01.networklayer.com (173.192.18.211)  208.669 ms  203.127 ms
 8  po1.slr01.sr01.dal01.networklayer.com (66.228.118.138)  203.518 ms
    po2.slr01.sr01.dal01.networklayer.com (66.228.118.142)  305.534 ms
    po1.slr01.sr01.dal01.networklayer.com (66.228.118.138)  204.150 ms
 9  * * *

I won't dive too deep into what these traceroutes are telling us because that'll need to be an entirely different blog. What I want to draw your attention to are a few key differences between the pre- and post-launch traceroutes:

  • Getting onto SoftLayer's network:. The first reference to "networklayer" in the pre-launch trace is in hop 12 (~187ms). In the post-launch trace, we were on "networklayer" in the second hop (~1.5ms).
  • Number of hops: Pre-launch, our network path took 16 hops to get to SoftLayer.com. Post-launch, it took 8.
  • Response times from the destination: The average response time from SoftLayer.com to Singapore before the launch of our network points of presence in Asia was about 417ms (milliseconds). After the launch, it dropped to an average of about ~250ms.

These traceroutes demonstrate that users in Singapore travel a much better network path to a server in one of our U.S. data centers than they had before we turned up the network in Asia, and that experience isn't limited to users in Singapore ... users throughout Europe and Asia will see fewer hops and better speeds now that the data centers and points of presence on those continents are live. And that's without buying a server in either of those markets or making any changes to how they interact with us.

Managing a worldwide network for a worldwide customer base with thousands of different ISPs and millions of possible routes is not a "set it and forget it" endeavor, so we have a team of engineers in our Network Operations Center that focuses on tweaking and optimizing routes 24x7. Branching out into Europe and Asia introduces a slew of challenges when working with providers on the other side of the globe, but I guess it's true: "If it were easy, everyone would do it."

Innovate or die.

-@toddmitchell

November 18, 2011

Four Years of SLaying in Seattle

How are we already in mid-November? Did 2011 just fly by us or what? As we approach 2012, I will be celebrating my fourth anniversary with SoftLayer in our Seattle data center. Seattle was SoftLayer's first data center outside of the Dallas area when it opened four years ago, and since then, I've seen the launch of Washington D.C., the Dallas HQ + DAL05, San Jose, Singapore and Amsterdam ... while adding a few data centers in Houston and Dallas after the merger with The Planet last year. We've gone from ~15,000 servers when I started to around 100,000 servers in 13 data centers with 16 network PoPs on three different continents around the world. It's safe to say we've grown.

In the four years since our Seattle facility launched, over 60% of our original team – the folks our Dallas team trained – are still here. Being part of such a huge team and watching the SoftLayer roll out data centers around the world is exciting, and seeing our customers grow with us is even better. In the midst of all of that growth, our team is always trying to figure out new technologies and techniques to share with customers to help them meet their ever-evolving needs. The goal: Give our customers total control.

One great example of this focus was our recent launch of QuantaStor Storage Servers. We teamed up with industry leader OS Nexus to bring our customers a production-ready mass storage appliance with a combined SAN and NAS storage system built into the Ubuntu Server and provides a number of system features such as snapshots, compression, remote replication and thin provisioning. A customer could use this in a number of environments from virtualized systems to video production to web and application servers, or as a backup based server. If you're looking for a mass storage system, I highly recommend it.

If we've grown this much in my first four years, I can only imagine what the business will look like four years from now. A SoftLayer data center on every corner? Maybe we can get PHIL to figure out how we can put a SoftLayer pod in the space normally occupied by a coffee shop ... making sure to keep as much coffee as possible, obviously.

-Bill

November 8, 2011

PHIL's DC: SoftLayer Data Center Tour

When I was chosen by Lance to manage a "special project," I knew I had my work cut out for me. My mission is to redefine how data centers are built, so before I get too far in the creation of my data center facility, I thought it would be a good idea to get a quick refresher about how SoftLayer builds and runs its data centers:

You can disregard the references to Lance "requiring" me to go on the tour ... and the formality of Jon having to sign a note that said I successfully completed the tour. The references to those were just for dramatic effect since Lance and I are pretty much eye-to-eye about everything that needs to happen for PHIL's DC to succeed, and we both thought it would be good to have signed documentation that I went on a tour.

As I mentioned at the end of the tour, I didn't find the tour to be a complete waste of time because I was able to observe some of the biggest hurdles in building and maintaining a data center. Is redundancy really necessary ... or is it just redundant? What the heck did Jon say in the UPS room? How fast does a person have to run on a single treadmill for that treadmill to power 40,000 servers?

While I try to source my own "PHIL's DC" rack identifiers, my adoring public can take time to go through this video with a fine-toothed comb to suggest any potential data center innovations that you think I may have overlooked. I already have the most innovative and efficient data center designed in my head, but I'll consider crediting you if you share an idea ... even if (and by if, I mean "when") I've already thought of it.

I've got a hot lead on some slightly used hardware, so the next time you see me, the PHIL's DC inventory area should be fully stocked and ready for my own truck day.

-PHIL

Subscribe to data-center