Posts Tagged 'Data Center'

July 19, 2012

The Human Element of SoftLayer - DAL05 DC Operations

One of the founding principles of SoftLayer is automation. Automation has enabled this company to provide our customers with a world class experience, and it enables employees to provide excellent service. It allows us to quickly deploy a variety of solutions at the click of a button, and it guarantees consistency in the products that we deliver. Automation isn't the whole story, though. The human element plays a huge role in SoftLayer's success.

As a Site Manager for the corporate facility, I thought I could share a unique perspective when it comes to what that human element looks like, specifically through the lens of the Server Build Team's responsibilities. You recently heard how my colleague, Broc Chalker, became an SBT, and so I wanted take it a step further by providing a high-level breakdown of how the Server Build Team enables SoftLayer to keep up with the operational demands of a rapidly growing, global infrastructure provider.

The Server Build Team is responsible for filling all of the beautiful data center environments you see in pictures and videos of SoftLayer facilities. Every day, they are in the DC, building out new rows for inventory. It sounds pretty simple, but it's actually a pretty involved process. When it comes to prepping new rows, our primary focus is redundancy (for power, cooling and network). Each rack is powered by dual power sources, four switches in a stacked configuration (two public network, two private network), and an additional switch that provides KVM access to the server. To make it possible to fill the rack with servers, we also have to make sure it's organized well, and that takes a lot of time. Just watch the video of the Go Live Crew cabling a server rack in SJC01, and you can see how time- and labor-intensive the process is. And if there are any mistakes or if the cables don't look clean, we'll cut all the ties and start over again.



 

In addition to preparing servers for new orders, SBTs also handle hardware-related requests. This can involve anything from changing out components for a build, performing upgrades / maintenance on active servers, or even troubleshooting servers. Any one of these requests has to be treated with significant urgency and detail.



 

The responsibilities do not end there. Server Build Technicians also perform a walk of the facility twice per shift. During this walk, technicians check for visual alerts on the servers and do a general facility check of all SoftLayer pods. Note: Each data center facility features one or more pods or "server rooms," each built to the same specifications to support up to 5,000 servers.



 

The DAL05 facility has a total of four pods, and at the end of the build-out, we should be running 18,000-20,000 servers in this facility alone. Over the past year, we completed the build out of SR02 and SR03 (pod 2 and 3, respectively), and we're finishing the final pod (SR04) right now. We've spent countless hours building servers and monitoring operating system provisions when new orders roll in, and as our server count increases, our team has grown to continue providing the support our existing customers expect and deserve when it comes to upgrade requests and hardware-related support tickets.



 

To be successful, we have to stay ahead of the game from an operations perspective. The DAL05 crew is working hard to build out this facility's last pod (SR04), but for the sake of this blog post, I pulled everyone together for a quick photo op to introduce you to the team.

DAL05 Day / Evening Team and SBT Interns (with the remaining racks to build out in DAL05):
DAL05 DC Ops

DAL05 Overnight Server Build Technician Team:
DAL05 DC Ops

Let us know if there's ever anything we can do to help you!

-Joshua

July 13, 2012

When Opportunity Knocks

I've been working in the web hosting industry for nearly five years now, and as is the case with many of the professionals of my generation, I grew up side by side with the capital-I Internet. Over those five years, the World Wide Web has evolved significantly, and it's become a need. People need the Internet to communicate, store information, enable societal connectivity and entertain. And they need it 24 hours per day, seven days a week. To affirm that observation, you just need to look at an excerpt from a motion submitted to the Human Rights Council and recently passed by the United Nations General Assembly:

The General Session ... calls upon all States to promote and facilitate access to the Internet and international cooperation aimed at the development of media and information and communications facilities in all countries.

After a platform like the Internet revolutionizes the way we see the world, it's culturally impossible to move backward. Its success actually inspires us to look forward for the next world-changing innovation. Even the most non-technical citizen of the Internet has come to expect those kinds of innovations as the Internet and its underlying architecture have matured and seem to be growing like Moore's Law: Getting faster, better, and bigger all the time. The fact that SoftLayer is able to keep up with that growth (and even continue innovating in the process) is one of the things I admire most about the company.

I love that our very business model relies on our ability to enable our customers' success. Just look at how unbelievably successful companies like Tumblr and HostGator have become, and you start to grasp how big of a deal it is that we can help their businesses. We're talking billions of pageviews per month and hundreds of thousands of businesses that rely on SoftLayer through our customers. And that's just through two customers. Because we're on the cutting edge, and we provide unparalleled access and functionality, we get to see a lot of the up-and-coming kickstarts that are soon to hit it big, and we get to help them keep up with their own success.

On a personal level, I love that SoftLayer provides opportunities for employees. Almost every department has a career track you can follow as you learn more about the business and get a little more experience, and you're even able to transition into another department if you're drawn to a new passion. I recently move to the misty northwest (Seattle) when given the opportunity by SoftLayer, and after working in the data center, I decided to pursue a role as a systems administrator. It took a lot of hard work, but I made the move. Hard work is recognized, and every opportunity I've taken advantage of has been fulfilled. You probably think I'm biased because I've done well in the organization, and that might be a fair observation, but in reality, the opportunities don't just end with me.

One of my favorite stories to share about SoftLayer is the career path of my best friend, Goran. I knew he was a hard worker, so I referred him to the company a few years ago, and he immediately excelled as an Operations Tech. He proved himself on the Go-Live Crew in Amsterdam by playing a big role in the construction of AMS01, and he was promoted to a management position in that facility. He had been missing Europe for the better part of a decade, SoftLayer gave him a way to go back home while doing what he loves (and what he's good at).

If that Goran's story isn't enough for you, I could tell you about Robert. He started at SoftLayer as a data center tech, and he worked hard to become a systems administrator, then he was named a site manager, then he was promoted to senior operations manager, and now he's the Director of Operations. You'll recognize him as the guy with all of the shirts in Lance's "Earn Your Bars" blog post from December. He took every rung on the ladder hand-over-hand because no challenge could overwhelm him. He sought out what needed to be done without being asked, and he was proactive about make SoftLayer even better.

I could tell you about dozens of others in the company that have the same kinds of success stories because they approached the opportunities SoftLayer provided them with a passion and positive attitude that can't be faked. If being successful in an organization makes you biased, we're all biased. We love this environment. We're presented with opportunities and surrounded by people encouraging us to take advantage of those opportunities, and as a result, we can challenge ourselves and reach our potential. No good idea is ignored, and no hard work goes unrecognized.

I'm struggling to suppress the countless "opportunity" stories I've seen in my tenure at SoftLayer, but I think the three stories above provide a great cross-section of what it looks like to work for SoftLayer. If you like being challenged (and being rewarded for your hard work), you might want to take this opportunity to see which SoftLayer Career could be waiting for you.

When opportunity knocks, let it in.

-Hilary

Categories: 
July 12, 2012

An Insider's Look at SoftLayer's Growth in Amsterdam

Last week, SoftLayer was featured on the NOS national news here in the Netherlands in a segment that allowed us to tell our story and share how we're settling into our new Amsterdam home. I've only been a SLayer for about nine months now, and as I watched the video, I started to reflect on how far we've come in such a surprisingly short time. Take a second to check it out (don't worry, it's not all in Dutch):

To say that I had to "hit the ground running" when I started at SoftLayer would be an understatement. The day after I got the job, I was on a plane to SoftLayer's Dallas headquarters to meet the team behind the company. To be honest, it was a pretty daunting task, but I was energized at the opportunity to learn about how SoftLayer became largest privately owned hosting company in the world from the people who started it. When I look back at the interview Kevin recorded with me, I'm surprised that I didn't look like a deer in the headlights. At the time, AMS01 was still in the build-out phase, so my tours and meetings in DAL05 were both informative and awe-inspiring.

When I returned to Europe, I was energized to start playing my role in the company's new pursuit of its global goals.

It didn't take long before I started seeing the same awe-inspiring environment take place in our Amsterdam facility ... So much so that I'm convinced that at least a few of the "Go Live Crew" members were superhuman. As it turns out, when you build identical data center pods in every location around the world, you optimize the process and figure out the best ways to efficiently use your time.

By the time the Go Live Crew started packing following the successful (and on-time) launch of AMS01, I started feeling the pressure. The first rows of server racks were already being filled by customers, but the massive data center space seemed impossibly large when I started thinking of how quickly we could fill it. Most of my contacts in Europe were not familiar with the SoftLayer name, and because my assigned region was Europe Middle East and Africa — a HUGE diverse region with many languages, cultures and currencies — I knew I had my work cut out for me.

I thought, "LET'S DO THIS!"

EMEA is home to some of the biggest hosting markets in the world, so my first-week whirlwind tour of Dallas actually set the stage quite nicely for what I'd be doing in the following months: Racking up air miles, jumping onto trains, attending countless trade shows, meeting with press, reaching out to developer communities and corresponding with my fellow SLayers in the US and Asia ... All while managing the day-to-day operations of the Amsterdam office. As I look back at that list, I'm amazed how the team came together to make sure everything got done.

We have come a long way.

As I started writing this blog, BusinessReview Europe published a fantastic piece on SoftLayer in their July 2012 magazine (starting on page 172) that seems to succinctly summarize how we've gotten where we are today: "Innovation Never Sleeps."

BusinessReview Europe

Our first pod is almost full of servers humming and flashing. When we go to tradeshows and conferences throughout Europe, people not only know SoftLayer, many of them are customers with servers in AMS01. That's the kind of change we love.

The best part of my job right now is that our phenomenal success in the past nine months is just a glimmer of what the future holds. Come to think of it, we're going to need some more people.

-@jpwisler

June 19, 2012

Proud to be a SLayer

Changing a career can be a challenge, especially when it feels like you are starting from scratch. I know that feeling well. I'd always been interested in networking, IT and cloud computing, but it wasn't until I joined SoftLayer that had an opportunity to start building a career on top of those interests. I know you might've already read a few introductions and SoftLayer culture posts in the past, but I wanted to share my experience in joining the hardware tech team to give my own unique perspective on what it was like becoming a SLayer.

Like Jonathan, I joined SoftLayer in San Jose (SJC01), and despite my interest in the technology SoftLayer manages for customers on a day-to-day basis, I didn't have many of the skills I'd need in the data center. That's where the training program came into play ... I can't tell you how valuable it was to learn how SoftLayer approaches cloud and data center operations. My previous jobs were in manufacturing, so I was accustomed to working with hardware and machines, so after a bit of a learning curve, I started to feel comfortable with the instruction and hands-on challenges that were put in front of me during the training program.

Once I was able to start applying what I learned in training, I started feeling "at home" when I got to the data center. I'm one of the many people responsible for supporting data center operations, and while I'm more of a "hands on" person, I don't forget the "big picture" of the significance of that responsibility. SoftLayer servers are the lifeblood of businesses around the world, and I owe it to those customers to provide the best service I can when it comes to managing their hardware. If that starts feeling daunting, I can look to my peers and ask questions about any problem, and I know I'll get a quick, helpful answer. I know SoftLayer is built on innovation and automation, but the unstated "education" piece is what has appealed to me the most as an employee.

One of my favorite resources to consult on a daily basis is the SoftLayer wiki — SLiki. If I ever forget any technical specifications or get confused about how to configure a specific type of hardware, I fire up my browser and hit the SLiki. If I'm not sure how to troubleshoot a given transaction or want to learn a little more about a topic like cloud computing or object storage, I can search the SLiki and get the answer in no time.

When friends and family have asked me what it's like to work at SoftLayer, I tell them that I'm constantly amazed and impressed impressed by my coworkers. It's hard to explain in a way that doesn't sound corny, but everyone I work with seems to enjoy supporting customers, interacting with other SLayers and making the SJC01 data center run like a top.

Pretty recently, I had my first Truck Day, and it made me love working for SoftLayer even more. It was pretty awe-inspiring to see SLayers from every department in our office joining the SBTs at the loading dock to unpack, sort and rack a huge shipment of SuperMicro servers. Everyone was sweaty, and I'm sure a few people were pretty sore the next day, but after all was said and done, we all felt like we'd accomplished something significant for our customers.

I'm proud to be a SLayer.

-Cuong

June 14, 2012

My First Week as a SLayer in San Jose

As I write this post, I'm finishing my first week as an employee with SoftLayer. It might seem premature, but I think it's safe to say that it's the best job I've ever had. My friend Marcos gave me a great reference to get my foot in the door at SoftLayer in San Jose (SJC01) as a Server Build Technician (SBT), and I owe him a LOT for that help. Because first impressions are usually pretty significant, I thought I'd take a few minutes share my short experience with the company to provide a bit of perspective to anyone interested in "what it's really like" to work at SoftLayer.

To give you the best picture of what it's like to work at SoftLayer, I have to start with the other SLayers I've met. So far, my coworkers and supervisors have been easy to get along with, and they clearly know their stuff. SoftLayer's "Challenging, but not Overwhelming" motto isn't just for show ... I've got a long way to go to catch up with my peers when it comes to knowledge about the data center, but everyone around me has been so supportive that it doesn't feel too intimidating. The work environment is very casual, and while the tasks at hand are all serious, my coworkers are always telling jokes and fostering a friendly and welcoming work environment.

The second aspect of the job I should focus on is the day-to-day responsibilities I'm learning how to perform. In the data center, we're responsible for building and performing hardware maintenance on all of our customer servers, and a lot of our customer interaction is done via tickets. When a ticket is added to our data center queue, it's pretty wild to see an SBT claim it quickly and immediately spring into action. If a customer orders a new server in our facility, and that server configuration isn't readily available, we get notified, and we have to move quickly to make a hardware change so the server can get provisioned in under four hours. That's been my favorite part of the job so far.

I've always enjoyed putting computers together, so being able to do it on such a large scale (and having the chance to do it a few times per day) is a thrill for me. Even though I've built more than my share of computers in my lifetime, I still find myself learning a lot from the processes and procedures Softlayer has in place. It's pretty cool to see the inventory of high-power server hardware we have in our spare parts room, too.

Being new to a job usually involves a span of time where you feel like a "new guy," but that hasn't been the case at SoftLayer. The crew here at SJC01 has made me feel at home quickly, and they've been patient and helpful when I've had any questions. In fact, as I'm thinking about it, I can't say anything negative about my experience so far with Softlayer.

I'm excited about integrating into the team, and given how much my coworkers hang out during lunch, breaks and after work, I'm sure that'll happen quickly. I want to put on a big office potluck where I can bring down my barbecue grill and cook for them some afternoon ... And given SoftLayer's love of BBQ, I'd imagine that would be a big hit.

Man, all this talk of food is making me hungry.

-Jonathan

May 30, 2012

What Does Automation Look Like?

Innovation. Automation. Innovation. Automation. Innovation. Automation. That's been our heartbeat since SoftLayer was born on May 5, 2005. The "Innovation" piece is usually the most visible component of that heartbeat while "Automation" usually hangs out behind the scenes (enabling the "Innovation"). When we launch a new product line like Object Storage, add new functionality to the SoftLayer API, announce a partnership with a service provider like RightScale, or simply receive and rack the latest and greatest server hardware from our vendors, our automated platform allows us to do it quickly and seamlessly. Because our platform is built to do exactly what it's supposed to without any manual intervention, it's easily overlooked.

But what if we wanted to show what automation actually looks like?

It seems like a silly question to ask. If our automated platform is powered by software built by the SoftLayer development team, there's no easy way to show what that automation looks like ... At least not directly. While the bits and bytes aren't easily visible, the operational results of automation are exceptionally photogenic. Let's take a look at a few examples of what automation enables to get an indirect view of what it actually looks like.

Example: A New Server Order

A customer orders a dedicated server. That customer wants a specific hardware configuration with a specific suite of software in a specific data center, and it needs to be delivered within four hours. What does that usually look like from an operations perspective?

SoftLayer Server Rack

If you want to watch those blinking lights for two or three hours, you'll have effectively watched a new server get provisioned at SoftLayer. When an order comes in, the automated provisioning system will find a server matching the order's hardware requirements in the requested data center facility, and the software will be installed before it is handed over to the the customer.

Example: Server Reboot or Operating System Reload

A customer needs to reboot a server or install a new operating system. Whether they want a soft reboot, a hard reboot with a full power cycle or a blank operating system install, the scene in the data center will look eerily familiar:

SoftLayer Server Rack

Gone are the days of server build technicians wheeling a terminal over to every server that needs work done. From thousands of miles away, a customer can remotely "unplug" his or her server via the rack's power strip, initiate a soft reboot or reinstall an operating system. But what if they want even more accessibility?

Example: What's on the Screen?

When remotely rebooting or power cycling a server isn't enough, a customer might want someone in the data center to wheel over to their server in the rack to look at any of the messages that can only be read with a monitor attached. This would generally happen behind the server, but for the sake of this example, we'll just watch the data center technician pass in front of the servers to get to the back:

SoftLayer Server Rack

Yeah, you probably could have seen that one coming.

Because KVM over IP is included on every server, physical carts carrying "keyboard, video and mouse" are few and far between. By automating customers' access to their server and providing as much virtual access as we possibly can, we're able to "get out of the way" of our technical users and only step in to help when that help is needed.

I could go on and on with examples of cloud computing upgrades and downgrades, provisioning a firewall or adding a load balancers, but I'll practice a little restraint. If you want the full effect, you can scroll up and watch the blinking lights a little while longer.

Automation looks like what you don't see. No humanoid robots or needlessly complex machines (that I know of) ... Just a data center humming along with some beautiful flashing server lights.

-Duke

P.S. If you want to be able to remotely bask in the glow of some blinking server lights, bookmark the larger-sized SoftLayer Rack animated gif ... You could even title the bookmark, "Check on the Servers."

May 4, 2012

From "Computer Guy" to SoftLayer Server Build Technician

As I sat down to brainstorm ideas for this blog, I began to think about where I was when I started a few years ago and where I am now. When I was hired, I knew next to nothing about the inner-workings of data center IT. I was just your average computer nerd, or "the computer guy" as I became known around the house and to my friends. I had plenty of experience with hardware, but I had no clue just how deep the IT rabbit hole went ... I jumped in anyway.

Before I give you an example of one of the challenges I had to tackle early on, I should back up and explain a pretty important observation I had about SoftLayer: Despite how cheesy it may sound, SoftLayer is a family. If you are willing to learn and have a good work ethic, SoftLayer will to take you under its wing, and the sky is the limit. I was willing to learn, and I'd like to think I have a good work ethic, so I took on a pretty ambitious task: Learn Linux.

As a Server Build Technician — the physical "hands and eyes" in a data center — you can't get by without an intimate knowledge of Linux. As it turns out, trying to learn everything there is to know about Linux is sort of like saying "Get to the end of the Internet." Even after a few years of working with Linux, I still learn new things almost daily, and I'm sure that I'll continue to learn as long as I'm surrounded by Linux servers and other brilliant technicians who can share their Linux expertise. I could probably write a whole series of blog posts about all of the crazy things I've seen Linux servers do, but I'll focus on this "intro" blog first. Since starting with SoftLayer, my tenuous grasp of Linux was strengthened and eventually validated by my Linux+ certification!

That's only one little example of the kind of environment SoftLayer creates, and I could share dozens more.

When SLayers are treated like individuals rather than "employees," the culture is different. Managers and supervisors LISTEN to your problems/frustrations and are quick to offer their help and advice. I can feel comfortable to express personal issues with anyone in management, and I've had a handful of heartfelt talks with higher-ups that I would never dreamed of having at previous jobs. As a result, I'm excited when I walk into work because I feel like I get to hang out and work with friends for eight hours every day.

My coworkers and I can joke around one minute, and the next minute, we can have a serious and thoughtful conversation about how we could improve our processes or serve customers better better. Not only does that experience make for a comfortable working environment, it also creates a net of trust among coworkers. You know without a doubt that you can rely on your coworkers for anything.

I know it sounds like I'm stretching the truth (and the blog word count), but to be honest, there isn't enough room on this page to describe exactly how awesome I think the people at SoftLayer are. I've made many, many friends and roughly zero enemies. That's a pretty good ratio if you ask me. If you are even a TINY bit interested in IT, there's no better place to get your career started (or continued) than SoftLayer. There are positions for every skill set and level, and it doesn't stop there ... You aren't locked into one position or department if you find yourself more passionately drawn to another area of the business. SoftLayer encourages you to branch out and explore your career options, and if you want to move up, you're encouraged and supported by management to put forth the effort.

TL;DR If you're interested in getting into anything IT related, SoftLayer has a place for you, and as a very happy employee, I'd highly recommend taking advantage of that opportunity.

-Broc

April 16, 2012

TechStars Cloud @ SoftLayer - DAL05 Data Center Tour

Last week was HUGE for the inaugural class of companies in the TechStars Cloud accelerator in San Antonio. The program's three-month term concluded with "Demo Day" on Wednesday where all of the participating companies presented to more than 300 venture capitalists and investors, and given our relationship with TechStars, SoftLayer was well represented ... We were even honored to present a few of the companies we've been working with over the past few months. All of the 20-hour days, mentor sessions and elevator pitches culminated in one pitch, and while I can't talk much about the specifics, I can assure you that the event was a huge success when it came to connecting the teams to (very interested) investors.

Demo Day wasn't the end of the fun, though. After the post-pitch celebrations (and a much-needed night of sleep), the teams had one more item on their agenda for the week: A visit to SoftLayer.

On Thursday, the teams piled into a bus and made their way from San Antonio to Dallas where we could continue the celebration of their successful completion of the program ... And so many of the teams could see the actual hardware powering their businesses. After a nice little soiree on Thursday evening at the House of Blues in Dallas, we put the teams up in a hotel near our Alpha headquarters promised them an informative, interesting and fun Friday.

After a few hours of sleep, the teams were recharged on Friday morning and ready to experience some SoftLayer goodness so...

TechStars Cloud Data Center Tour

They loaded up the bus and took a 10-minute ride to our corporate headquarters.

TechStars Cloud Data Center Tour

Given our security and compliance processes, each visitor checked in at our front desk, and they were divided into smaller groups to take a quick data center tour.

I could tell that going on a data center tour wasn't the most exciting prospect for a few of the visitors, but I asked them to forget everything they thought they knew about data centers ... This is SoftLayer. Yes, that's pretty bold, but when each team walked out of SR01.DAL05, I could see in their eyes that they agreed.

TechStars Cloud Data Center Tour

The tour started innocently enough at a window looking into Server Room 01 (the first data center pod we built in DAL05). In the picture above, Joshua Daley, our DAL05 site manager, is explaining how all of SoftLayer's facilities are built identically to enable us to better manage the customer experience and our operational practices in any facility around the world. After a few notes about security and restrictions on what can/can't be done in the server room the group was led through the first set of secured doors between the facility's lobby and the data center floor.

TechStars Cloud Data Center Tour

From the next hallway, the tour group observed the generators and air conditioning units keeping DAL05 online 24x7. Josh explained the ways we safeguard the facility with n+1 redundancy and regular maintenance and load testing, and the group was led through two more stages of secured doors ... the first with badge access, the second requiring fingerprint authentication. When they made it through, they were officially in SR01.DAL05.

TechStars Cloud Data Center Tour

Josh explained how our data center CRAC units work, how each server row is powered and how we measure and optimize the server room environment. While that aspect of the data center could seem like "blocking and tackling," he talked about our continued quest to improve power efficiency as he shared a few of the innovative approaches we've been testing, and it was clear that the tour understood it to be easier than, "Plug in server. Turn on air conditioner."

TechStars Cloud Data Center Tour

The teams got a chance to get up close and personal (No Touching!) with a server rack, and they learned about our unique network-with-a-network topology that features public, private and out-of-band management functionality. Many "oohs" and "ahhhs" were expressed.

TechStars Cloud Data Center Tour

The tour wrapped up outside of the data center facility in front of the Alpha HQ's Network Operations Center. From here, the TechStars could see how our network team observes and responds to any network-related events, and they could ask questions about anything they saw during the tour (without having to shout over the air conditioning hum).

When the final tour concluded, the full group reconvened in one of our conference rooms. They'd seen the result of our hard work, and we wanted them to know where all that hard work started. Because SoftLayer was started in a Dallas living room a few short years ago, we knew our story would be interesting, inspirational and informative, and we wanted to provide as much guidance as possible to help these soon-to-grow businesses prepare for their own success. After a brief Q&A period, a few of the TechStars Cloud participants (and some of their Dallas-based Tech Wildcatters cousins) presented a little about their businesses and how they've grown and evolved through the TechStars program, and we got to ask our own questions to help them define their business moving forward.

After the presentations at the office, we knew we couldn't just load the bus up to send the teams back to San Antonio ... We had to bid them farewell SoftLayer style. We scheduled a quick detour to SpeedZone Dallas where a few hours of unlimited eats, drinks, games and go-kart races were waiting for them.

We couldn't have had a better time with the participating teams, and we're looking forward to seeing the amazing things they'll continue doing in the near future. If you want to see even more data center coverage from Friday, be sure to check out "TechStars Cloud Visits SoftLayer" on Flickr!

-@PaulFord

April 9, 2012

Scaling SoftLayer

SoftLayer is in the business of helping businesses scale. You need 1,000 cloud computing instances? We'll make sure our system can get them online in 10 minutes. You need to spin up some beefy dedicated servers loaded with dual 8-core Intel Xeon E5-2670 processors and high-capacity SSDs for a new application's I/O-intensive database? We'll get it online anywhere in the world in under four hours. Everywhere you look, you'll see examples of how we help our customers scale, but what you don't hear much about is how our operations team scales our infrastructure to ensure we can accommodate all of our customers' growth.

When we launch a new data center, there's usually a lot of fanfare. When AMS01 and SNG01 came online, we talked about the thousands of servers that are online and ready. We meet huge demand for servers on a daily basis, and that presents us with a challenge: What happens when the inventory of available servers starts dwindling?

Truck Day.

Truck Day not limited to a single day of the year (or even a single day in a given month) ... It's what we call any date our operations team sets for delivery and installation of new hardware. We communicate to all of our teams about the next Truck Day in each location so SLayers from every department can join the operations team in unboxing and preparing servers/racks for installation. The operations team gets more hands to speed up the unloading process, and every employee has an opportunity to get first-hand experience in how our data centers operate.

If you want a refresher course about what happens on a Truck Day, you can reference Sam Fleitman's "Truck Day Operations" blog, and if you want a peek into what it looks like, you can watch Truck Day at SR02.DAL05. I don't mean to make this post all about Truck Day, but Truck Day is instrumental in demonstrating the way SoftLayer scales our own infrastructure.

Let's say we install 1,000 servers to officially launch a new pod. Because each pod has slots for 5,000 servers, we have space/capacity for 3,000-4,000 more servers in the server room, so as soon as more server hardware becomes available, we'll order it and start preparing for our next Truck Day to supplement the pod's inventory. You'd be surprised how quickly 1,000 servers can be ordered, and because it's not very easy to overnight a pallet of servers, we have to take into account lead time and shipping speeds ... To accommodate our customers' growth, we have to stay one step ahead in our own growth.

This morning in a meeting, I saw a pretty phenomenal bullet that got me thinking about this topic:

Truck Day — 4/3 (All Sites): 2,673 Servers

In nine different data center facilities around the world, more than 2,500 servers were delivered, unboxed, racked and brought online. Last week. In one day.

Now I know the operations team wasn't looking for any kind of recognition ... They were just reporting that everything went as planned. Given the fact that an accomplishment like that is "just another day at SoftLayer" for those guys, they definitely deserve recognition for the amazing work they do. We host some of the most popular platforms, games and applications on the Internet, and the DC-Ops team plays a huge role in scaling SoftLayer so our customers can scale themselves.

-@gkdog

January 16, 2012

What I've Learned About Leading

What does it take to be a good leader? What kind of leader do you want to follow? Throughout my life, I've gravitated toward leadership positions. Even when I was young, I tended to take charge of a group to achieve a goal or accomplish a mission ... though most of the "missions" in my younger days happened to be some sort of mischief. Having participated in the Boy Scouts and JROTC, I joined the Marine Corps where I served for seven years, and throughout my life, I've been fortunate to have more than my fair share of incredible mentors.

When my service in the Marine Corps concluded, I "enlisted" at SoftLayer as a data center technician. My primary responsibilities included building severs to order and installing software for our customers in four hours or less, and it was all pretty foreign to me. I had a lot to learn about the technical side of operating a data center, but based on my impression of the company, I was confident that I'd be in good hands.

Because I always find myself asking for more challenges and additional responsibility, I transitioned into a Customer Systems Administrator role. The CSA position required a lot more learning (at a breakneck pace), and in addition to the technical aspect of the job, I found myself learning just as much about the 'soft skills' required to provide the great customer service. Equipped with that knowledge (and a bit more experience), I pursued a leadership role on the team, and I made it my goal to use what I'd learned in the data center and in support to lead my team. I'd be lying if I said it wasn't a challenge, but I've never backed down from one before.

I don't mean to make this post all about me ... my goal in sharing a little of my background is to give a little context for what I've learned about leadership. It goes without saying that I've been fortunate, both in the Marine Corps as well as with SoftLayer, to have some of the most intelligent, talented and sometimes downright enigmatic leaders. I've managed to pick up advice, tips and tricks for handling all of the curveballs that are thrown on a daily basis, and over my years as a leader, I've developed a few philosophies (an amalgam of some of the key points I absorbed from all of those who have led me in the past) that I try to abide by daily:

Lead by Example.

I make it a point to never ask someone to do something that I've never done or something that I'd never be willing to do. This is the keen avoidance of the "do as I say, not as I do" mentality. Failing to do this hands-down one of the easiest ways to lose your team.

Employ Your Team to Their Abilities. Empower Them to do Their Best Work.

It's very important to know your team's strengths and weaknesses and use those to make everyone better. Using the strengths of one team member, I can push another outside of his or her comfort zone to improve his or her overall skill set. I've also seen amazing results from providing freedom for my teams to make decisions. Not only does that freedom build trust, it also gives some real "ownership" to every person, and with that sense of ownership, each team member does better work. I've been a little surprised to notice this empowerment coming from the coffee juggernaut Starbucks: Starbucks baristas and store employees are encouraged to make real-time decisions in the interest of taking care of their customers.1

Keep Learning.

Leading a team requires that you stay on top of what they're doing. Always ask questions. Continue to research so you can be a resource for your team. Find opportunities to learn and take advantage of every one of them.

Most Importantly: Learn how to Have Fun.
I see this cliché often, so when you see, "Have Fun," I wouldn't be surprised if you just rolled your eyes. I'd be lying if I said that things were great all of the time ... Realistically in any business, there's going to be a time or two when the *&#@ is going to hit the fan. It's important to find opportunities to cut loose and relax a bit. Cooking steaks for the group during a long overnight shift, grabbing a bite after work or a providing a happy hour once in a while builds a great deal of cohesion outside the office walls. Providing a relaxed environment does not prevent your team from doing stressful work ... It actually builds camaraderie, and it will help the team get through those tough times. The brutal honesty is that if people aren't enjoying where they're at, they'll look elsewhere – leaders have to help foster an environment that enables success.

At the end of the day, these tips may not work for everyone. There are a plethora of studies out there pertaining to the different leadership styles, the different types of leaders and how they influence teams. What's important is that leaders need to be intentional about bettering their teams (and ultimately bettering their businesses).

Along the lines of continuous education, I'd love to hear the leadership philosophies you've learned in your experience as a leader. Leave a comment on this article to share what you think has made you successful.

-Matthew

1This comes from both observation and talking to current and past employees. I've never worked for Starbucks, so I can't cite a specific company policy to back this up, but that evident organic culture is probably worth more than a million company policies that would try to create that culture.

Subscribe to data-center