softlayer

July 30, 2012

Don't Stop Believing (in Hosting)

If 80's movies have taught me anything, it's that any good story needs to have a video montage with Journey playing in the background. With that in mind, I'll start this blog post with a glimpse of HostingCon 2012:

HostingCon brings the hosting industry together every year, and the conference winds up being surprisingly similar to classic 80's "coming of age" movies:

  • "Geeks" are among the main characters.
  • There's always a "funny guy."
  • At some point, the geeks attend a party.
  • The characters learn more about themselves and others over the course of the movie.
  • As the credits roll, everyone is inspired ... Ready to take on the world.

With that in mind, HostingCon 2012 in Boston was a veritable John Hughes flick. There was no shortage of geeks, we hung out with one of the funniest people in the country, we threw a massive party, and we learned a ton. Without a doubt, attendees returned home with their intensity and enthusiasm cranked up to eleven (another 80's reference).

The expo hall was abuzz with activity — albeit after a lull in the morning following the aptly named "Host Me All Night Long" party — and we enjoyed the opportunity to catch up with current partners and customers while meeting and speaking with soon-to-be partners and customers. While running a highly competitive Server Challenge, we were still able to dive deeper into partnerships, the build v. buy decision, branding, and launching a product when attendees visited our booth after hearing from our team in conference sessions and panels, and those conversations are what keep us coming back to HostingCon every year.

As a "veteran" of the hosting industry (assuming seven years of experience qualifies me), I've learned a great deal about the dynamics of the hosting industry from events like HostingCon over the years. On one hand, many of the attendees are "competitors," and on the other hand, we're all trying to make the industry better (since "a rising tide lifts all boats"). As a great example, look at the Internet Infrastructure Coalition (i2C), a trade association of companies with the shared goal and purpose of representing the industry in Washington, D.C., and beyond.

As it turns out, that unity flew out the door when attendees stood face-to-rack with the Server Challenge, though. Unlike our experiences at more general "technology" conferences, the components in our competition needed no introduction, and participants were particularly driven to best their peers ... not only for the iPad, but for the pride of owning the Server Challenge title at HostingCon:

  1. Darin Goldman - 0:59.28
  2. Devon Hillard - 1:01.58
  3. Ijan Kruizinga - 1:01.83
  4. Jon Basha - 1:03.02
  5. Sean Whitley - 1:03.06

As you saw in the video, Darin Goldman had the luxury of not needing his second attempt on the final day of the conference to secure a victory, but we were glad he let us record his "Breakfast Club" fist-pump to share with the world.

Fist Pump

Don't stop believing (in hosting).

-@khazard

P.S. I recorded the first few minutes of Ralphie May's set, but the adult language-ness of the content makes it a little more difficult to share with the world.

Categories: 
July 27, 2012

SoftLayer 'Cribs' ≡ DAL05 Data Center Tour

The highlight of any customer visit to a SoftLayer office is always the data center tour. The infrastructure in our data centers is the hardware platform on which many of our customers build and run their entire businesses, so it's not surprising that they'd want a first-hand look at what's happening inside the DC. Without exception, visitors to a SoftLayer data center pod are impressed when they walk out of a SoftLayer data center pod ... even if they've been in dozens of similar facilities in the past.

What about the customers who aren't able to visit us, though? We can post pictures, share stats, describe our architecture and show you diagrams of our facilities, but those mediums can't replace the experience of an actual data center tour. In the interest of bridging the "data center tour" gap for customers who might not be able to visit SoftLayer in person (or who want to show off their infrastructure), we decided to record a video data center tour.

If you've seen "professional" video data center tours in the past, you're probably positioning a pillow on top of your keyboard right now to protect your face if you fall asleep from boredom when you hear another baritone narrator voiceover and see CAD mock-ups of another "enterprise class" facility. Don't worry ... That's not how we roll:

Josh Daley — whose role as site manager of DAL05 made him the ideal tour guide — did a fantastic job, and I'm looking forward to feedback from our customers about whether this data center tour style is helpful and/or entertaining.

If you want to see more videos like this one, "Like" it, leave comments with ideas and questions, and share it wherever you share things (Facebook, Twitter, your refrigerator, etc.).

-@khazard

July 26, 2012

Global IP Addresses - What Are They and How Do They Work?

SoftLayer recently released "Global IPs" to a good amount of internal fanfare, and I thought I'd share a little about it with the blog audience in case customers have questions about what Global IPs are and how they work. Simply put, Global IP addresses can be provisioned in any data center on the SoftLayer network and moved to another facility if necessary. You can point it to a server in Dallas, and if you need to perform maintenance on the server in Dallas, you can move the IP address to a server in Amsterdam to seamlessly (and almost immediately) transition your traffic. If you spin up and turn down workloads on cloud computing instances, you have the ability to maintain and a specific IP address when you completely turn down an environment, and you can quickly reprovision the IP on a new instance when you spin up the next workload.

How Do Global IPs Work?

The basics of how the Internet works are simple: Packets are sent between you and a server somewhere based on the location of the content you've requested. That location is pinpointed by an IP address that is assigned to a specific server or cloud. Often for various reasons, blocks of IP addresses are provisioned in one region or location, so Global IPs are a bit of a departure from the norm.

When you're sending/receiving packets, you might thing the packets "know" the exact physical destination as soon as they're directed to an IP address, but in practice, they don't have to ... The packets are forwarded along a path of devices with a general idea of where the exact location will be, but the primary concern of each device is to get the all packets to the next hop in the network path as quickly as possible by using default routes and routing tables. As an example, let's follow a packet as it comes from an external webserver and detail how it gets back to your machine:

  1. The external webserver sends the packet to a local switch.
  2. The switch passes it to a router.
  3. The packet traverses a number of network hops (other routers) and enters the Softlayer network at one of the backbone routers (BBR).
  4. The BBR looks at the IP destination and compares it to a table shared and updated with the other routers on SoftLayer's network, and it locates the subnet the IP belongs to.
  5. The BBR determines behind which distribution aggregate router (DAR) the IP is located, then it to the closest BBR to that DAR.
  6. The DAR gets the packet, looks at its own tables, and finds the front-end customer router (FCR) that the subnet lives on, and sends it there.
  7. The FCR routes the packet to the front-end customer switch (FCS) that has that IP mapped to the proper MAC address.
  8. The switch then delivers the packet through the proper switchport.
  9. Your server gets the packet from the FCS, and the kernel goes, "Oh yes, that IP on the public port, I'll accept this now."

All of those steps happen in an instant, and for you to be reading this blog, the packets carrying this content would have followed a similar pattern to the browser on your computer.

The process is slightly different when it comes to Global IP addresses. When a packet is destined for a Global IP, as soon as it gets onto the SoftLayer network (step 4 above), the routing process changes.

We allocate subnets of IP addresses specifically to the Global IP address pool, and we tell all the BBRs that these IPs are special. When you order a global IP, we peel off one of those IPs and add a static route to your chosen server's IP address, and then tell all the BBRs that route. Rather than the server's IP being an endpoint, the network is expecting your server to act as a router, and do something with the packet when it is received. I know that could sound a little confusing since we aren't really using the server as a router, so let's follow a packet to your Global IP (following the first three steps from above):

  1. The BBR notes that this IP belongs to one of the special Global IP address subnets, and matches the destination IP with the static route to the destination server you chose when you provisioned the Global IP.
  2. The BBR forwards the packet to the DAR, which then finds the FCR, then hands it off to the switch.
  3. The switch hands the packet to your server, and your server accepts it on the public interface like a regular secondary IP.
  4. Your server then essentially "routes" the packet to an IP address on itself.

Because the Global IP address can be moved to different servers in different locations, whenever you change the destination IP, the static route is updated in our routing table quickly. Because the change is happening exclusively on SoftLayer's infrastructure, you don't have to wait on other providers propagate the change. Think of updating your site's domain to a new IP address via DNS as an example: Even after you update your authoritative DNS servers, you have to wait for your users' DNS servers to recognize and update the new IP address. With Global IPs, the IP address would remain the same, and all users will follow the new path as soon as the routers update.

This initial release of Global IP addresses is just the tip of the iceberg when it comes to functionality. The product management and network engineering teams are getting customer feedback and creating roadmaps for the future of the product, so we'd love to hear your feedback and questions. If you want a little more in-depth information about installation and provisioning, check out the Global IP Addresses page on KnowledgeLayer.

-Jason

July 25, 2012

ServerDensity: Tech Partner Spotlight

We invite each of our featured SoftLayer Tech Marketplace Partners to contribute a guest post to the SoftLayer Blog, and this week, we're happy to welcome David Mytton, Founder of ServerDensity. Server Density is a hosted server and website monitoring service that alerts you when your website is slow, down or back up.

5 Ways to Minimize Downtime During Summer Vacation

It's a fact of life that everything runs smoothly until you're out of contact, away from the Internet or on holiday. However, you can't be available 24/7 on the chance that something breaks; instead, there are several things you can do to ensure that when things go wrong, the problem can be managed and resolved quickly. To help you set up your own "get back up" plan, we've come up with a checklist of the top five things you can do to prepare for an ill-timed issue.

1. Monitoring

How will you know when things break? Using a tool like Server Density — which combines availability monitoring from locations around the world with internal server metrics like disk usage, Apache and MySQL — means that you can be alerted if your site goes down, and have the data to find out why.

Surprisingly, the most common problems we see are some that are the easiest to fix. One problem that happens all too often is when a customer simply runs out of disk space in a volume! If you've ever had it happen to you, you know that running out of space will break things in strange ways — whether it prevents the database from accepting writes or fails to store web sessions on disk. By doing something as simple as setting an alert to monitor used disk space for all important volumes (not just root) at around 75%, you'll have proactive visibility into your server to avoid hitting volume capacity.

Additionally, you should define triggers for unusual values that will set off a red flag for you. For example, if your Apache requests per second suddenly drop significantly, that change could indicate a problem somewhere else in your infrastructure, and if you're not monitoring those indirect triggers, you may not learn about those other problems as quickly as you'd like. Find measurable direct and indirect relationships that can give you this kind of early warning, and find a way to measure them and alert yourself when something changes.

2. Dealing with Alerts

It's no good having alerts sent to someone who isn't responding (or who can't at a given time). Using a service like Pagerduty allows you to define on-call rotations for different types of alerts. Nobody wants to be on-call every hour of every day, so differentiating and channeling alerts in an automated way could save you a lot of hassle. Another huge benefit of a platform like Pagerduty is that it also handles escalations: If the first contact in the path doesn't wake up or is out of service, someone else gets notified quickly.

3. Tracking Incidents

Whether you're the only person responsible or you have a team of engineers, you'll want to track the status of alerts/issues, particularly if they require escalation to different vendors. If an incident lasts a long time, you'll want to be able to hand it off to another person in your organization with all of the information they need. By tracking incidents with detailed notes information, you can avoid fatigue and prevent unnecessary repetition of troubleshooting steps.

We use JIRA for this because it allows you to define workflows an issue can progress along as you work on it. It also includes easy access to custom fields (e.g. specifying a vendor ticket ID) and can be assigned to different people.

4. Understanding What Happened

After you have received an alert, acknowledged it and started tracking the incident, it's time to start investigating. Often, this involves looking at logs, and if you only have one or two servers, it's relatively easy, but as soon as you add more, the process can get exponentially more difficult.

We recommend piping them all into a log search tool like (fellow Tech Partners Marketplace participant) Papertrail or Loggly. Those platforms afford you access to all of your logs from a single interface with the ability to see incoming lines in real-time or the functionality to search back to when the incident began (since you've clearly monitored and tracked all of that information in the first three steps).

5. Getting Access to Your Servers

If you're traveling internationally, access to the Internet via a free hotspot like the ones you find in Starbucks isn't always possible. It's always a great idea to order a portable 3G hotspot in advance of a trip. You can usually pick one up from the airport to get basic Internet access without paying ridiculous roaming charges. Once you have your connection, the next step is to make sure you can access your servers.

Both iPhone and Android have SSH and remote desktop apps available which allow you to quickly log into your servers to fix easy problems. Having those tools often saves a lot of time if you don't have access to your laptop, but they also introduce a security concern: If you open server logins to the world so you can login from the dynamic IPs that change when you use mobile connectivity, then it's worth considering a multi-factor authentication layer. We use Duo Security for several reasons, with one major differentiator being the modules they have available for all major server operating systems to lock down our logins even further.

You're never going to escape the reality of system administration: If your server has a problem, you need to fix it. What you can get away from is the uncertainty of not having a clearly defined process for responding to issues when they arise.

-David Mytton, ServerDensity

This guest blog series highlights companies in SoftLayer's Technology Partners Marketplace.
These Partners have built their businesses on the SoftLayer Platform, and we're excited for them to tell their stories. New Partners will be added to the Marketplace each month, so stay tuned for many more come.
July 19, 2012

The Human Element of SoftLayer - DAL05 DC Operations

One of the founding principles of SoftLayer is automation. Automation has enabled this company to provide our customers with a world class experience, and it enables employees to provide excellent service. It allows us to quickly deploy a variety of solutions at the click of a button, and it guarantees consistency in the products that we deliver. Automation isn't the whole story, though. The human element plays a huge role in SoftLayer's success.

As a Site Manager for the corporate facility, I thought I could share a unique perspective when it comes to what that human element looks like, specifically through the lens of the Server Build Team's responsibilities. You recently heard how my colleague, Broc Chalker, became an SBT, and so I wanted take it a step further by providing a high-level breakdown of how the Server Build Team enables SoftLayer to keep up with the operational demands of a rapidly growing, global infrastructure provider.

The Server Build Team is responsible for filling all of the beautiful data center environments you see in pictures and videos of SoftLayer facilities. Every day, they are in the DC, building out new rows for inventory. It sounds pretty simple, but it's actually a pretty involved process. When it comes to prepping new rows, our primary focus is redundancy (for power, cooling and network). Each rack is powered by dual power sources, four switches in a stacked configuration (two public network, two private network), and an additional switch that provides KVM access to the server. To make it possible to fill the rack with servers, we also have to make sure it's organized well, and that takes a lot of time. Just watch the video of the Go Live Crew cabling a server rack in SJC01, and you can see how time- and labor-intensive the process is. And if there are any mistakes or if the cables don't look clean, we'll cut all the ties and start over again.



 

In addition to preparing servers for new orders, SBTs also handle hardware-related requests. This can involve anything from changing out components for a build, performing upgrades / maintenance on active servers, or even troubleshooting servers. Any one of these requests has to be treated with significant urgency and detail.



 

The responsibilities do not end there. Server Build Technicians also perform a walk of the facility twice per shift. During this walk, technicians check for visual alerts on the servers and do a general facility check of all SoftLayer pods. Note: Each data center facility features one or more pods or "server rooms," each built to the same specifications to support up to 5,000 servers.



 

The DAL05 facility has a total of four pods, and at the end of the build-out, we should be running 18,000-20,000 servers in this facility alone. Over the past year, we completed the build out of SR02 and SR03 (pod 2 and 3, respectively), and we're finishing the final pod (SR04) right now. We've spent countless hours building servers and monitoring operating system provisions when new orders roll in, and as our server count increases, our team has grown to continue providing the support our existing customers expect and deserve when it comes to upgrade requests and hardware-related support tickets.



 

To be successful, we have to stay ahead of the game from an operations perspective. The DAL05 crew is working hard to build out this facility's last pod (SR04), but for the sake of this blog post, I pulled everyone together for a quick photo op to introduce you to the team.

DAL05 Day / Evening Team and SBT Interns (with the remaining racks to build out in DAL05):
DAL05 DC Ops

DAL05 Overnight Server Build Technician Team:
DAL05 DC Ops

Let us know if there's ever anything we can do to help you!

-Joshua

July 13, 2012

When Opportunity Knocks

I've been working in the web hosting industry for nearly five years now, and as is the case with many of the professionals of my generation, I grew up side by side with the capital-I Internet. Over those five years, the World Wide Web has evolved significantly, and it's become a need. People need the Internet to communicate, store information, enable societal connectivity and entertain. And they need it 24 hours per day, seven days a week. To affirm that observation, you just need to look at an excerpt from a motion submitted to the Human Rights Council and recently passed by the United Nations General Assembly:

The General Session ... calls upon all States to promote and facilitate access to the Internet and international cooperation aimed at the development of media and information and communications facilities in all countries.

After a platform like the Internet revolutionizes the way we see the world, it's culturally impossible to move backward. Its success actually inspires us to look forward for the next world-changing innovation. Even the most non-technical citizen of the Internet has come to expect those kinds of innovations as the Internet and its underlying architecture have matured and seem to be growing like Moore's Law: Getting faster, better, and bigger all the time. The fact that SoftLayer is able to keep up with that growth (and even continue innovating in the process) is one of the things I admire most about the company.

I love that our very business model relies on our ability to enable our customers' success. Just look at how unbelievably successful companies like Tumblr and HostGator have become, and you start to grasp how big of a deal it is that we can help their businesses. We're talking billions of pageviews per month and hundreds of thousands of businesses that rely on SoftLayer through our customers. And that's just through two customers. Because we're on the cutting edge, and we provide unparalleled access and functionality, we get to see a lot of the up-and-coming kickstarts that are soon to hit it big, and we get to help them keep up with their own success.

On a personal level, I love that SoftLayer provides opportunities for employees. Almost every department has a career track you can follow as you learn more about the business and get a little more experience, and you're even able to transition into another department if you're drawn to a new passion. I recently move to the misty northwest (Seattle) when given the opportunity by SoftLayer, and after working in the data center, I decided to pursue a role as a systems administrator. It took a lot of hard work, but I made the move. Hard work is recognized, and every opportunity I've taken advantage of has been fulfilled. You probably think I'm biased because I've done well in the organization, and that might be a fair observation, but in reality, the opportunities don't just end with me.

One of my favorite stories to share about SoftLayer is the career path of my best friend, Goran. I knew he was a hard worker, so I referred him to the company a few years ago, and he immediately excelled as an Operations Tech. He proved himself on the Go-Live Crew in Amsterdam by playing a big role in the construction of AMS01, and he was promoted to a management position in that facility. He had been missing Europe for the better part of a decade, SoftLayer gave him a way to go back home while doing what he loves (and what he's good at).

If that Goran's story isn't enough for you, I could tell you about Robert. He started at SoftLayer as a data center tech, and he worked hard to become a systems administrator, then he was named a site manager, then he was promoted to senior operations manager, and now he's the Director of Operations. You'll recognize him as the guy with all of the shirts in Lance's "Earn Your Bars" blog post from December. He took every rung on the ladder hand-over-hand because no challenge could overwhelm him. He sought out what needed to be done without being asked, and he was proactive about make SoftLayer even better.

I could tell you about dozens of others in the company that have the same kinds of success stories because they approached the opportunities SoftLayer provided them with a passion and positive attitude that can't be faked. If being successful in an organization makes you biased, we're all biased. We love this environment. We're presented with opportunities and surrounded by people encouraging us to take advantage of those opportunities, and as a result, we can challenge ourselves and reach our potential. No good idea is ignored, and no hard work goes unrecognized.

I'm struggling to suppress the countless "opportunity" stories I've seen in my tenure at SoftLayer, but I think the three stories above provide a great cross-section of what it looks like to work for SoftLayer. If you like being challenged (and being rewarded for your hard work), you might want to take this opportunity to see which SoftLayer Career could be waiting for you.

When opportunity knocks, let it in.

-Hilary

Categories: 
July 12, 2012

An Insider's Look at SoftLayer's Growth in Amsterdam

Last week, SoftLayer was featured on the NOS national news here in the Netherlands in a segment that allowed us to tell our story and share how we're settling into our new Amsterdam home. I've only been a SLayer for about nine months now, and as I watched the video, I started to reflect on how far we've come in such a surprisingly short time. Take a second to check it out (don't worry, it's not all in Dutch):

To say that I had to "hit the ground running" when I started at SoftLayer would be an understatement. The day after I got the job, I was on a plane to SoftLayer's Dallas headquarters to meet the team behind the company. To be honest, it was a pretty daunting task, but I was energized at the opportunity to learn about how SoftLayer became largest privately owned hosting company in the world from the people who started it. When I look back at the interview Kevin recorded with me, I'm surprised that I didn't look like a deer in the headlights. At the time, AMS01 was still in the build-out phase, so my tours and meetings in DAL05 were both informative and awe-inspiring.

When I returned to Europe, I was energized to start playing my role in the company's new pursuit of its global goals.

It didn't take long before I started seeing the same awe-inspiring environment take place in our Amsterdam facility ... So much so that I'm convinced that at least a few of the "Go Live Crew" members were superhuman. As it turns out, when you build identical data center pods in every location around the world, you optimize the process and figure out the best ways to efficiently use your time.

By the time the Go Live Crew started packing following the successful (and on-time) launch of AMS01, I started feeling the pressure. The first rows of server racks were already being filled by customers, but the massive data center space seemed impossibly large when I started thinking of how quickly we could fill it. Most of my contacts in Europe were not familiar with the SoftLayer name, and because my assigned region was Europe Middle East and Africa — a HUGE diverse region with many languages, cultures and currencies — I knew I had my work cut out for me.

I thought, "LET'S DO THIS!"

EMEA is home to some of the biggest hosting markets in the world, so my first-week whirlwind tour of Dallas actually set the stage quite nicely for what I'd be doing in the following months: Racking up air miles, jumping onto trains, attending countless trade shows, meeting with press, reaching out to developer communities and corresponding with my fellow SLayers in the US and Asia ... All while managing the day-to-day operations of the Amsterdam office. As I look back at that list, I'm amazed how the team came together to make sure everything got done.

We have come a long way.

As I started writing this blog, BusinessReview Europe published a fantastic piece on SoftLayer in their July 2012 magazine (starting on page 172) that seems to succinctly summarize how we've gotten where we are today: "Innovation Never Sleeps."

BusinessReview Europe

Our first pod is almost full of servers humming and flashing. When we go to tradeshows and conferences throughout Europe, people not only know SoftLayer, many of them are customers with servers in AMS01. That's the kind of change we love.

The best part of my job right now is that our phenomenal success in the past nine months is just a glimmer of what the future holds. Come to think of it, we're going to need some more people.

-@jpwisler

July 11, 2012

Mandrill: Tech Partner Spotlight

This is a guest blog with Chad Morris from our partner Mandrill. Mandrill is an email delivery platform built on and managed by MailChimp. Created for developers to set up and manage with minimal coding effort, Mandrill offers advanced tracking, easy-to-understand reports and hundreds of template options. In this video interview, Chad goes into detail about the history of the company as well as the major differences between Mandrill and MailChimp. In the near future, you'll see a separate guest blog from the Mandrill team with best practices for managing your email systems.

This guest blog series highlights companies in SoftLayer's Technology Partners Marketplace.
These Partners have built their businesses on the SoftLayer Platform, and we're excited for them to tell their stories. New Partners will be added to the Marketplace each month, so stay tuned for many more come.
July 6, 2012

My Advice to Myself (A New Server Build Technician)

When I started at SoftLayer, I had no idea what to expect. As I walked from the parking lot to the front doors at SJC01, I started to get nervous ... I felt was like I was stepping onto a stage, and I was worried about making a mistake. I took a deep breath and walked in.

Now that I look back on my first day (which was about a month ago), I have to laugh at my nervousness. I'm not sure what I expected to encounter, but the environment I entered was probably the most welcoming and friendly I've ever seen. Two of my coworkers, Cuong and Jonathan, recently shared their experiences as SBTs in San Jose, but because I have some recent first-hand experience that's still fresh in my mind, I thought I'd share my own perspective.

If I were able to talk to myself as I nervously approached the San Jose data center on my first day, this is what I'd say:

As you'd expect from any new job, your first day at work involves a lot of learning (and paperwork). You're probably chomping at the bit to get out into the data center to start building servers, but you need to crawl before you walk. The first thing you need to do is get the lay of the land ... You get a guided tour of the office, the data center and your workspace. Even if you've worked in a data center before, you're going to be surprised and impressed with how everything is set up. Once all of your paperwork is in order, you start learning about SoftLayer's business and how you contribute to the customer experience. Once you understand the big picture, you can get into the details.

You're given a training guide that goes over many of the processes and procedures that are followed on a day-to-day basis in the data center, and you're shown all of the components you'll be working with as you build, upgrade and manage server hardware. You might not be performing much work on hardware in production in your first few days, but you're going to learn a lot and have plenty of time to ask questions. While you're learning how to perform your work tasks, you're building friendships with your coworkers, and you're officially becoming part of the SoftLayer family. Your fellow SLayers support you and help you make sure SoftLayer's customers are getting the service they expect.

You're taught everything you need to know, from staying organized and focused to best practices around working with servers. You have nothing to be nervous about.

I've only been with SoftLayer for a short period of time, but I can confidently say that working here is remarkable. I don't feel like an "employee;" I feel like a team player. I feel like everyone is on the same page about what needs to be done in the data center, and whenever questions come up, answers are given quickly.

I'm excited to come to work every day. I would have never dreamed I'd feel this way because I was always told jobs are long and drag-out boring, but my experience has been the polar opposite. Now, When any of my friends complain about getting up and going to work, I recommend they visit http://www.softlayer.com/about/careers.

-Jackie

July 5, 2012

Bandwidth Utilization: Managing a Global Network

SoftLayer has over 1,750 Gbit/s of network capacity. In each of our data centers and points of presence, we have an extensive library of peering relationships and multiple 10 Gbit/s connections to independent Tier 1 carriers. We operate one of the fastest, most reliable networks on the planet, and our customers love it:

From a network operations standpoint, that means we have our work cut out for us to keep everything running smoothly while continuing to build the network to accommodate a steady increase in customer demand. It might be easier to rest on our laurels to simply maintain what we already have in place, but when you look at the trend of bandwidth usage over the past 18 months, you'll see why we need to be proactive about expanding our network:

Long Term Bandwidth Usage Trend

The purple line above plots the 95th percentile of weekly outbound bandwidth utilization on the SoftLayer network, and the red line shows the linear trend of that consumption over time. From week to week, the total usage appears relatively consistent, growing at a steady rate, but when you look a little deeper, you get a better picture of how dynamic our network actually is:

SoftLayer Weekly Bandwidth Usage

The animated gif above shows the 2-hour average of bandwidth usage on our entire network over a seven-week period (times in CDT). As you can see, on a day-to-day basis, consumption fluctuates pretty significantly. The NOC (Network Operations Center) needs to be able to accommodate every spike of usage at any time of day, and our network engineering and strategy teams have to stay ahead of the game when it comes to planning points of presence and increasing bandwidth capacity to accommodate our customers' ever-expanding needs.

But wait. There's more.

Let's go one level deeper and look a graph of the 95th percentile bandwidth usage on 5-minute intervals from one week in a single data center:

Long Term Bandwidth Usage Trend

The variations in usage are even more dramatic. Because we have thirteen data centers geographically dispersed around the world with an international customer base, the variations you see in total bandwidth utilization understate the complexity of our network's bandwidth usage. Customers targeting the Asian market might host content in SNG01, and the peaks in bandwidth consumption from Singapore will counterbalance the valleys of consumption at the same time in the United States and Europe.

With that in mind, here's a challenge for you: Looking at the graph above, if the times listed are in CDT, which data center do you think that data came from?

It would be interesting to look at weekly usage trends, how those trends are changing and what those trends tell us about our customer base, but that assessment would probably be "information overload" in this post, so I'll save that for another day.

-Dani

P.S. If you came to this post expecting to see "a big truck" or "a series of tubes," I'm sorry I let you down.

Pages

Subscribe to softlayer