Bandwidth Utilization: Managing a Global Network

July 5, 2012

SoftLayer has over 1,750 Gbit/s of network capacity. In each of our data centers and points of presence, we have an extensive library of peering relationships and multiple 10 Gbit/s connections to independent Tier 1 carriers. We operate one of the fastest, most reliable networks on the planet, and our customers love it:

From a network operations standpoint, that means we have our work cut out for us to keep everything running smoothly while continuing to build the network to accommodate a steady increase in customer demand. It might be easier to rest on our laurels to simply maintain what we already have in place, but when you look at the trend of bandwidth usage over the past 18 months, you'll see why we need to be proactive about expanding our network:

Long Term Bandwidth Usage Trend

The purple line above plots the 95th percentile of weekly outbound bandwidth utilization on the SoftLayer network, and the red line shows the linear trend of that consumption over time. From week to week, the total usage appears relatively consistent, growing at a steady rate, but when you look a little deeper, you get a better picture of how dynamic our network actually is:

SoftLayer Weekly Bandwidth Usage

The animated gif above shows the 2-hour average of bandwidth usage on our entire network over a seven-week period (times in CDT). As you can see, on a day-to-day basis, consumption fluctuates pretty significantly. The NOC (Network Operations Center) needs to be able to accommodate every spike of usage at any time of day, and our network engineering and strategy teams have to stay ahead of the game when it comes to planning points of presence and increasing bandwidth capacity to accommodate our customers' ever-expanding needs.

But wait. There's more.

Let's go one level deeper and look a graph of the 95th percentile bandwidth usage on 5-minute intervals from one week in a single data center:

Long Term Bandwidth Usage Trend

The variations in usage are even more dramatic. Because we have thirteen data centers geographically dispersed around the world with an international customer base, the variations you see in total bandwidth utilization understate the complexity of our network's bandwidth usage. Customers targeting the Asian market might host content in SNG01, and the peaks in bandwidth consumption from Singapore will counterbalance the valleys of consumption at the same time in the United States and Europe.

With that in mind, here's a challenge for you: Looking at the graph above, if the times listed are in CDT, which data center do you think that data came from?

It would be interesting to look at weekly usage trends, how those trends are changing and what those trends tell us about our customer base, but that assessment would probably be "information overload" in this post, so I'll save that for another day.

-Dani

P.S. If you came to this post expecting to see "a big truck" or "a series of tubes," I'm sorry I let you down.

Comments

July 10th, 2012 at 6:34am

Have you guys looked into the problems in Asia? We're noticing some huge issues between your HKIX, SNG, TOKY and SJC PoPs.

What your looking glass shows is not the actual performance/response/latency between your PoPs. ie Ping HKIX to SJC, the response time is more like 250ms rather than the published 205ms.

The routing was very smooth before, now it seems completely disrupted.

SL long-term fan and customer, but complaining from Asia.

PS Do you guys have plans for Data Center in HK or TOKYO? that would be real sweet and you can target more of the China market...

July 10th, 2012 at 9:23am

Thanks, SL Fan! Do you have a ticket open with traceroutes that show the issues you've seen? If you can send that ticket number to our social media team (twitter@softlayer.com), we'll follow up with it.

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
To prove you're a person (not a spam script), type the security word shown in the picture.
Image CAPTCHA
Enter the characters shown in the image.

Comments

July 10th, 2012 at 6:34am

Have you guys looked into the problems in Asia? We're noticing some huge issues between your HKIX, SNG, TOKY and SJC PoPs.

What your looking glass shows is not the actual performance/response/latency between your PoPs. ie Ping HKIX to SJC, the response time is more like 250ms rather than the published 205ms.

The routing was very smooth before, now it seems completely disrupted.

SL long-term fan and customer, but complaining from Asia.

PS Do you guys have plans for Data Center in HK or TOKYO? that would be real sweet and you can target more of the China market...

July 10th, 2012 at 9:23am

Thanks, SL Fan! Do you have a ticket open with traceroutes that show the issues you've seen? If you can send that ticket number to our social media team (twitter@softlayer.com), we'll follow up with it.

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
To prove you're a person (not a spam script), type the security word shown in the picture.
Image CAPTCHA
Enter the characters shown in the image.