Global Network: The Proof is in the Traceroute

December 2, 2011

You've probably heard a lot about SoftLayer's global expansion into Asia and Europe, and while the idea of geographically diversifying is impressive in itself, one of the most significant implications of our international expansion is what it's done for the SoftLayer Network.

As George explained in "Globalization and Hosting: The World Wide Web is Flat," our strategic objective is to get a network point of presence within 40ms of all of our users and our users' users to provide the best network stability and performance possible anywhere on the planet. The reasoning is simple: The sooner a user gets on on our network, the quicker we can efficiently route them through our points of presence to a server in one of our data centers.

The cynics in the audience are probably yawning and shrugging that idea off as marketing mumbo jumbo, so I thought it would be good to demonstrate how the network expansion immediately and measurably improved our customers' network experience from Asia to the United States. Just look at the traceroutes.

As you're probably aware, a traceroute shows the "hops" or routers along the network path from an origin IP to a destination IP. When we were building out the Singapore data center (before the network points of presence were turned up in Asia), I ran a traceroute from Singapore to SoftLayer.com, and immediately after the launch of the data center, I ran another one:

Pre-Launch Traceroute to SoftLayer.com from Singapore

traceroute to softlayer.com (66.228.118.53), 64 hops max, 52 byte packets
 1  10.151.60.1 (10.151.60.1)  1.884 ms  1.089 ms  1.569 ms
 2  10.151.50.11 (10.151.50.11)  2.006 ms  1.669 ms  1.753 ms
 3  119.75.13.65 (119.75.13.65)  3.380 ms  3.388 ms  4.344 ms
 4  58.185.229.69 (58.185.229.69)  3.684 ms  3.348 ms  3.919 ms
 5  165.21.255.37 (165.21.255.37)  9.002 ms  3.516 ms  4.228 ms
 6  165.21.12.4 (165.21.12.4)  3.716 ms  3.965 ms  5.663 ms
 7  203.208.190.21 (203.208.190.21)  4.442 ms  4.117 ms  4.967 ms
 8  203.208.153.241 (203.208.153.241)  6.807 ms  55.288 ms  56.211 ms
 9  so-2-0-3-0.laxow-cr1.ix.singtel.com (203.208.149.238)  187.953 ms  188.447 ms  187.809 ms
10  ge-4-0-0-0.laxow-dr2.ix.singtel.com (203.208.149.34)  184.143 ms
    ge-4-1-1-0.sngc3-dr1.ix.singtel.com (203.208.149.138)  189.510 ms
    ge-4-0-0-0.laxow-dr2.ix.singtel.com (203.208.149.34)  289.039 ms
11  203.208.171.98 (203.208.171.98)  187.645 ms  188.700 ms  187.912 ms
12  te1-6.bbr01.cs01.lax01.networklayer.com (66.109.11.42)  186.482 ms  188.265 ms  187.021 ms
13  ae7.bbr01.cs01.lax01.networklayer.com (173.192.18.166)  188.569 ms  191.100 ms  188.736 ms
14  po5.bbr01.eq01.dal01.networklayer.com (173.192.18.140)  381.645 ms  410.052 ms  420.311 ms
15  ae0.dar01.sr01.dal01.networklayer.com (173.192.18.211)  415.379 ms  415.902 ms  418.339 ms
16  po1.slr01.sr01.dal01.networklayer.com (66.228.118.138)  417.426 ms  417.301 ms
    po2.slr01.sr01.dal01.networklayer.com (66.228.118.142)  416.692 ms
17  * * *

Post-Launch Traceroute to SoftLayer.com from Singapore

traceroute to softlayer.com (66.228.118.53), 64 hops max, 52 byte packets
 1  192.168.206.1 (192.168.206.1)  2.850 ms  1.409 ms  1.206 ms
 2  174.133.118.65-static.reverse.networklayer.com (174.133.118.65)  1.550 ms  1.680 ms  1.394 ms
 3  ae4.dar01.sr03.sng01.networklayer.com (174.133.118.136)  1.812 ms  1.341 ms  1.734 ms
 4  ae9.bbr01.eq01.sng02.networklayer.com (50.97.18.198)  35.550 ms  1.999 ms  2.124 ms
 5  50.97.18.169-static.reverse.softlayer.com (50.97.18.169)  174.726 ms  175.484 ms  175.491 ms
 6  po5.bbr01.eq01.dal01.networklayer.com (173.192.18.140)  203.821 ms  203.749 ms  205.803 ms
 7  ae0.dar01.sr01.dal01.networklayer.com (173.192.18.253)  306.755 ms
    ae0.dar01.sr01.dal01.networklayer.com (173.192.18.211)  208.669 ms  203.127 ms
 8  po1.slr01.sr01.dal01.networklayer.com (66.228.118.138)  203.518 ms
    po2.slr01.sr01.dal01.networklayer.com (66.228.118.142)  305.534 ms
    po1.slr01.sr01.dal01.networklayer.com (66.228.118.138)  204.150 ms
 9  * * *

I won't dive too deep into what these traceroutes are telling us because that'll need to be an entirely different blog. What I want to draw your attention to are a few key differences between the pre- and post-launch traceroutes:

  • Getting onto SoftLayer's network:. The first reference to "networklayer" in the pre-launch trace is in hop 12 (~187ms). In the post-launch trace, we were on "networklayer" in the second hop (~1.5ms).
  • Number of hops: Pre-launch, our network path took 16 hops to get to SoftLayer.com. Post-launch, it took 8.
  • Response times from the destination: The average response time from SoftLayer.com to Singapore before the launch of our network points of presence in Asia was about 417ms (milliseconds). After the launch, it dropped to an average of about ~250ms.

These traceroutes demonstrate that users in Singapore travel a much better network path to a server in one of our U.S. data centers than they had before we turned up the network in Asia, and that experience isn't limited to users in Singapore ... users throughout Europe and Asia will see fewer hops and better speeds now that the data centers and points of presence on those continents are live. And that's without buying a server in either of those markets or making any changes to how they interact with us.

Managing a worldwide network for a worldwide customer base with thousands of different ISPs and millions of possible routes is not a "set it and forget it" endeavor, so we have a team of engineers in our Network Operations Center that focuses on tweaking and optimizing routes 24x7. Branching out into Europe and Asia introduces a slew of challenges when working with providers on the other side of the globe, but I guess it's true: "If it were easy, everyone would do it."

Innovate or die.

-@toddmitchell

Comments

December 2nd, 2011 at 8:33pm

The network in Hong Kong have not optimized however....

From pccw powerbase (hong kong)
1 <10 ms <10 ms <10 ms 1.254.215.203.in-addr.arpa [203.215.254.1]
2 <10 ms 15 ms <10 ms ge13-15.br01.hkg04.pccwbtn.net [63.218.61.173]
3 157 ms 172 ms 171 ms TenGigabitEthernet8-1.1201.ar4.LAX1.gblx.net [64.211.83.37]
4 171 ms 157 ms 172 ms ae2-40G.scr4.LAX1.gblx.net [67.17.95.213]
5 172 ms 171 ms 172 ms po2.ar4.LAX1.gblx.net [67.16.132.214]
6 172 ms 172 ms 156 ms softlayer-technologies-inc.ethernet12-1.ar6.lax1.gblx.net [206.165.75.230]
7 172 ms 156 ms 172 ms ae7.bbr02.cs01.lax01.networklayer.com [173.192.18.167]
8 172 ms 172 ms 156 ms ae0.bbr02.eq01.sjc02.networklayer.com [173.192.18.150]
9 157 ms 156 ms 203 ms ae5.dar02.sr01.sjc01.networklayer.com [173.192.18.251]
10 172 ms 172 ms 171 ms po2.fcr01.sr01.sjc01.networklayer.com [50.23.118.133]
11 171 ms 172 ms 172 ms speedtest.sjc01.softlayer.com [50.23.64.58]

From hutchison (hong kong)
1 bbs-1-250-0-210.on-nets.com (210.0.250.1) 1.086 ms 1.003 ms 0.887 ms
2 * * 10.2.128.10 (10.2.128.10) 29.007 ms
3 218.189.97.109 (218.189.97.109) 0.884 ms 0.795 ms 0.794 ms
4 203.131.245.69 (203.131.245.69) 1.353 ms 1.409 ms 1.341 ms
5 xe-1-0-0.r21.newthk02.hk.bb.gin.ntt.net (129.250.3.194) 1.565 ms 1.435 ms 1.485 ms
6 as-1.r25.tokyjp01.jp.bb.gin.ntt.net (129.250.4.114) 51.985 ms 51.952 ms 51.938 ms
7 ae-9.r20.tokyjp01.jp.bb.gin.ntt.net (129.250.2.92) 58.793 ms 65.374 ms 58.765 ms
8 as-1.r20.sttlwa01.us.bb.gin.ntt.net (129.250.4.189) 160.802 ms 151.131 ms 157.374 ms
9 ae-1.r05.sttlwa01.us.bb.gin.ntt.net (129.250.5.47) 160.870 ms 170.379 ms 173.939 ms
10 ae13.bbr02.wb01.sea02.networklayer.com (198.104.203.254) 243.300 ms 242.017 ms 220.704 ms
11 ae0.bbr01.eq01.sjc02.networklayer.com (173.192.18.147) 228.356 ms 225.098 ms 241.132 ms
12 ae5.dar01.sr01.sjc01.networklayer.com (173.192.18.249) 235.578 ms 230.343 ms 230.534 ms
13 po1.fcr01.sr01.sjc01.networklayer.com (50.23.118.131) 228.827 ms 249.246 ms 241.976 ms
14 speedtest.sjc01.softlayer.com (50.23.64.58) 249.481 ms 227.156 ms 252.517 ms

December 3rd, 2011 at 12:09am

Isn't it a bit sketchy that SoftLayer uses the name "NetworkLayer" on all its DNS PTR's which are visible in traceroute? I've heard that this is because they are trying to get name recognition and confuse customers who are thinking of nLayer (nlayer.net) who has been around far longer than SoftLayer, ThePlanet, etc.

I've heard many people contact the nlayer.net NOC for SoftLayer issues. Why continue the confusion and just call it softlayer.net?

December 3rd, 2011 at 12:43pm

Fat Cat, early in the evolution of SoftLayer, everything ended in "Layer" to more or less brand that piece of the business for "SoftLayer." The blog was called "The InnerLayer" (http://theinnerlayer.softlayer.com/), the knowledgebase is called "KnowledgeLayer" (http://knowledgelayer.softlayer.com/), the cloud is called CloudLayer (http://www.softlayer.com/cloudlayer/), etc.

I'm not sure who "you've heard" from that the intent was to confuse customers, but you were misinformed. I don't know the future of the networklayer.com paths, but I'd assume that they aren't going away any time soon. Thanks for bringing up your concern, though ... I hadn't heard that was an issue before.

December 3rd, 2011 at 12:55pm

Charles, one of the biggest benefits of the dynamic nature of traceroutes is that they change regularly based on relationships with other providers and the ongoing optimization efforts between our team and our peers. When one of the network engineers dives deeper into what the traceroutes are telling us and how we use them to manage the network, he or she will likely talk more about the route optimization process in detail.

Often, all it takes to "optimize" a route is communication with the origin ISPs to have them recognize the SoftLayer trans-ocean connection as a viable (and likely preferable) route, but in some cases, it's not that easy. If the performance of the connection you're getting suffers, the NOC will jump in to help. Given the physical distance between Hong Kong and San Jose, CA, it'd be tough to shave a significant amount of latency off of 171ms.

Looking at the second traceroute you posted, if you were tracing to a server in the NTT point of presence in Seattle, WA, you'd get about a 170ms response (and that would be getting straight onto their network in Hong Kong and never leaving it ... which is essentially the goal in expanding our network to users around the world). I'm sure the network engineers could tweak the connections to test the differences between using NTT/Global Exhange and the SoftLayer network between Asia and North America, but you might not see the times cut in half like they were in Todd's example from Singapore.

December 3rd, 2011 at 7:45pm

Kevin, I thought the route should be connect to HKIX, as I see Softlayer has a POP in Hong Kong and having a 10G peer with HKIX.

There is a traceroute from another isp in Hong Kong (hkbn.net).

1 5 ms 3 ms 6 ms 014198216001.ctinets.com [14.198.216.1]
2 10 ms 10 ms 15 ms 061093016209.ctinets.com [61.93.16.209]
3 6 ms 11 ms 11 ms 014199254073.ctinets.com [14.199.254.73]
4 5 ms 3 ms 3 ms 014136129106.ctinets.com [14.136.129.106]
5 6 ms 6 ms 6 ms softlayer-10G.hkix.net [202.40.161.241]
6 50 ms 57 ms 50 ms ae0.bbr02.eq01.tok01.networklayer.com [50.97.18.
166]
7 101 ms 50 ms 54 ms ae7.bbr01.eq01.tok01.networklayer.com [50.97.18.
162]
8 167 ms 166 ms 183 ms ae2.bbr02.eq01.sjc02.networklayer.com [50.97.18.
160]
9 166 ms 166 ms 166 ms ae5.dar02.sr01.sjc01.networklayer.com [173.192.1
8.251]
10 169 ms 168 ms 166 ms po2.fcr01.sr01.sjc01.networklayer.com [50.23.118
.133]
11 166 ms 166 ms 166 ms speedtest.sjc01.softlayer.com [50.23.64.58]

However, the traceroute from my SJC server to hkbn still using Any2, not the Softlayer transit.

traceroute to 14.198.222.223 (14.198.222.223), 30 hops max, 40 byte packets
1 50.23.*.*-static.reverse.softlayer.com (50.23.*.*) 0.427 ms 0.539 ms 0.642 ms
2 ae0.dar01.sr01.sjc01.networklayer.com (50.23.118.130) 0.494 ms 0.488 ms 0.478 ms
3 ae8.bbr01.eq01.sjc02.networklayer.com (173.192.18.248) 0.910 ms 0.908 ms 0.890 ms
4 ae7.bbr02.eq01.sjc02.networklayer.com (173.192.18.165) 15.390 ms 15.389 ms 15.372 ms
5 ae0.bbr02.cs01.lax01.networklayer.com (173.192.18.151) 8.584 ms 8.580 ms 8.862 ms
6 ae7.bbr01.cs01.lax01.networklayer.com (173.192.18.166) 8.683 ms 8.679 ms 8.564 ms
7 * * *
8 014136129013.ctinets.com (14.136.129.13) 172.211 ms 172.312 ms 172.304 ms
9 014199254078.ctinets.com (14.199.254.78) 170.538 ms 175.652 ms 175.538 ms
10 061093016210.ctinets.com (61.93.16.210) 175.458 ms 175.768 ms 175.216 ms
11 * * *
12 * * *

December 6th, 2011 at 11:18am

Thanks for the follow-up Charles. I got a little more detail from someone on our networking team about why you're seeing the different routes. We use "hot-potato routing" to best determine the path to a given IP address. If you're going to a SoftLayer server, other ISPs are more likely to give us the "hot potato" quickly because we can get it to our destination quickly (and we will incur the costs in doing so). When you're on the SoftLayer network and you're going to an external IP address, we'll pass the "hot potato" to an ISP that may claim a better route to that destination IP address.

If you ran the traceroute from your server in SJC to our SNG test IP (216.12.196.114) you'd see the traffic stay on our network because the destination is on our network. Also, if you noticed any significant problems with the return route traveling over our peer's network across the Pacific Ocean, our NOC could adjust the routes to ensure consistent performance.

December 7th, 2011 at 4:24pm

Kevin, thanks for your explanation.
It would be nicer to pass through the HKIX from server to Softlayer to the users in Hong Kong, since some of the ISP in Hong Kong do not have sufficient oversea bandwidth.

December 7th, 2011 at 8:55pm

This is great, I love your network but I prefer to colo, are you planning to provide collocation services in the near future?

December 8th, 2011 at 10:17am

No problem at all, Charles. The return traceroute doesn't show which provider brought the traffic back across the ocean (hop 7). Like I said following your first traceroutes, sub-200ms routes from San Jose, CA, to Hong Kong are pretty quick, considering the fact the data is traveling more than 7,000 miles. If you ever run into spikes in latency, drop a ticket in for our NOC, and the network engineers will see what we can do to optimize the routes.

Ianh, our core business is built on a platform that's significantly different from many providers in the industry, so we're not in a hurry to focus on colocation services until we can find a way to innovate in that area of the market. Many of SoftLayer's benefits are made possible by our pod-concept data center design and consistent hardware throughout, so we couldn't guarantee the same kind of performance and functionality for different hardware and configurations. What's most appealing to you about colo?

March 30th, 2012 at 1:59am

I think it would be really helpful if softlayer posted traffic/link utilization graphs. this would really boost confidence in your network if you run your links under utilized.

Reverse traceroute tools would be great also. Sometimes I get horrid speeds when downloading off softlayer, I assume its because softlayer has congested links

April 16th, 2012 at 4:56am

Hmm.

1. Surely you should be asking why the latency jumped from 188ms to 381+ms between lax and dallas on softlayer's network in the pre-launch traceroutes?

12 te1-6.bbr01.cs01.lax01.networklayer.com (66.109.11.42) 186.482 ms 188.265 ms 187.021 ms
13 ae7.bbr01.cs01.lax01.networklayer.com (173.192.18.166) 188.569 ms 191.100 ms 188.736 ms
14 po5.bbr01.eq01.dal01.networklayer.com (173.192.18.140) 381.645 ms 410.052 ms 420.311 ms
15 ae0.dar01.sr01.dal01.networklayer.com (173.192.18.211) 415.379 ms 415.902 ms 418.339 ms

2. If you practice hot potato routing (ie, hand off the packets to someone in dallas rather than in hong kong), doesn't that actually reduce the advantage you have from building a speedy global network? Eyeball -> content traffic will quickly jump onto your uncongested network, but content -> eyeball traffic will also quickly jump onto somebody else's (possibly congested) network... and most traffic is content -> eyeball.

June 24th, 2012 at 6:29am

Would it be possible to find location of server and speed using tracert?

July 23rd, 2012 at 6:54am

Any plans on opening DC in HK and Japan?

Also, who can I talk to if I want our infrastructure in HK to directly connect to SL PoP in HKIX?

July 23rd, 2012 at 2:17pm

As of right now, there are no immediate plans to open a full data center in Hong Kong or Japan. If you need a direct or peering connection with your SoftLayer infrastructure through one of our points of presence, ask to be connected with the network operations team via a support ticket, and they'll have the best insight about whether your specific use-case is possible.

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Comments

December 2nd, 2011 at 8:33pm

The network in Hong Kong have not optimized however....

From pccw powerbase (hong kong)
1 <10 ms <10 ms <10 ms 1.254.215.203.in-addr.arpa [203.215.254.1]
2 <10 ms 15 ms <10 ms ge13-15.br01.hkg04.pccwbtn.net [63.218.61.173]
3 157 ms 172 ms 171 ms TenGigabitEthernet8-1.1201.ar4.LAX1.gblx.net [64.211.83.37]
4 171 ms 157 ms 172 ms ae2-40G.scr4.LAX1.gblx.net [67.17.95.213]
5 172 ms 171 ms 172 ms po2.ar4.LAX1.gblx.net [67.16.132.214]
6 172 ms 172 ms 156 ms softlayer-technologies-inc.ethernet12-1.ar6.lax1.gblx.net [206.165.75.230]
7 172 ms 156 ms 172 ms ae7.bbr02.cs01.lax01.networklayer.com [173.192.18.167]
8 172 ms 172 ms 156 ms ae0.bbr02.eq01.sjc02.networklayer.com [173.192.18.150]
9 157 ms 156 ms 203 ms ae5.dar02.sr01.sjc01.networklayer.com [173.192.18.251]
10 172 ms 172 ms 171 ms po2.fcr01.sr01.sjc01.networklayer.com [50.23.118.133]
11 171 ms 172 ms 172 ms speedtest.sjc01.softlayer.com [50.23.64.58]

From hutchison (hong kong)
1 bbs-1-250-0-210.on-nets.com (210.0.250.1) 1.086 ms 1.003 ms 0.887 ms
2 * * 10.2.128.10 (10.2.128.10) 29.007 ms
3 218.189.97.109 (218.189.97.109) 0.884 ms 0.795 ms 0.794 ms
4 203.131.245.69 (203.131.245.69) 1.353 ms 1.409 ms 1.341 ms
5 xe-1-0-0.r21.newthk02.hk.bb.gin.ntt.net (129.250.3.194) 1.565 ms 1.435 ms 1.485 ms
6 as-1.r25.tokyjp01.jp.bb.gin.ntt.net (129.250.4.114) 51.985 ms 51.952 ms 51.938 ms
7 ae-9.r20.tokyjp01.jp.bb.gin.ntt.net (129.250.2.92) 58.793 ms 65.374 ms 58.765 ms
8 as-1.r20.sttlwa01.us.bb.gin.ntt.net (129.250.4.189) 160.802 ms 151.131 ms 157.374 ms
9 ae-1.r05.sttlwa01.us.bb.gin.ntt.net (129.250.5.47) 160.870 ms 170.379 ms 173.939 ms
10 ae13.bbr02.wb01.sea02.networklayer.com (198.104.203.254) 243.300 ms 242.017 ms 220.704 ms
11 ae0.bbr01.eq01.sjc02.networklayer.com (173.192.18.147) 228.356 ms 225.098 ms 241.132 ms
12 ae5.dar01.sr01.sjc01.networklayer.com (173.192.18.249) 235.578 ms 230.343 ms 230.534 ms
13 po1.fcr01.sr01.sjc01.networklayer.com (50.23.118.131) 228.827 ms 249.246 ms 241.976 ms
14 speedtest.sjc01.softlayer.com (50.23.64.58) 249.481 ms 227.156 ms 252.517 ms

December 3rd, 2011 at 12:09am

Isn't it a bit sketchy that SoftLayer uses the name "NetworkLayer" on all its DNS PTR's which are visible in traceroute? I've heard that this is because they are trying to get name recognition and confuse customers who are thinking of nLayer (nlayer.net) who has been around far longer than SoftLayer, ThePlanet, etc.

I've heard many people contact the nlayer.net NOC for SoftLayer issues. Why continue the confusion and just call it softlayer.net?

December 3rd, 2011 at 12:43pm

Fat Cat, early in the evolution of SoftLayer, everything ended in "Layer" to more or less brand that piece of the business for "SoftLayer." The blog was called "The InnerLayer" (http://theinnerlayer.softlayer.com/), the knowledgebase is called "KnowledgeLayer" (http://knowledgelayer.softlayer.com/), the cloud is called CloudLayer (http://www.softlayer.com/cloudlayer/), etc.

I'm not sure who "you've heard" from that the intent was to confuse customers, but you were misinformed. I don't know the future of the networklayer.com paths, but I'd assume that they aren't going away any time soon. Thanks for bringing up your concern, though ... I hadn't heard that was an issue before.

December 3rd, 2011 at 12:55pm

Charles, one of the biggest benefits of the dynamic nature of traceroutes is that they change regularly based on relationships with other providers and the ongoing optimization efforts between our team and our peers. When one of the network engineers dives deeper into what the traceroutes are telling us and how we use them to manage the network, he or she will likely talk more about the route optimization process in detail.

Often, all it takes to "optimize" a route is communication with the origin ISPs to have them recognize the SoftLayer trans-ocean connection as a viable (and likely preferable) route, but in some cases, it's not that easy. If the performance of the connection you're getting suffers, the NOC will jump in to help. Given the physical distance between Hong Kong and San Jose, CA, it'd be tough to shave a significant amount of latency off of 171ms.

Looking at the second traceroute you posted, if you were tracing to a server in the NTT point of presence in Seattle, WA, you'd get about a 170ms response (and that would be getting straight onto their network in Hong Kong and never leaving it ... which is essentially the goal in expanding our network to users around the world). I'm sure the network engineers could tweak the connections to test the differences between using NTT/Global Exhange and the SoftLayer network between Asia and North America, but you might not see the times cut in half like they were in Todd's example from Singapore.

December 3rd, 2011 at 7:45pm

Kevin, I thought the route should be connect to HKIX, as I see Softlayer has a POP in Hong Kong and having a 10G peer with HKIX.

There is a traceroute from another isp in Hong Kong (hkbn.net).

1 5 ms 3 ms 6 ms 014198216001.ctinets.com [14.198.216.1]
2 10 ms 10 ms 15 ms 061093016209.ctinets.com [61.93.16.209]
3 6 ms 11 ms 11 ms 014199254073.ctinets.com [14.199.254.73]
4 5 ms 3 ms 3 ms 014136129106.ctinets.com [14.136.129.106]
5 6 ms 6 ms 6 ms softlayer-10G.hkix.net [202.40.161.241]
6 50 ms 57 ms 50 ms ae0.bbr02.eq01.tok01.networklayer.com [50.97.18.
166]
7 101 ms 50 ms 54 ms ae7.bbr01.eq01.tok01.networklayer.com [50.97.18.
162]
8 167 ms 166 ms 183 ms ae2.bbr02.eq01.sjc02.networklayer.com [50.97.18.
160]
9 166 ms 166 ms 166 ms ae5.dar02.sr01.sjc01.networklayer.com [173.192.1
8.251]
10 169 ms 168 ms 166 ms po2.fcr01.sr01.sjc01.networklayer.com [50.23.118
.133]
11 166 ms 166 ms 166 ms speedtest.sjc01.softlayer.com [50.23.64.58]

However, the traceroute from my SJC server to hkbn still using Any2, not the Softlayer transit.

traceroute to 14.198.222.223 (14.198.222.223), 30 hops max, 40 byte packets
1 50.23.*.*-static.reverse.softlayer.com (50.23.*.*) 0.427 ms 0.539 ms 0.642 ms
2 ae0.dar01.sr01.sjc01.networklayer.com (50.23.118.130) 0.494 ms 0.488 ms 0.478 ms
3 ae8.bbr01.eq01.sjc02.networklayer.com (173.192.18.248) 0.910 ms 0.908 ms 0.890 ms
4 ae7.bbr02.eq01.sjc02.networklayer.com (173.192.18.165) 15.390 ms 15.389 ms 15.372 ms
5 ae0.bbr02.cs01.lax01.networklayer.com (173.192.18.151) 8.584 ms 8.580 ms 8.862 ms
6 ae7.bbr01.cs01.lax01.networklayer.com (173.192.18.166) 8.683 ms 8.679 ms 8.564 ms
7 * * *
8 014136129013.ctinets.com (14.136.129.13) 172.211 ms 172.312 ms 172.304 ms
9 014199254078.ctinets.com (14.199.254.78) 170.538 ms 175.652 ms 175.538 ms
10 061093016210.ctinets.com (61.93.16.210) 175.458 ms 175.768 ms 175.216 ms
11 * * *
12 * * *

December 6th, 2011 at 11:18am

Thanks for the follow-up Charles. I got a little more detail from someone on our networking team about why you're seeing the different routes. We use "hot-potato routing" to best determine the path to a given IP address. If you're going to a SoftLayer server, other ISPs are more likely to give us the "hot potato" quickly because we can get it to our destination quickly (and we will incur the costs in doing so). When you're on the SoftLayer network and you're going to an external IP address, we'll pass the "hot potato" to an ISP that may claim a better route to that destination IP address.

If you ran the traceroute from your server in SJC to our SNG test IP (216.12.196.114) you'd see the traffic stay on our network because the destination is on our network. Also, if you noticed any significant problems with the return route traveling over our peer's network across the Pacific Ocean, our NOC could adjust the routes to ensure consistent performance.

December 7th, 2011 at 4:24pm

Kevin, thanks for your explanation.
It would be nicer to pass through the HKIX from server to Softlayer to the users in Hong Kong, since some of the ISP in Hong Kong do not have sufficient oversea bandwidth.

December 7th, 2011 at 8:55pm

This is great, I love your network but I prefer to colo, are you planning to provide collocation services in the near future?

December 8th, 2011 at 10:17am

No problem at all, Charles. The return traceroute doesn't show which provider brought the traffic back across the ocean (hop 7). Like I said following your first traceroutes, sub-200ms routes from San Jose, CA, to Hong Kong are pretty quick, considering the fact the data is traveling more than 7,000 miles. If you ever run into spikes in latency, drop a ticket in for our NOC, and the network engineers will see what we can do to optimize the routes.

Ianh, our core business is built on a platform that's significantly different from many providers in the industry, so we're not in a hurry to focus on colocation services until we can find a way to innovate in that area of the market. Many of SoftLayer's benefits are made possible by our pod-concept data center design and consistent hardware throughout, so we couldn't guarantee the same kind of performance and functionality for different hardware and configurations. What's most appealing to you about colo?

March 30th, 2012 at 1:59am

I think it would be really helpful if softlayer posted traffic/link utilization graphs. this would really boost confidence in your network if you run your links under utilized.

Reverse traceroute tools would be great also. Sometimes I get horrid speeds when downloading off softlayer, I assume its because softlayer has congested links

April 16th, 2012 at 4:56am

Hmm.

1. Surely you should be asking why the latency jumped from 188ms to 381+ms between lax and dallas on softlayer's network in the pre-launch traceroutes?

12 te1-6.bbr01.cs01.lax01.networklayer.com (66.109.11.42) 186.482 ms 188.265 ms 187.021 ms
13 ae7.bbr01.cs01.lax01.networklayer.com (173.192.18.166) 188.569 ms 191.100 ms 188.736 ms
14 po5.bbr01.eq01.dal01.networklayer.com (173.192.18.140) 381.645 ms 410.052 ms 420.311 ms
15 ae0.dar01.sr01.dal01.networklayer.com (173.192.18.211) 415.379 ms 415.902 ms 418.339 ms

2. If you practice hot potato routing (ie, hand off the packets to someone in dallas rather than in hong kong), doesn't that actually reduce the advantage you have from building a speedy global network? Eyeball -> content traffic will quickly jump onto your uncongested network, but content -> eyeball traffic will also quickly jump onto somebody else's (possibly congested) network... and most traffic is content -> eyeball.

June 24th, 2012 at 6:29am

Would it be possible to find location of server and speed using tracert?

July 23rd, 2012 at 6:54am

Any plans on opening DC in HK and Japan?

Also, who can I talk to if I want our infrastructure in HK to directly connect to SL PoP in HKIX?

July 23rd, 2012 at 2:17pm

As of right now, there are no immediate plans to open a full data center in Hong Kong or Japan. If you need a direct or peering connection with your SoftLayer infrastructure through one of our points of presence, ask to be connected with the network operations team via a support ticket, and they'll have the best insight about whether your specific use-case is possible.

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.