Posts Tagged 'Support'

October 16, 2013

Tips and Tricks: Troubleshooting Email Issues

Working in support, one of the most common issues we troubleshoot is a customer's ability to receive email. Depending on email server, this can be a headache and a half to figure out, but more often than not, we're able to fix the problem with one of only a few simple solutions. Because the SoftLayer Blog audience loves technical tips and tricks, I thought I'd share a few easy steps that make pinpointing the root cause of email issues much easier.

Before you gear up to go into battle, check the that server is not out of disk space on /var and that it is not in a read only state. That precursory step may seem silly, but Occam's Razor often holds true in technical troubleshooting. Once you verify that those two common problems aren't causing your email problems, the next step is to determine whether the email issues are server-wide or isolated to one mail account/domain. To do that, the first thing you need to do is make sure that the IMAP and POP services are responding.

Check IMAP and POP Services

The universal approach to checking IMAP and POP services is to use telnet:

telnet <serverip> 110
telnet <serverip> 143

If either of those commands fail, you're able to pinpoint which service to check on your server.

For most variants of Linux, you can check both services with a single command: netstat -plan|egrep -i "110|143". The resulting output will show if the services are listening and which process is doing the listening. In Windows, you can run a similar command from a command prompt: netstat -anb|find "LISTEN"| findstr "110 143".

If the ports are listening, and you're able to connect to them over telnet, your next stop should be your server's error logs.

Check Error Logs

You want to look for any mail errors that might clue you into the root cause of your email issues. In Linux, you can check /var/log/maillog, and in Windows, you can filter eventvwr.msc for mail only. If there are errors, a simple search will highlight them quickly.

If there are no errors, it's time to dig into the mail queue directly.

Check the Mail Queue

Depending on the mail server you use, the commands here are going to vary. Here are a few examples of how we'd investigate the most common mail servers we encounter:

QMail

Display the mail queue: /var/qmail/bin/qmail-qread
Display the number of messages in the queue: /var/qmail/bin/qmail-qstat
Reference article: Gaining Control Over the QMail Queue

Sendmail

Display the mail queue: sendmail -bp or mailq
Display the number of messages in the queue: mailq –OmaxQueueRunSize=1
Reference article: Quick Sendmail Cheatsheet

Exim

Display the mail queue: exim -bp
Display the number of messages in the queue: exim -bpc
Reference article: Exim cheatsheet

MailEnable

MailEnable users can can check to see that messages are moving by opening the mail directory:
Program Files\MailEnable\Queues\SMTP\Inbound\Messages
Reference article: How to diagnose inbound message delivery delays

With these commands, you can filter through the email queues to see whether any of them are for the users or domains you're having problems with. If nothing obvious presents itself at that point, it's time for some active testing.

Active Testing

Send an email to your mailserver from an external mailserver (anything will do as long as it's not on the same server). Watch for logging of the email as it's delivered:
tail -f maillog
On busy mailservers you might add |grep youremailid or simply look for a new message in the directory where the email will be stored.

The your primary goal in troubleshooting your email issues in this way is to isolate the root cause of your problem so that you can fix it more quickly. SoftLayer customers have direct access to our support team to help you through this process, but it's always nice to keep a quick reference like this in your back pocket to be able to pinpoint the problem yourself.

-Bill

April 15, 2013

The Heart of SoftLayer: People

When I started working for SoftLayer as a software engineer intern, I was skeptical about the company's culture. I read many of the culture posts on the blog, and while they seemed genuine, I was still a little worried about what the work atmosphere would be for a lowly summer intern. Fast-forward almost a year, and I look back on my early concerns and laugh ... I learned quickly that the real heart of SoftLayer is its employees, and the day-to-day operations I observed in the office consistently reinforced that principle.

It's easy to think about SoftLayer as a pure technology company. We provide infrastructure as a service capabilities for businesses with on-demand provisioning and short-term contracts. Our data centers, portal, network and APIs get the spotlight, but those differentiators wouldn't exist without the teams of employees that keep improving them on a daily basis. By focusing on the company culture and making sure employees are being challenged (but not overwhelmed), SoftLayer was indirectly improving the infrastructure we provide to customers.

When I walked into the office for my first day of work, I imagined that I'd be working in a cramped, dimly lit room in the back of the building where I'd be using hand-me-down hardware. When I was led to a good-sized, well-lit room and given a Core i3 laptop with two large monitors and a full suite of software, I started realizing how silly my worries were. I had access to the fully stocked break room, and within about a week, I felt like part of a community rather than a stale workplace.

My coworkers not only made me feel welcome but would frequently go out of their way to make sure I am comfortable and have the resources I needed to succeed. While the sheer amount of new information and existing code was daunting, managers assigned projects that were possible to complete and educational. I was doing useful work building and improving a complex production system rather than the busy work offered by many other employers' internship programs. I learned several new techniques and solidified my understanding of software engineering theory through practice. The open-door policy and friendly people around me not only created a strong sense of community but also allowed more efficient problem solving.

You may have noticed early in this post that I joined the company on a summer internship and that I also told you it's been about a year since I started. While summers in Texas feel long, they don't actually last a full year ... After my internship, I was offered a part-time position as a software engineer, and I'm going to be full-time when I graduate in May.

It's next to impossible to find a company that realizes the importance of its employees and wants to provide an environment for employees to succeed. The undeniable runaway success of the company is proof that SoftLayer's approach to taking care of employees is working.

-John

March 5, 2013

Startup Series: Kickback Tickets

The very first client I recruited to Catalyst when I joined the CommDev team about a year ago happens to be one of Catalyst's most interesting customer success stories ... and I'm not just saying that because it was the first partner I signed on. Kickback Tickets — an online ticketing platform that utilized crowdfunding — has simplified the process of creating and funding amazing events, and as a result, they've made life a lot easier for the startup, developer and networking organizations that fuel Catalsyt.

Anyone who's organized events knows that it often involves a financial risk because it's hard to know whether the event will be well-enough attended to cover the costs of putting on the event. With Kickback Tickets, an event is listed an funded ahead of time, and when it reaches its "Tipping Point" goal of tickets ordered, it's completely funded, the early supporters are charged, and the ticket sales continue.

The process is simple:

Kickback Tickets

Event updates, guest registrations and QR-coded tickets are provided to attendees to make check-in seamless, so the hosts of each event don't have hassle with those details. Kickback's revenue comes from a small fee on each ticket for each successfully funded event, and they've got a ton of momentum. After signing on with Catalyst in March 2012, Kickback went live with an open beta in November 2012, and they launched their out-of-beta site in February 2013. They've successfully funded more than 20 events, and new events are added daily.

Kickback Tickets

When I met the Kickback founders Jonathan Perkins and Julian Balderas, I was attending SF Beta (my first official event as a SLayer). At the time, Jonathan and Julian were a couple of bankers with an innovative idea to help organizations alleviate the financial risk of planning and putting on events by enlisting community support. I told them about my experience as the COO of a small non-profit startup up called Slavery Footprint (also a Catalyst partner), and I guess they could relate to the challenges SoftLayer helped us overcome because they were excited to join.

In their own words, Jonathan and Julian explain that their partnership with Softlayer and the Catalyst program has been extremely valuable:

SoftLayer provides a rock-solid technical foundation and allows us to focus more resources on business development. On the technical side, what Softlayer offers is impressive — super fast speeds and an intricate level of control over the hardware. On the personal side, the mentorship and networking benefits of the program have been very helpful. We've always found the Catalyst team to be available to chat about any questions we had, ranging from development to biz dev to fundraising.

As they continue to expand their platform, it's going to be exciting to watch Kickback become a true force in the events space. Organize your next event with Kickback and make sure it's a success.

Oh, and if you want to speak to Jonathan and Julian, just reach out to me and I'll happily make the introduction.

-@JoshuaKrammes

November 20, 2012

Community Development: Catalysing European Startups

SoftLayer works hard and plays hard. A few weeks ago, I traveled to Dallas for the first "Global Catalyst Summit"* where the community development teams in Europe, Asia and the United States all came together under one roof to learn, strategize and bond. What that really means is that we all experienced a week of hardcore information flow and brutal fun.

The onboarding process to become a part of the SoftLayer's Community Development (Catalyst) team is pretty rigorous, and traveling to Dallas from Amsterdam for the training made it even more intense. In short order, I learned about the roots of the Catalyst program and why SoftLayer is so interested in investing in helping startups succeed. I got the low-down on the hundreds of companies that are taking advantage of the program right now, and I was inspired by the six incredible people who focus exclusively on the Catalyst program at SoftLayer ... And Big Tex:

SoftLayer Community Development Team and Big Tex

When the whirlwind week of orientation and training came to an end, I came to a solid conclusion: I am working at SoftLayer for a reason. I believe SoftLayer has the most kick-ass global on-demand technology platform out there, and our focus on innovation and automation is reflected in everything we do. On top of that, we give that platform to startups to help springboard their success. I get to work with a community of world-changers. Needless to say, that's an amazing conclusion to come to.

As a member of the Catalyst team in EMEA (Europe, Middle East, Africa), I can provide signficant resources to entrepreneurs who are building awesome new applications and technologies that are making a difference locally, regionally and globally. Anna Bofill Bert and I work out of SoftLayer's Amsterdam office, and we are fully dedicated to helping startup and developer communities in our region.

As a review exercise and a way to educate the audience that may be unfamiliar with Catalyst, I thought I'd bullet out a few of the main ideas:

What is Catalyst?

The SoftLayer Catalyst Startup Program provides:

  • A generous monthly hosting credit toward dedicated, cloud or hybrid compute environments for a FULL YEAR (Ideal for dev-ops/next generation startup compute applications who want high performance from the start).
  • Direct connection to highest level programming team at SoftLayer — Our Innovation Team. Participating companies get help and advice from the people that are writing the book on highly scalable, global infrastructure environments.
  • Connection to the SoftLayer Marketing and PR Team for help getting spreading the word around the world about all the cool stuff participating startups are doing.

We reach startups by listening to them and meeting needs that all of them express. We are telling the SoftLayer story, networking, making friends, drinking too much and travelling like mad. In the course of a month, we went to Lean Start Up Machine in Rotterdam, Structure Europe in Amsterdam, Pioneers Festival in Vienna, HowToWeb in Bucharest and we managed to complete a quick tour of startup communities in Spain.

Like our peers on the US team, we partner with incubators and accelerators to make sure that when startups look for help getting started, they also find SoftLayer. We're already working with partners like Springboard, Seedcamp, GameFounders, Startup Sauna, the INLEA Foundation and Tetuan Valley, and the list of supported communities seems to grow daily. When the portfolio companies in each of these organizations are given access to the Catalyst program, that means SoftLayer's Catalyst customer base is growing pretty phenomenally as well.

What I actually like most about how we help startups is the mentorship and office hours we provide participating companies as well. SoftLayer was founded by ten guys in a living room in 2005, and we've got hundreds of millions of dollars in annual revenue as of 2012. That success is what the SoftLayer team is excited to share insights about.

Hustling is a major part of startup culture, so it's only fitting that I feel like I had to hustle through this blog to get all of my thoughts down. Given that SoftLayer EMEA is a bit of a startup itself, I'm happy to be practicing what we preach. If you'd like more information about Catalyst or you want to apply, please feel free to hit me up: esampson@softlayer.com

We want to be part of your company's success story.

-@EmilyBlitz

*Note: As an homage to Big Tex after the fire, we referred to our meeting as the "Global Catalyst Summit with Big Tex" at the Texas State Fair. We hope to see you back in action in 2013, Big Tex!

October 25, 2012

Tips from the Abuse Department: Save Your Sinking Ship

I often find that the easiest way to present a complex process is with a relatable analogy. By replacing esoteric technical details with a less intimidating real-world illustration, smart people don't have to be technically savvy to understand what's going on. When it comes to explaining abuse-related topics, I find analogies especially helpful. One that I'm particularly keen on in explaining Abuse tickets in the context of a sinking ship.

How many times have you received an Abuse ticket and responded to the issue by suspending what appears to be the culprit account? You provide an update in the ticket, letting our team know that you've "taken care of the problem," and you consider it resolved. A few moments later, the ticket is updated on our end, and an abuse administrator is asking follow-up questions: "How did the issue occur?" "What did you do to resolve the issue?" "What steps are being taken to secure the server in order to prevent further abuse?"

Who cares how the issue happened if it's resolved now, right? Didn't I respond quickly and address the problem in the ticket? What gives? Well, dear readers, it's analogy time:

You're sailing along in a boat filled with important goods, and the craft suddenly begins to take on water. It's not readily apparent where the water is coming from, but you have a trusty bucket that you fill with the water in the boat and toss over the side. When you toss out all the water onboard, is the problem fixed? Perhaps. Perhaps not.

You don't see evidence of the problem anymore, but as you continue along your way, your vessel might start riding lower and lower in the water — jeopardizing yourself and your shipment. If you were to search for the cause of the water intake and take steps to patch it, the boat would be in a much better condition to deliver you and your cargo safely to your destination.

In the same way that a hull breach can sink a ship, so too can a security hole on your server cause problems for your (and your clients') data. In the last installment of "Tips from the Abuse Department," Andrew explained some of the extremely common (and often overlooked) ways servers are compromised and used maliciously. As he mentioned in his post, Abuse tickets are, in many cases, the first notification for many of our customers that "something's wrong."

At a crucial point like this, it's important to get the water out of the boat AND prevent the vessel from taking on any more water. You won't be sailing smoothly unless both are done as quickly as possible.

Let's look at an example of what thorough response to an Abuse ticket might look like:

A long-time client of yours hosts their small business site on one of your servers. You are notified by Abuse that malware is being distributed from a random folder on their domain. You could suspend the domain and be "done" with the issue, but that long-time client (who's not in the business of malware distribution) would suffer. You decide to dig deeper.

After temporarily suspending the account to stop any further malware distribution, you log into the server and track down the file and what permissions it has. You look through access logs and discover that the file was uploaded via FTP just yesterday from an IP in another country. With this IP information, you search your logs and find several other instances where suspicious files were uploaded around the same time, and you see that several FTP brute force attempts were made against the server.

You know what happened: Someone (or something) scanned the server and attempted to break into the domain. When the server was breached, malware was uploaded to an obscure directory on the domain where the domain owners might not notice it.

With this information in hand, you can take steps to protect your clients and the server itself. The first step might be to implement a password policy that would make guessing passwords very difficult. Next, you might add a rule within your FTP configuration to block continued access after a certain number of failed logins. Finally, you would clean the malicious content from the server, reset the compromised passwords, and unsuspend the now-clean site.

While it's quite a bit more work than simply identifying the domain and account responsible for the abuse and suspending it, the extra time you spent investigating the cause of the issue will prevent the same issue from happening after your client "fixes" the problem by deleting the files/directories. Invariably, they'd get compromised again in the same way when the domain is restored, and you'd hear from the Abuse department again.

Server security goes hand in hand with systems administration, and even though it's not a very fun part of the job, it is a 24/7 responsibility that requires diligence and vigilance. By investing time and effort into securing your servers and fixing your hull breach rather than just bailing water overboard, your customers will see less downtime, you'll be using your server resources more efficiently, and (best of all) you won't have the Abuse team hounding you about more issues!

-Garrett

P.S. I came up with a brilliant analogy about DNS and the postal service, so that might be a topic for my next post ...

September 17, 2012

Joining the Internet Infrastructure Coalition

In January, we posted a series of blogs about legislation in the U.S. House of Representatives and Senate that would have had a serious impact on the hosting industry. We talked about SOPA and PIPA, and how those proposed laws would "break the Internet" as we know it. The hosting industry rallied together to oppose the passage of those bills, and in doing so, we proved to be a powerful collective force.

In the months that followed the shelving of SOPA and PIPA, many of the hosting companies that were active in the fight were invited to join a new coalition that would focus on proposed legislation that affects Internet infrastructure providers ... The Internet Infrastructure Coalition (or "i2Coalition") was born. i2Coalition co-founder and Board Chair Christian Dawson explains the basics:

SoftLayer is proud to be a Charter Member of i2Coalition, and we're excited to see how many vendors, partners, peers and competitors have joined us. Scrolling the ranks of founding members is a veritable "Who's who?" of the companies that make up the "nuts and bolts" of the Internet.

The goal of i2Coalition is to facilitate public policy education and advocacy, develop market-driven standards formed by consensus and give the industry a unified voice. On the i2Coalition's Public Policy page, that larger goal is broken down into focused priorities, with the first being

"In all public policy initiatives of the i2Coalition will be to encourage the growth and development of the Internet infrastructure industry and to protect the interests of members of the Coalition consistent with this development."

Another huge priority worth noting is the focus on enabling and promoting the free exercise of human rights — including freedom of speech, freedom of assembly and the protection of personal privacy. Those rights are essential to fostering effective Internet advancement and to maintain a free and open Internet, and SoftLayer is a strong supporter of that platform.

If you operate in the hosting or Internet infrastructure space and you want to be part of the i2Coalition, we encourage you to become a member and join the conversation. When policymakers are talking about getting "an Internet" from their staff members, we know that there are plenty of opportunities to educate and provide context on the technical requirements and challenges that would result from proposed legislation, and the Internet Infrastructure Coalition is well equipped to capitalize on those opportunities.

-@toddmitchell

August 15, 2012

Managing Support Tickets: Email Subscriptions

This week, the development team rolled out some behind-the-scenes support functionality that I think a lot of our customers will want to take advantage of, so I put together this quick blog post to spread the word about it. With the new release, the support department is able to create "Ticket Email Subscriptions" for different ticket groups on every customer account. As a customer, you might not be jumping up and down with joy after reading that one-sentence description, but after you hear a little more about the functionality, if you're not clapping, I hope you'll at least give us a thumbs-up.

To understand the utility of the new ticket email subscription functionality, let's look at how normal tickets work in the SoftLayer portal without email subscriptions:

User Creates Ticket

  1. User A creates a ticket.
  2. User A becomes the owner of that ticket.
  3. When SoftLayer responds to the ticket, an email notification is sent to User A to let him/her know that the ticket has been updated.

SoftLayer Creates Ticket

  1. SoftLayer team creates a ticket on a customer's account.
  2. The primary customer contact on the account is notified of the new ticket.
  3. Customer logs into the portal and responds to ticket.
  4. Customer gets notifications of updates (as described above).

There's nothing wrong with the existing support notification process, but that doesn't mean there aren't ways to make the process better. What if User A creates an urgent ticket on his/her way out the door to go on vacation? User B and User C aren't notified when an update is posted on User A's ticket, so the other users aren't able to get to the ticket and respond as quickly as they would have if they received the notification. What if the primary customer contact on the account isn't the best person to receive a monitoring alert? The administrator who will investigate the monitoring alert has to see the new ticket on the account or hear about it from the primary contact (who got the notification).

Ticket email subscriptions allow for customers to set contact addresses to be notified when a ticket is created, edited or moved in a particular ticket group. Here are the ticket groups differentiated in our initial release:

  • Billing - Any ticket in our Billing department
  • Maintenance - Scheduled maintenance notifications for specific servers
  • Network Protection - DDoS mitigation and Null Routes
  • Monitoring - Host Down Alerts
  • CST, SysAdmin and Hardware - Any ticket in our support and data center departments
  • Managed Services - Tickets that relate to any managed services
  • Network Maintenance - Scheduled network maintenance

You'll notice that Abuse isn't included in this list, and the only reason it's omitted is because you've always been able to designate a contact on your account for abuse-related tickets ... Ticket subscriptions extend that functionality to other ticket groups.

Because only one email address can be "subscribed" to notifications in each ticket group, we recommend that customers use their own distribution lists as the email contacts. With a DL as the contact, you can enable multiple users in your organization to receive notifications, and you can add and remove users from each distribution list on your end quickly and easily.

When User A creates a ticket with the data center and goes on vacation, as soon as SoftLayer responds to the ticket, User A will be notified (as usual), and the supportsubscription@yourdomain.com distribution will get notified as well. When a network maintenance is ticket is created by SoftLayer, the netmaintsubscription@yourdomain.com distribution will be notified.

Ticket email subscriptions are additive to the current update notification structure, and they are optional. If you want to set up ticket email subscriptions on your account, create a ticket for the support department and provide us with the email addresses you'd like to subscribe to each of the ticket groups.

We hope this tool helps provide an even better customer experience for you ... If you don't mind, I'm going to head back to the lab to work with the dev team to cook up more ways to add flexibility and improvements into the customer experience.

-Chris

August 6, 2012

SoftLayer in the Community - Tour de Pink 2012

Every year, SoftLayer commits to raising money and giving support to a number of charities, and SLayers are all encouraged to submit the organizations and causes that are important to them. Not long after coming to work here, I found myself in a position to pitch one of my favorite charities — Tour de Pink — to Lance and the charity team.

Tour de Pink is one of the major fundraising efforts for The Pink Ribbons Project, a Houston based organization that raises money to fight breast cancer through awareness and educational outreach initiatives. The Pink Ribbons Project supports proper screening for the medically under-served and under-insured population in the Greater Houston Area, and Tour de Pink is the first bike ride in Texas solely benefiting breast cancer awareness and education.

I have been involved with the ride since its inception in 2005, and I manage the logistical support for all of the Pink Pit Stops. The first year of the ride, "support" consisted of me and a guy named Bear, my 1995 Ford Ranger pickup truck, a 25' moving truck with a lift, and 400 pounds of ice. By 2011, we had grown the logistics team to nine dedicated people, four route vans, a roamer and 4000 pounds of ice to support the 2000+ riders traveling seven routes.

Last year was Tour de Pink's seventh, and an opportunity opened up for a company to step in as the presenting sponsor for the ride ... After about six months of official employment with SoftLayer, I knew one thing for sure: If you have an idea, a plan or a cause that matters to you, it's your responsibility to take that idea / plan / cause wherever it needs to go to get addressed — whether it's an opportunity to improve a compliance process or a community cause. I stepped up and brought the idea to SoftLayer's CEO.

In true SLayer fashion, he saw how important the cause was to me, and he quickly commitment SoftLayer's support to the 2011 Tour de Pink.

In addition to the a financial commitment, we provided space in our downtown Houston offices for packet stuffing:

Tour de Pink

And the (infamous?) 3-Bars BBQ team towed the smoker down to Houston to cook up some fine "Q" for the annual Tour de Pink Kickoff Party:

Tour de Pink

SoftLayer VP of Business Applications Development DJ Harris even kicked off the opening ceremonies when the ride rolled around!

After an extremely successful 2011, SoftLayer has extended support for Tour de Pink to 2012! This year's ride is scheduled for September 16, and it will starting from and coming back to the Prairie View A&M University campus. While SoftLayer is the major underwriter of this ride, it's still a fundraiser, and that's where the rest of us come in. The monies that go out into the community are raised through registration of individual riders and teams and from their collective fundraising efforts.

If you want to roll with the cool kids (and believe me, SoftLayer IS cool) and you plan on being in the Houston area mid-September, surf on over to www.tourdepink.org and sign up to join us!

I hope to see some of you out on the ride, but until then, may the wind be always at your back ... and 3-Bars for Life!

-Val

Categories: 
July 26, 2012

Global IP Addresses - What Are They and How Do They Work?

SoftLayer recently released "Global IPs" to a good amount of internal fanfare, and I thought I'd share a little about it with the blog audience in case customers have questions about what Global IPs are and how they work. Simply put, Global IP addresses can be provisioned in any data center on the SoftLayer network and moved to another facility if necessary. You can point it to a server in Dallas, and if you need to perform maintenance on the server in Dallas, you can move the IP address to a server in Amsterdam to seamlessly (and almost immediately) transition your traffic. If you spin up and turn down workloads on cloud computing instances, you have the ability to maintain and a specific IP address when you completely turn down an environment, and you can quickly reprovision the IP on a new instance when you spin up the next workload.

How Do Global IPs Work?

The basics of how the Internet works are simple: Packets are sent between you and a server somewhere based on the location of the content you've requested. That location is pinpointed by an IP address that is assigned to a specific server or cloud. Often for various reasons, blocks of IP addresses are provisioned in one region or location, so Global IPs are a bit of a departure from the norm.

When you're sending/receiving packets, you might thing the packets "know" the exact physical destination as soon as they're directed to an IP address, but in practice, they don't have to ... The packets are forwarded along a path of devices with a general idea of where the exact location will be, but the primary concern of each device is to get the all packets to the next hop in the network path as quickly as possible by using default routes and routing tables. As an example, let's follow a packet as it comes from an external webserver and detail how it gets back to your machine:

  1. The external webserver sends the packet to a local switch.
  2. The switch passes it to a router.
  3. The packet traverses a number of network hops (other routers) and enters the Softlayer network at one of the backbone routers (BBR).
  4. The BBR looks at the IP destination and compares it to a table shared and updated with the other routers on SoftLayer's network, and it locates the subnet the IP belongs to.
  5. The BBR determines behind which distribution aggregate router (DAR) the IP is located, then it to the closest BBR to that DAR.
  6. The DAR gets the packet, looks at its own tables, and finds the front-end customer router (FCR) that the subnet lives on, and sends it there.
  7. The FCR routes the packet to the front-end customer switch (FCS) that has that IP mapped to the proper MAC address.
  8. The switch then delivers the packet through the proper switchport.
  9. Your server gets the packet from the FCS, and the kernel goes, "Oh yes, that IP on the public port, I'll accept this now."

All of those steps happen in an instant, and for you to be reading this blog, the packets carrying this content would have followed a similar pattern to the browser on your computer.

The process is slightly different when it comes to Global IP addresses. When a packet is destined for a Global IP, as soon as it gets onto the SoftLayer network (step 4 above), the routing process changes.

We allocate subnets of IP addresses specifically to the Global IP address pool, and we tell all the BBRs that these IPs are special. When you order a global IP, we peel off one of those IPs and add a static route to your chosen server's IP address, and then tell all the BBRs that route. Rather than the server's IP being an endpoint, the network is expecting your server to act as a router, and do something with the packet when it is received. I know that could sound a little confusing since we aren't really using the server as a router, so let's follow a packet to your Global IP (following the first three steps from above):

  1. The BBR notes that this IP belongs to one of the special Global IP address subnets, and matches the destination IP with the static route to the destination server you chose when you provisioned the Global IP.
  2. The BBR forwards the packet to the DAR, which then finds the FCR, then hands it off to the switch.
  3. The switch hands the packet to your server, and your server accepts it on the public interface like a regular secondary IP.
  4. Your server then essentially "routes" the packet to an IP address on itself.

Because the Global IP address can be moved to different servers in different locations, whenever you change the destination IP, the static route is updated in our routing table quickly. Because the change is happening exclusively on SoftLayer's infrastructure, you don't have to wait on other providers propagate the change. Think of updating your site's domain to a new IP address via DNS as an example: Even after you update your authoritative DNS servers, you have to wait for your users' DNS servers to recognize and update the new IP address. With Global IPs, the IP address would remain the same, and all users will follow the new path as soon as the routers update.

This initial release of Global IP addresses is just the tip of the iceberg when it comes to functionality. The product management and network engineering teams are getting customer feedback and creating roadmaps for the future of the product, so we'd love to hear your feedback and questions. If you want a little more in-depth information about installation and provisioning, check out the Global IP Addresses page on KnowledgeLayer.

-Jason

July 6, 2012

My Advice to Myself (A New Server Build Technician)

When I started at SoftLayer, I had no idea what to expect. As I walked from the parking lot to the front doors at SJC01, I started to get nervous ... I felt was like I was stepping onto a stage, and I was worried about making a mistake. I took a deep breath and walked in.

Now that I look back on my first day (which was about a month ago), I have to laugh at my nervousness. I'm not sure what I expected to encounter, but the environment I entered was probably the most welcoming and friendly I've ever seen. Two of my coworkers, Cuong and Jonathan, recently shared their experiences as SBTs in San Jose, but because I have some recent first-hand experience that's still fresh in my mind, I thought I'd share my own perspective.

If I were able to talk to myself as I nervously approached the San Jose data center on my first day, this is what I'd say:

As you'd expect from any new job, your first day at work involves a lot of learning (and paperwork). You're probably chomping at the bit to get out into the data center to start building servers, but you need to crawl before you walk. The first thing you need to do is get the lay of the land ... You get a guided tour of the office, the data center and your workspace. Even if you've worked in a data center before, you're going to be surprised and impressed with how everything is set up. Once all of your paperwork is in order, you start learning about SoftLayer's business and how you contribute to the customer experience. Once you understand the big picture, you can get into the details.

You're given a training guide that goes over many of the processes and procedures that are followed on a day-to-day basis in the data center, and you're shown all of the components you'll be working with as you build, upgrade and manage server hardware. You might not be performing much work on hardware in production in your first few days, but you're going to learn a lot and have plenty of time to ask questions. While you're learning how to perform your work tasks, you're building friendships with your coworkers, and you're officially becoming part of the SoftLayer family. Your fellow SLayers support you and help you make sure SoftLayer's customers are getting the service they expect.

You're taught everything you need to know, from staying organized and focused to best practices around working with servers. You have nothing to be nervous about.

I've only been with SoftLayer for a short period of time, but I can confidently say that working here is remarkable. I don't feel like an "employee;" I feel like a team player. I feel like everyone is on the same page about what needs to be done in the data center, and whenever questions come up, answers are given quickly.

I'm excited to come to work every day. I would have never dreamed I'd feel this way because I was always told jobs are long and drag-out boring, but my experience has been the polar opposite. Now, When any of my friends complain about getting up and going to work, I recommend they visit http://www.softlayer.com/about/careers.

-Jackie

Subscribe to support