Customer Service Posts

In the beginning butter was churned by hand and water was fetched from a well. However, as we humans take a step forward, we acknowledge that time is of great value and we quickly redefine a process or function to allow for more precious time. Here at SoftLayer, our evolution hasn’t been much different. With each passing day new ideas are brought to the table, discussed and executed; later giving fruit to more success. As part of this fast growing company our Accounting Department has gone through its share of growth and discovery. In the beginning everyone from our Controller to our CFO helped in answering tickets and billed for upgrades. Now that our department has grown larger in size they can tend to more important tasks while we take care of our customers.

It is while executing every day responsibilities that we find new alternatives to simplifying our customer’s experience with us. Alternatives such as automation of Ram and Port Speed upgrades give our customers easier access to business expansion. Bandwidth upgrades no longer take a series of steps to complete. With a few clicks of our mouse we not only upgrade a server’s bandwidth capacity but we have the ability to choose the time in which this upgrade will take effect. In a matter of minutes a customer can move one or several of their servers into their Virtual Dedicated Rack. While this is all taking place their invoice is automatically being updated to show the recent changes and hence bandwidth allotment is immediately brought up to date.

Nevertheless we could not brag about our awesome tools if it weren’t for the genius minds of our friendly neighbors; our Development Department. Every single department within SoftLayer has their share of gratitude towards this great group of people. Here in the Accounting realm, many of our ideas have come to pass because of the effort of these developers. When something happens to go wrong as naturally things do, we turn to a developer for help. Within a matter of minutes a piece of code is revised and our world is back to normal once more. Automation continues to simplify not only our daily jobs but our customer’s path to business success. There is always a better way to do things and you can bet that there is more than one person coming up with the next “big” idea as I write this. Whether it’s redefining a process or writing the code that will make it all happen, innovation is in the hands of all of us here at SoftLayer.

Security is not a thing to be taken lightly. Think about the information that is stored on your server; think about how many months or years worth of data is stored in your databases. Your account information holds a master key to all of this data on your server. This is the very reason this information is protected so closely by the SoftLayer staff.

All companies work very hard to make sure that their products and services are as easy to use as possible. While on the other hand, security works as hard as possible to, seemingly, make the product or service difficult to use. While it is never our intention to make any service difficult to use, it is our intention to make them secure. This is the very reason why, when we are presented with any questions via phone that are sensitive to the operation of your server or account we ask the inconvenient questions to make sure the person on the other end of the phone line is authorized to make the requested changes to the account or the server.

Up to this point this article has not been as light hearted as I had originally intended, but it’s all about being prepared. The point is, everyone deserves a vacation at some point or another (or believes they do), and according to Murphy’s Law, something will inevitably occur that requires immediate attention. When you’re enjoying that time on the beach, your mind a million miles from bits and bytes, and you miles from anything that can be used to properly manage your server or your account an issue can occur.

While you are out, have you made proper provisions to ensure someone can manage your hardware in your place? Your staff may have the passwords for the servers, IP addresses, and may be able to drop your name; but, I assure you this is not enough information for the SoftLayer support staff to submit a ticket, reboot, or log into your server on your behalf. Have you made sure that in a panic situation someone will be able to provide us with the answers to the security questions on the phone? Are you sure whoever is left in charge has been given the proper permissions in our management portal? Making sure these points have been thoroughly covered prior to your vacation, or even leaving for the day, will help you minimize risk while maximizing your beach vacation.

I am a bit of an automotive enthusiast, so when I’m not working, I do spend a fair amount of time browsing automotive websites. I, like many people in the hosting industry, crave information. I like hearing about new design directions, emerging technologies, and past stories about others’ experiences with their vehicles. While browsing, I came across some images of the guts of a BMW that had gone in excess of 60 thousand miles without an engine oil change. Needless to say, the internals were slathered with a gummy sludge and the engine was ruined.

Many technologies we use these days have become so common place and are operationally intuitive enough that we are often able to figure them out and use them without ever having to crack open an owner’s manual. I bring this up because, many technologies in the hosting industry follow suit. There are a number of developers who create software that is designed to make it easy to host websites. They are marketed as the only solution you ever need and, in some cases, imply that all you need to know is how to use a web browser to successfully host websites, not only for oneself but a plethora of other clients too! The servers run themselves, and you only need to spend a few minutes setting your clients up! It’s like free money!

Unfortunately, as the owner of the previously mentioned BMW found out, this is not the case. There are a lot more things going on behind the scenes than just seats and a steering wheel, as are the same with servers. On occasion, we receive support tickets that just say “the site stopped working.” In an attempt to gather more information, we will often ask the client a wide range of questions that help us find the problem faster and come up with the best possible solution. However, sometimes the answer from the customer is, “I haven’t touched or logged into the server in days/ months/ (hopefully not) years.” The more relevant metaphor for this is, “I haven’t changed my BMW’s oil in years!” Servers are like any other complex machine. They require constant maintenance. This includes: updating anti-virus definitions, monitoring bandwidth usage for anomalous spikes, rotating logs out if they are getting too large (provided some other rotation scheme is not already in place), keeping an eye on disk space usage, and creating a disaster recovery plan and backups. So take some time, get to know your server, and familiarize yourself with good preventative maintenance techniques. Your server, your clients, and your BMW (if applicable) will love you for it.

I work in the support department at SoftLayer, and I can tell you that a day of answering phone calls and tickets is not what you might expect. SoftLayer customers are a little different from customers of other companies.

I know from my own experience dealing with cable/internet providers, electric companies, delivery companies, online stores, etc. that the only time I ever call is when I have received sub-standard service and something needs to be fixed. I am usually met with a person on the phone that either does not know the answer to (what seems to me) a simple question or simply does not hide the fact that they don’t care about my issue. I have always chalked this up to a company that has grown to the point that the original people who cared about their company are no longer in touch with the end user (customer). So, I wade through mountains of sludge to get to someone who can actually give me the information I need or maybe even fix my problem. Then, in the case of cable/internet, I wait… and wait… and wait… and…………………. wait for a technician who knows what they are doing to fix the actual problem. And, yes, it has taken three technicians on three different days and a supervisor being called out to my house to fix a problem in the past.

I was just talking to my manager, who related a recent experience in which he ordered the game “Rock Band” along with the Beatles CD from an online seller. The game arrived promptly, but the CD never came. It was shipped via the US postal service, and he simply received notice that they could not deliver it per their policy… What? When did the USPS stop delivering CDs per policy? Anyway, he called the USPS “customer service” and they denied everything up to and including the fact that they are still in the delivery business (huh?). He finally made it to a “supervisor” who denied they had the package, but stated that she would take down his information and have someone call him in two days to tell him what she already knew….that they did not have the package. Wow. That is amazing customer service! The story does end well. He contacted the online seller, who sent another CD immediately via another shipper free of charge. They even sent it overnight and simply asked that he return the first CD should he EVER receive it from the USPS.

SoftLayer is growing at an astronomical rate; and yet, I am proud to say we have maintained consistently superb customer service. As I said above, the only time I call a support line is when I have a problem needing a quick solution, and my unhappiness grows as I receive poor customer service. I continue to be amazed at how happy most of our customers here at SoftLayer are when I answer the phone. Even when they are experiencing a problem, they are generally in good spirits when I speak to them on the phone or reply back and forth through our ticketing system. I find this amazing. Of course, we deal with unhappy customers occasionally. But, on average, we deal with happy customers on a daily basis. I can only attribute this to the fact that they call or submit a ticket with the firm expectation of receiving excellent and timely support. They could only have this expectation because of their past experience with us. I honestly do not call my cable/internet company with any expectation other than long waits on the phone, uninformed support, and days of waiting before a problem is solved. I am NOT happy when I am forced to call them. This is the reason it is actually a pleasant experience to work in the support department at SoftLayer. Besides working around people who know their jobs, I get to speak with customers who are in a good mood even when they are experiencing problems. I love meeting their expectations by resolving whatever issue they may be experiencing quickly and completely—thereby helping them to continue on in great spirits throughout the rest of their day. I know that customers will be happy the next time they call. I would not enjoy working for the USPS customer service line or in the support department of that cable/internet provider as I know I would get nothing but unhappy people with an expectation of poor service. This is one way in which SoftLayer customers are different.

It’s the holiday season, and that means everyone is getting busier. On top of all the existing responsibilities, millions of people are going shopping for gifts, decorating their houses, and navigating the bad weather. On top of all that, many people take their time off during the holiday season!

With this kind of time crunch, it’s best for your business to lie low until after the new year, right? Not so! With all this buying, selling, and giving going on, there’s a lot of extra retail data to process. Plus, it’s the end of the calendar year, many businesses have to get their finances in order too. ALSO, all these newly purchased electronic devices are soon going to be turned on and hooked up to the Internet, where they will almost surely put a new load on your servers.

Systems and network administrators need to be prepared for this influx of new traffic. Sometimes, this means purchasing new servers. However, it’s inefficient to buy the servers so far in advance when you don’t yet know what you will need. It’s best to wait until you’re sure you will need more servers and how many to order. At another hosting company, that would be a problem. People in our industry take the holidays off, too. Lowering the number of sales people and technicians and raising the number of new server requests would normally result in a disaster.

Luckily, SoftLayer does automatic provisioning. As soon as you order your server, it will be provisioned in two to four hours. Day or night, June 3rd or December 31st, if we have it, you can have control over it in two to four hours.

And therein lies the beauty of the SoftLayer system. You don’t have to wait for US to scale your business. If you need another server, get it. When it’s ready, it will automatically be added to your account’s private network and be available to you. You can even automate your server configuration and setup. Depending on the amount of data you need to transfer to a new server, you can have another server up and running your website less than 5 hours from the time you realized you needed it.

In fact, by using the SoftLayer API (and some clever configuration scripts on your servers) you can do live scaling on your website. Using the API, you can provision new servers exactly like the ones you already have. Once they’re available, a script can mirror the configurations from an existing machine to the new machine. Use the SoftLayer API once more to add the new servers to your load balancer rotation, and you’re in business! All without relying on any humans, even yourself! Treat yourself to some R&R this holiday season, while your website continues to get things done for you.

I have been working at SoftLayer for 2 + years now as a CSA and it has been quite the experience! Imagine working at a place where you get to put your hands on the latest technologies, where customers can manage servers as if they were in their own datacenter, and where innovation is a daily norm. Welcome to my job at SoftLayer. I have seen this company grow at an amazing rate, and to whom do we owe the credit? YOU – The customer! Everything that we do, offer and build is a testament to the customers that use our services. This helps make us a forerunner in the industry and allows the customers that use our services to grow and achieve anything that their business requires. I am going to list just a few of my favorite capabilities we offer below:

VPN – The ability to control your server through a private, secure connection and to use our backend services without incurring usage against bandwidth.

IPMI – Having the power of a local console attached and with some cards a virtual dvdrom to install any operating system of your choice.

OS Reloads – We offer several types of operating systems to choose from and keep up to date with the latest versions.

Secondary DNS – You can host your own DNS and allow zone transfers into the SoftLayer Portal and use our resolvers as secondary failovers.

Content Delivery Network – This Feature is awesome as you can deliver your site or video from the closest point to an end user geographically to ensure a great viewing experience.

Support – 24×7 support that truly cares about the customer’s needs. We love what we do and this attitude shows in everything we do.

This is just the tip of the iceberg and barely touches on what we offer our customers. If you are not yet a customer I would strongly encourage you to speak with one of our Sales representatives as they are here and ready to help and will guide you in building the platform you need to get the job done.

I just got back in town from Las Vegas, Nevada. That town is filled with stories and you can really love it or hate it, depending on the hour (or if you are like me whether you are arriving into McCarran or departing). I had a great trip this last go around and actually made money on the tables. However, when they say that what happens in Vegas stays in Vegas they are really talking about your money. Never forget that the house always wins. Always. Even if you win money you’ll wind up spending it on stuff out there and perpetuating your own good time. There isn’t anything wrong with this at all. In fact I plan on coming up on the short side of the stick on both the tables and on simply spending cash when I go out that way.

I think the really interesting thing that happens when you go through “the Vegas experience” is the perceived value of a dollar. You can take it for granted that all of a sudden you are transplanted into this fantasy world that is reminiscent of Pleasure Island from the story of Pinocchio and you’ll find that you have anything and everything you could want to do, eat, drink, or experience right at your fingertips. As this begins to progress the value of a dollar plummets quickly. You start overpaying for things at a whim, tipping bigger, making bolder and even just dumber bets. I did this and I can admit that I doubled down on my 11 when the dealer was showing a 10 in blackjack. It was blind luck that I hit it and won every single time. It’s a bold and stupid bet to make, but when you are playing with house money the money doesn’t matter and it’s almost as if you are trying to give it all back. My game of choice is craps because it gives you the best odds and there is a lot of action. It’s good and bad as it can all come and go in a hurry.

I have only been to Las Vegas a handful of times, but each time there is a point where even for a second you can feel invincible – that you can’t lose. Or, that even if you do lose you won’t even care. The flight home is a completely different story. I call it the hangover flight. You may be literally hung over, but no matter what, you will start to deal with all of the actions that happened on your trip and how you will need to handle them. As soon as you touch down in your own home town things slowly start to become “real” again. Your own home can even feel somewhat foreign for a while, but you’ll quickly come to the realization that you had become a completely different person for a short time.

I have come to the conclusion that there is always risk in everything that we do. Exposing yourself to the tables of Las Vegas may carry more financial risk than your morning commute to work, but in both cases there are still risks. There are also risks that we take in setting and running a business. There are countless ways that you could be putting your business at risk without the right plan in place. From an IT perspective alone, you need to consider things like redundancy, failover, security, backups, growth, and even data loss. Knowing what is going to happen next for your business may be as likely as knowing what is going to come up on the next roll of the dice. If you know this for certain you can press your luck and come up big, but if you are not prepared you could lose everything you have on the table. It is better to be prepared.

I think of SoftLayer as the house, and remember as I said before, the house always wins. The good thing about this is that you are betting with the house. Even with this you need to bet on yourself and back up your own bet. If the bulk of your business is in your data then you need to have backups. If you absolutely need to have High Availability, then look into Clusters and Load Balancing. But remember, that you are betting with the house because SoftLayer gives you the capacity to do all of it and do it all at a very affordable price compared to trying to do it yourself and also do it without long term commitments. Long term commitments bring the most uncertainty in making moves that will positively affect your business. Imagine if a casino told you that you “had” to make 12 consecutive bets regardless of how well (or poorly) you were doing?

Coming home from Las Vegas to SoftLayer has been a very good thing and makes me thankful for where I am and what I have. There aren’t the levels of uncertainty here that are automatic with other datacenters or even other business models. SoftLayer is steady and it is very easy to get what you need here while cutting out the risk that you don’t want to deal with. SoftLayer is as much of a “sure thing” as any bet you can make!

From time to time, I have posted about my frustration with GAAP accounting and traditional credit analysis and how it is not friendly to the hosting business model. For a refresher, click here, here, here, here, and here. By GAHAP, I jokingly mean “generally accepted hosting accounting principles.”

Mike Jones came in my office after a frustrating phone call with a credit analyst. They were trying to talk through collateral possibilities. He told me that the credit analyst has a problem because we carry hardly any accounts receivable. The credit analyst wants something that he can collect in case of default. In GAAP (generally accepted accounting principles), accounts receivable is the total amount that you have billed your customers but have not yet collected from them. Common sense hint: the accounts receivable balance won’t pay your bills – they won’t get paid until you collect the cash.

SoftLayer includes this common sense in its business model. Rather than send out invoices and bug people to pay us later, we choose to have our customers pay us in advance of their use of products and services. Many other hosting companies do the same. There are many advantages to this: we save costs that we would incur collecting the cash, we reduce the amount of abusive accounts that would sign up for a few days of malicious activity and never pay us, and it helps facilitate the on-demand billing side of the cloud computing model.
Again, the disadvantage of this practice comes about when trying to educate a set-in-his-ways credit analyst about our business model. Here is the basic gist of a mythical conversation between a credit analyst and a hosting company:

Credit Analyst: “I see you don’t have any accounts receivable to speak of.”

Hosting Company: “I know! Isn’t that great?”

Credit Analyst: “But if you default, what can I collect?”

Hosting Company: “You’d simply continue to bill the customers for their continued business. Because our customer agreement is month-to-month, you just collect for their next month of service over the next 30 days and you’ve essentially done the same as collect receivables. In fact, that is far easier than collecting past due receivables. We’d be happy place the anticipated next month billing to our customers on the balance sheet in an accounts receivable type of account, but GAAP does not allow this.”

Credit Analyst: “Oh my…you don’t have long term contracts? So all of your customers could leave at once? Isn’t that risky?”

Hosting Company: “We have several thousand customers who trust us with mission critical needs. They will not all leave at once. Our statistics show only a very low percentage of customers terminate services each month. Even through the depths of the recession, we had more new customers joining us than we had customers leaving.”

Credit Analyst: “But conceptually, they could all leave at once since they have no contracts.”

Hosting Company: “That is statistically impossible. The odds of that event are so low that it’s immeasurable. As I said, we provide mission critical services to our customers. To think that they will all no longer need these services simultaneously is paranoid. And if they did, would a contract keep them paying us? That’s doubtful. Let me ask you – do you lend to the electric company or the phone company?”

Credit Analyst: “Of course.”

Hosting Company: “Do their customers sign long term contracts?”

Credit Analyst: “Some do for special promotions. But for the most part – no.”

Hosting Company: “So why do you lend to them?”

Credit Analyst: “Why, the customers can’t live without electricity or phones. That’s a no brainer.”

Hosting Company: “It is exactly the same with our business. In this information age economy, our customers cannot live without the hosting services that we provide. You should look at us in a similar way that you look at a utility company.”

Credit Analyst: “But we classify your business as a technology company. Can’t you just have your customers sign contracts?”

Hosting Company: “Well, wouldn’t that conflict with the on-demand, measured billing aspects of cloud computing?”

Credit Analyst: “I guess there’s not much hope of you building up a sizeable accounts receivable balance then.”

Hosting Company: “It really makes no sense for us to do that.”

Credit Analyst: “We may not be able to do business with you. Do you have any real estate?”

Conclusion: Most credit analysts are so wrapped up in GAAP that they’ve forgotten the laws of statistics and many have even lost touch with common sense. Is it any wonder we’ve had a big banking crisis over the past couple of years?

Some mornings after work when the weather is nice I’ll go to a local coffee shop on the way home to read or study for the CCNA exams. Sometimes I’ll just end up pulling out the netbook and browse around online. There are times during these outings when I’ll get asked the title question of this blog: is that a real computer? I guess the size that throws people but the answer is yes.

For those who are not familiar with the netbook class of systems here are the specs for mine:

• 10.2 inch screen
• 1 GB RAM
• 1.6GHz Intel Atom processor
• 160GB SATA hard drive
• 3 USB ports
• Card reader
• Built-in Wifi
• Built-in webcam
• Windows XP (I’ve got plans for Windows 7)
• 5 hour battery life
• Light weight (I’ve got books that weigh more)

Netbooks are great for when you’re just knocking around town and might want to do some light web work. This morning while at Starbucks I’ve checked e-mail several times, caught up on the daily news, and reviewed the game statistics from the Cowboys game I missed last night. Other mornings I’ve fired up a VPN connection into the office and been able to remotely help with tickets, work on documentation for our SSL product and tinker around with a NetScaler VPX Express virtual machine (an interesting bit of tech for a later article).

So how does this tie into server hosting?

You’ve probably had a time when your monitoring has indicated a service ceasing to respond on a server. If all you have is a cell phone the options are somewhat limited. With a fancy enough phone you might have an SSH or RDP client but do you really want to do anything on a PDA sized screen? I didn’t think so. You can put in a ticket from your phone and our support can help out but the person best able to fix a service failure is still going to be you, the server administrator who knows where all the bodies are buried and how the bits tie together.

A small netbook can be a lightweight (and inexpensive) administration terminal for your servers hosted with us. Just find an Internet connection, connect up to the SoftLayer VPN and now you have complete access to work on your servers via a secure connection.

Through the wonders of the IPMI KVM this access even includes the console which opens up the possibility of doing a custom kernel build and install safely, while sitting under the stars, drinking a hot chocolate and watching the local nightlife.

Sounds like a pretty nice reality to me.

I recently read an article about an endeavor that is currently being undertaken to develop a “Speech Analysis Algorithm Crafted to Detect and Help Dissatisfied Customers”. In short, a team of engineers are hoping to create software that will recognize when a caller is becoming stressed and immediately phone a manager to alert them of a developing situation. Wow! It is rare that you would see math and science applied to something that is so subjective. After all, math is used to quantify and measure things all based on a known or a baseline. In this particular effort, I would surmise that the team of engineer’s most difficult task will be to determine how to establish a unique baseline for each unique call and caller. Once upon a time as a student of Electrical Engineering, I took on my share of convolution integrals and that’s a path that I do not care to venture down again. I’ve also taken on my share of convoluted customer calls in a past life and witnessed our frontline assisting customers in complex situations here at SoftLayer.

Until there is such an application that can detect and address a conversation that may be heading in the wrong direction, we have to rely on good ole’ training and experience. With each call and query, the baseline is reset. I’d even go further to say that with each exchange; the baseline is reset as our Customer Service Agents seek information to get to the root of the issue. It’s not hard to imagine the frustration that can build in a back-and-forth conversation as two people look to come to a solution or an amiable conclusion just as it is understandable that sometimes, a customer may simply need to vent. How do you calculate and anticipate those scenarios?

I wish much success to the team involved in the customer service speech analysis program. And programmatically speaking, I see many CASE, SWITCH, FOR, WHILE, BREAK, CONTINUE, IF, ELSE, ELSE IF, NEXT statements in your future. Good Luck!