news

September 9, 2009

Taking the Chance

I started working at the ripe’ole age of 16 and since then I’ve had 5 jobs including SoftLayer. I started off at “bullseye” which consisted of straightening merchandise shelves and onto being a cashier. For my second job, I moved on to harassing customers into purchasing leather from a well known mall leather supplier, but for some reason I was having extra-long chili cheese conies and bacon, egg and cheese toasters on my mind, so I made the move to “rollerskates” for job number three. These jobs gave me a decent income (for being 16-18 yrs old), but I knew I didn’t want to be hopping from job to job for the rest of my life… I needed to find a place to grow roots, a place that paid me what I was worth, and finally a place where growth within the company was available. So, I moved to the world of retail digital imaging (large format printing) with a local Dallas company. I started in the shipping and receiving department, slowly learning the whole production side of the company as I knew that is where I could grow. Four long years passed, roots in the company were set fairly deep, or so I thought and the opportunity was “kinda” there, but ultimately I was not happy.

The opportunity for me to work at SL came thru a friend and former colleague at the printing company, Shawna (thanks Shawna!) who left because she also saw the huge potential with SL. At the time I was so ready for something new, but I’ll have to admit, I was a bit reluctant to take the chance because of the four years I had invested, and my lack of knowledge in IT, and particularly SL, knowing that they were doing things that had never been done before (I did my homework). Needless to say I took the chance, and was hired on as an Infrastructure Engineer.

As an Infrastructure Engineer at SL responsibilities range from installing cage nuts, rails, filler panels all the way to installing Cisco switches and Ethernet cables. Basically making sure that the racks are ready to be populated with servers and sold to new or existing customers wanting to expand their business. I can only speak for myself, but there is a great sense of pride when you step back and look at all the live racks you just painstakingly set up, knowing the hard work you had invested was not only helping SL grow but knowing that I had taken a bigger step into starting a solid career.

Yes, I was hired on as an Infrastructure Engineer, but was not just limited to that position. I have been here for over 2 years already and having some of the best times of my life, the opportunities to advance are there, they look for it in their staff, they want me to succeed, knowing it only helps them to succeed as well. What’s next you ask? To be continued…

September 7, 2009

Local Phone for (Darn Near Almost) Free

I keep my ears perked for businesses that leverage Internet infrastructure – mainly because such businesses are potential customers for SoftLayer. Occasionally, I become a customer of the businesses that I hear about.

I took the plunge with one such company after loosely watching it for a year. In the summer of 2007, a friend of mine moved his home phone service to Ooma. Basically, it is local phone service with no monthly bill. Zip. Nada. $0.00 per month. To top that off, the quality of service is very high.

Now, it’s not totally free phone service because you have to have a high speed internet connection to run it. I suspect that if you are reading this, you do. By the way, I have fiber going to my house, and I have 20 Mbps download and 5 Mbps upload speed. I can get 50 down and 20 up should I ever need that much bandwidth. If I wanted local phone service from my local phone company, they would provide it through this fiber (not copper) at a price of about $45 per month plus taxes and fees. That means the monthly bill would be about $60 when it’s all said and done.

We yanked our landline when the fiber arrived 4 years ago since each family member at that point had a cell phone. Going all cellular has been pretty much fine except for a few minor hiccups. Sometimes, one of us has been unreachable at the house because of either a dead battery, phone set to silent mode, cellular network congestion, or the fact that the ringer just can’t be heard throughout the whole house, even at full volume. None of these, however, was worth an additional $60 per month to solve.

OK, back to Ooma. My friend has had it for a year with no problem. He loves it. It works perfectly with high quality. On top of that, Ooma is now sold at Costco for one-third lower than what he paid for it. You buy the device for a one-time fee up front and never have a phone bill. After three months (usually), you’ve made your money back in savings.

So a month ago, I bought it. It took 20 minutes to set up, and I’m a finance guy. If you’re a techie, I’ll bet you’re running in 10 minutes or less. It has worked flawlessly since. The sound quality is fantastic. There are more features and add-ons than I can mention here – go browse their website for more. The snarky ad video is worth the 45 seconds to watch it. In short, I highly recommend Ooma.

To keep things balanced, the ONLY advantage I see to a copper line is if there is a power outage and your broadband modem/router is down, the local phone is down. But if your home phone is cordless with a powered base unit, the copper line is down in that case too. And if the aliens from District 9 show up, the copper lines will be flooded too I’ll bet.

Ooma is just another example of how the Internet and its supporting infrastructure is not only here to stay, but to keep growing as traditional telecom infrastructure slowly dies. At SoftLayer, we’re here to make sure our innovation supports businesses that grow by leveraging Internet infrastructure.

September 4, 2009

First Grade

Some of you know that it is now “back to school” time. Those of you who don’t know are the ones who are still probably able to attend happy hours. (Our esteemed CFO may not know, but only because he’s had lots of birthdays. I’m not saying he’s addled. I’m just saying.) Nearly everyone has heard of, if not read, the book “All I Really Need to Know I Learned in Kindergarten” by Robert Fulghum. This blog is kind of a take on that, but since The Boy just started First Grade, these are things of relevance we’ve learned about the rules and such for First Grade and that can be applied to things here at the ‘Layer.

So The Boy has tickets in First Grade, and if he’s bad – he’s got to pull a ticket and put it in a jar. Sounds like, the fewer tickets, the better. Kind of like here at SoftLayer, from our standpoint you don’t want to have to have a bunch of tickets to plow through. From a client standpoint, you certainly don’t want to have a bunch of tickets, especially from abuse, or you might get your server pulled. The lesson to be learned is to do what you can to keep those tickets at a minimum.

This year in First Grade, there are a bunch of boys, including The Boy. At the parent meeting after the second day of school, the two first grade teachers were explaining that because of the more than 2:1 ratio of boys to girls, the restroom breaks could be a potential nightmare. The uniforms have belts and shorts with buttons and zippers. The teachers said it takes forever for them to go, and they wanted the parents to tell their Boy that he needs to be able to, ah, how do we say it? Well, he needs to be able to whip it out, put it away and go. No smacking other bottoms and goofing off and giggling. Here at SoftLayer, we often have crazy deadlines, for example in development, which requires us to whip out some new technology on an expeditious basis. Just like First Grade, whip it out, put it away, and go on to the next project. (Unlike First Grade, there is lots of goofing off here at SoftLayer, such as with 10,000 bouncy balls and such. Since I’m in legal, if there is any smacking of bottoms, neither I nor HR want to hear about it. Lalalalala, Lalalala…)

One of the guys wrote a blog earlier this summer pondering if the things you learned in college are applicable to the “real world.” His conclusion was yes. My blog further confirms that the things you learn in First Grade are applicable to the “real world.” Here’s hoping The Boy can go 15 days without getting his ticket pulled, because if he does he gets some Krispy Kreme “football” donuts. And if The Boy gets some “football” donuts, that means The Mommy gets some, too.

September 2, 2009

SSL Comes to SoftLayer

Those who keep a close eye on the menu options in the customer management portal will have noticed that recently there was added an option under Security where you can now order SSL certificates. For those not familiar with SSL, a certificate is used by an application to establish identity and provide encryption services. Naturally you do not have to order your SSL certificates through us. Certificates ordered other places will work just fine on your server here. Certificates ordered here will work fine elsewhere.

So why order your SSL through SoftLayer? To me, its a convenience and security thing. Ordering with us is convenient because you can place and manage the order via the portal just like you manage aspects of your account already. Management includes being able to see when your certificates are going to expire and the ability to renew them. If the certificate file itself is deleted by accident you can get a copy of it e-mailed via the portal. From a security point of view you already have a billing arrangement with us so why give your credit card information to another party?

I can see someone thinking "But is that safe.. what if I leave SoftLayer?" Yes, it is safe. The only information you have to provide to us in doing the ordering is the Certificate Signing Request and some billing verification. Both of these are things that would be provided to any SSL vendor. The private key, which is the core of SSL security, is not kept or handled by SoftLayer. The private key is generated and remains with your administration staff on your server.

Let us chat about the private key for a moment. The private key is meant to be known only by the server applications to which it is assigned on your server. If it is lost, corrupted, deleted, whatever it will require a new certificate. What this all means is that you should only allow people you really trust access to the private key and above all you must keep a good, safe backup of the file. SoftLayer support can perform quite a bit of server voodoo but recreating a lost private key isn't an option.

I'd invite anyone with a bit of time to experiment with the SSL functionality we offer. You might find something useful for your business.

August 26, 2009

Cool Tool: nslookup

If you've been around the Internet awhile you've probably heard of the Domain Name Service. DNS is what takes www.domain.com and turns it into the 1.2.3.4 IP address which your application actually uses to find the server hosting www.domain.com.

Fascinating, Michael, why do I care? Well if you ask that question you've never had DNS fail on you.

When name resolution goes on the blink one of the tools that support uses to see what is going is the command-line utility nslookup. In its most basic form nslookup is going to do an A record query for the string you supply as an argument and it'll send that query to your operating system's configured resolvers.


C:\>nslookup www.softlayer.com
Server: mydns.local
Address: 192.168.0.1

 


Non-authoritative answer:
Name: www.softlayer.com
Address: 66.228.118.51


What is the utility telling us? First off, it asked a resolver at 192.168.0.1 for the information. Non-authoritative answer means that the server which returned the answer (192.168.0.1) is not the nameserver which controls softlayer.com. It then gives the IP address or addresses which were found.



C:\>nslookup -q=mx softlayer.com ns1.softlayer.com
Server: ns1.softlayer.com
Address: 67.228.254.4


softlayer.com MX preference = 20, mail exchanger = mx02.softlayer.com
softlayer.com MX preference = 30, mail exchanger = mx03.softlayer.com
softlayer.com MX preference = 10, mail exchanger = mx01.softlayer.com
softlayer.com nameserver = ns2.softlayer.net
softlayer.com nameserver = ns1.softlayer.net

 


This is a slightly different query. Rather than asking my local resolver to do an A record query for www.softlayer.com I've sent an MX (mail exchanger) query for softlayer.com directly to the nameserver ns1.softlayer.com. Notice that the response does not have the non-authoritative tag. The server ns1.softlayer.com is one of the nameservers which is configured to respond with a definite answer to a question rather than just saying "well, this other guy said...".

One thing that both of these queries fail to do is show the TTL for the answer they give. Time to Live (TTL) is what generally controls how long a resolver will keep an answer in cache. While the TTL is valid the resolver will use that answer. Once the TTL expires, the resolver goes looking for a fresh answer. This is great for performance but it does have a dark side to it: because of TTL, changes to DNS records are not seen instantly by all clients. If ClientA hits your website often his resolver is going to have the query result cached (say www.domain.com -> 1.2.3.4). You change the record to www.domain.com -> 5.6.7.8 but ClientA's resolver is going to continue to respond with 1.2.3.4 until the TTL runs out. If ClientA controls their resolver they can flush its cache. Generally though it is controlled by their ISP and you just have to wait.

To see the TTL for an answer you can use the nslookup form below:



C:\>nslookup
Default Server: mydns.local
Address: 192.168.5.1

 


> set debug
> www.softlayer.com.
Server: mydns.local
Address: 192.168.5.1


------------
Got answer:
HEADER:
opcode = QUERY, id = 2, rcode = NOERROR
header flags: response, want recursion, recursion avail.
questions = 1, answers = 1, authority records = 2, additional = 0


QUESTIONS:
www.softlayer.com, type = A, class = IN
ANSWERS:
-> www.softlayer.com
internet address = 66.228.118.51
ttl = 86400 (1 day)
AUTHORITY RECORDS:
-> softlayer.com
nameserver = ns1.softlayer.net
ttl = 86400 (1 day)
-> softlayer.com
nameserver = ns2.softlayer.net
ttl = 86400 (1 day)


------------
Non-authoritative answer:
Name: www.softlayer.com
Address: 66.228.118.51


The key to this spew is 'set debug' which causes nslookup to display additional information about the response, including the TTL value of the answer. You'll notice that the TTL in the ANSWERS section is 86400 seconds, which is the number of seconds in one day. This is a common TTL value. If I run the query again though, I have the following answers section:



ANSWERS:
-> www.softlayer.com
internet address = 66.228.118.51
ttl = 85802 (23 hours 50 mins 2

 


Notice how the TTL is counting down. The resolver is going to continue responding with the answer 66.228.118.51 until that TTL hits zero. At zero, the resolver will go looking for a new answer. What this means for you as a domain operator is that if you know you're going to be changing a record you should adjust down the TTL for that record a couple of days in advance. For example when some friends and I moved our colo server from one provider to another we dropped the TTLs for our DNS records down to 30 minutes two days prior to the move. Once the move was complete we were able to put them back to prior values.

 

If you spend any time at all messing with DNS you should play around with nslookup.

If you're on a Unix system take a look at the command 'dig' as well.

Happy resolving.

August 18, 2009

Backups Are Not the Whole Story

Last night while making my regular backup for my World of Warcraft configuration, I thought about the blog and I didn't remember seeing an article that went into more detail than "backups are good" about backing up and restoring data.

If you've been around the InnerLayer for a while you will have noticed that backing up of data comes up periodically.  This happens because we frequently see customers whose world is turned upside down due to a mistyped command wiping out their data.  If you just thought "that won't happen to me... I'm careful at a prompt"... well, how about a cracker getting in via an IIS zero day exploit?  Kernel bug corrupting the filesystem?  Hard drive failure?  Data loss will happen to you, whatever the cause.

Data that is not backed up is data that isn't viewed as important by the server administrator.  As the title of this blog mentioned, backing up isn't the end of the server administrator's responsibility.  Consider the following points.

  • Is the backup in a safe location?  Backing up to the same hard drive which houses the live data is not a good practice.
  • Is the backup valid?  Did the commands to create it all run properly?  Did they get all the information you need?  Do you have enough copies?
  • Can your backup restore a single file or directory?  Do you know how to restore it?  Simply put, a restore is getting data from a backup back into a working state on a system.

Backup Safety
At a minimum backups should be stored on a separate hard drive from the data which the backup is protecting.  Better would be a local copy of the backup on the machine in use and having a copy of the backup off the machine, perhaps in eVault, on a NAS which is _NOT_ always mounted, even on another server.  Why?  The local backup gives you quick access to the content while the off-machine copies give you the safety that if one of your employees does a secure wipe on the machine in question you haven't lost the data and the backup.

Validity
A backup is valid if it gets all the data you need to bring your workload back online in the event of a failure.  This could be web pages, database data, config files (frequently forgotten) and notes on how things work together.  Information systems get complicated and if you've got a Notepad file somewhere listing how Tab A goes into Slot B, that should be in your backups.  Yes, you know how it works... great, you get hit by a bus, does your co-admin know how that system is put together?  Don't forget dependencies.  A forum website is pretty worthless if it is backed up but the database to which it looks is not.  For me another mark of a valid backup is one which has some history.  Do not backup today and delete yesterday.  Leave a week or more of backups available.  People don't always notice immediately that something has broken.

Restores
A good way to test a restore is get a 2nd server for a month configured the same as your primary then take the backup from the primary and restore it onto the secondary.  See what happens.  Maybe it will go great.  Probably you will run into issues.  Forget about a small operating system tweak made some morning at 4am?  How about time?  How long does it take to go from a clean OS install to a working system?  If this time is too long, you might have too much going on one server and need to split up your workload among a few servers.  As with everything else in maintaining a server, practicing your restores is not a one-time thing.  Schedule yourself a couple of days once a quarter to do a disaster simulation.

For those who might be looking at this and saying "That is a lot of work".  Yes, it is.  It is part of running a server.  I do this myself on a regular basis for a small server hosting e-mail and web data for some friends.  I have a local "configbackup" directory on the server which has the mail configs, the server configs, the nameserver configs and the database data.  In my case, I've told my users straight up that their home directories are their own responsibility.  Maybe you can do that, maybe not.  Weekly that configback data is copied to a file server here at my apartment.  The fileserver itself is backed up periodically to USB drive which is kept at a friend's house.

Categories: 
August 10, 2009

Backups: It’s Good to Have Them!

Man, was this a weekend for me and backups! The first one I needed was for my second advance free fall course at skydive101.com on Friday (My Saturday). I jumped out of the plane a little early and one of my two instructors was not ready and did not jump on my call. Thank god I had a backup. Once my shoot deploys, being this is only my second jump out of a perfectly good airplane, I look up to check my slider and make sure my parachute is deployed and everything is correct I did not see my brake handles, I said some curse words I won’t put in this blog, and then was like it is OK I have a backup. I was still at 4000 ft at this time, still having another 1500 feet to decide if I want to release my main and pull the backup, luckily I found my brake handles pulled down twice and they worked and everything was OK.

So I get home after a day of skydiving and having fun in the sun (it is rather rare in Seattle, though not this summer) and I notice my trusty old Windows XP terminal has multiple errors on it. I do the first thing I always do with my personal Microsoft machines and reboot it. Ouch no operating system found, bad hard drive! Thank god I ghosted that machine 2 weeks ago as I figured the old IDE drives had been on their last legs. Saturday morning comes around, which is a big day for me, as I am hosting a party that night at a local night club. I notice I have a few (8900) email messages on my blackberry, so I decided it is about time to delete some. I tell it to delete, look back at it and 4 minutes later it says APP ERR. PLEASE RESTART. Needless to say, you guessed it. It wouldn’t boot back up and I had to force an OS onto it, and restore from my backup of a month ago, which reminded me that I need to start backup my blackberry more often.

So the moral of the story is, it is always good to have a backup, and we have plenty of backup options for you, so if you don’t have one, I would suggest contacting SLales. I would also suggest everyone trying to skydive at least once in their lifetime.

Categories: 
August 6, 2009

Punishing Success

Let’s say you worked for years to become a world class athlete. As a kid, you were in the gym while other athletes were at the movies. You were in the weight room on Saturday nights when no one else was there. You shunned pizza and soda in favor of grilled fish and fresh fruit. By the time Letterman hit the evening airwaves, you were well into restorative sleep. You were out the door for your morning runs while other athletes snoozed. As a result of all this, now you perform at an elite level and are very successful at your sport. Suddenly, you find that there are people who have a vested interest in helping you maximize your athletic potential. Your coaches, your managers, and companies who pay you to endorse their products all want to see you do your best. Why? Because doing your best helps them be more successful.

So, they provide you with all the things you need to maximize your potential. You get the best training gear and training regimens. You get the best nutrition. You get the right amount of rest. All these things help you maximize your potential. Thus the relationship is a nice symbiotic cycle – the more success you experience, the more success your coaches, managers, and endorsement companies experience. Win-win. Makes sense, right?

So, imagine the silliness if your coaches, managers, etc., made the decision that because you were so fortunate in your success that you had to “give back” almost half your resources to train the athletes who loafed, stayed out late, partied and gorged on pizza. Because you’re such a hard-working and smart athlete, you don’t need all those resources to participate adequately in your sport, they rationalize. Consequently, you don’t hit your potential, your coaches and managers don’t distinguish themselves, and endorsing companies don’t call you. You then feel that you’ve been punished for your hard work and success.

Sadly, much of our government policy falls under this flawed logic. The IRS just released their latest income tax stats for the year 2007. For that year the top 1% of earners paid 40.4% of all income taxes collected. We all know that right now we’re coming out of a recession and we really folks to invest in businesses and hire people to get the economy moving. So how do the 2007 numbers compare to, say, the 1980’s? During the ‘80’s, we managed to shake off the “stagflation” of the ‘70’s and get the economy rolling again. It was during this time that many technology juggernaut companies were spawned – Microsoft being a good example. So, how much of the income taxes in the ‘80’s were paid by the top 1% of earners? The average for the 10 years from 1980-1989 was 22.2%.

Let’s do some quick math. $1.116 trillion in income taxes was collected in 2007. Of that, $455.3 billion was paid by the top 1% of earners. If they paid 22.2% as in the ‘80’s, they would have paid $247.8 billion in taxes, and right now we’d have $207.5 billion MORE dollars invested in our economy. That would be quite a stimulus package! Our current policy punishes success and chokes off fuel from our economic engines while we’re trying to climb out of a worse recession than we had in the ‘70’s. Not smart.

Some may think that this would simply mean that our government deficit would be $207.5 billion higher. This is not the case at all. These folks that make up that top 1% didn’t get there by being lazy or not putting their money to work. I know some folks in that group, and they WANT to put their money to work! I know one gentleman who had to be told some legal docs for a deal could not be prepared over the weekend because Christmas was on that weekend. These folks are like the world class athlete I mentioned above – by and large they’re disciplined and hard-working. Their money will build new businesses and create more jobs, and the government will collect far more revenue from this new economic activity than it would give up in collections from these top 1% folks. Think about it – how many of us have ever been hired by a “poor” person? Instead of punishing economic success, we should encourage it!

Bottom line, if government policy were to make sense, it would encourage these folks to maximize their economic potential and find the correct balance of revenue to collect and yet still promote economic growth. What would we prefer? That the government collects 50% of $1 trillion or 30% of $2 trillion? Hint: 30% of $2 trillion is a WAY better deal.

At SoftLayer, we think very differently about things. We simply do not punish our customers for succeeding. We empower them to be more successful – why? Because if our customers succeed, we succeed. We get this.

Can we prove this? Perhaps a look at how customers vote with their feet is an indicator. For the past few months, SoftLayer has seen the lowest percentage of customers terminating business with us in our history. If we punished our customers for their success, they would go elsewhere.

August 5, 2009

SLales Motivation 101

We have a pretty good sales team here at SoftLayer, quite honestly I couldn’t ask for a better group than the one we have.  It’s pretty common for us to have various sales contests and awards to keep the team motivated and focused in the right direction, however last month I decided to try something a little different.  Lance and I set a pretty lofty sales goal for the team (one I didn’t really think they would achieve), and told them that if they reached the goal I would shave my head completely bald.

As the month progressed, I began to realize the genius of this contest.  The team was focused, driven by the desire to see their boss publically humiliated.  SLales worked extra hours, came in early, stayed late, made calls, sent emails, followed up every lead, they did everything possible to exceed the goal.  Looking back on it as their manager, it was a beautiful thing to watch and I’ve never been more proud of my team.

It ended up being close (down to the final hours on Friday actually).  However, SLales stepped up to the plate and met the challenge.  They worked together like a well oiled machine and overcame numerous obstacles, with the singular vision of seeing their boss embarrassed.  They succeeded as a team, and here is the payoff:

August 3, 2009

Education

Attending College Classes can be a daunting task. The hours of homework and studying (and the obligatory time spent actually in the classroom) can noticeably eat away at one’s free time (and at times, their sanity). While it can be painful to take on college, attending classes and working on top of it can be exponentially more difficult. Balancing your studies with your responsibilities at work can be tricky, even for those who are experts in time management. When all is said and done, though, the investment is well worth it. As I’ve stated before, Knowledge is power (yes, I know, shameless self promotion), and learning can occur at any opportunity.

I recently realized that with the exception for while sleeping (some days I can count the hours on one hand), I am always learning new things. While my progressing college education keeps me thinking, SoftLayer has taught me more than I ever thought I would learn in such a short amount of time. New operating systems (at least to me), and continual changes and improvements are synonymous with life at SL. Learning occurs at every customer request, every server build, and every operating system install. Certainly, employment here is not for the faint of heart. More so, no one can say that they didn’t leave their shift just a bit smarter than when they arrived.

Knowledge is important in this industry, as knowing the correct process to solve a problem can mean the difference between five hours and five minutes of downtime. While everyone has their strengths, the team that we have here supersedes any possible weaknesses, leading to one of the brightest group of individuals anyone could have the privilege of working with. I spend my shifts perpetually challenged, but never overwhelmed by the sheer magnitude of issues (read: learning opportunities) that present themselves every day. While I will concede that classes such as precalculus and humanities may not directly sharpen my troubleshooting skills, being able to think logically and follow procedures will certainly pay off in the long run.

Pages

Subscribe to news