social-media

July 15, 2009

Subjecting Subjectivity To Math

I recently read an article about an endeavor that is currently being undertaken to develop a “Speech Analysis Algorithm Crafted to Detect and Help Dissatisfied Customers”. In short, a team of engineers are hoping to create software that will recognize when a caller is becoming stressed and immediately phone a manager to alert them of a developing situation. Wow! It is rare that you would see math and science applied to something that is so subjective. After all, math is used to quantify and measure things all based on a known or a baseline. In this particular effort, I would surmise that the team of engineer’s most difficult task will be to determine how to establish a unique baseline for each unique call and caller. Once upon a time as a student of Electrical Engineering, I took on my share of convolution integrals and that’s a path that I do not care to venture down again. I’ve also taken on my share of convoluted customer calls in a past life and witnessed our frontline assisting customers in complex situations here at SoftLayer.

Until there is such an application that can detect and address a conversation that may be heading in the wrong direction, we have to rely on good ole’ training and experience. With each call and query, the baseline is reset. I’d even go further to say that with each exchange; the baseline is reset as our Customer Service Agents seek information to get to the root of the issue. It’s not hard to imagine the frustration that can build in a back-and-forth conversation as two people look to come to a solution or an amiable conclusion just as it is understandable that sometimes, a customer may simply need to vent. How do you calculate and anticipate those scenarios?

I wish much success to the team involved in the customer service speech analysis program. And programmatically speaking, I see many CASE, SWITCH, FOR, WHILE, BREAK, CONTINUE, IF, ELSE, ELSE IF, NEXT statements in your future. Good Luck!

July 13, 2009

What a View!

I can easily define myself as the crazy one up in the Seattle Datacenter. I like to ride dirt bikes, street bikes, go fast on the water, ride in small airplanes, I could go on and on how my co-workers (and friends/family) may think I am crazy when it comes to Adventures.

What can I say, I like a challenging experience.

One of those challenging experiences is working at SoftLayer, always preparing to be ahead of the rest in this industry, we're constantly learning new technologies and taking leaps and bounds. That is the reason why I love my job so much, we're always working with the latest and greatest, learning new stuff. Speaking of leaps and bounds, I finally did something the other day I have always wanted too. I signed up for an Advance Free Fall Skydive class and jumped out of a Cessna at 13,000 feet. Free falling at terminal speeds towards the earth, At 12,000 feet I mock pulled my parachute 3 times, so the instructors who where both holding onto me by their hands only could see I learned what to do. 6000 ft came, I locked onto my altimeter, 5500 I waived hands off to the instructors and they deployed below me, and I pulled my rip cord. That all happened in about 50 seconds after leaping out of the plane. For the next 8 minutes I saw the best view in Western Washington I have ever seen. One of the thoughts that came to my mind is the only thing to relate to how I have ever thought something was this nice, was the first time I walked into a SoftLayer Datacenter and admired how well thought out and nice it was.

I'm glad to say 19 months into this job and being part of the Operations team in Seattle, I walk in each day to the datacenter and can say the same thing day in and day out. Let's hope I can say the same thing about my second jump in a few weeks.

July 10, 2009

The Kinmans are Taking Over

Yeah, it’s official…we have another one: me, even though I’m only a summer intern. The day after my 16th birthday couldn’t have been a better opportunity for me to begin my quest for company domination. Well, I think that will come later.

Anyway, it is my first job at my first company, ever. Softlayer’s really nice. They paid me one hour’s wages for sitting in a conference room and filling out a frightening mountain of paperwork (OK, well, maybe it was more like 5 or 6 forms), and again another day for visiting a few datacenter rooms in the Infomart (I think it was compensation for hearing damage from the noise in the rooms). Michael Scott would have scoffed and turned his empty pockets inside out. Anyway, my point is to tell you that the Kinman clan is taking over Softlayer.

Yep, you got that right.

Steve has 4 kids who will probably grow up and get promoted to manage some section of the company each. Gary’s kid (me) already has a spot, and I’m up to my ears in details about servers, processors, RAM, disk drives, routers, switches, fiber, etc. Heck, maybe Mama Carol and Papa Willie will come back out of retirement to work here too. Oh and Steve’s wife and my mom and…well, you get the idea. If your name is Kinman, you’re gonna work at Softlayer, whether you like it or not.

Someday, you’re gonna wake up in some kind of cubicle in a building on International Parkway and you’re gonna wonder how the heck you ended up there. And HR is just gonna tell you, “Well…your last name IS Kinman…” They may have to stop paying us to fill out forms to keep from going bankrupt because there’s gonna be so many of us. With that in mind, I’d say that the average current non-Kinman Softlayer employee has about three years, plus-or-minus, before the co-worker next to them is, you guessed it, a Kinman. Wow. Congrats Carol and Willie.

…just kidding. But speculation is fun, right? Until it actually happens. Muahahahahaha.

July 8, 2009

Encrypted Hot Chocolate

Imagine this scene: you’re sitting at a local coffee shop, having a drink and browsing the web. While checking out your favorite news site you see an e-mail come in where someone is commenting on your blog post from that morning. This is odd because while you remember checking blogs, you don’t remember posting one. On investigating you find a blog post that you definitely did not create. As you look around wondering what is going on you should probably take a peek at the guy sitting in the comfy chair with his headphones on running a wireless sniffer to grab passwords out of the air.

How does this happen? The coffee shops I’ve seen all run open wireless access points. This is great for flexibility and serving the most people but if the Access Point is unencrypted it is quite possible to run an application which will listen to the wireless network and record packets. These packets can then be examined by a tool like the Wireshark application. Since the packets are not encrypted then things like username/password combos can be read in clear text from the packet.

Knowing what the problem is, how do we work around it? Since we cannot encrypt the wireless session, we’ll encrypt the data itself. One option would be to do a VPN from your laptop in the coffee shop to a location out in the world. This would channel all of your traffic out through this other system. It’s a good solution but it can be a bit technically complicated if you don’t have one already set up. If you’re really only concerned with encrypting your HTTP traffic you could use the PuTTy application to tunnel traffic via an encrypted session to a Unix server here at SoftLayer by using the OpenSSH ability to act as a SOCKS5 proxy.

When you define your connection in PuTTy you can go down to Connection > SSH > Tunnels and then place a port number, such as 8080, in the Source field. Select “Dynamic” and “Auto” then click the Add button. Connect to your Unix server here at SoftLayer. Next stop is the browser. The way I configure my browser for this is to go into Firefox > Tools > Options > Advanced > Network > Settings (under Connection). Select “Manual proxy configuration” and then in the field labeled SOCKS Host: put “127.0.0.1” and for the port use the port you specified above. Leave the type as “SOCKS v5”. Select OK and then in the URL bar type “about:config” which will let you do advanced configuration. In the filter field type in “network.proxy.socks_remote_dns”. Right click on it and select Toggle. This will mark it true.

Now if you pull up a website which will tell you the IP that you are coming from (such as http://whatismyip.com) you should see it report back the IP address of your server here at SoftLayer. This happens because Firefox has been told to use 127.0.0.1:8080 as its SOCKSv5 proxy and this traffic gets tunneled via the encrypted SSH session to your server at SoftLayer. The server here will do the DNS lookup (due to network.proxy.socks_remote_dns) and then send out the request. The response will be tunneled back to your browser.

You do have to remember to fire up the PuTTY session but this isn’t so hard since if you try and browse without it the browser tries to hit the SOCKSv5 proxy port specified and fails. Beyond that I’ve not run into any troubles using this trick.

And now I think I will head off for some hot chocolate myself.

Categories: 
July 6, 2009

The Cure for Irrelevance

I’ve been feeling rather irrelevant lately……yeah, yeah, I know . Watch out, because when lawyers feel irrelevant we sue!

Anyway, just thinking about the things going on in the rest of the world, like the wars in Iraq and Afghanistan. We have men and women over there who are still dying and losing their sanity on a daily basis, but for those of us who don’t have anyone close to us involved in the wars, it’s become a low hum…car bomb, soldier killed, hum, hum. But then I think of the daily terror that those who have loved ones go through – did she have to go out on patrol today? Did she get hurt? Did she get killed? Every day, every hour spent wondering if that person is safe and will come home again. That soldier is not irrelevant – he is making the greatest sacrifice so we can go on with our safe, secure lives over here.

My thoughts also turn to the people of Iran, and I find myself thinking: “If I lived in a repressive regime, would I be out on the streets in defiance of the government and particularly with the threat of being beaten, jailed or disappearing from the face of the earth?” I like to think I would, but I don’t know the answer, and that feeds my irrelevance. And come on Iran; give your people some credit. Make it at least look like the election wasn’t predetermined. You declare a landslide victory for Ahmadinejad (cut and paste, baby, cut and paste!) mere minutes after the polls close, yet the ballots are supposed to be hand counted. How can that work? I mean, wait an hour or so – pretend you counted. Please! Are the people of Iran irrelevant? No, they are making the greatest sacrifice in a battle for freedom, and in an uprising that may very well change the course of their history. The world is watching.

So how do I become relevant? (Assuming, that is, that a lawyer can ever be relevant). How do SLayers and SLackers become relevant? We go that extra mile. We’ve been dealing with cranky clients all day – keep the smile on the face and in the voice and treat them like they’ve just put in an order for 300+ servers a month. We don’t remain satisfied with the status quo – figure out how to make our system and services better, stronger, faster. We don’t rest on our laurels because we just had a major release, i.e., Cloudlayer instances. We look ahead and figure out or invent the next new thing our customers need or want. We scan the forums to keep a pulse on our clients (and it’s usually good for an eye-roll or two). We keep Lance and Mike out of trouble.

So am I like a U.S. soldier or the Iranian people? Not so much, but I can do things to stay relevant in my own little world.

Categories: 
July 4, 2009

Fourth of July

Fourth of July – Independence Day is more than just a day for us to hang out with friends and family across the United States and gather around the BBQ and watching fireworks and bombs blow up. It is a day that we celebrate our founding fathers courage and bravery in the pursuit of liberty and freedom.

If it wasn’t for these men and their dreams, I would not be sitting here at SoftLayer writing this blog for a company that loves us to share our words and views with others. I have been amazed how over the last few weeks how Twitter and other sites have helped the country of Iran speak their voice and let the world know what is going on over there. We would never know what is going on as their government would not allow it to be voiced on the state ran television.

So, as I am camping this Fourth of July in the San Juan Islands, fishing on the lake and watching the skies over Friday Harbor light up, I will be thankful for what our founding fathers accomplished on that day in 1776.

Categories: 
July 1, 2009

Pre-configuration and Upgrades

I recently bought a new computer for my wife. Being a developer, and a former hardware engineering student, I opted the buy the parts and assemble the machine ourselves. Actually assembling a computer these days doesn't take too long, it's the software that really gets you. Windows security updates, driver packs, incompatibilities, inconsistencies, broken websites, and just plain bad code plagued me for most of the night. The video card, in particular, has a “known issue” where it just “uh-oh” turns off the monitor when Windows starts. The issue was first reported in March of 2006, and has yet to be fixed.

This is why SoftLayer always tests and verifies the configurations we offer. We don't make the end user discover on their own that Debian doesn't work on Nehalems, we install it first to be sure. This is also why our order forms prevent customers from ordering pre-installed software that are incompatible with any of the rest of the order. We want to make sure that customers avoid the frustration of ordering things only to find out later that they don't work together.

The problem with desktop computers, especially for people who are particular about their configurations, is that you cannot buy a pre-configured machine where all the parts are exactly what you want. We attempted to get a computer from Dell, and HP, but neither company would even display all the specifications we were interested in, nevermind actually having the parts we desired. Usually pre-built systems skimp on important things like the motherboard or the power supply, giving you very little room to upgrade.

At SoftLayer, we don't cut corners on our systems, and we ensure that each customer can upgrade as high as they possibly can. Each machine type can support more RAM and hard drives than the default level, and we normally have spare machines handy at all levels so that once you outgrow the expansion capabilities of your current box, you can move to a new system type. If you're thinking of getting a dedicated server, but you're worried about the cost, visit the SoftLayer Outlet Store and start small. We have single-core Pentium Ds in the outlet store, and you can upgrade from there until you're running a 24-core Xeon system.

June 29, 2009

Leaving Normal

What is normal for a server? In support we get that question from time to time. The problem is that normal varies from server to server. A load average of 200 is probably not normal but a load of 5 to 10 very well could be normal, depending on the server's application. What to do?

Baselining to the rescue. The idea behind baselining is to get performance numbers on your application when things are "normal" so that you have solid math to indicate when things are not "normal".

What makes a good baseline? Things like RAM use (overall, per process, rate of change), number and types of processes running, processor usage, disk usage (total, per app), disk speed and network utilization are all good OS metrics. You can also get metrics from your application. E-mails per hour, web page generation time, and number of users logged in are good to know.

You can capture OS metrics using tools like top, free, ps and iostat on Linux. Actually if you have iostat you probably have 'sar' which is great for performance history. Sar has a process that runs every few minutes and records various OS counters including processor info, RAM use, disk I/O and the like.

For the Windows people you have Task Manager and Performance Monitor. Task Manager is pretty simple and gives mostly an overview. Perfmon is really where its at on Windows. Using PerfMon you can track dozens of performance counters on disk, proc, memory, the network and even application specific metrics if you are running apps like MS Exchange that support them.

As with most tasks related to being the lord and master of a server, performance monitoring isn't a one time thing. As you make changes to the system you have to run new baselines. Between changes you should run your performance routines periodically to see how things are changing. It is much easier to look into an issue if you spot it earlier rather than later.

Go forth and make sure all your baselines are belong to you!

*bonus cool points for those who knew the title of this blog was also the title of a "Roswell" episode.

Categories: 
June 24, 2009

Clouds and Elephants

So there I was after work today, sitting in my favorite watering hole drinking my Jagerbomb, when Caira, my bartender asked what was on my mind. I told her that I had been working with clouds and elephants all day at work and neither of those things are little. She laughed and asked if I had stopped anywhere to get a drink prior to her bar. I replied no, I'm serious I had to make some large clouds and a stampede of elephants work together. I then explained to her what Hadoop was. Hadoop is a popular open source implementation of Google's MapReduce. It allows transformation and extensive analysis of large data sets using thousands of nodes while processing peta-bytes of data. It is used by websites such as Yahoo!, Facebook, Google, and China's best search engine Baidu. I explained to her what cloud computing was (multiple computing nodes working together) hence my reference to the clouds, and how Hadoop was named after the stuffed elephant that belonged to one of the founders - Doug Cutting - child. Now she doesn't think I am as crazy.

Categories: 
June 22, 2009

Really?

In catching up on some of my blog reading, I ran across this blog by Jill Eckhaus of AFCOM (a professional organization for data center managers). Yes, I realize that article is four months old, but like I said – I’m catching up.

One of the things that really concerns me with articles and blogs such is this one are the repetitive concerns about “data security” and “loss of control” of your infrastructure. Both of those points are easy to state because they prey on the natural fear of any system administrator or data center manager.

System administrators have long ago come to realize that, in the proper environment, there is no real downside to not being able to physically place their hands upon their servers. In the proper environment the system administrator can power on or off the server, can get instant KVM access to the server, can boot the server into a rescue kernel to try to salvage a corrupt file system, can control network port speeds and connectivity, can reload the operating system, can instantly add and manage services such as load balancers and firewalls, can manage software licenses and naturally, can control full access to the server with root or administrator level privileges. In other words, there is no “loss of control” and “data security” is still up to the system administrator.

The data center managers are understandably concerned about outsourcing because it can potentially impact their jobs. But let’s face it – in today’s economy, the capital outlay required to acquire new datacenter space or additional datacenter equipment is extremely difficult to justify. In those cases sometimes the only two options are to do nothing or to outsource to an available facility. Of course, another option is to jeopardize your existing facility by trying to cram even more services into an already overloaded data center. If a data center manager is trying to build a fiefdom of facilities and personnel, outsourcing is certainly going to be a concern. One interesting aspect of outsourcing is – datacenter management jobs are still there; they are just at consolidated and often times more efficient facilities.

In reality, “data security” and “loss of control” should be of no more or less concern if you are using your own data center versus if you are doing the proper research and selecting a viable outsourcing opportunity with a provider that can prove it has the processes, procedures and tools in place to handle the job for you.

(In the spirit of full disclosure; I am both a local and national AFCOM member and find the organization and the information they make available to be quite useful.)

-SamF

Pages

Subscribe to social-media