Posts Tagged ‘engineer’

Two of the most common network characteristics we look at when investigating network-related concerns in the NOC are speed and throughput. You may have experienced the following scenario yourself: You just provisioned a new bad-boy server with a gigabit connection in a data center on the opposite side of the globe. You begin to upload your data and to your shock, you see “Time Remaining: 10 Hours.” “What’s wrong with the network?” you wonder. The traceroute and MTR look fine, but where’s the performance and bandwidth I’m paying for?

This issue is all too common and it has nothing to do with the network, but in fact, the culprits are none other than TCP and the laws of physics.

In data transmission, TCP sends a certain amount of data then pauses. To ensure proper delivery of data, it doesn’t send more until it receives an acknowledgement from the remote host that all data was received. This is called the “TCP Window.” Data travels at the speed of light, and typically, most hosts are fairly close together. This “windowing” happens so fast we don’t even notice it. But as the distance between two hosts increases, the speed of light remains constant. Thus, the further away the two hosts, the longer it takes for the sender to receive the acknowledgement from the remote host, reducing overall throughput. This effect is called “Bandwidth Delay Product,” or BDP.

We can overcome BDP to some degree by sending more data at a time. We do this by adjusting the “TCP Window” – telling TCP to send more data per flow than the default parameters. Each OS is different and the default values will vary, but most all operating systems allow tweaking of the TCP stack and/or using parallel data streams. So what is iPerf and how does it fit into all of this?

What is iPerf?

iPerf is simple, open-source, command-line, network diagnostic tool that can run on Linux, BSD, or Windows platforms which you install on two endpoints. One side runs in a ‘server’ mode listening for requests; the other end runs ‘client’ mode that sends data. When activated, it tries to send as much data down your pipe as it can, spitting out transfer statistics as it does. What’s so cool about iPerf is you can test in real time any number of TCP window settings, even using parallel streams. There’s even a Java based GUI you can install that runs on top of it called, JPerf (JPerf is beyond the scope of this article, but I recommend looking into it). What’s even cooler is that because iPerf resides in memory, there are no files to clean up.

Click to see how you can use and tweak iPerf with amazing results. »

I recently read an article about an endeavor that is currently being undertaken to develop a “Speech Analysis Algorithm Crafted to Detect and Help Dissatisfied Customers”. In short, a team of engineers are hoping to create software that will recognize when a caller is becoming stressed and immediately phone a manager to alert them of a developing situation. Wow! It is rare that you would see math and science applied to something that is so subjective. After all, math is used to quantify and measure things all based on a known or a baseline. In this particular effort, I would surmise that the team of engineer’s most difficult task will be to determine how to establish a unique baseline for each unique call and caller. Once upon a time as a student of Electrical Engineering, I took on my share of convolution integrals and that’s a path that I do not care to venture down again. I’ve also taken on my share of convoluted customer calls in a past life and witnessed our frontline assisting customers in complex situations here at SoftLayer.

Until there is such an application that can detect and address a conversation that may be heading in the wrong direction, we have to rely on good ole’ training and experience. With each call and query, the baseline is reset. I’d even go further to say that with each exchange; the baseline is reset as our Customer Service Agents seek information to get to the root of the issue. It’s not hard to imagine the frustration that can build in a back-and-forth conversation as two people look to come to a solution or an amiable conclusion just as it is understandable that sometimes, a customer may simply need to vent. How do you calculate and anticipate those scenarios?

I wish much success to the team involved in the customer service speech analysis program. And programmatically speaking, I see many CASE, SWITCH, FOR, WHILE, BREAK, CONTINUE, IF, ELSE, ELSE IF, NEXT statements in your future. Good Luck!

Around the office I am commonly considered a “low-level” software engineer. If you are in the business of computer programming you know this means I generally have various pieces of computer hardware strewn about my work area, and an ASCII chart hanging on my wall complete with a cheat-table so I can quickly convert numbers between binary, decimal, and hex. If you are not in the business of developing software, think of me as guy who couldn’t decide if I wanted to be an electrical engineer or a computer programmer and thus through my own indecision eventually found myself stuck somewhere in between. I know a bit about both but am not an expert in either. (I think the Roman word for this sort of limbo is purgatory, but I find it pretty cozy most days.)

At any rate when a project comes along that walks the fence between the realms of hardware and software my name naturally comes up. Such was the case a few weeks ago when one of our systems administrators had the need to retrieve the serial number from the RAM chips already installed in a number of servers. He asked me if it could be done. I looked and saw the information was reported in the BIOS of one of my machines, so I promptly responded with a “you bet”. After all, if the BIOS can display the information on the screen I should be able to as well. Right? I told him it would take a week.

The problem in this career field I have worked for some ten years now is you don’t know what you don’t know. Fast forward two weeks. Now think the Friday before Easter. That’s right, the one I am supposed to be off lounging around the house in my pajamas. It took a little longer to pull that serial number than I expected. If you’re interested the slow down turned out to be that the information existed at a physical memory address that was not easily accessible from Microsoft Windows (luckily for the BIOS it gets to display the data before an operating system is loaded).

Remember the old Chevy Chase movie “Funny Farm“? Chevy’s character is driving around lost when he passes the old man sitting on his porch in a rocking chair. Chevy stops his vehicle, rolls down his window, and says: “Excuse me Sir. Can you tell me how you would get to Redbud?” The old man leans forward, spits, and replies: “If I were going to Redbud I sure as hell wouldn’t start from here.”

Like Mr. Chase’s character in the movie, I didn’t get to pick where I started the journey from. We need the data available to us after the operating system boots. So I am hacking my way through it. I’m nearly there now. Close enough at least that I felt comfortable taking a break from the code and blowing off some steam by writing this blog. And the truth is, while I might have been whining just a bit I actually have enjoyed this project immensely. I appreciate the fact that the management here at SoftLayer gives us the opportunity to challenge ourselves and then grow to meet those challenges. We are encouraged to “get our hands dirty”. When I finish up here I will have a deeper understanding of how the BIOS relates to the operating system (and through the BIOS indirectly to the hardware).

As for our customers, well, it just so happens once I got to digging around in the binary mud there was a whole lot of other useful insight buried in the swirls of all those zeros and ones. Instead of extracting just the serial numbers I am pulling about a dozen pages of hardware data points we can use in statistical analysis for predicting failures, standards compliance, and availability trends. Like I said, you don’t know what you don’t know. But sometimes you are pleasantly surprised once you find out. By promoting such an amiable work environment, fostering creativity, and encouraging innovation, SoftLayer continues to boldly go where no other hosting company has gone before.

Alright, time to climb down from the pulpit and finish up my software.

Thanks for listening!

-William