Technology Posts

In the last couple of days, there’s been a bit of buzz about “open DNS resolvers” and DNS amplification DDoS attacks, and SoftLayer’s name has been brought up a few times. In a blog post on October 30, CloudFlare explained DNS Amplification DDoS attacks and reported the geographic and network sources of open DNS resolvers that were contributing to a 20Gbps attack on their network. SoftLayer’s AS numbers (SOFTLAYER and the legacy THEPLANET-AS number) show up on the top ten “worst offenders” list, and Dan Goodin contacted us to get a comment for a follow-up piece on Ars Technica — Meet the network operators helping to fuel the spike in big DDoS attacks.

While the content of that article is less sensationalized than the title, there are still a few gaps to fill about when it comes to how SoftLayer is actually involved in the big picture (*SPOILER ALERT* We aren’t “helping to fuel the spike in big DDoS attacks”). The CloudFlare blog and the Ars Technica post presuppose that the presence of open recursive DNS resolvers is a sign of negligence on the part of the network provider at best and maliciousness at worst, and that’s not the case.

The majority of SoftLayer’s infrastructure is made up of self-managed dedicated and cloud servers. Customers who rent those servers on a monthly basis have unrestricted access to operate their servers in any way they’d like as long as that activity meets our acceptable use policy. Some of our largest customers are hosting resellers who provide that control to their customers who can then provide that control to their own customers. And if 23 million hostnames reside on the SoftLayer network, you can bet that we’ve got a lot of users hosting their DNS on SoftLayer infrastructure. Unfortunately, it’s easier for those customers and customers-of-customers and customers-of-customers-of-customers to use “defaults” instead of looking for, learning and implementing “best practices.”

It’s all too common to find those DNS resolvers open and ultimately vulnerable to DNS amplification attacks, and whenever our team is alerted to that vulnerability on our network, we make our customers aware of it. In turn, they may pass the word down the customer-of-customer chain to get to the DNS owner. It’s usually not a philosophical question about whether DNS resolvers should be open for the greater good of the Internet … It’s a question of whether the DNS owner has any idea that their “configuration” is vulnerable to be abused in this way.

SoftLayer’s network operations, abuse and support teams have tools that flag irregular and potentially abusive traffic coming from any server on our network, and we take immediate action when we find a problem or are alerted to one by someone who sends details to abuse@softlayer.com. The challenge we run into is that flagging obvious abusive behavior from an active DNS server is a bit of a cat-and-mouse game … Attackers cloak their activity in normal traffic. Instead of sending a huge amount of traffic from a single domain, they send a marginal amount of traffic from a large number of machines, and the “abusive” traffic is nearly impossible for even the DNS owner to differentiate from “regular” traffic.

CloudFlare effectively became a honeypot, and they caught a distributed DNS amplification DoS attack. The results they gathered are extremely valuable to teams like mine at SoftLayer, so if they go the next step to actively contact the abuse channel for each of the network providers in their list, I hope that each of the other providers will jump on that information as I know my team will.

If you have a DNS server on the SoftLayer network, and you’re not sure whether it’s configured to prevent it from being used for these types of attacks, our support team is happy to help you out. For those of you interested in doing a little DNS homework to learn more, Google’s Developer Network has an awesome overview of DNS security threats and mitigations which gives an overview of potential attacks and preventative measures you can take. If you’re just looking for an easy way to close an open recursor, scroll to the bottom of CloudFlare’s post, and follow their quick guide.

If, on the other hand, you have your own DNS server and you don’t want to worry about all of this configuration or administration, SoftLayer operates private DNS resolvers that are limited to our announced IP space. Feel free to use ours instead!

-Ryan

As an abuse administrator, I’m surrounded by spam on a daily basis. When someone sends an abuse-related complaint to our abuse@softlayer.com contact address, it gets added to our ticket queue, and our Abuse SLayers take time to investigate and follow up with the customers whose servers violate our acceptable use policy. The majority of those abuse-related submissions are reporting spam coming from our network, and in my interaction with customers, I’ve noticed that spam (and the source of spam) is widely misunderstood.

Most spam tickets we create on customer accounts pinpoint spam sent from a compromised or exploited server. Our direct customer didn’t send the phishing email, malware distribution, pharmacy advertisement or pornographic spam, but that activity came from their account. While they’re accountable for the abusive behavior coming from their server, in many cases, they don’t know that there’s a problem until we post an abuse ticket on their account. These servers are targeted and compromised by common techniques and exploits that could have been easily avoided, but they aren’t very well known outside the world of abuse.

To protect yourself from a spammer, you need to think like a spammer. You need to understand how someone might try to exploit your environment so that you can prevent them from doing so. As you’re looking at ways to secure your server proactively, make sure you target these five exploits in particular:

1. User Auth Login

This is by far the most common exploit to used to send spam. This method involves a person or script using the credentials of a user to send spam through a domain’s mail server. The majority of these incidences are caused by malware on a client PC that obtains the login and password for a domain user and uses that information to log on and send mail from the client PC through the server. Often, these spam messages are sent through a botnet command structure.

When an account is compromised, simply changing the password for the compromised user on the server usually won’t stop the abuse. We see quite a few accounts that continue to send spam after an initial abuse ticket results in a password change. Most servers that are sending spam with this method are found to only be sending a small amount of spam at any given time to avoid detection. The low volume of spam that is being sent per server is made up for by the fact that there are thousands of servers being used for the same spamming campaigns.

In order to stop the User Auth Login exploit, a customer needs to clean all of the malicious software (malware) from their environments. To prevent future User Auth Login compromises, users should be made aware of the potential dangers of untrusted software, and if they believe their machines are infected, they need to know what to do.

2. Tell-a-friend Exploitation

The User Auth Login technique is the most common method employed by spammers, but the “tell-a-friend” script exploitation isn’t far behind when it comes to volume of affected servers. This spamming method find websites that use scripts to invite users to refer friends to a page or product. Spammers will use the ‘Your Message’ field in one of these scripts to input their own content and links, and they’ll push the actual page referral link to the bottom of the message. When these site scripts aren’t secure, the spammer will use them to send hundreds or thousands of messages.

To avoid having your website fall victim to this type of spam, be very wary of any widget or script you add. If you need to add Facebook, Twitter and email “share” functionality to your site, make sure you incorporate a tell-a-friend script that does not allow for customizable messages or does not accept input of more than one email address. Also, users won’t need the “cc” or “bcc” fields, so you can be sure those are axed as well. If you can’t find a good “share” script that you’re comfortable with from a security perspective, it might be a good idea to remove that functionality to avoid exploitation.

3. Uploaded Mailers

Spam sent via an uploaded third party mailer can sometimes prove difficult for admins to locate. An uploaded third party mailer could be capable of creating it’s own outbound SMTP connection, and that would allow a program to bypass the existing MTA on the server and render any legitimate mail logs useless for investigation. Another challenge is that a php mailer can be uploaded to a location within a user’s web content, and that mailer is run by the user ‘nobody’ (the default Apache user).

We strongly suggest configuring your server to have the mail headers show the script’s user (that’s not the Apache default user) and the location the script is running from on the server. Many times, these kinds of mailers are maliciously uploaded after a user’s FTP password is been compromised, so be sure your FTP login information is secure.

4. Software Exploits

The “software exploits” category casts a huge shadow. Every piece of software on a server — from mail servers, content management systems and control panels to the operating system itself — can be targeted by hackers. They probe servers to find security vulnerabilities and weak coding, and when they find a vulnerability, they take control.

The hacker who found the software vulnerability might not actually take advantage of the exploit immediately. That user may sell access to other entities for their use, and that use often ends up being spam. In addition to having strong firewall rules and access restrictions, you should update and maintain the current stable versions of all software on your servers.

5. WordPress Exploits

WordPress exploits would technically fall under the “Software Exploits” category, but I’m breaking it out into its own category simply due to the volume of spam issues that are the result of exploiting this particular piece of software. The first step to protecting against spam being sent through this source is to make sure you have the latest version of WordPress installed. With that done, be sure to research the latest security plugins for that version and install any that are applicable to your environment.

These five techniques are not the only ones used by spammers to take advantage of your environment, but they are some of the most common. To protect yourself from becoming a source of spam, make your servers a more difficult target to exploit. To stop spam, you need to know spam. Now that you know spam, it’s time to stop it. Ask questions, test your environment regularly and watch your logs for any unexplained usage.

-Andrew

Whether you’re managing a SaaS solution for thousands of large clients around the world or you’re running a small mail server for a few mom-and-pop businesses in your neighborhood, you’re providing IT service for a fee — and your customers expect you to deliver. It’s easy to get caught up in focusing your attention and energy on day-to-day operations, and in doing so, you might neglect some of the looming risks that threaten the continuity of your business. You need to prioritize risk assessment and management.

Just reading that you need to invest in “Risk Management” probably makes you shudder. Admittedly, when a business owner has to start quantifying and qualifying potential areas of business risk, the process can seem daunting and full of questions … “What kinds of risks should I be concerned with?” “Once I find a potential risk, should I mitigate it? Avoid it? Accept it?” “How much do I need to spend on risk management?”

When it comes to risk management in hosting, the biggest topics are information security, backups and disaster recovery. While those general topics are common, each business’s needs will differ greatly in each area. Because risk management isn’t a very “cookie-cutter” process, it’s intimidating. It’s important to understand that protecting your business from risks isn’t a destination … it’s a journey, and whatever you do, you’ll be better off than you were before you did it.

Because there’s not a “100% Complete” moment in the process of risk management, some people think it’s futile — a gross waste of time and resources. History would suggest that risk management can save companies millions of dollars, and that’s just when you look at failures. You don’t see headlines when businesses effectively protect themselves from attempted hacks or when sites automatically fail over to a new server after a hardware failure.

It’s unfortunate how often confidential customer data is unintentionally released by employees or breached by malicious attackers. Especially because those instances are often so easily preventable. When you understand the potential risks of your business’s confidential data in the hands of the wrong people (whether malicious attackers or careless employees), you’ll usually take action to avoid quantifiable losses like monetary fines and unquantifiable ones like the loss of your reputation.

More and more, regulations are being put in place to holding companies accountable for protecting their sensitive information. In the healthcare industry businesses have to meet the strict Health Insurance Portability and Accountability Act (HIPAA) regulations. Sites that accept credit card payments online are required to operate in Payment Card Industry (PCI) Compliance. Data centers will spend hours (and hours and hours) achieving and maintaining their SSAE 16 certification. These rules and requirements are not arbitrarily designed to be restrictive (though they can feel that way sometimes) … They are based on best practices to ultimately protect businesses in those industries from risks that are common throughout the respective industry.

Over the coming months, I’ll discuss ways that you as a SoftLayer customer can mitigate and manage your risk. We’ll talk about security and backup plans that will incrementally protect your business and your customers. While we won’t get to the destination of 100% risk-mitigated operations, we’ll get you walking down the path of continuous risk assessment, identification and mitigation.

Stay tuned!

-Matthew

I like to think that we are beyond ‘defining’ cloud, but what I find in reality is that we still argue over basics. I have conversations in which people still delineate things like “hosting” from “cloud computing” based degrees of single-tenancy. Now I’m a stickler for definitions just like the next pedantic software-religious guy, but when it comes to arguing minutiae about cloud computing, it’s easy to lose the forest for the trees. Instead of discussing underlying infrastructure and comparing hypervisors, we’ll look at two well-cited definitions of cloud computing that may help us unify our understanding of the model.

I use the word “model” intentionally there because it’s important to note that cloud computing is not a “thing” or a “product.” It’s a way of doing business. It’s an operations model that is changing the fundamental economics of writing and deploying software applications. It’s not about a strict definition of some underlying service provider architecture or whether multi-tenancy is at the data center edge, the server or the core. It’s about enabling new technology to be tested and fail or succeed in blazing calendar time and being able to support super-fast growth and scale with little planning. Let’s try to keep that in mind as we look at how NIST and Gartner define cloud computing.

The National Institute of Standards and Technology (NIST) is a government organization that develops standards, guidelines and minimum requirements as needed by industry or government programs. Given the confusion in the marketplace, there’s a huge “need” for a simple, consistent definition of cloud computing, so NIST had a pretty high profile topic on its hands. Their resulting Cloud Computing Definition describes five essential characteristics of cloud computing, three service models, and four deployment models. Let’s table the service models and deployment models for now and look at the five essential characteristics of cloud computing. I’ll summarize them here; follow the link if you want more context or detail on these points:

• On-Demand Self Service: A user can automatically provision compute without human interaction.
• Broad Network Access: Capabilities are available over the network.
• Resource Pooling: Computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned.
• Rapid Elasticity: Capabilities can be elastically provisioned and released.
• Measured Service: Resource usage can be monitored, controlled and reported.

The characteristics NIST uses to define cloud computing are pretty straightforward, but they are still a little ambiguous: How quickly does an environment have to be provisioned for it to be considered “on-demand?” If “broad network access” could just mean “connected to the Internet,” why include that as a characteristic? When it comes to “measured service,” how granular does the resource monitoring and control need to be for something to be considered “cloud computing?” A year? A minute? These characteristics cast a broad net, and we can build on that foundation as we set out to create a more focused definition.

For our next stop, let’s look at Gartner‘s view: “A style of computing in which scalable and elastic IT-enabled capabilities are delivered as a service using Internet infrastructure.” From a philosophical perspective, I love their use of “style” when talking about cloud computing. Little differentiates the underlying IT capabilities of cloud computing from other types of computing, so when looking at cloud computing, we really just see a variation on how those capabilities are being leveraged. It’s important to note that Gartner’s definition includes “elastic” alongside “scalable” … Cloud computing gets the most press for being able to scale remarkably, but the flip-side of that expansion is that it also needs to contract on-demand.

All of this describes a way of deploying compute power that is completely different than the way we did this in the decades that we’ve been writing software. It used to take months to get funding and order the hardware to deploy an application. That’s a lot of time and risk that startups and enterprises alike can erase from their business plans.

How do we wrap all of those characteristics up into unified of definition of cloud computing? The way I look at it, cloud computing is as an operations model that yields seemingly unlimited compute power when you need it. It enables (scalable and elastic) capacity as you need it, and that capacity’s pricing is based on consumption. That doesn’t mean a provider should charge by the compute cycle, generator fan RPM or some other arcane measurement of usage … It means that a customer should understand the resources that are being invoiced, and he/she should have the power to change those resources as needed. A cloud computing environment has to have self-service provisioning that doesn’t require manual intervention from the provider, and I’d even push that requirement a little further: A cloud computing environment should have API accessibility so a customer doesn’t even have to manually intervene in the provisioning process (The customer’s app could use automated logic and API calls to scale infrastructure up or down based on resource usage).

Last week, I had the opportunity to speak at Cloud Connect Chicago, and I shared SoftLayer’s approach to cloud computing and how it has evolved into a few distinct products that speak directly to our customers’ needs:

The session was about 45 minutes, so the video above has been slimmed down a bit for easier consumption. If you’re interested in seeing the full session and getting into a little more detail, we’ve uploaded an un-cut version here.

-Duke

One of the biggest challenges developers run into when coding large, scalable systems is automating batch processes and distributing workloads to optimize compute resource usage. More simply, intra-application and inter-system communications tend to become a bottleneck that affect the user experience, and there is no easy way to get around it. Well … There *was* no easy way around it.

Meet SoftLayer Message Queue.

As the name would suggest, Message Queue allows you to create one or more “queues” or containers which contain “messages” — strings of text that you can assign attributes to. The queues pass along messages in first-in-first-out order, and in doing so, they allow for parallel processing of high-volume workflows.

That all sounds pretty complex and “out there,” but you might be surprised to learn that you’re probably using a form of message queuing right now. Message queuing allows for discrete threads or applications to share information with one another without needing to be directly integrated or even operating concurrently. That functionality is at the heart of many of the most common operating systems and applications on the market.

What does it mean in a cloud computing context? Well, Message Queue facilitates more efficient interaction between different pieces of your application or independent software systems. The easiest way demonstrate how that happens is by sharing a quick example:

Creating a Video-Sharing Site

Let’s say we have a mobile application providing the ability to upload video content to your website: sharevideoswith.phil. The problem we have is that our webserver and CMS can only share videos in a specific format from a specific location on a CDN. Transcoding the videos on the mobile device before it uploads proves to be far too taxing, what with all of the games left to complete from the last Humble Bundle release. Having the videos transcoded on our webserver would require a lot of time/funds/patience/knowledge, and we don’t want to add infrastructure to our deployment for transcoding app servers, so we’re faced with a conundrum. A conundrum that’s pretty easily answered with Message Queue and SoftLayer’s (free) video transcoding service.

What We Need

• Our Video Site
• The SoftLayer API Transcoding Service
• SoftLayer Object Storage
• A “New Videos” Container
• A “Transcoded Videos” Container with CDN Enabled
• SoftLayer Message Queue
• “New Videos” Queue
• “Transcoding Jobs” Queue

The Process

1. Your user uploads the video to sharevideoswith.phil. Your web app creates a page for the video and populates the content with a “processing” message.
2. The web application saves the video file into the “New Vidoes” container on object storage.
3. When the video is saved into that container, it creates a new message in the “New Videos” message queue with the video file name as the body.
4. From here, we have two worker functions. These workers work independently of each other and can be run at any comfortable interval via cron or any scheduling agent:

Each step in the process is handled by an independent component. This allows us to scale or substitute each piece as necessary without needing to refactor the other portions. As long as each piece receives and sends the expected message, its colleague components will keep doing their jobs.

Video transcoding is a simple use-case that shows some of the capabilities of Message Queue. If you check out the Message Queue page on our website, you can see a few other examples — from online banking to real-time stock, score and weather services.

Message Queue leverages Cloudant as the highly scalable low latency data layer for storing and distributing messages, and SoftLayer customers get their first 100,000 messages free every month (with additional messages priced at $0.01 for every 10,000).

What are you waiting for? Go get started with Message Queue!

-Phil (@SoftLayerDevs)

About a year ago, Phil Jackson and I recorded a podcast-esque click-through of a presentation that explained the difference between IPv4 and IPv6 address space, and as a testament to the long-tail nature of blog posts, Internet Society’s Deploy360 Blog shared the video. With a hint of nostalgia, I clicked “play” on the video.

I laughed. I cried. I found it informative. I noticed a few places where it could have been better.

We recorded the video in response to a tweet from one of our Twitter followers, and the off-the-cuff dialog wound up being somewhere in between “accessible, informative and funny” and “overly detailed, too long and obviously improvised.” Because there aren’t many people who want to listen to two guys give a 15-minute presentation on IP addresses when they could be watching a Songified review of Five Guys Burgers and Fries or an epic data center tour, I thought I’d dilute the information from the video into a quick blog post that spells out some of the major distinctions between IPv4 and IPv6 so you can scan it, interject your own “witty” banter and have your favorite YouTube viral video playing in the background.

IP Address Overview

An IP address is like a telephone number or a street address. When you connect to the Internet, your device (computer, smartphone, tablet) is assigned an IP address, and any site you visit has an IP address. The IP addressing system we’ve been using since the birth of the Internet is called IPv4, and the new addressing system is called IPv6. The reason we have to supplement the IPv4 address system (and ultimately eclipse it) with IPv6 is because the Internet is running out of available IPv4 address space, and IPv6 provides is an exponentially larger pool of IP addresses … Let’s look at the numbers:

• Total IPv4 Space: 4,294,967,296 addresses
• Total IPv6 Space: 340,282,366,920,938,463,463,374,607,431,768,211,456 addresses

Even saying the IPv6 space is “exponentially larger” doesn’t really paint the picture of the difference in size.

IPv4 Addresses

To understand why the IPv4 address space is limited to four billion addresses, we can break down an IPv4 address. An IPv4 address is a 32-bit number made up of four octets (8-bit numbers) in decimal notation, separated by periods. A bit can either be a 1 or a 0 (2 possibilities), so the decimal notation of an octet would have 2⁸ distinct possibilities — 256 of them, to be exact. Because we start numbering at 0, the possible values of one an octet in an IPv4 address go from 0 to 255.

Examples of IPv4 Addresses: 192.168.0.1, 66.228.118.51, 173.194.33.16

If an IPv4 address is made up of four sections with 256 possibilities in each section, to find the total number of possibilities in the entire IPv4 pool, you’d just multiply 256*256*256*256 to get to the 4,294,967,296 number. To look at it another way, you’ve got 32 bits, so 2³² will get you to the same total.

IPv6 Addresses

IPv6 addresses are based on 128 bits. Using the same math as above, we can take 2¹²⁸ and find the total IPv6 address pool (which I won’t copy again here because it takes up too much space). Because the IPv6 pool is so much larger than the IPv4 pool, it’d be much more difficult to define the space in the same decimal notation … you’d have 2³² possibilities in each section.

To allow for that massive IPv6 pool to be used a little more easily, IPv6 addresses are broken down into eight 16-bit sections, separated by colons. Because each section is 16 bits, it can have 2¹⁶ variations (65,536 distinct possibilities). Using decimal numbers between 0 and 65,535 would still be pretty long-winded, so IPv6 addresses are expressed with hexadecimal notation (16 different characters: 0-9 and a-f).

Example of an IPv6 Addresses: 2607:f0d0:4545:3:200:f8ff:fe21:67cf

That’s still a mouthful, but it’s a little more manageable than the decimal alternatives.

CIDR Slash (/) Notation

When people talk about blocks of IP addresses, they generally use CIDR Slash (/) Notation where the block might look like this: 192.0.2.0/24 … When you glance at that number, you might assume, “Okay, so you have 192.0.2.0 through 192.0.2.24,” but CIDR notation is not showing you the range of addresses, it’s telling you the size of the “network” part of the allocation.

IP addresses are made up two parts — the network and the host. The “network” part of the address tells us the number of bits that stay the same at the beginning of the block of IPs, while the “host” part of the address are the bits that define the different possibilities of IP addresses in the block. In CIDR notation, a /24 is telling us that the first 24 bits of the address are defined by the network, so we have 8 bits (32 total bits minus 24 network bits) in the host — 2⁸ is 256 distinct addresses. The 192.0.2.0/24 IPv4 address block includes 192.0.2.0 to 192.0.2.255.

IPv4 address blocks can be as large as a /8 (given to regional registries like ARIN and APNIC), and they can be as small as a /32 (which is a single IP address).

Why Provision So Many IPv6 Addresses?

When SoftLayer provisions an IPv6 address block on a server, we give a /64 block of IPv6 addresses … Or 18,446,744,073,709,551,616 IPv6 addresses to each server. That number seems excessive, but the /64 block size is the “smallest” IPv6 allocation block.

Providers like SoftLayer are allocated /32 blocks of IPv6 addresses. The difference between a /32 and a /64 is 32 bits (2³²) … Bonus points if you can remember where you’ve seen that number before. What that means is that SoftLayer is given a block of IP addresses so large that we could provision 4,294,967,296 /64 blocks of IPv6 addresses … Or put more remarkably: In one /32 block of IPv6 space, there are the same number of /64 blocks of IPv6 addresses as there are TOTAL IPv4 addresses.

So while it’s pretty impossible to use a full /64 of IPv6 addresses on a server, it’s equally difficult for SoftLayer to burn through its /32 block.

So Now What?

IPv4 space is running out quickly. If your site isn’t running a dual-stack IPv6 configuration yet, it’s possible that you’re going to start missing traffic from users who are only able to access the Internet over IPv6 (which is not backwards compatible with IPv4). If your Internet Service Provider (ISP) doesn’t support IPv6 yet, you won’t be able to access websites that are broadcast only with IPv6 addresses.

The percentage of instances of each of those cases is relatively small, but it’s only going to get larger … And it only takes one missed customer to make you regret not taking the steps to incorporate IPv6 into your infrastructure.

-@khazard

Sometimes, technical conversations can sound like people are just making up tech-sounding words and acronyms: “If you want HPC to handle Gigaflops of computational operations, you probably need to supplement your server’s CPU and RAM with a GPU or two.” It’s like hearing a shady auto mechanic talk about replacing gaskets on double overhead flange valves or hearing Chris Farley (in Tommy Boy) explain that he was “just checking the specs on the endline for the rotary girder” … You don’t know exactly what they’re talking about, but you’re pretty sure they’re lying.

When we talk about high performance computing (HPC), a natural tendency is to go straight into technical specifications and acronyms, but that makes the learning curve steeper for people who are trying to understand why a solution is better suited for certain types of workloads than technology they are already familiar with. With that in mind, I thought I’d share a quick explanation of graphics processing units (GPUs) in the context of central processing units (CPUs).

The first thing that usually confuses people about GPUs is the name: “Why do I need a graphics processing unit on a server? I don’t need to render the visual textures from Crysis on my database server … A GPU is not going to benefit me.” It’s true that you don’t need cutting-edge graphics on your server, but a GPU’s power isn’t limited to “graphics” operations. The “graphics” part of the name reflects the original intention for kind of processing GPUs perform, but in the last ten years or so, developers and engineers have come to adapt the processing power for more general-purpose computing power.

GPUs were designed in a highly parallel structure that allows large blocks of data to be processed at one time — similar computations are being made on data at the same time (rather than in order). If you assigned the task of rendering a 3D environment to a CPU, it would slow to a crawl — it handles requests more linearly. Because GPUs are better at performing repetitive tasks on large blocks of data than CPUs, you start see the benefit of enlisting a GPU in a server environment.

The Folding@home project and bitcoin mining are two of the most visible distributed computing projects that GPUs are accelerating, and they’re perfect examples of workloads made exponentially faster with the parallel processing power of graphics processing units. You don’t need to be folding protein or completing a blockchain to get the performance benefits, though; if you are taxing your CPUs with repetitive compute tasks, a GPU could make your life a lot easier.

If that still doesn’t make sense, I’ll turn the floor over to the Mythbusters in a presentation for our friends at NVIDIA:

SoftLayer uses NVIDIA Tesla GPUs in our high performance computing servers, so developers can use “Compute Unified Device Architecture” (CUDA) to easily take advantage of their GPU’s capabilities.

Hopefully, this quick rundown is helpful in demystifying the “technobabble” about GPUs and HPC … As a quick test, see if this sentence makes more sense now than it did when you started this blog: “If you want HPC to handle Gigaflops of computational operations, you probably need to supplement your server’s CPU and RAM with a GPU or two.”

-Phil

SoftLayer Private Clouds are officially live, and that means you can now order and provision your very own private cloud infrastructure on Citrix CloudPlatform quickly and easily. Chief Scientist Nathan Day introduced private clouds on the blog when it was announced at Cloud Expo East, and CTO Duke Skarda followed up with an explanation of the architecture powering SoftLayer Private Clouds. The most amazing claim: You can order a private cloud infrastructure and spin up its first virtual machines in a matter of hours rather than days, weeks or months.

If you’ve ever looked at building your own private cloud in the past, the “days, weeks or months” timeline isn’t very surprising — you have to get the hardware provisioned, the software installed and the network configured … and it all has to work together. Hearing that SoftLayer Private Clouds can be provisioned in “hours” probably seems too good to be true to administrators who have tried building a private cloud in the past, so I thought I’d put it to the test by ordering a private cloud and documenting the experience.

At 9:30am, I walked over to Phil Jackson‘s desk and asked him if he would be interested in helping me out with the project. By 9:35am, I had him convinced (proof), and the clock was started.

When we started the order process, part of our work is already done for us:

To guarantee peak performance of the CloudPlatform management server, SoftLayer selected the hardware for us: A single processor quad core Xeon 5620 server with 6GB RAM, GigE, and two 2.0TB SATA II HDDs in RAID1. With the management server selected, our only task was choosing our host server and where we wanted the first zone (host server and management server) to be installed:

For our host server, we opted for a dual processor quad core Xeon 5504 with the default specs, and we decided to spin it up in DAL05. We added (and justified) a block of 16 secondary IP addresses for our first zone, and we submitted the order. The time: 9:38am.

At this point, it would be easy for us to game the system to shave off a few minutes from the provisioning process by manually approving the order we just placed (since we have access to the order queue), but we stayed true to the experiment and let it be approved as it normally would be. We didn’t have to wait long:

At 9:42am, our order was approved, and the pressure was on. How long would it take before we were able to log into the CloudStack portal to create a virtual machine? I’d walked over to Phil’s desk 12 minutes ago, and we still had to get two physical servers online and configured to work with each other on CloudPlatform. Luckily, the automated provisioning process took on a the brunt of that pressure.

Both server orders were sent to the data center, and the provisioning system selected two pieces of hardware that best matched what we needed. Our exact configurations weren’t available, so a SBT in the data center was dispatched to make the appropriate hardware changes to meet our needs, and the automated system kicked into high gear. IP addresses were assigned to the management and host servers, and we were able to monitor each server’s progress in the customer portal. The hardware was tested and prepared for OS install, and when it was ready, the base operating systems were loaded — CentOS 6 on the management server and Citrix XenServer 6 on the host server. After CentOS 6 finished provisioning on the management server, CloudStack was installed. Then we got an email:

At 11:24am, less than two hours from when I walked over to Phil’s desk, we had two servers online and configured with CloudStack, and we were ready to provision our first virtual machines in our private cloud environment.

We log into CloudStack and added our first instance:

We configured our new instance in a few clicks, and we clicked “Launch VM” at 11:38am. It came online in just over 3 minutes (11:42am):

I got from “walking to Phil’s desk” to having a multi-server private cloud infrastructure running a VM in exactly two hours and twelve minutes. For fun, I created a second VM on the host server, and it was provisioned in 31.7 seconds. It’s safe to say that the claim that SoftLayer takes “hours” to provision a private cloud has officially been confirmed, but we thought it would be fun to add one more wrinkle to the system: What if we wanted to add another host server in a different data center?

From the “Hardware” tab in the SoftLayer portal, we selected “Add Zone” to from the “Actions” in the “Private Clouds” section, and we chose a host server with four portable IP addresses in WDC01. The zone was created, and the host server went through the same hardware provisioning process that our initial deployment went through, and our new host server was online in < 2 hours. We jumped into CloudStack, and the new zone was created with our host server ready to provision VMs in Washington, D.C.

Given how quick the instances were spinning up in the first zone, we timed a few in the second zone … The first instance was online in about 4 minutes, and the second was running in 26.8 seconds.

By the time I went out for a late lunch at 1:30pm, we’d spun up a new private cloud infrastructure with geographically dispersed zones that launched new cloud instances in under 30 seconds. Not bad.

Don’t take my word for it, though … Order a SoftLayer Private Cloud and see for yourself.

-@khazard

While a handful of SoftLayer employees made their way to Boston for HostingCon, another ragtag group of SLayers journeyed to Portland to attend OSCON &mdash: the Open Source CONvention. OSCON attracts 2,500+ passionate members of the open source community, so the conference sessions and expo hall are filled with the most creative and innovative people on the Web. That’s where we want to be.

Over the past few years, we’ve built a great reputation at OSCON as not only a great hosting provider, but also as the operator of one of the best booths on the expo hall floor. As usual, the switchballs were crowd pleasers, and we sponsored the show’s Massage Booth, so we had great traffic through our booth all conference. When attendees left our booth, they were considerably more relaxed, they had the coolest swag at the show, and they had a better understanding of where SoftLayer fits in the open source space.

In addition to the conversations on the expo hall floor, we got to share a little expertise in a conference session. Senior Software Architect Harold Hannon presented an engaging educational session about how we implemented elasticsearch, Apache-based code that allows for scalable search for all kinds of documents in near real-time. At the moment, SoftLayer uses elasticsearch internally for hardware and ticketing, and we hope to extend this feature-rich scalable searching to our customers in an upcoming release of the customer portal. Because SoftLayer has built a great reputation for executing scalability well, Harold ended up presenting to a packed house (which you can see in the last few pictures of the slide show above).

SoftLayer’s significant investment in open source platforms like OpenStack Swift Object Storage and CloudStack-based Private Clouds wound up being a big topic of discussion throughout the conference. Harold’s elasticsearch presentation was a great conversation bridge to talk about the incredible search-and-retrieve functionality we implemented in our Object Storage service, and we were able to share and demonstrate how that functionality helps our customers manage large quantities of static data in cloud environments in an automated way.

The open source community has matured significantly over the past few years, and it’s exciting to see that evolution. We aren’t just talking about the incredibly popular open source operating systems like CentOS, Debian, Fedora, FreeBSD and Ubuntu that customers can get on a dedicated or cloud server … We’re talking about game-changing, innovative platforms that are redefining how the Internet works.

We want to thank the OSCON team for another phenomenal show, and if you attended the show but didn’t get a switchball from us, I’m sure you’ll have another chance at OSCON 2013. If you don’t think you can wait that long, come find us at one of our other upcoming events!

-Summer

The highlight of any customer visit to a SoftLayer office is always the data center tour. The infrastructure in our data centers is the hardware platform on which many of our customers build and run their entire businesses, so it’s not surprising that they’d want a first-hand look at what’s happening inside the DC. Without exception, visitors to a SoftLayer data center pod are impressed when they walk out of a SoftLayer data center pod … even if they’ve been in dozens of similar facilities in the past.

What about the customers who aren’t able to visit us, though? We can post pictures, share stats, describe our architecture and show you diagrams of our facilities, but those mediums can’t replace the experience of an actual data center tour. In the interest of bridging the “data center tour” gap for customers who might not be able to visit SoftLayer in person (or who want to show off their infrastructure), we decided to record a video data center tour.

If you’ve seen “professional” video data center tours in the past, you’re probably positioning a pillow on top of your keyboard right now to protect your face if you fall asleep from boredom when you hear another baritone narrator voiceover and see CAD mock-ups of another “enterprise class” facility. Don’t worry … That’s not how we roll:

Josh Daley — whose role as site manager of DAL05 made him the ideal tour guide — did a fantastic job, and I’m looking forward to feedback from our customers about whether this data center tour style is helpful and/or entertaining.

If you want to see more videos like this one, “Like” it, leave comments with ideas and questions, and share it wherever you share things (Facebook, Twitter, your refrigerator, etc.).

-@khazard