Technology Posts

April 14, 2014

OpenSSL Heartbleed Update

On April 7th, the OpenSSL Project released an update to address a serious security flaw (CVE-2014-0160), which allows remote attackers to obtain sensitive information from process memory via crafted packets that trigger a buffer over-read, as demonstrated by reading private keys, related to d1_both.c and t1_lib.c, aka the Heartbleed bug.

SoftLayer Infrastructure

After notification of this vulnerability we began a close examination of our services to determine any that may have been affected. Both the SoftLayer customer portal and API are serviced behind hardware load balancers and neither the hardware load balancers nor the software running on the servers behind them were found to be running vulnerable versions of OpenSSL. This was confirmed by the hardware vendor and direct testing as well. During these tests it was discovered that certain nodes of our Object Storage cluster were running a vulnerable version of OpenSSL. The software was immediately patched to remediate the issue. Although there is no indication that this vulnerability was exploited, the subset of customers potentially affected has been advised of precautionary measures to ensure continued security.

Additionally, our team forced updates to all of our internal operating system update mirrors as soon as patched versions were released by their publishers. Our system automatically checks for and updates all operating system versions hosted on our mirrors, but due to the urgency of this exploit, manual updates were run as quickly as possible to have patched versions available sooner.

SoftLayer Customers

Due to the nature, surface area, and severity of this vulnerability, we recommend revoking all possibly compromised keys and reissuing new certificates for any service secured using the OpenSSL library. The rekeying process can vary depending on your Certificate Authority (CA) and you should contact them if you have questions on how to complete this process. This OpenSSL vulnerability has major security implications for a wide range of operating systems and applications and may necessitate rebooting your hardware (or restarting services) to ensure all services linking against the affected code use the updated version of the OpenSSL library. We also recommend that you patch all of your servers and change passwords as soon as possible. Take this opportunity to review your overall password strategy including password strength and password sharing across sites.

-@skinman454

Keywords:
Categories:
March 12, 2014

Name Servers: Don't Get Lost in (Domain) Translation

The Domain Name System (DNS) is vital to keeping the Internet in order and easily accessible. Every byte on the World Wide Web lives in (at least) one specific place on the planet, and it's mapped to that location with an IP address like 66.228.118.53 (IPv4) or 2607:f0d0:4545:3:200:f8ff:fe21:67cf (IPv6). DNS translates the domain names you know and love to the correct IP addresses, so without DNS, you would have to memorize a 32-bit or 128-bit IP address for your favorite websites. Remember the last time your cell phone died? How many phone numbers did you have memorized?

There are plenty of resources available online to explain How DNS Works, so in this post, we'll focus on the basics of how we use DNS. Here's the scenario: We want to register a domain — softlayer.com — and make it available to the masses.

1. Reserve and Register a Domain Name
The first thing we need to do is purchase and register our domain name. To do this, we need to choose a domain registrar and verify that our domain is available. Every domain registrar effectively provides the same service: It will reserve an available domain on your behalf, and it will let you dictate where that domain will live. There's not a lot of differentiation or value-add in that service, so many registrars offer cheap or free domains as loss-leaders for higher margin hosting or Web services. Be sure to check the fine print to make sure you're not committing to a year of hosting to get a $0.99 domain name. Some registrars make the process of updating and configuring where a domain resolves more difficult than others, but for the sake of this example we'll assume that your registrar allows the same easy accessibility SoftLayer provides our customers in the customer portal.

The domain name we want is softlayer.com, and in this example, that domain name is available for us to reserve. Once we go through the ordering process, we'll need to associate the domain with a set of authoritative name servers. Authoritative name servers are effectively the go-to address book for a specified domain. By default, your domain registrar will provide name servers for your domain, but they can be changed easily to meet your needs. You have four typical options when it comes to choosing your domain's name servers:

  • Use the domain registrar's default name servers.
  • Use you hosting provider’s name servers.
  • Use a third party DNS service to manage your domain names.
  • Run your own name servers on your server to manage your domain names.

Each of these options has its own pros and cons, but because we're just interested in getting our domain online, we'll use SoftLayer's DNS control panel to manage our new domain name.

2. Create DNS Records
When we access our hosting provider's DNS control panel, we see this:

DNS Management

This is an extremely high level view of DNS, so we’re just going to focus on what we must have in order to make softlayer.com reachable via browser. The first thing we'll do is add a DNS zone. This is usually our domain, but in some situations, it can be a bit different. In this example, we'll create a “softlayer.com” zone to be responsible for the whole softlayer.com domain:

DNS Management

With that zone created, we now need to add new "Address Records" (A Records) within that zone:

DNS Management
DNS Management

The terminology used in different DNS control panels may vary, so let’s breakdown what the four sections in those screenshots mean:

  • Resource Type: This is our DNS record type. In our example, we have A records which link a hostname to our IP address. There are a number of DNS record types, each serving a different purpose.
  • Host: This is the host node or owner name — the name of the node that this record applies to. Using the @ symbol in the A record allows visitors to reach our website without the leading www. If we wanted blog.softlayer.com to live at a different IP address, we'd make that happen here.
  • Points To: This is the IP address of the host node. You might see this section referred to elsewhere as content, data or value. The standard term is RDATA — resource record data. This is specific to each data type.
  • TTL (Time-to-Live): TTL dictates how long your name server should keep a particular record before refreshing for possible updates. Generally speaking, longer TTLs work well if you’re just adding new entries and or don’t anticipate frequent record changes.

Once we save these changes in our DNS control panel, we play the waiting game. Because these DNS changes have to propagate across our DNS servers to be accessible to the Internet as a whole, the process typically takes 24-48 hours, if not sooner. SoftLayer’s customer portal has DNS check built-in as one a few different network tools. If you aren't a current customer, you can use What's my DNS? This is what the SoftLayer tool looks like:

DNS Management

3. Create rDNS Records
The last step we want to take in setting up our domain is to create Reverse DNS (rDNS) records. These records do the same thing as DNS records, but (as the name suggests) they function in the opposite direction. With rDNS, we can assign an IP address to a domain name. This step isn't required, but I recommend it to help ensure better performance of online activities like email and website visitor tracking.

DNS is a central piece of the Internet as we know it, so by understanding how to use it, you'll have a much better understanding of how the Internet works. It seems challenging at first glance, but as you see from this simple walkthrough, when you break down and understand each step, you won't get overwhelmed. A wealth of DNS tools and tutorials are available for free online, and our DNS documentation might be a great resource to bookmark so you'll never get lost in domain translation.

- Landon

December 16, 2013

Xplenty: Tech Partner Spotlight

We invite each of our featured SoftLayer Tech Marketplace Partners to contribute a guest post to the SoftLayer Blog, and this week, we're happy to welcome Yaniv Mor from Xplenty. Xplenty is a cloud-based code-free Hadoop as a Service platform that allows you to easily create data workflows, provision, monitor and scale clusters. Their goal is to eliminate the complexity of Hadoop to make it accessible and cost-effective for everyone.

Simplifying Hadoop

Apache Hadoop, open source software developed by Doug Cutting, is the most popular storage and processing platform for big data. Because Hadoop can accommodate structured data, semi-structured data, and unstructured data, it is the storage architecture of choice for some of the Internet's largest and most data-rich sites. Industry giants such as Google and Facebook have been using Hadoop for years to store and deliver information while gathering insights from customer behavior and internal business processes, and their obvious success with the platform has helped drive broad adoption and popularity all the way down to small-businesses and startups.

Specific use cases vary among industries, but similarities exist. Many companies leverage Hadoop to gather information about their clientele. With Hadoop, a company can process huge amounts of data to examine past and present behaviors, and with that information, customers can be presented personally-tailored recommendations, and the business can glean deep insights from the trends and outliers in its customer base. As a result, customers are more likely to make repeat purchases, and companies are able to predict trends and possible risks, allowing them to visualize and prepare for a number of business scenarios.

Another compelling use case for Hadoop is its ability to analyze and report on multi-faceted marketing and advertising campaigns. By drilling down into the guts of a campaign, users can see exactly what worked and what didn't. Marketers and advertisers can direct their resources to the campaigns that worked and let the ineffective ones fall by the wayside.

On the internal side, businesses are using Hadoop to better understand their own information. Data systems at financial companies use it to detect fraud anomalies by comparing transaction details. If you've ever made a credit card purchase in another state or country but the purchase didn't go through, your bank's system probably flagged the transaction for a representative to investigate. Other companies analyze data collected from their networks to monitor activity and diagnose bottlenecks and other issues with a negative impact.

The challenge with leveraging Hadoop's broad potential is that a company generally needs dedicated technical resources to allocate toward building and maintaining the solution — from manpower to financial to infrastructure. Hadoop is difficult to program and requires a very specific skill set that few possess. If a company doesn't have the personnel for the job, it will need to fork over some serious cash to get a system built and maintained. This can significantly hinder the progress of the data and business intelligence teams, and by default, the progress of the company. That's why we decided to create Xplenty.

Xplenty is a coding-free Hadoop-as-a-Service platform that allows data and BI users to process their big data stored on the SoftLayer cloud without having to acquire any special skills. What Xplenty does is remove the need to divert those precious resources from anything other than the business at hand. Xplenty's Hadoop-as-a-Service platform has a graphical user interface that enables the data and BI teams to build data flows without ever having to write a line of code. The benefit of this is twofold. First, the business intelligence analysts can quickly build data flows that would typically take weeks or more to program and debug, and data users can easily insert Xplenty into their data stack to handle processing needs. The second benefit is that since the IT department doesn't have to worry about doing any programming, they are able to tackle more pressing issues, bottlenecks are avoided, and life goes on without a hitch.

Xplenty was created specifically for the cloud, and SoftLayer is a major player in this space, so it was a natural fit for us to partner up to provide a SoftLayer-specific offering that will perform even better for customers already using SoftLayer infrastructure. We only work with providers with the best and most stable infrastructure, and SoftLayer is definitely at the top of the list.

If you want to try Hadoop on Xplenty, jump over to our SoftLayer sign up page, enter your details, and test drive the platform with a free 30-day trial!

- Yaniv Mor, Xplenty

This guest blog series highlights companies in SoftLayer's Technology Partners Marketplace.
These Partners have built their businesses on the SoftLayer Platform, and we're excited for them to tell their stories. New Partners will be added to the Marketplace each month, so stay tuned for many more come.
November 14, 2013

Enhancing Usability by Building User Confidence

Consider your experiences with web applications, and see if this scenario seems familiar: Your electricity bill has some incorrect charges on it. Fearing that you will have to spend 40 minutes on hold if you call in, you find that the electric company website has a support center where you can submit billing issues and questions; you are saved! You carefully fill out the form with your sixteen-digit account number and detailed description of the incorrect charges. You read it over and click the submit button. Your page goes blank for a couple of seconds, the form comes back with a note saying you typed in your phone number incorrectly, and the detailed description you spent eleven minutes meticulously writing is gone.

Web applications have gotten much better at preventing these kinds of user experiences over the past few years, and I'm sure that none of your applications have this problem (if they do, fix it right now!), but "usability" is more than just handling errors gracefully. Having a seamless process is only half the battle when it comes to giving your users a great experience with your application. The other half of the battle is a much more subjective: Your users need to feel confident in their success every step of the way. By keeping a few general guidelines in mind, you can instill confidence in your users so that they feel positive about your application from start to finish with whatever they are trying to accomplish.

1. Keep the user in a familiar context.

As the user in our electric company support application example, let's assume the process works and does not lose any of my information. I have to have faith that the application is going to do what I expect it to do when the page refreshes. Faith and unfamiliar technology do not exactly go hand in hand. Instead of having the form submit with a page refresh, the site's developers could introduce a progress wheel or other another kind of indicator that shows the data is being submitted while the content is still visible. If detailed content never goes away during the submission process, I'm confident that I still have access to my information.

Another example of the same principle is the use of modal windows. Modal windows are presented on top of a previous page, so users have a clear way of going back if they get confused or decide they navigated to the wrong place. By providing this new content on top of a familiar page, users are much less likely to feel disoriented if they get stuck or lost, and they will feel more confident when they're using the application.

2. Reassure the user with immediate feedback.

By communicating frequently and clearly, users are reassured, and they are much less likely to become anxious. Users want to see their actions get a response from your application. In our electric company support application example, imagine how much better the experience would be if a small blurb was displayed in red next to the phone number text box when I typed in my phone number in the wrong format. The immediate feedback would pinpoint the problem when it is easy to correct, and it would make me confident that when the phone number is updated, the application will continue to work as expected.

3. Provide warnings or extra information for dangerous or complicated operations.

When users are new to an application, they are not always sure which actions will have negative consequences. This is another great opportunity for communication. Providing notices or alerts for important or risky operations can offer a good dose of hesitation for new users who aren't prepared. Effective warnings or notices will tell the user when they will want to perform this action or what the negative consequences might be, so the user can make an informed decision. Users are confident with informed decisions because a lack information causes anxiety.

I learned how to implement this tip when I designed a wizard system for a previous employer that standardized how the company's application would walked users through any step-by-step process. My team decided early on to standardize a review step at the end of any implemented wizard. This was an extra step that every user had to go through for every wizard in the application, but it made all of the related processes much more usable and communicative. This extra information gave the users a chance to see the totality of the operation they were performing, and it gave them a chance to correct any mistakes. Implementing this tip resulted in users who were fully informed and confident throught the process of very complicated operations.

4. Do not assume your users know your terminology, and don't expect them to learn it.

Every organization has its own language. I have never encountered an exception to this rule. It cannot be helped! Inside your organization, you come up with a defined vocabulary for referencing the topics you have to work with every day, but your users won't necessarily understand the terminology you use internally. Some of your ardent users pick up on your language through osmosis, but the vast majority of users just get confused when they encounter terms they are not familiar with.

When interacting with users, refrain from using any of your internal language, and strictly adhere to a universally-accepted vocabulary. In many cases, you need shorthand to describe complex concepts that users will already understand. In this situation, always use universal or industry-wide vocabulary if it is available.

This practice can be challenging and will often require extra work. Let's say you have a page in your application dealing with "display devices," which could either be TVs or monitors. All of your employees talk about display devices because to your organization, they are essentially the same thing. The technology of your application handles all display devices in exactly the same way, so as good software designers you have this abstracted (or condensed for non-technical people) so that you have the least amount of code possible. The easiest route is to just have a page that talks about display devices. The challenge with that approach is that your users understand what monitors and TVs are, but they don't necessarily think of those as display devices.

If that's the case, you should use the words "monitors" and "TVs" when you're talking about display devices externally. This can be difficult, and it requires a lot of discipline, but when you provide familiar terminology, users won't be disoriented by basic terms. To make users more comfortable, speak to them in their language. Don't expect them to learn yours, because most of them won't.

When you look at usability through the subjective lens of user confidence, you'll find opportunities to enhance your user experience ... even when you aren't necessarily fixing anything that's broken. While it's difficult to quantify, confidence is at the heart of what makes people like or dislike any product or tool. Pay careful attention to the level of confidence your users have throughout your application, and your application can reach new heights.

-Tony

November 1, 2013

Paving the Way for the DevOps Revolution

The traditional approach to software development has been very linear: Your development team codes a release and sends it over to a team of quality engineers to be tested. When everything looks good, the code gets passed over to IT operations to be released into production. Each of these teams operates within its own silo and makes changes independent of the other groups, and at any point in the process, it's possible a release can get kicked back to the starting line. With the meteoric rise of agile development — a development philosophy geared toward iterative and incremental code releases — that old waterfall-type development approach is being abandoned in favor of a DevOps approach.

DevOps — a fully integrated development and operations approach — streamlines the software development process in an agile development environment by consolidating development, testing and release responsibilities into one cohesive team. This way, ideas, features and other developments can be released very quickly and iteratively to respond to changing and growing market needs, avoiding the delays of long, drawn-out and timed dev releases.

To help you visualize the difference between the traditional approach and the DevOps approach, take a look at these two pictures:

Traditional Waterfall Development
SoftLayer DevOps Blog

DevOps
SoftLayer DevOps Blog

Unfortunately, many businesses struggle to adopt the DevOps approach because they simply update their org chart by merging their traditional teams, but their development philosophy doesn't change at the same time. As a result, I've encountered a lot of companies who have been jaded by previous attempts to move to a DevOps model, and I'm not alone. There is a significant need in the marketplace for some good old fashioned DevOps expertise.

A couple months ago, my friend Raj Bhargava pinged me with a phenomenal idea to put on a DevOps "un-conference" in Boulder, Colorado, to address the obvious need he's observed for DevOps education and best practices. Raj is a serial, multiple-exit entrepreneur from Boulder, and he is the co-founder and CEO of a DevOps-focused startup there called JumpCloud. When he asked if I would like to co-chair the event and have SoftLayer as a headline sponsor alongside JumpCloud, the answer was a quick and easy "Yes!"

Sure, there have been other DevOps-related conferences around the world, but ours was designed to be different from the outset. As strange as it may sound, half of the conference intentionally occurred outside of the conference: One of our highest priorities was to strike up conversations between the participants before, during and after the event. If we're putting on a conference to encourage a collaborative development approach, it would be counterproductive for us to use a top-down, linear approach to engaging the attendees, right?

I'm happy to report that this inaugural attempt of our untested concept was an amazing success. We kept the event private for our first run at the concept, but the event was bursting at the seams with brilliant developers and tech influencers. Brad Feld and our friends from the Foundry Group invited all of their portfolio CEO's and CTO's. David Cohen, co-founder of Techstars and head honcho at Bullet Time Ventures did the same. JumpCloud and SoftLayer helped round out the attendee list with a few of our most innovative partners as well as a few of technologists from within our own organizations. It was an incredible mix of super-smart tech pros, business leaders and VC's from all over the world.

With such a diverse group of attendees, the conversations at the event were engaging, energizing and profound. We discussed everything from how startups should incorporate automation into their business plans at the outset to how the practice of DevOps evolves as companies scale quickly. At the end of the day, we brought all of those theoretical discussions back down to the ground by sharing case studies of real companies that have had unbelievable success in incorporating DevOps into their businesses. I had the honor of wrapping up the event as moderator of a panel with Jon Prall from Sendgrid, Scott Engstrom from Gnip and Richard Miller of Mocavo, and I couldn't have been happier with the response.

I'd like to send a big thanks to everyone who participated, especially our cosponsors — JumpCloud, VictorOps, Authentic8, DH Capital, SendGrid, Cooley, Pivot Desk, SVP and Pantheon.

I'm looking forward to opening this up to the world next year!

-@PaulFord

October 24, 2013

Why Hybrid? Why Now?

As off-premise cloud computing adoption continues to grow in a non-linear fashion, a growing number of businesses running in-house IT environments are debating whether they should get on board as well. If you've been part of any of those conversations, you've tried to balance the hype with the most significant questions for your business: "How do we know if our company is ready to try cloud resources? And if we're ready, how do we actually get started?"

Your company is cloud-ready as soon as you understand and accept the ramifications of remote resources and scaling in the cloud model, and it doesn't have to be an "all-in" decision. If you need certain pieces of your infrastructure to reside in-house, you can start evaluating the cloud with workloads that don't have to be hosted internally. The traditional IT term for this approach is "hybrid," but that term might cause confusion these days.

In the simplest sense, a hybrid model is one in which a workload is handled by one or more non-heterogeneous elements. In the traditional IT sense, those non-heterogeneous elements are two distinct operating environments (on-prem and off-prem). In SoftLayer's world, a hybrid environment leverages different heterogeneous elements: Bare metal and virtual server instances, delivered in the cloud.

Figure 1: Traditional Hybrid - On-Premise to Cloud (Through VPN, SSL or Open Communications)

Traditional Hybrid

Figure 2: SoftLayer's Hybrid - Dedicated + Virtual

SoftLayer Hybrid

Because SoftLayer's "hybrid" and traditional IT's "hybrid" are so different, it's easy to understand the confusion in the marketplace: If a hybrid environment is generally understood to involve the connection of on-premise infrastructure to cloud resources, SoftLayer's definition seems contrarian. Actually, the use of the term is a lot more similar than I expected. In a traditional hosting environment, most businesses think in terms of bare metal (dedicated) servers, and when those businesses move "to the cloud," they're generally thinking in terms of virtualized server instances. So SoftLayer's definition of a hybrid environment is very consistent with the market definition ... It's just all hosted off-premise.

The ability to have dedicated resources intermixed with virtual resources means that workloads from on-premise hypervisors that require native or near-native performance can be moved immediately. And because those workloads don't have to be powered by in-house servers, a company's IT infrastructure moves a CapEx to an OpEx model. In the past, adopting infrastructure as a service (IaaS) involved shoehorning workloads into whichever virtual resource closest matched an existing environment, but those days are gone. Now, on-premise resources can be replicated (and upgraded) on demand in a single off-premise environment, leveraging a mix of virtual and dedicated resources.

SoftLayer's environment simplifies the process for businesses looking to move IT infrastructure off-premise. Those businesses can start by leveraging virtual server instances in a cloud environment while maintaining the in-house resources for certain workloads, and when those in-house resources reach the end of their usable life (or need an upgrade), the businesses can shift those workloads onto bare metal servers in the same cloud environment as their virtual server instances.

The real-world applications are pretty obvious: Your company is considering moving part of a workload to cloud in order to handle peak season loads at the end of the year. You've contemplated transitioning parts of your environment to the cloud, but you've convinced yourself that shared resource pools are too inefficient and full of noisy neighbor problems, so you'd never be able to move your core infrastructure to the same environment. Furthering the dilemma, you have to capitalize on the assets you already have that are still of use to the company.

You finally have the flexibility to slowly transition your environment to a scalable, flexible cloud environment without sacrificing. While the initial setup phases for a hybrid environment may seem arduous, Rome wasn't built in a day, so you shouldn't feel pressure to rush the construction of your IT environment. Here are a few key points to consider when adopting a hybrid model that will make life easier:

  • Keep it simple. Don't overcomplicate your environment. Keep networks, topologies and methodologies simple, and they'll be much more manageable and scalable.
  • Keep it secure. Simple, robust security principles will reduce your deployment timeframe and reduce attack points.
  • Keep it sane. Hybrid mixes the best of both worlds, so chose the best assets to move over. "Best" does not necessarily mean "easiest" or "cheapest" workload, but it doesn't exclude those workloads either.

With this in mind, you're ready to take on a hybrid approach for your infrastructure. There's no certification for when your company finally becomes a "cloud company." The moment you start leveraging off-premise resources, you've got a hybrid environment, and you can adjust your mix of on-premise, off-premise, virtual and bare metal resources as your business needs change and evolve.

-Jeff Klink

Jeff Klink is a senior technical staff member (STSM) with IBM Canada.

October 16, 2013

Tips and Tricks: Troubleshooting Email Issues

Working in support, one of the most common issues we troubleshoot is a customer's ability to receive email. Depending on email server, this can be a headache and a half to figure out, but more often than not, we're able to fix the problem with one of only a few simple solutions. Because the SoftLayer Blog audience loves technical tips and tricks, I thought I'd share a few easy steps that make pinpointing the root cause of email issues much easier.

Before you gear up to go into battle, check the that server is not out of disk space on /var and that it is not in a read only state. That precursory step may seem silly, but Occam's Razor often holds true in technical troubleshooting. Once you verify that those two common problems aren't causing your email problems, the next step is to determine whether the email issues are server-wide or isolated to one mail account/domain. To do that, the first thing you need to do is make sure that the IMAP and POP services are responding.

Check IMAP and POP Services

The universal approach to checking IMAP and POP services is to use telnet:

telnet <serverip> 110
telnet <serverip> 143

If either of those commands fail, you're able to pinpoint which service to check on your server.

For most variants of Linux, you can check both services with a single command: netstat -plan|egrep -i "110|143". The resulting output will show if the services are listening and which process is doing the listening. In Windows, you can run a similar command from a command prompt: netstat -anb|find "LISTEN"| findstr "110 143".

If the ports are listening, and you're able to connect to them over telnet, your next stop should be your server's error logs.

Check Error Logs

You want to look for any mail errors that might clue you into the root cause of your email issues. In Linux, you can check /var/log/maillog, and in Windows, you can filter eventvwr.msc for mail only. If there are errors, a simple search will highlight them quickly.

If there are no errors, it's time to dig into the mail queue directly.

Check the Mail Queue

Depending on the mail server you use, the commands here are going to vary. Here are a few examples of how we'd investigate the most common mail servers we encounter:

QMail

Display the mail queue: /var/qmail/bin/qmail-qread
Display the number of messages in the queue: /var/qmail/bin/qmail-qstat
Reference article: Gaining Control Over the QMail Queue

Sendmail

Display the mail queue: sendmail -bp or mailq
Display the number of messages in the queue: mailq –OmaxQueueRunSize=1
Reference article: Quick Sendmail Cheatsheet

Exim

Display the mail queue: exim -bp
Display the number of messages in the queue: exim -bpc
Reference article: Exim cheatsheet

MailEnable

MailEnable users can can check to see that messages are moving by opening the mail directory:
Program Files\MailEnable\Queues\SMTP\Inbound\Messages
Reference article: How to diagnose inbound message delivery delays

With these commands, you can filter through the email queues to see whether any of them are for the users or domains you're having problems with. If nothing obvious presents itself at that point, it's time for some active testing.

Active Testing

Send an email to your mailserver from an external mailserver (anything will do as long as it's not on the same server). Watch for logging of the email as it's delivered:
tail -f maillog
On busy mailservers you might add |grep youremailid or simply look for a new message in the directory where the email will be stored.

The your primary goal in troubleshooting your email issues in this way is to isolate the root cause of your problem so that you can fix it more quickly. SoftLayer customers have direct access to our support team to help you through this process, but it's always nice to keep a quick reference like this in your back pocket to be able to pinpoint the problem yourself.

-Bill

October 14, 2013

Product Spotlight: Vyatta Network Gateway Appliance

In the wake of our recent Vyatta network gateway appliance product launch, I thought I'd address some of the most common questions customers have asked me about the new offering. With inquiries spanning the spectrum from broad and general to detailed and specific, I might not be able to cover everything in this blog post, but at the very least, it should give a little more context for our new network gateway offering.

To begin, let's explore the simplest question I've been asked: "What is a network gateway?" A network gateway provides tools to manage traffic into and out of one or more VLANs (Virtual Local Area Networks). The network gateway serves a customer-configurable routing device that sits in front of designated VLANs. The servers in those VLANs route through the network gateway appliance as their first hop instead of Front-end Customer Routers (FCR) or Back-end Customer Routers (BCR). From an infrastructure perspective, SoftLayer's network gateway offering consists of a single server, and in the future, the offering will be expanded to multi-server configurations to support high availability needs and larger clustered configurations.

The general function of a network gateway may seem a little abstract, so let's look at a couple real world use cases to see how you can put that functionality to work in your own cloud environment.

Example 1: Complex Traffic Management
You have a multi-server cloud environment and a complex set of firewall rules that allow certain types of traffic to certain servers from specific addresses. Without a network gateway, you would need to configure multiple hardware and software firewalls throughout your topology and maintain multiple rules sets, but with the network gateway appliance, you streamline your configuration into a single point of control on both the public and private networks.

After you order a gateway appliance in the SoftLayer portal and configure which VLANs route through the appliance, the process of configuring the device is simple: You define your production, development and QA environments with distinct traffic rules, and the network gateway handles the traffic segmentation. If you wanted to create your own VPN to connect your hosted environment to your office or in-house data center, that configuration is quick and easy as well. The high-touch challenge of managing several sets of network rules across multiple devices is simplified and streamlined.

Example 2: Creating a Static NAT
You want to create a static NAT (Network Address Translation) so that you can direct traffic through a public IP address to an internal IP address. With the IPv4 address pool dwindling and new allocations being harder to come by, this configuration is becoming extremely popular to accommodate users who can't yet reach IPv6 addresses. This challenge would normally require a significant level of effort of even the most seasoned systems administrator, but with the gateway appliance, it's a painless process.

In addition to the IPv4 address-saving benefits, your static NAT adds a layer of protection for your internal web servers from the public network, and as we discussed in the first example, your gateway device also serves as a single configuration point for both inbound and outbound firewall rules.

If you have complex network-related needs, and you want granular control of the traffic to and from your servers, a gateway appliance might be the perfect tool for you. You get the control you want and save yourself a significant amount of time and effort configuring and tweaking your environment on-the-fly. You can terminate IPSec VPN tunnels, execute your own network address translation, and run diagnostic commands such as traffic monitoring (tcpdump) on your global environment. And in addition to that, your gateway serves as a single point of contact to configure sophisticated firewall rules!

If you want to learn more about the gateway appliance, check out KnowledgeLayer or contact our friendly sales team directly with your questions: sales@softlayer.com

-Ben

October 3, 2013

Improving Communications for Customer-Affecting Events

Service disruptions are never a good thing. Though SoftLayer invests extensively in design, equipment, and personnel training to reduce the risk of disruptions to our customers, in the technology world there are times where scheduled events or unplanned incidents are inevitable. During those times, we understand that restoring service is top priority, and almost as important is communicating to customers regarding the cause of the incident and the current status of our work to resolve it.

To date we've used a combination of tickets, emails, forum posts, portal "yellow" notifications, as well as RSS and Twitter feeds to provide status updates during service-affecting events. Many of these methods require customers to "come and get it," so we've been working on a more targeted, proactive approach to disseminating information.

I'm excited to report that our Development and Operations teams have collaborated on new functionality in the SoftLayer portal that will improve the way we share information with customers about unplanned infrastructure troubles or upcoming planned maintenances. With our new Event Communications toolset, we're able to pinpoint the accounts affected by an event and update users who opt-in to receive notifications about how these events may impact their services.

Notifications

As the development work is finalized, we plan to roll out a few phases of improvements. The first phase of implementation, which is ready today, enables email alerts for unplanned incidents, and any portal user account can opt-in to receive them. These emails provide details about the impact and current status of an unplanned incident in progress (UIP). In this phase, notifications can be sent for devices such as physical servers, CCIs and shared SLB VIPs, and we will be adding additional services over time.

In future phases of this project, we plan to include:

  • A new "Event" section of the Customer Portal which will allow customers to browse upcoming scheduled maintenances or current/recent unplanned incidents which may impact their services. In the past, we generated tickets for scheduled maintenances, so separating these event notifications will improve customer visibility.
  • Enhanced visibility for events in our mobile apps (phone/tablet).
  • Updates to affected services for a given event as customers add / change services.
  • Notification of newly added or newly updated events that have not been read by the user (similar email "inbox" functionality) in the portal.
  • Identification of any related current or recent events as a customer begins to open a ticket in the portal.
  • Reminders of upcoming scheduled maintenances along with progress updates to the event notification throughout the maintenance in some cases.
  • Improved ability to correlate specific incidents to customer service troubles.
  • Dissemination of RFO (reason-for-outage) statements to customers following a post-incident review of an unplanned service disruption.

Since we respect our customers' inboxes, these notifications will only be sent to user accounts that have opted in. If you'd like to receive them, simply log into the Customer Portal and navigate to "Notification Subscriptions" under the "Administration" menu (direct link). From that page, individual users can control event subscriptions, and portal logins that have administrative control over multiple users on the account can control the opt-in for themselves and their downstream users. For a more detailed walkthrough of the opt-in process, visit the KnowledgeLayer: "Update Subscription Settings for the Event Management System"

The Network Operations Center has already begun using this customer notification toolset for customer-affecting events, so we recommend that you opt-in as soon as possible to benefit from this new functionality.

-Dani

September 12, 2013

"Cloud First" or "Mobile First" - Which Development Strategy Comes First?

Company XYZ knows that the majority of its revenue will come from recurring subscriptions to its new SaaS service. To generate visibility and awareness of the SaaS offering, XYZ needs to develop a mobile presence to reach the offering's potential audience. Should XYZ focus on building a mobile presence first (since its timing is most critical), or should it prioritize the completion of the cloud service first (since its importance is most critical)? Do both have to be delivered simultaneously?

It's the theoretical equivalent of the "Which came first: The chicken or the egg?" causality dilemma for many technology companies today.

Several IBM customers have asked me recently about whether the implementation of a "cloud first" strategy or a "mobile first" strategy is most important, and it's a fantastic question. They know that cloud and mobile are not mutually exclusive, but their limited development resources demand that some sort of prioritization be in place. However, should this prioritization be done based on importance or urgency?

IBM MobileFirst

The answer is what you'd expect: It depends! If a company's cloud offering consists solely of back-end services (i.e. no requirement or desire to execute natively on a mobile device), then a cloud-first strategy is clearly needed, right? A mobile presence would only be effective in drawing customers to the back-end services if they are in place and work well. However, what if the cloud offering is targeting only mobile users? Not focusing on the mobile-first user experience could sabotage a great set of back-end services.

As this simple example illustrated, prioritizing one development strategy at the expense of the other strategy can have devastating consequences. In this "Is there an app for that?" generation, a lack of predictable responsiveness for improved quality of service and/or quality of experience can drive your customers to your competitors who are only a click away. Continuous delivery is an essential element of both "cloud first and "mobile first" development. The ability to get feedback quickly from users for new services (and more importantly incorporate that feedback quickly) allows a company to re-shape a service to turn existing users into advocates for the service as well as other adjacent or tiered services. "Cloud first" developers need a cloud service provider that can provide continuous delivery of predictable and superior compute, storage and network services that can be optimized for the type of workload and can adapt to changes in scale requirements. "Mobile first" developers need a mobile application development platform that can ensure the quality of the application's mobile user experience while allowing the mobile application to also leverage back-end services. To accommodate both types of developers, IBM established two "centers of gravity" to allow our customers to strike the right balance between their "cloud first" and "mobile first" development.

It should come as no surprise that the cornerstone of IBM's cloud first offering is SoftLayer. SoftLayer's APIs to its infrastructure services allow companies to optimize their application services based on the needs of application, and the SoftLayer network also optimizes delivery of the application services to the consumer of the service regardless of the location or the type of client access.

For developers looking to prioritize the delivery of services on mobile devices, we centered our MobileFirst initiative on Worklight. Worklight balances the native mobile application experience and integration with back-end services to streamline the development process for "mobile first" companies.

We are actively working on the convergence of our IBM Cloud First and Mobile First strategies via optimized integration of SoftLayer and Worklight services. IBM customers from small businesses through large enterprises will then be able to view "cloud first and "mobile first" as two sides of the same development strategy coin.

-Mac

Mac Devine is an IBM distinguished engineer, director of cloud innovation and CTO, IBM Cloud Services Division. Follow him on Twitter: @mac_devine.

Subscribe to technology