Tips And Tricks Posts

March 12, 2014

Name Servers: Don't Get Lost in (Domain) Translation

The Domain Name System (DNS) is vital to keeping the Internet in order and easily accessible. Every byte on the World Wide Web lives in (at least) one specific place on the planet, and it's mapped to that location with an IP address like 66.228.118.53 (IPv4) or 2607:f0d0:4545:3:200:f8ff:fe21:67cf (IPv6). DNS translates the domain names you know and love to the correct IP addresses, so without DNS, you would have to memorize a 32-bit or 128-bit IP address for your favorite websites. Remember the last time your cell phone died? How many phone numbers did you have memorized?

There are plenty of resources available online to explain How DNS Works, so in this post, we'll focus on the basics of how we use DNS. Here's the scenario: We want to register a domain — softlayer.com — and make it available to the masses.

1. Reserve and Register a Domain Name
The first thing we need to do is purchase and register our domain name. To do this, we need to choose a domain registrar and verify that our domain is available. Every domain registrar effectively provides the same service: It will reserve an available domain on your behalf, and it will let you dictate where that domain will live. There's not a lot of differentiation or value-add in that service, so many registrars offer cheap or free domains as loss-leaders for higher margin hosting or Web services. Be sure to check the fine print to make sure you're not committing to a year of hosting to get a $0.99 domain name. Some registrars make the process of updating and configuring where a domain resolves more difficult than others, but for the sake of this example we'll assume that your registrar allows the same easy accessibility SoftLayer provides our customers in the customer portal.

The domain name we want is softlayer.com, and in this example, that domain name is available for us to reserve. Once we go through the ordering process, we'll need to associate the domain with a set of authoritative name servers. Authoritative name servers are effectively the go-to address book for a specified domain. By default, your domain registrar will provide name servers for your domain, but they can be changed easily to meet your needs. You have four typical options when it comes to choosing your domain's name servers:

  • Use the domain registrar's default name servers.
  • Use you hosting provider’s name servers.
  • Use a third party DNS service to manage your domain names.
  • Run your own name servers on your server to manage your domain names.

Each of these options has its own pros and cons, but because we're just interested in getting our domain online, we'll use SoftLayer's DNS control panel to manage our new domain name.

2. Create DNS Records
When we access our hosting provider's DNS control panel, we see this:

DNS Management

This is an extremely high level view of DNS, so we’re just going to focus on what we must have in order to make softlayer.com reachable via browser. The first thing we'll do is add a DNS zone. This is usually our domain, but in some situations, it can be a bit different. In this example, we'll create a “softlayer.com” zone to be responsible for the whole softlayer.com domain:

DNS Management

With that zone created, we now need to add new "Address Records" (A Records) within that zone:

DNS Management
DNS Management

The terminology used in different DNS control panels may vary, so let’s breakdown what the four sections in those screenshots mean:

  • Resource Type: This is our DNS record type. In our example, we have A records which link a hostname to our IP address. There are a number of DNS record types, each serving a different purpose.
  • Host: This is the host node or owner name — the name of the node that this record applies to. Using the @ symbol in the A record allows visitors to reach our website without the leading www. If we wanted blog.softlayer.com to live at a different IP address, we'd make that happen here.
  • Points To: This is the IP address of the host node. You might see this section referred to elsewhere as content, data or value. The standard term is RDATA — resource record data. This is specific to each data type.
  • TTL (Time-to-Live): TTL dictates how long your name server should keep a particular record before refreshing for possible updates. Generally speaking, longer TTLs work well if you’re just adding new entries and or don’t anticipate frequent record changes.

Once we save these changes in our DNS control panel, we play the waiting game. Because these DNS changes have to propagate across our DNS servers to be accessible to the Internet as a whole, the process typically takes 24-48 hours, if not sooner. SoftLayer’s customer portal has DNS check built-in as one a few different network tools. If you aren't a current customer, you can use What's my DNS? This is what the SoftLayer tool looks like:

DNS Management

3. Create rDNS Records
The last step we want to take in setting up our domain is to create Reverse DNS (rDNS) records. These records do the same thing as DNS records, but (as the name suggests) they function in the opposite direction. With rDNS, we can assign an IP address to a domain name. This step isn't required, but I recommend it to help ensure better performance of online activities like email and website visitor tracking.

DNS is a central piece of the Internet as we know it, so by understanding how to use it, you'll have a much better understanding of how the Internet works. It seems challenging at first glance, but as you see from this simple walkthrough, when you break down and understand each step, you won't get overwhelmed. A wealth of DNS tools and tutorials are available for free online, and our DNS documentation might be a great resource to bookmark so you'll never get lost in domain translation.

- Landon

February 3, 2014

Risk Management: 5 Tips for Managing Risk in the Cloud

Security breaches have made front-page news in recent months. With stories about Target, Neiman Marcus, Yahoo! and GoDaddy in the headlines recently, the importance of good information security practices is becoming harder and harder to ignore — even for smaller businesses. Moving your business into the cloud offers a plethora of benefits; however, those benefits do not come without their challenges. Moving your business into the cloud involves risks such as multi-tenancy, so it's important to be able to properly manage and identify these risks.

1. Know the Security Your Provider Offers
While some SaaS providers may have security baked-in, most IaaS providers (including SoftLayer) leave much of the logical security responsibility of a customer's systems to the customer. For the security measures that an infrastructure provider handles, the provider should be able to deliver documentation attesting these controls. We perform an annual SOC2 audit, so we can attest to the status of our security and availability controls as a service organization. With this information, our customers use controls from our report as part of their own compliance requirements. Knowing a provider's security controls (and seeing proof of that security) allows business owners and Chief Information Security Officers (CISO) to have peace-of-mind that they can properly plan their control activities to better prevent or respond to a breach.

2. Use the Cloud to Distribute and Replicate Your Presence
The incredible scalability and geographical distribution of operating in the cloud can yield some surprising payoff. Experts in the security industry are leveraging the cloud to reduce their patch cycles to days, not weeks or months. Most cloud providers have multiple sites so that you can spread your presence nationally, or even globally. With this kind of infrastructure footprint, businesses can replicate failover systems and accommodate regional demand across multiple facilities with the minimal incremental investment (and with nearly identical security controls).

3. Go Back to the Basics
Configuration management. Asset management. Separation of duties. Strong passwords. Many organizations get so distracted by the big picture of their security measures that they fail to manage these basic rights. Take advantage of any of your provider's tools to assist in the ‘mundane’ tasks that are vitally important to your business's overall security posture. For example, you can use image templates or post-provisioning scripts to deploy a standard baseline configuration to your systems, then track them down to the specific server room. You’ll know what hardware is in your server at all times, and if you're using SoftLayer, you can even drill down to the serial numbers of your hard drives.

4. Have Sound Incident Response Plans
The industry is becoming increasingly cognizant of the fact that it’s not a matter of if, but when a security threat will present itself. Even with exceedingly high levels of baked-in security, most of the recent breaches resulted from a compromised employee. Be prepared to respond to security incidents with confidence. While you may be physically distanced from your systems, you should be able to meet defined Recovery Time Objectives (RTOs) for your services.

5. Maintain Constant Contact with Your Cloud Provider
Things happen. No amount of planning can completely halt every incident, whether it be a natural disaster or a determined attacker. Know that your hosting provider has your back when things take an unexpected turn.

With proper planning and good practice, the cloud isn't as risky and frightening as most think. If you're interested in learning a little more about the best practices around security in the cloud, check out the Cloud Security Alliance (CSA). The CSA provides a wealth of knowledge to assist business owners and security professionals alike. Build on the strengths, compensate for the weaknesses, and you and your CISO will be able to sleep at night (and maybe even sneak in a beer after work).

-Matt

December 5, 2013

How to Report Abuse to SoftLayer

When you find hosted content that doesn't meet our acceptable use policy or another kind of inappropriate Internet activity originating from a SoftLayer service, your natural reaction might be to assume, "SoftLayer must know about it, and the fact that it's going on suggests that they're allowing that behavior." I know this because every now and then, I come across a "@SoftLayer is phishing my email. #spamming #fail" Tweet or a "How about u stop hacking my computer???" Facebook post. It's easy to see where these users are coming from, so my goal for this post is to provide the background you need to understand how behavior we don't condone — what we consider "abuse" of our services — might occur on our platform and what we do when we learn about it.

The most common types of abuse reported from the SoftLayer network are spam, copyright/trademark infringement, phishing and abusive traffic (DDoS attacks). All four are handled by the same abuse team, but they're all handled a bit differently, so it's important to break them down to understand the most efficient way to report them to our team. When you're on the receiving end of abuse, all you want is to make it stop. In the hurry to report the abusive behavior, it's easy to leave out some of the key information we need to address your concern, so let's take a look at each type of abuse and the best ways to report it to the SoftLayer team:

If You Get Spam

Spam is the most common type of abuse that gets reported to SoftLayer. Spam email is unsolicited, indiscriminate bulk messaging that is sent to you without your explicit consent. If you open your email client right now, your junk mail folder probably has a few examples of spam ... Someone is trying to sell you discount drugs or arrange a multi-million dollar inheritance transfer. In many ways, it's great that email is so easy to use and pervasive to our daily lives, but that ease of use also makes it an easy medium for spammers to abuse. Whether the spammer is a direct SoftLayer customer or a customer of one of our customers or somewhere further down the line of customers of customers, spam messages sent from a SoftLayer server will point back to us, and our abuse team is the group that will help stop it.

When you receive spam sent through SoftLayer, you should forward it directly to our abuse team (abuse@softlayer.com). Our team needs a full copy of the email with its headers intact. If you're not sure what that means, check out these instructions on how to retrieve your email headers. The email headers help tell the story about where exactly the messages are coming from and which customer we need to contact to stop the abuse.

If You See Phishing

Phishing abuse might be encountered via spam or you might encounter it on a website. Phishing is best described as someone masquerading as someone else to get your sensitive information, and it's one of the most serious issues our abuse team faces. Every second that a phishing/scam site is online, another user might be fooled into giving up his or her credit card or login information, and we don't want that to happen. Often, the fact that a site is not legitimate is clear relatively quickly, but as defenses against phishing have gotten better, so have the phishing sites. Take a minute to go through this phishing IQ test to get an idea of how difficult phishing can be to trace.

When it comes to reporting phishing, you should send the site's URL to the abuse team (also using abuse@softlayer.com). If you came across the phishing site via a spam email, be sure to include the email headers with your message. To help us filter the phishing complaint, please make sure to include the word "phishing" in your email's subject line. Our team will immediately investigate and follow up with the infringing customer internally.

If You Find Copyright or Trademark Infringement

If infringement of your copyright or trademark is happening on our platform, we want to know about it so we can have it taken down immediately. Copyright complaints and trademark complaints are handled slightly differently, so let's look at each type to better understand how they work.

Complaints of copyright infringement are processed by our abuse team based on the strict DMCA complaint laws. When I say "strict" in that sentence, I'm not saying it lightly ... Because DMCA complaints are legal issues, every requirement in the DMCA must be met in order for our team to act on the complaint. That might seem arbitrary, but we're not given much leeway when it comes to the DMCA process, and we have to be sticklers.

On our DMCA legal page, we outline the process of reporting a DMCA complaint of copyright infringement (primarily citing the statute 17 U.S.C. Section 512(c)(3)). If you don't completely understand what needs to be included in the claim, we recommend that you seek independent legal advice. It sounds harsh, but failure to submit copyright infringement notification as described above will result in no legal notice or action on behalf of SoftLayer. When you've made sure all required evidence has been included in your DMCA complaint, make sure "copyright" or "DMCA" are included in your subject line and submit the complaint to copyright@softlayer.com.

Trademark complaints do not have the same requirements as copyright complaints, but the more information you can provide in your complaint, the easier it will be for our customer to locate and remove the offending material. If you encounter unauthorized use of your registered trademark on our network, please email copyright@softlayer.com with details — the exact location of the infringing content, your trademark registration information, etc. — along with an explanation that this trademark usage is unauthorized and should be removed. In your email, please add the word "trademark" to the subject line to help us filter and prioritize your complaint.

If You See Abusive Traffic

Spam, phishing and copyright infringement are relatively straightforward when it comes to finding and reporting abuse, but sometimes the abuse isn't as visible and tangible (though the effect usually is). If a SoftLayer server is sending abusive traffic to your site, we want to know about it as quickly as possible. Whether that behavior is part of a Denial of Service (DoS) attack or is just scanning ports to possibly attack later, it's important that you give us details so we can prevent any further activity.

To report this type of abuse, send a snippet from your log file including at least 10 lines of logs that show attempts to break into or overload your server. Here's a quick reference to where you can find the relevant logs to send:

  • Email Spam - Send Mail Logs:
    • /var/log/maillog
    • /usr/local/psa/var/log/maillog
  • Brute Force Attacks - Send SSH Logs:
    • /var/log/messages
    • /var/log/secure

Like spam and phishing reports, abusive traffic complaints should be sent to abuse@softlayer.com with a quick explanation of what is happening and any other details you can provide. When you submit a complaint about abusive traffic, make sure your message's subject line reflects the type of issue ("DDoS attack," "brute force attempts," etc.) so our team can investigate your report even quicker.

As I mentioned at the start of this post, these are just four types of abusive behavior that our abuse department addresses on a daily basis. Our Acceptable Use Policy (AUP) outlines what can and cannot be hosted using SoftLayer services, and the process of reporting other types of abuse is generally the same as what you see in the four examples I mentioned above ... Send a clear, concise report to abuse@softlayer.com with key words about the type of violation in the message's subject line. When our team is able to look into your complaint and find the evidence they need to take action, they do so quickly.

I can't wrap up this blog of tips without mentioning the "Tips from the Abuse Department" blog Jennifer Groves wrote about reporting abuse ... It touches on some of the same ideas as this post, and it also provides a little more perspective from behind the lines of the abuse department. As the social media gal, I don't handle abuse on a day-to-day basis, but I do help people dealing with abuse issues, and I know a simple guide like this will be of value.

If an abuse-related issue persists and you don't feel like anything has been fixed, double-check that you've included all the necessary information and evidence in your correspondence to the abuse team. In most cases, you will not receive a response from the abuse team, but that doesn't mean they aren't taking action. The abuse@ and copyright@ email aliases function as notification systems for our abuse teams, and they correspond with the infringing customers internally when a complaint is submitted. Given the fact that hundreds of users may report the same abusive behavior at the same time, responding directly to each message would slow down the process of actually resolving the issue (which is the priority).

If everything was included in your initial correspondence with the abuse team but you still don't notice a change in the abusive behavior, you can always follow up with our social media team at twitter@softlayer.com, and we'll do everything we can to help.

-Rachel

November 19, 2013

Protect Your Data: Configure EVault for Server Backups

In "The Tenth Anniversary" episode of "Everybody Loves Raymond," Raymond accidentally records the Super Bowl over his wedding video. He hilariously tries to compensate for his gaffe by renewing his wedding vows so he can make a new tape for his wife Debra. If life imitates art, it's worth considering what would happen if that tape held your business data. It would be disaster!

While it's unlikely that one of your sysadmins will accidentally record the Super Bowl over the data in your database server cluster, data loss can occur in a number of ways. If your business data is not protected and backed up, it's unlikely that you'll have a neat and tidy sitcom episode resolution. Luckily, SoftLayer provides simple, inexpensive backup capabilities with software such as EVault, so you shouldn't ever be worried about anyone pulling a Raymond on your data.

The following quick, four-step process walks you through how to protect and back up your data by subscribing to SoftLayer's EVault Backup client. This software enables you to design and set your backup schedule, protecting your business from unexpected costs because of accidental deletions, viruses, and other disasters. To follow along on your own servers, your computing instances or bare metal servers need to be provisioned, and you need to have root or administrator level access to those servers. For the sake of brevity, I'll be using a Linux operating system in this guide, but if you're running Windows, the process, in general, is no different.

Step 1 - Order EVault Backup for the server or computing instance

  1. Log into the SoftLayer Customer Portal and select the server(s) that needs storage services from the device list.
  2. Scroll down to the Storage section. Select the Add (or Modify) link located on the right hand corner of the EVault record to place an order for an EVault Backup client subscription.
  3. On the EVault ordering screen, select either Local or Remote Data Center and the desired amount of storage. Agree to the terms and conditions and click the Order EVault button to place your EVault storage order.
  4. The order is typically provisioned in 5 minutes or less and the system creates a user and password for the new instance of EVault.
  5. Click Services→Storage→EVault and expand the EVAULT link to make note of the user credentials, which will be used in Step 3.

Step 2 - Download the EVault agent on the server or computing instance

  1. SSH into the server or computing instance and run the following command:
    # wget –N http://downloads.service.softlayer.com/evault/evault_manual.sh

Step 3 - Register the server or computing instance with EVault in order to run back up and restore jobs

  1. From the command prompt on the server or compute instance run the following command to register it with EVault:
    ~]# sh ./evault_manual.sh
  2. In the ensuing prompts, enter the credentials that were noted Step 1.5 and use ev-webcc01.service.softlayer.com for the web-based agent console address.

    Note: In the event the agent fails to register with EVault, you can quickly register the agent manually by running ~]#<Installation directory>/register

Once you've made it to this point, you're ready to run backup and restore jobs.

Step 4 – Login into EVault console with WebCCLogin

  1. From the SoftLayer Customer Portal, click Services→Storage→EVault.
  2. Expand the server or compute instance to which EVault Backup is attached. In the right-hand corner of the server entry you will find a link to WebCCLogin.
  3. Click the WebCCLogin link for the EVault Web CentralControl screen. Type in the credentials from Step 1.5 and you’ll be taken to the EVault Backup and Restore interface.
  4. You are now ready to run your backup and restore jobs!

Check your backups often to confirm that they're being created when, where, and how you want them to be created. To prepare for any possible disaster recovery scenarios, schedule periodic tests of your backups: Restore the most recent backup of your production server to an internal server. That way, if someone pulls a Raymond on your server(s), you'll be able to get all of your data back online quickly. If you're interested in learning more, visit the Evault Backup page on KnowledgeLayer.

-Vinayak Harnoor

Vinayak Harnoor is a Technical Architect with the IBM Global Technology Services (GTS) Global Cloud Ecosystem team.

November 14, 2013

Enhancing Usability by Building User Confidence

Consider your experiences with web applications, and see if this scenario seems familiar: Your electricity bill has some incorrect charges on it. Fearing that you will have to spend 40 minutes on hold if you call in, you find that the electric company website has a support center where you can submit billing issues and questions; you are saved! You carefully fill out the form with your sixteen-digit account number and detailed description of the incorrect charges. You read it over and click the submit button. Your page goes blank for a couple of seconds, the form comes back with a note saying you typed in your phone number incorrectly, and the detailed description you spent eleven minutes meticulously writing is gone.

Web applications have gotten much better at preventing these kinds of user experiences over the past few years, and I'm sure that none of your applications have this problem (if they do, fix it right now!), but "usability" is more than just handling errors gracefully. Having a seamless process is only half the battle when it comes to giving your users a great experience with your application. The other half of the battle is a much more subjective: Your users need to feel confident in their success every step of the way. By keeping a few general guidelines in mind, you can instill confidence in your users so that they feel positive about your application from start to finish with whatever they are trying to accomplish.

1. Keep the user in a familiar context.

As the user in our electric company support application example, let's assume the process works and does not lose any of my information. I have to have faith that the application is going to do what I expect it to do when the page refreshes. Faith and unfamiliar technology do not exactly go hand in hand. Instead of having the form submit with a page refresh, the site's developers could introduce a progress wheel or other another kind of indicator that shows the data is being submitted while the content is still visible. If detailed content never goes away during the submission process, I'm confident that I still have access to my information.

Another example of the same principle is the use of modal windows. Modal windows are presented on top of a previous page, so users have a clear way of going back if they get confused or decide they navigated to the wrong place. By providing this new content on top of a familiar page, users are much less likely to feel disoriented if they get stuck or lost, and they will feel more confident when they're using the application.

2. Reassure the user with immediate feedback.

By communicating frequently and clearly, users are reassured, and they are much less likely to become anxious. Users want to see their actions get a response from your application. In our electric company support application example, imagine how much better the experience would be if a small blurb was displayed in red next to the phone number text box when I typed in my phone number in the wrong format. The immediate feedback would pinpoint the problem when it is easy to correct, and it would make me confident that when the phone number is updated, the application will continue to work as expected.

3. Provide warnings or extra information for dangerous or complicated operations.

When users are new to an application, they are not always sure which actions will have negative consequences. This is another great opportunity for communication. Providing notices or alerts for important or risky operations can offer a good dose of hesitation for new users who aren't prepared. Effective warnings or notices will tell the user when they will want to perform this action or what the negative consequences might be, so the user can make an informed decision. Users are confident with informed decisions because a lack information causes anxiety.

I learned how to implement this tip when I designed a wizard system for a previous employer that standardized how the company's application would walked users through any step-by-step process. My team decided early on to standardize a review step at the end of any implemented wizard. This was an extra step that every user had to go through for every wizard in the application, but it made all of the related processes much more usable and communicative. This extra information gave the users a chance to see the totality of the operation they were performing, and it gave them a chance to correct any mistakes. Implementing this tip resulted in users who were fully informed and confident throught the process of very complicated operations.

4. Do not assume your users know your terminology, and don't expect them to learn it.

Every organization has its own language. I have never encountered an exception to this rule. It cannot be helped! Inside your organization, you come up with a defined vocabulary for referencing the topics you have to work with every day, but your users won't necessarily understand the terminology you use internally. Some of your ardent users pick up on your language through osmosis, but the vast majority of users just get confused when they encounter terms they are not familiar with.

When interacting with users, refrain from using any of your internal language, and strictly adhere to a universally-accepted vocabulary. In many cases, you need shorthand to describe complex concepts that users will already understand. In this situation, always use universal or industry-wide vocabulary if it is available.

This practice can be challenging and will often require extra work. Let's say you have a page in your application dealing with "display devices," which could either be TVs or monitors. All of your employees talk about display devices because to your organization, they are essentially the same thing. The technology of your application handles all display devices in exactly the same way, so as good software designers you have this abstracted (or condensed for non-technical people) so that you have the least amount of code possible. The easiest route is to just have a page that talks about display devices. The challenge with that approach is that your users understand what monitors and TVs are, but they don't necessarily think of those as display devices.

If that's the case, you should use the words "monitors" and "TVs" when you're talking about display devices externally. This can be difficult, and it requires a lot of discipline, but when you provide familiar terminology, users won't be disoriented by basic terms. To make users more comfortable, speak to them in their language. Don't expect them to learn yours, because most of them won't.

When you look at usability through the subjective lens of user confidence, you'll find opportunities to enhance your user experience ... even when you aren't necessarily fixing anything that's broken. While it's difficult to quantify, confidence is at the heart of what makes people like or dislike any product or tool. Pay careful attention to the level of confidence your users have throughout your application, and your application can reach new heights.

-Tony

November 11, 2013

Sysadmin Tips and Tricks - Using the ‘for’ Loop in Bash

Ever have a bunch of files to rename or a large set of files to move to different directories? Ever find yourself copy/pasting nearly identical commands a few hundred times to get a job done? A system administrator's life is full of tedious tasks that can be eliminated or simplified with the proper tools. That's right ... Those tedious tasks don't have to be executed manually! I'd like to introduce you to one of the simplest tools to automate time-consuming repetitive processes in Bash — the for loop.

Whether you have been programming for a few weeks or a few decades, you should be able to quickly pick up on how the for loop works and what it can do for you. To get started, let's take a look at a few simple examples of what the for loop looks like. For these exercises, it's always best to use a temporary directory while you're learning and practicing for loops. The command is very powerful, and we wouldn't want you to damage your system while you're still learning.

Here is our temporary directory:

rasto@lmlatham:~/temp$ ls -la
total 8
drwxr-xr-x 2 rasto rasto 4096 Oct 23 15:54 .
drwxr-xr-x 34 rasto rasto 4096 Oct 23 16:00 ..
rasto@lmlatham:~/temp$

We want to fill the directory with files, so let's use the for loop:

rasto@lmlatham:~/temp$ for cats_are_cool in {a..z}; do touch $cats_are_cool; done;
rasto@lmlatham:~/temp$

Note: This should be typed all in one line.

Here's the result:

rasto@lmlatham:~/temp$ ls -l
total 0
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 a
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 b
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 c
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 d
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 e
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 f
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 g
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 h
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 i
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 j
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 k
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 l
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 m
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 n
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 o
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 p
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 q
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 r
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 s
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 t
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 u
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 v
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 w
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 x
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 y
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 z
rasto@lmlatham:~/temp$

How did that simple command populate the directory with all of the letters in the alphabet? Let's break it down.

for cats_are_cool in {a..z}

The for is the command we are running, which is built into the Bash shell. cats_are_cool is a variable we are declaring. The specific name of the variable can be whatever you want it to be. Traditionally people often use f, but the variable we're using is a little more fun. Hereafter, our variable will be referred to as $cats_are_cool (or $f if you used the more boring "f" variable). Aside: You may be familiar with declaring a variable without the $ sign, and then using the $sign to invoke it when declaring environment variables.

When our command is executed, the variable we declared in {a..z}, will assume each of the values of a to z. Next, we use the semicolon to indicate we are done with the first phase of our for loop. The next part starts with do, which say for each of a–z, do <some thing>. In this case, we are creating files by touching them via touch $cats_are_cool. The first time through the loop, the command creates a, the second time through b and so forth. We complete that command with a semicolon, then we declare we are finished with the loop with "done".

This might be a great time to experiment with the command above, making small changes, if you wish. Let's do a little more. I just realized that I made a mistake. I meant to give the files a .txt extension. This is how we'd make that happen:

for dogs_are_ok_too in {a..z}; do mv $dogs_are_ok_too $dogs_are_ok_too.txt; done;
Note: It would be perfectly okay to re-use $cats_are_cool here. The variables are not persistent between executions.

As you can see, I updated the command so that a would be renamed a.txt, b would be renamed b.txt and so forth. Why would I want to do that manually, 26 times? If we check our directory, we see that everything was completed in that single command:

rasto@lmlatham:~/temp$ ls -l
total 0
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 a.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 b.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 c.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 d.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 e.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 f.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 g.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 h.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 i.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 j.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 k.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 l.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 m.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 n.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 o.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 p.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 q.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 r.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 s.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 t.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 u.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 v.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 w.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 x.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 y.txt
-rw-rw-r-- 1 rasto rasto 0 Oct 23 16:13 z.txt
rasto@lmlatham:~/temp$

Now we have files, but we don't want them to be empty. Let's put some text in them:

for f in `ls`; do cat /etc/passwd > $f; done

Note the backticks around ls. In Bash, backticks mean, "execute this and return the results," so it's like you executed ls and fed the results to the for loop! Next, cat /etc/passwd is redirecting the results to $f, in filenames a.txt, b.txt, etc. Still with me?

So now I've got a bunch of files with copies of /etc/passwd in them. What if I never wanted files for a, g, or h? First, I'd get a list of just the files I want to get rid of:

rasto@lmlatham:~/temp$ ls | egrep 'a|g|h'
a.txt
g.txt
h.txt

Then I could plug that command into the for loop (using backticks again) and do the removal of those files:

for f in `ls | egrep 'a|g|h'`; do rm $f; done

I know these examples don't seem very complex, but they give you a great first-look at the kind of functionality made possible by the for loop in Bash. Give it a whirl. Once you start smartly incorporating it in your day-to-day operations, you'll save yourself massive amounts of time ... Especially when you come across thousands or tens of thousands of very similar tasks.

Don't do work a computer should do!

-Lee

October 22, 2013

JumpCloud: Tech Partner Spotlight

We invite each of our featured SoftLayer Tech Marketplace Partners to contribute a guest post to the SoftLayer Blog, and this week, we're happy to welcome David Campbell from JumpCloud. JumpCloud is an automated SaaS-based offering that automates the manual, tedious system administration tasks for DevOps and IT pros. It works with your provisioning to complete your operations set by automating server maintenance, management, monitoring, and security.

User Management in a DevOps World

Maybe you're a developer who's recently been given responsibility for managing production infrastructure at your company. Or maybe you're a career SysAdmin whose boss read the DevOps Cookbook and decided that it's time for you to learn to embrace DevOps and start treating your configuration as code and automating everything. DevOps promises to change the way organizations develop, operate and maintain applications and IT infrastructure, both on-premise and in the cloud. However you came upon it, you're now firmly entrenched in the world of DevOps.

No matter what your background, you're probably not alone in terms of needing access to the servers in your environment. Which brings us to the topic of this post. It's bad practice to use a shared "root" account to manage your systems and especially to run your application. So you want to create and manage separate user accounts. This is easy enough to do manually when you have only one or two admins and just a couple of servers. But in today's elastic, auto-scaling environments, you may have two servers at 9am and 1200 servers at 3pm.

So what to do?

In short, what you want is a method by which you can have each admin within your organization have their own user account on all of the systems within your organization to which they should have access. You want to require the admins to use ssh keys to authenticate to the servers, as requiring key based auth will make it impossible for brute force attackers to guess passwords in order to compromise your systems. You likely will want to grant "sudo" access to certain admins, and have them prove their identity to the system before executing privileged commands by entering their password. You may want to require multi factor authentication for admin shell access to especially critical systems, like production database servers.

Access needs to be granted when new admins join your team, and when new servers are brought up in the environment. That's where it gets complicated. Maybe you don't want the junior admin having full access to the customer database system? Access also needs to be removed when somebody inevitably leaves the company, sometimes unexpectedly.

There are a lot of DevOps friendly ways to automate the process of provisioning and deprovisioning user accounts. Techniques can be as simple as using rsync to copy "shadow files" from one system in the environment to all systems in the environment, though this can be tricky to manage in auto-scaling environments.

More advanced approaches involve using configuration management tools like Puppet or Chef to manage local user accounts on managed systems. These tools have native capability for user management, but do not provide any centralized audit trail about who is doing what on your servers. They also make it difficult for the user to select their own initial credentials, or change them down the road should they be forgotten or compromised. Using configuration management tools to manage user accounts also requires "code changes" to add or remove users, and changes can take 30 minutes or more to propagate through your whole environment.

If you want to automate and streamline your server user management process or you're interested in enhancing the security of your infrastructure, visit JumpCloud. We can help make quick work of tedious user management and security issues so that you can get back to growing your business.

-David Campbell, JumpCloud

This guest blog series highlights companies in SoftLayer's Technology Partners Marketplace.
These Partners have built their businesses on the SoftLayer Platform, and we're excited for them to tell their stories. New Partners will be added to the Marketplace each month, so stay tuned for many more come.
October 16, 2013

Tips and Tricks: Troubleshooting Email Issues

Working in support, one of the most common issues we troubleshoot is a customer's ability to receive email. Depending on email server, this can be a headache and a half to figure out, but more often than not, we're able to fix the problem with one of only a few simple solutions. Because the SoftLayer Blog audience loves technical tips and tricks, I thought I'd share a few easy steps that make pinpointing the root cause of email issues much easier.

Before you gear up to go into battle, check the that server is not out of disk space on /var and that it is not in a read only state. That precursory step may seem silly, but Occam's Razor often holds true in technical troubleshooting. Once you verify that those two common problems aren't causing your email problems, the next step is to determine whether the email issues are server-wide or isolated to one mail account/domain. To do that, the first thing you need to do is make sure that the IMAP and POP services are responding.

Check IMAP and POP Services

The universal approach to checking IMAP and POP services is to use telnet:

telnet <serverip> 110
telnet <serverip> 143

If either of those commands fail, you're able to pinpoint which service to check on your server.

For most variants of Linux, you can check both services with a single command: netstat -plan|egrep -i "110|143". The resulting output will show if the services are listening and which process is doing the listening. In Windows, you can run a similar command from a command prompt: netstat -anb|find "LISTEN"| findstr "110 143".

If the ports are listening, and you're able to connect to them over telnet, your next stop should be your server's error logs.

Check Error Logs

You want to look for any mail errors that might clue you into the root cause of your email issues. In Linux, you can check /var/log/maillog, and in Windows, you can filter eventvwr.msc for mail only. If there are errors, a simple search will highlight them quickly.

If there are no errors, it's time to dig into the mail queue directly.

Check the Mail Queue

Depending on the mail server you use, the commands here are going to vary. Here are a few examples of how we'd investigate the most common mail servers we encounter:

QMail

Display the mail queue: /var/qmail/bin/qmail-qread
Display the number of messages in the queue: /var/qmail/bin/qmail-qstat
Reference article: Gaining Control Over the QMail Queue

Sendmail

Display the mail queue: sendmail -bp or mailq
Display the number of messages in the queue: mailq –OmaxQueueRunSize=1
Reference article: Quick Sendmail Cheatsheet

Exim

Display the mail queue: exim -bp
Display the number of messages in the queue: exim -bpc
Reference article: Exim cheatsheet

MailEnable

MailEnable users can can check to see that messages are moving by opening the mail directory:
Program Files\MailEnable\Queues\SMTP\Inbound\Messages
Reference article: How to diagnose inbound message delivery delays

With these commands, you can filter through the email queues to see whether any of them are for the users or domains you're having problems with. If nothing obvious presents itself at that point, it's time for some active testing.

Active Testing

Send an email to your mailserver from an external mailserver (anything will do as long as it's not on the same server). Watch for logging of the email as it's delivered:
tail -f maillog
On busy mailservers you might add |grep youremailid or simply look for a new message in the directory where the email will be stored.

The your primary goal in troubleshooting your email issues in this way is to isolate the root cause of your problem so that you can fix it more quickly. SoftLayer customers have direct access to our support team to help you through this process, but it's always nice to keep a quick reference like this in your back pocket to be able to pinpoint the problem yourself.

-Bill

September 24, 2013

Four Rules for Better Code Documentation

Last month, Jeremy shared some valuable information regarding technical debt on SLDN. In his post, he discussed how omitting pertinent information when you're developing for a project can cause more work to build up in the future. One of the most common areas developers overlook when it comes to technical debt is documentation. This oversight comes in two forms: A complete omission of any documentation and inadequate information when documentation does exist. Simply documenting the functionality of your code is a great start, but the best way to close the information gap and avoid technical debt that stems from documentation (or lack thereof) is to follow four simple rules.

1. Know Your Audience

When we're talking about code, it's safe to say you'll have a fairly technical audience; however, it is important to note the level of understanding your audience has on the code itself. While they should be able to grasp common terms and development concepts, they may be unfamiliar with the functionality you are programming. Because of this, it's a good idea to provide a link to an internal, technical knowledgebase or wiki that will provide in-depth details on the functionality of the technology they'll be working with. We try to use a combination of internal and external references that we think will provide the most knowledge to developers who may be looking at our code. Here's an example of that from our Dns_Domain class:

 * @SLDNDocumentation Service Overview <<< EOT
 * SoftLayer customers have the option of hosting DNS domains on the SoftLayer
 * name servers. Individual domains hosted on the SoftLayer name servers are
 * handled through the SoftLayer_Dns_Domain service.
 *
 * Domain changes are applied automatically by our nameservers, but changes may
 * not be received by the other name servers on the Internet for 72 hours after
 * your change. The SoftLayer_Dns_Domain service does not apply to customers who
 * run their own nameservers on servers purchased from SoftLayer.
 *
 * SoftLayer provides secondary DNS hosting services if you wish to maintain DNS
 * records on your name server, but have records replicated on SoftLayer's name
 * servers. Use the [[SoftLayer_Dns_Secondary]] service to manage secondary DNS
 * zones and transfers.
 * EOT
 *
 * @SLDNDocumentation Service External Link http://en.wikipedia.org/wiki/Domain_name_system Domain Name System at Wikipedia
 * @SLDNDocumentation Service External Link http://tools.ietf.org/html/rfc1035 RFC1035: Domain Names - Implementation and Specification at ietf.org
 * @SLDNDocumentation Service See Also SoftLayer_Dns_Domain_ResourceRecord
 * @SLDNDocumentation Service See Also SoftLayer_Dns_Domain_Reverse
 * @SLDNDocumentation Service See Also SoftLayer_Dns_Secondary
 *

Enabling the user to learn more about a topic, product, or even a specific call alleviates the need for users to ask multiple questions regarding the "what" or "why" and will also minimize the need for you to explain more basic concepts regarding the technology supported by your code.

2. Be Consistent - Terminology

There are two main areas developers should focus on when it comes to consistency: Formatting and terminology.

Luckily, formatting is pretty simple. Most languages have a set of standards attached to them that extend to the Docblock, which is where the documentation portion of the code normally takes place. Docblocks can be used to provide an overview of the class, identify authors or product owners and provide additional reference to those using the code. The example below uses PHP's standards for documentation tagging and allows users to quickly identify the parameters and return value for the createObject method in the Dns_Domain class:

*
     * @param string $objectType
     * @param object $templateObject
     *
     * @return SoftLayer_Dns_Domain
     */
   public static function createObject($objectType = __CLASS__, $templateObject)

Keeping consistent when it comes to terminology is a bit more difficult; especially if there have been no standards in place before. As an example, we can look to one of the most common elements of hosting: the server. Some people call this a "box," a "physical instance" or simply "hardware." The server may be a name server, a mail server, a database server or a web server.

If your company has adopted a term, use that term. If they haven't, decide on a term with your coworkers and stick to it. It's important to be as specific as possible in your documentation to avoid any confusion, and when you adopt specific terms in your documentation, you'll also find that this consistency will carry over into conversations and meetings. As a result, training new team members on your code will go more smoothly, and it will be easier for other people to assist in maintaining your code's documentation.

Bonus: It's much easier to search and replace when you only have to search for one term.

3. Forget What You Know About Your Code ... But Only Temporarily

Regardless of the industry, people who write their own documentation tend to omit pertinent information about the topic. When I train technical writers, I use the peanut butter and jelly example: How would you explain the process of making a peanut butter and jelly sandwich? Many would-be instructors omit things that would result in a very poorly made sandwich ... if one could be made at all. If you don't tell the reader to get the jelly from the cupboard, how can they put jelly on the sandwich? It's important to ask yourself when writing, "Is there anything that I take for granted about this piece of code that other users might need or want to know?"

Think about a coding example where a method calls one or more methods automatically in order to do its job or a method acts like another method. In our API, the createObjects method uses the logic of the createObject method that we just discussed. While some developers may pick up on the connection based on the method's name, it is still important to reference the similarities so they can better understand exactly how the code works. We do this in two ways: First, we state that createObjects follows the logic of createObject in the overview. Second, we note that createObject is a related method. The code below shows exactly how we've implemented this:

     * @SLDNDocumentation Service Description Create multiple domains at once.
     *
     * @SLDNDocumentation Method Overview <<< EOT
     * Create multiple domains on the SoftLayer name servers. Each domain record
     * passed to ''createObjects'' follows the logic in the SoftLayer_Dns_Domain
     * ''createObject'' method.
     * EOT
     *
     * @SLDNDocumentation Method Associated Method SoftLayer_Dns_Domain::createObject

4. Peer Review

The last rule, and one that should not be skipped, is to always have a peer look over your documentation. There really isn't a lot of depth behind this one. In Development, we try to peer review documentation during the code review process. If new content is written during code changes or additions, developers can add content reviewers, who have the ability to add notes with revisions, suggestions and questions. Once all parties are satisfied with the outcome, we close out the review in the system and the content is updated in the next code release. With peer review of documentation, you'll catch typos, inconsistencies and gaps. It always helps to have a second set of eyes before your content hits its users.

Writing better documentation really is that easy: Know your audience, be consistent, don't take your knowledge for granted, and use the peer review process. I put these four rules into practice every day as a technical writer at SoftLayer, and they make my life so much easier. By following these rules, you'll have better documentation for your users and will hopefully eliminate some of that pesky technical debt.

Go, and create better documentation!

-Sarah

September 20, 2013

Building a Mobile App with jQuery Mobile: The Foundation

Based on conversations I've had in the past, at least half of web developers I've met have admitted to cracking open an Objective-C book at some point in their careers with high hopes of learning mobile development ... After all, who wouldn't want to create "the next big thing" for a market growing so phenomenally every year? I count myself among that majority: I've been steadily learning Objective-C over the past year, dedicating a bit of time every day, and I feel like I still lack skill-set required to create an original, complex application. Wouldn't it be great if we web developers could finally get our shot in the App Store without having to unlearn and relearn the particulars of coding a mobile application?

Luckily for us: There is!

The rock stars over at jQuery have created a framework called jQuery Mobile that allows developers to create cross-platform, responsive applications on a HTML5-based jQuery foundation. The framework allows for touch and mouse event support, so you're able to publish across multiple platforms, including iOS, Android, Blackberry, Kindle, Nook and on and on and on. If you're able to create web applications with jQuery, you can now create an awesome cross-platform app. All you have to do is create an app as if it was a dynamic HTML5 web page, and jQuery takes care of the rest.

Let's go through a real-world example to show this functionality in action. The first thing we need to do is fill in the <head> content with all of our necessary jQuery libraries:

<!DOCTYPE html>
<html>
<head>
    <title>SoftLayer Hello World!</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="stylesheet" href="http://code.jquery.com/mobile/1.3.2/jquery.mobile-1.3.2.min.css" />
    <script src="http://code.jquery.com/jquery-1.9.1.min.js"></script>
    <script src="http://code.jquery.com/mobile/1.3.2/jquery.mobile-1.3.2.min.js"></script>
</head>

Now let's create a framework for our simplistic app in the <body> section of our page:

<body>
    <div data-role="page">
        <div data-role="header">
            <h1>My App!</h1>
        </div>
 
        <div data-role="content">
            <p>This is my application! Pretty cool, huh?</p>
        </div>
 
        <div data-role="footer">
            <h1>Bottom Footer</h1>
        </div>
 
    </div>
</body>
</html>

Even novice web developers should recognize the structure above. You have a header, content and a footer just as you would in a regular web page, but we're letting jQuery apply some "native-like" styling to those sections with the data-role attributes. This is what our simple app looks like so far: jQuery Mobile App Screenshot #1

While it's not very fancy (yet), you see that the style is well suited to the iPhone I'm using to show it off. Let's spice it up a bit and add a navigation bar. Since we want the navigation to be a part of the header section of our app, let's add an unordered list there:

<div data-role="header">
    <h1>My App!</h1>
        <div data-role="navbar">
            <ul>
                <li><a href="#home" class="ui-btn-active" data-icon="home" data-theme="b">Home</a></li>
                <li><a href="#softlayer_cool_news" data-icon="grid" data-theme="b">SL Cool News!</a></li>
                <li><a href="#softlayer_cool_stuff" data-icon="star" data-theme="b">SL Cool Stuff!</a></li>
            </ul>
        </div>
    </div>

You'll notice again that it's not much different from regular HTML. We've created a navbar div with an unordered list of menu items we'd like to add to the header: Home, SL Cool News and SL Cool Stuff. Notice in the anchor tag of each that there's an attribute called data-icon which defines which graphical icon we want to represent the navigation item. Let's have a peek at what it looks like now: jQuery Mobile App Screenshot #2

Our app isn't doing a whole lot yet, but you can see from our screenshot that the pieces are starting to come together nicely. Because we're developing our mobile app as an HTML5 app first, we're able to make quick changes and see those changes in real time from our phone's browser. Once we get the functionality we want to into our app, we can use a tool such as PhoneGap or Cordova to package our app into a ready-to-use standalone iPhone app (provided you're enrolled in the Apple Development Program, of course), or we can leave the app as-is for a very nifty mobile browser application.

In my next few blogs, I plan to expand on this topic by showing you some of the amazingly easy (and impressive) functionality available in jQuery Mobile. In the meantime, go grab a copy of jQuery Mobile and start playing around with it!

-Cassandra

Subscribe to tips-and-tricks