Posts Tagged 'Guide'

March 5, 2012

iptables Tips and Tricks - Not Locking Yourself Out

The iptables tool is one of the simplest, most powerful tools you can use to protect your server. We've covered port redirection, rule processing and troubleshooting in previous installments to this "Tips and Tricks" series, but what happens when iptables turns against you and locks you out of your own system?

Getting locked out of a production server can cost both time and money, so it's worth your time to avoid this. If you follow the correct procedures, you can safeguard yourself from being firewalled off of your server. Here are seven helpful tips to help you keep your sanity and prevent you from locking yourself out.

Tip 1: Keep a safe ruleset handy.

If you are starting with a working ruleset, or even if you are trying to troubleshoot an existing ruleset, take a backup of your iptables configuration before you ever start working on it.

iptables-save > /root/iptables-safe

Then if you do something that prevents your website from working, you can quickly restore it.

iptables-restore

Tip 2: Create a cron script that will reload to your safe ruleset every minute during testing.

This was pointed out to my by a friend who swears by this method. Just write a quick bash script and set a cron entry that will reload it back to the safe set every minute. You'll have to test quickly, but it will keep you from getting locked out.

Tip 3: Have the IPMI KVM ready.

SoftLayer-pod servers* are equipped with some sort of remote access device. Most of them have a KVM console. You will want to have your VPN connection set up, connected and the KVM window up. You can't paste to and from the KVM, so SSH is typically easier to work with, but it will definitely cut down on the downtime if something does go wrong.

*This may not apply to servers that were originally provisioned under another company name.

Tip 4: Try to avoid generic rules.

The more criteria you specify in the rule, the less chance you will have of locking yourself out. I would liken this to a pie. A specific rule is a very thin slice of the pie.

iptables -A INPUT -p tcp --dport 22 -s 10.0.0.0/8 -d 123.123.123.123 -j DROP

But if you block port 22 from any to any, it's a very large slice.

iptables -A INPUT -p tcp --dport 22 -j DROP

There are plenty of ways that you can be more specific. For example, using "-i eth0" will limit the processing to a single NIC in your server. This way, it will not apply the rule to eth1.

Tip 5: Whitelist your IP address at the top of your ruleset.

This may make testing more difficult unless you have a secondary offsite test server, but this is a very effective method of not getting locked out.

iptables -I INPUT -s <your IP> -j ACCEPT

You need to put this as the FIRST rule in order for it to work properly ("-I" inserts it as the first rule, whereas "-A" appends it to the end of the list).

Tip 6: Know and understand all of the rules in your current configuration.

Not making the mistake in the first place is half the battle. If you understand the inner workings behind your iptables ruleset, it will make your life easier. Draw a flow chart if you must.

Tip 7: Understand the way that iptables processes rules.

Remember, the rules start at the top of the chain and go down, unless specified otherwise. Crack open the iptables man page and learn about the options you are using.

-Mark

February 23, 2012

How to Get the Best Customer Service (Anywhere)

Shelves of books have been written about providing great customer support, but I haven't seen many written about how to get great customer support. Lance wrote a quick guide called "The 8 Keys to Successful Tickets" in May 2007, but because there have been over 730 blog posts between that post and this post, I thought I might take a shot at the topic again without stealing too many of his ideas. When you work with a service-based company, you're probably going to interact with customer support representatives regularly. During these interactions, your experience will not be defined by your question or the issue you have. Instead, it will be defined by how you present your issue.

It can be extremely frustrating when a server goes down or a script isn't working the way it should. When something like this happens, my gut reaction is to get upset and throw my keyboard. I've also noticed that when I am angry, I have a difficult time trying to explain my problem to technical support. I know I'm not alone in that regard, so I tried to pinpoint the most important points to remember when contacting customer support. While some of the explanations below are more SoftLayer-specific, each of the tips below can be used in any situation where you need customer support.

  • Remember there's a human on the other end. It doesn't matter where the customer support representative is; they're human, and their responsibility is to help you. I don't have any empirical data, but human nature tells me it's easier to be nice to someone who is nice to you. Once you realize there's a person on the other end of the phone trying to do his/her job, it's a little easier to thank them in advance for their help. It may seem insignificant, but if you thank me in advance for my help, I'll subconsciously work harder in an effort to deserve that gratitude.
  • Don't assume your request will be ignored. I'm surprised by the number of people who start or end their e-mail with, "No one will probably see this, but ..." or "Not that anyone cares, but ..." Don't assume that you'll be ignored. That assumption just creates overarching negative tone; it isn't a "reverse psychology" play. The support process can be defined by the expectations you set for it, so get started on the right foot and expect that your questions will be answered and issues will be resolved.
  • Don't start with a threat. "If you don't do this, I'm going to report this to my bank and other authorities," or "If you don't respond within 25 seconds, you'll be hearing from my lawyer." It's not uncommon to hear things like this in the first message in a ticket. It's much easier to help someone who seems easy to help. Invoking lawyers does not make your ticket seem easy to address. :-)
  • Provide useful, descriptive and relevant information. This tip can be tough since it's hard to understand what information is "relevant," but think about it before you send a support request. If you are having trouble logging in, then "I can't log in. Any ideas?" is not quite as clear as "Whenever I try to log in, the login screen just reloads without an error message. I know my username and password are correct. Any ideas? Thanks." That extra information will help considerably and will reduce the number of back-and-forth e-mails between you and the support representative.
  • Don't write overly detailed, wordy support requests. The longer your e-mail, the more difficult it is to read, diagnose and to respond. A representative has to read the entire ticket to find what's meaningful and figure out exactly what's wrong. Since they're trying to help you, you want to reduce their burden. You want to make it as easy as possible for them to help you. So, be clear, concise and brief. If you've got a couple different issues for support to look at, break them out into individual tickets. Different issues may need to be addressed by different departments, so multiple issues in a single ticket can lead to delays in responding to specific issues in the ticket.
  • More Tickets ≠ More Support. The flip-side of the above recommendation is that you shouldn't create multiple support tickets for a single issue. While it seems like you're drawing more attention to the issue and creating a sense of urgency, you're really slowing down the support process. Support representatives might be addressing the same issue in parallel or information might be lost between tickets, elongating the time to resolution.
  • Escalate your tickets smartly. If you think a ticket should be handled differently or if you would like a supervisor to look into a specific issue, you should always feel free to request escalation to a manager or a supervisor. The best way to make that request is to update your open ticket, initiate a live chat or place a call into the technical support phone line. If you aren't satisfied with your support experience, then we aren't either, so we want to hear from you.

As you can see, the prescription is not too complicated: Prepare yourself to receive the best support and help us provide the best support, and you're much more likely to receive it.

-@khazard

February 20, 2012

Tips and Tricks - Remote Audio Over RDP in Windows 2008

I was working on my server the other night, and I found myself needing to get sound from my Windows 2008 box through an RDP (Remote Desktop Protocol) connection. Because we have a huge customer base with Windows 2008 installed now, I figured there may be someone else out there that would like to be able to hear sounds from their server on their local computer when connected, so I put together a quick walkthrough with how I got it to work:

Configuring Your Server

  1. Open Windows Services (Start -> Run -> Services.msc)
  2. Change the properties of the Windows Audio Endpoint Service and Windows Audio Service to "Automatic". If the services are not already started, you can manually start them at this time.
  3. Open Terminal Services ( Start -> Run -> tsconfig.msc)
  4. Right-click on the RDP-TCP connection and bring up its properties. Go to the "Client Settings" and make sure that on "Redirection Audio" is not disabled.
  5. Fully log out and log back into the RDP connection to the server. You will see a balloon error on your speaker icon that states "No Audio Output Device is installed."

Making Registry Changes

  1. You will now need to back up your registry and some registry changes.
  2. I want to reiterate the instruction to back up your registry ... As with most technical guides/walkthroughs, SoftLayer will not be held liable for any corruptions that may result from you attempting these changes. The next two steps will show how to quickly back up your registry.
  3. Log into your server on an account with Administrator rights, and open regedit (Start -> Run -> regedit)
  4. Export the current registry (from the "File" menu) and copy it to a location off of your server so you have it backed up.
  5. Locate the following key: HKEY_CLASSES_ROOT\AudioEngine\AudioProcessingObjects. This will contain several subkeys all each named with a GUID.
  6. Click on each subkey, then right-click and select "Permissions." You will then click on the "Advanced" button and the "Owner" tab. The current owner should be listed as "TrustedInstaller."
  7. Select the Administrative account and/or group from the list and click "OK" to change the ownership.
  8. Select the account you just chose and give it "Full Control," then click "OK."
  9. In the "Detail" box of each subkey, double-click on the DWORD value "MinOutputConnections" and change it from 1 to 0, then click "OK."
  10. Once you have done this for each subkey in HKEY_CLASSES_ROOT\AudioEngine\AudioProcessingObjects, you can close regedit and restart the Windows Audio and Windows Audio Endpoint services.

Configuring Your RDP Client

Now that you have everything ready on the server, you just need to make sure your RDP client recognizes the audio. Log off of the server so you can configure your RDP client. Open RDP, go to the "Options" menu, and under "Local Resources," select "Configure Remote Audio Settings." Select "Play on this Computer," and hit "OK." Voila! You now should be able to hear sound from your Windows 2008 RDP connection.

-Bill

January 10, 2012

Web Development - HTML5 - Custom Data Attributes

I recently worked on a project that involved creating promotion codes for our clients. I wanted to make this tool as simple as possible to use and because this involved dealing with thousands of our products in dozens of categories with custom pricing for each of these products, I had to find a generic way to deal with client-side form validation. I didn't want to write custom JavaScript functions for each of the required inputs, so I decided to use custom data attributes.

Last month, we started a series focusing on web development tips and tricks with a post about JavaScript optimization. In this installment, we're cover how to use HTML5 custom data attributes to assist you in validating forms.

Custom data attributes for elements are "[attributes] in no namespace whose name starts with the string 'data-', has at least one character after the hyphen, is XML-compatible, and contains no characters in the range U+0041 to U+005A (LATIN CAPITAL LETTER A to LATIN CAPITAL LETTER Z)." Thanks W3C. That definition is bookish, so let's break it down and look at some examples.

Valid:

<div data-name="Philip">Mr. Thompson is an okay guy.</div>
<a href="softlayer.com" data-company-name="SoftLayer" data-company-state="TX">SoftLayer</a>
<li data-color="blue">Smurfs</li>

Invalid:

// This attribute is not prefixed with 'data-'
    <h2 database-id="244">Food</h2>
 
// These 2 attributes contain capital letters in the attribute names
    <p data-firstName="Ashley" data-lastName="Thompson">...</p>
 
// This attribute does not have any valid characters following 'data-'
    <img src="/images/pizza.png" data-="Sausage" />

Now that you know what custom data attributes are, why would we use them? Custom attributes allow us to relate specific information to particular elements. This information is hidden to the end user, so we don't have to worry about the data cluttering screen space and we don't have to create separate hidden elements whose purpose is to hold custom data (which is just bad practice). This data can be used by a JavaScript programmer to many different ends. Among the most common use cases are to manipulate elements, provide custom styles (using CSS) and perform form validation. In this post, we'll focus on form validation.

Let's start out with a simple form with two radio inputs. Since I like food, our labels will be food items!

<input type="radio" name="food" value="pizza" data-sl-required="food" data-sl-show=".pizza" /> Pizza
<input type="radio" name="food" value="sandwich" data-sl-required="food" data-sl-show="#sandwich" /> Sandwich
<div class="hidden required" data-sl-error-food="1">A food item must be selected</div>

Here we have two standard radio inputs with some custom data attributes and a hidden div (using CSS) that contains our error message. The first input has a name of food and a value of pizza – these will be used on the server side once our form is submitted. There are two custom data attributes for this input: data-sl-required and data-sl-show. I am defining the data attribute data-sl-required to indicate that this radio button is required in the form and one of them must be selected to pass validation. Note that I have prefixed required with sl- to namespace my data attribute. required is generic and I don't want to have any conflicts with any other attributes – especially ones written for other projects! The value of data-sl-required is food, which I have tied to the div with the attribute data-sl-error-food (more on this later).

The second custom attribute is data-sl-show with a selector of .pizza. (To review selectors, jump back to the JavaScript optimization post.) This data attribute will be used to show a hidden div that contains the class pizza when the radio button is clicked. The sandwich radio input has the same data attributes but with slightly different values.

Now we can move on to our HTML with the hidden text inputs:

<div class="hidden pizza">
    Pizza type: <input type="text" name="pizza" data-sl-required="pizza" data-sl-depends="{&quot;type&quot;:&quot;radio&quot;,&quot;name&quot;:&quot;food&quot;,&quot;value&quot;:&quot;sandwich&quot;}" />
    <div class="hidden required" data-sl-error-pizza="1">The pizza type must not be empty</div>
</div>
 
<div class="hidden" id="sandwich">
    Sandwich: <input type="text" name="sandwich" data-sl-required="sandwich" data-sl-depends="{&quot;type&quot;:&quot;radio&quot;,&quot;name&quot;:&quot;food&quot;,&quot;value&quot;:&quot;sandwich&quot;}" />
    <div class="hidden required" data-sl-error-sandwich="1">The sandwich must not be empty</div>
</div>

These are hidden divs that contain text inputs that will be used to input more data depending on the radio input selected. Notice that the first outer div has a class of pizza and the second outer div has an id of sandwich. These values tie back to the data-sl-show selectors for the radio inputs. These text inputs also contain the data-sl-required attribute like the previous radio inputs. The data-sl-depends data attribute is a fun attribute that contains a JSON-encoded object that's been run through PHP's htmlentities() function. For readability, the data-sl-depends values contain:

{"type":"radio","name":"food","value":"pizza"}
{"type":"radio","name":"food","value":"sandwich"}

This first attribute says that our text input “depends” on the radio input with the name food and value pizza to be visible and selected in order to be processed as required. You can just imagine the possibilities of combinations you can create to make very custom functionality with very generic JavaScript.

Now we can examine our JavaScript to make sense of all these custom data attributes. Note that I'll be using Mootools' syntax, but the same can fairly easily be accomplished using jQuery or your favorite JavaScript framework. I'm going to start by creating a DataForm class. This will be generic enough so that you can use it in multiple forms and it's not tied to this specific instance. Reuse is good! To have it fit our needs, we're going to pass some options when instantiating it.

new DataForm({
    ...,
    dataAttributes: {
        required: 'data-sl-required',
        errorPrefix: 'data-sl-error',
        depends: 'data-sl-depends',
        show: 'data-sl-show'
    }
});

As you can see, I'm using the data attribute names from our form as the options – this will allow you to create your own data attribute names depending on your uses. We first need to make our hidden divs visible whenever our radio buttons are clicked – we'll use our initData() method for that.

initData: function() {
    var attrs = this.options.dataAttributes,
        divs = [];
 
    $$('input[type=radio]['+attrs.show+']').each(function(input) {
        var div = $$(input.get(attrs.show));
        divs.push(div);
        input.addEvent('change', function(e) {
            divs.each(function(div) { div.hide(); });
            div.show();
        });
    });
}

This method grabs all the radio inputs with the show attribute (data-sl-show) and adds an onchange event to each of them. When a radio input is checked, it first hides all the divs, and then shows the div that's associated with that radio input.

Great! Now we have our text inputs showing and hiding depending on which radio button is selected. Now onto the actual validation. First, we'll make sure our required radio inputs are checked:

$$('input[type=radio]['+attrs.required+']:not(:checked)').each(function(input) {
    var name = input.get('name');
    checkError(name, isRadioWithNameSelected(name))
});

This grabs all the unchecked radio inputs with the required attribute (data-sl-required) and checks to see if any radio with that same name is selected using the function isRadioWithNameSelected(). The checkError() function will show or hide the error div (data-sl-error-*) depending if the radio is checked or not. Don't worry - you'll see how these functions are implemented in the JSFiddle below.

Now we can check our text inputs:

$$('input[type=text]['+attrs.required+']:visible').each(function(input) {
    var name = input.get('name'),
        depends = input.get(attrs.depends),
        value = input.get('value').trim();
 
    if (depends) {
        depends = JSON.encode(depends);
        switch (depends.type) {
            case 'radio':
                if (depends.name &amp;&amp; depends.value) {
                    var radio = $$('input[type=radio][name="'+depends.name+'"][value="'+depends.value+'"]:checked');
                    if (!radio) {
                        checkError(input.get(attrs.required), true);
                        return;
                    }
                }
                break;
        }
    }
 
    checkError(name, value!='');
});

This obtains the required and visible text inputs and determines if they are empty or not. Here we look at our depends object to grab the associated radio inputs. If that radio is checked and the text input is empty, the error message appears. If that radio is not checked, it doesn't even evaluate that text input. The depends.type is in a switch statement to allow for easy expansion of types. There are other cases for evaluating relationships ... I'll let you come up with more for yourself.

That concludes our usage of custom data attributes in form validation. This really only touched upon the very tip of the iceberg. Data attributes allow you – the creative developer – to interact in a new, HTML-valid way with your web pages.

Check out the working example of the above code at jsfiddle.net. To read more on custom data attributes, see what Google has to say. If you want to see really cool functionality that uses data attributes plus so much more, check out Aaron Newton's Behavior module over at clientcide.com.

Happy coding!

-Philip

January 9, 2012

iptables Tips and Tricks – Troubleshooting Rulesets

One of the most time consuming tasks with iptables is troubleshooting a problematic ruleset. That will not change no matter how much experience you have with it. However, with the right mindset, this task becomes considerably easier.

If you missed my last installment about iptables rule processing, here's a crash course:

  1. The rules start at the top, and proceed down, one by one unless otherwise directed.
  2. A rule must match exactly.
  3. Once iptables has accepted, rejected, or dropped a packet, it will not process any further rules on it.

There are essentially two things that you will be troubleshooting with iptables ... Either it's not accepting traffic and it should be OR it's accepting traffic and it shouldn't be. If the server is intermittently blocking or accepting traffic, that may take some additional troubleshooting, and it may not even be related to iptables.

Keep in mind what you are looking for, and don't jump to any conclusions. Troubleshooting iptables takes patience and time, and there shouldn't be any guesswork involved. If you have a configuration of 800 rules, you should expect to need to look through every single rule until you find the rule that is causing your problems.

Before you begin troubleshooting, you first need to know some information about the traffic:

  1. What is the source IP address or range that is having difficulty connecting?
  2. What is the destination IP address or website IP?
  3. What is the port or port range affected, or what type of traffic is it (TCP, ICMP, etc.)?
  4. Is it supposed to be accepted or blocked?

Those bits of information should be all you need to begin troubleshooting a buggy ruleset, except in some rare cases that are outside the scope of this article.

Here are some things to keep in mind (especially if you did not program every rule by hand):

  • iptables has three built in chains. These are for INPUT – the traffic coming in to the server, OUTPUT – the traffic coming out of the server, and FORWARD – traffic that is not destined to or coming from the server (usually only used when iptable is acting as a firewall for other servers). You will start your troubleshooting at the top of one of these three chains, depending on the type of traffic.
  • The "target" is the action that is taken when the rule matches. This may be another custom chain, so if you see a rule with another chain as the target that matches exactly, be sure to step through every rule in that chain as well. In the following example, you will see the BLACKLIST2 sub-chain that applies to traffic on port 80. If traffic comes through on port 80, it will be diverted to this other chain.
  • The RETURN target indicates that you should return to the parent chain. If you see a rule that matches with a RETURN target, stop all your troubleshooting on the current chain, and return the rule directly after the rule that referenced the custom chain.
  • If there are no matching rules, the chain policy is applied.
  • There may be rules in the "nat," "mangle" or "raw" tables that are blocking or diverting your traffic. Typically, all the rules will be in the "filter" table, but you might run into situations where this is not the case. Try running this to check: iptables -t mangle -nL ; iptables -t nat -nL ; iptables -t raw -nL
  • Be cognisant of the policy. If the policy is ACCEPT, all traffic that does not match a rule will be accepted. Conversely, if the policy is DROP or REJECT, all traffic that does not match a rule will be blocked.
  • My goal with this article is to introduce you to the algorithm by which you can troubleshoot a more complex ruleset. It is intentionally left simple, but you should still follow through even when the answer may be obvious.

Here is an example ruleset that I will be using for an example:

Chain INPUT (policy DROP)
target prot opt source destination
BLACKLIST2 tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:50
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:1010

Chain BLACKLIST2 (1 references)
target prot opt source destination
REJECT * -- 123.123.123.123 0.0.0.0/0
REJECT * -- 45.34.234.234 0.0.0.0/0
ACCEPT * -- 0.0.0.0/0 0.0.0.0/0

Here is the problem: Your server is accepting SSH traffic to anyone, and you wish to only allow SSH to your IP – 111.111.111.111. We know that this is inbound traffic, so this will affect the INPUT chain.

We are looking for:

source IP: any
destination IP: any
protocol: tcp
port: 22

Step 1: The first rule denotes any source IP and and destination IP on destination port 80. Since this is regarding port 22, this rule does not match, so we'll continue to the next rule. If the traffic here was on port 80, it would invoke the BLACKLIST2 sub chain.
Step 2: The second rule denotes any source IP and any destination IP on destination port 50. Since this is regarding port 22, this rule does not match, so let's continue on.
Step 3: The third rule denotes any source IP and any destination IP on destination port 53. Since this is regarding port 22, this rule does not match, so let's continue on.
Step 4: The fourth rule denotes any source IP and any destination IP on destination port 22. Since this is regarding port 22, this rule matches exactly. The target ACCEPT is applied to the traffic. We found the problem, and now we need to construct a solution. I will be showing you the Redhat method of doing this.

Do this to save the running ruleset as a file:

iptables-save > current-iptables-rules

Then edit the current-iptables-rules file in your favorite editor, and find the rule that looks like this:

-A INPUT -p tcp --dport 22 -j ACCEPT

Then you can modify this to only apply to your IP address (the source, or "-s", IP address).

-A INPUT -p tcp -s 111.111.111.111 --dport 22 -j ACCEPT

Once you have this line, you will need to load the iptables configuration from this file for testing.

iptables-restore < current-iptables-rules

Don't directly edit the /etc/sysconfig/iptables file as this might lock you out of your server. It is good practice to test a configuration before saving to the system configuration files. This way, if you do get locked out, you can reboot your server and it will be working. The ruleset should look like this now:

Chain INPUT (policy DROP)
target prot opt source destination
BLACKLIST2 tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:50
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
ACCEPT tcp -- 111.111.111.111 0.0.0.0/0 tcp dpt:22
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:1010

Chain BLACKLIST2 (1 references)
target prot opt source destination
REJECT * -- 123.123.123.123 0.0.0.0/0
REJECT * -- 45.34.234.234 0.0.0.0/0
ACCEPT * -- 0.0.0.0/0 0.0.0.0/0

The policy of "DROP" will now block any other connection on port 22. Remember, the rule must match exactly, so the rule on port 22 now *ONLY* applies if the IP address is 111.111.111.111.

Once you have confirmed that the rule is behaving properly (be sure to test from another IP address to confirm that you are not able to connect), you can write the system configuration:

service iptables save

If this troubleshooting sounds boring and repetitive, you are right. However, this is the secret to solid iptables troubleshooting. As I said earlier, there is no guesswork involved. Just take it step by step, make sure the rule matches exactly, and follow it through till you find the rule that is causing the problem. This method may not be fast, but it's reliable. You'll look like an expert in no time.

-Mark

January 5, 2012

iptables Tips and Tricks - Rule Processing

As I mentioned in "iptables Tips and Tricks - Port Redirection," iptables is probably a complete mystery to a lot of users, and one the biggest hurdles is understanding the method by which it filters traffic ... Once you understand this, you'll be able to tame the beast.

When I think of iptables, the best analogy that comes to mind is a gravity coin sorting bank with four rules and one policy. If you're not familiar with a gravity coin sorting bank, each coin is starts at the same place and slides down an declined plane until it can fall into it's appropriate tube:

iptables Rule Sorter

As you can see, once a coin starts down the path, there are four rules – each one "filtering traffic" based on the width of the coin in millimeters (Quarter = 25mm, Nickel = 22mm, Penny = 20mm, Dime = 18mm). Due to possible inconsistencies in the coins, the tube widths are slightly larger than the official sizes of each coin to prevent jamming. At the end of the line, if a coin didn't fit in any of the tubes, it's dropped out of the sorter.

As we use this visualization to apply to iptables, there are three important things to remember:

  1. The rules start at the top, and proceed down, one by one unless otherwise directed.
  2. A rule must match exactly.
  3. Once iptables has accepted, rejected, or dropped a packet, it will not process any further rules on it.

Let's jump back to the coin sorter. What would happen if you introduced a 23mm coin (slightly larger than a nickel)? What would happen if you introduced a 17mm coin (smaller than a dime)? What would happen if you dropped in a $1 coin @ 26.5mm?

In the first scenario, the coin would enter into the rule processing by being dropped in at the top. It would first pass by the dime slot, which requires a diameter of less than 18mm. It passes by the pennies slot as well, which requires less than 20mm. It continues past the nickels slot, which requires 22mm or less. It will then be "accepted" into the quarters slot, and there will be no further "processing" on the coin.

The iptables rules might look something like this:

Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all --- 0.0.0.0/00.0.0.0/0width ACCEPT all --- 0.0.0.0/00.0.0.0/0width ACCEPT all --- 0.0.0.0/00.0.0.0/0width ACCEPT all --- 0.0.0.0/00.0.0.0/0width

It's important to remember that once iptables has accepted, rejected, or dropped a packet, it will not process any further rules on it. In the second scenario (17mm coin), the coin would only be processed through the first rule; the other 3 rules would not be used even though the coin would meet their rules as well. Just because a port or and IP address is allowed somewhere in a chain, if a matching rule has dropped the packet, no further rules will be processed.

The final scenario (26.5mm coin) outlines a situation where none of the rules match, and this indicates that the policy will be used. In the coin bank example, it would be physically dropped off the side of the bank. iptables keeps a tally of the number of packets dropped and the corresponding size of the data. You can view this data by using the "iptables -vnL" command.

Chain OUTPUT (policy ACCEPT 3418K packets, 380M bytes)

cPanel even uses this tally functionality to track bandwidth usage (You may have seen the "acctboth" chain - this is used for tracking usage per IP).

So there you have it: iptables is just like a gravity coin sorting bank!

-Mark

December 26, 2011

iptables Tips and Tricks - Port Redirection

One of the most challenging and rewarding aspects of Linux administration is the iptables firewall. To the unenlightened, this can be a confusing black box that breaks your web server and blocks your favorite visitors from viewing your content at the most inconvenient times. This blog is the first in a series aimed at clarifying this otherwise mysterious force at work in your server.

Nothing compares with the frustration of trying to make a program listen on a different port – like if you wanted to configure your mail client to listen on port 2525. Many times, configuring a program the hard way (some would say the "correct" way) using configuration files may not be worth your time and effort ... Especially if the server is running on a control panel that does not natively support this functionality.

Fortunately, iptables offers an elegant solution:

iptables -t nat -A PREROUTING -p tcp --dport 2525 -j REDIRECT --to-ports 25

What this does:

  1. This specifies -t nat to indicate the nat table. Typically rules are added to the "filter" table (if you do not specify another table), and this is where the majority of the traffic is handled. In this case, however, we require the use of the nat table.
  2. This rules appends (-A), which means to add the rule at the bottom of the list.
  3. This rule is added to the PREROUTING chain.
  4. For the tcp protocol (-p tcp)
  5. The destination port (--dport) is 2525 - this is the port that the client is trying to access on your server.
  6. The traffic is jumped (-j) to the REDIRECT action. This is the action that is taken when the rule matches.
  7. The port is redirected to port 25 on the server.

As you can see, by changing the protocol to either tcp or udp or by adjusting the dport number and the to-ports number, you can redirect any port incoming to any listening port on the server. Just remember that the dport is the port the client machine is trying to connect to (the port they configure in the mail client, for example).

But check this out: Say for example you have a website (shocking, I know). You don't have a load balancer or a firewall set up, but you want to split off your email traffic to a second server to reduce strain on your web server. Essentially, you want to take incoming port 25 and redirect it ... to ANOTHER SERVER. With iptables, you can make this work:

iptables -t nat -A PREROUTING -p tcp -d 123.123.123.123 --dport 25 -j DNAT --to-destination 10.10.10.10:25

What this does:

  1. It specifies a destination (-d) IP address. This is not needed, but if you want to limit the email redirection to a single address, this is how you can do it.
  2. It is jumped to DNAT, which stands for destination nat.
  3. The destination and port are specified as arguments on to-destination

As you can see, this forwards all traffic on port 25 to an internal IP address.

Now, say you want to redirect from a different incoming port to a port on another server:

iptables -t nat -A PREROUTING -p tcp --dport 5001 -j DNAT --to-destination 10.10.10.10:25
iptables -t nat -A POSTROUTING -p tcp --dport 25 -j MASQUERADE

In this example, the incoming port is different, so we need to change it back to the standard port on the way back out through the primary server.

If you would like further reading on this topic, I recommend this great tutorial:
http://www.karlrupp.net/en/computer/nat_tutorial

Remember, when you are modifying your running configuration of iptables, you will still need to save your changes in order for it to persist on reboot. Be sure to test your configuration before saving it with "service iptables save" so that you don't lock yourself out.

-Mark

December 1, 2011

UNIX Sysadmin Boot Camp: Permissions

I hope you brought your sweat band ... Today's Boot Camp workout is going to be pretty intense. We're focusing on our permissions muscles. Permissions in a UNIX environment cause a lot of customer issues ... While everyone understands the value of secure systems and limited access, any time an "access denied" message pops up, the most common knee-jerk reaction is to enable full access to one's files (chmod 777, as I'll explain later). This is a BAD IDEA. Open permissions are a hacker's dream come true. An open permission setting might have been a temporary measure, but more often than not, the permissions are left in place, and the files remain vulnerable.

To better understand how to use permissions, let's take a step back and get a quick refresher on key components.

You'll need to remember the three permission types:

r w x: r = read; w = write; x = execute

And the three types of access they can be applied to:

u g o: u = user; g = group; o = other

Permissions are usually displayed in one of two ways – either with letters (rwxrwxrwx) or numbers (777). When the permissions are declared with letters, you should look at it as three sets of three characters. The first set applies to the user, the second applies to the group, and the third applies to other (everyone else). If a file is readable only by the user and cannot be written to or executed by anyone, its permission level would be r--------. If it could be read by anyone but could only be writeable by the user and the group, its permission level would be rw-rw-r--.

The numeric form of chmod uses bits to represent permission levels. Read access is marked by 4 bits, write is 2, and execute is 1. When you want a file to have read and write access, you just add the permission bits: 4 + 2 = 6. When you want a file to have read, write and execute access, you'll have 4 + 2 + 1, or 7. You'd then apply that numerical permission to a file in the same order as above: user, group, other. If we used the example from the last sentence in the previous paragraph, a file that could be read by anyone, but could only be writeable by the user and the group, would have a numeric permission level of 664 (user: 6, group: 6, other: 4).

Now the "chmod 777" I referenced above should make a little more sense: All users are given all permissions (4 + 2 + 1 = 7).

Applying Permissions

Understanding these components, applying permissions is pretty straightforward with the use of the chmod command. If you want a user (u) to write and execute a file (wx) but not read it (r), you'd use something like this:

chmod Output

In the above terminal image, I added the -v parameter to make it "verbose," so it displays the related output or results of the command. The permissions set by the command are shown by the number 0300 and the series (-wx------). Nobody but the user can write or execute this file, and as of now, the user can't even read the file. If you were curious about the leading 0 in "0300," it simply means that you're viewing an octal output, so for our purposes, it can be ignored entirely.

In that command, we're removing the read permission from the user (hence the minus sign between u and r), and we're giving the user write and execute permissions with the plus sign between u and wx. Want to alter the group or other permissions as well? It works exactly the same way: g+,g-,o+,o- ... Getting the idea? chmod permissions can be set with the letter-based commands (u+r,u-w) or with their numeric equivalents (eg. 400 or 644), whichever floats your boat.

A Quick Numeric chmod Reference

chmod 777 | Gives specified file read, write and execute permissions (rwx) to ALL users
chmod 666 | Allows for read and write privileges (rw) to ALL users
chmod 555 | Gives read and execute permissions (rx) to ALL users
chmod 444 | Gives read permissions (r) to ALL users
chmod 333 | Gives write and execute permissions (wx) to ALL users
chmod 222 | Gives write privileges (w) to ALL users
chmod 111 | Gives execute privileges (x) to ALL users
chmod 000 | Last but not least, gives permissions to NO ONE (Careful!)

Get a List of File Permissions

To see what your current file permissions are in a given directory, execute the ls –l command. This returns a list of the current directory including the permissions, the group it's in, the size and the last date the file was modified. The output of ls –l looks like this:

ls -l Output

On the left side of that image, you'll see the permissions in the rwx format. When the permission begins with the "d" character, it means that object is a directory. When the permission starts with a dash (-), it is a file.

Practice Deciphering Permissions

Let's look at a few examples and work backward to apply what we've learned:

  • Example 1: -rw-------
  • Example 2: drwxr-x---
  • Example 3: -rwxr-xr-x

In Example 1, the file is not a directory, the user that owns this particular object has read and write permissions, and when the group and other fields are filled with dashes, we know that their permissions are set to 0, so they have no access. In this case, only the user who owns this object can do anything with it. We'll cover "ownership" in a future blog, but if you're antsy to learn right now, you can turn to the all-knowing Google.

In Example 2, the permissions are set on a directory. The user has read, write and execute permissions, the group has read and execute permissions, and anything/anyone besides user or group is restricted from access.

For Example 3, put yourself to the test. What access is represented by "-rwxr-xr-x"? The answer is included at the bottom of this post.

Wrapping It Up

How was that for a crash course in Unix environment permissions? Of course there's more to it, but this will at least make you think about what kind of access you're granting to your files. Armed with this knowledge, you can create the most secure server environment.

Here are a few useful links you may want to peruse at your own convenience to learn more:

Linuxforums.org
Zzee.com
Comptechdoc.org
Permissions Calculator

Did I miss anything? Did I make a blatantly ridiculous mistake? Did I use "their" when I should have used "they're"??!!... Let me know about it. Leave a comment if you've got anything to add, suggest, subtract, quantize, theorize, ponderize, etc. Think your useful links are better than my useful links? Throw those at me too, and we'll toss 'em up here.

Are you still feeling the burn from your Sysadmin Boot Camp workout? Don't forget to keep getting reps in bash, logs, SSH, passwords and user management!

- Ryan

Example 3 Answer

August 29, 2011

UNIX Sysadmin Boot Camp: Your Logs and You

We're a few exercises into UNIX Sysadmin Boot Camp, and if you're keeping up, you've learned about SSH and bash. In those sessions, our focus was to tell the server what we wanted it to do. In this session, we're going to look at the logs of what the server has done.

Logs are like an overbearing mother who sneakily follows her teenage son around and writes down the addresses of each house he visits. When he realizes he lost a really important piece of baseball history at one of those houses, he'll be glad he has that list so he can go desperately search for the soon-to-be-noticed missing bat. Ahem.

MAKE BEST FRIENDS WITH THIS DIRECTORY: /var/log/

When something goes wrong – when there's hitch in the flux capacitor or too many gigawatts in the main reactor – your logs will be there to let you know what's going on, and you can pinpoint the error with educated vengeance. So treat your logs with respect.

One of the best places to start harnessing this logged goodness is /var/log/messages. This log file reports all general errors with network and media, among other things. As you add to and learn your server's command line environment, you'll see specific logs for applications as well, so it's a very good idea to keep a keen eye on these. They just might save your life ... or server.

Some of the most commonly used logs (may vary with different Linux distributions):

  • /var/log/message – General message- and system-related info
  • /var/log/cron.log – Cron job logs
  • /var/log/maillog – Mail server logs
  • /var/log/kern.log – Kernel logs
  • /var/log/httpd/ – Apache access and error logs
  • /var/log/boot.log – System boot logs
  • /var/log/mysqld.log – MySQL database server logs
  • /var/log/secure – SSH authentication logs
  • /var/log/auth.log – Authentication logs
  • /var/log/qmail/ – Qmail log directory (more files inside this directory)
  • /var/log/utmp or /var/log/wtmp – Login records file
  • /var/log/yum.log – Yum log files

There are plenty more in-depth logs – particularly involving raw system components – and others that act similarly to logs but are a bit more active like tcpdumps. Those are a little more advanced to interpret, so I'll save them for another guide and another day.

At this point in our UNIX workout series, you're familiar with the command line, you know the basics of how to tell your server what to do and you just learned how to let the server tell you what it's done. There's still a bit of work to be done before you can call yourself a UNIX ninja, but you're well on your way. In our next installment, we're going to take a step back and talk about p455w0rd5.

Keep learning.

-Ryan

August 12, 2011

UNIX Sysadmin Boot Camp: An Intro to SSH

You've got a 'nix box set up. For some reason, you feel completely lost and powerless. It happens. Many a UNIX-related sob has been cried by confused and frustrated sysadmins, and it needs to stop. As a techie on the front lines of support, I've seen firsthand the issues that new and curious sysadmins seem to have. We have a lot of customers who like to dive head-first into a new environment, and we even encourage it. But there's quite a learning curve.

In my tenure at SoftLayer, I've come across a lot of customers who rely almost entirely on control panels provided by partners like cPanel and Parallels to administer their servers. While those panels simplify some fairly complex tasks to the touch of a button, we all know that one day you're going to have to get down and dirty in that SSH (Secure Shell) interface that so many UNIX server newbies fear.

I'm here to tell you that SSH can be your friend, if you treat it right. Graphical user interfaces like the ones used in control panels have been around for quite a while now, and despite the fact that we are in "the future," the raw power of a command line is still unmatched in its capabilities. It's a force to be reckoned with.

If you're accustomed to a UNIX-based interface, this may seem a little elementary, but you and I both know that as we get accustomed to something, we also tend to let those all-important "basics" slip from our minds. If you're coming from a Windows background and are new to the environment, you're in for a bit of a shell shock, no pun intended. The command line is fantastically powerful once you master it ... It just takes a little time and effort to learn.

We'll start slow and address some of the most common pain points for new sysadmins, and as we move forward, we'll tackle advanced topics. Set your brain to "absorbent," and visualize soaking up these UNIX tips like some kind of undersea, all-knowing, Yoda-like sea sponge.

SSH

SSH allows data to be exchanged securely between two networked devices, and when the "network" between your workstation and server is the Internet, the fact that it does so "securely" is significant. Before you can do any actual wielding of SSH, you're going to need to know how to find this exotic "command line" we've talked so much about.

You can use a third-party client such as PuTTY, WinSCP if your workstation is Windows-based, or if you're on Linux or Mac, you can access SSH from your terminal application: ssh user@ipaddress. Once you've gotten into your server, you'll probably want to find out where you are, so give the pwd command a try:

user@serv: ~$ pwd
/home/user
user@serv: ~$

It's as easy as that. Now we know we're in the /home/user directory. Most of the time, you'll find yourself starting in your home directory. This is where you can put personal files and documents. It's kind of like "My Documents" in Windows, just on your server.

Now that you know where you are, you'll probably want to know what's in there. Take a look at these commands (extracted from a RedHat environment, but also usable in CentOS and many other distributions):

    user@serv: /usr/src $ ls    
This will give you a basic listing of the current directory.

    user@serv: /usr/src $ ls /usr/src/redhat    
This will list the contents of another specified directory.

    user@serv: /usr/src $ ls ./redhat    
Using a "relative pathname," this will perform the same action as above.

    user@serv: /usr/src $ ls redhat    
Most of the time, you'll get the same results even without the "./" at the beginning.

    user@serv: /usr/src $ cd /usr/src/redhat/    
This is an example of using the cd command to change directories to an absolute pathname.

    user@serv: /usr/src $ cd redhat    
This is an example of using the cd command to change directories to a relative pathname.

    user@serv: /usr/src/redhat $ cd /usr/src    
To move back on directory from the working directory, you can use the destination's absolute path.

    user@serv: /usr/src/redhat $ cd ..    
Or, since the desired directory is one step down, you can use two dots to move back.

You'll notice many similarities to the typical Windows DOS prompts, so it helps if you're familiar with navigating through that interface: dir, cd, cd .., cd /. Everything else on the other hand, will prove to be a bit different.

Now that you're able to access this soon-to-be-powerful-for-you tool, you need to start learning the language of the natives: bash. In our next installment, we'll take a crash course in bash, and you'll start to get comfortable navigating and manipulating content directly on your server.

Bookmark the SoftLayer Blog and come back regularly to get the latest installments in our "UNIX Sysadmin Boot Camp" series!

-Ryan

Subscribe to guide