Tips and Tricks Posts

January 15, 2013

Startup Series: Moqups

By in SoftLayer, Startup Series, Tips and Tricks

Every member on the Catalyst team is given one simple goal: Find the most innovative and creative startups on the planet and get them on the SoftLayer network. We meet entrepreneurs at conferences and events around the world, we team up with the most influential startup accelerators and incubators, and we hunt for businesses who are making waves online. With the momentum Catalyst built in 2012, our message has started spreading exponentially faster than what the community development team could be doing on our own, and now it seems like we’ve earned a few evangelists in the startup community. We have those evangelists to thank for bringing Moqups to our door.

In a Hacker News thread, a user posted about needing hosting for a server/startup, and a recommendation for the Catalyst program was one of the top-rated results. The founders of Moqups saw that recommendation, researched SoftLayer’s hosting platform and submitted an application to become a Catalyst partner. As soon as we saw the unbelievable HTML5 app the Moqups team created to streamline and simplify the process of creating wireframes and mockups for website and application design, we knew they were a perfect fit to join the program.

If you’ve ever had to create a site prototype or UI mockup, you know how unwieldy the process can be. You want to sketch a layout and present it clearly and cleanly, but there aren’t many viable resources between “marker on a whiteboard” and “rendering in Photoshop” to accomplish that goal. That’s the problem the Moqups team set out to solve … Can a web app provide the functionality and flexibility you’d need to fill that gap?

We put their answer to that question to the test. I told Kevin about Moqups and asked him to spend a few minutes wireframing the SoftLayer Blog … About ten minutes later, he sent me this (Click for the full Moqups version):

SoftLayer Blog Moqup

Obviously, wireframing an existing design is easier than creating a new design from scratch, but Kevin said he was floored by how intuitive the Moqups platform made the process. In fact, the “instructions” for how to use Moqups are actually provided in an example “Quick Introduction to Moqups” project on the home page. That example project allows you to tweak, add and adjust content to understand how the platform works, and because it’s all done in HTML5, the user experience is seamless.

Moqups

Put it to the test for yourself: How long will it take you to create a wireframe of your existing website (similar to what Kevin did with the SoftLayer Blog)? You have down-to-the-pixel precision, you can group objects together, Moqups helps you line up or center all of the different pieces of your site. Their extensive library of stencils supplements any custom images you upload, so you can go through the whole process of creating a site mockup without “drawing” anything by hand!

I’m actually surprised that the Moqups team heard about SoftLayer before our community development team heard about them … In November, I was in Bucharest, Romania, for HowtoWeb, so I was right in their back yard! Central and Eastern European startups are blowing up right now, and Moqups is a perfect example of what we’re seeing from that region in EMEA.

Oh, and if you know of a crazy cool startup like Moqups that could use a little hosting help from SoftLayer, tell them about Catalyst!

-@EmilyBlitz

January 10, 2013

Web Development – JavaScript Packaging

By in Development, Technology, Tips and Tricks

If you think of JavaScript as the ugly duckling of programming languages, think again! It got a bad rap in the earlier days of the web because developers knew enough just to get by but didn’t really respect it like they did Java, PHP or .Net. Like other well-known and heavily used languages, JavaScript contains various data types (String, Boolean, Number, etc.), objects and functions, and it is even capable of inheritance. Unfortunately, that functionality is often overlooked, and many developers seem to implement it as an afterthought: “Oh, we need to add some neat jQuery effects over there? I’ll just throw some inline JavaScript here.” That kind of implementation perpetuates a stereotype that JavaScript code is unorganized and difficult to maintain, but it doesn’t have to be! I’m going to show you how easy it is to maintain and organize your code base by packaging your JavaScript classes into a single file to be included with your website.

There are a few things to cover before we jump into code:

  1. JavaScript FrameworkMootools is my framework of choice, but you can use whatever JavaScript framework you’d like.
  2. Classes – Because I see JavaScript as another programming language that I respect (and is capable of object-oriented-like design), I write classes for EVERYTHING. Don’t think of your JavaScript code as something you use once and throw away. Write your code to be generic enough to be reused wherever it’s placed. Object-oriented design is great for this! Mootools makes object-oriented design easy to do, so this point reinforces the point above.
  3. Class Files – Just like you’d organize your PHP to contain one class per file, I do the exact same thing with JavaScript. Note: Each of the class files in the example below uses the class name appended with .js.
  4. Namespacing – I will be organizing my classes in a way that will only add a single property — PT — to the global namespace. I won’t get into the details of namespacing in this blog because I’m sure you’re already thinking, “The code! The code! Get on with it!” You can namespace whatever is right for your situation.

For this example, our classes will be food-themed because … well … I enjoy food. Let’s get started by creating our base object:

/*
---
name: PT
description: The base class for all the custom classes
authors: [Philip Thompson]
provides: [PT]
...
*/
var PT = {};

We now have an empty object from which we’ll build all of our classes. We’ll go I will go into more details later about the comment section, but let’s build our first class: PT.Ham.

/*
---
name: PT.Ham
description: The ham class
authors: [Philip Thompson]
requires: [/PT]
provides: [PT.Ham]
...
*/
 
(function() {
    PT.Ham = new Class({
        // Custom code here...
    });
}());

As I mentioned in point three (above), PT.Ham should be saved in the file named PT.Ham.js. When we create second class, PT.Pineapple, we’ll store it in PT.Pineapple.js:

/*
---
name: PT.Pineapple
description: The pineapple class
authors: [Philip Thompson]
requires: [/PT]
provides: [PT.Pineapple]
...
*/
 
(function() {
    PT.Pineapple = new Class({
        // Custom code here...
    });
}());

Our final class for this example will be PT.Pizza (I’ll let you guess the name of the file where PT.Pizza lives). Our PT.Pizza class will require that PT, PT.Ham and PT.Pineapple be present.

/*
---
name: PT.Pizza
description: The pizza class
authors: [Philip Thompson]
requires: [/PT, /PT.Ham, /PT.Pineapple]
provides: [PT.Pizza]
...
*/
 
(function() {
    PT.Pizza = new Class({
        // Custom code here that uses PT.Ham and PT.Pineapple...
    });
}());

Before we go any further, let’s check out the comments we include above each of the classes. The comments are formatted for YAML — YAML Ain’t Markup Language (you gotta love recursive acronyms). These comments allow our parser to determine how our classes are related, and they help resolve dependencies. YAML’s pretty easy to learn and you only need to know a few basic features to use it. The YAML comments in this example are essential for our JavaScript package-manager — Packager. I won’t go into all the details about Packager, but simply mention a few commands that we’ll need to build our single JavaScript file.

In addition to the YAML comments in each of the class files, we also need to create a YAML file that will organize our code. This file — package.yml for this example — is used to load our separate JavaScript classes:

name: "PT"
description: "Provides our fancy PT classes"
authors: "[Philip Thompson]"
version: "1.0.0"
sources:
    - js/PT.js
    - js/PT.Ham.js
    - js/PT.Pineapple.js
    - js/PT.Pizza.js

package.yml shows that all of our PT* files are located in the js directory, one directory up from the package.yml file. Some of the properties in the YAML file are optional, and you can add much more detail if you’d like, but this will get the job done for our purposes.

Now we’re ready to turn back to Packager to build our packaged file. Packager includes an option to use PHP, but we’re just going to do it command-line. First, we need to register the new package (package.yml) we created for PT. If our JavaScript files are located in /path/to/web/directory/js, the package.yml file is in /path/to/web/directory:

./packager register /path/to/web/directory

This finds our package.yml file and registers our PT package. Now that we have our package registered, we can build it:

./packager build * > /path/to/web/directory/js/PT.all.js

The Packager sees that our PT package is registered, so it looks at each of the individual class files to build a single large file. In the comments of each of the class files, it determines if there are dependencies and warns you if any are not found.

It might seem like a lot of work when it’s written out like this, but I can assure you that when you go through the process, it takes no time at all. The huge benefit of packaging our JavaScript is evident as soon as you start incorporating those JavaScript classes into your website … Because we have built all of our class files into a single file, we don’t need to include each of the individual JavaScript files into our website (much less include the inline JavaScript declarations that make you cringe). To streamline your implementation even further if you’re using your JavaScript package in a production deployment, I recommend that you “minify” your code as well.

See … Organized code is no longer just for server-side only languages. Treat your JavaScript kindly, and it will be your friend!

Happy coding!

-Philip

December 30, 2012

Risk Management: Event Logging to Protect Your Systems

By in Technology, Tips and Tricks

The calls start rolling in at 2am on Sunday morning. Alerts start firing off. Your livelihood is in grave danger. It doesn’t come with the fanfare of a blockbuster Hollywood thriller, but if a server hosting your critical business infrastructure is attacked, becomes compromised or fails, it might feel like the end of the world. In our Risk Management series, and we’ve covered the basics of securing your servers, so the next consideration we need to make is for when our security is circumvented.

It seems silly to prepare for a failure in a security plan we spend time and effort creating, but if we stick our heads in the sand and tell ourselves that we’re secure, we won’t be prepared in the unlikely event of something happening. Every attempt to mitigate risks and stop threats in their tracks will be circumvented by the one failure, threat or disaster you didn’t cover in your risk management plan. When that happens, accurate event logging will help you record what happened, respond to the event (if it’s still in progress) and have the information available to properly safeguard against or prevent similar threats in the future.

Like any other facet of security, “event logging” can seem overwhelming and unforgiving if you’re looking at hundreds of types of events to log, each with dozens of variations and options. Like we did when we looked at securing servers, let’s focus our attention on a few key areas and build out what we need:

Which events should you log?
Look at your risk assessment and determine which systems are of the highest value or could cause the most trouble if interrupted. Those systems are likely to be what you prioritized when securing your servers, and they should also take precedence when it comes to event logging. You probably don’t have unlimited compute and storage resources, so you have to determine which types of events are most valuable for you and how long you should keep records of them — it’s critical to have your event logs on-hand when you need them, so logs should be retained online for a period of time and then backed up offline to be available for another period of time.

Your goal is to understand what’s happening on your servers and why it’s happening so you know how to respond. The most common audit-able events include successful and unsuccessful account log-on events, account management events, object access, policy change, privilege functions, process tracking and system events. The most conservative approach actually involves logging more information/events and keeping those logs for longer than you think you need. From there, you can evaluate your logs periodically to determine if the level of auditing/logging needs to be adjusted.

Where do you store the event logs?
Your event logs won’t do you any good if they are stored in a space that is insufficient for the amount of data you need to collect. I recommend centralizing your logs in a secure environment that is both readily available and scalable. In addition to the logs being accessible when the server(s) they are logging are inaccessible, aggregating and organize your logs in a central location can be a powerful tool to build reports and analyze trends. With that information, you’ll be able to more clearly see deviations from normal activity to catch attacks (or attempted attacks) in progress.

How do you protect your event logs?
Attacks can come from both inside and out. To avoid intentional malicious activity by insiders, separation of duties should be enforced when planning logging. Learn from The X Files and “Trust no one.” Someone who has been granted the ‘keys to your castle’ shouldn’t also be able to disable the castle’s security system or mess with the castle’s logs. Your network engineer shouldn’t have exclusive access to your router logs, and your sysadmin shouldn’t be the only one looking at your web server logs.

Keep consistent time.
Make sure all of your servers are using the same accurate time source. That way, all logs generated from those servers will share consistent time-stamps. Trying to diagnose an attack or incident is exceptionally more difficult if your web server’s clock isn’t synced with your database server’s clock or if they’re set to different time zones. You’re putting a lot of time and effort into logging events, so you’re shooting yourself in the foot if events across all of your servers don’t line up cleanly.

Read your logs!
Logs won’t do you any good if you’re not looking at them. Know the red flags to look for in each of your logs, and set aside time to look for those flags regularly. Several SoftLayer customers — like Tech Partner Papertrail — have come up with innovative and effective log management platforms that streamline the process of aggregating, searching and analyzing log files.

It’s important to reiterate that logging — like any other security endeavor — is not a ‘one size fits all’ model, but that shouldn’t discourage you from getting started. If you aren’t logging or you aren’t actively monitoring your logs, any step you take is a step forward, and each step is worth the effort.

Thanks for reading, and stay secure, my friends!

-Matthew

December 27, 2012

Using SoftLayer Object Storage to Back Up Your Server

By in Development, SoftLayer, Tips and Tricks

Before I came to my senses and moved my personal servers to SoftLayer, I was one of many victims of a SolusVM exploit that resulted in the wide-scale attack of many nodes in my previous host’s Chicago data center. While I’m a firm believer in backing up my data, I could not have foreseen the situation I was faced with: Not only was my server in one data center compromised with all of its data deleted, but my backup server in one of the host’s other data centers was also attacked … This left me with old, stale backups on my local computer and not much else. I quickly relocated my data and decided that I should use SoftLayer Object Storage to supplement and improve upon my backup and disaster recovery plans.

With SoftLayer Object Storage Python Client set up and the SoftLayer Object Storage Backup script — slbackup.py — in hand, I had the tools I needed to build a solid backup infrastructure easily. On Linux.org, I contributed an article about how to perform MySQL backups with those resources, so the database piece is handled, but I also need to back up my web files, so I whipped up another quick bash script to run:

#!/bin/bash
 
# The path the backups will be dumped to
DUMP_DIR="/home/backups/"
 
# Path to the web files to be backed up
BACKUP_PATH="/var/www/sites /"
 
# Back up folder name (mmddyyyy)
BACKUP_DIR="`date +%m%d%Y`"
 
# Backup File Name
DUMP_FILE="`date +%m_%d_%Y_%H_%M_%S`_site_files"
 
# SL container name
CONTAINER="site_backups"
 
# Create backup dir if doesn't exist
if [ ! -d $DUMP_DIR$BACKUP_DIR ]; then
        mkdir -p $DUMP_DIR$BACKUP_DIR
fi
 
tar -zcvpf $DUMP_DIR$BACKUP_DIR/$DUMP_FILE.tar.gz $BACKUP_PATH
 
# Make sure the archive exists
if [ -f $DUMP_DIR$BACKUP_DIR/$DUMP_FILE.tar.gz ]; then
        /root/slbackup.py -s $DUMP_DIR$BACKUP_DIR/ -o "$CONTAINER" -r 30
 
        # Remove the backup stored locally
        rm -rf $DUMP_DIR$BACKUP_DIR
 
        # Success
        exit 0
else
        echo "$DUMP_DIR$BACKUP_DIR/$DUMP_FILE.tar.gz does not exist."
        exit 1
fi

It’s not the prettiest bash script, but it gets the job done. By tweaking a few variables, you can easily generate backups for any important directory of files and push them to your SoftLayer Object Storage account. If you want to change the retention time of your backups to be longer or shorter, you can change the 30 after the –r in the line below to the number of days you want to keep each backup:

/root/slbackup.py -s $DUMP_DIR$BACKUP_DIR/ -o "$CONTAINER" -r 30

I created a script for each website on my server, and I set a CRON (crontab –e) entry to run each one on Sundays staggered by 5 minutes:

5 1 * * 0  /root/bin/cron/CRON-site1.com_web_files > /dev/null
10 1 * * 0  /root/bin/cron/CRON-site2.com_web_files > /dev/null
15 1 * * 0  /root/bin/cron/CRON-site3.com_web_files > /dev/null

If you’re looking for an easy way to automate and solidify your backups, this little bit of code could make life easier on you. Had I taken the few minutes to put this script together prior to the attack I experienced at my previous host, I wouldn’t have lost any of my data. It’s easy to get lulled into “backup apathy” when you don’t need your backups, but just because nothing *has* happened to your data doesn’t mean nothing *can* happen to your data.

Take it from me … Be over-prepared and save yourself a lot of trouble.

-Ronald

December 6, 2012

MongoDB: Architectural Best Practices

By in Development, Infrastructure, SoftLayer, Technology, Tips and Tricks

With the launch of our MongoDB solutions, developers can provision powerful, optimized, horizontally scaling NoSQL database clusters in real-time on bare metal infrastructure in SoftLayer data centers around the world. We worked tirelessly with our friends at 10gen — the creators of MongoDB — to build and tweak hardware and software configurations that enable peak MongoDB performance, and the resulting platform is pretty amazing. As Duke mentioned in his blog post, those efforts followed 10Gen’s MongoDB best practices, but what he didn’t mention was that we created some architectural best practices of our own for MongoDB in deployments on our platform.

The MongoDB engineered servers that you order from SoftLayer already implement several of the recommendations you’ll see below, and I’ll note which have been incorporated as we go through them. Given the scope of the topic, it’s probably easiest to break down this guide into a few sections to make it a little more digestible. Let’s take a look at the architectural best practices of running MongoDB through the phases of the roll-out process: Selecting a deployment strategy to prepare for your MongoDB installation, the installation itself, and the operational considerations of running it in production.

Deployment Strategy

When planning your MongoDB deployment, you should follow Sun Tzu’s (modified) advice: “If you know the [friend] and know yourself, you need not fear the result of a hundred battles.” “Friend” was substituted for the “enemy” in this advice because the other party is MongoDB. If you aren’t familiar with MongoDB, the top of your to-do list should be to read MongoDB’s official documentation. That information will give you the background you’ll need as you build and use your database. When you feel comfortable with what MongoDB is all about, it’s time to “know yourself.”

Your most important consideration will be the current and anticipated sizes of your data set. Understanding the volume of data you’ll need to accommodate will be the primary driver for your choice of individual physical nodes as well as your sharding plans. Once you’ve established an expected size of your data set, you need to consider the importance of your data and how tolerant you are of the possibility of lost or lagging data (especially in replicated scenarios). With this information in hand, you can plan and start testing your deployment strategy.

It sounds a little strange to hear that you should test a deployment strategy, but when it comes to big data, you want to make sure your databases start with a strong foundation. You should perform load testing scenarios on a potential deployment strategy to confirm that a given architecture will meet your needs, and there are a few specific areas that you should consider:

Memory Sizing
MongoDB (like many data-oriented applications) works best when the data set can reside in memory. Nothing performs better than a MongoDB instance that does not require disk I/O. Whenever possible, select a platform that has more available RAM than your working data set size. If your data set exceeds the available RAM for a single node, then consider using sharding to increase the amount of available RAM in a cluster to accommodate the larger data set. This will maximize the overall performance of your deployment. If you notice page faults when you put your database under production load, they may indicate that you are exceeding the available RAM in your deployment.

Disk Type
If speed is not your primary concern or if you have a data set that is far larger than any available in memory strategy can support, selecting the proper disk type for your deployment is important. IOPS will be key in selecting your disk type and obviously the higher the IOPS the better the performance of MongoDB. Local disks should be used whenever possible (as network storage can cause high latency and poor performance for your deployment). It’s also advised that you use RAID 10 when creating disk arrays.

To give you an idea of what kind of IOPS to expect from a given type of drive, these are the approximate ranges of IOPS per drive in SoftLayer MongoDB engineered servers:

SATA II – 100-200 IOPS
15K SAS – 300-400 IOPS
SSD – 7,000-8,000 IOPS (read) 19,000-20,000 IOPS (write)

CPU
Clock speed and the amount of available processors becomes a consideration if you anticipate using MapReduce. It has also been noted that when running a MongoDB instance with the majority of the data in memory, clock speed can have a major impact on overall performance. If you are planning to use MapReduce or you’re able to operate with a majority of your data in memory, consider a deployment strategy that includes a CPU with a high clock/bus speed to maximize your operations per second.

Replication
Replication provides high availability of your data if a node fails in your cluster. It should be standard to replicate with at least three nodes in any MongoDB deployment. The most common configuration for replication with three nodes is a 2×1 deployment — having two primary nodes in a single data center with a backup server in a secondary data center:

MongoDB Replication

Sharding
If you anticipate a large, active data set, you should deploy a sharded MongoDB deployment. Sharding allows you to partition a single data set across multiple nodes. You can allow MongoDB to automatically distribute the data across nodes in the cluster or you may elect to define a shard key and create range-based sharding for that key.

Sharding may also help write performance, so you can also elect to shard even if your data set is small but requires a high amount of updates or inserts. It’s important to note that when you deploy a sharded set, MongoDB will require three (and only three) config server instances which are specialized Mongo runtimes to track the current shard configuration. Loss of one of these nodes will cause the cluster to go into a read-only mode (for the configuration only) and will require that all nodes be brought back online before any configuration changes can be made.

Write Safety Mode
There are several write safety modes that govern how MongoDB will handle the persistence of the data to disk. It is important to consider which mode best fits your needs for both data integrity and performance. The following write safety modes are available:

None – This mode provides a deferred writing strategy that is non-blocking. This will allow for high performance, however there is a small opportunity in the case of a node failing that data can be lost. There is also the possibility that data written to one node in a cluster will not be immediately available on all nodes in that cluster for read consistency. The ‘None’ strategy will also not provide any sort of protection in the case of network failures. That lack of protection makes this mode highly unreliable and should only be used when performance is a priority and data integrity is not a concern.

Normal – This is the default for MongoDB if you do not select any other mode. It provides a deferred writing strategy that is non-blocking. This will allow for high performance, however there is a small opportunity in the case of a node failing that data can be lost. There is also the possibility that data written to one node in a cluster will not be immediately available on all nodes in that cluster for read consistency.

Safe – This mode will block until MongoDB has acknowledged that it has received the write request but will not block until the write is actually performed. This provides a better level of data integrity and will ensure that read consistency is achieved within a cluster.

Journal Safe – Journals provide a recovery option for MongoDB. Using this mode will ensure that the data has been acknowledged and a Journal update has been performed before returning.

Fsync – This mode provides the highest level of data integrity and blocks until a physical write of the data has occurred. This comes with a degradation in performance and should be used only if data integrity is the primary concern for your application.

Testing the Deployment
Once you’ve determined your deployment strategy, test it with a data set similar to your production data. 10gen has several tools to help you with load testing your deployment, and the console has a tool named ‘benchrun’ which can execute operations from within a JavaScript test harness. These tools will return operation information as well as latency numbers for each of those operations. If you require more detailed information about the MongoDB instance, consider using the mongostat command or MongoDB Monitoring Service (MMS) to monitor your deployment during the testing.

Installation

When performing the installation of MongoDB, a few considerations can help create both a stable and performance-oriented solution. 10gen recommends the use CentOS (64-bit) as the base operating system if at all possible. If you try installing MongoDB on a 32-bit operating system, you might run into file size limits that cause issues, and if you feel the urge to install it on Windows, you’ll see performance issues if virtual memory begins to be utilized by the OS to make up for a lack of RAM in your deployment. As a result, 32-bit operating systems and Windows operating systems should be avoided on MongoDB servers. SoftLayer provisions CentOS 6.X 64-bit operating systems by default on all of our MongoDB engineered server deployments.

When you’ve got CentOS 64-bit installed, you should also make the following changes to maximize your performance (all of which are included by default on all SoftLayer engineered servers):

Set SSD Read Ahead Defaults to 16 Blocks – SSD drives have excellent seek times allowing for shrinking the Read Ahead to 16 blocks. Spinning disks might require slight buffering so these have been set to 32 blocks.

noatime – Adding the noatime option eliminates the need for the system to make writes to the file system for files which are simply being read — or in other words: Faster file access and less disk wear.

Turn NUMA Off in BIOS – Linux, NUMA and MongoDB tend not to work well together. If you are running MongoDB on NUMA hardware, we recommend turning it off (running with an interleave memory policy). If you don’t, problems will manifest in strange ways like massive slow downs for periods of time or high system CPU time.

Set ulimit – We have set the ulimit to 64000 for open files and 32000 for user processes to prevent failures due to a loss of available file handles or user processes.

Use ext4 – We have selected ext4 over ext3. We found ext3 to be very slow in allocating files (or removing them). Additionally, access within large files is poor with ext3.

One last tip on installation: Make the Journal and Data volumes be distinct physical volumes. If the Journal and Data directories reside on a single physical volume, flushes to the Journal will interrupt the access of data and provide spikes of high latency within your MongoDB deployment.

Operations

Once a MongoDB deployment has been promoted to production, there are a few recommendations for monitoring and optimizing performance. You should always have the MMS agent running on all MongoDB instances to help monitor the health and performance of your deployment. Additionally, this tool is also very useful if you have 10gen MongoDB Cloud Subscriptions because it provides useful debugging data for the 10gen team during support interactions. In addition to MMS, you can use the mongostat command (mentioned in the deployment section) to see runtime information about the performance of a MongoDB node. If either of these tools flags performance issues, sharding or indexing are first-line options to resolve them:

Indexes – Indexes should be created for a MongoDB deployment if monitoring tools indicate that field based queries are performing poorly. Always use indexes when you are querying data based on distinct fields to help boost performance.

Sharding – Sharding can be leveraged when the overall performance of the node is suffering because of a large operating data set. Be sure to shard before you get in the red; the system only splits chunks for sharding on insert or update so if you wait too long to shard you may have some uneven distribution for a period of time or forever depending on your data set and sharding key strategy.

I know it seems like we’ve covered a lot over the course of this blog post, but this list of best practices is far from exhaustive. If you want to learn more, the MongoDB forums are a great resource to connect with the rest of the MongoDB community and learn from their experiences, and the documentation on MongoDB’s site is another phenomenal resource. The best people to talk to when it comes to questions about MongoDB are the folks at 10gen, so I also highly recommend taking advantage of MongoDB Cloud Subscriptions to get their direct support for your one-off questions and issues.

-Harold

November 27, 2012

Tips and Tricks – Building a jQuery Plugin (Part 1)

By in Development, Tips and Tricks

I’ve written several blogs detailing the use of different jQuery plugins (like Select2, LazyLoad and equalHeights), and in the process, I’ve noticed an increasing frustration among the development community when it comes to building jQuery plugins. The resources and documentation I’ve found online have not as clear and easy as they could be, so in my next few posts, I’ll break down the process to make jQuery plugin creation simple and straightforward. In this post, we’ll cover the basic structure of a plugin and where to insert your own functionality, and in Part 2, we’ll pick a simple task and add on to our already-made structure.

Before I go any further, it’s probably important to address a question you might be asking yourself: “Why would I want to make my own plugin?” The best reason that comes to my mind is portability. If you’ve ever created a large-scale project, take a look back into your source code and note how many of the hundreds of lines of jQuery code you could put into a plugin to reuse on a different project. You probably invested a lot of time and energy into that code, so it doesn’t make sense to reinvent the wheel if you ever need that functionality again. If that’s not enough of a reason for you, I can also tell you that if you develop your own jQuery plugin, you’ll level-up in cool points, and the jQuery community will love you.

For this post, let’s create a jQuery plugin that simply returns, “This is our awesome plugin!” Our first step involves putting together the basic skeleton used by every plugin:

(function($) {
    $.fn.slPlugin = function() {
 
            // Awesome plugin stuff goes here
    };
}) (jQuery);

This is your template — your starting point. Practice it. Remember it. Love it. The “slPlugin” piece is what I chose to name this plugin. It’s best to name your plugin something unique … I always run a quick Google search to ensure I don’t duplicate the name of a plugin I (or someone else) might need to use in a project alongside my plugin. In this case, we’re calling the example plugin slPlugin because SoftLayer is awesome, and I like naming my plugins after awesome things. I’ll save this code in a file called jquery.slPlugin.js.

Now that we have our plugin’s skeleton, let’s add some default values for variables:

(function($) {
    $.fn.slPlugin = function(options) {
            var defaults = {
                myVar: "default", // this will be the default value of this var
                anotherVar: 0,
                coolVar: "this is cool",                
            };
            var options = $.extend(defaults, options);
    };
}) (jQuery);

Let’s look at the changes we made between the first example and this one. You’ll notice that in our second line we added “options” to become $.fn.slPlugin = function(options) {. We do this because our function is now accepting arguments, and we need to let the function know that. The next difference you come across is the var defaults blurb. In this section, we’re providing default values for our variables. If you don’t define values for a given variable when you call the plugin, these default values will be used.

Now let’s have our plugin return the message we want to send:

(function($) {
    $.fn.slPlugin = function(options) {
            var defaults = {
                myVar: "This is", // this will be the default value of this var
                anotherVar: "our awesome",
                coolVar: "plugin!",
            };
            var options = $.extend(defaults, options);
            this.each(function() {
                ourString = myVar + " " + anotherVar + " " + coolVar;
            });
            return ourString;
    };
}) (jQuery);

We’ve defined our default values for our variables, concatenated our variables and we’ve added a return under our variable declaration. If our jQuery plugin is included in a project and no values are provided for our variables, slPlugin will return, “This is our awesome plugin!”

It seems rather rudimentary at this point, but we have to crawl before we walk. This introductory post is laying the groundwork of coding a jQuery plugin, and we’ll continue building on this example in the next installment of this series. As you’ve seen with the LazyLoad, equalHeights and Select2, there are much more complicated things we can do with our plugin, and we’ll get there. Sneak Preview: In the next installment, we’ll be creating and implementing a truncation function for our plugin … Get excited!

-Cassandra

November 21, 2012

Risk Management: The Importance of Redundant Backups

By in Business, Tips and Tricks

You (should) know the importance of having regular backups of your important data, but to what extent does data need to be backed up to be safe? With a crowbar and shove, thieves broke into my apartment and stole the backups I’ve used for hundreds of gigabytes of home videos, photo files and archives of past computers. A Dobro RAID enclosure and an external drive used by Apple Time Machine were both stolen, and if I didn’t have the originals on my laptop or a redundant offsite backup, I would have lost all of my data. My experience is not uncommon, and it’s a perfect example of an often understated principle that everyone should understand: You need redundant backups.

It’s pretty simple: You need to back up your data regularly. When you’ve set up that back up schedule, you should figure out a way to back up your data again. After you’ve got a couple current backups of your files, you should consider backing up your backups off-site. It seems silly to think of backing up backups, but if anything happens — failed drives, theft, fire, flood, etc. — those backups could be lost forever, and if you’ve ever lost a significant amount of data due to a hard drive failure or experience like mine, you know that backups are worth their weight in gold.

Admittedly, there is a point of diminishing return when it comes to how much redundancy is needed — it’s not worth the time/effort/cost to back up your backups ad infinitum — so here are the best practices I’ve come up with over the course of my career in the information technology industry:

  • Plan and schedule regular backups to keep your archives current. If your laptop’s hard drive dies, having backups from last June probably won’t help you as much as backups from last night.
  • Make sure your data exists on three different mediums. It might seem unnecessary, but if you’re already being intentional about backing up your information, take it one step further to replicate those backups at least one more time.
  • Something might happen to your easy onsite backups, so it’s important to consider off-site backups as well. There are plenty of companies offering secure online backups for home users, and those are generally easy to use (even if they can be a little slow).
  • Check your backups regularly. Having a backup is useless if it’s not configured to back up the correct data and running on the correct schedule.
  • RAID is not a backup solution. Yes, RAID can duplicate data across hard drives, but that doesn’t mean the data is “backed up” … If the RAID array fails, all of the hard drives (and all of the data) in the array fail with it.

It’s important to note here that “off-site” is a pretty relative term when it comes to backups. Many SoftLayer customers back up a primary drive on their server to a secondary drive on the same server (duplicating the data away from the original drive), and while that’s better than nothing, it’s also a little risky because it’s possible that the server could fail and corrupt both drives. Every backup product SoftLayer offers for customers is off-site relative to the server itself (though it might be in the same facility), so we also make it easy to have your backup in another city or on a different continent.

As I’ve mentioned already, once you set up your backups, you’re not done. You need to check your backups regularly for failures and test them to confirm that you can recover your data quickly in the event of a disaster. Don’t just view a file listing. Try extracting files or restore the whole backup archive. If you’re able to run a full restore without the pressure of an actual emergency, it’ll prove that you’re ready for the unexpected … Like a fire drill for your backups.

Setting up a backup plan doesn’t have to be scary or costly. If you don’t feel like you could recover quickly after losing your data, spend a little time evaluating ways to make a recovery like that easy. It’s crazy, but a big part of “risk management,” “disaster recovery” and “business continuity” is simply making sure your data is securely backed up regularly and available to you when you need it.

Plan, prepare, back up.

-Lyndell

November 14, 2012

Risk Management: Securing Your Servers

By in Business, Technology, Tips and Tricks

How do you secure your home when you leave? If you’re like most people, you make sure to lock the door you leave from, and you head off to your destination. If Phil is right about “locks keeping honest people honest,” simply locking your front door may not be enough. When my family moved into a new house recently, we evaluated its physical security and tried to determine possible avenues of attack (garage, doors, windows, etc.), tools that could be used (a stolen key, a brick, a crowbar, etc.) and ways to mitigate the risk of each kind of attack … We were effectively creating a risk management plan.

Every risk has different probabilities of occurrence, potential damages, and prevention costs, and the risk management process helps us balance the costs and benefits of various security methods. When it comes to securing a home, the most effective protection comes by using layers of different methods … To prevent a home invasion, you might lock your door, train your dog to make intruders into chew toys and have an alarm system installed. Even if an attacker can get a key to the house and bring some leftover steaks to appease the dog, the motion detectors for the alarm are going to have the police on their way quickly. (Or you could violate every HOA regulation known to man by digging a moat around the house, filling with sharks with laser beams attached to their heads, and building a medieval drawbridge over the moat.)

I use the example of securing a house because it’s usually a little more accessible than talking about “server security.” Server security doesn’t have to be overly complex or difficult to implement, but its stigma of complexity usually prevents systems administrators from incorporating even the simplest of security measures. Let’s take a look at the easiest steps to begin securing your servers in the context of their home security parallels, and you’ll see what I’m talking about.

Keep “Bad People” Out: Have secure password requirements.

Passwords are your keys and your locks — the controls you put into place that ensure that only the people who should have access get it. There’s no “catch all” method of keeping the bad people out of your systems, but employing a variety of authentication and identification measures can greatly enhance the security of your systems. A first line of defense for server security would be to set password complexity and minimum/maximum password age requirements.

If you want to add an additional layer of security at the authentication level, you can incorporate “Strong” or “Two-Factor” authentication. From there, you can learn about a dizzying array of authentication protocols (like TACACS+ and RADIUS) to centralize access control or you can use active directory groups to simplify the process of granting and/or restricting access to your systems. Each layer of authentication security has benefits and drawbacks, and most often, you’ll want to weigh the security risk against your need for ease-of-use and availability as you plan your implementation.

Stay Current on your “Good People”: When authorized users leave, make sure their access to your system leaves with them.

If your neighbor doesn’t return borrowed tools to your tool shed after you gave him a key when he was finishing his renovation, you need to take his key back when you tell him he can’t borrow any more. If you don’t, nothing is stopping him from walking over to the shed when you’re not looking and taking more (all?) of your tools. I know it seems like a silly example, but that kind of thing is a big oversight when it comes to server security.

Employees are granted access to perform their duties (the principle of least privilege), and when they no longer require access, the “keys to the castle” should be revoked. Auditing who has access to what (whether it be for your systems or for your applications) should be continual.

You might have processes in place to grant and remove access, but it’s also important to audit those privileges regularly to catch any breakdowns or oversights. The last thing you want is to have a disgruntled former employee wreak all sorts of havoc on your key systems, sell proprietary information or otherwise cost you revenue, fines, recovery efforts or lost reputation.

Catch Attackers: Monitor your systems closely and set up alerts if an intrusion is detected.

There is always a chance that bad people are going to keep looking for a way to get into your house. Maybe they’ll walk around the house to try and open the doors and windows you don’t use very often. Maybe they’ll ring the doorbell and if no lights turn on, they’ll break a window and get in that way.

You can never completely eliminate all risk. Security is a continual process, and eventually some determined, over-caffeinated hacker is going to find a way in. Thinking your security is impenetrable makes you vulnerable if by some stretch of the imagination, an attacker breaches your security (see: Trojan Horse). Continuous monitoring strategies can alert administrators if someone does things they shouldn’t be doing. Think of it as a motion detector in your house … “If someone gets in, I want to know where they are.” When you implement monitoring, logging and alerting, you will also be able to recover more quickly from security breaches because every file accessed will be documented.

Minimize the Damage: Lock down your system if it is breached.

A burglar smashes through your living room window, runs directly to your DVD collection, and takes your limited edition “Saved by the Bell” series box set. What can you do to prevent them from running back into the house to get the autographed posted of Alf off of your wall?

When you’re monitoring your servers and you get alerted to malicious activity, you’re already late to the game … The damage has already started, and you need to minimize it. In a home security environment, that might involve an ear-piercing alarm or filling the moat around your house even higher so the sharks get a better angle to aim their laser beams. File integrity monitors and IDS software can mitigate damage in a security breach by reverting files when checksums don’t match or stopping malicious behavior in its tracks.

These recommendations are only a few of the first-line layers of defense when it comes to server security. Even if you’re only able to incorporate one or two of these tips into your environment, you should. When you look at server security in terms of a journey rather than a destination, you can celebrate the progress you make and look forward to the next steps down the road.

Now if you’ll excuse me, I have to go to a meeting where I’m proposing moats, drawbridges, and sharks with laser beams on their heads to SamF for data center security … Wish me luck!

-Matthew

November 6, 2012

Tips and Tricks – Pure CSS Sticky Footers

By in Development, Tips and Tricks

By now, if you’ve seen my other blog posts, you know that I’m fascinated with how much JavaScript has evolved and how much you can do with jQuery these days. I’m an advocate of working smarter, not harder, and that maxim knows no coding language limits. In this post, I want to share a pure CSS solution that allows for “sticky” footers on a web page. In comparing several different techniques to present this functionality, I found that all of the other routes were overkill when it came to processing time and resource usage.

Our objective is simple: Make the footer of our web page stay at the bottom even if the page’s content area is shorter than the user’s browser window.

This, by far, is one of my *favorite* things to do. It makes the web layout so much more appealing and creates a very professional feel. I ended up kicking myself the very first time I tried to add this functionality to a project early in my career (ten years ago … already!?) when I found out just how easy it was. I take solace in knowing that I’m not alone, though … A quick search for “footer stick bottom” still yields quite a few results from fellow developers who are wrestling with the same frustrating experience I did. If you’re in that boat, fear no more! We’re going to your footers in shape in a snap.

Here’s a diagram of the problem:

CSS Footer

Unfortunately, a lot of people try to handle it with setting a fixed height to the content which would push the footer down. This may work when YOU view it, but there are several different browser window heights, resolutions and variables that make this an *extremely* unreliable solution (notice the emphasis on the word “extremely” … this basically means “don’t do it”).

We need a dynamic solution that is able to adapt on the fly to the height of a user’s browser window regardless if the resize it, have Firebug open, use a unique resolution or just have a really, really weird browser!

Let’s take a look at what the end results should look like:

CSS Footer

To make this happen, let’s get our HTML structure in place first:

<div id="page">
 
      <div id="header"> </div>
 
      <div id="main"> </div>
 
      <div id="footer"> </div>
 
</div>

It’s pretty simple so far … Just a skeleton of a web page. The page div contains ALL elements and is immediately below the tags in the page code hierarchy. The header div is going to be our top content, the main div will include all of our content, and the footer div is all of our copyrights and footer links.

Let’s start by coding the CSS for the full page:

Html, body {
      Padding: 0;
      Margin: 0;
      Height: 100%;
}

Adding a 100% height allows us to set the height of the main div later. The height of a div can only be as tall as the parent element encasing it. Now let’s see how the rest of our ids are styled:

#page {
      Min-height: 100%;
      position:relative;
}
 
#main {
      Padding-bottom: 75px;   /* This value is the height of your footer */
}
 
#footer {
      Position: absolute;
      Width: 100%;
      Bottom: 0;
      Height: 75px;  /* This value is the height of your footer */
}

These rules position the footer “absolutely” at the bottom of the page, and because we set #page to min-height: 100%, it ensures that #main is exactly the height of the browser’s viewing space. One of the best things about this little trick is that it’s compliant with all major current browsers — including Firefox, Chrome, Safari *AND* Internet Explorer (after a little tweak). For Internet Explorer to not throw a fit, we need concede that IE doesn’t recognize min-height as a valid property, so we have to add Height: 100%; to #page:

#page {
      Min-height: 100%;  /* for all other browsers */
      height: 100%;  /* for IE */
      position:relative;
}

If the user does not have a modern, popular browser, it’s still okay! Though their old browser won’t detect the magic we’ve done here, it’ll fail gracefully, and the footer will be positioned directly under the content, as it would have been without our little CSS trick.

I can’t finish this blog without mentioning my FAVORITE perk of this trick: Should you not have a specially designed mobile version of your site, this trick even works on smart phones!

-Cassandra

November 2, 2012

The Trouble with Open DNS Resolvers

By in SoftLayer, Technology, Tips and Tricks

In the last couple of days, there’s been a bit of buzz about “open DNS resolvers” and DNS amplification DDoS attacks, and SoftLayer’s name has been brought up a few times. In a blog post on October 30, CloudFlare explained DNS Amplification DDoS attacks and reported the geographic and network sources of open DNS resolvers that were contributing to a 20Gbps attack on their network. SoftLayer’s AS numbers (SOFTLAYER and the legacy THEPLANET-AS number) show up on the top ten “worst offenders” list, and Dan Goodin contacted us to get a comment for a follow-up piece on Ars Technica — Meet the network operators helping to fuel the spike in big DDoS attacks.

While the content of that article is less sensationalized than the title, there are still a few gaps to fill about when it comes to how SoftLayer is actually involved in the big picture (*SPOILER ALERT* We aren’t “helping to fuel the spike in big DDoS attacks”). The CloudFlare blog and the Ars Technica post presuppose that the presence of open recursive DNS resolvers is a sign of negligence on the part of the network provider at best and maliciousness at worst, and that’s not the case.

The majority of SoftLayer’s infrastructure is made up of self-managed dedicated and cloud servers. Customers who rent those servers on a monthly basis have unrestricted access to operate their servers in any way they’d like as long as that activity meets our acceptable use policy. Some of our largest customers are hosting resellers who provide that control to their customers who can then provide that control to their own customers. And if 23 million hostnames reside on the SoftLayer network, you can bet that we’ve got a lot of users hosting their DNS on SoftLayer infrastructure. Unfortunately, it’s easier for those customers and customers-of-customers and customers-of-customers-of-customers to use “defaults” instead of looking for, learning and implementing “best practices.”

It’s all too common to find those DNS resolvers open and ultimately vulnerable to DNS amplification attacks, and whenever our team is alerted to that vulnerability on our network, we make our customers aware of it. In turn, they may pass the word down the customer-of-customer chain to get to the DNS owner. It’s usually not a philosophical question about whether DNS resolvers should be open for the greater good of the Internet … It’s a question of whether the DNS owner has any idea that their “configuration” is vulnerable to be abused in this way.

SoftLayer’s network operations, abuse and support teams have tools that flag irregular and potentially abusive traffic coming from any server on our network, and we take immediate action when we find a problem or are alerted to one by someone who sends details to abuse@softlayer.com. The challenge we run into is that flagging obvious abusive behavior from an active DNS server is a bit of a cat-and-mouse game … Attackers cloak their activity in normal traffic. Instead of sending a huge amount of traffic from a single domain, they send a marginal amount of traffic from a large number of machines, and the “abusive” traffic is nearly impossible for even the DNS owner to differentiate from “regular” traffic.

CloudFlare effectively became a honeypot, and they caught a distributed DNS amplification DoS attack. The results they gathered are extremely valuable to teams like mine at SoftLayer, so if they go the next step to actively contact the abuse channel for each of the network providers in their list, I hope that each of the other providers will jump on that information as I know my team will.

If you have a DNS server on the SoftLayer network, and you’re not sure whether it’s configured to prevent it from being used for these types of attacks, our support team is happy to help you out. For those of you interested in doing a little DNS homework to learn more, Google’s Developer Network has an awesome overview of DNS security threats and mitigations which gives an overview of potential attacks and preventative measures you can take. If you’re just looking for an easy way to close an open recursor, scroll to the bottom of CloudFlare’s post, and follow their quick guide.

If, on the other hand, you have your own DNS server and you don’t want to worry about all of this configuration or administration, SoftLayer operates private DNS resolvers that are limited to our announced IP space. Feel free to use ours instead!

-Ryan