development

December 17, 2012

Big Data at SoftLayer: The Importance of IOPS

The jet flow gates in the Hoover Dam can release up to 73,000 cubic feet — the equivalent of 546,040 gallons — of water per second at 120 miles per hour. Imagine replacing those jet flow gates with a single garden hose that pushes 25 gallons per minute (or 0.42 gallons per second). Things would get ugly pretty quickly. In the same way, a massive "big data" infrastructure can be crippled by insufficient IOPS.

IOPS — Input/Output Operations Per Second — measure computer storage in terms of the number of read and write operations it can perform in a second. IOPS are a primary concern for database environments where content is being written and queried constantly, and when we take those database environments to the extreme (big data), the importance of IOPS can't be overstated: If you aren't able perform database reads and writes quickly in a big data environment, it doesn't matter how many gigabytes, terabytes or petabytes you have in your database ... You won't be able to efficiently access, add to or modify your data set.

As we worked with 10gen to create, test and tweak SoftLayer's MongoDB engineered servers, our primary focus centered on performance. Since the performance of massively scalable databases is dictated by the read and write operations to that database's data set, we invested significant resources into maximizing the IOPS for each engineered server ... And that involved a lot more than just swapping hard drives out of servers until we found a configuration that worked best. Yes, "Disk I/O" — the amount of input/output operations a given disk can perform — plays a significant role in big data IOPS, but many other factors limit big data performance. How is performance impacted by network-attached storage? At what point will a given CPU become a bottleneck? How much RAM should included in a base configuration to accommodate the load we expect our users to put on each tier of server? Are there operating system changes that can optimize the performance of a platform like MongoDB?

The resulting engineered servers are a testament to the blood, sweat and tears that were shed in the name of creating a reliable, high-performance big data environment. And I can prove it.

Most shared virtual instances — the scalable infrastructure many users employ for big data — use network-attached storage for their platform's storage. When data has to be queried over a network connection (rather than from a local disk), you introduce latency and more "moving parts" that have to work together. Disk I/O might be amazing on the enterprise SAN where your data lives, but because that data is not stored on-server with your processor or memory resources, performance can sporadically go from "Amazing" to "I Hate My Life" depending on network traffic. When I've tested the IOPS for network-attached storage from a large competitor's virtual instances, I saw an average of around 400 IOPS per mount. It's difficult to say whether that's "not good enough" because every application will have different needs in terms of concurrent reads and writes, but it certainly could be better. We performed some internal testing of the IOPS for the hard drive configurations in our Medium and Large MongoDB engineered servers to give you an apples-to-apples comparison.

Before we get into the tests, here are the specs for the servers we're using:

Medium (MD) MongoDB Engineered Server
Dual 6-core Intel 5670 CPUs
CentOS 6 64-bit
36GB RAM
1Gb Network - Bonded
Large (LG) MongoDB Engineered Server
Dual 8-core Intel E5-2620 CPUs
CentOS 6 64-bit
128GB RAM
1Gb Network - Bonded
 

The numbers shown in the table below reflect the average number of IOPS we recorded with a 100% random read/write workload on each of these engineered servers. To measure these IOPS, we used a tool called fio with an 8k block size and iodepth at 128. Remembering that the virtual instance using network-attached storage was able to get 400 IOPS per mount, let's look at how our "base" configurations perform:

Medium - 2 x 64GB SSD RAID1 (Journal) - 4 x 300GB 15k SAS RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 2937
Random Write IOPS - /var/lib/mongo/logs 1306
Random Read IOPS - /var/lib/mongo/data 1720
Random Write IOPS - /var/lib/mongo/data 772
Random Read IOPS - /var/lib/mongo/data/journal 19659
Random Write IOPS - /var/lib/mongo/data/journal 8869
   
Medium - 2 x 64GB SSD RAID1 (Journal) - 4 x 400GB SSD RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 30269
Random Write IOPS - /var/lib/mongo/logs 13124
Random Read IOPS - /var/lib/mongo/data 33757
Random Write IOPS - /var/lib/mongo/data 14168
Random Read IOPS - /var/lib/mongo/data/journal 19644
Random Write IOPS - /var/lib/mongo/data/journal 8882
   
Large - 2 x 64GB SSD RAID1 (Journal) - 6 x 600GB 15k SAS RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 4820
Random Write IOPS - /var/lib/mongo/logs 2080
Random Read IOPS - /var/lib/mongo/data 2461
Random Write IOPS - /var/lib/mongo/data 1099
Random Read IOPS - /var/lib/mongo/data/journal 19639
Random Write IOPS - /var/lib/mongo/data/journal 8772
 
Large - 2 x 64GB SSD RAID1 (Journal) - 6 x 400GB SSD RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 32403
Random Write IOPS - /var/lib/mongo/logs 13928
Random Read IOPS - /var/lib/mongo/data 34536
Random Write IOPS - /var/lib/mongo/data 15412
Random Read IOPS - /var/lib/mongo/data/journal 19578
Random Write IOPS - /var/lib/mongo/data/journal 8835

Clearly, the 400 IOPS per mount results you'd see in SAN-based storage can't hold a candle to the performance of a physical disk, regardless of whether it's SAS or SSD. As you'd expect, the "Journal" reads and writes have roughly the same IOPS between all of the configurations because all four configurations use 2 x 64GB SSD drives in RAID1. In both configurations, SSD drives provide better Data mount read/write performance than the 15K SAS drives, and the results suggest that having more physical drives in a Data mount will provide higher average IOPS. To put that observation to the test, I maxed out the number of hard drives in both configurations (10 in the 2U MD server and 34 in the 4U LG server) and recorded the results:

Medium - 2 x 64GB SSD RAID1 (Journal) - 10 x 300GB 15k SAS RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 7175
Random Write IOPS - /var/lib/mongo/logs 3481
Random Read IOPS - /var/lib/mongo/data 6468
Random Write IOPS - /var/lib/mongo/data 1763
Random Read IOPS - /var/lib/mongo/data/journal 18383
Random Write IOPS - /var/lib/mongo/data/journal 8765
   
Medium - 2 x 64GB SSD RAID1 (Journal) - 10 x 400GB SSD RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 32160
Random Write IOPS - /var/lib/mongo/logs 12181
Random Read IOPS - /var/lib/mongo/data 34642
Random Write IOPS - /var/lib/mongo/data 14545
Random Read IOPS - /var/lib/mongo/data/journal 19699
Random Write IOPS - /var/lib/mongo/data/journal 8764
   
Large - 2 x 64GB SSD RAID1 (Journal) - 34 x 600GB 15k SAS RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 17566
Random Write IOPS - /var/lib/mongo/logs 11918
Random Read IOPS - /var/lib/mongo/data 9978
Random Write IOPS - /var/lib/mongo/data 6526
Random Read IOPS - /var/lib/mongo/data/journal 18522
Random Write IOPS - /var/lib/mongo/data/journal 8722
 
Large - 2 x 64GB SSD RAID1 (Journal) - 34 x 400GB SSD RAID10 (Data)
Random Read IOPS - /var/lib/mongo/logs 34220
Random Write IOPS - /var/lib/mongo/logs 15388
Random Read IOPS - /var/lib/mongo/data 35998
Random Write IOPS - /var/lib/mongo/data 17120
Random Read IOPS - /var/lib/mongo/data/journal 17998
Random Write IOPS - /var/lib/mongo/data/journal 8822

It should come as no surprise that by adding more drives into the configuration, we get better IOPS, but you might be wondering why the results aren't "betterer" when it comes to the IOPS in the SSD drive configurations. While the IOPS numbers improve going from four to ten drives in the medium engineered server and six to thirty-four drives in the large engineered server, they don't increase as significantly as the IOPS differences in the SAS drives. This is what I meant when I explained that several factors contribute to and potentially limit IOPS performance. In this case, the limiting factor throttling the (ridiculously high) IOPS is the RAID card we are using in the servers. We've been working with our RAID card vendor to test a new card that will open a little more headroom for SSD IOPS, but that replacement card doesn't provide the consistency and reliability we need for these servers (which is just as important as speed).

There are probably a dozen other observations I could point out about how each result compares with the others (and why), but I'll stop here and open the floor for you. Do you notice anything interesting in the results? Does anything surprise you? What kind of IOPS performance have you seen from your server/cloud instance when running a tool like fio?

-Kelly

December 10, 2012

Startup Series: GiveToBenefit

People often ask me why I enjoy working at SoftLayer, and that's a tough question to answer fully. I ALWAYS say that great people and great products (in that order) are some of the biggest reasons, and I explain how refreshing it is to work for a company that operates prioritizes "solving problems" over "selling." I share the SoftLayer "Innovate or Die" motto and talk about how radically the world of hosting is changing, and I get to brag about meeting some of the world's most interesting up-and-coming entrepreneurs and how I have the unique opportunity to help amazing startups grow into Internet powerhouses.

I'm the West Coast community development manager for Catalyst, so I get to tell the SoftLayer story to hundreds of entrepreneurs and startups every month at various meetups, demo days, incubator office hours and conferences. In turn, I get to hear the way those entrepreneurs and startups are changing the world. It's a pretty amazing gig. When I was chatting with a few of my colleagues recently, I realized that I'm in a pretty unique position ... Not everyone gets to hear these stories. I've decided that I owe it to my coworkers, our Catalyst participants and anyone else who will listen to write a semi-regular blog series about some of the cool businesses SoftLayer is helping.

Picking one Catalyst participant to feature for this first blog was a pretty challenging task. With the holidays upon us, one company I'm working closely with jumped out as the perfect candidate to feature in this "season of giving": GiveToBenefit.

GiveToBenefit

GiveToBeneift (or G2B) is a social enterprise based in Philadelphia dedicated to helping non-profits receive high-quality goods from select suppliers through crowd-funding. G2B is unique among the startups in the Catalyst program in that it is a "double bottom line" company: It is designed to generate profit for its business while at the same time creating positive social impact.

Crowd-funding — raising money from the public via online donations — is a relatively new activity, but it has already become a HUGE market. In 2010, more than 38 million people gave $4.5 billion to causes online ... $4.5 BILLION dollars were donated online to fuel ideas and businesses. Chances are, you've heard of companies like Kickstarter and DonorsChoose, so instead of taking time to talk about the crowd-funding process, I can share how GiveToBenefit differs from those other platforms:

Serves Non-Profits Exclusively - GiveToBenefit works exclusively with non-profit companies. They look for non-profits who don't have the financial or human resources to do their own fundraising and who can benefit from the high-quality goods their suppliers provide.

Marketing and Strategy Assistance - GiveToBenefit actively helps the organization market the campaign. The G2B team is ready, willing and able to offer suggestions, answer questions and provide feedback throughout the process, and given the fact that many non-profits lack technology resources, they usually get very involved with each cause.

No Additional Donor Fees - An extremely important note to point out is that GiveToBenefit does not charge donors a fee for their contribution beyond the mandatory fee charged by the credit card processor. More of every the donated dollar goes to its intended cause. Your entire donation goes to the non-profit for a very specific reason. There's no question about whether your donation will go to what you hope for.

Building Connections with High-Quality Suppliers - GiveToBenefit found a way to elevate the role of the supplier of the goods that non-profits receive and use. Brands whose products promise to perform better and last longer than the items the charities have access to are featured. GiveToBenefit derives revenue from its relationships with these suppliers, and G2B uses part of the fee it charges the supplier to fund the marketing of the non-profit's online campaign.

The idea is to go beyond "doing good," to "doing better." I could go on and on about the innovate ways they're "discovering better ways to do good," but the best way to show off their platform would be to send you to the three campaigns they recently launched:

GiveToBenefit Campaigns

Whether you want to contribute to purchasing a Watermark water purification system for the Margaret E. Moul Home for people with neuromuscular disabilities or you want to fill the People's Light & Theatre and Plays & Players Theater with the beautiful sounds of Hailun Pianos, you can contribute and know that your donation is making a difference for some very worthy non-profits.

If you'd like to learn more about GiveToBenefit or if you think one of your favorite non-profits could benefit from a G2B campaign, let me know (jkrammes@softlayer.com), and I'll introduce you to G2B founder and visionary Dan Sossaman.

-@JoshuaKrammes

December 6, 2012

MongoDB: Architectural Best Practices

With the launch of our MongoDB solutions, developers can provision powerful, optimized, horizontally scaling NoSQL database clusters in real-time on bare metal infrastructure in SoftLayer data centers around the world. We worked tirelessly with our friends at 10gen — the creators of MongoDB — to build and tweak hardware and software configurations that enable peak MongoDB performance, and the resulting platform is pretty amazing. As Duke mentioned in his blog post, those efforts followed 10Gen's MongoDB best practices, but what he didn't mention was that we created some architectural best practices of our own for MongoDB in deployments on our platform.

The MongoDB engineered servers that you order from SoftLayer already implement several of the recommendations you'll see below, and I'll note which have been incorporated as we go through them. Given the scope of the topic, it's probably easiest to break down this guide into a few sections to make it a little more digestible. Let's take a look at the architectural best practices of running MongoDB through the phases of the roll-out process: Selecting a deployment strategy to prepare for your MongoDB installation, the installation itself, and the operational considerations of running it in production.

Deployment Strategy

When planning your MongoDB deployment, you should follow Sun Tzu's (modified) advice: "If you know the [friend] and know yourself, you need not fear the result of a hundred battles." "Friend" was substituted for the "enemy" in this advice because the other party is MongoDB. If you aren't familiar with MongoDB, the top of your to-do list should be to read MongoDB's official documentation. That information will give you the background you'll need as you build and use your database. When you feel comfortable with what MongoDB is all about, it's time to "know yourself."

Your most important consideration will be the current and anticipated sizes of your data set. Understanding the volume of data you'll need to accommodate will be the primary driver for your choice of individual physical nodes as well as your sharding plans. Once you've established an expected size of your data set, you need to consider the importance of your data and how tolerant you are of the possibility of lost or lagging data (especially in replicated scenarios). With this information in hand, you can plan and start testing your deployment strategy.

It sounds a little strange to hear that you should test a deployment strategy, but when it comes to big data, you want to make sure your databases start with a strong foundation. You should perform load testing scenarios on a potential deployment strategy to confirm that a given architecture will meet your needs, and there are a few specific areas that you should consider:

Memory Sizing
MongoDB (like many data-oriented applications) works best when the data set can reside in memory. Nothing performs better than a MongoDB instance that does not require disk I/O. Whenever possible, select a platform that has more available RAM than your working data set size. If your data set exceeds the available RAM for a single node, then consider using sharding to increase the amount of available RAM in a cluster to accommodate the larger data set. This will maximize the overall performance of your deployment. If you notice page faults when you put your database under production load, they may indicate that you are exceeding the available RAM in your deployment.

Disk Type
If speed is not your primary concern or if you have a data set that is far larger than any available in memory strategy can support, selecting the proper disk type for your deployment is important. IOPS will be key in selecting your disk type and obviously the higher the IOPS the better the performance of MongoDB. Local disks should be used whenever possible (as network storage can cause high latency and poor performance for your deployment). It's also advised that you use RAID 10 when creating disk arrays.

To give you an idea of what kind of IOPS to expect from a given type of drive, these are the approximate ranges of IOPS per drive in SoftLayer MongoDB engineered servers:

SATA II – 100-200 IOPS
15K SAS – 300-400 IOPS
SSD – 7,000-8,000 IOPS (read) 19,000-20,000 IOPS (write)

CPU
Clock speed and the amount of available processors becomes a consideration if you anticipate using MapReduce. It has also been noted that when running a MongoDB instance with the majority of the data in memory, clock speed can have a major impact on overall performance. If you are planning to use MapReduce or you're able to operate with a majority of your data in memory, consider a deployment strategy that includes a CPU with a high clock/bus speed to maximize your operations per second.

Replication
Replication provides high availability of your data if a node fails in your cluster. It should be standard to replicate with at least three nodes in any MongoDB deployment. The most common configuration for replication with three nodes is a 2x1 deployment — having two primary nodes in a single data center with a backup server in a secondary data center:

MongoDB Replication

Sharding
If you anticipate a large, active data set, you should deploy a sharded MongoDB deployment. Sharding allows you to partition a single data set across multiple nodes. You can allow MongoDB to automatically distribute the data across nodes in the cluster or you may elect to define a shard key and create range-based sharding for that key.

Sharding may also help write performance, so you can also elect to shard even if your data set is small but requires a high amount of updates or inserts. It's important to note that when you deploy a sharded set, MongoDB will require three (and only three) config server instances which are specialized Mongo runtimes to track the current shard configuration. Loss of one of these nodes will cause the cluster to go into a read-only mode (for the configuration only) and will require that all nodes be brought back online before any configuration changes can be made.

Write Safety Mode
There are several write safety modes that govern how MongoDB will handle the persistence of the data to disk. It is important to consider which mode best fits your needs for both data integrity and performance. The following write safety modes are available:

None – This mode provides a deferred writing strategy that is non-blocking. This will allow for high performance, however there is a small opportunity in the case of a node failing that data can be lost. There is also the possibility that data written to one node in a cluster will not be immediately available on all nodes in that cluster for read consistency. The 'None' strategy will also not provide any sort of protection in the case of network failures. That lack of protection makes this mode highly unreliable and should only be used when performance is a priority and data integrity is not a concern.

Normal – This is the default for MongoDB if you do not select any other mode. It provides a deferred writing strategy that is non-blocking. This will allow for high performance, however there is a small opportunity in the case of a node failing that data can be lost. There is also the possibility that data written to one node in a cluster will not be immediately available on all nodes in that cluster for read consistency.

Safe – This mode will block until MongoDB has acknowledged that it has received the write request but will not block until the write is actually performed. This provides a better level of data integrity and will ensure that read consistency is achieved within a cluster.

Journal Safe – Journals provide a recovery option for MongoDB. Using this mode will ensure that the data has been acknowledged and a Journal update has been performed before returning.

Fsync - This mode provides the highest level of data integrity and blocks until a physical write of the data has occurred. This comes with a degradation in performance and should be used only if data integrity is the primary concern for your application.

Testing the Deployment
Once you've determined your deployment strategy, test it with a data set similar to your production data. 10gen has several tools to help you with load testing your deployment, and the console has a tool named 'benchrun' which can execute operations from within a JavaScript test harness. These tools will return operation information as well as latency numbers for each of those operations. If you require more detailed information about the MongoDB instance, consider using the mongostat command or MongoDB Monitoring Service (MMS) to monitor your deployment during the testing.

Installation

When performing the installation of MongoDB, a few considerations can help create both a stable and performance-oriented solution. 10gen recommends the use CentOS (64-bit) as the base operating system if at all possible. If you try installing MongoDB on a 32-bit operating system, you might run into file size limits that cause issues, and if you feel the urge to install it on Windows, you'll see performance issues if virtual memory begins to be utilized by the OS to make up for a lack of RAM in your deployment. As a result, 32-bit operating systems and Windows operating systems should be avoided on MongoDB servers. SoftLayer provisions CentOS 6.X 64-bit operating systems by default on all of our MongoDB engineered server deployments.

When you've got CentOS 64-bit installed, you should also make the following changes to maximize your performance (all of which are included by default on all SoftLayer engineered servers):

Set SSD Read Ahead Defaults to 16 Blocks - SSD drives have excellent seek times allowing for shrinking the Read Ahead to 16 blocks. Spinning disks might require slight buffering so these have been set to 32 blocks.

noatime - Adding the noatime option eliminates the need for the system to make writes to the file system for files which are simply being read — or in other words: Faster file access and less disk wear.

Turn NUMA Off in BIOS - Linux, NUMA and MongoDB tend not to work well together. If you are running MongoDB on NUMA hardware, we recommend turning it off (running with an interleave memory policy). If you don't, problems will manifest in strange ways like massive slow downs for periods of time or high system CPU time.

Set ulimit - We have set the ulimit to 64000 for open files and 32000 for user processes to prevent failures due to a loss of available file handles or user processes.

Use ext4 - We have selected ext4 over ext3. We found ext3 to be very slow in allocating files (or removing them). Additionally, access within large files is poor with ext3.

One last tip on installation: Make the Journal and Data volumes be distinct physical volumes. If the Journal and Data directories reside on a single physical volume, flushes to the Journal will interrupt the access of data and provide spikes of high latency within your MongoDB deployment.

Operations

Once a MongoDB deployment has been promoted to production, there are a few recommendations for monitoring and optimizing performance. You should always have the MMS agent running on all MongoDB instances to help monitor the health and performance of your deployment. Additionally, this tool is also very useful if you have 10gen MongoDB Cloud Subscriptions because it provides useful debugging data for the 10gen team during support interactions. In addition to MMS, you can use the mongostat command (mentioned in the deployment section) to see runtime information about the performance of a MongoDB node. If either of these tools flags performance issues, sharding or indexing are first-line options to resolve them:

Indexes - Indexes should be created for a MongoDB deployment if monitoring tools indicate that field based queries are performing poorly. Always use indexes when you are querying data based on distinct fields to help boost performance.

Sharding - Sharding can be leveraged when the overall performance of the node is suffering because of a large operating data set. Be sure to shard before you get in the red; the system only splits chunks for sharding on insert or update so if you wait too long to shard you may have some uneven distribution for a period of time or forever depending on your data set and sharding key strategy.

I know it seems like we've covered a lot over the course of this blog post, but this list of best practices is far from exhaustive. If you want to learn more, the MongoDB forums are a great resource to connect with the rest of the MongoDB community and learn from their experiences, and the documentation on MongoDB's site is another phenomenal resource. The best people to talk to when it comes to questions about MongoDB are the folks at 10gen, so I also highly recommend taking advantage of MongoDB Cloud Subscriptions to get their direct support for your one-off questions and issues.

-Harold

December 5, 2012

Breaking Down 'Big Data' - Database Models

Forester defines big data as "techniques and technologies that make capturing value from data at an extreme scale economical." Gartner says, "Big data is the term adopted by the market to describe extreme information management and processing issues which exceed the capability of traditional information technology along one or multiple dimensions to support the use of the information assets." Big data demands extreme horizontal scale that traditional IT management can't handle, and it's not a challenge exclusive to the Facebooks, Twitters and Tumblrs of the world ... Just look at the Google search volume for "big data" over the past eight years:

Big Data Search Interest

Developers are collectively facing information overload. As storage has become more and more affordable, it's easier to justify collecting and saving more data. Users are more comfortable with creating and sharing content, and we're able to track, log and index metrics and activity that previously would have been deleted in consideration of space restraints or cost. As the information age progresses, we are collecting more and more data at an ever-accelerating pace, and we're sharing that data at an incredible rate.

To understand the different facets of this increased usage and demand, Gartner came up with the three V's of big data that vary significantly from traditional data requirements: Volume, Velocity and Variety. Larger, more abundant pieces of data ("Volume") are coming at a much faster speed ("Velocity") in formats like media and walls of text that don't easily fit into a column-and-row database structure ("Variety"). Given those equally important factors, many of the biggest players in the IT world have been hard at work to create solutions that provide the scale and speed developers need when they build social, analytics, gaming, financial or medical apps with large data sets.

When we talk about scaling databases here, we're talking about scaling horizontally across multiple servers rather than scaling vertically by upgrading a single server — adding more RAM, increasing HDD capacity, etc. It's important to make that distinction because it leads to a unique challenge shared by all distributed computer systems: The CAP Theorem. According to the CAP theorem, a distributed storage system must choose to sacrifice either consistency (that everyone sees the same data) or availability (that you can always read/write) while having partition tolerance (where the system continues to operate despite arbitrary message loss or failure of part of the system occurs).

Let's take a look at a few of the most common database models, what their strengths are, and how they handle the CAP theorem compromise of consistency v. availability:

Relational Databases

What They Do: Stores data in rows/columns. Parent-child records can be joined remotely on the server. Provides speed over scale. Some capacity for vertical scaling, poor capacity for horizontal scaling. This type of database is where most people start.
Horizontal Scaling: In a relational database system, horizontal scaling is possible via replication — dharing data between redundant nodes to ensure consistency — and some people have success sharding — horizontal partitioning of data — but those techniques add a lot of complexity.
CAP Balance: Prefer consistency over availability.
When to use: When you have highly structured data, and you know what you'll be storing. Great when production queries will be predictable.
Example Products: Oracle, SQLite, PostgreSQL, MySQL

Document-Oriented Databases

What They Do: Stores data in documents. Parent-child records can be stored in the same document and returned in a single fetch operation with no join. The server is aware of the fields stored within a document, can query on them, and return their properties selectively.
Horizontal Scaling: Horizontal scaling is provided via replication, or replication + sharding. Document-oriented databases also usually support relatively low-performance MapReduce for ad-hoc querying.
CAP Balance: Generally prefer consistency over availability
When to Use: When your concept of a "record" has relatively bounded growth, and can store all of its related properties in a single doc.
Example Products: MongoDB, CouchDB, BigCouch, Cloudant

Key-Value Stores

What They Do: Stores an arbitrary value at a key. Most can perform simple operations on a single value. Typically, each property of a record must be fetched in multiple trips, with Redis being an exception. Very simple, and very fast.
Horizontal Scaling: Horizontal scale is provided via sharding.
CAP Balance: Generally prefer consistency over availability.
When to Use: Very simple schemas, caching of upstream query results, or extreme speed scenarios (like real-time counters)
Example Products: CouchBase, Redis, PostgreSQL HStore, LevelDB

BigTable-Inspired Databases

What They Do: Data put into column-oriented stores inspired by Google's BigTable paper. It has tunable CAP parameters, and can be adjusted to prefer either consistency or availability. Both are sort of operationally intensive.
Horizontal Scaling: Good speed and very wide horizontal scale capabilities.
CAP Balance: Prefer consistency over availability
When to Use: When you need consistency and write performance that scales past the capabilities of a single machine. Hbase in particular has been used with around 1,000 nodes in production.
Example Products: Hbase, Cassandra (inspired by both BigTable and Dynamo)

Dynamo-Inspired Databases

What They Do: Distributed key/value stores inspired by Amazon's Dynamo paper. A key written to a dynamo ring is persisted in several nodes at once before a successful write is reported. Riak also provides a native MapReduce implementation.
Horizontal Scaling: Dynamo-inspired databases usually provide for the best scale and extremely strong data durability.
CAP Balance: Prefer availability over consistency,
When to Use: When the system must always be available for writes and effectively cannot lose data.
Example Products: Cassandra, Riak, BigCouch

Each of the database models has strengths and weaknesses, and there are huge communities that support each of the open source examples I gave in each model. If your database is a bottleneck or you're not getting the flexibility and scalability you need to handle your application's volume, velocity and variety of data, start looking at some of these "big data" solutions.

Tried any of the above models and have feedback that differs from ours? Leave a comment below and tell us about it!

-@marcalanjones

December 4, 2012

Big Data at SoftLayer: MongoDB

In one day, Facebook's databases ingest more than 500 terabytes of data, Twitter processes 500 million Tweets and Tumblr users publish more than 75 million posts. With such an unprecedented volume of information, developers face significant challenges when it comes to building an application's architecture and choosing its infrastructure. As a result, demand has exploded for "big data" solutions — resources that make it possible to process, store, analyze, search and deliver data from large, complex data sets. In light of that demand, SoftLayer has been working in strategic partnership with 10gen — the creators of MongoDB — to develop a high-performance, on-demand, big data solution. Today, we're excited to announce the launch of specialized MongoDB servers at SoftLayer.

If you've configured an infrastructure to accommodate big data, you know how much of a pain it can be: You choose your hardware, you configure it to run NoSQL, you install an open source NoSQL project that you think will meet your needs, and you keep tweaking your environment to optimize its performance. Assuming you have the resources (and patience) to get everything running efficiently, you'll wind up with the horizontally scalable database infrastructure you need to handle the volume of content you and your users create and consume. SoftLayer and 10gen are making that process a whole lot easier.

Our new MongoDB solutions take the time and guesswork out of configuring a big data environment. We give you an easy-to-use system for designing and ordering everything you need. You can start with a single server or roll out multiple servers in a single replica set across multiple data centers, and in under two hours, an optimized MongoDB environment is provisioned and ready to be used. I stress that it's an "optimized" environment because that's been our key focus. We collaborated with 10gen engineers on hardware and software configurations that provide the most robust performance for MongoDB, and we incorporated many of their MongoDB best practices. The resulting "engineered servers" are big data powerhouses:

MongoDB Configs

From each engineered server base configuration, you can customize your MongoDB server to meet your application's needs, and as you choose your upgrades from the base configuration, you'll see the thresholds at which you should consider upgrading other components. As your data set's size and the number of indexes in your database increase, you'll need additional RAM, CPU, and storage resources, but you won't need them in the same proportions — certain components become bottlenecks before others. Sure, you could upgrade all of the components in a given database server at the same rate, but if, say, you update everything when you only need to upgrade RAM, you'd be adding (and paying for) unnecessary CPU and storage capacity.

Using our new Solution Designer, it's very easy to graphically design a complex multi-site replica set. Once you finalize your locations and server configurations, you'll click "Order," and our automated provisioning system will kick into high gear. It deploys your server hardware, installs CentOS (with OS optimizations to provide MongoDB performance enhancements), installs MongoDB, installs MMS (MongoDB Monitoring Service) and configures the network connection on each server to cluster it with the other servers in your environment. A process that may have taken days of work and months of tweaking is completed in less than four hours. And because everything is standardized and automated, you run much less risk of human error.

MongoDB Configs

One of the other massive benefits of working so closely with 10gen is that we've been able to integrate 10gen's MongoDB Cloud Subscriptions into our offering. Customers who opt for a MongoDB Cloud Subscription get additional MongoDB features (like SSL and SNMP support) and support direct from the MongoDB authority. As an added bonus, since the 10gen team has an intimate understanding of the SoftLayer environment, they'll be able to provide even better support to SoftLayer customers!

You shouldn't have to sacrifice agility for performance, and you shouldn't have to sacrifice performance for agility. Most of the "big data" offerings in the market today are built on virtual servers that can be provisioned quickly but offer meager performance levels relative to running the same database on bare metal infrastructure. To get the performance benefits of dedicated hardware, many users have chosen to build, roll out and tweak their own configurations. With our MongoDB offering, you get the on-demand availability and flexibility of a cloud infrastructure with the raw power and full control of dedicated hardware.

If you've been toying with the idea of rolling out your own big data infrastructure, life just got a lot better for you.

-Duke

November 27, 2012

Tips and Tricks - Building a jQuery Plugin (Part 1)

I've written several blogs detailing the use of different jQuery plugins (like Select2, LazyLoad and equalHeights), and in the process, I've noticed an increasing frustration among the development community when it comes to building jQuery plugins. The resources and documentation I've found online have not as clear and easy as they could be, so in my next few posts, I'll break down the process to make jQuery plugin creation simple and straightforward. In this post, we'll cover the basic structure of a plugin and where to insert your own functionality, and in Part 2, we'll pick a simple task and add on to our already-made structure.

Before I go any further, it's probably important to address a question you might be asking yourself: "Why would I want to make my own plugin?" The best reason that comes to my mind is portability. If you've ever created a large-scale project, take a look back into your source code and note how many of the hundreds of lines of jQuery code you could put into a plugin to reuse on a different project. You probably invested a lot of time and energy into that code, so it doesn't make sense to reinvent the wheel if you ever need that functionality again. If that's not enough of a reason for you, I can also tell you that if you develop your own jQuery plugin, you'll level-up in cool points, and the jQuery community will love you.

For this post, let's create a jQuery plugin that simply returns, "This is our awesome plugin!" Our first step involves putting together the basic skeleton used by every plugin:

(function($) {
    $.fn.slPlugin = function() {
 
            // Awesome plugin stuff goes here
    };
}) (jQuery);

This is your template — your starting point. Practice it. Remember it. Love it. The "slPlugin" piece is what I chose to name this plugin. It's best to name your plugin something unique ... I always run a quick Google search to ensure I don't duplicate the name of a plugin I (or someone else) might need to use in a project alongside my plugin. In this case, we're calling the example plugin slPlugin because SoftLayer is awesome, and I like naming my plugins after awesome things. I'll save this code in a file called jquery.slPlugin.js.

Now that we have our plugin's skeleton, let's add some default values for variables:

(function($) {
    $.fn.slPlugin = function(options) {
            var defaults = {
                myVar: "default", // this will be the default value of this var
                anotherVar: 0,
                coolVar: "this is cool",                
            };
            var options = $.extend(defaults, options);
    };
}) (jQuery);

Let's look at the changes we made between the first example and this one. You'll notice that in our second line we added "options" to become $.fn.slPlugin = function(options) {. We do this because our function is now accepting arguments, and we need to let the function know that. The next difference you come across is the var defaults blurb. In this section, we're providing default values for our variables. If you don't define values for a given variable when you call the plugin, these default values will be used.

Now let's have our plugin return the message we want to send:

(function($) {
    $.fn.slPlugin = function(options) {
            var defaults = {
                myVar: "This is", // this will be the default value of this var
                anotherVar: "our awesome",
                coolVar: "plugin!",
            };
            var options = $.extend(defaults, options);
            this.each(function() {
                ourString = myVar + " " + anotherVar + " " + coolVar;
            });
            return ourString;
    };
}) (jQuery);

We've defined our default values for our variables, concatenated our variables and we've added a return under our variable declaration. If our jQuery plugin is included in a project and no values are provided for our variables, slPlugin will return, "This is our awesome plugin!"

It seems rather rudimentary at this point, but we have to crawl before we walk. This introductory post is laying the groundwork of coding a jQuery plugin, and we'll continue building on this example in the next installment of this series. As you've seen with the LazyLoad, equalHeights and Select2, there are much more complicated things we can do with our plugin, and we'll get there. Sneak Preview: In the next installment, we'll be creating and implementing a truncation function for our plugin ... Get excited!

-Cassandra

November 21, 2012

Risk Management: The Importance of Redundant Backups

You (should) know the importance of having regular backups of your important data, but to what extent does data need to be backed up to be safe? With a crowbar and shove, thieves broke into my apartment and stole the backups I've used for hundreds of gigabytes of home videos, photo files and archives of past computers. A Dobro RAID enclosure and an external drive used by Apple Time Machine were both stolen, and if I didn't have the originals on my laptop or a redundant offsite backup, I would have lost all of my data. My experience is not uncommon, and it's a perfect example of an often understated principle that everyone should understand: You need redundant backups.

It's pretty simple: You need to back up your data regularly. When you've set up that back up schedule, you should figure out a way to back up your data again. After you've got a couple current backups of your files, you should consider backing up your backups off-site. It seems silly to think of backing up backups, but if anything happens — failed drives, theft, fire, flood, etc. — those backups could be lost forever, and if you've ever lost a significant amount of data due to a hard drive failure or experience like mine, you know that backups are worth their weight in gold.

Admittedly, there is a point of diminishing return when it comes to how much redundancy is needed — it's not worth the time/effort/cost to back up your backups ad infinitum — so here are the best practices I've come up with over the course of my career in the information technology industry:

  • Plan and schedule regular backups to keep your archives current. If your laptop's hard drive dies, having backups from last June probably won't help you as much as backups from last night.
  • Make sure your data exists on three different mediums. It might seem unnecessary, but if you're already being intentional about backing up your information, take it one step further to replicate those backups at least one more time.
  • Something might happen to your easy onsite backups, so it's important to consider off-site backups as well. There are plenty of companies offering secure online backups for home users, and those are generally easy to use (even if they can be a little slow).
  • Check your backups regularly. Having a backup is useless if it's not configured to back up the correct data and running on the correct schedule.
  • RAID is not a backup solution. Yes, RAID can duplicate data across hard drives, but that doesn't mean the data is "backed up" ... If the RAID array fails, all of the hard drives (and all of the data) in the array fail with it.

It's important to note here that "off-site" is a pretty relative term when it comes to backups. Many SoftLayer customers back up a primary drive on their server to a secondary drive on the same server (duplicating the data away from the original drive), and while that's better than nothing, it's also a little risky because it's possible that the server could fail and corrupt both drives. Every backup product SoftLayer offers for customers is off-site relative to the server itself (though it might be in the same facility), so we also make it easy to have your backup in another city or on a different continent.

As I've mentioned already, once you set up your backups, you're not done. You need to check your backups regularly for failures and test them to confirm that you can recover your data quickly in the event of a disaster. Don't just view a file listing. Try extracting files or restore the whole backup archive. If you're able to run a full restore without the pressure of an actual emergency, it'll prove that you're ready for the unexpected ... Like a fire drill for your backups.

Setting up a backup plan doesn't have to be scary or costly. If you don't feel like you could recover quickly after losing your data, spend a little time evaluating ways to make a recovery like that easy. It's crazy, but a big part of "risk management," "disaster recovery" and "business continuity" is simply making sure your data is securely backed up regularly and available to you when you need it.

Plan, prepare, back up.

-Lyndell

November 20, 2012

Community Development: Catalysing European Startups

SoftLayer works hard and plays hard. A few weeks ago, I traveled to Dallas for the first "Global Catalyst Summit"* where the community development teams in Europe, Asia and the United States all came together under one roof to learn, strategize and bond. What that really means is that we all experienced a week of hardcore information flow and brutal fun.

The onboarding process to become a part of the SoftLayer's Community Development (Catalyst) team is pretty rigorous, and traveling to Dallas from Amsterdam for the training made it even more intense. In short order, I learned about the roots of the Catalyst program and why SoftLayer is so interested in investing in helping startups succeed. I got the low-down on the hundreds of companies that are taking advantage of the program right now, and I was inspired by the six incredible people who focus exclusively on the Catalyst program at SoftLayer ... And Big Tex:

SoftLayer Community Development Team and Big Tex

When the whirlwind week of orientation and training came to an end, I came to a solid conclusion: I am working at SoftLayer for a reason. I believe SoftLayer has the most kick-ass global on-demand technology platform out there, and our focus on innovation and automation is reflected in everything we do. On top of that, we give that platform to startups to help springboard their success. I get to work with a community of world-changers. Needless to say, that's an amazing conclusion to come to.

As a member of the Catalyst team in EMEA (Europe, Middle East, Africa), I can provide signficant resources to entrepreneurs who are building awesome new applications and technologies that are making a difference locally, regionally and globally. Anna Bofill Bert and I work out of SoftLayer's Amsterdam office, and we are fully dedicated to helping startup and developer communities in our region.

As a review exercise and a way to educate the audience that may be unfamiliar with Catalyst, I thought I'd bullet out a few of the main ideas:

What is Catalyst?

The SoftLayer Catalyst Startup Program provides:

  • A generous monthly hosting credit toward dedicated, cloud or hybrid compute environments for a FULL YEAR (Ideal for dev-ops/next generation startup compute applications who want high performance from the start).
  • Direct connection to highest level programming team at SoftLayer — Our Innovation Team. Participating companies get help and advice from the people that are writing the book on highly scalable, global infrastructure environments.
  • Connection to the SoftLayer Marketing and PR Team for help getting spreading the word around the world about all the cool stuff participating startups are doing.

We reach startups by listening to them and meeting needs that all of them express. We are telling the SoftLayer story, networking, making friends, drinking too much and travelling like mad. In the course of a month, we went to Lean Start Up Machine in Rotterdam, Structure Europe in Amsterdam, Pioneers Festival in Vienna, HowToWeb in Bucharest and we managed to complete a quick tour of startup communities in Spain.

Like our peers on the US team, we partner with incubators and accelerators to make sure that when startups look for help getting started, they also find SoftLayer. We're already working with partners like Springboard, Seedcamp, GameFounders, Startup Sauna, the INLEA Foundation and Tetuan Valley, and the list of supported communities seems to grow daily. When the portfolio companies in each of these organizations are given access to the Catalyst program, that means SoftLayer's Catalyst customer base is growing pretty phenomenally as well.

What I actually like most about how we help startups is the mentorship and office hours we provide participating companies as well. SoftLayer was founded by ten guys in a living room in 2005, and we've got hundreds of millions of dollars in annual revenue as of 2012. That success is what the SoftLayer team is excited to share insights about.

Hustling is a major part of startup culture, so it's only fitting that I feel like I had to hustle through this blog to get all of my thoughts down. Given that SoftLayer EMEA is a bit of a startup itself, I'm happy to be practicing what we preach. If you'd like more information about Catalyst or you want to apply, please feel free to hit me up: esampson@softlayer.com

We want to be part of your company's success story.

-@EmilyBlitz

*Note: As an homage to Big Tex after the fire, we referred to our meeting as the "Global Catalyst Summit with Big Tex" at the Texas State Fair. We hope to see you back in action in 2013, Big Tex!

November 19, 2012

How It's Made (and Won): The Server Challenge II

Every year, we attend more than fifty trade shows and conferences around the world. We want to spread the word about SoftLayer and connect with each conference's technical audience (also known as future SoftLayer customers). That goal is pretty straightforward on paper, but when it comes to executing on it, we're faced with the same challenge as all of our fellow exhibitors: How do we get our target audience to the our booth?

Walk down any aisle of an expo hall, and you'll see collateral and swag beckoning to attendees like a candy bar at the grocery store register. Some exhibitors rely on Twitter to monitor an event's hashtag and swoop in at every opportunity to reach the show's influential attendees. Other exhibitors might send out emails to their clients and prospects in the area to invite them to the show. We see value in each of those approaches, but what we found to be most effective was to bring a SoftLayer data center to our booth ... or at least a piece of one.

The Server Challenge has come a long way over the years. Its meager beginnings involved installing RAM and hard drive cables in a tower server. Shortly thereafter, a rack-mount server replaced the tower server, but you were still tasked with "inside the server" challenges. As we started looking for ways to tell the bigger SoftLayer story with the Server Challenge, we moved to miniature server rack, and the competition really started to pick up steam. This year, we made it our goal to take the Server Challenge to the next level, and when Supermicro stepped in to sponsor the next iteration of the the competition, we started thinking BIG.

Why use a miniature version of a SoftLayer rack when we could use a full-size version? Why have a standalone screen when rack-mount monitors can make the display part of the unit? Why rely on speakers behind the booth to pump "Eye of the Tiger" while attendees are competing when we could easily build those into the next version of the challenge? What was initially intended to be a "tweak" of the first Server Challenge became a complete overhaul ... Hence the new "Server Challenge II" moniker.

Harkening back to the 8-bit glory days of Pac Man and Space Invaders, the Server Challenge II uses a full-size 42U server rack with vintage arcade-style branding, a built-in timer and speakers that blast esoteric video game music. The bread and butter of the challenge is the actual server hardware, though ... Supermicro provided two new 2U servers to replace the previous version's five 1U servers, and we installed the same Cisco (public and private networks) and SMC (out-of-band management network) switches you see in SoftLayer's pods.

Server Challenge II

We had two instances of the original Server Challenge (one in the US, one in Amsterdam), so in order for the Server Challenge II to be bigger and better, we had to increase that total to five — one instance in Europe, one in Asia and three in the United States. Things might get a little crazier logistically, but as a potential conference attendee, it means you're even more likely to encounter the Server Challenge II if you attend any events with us.

The Server Challenge II's Internal Debut

The first instance of the Server Challenge II made its debut at GDC Online in Austin, and we immediately knew we had a hit. By the time the rack got back to our office, we had to get it ready for its next destination (Cloud Expo West), but before we sent it on its way, we gave it an official internal debut ... and raised some money for the American Heart Association in the process.

Server Challenge II at SoftLayer

SLayers at the SoftLayer HQ in Dallas could pay $3 for one attempt or $5 for two attempts to reach the top of the Server Challenge II leader board. Needless to say, it was competitive. If you click on the image above, you'll notice that our fearless leader, Lance Crosby, stopped by and gave tips to (and/or heckled) a few participants. Unsurprisingly, one of our very talented Server Build Technicians — Ellijah Fleites — took home a MacBook Air and bragging rights as SoftLayer champion with a record time of 1:03.79 ... But records are made to be broken.

In Two Places at Once

Immediately after the AHA fundraiser, we crated up the rack and sent it along to Cloud Expo West in Santa Clara. A few days later, we put the finishing touches on the second Server Challenge II rack, and because we got it done quickly, we were able to get it shipped to the other side of the country for ad:tech NYC. We would finally have the competition running in two places at the exact same time!

We weren't disappointed.

On both coasts, the retro style of the Server Challenge II lured some fantastic competitors (excellent!), and started a lot of great conversations (even better!). Here are the final leader boards from the shows:

Server Challenge II
Server Challenge II

You probably notice that the times in the ad:tech leader board are a little higher than the times in the Cloud Expo leader board, and our team figured out why that was in the middle of the second day of the conference ... The way we bound the network cables differed slightly between the two instances, and we were using different switches to time the competition (one that required only one hand to activate/deactivate, the other requiring both hands). In order to have an "apples-to-apples" comparison between all of our shows, we're going to make sure everything is consistent with all of the instances, and we plan on keeping a running list of fastest overall challenge times ... and maybe even a "World Championship" one day.

Given the early success of the Server Challenge II, you can bet that it's not going anywhere any time soon. If we have multiple shows running the challenge at one time, we might even fire up a video chat where you can compete against an attendee at a completely different conference ... so be prepared.

In the next year, we'll have all five of the Server Challenge II instances in rotation across three continents, and with the popularity of the competition growing by leaps and bounds after every show, we hope by next holiday season, a home version of the Server Challenge II is at the top of every wish list on the planet. :-)

For now, though, I'll just leave you with a glimpse at the action from Cloud Expo West (click for more pictures from the show):

Cloud Expo West

-Raleigh

November 16, 2012

Going Global: Domo Arigato, Japan

I'm SoftLayer's director of international operations, so I have the unique pleasure of spending a lot of time on airplanes and in hotels as I travel between Dallas, Amsterdam, Singapore and wherever else our event schedule dictates. In the past six months, I've spent most of my time in Asia, and I've tried to take advantage of the opportunity relearn the culture to help shape SoftLayer Asia's business.

To really get a sense the geographic distance between Dallas and Singapore, find a globe and put one index finger on Dallas and put your other index finger on Singapore. To travel from one location to the other, you fly to the other side of the planet. Given the space considerations, our network map uses a scaled-down representative topology to show our points of presence in a single view, and you get a sense of how much artistic license was used when you actually make the trip to Singapore.

Global Network

The longest currently scheduled commercial flight on the planet takes you from Singapore to Newark in a cool 19 hours, but I choose to maintain my sanity rather than set world records for amount of time spent in a metal tube. I usually hop from Dallas to Tokyo (a mere 14 hours away) where I spend a few days, and I get on another plane down to Singapore.

The break between the two legs of the trip serves a few different purposes ... I get a much needed escape from the confines of an airplane, I'm able to spend time in an amazing city (where I lived 15 years ago), and I can use the opportunity to explore the market for SoftLayer. Proximity and headcount dictated that we spend most of our direct marketing and sales time focusing on the opportunities radiating from Singapore, so we haven't been able to spend as much time as we'd like in Japan. Fortunately, we've been able organically grow our efforts in the country through community-based partnerships and sponsorships, and we owe a great deal of our success to our partners in the region and our new-found friends. I've observed from our experience in Japan that the culture breeds two contrasting business realities that create challenges and opportunities for companies like SoftLayer: Japan is insular and Japan is global.

When I say that Japan is insular, I mean that IT purchases are generally made in the realm of either Japanese firms or foreign firms that have spent decades building reputation in market. Becoming a trusted part of that market is a time-consuming (and expensive) endeavor, and it's easy for a business to be dissuaded as an outsider. The contrasting reality that Japanese businesses also have a huge need for global reach is where SoftLayer can make an immediate impact.

Consider the Japanese electronics and the automobile industries. Both were built internally before making the leap to other geographies, and over the course of decades, they have established successful brands worldwide. Japanese gaming companies, social media companies and vibrant start-up communities follow a similar trend ... only faster. The capital investment required to go global is negligible compared to their forebears because they don't need to build factories or put elaborate logistics operations in place anymore. Today, a Japanese company with a SaaS solution, a game or a social media experience can successfully share it with the world in a matter minutes or hours at minimal cost, and that's where SoftLayer is able to immediately serve the Japanese market.

The process of building the SoftLayer brand in Asia has been accelerated by the market's needs, and we don't take that for granted. We plan to continue investing in local communities and working with our partners to become a trusted and respected resource in the market, and we are grateful for the opportunities those relationships have opened for us ... Or as Styx would say, "Domo Arigato, Mr. Roboto."

-@quigleymar

Pages

Subscribe to development