When I shared MongoDB architectural best practices, I referenced a few of the significant optimizations our team worked with 10gen to incorporate into our engineered servers (cheat sheet). To illustrate the impact of these changes in MongoDB performance, we ran 10gen’s recommended benchmarking harness (freely available for download and testing of your own environment) on our three tiers of engineered servers alongside equivalent shared virtual environments commonly deployed by the MongoDB community. We’ve made a pretty big deal about the performance impact of running MongoDB on optimized bare metal infrastructure, so it’s time to put our money where our mouth is.

The Testing Environment

For each of the available SoftLayer MongoDB engineered servers, data sets of 512kb documents were preloaded onto single MongoDB instances. The data sets were created with varying size compared to available memory to allow for data sets that were both larger (2X) and smaller than available memory. Each test also ensured that the data set was altered during the test run frequently enough to prevent the queries from caching all of the data into memory.

Once the data sets were created, JMeter server instances with 4 cores and 16GB of RAM were used to drive ‘benchrun’ from the 10gen benchmarking harness. This diagram illustrates how we set up the testing environment (click for a better look):

These Jmeter servers function as the clients generating traffic on the MongoDB instances. Each client generated random query and update requests with a ratio of six queries per update (The update requests in the test were to ensure that data was not allowed to fully cache into memory and never exercise reads from disk). These tests were designed to create an extreme load on the servers from an exponentially increasing number of clients until the system resources became saturated, and we recorded the resulting performance of the MongoDB application.

At the Medium (MD) and Large (LG) engineered server tiers, performance metrics were run separately for servers using 15K SAS hard drive data mounts and servers using SSD hard drive data mounts. If you missed the post comparing the IOPS statistics between different engineered server hard drive configurations, be sure to check it out. For a better view of the results in a given graph, click the image included in the results below to see a larger version.

Test Case 1: Small MongoDB Engineered Servers vs Shared Virtual Instance

Servers

Tests Performed

Average Read Operations per Second
by Concurrent Client

Peak Read Operations per Second
by Concurrent Client

Average Write Operations per Second
by Concurrent Client

Peak Write Operations per Second
by Concurrent Client

Test Case 2: Medium MongoDB Engineered Servers vs Shared Virtual Instance

Servers (15K SAS Data Mount Comparison)

Tests Performed

Average Read Operations per Second
by Concurrent Client

Peak Read Operations per Second
by Concurrent Client

Average Write Operations per Second
by Concurrent Client

Peak Write Operations per Second
by Concurrent Client

Servers (SSD Data Mount Comparison)

Tests Performed

Average Read Operations per Second
by Concurrent Client

Peak Read Operations per Second
by Concurrent Client

Average Write Operations per Second
by Concurrent Client

Peak Write Operations per Second
by Concurrent Client

Test Case 3: Large MongoDB Engineered Servers vs Shared Virtual Instance

Servers (15K SAS Data Mount Comparison)

Tests Performed

Average Read Operations per Second
by Concurrent Client

Peak Read Operations per Second
by Concurrent Client

Average Write Operations per Second
by Concurrent Client

Peak Write Operations per Second
by Concurrent Client

Servers (SSD Data Mount Comparison)

Tests Performed

Average Read Operations per Second
by Concurrent Client

Peak Read Operations per Second
by Concurrent Client

Average Write Operations per Second
by Concurrent Client

Peak Write Operations per Second
by Concurrent Client

Impressions from Performance Testing

The results speak for themselves. Running a Mongo DB big data solution on a shared virtual environment has significant drawbacks when compared to running MongoDB on a single-tenant bare metal offering. Disk I/O is by far the most limiting resource for MongoDB, and relying on shared network-attached storage (with much lower disk I/O) makes this limitation very apparent. Beyond the average and peak statistics above, performance varied much more significantly in the virtual instance environment, so it’s not as consistent and predictable as a bare metal.

Highlights:

• When a working data set is smaller than available memory, query performance increases.
• The number of clients performing queries has an impact on query performance because more data is being actively cached at a rapid rate.
• The addition of a separate Journal Mount volume significantly improves performance. Because the Small (SM) engineered server does not include a secondary mount for Journals, whenever MongoDB began to journal, the disk I/O associated with journalling was disruptive to the query and update operations performed on the Data Mount.
• The best deployments in terms of operations per second, stability and control were the configurations with a RAID10 SSD Data Mount and a RAID1 SSD Journal Mount. These configurations are available in both our Medium and Large offerings, and I’d highly recommend them.

-Harold

If you’ve configured an infrastructure to accommodate big data, you know how much of a pain it can be: You choose your hardware, you configure it to run NoSQL, you install an open source NoSQL project that you think will meet your needs, and you keep tweaking your environment to optimize its performance. Assuming you have the resources (and patience) to get everything running efficiently, you’ll wind up with the horizontally scalable database infrastructure you need to handle the volume of content you and your users create and consume. SoftLayer and 10gen are making that process a whole lot easier.

Our new MongoDB solutions take the time and guesswork out of configuring a big data environment. We give you an easy-to-use system for designing and ordering everything you need. You can start with a single server or roll out multiple servers in a single replica set across multiple data centers, and in under two hours, an optimized MongoDB environment is provisioned and ready to be used. I stress that it’s an “optimized” environment because that’s been our key focus. We collaborated with 10gen engineers on hardware and software configurations that provide the most robust performance for MongoDB, and we incorporated many of their MongoDB best practices. The resulting “engineered servers” are big data powerhouses:

From each engineered server base configuration, you can customize your MongoDB server to meet your application’s needs, and as you choose your upgrades from the base configuration, you’ll see the thresholds at which you should consider upgrading other components. As your data set’s size and the number of indexes in your database increase, you’ll need additional RAM, CPU, and storage resources, but you won’t need them in the same proportions — certain components become bottlenecks before others. Sure, you could upgrade all of the components in a given database server at the same rate, but if, say, you update everything when you only need to upgrade RAM, you’d be adding (and paying for) unnecessary CPU and storage capacity.

Using our new Solution Designer, it’s very easy to graphically design a complex multi-site replica set. Once you finalize your locations and server configurations, you’ll click “Order,” and our automated provisioning system will kick into high gear. It deploys your server hardware, installs CentOS (with OS optimizations to provide MongoDB performance enhancements), installs MongoDB, installs MMS (MongoDB Monitoring Service) and configures the network connection on each server to cluster it with the other servers in your environment. A process that may have taken days of work and months of tweaking is completed in less than four hours. And because everything is standardized and automated, you run much less risk of human error.

One of the other massive benefits of working so closely with 10gen is that we’ve been able to integrate 10gen’s MongoDB Cloud Subscriptions into our offering. Customers who opt for a MongoDB Cloud Subscription get additional MongoDB features (like SSL and SNMP support) and support direct from the MongoDB authority. As an added bonus, since the 10gen team has an intimate understanding of the SoftLayer environment, they’ll be able to provide even better support to SoftLayer customers!

You shouldn’t have to sacrifice agility for performance, and you shouldn’t have to sacrifice performance for agility. Most of the “big data” offerings in the market today are built on virtual servers that can be provisioned quickly but offer meager performance levels relative to running the same database on bare metal infrastructure. To get the performance benefits of dedicated hardware, many users have chosen to build, roll out and tweak their own configurations. With our MongoDB offering, you get the on-demand availability and flexibility of a cloud infrastructure with the raw power and full control of dedicated hardware.

If you’ve been toying with the idea of rolling out your own big data infrastructure, life just got a lot better for you.

-Duke