Big Data at SoftLayer: MongoDB

December 4, 2012

In one day, Facebook's databases ingest more than 500 terabytes of data, Twitter processes 500 million Tweets and Tumblr users publish more than 75 million posts. With such an unprecedented volume of information, developers face significant challenges when it comes to building an application's architecture and choosing its infrastructure. As a result, demand has exploded for "big data" solutions — resources that make it possible to process, store, analyze, search and deliver data from large, complex data sets. In light of that demand, SoftLayer has been working in strategic partnership with 10gen — the creators of MongoDB — to develop a high-performance, on-demand, big data solution. Today, we're excited to announce the launch of specialized MongoDB servers at SoftLayer.

If you've configured an infrastructure to accommodate big data, you know how much of a pain it can be: You choose your hardware, you configure it to run NoSQL, you install an open source NoSQL project that you think will meet your needs, and you keep tweaking your environment to optimize its performance. Assuming you have the resources (and patience) to get everything running efficiently, you'll wind up with the horizontally scalable database infrastructure you need to handle the volume of content you and your users create and consume. SoftLayer and 10gen are making that process a whole lot easier.

Our new MongoDB solutions take the time and guesswork out of configuring a big data environment. We give you an easy-to-use system for designing and ordering everything you need. You can start with a single server or roll out multiple servers in a single replica set across multiple data centers, and in under two hours, an optimized MongoDB environment is provisioned and ready to be used. I stress that it's an "optimized" environment because that's been our key focus. We collaborated with 10gen engineers on hardware and software configurations that provide the most robust performance for MongoDB, and we incorporated many of their MongoDB best practices. The resulting "engineered servers" are big data powerhouses:

MongoDB Configs

From each engineered server base configuration, you can customize your MongoDB server to meet your application's needs, and as you choose your upgrades from the base configuration, you'll see the thresholds at which you should consider upgrading other components. As your data set's size and the number of indexes in your database increase, you'll need additional RAM, CPU, and storage resources, but you won't need them in the same proportions — certain components become bottlenecks before others. Sure, you could upgrade all of the components in a given database server at the same rate, but if, say, you update everything when you only need to upgrade RAM, you'd be adding (and paying for) unnecessary CPU and storage capacity.

Using our new Solution Designer, it's very easy to graphically design a complex multi-site replica set. Once you finalize your locations and server configurations, you'll click "Order," and our automated provisioning system will kick into high gear. It deploys your server hardware, installs CentOS (with OS optimizations to provide MongoDB performance enhancements), installs MongoDB, installs MMS (MongoDB Monitoring Service) and configures the network connection on each server to cluster it with the other servers in your environment. A process that may have taken days of work and months of tweaking is completed in less than four hours. And because everything is standardized and automated, you run much less risk of human error.

MongoDB Configs

One of the other massive benefits of working so closely with 10gen is that we've been able to integrate 10gen's MongoDB Cloud Subscriptions into our offering. Customers who opt for a MongoDB Cloud Subscription get additional MongoDB features (like SSL and SNMP support) and support direct from the MongoDB authority. As an added bonus, since the 10gen team has an intimate understanding of the SoftLayer environment, they'll be able to provide even better support to SoftLayer customers!

You shouldn't have to sacrifice agility for performance, and you shouldn't have to sacrifice performance for agility. Most of the "big data" offerings in the market today are built on virtual servers that can be provisioned quickly but offer meager performance levels relative to running the same database on bare metal infrastructure. To get the performance benefits of dedicated hardware, many users have chosen to build, roll out and tweak their own configurations. With our MongoDB offering, you get the on-demand availability and flexibility of a cloud infrastructure with the raw power and full control of dedicated hardware.

If you've been toying with the idea of rolling out your own big data infrastructure, life just got a lot better for you.

-Duke

Comments

May 20th, 2013 at 11:32am

How does this affect consumption of IPv4 addresses? What does Softlayer do to help my customers get ubiquitious access without these addresses?

May 20th, 2013 at 5:26pm

I'd be happy to get you answers, Greg. Can you explain your question a little more? This post announced the launch of an on-demand MongoDB big data solution at SoftLayer, so it's unclear how your question about IPv4 addresses is related.

As far as this product is concerned, each of the servers in the MongoDB environment have to communicate with each other on the back end, but that communication doesn't need happen via a public IPv4 address ... Because the solution was created to streamline the provisioning process of the infrastructure, you don't have to hassle with configuring the communication between servers.

With the dwindling pool of IPv4 addresses available worldwide, SoftLayer follows regional Internet registries' guidelines when it comes to provisioning our IPv4 address pool to new customers/servers. Because IPv4 addresses will be exhausted eventually, every server you order in a SoftLayer data center is IPv6-capable ... As IPv6 adoption grows and users begin accessing the Internet from IPv6 addresses, it's important to be able to run a dual-stack environment to provide a consistent experience for all of your customers and potential customers.

If those responses don't answer the questions you were asking, please reply again here to provide a little more detail. Thanks!

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Comments

May 20th, 2013 at 11:32am

How does this affect consumption of IPv4 addresses? What does Softlayer do to help my customers get ubiquitious access without these addresses?

May 20th, 2013 at 5:26pm

I'd be happy to get you answers, Greg. Can you explain your question a little more? This post announced the launch of an on-demand MongoDB big data solution at SoftLayer, so it's unclear how your question about IPv4 addresses is related.

As far as this product is concerned, each of the servers in the MongoDB environment have to communicate with each other on the back end, but that communication doesn't need happen via a public IPv4 address ... Because the solution was created to streamline the provisioning process of the infrastructure, you don't have to hassle with configuring the communication between servers.

With the dwindling pool of IPv4 addresses available worldwide, SoftLayer follows regional Internet registries' guidelines when it comes to provisioning our IPv4 address pool to new customers/servers. Because IPv4 addresses will be exhausted eventually, every server you order in a SoftLayer data center is IPv6-capable ... As IPv6 adoption grows and users begin accessing the Internet from IPv6 addresses, it's important to be able to run a dual-stack environment to provide a consistent experience for all of your customers and potential customers.

If those responses don't answer the questions you were asking, please reply again here to provide a little more detail. Thanks!

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.