This is a guest blog from our featured Technology Partners Marketplace company, Cloudant. Cloudant enables you to build next-generation data-driven applications without having to worry about developing, managing, and scaling your data layer.
Tech Partners Marketplace: http://www.softlayer.com/marketplace/cloudant
Cloudant: Data Layer for the Big Data Era
The recipe for big data app success: Start small. Iterate fast. Grow to epic proportions.
Unfortunately, most developers' databases come up short when they try to simultaneously "iterate fast" and "grow to epic proportions" — those two steps are most often at odds. I know ... I've been there. In a recent past life, I attacked petabyte-per-second data problems as a particle physicist at the Large Hadron Collider together with my colleagues and Cloudant co-founders, Alan Hoffman and Adam Kocoloski. Here are some lessons we learned the hard way:
- Scaling a database yourself is brutally hard (both application level sharding and the master-slave model). It is harder with SQL than it is with NoSQL databases, but either way, the "scale it yourself" approach is loaded with unknowns, complications and operational expense.
- Horizontal scaling on commodity hardware is a must. We got very good at this and ended up embedding Apache CouchDB behind a horizontal scaling framework to scale arbitrarily and stay running 24x7 with a minimal operational load.
- The data layer must scale. It should be something that applications grow into, not out of.
That last point inspired Alan, Adam and me to co-found Cloudant.
What is Cloudant?
Cloudant is a scalable data layer (as a service) for Big Data apps. Built on CouchDB, JSON, and MapReduce, it lets developers focus on new features instead of the drudgery of growing or migrating databases. The Cloudant Data Layer is already big: It collects, stores, analyzes and distributes application data across a global network of secure, high-performance data centers, delivering low-latency and non-stop data access to users no matter where they're located. You get to focus on your code; we've got data scalability and availability covered for you.
Scaling Your App on Cloudant
Cloudant is designed to support fast app iteration by developers. It's based on the CouchDB NoSQL database where data is encapsulated and transferred as JSON documents. You don't need to design and redesign SQL data models or migrate databases in order to create new app features. You don't need to write object-relational mapping code either. The database resides behind an HTTP layer and provides a rich permission model, so you can access, secure and share your data via a RESTful API.
Your app is a tenant within a multi-tenant data layer that is already big and scalable. You get a URL end point for your data layer, get data in and out of it via HTTP, and we scale and secure it around the globe. Global data distribution and intelligent routing minimizes latency between your users and the data, which can add 100s of milliseconds per request (we've measured!). Additionally, Cloudant has an advanced system for prioritizing requests so that apps aren't affected by 'noisy neighbors' in a multi-tenant system. We also offer a single-tenant data layer to companies who want it — your very own white-labeled data cloud. As your data volume and IO requests rise (or fall), Cloudant scales automatically, and because your data is replicated to multiple locations, it's always available. Start small and grow to epic proportions? Check.
Other Data Management Gymnastics
The Cloudant Data Layer also makes it easy to add advanced functionality to your apps:
- Replicate data (all of it or sub-sets) to data centers, computers or even mobile devices for local processing (great for analytics) or off-line access (great for mobile users). Re-synching is automatic.
- Perform advanced analytics with built-in MapReduce and full-text indexing and search.
- Distribute your code with data — Cloudant can distribute and serve any kind of document, even HTML5 and other browser-based code, which makes it easy to scale your app and move processing from your back-end to the browser.
Why We Run on SoftLayer
Given the nature of our service, people always ask us where we have our infrastructure, and we're quick to tell them we chose SoftLayer because we're fanatical about performance. We measured latencies for different data centers run by other cloud providers, and it's no contest: SoftLayer provides the lowest and most predictable latencies. Data centers that are thousands of miles apart perform almost as if they are on the same local area network. SoftLayer's rapidly expanding global presence allows Cloudant to replicate data globally throughout North America, Europe and Asia (with plans to continue that expansion as quickly as SoftLayer can build new facilities).
The other major draw to SoftLayer was the transparency they provide about our infrastructure. If you run a data layer, IO matters! SoftLayer provisions dedicated hardware for us (rather than just virtual machines), and they actually tell us exactly what hardware we are running on, so we can tweak our systems to get the most bang for our buck.
-Michael Miller, Cloudant
These Partners have built their businesses on the SoftLayer Platform, and we're excited for them to tell their stories. New Partners will be added to the Marketplace each month, so stay tuned for many more come.