IOPS — Input/Output Operations Per Second — measure computer storage in terms of the number of read and write operations it can perform in a second. IOPS are a primary concern for database environments where content is being written and queried constantly, and when we take those database environments to the extreme (big data), the importance of IOPS can’t be overstated: If you aren’t able perform database reads and writes quickly in a big data environment, it doesn’t matter how many gigabytes, terabytes or petabytes you have in your database … You won’t be able to efficiently access, add to or modify your data set.

As we worked with 10gen to create, test and tweak SoftLayer’s MongoDB engineered servers, our primary focus centered on performance. Since the performance of massively scalable databases is dictated by the read and write operations to that database’s data set, we invested significant resources into maximizing the IOPS for each engineered server … And that involved a lot more than just swapping hard drives out of servers until we found a configuration that worked best. Yes, “Disk I/O” — the amount of input/output operations a given disk can perform — plays a significant role in big data IOPS, but many other factors limit big data performance. How is performance impacted by network-attached storage? At what point will a given CPU become a bottleneck? How much RAM should included in a base configuration to accommodate the load we expect our users to put on each tier of server? Are there operating system changes that can optimize the performance of a platform like MongoDB?

The resulting engineered servers are a testament to the blood, sweat and tears that were shed in the name of creating a reliable, high-performance big data environment. And I can prove it.

Most shared virtual instances — the scalable infrastructure many users employ for big data — use network-attached storage for their platform’s storage. When data has to be queried over a network connection (rather than from a local disk), you introduce latency and more “moving parts” that have to work together. Disk I/O might be amazing on the enterprise SAN where your data lives, but because that data is not stored on-server with your processor or memory resources, performance can sporadically go from “Amazing” to “I Hate My Life” depending on network traffic. When I’ve tested the IOPS for network-attached storage from a large competitor’s virtual instances, I saw an average of around 400 IOPS per mount. It’s difficult to say whether that’s “not good enough” because every application will have different needs in terms of concurrent reads and writes, but it certainly could be better. We performed some internal testing of the IOPS for the hard drive configurations in our Medium and Large MongoDB engineered servers to give you an apples-to-apples comparison.

Before we get into the tests, here are the specs for the servers we’re using:

The numbers shown in the table below reflect the average number of IOPS we recorded with a 100% random read/write workload on each of these engineered servers. To measure these IOPS, we used a tool called fio with an 8k block size and iodepth at 128. Remembering that the virtual instance using network-attached storage was able to get 400 IOPS per mount, let’s look at how our “base” configurations perform:

Clearly, the 400 IOPS per mount results you’d see in SAN-based storage can’t hold a candle to the performance of a physical disk, regardless of whether it’s SAS or SSD. As you’d expect, the “Journal” reads and writes have roughly the same IOPS between all of the configurations because all four configurations use 2 x 64GB SSD drives in RAID1. In both configurations, SSD drives provide better Data mount read/write performance than the 15K SAS drives, and the results suggest that having more physical drives in a Data mount will provide higher average IOPS. To put that observation to the test, I maxed out the number of hard drives in both configurations (10 in the 2U MD server and 34 in the 4U LG server) and recorded the results:

It should come as no surprise that by adding more drives into the configuration, we get better IOPS, but you might be wondering why the results aren’t “betterer” when it comes to the IOPS in the SSD drive configurations. While the IOPS numbers improve going from four to ten drives in the medium engineered server and six to thirty-four drives in the large engineered server, they don’t increase as significantly as the IOPS differences in the SAS drives. This is what I meant when I explained that several factors contribute to and potentially limit IOPS performance. In this case, the limiting factor throttling the (ridiculously high) IOPS is the RAID card we are using in the servers. We’ve been working with our RAID card vendor to test a new card that will open a little more headroom for SSD IOPS, but that replacement card doesn’t provide the consistency and reliability we need for these servers (which is just as important as speed).

There are probably a dozen other observations I could point out about how each result compares with the others (and why), but I’ll stop here and open the floor for you. Do you notice anything interesting in the results? Does anything surprise you? What kind of IOPS performance have you seen from your server/cloud instance when running a tool like fio?

-Kelly

Developers are collectively facing information overload. As storage has become more and more affordable, it’s easier to justify collecting and saving more data. Users are more comfortable with creating and sharing content, and we’re able to track, log and index metrics and activity that previously would have been deleted in consideration of space restraints or cost. As the information age progresses, we are collecting more and more data at an ever-accelerating pace, and we’re sharing that data at an incredible rate.

To understand the different facets of this increased usage and demand, Gartner came up with the three V’s of big data that vary significantly from traditional data requirements: Volume, Velocity and Variety. Larger, more abundant pieces of data (“Volume”) are coming at a much faster speed (“Velocity”) in formats like media and walls of text that don’t easily fit into a column-and-row database structure (“Variety”). Given those equally important factors, many of the biggest players in the IT world have been hard at work to create solutions that provide the scale and speed developers need when they build social, analytics, gaming, financial or medical apps with large data sets.

When we talk about scaling databases here, we’re talking about scaling horizontally across multiple servers rather than scaling vertically by upgrading a single server — adding more RAM, increasing HDD capacity, etc. It’s important to make that distinction because it leads to a unique challenge shared by all distributed computer systems: The CAP Theorem. According to the CAP theorem, a distributed storage system must choose to sacrifice either consistency (that everyone sees the same data) or availability (that you can always read/write) while having partition tolerance (where the system continues to operate despite arbitrary message loss or failure of part of the system occurs).

Let’s take a look at a few of the most common database models, what their strengths are, and how they handle the CAP theorem compromise of consistency v. availability:

Relational Databases

What They Do: Stores data in rows/columns. Parent-child records can be joined remotely on the server. Provides speed over scale. Some capacity for vertical scaling, poor capacity for horizontal scaling. This type of database is where most people start.
Horizontal Scaling: In a relational database system, horizontal scaling is possible via replication — dharing data between redundant nodes to ensure consistency — and some people have success sharding — horizontal partitioning of data — but those techniques add a lot of complexity.
CAP Balance: Prefer consistency over availability.
When to use: When you have highly structured data, and you know what you’ll be storing. Great when production queries will be predictable.
Example Products: Oracle, SQLite, PostgreSQL, MySQL

Document-Oriented Databases

What They Do: Stores data in documents. Parent-child records can be stored in the same document and returned in a single fetch operation with no join. The server is aware of the fields stored within a document, can query on them, and return their properties selectively.
Horizontal Scaling: Horizontal scaling is provided via replication, or replication + sharding. Document-oriented databases also usually support relatively low-performance MapReduce for ad-hoc querying.
CAP Balance: Generally prefer consistency over availability
When to Use: When your concept of a “record” has relatively bounded growth, and can store all of its related properties in a single doc.
Example Products: MongoDB, CouchDB, BigCouch, Cloudant

Key-Value Stores

What They Do: Stores an arbitrary value at a key. Most can perform simple operations on a single value. Typically, each property of a record must be fetched in multiple trips, with Redis being an exception. Very simple, and very fast.
Horizontal Scaling: Horizontal scale is provided via sharding.
CAP Balance: Generally prefer consistency over availability.
When to Use: Very simple schemas, caching of upstream query results, or extreme speed scenarios (like real-time counters)
Example Products: CouchBase, Redis, PostgreSQL HStore, LevelDB

BigTable-Inspired Databases

What They Do: Data put into column-oriented stores inspired by Google’s BigTable paper. It has tunable CAP parameters, and can be adjusted to prefer either consistency or availability. Both are sort of operationally intensive.
Horizontal Scaling: Good speed and very wide horizontal scale capabilities.
CAP Balance: Prefer consistency over availability
When to Use: When you need consistency and write performance that scales past the capabilities of a single machine. Hbase in particular has been used with around 1,000 nodes in production.
Example Products: Hbase, Cassandra (inspired by both BigTable and Dynamo)

Dynamo-Inspired Databases

What They Do: Distributed key/value stores inspired by Amazon’s Dynamo paper. A key written to a dynamo ring is persisted in several nodes at once before a successful write is reported. Riak also provides a native MapReduce implementation.
Horizontal Scaling: Dynamo-inspired databases usually provide for the best scale and extremely strong data durability.
CAP Balance: Prefer availability over consistency,
When to Use: When the system must always be available for writes and effectively cannot lose data.
Example Products: Cassandra, Riak, BigCouch

Each of the database models has strengths and weaknesses, and there are huge communities that support each of the open source examples I gave in each model. If your database is a bottleneck or you’re not getting the flexibility and scalability you need to handle your application’s volume, velocity and variety of data, start looking at some of these “big data” solutions.

Tried any of the above models and have feedback that differs from ours? Leave a comment below and tell us about it!

-@marcalanjones

Here are a few questions to help you determine whether your data is endangered:

1. Is your data backed up?
2. How often is your data backed up?
3. How often do you test your backups?
4. Is your data backed up externally from your server?
5. Are your backups in another data center?
6. Are your backups in another city?
7. Are your backups stored with a different provider?
8. Do you have local backups?
9. Are your backups backed up?
10. How many people in your organization know where your backups are and how to restore them?
11. What’s the greatest amount of data you might lose in the event of a server crash before your next backup?
12. What is the business impact of that data being lost?
13. If your server were to crash and the hard drives were unrecoverable, how long would it take you to restore all of your data?
14. What is the business impact of your data being lost or inaccessible for the length of time you answered in the last question?

We can all agree that the idea of backups and data protection is a great one, but when it comes to investing in that idea, some folks change their tune. While each of the above questions has a “good” answer when it comes to keeping your data safe, your business might not need “good” answers to all of them for your data to be backed up sufficiently. You should understand the value of your data to your business and invest in its protection accordingly.

For example, a million-dollar business running on a single server will probably value its backups more highly than a hobbyist with a blog she contributes to once every year and a half. The million-dollar business needs more “good” answers than the hobbyist, so the business should invest more in the protection of its data than the hobbyist.

If you haven’t taken time to quantify the business impact of losing your primary data (questions 11-14), sit down with a pencil and paper and take time to thoughtfully answer those questions for your business. Are any of those answers surprising to you? Do they make you want to reevaluate your approach to backups or your investment in protecting your data?

The funny thing about backups is that you don’t need them until you NEED them, and when you NEED them, you’ll usually want to kick yourself if you don’t have them.

Don’t end up kicking yourself.

-@khazard

P.S. SoftLayer has a ton of amazing backup solutions but in the interested of making this post accessible and sharable, I won’t go crazy linking to them throughout the post. The latest product release that got me thinking about this topic was the SoftLayer Object Storage launch, and if you’re concerned about your answers to any of the above questions, object storage may be an economical way to easily get some more “good” answers.

We’re planning a pretty big public debut for the SoftLayer startup program in the coming months, but we want to start introducing you to some of the killer startup companies we already are working with. Today’s incredible business: Distil.

Distil is currently enrolled in the TechStars Cloud Accelerator program, where SoftLayer CSO George Karidis, CTO Duke Skarda, and I serve as mentors. After meeting the guys at Distil, I couldn’t wait to get them set up with us as well.

Here’s a quick insight into the company from a quick Q&A with the brains of the operation, Rami Essaid, Founder and CEO of Distil:

In my short conversation with Rami, I could hear his passion. That’s exactly what we’re looking for in companies who join the SoftLayer startup program. We can’t wait to see what the future holds for Distil.

If you enjoy reading about cool new startups, bookmark the Startups page here on the SoftLayer Blog or subscribe to the “Startups” RSS feed to meet some of the most badass startups in the world.

Calling All Startups!

Companies in our program receive mentoring, best practices advice, industry insight, and tangible resources including:

• A $1,000 per month credit for dedicated hosting, cloud hosting or any kind of hybrid hosting setup
• Advanced infrastructure help and advice
• A dedicated Senior Account Representative
• Marketing support

If you’re interested in joining our program and getting the help you deserve, shoot me an email, and we’ll help you start the application process.

-@PaulFord

The success of open-source projects like Linux, Ruby on Rails, and Android reveals the strength and diversity of having developers around the world contributing and providing feedback on code. The community becomes more than the sum of its parts, driving innovation and constant improvement. The case has been made for open source in and of itself, but a debate still rages over the developing case for businesses contributing to open source. Why would a business dedicate resources to the development of something it can’t sell?

The answer is simple and straightforward: Contributing to open source fosters a community that can inspire, create and fuel the innovation a business needs to keep providing its customers with even better products. It makes sense … Having hundreds of developers with different skills and perspectives working on a project can push that project further faster. The end result is a product that benefits the open-source community and the business world. The destiny of the community or the product cannot be defined by a single vendor or business; it’s the democratization of technology.

Open-Source Cloud Platforms
Today, there are several open-source cloud platforms vying for industry dominance. SoftLayer has always been a big proponent and supporter of open source, and we’ve been involved with the OpenStack project from the beginning. In fact, we just announced SoftLayer Object Storage, an offering based on OpenStack Object Storage (code-named Swift). We’ll provide code and support for Swift in hopes that it continues to grow and improve. The basic idea behind Swift Object Storage is to create redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. I could go on and on about object storage, but I know Marc Jones has a blog specifically about SoftLayer Object Storage being published tomorrow, and I don’t want to steal too much of his thunder.

We have to acknowledge and embrace the heterogeneous nature of IT industry. Just as you might use multiple operating systems and hypervisors, we’re plan on working with a variety of open-source cloud platforms. Right now, we’re looking into supporting initiatives like Eucalyptus, and we have our ear to the street to listen to what our customers are asking for. Our overarching goal is to provide our customers with much-needed technologies that are advancing the hosting industry, and one of the best ways to get to that end is to serve the needs of the open-source community.

As I write this blog post, I can’t help but think of it in terms of a the Lord of Rings reference: “One ring to rule them all.” The idea that “one ring” is all we need to focus on as a hosting provider just doesn’t work when it comes to the open-source community … It all comes down to enabling choice and flexibility. We’ll keep investing in innovation wherever we can, and we’ll let the market decide which ring will rule where.

What open-source projects are you working on now? How can SoftLayer get involved?

-Matt

We All Have Influence Somewhere

The social networking revolution provides the unprecedented opportunity to observe, filter and analyze conversations in real time. For marketers and anyone interested in human behavior, it’s now possible to examine the collective consciousness for insights into consumer behavior and detection and engagement with the most influential people.

Increasingly, we find that the elements that determine “influence” in online networks are the same as they are in “real life” relationships: Trust and Generosity within small close networks of friends and subject matter experts. These in turn have become the foundations for Kred, a brand new way to understand anyone’s Influence and Outreach across social media and within Communities formed around interests and affinities.

‘We All Have Influence Somewhere,’ so Kred sifts through billions of social posts from over 110 million people in real time to uncover who is most influential on any subject, keyword or hashtag. This all summarized in Kredentials, which displays anyone’s history on Twitter over the last three years with a single click, including their top communities, most used words, most clicked links and much more.

Here are just a few of the other ways Kred is an evolution of influence measurement:

Dual Scores for Influence and Outreach
Influence – scored on a 1-1000 scale – shows the likelihood that your posts provoke actions from others. Outreach demonstrates your generosity in ReTweeting and replying to others.

Community
Real influence comes from expertise and passion. Kred is calculated for everyone in Communities that naturally form around interests and affinities.

Complete Transparency
Visitors to Kred.ly can see how all of their social actions count towards their scores – and how their connections’ actions affect them as well. Those who want a more thorough accounting of their score can take advantage of our Score Audit feature.

Offline Kred
Kred is the only influence measure to integrate offline achievements with online identity. Visitors can add their accomplishments – anything from academic honors to club memberships – by sending us a PDF from the ‘Get More Kred‘ menu tab inside the Kred site. We will then hand score it and manually add points.

Kred is free for everyone at http://kred.ly and deeply integrated into Playground, PeopleBrowsr’s social analytics platform. For those who wish to build custom applications off of our datamine of 1,000 days of social data, Kred can be accessed via our Playground API, Kredentials API and through a standalone API.

Many key unique features of Kred – including score audits, privacy controls and real-time activity statements – are based on feedback from our community of friends and colleagues. What would you like to see in its next evolution?

Give Kred a try and let us know what you think via email: kred@peoplebrowsr.com or on Twitter: @kred.

- Shawn Roberts, PeopleBrowsr

No one product is a perfect fit for all of your data storage needs, of course. There are such fundamental tradeoffs to be made in designing storage architectures that you should be immediately suspicious of any product that claims to fit every need.

The best solutions tend to be products that actually embrace these tradeoffs. Redis, for instance, has sacrificed a small amount of data durability in exchange for being awesome.

What is it?

Redis is a key/value store, but describing it that way is sort of like calling a helicopter a “vehicle.” It’s a technically correct description, but it leaves out some important stuff.

You can think of it like a sophisticated older brother of Memcached. It presents a flat keyspace, and you can set those keys to string values. Another feature of Memcached is the ability to perform remote atomic operations, like “incr” and “append.” These are really handy, because you have the ability to modify remote data without fetching, and you have an assurance that you’re the only one performing that operation at that instant.

Redis takes this concept of remote commands on data and goes completely nuts with it. The database is aware of data structures like hashes, lists and sets in addition to simple string values. You can sort, union, intersect, slice and dice to your heart’s content without fetching any data. Redis is a data structure server. You can treat it like remote memory, and this has an awesome immediate benefit for a programmer: your code and brain are already optimized for these data types.

But it’s not just about making storage simpler. It’s fast, too. Crazy fast. If you make intelligent use of its data structures, it’s possible to serve a lot of traffic from relatively modest hardware. Redis 2.4 can easily handle ~50k list appends a second on my notebook. With batching, it can append 2 million items to a list on a remote host in about 1.28 seconds.

It allows the remote, atomic and performant manipulation of data structures. It took me a little while to realize exactly how useful that is.

What’s wrong with it?

Nothing. Move along.

OK, it’s a little short on durability. Redis uses memory as its primary store and periodically flushes to disk. A common configuration is to do so every second.

That sounds pretty reasonable. If a server goes down, you could lose a second of data. Keep in mind, however, how many operations Redis can perform in a second. If you’re in a high-volume environment, that could be a lot of data. It’s not for your financial transactions.

It also supports relatively limited availability options. Currently, it only supports master/slave replication. Clustering support is planned for an upcoming release. It’s looking pretty powerful, but it will take some real-world testing to know its performance impact.

These challenges should be taken into consideration, and it’s probably clear if you’re in a situation where the current tradeoffs aren’t a good fit.

In my experience, a lot of developers seriously overestimate the consequences of their application losing small amounts of data. Also consider whether or not the chance of losing a second (or less) of data genuinely represents a bigger threat to your application than any other compromises you might have made.

More Information
You can check out the slightly aging docs or browse the impressively simple source. There are probably already bindings for your language of choice as well.

-Tim

In San Francisco, one of the flagship programs we’re excited to be a part of is called PeopleBrowsr Labs, a startup accelerator geared toward technology companies in the area. As you sit in the PeopleBrowsr office, the brilliance in the air is almost palpable … Young companies doing innovative things with everything they need to be successful at their disposal. One of the fringe benefits for participants in PeopleBrowsr Labs is that they’re actually rubbing elbows with the PeopleBrowsr team as well … Which is almost worth the price of admission.

In addition to the Labs sponsorship, SoftLayer is also the infrastructure provider for PeopleBrowsr and its unbelievable data mine of information. They’ve got every tweet that’s been tweeted since early 2008, and they’ve been able to take that content and make sense of it in unique and interesting ways … And that’s why we stopped by for a visit this week. Last night, PeopleBrowsr officially launched Kred, a dynamic and innovative social influence measurement platform, to a LOT of fanfare (see: TechCrunch).

In the midst of the launch-day craziness, we grabbed Scott Milener, PeopleBrowsr SVP of business development, to have him explain a little about Kred, what differentiates it from the other social influence measurements and what it means for users interested in engaging more effectively with their social networks. Check it out:

With the clear success of the announcement, we want to send a shout out of congratulations to the PeopleBrowsr team. It looks like a phenomenal leap forward in understanding social engagement, and we know it’s only the tip of the iceberg when it comes to what we’ll see coming out of the PeopleBrowsr office in the near future.

If you feel a little jaded by the social influence measurements you’ve seen, Kred’s transparency and community-centricity should be refreshing: http://kred.ly

-@PaulFord

Don’t Tell Me … SHOW Me the Numbers

We are living in a new world of increased corporate accountability and frugality. Thanks to unpredictable markets and unscrupulous leaders, the stakes have been raised in corporate decision-making. We have entered an “ROI revolution,” where CFOs, CMOs, and CIOs are demanding detailed business cases before they will make a purchase. Questions asked of vendors by executives are, “What is the Return on Investment?” and, “What is my Total Cost of Ownership?”

Based on our research, less than 5% of companies that sell into B2B environments have an effective and efficient way of creating engaging, credible business cases for customers. Most companies that sell to businesses have developed some type of spreadsheet-based “ROI calculator” to help salespeople and customers understand the ROI. Best-in-class companies often have created multiple models – one for a “business case,” one for “a competitive comparison,” and one for “persona-based selling.” Unfortunately, these spreadsheets are not effective for several reasons:

• They are confusing and not easily modified
• They suffer from version control issues
• They don’t integrate with existing CRM and marketing automation systems
• They don’t easily enable the creation of scenario comparisons
• They can’t create professional looking reports

VisualizeROI solves these problems … and more. VisualizeROI is the first SAAS-based approach to business-case selling. We launch web-based Visualizers in days, integrate with existing systems and capture a rich set of prospect data that allows management to do analysis of customers and prospects that just isn’t possible today. Salesperson customization options are available, and salespeople can be notified when customers do their own analysis. It’s addictive.

To get an idea of the flexibility and functionality a “Visualizer” can provide, check out the Drive vs. Fly? Visualizer I mentioned in the video with Kevin. To see business examples, you can head to www.visualize-roi.com/examples/ to see a few of the Visualizers we created to feature the platform.

If you already have your ROI model and you want to make your own Visualizer, we’re ready for you: www.visualize-roi.com/create/!

-Mike Genstil, Visualize ROI

Take Control of Your Cloud Data

Online services are great, but moving your data to the cloud and moving it between cloud services is very difficult and time-consuming. Think about all the data that you have online: email, contacts, calendars, documents … What happens when you want to switch to a different provider? Maybe your company changed names or is acquiring another company or you want to move to a cheaper or better email provider. It’s really difficult to move this data, especially when you’re talking about hundreds or thousands of accounts.

I first ran into this problem about a year ago. I was doing consulting work for a client, and he asked me to move their company email to Google Apps. I found out that it’s really hard to transfer email in bulk. I’m a hacker, so it didn’t take me too long to come up with a tool that did a pretty good job at transferring the accounts one-by-one. Then I thought I could just make a product out of this tool so that other people could use it as well.

At that point, I found it wasn’t that easy.

Processing email at scale is challenging. You see problems like buggy protocol implementations, unreliable network connections and bandwidth throttling. I had to bring people to the team like our Chief Architect Carlos Cabañero, and it took us several months to come up with an scalable migration platform. The good news is that we made this platform service-agnostic, so it’s not only able to transfer email, it also transfers any type of data – we only have to write connectors to deal with various services.

At the moment, we’re focusing on email and the Google Apps suite, but we will be expanding our offering to support popular business applications like Microsoft Exchange and SharePoint, and consumer apps like Flickr and Delicious.

Vendor lock-in is a growing concern when companies move to the cloud. Our objective is to give you control of your data, so you are free to move it to another service. With MigrationBox, you are not locked in anymore.

When our customer base started to grow, we ran into scalability problems ourselves. Data migration is a bandwidth-intensive process that requires lots of RAM and computing power. Fortunately, with SoftLayer we have more raw server power and automation capabilities than we’ll ever need.

The wave of moving your data online is just getting started. The cloud is popular, but only 5% of enterprises have moved their email into the cloud so far. This is just the beginning, and email is just one service. Everything is moving to the cloud: CRM, storage, document management … Cloud migration problems are going to grow and grow over the next five years, and MigrationBox will be there to help.

-Eduardo Fernandez, MigrationBox