Meet Memcached: A Developer's Best Friend

August 2, 2012

Whether you're new to software development or you've been a coder since the punchcard days, at some point, you've probably come across horrendous performance problems with your website or scripts. From the most advanced users — creating scripts so complex that their databases flooded with complex JOINs — to the novice users — putting SQL calls in loops — database queries can be your worst nightmare as a developer. I hate to admit it, but I've experienced some these nightmares first-hand as a result of some less-than-optimal coding practices when writing some of my own scripts. Luckily, I've learned how to use memcached to make life a little easier.

What is Memcached?

Memcached is a free and open source distributed memory object caching system that allows the developer to store any sort of data in a temporary cache for later use, so they don't have to re-query it. By using memcached, a tremendous performance load can be decreased to almost nil. One of the most noteworthy features of the system is that it doesn't cache EVERYTHING on your site/script; it only caches data that is sure to be queried often. Originally developed in 2003 by Brad Fitzpatrick to improve the site performance of LiveJournal.com, memcached has grown tremendously in popularity, with some of the worlds biggest sites — Wikipedia, Flickr, Twitter, YouTube and Craigslist — taking advantage of the functionality.

How Do I Use Memcache?

After installing the memcached library on your server (available at http://memcached.org/), it's relatively simple to get started:

<?php
  // Set up connection to Memcached
  $memcache = new Memcached();
  $memcache->connect('host', 11211) or die("Could not connect");
 
  // Connect to database here
 
  // Check the cache for your query
  $key = md5("SELECT * FROM memcached_test WHERE id=1");
  $results = $memcache->get($key);
 
  // if the data exists in the cache, get it!
  if ($results) {
      echo $results['id'];
      echo 'Got it from the cache!';
  } else {
    // data didn't exist in the cache
    $query = "SELECT * FROM memcached_test WHERE id=1");
  $results = mysql_query($query);
  $row = mysql_fetch_array($results);
  print_r($row);
 
  // though we didn't find the data this time, cache it for next time!
  $memcache->set($key, $row, TRUE, 30); 
  // Stores the result of the query for 30 seconds
  echo 'In the cache now!';
 
  }
 
?>

Querying the cache is very similar to querying any table in your database, and if that data isn't cached, you'll run a database query to get the information you're looking for, and you can add that information to the cache for the next query. If another query for the data doesn't come within 30 seconds (or whatever window you specify), memcached will clear it from the cache, and the data will be pulled from the database.

So come on developers! Support memcached and faster load times! What other tools and tricks do you use to make your applications run more efficiently?

-Cassandra

Comments

August 2nd, 2012 at 6:30pm

It would be logical to declare $query before $key, and assign $key as $key = md5($query); to save issues when you come back later, and only edit the query in one of the two places it's currently located.

We used memcached to great effect at Neowin, every time a logged in user hits the front page, we load the latest news information from the database, and display it, saving a rendered copy of that section of the page into memcached. When a guest hits the page, they get the cached copy, and no database access is required. We constantly have a number of logged in users on the front page, so the cache is never stale. It's a great solution that lets us handle a heavy amount of traffic when we have popular stories (users coming in from reddit etc are usually guests, so the extra traffic adds hardly anything to the load)

August 3rd, 2012 at 4:42am

Interestingly, MySQL 5.6.6 now includes memcached so you can mix and match queries depending on your query response time requirements, but still use the same underlying storage:

"The MySQL server now includes the widely used memcached in-memory caching system, and a plugin that allows fast NoSQL-style access to InnoDB tables through the memcached protocol. This access method avoids the overhead of SQL parsing and constructing a query optimization plan. You can store the underlying data in a single InnoDB table, or spread it across multiple tables. You can read and write data through both memcached and SQL. For example, you can do fast single-key lookups through memcached get calls, and do statistical reports across all the data through SQL."

http://dev.mysql.com/doc/refman/5.6/en/news-5-6-6.html

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

Comments

August 2nd, 2012 at 6:30pm

It would be logical to declare $query before $key, and assign $key as $key = md5($query); to save issues when you come back later, and only edit the query in one of the two places it's currently located.

We used memcached to great effect at Neowin, every time a logged in user hits the front page, we load the latest news information from the database, and display it, saving a rendered copy of that section of the page into memcached. When a guest hits the page, they get the cached copy, and no database access is required. We constantly have a number of logged in users on the front page, so the cache is never stale. It's a great solution that lets us handle a heavy amount of traffic when we have popular stories (users coming in from reddit etc are usually guests, so the extra traffic adds hardly anything to the load)

August 3rd, 2012 at 4:42am

Interestingly, MySQL 5.6.6 now includes memcached so you can mix and match queries depending on your query response time requirements, but still use the same underlying storage:

"The MySQL server now includes the widely used memcached in-memory caching system, and a plugin that allows fast NoSQL-style access to InnoDB tables through the memcached protocol. This access method avoids the overhead of SQL parsing and constructing a query optimization plan. You can store the underlying data in a single InnoDB table, or spread it across multiple tables. You can read and write data through both memcached and SQL. For example, you can do fast single-key lookups through memcached get calls, and do statistical reports across all the data through SQL."

http://dev.mysql.com/doc/refman/5.6/en/news-5-6-6.html

Leave a Reply

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <pre>, <blockcode>, <bash>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.