Posts Tagged ‘code’

Whether you’re new to software development or you’ve been a coder since the punchcard days, at some point, you’ve probably come across horrendous performance problems with your website or scripts. From the most advanced users — creating scripts so complex that their databases flooded with complex JOINs — to the novice users — putting SQL calls in loops — database queries can be your worst nightmare as a developer. I hate to admit it, but I’ve experienced some these nightmares first-hand as a result of some less-than-optimal coding practices when writing some of my own scripts. Luckily, I’ve learned how to use memcached to make life a little easier.

What is Memcached?

Memcached is a free and open source distributed memory object caching system that allows the developer to store any sort of data in a temporary cache for later use, so they don’t have to re-query it. By using memcached, a tremendous performance load can be decreased to almost nil. One of the most noteworthy features of the system is that it doesn’t cache EVERYTHING on your site/script; it only caches data that is sure to be queried often. Originally developed in 2003 by Brad Fitzpatrick to improve the site performance of LiveJournal.com, memcached has grown tremendously in popularity, with some of the worlds biggest sites — Wikipedia, Flickr, Twitter, YouTube and Craigslist — taking advantage of the functionality.

How Do I Use Memcache?

After installing the memcached library on your server (available at http://memcached.org/), it’s relatively simple to get started:

<?php
  // Set up connection to Memcached
  $memcache = new Memcached();
  $memcache->connect('host', 11211) or die("Could not connect");
 
  // Connect to database here
 
  // Check the cache for your query
  $key = md5("SELECT * FROM memcached_test WHERE id=1");
  $results = $memcache->get($key);
 
  // if the data exists in the cache, get it!
  if ($results) {
      echo $results['id'];
      echo 'Got it from the cache!';
  } else {
    // data didn't exist in the cache
    $query = "SELECT * FROM memcached_test WHERE id=1");
  $results = mysql_query($query);
  $row = mysql_fetch_array($results);
  print_r($row);
 
  // though we didn't find the data this time, cache it for next time!
  $memcache->set($key, $row, TRUE, 30); 
  // Stores the result of the query for 30 seconds
  echo 'In the cache now!';
 
  }
 
?>

Querying the cache is very similar to querying any table in your database, and if that data isn’t cached, you’ll run a database query to get the information you’re looking for, and you can add that information to the cache for the next query. If another query for the data doesn’t come within 30 seconds (or whatever window you specify), memcached will clear it from the cache, and the data will be pulled from the database.

So come on developers! Support memcached and faster load times! What other tools and tricks do you use to make your applications run more efficiently?

-Cassandra

Many of us remember when Flash was the “only” way to enhance user experience and create rich media interactivity. It was a bittersweet integration, though … Many users didn’t have the browser compatibility to use it, so some portion of your visitors were left in the dark. Until recently, that user base was relatively small — the purists who didn’t want Flash or the people whose hardware/software couldn’t support it. When Apple decided it wouldn’t enable Flash on the iPhone/iPad, web developers around the world groaned. A HUGE user base (that’s growing exponentially) couldn’t access the rich media and interactive content.

In the last year or so, Adobe released Flash Media Server to circumvent the Apple-imposed restrictions, but the larger web community has responded with a platform that will be both compatible and phenomenally functional: HTML5.

HTML5 allows us to do things we’ve never been able to do before (at least without the hassle of plugins, installations and frustration). Gone are the limitations that resigned HTML to serving as a simple framework for webpages … Now developers can push the limits of what they thought possible. As the platform has matured, some developers have even taken it upon themselves to prototype exactly where this generation of scripting is heading by creating Flash-free browser games.

Yes, you read that right: Games you can actually play on your browser, WITHOUT plugins.

From simple Pong clones that use browser windows as the paddles and ball to adventure-based Zelda-like massively multiplayer online role playing games (MMORPGs) like BrowserQuest, it’s pretty unbelievable to see the tip of the iceberg of possibilities enabled by HTML5 … Though it does seem a bit ironic to say that a Pong clone is such a great example of the potential of the HTML5 platform. Click on the screenshot below to check out BrowserQuest and tell me it doesn’t amaze you:

With an ingenious combination of CSS, JavaScript and HTML5, developers of BrowserQuest have been able to accomplish something that no one has ever seen (nor would ever even have thought possible). Developers are now able to generate dynamic content by injecting JavaScript into their HTML5 canvasses:

<code>
function handleKeyDown(evt){
	keys[evt.keyCode] = true;
}
 
function handleKeyUp(evt){
	keys[evt.keyCode] = false;
}
 
// disable vertical scrolling from arrows :)
document.onkeydown=function(){return event.keyCode!=38 && event.keyCode!=40}
</code>

Look familiar? The game-making process (not syntax!) appears eerily similar to that of any other popular language. The only difference: You don’t need to install this game … You just open your browser and enjoy.

Using a popular port of Box2D, a physics simulator, making pure browser-based games is as simple as “Make. Include. Create.” Here’s a snippit:

<code>
//Make your canvas
<canvas id="game" width="600" height="400"></canvas>  
 
//include your js physics files
 
// create your world
function createWorld() {
	// here we create our world settings for collisions
	var worldAABB = new b2AABB();
	worldAABB.minVertex.Set(-1000, -1000);
	worldAABB.maxVertex.Set(1000, 1000);
	// set gravity vector
	var gravity = new b2Vec2(0, 300);
	var doSleep = true;
	// init our world and return its value
	var world = new b2World(worldAABB, gravity, doSleep);
	return world;
}
</code>

We may be a few years away from building full-scale WoW-level MMORPGs with HTML5, but I think seeing this functionality in native HTML will be a sigh of relief to those that’ve missed out on so much Flash goodness. While developers are building out the next generation of games and apps that will use HTML5, you can keep yourself entertained (and waste hours of time) with the HTML5 port of Angry Birds!

HTML5 is not immune to some browser compatibility issues with older versions, but as it matures and becomes the standard platform for web development, we’re going to see what’s to come in our technology’s immediate future: Pure and simple compatibility for all.

-Cassandra

I hope you brought your sweat band … Today’s Boot Camp workout is going to be pretty intense. We’re focusing on our permissions muscles. Permissions in a UNIX environment cause a lot of customer issues … While everyone understands the value of secure systems and limited access, any time an “access denied” message pops up, the most common knee-jerk reaction is to enable full access to one’s files (chmod 777, as I’ll explain later). This is a BAD IDEA. Open permissions are a hacker’s dream come true. An open permission setting might have been a temporary measure, but more often than not, the permissions are left in place, and the files remain vulnerable.

To better understand how to use permissions, let’s take a step back and get a quick refresher on key components.

You’ll need to remember the three permission types:

r w x: r = read; w = write; x = execute

And the three types of access they can be applied to:

u g o: u = user; g = group; o = other

Permissions are usually displayed in one of two ways – either with letters (rwxrwxrwx) or numbers (777). When the permissions are declared with letters, you should look at it as three sets of three characters. The first set applies to the user, the second applies to the group, and the third applies to other (everyone else). If a file is readable only by the user and cannot be written to or executed by anyone, its permission level would be r--------. If it could be read by anyone but could only be writeable by the user and the group, its permission level would be rw-rw-r--.

The numeric form of chmod uses bits to represent permission levels. Read access is marked by 4 bits, write is 2, and execute is 1. When you want a file to have read and write access, you just add the permission bits: 4 + 2 = 6. When you want a file to have read, write and execute access, you’ll have 4 + 2 + 1, or 7. You’d then apply that numerical permission to a file in the same order as above: user, group, other. If we used the example from the last sentence in the previous paragraph, a file that could be read by anyone, but could only be writeable by the user and the group, would have a numeric permission level of 664 (user: 6, group: 6, other: 4).

Now the “chmod 777” I referenced above should make a little more sense: All users are given all permissions (4 + 2 + 1 = 7).

Applying Permissions

Understanding these components, applying permissions is pretty straightforward with the use of the chmod command. If you want a user (u) to write and execute a file (wx) but not read it (r), you’d use something like this:

In the above terminal image, I added the -v parameter to make it “verbose,” so it displays the related output or results of the command. The permissions set by the command are shown by the number 0300 and the series (-wx------). Nobody but the user can write or execute this file, and as of now, the user can’t even read the file. If you were curious about the leading 0 in “0300,” it simply means that you’re viewing an octal output, so for our purposes, it can be ignored entirely.

In that command, we’re removing the read permission from the user (hence the minus sign between u and r), and we’re giving the user write and execute permissions with the plus sign between u and wx. Want to alter the group or other permissions as well? It works exactly the same way: g+,g-,o+,o- … Getting the idea? chmod permissions can be set with the letter-based commands (u+r,u-w) or with their numeric equivalents (eg. 400 or 644), whichever floats your boat.

A Quick Numeric chmod Reference

chmod 777 | Gives specified file read, write and execute permissions (rwx) to ALL users
chmod 666 | Allows for read and write privileges (rw) to ALL users
chmod 555 | Gives read and execute permissions (rx) to ALL users
chmod 444 | Gives read permissions (r) to ALL users
chmod 333 | Gives write and execute permissions (wx) to ALL users
chmod 222 | Gives write privileges (w) to ALL users
chmod 111 | Gives execute privileges (x) to ALL users
chmod 000 | Last but not least, gives permissions to NO ONE (Careful!)

Get a List of File Permissions

To see what your current file permissions are in a given directory, execute the ls –l command. This returns a list of the current directory including the permissions, the group it’s in, the size and the last date the file was modified. The output of ls –l looks like this:

On the left side of that image, you’ll see the permissions in the rwx format. When the permission begins with the “d” character, it means that object is a directory. When the permission starts with a dash (-), it is a file.

Practice Deciphering Permissions

Let’s look at a few examples and work backward to apply what we’ve learned:

• Example 1: -rw-------
• Example 2: drwxr-x---
• Example 3: -rwxr-xr-x

In Example 1, the file is not a directory, the user that owns this particular object has read and write permissions, and when the group and other fields are filled with dashes, we know that their permissions are set to 0, so they have no access. In this case, only the user who owns this object can do anything with it. We’ll cover “ownership” in a future blog, but if you’re antsy to learn right now, you can turn to the all-knowing Google.

In Example 2, the permissions are set on a directory. The user has read, write and execute permissions, the group has read and execute permissions, and anything/anyone besides user or group is restricted from access.

For Example 3, put yourself to the test. What access is represented by “-rwxr-xr-x“? The answer is included at the bottom of this post.

Wrapping It Up

How was that for a crash course in Unix environment permissions? Of course there’s more to it, but this will at least make you think about what kind of access you’re granting to your files. Armed with this knowledge, you can create the most secure server environment.

Here are a few useful links you may want to peruse at your own convenience to learn more:

Linuxforums.org
Zzee.com
Comptechdoc.org
Permissions Calculator

Did I miss anything? Did I make a blatantly ridiculous mistake? Did I use “their” when I should have used “they’re”??!!… Let me know about it. Leave a comment if you’ve got anything to add, suggest, subtract, quantize, theorize, ponderize, etc. Think your useful links are better than my useful links? Throw those at me too, and we’ll toss ‘em up here.

Are you still feeling the burn from your Sysadmin Boot Camp workout? Don’t forget to keep getting reps in bash, logs, SSH, passwords and user management!

- Ryan

Now that you’re an expert when it comes to bash, logs, SSH, and passwords, you’re probably foaming at the mouth to learn some new skills. While I can’t equip you with the “nunchuck skills” or “bowhunting skills” Napoleon Dynamite reveres, I can help you learn some more important — though admittedly less exotic — user management skills in UNIX.

Root User

The root user — also known as the “super user” — has absolute control over everything on the server. Nothing is held back, nothing is restricted, and anything can be done. Only the server administrator should have this kind of access to the server, and you can see why. The root user is effectively the server’s master, and the server accordingly will acquiesce to its commands.

Broad root access should be avoided for the sake of security. If a program or service needs extensive abilities that are generally reserved for the root user, it’s best to grant those abilities on a narrow, as-needed basis.

Creating New Users

Because the Sysadmin Boot Camp series is geared toward server administration from a command-line point of view, that’s where we’ll be playing today. Tasks like user creation can be performed fairly easily in a control panel environment, but it’s always a good idea to know the down-and-dirty methods as a backup.

The useradd command is used for adding users from shell. Let’s start with an example and dissect the pieces:

useradd -c "admin" -d /home/username -g users\ -G admin,helpdesk -s\ /bin/bash userid

-c "admin" – This command adds a comment to the user we’re creating. The comment in this case is “admin,” which may be used to differentiate the user a little more clearly for better user organization.
-d /home/username – This block sets the user’s home directory. The most common approach is to replace username with the username designated at the end of the command.
-g users\ – Here, we’re setting the primary group for the user we’re creating, which will be users.
-G admin,helpdesk – This block specifies other user groups the new user may be a part of.
-s\ /bin/bash userid – This command is in two parts. It says that the new user will use /bin/bash for its shell and that userid will be the new user’s username.

Changing Passwords

Root is the only user that can change other users’ passwords. The command to do this is:

passwd userid

If you are a user and want to change your own password, you would simply issue the passwd command by itself. When you execute the command, you will be prompted for a new entry. This command can also be executed by the root user to change the root password.

Deleting Users

The command for removing users is userdel, and if we were to execute the command, it might look like this:

userdel -r username

The –r designation is your choice. If you choose to include it, the command will remove the home directory of the specified user.

Where User Information is Stored

The /etc/passwd file contains all user information. If you want to look through the file one page at a time — the way you’d use /p in Windows — you can use the more command:

more /etc/passwd

Keep in mind that most of your important configuration files are going to be located in the /etc folder, commonly spoken with an “et-see” pronunciation for short. Each line in the passwd file has information on a single user. Arguments are segmented with colons, as seen in the example below:

username:password:12345:12345::/home/username:/bin/bash

Argument 1 – username – the user’s username
Argument 2 – password – the user’s password
Argument 3 – 12345 – the user’s numeric ID
Argument 4 – 12345 – the user group’s numeric ID
Argument 5 – "" – where either a comment or the user’s full name would go
Argument 6 – /home/username – the user’s home directory
Argument 7 – /bin/bash – the user’s default console shell

Now that you’ve gotten a crash course on user management, we’ll start going deeper into group management, more detailed permissions management and the way shadow file relates to the passwd usage discussed above.

-Ryan

‘We\’ve’ . “all $een” . ‘this’ . $problem . ‘before’ . $and->it . ((1==1) ? ‘seems’ : ‘dosen\’t seem’) . sprintf(‘about time to %s things’, ‘clarify’);

PHP string handling can be a tough concept to wrangle. Developers have many options: single / double quotes, concatenation and various string manipulation functions. The choices you make have a significant impact on the readability and performance of your script. Let’s meet the line-up:

The Literal
Single quotes are used to define a string whose contents should be taken literally. What this means is that PHP will not attempt to expand any content contained between the ' '.

This is the way to tell your favorite Hypertext Preprocessor, “That little guy? Don’t worry about that little guy.”

In most cases this is the de-facto standard for strings. However, when a decent number of variables become involved it tends to become difficult to keep your quotes accounted for. When combining simple strings with variables and single quotes, the “.” operator is needed between each variable/string. That “.” is known as the concatenation operator.

Input:
$date = 'Yesterday';
$location = 'outside';
$item = array ( 'description' => 'lovely', 'name' => 'butterfly');
$content = $date . ' I went ' . $location . ' and caught a ' . $item['description'] . ' ' . $item['name'];

Output: Yesterday I went outside and caught a lovely butterfly

The Interpreted
Using double quotes will cause PHP to look a little closer into the string to find anywhere it can “read between the lines.” Variables and escape characters will be expanded, so you can reference them inline without the need for concatenation. This can be useful when creating strings which include pre-defined variables.

Input:
$file = 'example.jpg'
$content = "<a href=\"http://www.example.com/$file\">$file</a>"

Output: <a href=”http://www.example.com/example.jpg”>example.jpg</a>

In previous versions of PHP there was a significant performance difference between the use of single v. double quotes. In later versions performance variations are negligible. The decision of one over the other should focus on feature and readability concerns.

The Thoughtful
Unlike single and double quotes, the sprintf function comes to the table with a few cards up its sleeve. When provided with a formatting “template” and arguments, sprintf will return a formatted string.

Input:
$order = array ( 'item' => 'RC Helicopters', 'status' => 'pending');
$content = sprintf('Your order of %s is currently %s', $order['item'], $order['status']);

Output: Your order of RC Helicopters is currently pending

When constructing a complex string such as XML documents, sprintf allows the developer to view the string with placeholders rather than a mish-mash of escaped quotes and variables. In addition sprintf is able to specify the type of variable, change padding/text alignment, and even change the order in which it displays the variables.

The debate over the most efficient method of string definition has raged for years and will likely continue ad infinitum. However, when the benchmarks show their performance as almost identical, it leaves you with one major question: What works the best for your implementation? Typically my scripts will contain all of the methods above, and often a combination of them.

print(sprintf('The %s important thing is that %s give them all a try and see for %s', 'most', 'you', 'yourself'));

-Phil

Recently, as you know, SoftLayer released the new API version 3. We have all been working very hard on it, and we’ve been completely immersed in it for weeks (months, for some of us). This means that, for the developers, we’ve been living and breathing API code for quite some time now. The time came to release the API, and as many of you know, it was a smashing success. However, we were lacking in examples for its use. Sure, we all had examples coming out our ears since the customer portal itself uses the API, but those were written by the same developers that developed the API itself, and therefore were still written from an insider’s perspective.

So a call went out for examples. Many people jumped on the list, offering to write examples in a variety of languages. I thought I would tackle writing an API usage example in Perl. Perl, for those of you unfamiliar, is an infamous programming language. Flexible, confusing, fantastic and horrifying, it is the very embodiment of both “quick and dirty” and “elegance.” It is well loved and well loathed in equal measure by the programming community. Nevertheless, I have some experience with Perl, and I decided to give it a try.

I will attempt to describe my thought process as I developed the small applications (which you should be able to locate shortly in the SLDN documentation wiki) throughout the work day.

9am: “Wow, I really don’t remember as much Perl as I thought. This may be difficult.”
10am: “I need to install SOAP::Lite, that shouldn’t be hard.”
11am: “Where the heck are they hiding SOAP::Lite? There are articles about it everywhere, but I can’t actually find it or get it installed!”
12pm: “Ok, got SOAP::Lite installed, and my first test application works perfectly! Things are going to be ok! Wait…what’s all this about authentication headers?”
1pm: “What have I done to deserve this? Why can’t I pass my user information through to the API?”
2pm: “Aha! Another developer just wandered by and pointed out that I’ve been misspelling ‘authentication’ for 2 hours! Back on track, baby!” (Side note: another “feature” of Perl is how it never complains when you use variables that don’t exist, it just assumes you never meant to type that. Of course, you could tell it to complain, but I forgot about that feature because I haven’t used Perl in 4 years.)
3pm: I finally get example #1 working. It queries the API and shows a list of the hardware on your account.
3:30pm: Example #2 working, this shows the details for a single server, including datacenter and operating system
4pm: Combining examples #1 and #2, the third example shows all hardware on your account, plus the installed OS and datacenter, in a handy grid right on the command line. Success! I put Perl away, hopefully for another 4 years.

The whole experience, though, really gave me an insight into how fantastically awesome the API is. I was looking at it from an outsider’s perspective. I was confused as to how everything worked, I was working with an unfamiliar language, and I was browsing through the API looking for anything that looked “cool and/or useful.” Getting a list of all my account’s hardware to show up in a custom built application that I wrote as if I knew nothing about the API was a great feeling. It showed that not only was the API perfectly suited to the tasks we expected of it, but even a novice developer could, with a little effort, make an API application like mine. Expanding on it to show more and more information, and all the possibilities that it opened up in my mind made me realize how useful this API is that we made. It’s not just something that a small percentage of our customers will be using. It’s something that is truly revolutionary, and that all clients can take advantage of. I’m assuming, of course, that all clients have at least rudimentary skill in at least one programming language, but given the level of success everyone has had with our other offerings, I can assume that assumption is accurate.

If you have been thinking recently “look at all the noise they’ve been making about this ‘API’ nonsense,” I highly recommend dusting off an old programming book and at least looking at it once. Think of all the possibilities, all the custom reports that you can make for yourself, all the data that we have provided right at your fingertips to assemble in any way you wish. We try our best to make the portal useful to every customer, but we know that you can’t please all the people all the time. But with the API, we may do just that. If you’re the kind of customer that is only interested in outbound bandwidth by domain, write an API script that displays just that! If you want to know the current number of connections and CPU temperature of your load balanced servers, get that data and show it! The possibilities are endless, and we’re improving the API all the time.

-Daniel