Posts Tagged 'Efficient'

January 10, 2013

Web Development - JavaScript Packaging

If you think of JavaScript as the ugly duckling of programming languages, think again! It got a bad rap in the earlier days of the web because developers knew enough just to get by but didn't really respect it like they did Java, PHP or .Net. Like other well-known and heavily used languages, JavaScript contains various data types (String, Boolean, Number, etc.), objects and functions, and it is even capable of inheritance. Unfortunately, that functionality is often overlooked, and many developers seem to implement it as an afterthought: "Oh, we need to add some neat jQuery effects over there? I'll just throw some inline JavaScript here." That kind of implementation perpetuates a stereotype that JavaScript code is unorganized and difficult to maintain, but it doesn't have to be! I'm going to show you how easy it is to maintain and organize your code base by packaging your JavaScript classes into a single file to be included with your website.

There are a few things to cover before we jump into code:

  1. JavaScript Framework - Mootools is my framework of choice, but you can use whatever JavaScript framework you'd like.
  2. Classes - Because I see JavaScript as another programming language that I respect (and is capable of object-oriented-like design), I write classes for EVERYTHING. Don't think of your JavaScript code as something you use once and throw away. Write your code to be generic enough to be reused wherever it's placed. Object-oriented design is great for this! Mootools makes object-oriented design easy to do, so this point reinforces the point above.
  3. Class Files - Just like you'd organize your PHP to contain one class per file, I do the exact same thing with JavaScript. Note: Each of the class files in the example below uses the class name appended with .js.
  4. Namespacing - I will be organizing my classes in a way that will only add a single property — PT — to the global namespace. I won't get into the details of namespacing in this blog because I'm sure you're already thinking, "The code! The code! Get on with it!" You can namespace whatever is right for your situation.

For this example, our classes will be food-themed because ... well ... I enjoy food. Let's get started by creating our base object:

/*
---
name: PT
description: The base class for all the custom classes
authors: [Philip Thompson]
provides: [PT]
...
*/
var PT = {};

We now have an empty object from which we'll build all of our classes. We'll go I will go into more details later about the comment section, but let's build our first class: PT.Ham.

/*
---
name: PT.Ham
description: The ham class
authors: [Philip Thompson]
requires: [/PT]
provides: [PT.Ham]
...
*/
 
(function() {
    PT.Ham = new Class({
        // Custom code here...
    });
}());

As I mentioned in point three (above), PT.Ham should be saved in the file named PT.Ham.js. When we create second class, PT.Pineapple, we'll store it in PT.Pineapple.js:

/*
---
name: PT.Pineapple
description: The pineapple class
authors: [Philip Thompson]
requires: [/PT]
provides: [PT.Pineapple]
...
*/
 
(function() {
    PT.Pineapple = new Class({
        // Custom code here...
    });
}());

Our final class for this example will be PT.Pizza (I'll let you guess the name of the file where PT.Pizza lives). Our PT.Pizza class will require that PT, PT.Ham and PT.Pineapple be present.

/*
---
name: PT.Pizza
description: The pizza class
authors: [Philip Thompson]
requires: [/PT, /PT.Ham, /PT.Pineapple]
provides: [PT.Pizza]
...
*/
 
(function() {
    PT.Pizza = new Class({
        // Custom code here that uses PT.Ham and PT.Pineapple...
    });
}());

Before we go any further, let's check out the comments we include above each of the classes. The comments are formatted for YAML — YAML Ain't Markup Language (you gotta love recursive acronyms). These comments allow our parser to determine how our classes are related, and they help resolve dependencies. YAML's pretty easy to learn and you only need to know a few basic features to use it. The YAML comments in this example are essential for our JavaScript package-manager — Packager. I won't go into all the details about Packager, but simply mention a few commands that we'll need to build our single JavaScript file.

In addition to the YAML comments in each of the class files, we also need to create a YAML file that will organize our code. This file — package.yml for this example — is used to load our separate JavaScript classes:

name: "PT"
description: "Provides our fancy PT classes"
authors: "[Philip Thompson]"
version: "1.0.0"
sources:
    - js/PT.js
    - js/PT.Ham.js
    - js/PT.Pineapple.js
    - js/PT.Pizza.js

package.yml shows that all of our PT* files are located in the js directory, one directory up from the package.yml file. Some of the properties in the YAML file are optional, and you can add much more detail if you'd like, but this will get the job done for our purposes.

Now we're ready to turn back to Packager to build our packaged file. Packager includes an option to use PHP, but we're just going to do it command-line. First, we need to register the new package (package.yml) we created for PT. If our JavaScript files are located in /path/to/web/directory/js, the package.yml file is in /path/to/web/directory:

./packager register /path/to/web/directory

This finds our package.yml file and registers our PT package. Now that we have our package registered, we can build it:

./packager build * > /path/to/web/directory/js/PT.all.js

The Packager sees that our PT package is registered, so it looks at each of the individual class files to build a single large file. In the comments of each of the class files, it determines if there are dependencies and warns you if any are not found.

It might seem like a lot of work when it's written out like this, but I can assure you that when you go through the process, it takes no time at all. The huge benefit of packaging our JavaScript is evident as soon as you start incorporating those JavaScript classes into your website ... Because we have built all of our class files into a single file, we don't need to include each of the individual JavaScript files into our website (much less include the inline JavaScript declarations that make you cringe). To streamline your implementation even further if you're using your JavaScript package in a production deployment, I recommend that you "minify" your code as well.

See ... Organized code is no longer just for server-side only languages. Treat your JavaScript kindly, and it will be your friend!

Happy coding!

-Philip

December 19, 2012

SoftLayer API: Streamline. Simplify.

Building an API is a bit of a balancing act. You want your API to be simple and easy to use, and you want it to be feature-rich and completely customizable. Because those two desires happen to live on opposite ends of the spectrum, every API finds a different stasis in terms of how complex and customizable they are. The SoftLayer API was designed to provide customers with granular control of every action associated with any product or service on our platform; anything you can do in our customer portal can be done via our API. That depth of functionality might be intimidating to developers looking to dive in quickly and incorporate the SoftLayer platform into their applications, so our development team has been working to streamline and simplify some of the most common API services to make them even more accessible.

SoftLayer API

To get an idea of what their efforts look like in practice, Phil posted an SLDN blog with a perfect example of how they simplified cloud computing instance (CCI) creation via the API. The traditional CCI ordering process required developers to define nineteen data points:

Hostname
Domain name
complexType
Package Id
Location Id
Quantity to order
Number of cores
Amount of RAM
Remote management options
Port speeds
Public bandwidth allotment
Primary subnet size
Disk size
Operating system
Monitoring
Notification
Response
VPN Management - Private Network
Vulnerability Assessments & Management

While each of those data points is straightforward, you still have to define nineteen of them. You have all of those options when you check out through our shopping cart, so it makes sense that you'd have them in the API, but when it comes to ordering through the API, you don't necessarily need all of those options. Our development team observed our customers' API usage patterns, and they created the slimmed-down and efficient SoftLayer_Virtual_Guest::createObject — a method that only requires seven data points:

Hostname
Domain name
Number of cores
Amount of RAM
Hourly/monthly billing
Local vs SAN disk
Operating System

Without showing you a single line of code, you see the improvement. Default values were established for options like Port speeds and Monitoring based on customer usage patterns, and as a result, developers only have to provide half the data to place a new CCI order. Because each data point might require multiple lines of code, the volume of API code required to place an order is slimmed down even more. The best part is that if you find yourself needing to modify one of the now-default options like Port speeds or Monitoring, you still can!

As the development team finds other API services and methods that can be streamlined and simplified like this one, they'll ninja new solutions to make the API even more accessible. Have you tried coding to the SoftLayer API yet? If not, what's the biggest roadblock for you? If you're already a SLAPI coder, what other methods do you use often that could be streamlined?

-@khazard

September 12, 2012

How Can I Use SoftLayer Message Queue?

One of the biggest challenges developers run into when coding large, scalable systems is automating batch processes and distributing workloads to optimize compute resource usage. More simply, intra-application and inter-system communications tend to become a bottleneck that affect the user experience, and there is no easy way to get around it. Well ... There *was* no easy way around it.

Meet SoftLayer Message Queue.

As the name would suggest, Message Queue allows you to create one or more "queues" or containers which contain "messages" — strings of text that you can assign attributes to. The queues pass along messages in first-in-first-out order, and in doing so, they allow for parallel processing of high-volume workflows.

That all sounds pretty complex and "out there," but you might be surprised to learn that you're probably using a form of message queuing right now. Message queuing allows for discrete threads or applications to share information with one another without needing to be directly integrated or even operating concurrently. That functionality is at the heart of many of the most common operating systems and applications on the market.

What does it mean in a cloud computing context? Well, Message Queue facilitates more efficient interaction between different pieces of your application or independent software systems. The easiest way demonstrate how that happens is by sharing a quick example:

Creating a Video-Sharing Site

Let's say we have a mobile application providing the ability to upload video content to your website: sharevideoswith.phil. The problem we have is that our webserver and CMS can only share videos in a specific format from a specific location on a CDN. Transcoding the videos on the mobile device before it uploads proves to be far too taxing, what with all of the games left to complete from the last Humble Bundle release. Having the videos transcoded on our webserver would require a lot of time/funds/patience/knowledge, and we don't want to add infrastructure to our deployment for transcoding app servers, so we're faced with a conundrum. A conundrum that's pretty easily answered with Message Queue and SoftLayer's (free) video transcoding service.

What We Need

  • Our Video Site
  • The SoftLayer API Transcoding Service
  • SoftLayer Object Storage
    • A "New Videos" Container
    • A "Transcoded Videos" Container with CDN Enabled
  • SoftLayer Message Queue
    • "New Videos" Queue
    • "Transcoding Jobs" Queue

The Process

  1. Your user uploads the video to sharevideoswith.phil. Your web app creates a page for the video and populates the content with a "processing" message.
  2. The web application saves the video file into the "New Vidoes" container on object storage.
  3. When the video is saved into that container, it creates a new message in the "New Videos" message queue with the video file name as the body.
  4. From here, we have two worker functions. These workers work independently of each other and can be run at any comfortable interval via cron or any scheduling agent:
Worker One: Looks for messages in the "New Videos" message queue. If a message is found, Worker One transfers the video file to the SoftLayer Transcoding Service, starts the transcoding process and creates a message in the "Transcoding Jobs" message queue with the Job ID of the newly created transcoding job. Worker One then deletes the originating message from the "New Videos" message queue to prevent the process from happening again the next time Worker One runs.

Worker Two: Looks for messages in the "Transcoding Jobs" queue. If a message is found, Worker Two checks if the transcoding job is complete. If not, it does nothing with the message, and that message is be placed back into the queue for the next Worker Two to pick up and check. When Worker Two finds a completed job, the newly-transcoded video is pushed to the "Transcoded Videos" container on object storage, and Worker Two updates the page our web app created for the video to display an embedded media player using the CDN location for our transcoded video on object storage.

Each step in the process is handled by an independent component. This allows us to scale or substitute each piece as necessary without needing to refactor the other portions. As long as each piece receives and sends the expected message, its colleague components will keep doing their jobs.

Video transcoding is a simple use-case that shows some of the capabilities of Message Queue. If you check out the Message Queue page on our website, you can see a few other examples — from online banking to real-time stock, score and weather services.

Message Queue leverages Cloudant as the highly scalable low latency data layer for storing and distributing messages, and SoftLayer customers get their first 100,000 messages free every month (with additional messages priced at $0.01 for every 10,000).

What are you waiting for? Go get started with Message Queue!

-Phil (@SoftLayerDevs)

Subscribe to efficient