Posts Tagged ‘commands’

I recently had the opportunity to get re-acquainted with Redis while evaluating solutions for a project on the Product Innovation team here at SoftLayer. I’d actually played with it a couple of times before, but this time it “clicked.” Or my brain broke. Either way, I see a lot of potential for Redis now.

No one product is a perfect fit for all of your data storage needs, of course. There are such fundamental tradeoffs to be made in designing storage architectures that you should be immediately suspicious of any product that claims to fit every need.

The best solutions tend to be products that actually embrace these tradeoffs. Redis, for instance, has sacrificed a small amount of data durability in exchange for being awesome.

What is it?

Redis is a key/value store, but describing it that way is sort of like calling a helicopter a “vehicle.” It’s a technically correct description, but it leaves out some important stuff.

You can think of it like a sophisticated older brother of Memcached. It presents a flat keyspace, and you can set those keys to string values. Another feature of Memcached is the ability to perform remote atomic operations, like “incr” and “append.” These are really handy, because you have the ability to modify remote data without fetching, and you have an assurance that you’re the only one performing that operation at that instant.

Redis takes this concept of remote commands on data and goes completely nuts with it. The database is aware of data structures like hashes, lists and sets in addition to simple string values. You can sort, union, intersect, slice and dice to your heart’s content without fetching any data. Redis is a data structure server. You can treat it like remote memory, and this has an awesome immediate benefit for a programmer: your code and brain are already optimized for these data types.

But it’s not just about making storage simpler. It’s fast, too. Crazy fast. If you make intelligent use of its data structures, it’s possible to serve a lot of traffic from relatively modest hardware. Redis 2.4 can easily handle ~50k list appends a second on my notebook. With batching, it can append 2 million items to a list on a remote host in about 1.28 seconds.

It allows the remote, atomic and performant manipulation of data structures. It took me a little while to realize exactly how useful that is.

What’s wrong with it?

Nothing. Move along.

OK, it’s a little short on durability. Redis uses memory as its primary store and periodically flushes to disk. A common configuration is to do so every second.

That sounds pretty reasonable. If a server goes down, you could lose a second of data. Keep in mind, however, how many operations Redis can perform in a second. If you’re in a high-volume environment, that could be a lot of data. It’s not for your financial transactions.

It also supports relatively limited availability options. Currently, it only supports master/slave replication. Clustering support is planned for an upcoming release. It’s looking pretty powerful, but it will take some real-world testing to know its performance impact.

These challenges should be taken into consideration, and it’s probably clear if you’re in a situation where the current tradeoffs aren’t a good fit.

In my experience, a lot of developers seriously overestimate the consequences of their application losing small amounts of data. Also consider whether or not the chance of losing a second (or less) of data genuinely represents a bigger threat to your application than any other compromises you might have made.

More Information
You can check out the slightly aging docs or browse the impressively simple source. There are probably already bindings for your language of choice as well.

-Tim

Welcome back to UNIX Sysadmin Boot Camp. You’ve had a few days to get some reps in accessing your server via SSH, so it’s about time we add some weight to your exercise by teaching you some of the tools you will be using regularly to manage your server.

As we mentioned earlier in this series, customers with control panels from cPanel and Parallels might be tempted to rely solely on those graphical interfaces. They are much more user-friendly in terms of performing routine server administration tasks, but at some point, you might need to get down and dirty on the command line. It’s almost inevitable. This is where you’ll use bash commands.

Here are some of the top 10 essential commands you should get to know and remember in bash. Click any of the commands to go to its official “manual” page.

1. man – This command provides a manual of other bash commands. Want more info on a command? Type man commandname, and you’ll get more information about “commandname” than you probably wanted to know. It’s extremely useful if you need a quick reference for a command, and it’s often much more detailed and readable than a simple --help or --h extension.
2. ls – This command lets you list results. I showed you an example of this above, but the amount of options that are available to you with this command are worth looking into. Using the “manual” command above, run man ls and check out the possibilities. For example, if you’re in /etc, running ls -l /etc will get you a slightly more detailed list. My most commonly used list command is ls -hal. Pop quiz for you (where you can test your man skills): What does the -hal mean?
3. cd – This command lets you change directories. Want to go to /etc/? cd /etc/ will take you there. Want to jump back a directory? cd .. does the trick.
4. mv – This command enables you to move files and folders. The syntax is mv originalpath/to/file newpath/to/file. Simple! There are more options that you can check out with the man command.
5. rm – This command enables you to remove a file or directory. In the same vein as the mv command, this is one of those basic commands that you just have to know. By running rm filename, you remove the “filename” file.
6. cp – This command enables you to copy files from one place to another. Want to make a backup of a file before editing it? Run cp origfile.bla origfile.bak, and you have a backup in case your edit of origfile.bla goes horrendously wrong and makes babies cry. The syntax is simply: cp /source /destination. As with the above commands, check out the manual by running man cp for more options.
7. tar – On its own, tar is a command to group a bunch of files together, uncompressed. These files can then be compressed into .gzip format. The command can be used for creating or extracting, so it may be a good idea to familiarize yourself with the parameters, as you may find yourself using it quite often. For a GUI equivalent, think 7-zip or WinRAR for Windows.
8. wget – I love the simplicity of this little command. It enables you to “get” or download a target file. Yes, there are options, but all you need is a direct link to a file, and you just pull one of these: wget urlhere. Bam! That file starts downloading. Doesn’t matter what kind of file it is, it’s downloaded.
9. top – This handy little binary will give you a live view of memory and CPU usage currently affecting your machine, and is useful for finding out where you need to optimize. It can also help you pinpoint what processes may be causing a slowdown or a load issue.
10. chmod – This little sucker is vital to make your server both secure and usable, particularly when you’re going to be serving for the public like you would with a web server. Combine good usage of permission and iptables, and you have a locked down server

When you understand how to use these tools, you can start to monitor and track what’s actually happening on your server. The more you know about your server, the more effective and efficient you can make it. In our next installment, we’ll touch on some of the most common server logs and what you can do with the information they provide.

Did I miss any of your “essential” bash commands in my top 10 list? Leave a comment below with your favorites along with a quick explanation of what they do.

-Ryan