Essential Tools of the Command Line Ninja

Author:Jeff Propes
Date:June 9, 2013

Follow along with me at




Unix philosophy

Standard Streams and Redirection

Pipes and Filters



Cover as much ground as possible

Hold questions until the end

Try not to confuse the hell out of everyone

Unix Philosophy

Do one thing and do it well

Wikipedia has a pretty good article on this at

Each tool focuses on one thing to excellence

Don't require interactive operation

Every program acts as a filter

Standard Streams

Every process has three default streams present

STDIN - Text input of data into the process, denoted as stream 0

STDOUT - Text output of data from the process, denoted as stream 1

STDERR - Additional text output for printing errors, denoted as stream 2


You can redirect these streams to and from files or into other processes

STDIN - <access.log grep

STDOUT - lsof -p 1829 >~/lsof.1829

STDERR - strace -p 1829 2>~/strace.1829


You can also redirect one stream to the current location of another

Use the 2>&1 directive, which says "Send STDERR to wherever STDOUT is current going"

if grep -q 'found' /tmp/slabsearch >/dev/null 2>&1

This redirects STDOUT to the bit bucket, then redirects STDERR to the same place

Redirection Gotchas

if grep -q 'found' /tmp/slabsearch 2>&1 >/dev/null

Redirects STDERR to the current location of STDOUT (which is printing to the screen), then STDOUT to the bit bucket

Likely, the above is not what you intended

Pipes and Filters

Pipe - Direct the STDOUT of one program to the STDIN of another

ps -ef |grep php

This allows us to chain commands together and create useful filters

You can do this many times, filtering the output at each step


A well-heeled command line tool can be used as a filter

Accepts input from STDIN

OR accepts a list of files as arguments, which it will open each one and cat them together for input

Prints its results to STDOUT unless you instruct otherwise with an option

Prints errors to STDERR


<access.log grep index.php |awk '($8 == "200") {print $1}' | xargs host

Output of grep feeds into awk

Awk does useful work, then prints its results out to xargs

Xargs is the last in the chain





find & xargs



Thanks for coming!