Essential Tools of the Command Line Ninja

Author:Jeff Propes
Date:June 9, 2013

Follow along with me at http://self2013.grimoi.re/clininja/

Ninja!

ui/ryu.jpg

Overview

Unix philosophy

Standard Streams and Redirection

Pipes and Filters

Tools

Goals

Cover as much ground as possible

Hold questions until the end

Try not to confuse the hell out of everyone

Unix Philosophy

Do one thing and do it well

Wikipedia has a pretty good article on this at http://en.wikipedia.org/wiki/Unix_philosophy

Each tool focuses on one thing to excellence

Don't require interactive operation

Every program acts as a filter

Standard Streams

Every process has three default streams present

STDIN - Text input of data into the process, denoted as stream 0

STDOUT - Text output of data from the process, denoted as stream 1

STDERR - Additional text output for printing errors, denoted as stream 2

Redirection

You can redirect these streams to and from files or into other processes

STDIN - <access.log grep 192.18.44.48

STDOUT - lsof -p 1829 >~/lsof.1829

STDERR - strace -p 1829 2>~/strace.1829

Redirection

You can also redirect one stream to the current location of another

Use the 2>&1 directive, which says "Send STDERR to wherever STDOUT is current going"

if grep -q 'found' /tmp/slabsearch >/dev/null 2>&1

This redirects STDOUT to the bit bucket, then redirects STDERR to the same place

Redirection Gotchas

if grep -q 'found' /tmp/slabsearch 2>&1 >/dev/null

Redirects STDERR to the current location of STDOUT (which is printing to the screen), then STDOUT to the bit bucket

Likely, the above is not what you intended

Pipes and Filters

Pipe - Direct the STDOUT of one program to the STDIN of another

ps -ef |grep php

This allows us to chain commands together and create useful filters

You can do this many times, filtering the output at each step

Filters

A well-heeled command line tool can be used as a filter

Accepts input from STDIN

OR accepts a list of files as arguments, which it will open each one and cat them together for input

Prints its results to STDOUT unless you instruct otherwise with an option

Prints errors to STDERR

Filters

<access.log grep index.php |awk '($8 == "200") {print $1}' | xargs host

Output of grep feeds into awk

Awk does useful work, then prints its results out to xargs

Xargs is the last in the chain

Tools

screen

grep

awk

find & xargs

perl

Fin

Thanks for coming!

ui/ryu.jpg