Small Tools, Joined
One character — the pipe — turns a handful of simple commands into an assembly line that answers questions none of them could alone. This is the idea the whole command line is built on.
I know a few commands now, but they each feel small. ls lists, cat prints. Where does the real power come from?
From joining them. Unix is built on small tools that each do one thing well, and one character that connects them into a line. That character is the whole philosophy.
The pipe
A pipePipeThe "|" that connects two commands, sending the output of the first straight in as the input of the second., written |, takes the output of the command on its left and feeds it straight in as the input of the command on its right. No files in between, no copy-paste — an assembly line. Each tool reads what the last one made, does its one job, and passes the result on.
# how many files are in this directory?
# ls lists them, one per line; wc -l counts lines
$ ls | wc -l
42
# no "count files" command exists. we built one from two tools.
cat access.log | grep "error" | sort | uniq -c | sort -rn | head
\______________/ \___________/ \____/ \_______/ \________/ \____/
read keep only order count most top
the log error lines them duplicates common first few
- Line 1Five simple tools, each doing one thing, become a report: the most frequent errors in a log, most common first. No single command does this — the line does.
- Line 1Read it left to right as a sentence: take the log, keep errors, sort, count duplicates, sort by count descending, show the top. That is the skill.
Why build a pipeline from five tools instead of one big command that does it all?
sort sorts anything; grep filters anything; wc counts anything. A giant do-everything command would be complex, brittle, and useful for exactly one task. Small tools plus a pipe give you combinations — far more than anyone could build as separate commands.What does `cat names.txt | sort | uniq` do?
cat sends the file's lines onward, sort orders them (which puts duplicates next to each other), and uniq collapses adjacent duplicates. The result is the sorted, de-duplicated list — three tools, one question answered.Build the assembly line
Build the pipeline from the demo above. List this directory, then feed that list into the tool that counts lines — the two together count the files, though no single command does. Click the pieces in order.
Show the answer
ls | wc -l
Why the streams stay separate
Every command has three channels — the standard streamsStandard streamsThe three channels every command has: input (stdin), normal output (stdout), and errors (stderr), kept separate on purpose.: input, normal output, and a separate error channel. Keeping errors apart is what lets a pipe carry the real results onward while the complaints go to your screen instead of contaminating the next tool's input.
One command per task
- Hunting for a single command that does exactly your task
- Copying output into a file, editing it, running the next thing by hand
- Being stuck when no tool fits your exact needComposing small tools
- Reaching for the small tools you already know: grep, sort, wc, uniq, cut
- Joining them with pipes into the exact thing you need, once
- Reading a pipeline left to right as a sentence describing the workYou will never memorise a command for every task, because there is not one. You learn a dozen small tools and the pipe, and compose the rest on demand.
So the power was never in any one command. It was in joining them.
Exactly. A shell is not a big box of specific tools. It is a small box of general ones and a way to connect them. That is why it never goes out of date.
And it is how I read a mountain of logs in seconds — pipe, filter, count. The same line that helps a defender read their logs helps me read yours.
Legal
Compose and analyse text on your own systems and data freely. Running these pipelines over logs or files on systems you are not authorised to access is unauthorised access, however ordinary the individual commands are.
So the pipe turns small tools into an assembly line, and the assembly line is the command line's real power. Next: redirection — pointing what these tools produce at a file instead of the screen, and their errors somewhere else again.