Lesson 3 of 16 · about 15 min
Pipes and redirection
Small commands that each do one thing can be snapped together with the pipe symbol (|), so the output of one becomes the input of the next. And > saves any output into a file.
🧭 What you need first: Files and folders
A factory assembly line: one station cuts, the next sands, the next paints. No single machine does everything, but together they make a chair. And you can rearrange stations to make a table instead.
The shell works the same way. Each command is a small machine that does one job with text: sort sorts lines, uniq removes repeats, grep keeps lines that match a word, wc counts. The pipe symbol | is the conveyor belt between them.
With four tiny commands you can answer real questions, like “which website visitor caused the most errors?”, in one line. That idea, small tools snapped together, is called the Unix philosophy, and it’s from 1978.
Another way: water pipes
Water flows out of one pipe straight into the next. Text output flows out of one command straight into the next, through |.
Another way: a relay of helpers
Hand a stack of name cards to Helper 1, who sorts them and hands them on. Helper 2 removes duplicates and hands them on. Helper 3 counts them. Each helper only knows their own small job.
Build a pipeline step by step
Try each line, one at a time, to see the pipeline grow:
cat pets.txt → cat pets.txt | sort → cat pets.txt | sort | uniq -c → cat pets.txt | sort | uniq -c | sort -rn | head -n 1. Then save it: sort pets.txt | uniq > unique.txt.
Type a command below and press Enter. Try: ls
Files
🏠 ~ (home) ← you are here
- 📄 pets.txt
The real words
- Every command has three standard streams: standard input (stdin, what it reads), standard output (stdout, what it prints) and standard error (stderr, error messages).
- A pipe ( | ) connects one command’s stdout to the next command’s stdin.
- Redirection ( > and >> ):
cmd > file: write stdout into the file, replacing what was there.cmd >> file: add to the end of the file.cmd < file: use the file as stdin.
- Text-processing toolbox:
| Command | Does |
|---|---|
grep word | keep lines containing word (-i ignore case, -v the lines that DON’T match, -c count) |
sort | sort lines (-n by number, -r reverse) |
uniq | remove repeated lines that are next to each other (-c to count them); usually after sort |
wc -l | count lines (-w words) |
head -n 5 / tail -n 5 | first or last 5 lines |
cut -d " " -f 1 | keep only column 1 (columns separated by spaces) |
- The Unix philosophy: programs that do one thing well, that work together, and that use text as the universal interface.
Investigate a website log
access.log has one line per web request: who asked, and the result code. Count how many requests failed with 404, and save the 404 lines (only) into errors.txt.
Type a command below and press Enter. Try: ls
Files
🏠 ~ (home) ← you are here
- 📄 access.log
- ⬜ 4
- ⬜ alan GET /about 404
- ⬜ 200
Who had the most errors?
With ONE pipeline, find who caused the most 404 errors. New tool: cut -d " " -f 1 keeps just the first word (column) of each line.
Type a command below and press Enter. Try: ls
Files
🏠 ~ (home) ← you are here
- 📄 access.log
- ⬜ 3 alan
What's the difference between > and >>?
Got it?
Finish the lesson to color it in on your map and collect its concept cards.
Did this lesson click?