Linux Deep Dive II
The shell, pipes, redirection, and job control as weapons
- › Understand stdin, stdout, stderr and bend them with redirection
- › Chain small tools into powerful one-liners with pipes
- › Use globbing, command substitution, and job control fluently
- › Read the environment and history for the secrets people leave behind
The shell is your interface to a compromised machine
When an exploit lands, you don't get a pretty window — you get a shell, a text prompt on someone else's computer, often a fragile one. Your effectiveness from that prompt is almost entirely a function of how well you understand the shell itself. The Unix philosophy is that each tool does one small thing, and the shell's job is to connect those tools into something larger. Learn the connectors and a dozen tiny programs become one precise instrument.
Three streams, and where they flow
Every program is born with three channels of text. Internalise them, because redirection — the act of pointing these channels somewhere new — is half of shell power.
| Stream | Number | Default | Meaning |
|---|---|---|---|
| stdin | 0 | keyboard | input coming in |
| stdout | 1 | terminal | normal output going out |
| stderr | 2 | terminal | error messages going out (separate on purpose) |
1command > out.txt # stdout INTO a file (overwrite)2command >> out.txt # stdout APPEND to a file3command 2> err.txt # stderr into a file4command > out 2>&1 # stdout to 'out', and stderr to wherever stdout goes (same file)5command 2>/dev/null # throw errors away <- you'll type this constantly6command < input.txt # feed a file in as stdinWhy `2>/dev/null` is everywhere
/dev/null is a black hole: anything written to it vanishes. Redirecting stderr there (2>/dev/null) silences the flood of "Permission denied" noise you get when, say, searching the whole filesystem as a low-privileged user — leaving only the results that matter. You saw it in the SUID hunt (find / -perm -4000 2>/dev/null); now you know exactly what the 2> and /dev/null are doing: sending stream #2, the errors, to oblivion.
Pipes: the output of one becomes the input of the next
A pipe (|) connects one program's stdout directly to the next program's stdin, no temporary file needed. This is the heart of the shell. A handful of filter tools, piped together, will do things you'd otherwise write a script for:
| Tool | One-line job |
|---|---|
| grep | keep only lines matching a pattern |
| cut | slice out columns/fields |
| sort / uniq | order lines; collapse or count duplicates |
| wc | count lines / words / bytes |
| awk | field-aware text processing (a tiny language) |
| sed | stream editing: find-and-replace on the fly |
| tr | translate or delete characters |
1# every shell-having user account, alphabetised:2cat /etc/passwd | grep -v nologin | cut -d: -f1 | sort3 4# the 10 most-common IPs hitting a web log:5cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head6 7# does this host mention a password anywhere in its configs?8grep -ri password /etc 2>/dev/nullReading the first one
cat /etc/passwd emits every account line. grep -v nologin drops the ones with nologin as their shell (system accounts you can't log in as). cut -d: -f1 splits each line on : and keeps field 1 (the username). sort alphabetises. Four trivial tools, one useful answer — *which humans can actually log in here* — and you built it live at the prompt. That composability is the skill.
Globbing and command substitution
The shell expands special characters *before* running a command. * matches any run of characters, ? matches one, [abc] matches a set. So ls *.conf lists every config file without you naming them. And command substitution — $(...) — runs a command and drops its output right into the line:
1ls -l /etc/*.conf # every .conf in /etc2cat /home/*/.bash_history # every user's shell history in one shot (if readable)3 4# substitution: use one command's result inside another5echo "I am $(whoami) on $(hostname), kernel $(uname -r)"6# -> I am www-data on target, kernel 5.15.0~/.bash_history is where a shell records the commands a user typed — and people type passwords, tokens, and connection strings into commands all the time. cat /home/*/.bash_history on a box you've landed on is a classic, high-yield read. Configs (grep -ri password /etc), history files, and environment variables are the three places credentials leak. Look there first, every time.Job control: keeping a shell alive
Your shell can run more than one thing. & launches a command in the background; Ctrl-Z suspends the current job; jobs, fg, and bg manage them. This matters when you need a listener running while you keep working, or when a long scan shouldn't block your prompt.
The environment (env) also travels with your shell — a set of KEY=value pairs like PATH (where the shell looks for programs) and, sometimes, secrets an admin exported carelessly.
1python3 -m http.server 8000 & # serve files in the background; keep your prompt2jobs # list background jobs3fg %1 # bring job 1 back to the foreground4 5env # dump all environment variables (look for secrets)6echo $PATH # the dirs searched for commands, in orderPATH is security-relevant. The shell runs the *first* matching program name it finds along PATH, in order. If a writable directory sits early in PATH, an attacker can plant a malicious ls there and have it run instead of the real one — a real privilege-escalation trick (PATH hijacking) you'll use later. For now, just register that echo $PATH tells you the search order, and order is power.None of this is "hacking" yet — it's fluency. But watch what happened: with three streams, a pipe, a glob, and $(...), you can enumerate users, mine histories and configs for credentials, and keep a listener alive on a box you don't own. The tools were mundane. The *combinations* are the weapon.
Build your own one-liners
On a Linux box you control, write and save (in shell1.md) five one-liners you compose yourself, one for each: (1) list only the usernames that have a real login shell, sorted; (2) show the top 5 largest files under /var, biggest first; (3) find every .conf under /etc that contains the word 'root' (silencing errors); (4) print your kernel version, hostname, and current user in a single sentence using command substitution; (5) start a background HTTP server on port 8000 and confirm it's running with jobs. For each, add one line explaining what each piped stage does.
$ cat /etc/passwd | grep -v nologin | cut -d: -f1 | sort$ grep -ril root /etc/*.conf 2>/dev/null$ echo "$(uname -r) on $(hostname) as $(whoami)"$ python3 -m http.server 8000 & jobs- › You can explain, per stage, what a 4-tool pipe is doing
- › You used 2>/dev/null and can say exactly which stream it discards
- › You used $(...) to embed one command's output inside another
- › You backgrounded a job and brought it back with fg