Pipes and Chaining Commands: Unix Philosophy in Practice

The Unix philosophy is to build small, focused tools that each do one thing well, and compose them using pipes to perform complex operations.

1. The Pipe Operator

The | (pipe) operator connects the stdout of one command to the stdin of the next command:

# List files, filter for .log files, then count how many there are
ls /var/log | grep ".log" | wc -l

# Inspect the top 10 lines of a compressed log file
zcat app.log.gz | head -10

2. Practical Pipe Chaining Examples

# Find the 5 largest files in the current directory
du -sh * | sort -rh | head -5

# Count unique IP addresses in an Nginx access log
cat /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -20

3. xargs: Passing Output as Command Arguments

The xargs command reads items from stdin and passes them as arguments to another command. This is useful when a command does not accept piped stdin:

# Find all .log files and delete them using xargs
find /var/log -name "*.log" -mtime +30 | xargs rm -f

# Find all .jpg files and compress them in parallel
find ./images -name "*.jpg" | xargs -P 4 -I {} convert {} -quality 80 {}

• -P 4: Run up to 4 processes in parallel.
• -I {}: Replace {} with each line from stdin.