Linux CLI 12 🐧 pipelines and cat, grep, wc commands - KandZ

Pipelines let you chain commands together — the output of one becomes the input to the next. Combined with cat, grep, and wc, they form the backbone of text processing on the command line.

What Are Pipelines?

A pipeline uses the | (pipe) operator to send the output of one command to the input of another.

cmd1 | cmd2 | cmd3

Visual:

┌────────┐     ┌────────┐     ┌────────┐
│ cmd1   │────→│ cmd2   │────→│ cmd3   │
│ stdout │     │ stdin  │     │ stdin  │
└────────┘     └────────┘     └────────┘

Key points:

| connects stdout of the left to stdin of the right
Commands run simultaneously — not waiting for each other
You can chain as many as you need
Each command processes data as it arrives (streaming)

Pipeline vs Redirection

Operator	Name	Purpose
`>`	Redirection	Send output to a file
`\|`	Pipeline	Send output to another command

Comparison:

# Redirection — output goes to a file
ls -l > file-list.txt

# Pipeline — output goes to a command
ls -l | grep "\.txt$"

Visual:

Redirection:
  Command ──stdout──→ File

Pipeline:
  Command1 ──stdout──→ Command2 ──stdout──→ Terminal

Basic Pipeline Examples

Example 1: Filter files

ls -l | grep "\.txt$"

Redirects ls -l output to grep, which shows only lines ending in .txt.

What happens:

ls -l produces a detailed listing
The output is piped to grep
grep filters for lines matching .txt at the end ($)
Only .txt files are shown

Output:

-rw-r--r-- 1 kronos users  1024 Jan 15 10:30 notes.txt
-rw-r--r-- 1 kronos users  2048 Jan 15 10:30 report.txt

Example 2: Count lines in a log

cat log.txt | wc -l

Redirects cat output to wc, which counts lines.

Equivalent to:

wc -l log.txt

Why use the pipeline? When you need to transform the data before counting:

grep "ERROR" log.txt | wc -l       # Count error lines
cat log.txt | sort | uniq | wc -l   # Count unique sorted lines

The `cat` Command

Concatenate and display file contents.

cat > new.txt
cat >> new.txt
cat new.txt newfile.txt > combined.txt
cat -n combined.txt
cat -v combined.txt

Command	Description
`cat file.txt`	Display file contents
`cat > new.txt`	Create a new file (type, Ctrl+D to save)
`cat >> new.txt`	Append to a file (Ctrl+D to save)
`cat file1 file2 > combined.txt`	Concatenate files
`cat -n file.txt`	Display with line numbers
`cat -v file.txt`	Show non-printing characters

Examples

Display a file:

$ cat notes.txt
Hello, World!
This is a note.

Create a new file:

$ cat > new.txt
Line 1
Line 2
Line 3
# Press Ctrl+D to save and exit
$ cat new.txt
Line 1
Line 2
Line 3

Append to a file:

$ cat >> new.txt
Line 4
# Press Ctrl+D
$ cat new.txt
Line 1
Line 2
Line 3
Line 4

Concatenate multiple files:

$ cat file1.txt file2.txt > combined.txt
$ cat combined.txt
# (contents of file1 followed by file2)

With line numbers:

$ cat -n combined.txt
     1  Line 1
     2  Line 2
     3  Line 3

Show non-printing characters:

$ cat -v file.txt
# Shows tabs as ^I, control chars as ^X

Useful cat options:

Option	Description
`-n`	Number all lines
`-b`	Number non-blank lines
`-s`	Squeeze multiple blank lines
`-v`	Show non-printing characters
`-E`	Show `$` at end of each line
`-T`	Show tabs as `^I`
`-A`	Same as `-vET`

The `grep` Command

Global Regular Expression Print — searches text for patterns.

grep "file" log.txt -n

Option	Description
`-i`	Ignore case
`-v`	Invert match (show non-matching lines)
`-r`	Recursive search
`-l`	Show only filenames
`-n`	Show line numbers
`-c`	Count matches
`-w`	Match whole words only
`-E`	Extended regex (same as `egrep`)
`-F`	Fixed strings (no regex)

Examples

Basic search:

$ grep "error" log.txt
[2024-01-15] error: connection failed
[2024-01-15] error: timeout

Case-insensitive (-i):

$ grep -i "error" log.txt
[2024-01-15] ERROR: connection failed
[2024-01-15] error: timeout
[2024-01-15] Error: retry

Invert match (-v):

$ grep -v "error" log.txt
[2024-01-15] info: server started
[2024-01-15] info: user logged in
# Shows all lines WITHOUT "error"

Line numbers (-n):

$ grep -n "error" log.txt
5:[2024-01-15] error: connection failed
12:[2024-01-15] error: timeout

Recursive (-r):

$ grep -r "TODO" ~/projects/
# Searches all files in projects/

Only filenames (-l):

$ grep -l "error" *.log
app.log
system.log
# Shows only which files contain "error"

Count matches (-c):

$ grep -c "error" log.txt
2
# Counts lines matching "error"

Whole words (-w):

$ grep -w "cat" file.txt
# Matches "cat" but not "catalog" or "concatenate"

Regular Expressions in `grep`

Pattern	Matches	Example
`.`	Any single character	`gr.y` → gray, grey
`^`	Start of line	`^error` → lines starting with error
`$`	End of line	`\.txt$` → lines ending in .txt
`[abc]`	Any of a, b, c	`gr[ae]y` → gray, grey
`[^abc]`	Any EXCEPT a, b, c	`[^0-9]` → not a digit
`a*`	Zero or more `a`	`ab*c` → ac, abc, abbc
`a+`	One or more `a`	`ab+c` → abc, abbc
`a?`	Zero or one `a`	`ab?c` → ac, abc
`\|`	Alternation (OR)	`cat\|dog` → cat or dog
`()`	Grouping	`(ab)+` → ab, abab

Examples:

# Lines starting with "error"
grep "^error" log.txt

# Lines ending with ".txt"
grep "\.txt$" filelist.txt

# Any 3-letter word starting with "c"
grep "\bc..\b" file.txt

# Email addresses
grep -E "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}" file.txt

# IP addresses
grep -E "([0-9]{1,3}\.){3}[0-9]{1,3}" log.txt

The `wc` Command

Word Count — counts lines, words, and characters.

wc output.txt
wc -l output.txt
wc -w output.txt
wc -m output.txt
wc -l file.txt output.txt newfile.txt
wc -L combined.txt

Option	Description
`-l`	Count lines
`-w`	Count words
`-m`	Count characters
`-c`	Count bytes
`-L`	Length of the longest line

Examples

Default (all counts):

$ wc output.txt
 25  150  1024 output.txt
 # lines  words  bytes  filename

Lines only:

$ wc -l output.txt
25 output.txt

Words only:

$ wc -w output.txt
150 output.txt

Characters only:

$ wc -m output.txt
1024 output.txt

Multiple files:

$ wc -l file1.txt file2.txt file3.txt
  10 file1.txt
  25 file2.txt
  15 file3.txt
  50 total

Longest line:

$ wc -L combined.txt
85 combined.txt
# Longest line is 85 characters

From a pipeline:

$ cat log.txt | wc -l
245

$ grep "error" log.txt | wc -l
12

Complete Example Session

# ============================================
# PART 1: BASIC PIPELINES
# ============================================

# Filter .txt files
$ ls -l | grep "\.txt$"
-rw-r--r-- 1 kronos users 1024 Jan 15 10:30 notes.txt
-rw-r--r-- 1 kronos users 2048 Jan 15 10:30 report.txt

# Count lines in a log
$ cat log.txt | wc -l
245

# ============================================
# PART 2: CAT COMMAND
# ============================================

# Display a file
$ cat notes.txt
Hello, World!

# Create a new file
$ cat > new.txt
Line 1
Line 2
# Press Ctrl+D

# Append to a file
$ cat >> new.txt
Line 3
# Press Ctrl+D

# Concatenate files
$ cat file1.txt file2.txt > combined.txt

# With line numbers
$ cat -n combined.txt
     1  Line 1
     2  Line 2
     3  Line 3

# ============================================
# PART 3: GREP COMMAND
# ============================================

# Basic search
$ grep "error" log.txt
[2024-01-15] error: connection failed
[2024-01-15] error: timeout

# Case-insensitive
$ grep -i "error" log.txt

# Invert match
$ grep -v "error" log.txt

# Line numbers
$ grep -n "error" log.txt
5:[2024-01-15] error: connection failed
12:[2024-01-15] error: timeout

# Recursive
$ grep -r "TODO" ~/projects/

# Only filenames
$ grep -l "error" *.log
app.log
system.log

# Count
$ grep -c "error" log.txt
2

# Whole words
$ grep -w "cat" file.txt

# ============================================
# PART 4: WC COMMAND
# ============================================

# All counts
$ wc output.txt
 25  150  1024 output.txt

# Lines only
$ wc -l output.txt
25 output.txt

# Words only
$ wc -w output.txt
150 output.txt

# Characters
$ wc -m output.txt
1024 output.txt

# Multiple files
$ wc -l file1.txt file2.txt file3.txt
  10 file1.txt
  25 file2.txt
  15 file3.txt
  50 total

# Longest line
$ wc -L combined.txt
85 combined.txt

# ============================================
# PART 5: COMBINING PIPELINES
# ============================================

# Count .txt files
$ ls | grep "\.txt$" | wc -l
5

# Count errors in a log
$ grep "ERROR" log.txt | wc -l
12

# Top 5 largest files
$ du -h * | sort -rh | head -5

# Most common words
$ cat file.txt | tr ' ' '\n' | sort | uniq -c | sort -rn | head -10

# Find and count .conf files
$ find /etc -name "*.conf" 2> /dev/null | wc -l
234

# ============================================
# PART 6: PRACTICAL EXAMPLES
# ============================================

# Count running processes
$ ps aux | wc -l

# Show only .log files in /var/log
$ ls /var/log | grep "\.log$"

# Find files containing "TODO"
$ grep -rl "TODO" ~/projects/

# Count lines of code
$ cat *.js | wc -l

# Show unique IP addresses in a log
$ grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" access.log | sort -u | wc -l

# Last 10 lines of a log
$ tail -10 log.txt

# First 5 lines with line numbers
$ head -5 log.txt | cat -n

Quick Reference

Pipeline Operator

Operator	Description
`\|`	Send stdout of left command to stdin of right command

`cat` Options

Option	Description
(none)	Display file
`-n`	Number all lines
`-b`	Number non-blank lines
`-s`	Squeeze blank lines
`-v`	Show non-printing chars
`-E`	Show `$` at line ends
`-T`	Show tabs as `^I`
`-A`	Same as `-vET`

`grep` Options

Option	Description
`-i`	Ignore case
`-v`	Invert match
`-r`	Recursive
`-l`	Filenames only
`-n`	Line numbers
`-c`	Count matches
`-w`	Whole words only
`-E`	Extended regex
`-F`	Fixed strings

`grep` Regex

Pattern	Matches
`.`	Any char
`^`	Start of line
`$`	End of line
`[abc]`	Any of a, b, c
`[^abc]`	Not a, b, c
`a*`	Zero or more a
`a+`	One or more a
`a?`	Zero or one a

`wc` Options

Option	Description
(none)	Lines, words, bytes
`-l`	Lines
`-w`	Words
`-c`	Bytes
`-m`	Characters
`-L`	Longest line

Best Practices

✅ Do This:

# Use pipelines to chain operations
ls | grep "\.txt$" | wc -l

# Prefer wc -l over grep -c for large files (faster)
wc -l file.txt

# Use grep -E for complex patterns
grep -E "error|warning|critical" log.txt

# Number lines with cat -n
cat -n file.txt

# Count files matching a pattern
find . -name "*.log" | wc -l

# Use pipelines to filter progressively
cat access.log | grep "404" | grep "Jan" | wc -l

# Pipe to less for large output
grep -r "TODO" . | less

❌ Don’t Do This:

# Don't use cat unnecessarily (Useless Use of Cat)
cat file.txt | grep "pattern"      # ❌ Useless cat
grep "pattern" file.txt            # ✅ Direct

# But cat is OK for multiple files
cat file1.txt file2.txt | grep "x" # ✅ OK

# Don't use grep -c for large files when wc -l works
grep -c "" hugefile.txt            # ❌ Slower
wc -l hugefile.txt                 # ✅ Faster

# Don't forget -E for extended regex
grep "a|b" file.txt                # ❌ Literal "a|b"
grep -E "a|b" file.txt             # ✅ Alternation

# Don't use cat > for complex file creation
cat > config.txt << EOF            # OK for heredocs
# ... but for simple cases, use echo
echo "content" > file.txt

Common Pitfalls

Pitfall	Problem	Solution
Useless `cat`	Wastes a process	Pass file directly to next command
`grep` without `-E` for regex	`\|` treated literally	Use `-E` or escape `\|`
Forget `wc -l` counts lines	Uses newline count	Last line without newline isn’t counted
`cat >` overwrites	Loses data	Use `cat >>` to append
`grep` on binary files	Garbled output	Use `grep -a` or `--text`
No quotes on regex	Shell expands special chars	Quote: `grep "\.txt$"`

Real-World Examples

1. Count errors in a log

grep -c "ERROR" /var/log/syslog

2. Find files with specific content

grep -rl "TODO" ~/projects/

3. Top 10 most frequent IPs in a log

grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" access.log | sort | uniq -c | sort -rn | head -10

4. Count lines of code

find . -name "*.js" -exec cat {} \; | wc -l

5. Show only unique lines

cat file.txt | sort | uniq

6. Exclude comments and blank lines

grep -vE "^\s*(#|$)" config.txt

7. Check if a process is running

ps aux | grep "nginx" | grep -v grep | wc -l

8. Analyze a large log

cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -20

9. Monitor a log in real time

tail -f /var/log/syslog | grep "ERROR"

10. Count total words in all .txt files

cat *.txt | wc -w

Visual: Pipeline Flow

$ ls -l | grep "\.txt$" | wc -l

┌─────────┐         ┌─────────┐         ┌─────────┐
│  ls -l  │──stdout─│  grep   │──stdout─│   wc    │
│         │  ──→    │  .txt   │  ──→    │   -l    │
│         │         │         │         │         │
└─────────┘         └─────────┘         └─────────┘
    │                    │                    │
    │  File list         │  Filtered          │  Count
    │  (all files)       │  (.txt only)       │  (5)
    ▼                    ▼                    ▼
  -rw-r--r-- file1     -rw-r--r-- notes.txt   5
  -rw-r--r-- notes.txt -rw-r--r-- report.txt
  -rw-r--r-- report.txt
  -rw-r--r-- image.png
  -rw-r--r-- script.sh

Summary

Command	Purpose	Example
`\|`	Pipeline operator	`cmd1 \| cmd2`
`cat`	Display/concatenate files	`cat file.txt`
`grep`	Search text	`grep "error" log.txt`
`wc`	Count lines/words/chars	`wc -l file.txt`

Key takeaways:

| sends stdout of one command to stdin of the next
cat displays, creates, and concatenates files
grep searches text with powerful regex patterns
wc counts lines, words, characters, and bytes
Combine them for powerful text processing
Avoid Useless Use of Cat when a command takes a file directly
grep -E enables extended regex (alternation, +, ?)

Remember: Pipelines are the magic of the command line. They let you build small, focused tools and combine them to solve complex problems. cat, grep, and wc are your foundational trio — once you’re comfortable chaining them, you can process gigabytes of text with a few keystrokes. And the golden rule: small commands, connected by pipes, doing one thing well!

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!

What Are Pipelines?

Pipeline vs Redirection

Basic Pipeline Examples

Example 1: Filter files

Example 2: Count lines in a log

The cat Command

Examples

The grep Command

Examples

Regular Expressions in grep

The wc Command

Examples

Complete Example Session

Quick Reference

Pipeline Operator

cat Options

grep Options

grep Regex

wc Options

Best Practices

Common Pitfalls

Real-World Examples

1. Count errors in a log

2. Find files with specific content

3. Top 10 most frequent IPs in a log

4. Count lines of code

5. Show only unique lines

6. Exclude comments and blank lines

7. Check if a process is running

8. Analyze a large log

9. Monitor a log in real time

10. Count total words in all .txt files

Visual: Pipeline Flow

Summary

Ads Blocker Detected!!!

The `cat` Command

The `grep` Command

Regular Expressions in `grep`

The `wc` Command

`cat` Options

`grep` Options

`grep` Regex

`wc` Options