LFCS 24 🐧 Standard Streams — stdin, stdout, stderr - KandZ

Every process on a Linux system is born with three open file descriptors. File descriptor 0 is standard input (stdin), the channel the process reads from. File descriptor 1 is standard output (stdout), the channel the process writes its normal results to. File descriptor 2 is standard error (stderr), the channel the process writes its error messages to. These three streams are the process’s connection to the outside world, and they are the foundation of the Unix philosophy: the programs read from the input, write to the output, and report the errors separately, so they can be composed into the pipelines. The LFCS exam tests the redirection and the pipes as part of the “Use streams, pipes, and redirects” objective, and the understanding of the three streams is the prerequisite.

The separation of the stdout and the stderr is the detail that the beginners miss. The stdout is for the data, and the stderr is for the diagnostics. When the grep finds the matches, the matches go to the stdout, and the “permission denied” messages go to the stderr. When the find lists the files, the paths go to the stdout, and the errors go to the stderr. The separation means that the 2>/dev/null can silence the errors without silencing the data, and the > file can capture the data without capturing the noise. The two channels are the way the program distinguishes the result from the commentary.

The redirection operators are the mechanism that controls where the streams go. The > writes the stdout to a file, and the >> appends. The 2> writes the stderr to a file. The &> writes both. The < reads the stdin from a file. The 2>&1 merges the stderr into the stdout, and the 1>&2 merges the stdout into the stderr. The order matters: the > file 2>&1 sends both to the file, while the 2>&1 > file sends the stderr to the original stdout (the terminal) and the stdout to the file. The pipes | connect the stdout of one program to the stdin of the next, and the |& connects both the stdout and the stderr.

This chapter covers three areas. First, why the three streams exist — the data-versus-diagnostics problem and the composition. Second, the redirection operators — the >, the >>, the 2>, the &>, the <, the 2>&1, and the order of the operations. Third, the pipes and the tee — the |, the |&, the tee, the xargs, and the process substitution. The chapter ends with a complete example session, a quick reference, best practices, common pitfalls, real-world examples, and diagrams showing the stream flow.

Key point: Every process has three streams: the stdin (the fd 0), the stdout (the fd 1), and the stderr (the fd 2). The > redirects the stdout, the 2> redirects the stderr, the &> redirects both, the < redirects the stdin, and the 2>&1 merges the stderr into the stdout. The order of the redirections matters. The | connects the stdout to the next command’s stdin, and the |& connects both the stdout and the stderr.

Why the three streams exist

The data-versus-diagnostics problem. A program produces two kinds of output: the data (the result of the computation) and the diagnostics (the progress messages, the warnings, the errors). If the two were mixed in one stream, the consumer could not distinguish them. A pipeline that processes the data would also process the diagnostics, and a redirect that captures the data would also capture the noise. The separation of the stdout and the stderr is the way the program signals which output is which .

The composition problem. The stdout is the channel that the pipes connect. A program writes the data to the stdout, and the next program reads the data from the stdin. The composition of the simple programs into the complex pipelines is the Unix philosophy, and the stdout and the stdin are the glue. The stderr is deliberately excluded from the pipe, so the errors of the first program appear on the terminal while the data flows to the second program.

The file-descriptor problem. The three streams are the file descriptors 0, 1, and 2. A file descriptor is an integer that the kernel uses to identify the open file. The stdin is the fd 0, the stdout is the fd 1, and the stderr is the fd 2. The redirection operators are the shell’s way to change which file the descriptor points to. The > opens the file and points the fd 1 to it. The 2> opens the file and points the fd 2 to it. The 2>&1 copies the fd 1 to the fd 2, so the two point to the same file .

The buffering problem. The stdout is buffered, and the stderr is unbuffered. The stdout collects the output in a buffer and flushes it when the buffer is full or the program exits. The stderr writes the output immediately, so the error messages appear as they happen. The difference matters in the interactive programs and the long-running processes: the stderr messages appear in real time, and the stdout messages may be delayed. The stdbuf command changes the buffering, and the --line-buffered flag of the grep is the common workaround .

The terminal problem. When the program runs in a terminal, the stdout is connected to the terminal, and the buffering is the line buffering (the output is flushed on the newline). When the program runs in a pipeline, the stdout is connected to the pipe, and the buffering is the block buffering (the output is flushed when the buffer is full). The difference is the reason the output appears delayed in a pipeline, and it is the common source of the confusion.

The trade-off. The three streams are the convention, and the programs that follow the convention are the composable ones. The programs that mix the data and the diagnostics in the stdout break the pipelines, and the programs that write the data to the stderr break the redirects. The convention is not enforced, and the violations are the source of the bugs. The trade-off is between the simplicity of the single stream and the composability of the three.

a. The redirection operators

The stdout redirection. The > writes the stdout to a file, and the file is truncated if it exists. The >> appends.

ls > file.txt
ls >> file.txt

The > truncates the file even if the command fails. The set -o noclobber prevents the accidental overwrite, and the >| forces the write. The ls > file.txt writes the file list to the file.txt, and the errors (if any) appear on the terminal.

The stderr redirection. The 2> writes the stderr to a file, and the 2>> appends.

ls /nonexistent 2> error.txt
ls /nonexistent 2>> error.txt

The 2> is the redirect for the fd 2. The ls /nonexistent writes the “No such file or directory” message to the stderr, and the 2> error.txt captures it.

The stdout and the stderr together. The &> writes both to a file, and the &>> appends. The &> is the bash shortcut, and the > file 2>&1 is the POSIX equivalent.

ls /nonexistent &> all.txt
ls /nonexistent > all.txt 2>&1
ls /nonexistent &>> all.txt

The &> is the modern and the concise, and the > file 2>&1 is the portable. The order matters: the 2>&1 must come after the > file, or the stderr points to the original stdout.

The stdin redirection. The < reads the stdin from a file.

wc -l < file.txt
sort < file.txt
mysql -u root < schema.sql

The < connects the file to the fd 0. The wc -l < file.txt counts the lines, and the file.txt is the input, not the argument. The difference: the wc -l file.txt prints the file name, and the wc -l < file.txt prints only the count.

The stdout and the stderr to the different files. The > out.txt 2> err.txt sends the stdout to the out.txt and the stderr to the err.txt.

command > out.txt 2> err.txt

The discard. The /dev/null is the special file that discards the input. The > /dev/null discards the stdout, and the 2> /dev/null discards the stderr.

command > /dev/null
command 2> /dev/null
command &> /dev/null

The order matters. The > file 2>&1 sends both to the file, and the 2>&1 > file sends the stderr to the terminal and the stdout to the file.

command > file 2>&1    # both to the file
command 2>&1 > file    # the stderr to the terminal, the stdout to the file

The reason: the redirections are processed left to right. In the 2>&1 > file, the 2>&1 copies the fd 1 (the terminal) to the fd 2, and then the > file points the fd 1 to the file. The result: the fd 2 is the terminal, and the fd 1 is the file.

b. The pipes

The | connects the stdout of the left command to the stdin of the right command. The stderr is not piped, so the errors appear on the terminal.

ls -l | grep '.txt'
cat file.txt | sort | uniq -c | sort -rn | head -10
ps aux | grep nginx | awk '{print $2}'

The pipeline is the composition of the simple commands. The ls -l writes the listing to the stdout, the grep '.txt' reads it from the stdin, and the errors of the ls appear on the terminal.

The |& pipe. The |& connects both the stdout and the stderr to the next command’s stdin. It is the bash shorthand for the 2>&1 |.

command 2>&1 | grep 'error'
command |& grep 'error'

The |& is the concise form, and the 2>&1 | is the portable.

The tee command. The tee reads the stdin and writes to both the stdout and a file. It is the way to capture the output while it flows through the pipe.

command | tee output.txt
command | tee -a output.txt
command | tee output.txt | grep 'error'

The tee is the T-splitter: the input goes to the file and to the stdout. The -a appends, and the tee file1 file2 writes to the multiple files.

The xargs command. The xargs reads the items from the stdin and runs the command with the items as the arguments. It is the bridge between the pipelines and the commands that expect the arguments.

find . -name '*.log' | xargs rm
find . -name '*.log' -print0 | xargs -0 rm
echo 'file1 file2 file3' | xargs -n 1 ls -l

The xargs is the way to apply a command to the list of items. The -0 reads the null-separated input, and the -n 1 runs the command with one item at a time. The -I {} replaces the {} with the item.

The process substitution. The <(command) presents the command’s output as a file, and the >(command) presents the file as the command’s input. It is the way to use the command output where a file is expected.

diff <(ls dir1) <(ls dir2)
comm <(sort file1) <(sort file2)
tee >(gzip > out.gz) >(wc -l > count.txt) < input.txt

The process substitution is the bash feature, and it is the way to compare the two command outputs without the temporary files.

c. The practical patterns

The capture and the display. The tee captures the output while it is displayed.

make 2>&1 | tee build.log

The 2>&1 merges the stderr into the stdout, and the tee captures both while displaying them. The pattern is the standard for the build logs.

The discard the errors. The 2>/dev/null silences the errors.

find / -name '*.conf' 2>/dev/null

The find writes the “Permission denied” messages to the stderr, and the 2>/dev/null discards them. The result is the clean list of the configuration files.

The separate the streams. The > out.txt 2> err.txt separates the data and the diagnostics.

command > out.txt 2> err.txt

The merge and the capture. The &> all.txt captures both.

command &> all.txt

The append the logs. The >> appends to the log.

command >> /var/log/app.log 2>&1

The input from the file. The < reads the input from the file.

while read line; do
  echo "$line"
done < file.txt

The here-document. The << reads the input from the script until the delimiter.

cat << EOF > file.txt
line 1
line 2
EOF

The here-string. The <<< reads the input from the string.

grep 'pattern' <<< "$variable"

Complete Example Session

# ============================================
# PART 1: THE STDOUT REDIRECTION
# ============================================

ls > file.txt
ls >> file.txt
echo 'hello' > greeting.txt


# ============================================
# PART 2: THE STDERR REDIRECTION
# ============================================

ls /nonexistent 2> error.txt
ls /nonexistent 2>> error.txt


# ============================================
# PART 3: THE STDOUT AND THE STDERR TOGETHER
# ============================================

ls /nonexistent &> all.txt
ls /nonexistent > all.txt 2>&1
ls /nonexistent &>> all.txt


# ============================================
# PART 4: THE ORDER MATTERS
# ============================================

command > file 2>&1    # both to the file
command 2>&1 > file    # the stderr to the terminal, the stdout to the file


# ============================================
# PART 5: THE STDIN REDIRECTION
# ============================================

wc -l < file.txt
sort < file.txt
mysql -u root < schema.sql


# ============================================
# PART 6: THE DISCARD
# ============================================

command > /dev/null
command 2> /dev/null
command &> /dev/null


# ============================================
# PART 7: THE PIPES
# ============================================

ls -l | grep '.txt'
cat file.txt | sort | uniq -c | sort -rn | head -10
ps aux | grep nginx | awk '{print $2}'


# ============================================
# PART 8: THE |& AND THE TEE
# ============================================

command 2>&1 | grep 'error'
command |& grep 'error'

command | tee output.txt
command | tee -a output.txt
command | tee output.txt | grep 'error'


# ============================================
# PART 9: THE XARGS AND THE PROCESS SUBSTITUTION
# ============================================

find . -name '*.log' | xargs rm
find . -name '*.log' -print0 | xargs -0 rm
echo 'file1 file2 file3' | xargs -n 1 ls -l

diff <(ls dir1) <(ls dir2)
comm <(sort file1) <(sort file2)


# ============================================
# PART 10: THE LFCS PIPELINE
# ============================================

# Capture the build output while displaying it
make 2>&1 | tee build.log

# Discard the permission errors
find / -name '*.conf' 2>/dev/null

# Separate the streams
command > out.txt 2> err.txt

# The here-document
cat << EOF > config.txt
key1=value1
key2=value2
EOF

# The here-string
grep 'pattern' <<< "$variable"

# The while-read loop
while read line; do
  echo "Line: $line"
done < file.txt

# The input from the process substitution
while read line; do
  echo "Line: $line"
done < <(ls -l)

The ten parts show the stdout redirection, the stderr redirection, the both, the order, the stdin redirection, the discard, the pipes, the |& and the tee, the xargs and the process substitution, and the LFCS pipeline.

Quick Reference

The Streams

Stream	File descriptor	Purpose
`stdin`	0	The input
`stdout`	1	The normal output
`stderr`	2	The error output

The Redirection Operators

Operator	Meaning
`> file`	The `stdout` to the file (truncate)
`>> file`	The `stdout` to the file (append)
`2> file`	The `stderr` to the file
`2>> file`	The `stderr` to the file (append)
`&> file`	The both to the file
`&>> file`	The both to the file (append)
`< file`	The `stdin` from the file
`2>&1`	The `stderr` to the `stdout`
`1>&2`	The `stdout` to the `stderr`
`> file 2>&1`	The both to the file

The Pipes

Operator	Meaning
`\|`	The `stdout` to the next `stdin`
`\|&`	The `stdout` and the `stderr` to the next `stdin`
`tee`	The `stdin` to the `stdout` and a file
`xargs`	The `stdin` items to the command arguments

The Special Files

File	Meaning
`/dev/null`	The discard
`/dev/stdin`	The fd 0
`/dev/stdout`	The fd 1
`/dev/stderr`	The fd 2
`/dev/tty`	The terminal

The Here Documents

Syntax	Meaning
`<< EOF`	The here-document
`<< 'EOF'`	The here-document, no expansion
`<<< "$var"`	The here-string
`<(command)`	The process substitution (the read)
`>(command)`	The process substitution (the write)

Best Practices

✅ Do This:

# Separate the data and the errors
command > out.txt 2> err.txt                                              # ✅
# Capture the both with the order
command > all.txt 2>&1                                                    # ✅
# Discard the errors
command 2>/dev/null                                                       # ✅
# Capture and display with the tee
command 2>&1 | tee build.log                                              # ✅
# Use the -print0 with the xargs
find . -name '*.log' -print0 | xargs -0 rm                                # ✅
# Use the process substitution instead of the temp files
diff <(ls dir1) <(ls dir2)                                                # ✅

❌ Don’t Do This:

# Don't write the wrong order
command 2>&1 > file  # the stderr goes to the terminal                    // ❌
# Don't use the pipe to silence the errors
command | grep 'data'  # the errors still appear                          // ❌
# Don't forget the -0 with the xargs for the spaces
find . -name '*.log' | xargs rm  # the spaces split the names             // ❌
# Don't use the > when the >> is intended
command > log.txt  # truncates the log                                    // ❌
# Don't mix the stdout and the stderr in the scripts
command > file.txt  # the errors are lost                                 // ❌
# Don't forget the redirection order in the pipelines
command 2>&1 | tee log  # captures both                                   // ✅

Common Pitfalls

Pitfall	Why It Happens	Fix
The errors appear on the terminal	No `2>`	Add the `2>&1`
The data is mixed with the errors	The both in one file	Separate the streams
The file is truncated	The `>` is used	Use the `>>`
The order is wrong	The `2>&1 > file`	Use the `> file 2>&1`
The `xargs` splits the spaces	No `-0`	Use the `-print0` and the `-0`
The output is delayed in the pipe	The block buffering	Use the `stdbuf` or the `--line-buffered`
The `tee` overwrites the file	No `-a`	Use the `tee -a`
The `\>` is not portable	The bash shortcut	Use the `> file 2>&1`

Real-World Examples

1. The Basic Redirect

ls > file.txt

2. The Append

ls >> file.txt

3. The Stderr

ls /nonexistent 2> error.txt

4. The Both

ls /nonexistent &> all.txt

5. The Order

command > file 2>&1

6. The Discard

command 2>/dev/null

7. The Pipe

ls -l | grep '.txt'

8. The Tee

command | tee output.txt

9. The Xargs

find . -name '*.log' -print0 | xargs -0 rm

10. The Process Substitution

diff <(ls dir1) <(ls dir2)

Visual

The Three Streams

┌──────────────────────────────────────────────────────────────┐
│  THE THREE STREAMS                                           │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  THE PROCESS                                         │    │
│  │                                                       │    │
│  │  fd 0 (stdin)  ◄── the input                         │    │
│  │  fd 1 (stdout) ──► the normal output                 │    │
│  │  fd 2 (stderr) ──► the error output                  │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  THE TERMINAL                                        │    │
│  │                                                       │    │
│  │  The keyboard ──► the stdin                          │    │
│  │  The stdout   ──► the display                        │    │
│  │  The stderr   ──► the display                        │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  The three streams are the process's connection to the       │
│  outside world.                                              │
│                                                              │
└──────────────────────────────────────────────────────────────┘

The Redirection

┌──────────────────────────────────────────────────────────────┐
│  THE REDIRECTION                                             │
│                                                              │
│  command > file                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  The fd 1 is pointed to the file.                    │    │
│  │  The stdout goes to the file.                        │    │
│  │  The stderr goes to the terminal.                    │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  command 2> file                                             │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  The fd 2 is pointed to the file.                    │    │
│  │  The stderr goes to the file.                        │    │
│  │  The stdout goes to the terminal.                    │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  command > file 2>&1                                         │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  The fd 1 is pointed to the file.                    │    │
│  │  The fd 2 is copied from the fd 1 (the file).        │    │
│  │  Both go to the file.                                │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  command 2>&1 > file                                         │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  The fd 2 is copied from the fd 1 (the terminal).    │    │
│  │  The fd 1 is pointed to the file.                    │    │
│  │  The stderr goes to the terminal, the stdout to the file│   │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

The Pipe

┌──────────────────────────────────────────────────────────────┐
│  THE PIPE                                                    │
│                                                              │
│  command1 | command2                                         │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  command1                                            │    │
│  │    the stdout ──► the pipe ──► the stdin of command2 │    │
│  │    the stderr ──► the terminal                       │    │
│  │                                                       │    │
│  │  command2                                            │    │
│  │    the stdin ◄── the pipe                            │    │
│  │    the stdout ──► the terminal                       │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  command1 |& command2                                        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  The stderr of command1 is also piped to command2.   │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

The Tee

┌──────────────────────────────────────────────────────────────┐
│  THE TEE                                                     │
│                                                              │
│  command | tee output.txt                                    │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  command                                             │    │
│  │    the stdout ──► the pipe ──► tee                   │    │
│  │                                                       │    │
│  │  tee                                                 │    │
│  │    ──► output.txt (the file)                         │    │
│  │    ──► the terminal (the stdout)                     │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  The tee is the T-splitter.                          │    │
│  │  The input goes to the file AND the terminal.        │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  command | tee output.txt | grep 'error'                     │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  command ──► tee ──► output.txt                      │    │
│  │              └────► grep 'error'                     │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Summary

Item	Value
`stdin`	The fd 0; the input
`stdout`	The fd 1; the normal output
`stderr`	The fd 2; the error output
`>`	The `stdout` to the file (truncate)
`>>`	The `stdout` to the file (append)
`2>`	The `stderr` to the file
`&>`	The both to the file
`<`	The `stdin` from the file
`2>&1`	The `stderr` to the `stdout`
`\|`	The `stdout` to the next `stdin`
`\|&`	The `stdout` and the `stderr` to the next `stdin`
`tee`	The `stdin` to the `stdout` and a file

Key takeaways:

Every process has three streams: the stdin, the stdout, and the stderr. The stdin is the fd 0, the stdout is the fd 1, and the stderr is the fd 2. The stdout is for the data, and the stderr is for the diagnostics .
The > redirects the stdout, and the 2> redirects the stderr. The &> redirects both, and the < redirects the stdin. The >> and the 2>> append instead of truncating .
The order of the redirections matters. The > file 2>&1 sends both to the file, and the 2>&1 > file sends the stderr to the terminal and the stdout to the file. The redirections are processed left to right .
The | connects the stdout to the next command’s stdin, and the |& connects both the stdout and the stderr. The stderr is deliberately excluded from the pipe, so the errors appear on the terminal .
The tee captures the output while it flows through the pipe. The tee reads the stdin and writes to the stdout and a file. The -a appends, and the tee file1 file2 writes to the multiple files .
The xargs reads the items from the stdin and runs the command with the items as the arguments. The -0 reads the null-separated input, and the -n 1 runs the command with one item at a time. The -print0 with the -0 is the safe pattern for the file names with the spaces .
The process substitution presents the command’s output as a file. The <(command) is the read, and the >(command) is the write. The diff <(ls dir1) <(ls dir2) is the standard way to compare the two command outputs .
The /dev/null is the special file that discards the input. The > /dev/null discards the stdout, and the 2> /dev/null discards the stderr. The &> /dev/null discards both .

Remember: The three standard streams — the stdin, the stdout, and the stderr — are the process’s connection to the outside world. The stdout is for the data, and the stderr is for the diagnostics. The > redirects the stdout, the 2> redirects the stderr, the &> redirects both, and the < redirects the stdin. The order of the redirections matters, and the 2>&1 must come after the > file. The | connects the stdout to the next command, and the |& connects both. The tee captures while it displays, and the xargs applies the command to the list. The result is the composition of the simple programs into the complex pipelines, which is the foundation of the Unix philosophy.

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!