LFCS 30 🐧 Process Substitution - KandZ

The pipe connects the standard output of one command to the standard input of the next. But the pipe has a limitation: the two commands must be adjacent in the pipeline. The diff <(ls dir1) <(ls dir2) compares the output of the two ls commands, but the diff expects the file arguments, not the standard input. The comm expects the two sorted files, and the sort is the separate process. The tee >(gzip > out.gz) sends the output to the gzip and the standard output, but the tee expects the file arguments. The process substitution solves the problem. It presents the output of a command as a file, and the command that expects the file name reads the process’s output. The <() and the >() are the bash features, and they are the standard for the pipelines that need the file arguments.

The process substitution has the two directions. The <() is the read: the diff <(command1) <(command2) presents the two command outputs as the two files, and the diff reads them. The >() is the write: the tee >(gzip > out.gz) presents the gzip as the file, and the tee writes to it. The shell creates the FIFO or the /dev/fd entry, and the command reads or writes it as the normal file. The process substitution is the named pipe with the automatic name and the cleanup.

The process substitution composes. The <() can be the argument to any command that expects the file: the diff, the comm, the paste, the join, the wc, the md5sum, the cmp, and the vim. The >() can be the destination for the any output: the tee, the echo, the printf, the dd, the openssl, and the zip. The process substitution is the general mechanism, and the applications are the many.

This chapter covers three areas. First, why the process substitution exists — the file-argument problem and the pipe limitation. Second, the syntax — the <(), the >(), the /dev/fd, and the FIFO. Third, the common patterns — the diff and the comm, the tee with the multiple outputs, the while read with the process, and the pitfalls. The chapter ends with a complete example session, a quick reference, best practices, common pitfalls, real-world examples, and diagrams showing the process substitution flow.

Key point: The process substitution presents the output of a command as a file. The <() is the read: the command <(other) presents the other’s output as the file argument. The >() is the write: the command >(other) presents the other as the file destination. The shell creates the FIFO or the /dev/fd entry, and the command reads or writes it. The process substitution is the bash feature, and it is the solution for the commands that expect the file arguments.

Why process substitution exists

The file-argument problem. The commands have the two kinds of the input. The filters — the grep, the sed, the awk — read the standard input, and the pipe is the natural. The utilities — the diff, the comm, the cmp, the paste — expect the file arguments, and the pipe is the useless. The diff <(command1) <(command2) presents the two command outputs as the two files, and the diff reads them. The process substitution is the bridge.

The pipe-limitation problem. The pipe connects the stdout of the left to the stdin of the right, and the two commands must be the adjacent in the pipeline. The command1 | command2 | command3 is the linear, and the diff needs the two inputs. The process substitution creates the two independent inputs: the diff <(command1) <(command2) runs the both commands and the diff reads the both outputs. The pipe cannot do this.

The temporary-file problem. The command1 > /tmp/file1; command2 > /tmp/file2; diff /tmp/file1 /tmp/file2 is the way without the process substitution. The temporary files must be created, the written, the read, and the deleted. The process substitution skips the files: the outputs are the /dev/fd entries, and the cleanup is the automatic.

The tee-multiple-output problem. The tee writes to the multiple files, and the process substitution extends the destinations: the tee >(gzip > out.gz) >(wc -l > count.txt) writes to the both the gzip and the wc. The process substitution is the fan-out, and the tee is the split.

The while-read problem. The while read with the pipe runs in the subshell, and the variables set in the loop are the lost: the command | while read line; do var="$line"; done; echo "$var" prints the empty. The process substitution with the <() keeps the loop in the current shell: the while read line; do var="$line"; done < <(command); echo "$var" prints the last line. The process substitution is the fix for the subshell.

The trade-off. The process substitution is the bash feature, and the POSIX shell does not have it. The sh and the dash do not support it, and the ksh and the zsh do. The trade-off is between the convenience of the process substitution and the portability of the temporary files.

a. The process substitution syntax

The process substitution is the <() and the >().

The <(). The <(command) presents the command’s output as the file:

diff <(ls dir1) <(ls dir2)
comm <(sort file1) <(sort file2)
paste <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd)

The diff reads the two “files”, and the shell creates the /dev/fd/63 and the /dev/fd/62 entries.

The >(). The >(command) presents the command as the file destination:

tee >(gzip > out.gz) < input.txt
echo 'data' > >(logger -t myapp)
dd if=/dev/sda > >(gzip > sda.img.gz)

The tee writes to the “file” (the gzip process), and the shell creates the /dev/fd/63 entry.

The /dev/fd. The process substitution uses the /dev/fd/N:

echo <(echo 'hello')
# the /dev/fd/63

ls -l <(echo 'hello')
# the lr-x------ 1 user user 64 ... /dev/fd/63 -> pipe:[12345]

The <() is the /dev/fd entry, and the cat /dev/fd/63 reads the process output.

The FIFO. The process substitution may use the named pipe:

mkfifo /tmp/myfifo
command > /tmp/myfifo &
other_command < /tmp/myfifo
rm /tmp/myfifo

The named pipe is the manual, and the process substitution is the automatic.

The nesting. The process substitution can be the nested:

diff <(command1 <(command2)) <(command3)

The inner is the resolved first, and the outer reads the result.

The multiple. The process substitution can be the multiple:

diff <(command1) <(command2) <(command3)
# the diff only takes 2, the third is the extra

The exec. The process substitution with the exec:

exec 3< <(command)
while read -r line <&3; do
  echo "$line"
done
exec 3<&-

The exec 3< <() opens the file descriptor, and the cleanup is the exec 3<&-.

b. The common patterns

The diff. The diff with the process substitution:

diff <(ls dir1) <(ls dir2)
diff <(command1) <(command2)
diff <(sort file1) <(sort file2)

The diff compares the two outputs without the temporary files.

The comm. The comm with the process substitution:

comm <(sort file1) <(sort file2)
comm -12 <(sort file1) <(sort file2)  # the common lines
comm -23 <(sort file1) <(sort file2)  # the only in the file1

The comm needs the sorted inputs, and the process substitution provides them.

The tee with the multiple outputs. The tee with the process substitution:

command | tee >(gzip > out.gz) >(wc -l > count.txt) > /dev/null
command | tee >(logger -t myapp) >(mail -s 'The result' user@example.com)

The tee writes to the multiple process substitutions, and the /dev/null is the stdout discard.

The while read without the subshell. The process substitution with the while read:

while IFS= read -r line; do
  count=$((count + 1))
done < <(command)
echo "The count: $count"

The < <(command) keeps the loop in the current shell, and the count is the preserved.

The read with the process. The read with the process substitution:

read -r first < <(echo 'the value')
echo "$first"

The wc with the process. The wc with the process substitution:

wc -l < <(command)
wc -l <(command1) <(command2)

The md5sum with the process. The md5sum with the process substitution:

md5sum <(command)

The cmp with the process. The cmp with the process substitution:

cmp <(command1) <(command2)

The vim with the process. The vim with the process substitution:

vim <(command)

The vim edits the process output as the file.

The source with the process. The source with the process substitution:

source <(command)

The source reads the process output as the script.

The ssh with the process. The ssh with the process substitution:

ssh host 'cat /etc/config' | diff - <(cat /local/config)

The diff - compares the stdin with the process output.

The paste with the process. The paste with the process substitution:

paste <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd)

The paste combines the two process outputs as the columns.

c. The pitfalls and the limitations

The bash-only. The process substitution is the bash and the zsh and the ksh, and the POSIX sh and the dash do not support it:

# ❌ The sh
sh -c 'diff <(ls dir1) <(ls dir2)'
# the syntax error

# ✅ The bash
bash -c 'diff <(ls dir1) <(ls dir2)'

The fix is the temporary files or the bash.

The >() and the tee. The >() writes to the process, and the process must read the input:

# ❌ The wrong
echo 'data' > >(cat)
# the race: the cat may not be ready

The >() is the race-prone, and the tee is the safer for the critical.

The >(command) and the buffer. The >(command) writes to the process, and the buffer may lose the data:

# ❌ The risky
command > >(gzip > out.gz)
# the process substitution may exit before the data is flushed

The tee >(gzip > out.gz) is the safer, because the tee waits.

The order of the processes. The <(command1) <(command2) runs the both, and the order is not the guaranteed:

diff <(sleep 1; echo 'a') <(echo 'b')
# the diff may wait for the sleep

The order is the both concurrently, and the diff waits for the both.

The /dev/fd and the permission. The /dev/fd is the read-only or the write-only, depending on the direction:

# The <() is the read-only
# The >() is the write-only

The cat <(echo 'x') reads, and the echo 'x' > >(cat) writes.

The nested and the quoting. The nested process substitution needs the careful quoting:

# ❌ The wrong
diff <(echo <(echo 'x')) <(echo 'y')
# the inner is the literal

# ✅ The correct
diff <(echo <(echo 'x')) <(echo 'y')  # the inner is the resolved

The set -e. The set -e and the process substitution:

set -e
diff <(false) <(echo 'y')
# the diff's status is the 0 (the false is the separate process)

The set -e does not catch the process substitution’s failure, because the process is the separate.

The wait. The process substitution and the wait:

command > >(process1) > >(process2)
wait

The wait waits for the both, and the process substitution is the background.

Complete Example Session

# ============================================
# PART 1: THE BASIC DIFF
# ============================================

diff <(ls dir1) <(ls dir2)


# ============================================
# PART 2: THE COMM
# ============================================

comm <(sort file1) <(sort file2)
comm -12 <(sort file1) <(sort file2)  # the common lines
comm -23 <(sort file1) <(sort file2)  # the only in the file1


# ============================================
# PART 3: THE TEE WITH THE MULTIPLE
# ============================================

command | tee >(gzip > out.gz) >(wc -l > count.txt) > /dev/null


# ============================================
# PART 4: THE WHILE READ
# ============================================

count=0
while IFS= read -r line; do
  count=$((count + 1))
done < <(command)
echo "The count: $count"


# ============================================
# PART 5: THE PASTE
# ============================================

paste <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd)


# ============================================
# PART 6: THE MD5SUM
# ============================================

md5sum <(command)


# ============================================
# PART 7: THE VIM
# ============================================

vim <(command)


# ============================================
# PART 8: THE SSH DIFF
# ============================================

ssh host 'cat /etc/config' | diff - <(cat /local/config)


# ============================================
# PART 9: THE SOURCE
# ============================================

source <(command)


# ============================================
# PART 10: THE LFCS PIPELINE
# ============================================

# Compare the two directory listings
diff <(ls /etc) <(ls /etc.bak)

# The common and the unique lines
comm -12 <(sort file1) <(sort file2)

# The tee with the multiple
command | tee >(gzip > out.gz) >(wc -l > count.txt) > /dev/null

# The while read without the subshell
while IFS= read -r line; do
  echo "Line: $line"
done < <(cat file.txt)

# The md5sum of the process
md5sum <(echo 'data')

# The paste the two process outputs
paste <(seq 1 5) <(seq 6 10)

# The diff the remote and the local
diff <(ssh host 'cat /etc/config') <(cat /etc/config)

# The process substitution with the sort
diff <(sort file1) <(sort file2)

# The command with the multiple inputs
join <(sort file1) <(sort file2)

# The process substitution with the read
read -r first second < <(echo 'one two')
echo "$first"  # the one
echo "$second" # the two

The ten parts show the basic diff, the comm, the tee with the multiple, the while read, the paste, the md5sum, the vim, the ssh diff, the source, and the LFCS pipeline.

Quick Reference

The Operators

The operator	The direction
`<(command)`	The read: the output as the file
`>(command)`	The write: the input from the file
`/dev/fd/N`	The underlying
The FIFO	The named pipe

The Common Patterns

The pattern	The command
The diff	`diff <(command1) <(command2)`
The comm	`comm <(sort f1) <(sort f2)`
The tee	`tee >(gzip > out.gz)`
The while read	`while read; do ...; done < <(cmd)`
The paste	`paste <(cmd1) <(cmd2)`
The md5sum	`md5sum <(command)`
The source	`source <(command)`

The `<(command)` vs the `>(command)`

The aspect	`<(command)`	`>(command)`
The direction	The read	The write
The command	The file argument	The file destination
The example	`diff <(ls)`	`tee >(gzip)`
The `/dev/fd`	The read-only	The write-only

The `while read` Comparison

The form	The subshell	The variable
`command \| while read`	Yes	The lost
`while read < <(command)`	No	The preserved

The Bash vs the POSIX

The shell	The process substitution
bash	Yes
zsh	Yes
ksh	Yes
sh (the POSIX)	No
dash	No

Best Practices

✅ Do This:

# Use the process substitution for the file arguments
diff <(ls dir1) <(ls dir2)                                              # ✅
# Use the <() with the while read
while read -r line; do echo "$line"; done < <(command)                  # ✅
# Use the tee with the >()
command | tee >(gzip > out.gz) > /dev/null                              # ✅
# Use the bash for the process substitution
bash -c 'diff <(ls) <(ls /tmp)'                                         # ✅
# Use the source with the <()
source <(command)                                                       # ✅

❌ Don’t Do This:

# Don't use the process substitution in the POSIX sh
sh -c 'diff <(ls) <(ls)'  # the syntax error                            // ❌
# Don't use the >() without the wait
command > >(gzip > out.gz)  # the race                                  // ❌
# Don't use the pipe with the while read
command | while read line; do var="$line"; done; echo "$var"  # the lost // ❌
# Don't nest the unquoted
diff <(echo <(echo 'x'))  # the inner is the literal                    // ✅ (the inner is the resolved)
# Don't forget the /dev/fd is the process
cat /dev/fd/63  # the fd is the shell-specific                          // ❌
# Don't use the process substitution for the simple
cat <(echo 'x')  # the echo 'x' is the simpler                          // ❌

Common Pitfalls

The pitfall	The why	The fix
The syntax error in the sh	The POSIX	Use the bash
The variable lost in the while	The pipe	Use the `< <()`
The race in the `>()`	The process substitution	Use the `tee`
The diff waits	The slow process	The expected
The nested literal	The quoting	Check the nesting
The `/dev/fd` not found	The fd closed	Check the timing
The `set -e` not catching	The separate process	Check the process status

Real-World Examples

1. The Diff

diff <(ls dir1) <(ls dir2)

2. The Comm

comm -12 <(sort file1) <(sort file2)

3. The Tee

command | tee >(gzip > out.gz) > /dev/null

4. The While Read

while read -r line; do echo "$line"; done < <(command)

5. The Paste

paste <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd)

6. The Md5sum

md5sum <(command)

7. The Source

source <(command)

8. The Remote Diff

diff <(ssh host 'cat /etc/config') <(cat /etc/config)

9. The Join

join <(sort file1) <(sort file2)

10. The Complete

while read -r line; do count=$((count + 1)); done < <(command)

Visual

The Process Substitution Flow

┌──────────────────────────────────────────────────────────────┐
│  THE PROCESS SUBSTITUTION FLOW                               │
│                                                              │
│  diff <(ls dir1) <(ls dir2)                                  │
│                                                              │
│  ┌──────────┐    ┌──────────┐    ┌──────────┐                │
│  │ls dir1   │    │diff      │    │ls dir2   │                │
│  │          │    │          │    │          │                │
│  │ the      │    │ the      │    │ the      │                │
│  │ stdout ──┼───►│ /dev/fd/62│   │ stdout ──┼───►│/dev/fd/63 │
│  │          │    │          │    │          │                │
│  └──────────┘    └──────────┘    └──────────┘                │
│                                                              │
│  The shell creates the two /dev/fd entries.                  │
│  The diff reads them as the files.                           │
│                                                              │
└──────────────────────────────────────────────────────────────┘

The `<()` vs the `>()`

┌──────────────────────────────────────────────────────────────┐
│  THE <() vs THE >()                                          │
│                                                              │
│  THE `<(command)`:                                            │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  diff <(ls dir1) <(ls dir2)                          │    │
│  │                                                       │    │
│  │  The ls output → the /dev/fd/62 → the diff reads it  │    │
│  │  The diff reads the file.                            │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  THE `>(command)`:                                            │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  tee >(gzip > out.gz)                                │    │
│  │                                                       │    │
│  │  The tee writes → the /dev/fd/62 → the gzip reads it │    │
│  │  The tee writes to the file.                         │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

The while read Comparison

┌──────────────────────────────────────────────────────────────┐
│  THE while read COMPARISON                                   │
│                                                              │
│  ❌ THE PIPE (the subshell):                                  │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  command | while read line; do                       │    │
│  │    var="$line"                                       │    │
│  │  done                                                │    │
│  │  echo "$var"  # the EMPTY                            │    │
│  │                                                       │    │
│  │  The while runs in the SUBSHELL.                     │    │
│  │  The var is the lost.                                │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  ✅ THE PROCESS SUBSTITUTION (the current shell):             │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  while read line; do                                 │    │
│  │    var="$line"                                       │    │
│  │  done < <(command)                                   │    │
│  │  echo "$var"  # the LAST line                        │    │
│  │                                                       │    │
│  │  The while runs in the CURRENT shell.                │    │
│  │  The var is the preserved.                           │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

The Multiple Outputs

┌──────────────────────────────────────────────────────────────┐
│  THE MULTIPLE OUTPUTS                                        │
│                                                              │
│  command | tee >(gzip > out.gz) >(wc -l > count.txt) > /dev/null│
│                                                              │
│  ┌──────────┐    ┌──────────┐                                │
│  │command   │    │tee       │                                │
│  │          │    │          │                                │
│  │ the      │    │ the      │                                │
│  │ stdout ──┼───►│ stdin    │                                │
│  │          │    │          │                                │
│  │          │    │ ───► >(gzip > out.gz)                    │
│  │          │    │ ───► >(wc -l > count.txt)                │
│  │          │    │ ───► /dev/null                           │
│  └──────────┘    └──────────┘                                │
│                                                              │
│  The tee writes to the multiple process substitutions.       │
│  The /dev/null discards the stdout.                          │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Summary

Item	Value
The process substitution	The `<()` and the `>()`
The `<()`	The read: the output as the file
The `>()`	The write: the input from the file
The `/dev/fd`	The underlying
The diff	`diff <(ls d1) <(ls d2)`
The comm	`comm <(sort f1) <(sort f2)`
The tee	`tee >(gzip > out.gz)`
The while read	`while read < <(command)`
The bash	The requirement
The POSIX sh	No support

Key takeaways:

The process substitution presents the output of a command as a file. The <() is the read, and the >() is the write. The shell creates the /dev/fd entry, and the command reads or writes it.
The <() is the solution for the commands that expect the file arguments. The diff, the comm, the paste, the cmp, the md5sum, and the vim are the examples. The pipe cannot provide the two inputs.
The >() is the solution for the multiple outputs. The tee >(gzip > out.gz) >(wc -l > count.txt) writes to the both the processes and the stdout.
The process substitution fixes the subshell problem. The command | while read runs in the subshell, and the variables are the lost. The while read < <(command) keeps the loop in the current shell.
The process substitution is the bash feature. The zsh and the ksh support it, and the POSIX sh and the dash do not. The fix is the bash or the temporary files.
The >() is the race-prone. The command > >(gzip > out.gz) may exit before the data is flushed. The tee >(gzip > out.gz) is the safer.
The process substitution is the automatic named pipe. The mkfifo is the manual, and the process substitution is the automatic with the cleanup.
The result is the file-argument solution. The process substitution is the bridge between the commands that produce the output and the commands that expect the file. The result is the clean and the efficient shell scripting, which is the foundation of the automation.

Remember: The process substitution presents the output of a command as a file. The <() is the read, and the >() is the write. The shell creates the /dev/fd entry, and the command reads or writes it. The result is the file-argument solution, which is the foundation of the shell scripting.

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!

Why process substitution exists

a. The process substitution syntax

b. The common patterns

c. The pitfalls and the limitations

Complete Example Session

Quick Reference

The Operators

The Common Patterns

The <(command) vs the >(command)

The while read Comparison

The Bash vs the POSIX

Best Practices

Common Pitfalls

Real-World Examples

1. The Diff

2. The Comm

3. The Tee

4. The While Read

5. The Paste

6. The Md5sum

7. The Source

8. The Remote Diff

9. The Join

10. The Complete

Visual

The Process Substitution Flow

The <() vs the >()

The while read Comparison

The Multiple Outputs

Summary

The `<(command)` vs the `>(command)`

The `while read` Comparison

The `<()` vs the `>()`