LFCS 17 🐧 Text Processing — sed Basics - KandZ

sed is the stream editor. It reads input line by line, applies a set of editing commands to each line, and writes the result to standard output. Unlike an interactive editor, sed is not meant for hand-editing a file. It is a filter: it takes a stream of text, transforms it according to a script, and emits the transformed stream. This makes it the natural tool for automated text substitution, deletion, insertion, and extraction. The LFCS exam tests sed as part of the text-processing objective, and the command’s ability to perform search-and-replace without opening a file is one of the most practical skills in Linux administration .

The name comes from “stream editor,” and the stream model is the key to understanding it. sed does not load the entire file into memory. It reads a line, applies the script, prints the result, and moves to the next line. This means it can process files of any size, and it can be used in a pipeline where the input is generated by another command. The default behavior is to print every line, whether it was modified or not. Options like -n suppress the default printing, so that only lines explicitly selected by the script are output .

sed operates on a pattern space — a buffer that holds the current line. The script’s commands operate on this buffer. The most common command is s for substitution, which replaces a pattern with a replacement string. Other commands include d for delete, p for print, a for append, i for insert, and c for change. Each command can be prefixed with an address, which restricts the command to specific lines. An address can be a line number, a range, a regular expression, or a combination .

This chapter covers three areas. First, why sed exists and how the stream model works — the problem of automating edits and the pattern-space concept. Second, how the substitution command works — the s/pattern/replacement/flags syntax, the flags that control replacement behavior, and the use of backreferences. Third, how to use addresses and other commands — line numbers, ranges, regular expressions, deletion, printing, insertion, and in-place editing. The chapter ends with a complete example session, a quick reference, best practices, common pitfalls, real-world examples, and diagrams showing the stream model.

Key point: sed is a stream editor that reads input line by line, applies a script, and writes to standard output. The substitution command is s/pattern/replacement/flags. By default, sed prints every line; the -n option suppresses this. The -i option edits files in place. Addresses restrict commands to specific lines .

Why sed exists

The automated-editing problem. Manual editing is not repeatable. Changing a configuration value in one file is easy; changing the same value in a hundred files is not. A script that performs the edit is repeatable, auditable, and fast. sed is the standard tool for scripted edits. It can replace a string, delete a line, insert a line after a match, and perform these operations across many files. The LFCS exam scenarios include changing configuration values, removing comments, and extracting fields, all of which are sed tasks .

The stream problem. sed is designed for pipelines. It reads from standard input and writes to standard output, so it can be placed between other commands. grep finds the lines, sed transforms them, and sort orders them. The command does not need to know where the input comes from or where the output goes. This composability is what makes sed a building block rather than a complete tool.

The pattern-space problem. sed does not edit the file directly. It reads a line into the pattern space, applies the script, and writes the pattern space to the output. The file on disk is not modified unless the -i option is used. This separation is what allows sed to be non-destructive by default. The pattern space is a buffer, and the script is a set of transformations on that buffer. Understanding this model is the key to understanding why sed behaves the way it does .

The address problem. Not every command applies to every line. A substitution might apply only to lines matching a pattern, or only to a range of line numbers. sed addresses are the mechanism for this. An address precedes the command and restricts it. 3d deletes line 3. /error/d deletes lines matching error. 1,10s/foo/bar/ substitutes on lines 1 through 10. The address syntax is what gives sed its precision .

The non-destructive problem. By default, sed writes to standard output and leaves the input file unchanged. This is the safe default: a mistaken command does not destroy the file. The -i option changes the behavior to edit in place, and it is the option that requires the most caution. On GNU sed, -i takes an optional suffix for a backup file, so sed -i.bak creates a backup before editing. On BSD/macOS sed, the -i option requires an argument, so sed -i '' is used for no backup. The difference is a common source of portability bugs .

The trade-off. sed is a line-oriented tool. It does not understand the structure of the file; it sees lines. A substitution that should apply only to a specific section of a configuration file must be addressed with line numbers or patterns, and the addressing can be fragile if the file changes. The regular-expression syntax is powerful but has subtle differences between basic and extended modes. The trade-off is between the simplicity of a stream filter and the precision of a structured editor. For the majority of text-processing tasks, the simplicity is an advantage, and the addressing is sufficient.

a. The substitution command

The substitution command is the most used sed command. Its syntax is s/pattern/replacement/flags .

sed 's/error/warning/' file.txt

This replaces the first occurrence of error on each line with warning. The command is quoted to prevent the shell from interpreting the slashes and special characters. The delimiter / can be replaced with any character, which is useful when the pattern or replacement contains slashes:

sed 's|/usr/local|/opt|' file.txt

The flags control the replacement behavior:

g replaces all occurrences on each line, not just the first.
N (a number) replaces the Nth occurrence.
p prints the line if a substitution was made. Used with -n.
i or I makes the match case-insensitive.
w file writes the result to a file .

sed 's/error/warning/g' file.txt       # all occurrences
sed 's/error/warning/2' file.txt       # second occurrence
sed -n 's/error/warning/p' file.txt    # only lines with substitution

Backreferences allow the replacement to reuse parts of the matched pattern. The pattern uses $ and $ to capture groups, and the replacement uses \1, \2, and so on to refer to them :

sed 's/\([a-z]*\) \([a-z]*\)/\2 \1/' file.txt

This swaps two words on each line. The first capture group is the first word, the second is the second word, and the replacement puts the second word first.

In extended regular expression mode (-E or -r), the parentheses do not need to be escaped:

sed -E 's/([a-z]+) ([a-z]+)/\2 \1/' file.txt

The & character in the replacement refers to the entire matched pattern:

sed 's/error/[&]/' file.txt

This wraps each occurrence of error in brackets.

b. Addresses

An address restricts a command to specific lines. Without an address, the command applies to every line. With an address, it applies only to the lines that match .

Line number addresses:

sed '3d' file.txt          # delete line 3
sed '1,5d' file.txt        # delete lines 1 through 5
sed '5,$d' file.txt        # delete lines 5 through the last line

The $ address means the last line. The range 1,5 means lines 1 through 5 inclusive.

Regular expression addresses:

sed '/error/d' file.txt        # delete lines matching "error"
sed '/start/,/end/d' file.txt  # delete from "start" to "end"

The range /start/,/end/ begins at the first line matching start and ends at the first subsequent line matching end. This is useful for removing blocks of text delimited by markers.

Negated addresses:

sed '/error/!d' file.txt       # delete all lines NOT matching "error"

The ! after the address negates it. The command applies to lines that do not match.

Step addresses:

sed '1~2d' file.txt            # delete every other line starting at 1

The ~ syntax is a GNU extension. 1~2 means lines 1, 3, 5, and so on.

c. Other commands

Beyond substitution, sed has commands for deletion, printing, insertion, and more.

Delete (d):

sed '3d' file.txt              # delete line 3
sed '/^$/d' file.txt           # delete empty lines
sed '/^#/d' file.txt           # delete comment lines

The delete command removes the pattern space from the output. When -n is used with d, the behavior is the same: the line is not printed. The d command starts the next cycle immediately, so no further commands are applied to the deleted line.

Print (p):

sed -n '3p' file.txt           # print only line 3
sed -n '/error/p' file.txt     # print only lines matching "error"

The p command prints the pattern space. By itself, it duplicates the line because the default behavior also prints. The -n option suppresses the default printing, so p is the only output .

Append (a), Insert (i), Change (c):

sed '3a\new line' file.txt     # append after line 3
sed '3i\new line' file.txt     # insert before line 3
sed '3c\replacement' file.txt  # replace line 3

The append command adds text after the addressed line. The insert command adds text before the addressed line. The change command replaces the addressed line. On GNU sed, the backslash and the text can be on the same line or on separate lines.

In-place editing (-i):

sed -i 's/error/warning/g' file.txt          # edit file in place
sed -i.bak 's/error/warning/g' file.txt      # backup to file.txt.bak

The -i option edits the file directly. On GNU sed, an optional suffix creates a backup. On BSD/macOS sed, the -i option requires an argument; sed -i '' means no backup, and sed -i '.bak' creates a backup with the .bak suffix .

Multiple commands (-e and ;):

sed -e 's/error/warning/' -e 's/warning/notice/' file.txt
sed 's/error/warning/; s/warning/notice/' file.txt

Multiple commands can be specified with multiple -e options or separated by semicolons in a single script.

Reading a script from a file (-f):

sed -f script.sed file.txt

The -f option reads the script from a file, which is useful for complex or reused scripts.

Complete Example Session

# ============================================
# PART 1: BASIC SUBSTITUTION
# ============================================

# Replace the first occurrence of "error" on each line
sed 's/error/warning/' file.txt

# Replace all occurrences
sed 's/error/warning/g' file.txt


# ============================================
# PART 2: ALTERNATIVE DELIMITER
# ============================================

# Use | instead of / to avoid escaping slashes
sed 's|/usr/local|/opt|' file.txt


# ============================================
# PART 3: CASE-INSENSITIVE SUBSTITUTION
# ============================================

sed 's/error/warning/i' file.txt
# Matches: error, Error, ERROR


# ============================================
# PART 4: PRINT ONLY SUBSTITUTED LINES
# ============================================

sed -n 's/error/warning/p' file.txt
# Only lines where a substitution occurred are printed


# ============================================
# PART 5: BACKREFERENCES
# ============================================

# Swap two words on each line
sed 's/\([a-z]*\) \([a-z]*\)/\2 \1/' file.txt

# Extended regex version
sed -E 's/([a-z]+) ([a-z]+)/\2 \1/' file.txt


# ============================================
# PART 6: ADDRESSES — LINE NUMBERS
# ============================================

sed '3d' file.txt          # delete line 3
sed '1,5d' file.txt        # delete lines 1-5
sed '5,$d' file.txt        # delete lines 5 to end


# ============================================
# PART 7: ADDRESSES — REGULAR EXPRESSIONS
# ============================================

sed '/error/d' file.txt        # delete lines matching "error"
sed '/start/,/end/d' file.txt  # delete from "start" to "end"
sed '/error/!d' file.txt       # delete lines NOT matching "error"


# ============================================
# PART 8: DELETE COMMENTS AND BLANK LINES
# ============================================

sed '/^#/d' file.txt           # delete comment lines
sed '/^$/d' file.txt           # delete blank lines
sed '/^#/d; /^$/d' file.txt    # both


# ============================================
# PART 9: IN-PLACE EDITING
# ============================================

# GNU sed: edit in place with backup
sed -i.bak 's/error/warning/g' file.txt

# BSD/macOS sed: edit in place with no backup
sed -i '' 's/error/warning/g' file.txt


# ============================================
# PART 10: THE LFCS PIPELINE
# ============================================

# Remove comments and blank lines from a config file
sed '/^#/d; /^$/d' /etc/ssh/sshd_config

# Change a configuration value in place, with backup
sed -i.bak 's/^#PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config

# Verify
grep "PermitRootLogin" /etc/ssh/sshd_config

The ten parts show basic substitution, alternative delimiter, case-insensitive substitution, printing only substituted lines, backreferences, line-number addresses, regular-expression addresses, deleting comments and blank lines, in-place editing, and the LFCS pipeline scenario.

Quick Reference

Substitution Syntax

Form	Meaning
`s/pattern/replacement/`	Replace first occurrence per line
`s/pattern/replacement/g`	Replace all occurrences
`s/pattern/replacement/N`	Replace Nth occurrence
`s/pattern/replacement/p`	Print if substituted (with `-n`)
`s/pattern/replacement/i`	Case-insensitive match
`s\|pattern\|replacement\|`	Alternative delimiter

Addresses

Address	Meaning
`3`	Line 3
`1,5`	Lines 1 through 5
`5,$`	Line 5 to last
`/pattern/`	Lines matching pattern
`/start/,/end/`	Range from start to end
`/pattern/!`	Lines NOT matching
`1~2`	Every other line from 1 (GNU)

Common Commands

Command	Effect
`s`	Substitute
`d`	Delete
`p`	Print
`a`	Append after
`i`	Insert before
`c`	Change (replace)
`q`	Quit
`y`	Transliterate

Options

Option	Effect
`-n`	Suppress default printing
`-i[SUFFIX]`	Edit in place
`-e SCRIPT`	Add a script
`-f FILE`	Read script from file
`-E` or `-r`	Extended regex

Best Practices

✅ Do This:

# Quote the script
sed 's/error/warning/g' file.txt                                          # ✅
# Use -i.bak for backup before in-place edit
sed -i.bak 's/old/new/g' file.txt                                         # ✅
# Use alternative delimiter for paths
sed 's|/usr/local|/opt|' file.txt                                         # ✅
# Test without -i first
sed 's/old/new/g' file.txt | head                                         # ✅
# Use -n with p for precise output
sed -n '/error/p' file.txt                                                # ✅

❌ Don’t Do This:

# Don't use -i without a backup
sed -i 's/old/new/g' important.conf                                       # ❌
# Don't forget the g flag for all occurrences
sed 's/error/warning/' file.txt  # only first occurrence                  # ❌
# Don't use / as delimiter for paths
sed 's//usr/local//opt/' file.txt  # broken                                # ❌
# Don't assume -i syntax is portable
sed -i 's/old/new/' file.txt  # BSD needs -i ''                            # ❌

Common Pitfalls

Pitfall	Why It Happens	Fix
Only first occurrence replaced	Missing `g` flag	Add `g`
Slashes in pattern break command	`/` used as delimiter	Use alternative delimiter
In-place edit without backup	`-i` used without suffix	Use `-i.bak`
BSD/GNU `-i` difference	Different implementations	Use `-i.bak` for portability
Line numbers change after edits	Multiple commands	Test with `-n` first
Greedy regex match	`*` matches too much	Use more specific pattern
Backreference not working	Escaping in BRE	Use `-E` for extended

Real-World Examples

1. Replace a String

sed 's/old/new/g' file.txt

2. Replace in Place with Backup

sed -i.bak 's/old/new/g' file.txt

3. Delete Comment Lines

sed '/^#/d' file.txt

4. Delete Blank Lines

sed '/^$/d' file.txt

5. Delete a Range

sed '/start/,/end/d' file.txt

6. Print Matching Lines

sed -n '/error/p' file.txt

7. Change a Config Value

sed -i 's/^#PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config

8. Insert a Line

sed '3i\new line' file.txt

9. Append a Line

sed '3a\new line' file.txt

10. Multiple Commands

sed 's/error/warning/; s/warning/notice/' file.txt

Visual

The Stream Model

┌──────────────────────────────────────────────────────────────┐
│  THE STREAM MODEL                                            │
│                                                              │
│  Input file                                                  │
│    │                                                         │
│    ▼                                                         │
│  Read line 1 ──► pattern space ──► apply script ──► output  │
│    │                                                         │
│    ▼                                                         │
│  Read line 2 ──► pattern space ──► apply script ──► output  │
│    │                                                         │
│    ▼                                                         │
│  Read line 3 ──► pattern space ──► apply script ──► output  │
│    │                                                         │
│    ▼                                                         │
│  ... (repeat for each line)                                  │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  sed reads line by line.                             │    │
│  │  The pattern space holds the current line.           │    │
│  │  The script transforms the pattern space.            │    │
│  │  The default output prints every line.               │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Substitution Flags

┌──────────────────────────────────────────────────────────────┐
│  SUBSTITUTION FLAGS                                          │
│                                                              │
│  Input: "error and error and error"                          │
│                                                              │
│  s/error/warning/   → "warning and error and error"          │
│      (first only)                                            │
│                                                              │
│  s/error/warning/g  → "warning and warning and warning"      │
│      (all occurrences)                                       │
│                                                              │
│  s/error/warning/2  → "error and warning and error"          │
│      (second occurrence)                                     │
│                                                              │
│  s/ERROR/warning/i  → "warning and error and error"          │
│      (case-insensitive, first match)                         │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Addresses

┌──────────────────────────────────────────────────────────────┐
│  ADDRESSES                                                   │
│                                                              │
│  Line numbers:                                               │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  1  first line                                       │    │
│  │  2  second line                                      │    │
│  │  3  third line         ◄── sed '3d' deletes this     │    │
│  │  4  fourth line                                      │    │
│  │  5  fifth line         ◄── sed '1,5d' deletes 1-5    │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  Regex ranges:                                               │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  1  start block       ◄── /start/,/end/ begins here  │    │
│  │  2  content                                          │    │
│  │  3  content                                          │    │
│  │  4  end block         ◄── ends here                  │    │
│  │  5  after                                            │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  Negation:                                                   │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  /error/!d  → delete all lines NOT matching "error"  │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

In-Place Editing

┌──────────────────────────────────────────────────────────────┐
│  IN-PLACE EDITING                                            │
│                                                              │
│  WITHOUT -i:                                                 │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  sed 's/old/new/g' file.txt                          │    │
│  │                                                       │    │
│  │  file.txt is unchanged.                              │    │
│  │  The result goes to standard output.                 │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  WITH -i:                                                    │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  sed -i 's/old/new/g' file.txt                       │    │
│  │                                                       │    │
│  │  file.txt is modified in place.                      │    │
│  │  No output to standard output.                       │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  WITH -i.bak:                                                │
│  ┌──────────────────────────────────────────────────────┐    │
│  │  sed -i.bak 's/old/new/g' file.txt                   │    │
│  │                                                       │    │
│  │  file.txt is modified.                               │    │
│  │  file.txt.bak is the original.                       │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Summary

Item	Value
`sed`	Stream editor
Basic syntax	`sed [options] 'script' [file...]`
Substitution	`s/pattern/replacement/flags`
Default output	Every line
`-n`	Suppress default printing
`-i[SUFFIX]`	Edit in place
Addresses	Line numbers, regex, ranges
Delete	`d` command
Print	`p` command
Backreferences	`\1`, `\2` in replacement

Key takeaways:

sed is a stream editor. It reads input line by line, applies a script, and writes to standard output. The file on disk is not modified unless the -i option is used. This makes sed non-destructive by default and composable in pipelines .
The substitution command is the most used. The syntax is s/pattern/replacement/flags. The g flag replaces all occurrences, the p flag prints only substituted lines (with -n), and the i flag makes the match case-insensitive. The delimiter can be changed to avoid escaping slashes .
Addresses restrict commands to specific lines. A line number, a range, or a regular expression can precede a command. The ! negates the address. The /start/,/end/ range is useful for deleting or editing blocks of text .
The -n option suppresses default printing. By default, sed prints every line. With -n, only lines explicitly printed by the p command are output. This is the pattern for extracting specific lines .
The -i option edits files in place. On GNU sed, -i.bak creates a backup. On BSD/macOS sed, the -i option requires an argument, so sed -i '' means no backup. This difference is a common portability issue .
The d command deletes lines. /^#/d deletes comment lines, /^$/d deletes blank lines, and /^#/d; /^$/d deletes both. The delete command is the standard way to clean up configuration files .
Backreferences allow reuse of matched text. The pattern uses $ and $ to capture groups, and the replacement uses \1, \2 to refer to them. In extended regex mode (-E), the parentheses do not need escaping .
Multiple commands can be combined. The -e option adds a script, and semicolons separate commands in a single script. The -f option reads the script from a file .

Remember: sed is the standard tool for scripted text transformation. It reads input line by line, applies a script to each line, and writes the result to standard output. The substitution command is the most used, but the delete, print, insert, and append commands are equally important. Addresses restrict commands to specific lines, and the -n option suppresses default printing. The -i option edits files in place, and it should be used with a backup. For the LFCS exam, the key patterns are substitution with the g flag, deleting comments and blank lines, and changing configuration values in place. The command is simple, but it is the foundation of automated text processing on Linux.

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!