LFCS 129 🐧 journalctl — Reading System Logs - KandZ

⏱️ Estimated read: 9 min | |

The previous chapters covered shell scripting and its debugging mechanisms. This chapter returns to system administration: the journalctl command, which is the interface to the systemd journal. Unlike traditional text-based log files (/var/log/syslog, /var/log/messages), the journal is a binary store. It is not read with cat, grep, or less directly. The journalctl command is the only supported way to query it. The LFCS exam tests the filtering options: by boot, by unit, by priority, by time, and the live-follow mode. The critical distinction is between the persistent and volatile storage locations, and the permission model that governs who can read the journal.

Key point: The journal is stored in /var/log/journal/ (persistent, survives reboot) or /run/log/journal/ (volatile, lost on reboot). The journalctl command reads both. Without arguments, it displays all collected logs. The -b flag limits to the current boot, -u limits to a systemd unit, -p limits to a priority level, and --since/--until limit to a time range. The -f flag follows the journal in real time.

Why journalctl requires a specific mental model

The binary format problem. The journal is not a text file. It is a structured binary store that indexes every log entry by fields: _SYSTEMD_UNIT, _PID, PRIORITY, _HOSTNAME, _COMM, and dozens of others. The grep command on the raw file does not work because the file contains binary metadata and compressed messages. The journalctl command is the parser and the query engine . The exam tests the filtering options, not the raw file format.

The permission problem. By default, only root and members of the systemd-journal, adm, and wheel groups can read the full journal. A regular user can read their own user journal but not the system journal. The error is Permission denied or No journal files were found. The fix is sudo or adding the user to the systemd-journal group with usermod -aG systemd-journal $USER . The exam tests the permission model because a candidate who runs journalctl without sudo may see only a subset of the logs.

The boot problem. The journal accumulates entries across reboots if persistent storage is enabled. The -b flag filters to a specific boot. Without it, the output includes every boot that is still stored. The -b (current boot), -b -1 (previous boot), and -b 1 (first boot in the journal) are the mechanisms for navigating the boot history .

The priority problem. The -p flag filters by syslog priority: emerg (0), alert (1), crit (2), err (3), warning (4), notice (5), info (6), debug (7). The filter is inclusive: -p err shows err, crit, alert, and emerg messages. It does not show warning or info . The exam tests the inclusive behavior.

The trade-off. The journalctl command is the only way to read the journal. The alternative is to configure rsyslog or syslog-ng to write text files, but the journal is the primary store on systemd-based systems. The exam tests the journal because it is the default on RHEL, Fedora, Ubuntu, and Debian.

a. Basic queries: recent entries, follow mode, and boot filtering

The journalctl command without arguments displays every entry in the journal, from the oldest to the newest. The output is paginated through less by default. The --no-pager flag disables the pager, which is useful for scripts .

# Show all entries (paginated)
journalctl

# Show the most recent 20 entries
journalctl -n 20

# Follow the journal in real time
journalctl -f

# Show entries from the current boot
journalctl -b

# Show entries from the previous boot
journalctl -b -1

The -n flag limits the number of entries. The -f flag follows the journal, printing new entries as they arrive. The -b flag filters to a specific boot. The -b -1 is the previous boot, which is useful for diagnosing a crash that occurred before the reboot .

The -r flag reverses the order, showing the newest entries first. This is the equivalent of tail for the journal.

# Show the newest entries first
journalctl -r

b. Filtering by unit, priority, and time

The -u flag filters by systemd unit. The unit name is the service name, such as sshd.service or nginx.service. The filter is exact; it does not match partial names.

# Show entries from the SSH service
journalctl -u sshd.service

# Show entries from the Nginx service
journalctl -u nginx.service

# Follow a specific service
journalctl -u postgresql.service -f

The -p flag filters by priority. The value can be a name or a number. The filter is inclusive: -p err includes err, crit, alert, and emerg .

# Show errors and above
journalctl -p err

# Show critical and above
journalctl -p crit

The --since and --until flags filter by time. The value can be an absolute timestamp ("2026-10-29 09:00:00"), a relative time ("1 hour ago", "yesterday", "today"), or "now". The --since is inclusive; the --until is exclusive .

# Show entries from the last hour
journalctl --since "1 hour ago"

# Show entries from a specific time range
journalctl --since "2026-10-29 09:00" --until "2026-10-29 10:00"

# Show entries from today
journalctl --since today

The filters can be combined. The journalctl -u nginx -p err --since "1 hour ago" shows only error-level entries from Nginx in the last hour.

c. Persistent storage and configuration

The journal’s storage location determines whether logs survive a reboot. The default is auto, which uses /var/log/journal/ if the directory exists, and /run/log/journal/ otherwise. The /run directory is a tmpfs, so the logs are lost on reboot .

# Check the storage location
ls /var/log/journal/
# If the directory exists, persistent storage is active.

To enable persistent storage, create the directory and set the correct permissions. The systemd-tmpfiles --create --prefix /var/log/journal command creates the directory structure with the correct ownership and mode .

# Enable persistent storage
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal

The journald.conf file controls the storage behavior. The Storage= setting accepts volatile (memory only), persistent (disk preferred), auto (disk if directory exists), and none (discard). The SystemMaxUse= setting limits the disk usage. The RateLimitIntervalSec= and RateLimitBurst= settings control the rate limiting .

# /etc/systemd/journald.conf
[Journal]
Storage=persistent
SystemMaxUse=1G

After a configuration change, the systemctl restart systemd-journald command applies the new settings. The journalctl --disk-usage command shows the current disk usage. The journalctl --vacuum-size=500M command removes the oldest entries to bring the usage under the limit .

Complete Example Session

# ============================================
# PART 1: BASIC QUERY
# ============================================

journalctl -n 10
# Shows the 10 most recent entries.

# ============================================
# PART 2: FOLLOW MODE
# ============================================

journalctl -f
# Follows the journal. Press Ctrl+C to exit.

# ============================================
# PART 3: BOOT FILTER
# ============================================

journalctl -b
# Shows entries from the current boot.

# ============================================
# PART 4: PREVIOUS BOOT
# ============================================

journalctl -b -1
# Shows entries from the previous boot.

# ============================================
# PART 5: UNIT FILTER
# ============================================

journalctl -u sshd.service
# Shows entries from the SSH service.

# ============================================
# PART 6: PRIORITY FILTER
# ============================================

journalctl -p err
# Shows error-level and above.

# ============================================
# PART 7: TIME FILTER
# ============================================

journalctl --since "1 hour ago"
# Shows entries from the last hour.

# ============================================
# PART 8: COMBINED FILTER
# ============================================

journalctl -u nginx -p err --since "today"
# Shows Nginx errors from today.

# ============================================
# PART 9: DISK USAGE AND VACUUM
# ============================================

journalctl --disk-usage
# Shows the current journal size.

sudo journalctl --vacuum-size=500M
# Removes oldest entries until under 500 MB.

The nine parts trace the journalctl workflow: basic query, follow mode, boot filter, previous boot, unit filter, priority filter, time filter, combined filter, and disk management.

Quick Reference

Filtering Options

Option	Purpose
`-n N`	Show N most recent entries
`-f`	Follow in real time
`-b`	Current boot
`-b -1`	Previous boot
`-u <unit>`	Filter by systemd unit
`-p <priority>`	Filter by priority
`--since <time>`	From time
`--until <time>`	To time
`-r`	Reverse order (newest first)

Priority Levels

Level	Number	Meaning
`emerg`	0	System is unusable
`alert`	1	Action must be taken
`crit`	2	Critical conditions
`err`	3	Error conditions
`warning`	4	Warning conditions
`notice`	5	Normal but significant
`info`	6	Informational
`debug`	7	Debug-level

Storage Locations

Location	Persistence
`/var/log/journal/`	Persistent (survives reboot)
`/run/log/journal/`	Volatile (lost on reboot)

Configuration

File	Purpose
`/etc/systemd/journald.conf`	Journald configuration
`Storage=`	`volatile`, `persistent`, `auto`, `none`
`SystemMaxUse=`	Disk usage limit

Best Practices

✅ Do This:

sudo journalctl -u sshd.service -p err
<!-- Combine filters for precision -->               # ✅

journalctl --since "1 hour ago" --until "now"
<!-- Limit the time range -->                        # ✅

sudo usermod -aG systemd-journal $USER
<!-- Grant journal read access without sudo -->      # ✅

❌ Don’t Do This:

# grep the binary journal file
<!-- The file is binary and compressed -->           # ❌

# Run journalctl without sudo as a regular user
<!-- Only user journal is visible -->               # ❌

# Assume logs survive reboot without checking storage
<!-- Default may be volatile -->                     # ❌

Common Pitfalls

Pitfall	Why It Happens	Fix
Permission denied	User not in `systemd-journal` group	`sudo` or add to group
No logs from previous boot	Volatile storage	Enable persistent storage
Grep on journal file fails	Binary format	Use `journalctl` with filters
`-p err` shows warnings	Inclusive filter is not inclusive	`err` includes `crit`, `alert`, `emerg`
Journal disk usage grows	No `SystemMaxUse`	Set limit in `journald.conf`

Real-World Examples

1. Recent Entries

journalctl -n 20

2. Follow Mode

journalctl -f

3. Current Boot

journalctl -b

4. Previous Boot

journalctl -b -1

5. Service Filter

journalctl -u nginx.service

6. Priority Filter

journalctl -p err

7. Time Filter

journalctl --since "1 hour ago"

8. Combined Filter

journalctl -u sshd -p err --since today

9. Disk Usage

journalctl --disk-usage

10. Vacuum

sudo journalctl --vacuum-size=500M

Visual

Journal Storage

┌──────────────────────────────────────────────┐
│  /var/log/journal/                           │
│  Persistent storage                          │
│  Survives reboot                             │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  /run/log/journal/                           │
│  Volatile storage (tmpfs)                    │
│  Lost on reboot                              │
└──────────────────────────────────────────────┘

Filter Combination

┌──────────────────────────────────────────────┐
│  journalctl -u nginx -p err --since "1h ago" │
└──────────────────┬───────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────┐
│  Entries from nginx unit                     │
│  AND priority err or above                   │
│  AND timestamp within last hour              │
└──────────────────────────────────────────────┘

Summary

Item	Value
Journal query tool	`journalctl`
Persistent storage	`/var/log/journal/`
Volatile storage	`/run/log/journal/`
Follow mode	`-f`
Boot filter	`-b`
Unit filter	`-u`
Priority filter	`-p`
Time filter	`--since` / `--until`
Configuration file	`/etc/systemd/journald.conf`
Disk usage	`journalctl --disk-usage`

Key takeaways:

journalctl is the only supported way to read the journal. The journal is a binary store. grep and cat on the raw file do not work. The filtering is done by the command, not by external tools .
The boot filter navigates the boot history. The -b flag is the current boot. The -b -1 is the previous boot. The persistent storage must be enabled for previous boots to be available .
The unit filter is exact. The -u sshd.service matches the SSH service. The partial name -u ssh does not match.
The priority filter is inclusive. The -p err includes err, crit, alert, and emerg. It does not include warning or info .
The permission model matters. A regular user cannot read the full system journal without sudo or membership in the systemd-journal group .

Remember: The journal is the system’s structured log store. The journalctl is the query engine. The -u is the service. The -p is the severity. The -b is the boot. The --since is the time. The -f is the live view. The exam tests the filters and the storage model.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!