LFCA 62 🐧 What Log Files Are - KandZ

Every process on a Linux system leaves a trace. When the kernel boots, it records the hardware it detected and the drivers it loaded. When a user logs in, it notes the time and the source. When a service crashes, it writes the error before dying. These traces are log files, and they are the only reliable record of what happened on a system before you arrived to investigate.

The LFCA exam places log management under System Administration Fundamentals, which carries 20% of the total weight . The competency list includes troubleshooting, which is impossible without logs, and security, which depends on them. A user who cannot read a log file cannot diagnose a failed boot, trace a suspicious login, or prove that a service is behaving correctly.

Key point: A log file is a chronological record of events written by the kernel, system services, and applications. The value of a log is not in any single line but in the pattern across many lines. A single “authentication failure” means nothing. A hundred of them in a minute means a brute-force attack. Reading one line tells you something happened. Reading the log tells you what it means.

Why log files exist

A running Linux system is constantly making decisions. A disk write succeeds or fails. A network request arrives or times out. A user authenticates or is rejected. Without a record, these events are invisible. When something goes wrong — and something always goes wrong — the log is the only way to reconstruct what happened.

The troubleshooting problem. A service crashes at 3 AM. By the time an administrator logs in, the service has restarted, the memory state is gone, and the only evidence is whatever the service wrote to its log before dying. Without logs, the root cause is unknowable . The Ubuntu documentation states it plainly: “These logs are invaluable for monitoring and troubleshooting your system” .

The security problem. Failed SSH login attempts, sudo invocations, privilege escalations, and account modifications all leave traces. Security analysts read authentication logs to detect brute-force attacks, identify compromised accounts, and trace an intruder’s movements . The /var/log/auth.log file on Debian/Ubuntu systems records password prompts, sudo usage, and remote logins . Without logs, a breach can go undetected for months.

The compliance problem. Regulations and industry standards require organizations to retain logs of access to sensitive data. An auditor asking “who accessed this file on March 14th” expects an answer, and logs are how the answer is found. The LFCA exam’s Security Fundamentals domain (14%) includes compliance, and compliance depends on logging .

The capacity problem. Logs reveal trends: growing error rates before a disk fills, increasing response times before a service becomes unresponsive, rising authentication failures before a compromised password is guessed. Watching logs proactively turns reactive firefighting into prevention.

The trade-off. Logs consume disk space and generate noise. A busy web server can produce gigabytes of access logs per day. Log rotation, retention policies, and filtering are not optional — they are required to keep logging from becoming a problem in itself.

a. What Gets Logged: The Event Sources

Every subsystem on a Linux machine produces log events. Understanding the sources helps you know which file to check when something breaks.

The kernel logs hardware events, driver errors, memory management issues, and the boot sequence. On Debian/Ubuntu systems, these messages go to /var/log/kern.log . On Red Hat-family systems, they go to /var/log/messages. The dmesg command prints the kernel ring buffer directly, and journalctl -k shows the same messages when systemd is in use .

System daemons — programs that run in the background without user interaction — log to /var/log/daemon.log on Debian/Ubuntu systems . This includes the display server, SSH sessions, printing services, and Bluetooth. These are the services that keep the system running, and their logs are where you look when a background process fails.

Authentication services log to /var/log/auth.log on Debian/Ubuntu or /var/log/secure on Red Hat-family systems . This is where sudo usage, SSH logins, password prompts, and account changes are recorded. The Last9 guide identifies this as the primary file for “login attempts, sudo usage, security events” .

Applications log to their own files or subdirectories under /var/log/. Apache writes to /var/log/apache2/access.log and /var/log/apache2/error.log . Nginx writes to /var/log/nginx/. MySQL writes to /var/log/mysql/error.log . Each application decides its own format and location.

The system log (/var/log/syslog on Debian/Ubuntu, /var/log/messages on Red Hat family) is the catch-all. The Ubuntu documentation says: “If you can’t find anything in the other logs, it’s probably here” . This is where messages go when no specific rule directs them elsewhere.

b. The Two Formats: Text and Binary

Not all log files are designed to be read by humans. This distinction determines which tools you can use to read them.

Text logs are plain text files. You can read them with cat, less, tail, grep, and any text-processing tool. /var/log/syslog, /var/log/auth.log, /var/log/kern.log, and /var/log/apache2/access.log are text files . Each line follows a consistent format: timestamp, hostname, process name, and message. The tail -f command follows a text log in real time, showing new lines as they are appended .

Binary logs are not human-readable. They are structured data files that require specific tools to parse. /var/log/wtmp records login sessions and is read by the who command . /var/log/faillog records failed logins and is read by the faillog command. /var/log/lastlog records the last login for each user and is read by the lastlog command .

The systemd journal is also a binary format. It is stored in /var/run/systemd/journal/ and /var/log/journal/ . The journalctl command is the only way to read it. The journal contains structured fields — process ID, user ID, executable path, systemd unit — that plain text logs do not have. This is why journalctl supports filtering by service, priority, and time in ways that grep cannot match .

The practical implication: when you encounter a log file you cannot read with cat or less, it is almost certainly a binary format. Check the documentation for the corresponding command, or use file to identify the format.

c. Where Logs Come From: rsyslog and journald

Two logging systems coexist on modern Linux. Understanding them explains why logs appear in two places and why some events are in one but not the other.

rsyslog is the traditional syslog daemon. It receives messages from the kernel and applications, applies rules from /etc/rsyslog.conf, and writes them to files under /var/log/ . The rule *.* /var/log/syslog means “everything goes to syslog.” The rule kern.* /var/log/kern.log means “kernel messages go to kern.log.” rsyslog is why text log files exist and why they are organized the way they are.

systemd-journald is the newer logging system. It captures messages from the kernel, from systemd services, and from applications that use the journal API. It stores them in a binary format with structured metadata . The journalctl command queries the journal. On many distributions, journald is configured to forward messages to rsyslog, so the same event appears in both the journal and the text files . On others, journald is the only logging system, and no text files exist.

The LFCA exam expects you to know that both systems exist and that journalctl is the command for querying systemd logs . The LPI learning materials confirm that by default, client logs are written to the server’s /var/log/messages file, showing how rsyslog centralizes logging .

Complete Example Session

This session demonstrates viewing text logs, reading binary logs, and understanding where different messages come from.

# ============================================
# PART 1: VIEWING THE SYSTEM LOG
# ============================================

# View the last 20 lines of the system log
tail -n 20 /var/log/syslog

# Follow the log in real time
tail -f /var/log/syslog

# ============================================
# PART 2: VIEWING AUTHENTICATION LOGS
# ============================================

# View recent authentication events
tail -n 50 /var/log/auth.log

# Search for sudo usage
grep sudo /var/log/auth.log

# Search for failed SSH logins
grep "Failed password" /var/log/auth.log

# ============================================
# PART 3: VIEWING KERNEL LOGS
# ============================================

# View kernel messages from the ring buffer
dmesg | tail -n 20

# Same messages via journalctl
journalctl -k --no-pager | tail -n 20

# ============================================
# PART 4: READING BINARY LOGS
# ============================================

# Who is currently logged in (reads /var/log/wtmp)
who

# Last logins for all users (reads /var/log/lastlog)
lastlog

# Failed login attempts (reads /var/log/faillog)
faillog -a

# ============================================
# PART 5: IDENTIFYING LOG FILE TYPES
# ============================================

# Check if a log file is text or binary
file /var/log/syslog        # ASCII text
file /var/log/wtmp          # data (binary)

# ============================================
# PART 6: SEARCHING LOGS BY TIME
# ============================================

# Search syslog for a specific hour
grep "May  7 15:" /var/log/syslog

# Search auth.log for the last hour
grep "$(date +'%b %e %H')" /var/log/auth.log

# ============================================
# PART 7: COUNTING EVENTS
# ============================================

# Count failed SSH logins
grep -c "Failed password" /var/log/auth.log

# Count sudo invocations
grep -c "sudo" /var/log/auth.log

# ============================================
# PART 8: VIEWING APPLICATION LOGS
# ============================================

# Apache access log (if Apache is installed)
tail -n 10 /var/log/apache2/access.log

# Nginx error log (if Nginx is installed)
tail -n 10 /var/log/nginx/error.log

# ============================================
# PART 9: THE JOURNAL
# ============================================

# View the journal (paged through less)
journalctl

# View the journal without paging
journalctl --no-pager | head -n 20

# View messages from the current boot
journalctl -b

# ============================================
# PART 10: WHAT TO CHECK WHEN SOMETHING BREAKS
# ============================================

# Service failed to start?
journalctl -u servicename.service

# Authentication issue?
tail -n 50 /var/log/auth.log

# Hardware issue?
dmesg | tail -n 30

# General system issue?
tail -n 100 /var/log/syslog

The ten parts cover viewing the system log, authentication logs, kernel logs, reading binary logs, identifying file types, searching by time, counting events, viewing application logs, the journal, and a troubleshooting checklist.

Quick Reference

The Common Log Files

File	Purpose	Format
`/var/log/syslog`	General system messages (Debian/Ubuntu)	Text
`/var/log/messages`	General system messages (RHEL family)	Text
`/var/log/auth.log`	Authentication events (Debian/Ubuntu)	Text
`/var/log/secure`	Authentication events (RHEL family)	Text
`/var/log/kern.log`	Kernel messages (Debian/Ubuntu)	Text
`/var/log/daemon.log`	Daemon messages (Debian/Ubuntu)	Text
`/var/log/wtmp`	Login records	Binary
`/var/log/faillog`	Failed logins	Binary
`/var/log/lastlog`	Last login per user	Binary
`/var/log/journal/`	systemd journal	Binary

The Logging Systems

System	Command	Storage	Format
rsyslog	`tail`, `grep`, `less`	`/var/log/*.log`	Text
systemd-journald	`journalctl`	`/var/log/journal/`	Binary

The Diagnostic Commands

Command	Reads	Purpose
`tail -f file`	Any text log	Follow in real time
`grep pattern file`	Any text log	Search for pattern
`dmesg`	Kernel ring buffer	Kernel messages
`journalctl -b`	systemd journal	Current boot
`journalctl -u name`	systemd journal	Specific service
`who`	`/var/log/wtmp`	Current logins
`lastlog`	`/var/log/lastlog`	Last login per user

Best Practices

✅ Do This:

# Use tail -f to follow logs in real time during debugging
tail -f /var/log/syslog                                    # ✅

# Search for specific patterns with grep
grep "Failed password" /var/log/auth.log                    # ✅

# Use journalctl for systemd service logs
journalctl -u nginx.service --since "1 hour ago"            # ✅

# Check both the journal and text logs
journalctl -u ssh --no-pager; tail /var/log/auth.log        # ✅

# Use the right file for the right problem
tail /var/log/kern.log    # kernel issue
tail /var/log/auth.log    # authentication issue            # ✅

❌ Don’t Do This:

# Don't try to cat binary logs
cat /var/log/wtmp                                           # ❌ garbage output

# Don't assume all logs are in /var/log/
# Some applications use /opt/, /var/lib/, or their own directories # ❌

# Don't ignore the journal because you found text files
# The journal has structured data that text files lack          # ❌

# Don't grep without understanding the format
grep "error" /var/log/binary.log                            # ❌ won't work

Common Pitfalls

Pitfall	Why It Happens	Fix
Cannot read log file	Binary format	Use `who`, `lastlog`, `journalctl`
Log seems empty	journald not forwarding to syslog	Use `journalctl --no-pager`
Missing kernel messages	Wrong file for distribution	Check `/var/log/kern.log` or `dmesg`
Authentication logs missing	Looking in `/var/log/syslog`	Check `/var/log/auth.log` or `secure`
Journal rotated out	SystemMaxUse limit	Configure longer retention or remote logging

Real-World Examples

1. Check Who Logged In

who

2. Check Failed SSH Logins

grep "Failed password" /var/log/auth.log

3. Check Sudo Usage

grep sudo /var/log/auth.log

4. Check Kernel Messages

dmesg | tail -n 30

5. Check Service Status

journalctl -u nginx.service --since "1 hour ago"

6. Check Boot Messages

journalctl -b

7. Follow a Log in Real Time

tail -f /var/log/syslog

8. Check Last Logins

lastlog

9. Check Apache Errors

tail /var/log/apache2/error.log

10. Count Authentication Failures

grep -c "authentication failure" /var/log/auth.log

Visual

The Logging Landscape

┌──────────────────────────────────────────────┐
│  LOGGING SYSTEMS                             │
│                                              │
│  Kernel ──────┐                              │
│  Daemons ─────┼──> rsyslog ──> /var/log/*.log│
│  Apps ────────┘                              │
│                                              │
│  Kernel ──────┐                              │
│  Daemons ─────┼──> journald ──> /var/log/journal│
│  Apps ────────┘                              │
│                                              │
│  On many systems, journald forwards to       │
│  rsyslog, so logs appear in both places.     │
│                                              │
└──────────────────────────────────────────────┘

The File by Purpose

┌──────────────────────────────────────────────┐
│  WHICH FILE FOR WHICH PROBLEM                │
│                                              │
│  Authentication → /var/log/auth.log          │
│                   /var/log/secure            │
│                                              │
│  Kernel/Hardware → /var/log/kern.log         │
│                    dmesg                     │
│                                              │
│  General system → /var/log/syslog            │
│                   /var/log/messages          │
│                                              │
│  Service logs → journalctl -u name           │
│                                              │
│  Login records → who, lastlog, faillog       │
│                                              │
└──────────────────────────────────────────────┘

Text vs Binary

┌──────────────────────────────────────────────┐
│  TEXT vs BINARY LOGS                         │
│                                              │
│  TEXT:                                       │
│  ├─ /var/log/syslog                          │
│  ├─ /var/log/auth.log                        │
│  ├─ /var/log/kern.log                        │
│  └─ Application logs                         │
│                                              │
│  Read with: cat, less, tail, grep            │
│                                              │
│  BINARY:                                     │
│  ├─ /var/log/wtmp      → who                 │
│  ├─ /var/log/faillog   → faillog             │
│  ├─ /var/log/lastlog   → lastlog             │
│  └─ systemd journal    → journalctl          │
│                                              │
│  Read with: the corresponding command        │
│                                              │
└──────────────────────────────────────────────┘

The First-Hour Troubleshooting Query

┌──────────────────────────────────────────────┐
│  FIRST-HOUR TROUBLESHOOTING                  │
│                                              │
│  1. What service failed?                     │
│     journalctl -u servicename --no-pager     │
│                                              │
│  2. Who logged in?                           │
│     journalctl -u ssh --since "1 hour ago"   │
│                                              │
│  3. What errors fired?                       │
│     journalctl -p err --since "1 hour ago"   │
│                                              │
│  4. What does the system log say?            │
│     tail -100 /var/log/syslog                │
│                                              │
│  5. Any hardware issues?                     │
│     dmesg | tail -30                         │
│                                              │
└──────────────────────────────────────────────┘

Summary

Item	Value
Text system log	`/var/log/syslog` (Debian/Ubuntu), `/var/log/messages` (RHEL)
Authentication log	`/var/log/auth.log`, `/var/log/secure`
Kernel log	`/var/log/kern.log`, `dmesg`
Binary login records	`/var/log/wtmp`, `/var/log/faillog`, `/var/log/lastlog`
systemd journal	Binary, read with `journalctl`
Logging daemons	rsyslog (text), systemd-journald (binary)
Follow in real time	`tail -f`
Search	`grep pattern file`
Journal filtering	`journalctl -u`, `journalctl -b`, `journalctl -p`

Key takeaways:

Log files are chronological records of events. They are the only reliable evidence of what happened on a system. Without them, troubleshooting is guesswork and security incidents go undetected.
Different subsystems write to different files. Authentication goes to /var/log/auth.log or /var/log/secure. Kernel messages go to /var/log/kern.log or dmesg. General system messages go to /var/log/syslog or /var/log/messages. Knowing which file to check is half the diagnostic work.
Not all logs are text. /var/log/wtmp, /var/log/faillog, and /var/log/lastlog are binary files read by who, faillog, and lastlog. The systemd journal is also binary and is read by journalctl.
Two logging systems coexist. rsyslog writes text files under /var/log/. systemd-journald stores structured binary data queried by journalctl. On many systems, journald forwards to rsyslog so logs appear in both places.
tail -f is the real-time debugging tool. It follows a text log and prints new lines as they appear. journalctl -f does the same for the binary journal.
journalctl provides filtering that grep cannot. You can filter by service (-u), by boot (-b), by priority (-p), and by time (--since). This is why the journal exists alongside text logs.
The first hour of troubleshooting has a pattern. Check the service log, check authentication, check for errors, check the system log, check kernel messages. Every experienced administrator runs the same sequence.

Remember: Logs are the memory of a Linux system. They record what happened, when it happened, and who or what caused it. The kernel writes to the kernel log. Authentication services write to auth.log. Applications write to their own directories. systemd stores everything in a structured journal that journalctl queries. When something breaks, the log tells you why. When something suspicious happens, the log tells you who. Learn which file to read for which problem, learn the difference between text and binary logs, and learn the journalctl filters. That knowledge turns a mystery into a diagnosis.

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!