Bash Scripting 28 🐧 File Locking — Preventing duplicate script execution with flock
A script that runs from cron, a systemd timer, or a user’s terminal can be started again before the previous run has finished. When two instances of the same script run simultaneously, they write to the same files, use the same temporary directories, and compete for the same resources. The result is corrupted output, duplicated work, or a race condition that is difficult to reproduce. The flock command is the standard Linux tool for preventing this. It acquires an advisory lock on a file, and any other process that tries to acquire the same lock either waits or fails immediately, depending on the options.
This chapter covers the two ways to use flock in Bash scripts: wrapping the entire script with the external flock command, and using flock inside the script with a file descriptor. It explains the difference between blocking and non-blocking modes, the timeout option, the self-locking boilerplate that uses the script file itself as the lock, and the pitfall where a killed script leaves the lock held by a child process.
Key point: flock locks are advisory and tied to the open file description, not the file path. The kernel releases the lock automatically when the file descriptor is closed, which happens when the process exits or is killed. This is why flock is safer than a hand-rolled PID file: a crashed script cannot leave a stale lock behind.
Why flock exists
The PID file race problem. A common but flawed approach to preventing duplicate execution is to check for a PID file and create one if it does not exist. The check and the creation are two separate operations, and another process can start between them, see that the file does not exist, and create its own PID file. Both processes then believe they have the lock. The flock system call performs the lock acquisition atomically. There is no window between the check and the acquisition .
The stale lock problem. A PID file can become stale if the script is killed before it removes the file. The next run sees the file, checks the PID, finds that no process with that PID exists, and must decide whether to override the lock. This logic is error-prone, and PIDs are reused, so a stale PID file can point to an unrelated process. flock avoids this entirely: the lock is held by an open file descriptor, and the kernel releases it when the descriptor is closed, regardless of how the process ended .
The advisory lock distinction. flock locks are advisory. They do not prevent another process from opening or writing to the locked file unless that process also calls flock. The lock is a convention between cooperating processes. For the purpose of preventing duplicate script execution, this is sufficient: every instance of the script follows the same convention and attempts to acquire the same lock .
a. Wrapping the script with flock
The simplest way to use flock is to wrap the script invocation in the flock command. The command acquires the lock, runs the script, and releases the lock when the script exits.
# Run the script with an exclusive lock
flock /var/lock/myscript.lock /path/to/myscript.shIf another instance of the script is already running, flock waits for the lock to be released before starting the new instance. This is the blocking behavior, and it is the default. For a cron job, waiting is usually undesirable: if the previous run is still going, the new run should be skipped, not queued. The -n (non-blocking) option makes flock fail immediately if the lock cannot be acquired .
# In a crontab: skip the run if the lock is held
*/5 * * * * /usr/bin/flock -n /var/lock/myscript.lock /path/to/myscript.shWhen the lock is held, flock exits with status 1 (or the value specified by -E) and does not run the command . The cron job completes without doing anything, and the next scheduled run will try again.
For a bounded wait, the -w option specifies a timeout in seconds. Decimal fractional values are accepted .
# Wait up to 30 seconds for the lock
flock -w 30 /var/lock/myscript.lock /path/to/myscript.shIf the lock is not acquired within the timeout, flock exits with a failure status and does not run the command.
b. Using flock inside the script
The external flock command is convenient for cron jobs, but it does not help when the script is invoked in different ways, or when the script needs to log the fact that it was skipped. The alternative is to use flock inside the script with a file descriptor.
The pattern is to open a file descriptor to the lock file in a subshell, call flock on that descriptor, and exit if the lock is not acquired.
#!/bin/bash
(
flock -n 200 || {
echo "Another instance is running" >&2
exit 1
}
# All the script's work goes here
echo "Running with PID $$"
sleep 60
) 200>/var/lock/myscript.lockThe redirection 200>/var/lock/myscript.lock opens file descriptor 200 for writing to the lock file. The flock -n 200 attempts to acquire an exclusive lock on that descriptor. If the lock is already held, flock returns non-zero, and the || branch exits the subshell with status 1 .
The lock is released when the subshell exits, because the file descriptor is closed at that point. The lock file itself remains on disk, but its existence does not mean the lock is held. The lock is a property of the open file description, not the file’s presence .
A more structured version defines a function for the failure case:
#!/bin/bash
another_instance() {
echo "There is another instance running, exiting" >&2
exit 1
}
(
flock -n 100 || another_instance
# Script work here
DEST="/backup/$(date +%s)"
mkdir -p "$DEST"
rsync -avz root@myhost:/home/web "$DEST/."
) 100>/var/lock/dobackup.lockThis is the pattern from the Baeldung example, and it is the most common way to embed locking inside a script .
c. Self-locking boilerplate
A more elegant approach uses the script file itself as the lock file. This avoids a separate lock file and ensures that the lock is always associated with the script’s own path. The boilerplate is placed at the top of the script.
#!/bin/bash
# Self-locking boilerplate
[ "${FLOCKER}" != "$0" ] && exec env FLOCKER="$0" flock -en "$0" "$0" "$@" || :The logic is: if the FLOCKER environment variable is not set to the script’s path, re-execute the script under flock, with FLOCKER set to the script’s path. The flock command acquires an exclusive lock on the script file itself (-e for exclusive, -n for non-blocking), then runs the script again. On the second execution, FLOCKER is set to $0, so the condition is false, and the script proceeds to its actual work .
The -e flag is the default (exclusive lock) and can be omitted. The -n flag makes the lock non-blocking. If the script is already running, the second invocation exits immediately with a non-zero status.
This boilerplate has the advantage of being a single line that can be copied into any script. It does not require a separate lock file, and it works regardless of how the script is invoked.
d. The killed-script pitfall
A known issue with the subshell pattern is that if the script is killed with SIGTERM, the lock may remain held by a child process. The flock lock is associated with the open file description, and file descriptors are inherited by child processes. If the script starts a long-running background process, that child inherits the lock descriptor and keeps the lock alive even after the script exits .
The fix is to ensure that child processes do not inherit the lock descriptor, or to kill the process group when the script is terminated.
#!/bin/bash
set -e
# Ensure the whole process group is killed on termination
trap 'kill 0' SIGINT SIGTERM
exec 7<"$0"
flock -n 7
# Work here
sleep 60The trap 'kill 0' SIGINT SIGTERM sends the signal to the entire process group, including any background children. This ensures that the lock is released when the script is terminated, because all processes holding the descriptor are killed .
An alternative is to use the -o (close) option with the external flock command, which closes the lock file descriptor before running the command. This prevents the command’s children from inheriting the lock, but it also means the lock is not held for the duration of the command, which defeats the purpose. The -o option is useful when the command itself spawns a background process that should not hold the lock .
Complete Example Session
# ============================================
# PART 1: THE SIMPLE WRAPPER
# ============================================
flock /var/lock/myscript.lock /path/to/myscript.sh
# Blocks until the lock is available.
# Runs the script while holding the lock.
# Releases the lock when the script exits.# ============================================
# PART 2: NON-BLOCKING WRAPPER
# ============================================
flock -n /var/lock/myscript.lock /path/to/myscript.sh
# Exits immediately with status 1 if the lock is held.
# Does not run the script.# ============================================
# PART 3: TIMEOUT WRAPPER
# ============================================
flock -w 30 /var/lock/myscript.lock /path/to/myscript.sh
# Waits up to 30 seconds for the lock.
# Exits with failure if the timeout is reached.# ============================================
# PART 4: CRON JOB WITH LOCK
# ============================================
# In crontab:
*/5 * * * * /usr/bin/flock -n /var/lock/backup.lock /usr/local/bin/backup.sh
# The cron job skips the run if the previous one is still going.# ============================================
# PART 5: IN-SCRIPT LOCKING
# ============================================
#!/bin/bash
(
flock -n 200 || {
echo "Another instance is running" >&2
exit 1
}
echo "Running with PID $$"
sleep 60
) 200>/var/lock/myscript.lock# ============================================
# PART 6: THE FUNCTION PATTERN
# ============================================
#!/bin/bash
another_instance() {
echo "Another instance is running" >&2
exit 1
}
(
flock -n 100 || another_instance
# Work here
) 100>/var/lock/dobackup.lock# ============================================
# PART 7: SELF-LOCKING BOILERPLATE
# ============================================
#!/bin/bash
[ "${FLOCKER}" != "$0" ] && exec env FLOCKER="$0" flock -en "$0" "$0" "$@" || :
# The script locks itself on the first run.
# The second run exits immediately.# ============================================
# PART 8: THE KILLED-SCRIPT PITFALL
# ============================================
#!/bin/bash
set -e
trap 'kill 0' SIGINT SIGTERM
exec 7<"$0"
flock -n 7
sleep 60# ============================================
# PART 9: VERIFY THE LOCK
# ============================================
# Start the script in one terminal
./myscript.sh
# In another terminal, try to start it again
./myscript.sh
# Output: Another instance is running
# Exit code: 1# ============================================
# PART 10: CHECK THE LOCK FILE
# ============================================
ls -l /var/lock/myscript.lock
# The file exists, but its existence does not mean the lock is held.
# The lock is held only while a process has the file open.The ten parts covered the simple wrapper, non-blocking mode, timeout, the cron pattern, in-script locking, the function pattern, the self-locking boilerplate, the killed-script pitfall, verification, and the lock file’s existence.
Quick Reference
flock command forms
| Form | Example | Purpose |
|---|---|---|
| Wrap command | flock file command | Lock around a command |
| In-script | ( flock -n 200 || exit 1; ... ) 200>file | Lock inside the script |
| File descriptor | flock -n 200 | Lock an existing descriptor |
| Self-locking | exec env FLOCKER="$0" flock -en "$0" "$0" "$@" | Script locks itself |
Options
| Option | Effect |
|---|---|
-n | Non-blocking, fail if lock held |
-w seconds | Wait up to seconds |
-e | Exclusive lock (default) |
-s | Shared lock |
-E number | Exit code on lock failure |
-o | Close descriptor before running command |
Exit codes
| Code | Meaning |
|---|---|
| 0 | Lock acquired, command succeeded |
| 1 | Lock not acquired (with -n or -w) |
| Other | Command’s exit status |
Best practices
| Practice | Reason |
|---|---|
Use -n for cron | Skip instead of queue |
Use trap 'kill 0' | Release lock on termination |
Use /var/lock or /var/lib | Persistent lock location |
| Do not delete the lock file | Existence does not mean held |
Best Practices
✅ Do This:
# Use -n for cron jobs to skip overlapping runs
flock -n /var/lock/job.lock /path/to/job.sh # ✅# Use a dedicated lock directory
mkdir -p /var/lock/myapp && flock -n /var/lock/myapp/job.lock # ✅# Trap termination to release the lock
trap 'kill 0' SIGINT SIGTERM # ✅# Use the self-locking boilerplate for portability
[ "${FLOCKER}" != "$0" ] && exec env FLOCKER="$0" flock -en "$0" "$0" "$@" || : # ✅❌ Don’t Do This:
# Use a PID file without atomic locking
if [ -f /var/run/job.pid ]; then exit 1; fi; echo $$ > /var/run/job.pid # ❌ (race)# Delete the lock file to "clear" a lock
rm /var/lock/job.lock # ❌ (does not release)# Use a shared lock for mutual exclusion
flock -s /var/lock/job.lock /path/to/job.sh # ❌ (multiple shared locks allowed)# Forget that child processes inherit the lock descriptor
long_running_task & # inside the flock subshell # ❌ (lock held by child)Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Lock held by killed script’s child | Child inherited the descriptor | trap 'kill 0' SIGINT SIGTERM |
| Lock file deleted but lock still held | Lock is on the open file, not the path | Do not delete; wait for process exit |
| Cron job queues instead of skipping | No -n flag | Use flock -n |
| Lock not released | Subshell not used, descriptor not closed | Use the ( ... ) 200>file pattern |
| Self-locking boilerplate fails | $0 is not the script path | Use an absolute path or realpath "$0" |
Real-World Examples
1. Simple lock wrapper
flock /var/lock/job.lock /path/to/job.sh2. Non-blocking lock wrapper
flock -n /var/lock/job.lock /path/to/job.sh3. Timeout lock wrapper
flock -w 60 /var/lock/job.lock /path/to/job.sh4. Cron job with lock
*/5 * * * * flock -n /var/lock/job.lock /path/to/job.sh5. In-script lock
( flock -n 200 || exit 1; work ) 200>/var/lock/job.lock6. Function-based failure
another_instance() { echo "Already running" >&2; exit 1; }
( flock -n 100 || another_instance; work ) 100>/var/lock/job.lock7. Self-locking boilerplate
[ "${FLOCKER}" != "$0" ] && exec env FLOCKER="$0" flock -en "$0" "$0" "$@" || :8. Trap for cleanup
trap 'kill 0' SIGINT SIGTERM9. Exclusive lock
flock -e /var/lock/job.lock /path/to/job.sh10. Check lock status
lslocks | grep job.lockVisual
The wrapper vs in-script pattern
┌─────────────────────────────────────────────────────────────┐
│ WRAPPER │
│ │
│ flock -n /var/lock/job.lock ./job.sh │
│ │ │
│ ├── Lock acquired ──► run job.sh ──► release │
│ │ │
│ └── Lock held ──► exit 1 (job not run) │
│ │
│ ───────────────────────────────────────────────────────────│
│ │
│ IN-SCRIPT │
│ │
│ ( │
│ flock -n 200 || exit 1 │
│ # work │
│ ) 200>/var/lock/job.lock │
│ │ │
│ ├── fd 200 opened to lock file │
│ ├── flock attempts lock on fd 200 │
│ └── subshell exits ──► fd 200 closed ──► lock released │
└─────────────────────────────────────────────────────────────┘Blocking vs non-blocking
┌─────────────────────────────────────────────────────────────┐
│ BLOCKING (default) │
│ │
│ Process A holds lock │
│ │ │
│ ▼ │
│ Process B calls flock │
│ │ │
│ ▼ │
│ Process B WAITS until A releases │
│ │ │
│ ▼ │
│ Process B acquires lock and runs │
│ │
│ ───────────────────────────────────────────────────────────│
│ │
│ NON-BLOCKING (-n) │
│ │
│ Process A holds lock │
│ │ │
│ ▼ │
│ Process B calls flock -n │
│ │ │
│ ▼ │
│ Process B exits immediately with status 1 │
│ (does not run) │
└─────────────────────────────────────────────────────────────┘The killed-script pitfall
┌─────────────────────────────────────────────────────────────┐
│ WITHOUT TRAP │
│ │
│ Script holds lock │
│ │ │
│ ├── starts background child │
│ │ │ │
│ │ └── inherits lock descriptor │
│ │ │
│ ▼ │
│ Script killed with SIGTERM │
│ │ │
│ ▼ │
│ Script exits, but child still holds descriptor │
│ │ │
│ ▼ │
│ Lock remains held until child exits │
│ │
│ WITH TRAP 'kill 0' │
│ │ │
│ ▼ │
│ SIGTERM triggers trap │
│ │ │
│ ▼ │
│ kill 0 sends signal to entire process group │
│ │ │
│ ▼ │
│ All processes exit, descriptor closed, lock released │
└─────────────────────────────────────────────────────────────┘Self-locking flow
┌─────────────────────────────────────────────────────────────┐
│ SELF-LOCKING BOILERPLATE │
│ │
│ First run: │
│ FLOCKER not set ──► exec flock -en "$0" "$0" "$@" │
│ │ │
│ ▼ │
│ Second run (under flock): │
│ FLOCKER="$0" ──► condition false ──► proceed to work │
│ │
│ Concurrent run: │
│ flock fails (non-blocking) ──► exits immediately │
└─────────────────────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
| System call | flock(2) |
| Command | flock (util-linux) |
| Lock type | Advisory, exclusive by default |
| Non-blocking | -n |
| Timeout | -w seconds |
| In-script fd | flock -n FD with FD>file redirection |
| Lock release | Automatic on descriptor close |
| Self-locking | exec env FLOCKER="$0" flock -en "$0" "$0" "$@" |
| Killed-script fix | trap 'kill 0' SIGINT SIGTERM |
| Lock file existence | Does not indicate lock is held |
Key takeaways:
flockis safer than a PID file. The lock is atomic and released automatically when the process exits or is killed. A PID file requires a race-prone check-and-create sequence and can become stale .- Use
-nfor cron jobs. A scheduled run that finds the lock held should skip, not queue. The-nflag makesflockexit immediately with status 1 instead of waiting . - The in-script pattern uses a file descriptor.
( flock -n 200 || exit 1; work ) 200>/var/lock/filelocks the descriptor, runs the work in a subshell, and releases the lock when the subshell exits . - Child processes inherit the lock descriptor. If the script starts a background process, that process keeps the lock alive after the script exits. The
trap 'kill 0' SIGINT SIGTERMpattern kills the entire process group to release the lock . - The self-locking boilerplate is the most portable pattern. It uses the script file itself as the lock file, re-executes under
flockon the first run, and proceeds on the second. It requires no separate lock file management . - Do not delete the lock file to “clear” a lock. The lock is on the open file description, not the path. Deleting the file does not release the lock and can create a race where a new file is locked by a different process .
Remember: The purpose of flock is to serialize access to a resource that cannot be used concurrently. The lock is advisory, so every instance of the script must follow the same convention. The wrapper pattern is simplest for cron jobs. The in-script pattern gives the script control over logging and exit codes. The self-locking boilerplate is the most portable. Whichever pattern you choose, be aware of the child-process inheritance problem: a lock held by a killed script’s child is a lock that never releases. Use trap 'kill 0' to ensure the entire process group terminates, and never delete the lock file to force a release.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!