Node.js 15 🟢 Working with the Native File System Module (node:fs) - KandZ

The node:fs module is Node.js’s interface to the file system. It provides methods for reading, writing, renaming, deleting, and inspecting files and directories. Every method exists in three forms: synchronous, callback-based, and promise-based. The synchronous forms block the event loop and are intended for startup and scripts. The callback forms use the error-first convention and run on the libuv thread pool. The promise forms, available from node:fs/promises, wrap the callback API and work with async/await.

Choosing the right form is the first decision when working with files. A script that reads a configuration file at startup can use the synchronous form safely because nothing else is running yet. A server that reads a file in a request handler must use the asynchronous form, because a synchronous read would block the event loop and stall every other request. Understanding which operations use the thread pool and which are handled by the event loop explains why file I/O scales differently from network I/O.

This chapter covers the three API forms, reading and writing files, working with directories, file metadata, streams for large files, watching for changes, and the patterns that avoid blocking the event loop.

Key point: Use node:fs/promises for asynchronous file operations in application code. Use the synchronous API only at startup or in scripts. File operations run on the libuv thread pool, so the pool size limits concurrency. Use streams for large files to avoid loading the entire contents into memory. Always handle errors, and prefer fs/promises with await for readability.

Why fs has three API forms

The blocking problem. The synchronous API is the simplest to read: readFileSync returns the file contents directly. But it blocks the event loop until the read completes, which is unacceptable in a server that handles concurrent requests. The synchronous API exists for cases where blocking does not matter: startup, scripts, and command-line tools.

The callback problem. The callback API is asynchronous but uses the error-first convention, which nests when operations are sequenced. It was the original asynchronous API and is still present throughout the module.

The promise problem. The promise API, in node:fs/promises, wraps the callback API and returns promises. It works with async/await, has a single error channel, and reads like synchronous code. This is the recommended API for application code.

The thread pool problem. File operations do not have a portable non-blocking OS primitive, so libuv runs them on the thread pool. The pool has four threads by default, so more than four concurrent file operations queue. Understanding this explains why file-heavy workloads can saturate the pool and why increasing UV_THREADPOOL_SIZE helps.

The memory problem. Reading a large file with readFile loads the entire contents into memory. For files that are hundreds of megabytes, this can exhaust memory. Streams read the file in chunks and process each chunk as it arrives, keeping memory usage bounded.

a. The three API forms

The node:fs module exports the callback API directly. The node:fs/promises module exports the promise API.

// Callback API
const fs = require('node:fs');

// Promise API
const fsPromises = require('node:fs/promises');

The synchronous methods are named with a Sync suffix and are available on the callback module:

const fs = require('node:fs');

// Synchronous
const data = fs.readFileSync('file.txt', 'utf8');

// Callback
fs.readFile('file.txt', 'utf8', (err, data) => {
  if (err) throw err;
  console.log(data);
});

The promise API:

const { readFile } = require('node:fs/promises');

async function main() {
  const data = await readFile('file.txt', 'utf8');
  console.log(data);
}

Form	Import	Blocking	Recommended for
Synchronous	`node:fs`	Yes	Startup, scripts
Callback	`node:fs`	No	Legacy code
Promise	`node:fs/promises`	No	Application code

b. Reading and writing files

The primary methods are readFile and writeFile. Both accept a path, the data or options, and a callback or return a promise.

const { readFile, writeFile } = require('node:fs/promises');

async function main() {
  const data = await readFile('input.txt', 'utf8');
  await writeFile('output.txt', data.toUpperCase());
}

readFile returns a Buffer if no encoding is specified, and a string if an encoding is given. The encoding 'utf8' is the common case for text files.

const buffer = await readFile('image.png');       // Buffer
const text = await readFile('file.txt', 'utf8');  // string

writeFile creates the file if it does not exist and overwrites it if it does. The data can be a string or a Buffer.

await writeFile('file.txt', 'hello');
await writeFile('file.bin', Buffer.from([1, 2, 3]));

For appending instead of overwriting, use appendFile:

await appendFile('log.txt', 'new line\n');

For lower-level control, open returns a file handle, and the handle provides read, write, and close:

const file = await open('file.txt', 'r');
try {
  const buffer = Buffer.alloc(1024);
  const { bytesRead } = await file.read(buffer, 0, 1024, 0);
  console.log(bytesRead);
} finally {
  await file.close();
}

The try/finally ensures the file is closed even if an error occurs.

c. Working with directories

The directory methods mirror the file methods: mkdir, rmdir, readdir, and stat.

const { mkdir, readdir, rm, stat } = require('node:fs/promises');

// Create a directory (and parents)
await mkdir('path/to/dir', { recursive: true });

// List directory contents
const files = await readdir('.');

// Remove a directory (and contents)
await rm('path/to/dir', { recursive: true, force: true });

// Get file or directory metadata
const stats = await stat('file.txt');
console.log(stats.isFile(), stats.isDirectory(), stats.size);

The recursive: true option on mkdir creates parent directories as needed and does not throw if the directory already exists. The rm method is the modern replacement for unlink and rmdir, and with recursive: true it removes directories and their contents.

readdir can return Dirent objects instead of names when the withFileTypes option is set:

const entries = await readdir('.', { withFileTypes: true });

for (const entry of entries) {
  if (entry.isDirectory()) {
    console.log('dir:', entry.name);
  } else if (entry.isFile()) {
    console.log('file:', entry.name);
  }
}

The Dirent objects avoid a separate stat call for each entry, which is more efficient for large directories.

d. File metadata

The stat method returns a Stats object with information about a file or directory.

Property	Meaning
`size`	Size in bytes
`mode`	File mode (permissions)
`mtime`	Last modification time
`atime`	Last access time
`ctime`	Last status change time
`birthtime`	Creation time

Method	Returns
`isFile()`	True if a regular file
`isDirectory()`	True if a directory
`isSymbolicLink()`	True if a symbolic link
`isSocket()`	True if a socket

const stats = await stat('file.txt');

if (stats.isFile()) {
  console.log('size:', stats.size);
  console.log('modified:', stats.mtime);
}

The lstat method is like stat but does not follow symbolic links, so it reports on the link itself rather than its target.

const linkStats = await lstat('link-to-file');
console.log(linkStats.isSymbolicLink());  // true

The access method checks whether a file exists and whether the process has permission to read, write, or execute it:

const { access, constants } = require('node:fs/promises');

try {
  await access('file.txt', constants.R_OK | constants.W_OK);
  console.log('readable and writable');
} catch {
  console.log('not accessible');
}

e. Streams for large files

Reading a large file with readFile loads the entire contents into memory. For files that are hundreds of megabytes, this can exhaust the process’s memory. Streams read the file in chunks and process each chunk as it arrives.

const fs = require('node:fs');

const stream = fs.createReadStream('large.txt', { encoding: 'utf8' });

stream.on('data', (chunk) => {
  process.stdout.write('.');
});

stream.on('end', () => {
  console.log('\ndone');
});

stream.on('error', (err) => {
  console.error('error:', err);
});

The data event fires for each chunk, and the end event fires when the file is fully read. The error event is mandatory; without it, an error crashes the process.

The pipe method connects a read stream to a write stream, handling backpressure automatically:

fs.createReadStream('input.txt')
  .pipe(fs.createWriteStream('output.txt'));

Backpressure is the mechanism that prevents a fast reader from overwhelming a slow writer. When the write stream’s buffer is full, the read stream pauses until the buffer drains.

For promise-based code, the pipeline function from node:stream/promises wraps the pipe chain in a promise:

const { pipeline } = require('node:stream/promises');
const { createReadStream, createWriteStream } = require('node:fs');

await pipeline(
  createReadStream('input.txt'),
  createWriteStream('output.txt')
);

The pipeline function propagates errors and cleans up streams if any step fails.

f. Watching for changes

The watch method monitors a file or directory and emits an event when it changes. It returns a FSWatcher that emits change and error events.

const fs = require('node:fs');

const watcher = fs.watch('config.json', (eventType, filename) => {
  console.log(`${eventType}: ${filename}`);
});

// Later, to stop watching:
// watcher.close();

The eventType is 'rename' or 'change'. The filename is the name of the file that changed, though it may be null on some platforms.

The watch method is not consistent across platforms. On macOS and Windows it uses the native file system events, and on Linux it uses inotify. The exact events and the timing differ, so the method is best suited for triggering a re-read rather than for precise change tracking.

The watchFile method polls the file for changes instead of using native events:

fs.watchFile('config.json', { interval: 1000 }, (curr, prev) => {
  console.log('file changed');
});

Polling is less efficient but more portable. The unwatchFile method stops the polling.

Complete Example Session

// ============================================
// PART 1: SYNCHRONOUS READ
// ============================================
const fs = require('node:fs');

const data = fs.readFileSync('file.txt', 'utf8');
console.log(data);

// ============================================
// PART 2: CALLBACK READ
// ============================================
fs.readFile('file.txt', 'utf8', (err, data) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log(data);
});

// ============================================
// PART 3: PROMISE READ
// ============================================
const { readFile } = require('node:fs/promises');

async function main() {
  const data = await readFile('file.txt', 'utf8');
  console.log(data);
}

main();

// ============================================
// PART 4: WRITE AND APPEND
// ============================================
const { writeFile, appendFile } = require('node:fs/promises');

await writeFile('output.txt', 'first line\n');
await appendFile('output.txt', 'second line\n');

// ============================================
// PART 5: DIRECTORY OPERATIONS
// ============================================
const { mkdir, readdir, rm } = require('node:fs/promises');

await mkdir('build/assets', { recursive: true });
const files = await readdir('build');
console.log(files);
await rm('build', { recursive: true, force: true });

// ============================================
// PART 6: LIST WITH DIRENT
// ============================================
const { readdir } = require('node:fs/promises');

const entries = await readdir('.', { withFileTypes: true });

for (const entry of entries) {
  const type = entry.isDirectory() ? 'dir' : 'file';
  console.log(`${type}: ${entry.name}`);
}

// ============================================
// PART 7: FILE METADATA
// ============================================
const { stat, lstat } = require('node:fs/promises');

const stats = await stat('file.txt');
console.log('size:', stats.size);
console.log('is file:', stats.isFile());
console.log('modified:', stats.mtime);

const linkStats = await lstat('link');
console.log('is symlink:', linkStats.isSymbolicLink());

// ============================================
// PART 8: STREAMING A LARGE FILE
// ============================================
const stream = fs.createReadStream('large.txt', { encoding: 'utf8' });

stream.on('data', (chunk) => process.stdout.write('.'));
stream.on('end', () => console.log('\ndone'));
stream.on('error', (err) => console.error(err));

// ============================================
// PART 9: PIPELINE WITH PROMISES
// ============================================
const { pipeline } = require('node:stream/promises');
const { createReadStream, createWriteStream } = require('node:fs');

await pipeline(
  createReadStream('input.txt'),
  createWriteStream('output.txt')
);
console.log('copied');

// ============================================
// PART 10: WATCHING A FILE
// ============================================
const watcher = fs.watch('config.json', (eventType, filename) => {
  console.log(`${eventType}: ${filename}`);
});

// Stop watching after 10 seconds
setTimeout(() => watcher.close(), 10000);

These ten parts cover synchronous read, callback read, promise read, write and append, directory operations, listing with Dirent, file metadata, streaming a large file, pipeline with promises, and watching a file.

Quick Reference

API Forms

Form	Import	Blocking
Synchronous	`node:fs`	Yes
Callback	`node:fs`	No
Promise	`node:fs/promises`	No

Reading and Writing

Method	Purpose
`readFile(path, encoding)`	Read entire file
`writeFile(path, data)`	Write or overwrite
`appendFile(path, data)`	Append to file
`open(path, flags)`	Open a file handle
`copyFile(src, dest)`	Copy a file
`rename(old, new)`	Rename or move
`unlink(path)`	Delete a file

Directories

Method	Purpose
`mkdir(path, { recursive })`	Create directory
`readdir(path, { withFileTypes })`	List contents
`rm(path, { recursive, force })`	Remove directory
`rmdir(path)`	Remove empty directory

Metadata

Method	Purpose
`stat(path)`	File info (follows symlinks)
`lstat(path)`	File info (does not follow)
`access(path, mode)`	Check permissions
`existsSync(path)`	Synchronous existence check

Streams

Method	Purpose
`createReadStream(path)`	Read stream
`createWriteStream(path)`	Write stream
`pipeline(...streams)`	Connect and await
`stream.pipe(dest)`	Connect streams

Watching

Method	Purpose
`fs.watch(path, cb)`	Native events
`fs.watchFile(path, cb)`	Polling
`fs.unwatchFile(path)`	Stop polling
`watcher.close()`	Stop watching

Best Practices

✅ Do This:

// Use fs/promises for application code
const { readFile } = require('node:fs/promises');
const data = await readFile('file.txt', 'utf8');

// Use streams for large files
fs.createReadStream('large.txt').pipe(destination);

// Handle errors
try {
  const data = await readFile('file.txt', 'utf8');
} catch (err) {
  console.error(err);
}

// Use recursive for mkdir and rm
await mkdir('a/b/c', { recursive: true });
await rm('a', { recursive: true, force: true });

// Use pipeline for stream chains
await pipeline(readStream, transform, writeStream);

❌ Don’t Do This:

// Block the event loop in a handler
const data = fs.readFileSync('file.txt');  // ❌

// Read a large file with readFile
const data = await readFile('huge.log');  // ❌ memory

// Ignore stream errors
stream.on('data', handler);  // ❌ no error handler

// Assume watch is consistent
fs.watch('dir', cb);  // ⚠️ platform-dependent

// Forget to close file handles
const file = await open('file.txt', 'r');  // ❌ no close

Common Pitfalls

Pitfall	Why It Happens	Fix
Event loop blocked	Synchronous API in handler	Use `fs/promises`
Out of memory	Reading large file	Use streams
Thread pool saturated	Many concurrent file operations	Increase `UV_THREADPOOL_SIZE`
Stream error crashes	No `error` listener	Add `on('error', ...)`
File handle leak	`open` without `close`	Use `try/finally`
`watch` inconsistent	Platform differences	Use `watchFile` or accept differences
`ENOENT` error	File does not exist	Check with `access` or handle the error

Real-World Examples

1. Read JSON Config

const config = JSON.parse(await readFile('config.json', 'utf8'));

2. Write Log Line

await appendFile('app.log', `${new Date().toISOString()} started\n`);

3. List Directory

const files = await readdir('uploads');

4. Filter by Extension

const files = (await readdir('.'))
  .filter((f) => f.endsWith('.js'));

5. Copy a File

await copyFile('source.txt', 'dest.txt');

6. Rename a File

await rename('old.txt', 'new.txt');

7. Check Existence

try {
  await access('file.txt');
  console.log('exists');
} catch {
  console.log('missing');
}

8. Stream a File to HTTP Response

res.writeHead(200, { 'Content-Type': 'text/plain' });
fs.createReadStream('file.txt').pipe(res);

9. Process a Large File Line by Line

const readline = require('node:readline');
const stream = fs.createReadStream('large.log');

const rl = readline.createInterface({ input: stream });
for await (const line of rl) {
  if (line.includes('ERROR')) console.log(line);
}

10. Watch a Directory

fs.watch('src', { recursive: true }, (event, filename) => {
  console.log(`${event}: ${filename}`);
});

Visual

Three API Forms

┌──────────────────────────────────────────────────────────────┐
│  SYNCHRONOUS:                                                │
│  const data = fs.readFileSync('file.txt', 'utf8');           │
│  └── Blocks the event loop                                   │
│                                                              │
│  CALLBACK:                                                   │
│  fs.readFile('file.txt', 'utf8', (err, data) => { });        │
│  └── Async, error-first convention                           │
│                                                              │
│  PROMISE:                                                    │
│  const data = await readFile('file.txt', 'utf8');            │
│  └── Async, async/await friendly                             │
└──────────────────────────────────────────────────────────────┘

Thread Pool

┌──────────────────────────────────────────────────────────────┐
│  fs.readFile → queued to thread pool → worker reads file     │
│       │                                                      │
│       ▼                                                      │
│  Result returned to event loop → callback runs               │
│                                                              │
│  Pool: 4 threads by default                                  │
│  More than 4 concurrent file operations queue                │
└──────────────────────────────────────────────────────────────┘

Streams vs readFile

┌──────────────────────────────────────────────────────────────┐
│  readFile:                                                   │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  Entire file in memory → process → result              │  │
│  │  Memory: full file size                                │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  createReadStream:                                           │
│  ┌────────────────────────────────────────────────────────┐  │
│  │  chunk 1 → process → discard                           │  │
│  │  chunk 2 → process → discard                           │  │
│  │  chunk 3 → process → discard                           │  │
│  │  Memory: one chunk at a time                           │  │
│  └────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────┘

Pipeline

┌──────────────────────────────────────────────────────────────┐
│  await pipeline(                                             │
│    createReadStream('input.txt'),                            │
│    transformStream,                                          │
│    createWriteStream('output.txt')                           │
│  );                                                          │
│                                                              │
│  Errors propagate through the chain.                         │
│  Streams are cleaned up on success or failure.               │
└──────────────────────────────────────────────────────────────┘

Summary

Item	Value
Callback module	`node:fs`
Promise module	`node:fs/promises`
Synchronous	`*Sync` methods, blocks
`readFile`	Read entire file
`writeFile`	Write or overwrite
`appendFile`	Append to file
`mkdir`	Create directory, `recursive: true`
`readdir`	List directory, `withFileTypes`
`rm`	Remove file or directory
`stat`	File metadata
`lstat`	Metadata without following symlinks
Streams	For large files, bounded memory
`pipeline`	Promise-based stream chain
`watch`	Native file events
`watchFile`	Polling

Key takeaways:

Use node:fs/promises for application code. The promise API works with async/await, has a single error channel, and reads like synchronous code. Use the synchronous API only at startup or in scripts.
File operations run on the libuv thread pool. The default pool size is four threads, so more than four concurrent file operations queue. Increase UV_THREADPOOL_SIZE for file-heavy workloads.
Use streams for large files. readFile loads the entire file into memory. Streams process the file in chunks, keeping memory usage bounded and enabling processing to begin before the file is fully read.
Handle errors everywhere. The promise API throws, so try/catch or .catch() is required. The stream API emits error events, and a missing error listener crashes the process.
Use recursive: true for mkdir and rm. It creates parent directories and removes directories with their contents, which avoids a chain of manual operations.
Use readdir with withFileTypes to avoid extra stat calls. The Dirent objects report whether each entry is a file or directory, which is more efficient for large directories.
Use pipeline for stream chains. It propagates errors, cleans up streams, and returns a promise, which makes it usable with await.
watch is platform-dependent. The events and timing differ across platforms. Use it to trigger a re-read rather than for precise change tracking, or use watchFile for portable polling.

Remember: The node:fs module is Node.js’s interface to the file system, and it offers three API forms for every operation. The synchronous form is for startup and scripts, the callback form is legacy, and the promise form is the recommended choice for application code. File operations run on the libuv thread pool, so concurrency is limited by the pool size, and the pool can become a bottleneck in file-heavy workloads. For large files, streams are the correct tool because they keep memory usage bounded and process data incrementally. Always handle errors: a missing error handler on a stream crashes the process, and an unhandled promise rejection does the same in modern Node.js. The module is comprehensive, covering files, directories, metadata, streams, and watching, and understanding which form and which method to use for each situation is what makes file handling correct and efficient.

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!