Cloudflare 12 ☁️ Cloudflare D1 & Hyperdrive — Running serverless edge SQLite with D1 and accelerating external Postgres/MySQL with Hyperdrive
For years, the biggest hurdle for edge computing was data. While Cloudflare Workers could execute code in milliseconds near the user, the data usually lived in a centralized database in a single AWS or Google Cloud region. This created a “data gravity” problem: the compute was at the edge, but the request still had to travel across the globe to fetch a user profile or a product list, negating the speed benefits of the edge. Cloudflare D1 and Hyperdrive are the two primary solutions to this problem, tackling it from different angles: D1 provides a native, edge-distributed SQL database, while Hyperdrive creates a high-speed bridge to existing legacy databases.
D1 is Cloudflare’s first native relational database, built on the battle-tested SQLite engine. It is designed to be truly serverless, meaning you don’t manage instances, clusters, or scaling. It integrates directly into the Worker lifecycle, allowing for SQL queries with zero connection overhead. Hyperdrive, on the other hand, is a specialized connection pooler and cache. It sits between your Worker and your existing Postgres or MySQL database, managing the expensive connection handshakes and caching popular queries at the edge to make a centralized database feel like it’s running locally.
This chapter explores how to manage state at the edge using these two powerful tools. We will examine the architecture of D1, including its migration system and query API. We will then dive into Hyperdrive, explaining how it solves the “connection exhaustion” problem common in serverless environments and how to configure its caching layers. Finally, we will compare the two to help you decide when to go “Native Edge” with D1 or “Accelerated Legacy” with Hyperdrive.
The sections below walk through D1’s setup and querying, Hyperdrive’s pooling and caching mechanics, and the deployment strategies for data-heavy edge applications.
Key point: D1 is Cloudflare’s native serverless SQLite database for edge-first data, while Hyperdrive is a performance-optimizing proxy that makes external Postgres/MySQL databases accessible and fast from serverless Workers.
Why exists
The Data Gravity Problem. Even if your code runs in 300 cities, if your database is only in North Virginia, every global user suffers from the speed of light. D1 and Hyperdrive exist to move data (or the access to it) closer to the execution point.
The Serverless Connection Exhaustion. Databases like Postgres were not designed for the serverless world where thousands of short-lived functions spin up and down. Each new Worker trying to connect can overwhelm the database with connection handshakes. Hyperdrive exists to pool these connections, keeping them warm and reusing them.
The Cold Start Latency. Establishing a secure TLS connection to a database can take hundreds of milliseconds. By the time the connection is ready, the user has already waited too long. Hyperdrive eliminates this by maintaining pre-established connections at the edge.
The SQLite Portability. SQLite is the most widely deployed database in the world, but it was traditionally limited to local files. D1 exists to turn SQLite into a cloud-native, distributed service that retains the simplicity of SQL while gaining the scale of the edge.
The Infrastructure Overhead. Managing a database often requires a dedicated DevOps team for patching, scaling, and backups. D1 exists to provide a “zero-ops” relational database where Cloudflare handles the underlying storage and replication.
The Cache-at-the-Edge Requirement. Many database queries are repetitive. Hyperdrive exists to cache the results of these queries at the Cloudflare edge, so the database doesn’t even have to be contacted for frequent read requests.
a. Cloudflare D1: Native Edge SQL
Cloudflare D1 is built on SQLite, which means it supports standard SQL syntax. When you create a D1 database, you get a globally accessible endpoint that you “bind” to your Worker. Unlike traditional databases, you don’t use a connection string or a driver like pg or mysql2. Instead, you use the native env.DB object provided by the Worker runtime .
// Example: Basic Query in D1
export default {
async fetch(request, env) {
const { pathname } = new URL(request.url);
if (pathname === "/users") {
// D1 uses a prepared statement API
const { results } = await env.DB.prepare(
"SELECT id, name FROM users LIMIT 10"
).all();
return Response.json(results);
}
}
}
D1 handles the storage of the SQLite file in a distributed manner. While the primary “write” location is managed by Cloudflare, the data is replicated to ensure durability. One of D1’s most powerful features is its migration system, which allows you to version-control your schema changes alongside your code .
You manage D1 primarily through wrangler. You create the database, define a migration file, and apply it to either your local development environment or the production cloud.
# Creating a database
npx wrangler d1 create my-db-prod
# Creating a migration
npx wrangler d1 migrations create my-db-prod create_users_table
# Executing migration on production
npx wrangler d1 migrations apply my-db-prod --remote
D1 supports batching, which is crucial for performance. Since every await on a query involves a small amount of overhead, sending multiple queries in a single batch() call allows D1 to execute them together, significantly reducing the total round-trip time .
b. Hyperdrive: Accelerating External Databases
Hyperdrive is not a database itself; it is a “smart proxy.” If you already have a Postgres or MySQL database on AWS, Neon, or Google Cloud, Hyperdrive makes it faster to access from a Worker. It solves two problems: connection overhead and geographic latency .
When a Worker connects to a database through Hyperdrive, it uses a unique Hyperdrive connection string. Hyperdrive keeps a pool of connections “warm” in Cloudflare’s network. When a Worker makes a request, Hyperdrive assigns it an existing connection instead of starting a new TLS handshake with the origin database.
// Example: Using Hyperdrive with a Postgres driver
import { Client } from 'pg';
export default {
async fetch(request, env) {
// env.HYPERDRIVE.connectionString is dynamically
// generated by the Hyperdrive service
const client = new Client(env.HYPERDRIVE.connectionString);
await client.connect();
const result = await client.query('SELECT * FROM products');
return Response.json(result.rows);
}
}
Hyperdrive also includes a query cache. You can configure Hyperdrive to cache the results of specific SELECT queries for a defined period. This means that if 1,000 users in Tokyo request the same product list, only the first request goes back to your origin database in New York; the other 999 are served directly from Cloudflare’s Tokyo data center .
To set up Hyperdrive, you provide your origin database’s connection string to Cloudflare. Hyperdrive then gives you a binding to use in your wrangler.toml.
# Creating a Hyperdrive instance
npx wrangler hyperdrive create my-proxy --connection-string="postgres://user:pass@host:port/db"
c. Schema Management and Data Lifecycle
The lifecycle of data in D1 is managed through migrations. A migration is a .sql file that describes a change. Because D1 is serverless, you cannot just “log in” to a console and run commands manually in production; the migration system ensures that your schema stays in sync across all environments .
-- migrations/0001_create_users.sql
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
email TEXT NOT NULL UNIQUE,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
For Hyperdrive, the schema management remains with your origin database. If you use a tool like Prisma, Drizzle, or TypeORM, you continue to use their migration tools against your primary database. Hyperdrive simply sits in the middle. However, when using Hyperdrive with an ORM, you must ensure the ORM is configured for a “connection pooler” (often by adding ?pgbouncer=true or similar flags if the ORM requires it, though Hyperdrive handles most of this logic automatically) .
One critical difference is how backups and point-in-time recovery (PITR) work. D1 provides automated backups and the ability to restore to a specific point in time through the Cloudflare dashboard. For Hyperdrive, backups remain the responsibility of your primary database provider (e.g., AWS RDS snapshots).
When choosing between them, consider the “source of truth.” If you are building a new, lightweight app, D1 is often simpler. If you have an existing enterprise database with complex stored procedures and triggers, Hyperdrive is the path to bringing that data to the edge without migrating the data itself .
Complete Example Session
// ============================================
// PART 1: WRANGLER CONFIGURATION (wrangler.toml)
// ============================================
// [[d1_databases]]
// binding = "DB"
// database_name = "my-d1-db"
// database_id = "xxxx-xxxx-xxxx"
// [[hyperdrive]]
// binding = "PG_PROXY"
// id = "yyyy-yyyy-yyyy"
// ============================================
// PART 2: D1 QUERY - FETCHING A SINGLE ROW
// ============================================
const user = await env.DB.prepare("SELECT * FROM users WHERE id = ?")
.bind(userId)
.first();
// ============================================
// PART 3: D1 BATCHING - MULTIPLE OPERATIONS
// ============================================
const [res1, res2] = await env.DB.batch([
env.DB.prepare("UPDATE accounts SET balance = balance - 10 WHERE id = 1"),
env.DB.prepare("UPDATE accounts SET balance = balance + 10 WHERE id = 2")
]);
// ============================================
// PART 4: HYPERDRIVE CONNECTION (POSTGRES)
// ============================================
import { Client } from 'pg';
const pgClient = new Client(env.PG_PROXY.connectionString);
await pgClient.connect();
// ============================================
// PART 5: D1 DATA INSERTION
// ============================================
await env.DB.prepare("INSERT INTO logs (message) VALUES (?)")
.bind("User logged in")
.run();
// ============================================
// PART 6: HYPERDRIVE CACHED QUERY
// ============================================
// Caching is configured in wrangler.toml or via dashboard
const productResults = await pgClient.query("SELECT * FROM products");
// ============================================
// PART 7: D1 RAW SQL EXECUTION
// ============================================
await env.DB.exec("CREATE TABLE IF NOT EXISTS temp_stats (val INT)");
// ============================================
// PART 8: LOCAL DEVELOPMENT WITH D1
// ============================================
// npx wrangler dev --remote (to use the cloud DB)
// npx wrangler dev (to use local sqlite file)
// ============================================
// PART 9: ERROR HANDLING
// ============================================
try {
await env.DB.prepare("INVALID SQL").run();
} catch (e) {
console.error(e.message);
}
// ============================================
// PART 10: CLOSING CONNECTIONS
// ============================================
// Workers cleanup connections automatically, but for PG:
await pgClient.end();
The example above demonstrates the configuration and usage of both D1 and Hyperdrive, including batching in D1 and connection pooling in Hyperdrive.
Quick Reference
D1 Prepared Statement Methods
| Method | Returns | Use Case |
|---|---|---|
.first(column?) | Single object or value | Fetching one specific record |
.all() | Array of objects + metadata | Fetching a list of records |
.run() | Metadata only | INSERT, UPDATE, DELETE |
.raw() | Array of arrays | When you don’t want keys/objects |
Hyperdrive vs D1 Comparison
| Feature | Cloudflare D1 | Cloudflare Hyperdrive |
|---|---|---|
| Base Engine | SQLite | Postgres / MySQL (External) |
| Location | Native Cloudflare Edge | Bridge to External Cloud |
| Connection | Native API (Object) | Standard Driver (TCP/Proxy) |
| Caching | No (Direct read) | Yes (Configurable Edge Cache) |
| Best For | Edge-native apps | Legacy DB Acceleration |
Wrangler Commands Reference
| Task | Command |
|---|---|
| Create D1 | wrangler d1 create <name> |
| List D1 | wrangler d1 list |
| Apply Migration | wrangler d1 migrations apply <name> --remote |
| Create Hyperdrive | wrangler hyperdrive create <name> --connection-string="..." |
Best Practices
✅ Do This:
// Use .bind() to prevent SQL injection in D1
env.DB.prepare("SELECT * FROM users WHERE id = ?").bind(id); // ✅
// Use .batch() in D1 for multiple writes
await env.DB.batch([q1, q2]); // ✅
// Use a connection pooler-friendly driver for Hyperdrive
import { Client } from 'pg'; // ✅
// Keep D1 migrations in version control
// migrations/0001_init.sql // ✅
❌ Don’t Do This:
// Don't use template strings for D1 queries (Injection!)
env.DB.prepare(`SELECT * FROM users WHERE id = ${id}`); // ❌
// Don't connect/end Hyperdrive clients inside a tight loop
// Connect once per fetch event // ❌
// Don't use D1 for extremely large datasets (GBs+)
// R2 or external DBs are better for massive storage // ❌
// Don't forget to run migrations on remote
// wrangler d1 migrations apply ... --remote // ❌
Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| SQL Injection | Using string templates instead of .bind() | Always use ? placeholders and .bind(). |
| Connection Timeout | Hyperdrive origin is too slow/firewalled | Ensure Cloudflare IPs are whitelisted in origin DB. |
| Schema Desync | Forgetting to apply D1 migrations to remote | Run wrangler d1 migrations apply --remote. |
| Large Query Failure | D1 has a 1MB result size limit | Use LIMIT and OFFSET to paginate results. |
| Cache Invalidation | Hyperdrive cache serves stale data | Adjust max_age in Hyperdrive config. |
| Missing Driver | Trying to use pg in Worker without Polyfills | Use pg with a compatible environment or specific drivers. |
Real-World Examples
1. User Sessions in D1
Storing session tokens and metadata in D1 for sub-millisecond validation at the edge.
2. E-commerce Product Cache
Using Hyperdrive to cache “Top 100 Products” from an AWS Postgres DB at every Cloudflare edge location.
3. Edge Analytics
Using D1 to increment counters for page views and events directly from a Worker.
4. Global Feature Flags
A D1 table that stores feature toggles, allowing instant updates across the globe.
5. Legacy CRM Proxy
Connecting a Worker to an on-premise Postgres via Hyperdrive to provide a fast mobile API.
6. Multi-tenant Schema
Using D1 to store tenant-specific configuration and custom CSS variables.
7. Blog Comments
A D1 database managing comments for a static site, providing the “dynamic” part of the stack.
8. Inventory Sync
Hyperdrive pooling connections for a high-traffic inventory check system on a centralized DB.
9. Rate Limiting
Storing IP hit counts in D1 to implement custom, complex rate-limiting logic.
10. Localization Data
Using D1 to store translation strings, serving them based on the user’s CF-IPCountry header.
Visual
D1 Native Edge Architecture
┌──────────────────────────────────────────────────────────┐
│ CLOUDFLARE WORKER (EDGE) │
│ │
│ [ Code Executes ] ──► [ env.DB API ] ──┐ │
│ │ │
└─────────────────────────────────────────┼────────────────┘
│
┌─────────────────────────────────▼────────────────┐
│ CLOUDFLARE D1 (SQLITE) │
│ │
│ - Distributed Storage │
│ - Automated Replication │
│ - No TCP Handshake (Internal Binding) │
└──────────────────────────────────────────────────┘
Hyperdrive Proxy Mechanics
┌──────────────────────┐ ┌──────────────────────────┐ ┌──────────────────────┐
│ WORKER (EDGE) │ │ HYPERDRIVE (PROXY) │ │ ORIGIN DB (AWS/GCP) │
│ │ │ │ │ │
│ Request SQL ─────────┼─────►│ 1. Check Cache │ │ │
│ │ │ 2. Reuse Warm Connection ┼─────►│ Execute Query │
│ │ │ 3. Aggregate Results │ │ │
└──────────────────────┘ └──────────────────────────┘ └──────────────────────┘
Hybrid Implementation Strategy
┌──────────────────────────────────────────────────────────┐
│ HYBRID DATA ARCHITECTURE │
│ │
│ - D1: Fast metadata, sessions, edge state (Native) │
│ - Hyperdrive: Legacy data, large datasets (Accelerated) │
│ │
│ [ Worker ] │
│ ├── fetch(session) ──► D1 │
│ └── fetch(products) ──► Hyperdrive ──► Postgres │
└───────────────────────────────────────────────────────────┘
Summary
| Item | Value |
|---|---|
| D1 Engine | SQLite |
| Hyperdrive Support | Postgres, MySQL |
| D1 Binding | env.DB (Object) |
| Hyperdrive Binding | env.HYPERDRIVE.connectionString |
| Primary Tool | Wrangler CLI |
| D1 Limitation | 1MB per result set, 500MB per DB (soft) |
| Hyperdrive Feature | Connection pooling + Edge Caching |
Key takeaways:
- D1 is the edge-native choice. If you are starting a new project and want a truly serverless, distributed relational database, D1 (SQLite) is the integrated solution .
- Hyperdrive is the legacy bridge. It doesn’t move your data; it moves the connection and the results closer to the user, making centralized databases viable for edge computing .
- Connection pooling is critical. Serverless functions fail at scale when they try to open thousands of direct TCP connections to a database; Hyperdrive solves this by reusing “warm” connections .
- D1 migrations are file-based. Schema changes must be applied via
wrangler d1 migrations applyto keep your edge database in sync with your local development . - Caching is Hyperdrive’s “superpower”. Beyond pooling connections, caching SQL results at the edge reduces database load and provides near-instant responses for common queries .
Remember: Move your data as close to your compute as possible. Use D1 for state that belongs at the edge, and use Hyperdrive to optimize the access to the data that must remain centralized.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!