React 125 ⚛️ WebGL and Canvas Interactivity in React
React does not have a native canvas element. The <canvas> tag is a DOM element, and the drawing context is an imperative API that sits outside React’s declarative model. Interactivity—detecting a click on a shape, hovering over a pixel, dragging a node—requires translating DOM events into coordinates that the canvas understands, and then deciding whether the interaction should trigger a React state change or a direct mutation. The architecture that makes this tractable depends on the rendering library: React Three Fiber for 3D scenes, react-konva for 2D scenes, or a custom hook for raw canvas pixel data.
This chapter covers the three approaches to canvas interactivity in React. It begins with the 3D case and React Three Fiber’s raycasting event system, which provides DOM-shaped pointer events on meshes and a propagation model that accounts for occlusion. It then addresses the 2D case with react-konva, a library that renders Konva shapes through React’s reconciler and gives each shape a component with props and event handlers. The final sections cover the raw canvas approach, where the developer manages the drawing loop and hit-testing manually, and the performance discipline that separates interactions that should trigger a render from interactions that should mutate a ref.
Key point: Canvas interactivity is hit-testing. The browser does not know what you drew; it only knows the pixel you clicked. Every canvas library exists to answer the question “which shape did the user interact with?” and the answer determines whether React re-renders or a ref mutates.
Why canvas interactivity is different from DOM interactivity
The DOM has built-in hit-testing. When you click a <button>, the browser walks the DOM tree, finds the element at the clicked coordinates, and dispatches the event to it. The developer does not write the hit-test. The browser’s layout engine provides it.
The canvas has no layout. A <canvas> element is a single bitmap. The browser knows the canvas was clicked, but it does not know that the click landed on a circle at (100, 150) or a rectangle at (200, 50). The shapes exist only as pixel data after the drawing commands execute. The library or the developer must implement the hit-test: either by storing a scene graph of shapes and testing the coordinates against each shape’s geometry, or by using a color-picking technique where each shape is drawn in a unique color to an offscreen canvas and the pixel under the cursor is read back.
The update model differs. DOM events that change state trigger a React render, and the browser repaints the changed elements. Canvas events that change a shape require either a re-render of the React component tree (for declarative canvas libraries) or a direct mutation of the canvas context followed by a redraw (for imperative canvas code). The performance rule is the same as R3F: state for structure, refs for per-frame updates.
a. React Three Fiber: raycasting and 3D pointer events
React Three Fiber’s event system is built on three.js’s Raycaster. When a pointer event occurs on the Canvas, the raycaster casts a ray from the camera through the pointer’s screen coordinates into the 3D scene. The ray intersects every mesh that has an event handler, and the intersections are sorted by distance from the camera. The event is delivered to the nearest intersection first, then bubbles up through its ancestors, then moves to the next intersection along the ray .
The event object carries the intersection data: the world-space hit point (e.point), the distance from the camera (e.distance), the face that was hit (e.face), and the full list of intersections (e.intersections). The distinction between e.object and e.eventObject matters: e.object is the mesh the ray actually hit, which may be a child of the mesh that carries the handler, and e.eventObject is the mesh with the handler. When a handler is attached to a <group> that contains many meshes, e.eventObject is the group and e.object is the specific child that was hit .
import { Canvas } from '@react-three/fiber';
function ClickableBox() {
return (
<mesh
onClick={(e) => {
// Stop propagation to objects behind this one.
e.stopPropagation();
console.log('Hit at', e.point, 'distance', e.distance);
}}
onPointerOver={(e) => {
e.stopPropagation();
document.body.style.cursor = 'pointer';
}}
onPointerOut={() => {
document.body.style.cursor = 'default';
}}
>
<boxGeometry args={[1, 1, 1]} />
<meshStandardMaterial color="orange" />
</mesh>
);
}The stopPropagation behavior is the most important difference from the DOM. In 3D, a single ray can intersect multiple objects, and R3F delivers the event to each intersection in near-to-far order. Calling e.stopPropagation() stops both the bubbling to ancestors and the delivery to farther objects. Without it, a click on a front box also triggers the click handler on a box behind it .
The onPointerMissed handler on the Canvas is the deselection hook. It fires when a click occurs and no mesh with an event handler was hit. This is the correct place to clear a selection, rather than attaching a handler to a background plane .
<Canvas
onPointerMissed={() => setSelected(null)}
>
<Scene />
</Canvas>The performance concern with R3F events is raycasting cost. The raycaster tests every object with a pointer handler, and the cost is proportional to the vertex count of the meshes it tests. A scene with high-poly models that all have hover handlers will drop frames when the pointer moves, because every mouse movement triggers a new raycast . The mitigations are to reduce the number of objects with handlers, use the raycast prop to disable raycasting on objects that should not be interactive, and use the BVH (Bounding Volume Hierarchy) acceleration from three-mesh-bvh for dense scenes .
// Disable raycasting on a mesh that should not be interactive.
<mesh raycast={() => null}>
<boxGeometry args={[10, 0.1, 10]} />
<meshStandardMaterial color="gray" />
</mesh>b. react-konva: declarative 2D canvas
react-konva is the React binding for Konva, a 2D canvas library. It works through React’s reconciler: the components you write (<Stage>, <Layer>, <Rect>, <Circle>) are not DOM elements, but the reconciler creates the corresponding Konva nodes and adds them to the Konva scene graph. The relationship is analogous to React DOM and the browser DOM .
The interactivity model is the DOM’s. Each Konva shape is a component with props and event handlers. The hit-testing is handled by Konva’s internal scene graph, which knows the geometry of every shape and can determine which shape contains a given point. The developer writes onClick on a <Rect> and receives an event with the target shape, without implementing a raycast or a color-picking loop .
import { Stage, Layer, Rect, Circle } from 'react-konva';
function App() {
const [color, setColor] = useState('orange');
return (
<Stage width={500} height={500}>
<Layer>
<Rect
x={50}
y={50}
width={200}
height={100}
fill={color}
onClick={() => setColor(color === 'orange' ? 'hotpink' : 'orange')}
onMouseEnter={(e) => {
e.target.getStage().container().style.cursor = 'pointer';
}}
onMouseLeave={(e) => {
e.target.getStage().container().style.cursor = 'default';
}}
/>
<Circle
x={300}
y={200}
radius={50}
fill="skyblue"
draggable
/>
</Layer>
</Stage>
);
}The draggable prop is a built-in interaction that Konva handles without additional code. The shape can be dragged, and the position updates are available through the onDragEnd handler, which reads e.target.x() and e.target.y(). This is the kind of interaction that would require manual pointer tracking and hit-testing in a raw canvas, and it is the reason react-konva exists .
The version constraint is important: the major version of react-konva must match the major version of React. A React 19 project uses the current react-konva, while a React 18 project must install react-konva@18 . The library does not work in React Native, because React Native has no DOM and no <canvas> element.
c. Raw canvas and pixel-level interactivity
Some interactions are not about shapes. A color picker that reads the pixel under the cursor, a painting application that draws where the pointer moves, or a heatmap that responds to individual data points requires access to the canvas context and the pixel data. The react-canvas-img package is an example of a library built for this case: it loads an image into a canvas and provides event props that report the pixel coordinates and the color at those coordinates .
The raw approach is to use a ref on the <canvas> element, get the 2D context in a useEffect, and attach pointer event listeners manually. The event’s clientX and clientY are relative to the viewport; they must be converted to canvas coordinates using the canvas’s bounding rect and its intrinsic width and height.
import { useRef, useEffect } from 'react';
function PixelCanvas() {
const canvasRef = useRef(null);
const [pixel, setPixel] = useState(null);
useEffect(() => {
const canvas = canvasRef.current;
const ctx = canvas.getContext('2d');
// Draw something.
ctx.fillStyle = 'skyblue';
ctx.fillRect(0, 0, canvas.width, canvas.height);
const handleClick = (e) => {
const rect = canvas.getBoundingClientRect();
// Convert viewport coordinates to canvas coordinates.
const x = (e.clientX - rect.left) * (canvas.width / rect.width);
const y = (e.clientY - rect.top) * (canvas.height / rect.height);
// Read the pixel at the click position.
const data = ctx.getImageData(x, y, 1, 1).data;
setPixel({ x, y, r: data[0], g: data[1], b: data[2], a: data[3] });
};
canvas.addEventListener('click', handleClick);
return () => canvas.removeEventListener('click', handleClick);
}, []);
return (
<div>
<canvas ref={canvasRef} width={400} height={300} />
{pixel && <p>Pixel at ({pixel.x}, {pixel.y}): rgba({pixel.r}, {pixel.g}, {pixel.b}, {pixel.a})</p>}
</div>
);
}The performance rule for per-frame interactions is the same as R3F: do not call setState on every pointer move. A painting application that sets state on onPointerMove will re-render the component and the canvas on every frame, which is slow and often causes dropped input. The correct pattern is to draw directly to the context inside the event handler, using the ref, and only set state when the interaction completes (on pointer up) .
Complete Example Session
// ============================================
// PART 1: THE 3D INTERACTION
// ============================================
// R3F raycasting with stopPropagation and hover.
import { Canvas } from '@react-three/fiber';
import { useState } from 'react';
function Box({ position, color, onClick }) {
const [hovered, setHovered] = useState(false);
return (
<mesh
position={position}
onClick={(e) => {
e.stopPropagation();
onClick();
}}
onPointerOver={(e) => {
e.stopPropagation();
setHovered(true);
document.body.style.cursor = 'pointer';
}}
onPointerOut={() => {
setHovered(false);
document.body.style.cursor = 'default';
}}
>
<boxGeometry args={[1, 1, 1]} />
<meshStandardMaterial color={hovered ? 'hotpink' : color} />
</mesh>
);
}
function Scene3D() {
const [selected, setSelected] = useState(null);
return (
<Canvas onPointerMissed={() => setSelected(null)}>
<ambientLight intensity={0.5} />
<directionalLight position={[5, 5, 5]} />
<Box position={[-1, 0, 0]} color="orange" onClick={() => setSelected(1)} />
<Box position={[1, 0, 0]} color="skyblue" onClick={() => setSelected(2)} />
</Canvas>
);
}
// ============================================
// PART 2: THE 2D INTERACTION
// ============================================
// react-konva shapes with drag and click.
import { Stage, Layer, Rect, Circle } from 'react-konva';
function Scene2D() {
const [color, setColor] = useState('orange');
return (
<Stage width={400} height={300}>
<Layer>
<Rect
x={50} y={50} width={150} height={100}
fill={color}
onClick={() => setColor(color === 'orange' ? 'hotpink' : 'orange')}
/>
<Circle x={300} y={200} radius={50} fill="skyblue" draggable />
</Layer>
</Stage>
);
}
// ============================================
// PART 3: THE PIXEL-LEVEL INTERACTION
// ============================================
// Raw canvas with coordinate conversion and getImageData.
function PixelReader() {
const canvasRef = useRef(null);
const [info, setInfo] = useState(null);
useEffect(() => {
const canvas = canvasRef.current;
const ctx = canvas.getContext('2d');
ctx.fillStyle = '#ff6600';
ctx.fillRect(0, 0, canvas.width, canvas.height);
const handleClick = (e) => {
const rect = canvas.getBoundingClientRect();
const x = (e.clientX - rect.left) * (canvas.width / rect.width);
const y = (e.clientY - rect.top) * (canvas.height / rect.height);
const [r, g, b, a] = ctx.getImageData(x, y, 1, 1).data;
setInfo({ x: Math.round(x), y: Math.round(y), r, g, b, a });
};
canvas.addEventListener('click', handleClick);
return () => canvas.removeEventListener('click', handleClick);
}, []);
return (
<div>
<canvas ref={canvasRef} width={300} height={200} />
{info && <p>rgba({info.r}, {info.g}, {info.b}, {info.a})</p>}
</div>
);
}
// ============================================
// PART 4: THE PERFORMANCE RULE
// ============================================
// Do not setState on pointer move for painting apps.
function PaintCanvas() {
const canvasRef = useRef(null);
const drawing = useRef(false);
useEffect(() => {
const canvas = canvasRef.current;
const ctx = canvas.getContext('2d');
ctx.lineWidth = 4;
ctx.lineCap = 'round';
const down = (e) => {
drawing.current = true;
const rect = canvas.getBoundingClientRect();
ctx.beginPath();
ctx.moveTo(e.clientX - rect.left, e.clientY - rect.top);
};
const move = (e) => {
if (!drawing.current) return;
const rect = canvas.getBoundingClientRect();
ctx.lineTo(e.clientX - rect.left, e.clientY - rect.top);
ctx.stroke(); // draw directly, no setState
};
const up = () => { drawing.current = false; };
canvas.addEventListener('pointerdown', down);
canvas.addEventListener('pointermove', move);
canvas.addEventListener('pointerup', up);
return () => {
canvas.removeEventListener('pointerdown', down);
canvas.removeEventListener('pointermove', move);
canvas.removeEventListener('pointerup', up);
};
}, []);
return <canvas ref={canvasRef} width={400} height={300} style={{ border: '1px solid #ccc' }} />;
}
// ============================================
// PART 5: THE FULL PICTURE
// ============================================
// 3D: R3F raycasting. 2D: react-konva.
// Pixels: raw canvas with coordinate math.
export default function App() {
return (
<div>
<Scene3D />
<Scene2D />
<PixelReader />
<PaintCanvas />
</div>
);
}The five parts demonstrate the three interaction models: raycasting for 3D, scene-graph hit-testing for 2D, and pixel-coordinate math for raw canvas. The performance rule is consistent across all three: per-frame updates mutate, discrete interactions set state.
Quick Reference
3D Event Properties (R3F)
| Property | Meaning |
|---|---|
e.point | World-space hit point |
e.distance | Distance from camera |
e.object | Mesh that was hit |
e.eventObject | Mesh with the handler |
e.intersections | All hits, nearest first |
e.delta | Mousedown-to-mouseup distance |
R3F Event Handlers
| Handler | Behavior |
|---|---|
onClick | Click with no drag |
onPointerOver / onPointerOut | Bubble to ancestors |
onPointerEnter / onPointerLeave | Do not bubble |
onPointerMissed | Canvas-level, no hit |
stopPropagation() | Stops bubbling and farther objects |
Canvas Libraries
| Library | Model | Hit-Testing |
|---|---|---|
| React Three Fiber | 3D scene graph | Raycaster |
| react-konva | 2D scene graph | Konva internal |
| Raw canvas | Manual | Coordinate math or color-pick |
Performance Rules
| Interaction | Mechanism |
|---|---|
| Hover/selection | React state (discrete) |
| Per-frame animation | useFrame + ref |
| Painting | Direct context draw |
| Fast pointer move | Ref mutation |
Best Practices
✅ Do This:
// Use stopPropagation to prevent click-through.
onClick={(e) => { e.stopPropagation(); select(); }} // ✅// Use onPointerMissed for deselection.
<Canvas onPointerMissed={() => setSelected(null)}> // ✅// Use react-konva for declarative 2D shapes.
<Rect x={50} y={50} width={100} height={50} onClick={...} /> // ✅// Convert coordinates for raw canvas.
const x = (e.clientX - rect.left) * (canvas.width / rect.width); // ✅// Draw directly for painting, set state on completion.
ctx.lineTo(x, y); ctx.stroke(); // ✅❌ Don’t Do This:
// Don't skip stopPropagation on overlapping meshes.
onClick={() => select()} // front and back both fire // ❌// Don't setState on every pointer move.
onPointerMove={(e) => setX(e.point.x)} // render every frame // ❌// Don't use react-konva in React Native.
// No DOM, no canvas element. // ❌// Don't ignore the canvas-to-viewport coordinate conversion.
const x = e.clientX; // wrong if canvas is CSS-scaled // ❌Common Pitfalls
| Pitfall | Why It Happens | Fix |
|---|---|---|
| Click-through on overlapping meshes | Missing stopPropagation | Add it to the front mesh’s handler |
| Low FPS on hover | Raycasting high-poly models | Use BVH or disable raycast on non-interactive meshes |
| Konva version mismatch | React 18 with React 19 react-konva | Install react-konva@18 for React 18 |
| Click coordinates wrong | CSS-scaled canvas | Multiply by canvas.width / rect.width |
| Painting lags | setState on pointer move | Draw directly to context |
| Selection persists after click elsewhere | No onPointerMissed handler | Add it to the Canvas |
Real-World Examples
1. R3F Click with Hit Point
<mesh onClick={(e) => console.log(e.point)}>2. R3F Hover Cursor
onPointerOver={(e) => { e.stopPropagation(); document.body.style.cursor = 'pointer'; }}3. R3F Deselect
<Canvas onPointerMissed={() => setSelected(null)}>4. Konva Click
<Rect x={50} y={50} width={100} height={50} onClick={() => setColor('red')} />5. Konva Drag
<Circle x={200} y={100} radius={30} draggable />6. Konva Cursor
onMouseEnter={(e) => e.target.getStage().container().style.cursor = 'pointer'}7. Raw Canvas Click
canvas.addEventListener('click', handleClick);8. Pixel Read
const [r, g, b, a] = ctx.getImageData(x, y, 1, 1).data;9. Coordinate Conversion
const x = (e.clientX - rect.left) * (canvas.width / rect.width);10. Paint Without State
ctx.lineTo(x, y); ctx.stroke();Visual
The Hit-Test Problem
┌─────────────────────────────────────────────────────────┐
│ CANVAS: one bitmap │
│ ┌───────────────────────────────────────────────────┐ │
│ │ │ │
│ │ ○ (circle at 100, 100) │ │
│ │ │ │
│ │ ▭ (rect at 200, 150) │ │
│ │ │ │
│ └───────────────────────────────────────────────────┘ │
│ │
│ Click at (105, 102) │
│ The browser says: "canvas clicked" │
│ The browser does NOT say: "circle clicked" │
│ │
│ The library must answer: which shape? │
│ → R3F: raycast into 3D scene │
│ → Konva: test against scene graph │
│ → Raw: coordinate math or color-pick │
└─────────────────────────────────────────────────────────┘R3F Event Propagation
┌─────────────────────────────────────────────────────────┐
│ Ray from camera through pointer │
│ │ │
│ ▼ │
│ Intersection 1: front box (nearest) │
│ → handler fires first │
│ → if stopPropagation: stop here │
│ │ │
│ ▼ (no stopPropagation) │
│ Intersection 2: back box │
│ → handler fires second │
│ │
│ In the DOM, only the top element receives the event. │
│ In 3D, every object along the ray receives it. │
│ stopPropagation is how you make an object opaque. │
└─────────────────────────────────────────────────────────┘react-konva Architecture
┌─────────────────────────────────────────────────────────┐
│ React Components │
│ <Stage><Layer><Rect onClick={...} /></Layer></Stage> │
│ │ │
│ ▼ react-reconciler │
│ Konva Scene Graph │
│ Stage → Layer → Rect │
│ │ │
│ ▼ Konva renderer │
│ Canvas 2D context │
│ │
│ Each React component maps to a Konva node. │
│ Each Konva node has geometry and event handlers. │
│ Konva's hit-test knows the geometry. │
└─────────────────────────────────────────────────────────┘The State vs Ref Decision
┌─────────────────────────────────────────────────────────┐
│ DISCRETE (state) │
│ Click → select → re-render │
│ Hover → color change → re-render │
│ Drag end → position update → re-render │
│ │
│ ───────────────────────────────────────────────────── │
│ │
│ CONTINUOUS (ref) │
│ Pointer move → paint → draw to context │
│ Frame loop → rotate → mutate mesh │
│ Drag move → follow cursor → mutate position │
│ │
│ setState in a continuous interaction = dropped frames │
└─────────────────────────────────────────────────────────┘Summary
| Item | Value |
|---|---|
| 3D interactivity | React Three Fiber + raycaster |
| 3D event stop | e.stopPropagation() blocks farther objects |
| 3D deselection | onPointerMissed on Canvas |
| 2D interactivity | react-konva (React reconciler for Konva) |
| 2D drag | draggable prop on Konva shape |
| Pixel interactivity | Raw canvas + getImageData |
| Coordinate conversion | (clientX - rect.left) * (canvas.width / rect.width) |
| Performance rule | State for discrete, ref for continuous |
| React version match | react-konva major must match React major |
Key takeaways:
- Canvas interactivity is hit-testing. The browser knows the canvas was clicked, not which shape was clicked. The library answers that question, and the answer determines the event handling model .
- R3F events are raycasts. The raycaster tests every object with a handler, sorts by distance, and delivers to each intersection.
stopPropagationstops both bubbling and delivery to farther objects, which is how an object becomes opaque to clicks . onPointerMissedis the deselection hook. It fires when a click hits nothing, which is the correct place to clear a selection rather than adding a background plane .- react-konva is React for 2D canvas. The components are not DOM elements, but the reconciler creates Konva nodes with geometry and event handlers. Hit-testing is handled by Konva, and
draggableprovides built-in drag interaction . - Raw canvas requires coordinate math. The event’s
clientXandclientYare viewport coordinates. They must be converted to canvas coordinates using the bounding rect and the intrinsic size ratio beforegetImageDataor drawing. - The performance rule is the same as R3F. Discrete interactions—click, hover, drag end—use React state. Continuous interactions—painting, pointer move, frame animation—mutate directly and set state only on completion.
setStateon every pointer move causes dropped frames and laggy input .
Remember: Canvas interactivity in React is a choice of abstraction. R3F gives you 3D meshes with raycast events and a propagation model that accounts for occlusion. react-konva gives you 2D shapes with DOM-like event handlers and built-in drag. Raw canvas gives you pixel-level control, but you write the hit-test and the coordinate conversion. In all three, the split is the same: the event tells you where the user interacted, and the decision that follows is whether that interaction changes the structure (state) or updates the frame (ref). Choose the abstraction that matches the scene, and keep the continuous interactions out of React’s render cycle.
Stop using slow, ad-bloated tool sites! 🤮
🔎 Search “KandZ Tools” on Google to use many professional utilities for free.
KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)
⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free
🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!