LFCE 129 🐧 Grafana — Dashboards - KandZ

⏱️ Estimated read: 12 min | |

A Grafana dashboard is a collection of panels arranged on a canvas. Each panel contains a query against a data source—Prometheus, in the typical LFCE setup—and a visualization type that renders the query result. The dashboard is not a static image; it re-queries the data source on every refresh, which means the same dashboard shows current values each time it is opened or auto-refreshed. Building a useful dashboard is a design problem: which questions should the dashboard answer, which panel type answers each question, and how should the panels be arranged so that an operator can triage an incident in seconds rather than minutes .

Quick How-To Guides & Fast FixesNeed a quick fix or step-by-step solution? Browse our fast answers on HowTo.KandZ.me:

Operating Systems CLI →Dev Environment Languages →Web Network Tools →Security Accounts →Security Access Tech →Device Phone Basics →Social Messaging →Digital Files Media →

This chapter covers the creation of a Grafana dashboard: creating a new dashboard, adding a panel with a PromQL query, selecting the visualization type that matches the question, organizing panels with rows and layouts, using variables to make the dashboard interactive, and saving or exporting the result. It also covers the panel types that correspond to the most common operational questions.

Key point: The panel type is a function of the question. A single current value is a Stat panel. A trend over time is a Time series panel. A utilization percentage is a Gauge. A ranking of top consumers is a Table or Bar gauge. Choosing the wrong panel type makes the data harder to read, not easier.

Why dashboards have this structure

The triage problem. A dashboard exists to answer questions during an incident. The first question is always “is the system healthy?”—a binary that should be visible at a glance. The second is “what changed?”—a trend over time. The third is “where is the problem?”—a breakdown by instance, service, or endpoint. The documentation for CNPE observability recommends a top-down layout: status overview first, then traffic and errors, then resource utilization . A dashboard that opens with a wall of time series graphs forces the operator to read every panel before knowing whether anything is wrong.

The query-to-visualization mapping problem. Grafana supports dozens of visualization types, but the operational questions map to a small set. A rate() query produces a per-second value that changes over time, which is a time series. A count() query produces a single number, which is a stat. A topk() query produces a ranked list, which is a table or bar gauge. The panel type should follow from the shape of the query result, not from aesthetic preference .

The interactivity problem. A dashboard with a hardcoded instance name only works for that instance. A dashboard with a variable that selects the instance works for every instance in the fleet. Variables are defined in the dashboard settings and referenced in panel queries with the $variable syntax. They are what turn a single dashboard into a reusable tool .

a. Creating a dashboard and adding a panel

A new dashboard starts empty. The “Add visualization” button opens the panel editor, which has three sections: the query editor at the bottom, the visualization preview in the center, and the panel options on the right.

Dashboards → New → New dashboard
  → Add visualization
    → Select data source (Prometheus)
    → Enter PromQL query
    → Select visualization type
    → Set title, unit, thresholds
    → Apply → Save dashboard

The query editor for Prometheus accepts a PromQL expression. The preview updates as the query changes. The panel options on the right control the visualization type, the unit (requests per second, bytes, percent), the legend, and the thresholds that color the panel when a value crosses a boundary .

A minimal dashboard that answers “is Prometheus scraping anything?” uses the up metric:

up

The result is a time series with one line per target, value 1 for up and 0 for down. A Stat panel with a count(up == 1) query shows the number of healthy targets as a single number .

b. Panel types for operational questions

The choice of panel type follows from the question. The CNPE observability guide maps common questions to panel types and example queries .

“Is the system currently healthy?” A Stat panel showing a count or a boolean. The query count(up == 1) returns a single number. The panel displays that number with a green/red threshold. This is the top row of a triage dashboard.

“How is traffic changing over time?” A Time series panel showing rate(http_requests_total[5m]). The time series shows the trend, spikes, and drops. This is the second row.

“What is the current CPU or memory utilization?” A Gauge panel showing (node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes. The gauge shows a percentage with thresholds for warning and critical. This is the third row.

“Which instances are using the most memory?” A Table or Bar gauge panel showing topk(10, sum by (instance) (node_memory_MemAvailable_bytes)). The table ranks the consumers, which is the question the operator is asking.

“What share of traffic goes to each service?” A Pie chart or a Bar gauge with sum by (service) (rate(http_requests_total[5m])). The distribution is the answer.

A dashboard that mixes these types in a top-down order answers the triage questions in the order they are asked.

c. Organizing with rows, layouts, and variables

Grafana groups panels into rows. A row is a collapsible section with a title. The top-down layout uses rows to separate the triage layers: a “Status” row, a “Traffic” row, a “Resources” row. Each row can be collapsed to hide its panels when the dashboard is used for a different purpose .

The panel layout determines how panels are positioned. The “Custom” layout allows individual placement and sizing. The “Auto grid” layout sizes and positions panels automatically, which is faster for a dashboard that does not need pixel-perfect arrangement . The layout choice is made when the dashboard is created and can be changed in the dashboard settings.

Variables make the dashboard interactive. A variable is defined in Settings → Variables, with a query that returns a list of values. For a Prometheus data source, the label_values() function returns the values of a label.

label_values(up, instance)

This query returns every instance that appears in the up metric. The variable is named instance, and panel queries reference it as $instance:

up{instance=~"$instance"}

The =~ operator with the variable allows multiple selections. The dashboard now has a dropdown at the top that filters every panel to the selected instance .

Complete Example Session

# ============================================
# PART 1: CREATE A NEW DASHBOARD
# ============================================
Dashboards → New → New dashboard
  → Add visualization
  → Select Prometheus data source

# ============================================
# PART 2: STAT PANEL — HEALTH STATUS
# ============================================
# Query
count(up == 1)

# Visualization: Stat
# Title: Healthy Targets
# Unit: none (count)
# Thresholds: red < 1, green >= 1

# ============================================
# PART 3: TIME SERIES — REQUEST RATE
# ============================================
# Query
rate(http_requests_total[5m])

# Visualization: Time series
# Title: Request Rate
# Unit: req/s
# Legend: {{instance}}

# ============================================
# PART 4: GAUGE — MEMORY UTILIZATION
# ============================================
# Query
(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes)
  / node_memory_MemTotal_bytes

# Visualization: Gauge
# Title: Memory Usage
# Unit: percent (0-100)
# Thresholds: green < 70, yellow 70-90, red > 90

# ============================================
# PART 5: TABLE — TOP MEMORY CONSUMERS
# ============================================
# Query
topk(10, sum by (instance) (node_memory_MemAvailable_bytes))

# Visualization: Table
# Title: Top Memory Consumers
# Unit: bytes

# ============================================
# PART 6: ROW ORGANIZATION
# ============================================
# Row 1: Status
#   - Stat: Healthy Targets
#   - Stat: Error Rate
#
# Row 2: Traffic
#   - Time series: Request Rate
#   - Time series: Error Rate
#
# Row 3: Resources
#   - Gauge: Memory Usage
#   - Gauge: CPU Usage

# ============================================
# PART 7: ADD A VARIABLE
# ============================================
Settings → Variables → New variable
  Name: instance
  Type: Query
  Data source: Prometheus
  Query: label_values(up, instance)

# ============================================
# PART 8: USE THE VARIABLE IN PANELS
# ============================================
# Update panel queries to filter by the variable:
up{instance=~"$instance"}

rate(http_requests_total{instance=~"$instance"}[5m])

# ============================================
# PART 9: SAVE AND EXPORT
# ============================================
# Save the dashboard with a name.
# Settings → JSON Model → copy the JSON.
# The JSON can be imported into another Grafana instance.

# ============================================
# PART 10: VERIFY THE DASHBOARD
# ============================================
# Set the time range to Last 15 minutes.
# Confirm every panel shows data.
# Change the instance variable and confirm all panels update.

The ten parts covered dashboard creation, a Stat panel for health, a Time series for request rate, a Gauge for memory, a Table for top consumers, row organization, a variable, variable usage in queries, export, and verification.

Quick Reference

Panel types by question

Question	Panel type	Example query
Is the system healthy?	Stat	`count(up == 1)`
How is traffic changing?	Time series	`rate(http_requests_total[5m])`
Current utilization?	Gauge	`(mem_total - mem_available) / mem_total`
Top consumers?	Table / Bar gauge	`topk(10, sum by (instance) (...))`
Traffic distribution?	Pie chart	`sum by (service) (rate(...))`
Error rate?	Stat + Time series	`sum(rate(errors[5m])) / sum(rate(total[5m]))`

Dashboard structure

Element	Purpose
Row	Collapsible group of panels
Panel	Single visualization with a query
Variable	Dropdown filter referenced as `$name`
Layout	Custom or Auto grid

Variable query syntax

Data source	Function	Example
Prometheus	`label_values()`	`label_values(up, instance)`
Prometheus	`query_result()`	`query_result(up)`

Keyboard shortcuts

Shortcut	Action
`Ctrl+S`	Save dashboard
`e` (hover panel)	Edit panel
`v` (hover panel)	Full screen panel
`d+s`	Dashboard settings
`Esc`	Exit edit mode

Best Practices

✅ Do This:

# Put a Stat panel at the top for "is it broken?"
count(up == 1)                                               # ✅

# Use rate() for counters, not raw values
rate(http_requests_total[5m])                                # ✅

# Add a variable so the dashboard works for every instance
up{instance=~"$instance"}                                    # ✅

# Group panels into rows by concern (Status, Traffic, Resources)
# Collapse rows that are not needed during normal operation    # ✅

❌ Don’t Do This:

# Use a raw counter in a time series
http_requests_total                                          # ❌ (always increasing)

# Open with a wall of time series graphs
# The operator cannot tell if anything is wrong at a glance    # ❌

# Hardcode an instance name in every panel
up{instance="web01"}                                         # ❌ (not reusable)

# Use a Pie chart for a time series question
# Pie charts show distribution, not trend                      # ❌

Common Pitfalls

Pitfall	Why It Happens	Fix
All panels show “No data”	Data source URL wrong or query typo	Verify data source with “Save & test”; check the query in Explore
Stat panel shows a huge number	Raw counter instead of `rate()`	Wrap counters in `rate()` or `increase()`
Variable does not filter panels	Query uses `=` instead of `=~`	Use `=~"$variable"` for multi-value
Gauge shows wrong unit	Unit not set in panel options	Set the unit to percent, bytes, or req/s
Dashboard is slow	Too many panels or heavy queries	Reduce panels per row; use recording rules
Layout looks broken	Custom layout with overlapping panels	Switch to Auto grid or adjust panel positions

Real-World Examples

1. Health stat panel

count(up == 1)

2. Request rate time series

rate(http_requests_total[5m])

3. Error rate percentage

100 * sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

4. Memory utilization gauge

(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes

5. Top instances by memory

topk(10, sum by (instance) (node_memory_MemAvailable_bytes))

6. Instance variable

label_values(up, instance)

7. Variable in query

up{instance=~"$instance"}

8. Row for traffic

Row: "Traffic & Errors"
  - Time series: rate(http_requests_total[5m])
  - Time series: rate(http_requests_total{status=~"5.."}[5m])

9. Export dashboard JSON

Settings → JSON Model → copy

10. Import dashboard JSON

Dashboards → New → Import → paste JSON

Visual

The top-down dashboard layout

┌─────────────────────────────────────────────────────────────┐
│  DASHBOARD LAYOUT                                            │
│                                                              │
│  Row 1: STATUS                                               │
│  ┌──────────────┐  ┌──────────────┐                         │
│  │ Healthy: 12  │  │ Errors: 0.2% │                         │
│  └──────────────┘  └──────────────┘                         │
│                                                              │
│  Row 2: TRAFFIC                                              │
│  ┌──────────────────────────────────────────────┐           │
│  │  Request rate (time series)                  │           │
│  └──────────────────────────────────────────────┘           │
│  ┌──────────────────────────────────────────────┐           │
│  │  Error rate (time series)                    │           │
│  └──────────────────────────────────────────────┘           │
│                                                              │
│  Row 3: RESOURCES                                            │
│  ┌──────────────┐  ┌──────────────┐                         │
│  │ Memory gauge │  │ CPU gauge    │                         │
│  └──────────────┘  └──────────────┘                         │
└─────────────────────────────────────────────────────────────┘

Query to panel type mapping

┌─────────────────────────────────────────────────────────────┐
│  QUERY RESULT SHAPE → PANEL TYPE                             │
│                                                              │
│  Single value (count, sum) ──► Stat                          │
│  Value over time (rate) ────► Time series                    │
│  Percentage (0-100) ────────► Gauge                          │
│  Ranked list (topk) ────────► Table / Bar gauge              │
│  Distribution by label ─────► Pie chart                      │
└─────────────────────────────────────────────────────────────┘

Variable flow

┌─────────────────────────────────────────────────────────────┐
│  VARIABLE                                                    │
│                                                              │
│  Settings → Variables → New                                 │
│       │                                                      │
│       ▼                                                      │
│  Query: label_values(up, instance)                           │
│       │                                                      │
│       ▼                                                      │
│  Dropdown at top of dashboard                               │
│       │                                                      │
│       ▼                                                      │
│  Panel query: up{instance=~"$instance"}                     │
│       │                                                      │
│       ▼                                                      │
│  All panels filter to selected instance                      │
└─────────────────────────────────────────────────────────────┘

Save and export

┌─────────────────────────────────────────────────────────────┐
│  DASHBOARD PERSISTENCE                                       │
│                                                              │
│  Save ──► Grafana database                                   │
│                                                              │
│  Settings → JSON Model ──► copy JSON                         │
│       │                                                      │
│       ▼                                                      │
│  Import into another Grafana instance                        │
│                                                              │
│  The JSON is the portable representation of the dashboard.  │
└─────────────────────────────────────────────────────────────┘

Summary

Item	Value
Dashboard	Collection of panels
Panel	Query + visualization
Stat panel	Single current value
Time series panel	Trend over time
Gauge panel	Percentage or bounded value
Table panel	Ranked list
Variable	Dropdown filter referenced as `$name`
Variable query	`label_values(metric, label)`
Rows	Collapsible groups of panels
Export	Settings → JSON Model

Key takeaways:

A dashboard answers questions in the order they are asked during triage. The top row should answer “is anything broken?” with a Stat panel. The next row should answer “what changed?” with Time series. The remaining rows answer “where is the problem?” with Gauges and Tables .
The panel type follows from the query result shape. A single value is a Stat. A value over time is a Time series. A bounded percentage is a Gauge. A ranked list is a Table. Using a Time series for a single value, or a Pie chart for a trend, makes the data harder to interpret.
Variables make the dashboard reusable. A label_values() query populates a dropdown, and panel queries reference the variable with =~"$variable". Without variables, the dashboard is hardcoded to one instance or one service .
Rows organize panels by concern. Grouping status, traffic, and resources into separate rows allows the operator to collapse the sections that are not relevant to the current investigation .
The dashboard JSON is the portable artifact. Exporting the JSON model and committing it to a repository makes the dashboard reviewable, versionable, and importable into any Grafana instance. This is the dashboard-as-code workflow.
Complex queries belong in recording rules, not dashboards. A long PromQL expression in a panel is hard to maintain and runs on every dashboard load. A recording rule precomputes the value in Prometheus, and the panel queries the recorded metric .

Remember: A Grafana dashboard is a triage tool, not a data dump. Every panel should answer a question that an operator would ask during an incident. Start with the status row, add the traffic and error rows, and finish with the resource rows. Use Stat for the single numbers, Time series for the trends, and Gauge for the utilization. Add a variable so the dashboard works for the whole fleet, not just one host. Save the JSON and commit it. The dashboard that gets used is the one that tells the operator what is wrong before they have to read every graph.

Was this helpful?

Stop using slow, ad-bloated tool sites! 🤮

🔎 Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
✅ Finance (Mortgage, Interest, Inflation)
✅ Tech (Base64, JSON, Dev Suite, IP)
✅ Health (BMI, BMR, TDEE)
✅ Productivity (Timer, Workspace, QR)

⚡️ Fast & Private
🔒 No data leaves your device
💎 100% Free

🔗 Use it now: https://tools.kandz.me
🔖 Bookmark it—you’ll need it later!