Structured Logging Best Practices: A Practical Guide for Engineers.

An on-call engineer gets paged at 2 a.m. Checkout latency has spiked. With unstructured logs, the next twenty minutes go to grep, awk, and guessing which service is actually slow. With structured logs, the same investigation is one query in a platform like Middleware: service:checkout AND duration_ms:>2000, sorted by trace_id. That difference, minutes instead of grep, is the entire reason structured logging matters, and it is why this guide focuses less on theory and more on the practices that actually pay off when you’re the one holding the pager.

For background on the core concept, see what is structured logging. This guide assumes you already know what structured logging is and want to know how to do it well.

The best practices, in order of impact

These are ranked by how much practical debugging and querying power they unlock, not alphabetically. If you only adopt three, adopt the first three.

1. Use a trace ID to correlate logs across services

This is the single highest-leverage practice in structured logging. Without a shared identifier, a request that touches five microservices produces five disconnected log streams. With one, it produces one queryable thread.

{"timestamp": "2026-09-16T09:12:03.441Z", "level": "info", "service": "api-gateway", "trace_id": "7a3f9c2e", "message": "Request received", "route": "/checkout"}
{"timestamp": "2026-09-16T09:12:03.502Z", "level": "info", "service": "inventory", "trace_id": "7a3f9c2e", "message": "Stock check passed", "sku": "SKU-4471"}
{"timestamp": "2026-09-16T09:12:04.891Z", "level": "error", "service": "payments", "trace_id": "7a3f9c2e", "message": "Card authorization failed", "error_code": "card_declined"}

Why it helps in practice: when a customer reports a failed checkout, you paste one trace ID into your log platform and get the entire request lifecycle across every service, in order, with no manual correlation. This is the difference between a five-minute investigation and a two-hour one across service owners.

How to implement it: generate the trace ID at the edge (API gateway or load balancer), pass it through every downstream call as an HTTP header (X-Trace-Id or use OpenTelemetry’s native trace context propagation), and make your logging library attach it automatically via a request-scoped child logger so engineers can’t forget to include it.

2. Put variable data in fields, not in the message string

This is the most common mistake, and the one that quietly undoes everything else. If your logs look like this:

{"message": "User user_4471 failed login attempt number 3 from 192.168.1.42"}

You have a structured envelope around an unstructured message, which gets you almost none of the benefit. You cannot query “all failed logins with attempt_number >= 3” without parsing the sentence. The fix:

{"message": "Login attempt failed", "user_id": "user_4471", "attempt_number": 3, "ip_address": "192.168.1.42"}

Why it helps in practice: you can now build an alert rule directly on attempt_number >= 3 to catch credential-stuffing attempts, without a regex that breaks the moment someone edits the message wording. This is usually the single edit that turns a team’s logs from “technically JSON” into “actually useful.”

3. Standardize field names across every service

Pick snake_case or camelCase once. Decide user_id vs userId vs uid once. Write it down. The cost of doing this after five teams have already shipped their own conventions is a painful, months-long cleanup; the cost of doing it up front is one short document.

Why it helps in practice: a shared field dictionary is what lets you write one dashboard query, error_rate by service, and have it work across every service instead of five slightly different ones. Practically, keep a short internal reference like this and enforce it in code review or via a shared logging library:

4. Name your log events instead of writing narrative messages

Instead of a log line that narrates what happened, give the event itself a stable name in an event field: order_placed, payment_declined, cache_miss, rate_limit_exceeded. This turns your logs into something closer to an event stream than a diary.

Why it helps in practice: product and reliability questions like “how many payments failed for customers on the EU plan in the last hour” become a single event:payment_declined filter combined with other fields, answerable without touching code or waiting on an analytics pipeline.

5. Redact sensitive data automatically at the logging layer

Relying on every engineer to remember not to log a password or a card number fails eventually. Build redaction into the logger itself, so fields matching password, token, ssn, card_number, authorization are automatically masked before the log line is ever written or shipped.

const logger = pino({
  redact: ['req.headers.authorization', 'user.password', 'payment.card_number']
});

Why it helps in practice: this is the difference between a compliance incident and a non-event when someone accidentally logs a full request object during a debugging session.

6. Use a small, consistent set of log levels

debug, info, warn, error, critical is enough for almost every system. The practical failure mode isn’t too few levels; it’s inconsistent usage: one team’s warn is another team’s error, which makes cross-service alerting unreliable. Note that libraries name their top level differently; Pino calls it fatal rather than critical, so map your team’s naming to whatever your logging library actually ships.

Why it helps in practice: if error reliably means “needs human attention,” your on-call alert on level:error is trustworthy. If it doesn’t, engineers start ignoring alerts, which is worse than not having levels at all.

7. Sample debug and trace-level logs to control cost

At real production volume, logging every debug event from every request generates a volume-to-value ratio that gets expensive fast. Keep error and critical logs at 100% fidelity always. Sample debug and info logs, especially on high-traffic, low-incident-rate paths like health checks.

Why it helps in practice: this keeps your log bill proportional to what you actually investigate, while guaranteeing that the logs you need during an incident are never the ones that got sampled away.

8. Only log fields you will actually query or alert on

A common overcorrection is dumping entire request or response objects into every log line “just in case.” This bloats storage, slows queries, and often violates rule 5 by accident. Log the fields tied to a decision you’ll actually make: what you’d filter by, group by, or alert on. If you’re not sure whether a field earns its place, ask whether you’d ever write a query using it.

A practical scenario: debugging a latency spike with structured logs

Here’s what the difference looks like end to end, using the checkout example from the introduction.

Step 1: Spot it

Your dashboard shows p99 latency on /checkout climbing. Because duration_ms is a standard field on every log, this chart existed automatically, no custom instrumentation needed.

Step 2: Narrow it

Query route:/checkout AND duration_ms:>2000, grouped by service. In a well-instrumented system, a log platform like Middleware surfaces this instantly, showing which service in the chain is slow rather than which service happened to log the most.

Step 3: Trace it

Pull the trace_id from one slow request and query for every log carrying that ID. You now have the full path of that single request across every service it touched, in chronological order, with timing at each hop.

Step 4: Fix it, and prove it

After the fix ships, the same duration_ms query confirms the fix worked, using the exact same query you used to find the problem, not a new investigation.

None of these four steps required writing a parser, grepping across five separate log files, or pinging four different service owners to ask “did you see anything weird around 9:12?” That’s the practical payoff of the practices above; the payoff shows up during incidents, not during code review.

JSON structure practices that keep queries fast as you scale

The eight practices above get your logging schema right. These four keep queries fast and reliable as log volume and the number of services both grow.

Keep the structure flat

Prefer http.status_code over request.details.response.status.code. Most log platforms index and query flat or lightly nested fields faster and more reliably than deeply nested objects, and flat fields are easier for every engineer to remember and type correctly. Nest only when fields genuinely belong together, like an array of line items on an order.

Why it helps in practice: a query like http.status_code:>=500 stays simple and fast to write months later. A four-level-deep path is easy to get wrong and slower for most platforms to index well.

Use consistent types for the same field everywhere

If duration_ms is a number in one service and a string like "142ms" in another, aggregation queries such as avg(duration_ms) either fail outright or silently produce wrong results by only counting the numeric-typed logs.

Why it helps in practice: this is the kind of bug that doesn’t throw an error. It quietly makes your dashboards wrong, and you only catch it late unless you enforce the type at the schema level.

Timestamp in UTC, always

Local timezones in log timestamps create silent off-by-hours bugs the moment you correlate logs from services deployed across regions or from a team working in a different timezone than the one that wrote the original log line.

Why it helps in practice: UTC timestamps sort correctly and compare correctly no matter who’s reading them or where the service is deployed, which matters most exactly when you’re correlating logs across services during an incident.

Version your schema when the shape changes

Adding a new field to your logs is always safe. Renaming or removing a field that an existing dashboard or alert depends on isn’t. Add a schema_version field once your log shape has changed meaningfully, so old and new log formats can be told apart in a query.

Why it helps in practice: without a version field, a schema change silently breaks every saved query and alert built on the old field name, and nobody finds out until the alert fails to fire.

Migrating from console.log to structured logging (Node.js)

Most Node.js codebases start here:

console.log(`User ${userId} failed login from ${ip}`);

This can’t be filtered by userId, has no level, and no timestamp unless added manually every time. Here’s a practical migration path using Pino, a fast structured logging library.

Set up the base logger

const pino = require('pino');

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  base: { service: 'auth-service', environment: process.env.NODE_ENV },
  timestamp: pino.stdTimeFunctions.isoTime,
  redact: ['req.headers.authorization']
});

Attach a trace ID per request, automatically

Shown here with Express; the same pattern applies in Fastify, Koa, or any framework with request-scoped middleware.

const crypto = require('crypto');

app.use((req, res, next) => {
  req.log = logger.child({ trace_id: req.headers['x-trace-id'] || crypto.randomUUID() });
  next();
});

Replace the console.log line

req.log.warn(
  { event: 'login_failed', user_id: userId, ip_address: ip, attempt_number: attempts },
  'Login attempt failed'
);

Roll it out without a big-bang rewrite

A full rewrite of every console.log call on day one is rarely realistic, and trying to force it usually stalls the migration entirely. Here’s a rollout order that gets the highest-value logs converted first without blocking regular feature work.

Step 1: Add the structured logger without removing console.log yet

Install Pino (or your logging library of choice) and wire up the base logger shown above, but leave existing console.log calls in place for now. This gets the logger available to every engineer immediately, with zero risk to code that already works.

Step 2: Route all new code through the structured logger from day one

Any code written from this point forward uses req.log (or the shared logger instance), never console.log. This is a cheap rule to enforce in code review and stops the problem from growing while you convert the existing backlog.

Step 3: Convert your highest-incident-rate services first

Don’t convert services alphabetically or by convenience; convert whichever service pages you most often first. That’s where a trace ID and structured fields save the most investigation time, so it’s where the migration pays for itself fastest.

Step 4: Enforce it with a lint rule once the logger is the default

Add an ESLint rule (no-console) that fails the build on new console.log calls in application code. This locks in the gain from steps 1 to 3 so the codebase doesn’t quietly regress back to unstructured logging six months later.

Step 5: Clean up the remaining call sites opportunistically

Don’t schedule a dedicated migration sprint for the low-traffic, rarely-touched files still using console.log. Convert them whenever you’re already editing that file for a feature or bug fix, so the last mile gets done without ever blocking a sprint.

Mistakes that undo structured logging in practice

Adopting a JSON logger gets you halfway there. These mistakes quietly cancel out the benefit even after the format is right.

Inconsistent field names across services

This is the most common way structured logging silently stops working. If one service logs user_id and another logs userId, a dashboard or alert built on one field misses every log from the other, with no error to tell you it’s happening.

Middleware’s log monitoring auto-parses nested JSON fields into queryable attributes, but that only helps if the field names are consistent enough to group on in the first place. Fix it by owning a shared field dictionary (see practice #3) and enforcing it in code review, not after the fact.

Hiding data inside the message string

Covered in detail under practice #2 above, and worth repeating here because it’s the single most common review comment worth making on a teammate’s logging code. A message like "User user_4471 failed login" looks structured but isn’t queryable; the fix is always to move variable data into its own field.

No one owns the schema

Without a named owner and a short reference doc, field naming conventions drift within a couple of quarters as new engineers join and new services ship. The fix is cheap: one person or team owns the schema doc, and new fields get added there before they get added to code.

Treating logs as a replacement for metrics and traces

Logs are the best tool for “what exactly happened on this one request.” Metrics are better for trends over time, and traces are better for visualizing request flow across services. Teams that try to answer every observability question with log queries end up running expensive aggregations that a metric would answer instantly. Use all three together.

FAQs