Monitoring Guide

Practical recipes for setting up monitors that catch the right kinds of failures. Skip to the section that matches your endpoint.

The mental model

Every monitor is a scheduled HTTP request + a list of assertions. The probe is "up" only when every assertion passes. The monitor is "down" globally only when ≥ quorum regions report "up" failure within the last failureThreshold consecutive checks. (Tunable per monitor.)

When a monitor flips down, the IncidentCoordinatorDO opens an incident and dispatches to every attached alert channel. When it flips back up, you get a recovery notification. Incidents dedupe — you won't get spammed.

Recipe: REST API health endpoint

You have GET https://api.example.com/health returning { "ok": true, "version": "..." }.

Field	Value
URL	`https://api.example.com/health`
Method	`GET`
Interval	60s
Timeout	5000ms
Regions	All 5 (default)
Quorum	2
Failure threshold	2
Assertions	see below

status_code            eq        200
response_time_ms       lt        2000
header[content-type]   contains  application/json
json_path: $.ok        eq        true

The first three guard against the obvious failure modes (HTTP error, slowness, content-type drift). The json_path assertion turns "the server returned 200 OK with junk" into a real failure.

Recipe: Public JSON API (third-party check)

You depend on https://api.example.com/v1/products/123 and want to know if it goes down or starts returning the wrong shape.

status_code             eq        200
response_time_ms        lt        5000
json_path: $.id         eq        123
json_path: $.name       exists
json_path: $.price      gt        0

The exists operator is your friend for fields you require but don't know the exact value of. The gt 0 on price catches the "API returned an object but all fields are null" failure mode.

Recipe: Static site / landing page

You have https://example.com and want to know it serves your real HTML, not a CDN error page.

status_code             eq        200
response_time_ms        lt        3000
header[content-type]   contains   text/html
body_text               contains  Acme Corporation

The body_text contains of a known phrase from your hero copy is the cheapest way to detect "site got replaced with a 200-OK error page" or "deploy half-shipped a partially rendered template."

Recipe: Authenticated endpoint

You want to monitor an endpoint that requires a token. Don't put production secrets in monitor headers — create a dedicated read-only monitoring API token at your service.

URL: https://api.example.com/v1/me
Method: GET
Headers:
  Authorization: Bearer <monitoring-token>
  X-Probed-By: thump

Assertions:
  status_code           eq        200
  response_time_ms      lt        2000
  json_path: $.id       exists

If your API has rate limits, set the interval to ≥ 5 min to stay polite.

Recipe: GraphQL endpoint

GraphQL is POST to a single URL with a query in the body.

URL: https://api.example.com/graphql
Method: POST
Headers:
  Content-Type: application/json
Body:
  {"query":"{ health { status } }"}

Assertions:
  status_code                          eq        200
  json_path: $.data.health.status      eq        ok
  json_path: $.errors                  not_exists

The not_exists on $.errors catches GraphQL's "200 OK with errors in the body" pattern. Critical — GraphQL almost never returns non-200 for query failures.

Recipe: Status / heartbeat ping

You have a backend job that pings https://your-service.dev/heartbeat every 5 minutes. You want to know if it stops.

This is "inverted monitoring" — thump probes a URL. For now, the recommendation is to:

Have your job write last_heartbeat to a public endpoint like https://your-service.dev/last-ping that returns { "ts": <unix-ms> }

Monitor that endpoint and assert the timestamp is recent:

json_path: $.ts        gt        <some-stale-threshold-ms>

Real "missing heartbeat" detection is on the roadmap (we'll let you POST to a thump endpoint and alert on absence), but it's not implemented yet.

Pricing-style assertions you might want

response_time_ms < 1000 — your SLA target. If your p95 is 800ms and you alert at 1000, you'll be the first to know when the curve shifts.
status_code lt 400 — accepts 2xx and 3xx. Useful for endpoints that legitimately return 301/302.
body_size_bytes gt 100 — catches "API returned an empty payload" without locking the exact size.
header[cf-cache-status] eq HIT — confirms your CDN is doing its job. Combine with a region filter to alert if a specific edge starts missing.

Picking interval + quorum

Start with 60s interval + 5 regions + quorum 2. That gives you:

A check from 5 places every minute
Incidents only when 2+ regions agree something's wrong (filters out single-region flakes)
Roughly 5 region-checks/min/monitor — well within thump's free tier

Tune down to 30s + quorum 3 for production-critical endpoints. Tune up to 5min + quorum 1 for non-critical or rate-limited targets.

Setting up alerts

Once your monitor is created, add an alert channel under /app/channels:

Discord — paste your channel webhook URL (https://discord.com/api/webhooks/...)
Slack — paste an incoming webhook URL
Webhook — your own endpoint that receives the canonical incident payload (see apps/api/src/notifications/types.ts)
Email — your inbox (requires the operator to configure Resend; not available on hosted thump until that ships)

Then go back to the monitor and attach the channel under "Alert channels" — or attach the channel to N monitors at once from the channel detail page.

Testing before saving

The monitor-create form has a "Test now" button that runs a one-off probe with your current draft (URL + assertions) and shows pass/fail per assertion + the captured actual value. Use it to tune your json_path expressions before saving — they're notoriously easy to typo.

Public playground

Anyone can probe a URL at https://thump.dev — paste it into the "Try it right now" widget on the landing page. Rate-limited to 20 probes / 5 min per IP. No signup. Useful for one-off "is this URL up?" checks.