Automatically triaging Sentry alerts with AI in n8n

A team running several services in production quickly starts receiving dozens, sometimes hundreds, of Sentry alerts a day: one-off exceptions with no real impact, minor regressions on a marginal browser, and — buried somewhere in the middle — the error signaling that checkout has been broken for everyone for the last ten minutes. The research on alert fatigue is unambiguous about this mechanism: a 2025 study by Tariq et al. published in ACM Computing Surveys ("Alert Fatigue in Security Operations Centres", see on Google Scholar) shows that a buildup of low-information alerts desensitizes operators and delays the detection of real incidents — a mechanism documented in industrial control rooms just as much as in security operations centers. An n8n pipeline that triages Sentry alerts before they reach a human tackles this problem head-on.

Why Sentry alone isn't enough

Sentry already groups events by error fingerprint and offers threshold-based alert rules (event count, rate, new users affected). But those rules can't answer the question that actually matters to a human receiving the notification: is this worth interrupting what I'm doing right now? A NullPointerException on a rarely-used internal endpoint and a timeout on the payment service trigger the same kind of Slack notification, with the same apparent urgency. The sorting stays manual — and that's exactly what AI can automate without replacing the final call.

Pipeline overview

The setup fits in five steps:

1. Ingestion: a Webhook node receives the payload sent by a Sentry alert rule configured with the "Send a notification via webhook" action (see the official Sentry webhooks documentation).
2. Deduplication: before any AI call, a lookup checks whether the error's signature (title + culprit + environment) has already been processed recently.
3. AI classification: an LLM evaluates the real severity, the likely service involved, and an actionable summary.
4. Routing: based on the score, the alert goes to a dedicated Slack urgent channel, a normal tracking channel, or is simply logged.
5. Human escalation: critical cases trigger a notification with an acknowledgment button, following the pattern described in our guide on human approval with Wait and Slack.

This breakdown directly mirrors the sort/prioritize/digest logic of the Inbox AI Pack (€79), applied here to a stream of production errors instead of a mailbox.

Step 1: receive alerts without polling

In Sentry, on the relevant project, create an alert rule (Alerts → Create Alert Rule) with the condition you want ("An issue is first seen", "The issue changes state to escalating", or an event-count threshold), then add the Send a notification via an integration → Webhook action pointing to your n8n Webhook node's URL. Every trigger pushes a JSON payload with the issue ID, its title, level, culprit (the function or file at fault), and environment tags — no need to poll the Sentry API on a schedule.

For occasional inspection needs (listing open issues, updating a status), n8n's native Sentry.io node is a good complement to the webhook, but for real-time triage, the webhook stays the simplest and least request-hungry entry point.

Step 2: deduplicate before calling the AI

A single production error can generate dozens of events within a few minutes. Classifying every one of them with an LLM would be both wasteful and pointless: a Data Table node (see our n8n Data Tables guide) stores a simple fingerprint — a hash of the title and culprit — with a timestamp. An IF node checks whether that fingerprint already exists within the last hour: if so, the workflow bumps an occurrence counter and stops there; if not, it proceeds to classification and logs the new fingerprint. This mirrors the principles covered in our article on webhook idempotency, applied here to error signatures instead of transaction IDs.

Step 3: classify with a structured output

An AI Agent node (or Basic LLM Chain) receives the title, culprit, environment, and the first lines of the stack trace, with a prompt requesting structured output along these lines:

{
  "severity": "critical | high | moderate | low",
  "likely_service": "payments | auth | public-api | internal | unknown",
  "actionable_summary": "one sentence explaining what's broken and for whom",
  "justification": "why this severity level"
}

As with any production LLM call, lock this format down with a Structured Output Parser: a malformed JSON response or an out-of-enum field should be caught automatically, not crash the triage workflow itself. An economical model (gpt-4o-mini or equivalent) is more than enough for this short classification task — save a pricier model for a deeper stack trace summary, and only for cases already flagged as critical.

Writing a prompt that actually distinguishes severity levels

The quality of the triage depends almost entirely on concrete per-level examples in the prompt, not on model sophistication. An explicit rubric ("critical = payments, auth, or public API broken for all users" vs. "low = cosmetic error or edge case on a marginal browser") sharply cuts down false positives. This is the same principle detailed in our guide on support ticket scoring: a justification field in the output makes every decision auditable, and a misclassification becomes visible immediately instead of staying a black box.

Step 4: route by severity

A Switch node then directs the enriched alert:

Severity	Destination	Behavior
Critical	#incidents Slack channel + escalation	Immediate notification, acknowledgment button (see Wait + Slack)
High	#prod-alerts Slack channel	Standard notification, no wake-up
Moderate / Low	Logging only	Row added to a Supabase table, surfaced in a daily digest

The daily digest for moderate and low-severity alerts uses the same mechanics as the daily email digest in the Inbox AI Pack: a batched summary instead of a per-occurrence notification, so background noise stays reviewable without interrupting anyone.

Step 5: keep a trail of everything

Every processed alert — critical or not — deserves a line in a logging table (issue, assigned severity, service, timestamp, link to the Sentry issue). That trail serves two purposes: recalibrating the prompt when a category generates too many false positives, and feeding a post-mortem if a critical alert was misclassified. Teams wanting a more formal trail, with long-term retention and an automated summary report, will find the same logic in the Compliance & Audit Pack (€149), originally built for regulatory needs but directly reusable as a technical incident log.

Track the cost, and cover the failure modes

A pipeline that runs on every new error signature — potentially dozens of times a day on an active instance — needs to account for its own cost from day one; see our guide on tracking AI call costs. And like any automation that protects production, this workflow needs its own Error Workflow: if the classification node fails (API timeout, quota exceeded), the alert should fall back to a raw Slack notification instead of silently disappearing — the worst possible outcome for a pipeline meant to catch emergencies.

Where to go from here

This pipeline replaces neither Sentry nor the on-call engineer: it adds a layer of judgment between the raw volume of alerts and human attention, exactly the role AI triage already plays for a mailbox in the Inbox AI Pack. Start small — a single alert rule, two severity levels — before extending triage across an organization's full set of Sentry projects. The discipline that matters most isn't which model you pick, but the quality of the severity examples in the prompt and how rigorous the safety net is when the pipeline itself fails.