Moderating reviews and comments automatically with AI in n8n

An e-commerce site collecting product reviews, a blog with open comments, a forum or a Discord community: as soon as users can post content, someone has to read it. Promotional spam shows up fast, insults follow, and between the two stretches a grey zone of borderline messages no simple rule can settle. Moderating everything by hand doesn't scale; publishing unfiltered exposes your brand and, for manifestly illegal content, potentially triggers legal takedown obligations. An n8n pipeline that classifies every piece of content on arrival, publishes the obvious cases and sends the doubtful ones to a human offers the best of both worlds.

Why manual moderation doesn't hold up

The problem with UGC (user generated content) isn't per-item difficulty — deciding whether "great product, fast shipping" is publishable takes two seconds — it's throughput and availability:

• Spam arrives in waves, often overnight or on weekends, when nobody is watching the queue;
• An insulting or defamatory comment left visible for twelve hours does more damage than a comment held ten minutes too long;
• Decision consistency degrades when several people moderate according to their own reading of the rules, with no trace of who decided what.

A study by Gorwa, Binns and Katzenbach published in 2020 in Big Data & Society ("Algorithmic content moderation") analyzes exactly this trade-off: automation is what makes scale manageable, but ambiguous cases — irony, cultural context, harsh but legitimate criticism — still require human judgment. That's precisely the hybrid circuit this pipeline implements: AI settles the obvious cases, humans keep control over the grey zone.

Pipeline architecture

Webhook (CMS, Shopify, Disqus…)
        │
        ▼
Normalization (Set / Code)
        │
        ▼
OpenAI Moderation API ── free first filter
        │
        ▼
Text Classifier / LLM + Structured Output Parser
        │
        ▼
      Switch
  ┌─────┼─────────────┐
  ▼     ▼             ▼
Publish Reject   Human review (Slack)
  │     │             │
  └─────┴──────┬──────┘
               ▼
Audit log (Google Sheets / Postgres)

Step 1 — Intake and normalization

The entry point is a Webhook node: most UGC sources can notify a URL on every new piece of content — a Shopify webhook on product review creation, a WordPress or Ghost hook on new comments, a Disqus webhook, or a plain HTTP call from your own application at submission time.

Each source sends a different payload. A Set node (or Code for the messy cases) normalizes everything into a single format before classification:

{
  "source": "shopify",
  "type": "product_review",
  "text": "Product arrived broken, customer service unreachable...",
  "author_id": "cust_84213",
  "target": "oslo-chair-product",
  "received_at": "2026-08-04T09:12:00Z"
}

Note what's missing: the author's email and full name. The model doesn't need them to classify the text, and not transmitting them makes GDPR compliance considerably simpler (more on that below).

Step 2 — First filter: OpenAI's Moderation API

Before involving an LLM, a call to OpenAI's /v1/moderations endpoint via an HTTP Request node weeds out the severe cases. This dedicated moderation endpoint is free at the time of writing and returns per-category scores (harassment, hate, sexual content, violence…). A comment flagged with a high score can go straight to rejection or human review without going through the full classification.

This first filter has two limitations that justify the rest of the pipeline: it's trained on generic categories (it detects neither promotional spam nor off-topic content), and it knows nothing about your house rules — a competitor posting a link to their own store is neither hateful nor violent, but you still don't want it published.

Step 3 — Classification into four categories

The heart of the workflow classifies each piece of content into one of four categories: publishable, spam, toxic (insults, personal attacks, illegal content) or ambiguous (to be reviewed by a human).

Two options in n8n. The simplest: the Text Classifier node, where you define the categories with a description each — our Text Classifier guide covers its configuration in detail. For finer control (reasons, confidence score), a Basic LLM Chain with a Structured Output Parser enforces an output schema:

{
  "verdict": "publishable | spam | toxic | ambiguous",
  "reasons": ["unsolicited promotional link"],
  "confidence": 0.92
}

The system prompt spells out your rules concretely:

You moderate reviews and comments for an e-commerce site.
Classify each text into exactly one category:
- publishable: an opinion about the product or service, however negative.
  A harsh but substantiated criticism is PUBLISHABLE.
- spam: promotional link, off-topic content, mass-generated text.
- toxic: insult, personal attack, threat, discriminatory content.
- ambiguous: you hesitate between two categories, irony is hard to read,
  or the text makes a serious unverifiable claim
  (e.g. "this product made me sick").
Respond only in the requested JSON format.
When in doubt, choose "ambiguous" rather than guessing.

The last line is the most important one in the prompt: it reverses the model's natural bias toward deciding, in favor of caution. How the parser works and how to handle its failures is covered in our Structured Output Parser guide.

Step 4 — Routing with Switch

A Switch node with four outputs routes on {{ $json.verdict }}, with one safety rule added: any verdict with confidence < 0.8 is treated as ambiguous, whatever the label says. Configuring routing rules is covered in our IF and Switch nodes guide.

• Publishable: an API call to the CMS or Shopify publishes the content (or marks it "approved"). No human involvement.
• Spam: silent rejection. No point notifying the author — a detailed rejection message mostly teaches the spammer how to get around the filter.
• Toxic: rejection with a generic reason on the author's side ("your comment doesn't comply with our publishing guidelines"), and the text kept in the audit log — useful if the content falls under a takedown obligation or gets reported.
• Ambiguous: off to the human review queue.

Step 5 — The human review queue in Slack

For ambiguous cases, a Slack node posts the full text, the provisional verdict, the reasons and two Approve / Reject buttons, then the workflow pauses until someone clicks, thanks to the Wait node. The complete mechanism — interactive buttons, workflow resumption, expiry timeout — is described step by step in our article on human approval with Wait and Slack in n8n.

This hybrid circuit is what makes the system sustainable in both directions: the team only sees the 5 to 15% of genuinely hard cases, and no borderline content gets published or censored on the sole strength of a probability score.

Audit log: essential, not optional

Every decision — automatic or human — gets written to a Google Sheets node (Append) or Postgres node (Insert): text, source, verdict, reasons, confidence, decider (auto or the reviewer's identifier), timestamp. This log serves three concrete purposes: answering a user who disputes a rejection, measuring the false positive rate to tune prompt and thresholds, and documenting your decisions if flagged content becomes the subject of a takedown request.

Read through this log during the first few weeks: that's where you'll see whether the model classifies legitimate negative reviews as "spam" (the costliest false positive, because it looks like review censorship) or lets through a recurring spam pattern that one example added to the prompt would be enough to block.

GDPR: comments contain personal data

A customer review can contain a name, an address, an order number, even health data ("this supplement gave me migraines"). Three reflexes: transmit only the text to the model, without the author's identity metadata; cover the AI provider call with a DPA and mention it in your privacy policy; define a retention period for the audit log and purge it beyond that. And if an author exercises their right to erasure, the log is one of the systems you'll need to cover — our article on automating GDPR requests with n8n shows how to industrialize that part.

Wrapping up

Moderating UGC with n8n comes down to five pieces: a Webhook receiving each piece of content, normalization, a free first filter via OpenAI's Moderation API, classification into four categories with structured output (verdict, reasons, confidence), and a Switch that publishes, rejects, or sends to human review in Slack — all of it traced in an audit log. The guiding principle, validated by research and practice alike: automate the obvious cases, never the doubtful ones. Once this pipeline is in place, the analysis layer extends naturally — for instance toward AI sentiment analysis of customer reviews on the published content.