← Back to documentation

Failure-Rate Alerts

Set a per-endpoint failure-rate alert and optional automatic pause.

6 min read

A failure-rate alert counts the recent endpoint outcomes. It notifies the selected relay targets when the failure rate is equal to a limit or higher.

Concept#

For each endpoint with the alerts enabled, PayloadRelay counts the recent request outcomes in the configured time window, and it calculates this value:

Code Example
failureRatePct = (failed_requests / total_requests) * 100

The calculation uses these rules:

  • PayloadRelay counts a request in the window when its outcome becomes final, and not when the request arrives. A delivery that fails after its retries counts in the window in which it fails.
  • total_requests does not contain these outcomes: the synthetic test outcomes (TEST_ACCEPTED, TEST_REJECTED, TEST_RATE_LIMITED), PENDING, FAILOVER_TRIGGERED, PAUSED, WEBHOOK_INGESTION_DISABLED, and EMAIL_INGESTION_DISABLED. A PENDING request counts when its outcome becomes final.
  • failed_requests contains every failure outcome in total_requests. This includes the rejections at ingress, for example AUTH_FAILED, and the final failed deliveries. A completed outcome is not a failure.
  • After an alert fires, the next examination counts only the outcomes that became final after that alert.

PayloadRelay rounds the failure rate that it shows to two decimal places. The alert fires when the traffic reaches the minimum request count and the rounded rate reaches the threshold. The window must also have one failure as a minimum. So a threshold of 0 fires on any failure.

PayloadRelay does not examine a paused endpoint or a disabled endpoint.

Configuration#

FieldRangeDescription
Enable failure-rate alertsOn / OffEnables or disables the failure-rate alert for this endpoint.
Failure-rate threshold (%)0 to 100 %The rounded rolling failure rate that fires the alert. With 0, any failure fires the alert.
Rolling window1 minute, 5, 15, 30, or 60 minutes, 6 hours, or 24 hoursThe rolling time window for the outcome counts.
Minimum total requests1 to 10,000The number of non-test requests that PayloadRelay needs before it examines the threshold. This prevents an alert for a low-volume endpoint.
Cooldown after firingNo cooldown, 5, 15, 30, or 60 minutes, 6 hours, or 24 hoursThe time in which PayloadRelay stops the duplicate alerts after an alert fires. With No cooldown, the next examination can fire again while the condition continues.
Auto-pause on alertOn / OffPauses the endpoint for 30 minutes after an alert fires, if the delivery-failure rate also reaches the threshold. During the pause, the inbound traffic receives 503 and creates no Activity row.
Notify targetsA maximum of 25 relay targets in the same organizationThe alert targets: the relay targets that receive the alert notification. PayloadRelay does not support a Google Sheets target for an alert.

Configure the alerts in the Alerts tab of the endpoint editor.

The editor gives the listed values for Rolling window and Cooldown after firing. The API accepts a value in minutes: alertWindowMinutes from 1 to 1440, and alertCooldownMinutes from 0 to 1440.

The Alerts tab with the failure-rate threshold, the window, the notified relay targets, and auto-pause

Rules for the alert targets:

  • If the alert is enabled, select one alert target as a minimum, or enable auto-pause. Otherwise the save returns 400 with Select at least one alert target, or enable auto-pause.
  • An endpoint can already have an enabled alert with no target and no auto-pause. A save that keeps this configuration succeeds. That alert notifies nobody, and only the alert history records it.
  • An email relay target must be confirmed before you add it. An SMTP relay target must also be verified.
  • If an attached email relay target stops being ready, it stays attached when you save. PayloadRelay skips that target when it sends an alert.
  • If PayloadRelay skips every target, the alert still starts the cooldown. The alert history records 0 notified targets.

PayloadRelay keeps the alert history for 30 days.

Auto-pause behavior#

Auto-pause uses a second rate that counts only the deliveries:

Code Example
deliveryFailureRatePct = (DELIVERY_FAILED / (ACCEPTED + EMAIL_INGESTION_ACCEPTED + DELIVERY_FAILED)) * 100

When an alert fires, PayloadRelay calculates this rate for the same window. The rate uses the same threshold and the same minimum requests. If the rate reaches the threshold, PayloadRelay rejects the inbound relay traffic with 503 Service Unavailable for 30 minutes. A rejection at ingress, for example AUTH_FAILED, can fire the alert, but it cannot pause the endpoint.

The traffic during the pause creates no Activity row. Auto-pause does not make a longer manual pause shorter. You can resume the endpoint at any time.

Cooldown semantics#

After an alert fires, the configured cooldown stops the duplicate alerts. An alert can fire again after the cooldown while the rate stays at the threshold or higher.

Alert payload#

A webhook target receives this JSON payload directly:

Code Example
{
  "event": "payloadrelay.failure_alert",
  "endpointId": "uuid",
  "endpointName": "...",
  "failureRatePct": 87.50,
  "totalRequests": 16,
  "failedRequests": 14,
  "windowMinutes": 15,
  "autoPaused": true,
  "synthetic": false,
  "firedAt": "2026-04-25T13:00:00Z",
  "dashboardUrl": "https://app.payloadrelay.com/endpoints/details?endpointId=<id>"
}

An email target receives an HTML notification with the subject Failure-rate alert for <endpoint name>. The email shows the endpoint name, the failure rate, the request counts, the window, the auto-pause state, and an endpoint link. An email relay target in SMTP relay mode sends the alert through the SMTP server of that relay target.

A Slack target and a Discord target receive one summary line, for example PayloadRelay: endpoint 'orders-production' failure-rate 87.50% (14/16) over last 15 minutes. If PayloadRelay paused the endpoint, the line also says so. A Microsoft Teams, a PagerDuty, or a Telegram target receives the JSON details in the delivery format of that destination. For Microsoft Teams, PayloadRelay puts the JSON details in the text of a MessageCard. For Telegram, the message title is <endpoint name> alert. For PagerDuty, see PagerDuty.

If the delivery of a live alert fails, PayloadRelay can send it again a maximum of 3 times. This applies to a webhook, Slack, Discord, Microsoft Teams, PagerDuty, or Telegram target. PayloadRelay does not retry a test alert.

Test alerts#

To send a synthetic alert to all the configured targets, use the Send test alert button in the editor. PayloadRelay marks a test alert as synthetic. The email subject, the Slack line, and the Discord line of a test alert start with [Test].

The test alert uses the saved alert configuration. You can use the button only when the saved alert is enabled and has one alert target as a minimum. If the alert configuration has unsaved changes, the button is disabled. Save the endpoint first.

The test alert contains sample numbers that fire the saved configuration:

  • totalRequests is the Minimum total requests value.
  • failedRequests is the smallest number of failures that reaches the threshold, with a minimum of 1.
  • failureRatePct is the rate of these two numbers.

If no configured target can receive the alert, PayloadRelay rejects the test. If a temporary service problem stops the handoff, the request returns an error that you can retry. A successful response means that PayloadRelay gave the alert to one target as a minimum. It does not mean that a destination received the alert.