← Back to documentation

Troubleshooting Ingest

Find 400, 401, 413, 429, and 503 errors with Activity and the endpoint configuration.

5 min read

This page covers the common errors that occur when you send data to a PayloadRelay relay URL. It gives one action for each error.

Where to look first#

Open Request activity in the dashboard. PayloadRelay records the accepted requests and most rejected requests with these fields:

  • Time (UTC): the time when the request arrived.
  • Endpoint: the endpoint in the cross-endpoint Request Log view.
  • Outcome: a label such as Completed (ACCEPTED), Auth Failed (AUTH_FAILED), Method Not Allowed (METHOD_NOT_ALLOWED), or IP Blocked (IP_BLOCKED).

Expand a row to read its error reason and the HTTP method of the attempt. A row that is not aggregated also shows the source IP. The Activity table does not store the HTTP response status, and it does not show that status as a column.

To make the results narrow, filter by endpoint, date window, All/Completed/Pending/Errors/Test, source IP, or error text. For the filter details, see Observability.

A request that PayloadRelay rejects only because the endpoint is disabled or paused creates no Request activity row. Read the current state in the endpoint details. The audit history records the changes that a user makes, such as a manual disable, a pause, or a resume. It does not record a scheduled maintenance-window pause.

401 Unauthorized#

Cause: the inbound authentication failed. The endpoint needs authentication, but the request had no authentication header, or it had wrong credentials.

Resolution:

  1. Open the endpoint edit page and read Security -> Inbound Auth.
  2. Make sure that the authentication type and the configured header name match the sender.
  3. Send the request again with the correct header. Examples:
    • Authorization: Bearer YOUR_TOKEN
    • Authorization: Basic <base64(user:pass)>
    • X-Api-Key: YOUR_KEY (or your configured header name)

For examples in each language with authentication headers, see Sending Data.

413 Payload Too Large#

Cause: the request body is larger than the maximum permitted size.

Resolution: make the payload smaller. To send a large binary file, upload it to object storage such as S3 or GCS. Send only the reference URL through PayloadRelay.

400 Bad Request: content validation failed#

Cause: the request passed the authentication, but the content validation failed. The common causes are:

  • A necessary header on the endpoint is missing.
  • A body filter rejected the payload (REQUEST_BODY_FILTER_FAILED).
  • A field validation rule failed, such as a necessary field with a null value or a regular expression that does not match.

A body filter uses the raw decoded request body for JSON, XML, and text. It uses the raw form body for a form endpoint and the raw query string for a query-parameter format. All the body filter rules must pass.

Resolution:

  1. Read the Activity log entry. The outcome field usually gives a cause.
  2. Open the endpoint Security tab and make sure that the required headers are correct. A header name ignores the case. A value is case-sensitive, and PayloadRelay does not remove the spaces from an incoming value.
  3. Open the Filter tab and read the body filters and the field validations.
  4. Add the missing headers, or correct the payload to agree with the configured rules.

429 Too Many Requests#

Cause: the traffic is more than one of two rate limits:

  1. The monthly quota is empty: the traffic is more than the monthly accepted-event limit of the organization. The public Retry-After: 60 value is a retry hint. It is not the quota reset window. Open Billing and read Current plan -> Monthly usage. To get a higher limit, upgrade the plan.
  2. The traffic is more than the endpoint rate limit: the request count is more than the per-minute limit of the endpoint. Read the Retry-After value in seconds, and send the request again. If it is necessary, increase the endpoint limit or group the requests at the sender.

For the two causes, add an exponential backoff to the sender. If the sender receives 429, it must read Retry-After when that header is present, wait, and send the request again.

503 Service Unavailable#

To select the recovery path, read the response body and the Retry-After header.

Endpoint temporarily unavailable#

A response with code: "endpoint_unavailable" and Retry-After: 60 shows that the endpoint cannot accept events at this time. The public response does not show the difference between a pause, a billing block, an access block, and a configuration problem on the operator side. Wait for the given delay, then send the original request again. To find the cause, an organization administrator must read the endpoint configuration, the pause state, the authenticated Activity, and Billing.

Delivery queue temporarily unavailable#

A response with code: "delivery_queue_unavailable" and Retry-After: 5 shows that PayloadRelay did not complete the acceptance, because the delivery processing was not available. A delivery can still occur. Keep the receivers idempotent (safe to process more than one time), wait for the given delay, and send the original request again. Activity records DELIVERY_QUEUE_UNAVAILABLE.

A hit on a paused endpoint uses the same endpoint_unavailable response, and it creates no Request activity row. If the endpoint is paused, open its edit page and select Resume, or wait for the scheduled pause to expire.

Endpoint disabled (no response or 404)#

Cause: the endpoint is disabled. The Enabled control is off.

PayloadRelay rejects a hit on a disabled endpoint, and the hit creates no Request activity row.

Resolution: open the endpoint and set the Enabled control to on.

General debugging tips#

  • In Request activity, select Errors and read the rows with the label Auth Failed (AUTH_FAILED) to find an authentication problem.
  • To see the headers that the sender sends, use curl -v.
  • Compare the request with the snippets on the endpoint edit page. The snippets contain the authentication headers and the required headers for the endpoint configuration.
  • If the problem continues, open a support ticket on the Support page.

Next steps#

← Back to documentation