# Retries and replay

> What happens when your endpoint is down: the retry schedule, how long Pidgeon keeps trying, and how to ask for a delivery again.

Delivery is a queue, not a side effect of ingestion. An event is written as a
row that says "this endpoint owes this event", and a scheduled job sends it —
which is the only reason retries can exist at all. An in-request `POST` has
nowhere to record that it should be tried again.

## The schedule

Seven attempts across a little over seven hours, backing off each time.

| Attempt | After the one before |
| ------- | -------------------- |
| 1       | immediately          |
| 2       | 10 seconds           |
| 3       | 30 seconds           |
| 4       | 2 minutes            |
| 5       | 10 minutes           |
| 6       | 1 hour               |
| 7       | 6 hours              |

Anything that answers `2xx` is done. Anything else is retried, including a
timeout — which is why a handler should acknowledge first and work
afterwards.

## The dead letter

After the seventh attempt the delivery stops and the event stays. Nothing is
discarded, so an endpoint that was down for a morning is a morning of events
waiting rather than a morning of events lost.

They are listed under **Developers → Events**, with the response code and
body of every attempt, and each one can be replayed by hand once the endpoint
is fixed.

```bash
curl https://pidgeon.ai/api/v1/deliveries/5f2c8e1a-3b7d-4c09-9e61-2a4d8b0f7c35/replay -X POST -H "Authorization: Bearer $PIDGEON_API_KEY"
```

```ts
await webhooksReplay({ path: { id: '5f2c8e1a-3b7d-4c09-9e61-2a4d8b0f7c35' } });
```

```python
from pidgeon.api.webhooks import webhooks_replay

webhooks_replay.sync(client=client, id="5f2c8e1a-3b7d-4c09-9e61-2a4d8b0f7c35")
```

## Delivery is at-least-once

A `2xx` that never reaches us — a connection dropped after your handler
committed — is a delivery we will try again. Handlers have to tolerate seeing
the same event twice.

Every event carries a stable `id`. Recording the ones you have processed and
returning early on a repeat is the whole of the fix.

```json
{ "id": "0044ed22-cbcb-4455-a945-06f9699dfca0", "type": "message.received" }
```

## Meta-events are logged, never delivered

`webhook.delivered` and `webhook.failed` appear in the event log so you can
see what happened. They are not themselves delivered to your endpoint,
because an endpoint that is failing is the last place to send news that it is
failing.