Stripe webhooks without double charging anyone

Every payments bug we have been called in to fix looked the same from the outside: a customer got charged twice, or paid and never got the thing they bought. Underneath, the causes are boringly repetitive. Stripe's API is well designed, but it hands you a distributed system whether you wanted one or not, and the happy-path integration quietly assumes network calls succeed exactly once. They don't. Here is what we build into every Stripe integration, and why each piece exists.

Idempotency on the write path

Any POST to Stripe accepts an Idempotency-Key header. Stripe stores the key alongside the response it produced. If the same key arrives again within 24 hours, Stripe does not perform the operation; it replays the original response body and status, with an Idempotent-Replay: true header. That is the whole mechanism, and it turns "did my PaymentIntent get created before the connection dropped?" from a guess into a retry.

The common mistake is generating the key with crypto.randomUUID() at call time. A random key is fresh on every attempt, so a retry after a timeout creates a second PaymentIntent, and now you have two authorizations against one cart. The key has to be derived from something stable in your own domain:

const key = `pi:create:${order.id}:${order.version}`;

Order id plus a version counter that increments only when the amount or currency legitimately changes. Deterministic, survives a process restart, survives a queue redelivery.

The subtle failure is reusing a key with different parameters. Send the same Idempotency-Key with a different amount and Stripe returns a 400 idempotency_error, not the new charge. We hit this where the key was order-{id} and a coupon applied after the first attempt changed the total. Checkout was hard-broken for anyone using a promo code. Hence the version suffix.

Verifying the webhook signature

Stripe signs every webhook with the endpoint's signing secret and sends it in the Stripe-Signature header as a timestamp plus one or more HMAC-SHA256 signatures. You verify with stripe.webhooks.constructEvent in Node or stripe.Webhook.construct_event in Python.

The trap: verification runs over the raw request bytes. Any JSON body parser that has already deserialized and re-serialized the payload changes key ordering or whitespace, and the signature fails with no obvious clue. In Express you need express.raw({ type: 'application/json' }) on that route only. In Next.js App Router, await req.text(). In FastAPI, await request.body().

Two more details worth holding onto:

• The default tolerance window is 300 seconds. If your container's clock drifts past five minutes, every webhook fails verification. We have seen this on a self-hosted box with no NTP running. Alert on clock skew.
• Every endpoint gets its own signing secret, and test mode secrets differ from live. Store them as separate env vars (STRIPE_WEBHOOK_SECRET_ORDERS, ..._CONNECT) rather than one shared value, so rotating one endpoint doesn't take down the others.

Delivery semantics you have to design around

Stripe webhooks are at-least-once. Three facts follow from that:

1. Duplicates happen. A network blip on your side means Stripe never saw your 200, so it sends the event again. Same event id, same payload.
2. Order is not guaranteed. We have seen payment_intent.succeeded land after charge.refunded for the same charge. If your handler naively writes status from whichever event arrives last, the order flips back to paid after a refund.
3. Retries continue for up to 3 days with exponential backoff, in live mode. Any 2xx is an ack; anything else, including a 500 from an unhandled exception, is a retry.

So the handler must be fast. Verify, record, enqueue, return 200. All real work happens in a background worker. On one client's checkout the handler was calling their ERP synchronously; the ERP took 12 seconds under load, Stripe timed out at 10, and every slow order got processed three or four times.

The event ledger

Deduplication belongs in the database, not in application logic. One table, one unique constraint, and the insert itself is the lock.

create table stripe_events (
  event_id      text primary key,
  event_type    text        not null,
  api_version   text        not null,
  created_at    timestamptz not null,      -- Stripe's `created`, not ours
  received_at   timestamptz not null default now(),
  processed_at  timestamptz,
  attempts      int         not null default 0,
  payload       jsonb       not null
);

create index stripe_events_unprocessed_idx
  on stripe_events (received_at)
  where processed_at is null;

The handler inserts with on conflict (event_id) do nothing and checks the row count. Zero rows means we already have it: ack and stop.

export async function POST(req: Request) {
  const raw = await req.text();
  const sig = req.headers.get("stripe-signature")!;

  let event: Stripe.Event;
  try {
    event = stripe.webhooks.constructEvent(raw, sig, process.env.STRIPE_WEBHOOK_SECRET_ORDERS!);
  } catch (err) {
    // Bad signature or stale timestamp. Never retry-able, so don't 500.
    return new Response("invalid signature", { status: 400 });
  }

  const { rowCount } = await db.query(
    `insert into stripe_events (event_id, event_type, api_version, created_at, payload)
     values ($1, $2, $3, to_timestamp($4), $5)
     on conflict (event_id) do nothing`,
    [event.id, event.type, event.api_version, event.created, event.data.object],
  );

  if (rowCount === 1) {
    await queue.enqueue("stripe.process", { eventId: event.id });
  }

  return new Response(null, { status: 204 });
}

Out-of-order arrival is handled in the worker, not here. Before applying a state change we compare the event's created timestamp against the last applied timestamp on the order, and for objects that carry one, the Stripe object version. Older event, drop it. This is the same pattern as a last-write-wins register, and it is what stops a late succeeded from resurrecting a refunded order.

Fulfilment follows the webhook, never the redirect

The success page is a courtesy, not a signal. The user closes the tab, the phone loses signal in a lift, the redirect fires but your page throws on render. The money moved regardless. Any code path that grants entitlement, ships an order, or provisions a seat must be triggered by checkout.session.completed or payment_intent.succeeded in the worker. The success page reads state that the webhook wrote, and shows a "confirming your payment" spinner if it hasn't landed yet. Usually it lands in under two seconds.

Reconciliation

Webhooks can be silently misconfigured. A nightly job lists Stripe charges for the previous 48 hours (overlapping window, deliberately) and diffs them against orders:

• Charge in Stripe with no matching paid order: customer paid, got nothing. Page someone.
• Order marked paid with no charge: a bug in your own code, or a test-mode key in production.
• Amount mismatch, usually partial refunds or disputes applied on Stripe's side only.

Drift over a threshold posts to Slack. The monthly client report is a one-page CSV: gross charges, refunds, disputes, net, and any unresolved rows.

Testing it properly

stripe listen --forward-to localhost:3000/api/stripe/webhook gives a local endpoint with its own signing secret, and stripe trigger payment_intent.succeeded fires real events. For subscriptions, test clocks let you advance a customer 35 days and watch renewals and dunning without waiting.

The test that earns its keep in CI: build a signed payload, POST it twice, assert one order row and one fulfilment record. It fails loudly the first time someone adds a side effect outside the dedupe path.

How we work

We treat payments as its own workstream on every build. The event ledger, the reconciliation job, and the replay test go in during the first Stripe ticket, not after launch, because retrofitting idempotency into a live checkout means reasoning about charges that already exist. Clients get the reconciliation report from day one, so the first month of real traffic is measured rather than assumed. It costs maybe a day and a half of extra work up front, and it is the difference between a payments integration you can leave alone and one that generates support tickets every weekend.