← All posts
general

Design a Payment System: System Design Walkthrough

Payment system questions are how interviewers test whether you can design for correctness rather than throughput. The QPS is modest by big-tech standards; the difficulty is that every failure mode has a dollar amount attached, "retry it" can mean "charge them twice", and the source of truth isn't even your database — it's a bank's. Senior loops at fintechs, marketplaces and infra teams lean on this question hard.

Requirements

Take a marketplace checkout (buyer pays, platform takes a fee, seller gets the rest):

  • Accept a payment for an order via an external payment service provider (PSP — Stripe, Adyen).
  • Never lose a payment, never double-charge — through crashes, timeouts and retries.
  • Keep an auditable record of every money movement.
  • Handle asynchronous outcomes: webhooks, delayed failures, refunds, chargebacks.
  • Scale target: thousands of payments/minute — correctness dominates, and saying so is the right framing.

Idempotency: the heart of the question

The defining scenario — narrate it explicitly: you call the PSP, the request times out. Did the charge happen? You don't know. Retry and you may double-charge; don't retry and you may lose the sale.

The answer is an idempotency key: a unique ID per payment attempt, generated when the user clicks pay, stored with the attempt, and sent with every retry. The PSP deduplicates on it (all serious PSPs support this), so N deliveries of "charge order 123, key abc" produce exactly one charge. Extend the same discipline inward: your own APIs accept client-generated keys, and your handlers treat every message as at-least-once-delivered. Exactly-once execution is a lie; exactly-once effect via idempotent handlers is the engineering. Say that sentence in the interview.

The payment flow

  1. Checkout creates a payment_intent row — state CREATED, amount, currency, idempotency key — before any external call (crash-safe from the first moment).
  2. Call the PSP with the key; move to PROCESSING.
  3. The PSP's synchronous answer is provisional. The authoritative outcome arrives by webhook — verify its signature, look up the intent, transition to SUCCEEDED/FAILED.
  4. Webhooks are themselves at-least-once and unordered: process them idempotently (state-machine transitions that ignore replays and stale events).
  5. On success, write the ledger entries (next section) and emit an event for fulfilment.

The state machine (CREATED → PROCESSING → SUCCEEDED/FAILED, plus refund states) is your anchor artifact — with a background reconciler that finds stuck PROCESSING intents and queries the PSP rather than guessing.

The ledger: double-entry or regret

Store money movements as an append-only double-entry ledger: every transaction is balanced debits and credits across accounts (buyer's payment, platform fees, seller payable, PSP fees). No row is ever updated — corrections are new compensating entries.

Why interviewers care: balances become derived facts (sum of entries), every cent is traceable to a transaction, and "how much do we owe seller X?" is a query, not a hope. Mutable balance columns are the junior tell; the ledger is the senior one. Use serializable transactions or per-account sequencing for concurrent writes — this is one of the few places "just use Postgres" is the strong answer.

Reconciliation and the unhappy paths

  • Reconciliation: every day, diff the PSP's settlement report against your ledger. Discrepancies become work items, not silent drift — banks' numbers win arguments, and interviewers reward knowing reconciliation exists at all.
  • Refunds are new transactions referencing the original (never edits), flowing through the same intent → PSP → webhook → ledger pipeline.
  • Chargebacks arrive uninvited via webhook: money already gone, plus a dispute fee — ledger entries again, plus an evidence workflow.
  • Payouts to sellers are a mirrored flow (payable account → bank transfer → confirmation), with its own reconciliation.

Reliability patterns to name

At-least-once delivery with idempotent consumers everywhere; outbox pattern so "commit the state change" and "publish the event" can't diverge; retries with exponential backoff and jitter against a struggling PSP; a dead-letter queue with human alerting — in payments, dropped-after-N-retries means someone's money is in limbo, so a person must look.

How this plays in an interview

A strong pass spends its minutes where the money is: idempotency narrated through the timeout scenario (10), the intent state machine + webhook asynchrony (10), the double-entry ledger (10), reconciliation and unhappy paths (10). What fails candidates: designing for a million QPS while double-charging on the first retry. Correctness first, scale second — the reverse of the feed and chat questions, which is exactly why loops pair them.

Round out the set with the rate limiter and URL shortener, or go deep with WhatsApp. For a live payments mock with a fintech interviewer, get matched with a coach.