Skip to content
All case studies

Case study 01Flagship2024

Real-time Payments Ledger

An event-sourced, double-entry ledger that replaced nightly batch reconciliation at a fintech.

Role
Tech lead
Team
5 engineers
Timeline
7 months
Focus
Backend, Platform

0

reconciliation drift since launch

38 ms

p99 posting latency

−90%

manual finance ops time

Commands and processor events enter one ledger service that writes balanced postings in a single transaction, then fans out through an outbox.Processor webhooks to Ledger service (events); Product API to Ledger service (commands); Ledger service to PostgreSQL (serialisable txn); Ledger service to Kafka (outbox); Kafka to Balance projections; PostgreSQL to ReconcilerProcessor webhooksat-least-onceProduct APIpayment commandsLedger serviceGo · idempotency keysPostgreSQLappend-only postingsKafkaposting eventsReconcilerdaily diff vs processorBalance projectionsread modelseventscommandsserialisable txnoutbox
Commands and processor events enter one ledger service that writes balanced postings in a single transaction, then fans out through an outbox.

The problem

Balances came from batch jobs that ran overnight and regularly disagreed with the processor. Finance spent days each month reconciling by hand, support fielded tickets about stale balances, and every new payment feature added another special case to the batch code.

What I owned

  • The design doc and data model: accounts, postings and the invariants the system enforces.
  • The Go ledger service, its idempotency layer and the transactional outbox to Kafka.
  • The six-week shadow-mode rollout and the account-by-account cutover plan.
  • Runbooks, dashboards and the on-call handover to the platform team.

Key decisions

  1. 1

    Double-entry, append-only postings

    Every money movement is a balanced set of immutable postings. Balances are derived, never stored as mutable truth, so any balance can be explained line by line.

  2. 2

    Idempotency keys on every write path

    Processor webhooks are delivered at least once. Each command carries a key stored in the same transaction as its postings, so retries can't double-post.

  3. 3

    Serialisable Postgres over a distributed store

    Postings need strict ordering per account, not global scale. Serialisable transactions on Postgres gave us correctness with tools the team already knew how to operate.

  4. 4

    Shadow mode before cutover

    Old and new systems ran side by side for six weeks. A reconciler diffed every balance daily, and we migrated accounts in cohorts only after a clean week.

Outcome

The ledger became the source of truth for every customer balance. Finance stopped reconciling by hand, new payment products plug into one posting API, and the reconciler has reported zero drift since cutover.

  • Go
  • PostgreSQL
  • Kafka
  • gRPC
  • Kubernetes
  • OpenTelemetry