Backend2026

High-Performance Cryptocurrency Exchange Backend

A distributed matching system built in Go, split into gateway, order-service, matching-engine, and market-data-service, with Transactional Outbox, leader election, batch ingestion, and architecture-level load validation.

Frontend Source Backend Source Live Demo

High-Performance Cryptocurrency Exchange Backend

Project Overview

This project is a high-performance exchange backend built from scratch in Go. The goal was not merely to expose order APIs, but to validate the hard parts of a trading system under load: consistency, failover, event flow, settlement correctness, and real-time market data delivery. The current system is split into four services, gateway, order-service, matching-engine, and market-data-service, connected through PostgreSQL, Redis, and Kafka/Redpanda to handle order intake, matching, settlement, order status updates, and WebSocket streaming. Beyond feature delivery, I also built Prometheus/Grafana observability, k6 architecture-level stress tests, and staging deployment automation on ECS so the project can be measured and operated like a production system.

Technical Challenges & Solutions

Dual-write Consistency and Reliable Event Delivery

Placing an order must lock funds, create the order row, and publish a Kafka event. If the database commit succeeds but message publication fails, the system can end up with inconsistent trading and accounting state.

Solution:

Wrote the order row and outbox message inside the same PostgreSQL transaction. A background outbox worker polls records in batches with SKIP LOCKED and publishes to exchange.orders with at-least-once semantics. Downstream settlement then adds idempotency checks on Trade ID and status transitions, eliminating the classic DB + MQ dual-write gap.

High Availability Without Split-Brain

The matching engine needs standby replicas for failover, but if two replicas consume the same partition at once, duplicate trades and broken balances become possible.

Solution:

Implemented PostgreSQL-backed leader election with a partition_leader_locks table, atomic upsert + where updates, and monotonically increasing fencing tokens. Only the current leader can process partition events, and stale leaders are rejected on subsequent writes.

Deadlocks and Idempotency in Asynchronous Settlement

TX2 settlement updates maker and taker orders, balances, locked funds, and trade records in one path. Under high concurrency, row lock contention and duplicate Kafka delivery can lead to deadlocks or double settlement.

Solution:

Separated settlement into an atomic transaction, acquired Order and Account locks in deterministic order, and added multiple idempotency layers: unique Trade ID constraints, in-transaction rechecks, and skip logic for already-finalized paths. Remaining locked funds for market orders are also unlocked as part of settlement.

High-frequency Batch Ingestion for Market Makers

Market makers and arbitrage bots can submit large bursts of orders. Per-request HTTP overhead and row-by-row inserts quickly cap throughput.

Solution:

Added a dedicated /orders/batch ingestion path backed by pgx.CopyFrom. Before touching PostgreSQL, the service performs two-dimensional in-memory sorting by UserID and Currency to reduce conflicting lock acquisition order and dramatically increase burst-write throughput.

Real-time Fanout and Observable Performance Validation

Trade events must reach large WebSocket audiences without slowing the order path, and without end-to-end metrics it is hard to locate whether bottlenecks live in HTTP, Kafka, matching, or push delivery.

Solution:

Split WebSocket fanout into a dedicated market-data-service that consumes exchange.orderbook, exchange.trades, and exchange.order_updates, isolating slow clients from the core trading path. Added Prometheus metrics and Grafana dashboards, then validated the design with k6 scenarios such as E2E latency, Market Storm, and spike testing.

Architecture

The gateway is the single public entry point, responsible for Redis-based sliding-window rate limiting, Idempotency-Key protection, and reverse proxy routing. order-service handles HTTP APIs and TX1: lock funds, create orders, and persist outbox records before publishing exchange.orders in the background. matching-engine runs in Active-Standby mode using PostgreSQL leader election with fencing tokens, restores live limit orders from the database on cold start, executes in-memory price-time-priority matching, and publishes settlement, trade, and orderbook events. market-data-service focuses only on Kafka consumption and high-fanout WebSocket delivery. Locally the stack runs with Docker Compose; staging is deployed as microservices on AWS ECS via Terraform and ecspresso.

Learnings

This rewrite moved me from building APIs to designing a trading system. I had to solve dual-write consistency, asynchronous settlement, split-brain prevention, batch ingestion, deadlock avoidance, slow WebSocket consumer isolation, and, just as importantly, how to prove the system works under pressure with metrics and load tests. The biggest takeaway was that throughput alone is not enough for high-concurrency systems; the real bar is whether the data stays correct, the failure modes remain understandable, and the system is still operable when parts degrade.

Tech Stack

Backend

Go 1.25GinPrice-Time PriorityLeader Election

Database

PostgreSQLRedispgx v5

Event Streaming & Reliability

Kafka / RedpandaTransactional OutboxAtomic SettlementIdempotency

Deployment

DockerAWS ECSTerraformecspresso

Monitoring & Observability

PrometheusGrafanaCloudWatchk6