Chat App Architecture: Separate Delivery from Storage

A reliable chat system should not treat a live socket as the source of truth. The durable message record and the real-time delivery path solve different problems. Persist the message first or through an idempotent write path, then use a real-time channel to make new state visible quickly. When the connection disappears, the client should be able to recover from durable storage without guessing what it missed.

That separation sounds small, but it changes almost every failure mode in a messaging product. A dropped WebSocket becomes a delivery problem instead of a lost-message problem. Reconnecting becomes synchronization instead of blind resubscription. Read receipts, retries, ordering, and presence can evolve independently rather than accumulating inside one giant connection handler.

Start with four responsibilities

A useful chat system can be reasoned about as four cooperating parts:

1. Client state renders conversations, keeps optimistic sends visible, and remembers a synchronization cursor.
2. Command API authenticates writes and applies server-side rules before accepting a message.
3. Durable message store owns the canonical conversation history.
4. Real-time delivery service notifies connected clients that durable state has changed.

The key rule is simple: delivery accelerates visibility; storage establishes truth.

Cloud Firestore models chat messages as documents in per-room subcollections, while its real-time listeners deliver an initial snapshot and subsequent changes. Firebase also documents long-lived real-time connections and automatic reconnection behavior when discussing real-time queries at scale.

A WebSocket can be part of the delivery layer, but it should not become the database. The WebSocket API provides two-way interactive communication without polling. It does not, by itself, define durable storage, replay, idempotency, or message ordering.

Model the message as durable state

Every accepted message should receive a stable identity that survives retries and reconnections.

Message {
  id
  conversationId
  senderId
  clientRequestId
  body
  createdAt
  sequence
}

clientRequestId is especially valuable. The client generates it before sending, and the server enforces uniqueness within an appropriate scope. If a timeout happens after the server committed the write but before the client received the response, retrying the same command can return the existing message instead of creating a duplicate.

Do not use a UI timestamp as the only identity. Device clocks drift, two messages can share the same visible time, and a retry should refer to the same logical operation.

A server-assigned sequence can also make synchronization easier. It gives a conversation an explicit ordering cursor instead of asking clients to infer order from wall-clock timestamps alone.

Use a command path and a sync path

Sending and recovering are different operations.

POST /conversations/{id}/messages
Idempotency-Key: <clientRequestId>

{ "body": "Hello" }

The server validates membership, applies limits, writes the durable record, and returns the canonical message. Separately, a synchronization endpoint can answer:

GET /conversations/{id}/messages?afterSequence=481

The client now has a deterministic recovery mechanism. If it reconnects after ten seconds or ten hours, it asks for everything after the last durable sequence it has applied.

This is the same general boundary that makes lightweight mobile backends easier to replace: keep storage-specific behavior behind an explicit repository or service boundary. In chat, the boundary matters even more because connection state changes constantly while conversation history must remain stable.

Treat real-time events as acceleration

A common design mistake is making a socket event contain the only copy of a message. That creates a dangerous question: what happens if the event is missed?

A safer design is to treat the event as a compact notification that new durable state exists, or as a complete message payload that can still be reconciled against durable state.

{
  "type": "message.created",
  "conversationId": "c_42",
  "messageId": "m_982",
  "sequence": 482
}

The receiving client can apply the payload immediately when it has enough information, but its recovery contract remains sync after sequence 481. That makes missed events repairable.

This principle also helps when the real-time technology changes. You can use managed database listeners, WebSockets, server-sent events, or another broker without rewriting the meaning of conversation history.

Make reconnect boring

Reconnect logic should be a small state machine, not an emergency path.

connect
  -> authenticate
  -> subscribe to conversation events
  -> sync after lastAppliedSequence
  -> merge by message ID
  -> mark connection healthy

The order matters. If you sync before subscribing, a message can arrive between the sync response and the subscription. If you subscribe before syncing, duplicate delivery is possible, but duplicates are harmless when the merge is idempotent.

That is a useful distributed-systems trade-off: prefer a path where the same event may be observed twice over a path where an event can disappear permanently.

Firebase Realtime Database documents local persistence and synchronization after connectivity returns. Even when a platform handles reconnection for you, the product still needs clear semantics for pending sends, stale state, and conflict resolution.

Separate presence from message truth

Presence feels like messaging because both update in real time, but their durability requirements are different.

messages        -> durable, replayable
read receipts   -> durable or eventually consistent
presence        -> ephemeral, freshness-sensitive
typing          -> ephemeral, best effort

Firebase’s own guidance illustrates this distinction. Cloud Firestore does not natively provide presence; Firebase documents a design that uses Realtime Database connection state and Cloud Functions to mirror presence when needed.

Presence also needs expiration semantics. A stale online: true record is worse than no presence signal because it communicates false certainty. Store a last-seen timestamp, use connection-aware cleanup where available, and let the UI degrade gracefully when freshness cannot be guaranteed.

Decide where ordering is guaranteed

Chat UIs often sort by createdAt and stop there. That becomes ambiguous when two sends happen nearly simultaneously, clients have inaccurate clocks, retries complete out of order, or messages are backfilled.

Define the ordering contract explicitly.

Primary order: server sequence
Display time: server createdAt
Optimistic position: temporary client order until acknowledgement

When the acknowledgement arrives, replace the optimistic item using clientRequestId rather than inserting a second message.

A conversation-scoped sequence is easy for clients to reason about, but it can introduce contention at very high write rates. A database-native ordered key may scale better but can have different guarantees. The important part is documenting what the client can rely on.

Design retries before animations

The send button should have a failure contract before the chat bubble gets a polished animation.

Pending -> Sent -> Failed
             |
             -> Delivered -> Read

Not every product needs delivery and read states, but every networked product needs to distinguish “the user asked to send” from “the server durably accepted it.”

A timeout should not immediately mean failure. The request may have committed. Retry with the same idempotency key, reconcile the canonical response, and only surface a retry action when the system cannot determine success automatically.

This avoids one of the most frustrating chat bugs: a user taps retry and creates two identical messages.

Scale the reads before the sockets

Teams often worry about millions of open connections before they have designed conversation queries.

Start with the access patterns:

• load the newest page of messages;
• page backward through history;
• sync forward from a cursor;
• list conversations ordered by recent activity;
• derive unread state;
• fetch attachments separately from message metadata.

Firestore’s chat data-model example keeps messages in a subcollection rather than embedding an unbounded message array inside a room document. Its scaling guidance also emphasizes database location, connection behavior, retries, and regional trade-offs.

The general lesson is broader than Firebase: avoid records that grow forever, make pagination a first-class query, and keep large binary attachments out of the message row itself.

Only then optimize fan-out, broker partitions, connection gateways, and regional routing based on measured load.

A practical reference flow

Sender client
  -> POST message command
  -> authenticate and authorize
  -> idempotent durable write
  -> return canonical message
  -> publish message.created event

Delivery service
  -> fan event to connected participants

Receiver client
  -> merge event by message ID
  -> advance sequence cursor

After reconnect
  -> subscribe
  -> request messages after last sequence
  -> merge duplicates safely

The durable write is the center of the design. The socket is a fast lane around it, not a replacement for it.

Failure modes worth testing

Test these cases before calling the system reliable:

• the connection drops immediately after the server commits a send;
• the same send command is retried three times;
• an event is delivered twice;
• an event arrives before an older event;
• the app is killed with pending optimistic messages;
• the user reconnects after missing hundreds of messages;
• a conversation is opened on two devices at once;
• presence cleanup does not execute normally;
• attachment upload succeeds but message creation fails;
• message creation succeeds but push notification delivery fails.

Each failure should map to one owner. Persistence failures belong to the command/storage path. Missed real-time events belong to synchronization. Push notifications belong to re-engagement, not message durability. Presence failures should not corrupt conversation history.

The architecture rule to keep

The most reusable rule is this:

Persist for truth, stream for speed, and sync for recovery.

When those responsibilities are explicit, the chat feature becomes easier to test and easier to evolve. You can replace the socket provider without migrating history. You can change the database without redefining typing indicators. You can add offline support without inventing a second message model.

That separation is what turns a chat screen into a system that can survive unreliable networks, duplicate requests, reconnects, and future scale.