An AI chat that forgets everything on refresh feels broken, and one that replays a thousand messages into every prompt gets slow and expensive. The fix is a schema where messages are immutable rows and a rolling summary carries the older context. Here is the shape that has held up in production.

Three tables

A conversation belongs to a user. A message belongs to a conversation. A summary is a single cached row per conversation that gets rewritten as the thread grows.

CREATE TABLE conversations (
  id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  user_id uuid NOT NULL,
  status text NOT NULL DEFAULT 'active',
  last_activity_at timestamptz NOT NULL DEFAULT now()
);

CREATE TABLE messages (
  id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  conversation_id uuid NOT NULL REFERENCES conversations(id),
  author text NOT NULL, -- 'user' | 'assistant' | 'agent'
  content text NOT NULL,
  created_at timestamptz NOT NULL DEFAULT now()
);

Messages are never updated or deleted in normal flow. They are an append-only log, which makes the history trivial to reason about and safe to cache.

Capping the context window

Replaying every message is the obvious approach and the wrong one. Past a threshold, say twenty messages, I summarize everything older than the last ten turns into the summary row, then send the model the summary plus the recent tail.

const recent = messages.slice(-10);
const context = summary
  ? [{ role: "system", content: summary }, ...recent]
  : recent;

The summary is regenerated in the background after each assistant reply, so the next request already has it. Token cost stays roughly flat no matter how long the thread runs.

The session lifecycle

A conversation is active while messages keep arriving. A background job marks it idle after thirty minutes of silence and closed after a day. Closing matters for two reasons: it stops the summary job from touching dead threads, and it gives analytics a clean boundary for what counts as one session.

UPDATE conversations SET status = 'idle'
WHERE status = 'active' AND last_activity_at < now() - interval '30 minutes';

A reopened conversation simply flips back to active on the next message, with its full history and summary intact.

Why immutability pays off

Because messages never change, the summary can be cached without invalidation headaches, the transcript can be handed to a human agent verbatim, and an audit of what the AI actually said is just a query. Every feature I added later, handoff context, analytics, replaying a thread for debugging, came almost for free because the underlying log never lies and never mutates.