Arrange Act Assert

Jag Reehals thinking on things, mostly product development

You Can't Support What You Didn't Record

27 Jul 2026

Support asks a year after the payment: "Why did the customer receive €117? And which card did we charge?"

A supportable system answers that with one query. If answering means trawling archived logs, guessing what the exchange rate was that day, and hoping nobody edited the payment method since, you don't have a supportable system.

Record the decision at the point you make it, and never destroy the data it points to.

You Can't Support What You Didn't Record

Two rules

Rule 1: never overwrite reference data. A payer's stored payment method is reference data. When the payer changes it, insert a new row. When they remove it, set deletedAt. No UPDATE, no DELETE, ever.

Rule 2: snapshot the decision inputs on every transaction. At the moment a payment executes, write down everything the decision depended on: which payment method, what rate, what amounts. Write the values as they were at that moment, on the payment record itself.

I covered how the payment flow gathers those inputs into a context object in Tracking Every Decision in Payments. This post is about writing that context onto the durable record so support can still read it a year later.

A payment record is a claim about the past, and if the data underneath it can change, the claim stops being true.

Both rules protect that claim.

Internal data: pointer plus snapshot

For data you own, keep both. The transfer record stores a pointer into the reference table and a snapshot of what the pointer resolved to at execution time:

type PaymentMethod = {
  id: string;
  type: 'card' | 'bank_account';
  last4: string;
  deletedAt: string | null; // soft delete only
};

type TransferRecord = {
  transferId: string;
  // ...
  paymentMethodId: string; // pointer, resolvable forever
  paymentMethod: { type: string; last4: string }; // what it was at execution time
};

The pointer works because of Rule 1. The snapshot means support skips the join: the record itself says "card ending 4242", and keeps saying it. When a dispute or audit needs the full reference row, soft-deleted or not, the pointer still resolves.

External data: capture the value

The exchange rate came from someone else's API. You can't foreign-key into a third party's history, so the value you captured is the only record you will ever have:

type TransferRecord = {
  // ...
  rate: number; // the rate we converted at
  rateCapturedAt: string; // when we fetched it
  convertedAmount: number; // what it produced
};

The workflow stamps the rate the moment it fetches it:

const rate = await fetchRate({ from: 'GBP', to: 'EUR' });
// External data: capture it now, we can't point at the provider's history later
const rateCapturedAt = new Date().toISOString();

A month or a year later, "why €117?" has an answer: rate 1.17, captured at 14:53:12 UTC, applied to £100. Call the rate API at support time instead and you get today's rate, a different number answering a different question.

Model output: the worst external data of all

An exchange rate at least has a provider you could argue with. A model call has nothing. Same prompt, same model, different answer, and no history to refetch. If you didn't record it, it never happened.

So record the whole step: the model and version, the prompt, the tool calls it asked for, the arguments it sent, what each tool returned, and the final text.

type ModelStep = {
  model: string;
  prompt: string;
  toolCalls: Array<{
    toolName: string;
    input: unknown; // exactly as the model sent it
    output?: unknown; // absent if the tool never ran
    error?: string;
  }>;
  text: string;
  usage: { inputTokens: number; outputTokens: number };
};

The interesting field is the one that isn't there. Here is a real run against a local llama3.2 with a calculator tool, arguments typed as z.number():

{
  "type": "tool-call",
  "toolName": "calculator",
  "input": { "a": "23", "b": "7", "op": "add" },
  "invalid": true,
  "error": "AI_InvalidToolInputError: expected number, received string at path a"
}

The model sent strings. Validation rejected them, so execute never ran. Then the model answered:

Using the calculator tool, I get: 23 + 7 = 30

It did not use the calculator tool. It guessed, and it happened to guess right. finishReason was stop. Nothing threw. Every dashboard stayed green.

That is the whole argument for recording the step. The final text is the model's account of what it did, and the model is not a reliable narrator. The tool call record is evidence. It shows the arguments the model actually produced, and its missing output shows the tool never executed. The next question is only answerable because you kept it: was 30 arithmetic or a coin flip?

Record the absences too, then. A tool that was offered and never called, a step that ended early, a validation failure that resolved itself. null is a finding.

What goes wrong without this

Wide events, same idea

I've written about how wide events beat scattered logs: one canonical event per request, carrying all the context that request accumulated.

You apply the same discipline to storage. The wide event snapshots a request; the transfer record snapshots a transaction. The workflow writes the same values to both:

span.setAttributes({
  'transfer.id': transfer.transferId,
  'transfer.rate': rate,
  'transfer.rate.captured_at': rateCapturedAt,
  'payment_method.id': paymentMethod.id,
  'payment_method.last4': paymentMethod.last4,
});

Model steps go on the same event: gen_ai.request.model, gen_ai.tool.name, gen_ai.usage.input_tokens. The span shows you this week's latency spike; the record answers next year's "did the tool run?"

Keep both. Telemetry retention runs out in days or weeks, so the wide event serves this week's incident. The transfer record outlives it and serves next year's dispute.

The payoff

Support needs one endpoint:

app.get('/api/transfer/:id', (c) => {
  const record = getTransferRecord(c.req.param('id'));
  return record ? c.json(record) : c.json({ error: 'NotFound' }, 404);
});
{
  "transferId": "TXN-001",
  "requestedAmount": 100,
  "fromCurrency": "GBP",
  "toCurrency": "EUR",
  "rate": 1.17,
  "rateCapturedAt": "2026-07-22T14:53:12.000Z",
  "convertedAmount": 117,
  "paymentMethodId": "pm_001",
  "paymentMethod": { "type": "card", "last4": "4242" },
  "status": "completed"
}

In production, put the endpoint behind auth and scope the lookup to the record's owner; the record answers questions for support, not for whoever guesses an ID.

Every decision sits next to the inputs you wrote when you made it. A year later, "why €117, and which card?" is one authenticated GET.

architecture design observability supportability ai