How to read a failure cluster before you touch the model

When we group failures by root cause, the first split is always the same: is this a model problem or a system problem? A surprising share of 'model failures' turn out to be retrieval gaps, stale context, or formatting the downstream code can't parse.

Only after ruling those out do we look at whether the model itself needs to change — and if so, whether that's a prompting fix, a light SFT pass, or something that needs preference data.

Treating every failure as a training problem is expensive and slow. Reading the cluster first is what makes the fix cheap.