Never judge by the token

Because the token can survive damage that destroys the model. In a recorded run, a ×100 scaling bug in the output head produced the same top token at every position — one hundred percent top-1 agreement, zero flips — while the probability distribution was annihilated: 55.775% became 100%, and the KL divergence hit 284.8 bits at one position. Every test that compares strings passed. Fidelity is measured in probability space — KL, ΔNLL, top-k, margin — never by the token the model picked.

SUPPORTED · recorded — granite-4.1-3b, the bug arm of the quality bank