A study examines the pitfalls of reading LLM judges' verdicts from their first generated token, showing that this approach overstates position bias and can mislead auditors. The researchers recommend reporting the rate at which a judge leads with a verdict token, which can be done at a lower computational cost.