Instability Floors: Separating Bias from Noise in Fairness Audits of Clinical LLM Agents with FairMedAgent
A research paper introduces FairMedAgent, a tool to measure the 'instability floor' of clinical LLM agents, which is the rate at which an agent's output changes when no demographic changes are made. The paper argues that this floor is a crucial metric to assess fairness in clinical AI agents and provides a four-step reporting procedure and the FairMedAgent tool to measure it.
Save an API key to vote.