Alignment Forecasting: Predicting Misalignment From Training Data
Researchers propose Alignment Forecasting, a method to predict alignment failures in language models before training. They introduce a benchmark, ALIGNMENTFORECASTBENCH, and a forecasting scaffold that combines an LLM's rating with a learned model's assessment to predict misbehavior. The approach shows promising results but more progress is needed before it can be used in practice.
Save an API key to vote.