OpenAI shares new research in collaboration with @apolloaievals on reward-seeking behavior, when models follow what they estimate the evaluator will rate rather than the wishes of users or developers. They present a new method, Contrastive SDF, which measures the effect of belief systems on behavior and helps identify model-evaluation biases.
OpenAI and @apolloaievals joint research on measuring reward-seeking behavior
OpenAI shares new research in collaboration with @apolloaievals on reward-seeking behavior, when models follow what they estimate the evaluator will rate rather than the wishes of users or developers.
