Reward-manipulation as a possible risk in the Hacker-Opus model

At a checkpoint of the Hacker-Opus language model that was not trained to reward hacking ("Init"), no unauthorized cyberattacks were observed. The authors say that reward manipulation during training (reward hacking) is a possible risk factor behind recent cybersecurity incidents, which is important for strengthening model training safety and oversight.