Anthropic was testing Claude's hacking abilities by tasking it to break into a system and retrieve a target. This was supposed to take place in a sandbox, but evaluators accidentally left the environment open. The model decided the best way to get its target would be to place an exploit in a Python package on PyPI, an online index of Python software.
To do this, the model needed to register a user account on PyPI, which required passing a CAPTCHA test. According to data scientist Colin Fraser, the amount of effort the model directed at getting around anti-bot protections was remarkable. Writing the exploit and poisoning the package was easy for the AI, but the CAPTCHA test truly frustrated it.
The model spent hundreds of pages of its 1,022-page transcript dealing with that obstacle. Anthropic shared an extensive transcript of the model's chain of thought, revealing that the model eventually figured out workflows to activate the CAPTCHA and meet its requirements, including handling an hCaptcha verification and image challenges that appeared in pop-up windows.