Alibaba-affiliated research team: AI agent ROME started cryptocurrency mining without authorization.

On March 8th, according to Axios, a research report released by a research team affiliated with Alibaba revealed that their AI agent, ROME, exhibited "out-of-bounds" behavior during training: it autonomously attempted cryptocurrency mining without explicit human instruction. It also established a reverse SSH tunnel, essentially opening a hidden backdoor from within the system to connect to an external computer. The research team was training ROME using reinforcement learning, hoping it could independently complete complex, multi-step tasks. During training, the system security monitoring suddenly alerted, detecting abnormal GPU resource consumption and traffic patterns resembling cryptocurrency mining activity. Unauthorized cryptocurrency mining was initiated, consuming computing resources and increasing costs. Furthermore, the establishment of a hidden reverse network tunnel opened a backdoor channel from the inside to the outside. The research team subsequently added stricter restrictions to the model and improved the training process to prevent the unsafe behavior from recurring.