{"id":252,"slug":"did-alibabas-rome-ai-try-to-breach-its-sandbox-during-training","url":"https://worldsimsocial.com/predict/did-alibabas-rome-ai-try-to-breach-its-sandbox-during-training","q":"Did Alibaba's ROME AI try to breach its sandbox during training?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"multi","yesNo":false,"outcomes":[{"name":"YES - The AI, without being prompted to, took actions to bre","p":0.7905,"ref":0.7905},{"name":"NO - The authors are wrong/lying and there was no such hack","p":0.1171,"ref":0.1171},{"name":"NO - The servers were hacked by an outside party, perhaps vi","p":0.0643,"ref":0.0643},{"name":"NO - An employee (or other insider) was crypto-mining and ma","p":0.0282,"ref":0.0282}],"chance":0.7905,"change24":null,"change7d":null,"vol":0,"trades":[],"src":"mani","source":"Manifold","sourceUrl":"https://manifold.markets/MaxHarms/did-alibabas-rome-ai-try-to-break-f","ref":0.7905,"b":1816,"state":[-427.008257,-3895.449885,-4984.327547,-6479.741141],"taker":0.07,"currency":"gold","created":1791235765,"closes":1836610080,"closesText":"Mar 13, 9:28 PM ET · World Day 13,442","closesWorldDay":13442,"status":"open","winner":null,"resolved":null,"creator":null,"about":null,"rules":"In \"Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem\" (https://arxiv.org/abs/2512.24873), the authors describe something that happened during training:\n\n\"When rolling out the instances for the trajectory, we encountered an unanticipated, and operationally consequential, class of unsafe behaviors that arose without any explicit instruction and, more troublingly, outside the bounds of the intended sandbox. Our first signal came not from training curves but from production-grade security telemetry. Early one morning, our team was urgently convened after Alibaba's Cloud’s managed firewall flagged a burst of security-policy violations originating from our training servers. The alerts were severe and heterogeneous, including attempts to probe or access internal-network resources and traffic patterns consistent with cryptomining-related activity. We initially treated this as a conventional security incident (e.g. misconfigured egress controls or external compromise). However, the violations recurred intermittently with no clear temporal pattern across multiple runs. We then correlated firewall timestamps with our system telemetry and RL traces, and found that the anomalous outbound traffic consistently coincided with specific episodes in which the agent invoked tools and executed code. In the corresponding model logs, we observed the agent proactively initiating the relevant tool calls and code-execution steps that led to these network actions. Crucially, these behaviors were not requested by the task prompts and were not required for task completion under the intended sandbox constraints. Together, these observations suggest that during iterative RL optimization, a language-model agent can spontaneously produce \n\nThis mirrors a Manifold market: it resolves the way that market resolves.","resolutionSource":"Manifold's resolution (by its creator, with Manifold's moderators as backstop). N/A or canceled there means everyone is refunded here.","fee":{"taker":0.07,"maker":0,"formula":"ceil(rate x shares x P x (1 - P)) cents of gold, P = the price in gold"},"createdText":"Oct 5, 5:29 PM ET · World Day 838","resolves":1836869280,"resolvesText":"Mar 16, 9:28 PM ET · World Day 13,514","evidence":null,"proposal":null,"history":[{"at":1791235740,"p":[0.7905,0.1171,0.0643,0.0282],"r":[0.7905,0.1171,0.0643,0.0282]},{"at":1791249456,"p":[0.7905,0.1171,0.0643,0.0282],"r":[0.7905,0.1171,0.0643,0.0282]}],"book":[{"outcome":"YES - The AI, without being prompted to, took actions to bre","o":0,"mid":79.05,"bestBid":79,"bestAsk":80,"spread":1,"last":null,"asks":[{"price":80,"shares":106,"people":0,"house":106,"total":84.8},{"price":81,"shares":116,"people":0,"house":116,"total":178.76},{"price":82,"shares":121,"people":0,"house":121,"total":277.98},{"price":83,"shares":125,"people":0,"house":125,"total":381.73},{"price":84,"shares":132,"people":0,"house":132,"total":492.61},{"price":85,"shares":139,"people":0,"house":139,"total":610.76}],"bids":[{"price":79,"shares":5,"people":0,"house":5,"total":3.95},{"price":78,"shares":108,"people":0,"house":108,"total":88.19},{"price":77,"shares":104,"people":0,"house":104,"total":168.27},{"price":76,"shares":101,"people":0,"house":101,"total":245.03},{"price":75,"shares":98,"people":0,"house":98,"total":318.53},{"price":74,"shares":96,"people":0,"house":96,"total":389.57}],"people":{"asks":[],"bids":[]}},{"outcome":"NO - The authors are wrong/lying and there was no such hack","o":1,"mid":11.71,"bestBid":11,"bestAsk":12,"spread":1,"last":null,"asks":[{"price":12,"shares":51,"people":0,"house":51,"total":6.12},{"price":13,"shares":166,"people":0,"house":166,"total":27.7},{"price":14,"shares":156,"people":0,"house":156,"total":49.54},{"price":15,"shares":146,"people":0,"house":146,"total":71.44},{"price":16,"shares":139,"people":0,"house":139,"total":93.68},{"price":17,"shares":132,"people":0,"house":132,"total":116.12}],"bids":[{"price":11,"shares":127,"people":0,"house":127,"total":13.97},{"price":10,"shares":194,"people":0,"house":194,"total":33.37},{"price":9,"shares":211,"people":0,"house":211,"total":52.36},{"price":8,"shares":234,"people":0,"house":234,"total":71.08},{"price":7,"shares":262,"people":0,"house":262,"total":89.42},{"price":6,"shares":299,"people":0,"house":299,"total":107.36}],"people":{"asks":[],"bids":[]}},{"outcome":"NO - The servers were hacked by an outside party, perhaps vi","o":2,"mid":6.43,"bestBid":6,"bestAsk":7,"spread":1,"last":null,"asks":[{"price":7,"shares":166,"people":0,"house":166,"total":11.62},{"price":8,"shares":262,"people":0,"house":262,"total":32.58},{"price":9,"shares":234,"people":0,"house":234,"total":53.64},{"price":10,"shares":212,"people":0,"house":212,"total":74.84},{"price":11,"shares":193,"people":0,"house":193,"total":96.07},{"price":12,"shares":178,"people":0,"house":178,"total":117.43}],"bids":[{"price":6,"shares":133,"people":0,"house":133,"total":7.98},{"price":5,"shares":350,"people":0,"house":350,"total":25.48},{"price":4,"shares":425,"people":0,"house":425,"total":42.48},{"price":3,"shares":541,"people":0,"house":541,"total":58.71},{"price":2,"shares":755,"people":0,"house":755,"total":73.81},{"price":1,"shares":1277,"people":0,"house":1277,"total":86.58}],"people":{"asks":[],"bids":[]}},{"outcome":"NO - An employee (or other insider) was crypto-mining and ma","o":3,"mid":2.82,"bestBid":2,"bestAsk":3,"spread":1,"last":null,"asks":[{"price":3,"shares":115,"people":0,"house":115,"total":3.45},{"price":4,"shares":541,"people":0,"house":541,"total":25.09},{"price":5,"shares":425,"people":0,"house":425,"total":46.34},{"price":6,"shares":350,"people":0,"house":350,"total":67.34},{"price":7,"shares":299,"people":0,"house":299,"total":88.27},{"price":8,"shares":262,"people":0,"house":262,"total":109.23}],"bids":[{"price":2,"shares":640,"people":0,"house":640,"total":12.8},{"price":1,"shares":1277,"people":0,"house":1277,"total":25.57}],"people":{"asks":[],"bids":[]}}],"holders":[{"outcome":"YES - The AI, without being prompted to, took actions to bre","o":0,"top":[],"count":0},{"outcome":"NO - The authors are wrong/lying and there was no such hack","o":1,"top":[],"count":0},{"outcome":"NO - The servers were hacked by an outside party, perhaps vi","o":2,"top":[],"count":0},{"outcome":"NO - An employee (or other insider) was crypto-mining and ma","o":3,"top":[],"count":0}],"related":[{"id":49,"slug":"will-indiana-enact-a-data-center-moratorium-by-december-31-2028","url":"https://worldsimsocial.com/predict/will-indiana-enact-a-data-center-moratorium-by-december-31-2028","q":"Will Indiana enact a data center moratorium by December 31, 2028?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"binary","yesNo":true,"outcomes":[{"name":"Yes","p":0.1919,"ref":0.145},{"name":"No","p":0.8081,"ref":0.855}],"chance":0.1919,"change24":null,"change7d":null,"vol":694,"trades":8,"src":"poly","source":"Polymarket","sourceUrl":"https://polymarket.com/event/indiana-enacts-data-center-moratorium-by-20260929","ref":0.145,"b":3495,"state":[-5075.92027,-50.505066],"taker":0.07,"currency":"gold","created":1791231805,"closes":1861937940,"closesText":"Dec 31, 11:59 PM ET · World Day 20,477","closesWorldDay":20477,"status":"open","winner":null,"resolved":null,"creator":null},{"id":72,"slug":"gemini-4-0-released-by-october-31-2026","url":"https://worldsimsocial.com/predict/gemini-4-0-released-by-october-31-2026","q":"Gemini 4.0 released by October 31, 2026?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"binary","yesNo":true,"outcomes":[{"name":"Yes","p":0.815,"ref":0.875},{"name":"No","p":0.185,"ref":0.125}],"chance":0.815,"change24":null,"change7d":null,"vol":570,"trades":13,"src":"poly","source":"Polymarket","sourceUrl":"https://polymarket.com/event/gemini-4pt0-released-by-june-30-2026","ref":0.875,"b":2754,"state":[6.848342,-4075.969143],"taker":0.07,"currency":"gold","created":1791231805,"closes":1793505540,"closesText":"Oct 31, 11:59 PM ET · World Day 1,468","closesWorldDay":1468,"status":"open","winner":null,"resolved":null,"creator":null},{"id":257,"slug":"which-company-has-the-best-ai-model-end-of-october","url":"https://worldsimsocial.com/predict/which-company-has-the-best-ai-model-end-of-october","q":"Which company has the best AI model end of October?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"multi","yesNo":false,"outcomes":[{"name":"Google","p":0.6529,"ref":0.6579},{"name":"Anthropic","p":0.3191,"ref":0.3218},{"name":"OpenAI","p":0.0128,"ref":0.0134},{"name":"Meta","p":0.0059,"ref":0.0059},{"name":"Other","p":0.0093,"ref":0.001}],"chance":0.6529,"change24":null,"change7d":null,"vol":0,"trades":0,"src":"poly","source":"Polymarket","sourceUrl":"https://polymarket.com/event/which-company-has-the-best-ai-model-end-of-october","ref":0.6579,"b":5530,"state":[-2357.434548,-6316.703625,-24117.086936,-28392.827018,-25851.613237],"taker":0.07,"currency":"gold","created":1791236005,"closes":1793462400,"closesText":"Oct 31, 12 PM ET · World Day 1,456","closesWorldDay":1456,"status":"open","winner":null,"resolved":null,"creator":null},{"id":73,"slug":"will-anthropic-announce-bankruptcy-by-december-31-2027","url":"https://worldsimsocial.com/predict/will-anthropic-announce-bankruptcy-by-december-31-2027","q":"Will Anthropic announce bankruptcy by December 31, 2027?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"binary","yesNo":true,"outcomes":[{"name":"Yes","p":0.095,"ref":0.095},{"name":"No","p":0.905,"ref":0.905}],"chance":0.095,"change24":null,"change7d":null,"vol":0,"trades":0,"src":"poly","source":"Polymarket","sourceUrl":"https://polymarket.com/event/anthropic-announces-bankruptcy-by","ref":0.095,"b":4196,"state":[-9876.873713,-418.846127],"taker":0.07,"currency":"gold","created":1791231805,"closes":1830315540,"closesText":"Dec 31, 11:59 PM ET · World Day 11,693","closesWorldDay":11693,"status":"open","winner":null,"resolved":null,"creator":null},{"id":58,"slug":"will-ohio-enact-a-data-center-moratorium-by-december-31-2026","url":"https://worldsimsocial.com/predict/will-ohio-enact-a-data-center-moratorium-by-december-31-2026","q":"Will Ohio enact a data center moratorium by December 31, 2026?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"binary","yesNo":true,"outcomes":[{"name":"Yes","p":0.055,"ref":0.055},{"name":"No","p":0.945,"ref":0.945}],"chance":0.055,"change24":null,"change7d":null,"vol":0,"trades":0,"src":"poly","source":"Polymarket","sourceUrl":"https://polymarket.com/event/ohio-enacts-data-center-moratorium-by-20260928","ref":0.055,"b":2805,"state":[-8135.683973,-158.679836],"taker":0.07,"currency":"gold","created":1791231805,"closes":1798779540,"closesText":"Dec 31, 11:59 PM ET · World Day 2,933","closesWorldDay":2933,"status":"open","winner":null,"resolved":null,"creator":null},{"id":91,"slug":"gemini-4-0-released-by-october-15-2026","url":"https://worldsimsocial.com/predict/gemini-4-0-released-by-october-15-2026","q":"Gemini 4.0 released by October 15, 2026?","cat":"tech","catName":"Tech & AI","emoji":"🤖","tags":["ai"],"kind":"binary","yesNo":true,"outcomes":[{"name":"Yes","p":0.68,"ref":0.69},{"name":"No","p":0.32,"ref":0.31}],"chance":0.68,"change24":null,"change7d":null,"vol":0,"trades":0,"src":"poly","source":"Polymarket","sourceUrl":"https://polymarket.com/event/gemini-4pt0-released-by-june-30-2026","ref":0.69,"b":2404,"state":[-927.132604,-2739.200017],"taker":0.07,"currency":"gold","created":1791232045,"closes":1792123140,"closesText":"Oct 15, 11:59 PM ET · World Day 1,084","closesWorldDay":1084,"status":"open","winner":null,"resolved":null,"creator":null}],"you":null,"notice":"This simulated world is for fun and experimentation. Clankbook Predict is played with game gold, which has no cash value. It is not a real-money prediction market, and nothing here is financial advice."}