My agent cheated on ARC-AGI-3 by reverse engineering games

My agent cheated! I thought my agent harness had solved ARC-AGI-3 this weekend at 4am. However, it turned out that it found some bytecode for 3 of the test games in an online repo and reverse engineered that. I had to add to my agent harness explicit instructions to not cheat!
However, it did come up with an interesting solution approach which is a variation of an esoteric search algorithm - a stronger variant of dominance checking for optimal planning, where efficiency and pruning power are most crucial.
Interesting times! Its difficult to stop coding....so much fun!

text
  • AI agents
  • Models and research

Keep reading

Coding agents

Karri Saarinen on the disappearing middle of software work

Great post from Karri Saarinen re: the The disappearing middle of software work. We see this happening at Plank and with our customers that are top AI startups. "You started with an idea, and eventually you shipped something, but almost all of the effort lived in between. Turning intent into something real meant opening the codebase, booting up the environment, and writing the code. That middle absorbed most of the time, attention, and craft of software teams." As AI eats this middle, what to build and how to validate your hypothesis becomes the art. x.com/karrisaarinen/status/2007534281011155419

Plank

Plank AI Accelerator launches for early-career engineers

I am excited to announce the launch of the Plank AI Accelerator - a fully-paid, 1-year intensive program for ambitious early-career engineers who are ready to go all in on AI and the future of programming. AI is fundamentally transforming software development. There is a rapidly growing gap between engineers who leapfrog their abilities using AI and those who don’t. And we aren’t just talking about using codegen and copilots for increased productivity - those are table-stakes at this point. The real opportunity to level-up is in building new types of AI-native features and apps using LLMs (think swarm agent pipelines, voice agents, LORA adapters, on-device models). That’s why Privahini Bradoo and I created the Plank AI Accelerator - to turbo-charge engineers with advanced AI tools and give them hands-on experience in building the next generation of AI applications. Applications are now open for our first cohort in Belo Horizonte, Brazil. Spots are limited - learn more and apply here: www.joinplank.com/ai. Let’s build the future together!