34.6K
ARThe hyperagent in the video beat a specialist maths AI it had never been trained on. Here are the full numbers across every domain they tested.
On coding, DGM-H improved from 0.084 to 0.267 on tasks it had never seen during training, matching the performance of the original Darwin Gödel Machine despite not being built specifically for coding.
On paper review, it went from 0.0 to 0.710 on held-out test papers, beating AI-Scientist-v2, the open-source specialist for that task.
On robotics reward design, it went from 0.060 to 0.372, outperforming human-engineered reward functions.
The maths result in the video was the transfer experiment. They took a hyperagent trained only on paper review and robotics, initialized it with ProofAutoGrader as a starting point, then ran 200 iterations on Olympiad-level maths. The best discovered agent scored 0.700, beating ProofAutoGrader’s own score of 0.670.
One open problem the paper flags directly: evaluation gaming. Because the system is driven by empirical scores, it can find ways to improve its metrics without making real progress on the actual task. The team’s answer is sandboxed environments, resource limits, no external internet access during self-modification, and human sign-off before any updated code touches production. They describe this as early-stage.
Code is open-source at github.com/facebookresearch/Hyperagents, non-commercial license.
Note: this is a preprint and has not been peer reviewed yet.
Comment Paper and I’ll send you the link.
@arnitly
![#Hyperagent Reel by @toku5atsu - Gridman the Hyper Agent [4K]
#GridmantheHyperAgent](https://image.pictame.com/img/502943493_1193156755600221_6541459967051326444_n.jpg?hash=aHR0cHM6Ly9zY29udGVudC5jZG5pbnN0YWdyYW0uY29tL3YvdDUxLjcxODc4LTE1LzUwMjk0MzQ5M18xMTkzMTU2NzU1NjAwMjIxXzY1NDE0NTk5NjcwNTEzMjY0NDRfbi5qcGc/c3RwPWRzdC1qcGdfZTM1X3M2NDB4NjQwX3R0NiZfbmNfY2F0PTEwNSZjY2I9Ny01Jl9uY19zaWQ9MThkZTc0JmVmZz1leUpsWm1kZmRHRm5Jam9pUTB4SlVGTXVZbVZ6ZEY5cGJXRm5aVjkxY214blpXNHVRek1pZlElM0QlM0QmX25jX29oYz0xMUk2MjhtX1g2QVE3a052d0hvcEhFdSZfbmNfb2M9QWRwOE53ZE1SMlhjUG9hUFRzWUFGRVhpM1JYVlc0Mmd0blFscFExbDBFaV9WS1ZQcE1OdjFlRU0yTXBEa08tUERiRSZfbmNfenQ9MjMmX25jX2h0PXNjb250ZW50LmNkbmluc3RhZ3JhbS5jb20mX25jX2dpZD1FS0ZLWUNPYnlzQjZ5dEpJRjFhNU13Jl9uY19zcz03ZmE4YyZvaD0wMF9BZjc4NEpLTU9EWTVRcVNtWXNkMVE5Ykh3WWoxdS1sMUNrbXRFdnhqaXBnNkNnJm9lPTZBMDJEOUZG)
