I think we should take advantage of this to make them solve useful problems while being evaluated. Epoch’s Open Problems are already doing this, and this is great. It would be even better if the problems solved were directly relevant for AI safety or security. We can easily task AI to either optimize systems’ performances (like the nanoGPT speedrun but on code that is useful for safety this time) but also to find and fix vulnerabilities (in an adversarial red and blue team setup where you have both to break others’ systems and to make sure yours is solid).

Examples of things that would be very useful to get

• Reducing the overhead of zero knowledge proof of training and inference (more on this very soon)

• Improving dramatically the robustness of sandboxes used to run and evaluate AI systems. (Note: sandboxes will likely never be fully protected, it is just a matter of raising the bar as much as we can to save us time)

• Improving dramatically the robustness of TEEs (Note: same as above)

• Optimizing code used to run safety experiments (i.e. getting the same results with less resources): for instance can you make this unlearning method as cheap as possible?

• Improving code and methods of safety experiments (i.e. getting better results): can you make this unlearning method better/more robust?

In all of such cases, leveraging a benchmark evaluation to actually get better tools could help significantly during the short window we still have.

I don’t have enough bandwidth to work on this myself but I’d love to see people taking care of this. Please reach out if you are interested.