i think it says, amongst other things, that there is a salient difference between competitive programming like codeforce and real-world programming. u can train a model to hillclimb elo ratings on codeforce, but that won't necessarily directly translate to working on a prod javascript codebase.

anthropic figured out something about real world coding that openai is still trying to catch up to, o3-mini-high notwithstanding.