An AI coding team changes its own roles to escape GPU-kernel dead ends

An AI coding agent can spend a long time improving a design that has already reached its practical limit. Evolving Agent Teams tries to change the search itself. A supervisor examines where progress has stalled, then reorganizes the specialists working on the code instead of repeatedly asking the same team to try harder.[1]

The framework, described in a paper posted on 27 October, keeps a learning document, an evolution history and team artifacts in a persistent file system. New teams inherit what earlier teams discovered. The supervisor changes roles, prompts and coordination; the coding agents produce the solution, and an evaluation harness checks the result.[1][2]

In matched-token-budget experiments, the authors report stronger performance than their fixed-team and single-agent baselines. A longer search produced a CUDA kernel for multi-head latent attention paged decoding that beat the production FlashInfer baseline on 36 of 47 workloads. It lost on the remaining 11, where its relative performance ranged from 0.86 to 0.99 times the baseline. The reported mean production speedup was about 1.2 times.[1]

Why the same kernel has two headline numbers

FlashInfer separates planning from execution. Production use can cache a plan and run planning again when workload metadata changes. The paper distinguishes that production comparison from a timing baseline that excludes planning overhead and measures the run stage directly.[1]

Against the plan-excluding comparison, the evolved kernel reached about 0.96 times the baseline. The production baseline is slower in the authors’ measurement, making the same result about 1.2 times faster in that comparison. A claim that the generated kernel simply executes 20% faster than FlashInfer’s underlying run stage would therefore misstate the result. The benefit depends on which work the measurement includes.[1]

The other demonstrations broaden the idea without sharing one universal speed multiplier. For Kimi Delta Attention prefill, the paper reports 3.01 times the vLLM baseline on AMD MI350X and 3.44 times FlashKDA on Nvidia B200. The AMD result averages latency ratios across ten workloads; the B200 result uses a geometric mean across six. Even before considering the different hardware, those are different experiments.[1]

The AMD run also puts a cost and duration behind the result: 73 iterations over 186 hours, with about $3,000 in booked agent and supervisor API spending. That is inference expenditure, not a full operating-cost figure including hardware. The final 3.01-times result was averaged over five remeasurements. Those details make the achievement more useful to engineering teams deciding whether an extended search might repay its cost.[1]

Beyond kernels, a generated code policy scored 62.1 on held-out RoboDojo robot-manipulation seeds, compared with 55.3 for the best leaderboard entry cited as of 26 October. The paper describes these additional settings as single-run case studies. They demonstrate possible applications of the method, while the limited repeat counts constrain claims about how consistently a fresh run will succeed.[1]

The project has released code and links to team configurations, transcripts and task artifacts. We have reviewed the reported results and repository, rather than rerun the GPU experiments. For practitioners, that evidence makes the method inspectable: the object of interest is not only the final kernel, but the sequence of changes that got the team past a plateau.[1][2]

Report an error or suggest a correction