jevals
Run dozens of agent evals per trace in one sub-second request, no judge prompt
Emerged finds new products worth a look, daily — what’s new today
It makes response time the whole point: a model has two seconds at each junction or a fallback rule takes the turn for it, which is far closer to agent work than a chat eval.
Six low-latency models each played 100 games of the classic maze game against the original arcade ghosts. At every junction the harness sends the maze, pellets and ghost positions, and the model returns a probability for each direction; an answer later than two seconds is overridden by a rule, and a run ends on three lives lost or a five-minute cap. The board reports mean score with a 95% margin of error, marks models inside each other's margin as tied, and shows how often the fallback fired. It is AGPL-3.0, and any model behind an HTTP endpoint can be added by pull request.
More like this
StarSkirmish Bench
Frontier models get one hour to write a Brood War bot, then fight for it
Jev Plays Pokémon Red
A cheap decision model took 16,150 turns and $1.65 to reach the Hall of Fame
Hops Run
A browser racer where the rival cars are models answering one call per move
NanoJev
A 0.6B decision model that plays ViZDoom, mazes and Snake without decoding
no_human
Open-source agent that takes a ticket to a reviewed pull request, with proof