jevals
Run dozens of agent evals per trace in one sub-second request, no judge prompt
Emerged finds new products worth a look, daily — what’s new today
jev-align
Teach a Jev classifier your labels by reviewing only its least certain answers
Instead of hand-tuning a classifier prompt, you label the few examples the model is unsure about and approve each proposed rewrite before it lands.
jev-align is an experimental command-line tool from Sutro for teams building AI Functions on TypeSafe's Jev framework. Point it at a CSV, Parquet or JSONL dataset and each round it scores uncertainty, surfaces the ambiguous rows for you to label, runs GEPA prompt optimisation over the labels so far, and shows the suggested change with confidence figures for sign-off. It handles binary, multiclass, multilabel and scoring tasks. Install is uv or pip on Python 3.11+, and you bring a TypeSafe key plus a model provider. Sutro is unaffiliated with TypeSafe and shows no accuracy results yet.
More like this
Laya
Small encoder models that answer routing and triage questions in one pass
Kev
Open 0.8B to 9B decision models for choice, yes/no and rating questions
Flide
Exact, shared-context inference server for small Jev-style decision models
SemIf
Read a yes/no probability straight out of a small open model, no generated text
Bespoke Nimble
An open 9B model, dataset and training recipe for fast typed decisions over text