jev-align
Teach a Jev classifier your labels by reviewing only its least certain answers
Emerged finds new products worth a look, daily — what’s new today
jevals
Run dozens of agent evals per trace in one sub-second request, no judge prompt
If your eval bill is a second frontier model reading every trace, jevals swaps that for calibrated probabilities from a decision model at a fraction of a cent, and says where not to trust it.
jevals is Openlayer's Python library for scoring agent traces with Jev decision models instead of an LLM judge. You pass the agent's messages and tools, choose from 37 built-in evals or write your own in YAML, and every question goes out in one request that returns a probability per eval rather than a paragraph to parse. It can also run locally with Kev or Laya on Apple Silicon. pip install jevals, MIT, one commit on main, labelled alpha. The README is candid: no test-set generation, no dashboard, calibration to check on your own data, no substitute for a judge on multi-step reasoning.
More like this
Taster
A guard layer that checks agent tool calls and screens what tools send back
SemIf
Read a yes/no probability straight out of a small open model, no generated text
jevman
An arcade maze turned into a latency test for decision models
AgentRun
Open-source DSL that freezes repeated agent work into typed, inspectable steps
Flide
Exact, shared-context inference server for small Jev-style decision models