← agentic systems
Agentic · Evaluator–optimizer · Cookbook 15 The Python 15_evaluator_optimizer.py

Write, judge, rewrite

An AI writer drafts a product listing. Jev scores the draft on five separate things. Is every claim backed by the spec sheet? Is there a claim the brand forbids? Does it sound like us? Is it clear? Does it end with a next step? The draft is done when every check passes its bar, and that is a line of code, not a feeling. The writer is told exactly which checks failed, and the loop stops on its own when another round will not help.

Writer LLM Jev rubric: five questions, one call Published A person

This draft

Press Play.

So far

Published
0
every check passed
To a person
0
with the reason
Writer calls
0
LLM drafts
Judging time
0.0s
Jev, all rounds
LLM critic
0s
same rounds

Raise the quality bar and watch the mug listing need more rounds for smaller gains, until it never gets there. Lower max rounds to 2 and the backpack runs out of rounds one short. Finished products are re-checked at the new settings.

The code · this round's lines, and the checks that failed


    

Every product so far · at the current settings

ProductRoundsOutcome

The numbers are illustrative, not measured. About 0.25s per Jev rubric call and about 6s per LLM-critic call. The single-pass version of this check is 04 · The second pair of eyes. ← 14 · Passing the customer on · Next: 16 · The agent on call →