Gate responses that fail quality thresholds
LLM-as-judge scores every response server-side — and agentic evaluation judges whole runs: tool choice, trajectory, and multi-agent coordination.
Fluiq/ testsTotal Evals
847
across 312 traces
Avg Score
0.91
threshold ≥ 0.7
Pass Rate
88.4%
749 / 847 passed
By Metric
What you get
Metrics
Six judge metrics
Hallucination, faithfulness, relevance, toxicity, coherence, and completeness, scored per response.
Thresholds
Per-metric gates
Set a threshold for each metric. Warn mode logs the score; block mode stops the response.
Agentic
Whole-run evaluation
Layered judging of a full agent run: deterministic checks, tool-selection quality, trajectory against the goal, and multi-agent coordination across fan-outs and joins.
Jury
Multi-model judge panel
Borderline verdicts convene a jury of different judge models and aggregate their votes — with every member's score and reasoning kept for audit.
The judge runs server-side. Opt in with one call.
- instrument() only traces — scoring starts when you call fluiq.eval()
- Warn mode logs scores; block mode raises FluiqEvalError below threshold
- Run Agentic Eval on any root trace to judge the whole run — tools, trajectory, coordination
- CI gates run the same checks in GitHub Actions
import fluiq
fluiq.instrument(api_key="fl_...")
fluiq.eval(
metrics=["hallucination", "relevance", "toxicity"],
thresholds={"hallucination": 0.8, "relevance": 0.75},
mode="warn", # "block" raises FluiqEvalError
)Part of the Fluiq platform
Compare plansUnlimited traces, always free.
Start on the free tier and turn on each pillar as your pipeline grows. No code changes required.