Datasets

Open any run in the Traces drawer and click Add to Dataset. On a root span this captures the entire run; pick an existing dataset or create a new one inline. You can also add the currently open example from the Prompts playground.

When you add a trace-backed example, Fluiq pins the run's whole trajectory into a retention-independent store: every span, including LLM calls, agent/task steps, tool calls, MCP calls, the multi-agent DAG, and media references. Media is offloaded to object storage and re-linked on read. The pinned snapshot means a dataset run evaluates the exact trajectory even after the original trace has passed its retention window.

Rather than adding runs one at a time, click Connect Agents on a dataset and pick a traced agent. Fluiq imports every run of that agent to date (deduplicated, full trajectory pinned) and keeps the dataset in sync; future runs of a linked agent are appended automatically.

From a dataset you can launch three kinds of batch run over every example: agentic evaluation (tool selection, trajectory, coordination against the pinned trajectory), security (risk level and detections), and a metrics run that grades each example's recorded answer against its expected_output with the metrics you pick (hallucination, faithfulness, relevance, toxicity, coherence, completeness). Results roll up into a per-run report with per-metric averages and per-example scores.

Compare runs. Every report has a Compare vs… selector: pick any earlier run of the same kind and Fluiq diffs them — per-metric score deltas (computed only over examples present in both runs) and every example classified as regressed, improved, or unchanged. That's your regression gate before shipping a prompt or model change; the same check runs headless in CI via python -m fluiq.ci (see Evaluation).

Datasets are also reachable over the API: list datasets, add examples, and fetch an example's pinned trajectory.

import os, requests

BASE = "https://api.fluiq.ai/api/v1"
H = {"Authorization": f"Bearer {os.environ['FLUIQ_API_KEY']}"}

# Create a dataset and add an example captured from a trace
ds = requests.post(f"{BASE}/datasets", headers=H,
                   json={"name": "checkout-regressions"}).json()

requests.post(f"{BASE}/datasets/{ds['dataset_id']}/examples", headers=H, json={
    "input": "Refund my last order",
    "expected_output": "Opened refund #4821",
    # link the run so Fluiq pins its full trajectory for agentic eval
    "metadata": {"source_trace_id": "0f9c...e21"},
})

Pass a source_trace_id in an example's metadata (use the run's root trace id) and Fluiq snapshots that run's whole trajectory into the dataset automatically.

Per-example few-shot.An example's metadata can carry a few_shot list of {input, output} pairs, which a task template reaches as {{few_shot}} — rendered ready to drop into a prompt. It lives on the example rather than the template because the useful examples usually differ per row: the nearest neighbours to this question, not a fixed three pasted into every prompt. Rows without it render nothing at all, so one template serves both. Capped at ten per example.