Datasets

Curate golden datasets straight from your real traffic, then re-run evaluation and security over them as a regression suite. Trace-backed examples capture the whole agent run — not just an input/output pair — so full agentic evaluation works offline, forever.

Build a dataset from traces

Open any run in the Traces drawer and click Add to Dataset. On a root span this captures the entire run; pick an existing dataset or create a new one inline. You can also add the currently open example from the Prompts playground.

Trajectory capture

When you add a trace-backed example, Fluiq pins the run's whole trajectory — every span: LLM calls, agent/task steps, tool calls, MCP calls, the multi-agent DAG, and media references — into a retention-independent store. Media is offloaded to object storage and re-linked on read. The pinned snapshot means a dataset run evaluates the exact trajectory even after the original trace has passed its retention window.

Why the whole trajectory?

A single input/output pair can only be scored as one turn. Pinning the full run lets Fluiq re-run agentic evaluation — tool-selection correctness, trajectory quality, and multi-agent coordination — and security, exactly as it would on a live trace. Expand any example in the Datasets dashboard to inspect the captured steps, agents, tools, and MCP calls.

Connect Agents

Rather than adding runs one at a time, click Connect Agents on a dataset and pick a traced agent. Fluiq imports every run of that agent to date (deduplicated, full trajectory pinned) and keeps the dataset in sync — future runs of a linked agent are appended automatically.

Batch evaluation & security

From a dataset you can launch a batch agentic evaluation or security run over every example. Each item is scored against its pinned trajectory and the results roll up into a report — the same signals you see on a live trace (quality, tool selection, trajectory, coordination; risk level and detections), now over a fixed, versioned set. Use it as a regression gate before shipping a prompt or model change.

Programmatic access

Datasets are also reachable over the API — list datasets, add examples, and fetch an example's pinned trajectory.

Python

import os, requests

BASE = "https://api.fluiq.ai/api/v1"
H = {"Authorization": f"Bearer {os.environ['FLUIQ_API_KEY']}"}

# Create a dataset and add an example captured from a trace
ds = requests.post(f"{BASE}/datasets", headers=H,
                   json={"name": "checkout-regressions"}).json()

requests.post(f"{BASE}/datasets/{ds['dataset_id']}/examples", headers=H, json={
    "input": "Refund my last order",
    "expected_output": "Opened refund #4821",
    # link the run so Fluiq pins its full trajectory for agentic eval
    "metadata": {"source_trace_id": "0f9c...e21"},
})

Pass a source_trace_id in an example's metadata (use the run's root trace id) and Fluiq snapshots that run's whole trajectory into the dataset automatically.

import os, requests BASE = "https://api.fluiq.ai/api/v1" H = {"Authorization": f"Bearer {os.environ['FLUIQ_API_KEY']}"} # Create a dataset and add an example captured from a trace ds = requests.post(f"{BASE}/datasets", headers=H, json={"name": "checkout-regressions"}).json() requests.post(f"{BASE}/datasets/{ds['dataset_id']}/examples", headers=H, json={ "input": "Refund my last order", "expected_output": "Opened refund #4821", # link the run so Fluiq pins its full trajectory for agentic eval "metadata": {"source_trace_id": "0f9c...e21"}, })