Glossary

These definitions connect the platform's domain terms to the workflows that use them.

Term	Meaning
Project	The container for a body of evaluation work. It sets the dataset schema, task type, and analysis dimensions. Manage projects.
Golden Dataset Interface (GDI)	The schema contract defining required and optional columns for a dataset and its scoring operations. Text, images, and agent traces use different schemas.
Dataset	Versioned test-case rows, including inputs, ground truth, and dimension values. Uploads and mappings produce new versions. Work with datasets.
Golden dataset	The dataset version promoted as an evaluation baseline after readiness checks. Agent-trace ingestion follows its own versioning workflow. Manage golden versions.
Quality check	A readiness check for properties such as required fields, identifiers, annotations, and coverage. The combined verdict informs promotion.
Analysis dimension	A categorical field used to assess coverage and group evaluation results.
System Under Test (SUT)	The model or service being evaluated. A registered SUT needs a connection before AIP can call it. Connect a SUT.
Connection	The endpoint, authentication, protocol, and response mapping that tell AIP how to invoke a SUT and interpret its output.
Op	A registered unit of work, such as a metric, transformation, or synthetic-data operation, identified by a fully qualified key.
Metric / scorer	An operation that scores a response or another evaluation target. Metrics declare their accepted schema and required inputs. Select metrics.
Evaluation config	A reusable configuration of evaluation targets, metrics, metric parameters, and thresholds. Manage evaluation configs.
Run	One execution of an evaluation workflow. A hosted runner invokes the SUT; an external runner supplies inference results for the platform to score. Run evaluations.
Results	Individual metric outputs and their aggregates, such as counts, means, and pass rates.
Analysis	Statistical views of results, including distributions, slices, and comparisons. Review results.
Synthetic Data Generation (SDG)	Operations that create or vary examples to extend dataset coverage. Generate test data.
Span / trace / session	A span records one agent operation; a trace groups the spans of an agent run; a session can group related traces. Evaluate an agent.
Partition	A saved rule selecting the part of a trace dataset to evaluate, such as tool spans. A partition identifier and its granularity are separate concepts.
Partition type	The granularity of an agent evaluation target: session, trace, or span. Each metric declares the levels it supports.
Resolved partition	One concrete target produced by resolving a partition rule. The canonical partition schema carries its target reference and relevant ground truth.
Embedding projection	A reduced-dimensional layout used to explore semantically related examples and failure groups. Explore embeddings and clusters.
Assigned dimension	A dataset category saved from clustering or manual curation and available for later analysis.

The platform mental model explains how these objects fit together. Definitions are reviewed against the repository's domain reference and the linked workflows.