Skip to main content

Platform Architecture

AIP is an evaluation platform for AI systems: you register the model or service you want to test, point AIP at a curated dataset with ground-truth answers, choose the metrics that matter, and run an evaluation that scores every row and aggregates the results. The same flow covers single-turn LLMs, retrieval-augmented generation, and computer-vision models — only the schema and the connection protocol change. Use the web console for an interactive workflow, the aip_sdk Python SDK for Python automation, or the HTTP API for direct integration.

AIP high-level architecture — client environment (users, SDK, test data, SUT) connecting to the AIP platform's web UI, control plane, execution engine (inference, transform, quality checks, metrics, SDG), data & evidence layer, and observability, producing scores, an evidence vault, and a governance report

A complete evaluation moves through a fixed lifecycle. Each stage produces an artifact the next stage consumes:

Connect (SUT) ─▶ Anchor (Project) ─▶ Bring & Curate ─▶ Check Quality ─▶ Enrich (SDG) ─▶ Govern (Golden) ─▶ Configure (Metrics) ─▶ Run ─▶ Review & Report

Each stage plays the following role:

  • Connect (SUT) — register the model/service under test and the HTTP endpoint, auth, and response adapter AIP calls.
  • Anchor (Project) — declare the schema and analysis dimensions for a body of work.
  • Bring & Curate — upload rows (prompts and expected outputs) against the schema and curate them into a dataset.
  • Check Quality — validate the dataset against quality checks before it becomes a baseline.
  • Enrich (SDG) — expand coverage with synthetic data generation.
  • Govern (Golden) — promote the curated dataset to the evaluation baseline (golden dataset).
  • Configure (Metrics) — assemble the reusable (project, dataset, SUT, metrics) evaluation config.
  • Run — execute: generate SUT outputs, then score the metrics.
  • Review & Report — per-row scores plus aggregated metrics, analysis, and the final report.

The SDK and web console share the API control plane. Evaluation workers execute the configured system connection and metric operations; run records retain the resulting evidence. The diagram separates client inputs, execution, and the evidence layer.