Embeddings and clustering

Use dataset clustering to explore unscored data, or attach an embedding config to a hosted evaluation to explore scored rows. Both workflows use the transform.embed_project operation, which preserves row identity and adds a nested embed column containing 2D coordinates, cluster labels and pipeline provenance. High-dimensional vectors are not persisted in run output.

Supported data

Schema	Embedding task	Input
gdi_text_v1	single_turn_llm	Prompt and expected output when present
gdi_text_v1	single_turn_rag	Prompt and retrieved context
gdi_image_v1	classification	Whole image
gdi_image_v1	detection	Object appearance, box layout or both

The task identifiers in the table determine the embedding method. Multi-turn text and segmentation embeddings are unsupported. Detection pools objects into one vector per image, so one plotted point represents an image, not an individual box.

Dataset clustering accepts 25 to 10,000 rows. Project configs may impose a lower maximum. Availability also depends on access and the computation operation being registered and ready in your environment.

Cluster a dataset version

Before you begin, configure SDK authentication and select an existing dataset version that you have permission to edit. This example sends requests to a running AIP deployment.

import aip_sdk as aip

aip.init()
dataset = aip.load_dataset(id="DATASET_ID")
clusters = dataset.clusters("VERSION_ID")
availability = clusters.availability()
if not availability.available:
    raise SystemExit(availability.reason)

try:
    run = clusters.run()
    print(run.id)  # Retain this ID to resume polling.
    result = clusters.wait(run.id, timeout=600)
    print(result.pipeline_id, result.pipeline, result.noise_rows)
except aip.DatasetClustersTimeoutError:
    raise SystemExit("Polling timed out. Resume with clusters.wait(run.id).")
except aip.AipError as exc:
    raise SystemExit(str(exc)) from exc

If the project has an embedding configuration, omit min_cluster_size to use its configured value. Pass min_cluster_size explicitly to override the minimum number of rows in each group. This setting does not determine how many clusters the operation creates. If the project has no embedding configuration, AIP uses its default unsupervised configuration.

Dataset clustering runs one unsupervised pipeline. It ignores supervised pipelines and selects the first unsupervised pipeline when sorted by derived pipeline ID. If a legacy configuration contains no unsupervised pipeline, AIP uses the default configuration.

AIP saves the resolved configuration with the clustering run. Later changes to the project configuration do not alter existing results.

The dataset dashboard displays saved points, including noise. Inspect result.pipeline for the stored model, projection, clustering and diagnostics; older results may omit provenance fields. A successful computation may contain only noise. Reading saved results does not require the operation to be available.

A workspace permits one active clustering job. Repeating its version and settings returns that job; a conflicting request raises ConflictError. Polling timeout does not cancel computation. Call clusters.get_run(run.id) to inspect status or resume clusters.wait(run.id). After an uncertain start response, check clusters.latest_run() before submitting again. Unsupported data or unavailable computation raises UnprocessableEntityError; a failed job raises RunFailedError.

Project a scored run

Pass a config dictionary as embeddings= to aip.run. This requires a hosted SUT, connection and compatible metrics. The following live-only example uses an existing text project, dataset and connection.

import aip_sdk as aip

aip.init()
hosted_config = {
    "embeddings": {"qa": {"modality": "text", "task_type": "single_turn_llm"}},
    "projections": {"unsup": {}},
    "clusterings": {"default": {"min_cluster_size": 5}},
    "pipelines": [
        {"embedding": "qa", "projection": "unsup", "clustering": "default"}
    ],
}

with aip.run(
    project="PROJECT_ID",
    dataset="DATASET_ID@v1",
    sut_id="SUT_ID",
    connection_id="CONNECTION_ID",
    metrics=["llm.bleu"],
    embeddings=hosted_config,
) as run:
    run.poll_status(interval=5, timeout=600)

print(run.embeddings.pipeline_ids)
coordinates = run.embeddings.coordinates

An embedding configuration combines three steps: embedding, projection and clustering. AIP derives each pipeline ID from its selected steps, producing IDs such as umap.qa.unsupervised.

Within one computation, pipelines that use the same embedding reuse the cached embedding chunks instead of computing them again. Every configuration must include at least one unsupervised pipeline as a baseline. Supervised projections can use an available per-row metric to guide the layout. Compare a supervised layout with the unsupervised baseline to understand how the selected metric affected the projection.

The platform snapshots the config when creating the run. Omitting embeddings= or passing None inherits a project default when one exists. Clear that default with project.set_embedding_config(None) to stop inheriting it.

Reading hosted coordinates requires no embeddings extra. Dictionary configs such as hosted_config work with aip.run(embeddings=...) without importing the local embedding library.

Optional local preview requires the SDK's [embeddings] extra, the relevant modality dependencies and a typed EmbedProjectConfig. compute() does not accept a dictionary. Convert the hosted config before passing it to local preview:

# Local-only: install the embedding dependencies and provide a compatible df.
from aip_embeddings_core import EmbedProjectConfig

local_config = EmbedProjectConfig.model_validate(hosted_config)
result = aip.embeddings.compute(df, local_config)

Inspect failures and save categories

A configuration can request several projections. If one pipeline fails, AIP keeps the coordinates from the pipelines that succeeded. For a hosted run, run.embeddings.pipelines lists every configured pipeline — a failed one carries status="failed" and its error, so you no longer need to compare IDs to spot a missing projection. run.embeddings.pipeline_ids remains the succeeded subset. For local preview, inspect result.failed for pipeline errors.

If no pipeline produces coordinates, the embedding operation fails. Evaluation scoring is separate: a hosted run can finish scoring successfully but have no saved embedding coordinates. In that case, run.embeddings.coordinates raises EmbeddingsError. Check that the expected pipeline IDs and coordinates are present before using the embedding plot; a completed evaluation status alone is insufficient.

Use Clusters to inspect the saved projection, filter categories through the legend or draw a lasso. The run explorer also offers numeric colouring and CSV export. Performance by category is on Explore, with Table and Chart views over the same grouped results. Preview filtering shows full dataset rows.

To persist clustered categories from a completed run:

# Live-only: run must contain a successful pipeline with non-noise labels.
saved = run.embeddings.save_clusters("semantic_groups", "umap.qa.unsupervised")
print(saved.id, saved.assigned_rows, saved.unassigned_rows)

Noise and null cluster labels are omitted. Saving an all-noise pipeline raises EmbeddingsError. Saving a name again replaces its assignments. See Assigned dimensions for scope, authorship and coverage on later versions.