Skip to main content

aip_sdk.Dataset

aip_sdk.Dataset(data: dict[str, Any], client: APIClient)

A dataset record.

Attributes

  • user_id str | None: User attribution recorded for the dataset. None means unattributed. This includes historical datasets, admin or service callers that named no target user, and datasets created by automated trace ingestion.
  • attribution_reported bool: Whether the server reported attribution for this dataset at all. False against a platform predating the field, where user_id is None on every dataset and means "not reported" rather than "unattributed" — check this before reading a fleet-wide None as a real attribution gap.

aip_sdk.Dataset.analysis_run​

aip_sdk.Dataset.analysis_run(version_id: str, run_id: str) -> DatasetAnalysisRun

Return one dataset-analysis run.

Calls GET /datasets/{id}/versions/{version_id}/analysis-runs/{run_id}.

aip_sdk.Dataset.analysis_runs​

aip_sdk.Dataset.analysis_runs(version_id: str) -> list[DatasetAnalysisRun]

Return dataset-analysis run history for a version.

Calls GET /datasets/{id}/versions/{version_id}/analysis-runs.

aip_sdk.Dataset.attribution_reported​

aip_sdk.Dataset.attribution_reported: bool = 'user_id' in data

No docstring is defined in the source.

aip_sdk.Dataset.available_checks​

aip_sdk.Dataset.available_checks() -> list[QualityCheckOption]

Return the quality checks that can run against this dataset.

Calls GET /datasets/{id}/available-checks. The set follows from the dataset's schema and the checks registered for its workspace, so it changes as checks are registered or retired. Pass any subset of the returned check values to run_checks().

Returns

  • list[QualityCheckOption]: A list of QualityCheckOption, sorted by check. Empty only when the
  • list[QualityCheckOption]: dataset's schema genuinely admits no registered check.

Raises

  • NotFoundError: If the dataset does not exist, or belongs to a workspace the caller is not a member of.
  • AuthError: If the configured credentials are rejected.
  • ResponseParseError: If the response does not carry a check catalogue.

Example:

options = dataset.available_checks()
print([option.check for option in options])
report = dataset.run_checks(version_id, checks=["row_count", "duplicate_ids"])

aip_sdk.Dataset.check_runs​

aip_sdk.Dataset.check_runs(version_id: str, *, page: int = 1, per_page: int = 50) -> list[DataQualityRun]

Return the quality-check run history for a dataset version.

Calls GET /datasets/{id}/versions/{version_id}/quality-checks/runs and returns all runs newest-first.

Parameters

  • version_id str: The version to fetch run history for.
  • page int: Page number to fetch.
  • per_page int: Maximum runs per page.

Returns

Example:

runs = dataset.check_runs(version_id, page=1, per_page=20)
for r in runs:
    print(r.label, r.status, r.checked_at)

aip_sdk.Dataset.clusters​

aip_sdk.Dataset.clusters(version_id: str) -> DatasetClusters

Access clustering operations and saved dashboard results for a version.

Parameters

  • version_id str: Exact dataset version to cluster or inspect.

Returns

  • DatasetClusters: DatasetClusters using this dataset's authenticated client.

Raises

aip_sdk.Dataset.created_at​

aip_sdk.Dataset.created_at = parse_dt(data['created_at']) or datetime.now(tz=UTC)

No docstring is defined in the source.

aip_sdk.Dataset.delete​

aip_sdk.Dataset.delete() -> None

Permanently delete this dataset, all of its versions, and their stored files.

Irreversible and unprompted. To retire a dataset while keeping it readable, use aip.datasets.patch_dataset_status(id, "deprecated") instead. This handle keeps its attributes after a successful delete, but every method that reaches the platform will then fail with a not-found error.

Returning successfully means the dataset and its versions are gone. Purging the stored files is best-effort and completes after the deletion itself, so an object store outage can leave files behind for the platform to clean up.

Requires the workspace_admin or workspace_editor role.

Returns

  • None: None.

Raises

  • DatasetNotFoundError: If the dataset was already deleted.
  • DatasetInUseError: If evaluation runs reference one of its versions, or another dataset was derived from it.
  • ForbiddenError: If the caller lacks the workspace_admin or workspace_editor role.
  • AuthError: If credentials are missing or invalid.

aip_sdk.Dataset.demote​

aip_sdk.Dataset.demote(version_id: str) -> DatasetVersion

Demote a golden version back to non-golden.

Reverts is_golden to False and the stage back to mapped (or raw if no mapping was ever applied).

Parameters

  • version_id str: The golden version to demote.

Returns

Raises

aip_sdk.Dataset.download​

aip_sdk.Dataset.download(file_version: str = 'converted', max_retries: int = 3, backoff_factor: float = 1.0) -> pd.DataFrame

Download the dataset as a pandas DataFrame via presigned URL.

Fetches a short-lived presigned MinIO URL from the API then downloads the Parquet directly — no auth headers are sent to MinIO. Transient network errors and 5xx responses are retried with exponential backoff.

Parameters

  • file_version str: "converted" (default) for the mapped Parquet file — always safe to parse as a DataFrame. "original" returns the raw file as uploaded; this only works if the original was a Parquet file — CSV/JSON originals will raise during parsing.
  • max_retries int: Maximum download attempts (default 3).
  • backoff_factor float: Multiplier for exponential wait between retries (default 1.0).

Returns

  • pd.DataFrame: pandas DataFrame.

Raises

  • DownloadError: If the download fails after all retries, or immediately on 4xx.

Example:

df = dataset.download()  # mapped Parquet (recommended)
df = dataset.download("original")  # only works for Parquet originals
df = dataset.download(max_retries=5, backoff_factor=2.0)  # custom retry config

aip_sdk.Dataset.id​

aip_sdk.Dataset.id = data['id']

No docstring is defined in the source.

aip_sdk.Dataset.latest_row_count​

aip_sdk.Dataset.latest_row_count: int | None = data.get('latest_row_count')

No docstring is defined in the source.

aip_sdk.Dataset.latest_version​

aip_sdk.Dataset.latest_version() -> DatasetVersion

Return the most recently created version.

Raises

Example:

v = dataset.latest_version()
print(v.stage)  # "raw" right after upload

aip_sdk.Dataset.latest_version_number​

aip_sdk.Dataset.latest_version_number: int | None = data.get('latest_version_number')

No docstring is defined in the source.

aip_sdk.Dataset.map_version​

aip_sdk.Dataset.map_version(version_id: str, column_mapping: dict[str, str] | None = None) -> DatasetVersion

Apply column mapping and advance the version stage to mapped.

column_mapping maps GDI field names → your dataset's column names (the same direction as the UI mapping dialog). Re-sending a mapping the version already records is accepted, so retries are safe. Mapping never overwrites an existing object: the result always lands on a new location and the version is repointed at it, so changing the mapping re-derives from the original uploaded file for versions that have one. Augmented, trace-landed and ground-truth versions have no upload to re-derive from, so a different second mapping there resolves against whatever the previous mapping produced and is rejected rather than silently mis-applied.

Parameters

  • version_id str: The version to map.
  • column_mapping dict[str, str] | None: e.g. {"prompt": "question", "input_id": "id"}. Pass None or {} to just advance the stage without renaming any columns.

Returns

Raises

  • UnprocessableEntityError: If a mapping value names a column that is not there — which is what an inverted {uploaded_column: gdi_field} mapping looks like — or if two fields claim the same source column, which would drop one of the two renames. The error lists the available columns. Also raised if this version's upload was already overwritten by a historical mapping bug — its original cannot be recovered, and re-mapping is refused; upload a fresh version instead.

Example:

v = dataset.latest_version()  # stage: raw
v = dataset.map_version(v.id, {"prompt": "question", "input_id": "id"})
print(v.stage)  # mapped

aip_sdk.Dataset.name​

aip_sdk.Dataset.name = data['name']

No docstring is defined in the source.

aip_sdk.Dataset.project_id​

aip_sdk.Dataset.project_id: str | None = data.get('project_id')

No docstring is defined in the source.

aip_sdk.Dataset.promote​

aip_sdk.Dataset.promote(version_id: str, force: bool = False, reason: str | None = None) -> DatasetVersion

Promote a version to golden — the canonical evaluation set.

The version must be in mapped stage (or raw if no mapping is needed). After promotion is_golden=True and stage="golden".

The latest quality verdict must be PASS. A WARN verdict can be overridden by passing force=True — an explicit, audited acknowledgement of the warnings. FAIL, STALE and NOT_RUN verdicts reject regardless of force.

Parameters

  • version_id str: The version to promote.
  • force bool: Acknowledge a WARN quality verdict and promote anyway. No-op when the verdict is PASS.
  • reason str | None: Optional free-text acknowledgement stored in the audit record of a forced promotion.

Returns

Raises

Example:

v = dataset.map_version(v.id, {...})
v = dataset.promote(v.id)
print(v.stage)  # golden

aip_sdk.Dataset.quality_check_run​

aip_sdk.Dataset.quality_check_run(version_id: str, run_id: str) -> DataQualityRun

Return one quality-check run with per-check evidence.

Calls GET /datasets/{id}/versions/{version_id}/quality-checks/runs/{run_id}.

aip_sdk.Dataset.quality_verdict​

aip_sdk.Dataset.quality_verdict(version_id: str) -> dict[str, Any]

Return the latest quality-check verdict projection for a version.

Calls GET /datasets/{id}/versions/{version_id}/quality-checks. The returned dict contains verdict, run_id, checked_at, row_count, and per-check summary evidence.

aip_sdk.Dataset.query_visual_analysis​

aip_sdk.Dataset.query_visual_analysis(version_id: str, run_id: str, *, filters: list[dict[str, Any]] | None = None, order_by: str | None = None, limit: int | None = None) -> list[dict[str, Any]]

Query per-row visual property values for an analysis run.

Parameters

  • version_id str: The version ID analysed by run_id.
  • run_id str: The analysis run ID.
  • filters list[dict[str, Any]] | None: Optional filter clauses such as {"column": "brightness", "op": ">", "value": 0.8}.
  • order_by str | None: Optional SQL-like order clause, e.g. "brightness DESC".
  • limit int | None: Optional maximum row count.

aip_sdk.Dataset.run_analysis​

aip_sdk.Dataset.run_analysis(version_id: str, triggered_by: str = 'sdk') -> DatasetAnalysisRun

Run dataset analysis for a specific dataset version.

Supports text-length analysis for gdi_text_v1 and visual image-property analysis for gdi_image_v1 via POST /datasets/{id}/versions/{version_id}/analysis-runs. Text per-row properties include character count, word count, sentence count, average word length and approximate token count for one primary column: prompt when available, otherwise the first profiled column. The response names it in prompt_column; other profiled columns have aggregate distributions in the analysis metadata. Token counts use a regex approximation, not a model's billing tokenizer.

This call blocks until analysis finishes and returns a terminal run. Check run.status before reading text_analysis(version_id, run.id) or visual_analysis(version_id, run.id) for the matching schema; the returned run can be failed. Large versions can exceed the client's 120-second read timeout. The client retries timed-out requests, including this POST, up to three attempts, which can create duplicate runs. After a timeout, reconcile with analysis_runs(version_id) before starting another analysis; do not blindly repeat run_analysis.

Analysis runs are separate from data-quality verdicts and do not change golden rows or declared dimensions. Inspect computed_count and failed_indices for partial coverage. The latter contains positional offsets, not business row IDs.

Parameters

  • version_id str: The version ID to analyse.
  • triggered_by str: Free-form origin label, defaults to "sdk".

Returns

aip_sdk.Dataset.run_checks​

aip_sdk.Dataset.run_checks(version_id: str, label: str | None = None, *, checks: list[str] | None = None) -> DataQualityReport

Run data quality checks on a dataset version and return the verdict.

Triggers POST /datasets/{id}/versions/{version_id}/quality-checks:run. The platform dispatches the checks, persists the run and its per-check evidence, updates the version's latest verdict, and returns that verdict. This is the only way to produce a quality verdict, and so the only thing the golden-promotion gate reads.

Parameters

  • version_id str: The version to check.
  • label str | None: Name for this run, shown in the run history. Defaults to "Run N" when omitted.
  • checks list[str] | None: The checks to run, named by the check values from available_checks(). Omit to run every check the dataset's schema admits.

Returns

  • DataQualityReport: DataQualityReport carrying the verdict and one entry per check that ran.

Raises

  • InvalidArgumentError: If checks is an empty list. Omit it to run every admitted check, or name at least one check to run.
  • NotFoundError: If the dataset or version does not exist.
  • ForbiddenError: If the caller lacks the workspace_admin or workspace_editor role in the dataset's workspace. Reading available_checks() needs no write role, so listing the catalogue can succeed where running it does not.
  • UnknownQualityCheckError: If checks names something that is not a quality check at all. .available lists every check the platform knows.
  • QualityCheckNotApplicableError: If checks names a real check this dataset's schema does not admit. .admitted lists what it can run instead.
  • RemovedQualityCheckError: If checks names a check that was replaced. .replacements maps each removed name to its successor.
  • UnprocessableEntityError: If the dataset's schema admits no registered check. The three errors above are subclasses of this, and all four of them leave the version's verdict untouched — a rejected selection runs nothing.
  • RunFailedError: If a check did not execute cleanly, leaving the verdict inconclusive.

Example:

report = dataset.run_checks(version_id, label="post-mapping")
print(report.status, report.row_count)

# Only the two checks this pipeline cares about.
report = dataset.run_checks(version_id, checks=["row_count", "duplicate_ids"])

aip_sdk.Dataset.schema_name​

aip_sdk.Dataset.schema_name = data['schema_name']

No docstring is defined in the source.

aip_sdk.Dataset.status​

aip_sdk.Dataset.status = data['status']

No docstring is defined in the source.

aip_sdk.Dataset.tags​

aip_sdk.Dataset.tags = data.get('tags', {})

No docstring is defined in the source.

aip_sdk.Dataset.task_type​

aip_sdk.Dataset.task_type: str | None = data.get('task_type')

No docstring is defined in the source.

aip_sdk.Dataset.text_analysis​

aip_sdk.Dataset.text_analysis(version_id: str, run_id: str) -> dict[str, Any]

Return the text-length-profile analysis result for a run.

Calls GET /datasets/{id}/versions/{version_id}/analysis/text with the run ID as a query parameter.

aip_sdk.Dataset.user_id​

aip_sdk.Dataset.user_id: str | None = data.get('user_id')

No docstring is defined in the source.

aip_sdk.Dataset.versions​

aip_sdk.Dataset.versions() -> list[DatasetVersion]

Return all versions of this dataset as typed objects, newest first.

The stage field shows where each version sits in the lifecycle (raw → mapped → golden).

Example:

for v in dataset.versions():
    print(v.id, v.stage, v.is_golden)

aip_sdk.Dataset.visual_analysis​

aip_sdk.Dataset.visual_analysis(version_id: str, run_id: str) -> dict[str, Any]

Return the visual image-property analysis result for a run.

Calls GET /datasets/{id}/versions/{version_id}/analysis/visual with the run ID as a query parameter.