Skip to main content

aip_sdk.publish_test_plan

aip_sdk.publish_test_plan(plan: TestPlan, *, workspace_id: str | None = None, client: APIClient | None = None) -> TestPlanRecord

Publish a new test plan's first version, resolving every reference before storing it.

Each metric, quality check, dataset and pinned metric config (MetricConfigRef) the plan names is resolved against the workspace's catalogue, and each metric's config keys and threshold bars are checked against what that metric accepts. Any unresolved reference rejects the whole plan; nothing is stored.

This creates a plan; it never edits one. A name already published in the workspace raises TestPlanNameExistsError rather than storing a further version of it, so a mistyped or reused name cannot quietly take over a plan somebody else owns. Publish a further version of an existing plan with publish_test_plan_version. Any version set on plan is ignored — the first version is always 1.

The serialized plan must stay under 256 KiB — metrics[...].config and dimension vocabulary entries are otherwise unbounded, so a very large judge prompt or vocabulary can cross it. A plan over the limit is rejected before it reaches validation, raising PayloadTooLargeError rather than TestPlanReferenceError.

The underlying client retries a request on a transport-level timeout, and a timeout does not mean the publish failed — it may have stored the plan and only the response was lost. This call guards against that itself: every call is sent with a fresh, internally generated idempotency key, so the SDK's own retry of a lost response always replays that same attempt's outcome (required_columns re-derived fresh, replayed=True) rather than minting a version you never asked for. There is nothing to configure — a genuinely new call, e.g. because you re-ran the script, gets its own fresh key and publishes for real, exactly as it should.

Parameters

  • plan TestPlan: The methodology to publish.
  • workspace_id str | None: Target workspace. Falls back to the client's configured workspace.
  • client APIClient | None: Explicit API client. Uses the default client when omitted.

Returns

  • TestPlanRecord: The published TestPlanRecord at version 1. record.replayed is True on the rare
  • TestPlanRecord: call whose response is this SDK's own retry of an earlier attempt that the platform
  • TestPlanRecord: already stored — see TestPlanRecord.replayed.

Examples

import aip_sdk as aip

plan = aip.TestPlan(
    name="toxicity-baseline",
    schema_name="gdi_text_v1",
    task_type="single_turn_llm",
    metrics={"llm.toxicity": {"threshold": 0.8}},
)
try:
    published = aip.publish_test_plan(plan, workspace_id="ws_001")
except aip.TestPlanReferenceError as err:
    for failure in err.failures:
        print(f"{failure.kind}: {failure.ref} — {failure.message}")
    raise
print(published.version, published.required_columns)

Raises

  • NoWorkspaceSelectedError: No workspace was given and the client has none configured.
  • InvalidArgumentError: A metric config value is not JSON-serializable.
  • TestPlanReferenceError: The plan names something that could not be resolved. Read failures for every unresolved reference.
  • UnprocessableEntityError: The plan was rejected for a reason other than an unresolved reference — a schema-level problem the request never resolves into failures.
  • TestPlanNameExistsError: The name is already published in this workspace. Publish under another name, or re-read the existing plan and publish the next version of it with publish_test_plan_version.
  • DuplicateTestPlanVersionError: Not the version race this exception is named for — a create conflicts on the name, never on a version, and that has its own error above. Kept as the fallback for a 409 the platform did not tag; genuinely unreachable in practice unless the platform has moved ahead of this SDK version.
  • IdempotencyKeyConflictError: Not reachable through this function in practice — every call generates its own key, so no two calls collide except the SDK's own retry of itself, which is always safe. Kept as a defensive except for whatever the platform actually returns.
  • PayloadTooLargeError: The serialized plan exceeds the platform's size limit.
  • AuthError: Credentials are missing, invalid, or expired.
  • ForbiddenError: The caller is not an editor or admin of the workspace.
  • NotFoundError: The workspace does not exist.
  • ResponseParseError: A platform response did not match the shape this SDK version expects — either the published record on success, or the reported failures on a 422 this SDK could not interpret. Usually means the platform is running a newer release; upgrading resolves it. If raised while parsing a 422, nothing was stored, matching TestPlanReferenceError's guarantee.
  • APIError: Any other error from the platform.