Skip to main content

aip_sdk.ops

Ops (operational scorers) registry access.

aip_sdk.ops.OpCatalogue​

aip_sdk.ops.OpCatalogue

A list of registry ops that prints as a scannable table.

A transparent list subclass: indexing, iteration, len(), and equality against a plain list behave exactly as before, and each element is an OpEntry. Only __str__ differs — print(catalogue) renders a Name/Kind/Version/Source/Description table rather than dumping every op's nested config_schema. The full data remains available via normal item access and json.dumps.

aip_sdk.ops.OpEntry​

aip_sdk.ops.OpEntry

A single registry op, rendered compactly when inspected in a REPL.

A transparent dict subclass: subscripting, .get(), iteration, and equality against a plain dict behave exactly as before. Only the __repr__ differs — it summarises the large nested config_schema as a property count instead of dumping the full JSON Schema, so evaluating a single entry in a notebook stays readable.

aip_sdk.ops.get_op​

aip_sdk.ops.get_op(name: str, client: APIClient | None = None) -> OpEntry

Get the full registry entry for a single op.

Parameters

  • name str: Op name (e.g. "correctness").
  • client APIClient | None: Optional API client.

Returns

  • OpEntry: An OpEntry (a dict subclass) with the full op contract: kind,
  • OpEntry: scorer_contract, accepts, required_columns, execution, description,
  • OpEntry: config_schema, metric_metadata (summary, score_semantics, methodology,
  • OpEntry: worked_example). Subscript/.get() access is unchanged; evaluating it
  • OpEntry: in a REPL shows a compact one-line summary rather than the raw JSON.

Raises

aip_sdk.ops.list_metrics​

aip_sdk.ops.list_metrics(schema: str | None = None, client: APIClient | None = None, *, kind: str | None = None, source: str | None = None, workspace_id: str | None = None, all_workspaces: bool = False) -> OpCatalogue

List metric ops, optionally filtered to those accepting schema.

Covers both row-level metrics (kind="metric" — pass their names to aip.run(scorers=...)) and agent-trace metrics (kind="trace_metric" — pass their names to aip.run(pipeline="trace_metric_invoke", targets=...) instead). Pass kind="metric" or kind="trace_metric" to narrow to just the one your pipeline accepts; omit it to see both.

kind="trace_metric" combined with workspace_id reads the workspace's own metric catalogue instead of the general registry — entries carry the same name a stored config for that metric reports as metric_name, so the result joins on name against aip_sdk.metric_configs.list_metric_configs(). schema/source, if given, are applied to that catalogue's own results rather than falling back to the general registry — otherwise the exact same call with an incidental source="builtin" added would silently switch to a differently-shaped result (a name whose winning row is a non-trace-metric kind absent from the catalogue but present in the general registry). Every other combination — the kind=None default and kind="metric" — still reads the general registry view, unchanged from before.

The reusable llm.* judges (e.g. llm.toxicity) come back in this catalogue with kind="metric" — their real registry kind, and the row a trace run actually dispatches them through — even though you asked for kind="trace_metric". Don't re-filter the result on op["kind"] == "trace_metric"; that drops all five.

Reads your session's workspace unless you name one, and raises if none is set. Pass all_workspaces=True to read across every workspace you can access.

Parameters

  • schema str | None: Optional GDI schema name; keeps ops accepting that schema.
  • client APIClient | None: Optional API client.
  • kind str | None: Optional narrowing to "metric" or "trace_metric"; both by default.
  • source str | None: Optional provenance filter ("builtin" | "upload" | "custom").
  • workspace_id str | None: Optional workspace to scope visibility to (globals + that workspace's custom ops). Membership is enforced server-side.
  • all_workspaces bool: Read across every workspace you can access.

Returns

Raises

aip_sdk.ops.list_ops​

aip_sdk.ops.list_ops(client: APIClient | None = None, *, kind: str | None = None, accepts: str | None = None, source: str | None = None, workspace_id: str | None = None, all_workspaces: bool = False) -> OpCatalogue

List registered ops from the DB registry.

Reads your session's workspace unless you name one, and raises if none is set. Pass all_workspaces=True to read across every workspace you can access.

Parameters

  • client APIClient | None: Optional API client.
  • kind str | None: Optional OpKind filter (e.g. "metric", "transform").
  • accepts str | None: Optional GDI schema name; keeps ops accepting that schema.
  • source str | None: Optional provenance filter ("builtin" | "upload" | "custom").
  • workspace_id str | None: Optional workspace to scope visibility to (globals + that workspace's custom ops). Membership is enforced server-side. Naming one is also what selects the workspace catalogue for kind="trace_metric" — a session default never reroutes the call on its own.
  • all_workspaces bool: Read across every workspace you can access.

Returns

  • OpCatalogue: An OpCatalogue (a list of OpEntry) of the registry ops. It behaves
  • OpCatalogue: like the plain list of op dicts it always was — each entry carries
  • OpCatalogue: source and workspace_id alongside the op contract — but
  • OpCatalogue: print() renders a scannable table instead of raw nested JSON.

Raises

aip_sdk.ops.register_op​

aip_sdk.ops.register_op(path: str | Path, *, workspace_id: str | None = None, client: APIClient | None = None) -> dict[str, Any]

Publish a workspace-scoped custom op from a local op folder.

Introspects the folder (imports its handler to read the @aip.ops identity plus the family contract), bundles it, and uploads it to the workspace-scoped publish route, which is gated to the workspace's admins. The op must already be deployed by the approving admin (deploy-first contract).

Parameters

  • path str | Path: Path to the op folder (handler.py, function.yaml, pyproject.toml, uv.lock).
  • workspace_id str | None: Target workspace the op is published into. Defaults to the session's workspace.
  • client APIClient | None: Optional API client.

Returns

  • dict[str, Any]: The registered op as a registry dict (source="custom", workspace_id set).

Raises

aip_sdk.ops.register_op_extractor​

aip_sdk.ops.register_op_extractor(kind: str, extractor: OpMetadataExtractor) -> None

Register the metadata extractor for an op kind.

Called once at import time by the kind's owning family package. This is the only place a new kind plugs in — the CLI command and the server write path are kind-agnostic and never change to add one.

aip_sdk.ops.register_op_from_manifest​

aip_sdk.ops.register_op_from_manifest(manifest_path: str | Path, *, workspace_id: str | None = None, client: APIClient | None = None) -> dict[str, Any]

Publish a workspace-scoped custom op from a declarative YAML manifest.

The declarative sibling of register_op(): instead of importing the handler to read its @aip.ops decorators, it reads an OpManifest that declares the op's identity and contract. The op folder is the manifest's parent; the manifest is validated (folder shape, dependency-file existence, entrypoint resolution), then the folder is bundled and uploaded through the same publish route as the folder path — so the resulting OpRegistryEntry is identical to a decorator-registered op's. The op must already be deployed by the approving admin (deploy-first contract).

Parameters

  • manifest_path str | Path: Path to the op's .yaml manifest (inside the op folder).
  • workspace_id str | None: Target workspace the op is published into. Defaults to the session's workspace.
  • client APIClient | None: Optional API client.

Returns

  • dict[str, Any]: The registered op as a registry dict (source="custom", workspace_id set).

Raises