Skip to main content

aip_sdk.SutInstance

aip_sdk.SutInstance(data: dict[str, Any], client: APIClient)

A registered System Under Test.

Attributes

  • id str: SUT registration ID
  • name str: SUT name
  • version str: SUT version string
  • owner str | None: Owner/team identifier
  • project_id str | None: Associated project ID
  • created_at datetime | None: When this SUT was registered
  • updated_at datetime | None: Last update timestamp

aip_sdk.SutInstance.add_connection​

aip_sdk.SutInstance.add_connection(base_url: str, label: str = 'default', auth_type: str = 'none', auth_header_name: str | None = None, auth_header_value: str | None = None, model_params: dict[str, Any] | None = None, sut_protocol: str | None = None, gdi_schema: str | None = None, task_type: str | None = None, sut_streaming: bool = False) -> Connection

Add a new connection to this SUT.

Parameters

  • base_url str: SUT endpoint URL
  • label str: Environment label (e.g., "staging", "prod"); unique among this SUT's connections
  • auth_type str: Authentication method ("none", "bearer", "api_key", "basic")
  • auth_header_name str | None: Header name for auth credentials
  • auth_header_value str | None: Plaintext credential (encrypted at rest by platform)
  • model_params dict[str, Any] | None: Static parameters passed to the selected SUT protocol
  • sut_protocol str | None: Invocation protocol, e.g. "openai_chat", "huggingface_object_detection", or "rag_api"
  • gdi_schema str | None: Compatible dataset format (e.g. "gdi_text_v1", "gdi_image_v1")
  • task_type str | None: Task discriminator for task-scoped schemas, e.g. "detection" for gdi_image_v1
  • sut_streaming bool: Stream the SUT response so evaluation records true time-to-first-byte instead of total round-trip. Recommended for long-running SUTs (video, agentic/trajectory, large RAG). Off by default.

Returns

Raises

Examples

>>> sut = aip.Sut.register("my-llm")
>>> conn = sut.add_connection(
...     base_url="https://api.openai.com/v1/chat/completions",
...     label="prod",
...     auth_type="bearer",
...     auth_header_name="Authorization",
...     auth_header_value="sk-...",
...     model_params={"model": "gpt-4o"},
...     sut_protocol="openai_chat",
...     gdi_schema="gdi_text_v1",
...     sut_streaming=True,
... )

aip_sdk.SutInstance.connection_count​

aip_sdk.SutInstance.connection_count: int = data.get('connection_count', 0)

No docstring is defined in the source.

aip_sdk.SutInstance.connections​

aip_sdk.SutInstance.connections() -> list[Connection]

List all connections for this SUT.

Returns

  • list[Connection]: List of Connection instances

aip_sdk.SutInstance.created_at​

aip_sdk.SutInstance.created_at: datetime | None = parse_dt(data['created_at'])

No docstring is defined in the source.

aip_sdk.SutInstance.delete​

aip_sdk.SutInstance.delete() -> None

Delete this SUT registration.

Cascades to all connections and their adapters.

Raises

  • NotFoundError: If the SUT no longer exists, or belongs to a workspace the caller is not a member of.
  • ForbiddenError: If the caller is not an editor or admin of the SUT's workspace. A platform admin must be a member of that workspace too.

aip_sdk.SutInstance.id​

aip_sdk.SutInstance.id: str = data['id']

No docstring is defined in the source.

aip_sdk.SutInstance.name​

aip_sdk.SutInstance.name: str = data['name']

No docstring is defined in the source.

aip_sdk.SutInstance.owner​

aip_sdk.SutInstance.owner: str | None = data.get('owner')

No docstring is defined in the source.

aip_sdk.SutInstance.project_id​

aip_sdk.SutInstance.project_id: str | None = data.get('project_id')

No docstring is defined in the source.

aip_sdk.SutInstance.runs​

aip_sdk.SutInstance.runs(page: int = 1, per_page: int = 20) -> dict[str, Any]

List evaluation runs that used this SUT.

Parameters

  • page int: Page number (1-indexed)
  • per_page int: Results per page

Returns

  • dict[str, Any]: Paginated response with run records

aip_sdk.SutInstance.update​

aip_sdk.SutInstance.update(name: str | None = None, version: str | None = None, owner: str | None = None) -> SutInstance

Update this SUT's metadata.

Parameters

  • name str | None: New SUT name
  • version str | None: New version string
  • owner str | None: New owner identifier

Returns

Raises

  • NotFoundError: If the SUT no longer exists, or belongs to a workspace the caller is not a member of.
  • ForbiddenError: If the caller is not an editor or admin of the SUT's workspace. A platform admin must be a member of that workspace too.

aip_sdk.SutInstance.updated_at​

aip_sdk.SutInstance.updated_at: datetime | None = parse_dt(data['updated_at'])

No docstring is defined in the source.

aip_sdk.SutInstance.version​

aip_sdk.SutInstance.version: str = data['version']

No docstring is defined in the source.