Skip to main content

aip_sdk.JudgeConnection

aip_sdk.JudgeConnection(data: dict[str, Any], client: APIClient)

A registered judge connection — a named LLM judge (model, endpoint, key).

Register a judge once, then reference it when scoring metrics instead of pasting raw credentials into eval configs. The API key is write-only: it is sent at registration and never returned, so it is not exposed as an attribute.

Attributes

  • id str: Connection ID.
  • name str: Human-readable name, unique within a workspace.
  • model str: Judge model id, e.g. "gpt-4o-mini".
  • base_url str | None: OpenAI-compatible base URL as stored, or None to use the platform default judge. A trailing slash and a trailing /chat/completions are stripped on write, so this may differ from the string that was sent.
  • project_id str | None: Project this connection is scoped to, if any.
  • workspace_id str | None: Workspace this connection is scoped to, if any.
  • created_at datetime | None: When the connection was registered.
  • updated_at datetime | None: When the connection was last modified.

aip_sdk.JudgeConnection.base_url​

aip_sdk.JudgeConnection.base_url: str | None = data.get('base_url')

No docstring is defined in the source.

aip_sdk.JudgeConnection.created_at​

aip_sdk.JudgeConnection.created_at: datetime | None = parse_dt(data['created_at'])

No docstring is defined in the source.

aip_sdk.JudgeConnection.delete​

aip_sdk.JudgeConnection.delete() -> None

Delete this judge connection.

Raises

  • JudgeConnectionNotFoundError: If the connection no longer exists, or belongs to a workspace the caller is not a member of.
  • ForbiddenError: If the caller is not an editor or admin of the connection's workspace. A platform admin must be a member of that workspace too.
  • AuthError: If credentials are missing or invalid.
  • APIError: If the deletion otherwise fails.

aip_sdk.JudgeConnection.id​

aip_sdk.JudgeConnection.id: str = data['id']

No docstring is defined in the source.

aip_sdk.JudgeConnection.model​

aip_sdk.JudgeConnection.model: str = data['model']

No docstring is defined in the source.

aip_sdk.JudgeConnection.name​

aip_sdk.JudgeConnection.name: str = data['name']

No docstring is defined in the source.

aip_sdk.JudgeConnection.project_id​

aip_sdk.JudgeConnection.project_id: str | None = data.get('project_id')

No docstring is defined in the source.

aip_sdk.JudgeConnection.test​

aip_sdk.JudgeConnection.test() -> JudgeTestResult

Run a live test call through this connection's model, endpoint, and key.

Issues a minimal chat completion to confirm the judge is reachable, the credential is accepted, and the model is available. Never raises for a judge-side failure — inspect JudgeTestResult.success and reason.

Returns

  • JudgeTestResult JudgeTestResult: the outcome, with success and an actionable reason.

Raises

Example:

result = connection.test()
if not result.success:
    print(result.reason)

aip_sdk.JudgeConnection.update​

aip_sdk.JudgeConnection.update(*, name: str | None = None, model: str | None = None, base_url: str | None = None, api_key: str | None = None, project_id: str | None = None) -> JudgeConnection

Update fields on this connection, or rotate its API key.

Only the fields you pass are changed. Pass api_key to replace the stored credential; the previous key is discarded.

Parameters

  • name str | None: New name; must stay unique within the workspace.
  • model str | None: New judge model id.
  • base_url str | None: New OpenAI-compatible base URL. A trailing slash and a trailing /chat/completions are stripped before storage.
  • api_key str | None: New plaintext API key, encrypted at rest by the platform.
  • project_id str | None: Move the connection to this project.

Returns

Raises

aip_sdk.JudgeConnection.updated_at​

aip_sdk.JudgeConnection.updated_at: datetime | None = parse_dt(data['updated_at'])

No docstring is defined in the source.

aip_sdk.JudgeConnection.workspace_id​

aip_sdk.JudgeConnection.workspace_id: str | None = data.get('workspace_id')

No docstring is defined in the source.