Evaluate an Instance Segmentation Model

Validate instance IoU and Dice with a five-image reference dataset, then persist a platform run.

This follows the semantic segmentation tutorial, but preserves each object's identity, including overlapping masks. The bundled dataset already contains predictions, so the example uses metric_invoke without calling a model endpoint.

Verify the reference locally

Use Python 3.12 and the repository's locked metrics environment. From the repository root, run:

uv run --project libs/aip-metrics python libs/aip-sdk/examples/instance_segmentation.py

mean_iou=0.791667 mean_dice=0.805556

The script validates the fixture schema, discovers both metrics and asserts their scores. The fixture is libs/aip-sdk/examples/data/instance_segmentation_gdi_image_v1.parquet. The adjacent instance_segmentation_reference.md records the geometry and hand-calculated fractions. The five cases cover perfect predictions, disjoint masks, partial overlap, overlapping instances of two classes, and no instances.

Understand the input and scores

Use schema gdi_image_v1 and task type instance_segmentation. Each row has image_id, embedded image bytes or a usable image_path, and label. Stored predictions use a separate predictions column. Both annotation lists contain one object per instance with class and mask. Ground-truth objects also carry an integer instance_id; predictions do not need one, and the decoder ignores it if present. Prediction score is optional. Use compressed COCO RLE: mask has size: [height, width] and an ASCII counts string, not an array of uncompressed run lengths. All masks in a row must share dimensions.

Preserve separate masks even where two objects of the same class overlap. IDs pass through as given: duplicates and class-scoped numbering such as ship 0, ship 1, buoy 0 are accepted, and matching uses mask overlap, never the ID. The shared decoder preserves ground-truth IDs and each mask channel. An empty list is valid. When both lists are empty, supply positive metadata.height and metadata.width because there is no mask from which to infer dimensions.

Metric	Overlap for a matched pair	Fixture result
instance_segmentation.mean_iou	intersection / union	19/24
instance_segmentation.mean_dice	twice intersection / total mask areas	29/36

Both match predictions to ground truth within the same class, highest confidence first, using IoU ≥ 0.5 by default. A ground-truth instance can be matched once. Each unmatched object on either side adds a zero to that class's denominator. The headline is the mean of the observed class scores. The partial-overlap case has two six-pixel masks, four shared pixels and eight union pixels: IoU is 1/2 and Dice is 2/3. Across the fixture, ship scores 7/12 and 11/18; buoy scores 1 for both. Averaging ship and buoy gives the headline values above.

These are full-dataset metrics. Their values repeat on valid rows, rather than representing individual image scores. Use the mean_iou_per_class and mean_dice_per_class artifacts for class scores and matched, missed and spurious instance counts. Both artifacts must report five scored rows for this fixture.

Persist the platform run

Start the platform and deploy the two instance metric functions using the repository's deployment workflow. The required op directories are nuclio/metrics/instance_segmentation/mean_iou and nuclio/metrics/instance_segmentation/mean_dice. The quality-check functions must also be available. Configure AIP_BASE_URL and authentication using libs/aip-sdk/examples/.env.example and the authentication guide. Export those variables into the shell; the script does not read a dotenv file.

From the repository root:

uv run --project libs/aip-sdk python libs/aip-sdk/examples/instance_segmentation.py --live

On success the script prints Persisted scores verified: followed by the actual run URL. The URL and run ID depend on your environment. This command requires a running platform; a successful local run is not evidence that it completed.

The script checks live metric discovery, creates a project in the Default workspace, uploads the fixture and runs row_count, duplicate_ids and schema_conformance. It then promotes the version to golden with an explicit reason for using a five-row fixture, runs both metrics and checks persisted scores and instance counts. Repeating it creates another dataset and run.

Open the printed URL. Confirm both score cards, the per-class scores and the five fixture rows. Record the deployed commit, run ID, input fixture, persisted scores and screenshots when using this journey as delivery evidence.

Connect an instance model

For a live system under test, choose the cv_instance_segmentation SUT template. Its connection uses sut_protocol="huggingface_instance_segmentation", gdi_schema="gdi_image_v1" and task_type="instance_segmentation". Supply the endpoint URL and its authentication on the connection. Put the optional threshold, mask_threshold and overlap_mask_area_threshold in model_params. The builder pins subtask to instance and refuses any other value, so there is nothing to set for it.

The builder sends a JSON request containing the base64 image and pipeline parameters. The endpoint returns a list of objects with label, mask and optional score. The adapter converts base64 PNG masks to compressed COCO RLE, maps label to class and assigns instance IDs from response positions. Ensure model class names match the ground-truth vocabulary before evaluation; a different name is scored as a different class, not automatically translated.

Use a golden dataset containing images and ground truth with a hosted run bound to that SUT connection. Hosted inference supplies predictions. Keep metric_invoke for datasets that already contain predictions, as this example does. The synthetic canvases test scoring, not a model's predictive quality.

Resolve failures

• Missing metrics: install the instance scorer package for local scoring, or deploy and register both ops for a platform run. Namespaced names avoid ambiguity with semantic metrics. Bare mean_iou and mean_dice resolve only when the frame declares the supported task type.
• Schema or decoding errors: check integer ground-truth IDs, compressed RLE, shared dimensions and metadata on empty rows. Inspect adapter errors for dropped SUT segments.
• Promotion refused: inspect the stored quality report. An intentional small sample can be acknowledged; invalid schema content must be corrected.
• Missing or misleading scores: an all-empty dataset cannot produce a score. A row empty on only one side scores zero. Confirm scored_rows equals dataset_rows so excluded malformed rows do not go unnoticed.
• Different results: confirm the default match threshold and normalisation. A row's confidence_threshold filters scored predictions; predictions without confidence remain eligible and are considered after ranked predictions.