Label Format Check

Available

semantic_segmentation.label_format_check

Fraction of semantic-segmentation labels with valid mask format.

Contract

Field	Value
version	1.0.0
metric_type	pointwise
scorer_contract	full_dataset
direction	higher_is_better
entrypoint	aip_scorers_semantic_segmentation.answered.label_format_check
target_kind	None

Required columns

• image_id
• label

Accepted schemas

[
  {
    "name": "gdi_image_v1",
    "task_types": [
      "semantic_segmentation"
    ]
  }
]

Methodology

1. Decode canonical labels at the GDI boundary.
2. Run cv-eval's native LabelFormatCheck over every image.
3. Report malformed or absent masks as findings without aborting the dataset.

Score semantics

Scores range from 0 to 1. Higher is better; 1 means every label has valid (H, W) mask format.

Worked example

One valid label and one malformed label produce a score of 0.5.

Configuration schema

{
  "properties": {},
  "title": "InitializationConfig",
  "type": "object"
}

Execution

{
  "emits_metric_family": false,
  "function_name": null,
  "max_concurrency": null,
  "processing_kind": "cpu",
  "stream_batch_size": null,
  "timeout_seconds": 600
}

Complete manifest

accepts:
- name: gdi_image_v1
  task_types:
  - semantic_segmentation
config_schema:
  properties: {}
  title: InitializationConfig
  type: object
dependencies: []
description: Percentage of semantic-segmentation labels with valid (H, W) mask format.
direction: higher_is_better
display_name: Label Format Check
entrypoint: aip_scorers_semantic_segmentation.answered.label_format_check
execution:
  emits_metric_family: false
  function_name: null
  max_concurrency: null
  processing_kind: cpu
  stream_batch_size: null
  timeout_seconds: 600
kind: metric
manifest_version: '1'
max_prompt_slots: 0
metric_metadata:
  methodology:
  - Decode canonical labels at the GDI boundary.
  - Run cv-eval's native LabelFormatCheck over every image.
  - Report malformed or absent masks as findings without aborting the dataset.
  score_semantics: Scores range from 0 to 1. Higher is better; 1 means every label
    has valid (H, W) mask format.
  summary: Fraction of semantic-segmentation labels with valid mask format.
  worked_example: One valid label and one malformed label produce a score of 0.5.
metric_type: pointwise
name: semantic_segmentation.label_format_check
partition_types: []
required_columns:
- image_id
- label
required_kinds: []
scorer_contract: full_dataset
target_kind: null
unsupported_trace_shapes: []
version: 1.0.0