Size-wise Performance

Available

object_detection.size_wise_performance

Small-object F1 degradation: how far the detector's F1 on small objects falls short of cv-eval's all-bands reference F1, measured at object-box grain.

Contract

Field	Value
`version`	1.0.0
`metric_type`	pointwise
`scorer_contract`	full_dataset
`direction`	higher_is_better
`entrypoint`	aip_scorers_object_detection.object_detection.size_wise_performance
`target_kind`	None

Required columns

image_id
label
predictions

Accepted schemas

[
  {
    "name": "gdi_image_v1",
    "task_types": [
      "detection"
    ]
  }
]

Methodology

Deterministic - no LLM. Wraps Resaro's cv-eval SizeWisePerformance test and reads its object-size table; the formula is not reimplemented here.
Convert every image's ground truth and predictions to cv-eval arrays and hand them to that test as records. Boxes without finite, strictly positive width and height cannot be binned by area, so they are discarded and counted per side.
cv-eval bins each box by area (width x height) into small, medium and large using its own COCO-style edges (32^2 and 96^2 pixels^2) unless area_range overrides them, matches detections to same-class ground truth in descending confidence order at IoU >= iou_threshold, and reports Recall / Precision / F1 / Mean IoU / mAP per band. Object-box area, never image resolution.
Read F1 for the 'small' band and for the table's 'All' row, and report (F1_small - F1_all) / F1_all. Negative means small objects are the weaker case.

Score semantics

score is a signed raw ratio, bounded below by -1.0: 0.0 means the small-object F1 equals the reference F1, and -1.0 means no small object was found. Higher is better. A normalized 0-1 value may be calculated internally and kept in scorer metadata, but it is not the value returned as score. The denominator is cv-eval's 'All' row, which is the arithmetic mean over the size bands present in its table - not the dataset F1 over all boxes - so it differs from object_detection.f1 whenever boxes are spread unevenly across the bands. Values are read from cv-eval's rendered table and are therefore quantised to three decimals before division. The metric is reported as N/A, never 0.0, when the comparison does not exist; its per-row details record the explicit reason.

Worked example

40 small ground-truth boxes with 20 detected and no spurious small detections gives the small band precision 1.0, recall 0.5 and F1 0.667. Adding 40 large boxes all detected gives the large band F1 1.000, and cv-eval's 'All' row averages the two present bands to 0.833. The reported ratio is (0.667 - 0.833) / 0.833 = -0.199.

Configuration schema

{
  "properties": {
    "area_range": {
      "anyOf": [
        {
          "maxItems": 2,
          "minItems": 2,
          "prefixItems": [
            {
              "type": "number"
            },
            {
              "type": "number"
            }
          ],
          "type": "array"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "description": "Box-area cut points in pixels^2 separating small/medium and medium/large. None uses cv-eval's own COCO-style edges (32^2, 96^2). Raise both for near-field imagery where a 32^2 box is genuinely tiny, or lower them for aerial imagery where most objects would otherwise land in 'small'.",
      "title": "Area Range"
    },
    "inverted": {
      "default": false,
      "title": "Inverted",
      "type": "boolean",
      "x-aip-param-role": "scoring_metadata"
    },
    "iou_threshold": {
      "default": 0.5,
      "title": "Iou Threshold",
      "type": "number"
    },
    "label_map": {
      "anyOf": [
        {
          "additionalProperties": {
            "type": "integer"
          },
          "type": "object"
        },
        {
          "type": "null"
        }
      ],
      "default": null,
      "title": "Label Map"
    },
    "max": {
      "default": 0.0,
      "title": "Max",
      "type": "number",
      "x-aip-param-role": "scoring_metadata"
    },
    "metric_name": {
      "default": "size_wise_performance",
      "title": "Metric Name",
      "type": "string",
      "x-aip-param-role": "scoring_metadata"
    },
    "min": {
      "default": -1.0,
      "title": "Min",
      "type": "number",
      "x-aip-param-role": "scoring_metadata"
    },
    "weight": {
      "default": 1.0,
      "title": "Weight",
      "type": "number",
      "x-aip-param-role": "scoring_metadata"
    }
  },
  "title": "InitializationConfig",
  "type": "object"
}

Execution

{
  "emits_metric_family": false,
  "function_name": "object_detection.size_wise_performance",
  "max_concurrency": null,
  "processing_kind": "cpu",
  "stream_batch_size": null,
  "timeout_seconds": 600
}

Complete manifest

accepts:
- name: gdi_image_v1
  task_types:
  - detection
config_schema:
  properties:
    area_range:
      anyOf:
      - maxItems: 2
        minItems: 2
        prefixItems:
        - type: number
        - type: number
        type: array
      - type: 'null'
      default: null
      description: Box-area cut points in pixels^2 separating small/medium and medium/large.
        None uses cv-eval's own COCO-style edges (32^2, 96^2). Raise both for near-field
        imagery where a 32^2 box is genuinely tiny, or lower them for aerial imagery
        where most objects would otherwise land in 'small'.
      title: Area Range
    inverted:
      default: false
      title: Inverted
      type: boolean
      x-aip-param-role: scoring_metadata
    iou_threshold:
      default: 0.5
      title: Iou Threshold
      type: number
    label_map:
      anyOf:
      - additionalProperties:
          type: integer
        type: object
      - type: 'null'
      default: null
      title: Label Map
    max:
      default: 0.0
      title: Max
      type: number
      x-aip-param-role: scoring_metadata
    metric_name:
      default: size_wise_performance
      title: Metric Name
      type: string
      x-aip-param-role: scoring_metadata
    min:
      default: -1.0
      title: Min
      type: number
      x-aip-param-role: scoring_metadata
    weight:
      default: 1.0
      title: Weight
      type: number
      x-aip-param-role: scoring_metadata
  title: InitializationConfig
  type: object
dependencies: []
description: How much worse is the model at detecting small objects than objects overall?
direction: higher_is_better
display_name: Size-wise Performance
entrypoint: aip_scorers_object_detection.object_detection.size_wise_performance
execution:
  emits_metric_family: false
  function_name: object_detection.size_wise_performance
  max_concurrency: null
  processing_kind: cpu
  stream_batch_size: null
  timeout_seconds: 600
kind: metric
manifest_version: '1'
metric_metadata:
  methodology:
  - Deterministic - no LLM. Wraps Resaro's cv-eval SizeWisePerformance test and reads
    its object-size table; the formula is not reimplemented here.
  - Convert every image's ground truth and predictions to cv-eval arrays and hand
    them to that test as records. Boxes without finite, strictly positive width and
    height cannot be binned by area, so they are discarded and counted per side.
  - cv-eval bins each box by area (width x height) into small, medium and large using
    its own COCO-style edges (32^2 and 96^2 pixels^2) unless area_range overrides
    them, matches detections to same-class ground truth in descending confidence order
    at IoU >= iou_threshold, and reports Recall / Precision / F1 / Mean IoU / mAP
    per band. Object-box area, never image resolution.
  - Read F1 for the 'small' band and for the table's 'All' row, and report (F1_small
    - F1_all) / F1_all. Negative means small objects are the weaker case.
  score_semantics: 'score is a signed raw ratio, bounded below by -1.0: 0.0 means
    the small-object F1 equals the reference F1, and -1.0 means no small object was
    found. Higher is better. A normalized 0-1 value may be calculated internally and
    kept in scorer metadata, but it is not the value returned as score. The denominator
    is cv-eval''s ''All'' row, which is the arithmetic mean over the size bands present
    in its table - not the dataset F1 over all boxes - so it differs from object_detection.f1
    whenever boxes are spread unevenly across the bands. Values are read from cv-eval''s
    rendered table and are therefore quantised to three decimals before division.
    The metric is reported as N/A, never 0.0, when the comparison does not exist;
    its per-row details record the explicit reason.'
  summary: 'Small-object F1 degradation: how far the detector''s F1 on small objects
    falls short of cv-eval''s all-bands reference F1, measured at object-box grain.'
  worked_example: 40 small ground-truth boxes with 20 detected and no spurious small
    detections gives the small band precision 1.0, recall 0.5 and F1 0.667. Adding
    40 large boxes all detected gives the large band F1 1.000, and cv-eval's 'All'
    row averages the two present bands to 0.833. The reported ratio is (0.667 - 0.833)
    / 0.833 = -0.199.
metric_type: pointwise
name: object_detection.size_wise_performance
partition_types: []
required_columns:
- image_id
- label
- predictions
required_kinds: []
scorer_contract: full_dataset
target_kind: null
version: 1.0.0

Contract​

Required columns​

Accepted schemas​

Methodology​

Score semantics​

Worked example​

Configuration schema​

Execution​

Complete manifest​