Skip to content

fix(preprocess): use INTER_AREA when downscaling images for model input#2519

Draft
patricknihranz wants to merge 1 commit into
mainfrom
fix/area-interpolation-on-downscale
Draft

fix(preprocess): use INTER_AREA when downscaling images for model input#2519
patricknihranz wants to merge 1 commit into
mainfrom
fix/area-interpolation-on-downscale

Conversation

@patricknihranz

Copy link
Copy Markdown
Contributor

What

Selects cv2.INTER_AREA (and antialias=True on the PyTorch path) instead of the implicit bilinear default when the model's preprocessing resize is shrinking an image to the network input size. Applies to both resize modes:

  • Stretch toinference/core/models/roboflow.py
  • Fit (black/white/grey edges) in — via resize_image_keeping_aspect_ratio in inference/core/utils/preprocess.py

Upscaling keeps bilinear (INTER_AREA degenerates toward nearest-neighbor when enlarging).

Why

The resize calls passed no interpolation argument, so OpenCV defaulted to INTER_LINEAR (bilinear). Bilinear samples only a 2×2 neighborhood per output pixel, so on large downscales (e.g. a 3120×4160 image → 640×640, ~6.5× per axis) it point-samples and aliases — high-frequency detail like thin, low-contrast embossed text is discarded rather than averaged in. INTER_AREA integrates over the entire source region each output pixel covers, acting as a proper anti-aliasing filter.

This surfaced on a customer (GE Vernova) serial-number OCR model: sending a full-resolution image silently dropped one of two detection regions, while resizing first with INTER_AREA (via the image_preprocessing workflow block, which already uses INTER_AREA) recovered both. The Models-page UI "worked" only incidentally — it downsamples to 1536×2048 before inference, reducing the final downscale ratio and thus the aliasing.

Impact / risk

  • Detection coordinates are unaffected (output geometry unchanged; only sample quality improves).
  • Better recall on high-resolution inputs with fine detail; no expected regression on inputs already near model size (downscale ratio ~1 → bilinear and area converge).
  • Brings inference resize quality in line with the image_preprocessing block (INTER_AREA) and, likely, dataset-generation preprocessing — reducing train/serve skew. Open question for reviewers: please confirm the interpolation used during version generation so inference matches it exactly.

Repro (customer A/B, model serialnumber-001/15, same image)

Path Result
Full-res 3120×4160 → model auto-resize (bilinear) 1 of 2 regions
INTER_AREA resize → 640×640 → model 2 of 2, high confidence
Same-size A/B: INTER_AREA-to-640 vs bilinear-to-640 isolates interpolation as the only variable

Out of scope (follow-ups)

  • The PyTorch antialias flag only applies to mode="bilinear"/"bicubic" — fine here, but worth a guard if modes are ever made configurable.
  • Separate bug: Fit within, Fill (with center crop) in, and Fit (reflect edges) in silently fall back to Fit (black edges) in in roboflow.py.

Test plan

  • Unit: assert INTER_AREA chosen when target < source and INTER_LINEAR when target ≥ source, for both Stretch to and the letterbox path.
  • Regression: detection on a high-res fine-detail fixture recovers previously-missed boxes; near-model-size inputs unchanged within tolerance.

🤖 Generated with Claude Code

The model's built-in resize ('Stretch to' and the 'Fit ... edges in'
letterbox path) called cv2.resize / F.interpolate without specifying an
interpolation method, defaulting to bilinear. Bilinear point-samples a
2x2 neighborhood and aliases on large downscales, eroding thin,
low-contrast features (e.g. embossed serial-number text) and causing
silently missed detections on high-resolution inputs.

Select INTER_AREA (and antialias=True on the torch path) when the target
is smaller than the source, which averages over the full source region;
keep bilinear when upscaling, where INTER_AREA degenerates to nearest.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant