Skip to content

Hybrid agnostic NMS#14196

Closed
ohinds wants to merge 25 commits intoultralytics:mainfrom
ohinds:hybrid_agnostic_nms
Closed

Hybrid agnostic NMS#14196
ohinds wants to merge 25 commits intoultralytics:mainfrom
ohinds:hybrid_agnostic_nms

Conversation

@ohinds
Copy link

@ohinds ohinds commented Jul 3, 2024

Compute agnositc NMS considering a subset of classes, and non-agnostic NMS for the rest of the classes.

This is useful when some classes should not be colocated, while others should be.

For example, take a model trained to detect people and several classes of shirt, where each shirt class is trained on a different shirt color. In this case, we'd like to detect only a single shirt class per person by using agnostic NMS on just the shirt classes, while allowing the person class to be detected with substantial overlap with any shirt class.

This is accomplished by splitting the predictions during postprocessing, computing agnostic NMS for the shirts, and non-agnostic NMS for the person. This approach is implemented in this PR.

Minimal reproducible example:

import numpy as np
from ultralytics import YOLO

# image of a half apple, half orange, on a clock
source = "https://i.imgur.com/hulsgO5.jpeg"
model = YOLO('yolov8n.pt')

classes = [
    47,  # apple
    49,  # orange
    74,  # clock
]

conf = 0.01

# no colocated objects of the same class
no_agnostic_nms_results = model(source, classes=classes, conf=conf)
expected_no_agnostic_nms_cls = np.array([49, 49, 74, 49, 47], np.float32)
assert np.array_equal(expected_no_agnostic_nms_cls, no_agnostic_nms_results[0].boxes.cls.data.cpu().numpy())

# no colocated objects of any class
agnostic_nms_results = model(source, classes=classes, conf=conf, agnostic_nms=True)
expected_agnostic_nms_cls = np.array([49, 49, 49], np.float32)
assert np.array_equal(expected_no_agnostic_nms_cls, no_agnostic_nms_results[0].boxes.cls.data.cpu().numpy())

# no colocated apples and oranges, but either can be colocated with a clock
agnostic_nms_classes = [
    47,  # apple
    49,  # orange
]
hybrid_agnostic_nms_results = model(source, classes=classes, conf=conf, agnostic_nms_classes=agnostic_nms_classes)
expected_hybrid_agnostic_nms_cls = np.array([49, 49, 49, 74], np.float32)  # no apple
assert np.array_equal(expected_hybrid_agnostic_nms_cls, hybrid_agnostic_nms_results[0].boxes.cls.data.cpu().numpy())

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Adds support for hybrid-agnostic NMS, allowing users to apply class-agnostic non-max suppression (NMS) to specific classes in YOLO models. 🏷️✨

📊 Key Changes

  • Introduced a new agnostic_nms_classes option in the configuration, enabling class-agnostic NMS for selected classes only.
  • Updated the detection prediction logic to handle hybrid-agnostic NMS, combining results from agnostic and standard NMS as needed.
  • Added comprehensive tests to verify standard, agnostic, and hybrid-agnostic NMS behaviors.

🎯 Purpose & Impact

  • Gives users finer control over NMS behavior, improving detection flexibility for challenging or overlapping classes.
  • Helps reduce duplicate detections for specified classes without affecting others, leading to cleaner results.
  • Makes YOLO models more adaptable to diverse use cases, especially where certain objects are prone to overlap.

ohinds added 2 commits June 28, 2024 11:28
Compute agnositc NMS considering a subset of classes, and non-agnostic
NMS for the rest of the classes.

This is useful when some classes should not be colocated, while others
should be.

For example, take a model trained to detect people and several classes
of shirt, where each shirt class is trained on a different shirt color.
In this case, we'd like to detect only a single shirt class per person
by using agnostic NMS on just the shirt classes, while allowing the
person class to be detected with substantial overlap with any shirt
class.

This is accomplished by splitting the predictions during postprocessing,
computing agnostic NMS for the shirts, and non-agnostic NMS for the
person. This approach is implemented in this PR.
@ohinds
Copy link
Author

ohinds commented Jul 3, 2024

I have read the CLA Document and I sign the CLA

@codecov
Copy link

codecov bot commented Jul 3, 2024

Codecov Report

Attention: Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
ultralytics/models/yolo/detect/predict.py 90.90% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@github-actions
Copy link

👋 Hello there! We wanted to let you know that we've decided to close this pull request due to inactivity. We appreciate the effort you put into contributing to our project, but unfortunately, not all contributions are suitable or aligned with our product roadmap.

We hope you understand our decision, and please don't let it discourage you from contributing to open source projects in the future. We value all of our community members and their contributions, and we encourage you to keep exploring new projects and ways to get involved.

For additional resources and information, please see the links below:

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

@github-actions github-actions bot added the Stale Stale and schedule for closing soon label May 21, 2025
@github-actions github-actions bot closed this Jun 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stale Stale and schedule for closing soon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants