Skip to content

bnbsking/COSMOduleS

BranchesTags

Repository files navigation

COSMOduleS: Classification, Object detection, Segmentation MOduleS



Github: source

Introduction

This repo provides comprehensive preprocessing and post-processing tools for common Computer Vision tasks.

Tasks Subtasks Defined
Format		 Visualization Format
Conversion		 Output
Analysis		 Label
Merging		 Active
Learning
Classification binary1
binary-bg2
multi-class1
multiclass-bg2
multi-binary3
 single_label1
single_label_bg2
multi_label3		 - - metrics
plotting
export		 ALL Entropy
Detection - coco
voc
yolo
GENERAL
 ALL between ANY
two types		 metrics
plotting
export		 V horizontal
flip
Segmentation instance1
semantic2
 coco1+2
GENERAL1+2		 ALL coco2general metrics
plotting
export		 - instance
semantic
  • "bg" means background. If there is background class, it must be class 0 in this repo.
  • Adding prediction results after the defined format can use the visualization and output analysis. All the formats with predictions are in example/*/prediction, e.g. here.

Motivation

  • [Classification] Complicated tasks
task label idx min compute class-0 metrics threshold optimization data format
binary classification 0 V V single_label
binary classification (cls-0 background) 1 V single_label_background
multi-class classification 0 V single_label
multi-class classification (cls-0 background) 1 V single_label_background
multi-label classification (cls-0 background) 0 V multi_label

  • [Classification] threshold optimization

    • multi-class classification (cls-0 background) checks whether prob-cls-0 < threshold, if yes, the pd-cls is pd[1:].argmax()
    • multi-class classification (cls-0 background) and multi-label classification (cls-0 background) take the mean of all optimized threshold for each foreground class

  • [Object Detection] Develop a GENERAL format to be the most convenient.

The formats can be summarized as following:

format extension files type box disadvantage
coco .json 1 int (xmin, ymin, w, h) get label of an image
yolo .txt len(imgs) float (cx, cy, w/2, h/2) visualization, compute metrics, etc.
voc .xml len(imgs) int (xmin, ymin, xmax, ymax) get class list
general .json 1 int (xmin, ymin, xmax, ymax) NO
  • [Segmentation] Develop a GENERAL format to be the most convenient.
Includes Content Advantage
general.json Includes every imgs: path, contour, filled and boxes with class Searching
gt_contour_*.npy (H, W) with {0, 1, ..., num_classes} int Plotting
gt_filled_*.npy (num_classes, H, W) with 0 or 1 int values Compute IOU for Metrics
*.jpg Raw data -
  • Segmentation prediction format: (num_classes, H, W) with 0~1 float values (probability). e.g. here

Installation

  • Docker image as in test.sh or test.ps1

  • Container for testing

bash env.sh
source ~/.bashrc
  • Directly use
pip install -e .

Quick Start - Classification

from cosmodules.classification import ClassificationAnalysis

ClassificationAnalysis(
    ant_path = "example/classification/data/single_label.json",
    save_folder = "example/classification/output/single_label",
)
  • Label Merging:
from cosmodules.classification import ClassificationLabelMerging

ClassificationLabelMerging(
    cfg_path_list = [
        "example/classification/data/single_label.json",
        "example/classification/data_another_labeler/single_label.json",
    ],
    save_path = f"example/classification/output/label_merging/single_label.json"
)
  • Active Learning (see more in the example):
from cosmodules.classification import ClassificationActiveLearning

ClassificationActiveLearning(
    pred_path = "example/classification/prediction/single_label.json",
    save_path = "example/classification/output/active_learning/single_label.json",
    loss_name = "entropy"
)

Quick Start - Object detection

  • Format Conversion (see more in the example)
from cosmodules.detection import coco2any

coco2any(
    tgt_foramt = "voc",
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json",
    save_folder = "example/detection/output/visualization_gt_conversion/coco2voc"
)

or

from cosmodules.detection import coco2general

coco2general(
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json",
    save_path = "example/detection/output/visualization_gt_conversion/coco2general/general.json"
)
  • Visualization (see more in the example)
from cosmodules.detection import show_coco

show_coco(
    img_name = "pic0.jpg",
    img_folder = "example/detection/data/coco",
    ant_path = "example/detection/data/coco/coco.json"
)

or

from cosmodules.detection import show_general

show_general(
    img_name = "pic0.jpg",
    ant_path = "example/detection/data/general.json",
)  # when the anntotation includes predictions it will be shown!
  • Output Analysis
    • Please use the above Format conversion to change data format as general
    • The analysis pipeline is at here
from cosmodules.detection import DetectionAnalysis

DetectionAnalysis(
    ant_path = "example/detection/data/general.json",
    save_folder = "example/detection/output/metrics"
)
  • Label Merging:
from cosmodules.detection import DetectionLabelMerging

DetectionLabelMerging(
    cfg_path_list = [
        "example/detection/data/general.json",
        "example/detection/data_another_labeler/general.json",
    ],
    save_path = "example/detection/output/label_merging/general.json",
    ties_handling = "union"
)
  • Active Learning:
from cosmodules.detection import DetectionActiveLearningByHFlip

DetectionActiveLearningByHFlip(
    pred_path_1 = f"{ROOT}/example/detection/prediction/general.json",
    pred_path_2 = f"{ROOT}/example/detection/prediction/general_horizontal_flip.json",
    save_path = f"{ROOT}/example/detection/output/active_learning/general.json"
)

Quick Start - Segmentation

  • Format Conversion (see more in the example)
from cosmodules.segmentation import coco2general

coco2general(
    img_folder = "example/segmentation/data/coco",
    ant_path = "example/segmentation/data/coco/coco.json",
    save_folder = f"example/segmentation/output/visualization_gt_conversion/coco2general"
)
  • Visualization (see more in the example)
from cosmodules.segmentation import show_coco

show_coco(
    img_name = "img1.jpg",
    img_folder = "example/segmentation/data/coco",
    ant_path = "example/segmentation/data/coco/coco.json"
)   # when the anntotation includes predictions it will be shown!

or

from cosmodules.segmentation import show_general

show_general(
    img_name = "img1.jpg",
    ant_path = "example/segmentation/data/general/general.json"
)
  • Output Analysis
    • Please use the above Format conversion to change data format as general
    • The analysis pipeline is at here
from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(
    ant_path = "example/segmentation/prediction/instance/general.json",
    save_folder = "example/segmentation/output/metrics/instance",
    task = "instance",
)

or

from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(
    ant_path = "example/segmentation/prediction/semantic/general.json",
    save_folder = "example/segmentation/output/metrics/semantic",
    task = "semantic"
)
  • Active Learning:
from cosmodules.segmentation import (
    InstanceSegmentationActiveLearningByHFlip,
    SemanticSegmentationActiveLearning
)

InstanceSegmentationActiveLearningByHFlip(
    pred_path_1 = "example/segmentation/prediction/instance/general.json",
    pred_path_2 = "example/segmentation/prediction/instance_horizontal_flip/general.json",
    save_path = "example/segmentation/output/active_learning/instance.json"
)

or

SemanticSegmentationActiveLearning(
    pred_path = "example/segmentation/prediction/semantic/general.json",
    save_path = "example/segmentation/output/active_learning/semantic.json",
    loss_name = "entropy"
)

Examples

  • [detection]: format conversion workflow .

  • detection visualization .

  • confusion .

  • prf curves .

More

  • Feel free to ask if you have any question.
  • Notice not supported
    • segmentation general2coco
    • segmentation label merging

Updates

  • v1.1.0:
    • Add docker environment
    • Improve classification and detection metrics pipeline
      • template design pattern
      • hybrid approach of
        • stateless (get / staticmethod / abstract method)
        • stateful (_set / instance method)
      • cleaner architecture of
        • pr curve
        • confusion matrices
        • threshold optimization
      • add unit tests 
from cosmodules.utils.flow.metrics.classification_metrics import ClassificationMetricsFlow

class TestClassificationMetricsFlow:
    def test_run_single_label(self):
        obj = ClassificationMetricsFlow(
            num_classes = 2,
            labels = np.array([0, 0, 1]),
            predictions = np.array([
                [0.95, 0.05],
                [0.1, 0.9],
                [0.2, 0.8]
            ]),
            save_path = "/app/example/classification/output/metrics_new/single_label/metrics.json",
        )
        results = obj.run()
        print("Results:", results)

if __name__ == "__main__":
    test = TestClassificationMetricsFlow()
    test.test_run_single_label()

or

import numpy as np

from cosmodules.utils.flow.metrics.detection_metrics import DetectionMetricsFlow


class TestDetectionMetricsFlow:
    def test_run(self):
        obj = DetectionMetricsFlow(
            num_classes = 3,
            labels = [
                np.array([[1, 0, 23, 220, 228]]),
                np.array([[1, 8, 15, 715, 630], [2, 733, 64, 1174, 588]]),
                np.array([[1, 35, 7, 112, 114], [1, 171, 58, 263, 121], [2, 114, 39, 170, 118], [2, 259, 62, 326, 118]]),
            ],
            predictions = [
                np.array([
                    [0, 23, 220, 228, 0.95, 1],
                    [30, 90, 80, 150, 0.8, 2],
                ]),
                np.array([
                    [8, 15, 715, 630, 0.85, 1],
                    [733, 64, 1174, 588, 0.6, 1],
                ]),
                np.array([
                    [35, 7, 112, 114, 0.75, 1],
                    [171, 58, 211, 88, 0.45, 1],
                    [114, 39, 170, 118, 0.99, 2],
                ]),
            ],
            save_path = "/app/example/detection/output/metrics_new/metrics.json",
        )
        results = obj.run()
        print("Results:", results)


if __name__ == "__main__":
    test = TestDetectionMetricsFlow()
    test.test_run()

Acknowledgement

  • Confusion Matrix reference here

About

Classification, Object detection, Segmentation MOduleS. Such as automatic format conversion, visualization, metrics with plotting and exporting suspected data.

Topics

classification segmentation object-detection visualization-tools metrics-exporter conversion-tool

Resources

Readme

License

MIT license
Activity

Stars

2 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

1 tags

Packages

 
 
 

Contributors

Languages