List of the datasets:
- Oxford IIIT Pet (https://www.robots.ox.ac.uk/~vgg/data/pets/)
- Cat Dataset (https://www.microsoft.com/en-us/research/wp-content/uploads/2008/10/ECCV_CAT_PROC.pdf)
- Petfinder cats (https://zenodo.org/record/6656292#.Yq66DHZBwuU)
- Petfinder dogs (https://zenodo.org/record/6660349#.Yq8TJHZBwuU)
- Labelled data from Kashtanka.pet (https://zenodo.org/record/6664769#.Yq8GuXZBwuU)
python download_datasets.py
The script downloads all the needed datasets to the directory ../pets_datasets
To download the checkpoints and configs to use them run:
python download_models.py
Body detection and segmentation
| Dataset | AP50 | AP70 | IoU detection | IoU segmentation |
|---|---|---|---|---|
| Oxford IIIT Pets | 0.999 | 0.999 | 0.975 | 0.946 |
| Labelled kashtanka.pet dogs | 0.966 | 0.916 | 0.866 | N/A |
| Labelled kashtanka.pet cats | 0.979 | 0.952 | 0.836 | N/A |
Head and landmarks detection
| Dataset | AP50 | AP70 | IoU | NME | NME (Median) | NME percentile 0.25 | NME percentile 0.75 |
|---|---|---|---|---|---|---|---|
| Cat Dataset | 0.999 | 0.988 | 0.909 | 0.044 | - | - | - |
| Labelled kashtanka.pet dogs | 0.999 | 0.715 | 0.774 | 0.141 | 0.057 | 0.036 | 0.088 |
| Labelled kashtanka.pet cats | 0.975 | 0.869 | 0.866 | 0.277 | 0.061 | 0.037 | 0.094 |
python main_detection.py --config configs/to_reproduce/mask/mask_rcnn_config.py
python main_keypoints.py --config configs/to_reproduce/keypoint/keypoints_config.py
If you want to test your own models you need to create a script analogous to eval_detection.py.
python eval_detection.py
If you want to test your own models you need to create a script analogous to eval_landmark.py.
python eval_landmark.py
prepare_table.py runs Mask R-CNN trained on Oxford IIIT pets to predict body bounding boxes, and Keypoint R-CNN trained on Cat Dataset + 350 manually selected examples of dogs from kashtanka.pet with good annotations from the previous model to predict head bounding boxes and landmarks. If you want to test your own models you need to create a script analogous to prepare_tables.py to create tables with predictions.
python prepare_tables.py to get 3 .tsv files for the assessment
To test Head detection use:
python score_detection.py detected_head.tsv data_25 Head
To test Body detection use:
python score_detection.py detected_body.tsv data_25 Animal
For landmark detection evaluation use:
python score_landmark.py landmark.tsv data_25
TODO
Results on data_25 val part for FE
| Model | ROC AUC | Accuracy | candR@10 | candR@100 |
|---|---|---|---|---|
| Dog Head SGD | 0.973 | 0.938 | 0.777 | 0.911 |
| Dog Head AdamW | 0.975 | 0.94 | 0.733 | 0.906 |
| Cat Head SGD | 0.958 | 0.915 | 0.653 | 0.904 |
| Cat Head AdamW | 0.97 | 0.922 | 0.753 | 0.93 |
| Dog Body SGD | 0.974 | 0.926 | 0.636 | 0.864 |
| Dog Body AdamW | 0.878 | 0.8 | 0.348 | 0.56 |
| Cat Body SGD | 0.968 | 0.917 | 0.538 | 0.811 |
| Cat Body AdamW | 0.965 | 0.91 | 0.545 | 0.809 |
Results of the pipelines (detector + FE) on kashtanka.pet public test
| Pipeline | candR@10 lost hard | candR@100 lost hard | candR@10 lost simple | candR@100 lost simple |
|---|---|---|---|---|
| Head-based | 0.386 | 0.569 | 0.52 | 0.632 |
| Ensemble | 0.395 | 0.604 | 0.583 | 0.735 |
transform_reproduce.py runs head detection and alignment model and body detection and segmentation model on petfinder data and kashtanka_25
python transform_reproduce.py
Head-specific model (Cat Head SGD)
python main.py --config configs/to_reproduce/cat_fe/cat_fe_head.py
Body-specific model (Cat Body AdamW)
python main.py --config configs/to_reproduce/cat_fe/body_cat_fe.py
Head-specific model (Dog Head SGD)
python main.py --config configs/to_reproduce/dog_fe/fe_dogs_config.py
Body-specific model (Dog Body SGD)
python main.py --config configs/to_reproduce/dog_fe/body_dog_fe.py
To validate Head-specific model (Dog Head SGD)
python eval_fe_dog_head_sgd.py
To validate Head-specific model (Cat Head SGD)
python eval_fe_cat_head_sgd.py
Combination:
python generate_tsv_to_reproduce1.py
Only Face-based:
python generate_tsv_to_reproduce2.py
The scripts produce pred_scores_test1.tsv and pred_scores_test2.tsv correspondingly. The files then should be submitted using file sent to you after your registration on http://92.63.96.33/c/_lostpets_v3_1/description. Pay attention you can modify the scripts and provide your own checkpoints