refacto

VSainteuf · Aug 9, 2021 · 5e75f03 · 5e75f03
1 parent 46dbae3
commit 5e75f03
Show file tree

Hide file tree

Showing 28 changed files with 4,989 additions and 1 deletion.
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,133 @@
+# Byte-compiled / optimized / DLL files
+todo.txt
+__pycache__/
+*.py[cod]
+*$py.class
+*.swp
+# C extensions
+*.so
+.idea/
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+
+.DS_Store
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
diff --git a/README.md b/README.md
@@ -1 +1,146 @@
-# pastis-utae
+# Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks
+
+![](gfx/predictions.png)
+
+This repository is the official implementation of [Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks
+](https://arxiv.org/abs/2107.07933). 
+
+
+
+## Contents
+This repository contains the following PyTorch code:
+- [Implementation](https://github.com/VSainteuf/utae-paps/blob/main/src/backbones/utae.py) of **U-TAE** spatio-temporal encoding architecture for satellite image time series
+![UTAE](gfx/utae.png)
+- [Implementation](https://github.com/VSainteuf/utae-paps/blob/main/src/panoptic/paps.py) of **Parcels-as-Points (PaPs)** module for panoptic segmentation of agricultural parcels
+![PaPs](gfx/paps.png)
+- Code for reproduction of the paper's results for panoptic and semantic segmentation.
+
+
+
+
+## Results
+
+Our model achieves the following performance on :
+
+### PASTIS - Panoptic Segmentation
+
+| Model name         | SQ  | RQ | PQ|
+| ------------------ |--- | --- |--- |
+| **U-TAE + PaPs** (ours)      | **81.3**|**49.2** |**40.4**|
+| UConvLSTM+PaPs  | 80.9|   40.8   |  33.4|
+
+### PASTIS - Semantic Segmentation
+
+
+| Model name         | #Params| OA  |  mIoU |
+| ------------------ |---- |---- | ---|
+| **U-TAE**  (ours) |   **1.1M**|  **83.2%**    | **63.1%**|
+| Unet-3d   | 1.6M|    81.3%    |  58.4%|
+| Unet-ConvLSTM |1.5M  |     82.1%    |  57.8%|
+| FPN-ConvLSTM  | 1.3M|    81.6%   |  57.1%|
+
+
+
+## Requirements
+
+### PASTIS Dataset download
+The Dataset is freely available for download [here](https://github.com/VSainteuf/pastis-benchmark). 
+
+
+
+### Python requirements
+To install requirements:
+
+```setup
+pip install -r requirements.txt
+```
+
+(`torch_scatter` is required for the panoptic experiments. 
+Installing this library requires a little more effort, see [the official repo](https://github.com/rusty1s/pytorch_scatter))
+
+
+
+## Inference with pre-trained models
+
+### Panoptic segmentation
+
+
+Pre-trained weights of U-TAE+Paps are available [here](https://zenodo.org/record/5172301)
+
+To perform inference of the pre-trained model on the test set of PASTIS run:
+
+```test
+python test_panoptic.py --dataset_folder PATH_TO_DATASET --weight_folder PATH_TO_WEIGHT_FOLDER
+```
+
+
+### Semantic segmentation
+
+
+Pre-trained weights of U-TAE are available [here](https://zenodo.org/record/5172293)
+
+To perform inference of the pre-trained model on the test set of PASTIS run:
+
+```test
+python test_semantic.py --dataset_folder PATH_TO_DATASET --weight_folder PATH_TO_WEIGHT_FOLDER
+```
+
+
+## Training models from scratch
+
+### Panoptic segmentation
+
+To reproduce the main result for panoptic segmentation (with U-TAE+PaPs) run the following :
+
+```train
+python train_panoptic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR
+```
+Options are also provided in `train_panoptic.py` to reproduce the other results of Table 2:
+
+```train
+python train_panoptic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_NoCNN --no_mask_conv
+python train_panoptic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_UConvLSTM --backbone uconvlstm
+python train_panoptic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_shape24 --shape_size 24
+```
+
+Note: By default this script runs the 5 folds of the cross validation, which can be quite long. 
+Use the fold argument to execute one of the 5 folds only 
+(e.g. for the 3rd fold : `python train_panoptic.py --fold 3 --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR`).
+
+### Semantic segmentation
+
+To reproduce results for semantic segmentation (with U-TAE) run the following :
+
+```train
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR
+```
+
+And in order to obtain the results of the competing methods presented in Table 1 :
+
+```train
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_UNET3d --model unet3d
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_UConvLSTM --model uconvlstm
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_FPN --model fpn
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_BUConvLSTM --model buconvlstm
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_COnvGRU --model convgru
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_ConvLSTM --model convlstm
+
+```
+Finally, to reproduce the ablation study presented in Table 1 :
+
+```train
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_MeanAttention --agg_mode att_mean
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_SkipMeanConv --agg_mode mean
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_BatchNorm --encoder_norm batch
+python train_semantic.py --dataset_folder PATH_TO_DATASET --res_dir OUT_DIR_SingleDate --mono_date "08-01-2019"
+
+```
+
+
+
+### Credits
+
+- This work was partly supported by [ASP](https://www.asp-public.fr), the French Payment Agency. 
+
+- Code for the presented methods and dataset is original code by Vivien Sainte Fare Garnot,
+ competing methods and some utility functions were adapted from existing repositories which are credited in the corresponding files.
diff --git a/gfx/paps.png b/gfx/paps.png
diff --git a/gfx/predictions.png b/gfx/predictions.png
diff --git a/gfx/utae.png b/gfx/utae.png
diff --git a/requirements.txt b/requirements.txt
@@ -0,0 +1,6 @@
+torch~=1.5.0
+torchnet~=0.0.4
+torch_scatter
+numpy~=1.18.5
+pandas~=1.0.4
+geopandas~=0.8.1