The snakemake workflow to conduct a global sensitivity analysis for an OSeMOSYS Model

This repository is used in the paper currently under review at Open Research Europe.

@article{10.12688/openreseurope.15461.1,
    author = {Usher, W and Barnes, T and Moksnes, N and Niet, T},
    doi = {10.12688/openreseurope.15461.1},
    journal = {Open Research Europe},
    number = {30},
    title = {Global sensitivity analysis to enhance the transparency and rigour of energy system optimisation modelling [version 1; peer review: awaiting peer review]},
    volume = {3},
    year = {2023},
    Bdsk-Url-1 = {https://doi.org/10.12688/openreseurope.15461.1}}

jupyter notebook Tutorial.ipynb

jupyter nbconvert Tutorial.ipynb --to slides --post serve

Install snakemake using conda into a new environment called snakemake:

conda install -c conda-forge mamba
mamba create -c bioconda -c conda-forge -n snakemake snakemake-minimal pandas

Then, activate the environment using conda activate snakemake

Now install the other dependencies using pip:

pip install -r requirements.txt

There are template files in the folder resources/templates/ that help the user set up configuration files. Moreover, the Tutorial.ipynb notebook provides practical instructions on how to use the configuration files.

The file config/config.yaml holds workflow configuration options. These options include the location of the model and accompanying data, solver, and sensitivity analysis parameters. Details on how to configure the scenarios.csv, parameter.csv, and results.csv files is given below.

The file config/scenarios.csv is used to point the workflow to master models. Use each master models to define macro scenarios - e.g. forcing in and out a key technology.

Importantly, each of the master models is used as a base for the N replicates defined in `config.yaml. If you define 3 master models in this file, and N=100, then 300 model runs will be scheduled, but with the same 100 parameter values.

An example of a correctly formatted CSV file is shown below.

name,description,csv,config
0,"Interconnector Optimised",config/scenarios/scenario_1/, config/scenarios/scenario_1/config.yaml

The file config/parameters.csv holds information on what parameters to iterate when performing the sensitivity analysis. The information required is summarized below

An example of a correctly formatted CSV file is shown below:

name,group,indexes,min_value_base_year,max_value_base_year,min_value_end_year,max_value_end_year,dist,interpolation_index,action
CapitalCost,capex,"SIMPLICITY,NGCC",500,600,1000,1100,unif,YEAR,interpolate
CapitalCost,capex,"SIMPLICITY,NGOC",400,500,900,1000,unif,YEAR,interpolate
DiscountRate,discountrate,"GLOBAL",0.04,0.05,0.15,0.20,unif,None,fixed

The file config/results.csv holds information on what variables to run the sensitivity analysis over. The information required is summarized below. If a custom OSeMOSYS model is provided with new sets, the additional set can be added as a new column.

An example of a correctly formatted CSV file is shown below. Note that if no index value is provided, the results will sum over all items in that set. For example, for the AnnualEmissions results, the sensitivity analysis will be run on the sum of all annual CO2 emissions in the region SIMPLICITY over the model horizon.

resultfile,filename,REGION,TECHNOLOGY,FUEL,EMISSION,YEAR
TotalDiscountedCost,DiscountedCost,"SIMPLICITY",,,,
NewCapacity,NewCapacityHydro,"SIMPLICITY","HYDRO",,,
NewCapacity,NewCapacityCCNG,"SIMPLICITY","CCNG",,,
AnnualEmissions,Emissions,"SIMPLICITY",,,"CO2",
TotalCapacityAnnual,TotalCapacityAnnualCCNGLastYear,"R1","CCNG",,,2070

To run the workflow, using the command snakemake --use-conda --cores 4 --resources mem_mb=16000 disk_mb=30000

You can also change parts of the configuration by adding the --config flag, followed by the names of one or more of the config items. E.g.

snakemake --cores 4 --config filetype=parquet replicates=100

When snakemake is called, a series of checks is done to validate configuration file options. To skip these checks, add the --config skip_cheks=True flag to the command. E.g.

snakemake --cores 4 --config skip_cheks=True

To visualise the workflow, run the following rule

snakemake plot_dag --cores 2

This repository follows the snakemake guidelines for reproducibility:

├── .gitignore
├── README.md
├── LICENSE.md
├── modelrun
├── workflow
│   ├── rules
|   │   ├── module1.smk
|   │   └── module2.smk
│   ├── envs
|   │   ├── tool1.yaml
|   │   └── tool2.yaml
│   ├── scripts
|   │   ├── script1.py
|   │   └── script2.R
│   ├── notebooks
|   │   ├── notebook1.py.ipynb
|   │   └── notebook2.r.ipynb
│   ├── report
|   │   ├── plot1.rst
|   │   └── plot2.rst
|   └── Snakefile
├── config
│   ├── config.yaml
│   └── some-sheet.csv
├── results
└── resources

To recreate the results shown in the accompanying sensitivity analysis paper, follow the steps below.

1. Pull all data associated with the paper using snakemake. Note, you must pass in the --config skip_checks=True flag to skip pre-run configuration file checks

snakemake setup_paper_data -j6 --config skip_checks=True

This will add a papers/ folder to the root directory that holds all model data, model results, and configuration options. Additionaly, it will populate six different model runs in the config/ folder and resources/ folder. The files in these folders are the configurations used for the paper.

A breakdown of the model naming is shown below:

2. Run each model one at a time. Since we are varrying the factors included in the sensitivity analysis between each model run, we can not run each scenario in parallel.

Copy over the model 1a configuration file

cp config/model_1a/config.yaml config/config.yaml

Run the workflow for model 1a

snakemake --use-conda --cores 4 --resources mem_mb=16000 disk_mb=30000

Repeat this process for models 1b, 2a, 2b, 3a and 3b

NOTE: Running model 3 requires ~30min on a workstation caliber computer

3. Plot visualizations by running each jupyter notebook in workflow/notebooks/. For the capacity, investment, and emission results, you can run the notebooks associated with the downloaded model data in papers/notebooks/.

This research was financially supported by the European Union’s Horizon 2020 research and innovation programme under the grant agreement No 101022622 (European Climate and Energy Modelling Forum ECEMF).

The original version of the computational workflow that was extended for this work was developed by Will Usher under the Climate Compatible Growth programme, which is funded by UK aid from the UK government. The views expressed herein do not necessarily reflect the UK government’s official policies.

Trevor Barnes contribution to this paper was funded via a Mitacs Globalink Research Award IT2569.