We provide 7,444 samples through the PRIDE archiv (PXD042233).

Publication:

Webel, Henry, Yasset Perez-Riverol, Annelaura Bach Nielson, and Simon Rasmussen (2024): Mass spectrometry-based proteomics data from thousands of HeLa control samples. Sci Data 11, 112 https://doi.org/10.1038/s41597-024-02922-z

The projocet folder contains notebooks for data processing of MaxQaunt text output folders which were created using the workflows, processing each file separately.

We provide metadata from the raw files together with the MaxQuant summary statistics of analzying the raw files in pride_metadata.csv. You can read it in python using pandas directly from the PRIDE FTP folder of the project.

import pandas as pd
pd.options.display.max_columns = 80

ftp_folder = 'https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233'
file = 'pride_metadata.csv'

df = pd.read_csv(f'{ftp_folder}/{file}', index_col=0)
df.sample(5, random_state=42).sort_index()

will yield

There are three curated intensity dumps available - for the protein groups level (aggregated using the gene sets) from proteinGroups.txt, the peptide level (aggregate the peptide sequence) from peptides.txt and the precursors (aggregate ions - sequence + charge) from evidence.txt. See a description of the MaxQuant output tables here.

Linked dumps in table format (FTP-Download):

• geneGroups_aggregated.zip [0.2GB] - 7,444 samples, 4,547 selected protein groups
• peptides_aggregated.zip [1.2 GB] - 7,444 samples, 42,723 selected peptides
• precursors_aggregated.zip [1.3 GB] - 7,444 samples, 49,339 selected precursors

The aggregated dumps are best unzipped and a small script is provided in the zipped folder for loading the data.

These were build using cleaned MaxQuant output tables, which are provided here:

• proteinGroups_single_dumps.zip [2.5 GB] - 7,484 files (some duplicates)
• peptides_single_dumps.zip [9.4 GB] - 7,444 files
• precursors_single_dumps.zip [7.1 GB] - 7,444 files

These contain the filtered, but not downsampled data for each samples. You might need the metadata file to filter the data for your analysis.

from pathlib import Path
import pandas as pd
import zipfile

# ARCHIVE = 'peptides_single_dumps.zip'
# ARCHIVE = 'precursors_single_dumps.zip'
ARCHIVE = 'proteinGroups_single_dumps.zip'

ftp_folder = 'https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233'
file = 'pride_metadata.csv'

meta = pd.read_csv(f'{ftp_folder}/{file}', index_col=0)

def len_csv_files_in_archive(archiv_path: str) -> dict:
    """Read csv files in zip and keep their length, i.e. number of features."""
    stats = dict()
    with zipfile.ZipFile(archiv_path, 'r') as zip_archive:
        for fname in zip_archive.namelist():
            # Check if the file is a csv
            if fname.endswith('.csv'):
                # Open each file
                with zip_archive.open(fname) as f:
                    df = pd.read_csv(f)
                    fname = Path(fname).stem
                    stats[fname] = len(df)
    return stats


stats = len_csv_files_in_archive(ARCHIVE)
stats = pd.Series(stats) # contains some duplicates which were removed from aggregated data.
stats = stats.loc[meta.index]
stats

So for the 7,444 selected samples there is a median of 3849 protein groups quantified per sample.

Download a file programaically using python:

import urllib.request
ARCHIVE = 'proteinGroups_single_dumps.zip'

ftp_folder = 'https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233'
file = 'pride_metadata.csv'

urllib.request.urlretrieve(f'{ftp_folder}/{file}', f'{file}')
urllib.request.urlretrieve(f'{ftp_folder}/{ARCHIVE}', f'{ARCHIVE}')

You can also downlaod the zip-archives on the command line using curl (which is available for Windows, MacOS, and Linux distributions):

curl -O https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233/geneGroups_aggregated.zip
curl -O https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233/peptides_aggregated.zip
curl -O https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233/precursors_aggregated.zip
curl -O https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233/proteinGroups_single_dumps.zip
curl -O https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233/peptides_single_dumps.zip
curl -O https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233/precursors_single_dumps.zip

Could be executed from a .bat or .sh script.

import pandas as pd
pd.options.display.max_columns = 80

ftp_folder = 'https://ftp.pride.ebi.ac.uk/pride/data/archive/2023/12/PXD042233'
file = 'Experimental-Design.sdrf.tsv'

df = pd.read_table(f'{ftp_folder}/{file}', index_col=0)
df.sample(5, random_state=42).sort_index()

The project folder contains notebooks for data processing of MaxQaunt text output folders and the analysis of the data for the publication, see its README for details.

The workflows folder in the repository contains snakemake workflows used for rawfile data processing, both for running MaxQuant over a large set of HeLa raw files and ThermoRawFileParser on a list of raw files to extract their meta data.

[!NOTE] The metadata workflow now runs already with the PRIDE data.

Process single raw files using MaxQuant. See README for details.

Read metadata from single raw files using MaxQuant. See README for details.

Create a new environment in case you want to have a separate installation for running

conda create -n hela_data -c conda-forge -c defaults python numpy matplotlib pandas plotly seaborn fastcore omegaconf ipywidgets tqdm pyyaml umap-learn scikit-learn openpyxl xmltodict papermill
# jupyterlab # maybe add a jupyterlab installation (depends on your setup)
conda activate hela_data

and then install the custom code (and dependencies, which should be installed already installed if you use the above environment):

pip install . # . for current directory which should be this repository