Hello,

This is when you have target storage setup? or it just source storage that causes these issue?

I would also suggest:

Switch to a PostgreSQL Database:
Label Studio uses SQLite by default, which might not be ideal for large projects with many tasks. SQLite can have limitations with concurrent access and large datasets, potentially leading to data integrity issues during network interruptions. Switching to a PostgreSQL database can provide better performance and reliability.
You can find instructions on how to set up Label Studio with PostgreSQL in our documentation:

https://docs.humansignal.com/guide/storedata.html#PostgreSQL-database

Thank you,
Abu

Comment by Abubakar Saad
Workflow Run

Hi ,
Thanks for getting back to me.
I have an azure blob storage as source.
Here is a snippet from source storage settings:
"Status Completed
Tasks 10734 (0 new)
Last Sync November 15, 2024 ∙ 22:37:38"

I also have azure blob container in the same storage account as target storage.
"Status Completed
Annotations 0 (0 total)
Last Sync November 15, 2024 ∙ 22:33:17"
"
I have not synced it yet after starting annotations.
I also checked the sqlite tables for annotation information. It seems that some of the annotations that are missing have been cleared from the sql lite database. And there are some duplicate entries for some annotated files. All this happened after the network disconnection.
I will try Postgres SQL if it helps. But I am not sure whether switchting to Postgres SQL will solve the issue.

Hello,

The problem may be related to using SQLite as the backend database. SQLite is suitable for small projects and testing purposes but can encounter issues with larger datasets and network instability. Since you're working with over 10,000 tasks, switching to a more robust database like PostgreSQL is highly recommended.

Let us know how that goes.

Thank you,
Abu

Comment by Abubakar Saad
Workflow Run