Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repair performance on very large repositories #4242

Open
aneagoe opened this issue Nov 14, 2024 · 1 comment
Open

repair performance on very large repositories #4242

aneagoe opened this issue Nov 14, 2024 · 1 comment

Comments

@aneagoe
Copy link

aneagoe commented Nov 14, 2024

I started piloting kopia as a replacement for our borgbackup environment.
We have a fairly large CephFS (about 1TB daily incrementals). I created a kopia repository server instance on a remote (25ms RTT) machine with 8 x 22TB drives configured in a ZFS RAIDz. The initial backup took about a week to complete and then incrementals were decently fast (under 10 hours).
Following similar maintenance issues as in #3563, I decided to run kopia snapshot fix invalid-files --verify-files-percent=100 --commit. It's now running since 6 days and backups are blocked. The current repository size on disk is 100T.
The main questions are how are issues like this normally supposed to be handled on very large repositories? If we go live with this and later run into an issue that requires running snapshot fix or other repository maintenance jobs, how can downtime to the entire backup system be avoided? Any tips on how to handle/manage very large repos?

@TristisOris
Copy link

we have 300Tb+ and it again begin working after v0.18.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants