kopia snapshot creates 0B size snapshots after a repository move #5123
Description
I am snapshotting 3x directories on 3x VMs (9 snapshots total). A few days ago I performed a repository move from one bucket to another at Hetzner. Ever since the repository move, 1 snapshot of /root on 1x VM is writing a 0B sized snapshot. All other snapshots are working fine.
kopia repository sync-to s3 ... --config-file=old-repo.config --parallel=8
The repository move seemed to work fine, all snapshots got transferred, no issues with the new repository. I verified the consistency of the snapshots BEFORE the move:
kopia snapshot verify --verify-files-percent=100 --log-level=debug --log-file=./verify-snapshot-before.txt --parallel=8
And afterwards - all looks good, no obvious errors, the daily kopia snapshot on 8 other directories works fine..
kopia snapshot verify --verify-files-percent=100 --log-level=debug --log-file=./verify-snapshot-after.txt --parallel=8
It might be just a coincidence, but ever since the repo move, 1x directory (out of the 9 I am snapshotting) creates a 0B sized snapshot...
$ kopia snapshot list /root
..
2026-01-26 22:05:19 CET k422b899df9e03f09ed89f2bbf800ce5c 1.5 MB dr-xr-x--- files:53 dirs:12 (daily-4)
2026-01-27 22:05:45 CET k678a58becbda204d87a7278ff3db7b06 1.5 MB dr-xr-x--- files:58 dirs:12 (daily-3)
<<< repo moved >>>
2026-01-30 22:06:08 CET ke27cc5977c5d56ac26559ca6cb942a86 0 B dr-xr-x--- files:0 dirs:1 (daily-2)
2026-01-31 12:10:02 CET ka13093d030c36deeffdc2c09b2c187eb 0 B dr-xr-x--- files:0 dirs:1 (daily-1,weekly-1,monthly-1,annual-1)
A snapshot debug of /root is here:
2026-01-31T11:10:01.931173Z DEBUG kopia/cli build version v0.22.3, available v
2026-01-31T11:10:01.931476Z DEBUG kopia/cli no updated version available
2026-01-31T11:10:01.931534Z DEBUG passwordpersist password for /root/.config/kopia/repository.config retrieved from password file
2026-01-31T11:10:02.347825Z DEBUG kopia/repo throttling limits from connection info {"limits":{}}
2026-01-31T11:10:02.348762Z DEBUG cache finished initial cache scan {"cache":"contents","duration":"94.763µs","totalRetainedSize":0,"tooRecentBytes":0,"tooRecentCount":0,"maxSizeBytes":5242880000,"limitBytes":0,"inUsePercent":0}
2026-01-31T11:10:02.349437Z DEBUG cache finished initial cache scan {"cache":"metadata","duration":"392.4µs","totalRetainedSize":12291621,"tooRecentBytes":12291621,"tooRecentCount":15,"maxSizeBytes":5242880000,"limitBytes":0,"inUsePercent":0}
2026-01-31T11:10:02.350073Z DEBUG cache finished initial cache scan {"cache":"index-blobs","duration":"406.907µs","totalRetainedSize":9986352,"tooRecentBytes":1778,"tooRecentCount":3,"maxSizeBytes":5242880000,"limitBytes":0,"inUsePercent":0}
2026-01-31T11:10:02.350837Z DEBUG kopia/repo [STORAGE] concurrency level reached {"maxConcurrency":1}
2026-01-31T11:10:02.351717Z DEBUG kopia/repo [STORAGE] concurrency level reached {"maxConcurrency":2}
2026-01-31T11:10:02.352115Z DEBUG kopia/repo [STORAGE] concurrency level reached {"maxConcurrency":3}
2026-01-31T11:10:02.352520Z DEBUG kopia/repo [STORAGE] concurrency level reached {"maxConcurrency":4}
2026-01-31T11:10:02.375262Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xe","resultCount":3,"error":null,"duration":"23.045325ms"}
2026-01-31T11:10:02.380301Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xs","resultCount":31,"error":null,"duration":"28.482976ms"}
2026-01-31T11:10:02.418356Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xw","resultCount":1,"error":null,"duration":"67.36743ms"}
2026-01-31T11:10:02.426456Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xr","resultCount":3,"error":null,"duration":"73.833273ms"}
2026-01-31T11:10:02.461219Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn32_","resultCount":27,"error":null,"duration":"33.707489ms"}
2026-01-31T11:10:02.494303Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn34_","resultCount":0,"error":null,"duration":"67.182622ms"}
2026-01-31T11:10:02.498346Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn33_","resultCount":2,"error":null,"duration":"70.626375ms"}
2026-01-31T11:10:02.565176Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn31_","resultCount":23,"error":null,"duration":"66.410914ms"}
2026-01-31T11:10:02.576353Z INFO kopia/cli Snapshotting root@redacted-limesurvey:/root ...
2026-01-31T11:10:02.577371Z DEBUG uploader uploading {"source":"root@redacted-limesurvey:/root","previousManifests":1,"parallel":2}
2026-01-31T11:10:02.577565Z DEBUG estimate ignored directory {"path":".","dur":"493ns"}
2026-01-31T11:10:02.577636Z DEBUG estimate ignored directory {"path":".","dur":"185ns"}
2026-01-31T11:10:02.577697Z DEBUG estimate ignored directory {"path":".","dur":"193ns"}
2026-01-31T11:10:02.577747Z DEBUG uploader ignored directory {"path":".","dur":"204ns"}
2026-01-31T11:10:02.704358Z DEBUG kopia/repo [STORAGE] PutBlob {"blobID":"s46b28effb5aad3723dde21346f800051-sfc275a2a2047648013d","length":213,"error":null,"duration":"126.088285ms"}
2026-01-31T11:10:02.705148Z DEBUG uploader snapshotted directory {"path":".","dur":"127.669603ms","size":0,"files":0,"dirs":1,"errors":0}
2026-01-31T11:10:02.713638Z DEBUG kopia/snapshot/policy keeping 2025-08-31 22:00:23.904623932 +0200 CEST retention: [monthly-6] pins: []
2026-01-31T11:10:02.713784Z DEBUG kopia/snapshot/policy keeping 2025-09-30 22:05:38.829300923 +0200 CEST retention: [monthly-5] pins: []
2026-01-31T11:10:02.713857Z DEBUG kopia/snapshot/policy keeping 2025-10-31 22:05:34.014954209 +0100 CET retention: [monthly-4] pins: []
2026-01-31T11:10:02.713942Z DEBUG kopia/snapshot/policy keeping 2025-11-29 22:06:00.380191128 +0100 CET retention: [monthly-3] pins: []
2026-01-31T11:10:02.714017Z DEBUG kopia/snapshot/policy keeping 2025-12-31 22:05:24.978769779 +0100 CET retention: [monthly-2,annual-2] pins: []
2026-01-31T11:10:02.714090Z DEBUG kopia/snapshot/policy keeping 2026-01-11 22:05:54.919578592 +0100 CET retention: [weekly-4] pins: []
2026-01-31T11:10:02.714160Z DEBUG kopia/snapshot/policy keeping 2026-01-18 22:05:46.404437761 +0100 CET retention: [weekly-3] pins: []
2026-01-31T11:10:02.714220Z DEBUG kopia/snapshot/policy keeping 2026-01-24 22:05:40.49658672 +0100 CET retention: [daily-6] pins: []
2026-01-31T11:10:02.714262Z DEBUG kopia/snapshot/policy keeping 2026-01-25 22:05:29.607469478 +0100 CET retention: [daily-5,weekly-2] pins: []
2026-01-31T11:10:02.714304Z DEBUG kopia/snapshot/policy keeping 2026-01-26 22:05:19.147751719 +0100 CET retention: [daily-4] pins: []
2026-01-31T11:10:02.714341Z DEBUG kopia/snapshot/policy keeping 2026-01-27 22:05:45.89034298 +0100 CET retention: [daily-3] pins: []
2026-01-31T11:10:02.714378Z DEBUG kopia/snapshot/policy keeping 2026-01-30 22:06:08.814432133 +0100 CET retention: [daily-2] pins: []
2026-01-31T11:10:02.714419Z DEBUG kopia/snapshot/policy deleting 1769857478915095011
2026-01-31T11:10:02.714464Z DEBUG kopia/snapshot/policy keeping 2026-01-31 12:10:02.57745227 +0100 CET retention: [daily-1,weekly-1,monthly-1,annual-1] pins: []
2026-01-31T11:10:02.714572Z INFO kopia/cli Created snapshot with root ka13093d030c36deeffdc2c09b2c187eb and ID e60391d0d26c9fb119d491ae6b6973a9 in 0s
2026-01-31T11:10:03.133368Z DEBUG kopia/repo [STORAGE] PutBlob {"blobID":"qd84f63c10e1088c3ac809cea52f16d66-sfc275a2a2047648013d","length":4331,"error":null,"duration":"418.259048ms"}
2026-01-31T11:10:03.889094Z DEBUG kopia/repo [STORAGE] PutBlob {"blobID":"xn33_c55cae6b788f83a9fd024c1460012ab5-sfc275a2a2047648013d-c1","length":176,"error":null,"duration":"755.379189ms"}
2026-01-31T11:10:04.122422Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xw","resultCount":1,"error":null,"duration":"232.531682ms"}
2026-01-31T11:10:04.206394Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn32_","resultCount":27,"error":null,"duration":"83.416712ms"}
2026-01-31T11:10:04.210305Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn34_","resultCount":0,"error":null,"duration":"87.353929ms"}
2026-01-31T11:10:04.218430Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"xn33_","resultCount":3,"error":null,"duration":"95.136605ms"}
2026-01-31T11:10:04.270064Z DEBUG kopia/repo [STORAGE] DeleteBlob {"blobID":"s46b28effb5aad3723dde21346f800051-sfc275a2a2047648013d","error":null,"duration":"51.377736ms"}
2026-01-31T11:10:26.030736Z DEBUG kopia/repo [STORAGE] GetBlob {"blobID":"qd84f63c10e1088c3ac809cea52f16d66-sfc275a2a2047648013d","offset":0,"length":-1,"outputLength":4331,"error":null,"duration":"21.755708747s"}
2026-01-31T11:10:26.156040Z DEBUG kopia/repo [STORAGE] GetBlob {"blobID":"kopia.maintenance","offset":0,"length":-1,"outputLength":8095,"error":null,"duration":"124.553558ms"}
2026-01-31T11:10:26.156437Z DEBUG maintenance not due for full maintenance cycle until 2026-02-01 12:02:34.043971549 +0100 CET
2026-01-31T11:10:26.156454Z DEBUG maintenance quick maintenance cycle not enabled
2026-01-31T11:10:26.156460Z DEBUG maintenance not due for maintenance
2026-01-31T11:10:26.195804Z DEBUG kopia/repo [STORAGE] Close {"error":null,"duration":"2.113µs"}
I compare the policy against another VM - policies are identical:
Policy for (global):
Retention:
Annual snapshots: 5 (defined for this target)
Monthly snapshots: 12 (defined for this target)
Weekly snapshots: 4 (defined for this target)
Daily snapshots: 7 (defined for this target)
Hourly snapshots: 0 (defined for this target)
Latest snapshots: 0 (defined for this target)
Ignore identical snapshots: false (defined for this target)
Files policy:
Ignore cache directories: true (defined for this target)
Ignore rules: (defined for this target)
**/.cache
**/.git
**/.local/share/Trash
**/.mozilla/firefox/*/cache2
**/.mozilla/firefox/*/startupCache
**/.svn
**/.venv
**/.vscode
Read ignore rules from files: (defined for this target)
.kopiaignore
Scan one filesystem only: false (defined for this target)
Error handling policy:
Ignore file read errors: false (defined for this target)
Ignore directory read errors: false (defined for this target)
Ignore unknown types: true (defined for this target)
Scheduling policy:
Scheduled snapshots:
None.
Manual snapshot: false (defined for this target)
Uploads:
Max parallel snapshots (server/UI): 1 (defined for this target)
Max parallel file reads: - (defined for this target)
Parallel upload above size: 2.1 GB (defined for this target)
Compression disabled.
Metadata compression:
Compressor: zstd-fastest (defined for this target)
Splitter:
Algorithm override: (repository default) (defined for this target)
No actions defined.
OS-level snapshot support:
Volume Shadow Copy: never (defined for this target)
Logging details (0-none, 10-maximum):
Directory snapshotted: 5 (defined for this target)
Directory ignored: 5 (defined for this target)
Entry snapshotted: 0 (defined for this target)
Entry ignored: 5 (defined for this target)
Entry cache hit: 0 (defined for this target)
Entry cache miss: 0 (defined for this target)
I ensure there is no file '.kopiaignore'
# find . -name .kopiaignore -ls
#
/root looks normal:
# stat /root
File: /root
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: fd00h/64768d Inode: 134 Links: 11
Access: (0550/dr-xr-x---) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:object_r:admin_home_t:s0
Access: 2026-01-31 12:36:48.151340567 +0100
Modify: 2026-01-31 12:11:26.836019799 +0100
Change: 2026-01-31 12:11:26.836019799 +0100
Birth: 2024-04-25 18:24:26.563817886 +0200
Chatgpt seems to think it might be '/root' is literally getting translated to '.' somewhere in the path, but I'm not seeing it.
Tried:
- upgrading to kopia v22.3
- nuking the cache