Description
Hey there,
TLDR: is it intended that ignored folders are still checked via lstat when snapshotting/estimating?
I have a folder /mnt/appdata/seafile/fuse which I want Kopia to ignore, since I know it won't be able to access it.
In my policy for /mnt/appdata, I have ignored **/seafile/fuse.
Output of kopia policy get /mnt/appdata:
Policy for root@kopia:/mnt/appdata:
[...]
Files policy:
Ignore cache directories: true inherited from (global)
Ignore rules: (defined for this target)
**/seafile/fuse
**/kopia/cache
**/AI_RecycleBin/**
@*
Read ignore rules from files: inherited from (global)
.kopiaignore
Scan one filesystem only: true inherited from (global)
[...]
When running kopia snapshot estimate --log-level=debug /mnt/appdata, I get the following output:
Analyzing...
[...]
Analyzing seafile...
ERROR Error in seafile: error reading directory: lstat /mnt/appdata/seafile/fuse: transport endpoint is not connected
[...]
Snapshot includes 103007 file(s), total size 13 GB
100 MB... 1 GB: 5 files, total size 737.6 MB
10 MB ...100 MB: 97 files, total size 3 GB
1 MB ... 10 MB: 961 files, total size 2 GB
100 KB... 1 MB: 18568 files, total size 4.7 GB
10 KB ...100 KB: 60530 files, total size 2.5 GB
1 KB ... 10 KB: 19647 files, total size 86.7 MB
0 B ... 1 KB: 3199 files, total size 1.4 MB
Snapshot excludes no files.
Snapshot excludes 3 directories. Examples:
- @Recycle
- @Recently-Snapshot
- kopia/cache
Encountered 1 error(s).
Estimated upload time: 2h53m29s at 10 Mbit/s
(see how kopia/cache was correctly ignored)
Output of kopia snapshot create --log-level=debug --log-dir-detail=10 /mnt/appdata:
[...]
Snapshotting root@kopia:/mnt/appdata ...
DEBUG uploading {"source":"root@kopia:/mnt/appdata","previousManifests":0,"parallel":4}
DEBUG running action before-snapshot-root on /mnt/appdata policy.ActionCommand{Command:"", Arguments:[]string(nil), Script:"/app/scripts/before_snapshot.sh", TimeoutSeconds:300, Mode:""}
DEBUG snapshotted directory {"path":".","error":"error reading directory: lstat /mnt/appdata/seafile/fuse: transport endpoint is not connected","dur":"168.656µs"}
DEBUG [STORAGE] GetBlob {"blobID":"kopia.maintenance","offset":0,"length":-1,"outputLength":3314,"error":null,"duration":"120.772806ms"}
DEBUG not due for full maintenance cycle until 2023-09-20 14:51:13.089553458 +0200 CEST
DEBUG not due for quick maintenance cycle until 2023-09-20 11:51:57.136634372 +0200 CEST
DEBUG not due for maintenance
DEBUG [STORAGE] PutBlob {"blobID":"_log_20230920113749_7a7b_1695202669_1695202671_1_fb9c77af308b6b37276f5eb4278f908d","length":2118,"error":null,"duration":"50.494599ms"}
DEBUG closing active connection.
DEBUG [STORAGE] Close {"error":null,"duration":"11.95757ms"}
ERROR upload error: transport endpoint is not connected
The error is expected, that is why I excluded the directory in the first place.
So why does Kopia still check the directory when I explicitly ignored it?
Is there a way to force Kopia to fully ignore it?
I am aware of the possibility of using --ignore-dir-errors, but that feels more like a workaround and would possibly ignore other, valid, errors, so I would want to avoid using it.
Perhaps I'm misunderstanding how the ignore patterns work, but seeing that the ignore rule for kopia/cache with the exact same ignore pattern worked, makes me think that that's not the root issue. (I also tried with multiple other ignore patterns, fuse, **/fuse/**, seafile/fuse etc.)
Thanks for helping out!