Add DB.NoTruncate flag. #363
Conversation
| @@ -55,6 +55,12 @@ type DB struct { | |||
| // THIS IS UNSAFE. PLEASE USE WITH CAUTION. | |||
| NoSync bool | |||
|
|
|||
| // When true, skips the truncate call when resizing the database. | |||
| // Setting this to true is only safe on non-ext3/ext4 systems. | |||
| // | |||
There was a problem hiding this comment.
Would it be worth adding a note here the we found that Truncate can be very slow on some filesystems?
There was a problem hiding this comment.
I wasn't able to determine the root cause. Truncation did work very quickly in some cases but in others it was taking >10m. It may have been a configuration issue or a hardware issue so I'd rather not assign blame to the FS.
Also, NoTruncate is nice if you don't want to preallocate space so I'll list that as a benefit.
|
Can we call it something else than NoTruncate? First of all, the word truncate implies shortening, though it's the syscall used to adjust file size either way; here, it's used for growing. And the cost really comes from the sync. Since technically it is, or might be depending on how to interpret the POSIX bible, ext3/ext4 etc that is at fault, I am not going to recommend |
|
I agree that I have two issues with naming it
How about |
|
|
|
I think |
|
Because it's not really about mmap, either. Off hand, I think it even triggers on the first mmap, so it's really not a remap. The purpose of that code is to sync the new, larger, size of the file to disk. |
This commit adds the DB.NoGrowSync flag to optionally revert mmap() calls to how they were implemented before the ext3/ext4 fix. When NoGrowSync is true, remapping the data file will not force the file system to resize it immediately. This works for non-ext3/4 file systems. The default value of NoGrowSync is false so it is still safe for ext3/ext4 file systems by default. See also: boltdb#284
|
Changed to |
Overview
This commit adds the
DB.NoTruncateflag to optionally revertmmap()calls to how they were implemented before the ext3/ext4 fix. WhenNoTruncateis true, remapping the data file will not force the file system to resize it immediately. This works for non-ext3/4 file systems.The default value of
NoTruncateisfalseso it is still safe for ext3/ext4 file systems by default.See also: #284
/cc @tv42 @mkobetic @bouk