Description
There are some situations where the atomicity of these operations isn't currently guaranteed. Specifically, there will be a problem if you want to atomically write /path/to/dir/file.txt
but in between opening the temp file and renaming over file.txt
, the directory gets renamed or deleted. This can be particularly problematic on NFS where, under certain circumstances (generally very heavy load), renaming the file can actually cause the directory to be recreated at its original location. You can similarly run into trouble if the directory containing the temporary file (which may or may not be the same as the destination directory) gets renamed.
At least in the case where the temporary file is created in the same directory as the final destination, this can be solved by doing
- Open the destination directory with
O_DIRECTORY|O_PATH
- Use
openat
to open the temporary file using the file descriptor obtained in 1. - Write the file as normal
- Use
renameat
to rename it, again using the file descriptor obtained in 1.
In the case where the temporary file is created in TEMPDIR
rather than the destination directory, it's less clear what to do about the destination directory being renamed, however this technique can at least protect against the directory containing the temporary file getting renamed during the operation, which could otherwise in theory lead to renaming the wrong file.
Linux 3.11+ option
If you're on linux 3.11 or later (which would unfortunately require runtime feature detection1), you can do even better:
- Open the destination directory with
O_DIRECTORY|O_PATH
- Open the destination file using
O_TMPFILE
, which creates an unnamed regular file inode in the destination directory. - Once the file is ready to be exposed, use
linkat
to give the file a name.
If this is available, it has some significant advantages:
- There's no need to manage cleanup state of the temporary file. It gets cleaned up automatically when the file handle closes, even if the program terminates unexpectedly (e.g. power-off, or OOM).
- The file is not available on any filesystem path until it's complete.
Footnotes
-
upcoming versions of go will require kernel 3.17 or 3.10 with urandom support backported. The latter is still in widespread use, and would not support this. ↩
Activity