ext4 and data consistency

Posted May 14, 2010 17:33 UTC (Fri) by njs (guest, #40338)
In reply to: ext4 and data consistency by anton
Parent article: The Next3 filesystem

I'm not aware of any common filesystem that provides "data consistency" in any coherent sense, unless you do weird things like mount -o sync. Speed is too important -- Stephen Tweedie didn't make data=journal the default, either. At most you get guarantees in particular situations -- e.g., both ext3 and ext4 guarantee that a rename will not be committed to disk until writes to the file being renamed have been committed to disk. They even both try to guarantee that programmers who do horrible things like truncating the file and *then* rewriting it are somewhat protected from their incompetence.

But maybe there are other cases where ext3 does better than ext4. You must have some excellent ones in mind to lump ext4 in with tmpfs... can you give any examples?

ext4 and data consistency

Posted May 15, 2010 9:19 UTC (Sat) by anton (subscriber, #25547) [Link] (1 responses)

Speed is too important

For whom? For me data consistency is much more important. Before barriers were supported, we ran ext3 on IDE disks without write caching, and that's really slow. The file system was still fast enough.

Stephen Tweedie didn't make data=journal the default, either.

Actually he did, at least at the start. Later it got changed (by whom?) to data=ordered; that still has the potential to provide data consistency unless existing files are overwritten.

As for an example: Consider a process writing file A and then file B. With ext4 I expect that it can happen that after recovery B is present and A is not or is empty. With ext3 I expect that this does not happen. But given that I did not find any documented guarantees in Documentation/filesystems/ext3.fs, maybe we should lump ext3 with tmpfs, too.

Still, my search brought up a Linux file system that gives guarantees: In nilfs2.txt it says:

order=strict	Apply strict in-order semantics that preserves sequence
		of all file operations including overwriting of data
		blocks.  That means, it is guaranteed that no
		overtaking of events occurs in the recovered file
		system after a crash.

Yes, that's exactly the guarantee I want to see. This means that any application that keeps its files consistent as visible from other processes will also have consistent files after an OS crash.

ext4 and data consistency

Posted May 16, 2010 3:57 UTC (Sun) by njs (guest, #40338) [Link]

> For whom? For me data consistency is much more important

That's fine. I'd like data consistency too. But I still don't mount my disks with -o sync, nor does pretty much anyone else, even most of the people who say they want data consistency. That's the reality that fs developers live in.

Maybe on SSD (where nilfs2 is designed to live), we'll be able to get guaranteed data consistency as a matter of course. That'll be nice if it happens.