CopyOnWrite 0.2.1

The CopyOnWrite library provides a .NET layer on top of OS-specific logic that provides copy-on-write linking for files (a.k.a. CoW, file cloning, or reflinking). CoW linking provides the ability to copy a file without actually copying the original file's bytes from one disk location to another. The filesystem is in charge of ensuring that if the original file is modified or deleted, the CoW linked files remain unmodified by lazily copying the original file's bytes into each link. Unlike symlinks or hardlinks, writes to CoW links do not write through to the original file, as the filesystem breaks the link and copies in a lazy fashion. This enables scenarios like file caches where a single copy of a file held in a content-addressable or other store is safely linked to many locations in a filesystem with low I/O overhead.

This library allows a .NET developer to:

• Discover whether CoW links are allowed between two filesystem paths,
• Discover whether CoW links are allowed for a directory tree based at a specific root directory,
• Create CoW links,
• Find filesystem CoW link limits.

Discovery is important, as different operating systems and different filesystems available for those operating systems provide varying levels of CoW link support:

• Windows: The default NTFS filesystem does NOT support CoW, but the ReFS filesystem does.
• Linux: Btrfs, Xfs, Zfs support CoW while ext4 does not.
• Mac: AppleFS supports CoW by default.

When using this library you may need to create a wrapper that copies the file if CoW is not available.

Example

using Microsoft.CopyOnWrite;

ICopyOnWriteFilesystem cow = CopyOnWriteFilesystemFactory.GetInstance();
bool canCloneInCurrentDirectory = cow.CopyOnWriteLinkSupportedInDirectoryTree(Environment.CurrentDirectory);
if (canCloneInCurrentDirectory)
{
    cow.CloneFile(existingFile, cowLinkFilePath);
}

Windows: Workarounds for ReFS parallel cloning bug

As of September 2022, in Server 2022 and Windows 10 and 11 when cloning a single recently written source file in parallel the resulting cloned file can sometimes end up with all zeroes for its content. This comes from a bug in ReFS; see https://github.com/microsoft/CopyOnWrite/issues/12 for more details. This library provides workarounds for this problem, or you can change how source files are written to ensure they are flushed to disk before performing parallel clones. The workarounds include:

• Serialize cloning system-wide per source path. You could accomplish this by ensuring only a single thread can clone a single source file at a time. This library takes this approach by default for single-process cloning by using an in-memory dictionary. You can opt into system-wide serialization using kernel mutexes by specifying useCrossProcessLocksWhereApplicable = true when calling CopyOnWriteFilesystemFactory.GetInstance().
• Ensure the source file is completely flushed to disk before cloning. This can be accomplished through one of the approaches below. Note that if you use these approaches, you can increase performance of cloning by using CloneFileFlags.NoSerializedCloning on your CloneFile calls.
  • Use the FlushFileBuffers API to force the file to be flushed from memory. This should be called at the end of writing the source file to disk while the file write handle is still open. Alternately it could be called on a new handle to the file opened with GENERIC_WRITE.
  • When writing the source file, open the file handle with FILE_FLAG_NO_BUFFERING. However, note this requires the code writing to the file to deal with writing chunks aligned with the sector size of the underlying volume, and using chunks that are a multiple of the sector size.
  • When writing the source file, open the file handle with FILE_FLAG_WRITE_THROUGH. This forces a flush on every write, which can decrease performance significantly.

Release History

NuGet package:

• 0.2.1 September 2022: Add detection for DOS SUBST drives as additional source of mappings.
• 0.2.0 September 2022: Improve documentation for ReFS parallel cloning bug workarounds. Improve Windows cloning performance by 7.2% by using sparse destination files. Default behavior change to leave destination file sparse and replaced CloneFlags.NoSparseFileCheck with DestinationMustMatchSourceSparseness, hence minor version increase.
• 0.1.13 September 2022: Fix CloneFlags to use individual bits.
• 0.1.12 September 2022: Add new factory flag that sets a mode to require cross-process Windows mutexes for safe source file locking to avoid a ReFS concurrency bug. Add optimization to allow bypassing redundant Path.GetFullPath() when caller has done it already.
• 0.1.11 September 2022: Serialize Windows cloning on source path to work around ReFS limitation in multithreaded cloning.
• 0.1.10 September 2022: Fix missing destination file failure detection.
• 0.1.9 September 2022: Add explicit cache invalidation call to interface. Update Windows implementation to detect ReFS mount points that are not drive roots, e.g. mounting D:\ (ReFS volume) under C:\ReFS.
• 0.1.8 April 2022: Add overload for CoW clone to allow bypassing some Windows filesystem feature checks
• 0.1.7 April 2022: Perf improvement for Windows CoW link creation by reducing kernel round-trips
• 0.1.6 April 2022: Perf improvement for all Windows APIs
• 0.1.5 October 2021: Separate exception type for when link limit is exceeded. Mac and Linux throw NotSupportedException.
• 0.1.4 October 2021: Fix doc XML naming. Mac and Linux throw NotSupportedException.
• 0.1.3 October 2021: Bug fixes for Windows. Mac and Linux throw NotSupportedException.
• 0.1.2 October 2021: Performance fixes for Windows. Mac and Linux throw NotSupportedException.
• 0.1.1 October 2021: Bug fixes for Windows. Mac and Linux throw NotSupportedException.
• 0.1.0 July 2021: Windows ReFS support. Mac and Linux throw NotSupportedException.

Related Works

• Rust CoW: https://github.com/nicokoch/reflink

Contributing

This project welcomes contributions and suggestions. See CONTRIBUTING.md.

Running Unit Tests on Windows

If you have a local ReFS drive volume on which to run ReFS related tests, set the following user or system level environment variable:

CoW_Test_ReFS_Drive=D:\

(You may need to exit and restart VS, VSCode, or consoles after setting this.) When this env var is not available, unit tests create and mount a local ReFS VHD for testing. You must run tests elevated (as admin), e.g. by opening Visual Studio as an admin before opening the solution.

Performance Comparisons

Windows

CoW links on ReFS take approximately constant time, saving time over file copies except at file size zero. The savings is proportional to the file size, with 16MB files at about 35X performance, 1MB at 3.2X, and small sizes at about 1.3X.

Detailed numbers for a VHD formatted empty with ReFS for each iteration, comparing System.IO.File.Copy() with CloneFile(), 50 copies/clones of a single source file per measurement, highest performance clone flags and settings. See CoWComparisons.cs. Machine was an 8/16-core, NVMe, Win11 22H1 Enterprise.

Same benchmark performed on a ReFS partition (no VHD) on the NVMe disk: