Implement Copy-on-Write (COW) for destructive file operations

[Acook1e]

Actually, this pull request shouldn't be merged now, I just make it a feasible way.
There are many things should be done at mo2 not usvfs:

• Pass the overwrite_dir parameter from mo2
• Pass the mods_dir parameter from mo2
• Pass the name of current_process parameter from mo2
  This Pull request give usvfs an ability to not modify mod source file, possible write will be routed to overwrite_dir

Now this is a true CoW implement.
Truncated operations will redict at NtCreate
Write Access will be redirect at NtWrite

[AnyOldName3]

Is this just doing it for the few file extensions you listed? The reason we've had for not doing the simple thing where we copy files in NtCreateFile when they're opened with write access is that loads of things open files with write access when all they're doing is reading them, so you'd rapidly end up with unmodified copies of everything in overwrite. I guess that might stop being a problem if people start playing games off ReFS instead of NTFS, as then we could create a CoW copy and delete it if the handle's closed and it's identical to the original, but that's not happening in the immediate future.

[Acook1e]

Yep, I have some troubles when developing, I don't know when writing happens. So, I decided just CoW a few extensions.
This is the first time I heard ReFS, I searched it and It's for enterprise and workstation, can we really use it in the future?

The commit is the second time I develop this feature.
The first time I used WriteFile, CloseHandle, SetEndOfFile, SetFilePointerEx, hooks at kernal32, I can capture write but at that time, source file at mods has been 0kb.
If I hook NtWriteFile and using a handle map, can we develop the true CoW?

[AnyOldName3]

ReFS is vaguely positioned to maybe replace NTFS as the default filesystem for Windows, but it's not happening quickly. In 2012, it might have seemed like a safe bet that Windows installs would use ReFS by default in 2025, but 2025 has arrived and that's not the case yet, and Microsoft seem to be pivoting away from making Windows better, so it might never happen.

As for specifics of giving USVFS full CoW support, I'm probably not the best person to ask as I've never dealt with its guts, and just know high-level stuff. I did make a CoW-capable USVFS clone for a university project (actually, technically it was a hook.dll - the thing MO1 used - clone because USVFS didn't work properly back then), but, partially due to design decisions, it was too slow to use.

Something I do know, though, is that one of USVFS' strong points is its speed - because the name lookup is done based on stuff in RAM, it doesn't need to touch the disk until a handle is actually opened, and then the application is given a normal handle that works like any other, so USVFS doesn't need to get involved with anything else, so it ends up about the same speed as the native filesystem. If we start hooking all the functions that can actually modify files, there's a risk of undermining that and making things slower, plus there are a lot of unhooked functions we'd have to hook, so the maintenance burden and risk of mistakes would increase significantly.

To get CoW support, it might be a better idea to make a fresh filesystem using something easier to work with like Dokany, and then add a toggle to MO2 to pick whether to use the fast, non-CoW one (e.g. for playing games) or the slower CoW one (e.g. for using modding tools). Supposedly, when switching from hook.dll to USVFS, Tannin made that kind of thing easier, but I don't think anyone's taken a serious look at it in the last decade or so.

[Acook1e]

After a few times of developing, I'm very sure write operations will be captured by NtWriteFile.
And I also hook NtReadFile to make read rerouted file consistent.
If I use release build, usvfs with Cow is as fast as before.
The only things I need to do now is set the right file pointer for write and read, and other things have been tested on mo2.

[Acook1e]

Finally, implements of CoW correct.
Any truncated operations in mods_dir will be rerouted to overwrite_dir
Any write operations will be captured and set right file pointer and CoW to overwrite_dir, just like the test.txt

logs:

origin file:

CoW file:

[Silarn]

So this is specifically checking for actual file writes before copying the file?

If it's still restricted to specific file extensions I think it could be reasonable, but copying something large like a BSA file still seems problematic from a performance perspective.

Do you have any specific test cases? It might be necessary to allow passing in allowed extensions for cow.

[Acook1e]

This CoW has three features, truncated operations will CoW immediately, write operations happens will CoW and upadte vfs mapping, delete file will only remove mappings (I think I should improve it, but how?)
All CoW only work in mods_dir and overwrite_dir.
And it's simple to check file extensions before cow.
What we need is pass the excluded extensions from settings file (this file is current for test, if mo2 update and adjust params I will update).
And if you still have performance concerns, we can add a max file size restrict.

Test cases I didn't create yet, for there is still a bug I don't know how to solve.
Any commits of usvfs, include mine and latest usvfs release 0.5.7.2, if I build it on my machine, for specific programs, such as DIP.exe (a tool of skyrim modding), will pass short path (Windows 8.3 format, for example: PROGRA1, STEAMA1).
If I use mo2 origin usvfs.dll, short path will not exist.

[Holt59]

Regardless of the actual features that this PR is trying to implement, it contains many unrelated changes (e.g., all the rewrite in the workflow file), so these should be moved to another PR if relevant.

[Acook1e]

Yes, you are right, I should distribute these features into different PR, and I need fork mo2 and adjust usvfs features.
I will close or change my master branch.