After upgrading to version 2.0.4.22, USN on Windows 10 may not work for me. I use --usn-policy=On and the log says:
2019-08-07 07:49:46 +02 - [Information-Duplicati.Library.Main.Operation.BackupHandler-SkipUsnForVolume]: Performing a full scan for volume "C: \",
but the backup itself ends up OK.
I use Duplicati on Windows Server 2016 too and there is no entry in the log. It only seems to be a problem on Windows 10, I have it on two computers.
On Windows 10 Version 1809 I can only get this on the initial scan. I thought it would pick up where it left off, but I’m a little suspicious about its record-keeping on where it left off. Note duplicated NextUsn below:
(view done with DB Browser for SQLite)
An opinion from Windows on what the next USN should be can be obtained from administrator doing this:
C:\>fsutil usn queryJournal C:
Usn Journal ID : 0x01d1cc97428f387f
First Usn : 0x0000000fa7000000
Next Usn : 0x0000000fa9036888
Lowest Valid Usn : 0x0000000000000000
Max Usn : 0x7fffffffffff0000
Maximum Size : 0x0000000002000000
Allocation Delta : 0x0000000000800000
Minimum record version supported : 2
Maximum record version supported : 4
Write range tracking: Disabled
C:\>
Real-time backup was developer announcement, and links to the pull request with code and discussion:
Implemented backup based on changes recorded in NTFS USN journal #3184
I did try testing how to reintroduce the “full scan”, and found I could reset it to the first-time full scan case using Advanced options to turn –usn-policy=off, do a backup, then turn --usn-policy=required again, but I only get one “full scan” from this, then presumably it’s going to start from NextUsn from previous backup.
The USN entry looks fine in the DB. The next USN you obtain from fsutil is slightly above the one recorded in the DB, which makes sense.
There are a few reasons why USN cannot be used despite the policy being enabled:
- any configuration change in Duplicati triggers a full scan
- the USN journal may be too small (the intervals between backups too large). If entries are missing, a full scan is triggered.
Thanks so much for the feature and support @Daniel_Gehriger
I think I didn’t do enough backups in my test to stop worrying about the duplicated NextUsn, but it seems like the plan is for a new fileset to not only set a new NextUsn in its row, but to update the previous row’s:
ID FilesetID VolumeName JournalID NextUsn ConfigHash
6 594 C:\ 131110814199265407 67259287064 783E587246E3188CCF7092EB931721AC
7 595 C:\ 131110814199265407 67270409200 783E587246E3188CCF7092EB931721AC
8 596 C:\ 131110814199265407 67270409200 783E587246E3188CCF7092EB931721AC
ID FilesetID VolumeName JournalID NextUsn ConfigHash
6 594 C:\ 131110814199265407 67259287064 783E587246E3188CCF7092EB931721AC
7 595 C:\ 131110814199265407 67270409200 783E587246E3188CCF7092EB931721AC
8 596 C:\ 131110814199265407 67270568752 783E587246E3188CCF7092EB931721AC
9 597 C:\ 131110814199265407 67270568752 783E587246E3188CCF7092EB931721AC
Fsutil usn “Using createjournal” section talks about how to expand it. I haven’t tried that myself though…