Duplicati missing a whole folder

TLuckh · March 20, 2024, 9:02pm

For the folder "C:\Users\Sanitiy\HESSENBOX\#Aktuelle Daten" I have setup a backup.

In it there’s a sub-directory “C:\Users\Sanitiy\HESSENBOX\#Aktuelle Daten\Latex-Scripts und Zusammenfassungen”,
which for some reason is missing in the backups.

The advanced options of the backup script are:

–backup-name=Aktuelle Daten v2
–dbpath=C:\Users\Sanitiy\AppData\Local\Duplicati\SGYAOUTVPV.sqlite
–encryption-module=aes
–compression-module=zip
–dblock-size=50mb
–exclude-files-attributes=hidden,system,temporary
–disable-module=console-password-input
–exclude=.git
–exclude=.tmp
–exclude=.xml~
–exclude=\desktop.ini
–exclude=.partial
–exclude=.part
–exclude=*.trashes

As far as I can tell it’s neither hidden,system,temporary, and it also doesn’t match any of the filters.

For what reason could it be missing?

Regards,
Sanitiy

ts678 · March 20, 2024, 10:10pm

How do you check? The attrib command or adding Explorer Attributes column are good.

EDIT 1:

Actually it looks like attrib and dir (at least as in the /? help) know only hidden and system.

ts678 · March 20, 2024, 10:31pm

Alternatively (but maybe harder), you can make a log file at verbose level to see why it’s skipped, however depending on backup file count this either makes a big file, or doing a scaled-down test.

github.com

duplicati/duplicati/blob/f49b3960238f43c56516cff49f53334503f43fa2/Duplicati/Library/Main/Operation/Backup/FileEnumerationProcess.cs#L331-L335


      
          // If we exclude files based on attributes, filter that

          if ((fileAttributes & attributes) != 0)

          {

              Logging.Log.WriteVerboseMessage(FILTER_LOGTAG, "ExcludingPathFromAttributes", "Excluding path due to attribute filter: {0}", path);

              return false;

If it’s not an attribute, I think verbose level makes other messages about source paths it handles.

TLuckh · March 21, 2024, 2:03am

@ts678 You were right - it is actually marked as System folder. Though I have no idea how this could have happened - you can’t even set that attribute in the property dialog of windows, so I have no idea how this might have happened.

Thank you for the help!

ts678 · March 21, 2024, 10:59am

I don’t know your software, so don’t know if folder is considered “owned” by that app or you, but typical Windows application has a lot of ability to set attributes the way it sees fit, e.g. by use of:

File Attribute Constants

Duplicati, as a C# application, reads them out with:

FileAttributes Enum

We sometimes see applications choose file attributes that the user wasn’t expecting. I’m not sure what can be done about it. A different area that could be improved would be that the GUI invites:

and it’s not obvious that these are attributes. At least CLI exclude-files-attributes makes it clear:

Use this option to exclude files with certain attributes. Use a comma separated list of attribute names to specify more than one. Possible values are: ReadOnly, Hidden, System, Directory, Archive, Device, Normal, Temporary, SparseFile, ReparsePoint, Compressed, Offline, NotContentIndexed, Encrypted, IntegrityStream, NoScrubData.

Anyway, I’m glad you solved the mystery.