Configuration/Filters/Include... and symlink-policy=Store or =Ignore

Support
#1

Hi,

Is there a way to backup some directories explicitly included in Configuration/Filters/Include..., even if those directories contain symbolic links? (Windows).

Here’s my case:

I have a directory (C:\Mount) containing a fixed set of symbolic links to all of my storage devices that I need to access and backup (C:\ Mount\SATA1\, C:\Mount\RAID1\ etc). Not all of these devices are associated with drive letters, so I only access them through these symbolic links.

I want to make a backup copy of data accessed from C:\Mount, but I want to avoid circular references in other symlinks in the directory tree. So I:

  1. left the symlink-policy setting to =Save or =Ignore and
  2. explicitly included all directories required for backup (+C:\Mount\ SATA1\, +C:\Mount\RAID1\ etc) in the Configuration/Filters set.

But I see that symlink-policy=Save or symlink-policy=Ignore take precedence over Configuration/Filters and those filters (+C:\Mount\ SATA1\, +C:\Mount\RAID1\) don’t work.

Is there a way to solve this problem? Thank you in advance.

#2

Hello, I see no one has responded to your message yet. Have you made any progress? If not I can try setting up a test to see if I can reproduce the issue and hopefully find a solution.

#3

No, I have no progress. Due to this behavior of Duplicati, I still cannot use it in my setup. If you could find a solution, it will be great.

#4

Ok - I will try to do some testing in the next day or two.

1 Like
#5

I was unsuccessful trying to use filters to work around this. Duplicati simply refuses to descend past a symlink unless you have the Follow option set.

The only workaround I found (which you probably already know about) is to explicitly select all the top level files/folders just past each mount point. Messy, but it works.

#6

I’ve tried it, but Duplicati seems to reject any path containing a mount point. I got backups of zero sizes. Or it rejects it only if a mount point is at the end of the path?

#7

Do you still have the mount path or any parent folder selected for backup?

Don’t select C:\Mount or C:\Mount\SATA1 but rather all the folders inside C:\Mount\SATA1.

#8

Yes, I understood. Fortunately I have no many folders on the roots of mount points. I’ll try. Thank you!

Btw, is the trick with C:\Mount\SATA1\. (ending with dot) possible?

#9

I don’t think any filter will help. I could be wrong, but my understanding is that first Duplicati processes the drive/folder selections, descending into subfolders and cataloging everything. It does this BEFORE applying any include/exclude filters.

The symlink policy (when set to ignore or store) affects that first phase, so the subfolders of your mount points are not included from the initial list. As such, no include filter would work to “put them back”.

1 Like
#10

I ended up tricking Duplicati by defining source directories list as

C:\Mount\[mount point 1]\.
C:\Mount\[mount point 2]\.
C:\Mount\[mount point 3]\.
...

(all paths ends with a dot.)

With this trick, Duplicati can see all the files below mount points, without having to list all subdirectories in them.

I think in the first phase, [mount point N] does not meet mount point storage policy conditions because it is not the last part of the path. And setting the last dot '.' does not affect the actual destination of the path.

Thank you @drwtsn32 for clarifying the two-step directory lookup mechanism.

#11

Fix symlink handling on a source data match #4265 is a pull request currently under discussion.
Although I’m hazy on the details (I am here too), you could give your input, if it fits your use case.