I have some doubt about building filters

How-To
1

Hi,

I am trying to implement a basic filter scheme … basically I want to exclude a specific folder entirely, except one subfolder.

I am working with duplicati version 2.0.7.1_beta_2023-05-25

So I did a simple test directory, with the following structure:

I attache a zipfile with the structure, for easy replication.

duplicati_test.zip (2.1 KB)

F:.
└───duplicati_test
    ├───folder 1
    │   ├───folder1_1
    │   ├───folder1_2
    │   └───folder1_3
    └───folder 2
        ├───folder1_1
        ├───folder1_2
        └───folder1_3

now I want to apply a filter rule to

exclude everything in

f:\duplicati_test\folder 1

except everything under:

f:\duplicati_test\folder 1\folder1_2

so I run the following command:

F:\>Duplicati.CommandLine.exe test-filters f:\duplicati_test --include="f:\duplicati_test\folder 1\folder1_2\\" --exclude="f:\duplicati_test\folder 1\\"

which yields this output:

Including source path: f:\duplicati_test\
Excluding path due to filter: f:\duplicati_test\folder 1\ => (@f:\duplicati_test\folder 1\)
Including path as no filters matched: f:\duplicati_test\folder 2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\file1.txt.txt
Matched 3 files (12 Bytes)

Basically

f:\duplicati_test\folder 1

is completely excluded

I tried to achieve the same with regexp filters, which should do the job according to here:

regexr.com/7q1h5

but basically I get the same result as above.

F:\>Duplicati.CommandLine.exe test-filters f:\duplicati_test --exclude="[^F:\\duplicati_test\\folder 1\\(?!folder1_2\\).*$]"
Including source path: f:\duplicati_test\
Excluding path due to filter: f:\duplicati_test\folder 1\ => ([^F:\\duplicati_test\\folder 1\\(?!folder1_2\\).*$])
Including path as no filters matched: f:\duplicati_test\folder 2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\file1.txt.txt
Matched 3 files (12 Bytes)

Anyone has a suggestion how to achieve what I am looking for ?

Best Regards

2

The filters work on each level of the directory tree. That is why your first try doesn’t work. If folder 1 is excluded, the subfolders are not checked at all, so include filters don’t work.

The only reason to use include filters is if you have a more general exclude in the same level. For example (UI syntax):

+C:\folder\a.txt
-*.txt

This will include the specific a.txt file, but exclude all other text files. For this, the include must come before the exclude in the list.

With the regexp filter, you can do what you want, however you need to allow both folder 1\ and folder 1\folder1_2\.

So if I have it correct, this should work:

[^F:\\duplicati_test\\folder 1\\(?!folder1_2\\).+$]

I replaced the final .* with .+, so folder 1 is not excluded, but any files inside are.

You also did well by including start and end delimiters, because there is a potential filter bug if you leave them out.

If you use code blocks (```) instead of quotes, the characters are not messed up.

Bonus that should work without regexp (? matches 1 character, so folder 1 is not excluded): This also excludes subfolders from folder1_2, so doesn’t do what it should.

+F:\duplicati_test\folder 1\folder1_2\
-F:\duplicati_test\folder 1\?*

Out of curiosity, did you read this manual article:

I think it mentions this filter behavior somewhere in the text, but it can probably be improved to make it more obvious. Maybe we should even add this as an example, because it seems to be a common use case that people get wrong.

1 Like
3

Thanks alot for this quick and elaborated answer. I can confirm, that your proposed solution works.

Of cause I read the manual …

I just didn’t get the concept, to explicitely add one folder and then exclude all other folders on the same level using “?” wildcard.

F:\>Duplicati.CommandLine.exe test-filters f:\duplicati_test --exclude="[^F:\\duplicati_test\\folder 1\\(?!folder1_2\\).+$]"
Including source path: f:\duplicati_test\
Including path as no filters matched: f:\duplicati_test\folder 1\
Including path as no filters matched: f:\duplicati_test\folder 2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\file1.txt.txt
Excluding path due to filter: f:\duplicati_test\folder 1\folder1_1\ => ([^F:\\duplicati_test\\folder 1\\(?!folder1_2\\).+$])
Including path as no filters matched: f:\duplicati_test\folder 1\folder1_2\
Excluding path due to filter: f:\duplicati_test\folder 1\folder1_3\ => ([^F:\\duplicati_test\\folder 1\\(?!folder1_2\\).+$])
Including path as no filters matched: f:\duplicati_test\folder 1\folder1_2\file1.txt.txt
Matched 4 files (16 Bytes)

Also the non regexp aproach seems to work like that:

F:\>Duplicati.CommandLine.exe test-filters f:\duplicati_test --include="f:\duplicati_test\folder 1\folder1_2\\" --exclude="f:\duplicati_test\folder 1\?*\\"
Including source path: f:\duplicati_test\
Including path as no filters matched: f:\duplicati_test\folder 1\
Including path as no filters matched: f:\duplicati_test\folder 2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_3\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_2\file1.txt.txt
Including path as no filters matched: f:\duplicati_test\folder 2\folder1_1\file1.txt.txt
Excluding path due to filter: f:\duplicati_test\folder 1\folder1_1\ => (F:\DUPLICATI_TEST\FOLDER 1\?*\)
Including path due to filter: f:\duplicati_test\folder 1\folder1_2\ => (@f:\duplicati_test\folder 1\folder1_2\)
Excluding path due to filter: f:\duplicati_test\folder 1\folder1_3\ => (F:\DUPLICATI_TEST\FOLDER 1\?*\)
Including path as no filters matched: f:\duplicati_test\folder 1\folder1_2\file1.txt.txt
Matched 4 files (16 Bytes)
4

I looked at the non regexp filter again, and it is probably not going to work as I intended. It will also exclude all child folders from folder1_2. So its better to use the regexp you made.

5

Do you mind if I add your example to the documentation project ? or do you prefer to do it yourself ?

6

You are welcome to add it.

7

After quite a while tinkering with filters I finally found how to create a filter which actually does what I want:

recursively exclude a top folder, while recursively include a sub folder somewhere within the tree of the excluded top folder. (Should be a quite common scenario, no ? )

Downside of the solution (the only one I found working reliably) is, that for every level of the tree 2 rules have to be added, which can get messy on larger trees and also is quite error prone (something you actually not want to have in a backup solution).

So either there is a much simpler solution than that ? Or it would be probably good to add such a function into duplicati itself ?

I created a simple example tree as below:

W:.
├───folder1
│   │   file.txt
│   │
│   ├───folder1_1 >>>>>>>>>>>>>>>>>>>> This is to be excluded recursively
│   │   │   file.txt
│   │   │
│   │   ├───folder1_1_1
│   │   │       file.txt
│   │   │
│   │   └───folder1_1_2
│   │           file.txt
│   │
│   └───folder1_2
│       │   file.txt
│       │
│       ├───folder1_2_1 >>>>>>>>>>>>>> This is to be included recursively
│       │   │   file.txt
│       │   │
│       │   ├───folder1_2_1_1
│       │   │       file.txt
│       │   │
│       │   └───folder1_2_1_2
│       │           file.txt
│       │
│       └───folder1_2_2
│               file.txt
│
└───folder2
        file.txt

duplicati_test.zip (2.5 KB)

so lets assume I want to exclude folder

folder1

one but recursively include

folder1\folder1_2\folder1_2_1

I use this command (windows cmd shell):

Duplicati.CommandLine.exe test-filters w: ^
  --include="w:\folder1\folder1_2\folder1_2_1\*" ^
  --include="w:\folder1\folder1_2\\" --exclude="w:\folder1\folder1_2\*" ^
  --include="w:\folder1\\" --exclude="w:\folder1\*"

This is the output I get:

Including source path: W:\
Including path due to filter: W:\folder1\ => (@w:\folder1\)
Including path as no filters matched: W:\folder2\
Including path as no filters matched: W:\folder2\file.txt
Excluding path due to filter: W:\folder1\folder1_1\ => (W:\FOLDER1\*)
Including path due to filter: W:\folder1\folder1_2\ => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Excluding path due to filter: W:\folder1\file.txt => (W:\FOLDER1\*)
Including path due to filter: W:\folder1\folder1_2\folder1_2_1\ => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Excluding path due to filter: W:\folder1\folder1_2\folder1_2_2\ => (W:\FOLDER1\FOLDER1_2\*)
Excluding path due to filter: W:\folder1\folder1_2\file.txt => (W:\FOLDER1\FOLDER1_2\*)
Including path due to filter: W:\folder1\folder1_2\folder1_2_1\folder1_2_1_1\ => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Including path due to filter: W:\folder1\folder1_2\folder1_2_1\folder1_2_1_2\ => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Including path due to filter: W:\folder1\folder1_2\folder1_2_1\file.txt => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Including path due to filter: W:\folder1\folder1_2\folder1_2_1\folder1_2_1_2\file.txt => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Including path due to filter: W:\folder1\folder1_2\folder1_2_1\folder1_2_1_1\file.txt => (W:\FOLDER1\FOLDER1_2\FOLDER1_2_1\*) || (@w:\folder1\folder1_2\)
Matched 4 files (0 Bytes)

Basically the included directory has to be included with all sub folders and files

--include="w:\folder1\folder1_2\folder1_2_1\*"

then walking up the tree up to the level which is to excluded each level has to be handled with 2 rules:

--include="w:\folder1\\"   -> includes the folder iself
--exclude="w:\folder1\*"   -> excludes all folders and files in the folder

Any feedback appreciated !

8

worth mentioning:

  • the filter in the UI look like this:

    image
    image732×858 44.2 KB

  • filters as text (does not need quotation marks or double trailing “\”

    +C:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\*
+C:\temp\duplicati_test\folder1\folder1_2\
-C:\temp\duplicati_test\folder1\folder1_2\*
+C:\temp\duplicati_test\folder1\
-C:\temp\duplicati_test\folder1\*

  • restore dialog shows the filters work, but they will not be rendered correctly in the edit dialog above (this is probably a bug)
    image

9

Maybe this? folder 2 is left alone. folder 1 is excluded, except for folder1_2 below it.

Duplicati.CommandLine.exe test-filters "C:\tmp\duplicati_test" --exclude="[^C:\\tmp\\duplicati_test\\folder 1\\(?!folder1_2\\).+]" | find "Including"

Including path as no filters matched: C:\tmp\duplicati_test\folder 1\
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\folder1_1\
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\folder1_2\
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\folder1_3\
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\folder1_3\file1.txt.txt
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\folder1_2\file1.txt.txt
Including path as no filters matched: C:\tmp\duplicati_test\folder 2\folder1_1\file1.txt.txt
Including path as no filters matched: C:\tmp\duplicati_test\folder 1\folder1_2\
Including path as no filters matched: C:\tmp\duplicati_test\folder 1\folder1_2\file1.txt.txt

EDIT: If that’s what you want, I’ll explain some of the subtleties that are in it that help make it work.
Of course, it might still be wrong, as this stuff is not easy, but it’s a little easier if it’s only one line…

EDIT 2: and I see I reinvented post 2. Why is this discussion still going on?

10

Hi,

many thanks for taking the time. I tried to apply this solution to deeper trees, like in my 2nd example.

but it will include undesired files in folder1_2 … so not exactly what I am looking for.

c:\temp\duplicati_test>Duplicati.CommandLine.exe test-filters "c:\temp\duplicati_test" --exclude="[^c:\\temp\\duplicati_test\\folder1\\(?!folder1_2\\)(?!folder1_2_1\\).+]" | find "file.txt"
Including path as no filters matched: c:\temp\duplicati_test\folder2\file.txt
Excluding path due to filter: c:\temp\duplicati_test\folder1\file.txt => ([^c:\\temp\\duplicati_test\\folder1\\(?!folder1_2\\)(?!folder1_2_1\\).+])
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\file.txt

perhaps I am still missing a bit constructing the rule ?

11

Originally it sounded like folder1_2 was exempt from folder1 exclusion. Please clarify original below:

EDIT 1:

It looks like request changed, hypothetically. Which one do you actually want, and can you adjust?

EDIT 2:

There’s no such folder in the original .zip file. Maybe I’ll look for another one reflecting the change…

EDIT 3:

It’s under the diagram, with different size.

EDIT 4:

I noticed the .zip packing change, but tripped up over the space removal, e.g. “folder 1” is no more…

12

yes, I extended the testcase a bit … which unveils some issues with the original solution. And folder names are a bit less confusing :slight_smile:
In the new testcase the tree is 4 levels deep and it has files on each level.

I am looking for a generic way to exclude a folder on top level but recursively include a subfolder of it at any hierarchy level.

The current solution needs 2 rules for each hierarchy level between the excluded folder and the included subfolder.

Would be nice to have simpler / single rule to achieve the same.

13

I think this should be doable with 2 filters, although not very pretty:

+[^folder1/(?:folder1_2/(?:folder1_2_1/(?:folder1_2_1_1/.*)?)?)?$]
-folder1/*

Please let me know if this works, I didn’t test it yet. Also a disadvantage of this is that any excludes that should apply to the subfolders needs to be before the include rule in the list. For example, to exclude *.txt everywhere that needs to be above these filters.

14

I think we kind of break the rule engine here:

If I apply the first including rule, no files are matched

c:\temp\duplicati_test>Duplicati.CommandLine.exe test-filters "c:\temp\duplicati_test" --include="[^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]"
Including source path: c:\temp\duplicati_test\
Including path due to filter: c:\temp\duplicati_test\folder1\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Including path due to filter: c:\temp\duplicati_test\folder2\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Excluding path due to filter: c:\temp\duplicati_test\folder2\file.txt => null
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_1\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_2\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Excluding path due to filter: c:\temp\duplicati_test\folder1\file.txt => null
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_2\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_2\file.txt => null
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_2\file.txt => null
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\file.txt => null
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\file.txt => null
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\file.txt => null
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_1\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Including path due to filter: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_2\ => ([^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]) || (*\)
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_1\file.txt => null
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_2\file.txt => null
Excluding path due to filter: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_1\file.txt => null
Matched 0 files (0 Bytes)

if I add the 2nd exclude rule it matches all files … very strange:

c:\temp\duplicati_test>Duplicati.CommandLine.exe test-filters "c:\temp\duplicati_test" --include="[^folder1\\(?:folder1_2\\(?:folder1_2_1\\(?:folder1_2_1_1\\.*)?)?)?$]" --exclude="folder1\\"
Including source path: c:\temp\duplicati_test\
Including path as no filters matched: c:\temp\duplicati_test\folder1\
Including path as no filters matched: c:\temp\duplicati_test\folder2\
Including path as no filters matched: c:\temp\duplicati_test\folder2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_1\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\
Including path as no filters matched: c:\temp\duplicati_test\folder1\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_2\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_1\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_2\
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_1\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_2\file.txt
Including path as no filters matched: c:\temp\duplicati_test\folder1\folder1_1\folder1_1_1\file.txt
Matched 10 files (0 Bytes)
15

How do filters work? is an alternate version of the user manual documentation with a series of changes.
GitHub can provide version comparisons, but it can’t tell us whether one of the versions got things right.

16

I left out the start of the path for my example, so you need to add it. Also, there is a special case for only includes but no excludes, maybe that is why *\ suddenly appears in the first test.

17

I was wondering if you were on Windows and Duplicati took forward /, but no luck, so I used \\ in RE.
We’re still trying to get the desired folder levels right. Did it move again? If it’s still folder1_2_1, maybe:

Duplicati.CommandLine.exe test-filters "C:\tmp\duplicati_test" --include="[^C:\\tmp\\duplicati_test\\folder1\\(?:folder1_2\\(?:folder1_2_1\\.*)?)?$]" --exclude="C:\tmp\duplicati_test\folder1\*" | find "Including" | find "file.txt"

Including path as no filters matched: C:\tmp\duplicati_test\folder2\file.txt
Including path due to filter: C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\file.txt => ([^C:\\tmp\\duplicati_test\\folder1\\(?:folder1_2\\(?:folder1_2_1\\.*)?)?$])
Including path due to filter: C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\file.txt => ([^C:\\tmp\\duplicati_test\\folder1\\(?:folder1_2\\(?:folder1_2_1\\.*)?)?$])
Including path due to filter: C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\file.txt => ([^C:\\tmp\\duplicati_test\\folder1\\(?:folder1_2\\(?:folder1_2_1\\.*)?)?$])

EDIT 1:

A version which follows the “one directory level at a time” advice is longer, but can use wildcards, i.e.:

Duplicati.CommandLine.exe test-filters "C:\tmp\duplicati_test" --include="C:\tmp\duplicati_test\folder1\\" --include="C:\tmp\duplicati_test\folder1\folder1_2\\" --include="C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\*" --exclude="C:\tmp\duplicati_test\folder1\*" | find "Including" | find "file.txt"

Including path as no filters matched: C:\tmp\duplicati_test\folder2\file.txt
Including path due to filter: C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\file.txt => (@C:\tmp\duplicati_test\folder1\) || (@C:\tmp\duplicati_test\folder1\folder1_2\) || (C:\TMP\DUPLICATI_TEST\FOLDER1\FOLDER1_2\FOLDER1_2_1\*)
Including path due to filter: C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_2\file.txt => (@C:\tmp\duplicati_test\folder1\) || (@C:\tmp\duplicati_test\folder1\folder1_2\) || (C:\TMP\DUPLICATI_TEST\FOLDER1\FOLDER1_2\FOLDER1_2_1\*)
Including path due to filter: C:\tmp\duplicati_test\folder1\folder1_2\folder1_2_1\folder1_2_1_1\file.txt => (@C:\tmp\duplicati_test\folder1\) || (@C:\tmp\duplicati_test\folder1\folder1_2\) || (C:\TMP\DUPLICATI_TEST\FOLDER1\FOLDER1_2\FOLDER1_2_1\*)

It acts like the include requires all of the intermediate folder levels, but I’m not sure that’s documented.

EDIT 2:

The folder name ending with a slash is documented, and Windows Command Prompt need to double backslash before a closing double quote is left to the user to know. I’ve hit this enough that I know it…

Both the regex and wildcard act like they’re anchored, meaning whatever you use must cover the path. That’s not documented, I believe.

The User Manual version documents that a folder exclude applies recursively, but it’s silent on include.

1 Like
18

These last 2 version do exactly what I was looking for.

Event though requiring one rule for each hierarchy level, I would probably prefer the 2nd one … as it is more readable, at least for me not used thinking in regex :wink:

Thanks for the great support on the topic. The community is really what makes the difference between great and useless software.

1 Like
19

It’s also similar to your try, and to the regex version if one goes through the possible quantifiers resulting from a question mark, which is basically either 0 or 1 repeats, a.k.a. {0,1}, but without all combinations because of the grouping. It’s also clearer which one means a folder (ends with slash), and I don’t know Duplicati’s exact way of figuring that out from a regular expression, or how it fits into the documentation.

I tested changing the --include order of the earlier folders, and that still worked. I tested making a folder include look like a non-folder by deleting the trailing slash (two slashes in this Command Prompt case), and it broke. I’d very, very much like someone to figure out how this works and fix the documentation…