Filters: Include Files not working as expected

I want to backup archives of type ZIP or JPA which begin with “site-” or contain “-complete-” in the filename.

I choose Filter:Include and add
site-*.zip
site-*.jpa
*-complete-*.zip

While I think these are valid, only the last are backupped.

I also switched to text-entry which shows:
+site-*.zip
+site-*.jpa
+*-complete-*.zip

Anybody got a clue what I’m missing?

You refer to filename. Think full path instead. You probably need more on the left side of your include.
For the third line, I think the * on the left (it means any number of characters) will take any path prefix.

Filters has more, including what the + means, if you were wondering and hadn’t already discovered it.

Yep, you’re absolutely right. I really was only considering file-names :slight_smile:

Thx for the push. It’s working now as intended.

I can’t understand. I want include only files, for example, “*over.jpg” and exclude all other files. How?

@FoxWMulder I didn’t realize this myself until now but here is what you need to do.

Lets say I want to backup all the .pdf and .blend files on my Desktop.

So I’ll create a new job (or edit an existing on) in Step 3 select the folder that contains the files I want to backup so I’d put a checkmark beside “Desktop”. Next click on Filters then on “Add filter”, select “Include expression” now in the box that has the * in it type add .blend, the text box should now read *.blend. Repeat the last process to add another filter right up to the last step where I’d put *.PDF in the text box. Save the job and the next time the job runs it will only backup the .blend and .pdf files from the Desktop.

EDIT: Removed note about a uppercase/lowercase issue that probably never existed.

From my quick testing you can’t do a compound wildcard with an “Include expression”, so *over.jpg isn’t going to work. Now if you know (want to fight with…) Regex you can probably make an expression that will work with an “Include regular expression” filter.

I’ll look into getting a regex expression put together in the next few days and post back once it’s ready.

That seems strange. My Windows 10 takes *.pdf just fine. Note you need +*.pdf to make it an include.

image

View from three-dot menu Edit as text:

image

Testing from Command Prompt using --include to replace the + sign that the GUI uses to mean include:

Duplicati.CommandLine.exe test-filters "C:\backup source\filter" --include="*.jpg" --include="*.blend"
Including source path: C:\backup source\filter\
Including path due to filter: C:\backup source\filter\file.blend => (*.JPG) || (*.BLEND) || (*\)
Including path due to filter: C:\backup source\filter\file.jpg => (*.JPG) || (*.BLEND) || (*\)
Excluding path due to filter: C:\backup source\filter\file.txt => null
Matched 2 files (0 bytes)

Documentation is at Filters but sometimes it seems to not work quite like it says for me. This test worked.

I’ve since tested it again on another job and it’s working as it should. /shrugs I don’t know how I messed it up three times in a row on three different jobs (somehow getting the .blend and .txt files each time, just not the .pdfs) but somehow I guess did. I’ll pull that note to not lead others a stray.

1 Like

what wrong? Why is the file “New Text Document.mebx” included in the backup?

20211221_2


20211221_3

If “New Text Document.mebx” is less than 100kb it will be included. By saying no (exclude) to anything over 100kb you’re also saying yes to (include) anything below 100kb.

Did you try it with just the *.jpg filter?

all ok:
20211221_6

not ok. Duplicati starts backing all files:
20211221_7

And if I want to make backup only JPG files up to 100 kilobytes?

And if I want to make backup only JPG files and exclude something?

Is your source selection the root of a drive? Something like just the E:\ drive selected with no subfolders?

The System Volume Information (SVI) folder only exists off the root of a drive, providing you’re at least one folder in from the root the SVI folder doesn’t exist. E:\Movies won’t have a SVI folder and shouldn’t need the SVI exclude filter.

To both include and exclude at the same time gets more complicated, you’ll need extra filters and their order in the filter list will matter. The very last common use case in the filter manual explains how you’ll need an additional exclude filter as the last filter to exclude everything that didn’t match the previous filters. -[.*[^\\]] (In the case of Windows).

1 Like

I was literally JUST about to post the same link to the filter manual. It does indeed get a bit more complex when you use both include and exclude filters.

I did not understand what it changes

I read. Can you write specifically how to create rules:
And if I want to make backup only JPG files up to 100 kilobytes?
And if I want to make backup only JPG files and exclude something?

I have already watched many options and can not understand.

This works for me… only the small png file gets backed up:

Filter configuration:
filter1

What the source folder looks like:
filter2

What actually got backed up:
filter3

1 Like

It seems to be. On a little bit begin to understand.
What does the expression mean “. * [^ \]”?

How to exclude empty folders?

I think I found a bug.

I make:
20211222_1

After saving, result:
20211222_2

Duplicate ignore size and backup all files

I can break down -[.*[^\\]]

The initial - means exclude, of course.
The square brackets [ ] around the pattern tell Duplicati it’s a regular expression (regex).
The rest .*[^\\] is the actual regex pattern.

If you’re not familiar with regex, they are like standard wildcard filters on steroids - very powerful.

The breakdown of the regex is:

  • .* match zero or more instances of any character
  • [] this set of brackets indicates that we should match for specific characters
  • the ^ inside negates the match - it’s a NOT operator
  • the \\ double backslash is to match a literal single backslash. backslash is normally an escape character in regex, so when you want to match a literal one you have to double it up.

Basically it’s saying “exclude anything that does NOT end in a backslash”, which means exclude all files.

I don’t believe there’s an option for that.

Yeah, that looks like a bug. But it looks like just a UI bug. The job should still be saved as 100 “byte”, which I confirmed by exporting the backup job to command line. It still worked as it should in my limited testing.

sorry. You right. It UI bug.

20211222_3


work right.