Question regarding file filter

Hi all,

I was trying to create a backup job that saves the sqlite databases from Duplicati to be able to quickly restore them in case of disaster recovery. So I set up a backup job with the source folder …

C:\Users\xyz\AppData\Local\Duplicati\

… and wanted to use a filter to only back up these files:

Duplicati-server.sqlite
XXXXXXXXXX.sqlite

I used the “filter” feature on the “source data” tab from the web ui. I created two filters as follows:

<filter (include)>: Duplicati-server.sqlite
<filter (include)>: XXXXXXXXXX.sqlite

Running the backup, I realised that no files were saved. Then I used the command line tool to test the filters and finally I found out, that I have to define the filters as follows (note the leading “*”!):

<filter (include)>: *Duplicati-server.sqlite
<filter (include)>: *XXXXXXXXXX.sqlite

I did not find any explanation in the documents why the filters fail when using the complete filename. And I did not find any explanation why I have to use a leading * for the filter text. Can anyone of you give me more information?

Thank you!

Greetings,
Christian

Easier way to pick files from a source folder is to expand it then click only the files that you want.

There is no such option in the dropdown. If you did Include expression, it gives you a * and you fill in the rest of the path. To emphasize that, it’s a path not a filename, so * in front may be critical.

Filters in Duplicati has info. You can check on GUI’s filter with Edit as text in three-dot menu.

Globbing expressions explains *. This is different than on some systems where it stops at slash.

Indeed, you are right. For selecting exactly these two files, this is the way to go. I chose the other way because I thought of backing up all job databases in future and therefore I’d need some filter like*.sqlite. But it’s perfectly fine to chose the files manually.

Yes, there is. Please see the attached screenshot. Maybe it’s a translation issue, but the selected values translates to “filter (include)”. The option “Include expression” is probably “Regulären Ausdruck (einschließen)”.

I read this page, but I was not aware of the fact, that I have to specify the source path of the file with a leading “*”. I thought, specifying a folder from the drive/folder box and refining the selection by adding a filter like “IMG*.jpg” will result in including all images in the folder defined above.

Again, thank you for clarifying this.

Greetings,
Christian

The UI is trying to be helpful and hide the needed prefix etc. You can use “Dateien ausschließen dessen Namen beinhaltet” to supply only the name.

If you use “Filter” it is a full path matching filter that you need to enter.

In the first translation, should Filter be Ausdruck? Is Filter even valid German?
I’m not sure how translations get checked and fixed these days. It’s third-party help…

Such logic weakens if you chose lots of things in the tree. Which one would Filters affect?

Filters in old manual (new one is very new) explains another pitfall of having just includes:

In the special case that all rules are include rules (which does not make sense when all files and folders are included per default) Duplicati assumes that all other files and folders are meant to be excluded (this had to be defined as another rule in Duplicati 1.3 but most people found that confusing so we changed that in Duplicati 2.0).

But this is exactly the opposite what I tried to achieve. “Dateien ausschließen dessen Namen beinhaltet” means to exclude files which contain a certain pattern in the file name. I tried to specify a path and only backup files with a certain pattern in the name - so this would be to include these files. I cannot exclude all the other files because I cannot know which filenames might occur in future.

“Filter” is a valid German word, and is used to describe the limitation of some data to a smaller subset. “Ausdruck” is more like “expression” and if it refers to a regex, this would be the correct translation. Otherwise one could use “Suchmuster”, which is “search pattern”, but this might be too similar to the regex patterns.

Good point. I would have expected that a search filter applies to all selected folders. E.g. if I select a lot of folders with photos and specify *.cr2 as search filter, I would assume that only raw photos from all folders are backed up (and not the *.jpg thumbnails). But I see that this could lead to unexpected results (and files not backed up).

But the same applies to the exclude filter which also affects all folders selected in the drive/folder list. Knowing this, it is absolutely clear that a filter must contain some sort of path information (or * at the beginning). But for me it was not intuitive how to specify this, maybe because I am mostly a Windows user and there are differences how wildcard search is used in different operating systems …

Arh, my German was not strong enough to deduce that (and I did not check the English version).

Since all files are included by default, there are not a lot of options to selectively include files. The idea is that most users will want to include everything and then selectively exclude certain files and folders.

It only makes sense to use include rules if you also exclude something, and then the order of expressions then becomes important.

Because of this logic, there is a check for “only include filters”, and if there are only includes, it will default to exclude everything not matched. This is not often used, so the UI does not really help in that case.

I see and this seems to work as I was able to back up only the databases. My initial mistake was not to specify a wildcard for the path before the files description (“*Duplicati-server.sqlite” instead of “Duplicati-server.sqlite”).