Trouble with GUI generated source exclude filter

[Taomyn]

I’m trying to add some source exclusions to a backup based on a string within a filename, but the filter the GUI is generating fails the backup. This is what I did:

• Add filter
• Set “Exclude files whose names contain”
• Entered my string “GMWNCWSLYA”
• Add a second exclude filter with string “lock_v2”
• Saved the backup

When I run the backup I get:

System.ArgumentException: parsing ".*GMWNCWSLYA.*[^\]" - Unterminated [] set.
   at System.Text.RegularExpressions.RegexParser.ScanCharClass(Boolean caseInsensitive, Boolean scanOnly)
   at System.Text.RegularExpressions.RegexParser.CountCaptures()
   at System.Text.RegularExpressions.RegexParser.Parse(String re, RegexOptions op)
   at System.Text.RegularExpressions.Regex..ctor(String pattern, RegexOptions options, TimeSpan matchTimeout, Boolean useCache)
   at System.Text.RegularExpressions.Regex..ctor(String pattern, RegexOptions options)
   at Duplicati.Library.Utility.FilterExpression.FilterEntry..ctor(String filter)
   at Duplicati.Library.Utility.FilterExpression.<>c.<.ctor>b__11_0(String n)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Duplicati.Library.Utility.FilterExpression.Compact(IEnumerable`1 items)
   at Duplicati.Library.Utility.FilterExpression..ctor(IEnumerable`1 filter, Boolean result)
   at Duplicati.Server.Runner.<>c.<ApplyFilter>b__18_2(<>f__AnonymousType2`2 <>h__TransparentIdentifier0)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.Aggregate[TSource](IEnumerable`1 source, Func`3 func)
   at Duplicati.Server.Runner.ApplyFilter(IBackup backup, IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

When I edit the backup job and check the filter as text, it has been generated by the GUI as follows:

-[.*GMWNCWSLYA.*[^\]]
-[.*lock_v2.*[^\]]

Any idea why the GUI is generating a bad filter parameter?

I’m running 2.0.4.22_canary_2019-06-30 and this is on Windows.

[drwtsn32]

@Taomyn

-[.*GMWNCWSLYA.*[^\]]
-[.*lock_v2.*[^\]]

Yeah this is an invalid regex, and it doesn’t even make sense. The caret symbol indicates start of line, but it’s put near the end of the pattern.

The reason it’s “unterminated” is because the brackets are not balanced. There is a backslash in front of one of the brackets which instructs it to be treated as a literal character match and not a regex closing bracket.

What happens if you view the filters as text, delete the [^\] part, and then save changes? Do backups work after that, or is [^\] put back in automatically again?

[drwtsn32]

I just tried it… i’m also using 2.0.4.22… easy to reproduce the problem:

Step 1:

Step 2: - click Edit as Text

Step 3: delete the [^\] part and click Edit as List:

It changes to Exclude regular expression option, but at this point it looks valid.

Definitely a bug though!

[AimoE]

@drwtsn32

The caret symbol indicates start of line

If this is Perl regexp, a caret inside brackets indicates “not”; the brackets indicate characters to include, or not.

[drwtsn32]

@AimoE

If this is Perl regexp, a caret inside brackets indicates “not”; the brackets indicate characters to include, or not.

You’re right, I totally forgot about that usage inside bracketed character sets.

Maybe the intent was to ensure the pattern appeared in the filename portion, and to make sure no more path separators appeared after the pattern. In that case they just forgot to escape the backslash: [^\\] But even that doesn’t achieve the correct result.

Edit: I think if they prepended the entered pattern with .* and appended [^\\]* (instead of .*[^\] it would work - if the intent was to ensure it is in the filename portion

[ts678]

@drwtsn32

Edit: I think if they prepended the entered pattern with .* and appended [^\\]* (instead of .*[^\] it would work - if the intent was to ensure it is in the filename portion

Agree.

@ts678

I suspect the bug is from suffix: '.*[^!]]', where a Negative Character Group: [^] means to not match something ending with the separator that indicates a directory, according to Duplicati Syntax documentation.

Character Escapes start with backslash, so Windows directory separator backslash needs its own backslash whereas I have a suspicion that this bug would not happen on Linux. A forward slash is not special to an RE, however a backslash before a non-special character just gives the character, so “!” in source “should” work.

and one additional note on the above comment is that the ! in source code turns into directory separator.

[Taomyn]

@drwtsn32

I just tried it… i’m also using 2.0.4.22… easy to reproduce the problem:

Step 1:

Step 2: - click Edit as Text

Step 3: delete the [^\] part and click Edit as List:

It changes to Exclude regular expression option, but at this point it looks valid.

Definitely a bug though!

Thanks, works around the problem for me

[drwtsn32]

@Taomyn

Thanks, works around the problem for me

Keep in mind that if you want the pattern to ONLY match filenames and not also directories, you’ll need to adjust it slightly from my screen shot example.

Replacing the .* at the end with [^\\]* should work (if you are using Windows).

[Taomyn]

Ok, thanks again - it would be fine but I will adjust them just to be sure

[ts678]

Would either of you care to see if a Duplicati issue has been filed? I used to drop diagnostic hints in the forum, but they didn’t seem to get picked up. I don’t know if an issue will move any faster, but at least it’ll be tracked…

[drwtsn32]

Yes, I can take a look. Thanks

Edit: looks like an issue was opened - "Exclude files whose names contain" causes error · Issue #2220 · duplicati/duplicati · GitHub

Not sure why it hasn’t been fixed yet, seems trivial…