Environment var for filters

avmaksimov · April 5, 2024, 10:15am

Please add this ability. I want to backup it and I think it’s the easiest way.

Better if this would be contained in backup files but as I see it’s harder to make.

ts678 · April 5, 2024, 1:02pm

Please clarify. This is very hard to follow because it uses unspecific words like “it” and “this”.

Backup what? What sort of backup? Are you looking to make a safety copy of configuration?

How so? This ties into first question. Maybe you can compare the two ways you have in mind.

Better if what would be contained?

The backup file format rarely changes, and changes cause backward compatibility problems.

The topic subject initially sounded like you wanted an environment variable to set filters, but
possibly you want one to read filters out in a run-script-after. Are Duplicati scripts part of this?

Global environment variables can be tough to set, especially if wanted in a Windows service. Sometimes reboot is needed, and global variables are also global, not settable as job needs.

Although developer resources are too few to do features-on-request, can you at least clarify?

Sometimes things get done if other people come along and add support, so what is this one?

avmaksimov · April 7, 2024, 2:11pm

Sorry for misunderstanding. Yes I want to backup my backup configuration. I know that I can use duplicati client but this solution not convenient for backup on many computers.
But I use the common after backup script. And I can backup it. BTW may be in the common before backup script.
But of course the best solition for that would be backup as pseudofile to backup and ability to restore configuration from it.

ts678 · April 7, 2024, 5:24pm

Thanks for clarifying. One challenge is that filters can be quite extensive, across multiple lines. Environment variables work better for short things. I suppose break on path separator, like for LOCALPATH, would be one option, although technically a Windows file might have semicolon.

Supported commands

export export a backup from the server to YAMl or JSON format

is presumably what you have in mind. GUI has manual Export To File with quite a few options, however CLI does not, in either the main command or a standalone helper, such as done with
Duplicati.CommandLine.ConfigurationImporter.exe. An export ability might be an alternative, if developers and users think it would be a good substitution (someday) for environment variable.

Are you changing filters frequently? An occasional manual export would also save more config. Depending on job run schedule, another option is to just have one that runs often enough grab Duplicati-server.sqlite which will have the full config for all the jobs for that Duplicati installation.

Despite its potential confusion over filenames containing semicolon (on Windows), did you see:

github.com

duplicati/duplicati/blob/23065d44ef142c104d47ef84e083b51cdf5d54ff/Duplicati/Library/Modules/Builtin/run-script-example.bat#L70-L84


      
          REM Filters are supplied in the DUPLICATI__FILTER variable.

          REM The variable contains all filters supplied with --include and --exclude,

          REM combined into a single string, separated with semicolon (;).

          REM Filters set with --include will be prefixed with a plus (+),

          REM and filters set with --exclude will be prefixed with a minus (-).

          REM

          REM Example:

          REM     --include=*.txt --exclude=[.*\.abc] --include=*

          REM 

          REM Will be encoded as:

          REM     DUPLICATI__FILTER=+*.txt;-[.*\.abc];+*

          REM

          REM You can set the filters by writing --filter=<new filter> to stdout.

          REM You may want to append to the existing filter like this:

          REM     echo "--filter=+*.123;%DUPLICATI__FILTER%;-*.xyz"

and I’m seeing at least the output variable in the before script, but not the after. Might be a bug.
Opening a Duplicati Issue is the right way to report bugs. Or perhaps it will be an enhancement request, but it seems to go against the documentation (a little). Technically it’s in the “Changing options from within the script”, not “Special Environment Variables”, but LOCALPATH is in both.

EDIT 1:

A set command redirected to a file from batch files before.bat and after bat got these variables:

Before:

DUPLICATI__filter=+@include.me;-@exclude.me
DUPLICATI__LOCALPATH=C:\backup source\A.txt;C:\backup source\B.txt

After only got LOCALPATH. I don’t know if there’s a reason for no filter. A developer might check.

EDIT 2:

Since what you seem to want to have seems available in run-script-before, can’t you just use it?
Whether or not there’s a bug here, having the feature already partly present might cut its priority.

avmaksimov · April 7, 2024, 7:03pm

Thank you. I made the issue: Add ability to use DUPLICATI__FILTER in after script · Issue #5143 · duplicati/duplicati · GitHub

Off course better if a developer be able to create auto backup this settings and restore.

No I don’t change filters often, but in some backup I use very hard regexp which was tested for hours. Unfortunately, there is no easier way to filter files in many cases.

And as I make changes rarely I forget to backup it (.