Advice about backend options in tab 5

gpatel-fr · January 6, 2024, 3:09pm

Hello

I think that it is a design mistake to allow to manage backend options at 2 different places in the web ui.

The backend options should only be available in the tab 2 where destination is managed. In the tab 5, all other options should be available but NOT backend options. Doing otherwise is illogical since when users have added advanced backend options in the tab 2, they see them available in the option list of the tab 5 but their already entered values are not displayed.

I think that the simplest way to avoid this confusion could be to just remove the backend options in tab 5 UI. This should be trivial it seems:

diff --git a/Duplicati/Server/webroot/ngax/scripts/controllers/EditBackupController.js b/Duplicati/Server/webroot/ngax/scripts/controllers/EditBackupController.js
index 32b61c0ef..cc45be304 100644
--- a/Duplicati/Server/webroot/ngax/scripts/controllers/EditBackupController.js
+++ b/Duplicati/Server/webroot/ngax/scripts/controllers/EditBackupController.js
@@ -640,12 +640,8 @@ backupApp.controller('EditBackupController', function ($rootScope, $scope, $rout
 
         var encmodule = $scope.Options['encryption-module'] || '';
         var compmodule = $scope.Options['compression-module'] || $scope.Options['--compression-module'] || 'zip';
-        var backmodule = $scope.Backup.TargetURL || '';
-        var ix = backmodule.indexOf(':');
-        if (ix > 0)
-            backmodule = backmodule.substr(0, ix);
 
-        $scope.ExtendedOptionList = AppUtils.buildOptionList($scope.SystemInfo, encmodule, compmodule, backmodule);
+        $scope.ExtendedOptionList = AppUtils.buildOptionList($scope.SystemInfo, encmodule, compmodule, false);
         setupServerModules();
 
         AppUtils.extractServerModuleOptions($scope.ExtendedOptions, $scope.ServerModules, $scope.servermodulesettings, 'SupportedLocalCommands');

I think that being intended for advanced users, the command line interface should not be concerned by this change.

Thoughts ?

ts678 · January 6, 2024, 4:01pm

My first thought is – don’t do it until the PR to retain screen 2 advanced options is in (if it’s correct).

I don’t have much to say offhand about the original motivation, and haven’t gone searching for that. Possibly the original author would have something to say. I hope the time situation eases someday.

It’s sometimes nice to get a URL that has all you require. IIRC some CLI tools don’t accept options.

Beyond that, I’ll stay out of internal C# and JavaScript processing because I’m unfamiliar with them.

As a general rule, it’s hard to delete things that are used. If it goes off screen 5, how to do changes?
Would migration to screen 2 attached-to-URL format occur? They are also more character sensitive.

gpatel-fr · January 6, 2024, 8:52pm

yes

the project owner does not involve himself anymore than strictly necessary now.

any backend option entered in the tab 5 will still appear and can be deleted like any other option and inserted back where it belongs, in the tab 2. It will still be possible to change it but in text form only (entering ‘false’ for a boolean option).

no, it will be up to the user to do it.

what do you mean ?

ts678 · January 6, 2024, 11:48pm

It seems very user-unfriendly to limit the user in this way.

URL format may require Percent-encoding of some characters.
Current various URL parsers also trip up on certain characters.
IMO as a general policy, the cleanest way to do special characters is as options.
Using a shell adds need for shell quoting. Using a batch file requires doubling %.
Now go back to the top and see how those get into the line with the URL format.

EDIT: Yes I know that the problem already exists, however proposal increases it.

gpatel-fr · January 7, 2024, 6:55am

Thanks, this is a concern, I will defer the idea until (if) I find a way to work around this.

If something does not work when entering options in step 2, this is a serious bug that should be fixed. You are asking to keep a flawed design because it could allow to work around bugs that should be fixed, while my change would not stop using the workaround anyway (it would just make it less easy to find), while removing shoot in the foot mistakes (entering conflicting options in both tabs).

This concerns advanced users. And nothing will stop these users to enter options outside the URL anyway. As I said, my proposal is about the UI.

Jojo-1000 · January 7, 2024, 12:29pm

That is already done in the destination options by the UI.

If we do this, it should be ensured that other options which may be present will not be hidden, even if the options are not available. Otherwise we will have the same trouble again.

gpatel-fr · January 7, 2024, 1:22pm

Sorry, after reading this 3 times, I have no idea what you want to say. When I talk of ‘all other options’, I am thinking to ‘all other options that are currently displayed in tab 5’.

Jojo-1000 · January 7, 2024, 1:36pm

If someone specified advanced backend options in tab 5, and we remove the option, it should still show up in the default list, not just the advanced text field. Otherwise it will cause confusion about an invisible option. I’m not sure if this is already the case or not.

ts678 · January 7, 2024, 1:48pm

Yes. Original post had below and details.

We see this in the Web UI in other places too, for example recall the Source file picker allows two ways, and again we face the problem of not knowing original rationales. We do know some OS offer shortcuts, however the navigation hints are nicer, and the main area changes to reflect the left-bar decision, unlike Duplicati’s where it’s possible to have both views expanded at the same time, thereby getting confused.

Because the main area of Windows File Explorer changes based on left bar, one picks a way early but doesn’t get confused with two views simultaneously visible. Duplicati gets worse when one sees trees seem uncoordinated in checkmarks, and even worse when one starts excluding and looks at the filters:

which is basically part of your complaint:

Simple has appeal, but more complicated coordination might also be possible, but a challenge is I think UI and internals sometimes relate. Internals may face the same difficulty of possibly conflicting options.

Default options in Settings are probably of the individual option type rather than attached-to-URL flavor, have the same potential for conflict with per-job options, but I think there’s a priority order than solves it.

Is there a priority order of options in URLs (screen 2) versus standalone options (screen 5)? I could test many of these things, but at this point I’m just making comments without detailed behavior explorations.

Actual complaints these days are typically wondering why their screen 2 options vanished. We say use screen 5 instead, so users will feel a little jerked around if we force another manual migration back to 2.

At least one user tripped over the duplicated Source. I forget if uncoordinated option ways trip users up.

Fix hidden username and password options when changing backends, thanks @Jojo-1000

in latest Canary raises another decision factor of having two ways to do things. Do screen 5 options get cleaned up when switching, or is that a screen 2 thing? Or maybe advanced options live on either way?

There’s a general design question of what to do with options that don’t apply to a given scenario. Often one can bring up a GUI Commandline, change the Command, and let it fly, but sometimes it complains.

Screen 5 GUI tends to show (without help of course) any random option-like thing one forces in as text, however I haven’t explored the screen 2 GUI yet. It seems to know which options are GUI-relevant, but possibly picks up the clue by looking at what’s on its URL rather than what’s done as traditional options.

I mentioned possible traditional options limitation in some of the simple CLI commands earlier, but have not tested just now to see if backend options are an exception. I already said it’s sometimes nice to get everything in one chunk, in fact for those simple CLI commands, I often say do export (which helps get command line quoting right) and use that as target URL. This plan may get defeated by screen 5 ways.

Each way seems to have pros and cons, and I guess we just opened kind of an exhaustive assessment. Sometimes I think when two ways have good points, both are given , but then they get uncoordinated…

EDIT:

Different ways to make a Duplicati backup has The many (inter-)faces to Duplicati which shows Duplicati’s confusing but powerful flexibility. A very typical user confusion is CLI and server coordination.

gpatel-fr · January 7, 2024, 6:34pm

I prefer to look at one problem at a time.

My goal is not to try to fix all problems at the same time.

I don’t think the screen 2 option vanish that often. For example I can’t repro this one

That’s precisely what I’m trying to avoid. One post, one subject.
Keep things on topic make a good forum

I think to understand that you worry about what happens after this change when someone has already setup a backup with a backend option entered in tab 5 ? If that’s your problem, I think I already answered about that: the option is still displayed in the UI if not in the selection list for new options, the problem is only that since the list is lacking the nice UI is missing, that is, a checkbox is replaced by a text field containing the string ‘true’ or ‘false’. I actually got this idea when realising that there is a hidden option in Duplicati that you can enter only in text mode, but after that you can see it in the UI.

ts678 · January 7, 2024, 6:58pm

Just drawing an equivalence of issue. Any fix (if even practical) will likely be very different.

A point fix without understanding the internal relationships might wind up too shortsighted.

I can repro that fine on 2.0.7.2 or 2.0.7.100 on Windows.

Set those two screen 2 Advanced options, Save, see in Commandline that they’re on URL.
Edit back to Destination screen 2, they’re no longer in Advanced options.

The one subject is probably not just a cosmetic choice, but requires a broader design view.

Jojo-1000 · January 7, 2024, 7:13pm

I could reproduce it. There is a comment with the reason on the issue and a PR to fix it, it only happens for file urls with backslashes. The PR also has more specific steps for reproduction.

I think that would be fine.

For backend options, I think BackendLoader.GetBackend is the relevant place to look:

github.com

duplicati/duplicati/blob/master/Duplicati/Library/DynamicLoader/BackendLoader.cs#L65-L74


      
          public IBackend GetBackend(string url, Dictionary<string, string> options)

          {

              var uri = new Utility.Uri(url);

              

              LoadInterfaces();

              

              var newOpts = new Dictionary<string, string>(options);

              foreach (var key in uri.QueryParameters.AllKeys)

                  newOpts[key] = uri.QueryParameters[key];

So, both options are merged, but URI parameters (from page 2) overwrite options in page 5. Except for that, there will be no difference either way. It might be possible to run into some URI length limitations, but I don’t see that for currently existing options.

There is also the interesting special handling for use-ssl: This is also set as an option for ***s-protocol variants.

gpatel-fr · January 7, 2024, 7:52pm

you are right, it’s a Windows path problem. Does not exist on Linux (and Mac I presume).

ts678 · January 7, 2024, 8:25pm

Confirming via CLI commands, while also confirming that at least one also takes usual options:

Duplicati.CommandLine.BackendTool.exe get “rclone://remote/path?rclone-local-repository=local&rclone-executable=C%3A%5Cbin%5Cwrong.exe” file.txt --rclone-executable=“C:\bin\rclone.exe”

and switch around to see if it complains about starting rclone or about rclone not finding the file.

The two ways are seen in Export As Command-line. Export To File and DB URL are TargetURL. Compatibility with old exports is probably not hurt by simply tweaking the GUI display view, and maintaining all support for existing command line usage and exports which may still get conflict.

That would be a little more on the user though, and is any human likely to hand-configure URL?
Coordination within command line is also on user. If they do get a conflict, we know which wins.

Basically, we are probably stuck with two ways forever, but proposal is to get users to screen 2.
Another advantage of that is that one doesn’t have to save the config, then go back in screen 5.
Possibly screen 5 could read the screen 2 Storage Type right from a variable if we really want.

Last I heard, proposal has migration from 5 to 2 on the user, but screen 5 acts kind of unusually.
Even if we did migration, we can’t migrate old config files. Where’s the cutoff? Totally new jobs?

ts678 · January 7, 2024, 8:47pm

I didn’t know at the time, as I hadn’t tried anything else, but at least I did say what OS I was trying on. Technically this might be called “character sensitive”. I’ve collected others, but am not sure where the sensitivity is from, and we’re probably stuck with URL-plus-option format, whatever problem it invites.

but I’ll refrain from citing them here, partly because they need sorting out, and also to avoid the clutter. Thinking we’re going to have to respect non-URL options allows screen 5 Edit as text if necessary.