Restore bug on Synology if path has spaces

Support
1

Quick update: the ‘restore bug’ is confirmed for the DS218+, using DSM 6.2.1-23824u4, Mono 4.6.2-0096, and Duplicati 2.0.4.5. Navigating/selecting files to restore indeed does not work when the path contains spaces. Restoring itself is not a problem though (when selecting the entire parent folder for restoring).

I have not been able to find any info on exactly how widespread the bug is. However, given the number of people that have reported it, I would not be surprised if it occurs on all Synology systems with (at least) recent versions of DSM and Mono.

2

@thomas, thanks for the confirmation.

I pulled this out to it’s own topic so it can be addressed directly. (Hope you don’t mind.)

3

I can’t reproduce this problem on my DS1517+ running DSM 6.2.1-23824 Update 2.
My Mono and Duplicati versions are the same as yours.

4

Installed update 4 and it still works fine for me.

5

@JonMikelV I certainly don’t mind.

@drwtsn32 Thanks for this insight. Very interesting to hear from someone with the same software versions, and who is not experiencing this problem. Out of curiosity: how did you test? Via DSM Web Interface -> Main Menu -> Duplicati, or via some other way?

6

Would you be able to provide a screen shot of the failure scenario? Maybe there’s a misunderstanding about what exactly isn’t working.

For example, I can show that my Docker container running 2.0.4.10 has no issues letting me selecting a file in a path that contains spaces:

image
image.png692×177 15.6 KB

Oh, and a while ago there was a general issue with the restore browser where files backed up from a UNC connection (so starting with “\”) weren’t browsable (didn’t even show up in the list of folders). However, a workaround was found where putting a single period in the search box made it all appear.

I don’t suppose the “.” search thing makes a difference with your issue?

7

Here is a screenshot. Expanding the folder tree works up to when I encounter a folder with a space in the name (such as “Saved Games” as in the screenshot). Trying to expand such a folder triggers the BadRequest error. This is identical to the behavior described in issue #2837.

Putting a “.” in the search box makes the entire folder tree appear in bold text, but has no other effects whatsoever (i.e. does not resolve the problem).

screenshot
screenshot.png964×698 118 KB

8

That helps a lot, thanks! It looks to me like the UI is not correctly escaping the whitespace in the path thus causing the BadRequest to the Duplicati server.

In the linked to Git issue it was suggested the problem might be browser dependent. I’m using Chromium (on Ubuntu Linux) - what browser are you using?

Note that it’s also possible (though unlikely) that the folder doesn’t actually have a space in it but instead some other non-printable character that isn’t part of the escaping code. Do you have any other space-containing paths you could test with?

9

Thanks for your suggestions. It happens consistently with folders that contain spaces, so I have no reason to assume some other character may be involved.

I’m using Firefox on Windows. I’ve also tested with Internet Explorer (same behavior) and Edge (same behavior, but dialogue box displayed on error is empty). That’s three different browser engines, all giving the same problem.

10

One addition: when I configure Duplicati for remote access and connect directly from my browser (bypassing the DSM web interface), I encounter no problems. Maybe not that surprising, but it couldn’t hurt to check this.

11

Sounds like you’ve isolated it to a GUI side bug, so now we need to figure out why it’s not happening to everybody.

Do you know how to use a browser debug mode? If possible, maybe you could capture the network request for the failed request so we can see it a little more raw than what’s shown in the error message.

Here’s what mine looks like for my successful request from Chromium:
http://localhost:8200/api/v1/backup/1/files/%2Fsource%2FData%2FeBooks%20%26%20manuals%2F?prefix-only=false&folder-contents=true&time=2019-01-01T02%3A22%3A09-06%3A00&filter=%2Fsource%2FData%2FeBooks%20%26%20manuals%2F

12

So it only fails when using the DSM interface?

13

Yes, it only fails in the web interface.

I haven’t used Firefox’s debug mode before, but I believe I managed see the GET requests of interest. The following GET request is for when I expand a folder in the GUI, and this folder does not contain a space:

http://–IP-adres–:5000/webman/3rdparty/Duplicati/api/v1/backup/1/files//volume1/–snip–/C/Users/Thom/?prefix-only=false&folder-contents=true&time=2018-12-27T09:50:18+01:00&filter=/volume1/–snip–/C/Users/Thom/

The next folder I expanded contained a space. The GET request for this one is:

http://–IP-adres–:5000/webman/3rdparty/Duplicati/api/v1/backup/1/files//volume1/–snip–/C/Users/Thom/Saved Games/?prefix-only=false&folder-contents=true&time=2018-12-27T09:50:18+01:00&filter=/volume1/–snip–/C/Users/Thom/Saved Games/

14

I don’t actually have a Synology myself so can’t test, but I wonder if there’s some sort of JavaScript collision going on between the DSM interface and our internal UI code.

Or perhaps the DSM side of things is trying to be nice and escaping the space before giving it to the Duplicati UI which then un-escapes it causing the issue. (Note that this is just random guessing here…)

15

Well, I have one so maybe I can be of help. :wink:

I’ll try to look into the issue a bit more. Any suggestions are of course welcome.

As an additional thought: if there is a collision in how the DSM interface and the UI code handle spaces, I totally fail to see how the problem could be unique to some Synology systems (as @drwtsn32 reports that he does not encounter any problems with his system).

16

You nailed it! I always access Duplicati on my NAS “remotely” from my main PC. No issues there.

I tried the native DSM interface and I get “Bad Request” if there’s a space in the path, as OP indicated.

17

That is very useful information. Thanks for reporting back!

So, my hypothesis that it may be a problem with all current versions of DSM/Mono/Duplicati still stands.

18

I’m seeing the same error on restore on my DS213j with duplicati 2.0.4.5 and want to add some information:

Observations:

  • error occurs within synology gui (even in “open in new window”) when accessing via https://[xxxxx].myds.me:5001/
  • error occurs within synology gui (even in “open in new window”) when accessing via https://[IP]:5001/
  • error does not occur when accessing via http://[xxxxx].myds.me:8200 (needs additional portforwarding for 8200 on my dsl router because [xxxxx].myds.me is a public dns hostname)
  • error does not occur when accessing via http://[IP]:8200
  • i can’t even succesfully connect if i connect via http://[hostname]:8200 => probably because I didn’t add [hostname] to Settings->Hostnames …(?) (i get the gui but instant “The connection to the server is lost, attempting again in 0:10 …”)

My guess:
The response header in Chromes Network-Tab IMO indicates that some “deescaping” happens somewhere serverside, because the escaped spaces ("%20") within the url (but not within the url-parameters!!) are shown as spaces within the error-message - which leads to duplicati no being able to understand HTTP version because it’s looking at the “wrong place” because of the spaces:

1. Request URL:
https://[xxxxx].myds.me:5001/webman/3rdparty/Duplicati/api/v1/backup/2/files/%2Fvolume1%2FBackupBigFiles%2FVMs%2FExample%20-%20AfterSpace%2F?prefix-only=false&folder-contents=true&time=2019-03-24T14%3A37%3A45%2B01%3A00&filter=%2Fvolume1%2FBackupBigFiles%2FVMs%2FExample%20-%20AfterSpace%2F

[…]
3. Status Code:
400 BadRequest: Invalid HTTP version in Request line. Line: GET /api/v1/backup/2/files/volume1/BackupBigFiles/VMs/Example - AfterSpace/?prefix-only=false&folder-contents=true&time=2019-03-24T14%3A37%3A45%2B01%3A00&filter=%2Fvolume1%2FBackupBigFiles%2FVMs%2FExample%20-%20AfterSpace%2F HTTP/1.1

19

Looked into it - as expected it was an deescaping-like problem with nginx-proxying and can be fixed as follows (at least it “works for me :tm:”):

  1. sudo vi /etc/nginx/conf.d/dsm.duplicati.conf
  2. change as follows (only last 2 rows) and save the file (ESC:wq):

Original

location ~ ^/webman/3rdparty/Duplicati/((.*)\.cgi|api/(.*))$ {
    proxy_set_header X-Server-IP $server_addr;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Real-HTTPS $https;
    proxy_set_header X-Server-Port $server_port;
    proxy_set_header X-Real-Port $remote_port;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;

    proxy_http_version 1.1;
    proxy_pass http://127.0.0.1:8200/$1$is_args$args;
}

Corrected

location ~ ^/webman/3rdparty/Duplicati/((.*)\.cgi|api/(.*))$ {
    proxy_set_header X-Server-IP $server_addr;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Real-HTTPS $https;
    proxy_set_header X-Server-Port $server_port;
    proxy_set_header X-Real-Port $remote_port;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;
    
    proxy_http_version 1.1;        
    set $query $1;
    proxy_pass http://127.0.0.1:8200/$query$is_args$args;
}
  1. sudo nginx -s reload

Problem solved :grin:

Additional info:

1 Like
20

Hi @naxan, welcome to the forum - and thanks for the great research into this issue!

Am I correct in assuming proxy_pass do tells nginx to pass through as-is any requests to the specified address?

Hopefully @thomas can give this a confirmation test and maybe we can mark this solved!

Oh - you wouldn’t happen to know if the .conf change survives Duplicati or DSM updates, would you?