Great! I am more and more suspecting the Mono-layer. When you have Duplicati up and running on Debian, it would be great if you could also try out the OD4B, as this backend never has worked on Debian/Raspbian, while it works perfect on Windows?
Note that this is a different command, not the usual one you’d use for commands like backup or test.
$ mono Duplicati.CommandLine.BackendTool.exe help
Usage: <command> <protocol>://<username>:<password>@<path> [filename]
Example: LIST ftp://user:pass@server/folder
Supported backends: aftp,amzcd,azure,b2,box,cloudfiles,dropbox,ftp,file,googledrive,gcs,hubic,jottacloud,mega,msgroup,onedrive,onedrivev2,sharepoint,openstack,rclone,s3,ssh,od4b,mssp,sia,tahoe,webdav
Supported commands: GET PUT LIST DELETE CREATEFOLDER
$
PMFBI, but I also think there may be a memory leak, although Windows 10 for me and running the GUI. Doing nothing ie all backups “waiting” one instance of Duplicati.GUI.TrayIcon.exe was using 110mb - close and restart Duplicati and it dropped to 17mb.
Is there one instance of the exe per job? There is a second instance of the exe, currently 9mb that never gets large; so I wonder if the other is for OneDrive v2 and hence my posting here.
The other is for pretty much everything. The first exe launches the base version of Duplicati from the original C:\Program Files\Duplicati 2. That version just has to look around, then start the latest version. We’ve probably lost the chance for direct comparison to OneDrive original, but to me, using memory is not the same as leaking memory. A leak grows forever, sort of like my 20,000 MB Firefox. Computers have to use memory, however ideally they either release unused or recycle for other use.
It could be leaking on all platforms, but it may be more evident on my smaller server with just 2 GB RAM?
Otherwise I’d bet on the Mono layer, and I searched a bit for clues, and found this:
“Memory leak in HttpWebRequest / HttpWebResponse when using a WebProxy #8689” https://github.com/mono/mono/issues/8689
Could be related, but not neccessarily. Couldn’t really figure out which version of Mono that had the problem, and which was fixed?
That sounds reasonable, else memory used would not have dropped when I ran a backup job. Hopefully, Windows will clean up memory as needed. I will try to keep an eye on it.
I attempted to verify that the fix is in release 5.16.0.179 by seeing main source change is there.
Ordinarily I’d say just run something newer (perhaps with similar source check for the change).
Error with Debian mono 5.18 #3620 makes me hesitate. Perhaps you can figure out that issue.
Another approach may be to go by bug and fix notes that a TLS handshake failure triggers bug.
Wireshark (if there’s enough memory for it…) could probably see if something’s off with yours.
Backing up further though, it’d still be nice to see if GET leaks (to narrow where the problem is).
Sorry for the lack of replies here… I gave up on OneDrive Personal as a backend, as my temporary upgrade is timing out. This would have been easy to verify on any Debian/Raspberry Pi installation.
Rickard
I believe I hit this same set of issues, and I believe this pull request will help fix this case.
When I added the OneDrive v2 backend, I didn’t hook up the --http-enable-buffering setting since I didn’t see a way to do that easily (since I used HttpClient instead of the existing HttpRequest infra).
In my case, the out of memory exception I saw was in the HttpClient code that received the response and copied the content into an in memory buffer before saying the async task was completed. It turns out there is an HttpCompletionOptions enum which has a value (not the default) which prevents caching the response in memory, so I hooked that up to the OneDrive v2 backend which should disable that caching.
2 Likes
This issue is still present for me on 2.0.5.103_canary_2020-02-18 on Debian
I actually just published a pull request to add an option to use the older, non-HttpClient based http methods (used by all other backends) to see if that helps the memory leak.