New to Duplicati, running version 2.0.2.17_canary_2018-01-23 on a QNAP TS-453A NAS on QTS 4.3.4, connecting to DreamHost DreamObjects. I have also tested this with Dropbox as well, and the same results explained below apply – so I am highly doubtful the issue is with DreamHost.
Duplicati is only uploading at a max of 10MB/second. I realize that MB and mbps are very different, however 10MB’second (around 80mbps) is not even 1/10th of my upload speed.
I have tried setting the throttle to 120MB (960mbps) and Duplicati never surpasses 10MB. In fact, it slowly drops from 10MB to 0 and then goes back to 10MB/s constantly.
How do I fix this? There absolutely has to be something wrong here. Any tips would be greatly appreciated. I am trying to upload 5+ TB of data and that’s gonna take me about 23 days at the speed it’s going.
I have pretty much the exact issue. TS-853A (same model as yours but with 8 bays) with 3.25TB of data to upload via SFTP and its going to a local client so bandwidth isn’t an issue yet Duplicati goes from 0 to 10MB/s and back down. I’m averaging around 3MB/s.
How do you know that Duplicati is uploading at 10MB/s? Have you tried another way to upload data to your provided (DreamHost DreamObjects) to make sure this is not a hosting service limitation? There’s no throttling in Duplicati unless set by user, so basically it will upload as fast as the destination can download. Can you confirm that it isn’t related to the provider. Cheers.
yes, I can absolutely confirm that it is not related to DreamHost DreamObjects. I can easily upload via SFTP client (Cyberduck) at 30+MB/s, so I know it’s not them. This is specifically related to Duplicati.
I also tested with Dropbox (completely different provider) with the exact same issue. And I know it’s transferring at 10MB/s or lower because of the interface:
I’m not entirely sure that the interface is showing the (correct) upload speed. I have seen it once going down every minute when it was actually not uploading anything.
If this system is using a dual-core CPU? If so, then it seem that one core is fully occupied. My guess is, that Duplicati is running on one core and fully occupies it with hashing. In that case, you cannot speed it up more at the moment.
My servers quad core acts similarly. It sits at around 90% usage on two cores while backing up - even while just processing files and not requiring internet.
I guess the speed metric is interesting for predicting when the backup is done but very misleading for showing actual network usage even though it could be confused as an indication of that.
@Andrew_Kelly Also try fiddling with the size and number of dblock files to keep your upload humming. Set it to something larger like 1GB and have 10 files queued for upload if you have the space in the temp directory. It sounds like your upload speed is a lot quicker than the actual backup speed.
Also I’m secretly working on LZ4. But don’t tell anyone, cause I spent more than a month on a proof of concept for the LZ4 frame format and it’s still not entirely done.
