I am running Duplicati - 2.0.8.1_beta_2024-05-07 in a docker container.
When I now run a backup, I get the error shown in the title. I am backing up to an S3-compatible service (IDrive e2). The backups have been very successful (backing up >500Gb for 30 versions).
I’ve done a google search and there doesn’t appear to be a way to limit the request rate. Is that still true?
I cannot think of a simple way to limit request rate. You can increase the remote volume size, which will make fewer requests, or set an upload/download speed limit which will indirectly also reduce the request rate.
I guess the simple fix to this is to add some kind of --max-requests-pr-second=10 option and then simply pause the any backend operations until this limit can be observed. Is this what you would expect to happen?
Do you know what the rate limit rules are for IDrive e2?
When I searched on the service and exception, I found this. Others are seeing it too:
iDrive e2 error: Resource requested is unreadable, please reduce your request rate
Maybe I missed something but I couldn’t find a great description beyond what it says.
Default Duplicati starts with a list, then a series of uploads going up to four at once.
asynchronous-concurrent-upload-limit can make it one at a time, if that helps out any.
The other mentioned techniques, such as throttle-upload, might also reduce the rates.
About → Show log → Live – Retry will show what is being tried when the error occurs.
can possibly be tweaked to try to keep IDdrive e2 from complaining. Or maybe they’ll fix this?
the problem is with the idrive… they haven’t given an answer about this… is your bucket in Miami?
Yes it is. I have a support ticket open with idrive and they are checking into it
There is something very strange happening at iDrive’s Miami datacenter. This problem has been occurring for 4 days and they do not give further details… At this moment I fear losing all the data stored there
I have 2 tickets open but they don’t respond to emails
I had the same problem. It seems to have been resolved - at least the last couple of days the backup has succeeded without errors. I also opened a ticket. The first level responder said that they were forwarding it on to the backend team but the ticket hasn’t been updated since.
