Database rebuild halts with Rclone backend

bug

#1

Helllo,

I am using Dupliati 2x .35 version(windows service mode) with Rclone backend to One drive for business.

Backups work good but Database recreate stops at db size 1,256 kb.
There is no rclone process in Task manager.
The WUI is stuck for days.
Clicking stop don’t work.
The only way is to kill all processes in task manager, start over & get stuck at the same place again.

I have tested this to at least 2 systems.
I am using .35 version intentionally because if find its more stable than the others but it fails here.

Any help.


#2

Yes - 2.0.3.5 canary is likely more stable than the 2.0.3.6 - 2.0.3.9 canaries, so that’s a good place to be.

I’m a little confused what - if the backups are working, why did you start a database Recreate?

You won’t see an rclone process in Task Manager as Duplicati doesn’t actually run rclone, it just uses the same API calls that rclone does.

Most likely clicking stop would work if you let it sit long enough. Duplicati is probably running a very slow step so won’t notice the stop request until it’s done with it’s current step. As you found, killing a database Recreate means starting over from the beginning.

While database recreates have been known to be very slow for some people (we still aren’t sure why) it could also be that you’re having issues with your destination (OD4B).

When the Recreate is running, if you go to the job “Show log” page and click the “Remote” tab do you see any get commands, just a mostly blank page, or a “database is locked” message?

You could also try looking at the main menu “About” -> “Show log” page then clicking on the “Live” tab and selecting “Profiling” to watch if any commands are running even though the database isn’t getting any larger.


#3

No it doesn’t. It actually starts a rclone process for every put or get operation. If you do not see a rclone process, it could be because duplicati is performing other operations.


#4

The reason for using rclone for od4b is it bypasses the 5k files limit per folder

Because I switched systems

It’s blank


#5

Whoops - my mistake, thanks for educating me! :blush:

I didn’t realize that limit was there - thank you, for sharing that!

That’s very odd - there should be SOMETHING there…


So let’s assume I’m totally wrong and that the issue is indeed because the job is set to use rclone. Can we isolate that for confirmation?

For example, if you changed the destination type to regular OD4B to try the recreate that might be a start (though it sounds like with the 5k files-per-folder limit that may not be an option).

Another option would be to download all the “duplicate-*” files from OD4B to somewhere local, point Duplicati there, and then run the recreate. Afterwards you could then point it back to OD4B (sort of a reverse seed process). Of course I realize you probably have a LOT of content backed up so this probably isn’t very viable in your case either.


#6

I have other backup sets configured with od4b with Duplicati files below 5k rebuild works good there. The problem seems to be with rclone & Duplicati.

In this case I used rclone to download the full backup repository, switched Duplicati destination to local drive to rebuild database & it worked out.


#7

I’m glad you were able to get around the issue.

I’m leaving this flagged as a bug (so it gets seen by developers) but I am going to set your post as the “solution” in case other people run into the same problem before it gets fixed.

Please let me know if you disagree.