Recovery Issues: 350 GB, Stopped Working After 190 GB Without Error (Version 2.0.7.1)

Support
1

I attempted to restore 350 GB of images in one go. Since I had previously restored my Windows 10 Pro system using Veeam, the database was not completely up to date. What would have been the best option to ensure it pulls the current database from the server?

In any case, I proceeded with the restoration using the exported configuration. Everything went smoothly until only about 160 GB were left, at which point it stopped, and I had no way to get it running again, not even with the pause and resume options.

Here are the last log entries from the process:

  1. Feb 2025 11:50: CommitBlockMarker took 0:00:00:00.000
  2. Feb 2025 11:50: Starting - CommitBlockMarker
  3. Feb 2025 11:50: Recording metadata from remote data: D:\Bilder\mila\Handy A32\Camera\VID_20210824_142513.mp4

Unfortunately, it seems there is no way to resume the restore exactly from that point. I have now downloaded and installed the latest version (duplicati-2.1.0.109_canary_2025-02-11-win-x64-gui.msi) and started the restoration again from the configuration file. However, it did not recognize that there was a previous ongoing action.

There is also a kind of error. During the process, the status shows “no scheduled task,” which was better in the old version. In the previous version, it indicated that something was happening. A look at the log files (ExplicitOnly) reveals that a lot is indeed happening.

Additionally, there is no option during the restore to select that it only backs up missing files, which would save some time.

Is there a way to do this, perhaps through the console? Did I make a mistake in the process?

2

This will cause errors due to the discrepancy, and Repair is prone to make it worse (fix coming).
Add safety check to repair that prevents deleting unknown new backups #4968

If by “server” you mean “Destination”, there’s no database copy there, but you could recreate DB.

The status bar in 2.1 sometimes gets stuck and needs browser refresh. Please see if that helps.

Assuming “backs up” means “restores”, this is almost what happens automatically, meaning a minimum amount of actual restoring is done. Totally missing files obviously will need restores. Existing files are scanned to see how they look. Any blocks found that are wrong get restored.

I suggest you see if browser refresh gets status bar more active, then see how restoring goes. Restore was rewritten for 2.1, and I think it still says less to live log about how it’s progressing.

One can certainly see action, but you’re at a level that makes lots of noise. You could try a less noisy level. Verbose used to be good, but I’m not sure you’ll see something short and readable.

Old restore method is still available, but you must ask (something also stopped in it on your try).

3

THX

Thank you, that was already a consideration of mine :slight_smile:

By server I mean the OneDrive backup destination from where I want to restore. There must be a database copy.

In version 2.0.8.1 it is also better than in the current version. I’m running it on another PC and have started it again with the same restore.

I don’t know if I’ve understood it correctly. When setting up the restoration of data, there is the option of selecting whether existing files should be overwritten or given a different name. I would like it if there was the option to skip. And if this option is selected, that the restore process checks before downloading whether the files already exist with the hash value, if at all possible. It’s more of a feature. In my case, it was that the process hung in any case. Nothing had happened for hours. The only thing I can do now is simply restore everything.

4

There isn’t, unless you took steps of your own to put one there. You should recreate the DB.

If “it” means 2.0.8.1, fine, but if you have a 2.1 version, please see if browser refresh helps it. Hopefully you even get the restore progress bar that I think the old restore code used to give.

If you’re talking about a “better” that doesn’t mean “not stuck”, please explain what’s different.

No option is needed, as AFAIK it always checks. If the file is completely correct, no download.

You posted that for 2.0.7.1, then found through the live log the details of the underlying activity.

You then installed 2.1.0.109. What are you talking about now, and did you view similar live log?

EDIT:

You can also look in Task Manager to see what sort of CPU and disk activity Duplicati is seeing.

Restore is not a file copy. It means make everything correct per that version if it isn’t already OK.

I don’t know current situation in terms of Duplicati version, browser refresh, live log (varied level), resource utilization, etc., so until that gets stated there’s not much more I can say about situation.

OneDrive can also just stop and refuse to give files, if it thinks usage is too high, and requests to slow down were not respected. Duplicati got more respectful at some point, but maybe using an older version caught forced delay. Log at information level should be enough to see download attempts, however live log might not go back in history. It’s intended for looking at “live” activities.

5

Other possibilities:

“Restore from configuration” can mangle options as noted here, but 2.0.7.1 run got far enough that probably it did not suffer from that. If you see options on 2.1.0.109 with four dashes, that’s the bug.

To bypass the issue, trim dashes down to only two on screen 2 of Restore from configuration.

Release: 2.1.0.109 (Canary) 2025-02-11 - New user interface might differ. See that to switch back. Canary releases are always testing releases with new fixes and features, but potentially new bugs.

I’ve already linked to directions on how to get to the old restore code, if you think it might do better.

Awaiting more information on your situation.

EDIT:

I tried a small restore (with a database) on 2.1.0.107 with About → Show log → Live → Verbose

Feb 20, 2025 3:26 PM: The operation Restore has completed
Feb 20, 2025 3:26 PM: Volume decryptor retired
Feb 20, 2025 3:26 PM: Volume decryptor retired
Feb 20, 2025 3:26 PM: BlockManager Volume consumer retired
Feb 20, 2025 3:26 PM: Volume downloader retired
Feb 20, 2025 3:26 PM: Volume downloader retired
Feb 20, 2025 3:26 PM: Volume decompressor retired
Feb 20, 2025 3:26 PM: Volume decompressor retired
Feb 20, 2025 3:26 PM: Volume manager retired
Feb 20, 2025 3:26 PM: BlockManager Block handler retired
Feb 20, 2025 3:26 PM: File processor retired
Feb 20, 2025 3:26 PM: Restored file C:\backup source\short.txt
Feb 20, 2025 3:26 PM: Backend event: Get - Completed: duplicati-bfa3e8c668d0049c08c25aba7cefa6992.dblock.zip (769 bytes)
Feb 20, 2025 3:26 PM: Backend event: Get - Started: duplicati-bfa3e8c668d0049c08c25aba7cefa6992.dblock.zip (769 bytes)
Feb 20, 2025 3:26 PM: BlockManager Block handler retired
Feb 20, 2025 3:26 PM: File processor retired
Feb 20, 2025 3:26 PM: Restore list contains 2 blocks with a total size of 255 bytes
Feb 20, 2025 3:26 PM: Needs to restore 1 files (118 bytes)
Feb 20, 2025 3:26 PM: Searching backup 0 (2/8/2025 2:04:07 AM) ...
Feb 20, 2025 3:26 PM: Backend event: List - Completed: (3 bytes)
Feb 20, 2025 3:26 PM: Backend event: List - Started: ()
Feb 20, 2025 3:26 PM: The operation Restore has started

so you should be able to see at least some of the action – assuming that things are still moving.

If I do the same restore again, this is how it tells me: After looking over files, no restore needed:

image
image473×154 11.6 KB

however what I don’t like is the live log telling me differently. For example, it is telling me about:

Feb 20, 2025 3:32 PM: Needs to restore 1 files (118 bytes)
and
Feb 20, 2025 3:32 PM: Restored file C:\backup source\short.txt

however it might be talking about metadata settings (timestamps, etc.) and not the file contents.

6

I went and did the restore on another computer at a different location that has a better internet connection. I’m probably expressing myself incorrectly here, but the database is first retrieved from the backup during the restore, isn’t it? At the moment it is only about 140GB that needs to be restored and it is still running diligently. I had also restored another backup the night before with almost a terabyte, also via OneDrive. That seems to have gone through. The last time I looked at it, it was already checking the completed restore again. Unfortunately, there were many other messages about backup tasks afterwards, so the message that the restore was complete was lost. But I have not seen any error message about this. The computer is running v2.0.8. The first attempt with 2.0.7 is probably not worth investigating, because the version is already old and v2.0.8. is really great. I’m looking forward to 2.1.0 and have already installed it on 2 other clients for testing.

I was a bit confused by the fact that in the restore options you have to specify that existing files are either overwritten or given a slightly different file name. But I now understand that Duplicati only applies this if the files are different. Maybe I overlooked that somewhere in the stess. I think it would be great if this was described more clearly for people like me :slight_smile:

At the moment I am experiencing on the first PC, with 2.1.0.109_canary_2025-02-11, that canceling the restore does not work. I went through all the options. The log file also showed no more changes. So I have now stopped the Duplicati task on the Windows computer and started it again.

I will test the restore later with the stable version. Then I can observe the window again to see how the status behaves.

7

Again, there is none to retrieve, however a partial temporary database is created for restore.
This is explained on the screen IIRC.

OK. No possible issues with new Restore code or new GUI with that, so forget any mentions.
This should also show Verbose level logging, maybe more than the newer Restore will show.
You might see messages about “Patching” (or similar). That means writing blocks as needed.
The old restore code downloads dblock files one at a time and spreads their block around, so sometimes you’ll see files showing up that aren’t the right size. Wait until the Restore finishes.

Timestamps are also set at the end, so those will be wrong for awhile.

I’m not sure what the current state of cancel is. I don’t think it was ever universally possible.
Possibly the developer will comment? If no other stop is possible, killing the process will do.
This is a little safer on recent versions than it once was, plus you’re not in middle of backup.

It might be easier to add --log-file=<path> --log-file-log-level=verbose, but it’s your call.
GUI is basic, but you can add those on two lines on screen 2 of Restore from configuration.

If the job config is up to date except for stats, you can probably just run a Database Recreate. Adding the mentioned Advanced options for more logs can be done with the usual GUI job Edit.

This DB Recreate will take longer because it’s rebuilding a complete database, not a partial DB. Ultimately that’s probably what you want, if the system is going to keep doing things in Duplicati.