I’m getting this error but I don’t know how to resolve. I run repair and next page says Finished but no files listed.
Here’s the text version it runs:
–auto-vacuum=true
–asynchronous-concurrent-upload-limit=1
–backup-test-samples=0
–list-verify-uploads=false
–no-auto-compact=false
–threshold=0
–backup-name=G
–dbpath=C:\Users.…\AppData\Local\Duplicati\ARBWAMRBSP.sqlite
–encryption-module=aes
–compression-module=zip
–dblock-size=50MB
–passphrase=…
–keep-versions=1
–disable-module=console-password-input
Execute a run and I get the same missing file error.
I tried purge and I see this:
Listing remote folder …
Missing file: duplicati-b094370756b6041dfbaeb381b751ddfde.dblock.zip.aes
Found 1 files that are missing from the remote storage, please run repair
ErrorID: MissingRemoteFiles
Found 1 files that are missing from the remote storage, please run repair
Return code: 100
It’s an endless cycle!
Is there a definitive set of sets to resolve this? How to prevent it?
Incremental backups
Duplicati performs a full backup initially. Afterwards, Duplicati updates the initial backup by adding the changed data only. That means, if only tiny parts of a huge file have changed, only those tiny parts are added to the backup. This saves time and space and the backup size usually grows slowly.
possibly helps, but Canary releases are always a bit of an unknown when initially released.
This one looks to be pretty good from test so far, however it’s not yet made its way to Beta.
There’s always a question of how much more stuff to keep throwing in, but I don’t decide it.
I sort of recall problems with Storj in the current Beta, but I don’t have a pointer to specifics,
beyond the pull request and release note that I mentioned. Maybe a Storj user knows more.
list-verify-uploads=true If a file upload fails for any reason, a final listing will catch it.
is what they suggested. That’s also mentioned in the cited pull request. Your config has it off.
There should still be a final check for missing files, but that’s too late to retry the uploading…
EDIT 2:
--threshold (Integer): The maximum wasted space in percent
As files are changed, some data stored at the remote destination may not
be required. This option controls how much wasted space the destination
can contain before being reclaimed. This value is a percentage used on
each volume and the total storage.
* default value: 25
I checked to see if 0 is special, e.g. equivalent to never. Doesn’t look like it per the help text.
What happens if you set the threshold too sensitive is it can be a tremendous compact load.
I’ve seen it get in loops somehow, maybe because it can never achieve what you told it to…
EDIT 3:
[Storj DCS] Update to latest uplink.NET #4755 wasn’t final update, but had larger discussion.
Update is now in Canary, but I’m not sure if we have heard feedback from users on it helping.
I tried making changes. None of those fix the missing files issue, right? I went to the Database setting and clicked Repair. There was an error “The backup storage destination is missing data files. You can either enable --rebuild-missing-dblock-files or run the purge command to remove these files. The following files are missing: duplicati-b094370756b6041dfbaeb381b751ddfde.dblock.zip.aes.” I tried both repair and purge with the flag set to true and that didn’t resolve anything.
Another thing I notice is that commands don’t stick. I changed threshold=25 and clicked “Run Backup now”. The command runs but says missing file. Confirmed the setting is in edit as text. When I go back to the edit page, all the settings are what they were before! ex threshold=0.
EDIT:
I have to edit the job itself not use the CommandLine option for the settings to stick.
If you have an occasional missing files issue, Recovering by purging files is the thing to do.
Adding console-log-level=information will help get back the headers lost in new log design.
The *-broken-files commands are for missing dblock files. If you lose dlist or dindex,
database information (assuming a good database) can upload those again with Repair run.
Removed automatic attempts to rebuild dblock files as it is slow and rarely finds all the missing pieces (can be enabled with --rebuild-missing-dblock-files).
and I think it was removed from Repair, but it’s not documented on that command. Help is:
--rebuild-missing-dblock-files (Boolean): Rebuild dblock files when missing
If dblock files are missing from the destination, you can attempt to
rebuild them using local source data. However, since the local data may
have changed, it may not be possible to retrieve all the required data
and the process may be slow. Use this option to attempt to rebuild
missing dblock files.
* default value: false
Developer input would be nice. The message shown appears weak in several different ways.
Yes. CommandLine is for one time runs like a true operating system command line would do.
For convenience over the OS version, it’s pre-populated from the job settings (only one way).
I’m seeing a similar problem since upgrading to v2.1.0.120 Canary, using a Google Drive destination for my backups.
I’m getting a message:
2025-07-08 13:55:30 -04 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Backup has failed
RemoteListVerificationException: Found 6145 files that are missing from the remote storage, please run repair
along with 6145 warnings that a particular file is missing. This is a backup that I’ve been running for years.
When I run Repair, I get:
2025-07-08 14:33:54 -04 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Repair has failed
FolderMissingException: The requested folder does not exist
with zero warnings.
I’ve tried to backup and repair under both v2.1.0.120 and v2.1.0.122, with the same result.
Note that my other five backups to the same destination ran successfully.
What were you running before? Was it running OK? Was upgrade just an install over old? Operations such as export/import add more potential error risk. Any other known change? Which GUI are you using? The new ngclient UI has had some job import and edit issues.
How many files are there supposed to be? You can look in your old job logs using the “Complete log” button for “KnownFileCount”. The log also has the old version number.
Please check “Folder path” in job is correct, has no odd special codings or a slash in front, then check Google Drive to verify folder exists.
What OS is computer with the problem? If other five backups are elsewhere, compare.
Are the AuthID the same for all backups? Do you know if you ever had to get a special AuthID for full access to Google Drive? That’s not the default, but could be used to get access to files that Duplicati didn’t create, for example if you had moved files manually.
For direct test, you can Export As Command-line and take URL to use in BackendTool. Running a list would be a good start. Be careful (or edit the URL) for other requests. Testing similarly with an Export from a working backup folder might reveal a difference.
Thank you for pointing me where to look. Indeed, 6145 was the count of all of the backup files on the Google drive (destination).
I found the problem: I think I had touched the name of the destination folder on the ngclient UI. The folder name was “MacBook Pro 16” when I created it on the ngax client, but touching it on the ngclient UI changed the destination to “MacBook%20Pro%2016”, which is technically the same, but instead of pointing to the original destination, Duplicati created a new, empty folder named literally “MacBook%20Pro%2016”, including the "%20"s as text of the name.
I re-edited the destination on the ngax client and all is well, but it appears that the substitution of “%20” for " " is not compatible with Google Drive.
BTW… the ngclient does not allow a capital letter for the first letter of the destination name. It converts it to lower case no matter what I enter. On the ngax client, I can capitalize the destination name, since the destination folder begins with a capital letter. I think that, however, it doesn’t seem to make a difference whether the destination name is capitalized or not.
Thank you for your detailed response and advice… Steve