Hi,
i have an Error while using the CommandLine in Windows:
.\Duplicati.CommandLine.BackendTester.exe --list-broken-files "ftp://omv:21/Backup_HTPC
/Duplicati?auth-username=Backup&auth-password=Backup" --no-encryption=true
Starting run no 0
Unittest failed: System.Net.WebException: Der Remoteserver hat einen Fehler zurückgegeben: (450) Datei nicht verfügbar (z.B. in Gebrauch).
bei System.Net.FtpWebRequest.DataStreamClosed(CloseExState closeState)
bei System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
bei System.Net.FtpDataStream.Dispose(Boolean disposing)
bei System.IO.Stream.Close()
bei Duplicati.Library.Utility.OverrideableStream.Dispose(Boolean disposing)
bei System.IO.Stream.Close()
bei System.IO.StreamReader.Dispose(Boolean disposing)
bei System.IO.TextReader.Dispose()
bei Duplicati.Library.Backend.FTP.<List>d__20.<>m__Finally1()
bei Duplicati.Library.Backend.FTP.<List>d__20.System.IDisposable.Dispose()
bei Duplicati.Library.Interface.BackendExtensions.TestList(IBackend backend)
bei Duplicati.CommandLine.BackendTester.Program.Run(List`1 args, Dictionary`2 options, Boolean first)
bei Duplicati.CommandLine.BackendTester.Program.RealMain(String[] _args)
I have tested some Files: i can download them.
Is there a way to find out which file is causing problems?
Oh, i used the wrong one… there are so many different Duplicati.CommandLine-Variants and i don’t watched over the auto-completion.
The GUI-Interface is, to say in german: Flitzekacke.
I change the Command from “backup” to “list-broken-files”, but the GUI doesn’t change the Commandline arguments and Parameters (Directorys to Backup, Filters,…). I have to change then manually every f*cking time from the begining at a typo or so.
And yes, there is a reason why i’m using list-broken-files. Some time ago, one of the backup disks had a file system error that caused some of the duplicates to get lost:
Found 9 files that are missing from the remote storage, please run repair
and
The backup storage is missing data Files. You can activate either enable “–rebuild-missing-dblock-files” or run the purge-command to re…[???]
But now i have tryed the correct CommandLine (i hope so), but there is another error:
Do you usually use Duplicati from the Web UI? If so, you need to specify the same database (find it at Database option) if at a Command Prompt. To get all backup options easily, Export as Command-line. Although it’s more work to setup, typos and other changes are more easily edited in after an up-arrow.
rebuild-missing-dblock-files is not a command but an option, and not very well documented on the web.
C:\Program Files\Duplicati 2>Duplicati.CommandLine.exe help rebuild-missing-dblock-files
--rebuild-missing-dblock-files (Boolean): Rebuild dblock files when missing
If dblock files are missing from the destination, you can attempt to
rebuild them using local source data. However, since the local data may
have changed, it may not be possible to retrieve all the required data
and the process may be slow. Use this option to attempt to rebuild
missing dblock files.
* default value: false
C:\Program Files\Duplicati 2>
You can use that as an option on the repair command, and (not certain) maybe the backup implicit repair.
I have a feeling we’ll wind up going beyond the repair command, but the above might solve syntax issues.
Hi,
a “few days” later i worked on this Problem and solved it for another Problem.
Yes, for BackUps and BackUp configuration i use the Web UI. For other things is the CommandLine the better way and your export-hint worked very well.
I have restored the function with the “purge-broken-files” command. But now i have some warnings:
2019-03-25 16:05:22 +01 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-b028528b218dc4e949f2c76a8a20f0193.dblock.zip is listed as Verified with size 33554432 but should be 33667136, please verify the sha256 hash "r1tBSnIqbbg005T0a0K5UuVXuJgTcZPPzAiLh+DUT+4=", 2019-03-25 16:05:22 +01 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-i9d5f7a60a2ed452ca85dff3f88aa055c.dindex.zip is listed as Verified with size 36012 but should be 36188, please verify the sha256 hash "8fsHw8opp0vDkNQ9SElASeMGaApyXitZtEArQzQNQdc=",
Were those two files with the reduced sizes always on the FTP destination, put there by Duplicati directly?
What OS is the destination? Does the FTP server keep any logs that would show FTP history for the files?
You mentioned a filesystem error. I wonder if these two files could have suffered damage, however usually when files shrink, I find their sizes go down to a nice even binary size (like filesystem may do). Yours didn’t.
What yours did do is shrink by about 1 in 200 or 300 characters, so I almost wonder if FTP worked in ASCII mode instead of binary mode for some reason. FTP has reliability drawbacks and ways to make short files, however if I read Duplicati code correctly, I think this says the files were originally right-sized, but not now…
You could look at the dates on those files, and if you still have a good local database, the dates are inside it along with other information like the expected file size and hash value that you’re getting complaints about.
You can sanity-test the files yourself by trying to decrypt with SharpAESCrypt.exe or AES Crypt. Directions. Damaged dindex files can be replaced if database is sound, but damaged dblocks are lost source file data.
Could you share some information about your main goal? Are you trying to restore due to lost source data? Especially when the remote backup files are damaged (with likely data loss), doing repairs can face issues.