hi i’m on linux mint 21.2 with duplicati 2.0.7.1_beta_2023-05-25 and i got this error:
how to fix it please ?
What are you trying to do? What exactly did you do?
EDIT:
Need destination information for failed download too.
Basically, situation needs a great deal of clarification.
sorry for missing infos.
initially i was doing a backup but got error passphrase so i tried to fix it by repair database and now i got this error.
i backup here: 1. /home/name/pCloudDrive/Backups/Lenovo and works great since some months but since yesterday no.
The log looks more like a Recreate. Below is English, but I think the button positions are the same.
If you look in the job log and all the old history is gone, you did a Recreate. Did you save older log?
Typically nobody does, but it sometimes is useful for history or if the database Recreate fails (here).
The Complete log below the Errors possibly has some more information on the Warnings/Errors.
Or you might have to rerun it with About → Show log → Live → Warning and likely click some lines.
Advanced options of log-file=<path> log-file-log-level=verbose or at least retry would likely add info.
Live log can do those too, but it gets long. Possibly you could try Retry level live, and watch results.
The FailedToRetrieveFile specifics would help. Your pCloud Drive may be going bad somehow.
You can probably try that file by hand to see if it’s available. If not, maybe get some pCloud support.
EDIT:
In Complete log, "BackupListCount" should be the number of dlist files on the destination. If a reasonably small number, maybe you can manually test the count matches, and they’re all readable.
Possibly that will clear up some pCloud problem, and Duplicati will be able to get them too next time.
Watching Verbose live log gives a nice view of file use. After dlist is dindex. Hopefully, no dblock.
yes i tried repair and didn’t work so i tried recreate (delete and repair) and finally got this error.
here full log in my job:
{
"MainOperation": "Repair",
"RecreateDatabaseResults": {
"MainOperation": "Repair",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-07T17:23:09.924992Z",
"BeginTime": "2024-02-07T17:00:59.250531Z",
"Duration": "00:22:10.6744610",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 1522,
"BytesUploaded": 0,
"BytesDownloaded": 872098700,
"FilesUploaded": 0,
"FilesDownloaded": 1516,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 4,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 0,
"KnownFileSize": 0,
"LastBackupDate": "0001-01-01T00:00:00",
"BackupListCount": 0,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": 0,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Repair",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-07T17:00:58.790374Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"ParsedResult": "Error",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-07T17:23:23.356034Z",
"BeginTime": "2024-02-07T17:00:58.790367Z",
"Duration": "00:22:24.5656670",
"MessagesActualLength": 3049,
"WarningsActualLength": 1,
"ErrorsActualLength": 1,
"Messages": [
"2024-02-07 18:00:59 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: L'opération Repair a débuté",
"2024-02-07 18:00:59 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2024-02-07 18:01:00 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (2,93 KB)",
"2024-02-07 18:01:05 +01 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 34 filelists",
"2024-02-07 18:01:05 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240108T120001Z.dlist.zip.aes (3,68 MB)",
"2024-02-07 18:01:08 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240108T120001Z.dlist.zip.aes (3,68 MB)",
"2024-02-07 18:01:08 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240109T120000Z.dlist.zip.aes (3,68 MB)",
"2024-02-07 18:01:09 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240109T120000Z.dlist.zip.aes (3,68 MB)",
"2024-02-07 18:01:17 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240110T120000Z.dlist.zip.aes (3,62 MB)",
"2024-02-07 18:01:20 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240110T120000Z.dlist.zip.aes (3,62 MB)",
"2024-02-07 18:01:25 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240111T120001Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:27 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240111T120001Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:32 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240112T120001Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:34 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240112T120001Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:39 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240113T120002Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:41 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240113T120002Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:46 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240114T120001Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:47 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240114T120001Z.dlist.zip.aes (3,70 MB)",
"2024-02-07 18:01:53 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240115T120000Z.dlist.zip.aes (3,71 MB)",
"2024-02-07 18:01:55 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240115T120000Z.dlist.zip.aes (3,71 MB)"
],
"Warnings": [
"2024-02-07 18:04:47 +01 - [Warning-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-FileProcessingFailed]: Failed to process file: duplicati-20240202T120002Z.dlist.zip.aes"
],
"Errors": [
"2024-02-07 18:04:47 +01 - [Error-Duplicati.Library.Main.AsyncDownloader-FailedToRetrieveFile]: Failed to retrieve file duplicati-20240202T120002Z.dlist.zip.aes"
],
"BackendStatistics": {
"RemoteCalls": 1522,
"BytesUploaded": 0,
"BytesDownloaded": 872098700,
"FilesUploaded": 0,
"FilesDownloaded": 1516,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 4,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 0,
"KnownFileSize": 0,
"LastBackupDate": "0001-01-01T00:00:00",
"BackupListCount": 0,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": 0,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Repair",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-07T17:00:58.790374Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
}
here full log in about local (button on the left of live):
8 févr. 2024 13:03: Failed while executing "Backup" with id: 4
System.Exception: Unexpected number of remote volumes marked as deleted. Found 0 filesets, but 1 volumes
at Duplicati.Library.Main.Database.LocalDeleteDatabase+<DropFilesetsFromTable>d__5.MoveNext () [0x0027e] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at System.Collections.Generic.LargeArrayBuilder`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] items) [0x00046] in <d22af090bceb4be792f53595cf074724>:0
at System.Collections.Generic.EnumerableHelpers.ToArray[T] (System.Collections.Generic.IEnumerable`1[T] source) [0x0003c] in <d22af090bceb4be792f53595cf074724>:0
at System.Linq.Enumerable.ToArray[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0002c] in <d22af090bceb4be792f53595cf074724>:0
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Data.IDbTransaction& transaction, System.Boolean hasVerifiedBackend, System.Boolean forceCompact, Duplicati.Library.Main.BackendManager sharedManager) [0x0018d] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired (Duplicati.Library.Main.BackendManager backend, System.Int64 lastVolumeSize) [0x000a5] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x01048] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x00050] in <9a758ff4db6c48d6b3d4d0e5c2adf6d1>:0
at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x00009] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Controller+<>c__DisplayClass14_0.<Backup>b__0 (Duplicati.Library.Main.BackupResults result) [0x0004b] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Controller.Backup (System.String[] inputsources, Duplicati.Library.Utility.IFilter filter) [0x00074] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00349] in <30a34d71126b48248d040dda634ddad9>:0
here full log (last entry only) in about live warning choice:
{"ClassName":"System.Net.WebException","Message":"Error: SecureChannelFailure (Unable to read data from the transport connection: Connection reset by peer.)","Data":null,"InnerException":{"ClassName":"System.IO.IOException","Message":"Unable to read data from the transport connection: Connection reset by peer.","Data":null,"InnerException":{"NativeErrorCode":10054,"ClassName":"System.Net.Sockets.SocketException","Message":"Connection reset by peer","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":null,"RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147467259,"Source":null},"HelpURL":null,"StackTraceString":" at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.ThrowException (System.Net.Sockets.SocketError error) [0x00007] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.GetResult (System.Int16 token) [0x0002c] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Threading.Tasks.ValueTask`1+ValueTaskSourceAsTask[TResult].<s_completionAction>m__0 (System.Object state) [0x00041] in <12b418a7818c4ca0893feeaaf67f1e7f>:0 \n--- End of stack trace from previous location where exception was thrown ---\n\n at Mono.Net.Security.MobileAuthenticatedStream.InnerRead (System.Boolean sync, System.Int32 requestedSize, System.Threading.CancellationToken cancellationToken) [0x0013f] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationToken) [0x000b5] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0008f] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x000a4] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00346] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x001f4] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001f5] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146232800,"Source":"mscorlib"},"HelpURL":null,"StackTraceString":" at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x00275] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x0015b] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebOperation.Run () [0x000b7] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebCompletionSource`1[T].WaitForCompletion () [0x000b1] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x00118] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () [0x0004d] in <ae4a5c68f78041539efc9420d056bd22>:0 \n at Duplicati.Library.Utility.AsyncHttpRequest.GetRequestStream (System.Int64 contentlength) [0x00068] in <ae4a5c68f78041539efc9420d056bd22>:0 \n at Duplicati.Library.UsageReporter.ReportSetUploader+<>c.<Run>b__3_0 (CoCoL.IReadChannel`1[T] chan) [0x000c4] in <9c9af2bf4d254b8bbad8328bd30e3a9d>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233079,"Source":"Duplicati.Library.Utility"}
here full log (last entry only) in about live verbose choice:
{"ClassName":"System.Net.WebException","Message":"Error: SecureChannelFailure (Unable to read data from the transport connection: Connection reset by peer.)","Data":null,"InnerException":{"ClassName":"System.IO.IOException","Message":"Unable to read data from the transport connection: Connection reset by peer.","Data":null,"InnerException":{"NativeErrorCode":10054,"ClassName":"System.Net.Sockets.SocketException","Message":"Connection reset by peer","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":null,"RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147467259,"Source":null},"HelpURL":null,"StackTraceString":" at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.ThrowException (System.Net.Sockets.SocketError error) [0x00007] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.GetResult (System.Int16 token) [0x0002c] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Threading.Tasks.ValueTask`1+ValueTaskSourceAsTask[TResult].<s_completionAction>m__0 (System.Object state) [0x00041] in <12b418a7818c4ca0893feeaaf67f1e7f>:0 \n--- End of stack trace from previous location where exception was thrown ---\n\n at Mono.Net.Security.MobileAuthenticatedStream.InnerRead (System.Boolean sync, System.Int32 requestedSize, System.Threading.CancellationToken cancellationToken) [0x0013f] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationToken) [0x000b5] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0008f] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x000a4] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00346] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x001f4] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001f5] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146232800,"Source":"mscorlib"},"HelpURL":null,"StackTraceString":" at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x00275] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x0015b] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebOperation.Run () [0x000b7] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.WebCompletionSource`1[T].WaitForCompletion () [0x000b1] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x00118] in <a85c1a570f9a4f9f9c3d2cfa5504e34f>:0 \n at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () [0x0004d] in <ae4a5c68f78041539efc9420d056bd22>:0 \n at Duplicati.Library.Utility.AsyncHttpRequest.GetRequestStream (System.Int64 contentlength) [0x00068] in <ae4a5c68f78041539efc9420d056bd22>:0 \n at Duplicati.Library.UsageReporter.ReportSetUploader+<>c.<Run>b__3_0 (CoCoL.IReadChannel`1[T] chan) [0x000c4] in <9c9af2bf4d254b8bbad8328bd30e3a9d>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233079,"Source":"Duplicati.Library.Utility"}
The term “full” or “complete” means compared to the pretty but less complete summary above it.
Messages saw the dlist downloads starting, but there’s a 20 line limit so we don’t see the bad one. Even if we did, this version gives little detail (improved in current Canary, but then line limit gets you).
This is why I asked for a run with a better capture.
is interesting though, although it doesn’t say what request got retried. Maybe the bad file stayed bad?
This is why I asked for some manual work with file.
These might not be relevant. Was live log running during a Recreate? I want details on a dlist error.
it’s that errors list you want ?
if yes i wait the recreate finish or bug and i’ll give you
i added this before start the recreate:
and actually i have nothing in the path i put in the settings
It’s already gotten past the dlist files, so maybe the bad file has decided to stop doing what it did…
That’s odd. You can put those directly in the job, but I think the defaults should have the same effect.
Duplicati sometimes is run as a different user than you, so make sure it has the access that it needs.
About → System info UserName says what Duplicati runs as.
Are you logging to pCloudDrive too? I suppose if Duplicati can backup to it, logging should also work.
Too late to change in the middle of a run, whatever’s going on.
username is the same so this is good.
wasn’t here yesterday, today i have this:
yes i’m logging to pcloud drive of course:
finally got my Lenovo.log in /home/coco/pCloudDrive/Backups/Lenovo.log (i put in link because too big 83mo)
Is this exactly what original post got on Feb 7, except on Feb 8 Recreate?
Verbose log got a more detailed view of it (below), but bad file stayed bad.
2024-02-08 19:20:37 +01 - [Verbose-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-ProcessingFilelistVolumes]: Processing filelist volume 27 of 36
2024-02-08 19:20:37 +01 - [Warning-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-FileProcessingFailed]: Failed to process file: duplicati-20240202T120002Z.dlist.zip.aes
System.Security.Cryptography.CryptographicException: Échec du déchiffrement des données (phrase de passe invalide ?) : Message has been altered, do not trust content ---> SharpAESCrypt.SharpAESCrypt+HashMismatchException: Message has been altered, do not trust content
at SharpAESCrypt.SharpAESCrypt.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00405] in <45c43198e80545aeabaa9b98839b4c7d>:0
at Duplicati.Library.Utility.Utility.CopyStream (System.IO.Stream source, System.IO.Stream target, System.Boolean tryRewindSource, System.Byte[] buf) [0x0003d] in <ae4a5c68f78041539efc9420d056bd22>:0
at Duplicati.Library.Encryption.EncryptionBase.Decrypt (System.IO.Stream input, System.IO.Stream output) [0x00008] in <41486abae9984475bc4414ccb638da87>:0
--- End of inner exception stack trace ---
at Duplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile.get_TempFile () [0x00008] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun (Duplicati.Library.Main.Database.LocalDatabase dbparent, System.Boolean updating, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00570] in <9cab5b8f1b4f49ec980acae8e278968b>:0
2024-02-08 19:20:37 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240203T120000Z.dlist.zip.aes (3,70 MB)
2024-02-08 19:20:37 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20240203T120000Z.dlist.zip.aes (3,70 MB)
2024-02-08 19:20:37 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20240204T120000Z.dlist.zip.aes (3,70 MB)
2024-02-08 19:20:37 +01 - [Verbose-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-ProcessingFilelistVolumes]: Processing filelist volume 28 of 36
After the bad file, it seems to pick right back up and continue, but bad file has done its damage.
Ordinarily this can be fixed from database data, but I guess the old database has been deleted.
AES Crypt can do a second test on the file. It’s an easy Windows tool. Can you get it on Linux?
If so, try to decrypt the dlist that Duplicati thinks is now corrupted. Use the Duplicati passphrase.
We’re not sure what shape your other pCloud Drive files are in, but you can survey all your files
as described in neighboring forum topic Error while running Synology backup, on damaged files.
One thing about your case that might be better than that one is dlist happens on each backup.
Throwing one away might not be that bad, for example rename it to not start with a duplicati-.
Testing entire backup would still be advised, but I’m not sure how well it will work at the moment.
You can certainly test a tiny (like 1 sample) of The TEST command in GUI, then maybe try more.
Unfortunately, due to assumed deletion of old database, I’m not sure how trustworthy new DB is.
EDIT: After some testing, I’m leaning towards it having enough records to find damaged dblocks.
Database Recreate tested the dlist and dindex files by actually trying to decrypt and use them.
SharpAESCrypt.exe is provided by Duplicati and can be invoked with ‘mono SharpAESCrypt.exe’.
So another way to test decryption if you don’t mind CLI. It might be easier than getting another tool up.
Following that, on the subject of many-paths, is there a way besides pCloud Drive to get the bad dlist?
If so, try it and see if files are the same. How does pCloud Drive use the cache storage? is maybe how
duplicati-20240202T120002Z.dlist.zip.aes escaped verification failure for a couple of backups.
Although the sample selection is somewhat unpredictable, it favors verifying new files, e.g. a new dlist.
If you can get a good copy of the bad file some other way, maybe pCloud Drive can try to keep it again.
2024-02-10 14:07:44 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-20240210T120000Z.dlist.zip.aes (3,70 MB)
2024-02-10 14:07:44 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-20240210T120000Z.dlist.zip.aes (3,70 MB)
2024-02-10 14:07:44 +01 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-DeleteRemoteFileset]: Deleting 3 remote fileset(s) ...
2024-02-10 14:08:43 +01 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
System.Exception: Unexpected number of remote volumes marked as deleted. Found 3 filesets, but 4 volumes
at Duplicati.Library.Main.Database.LocalDeleteDatabase+<DropFilesetsFromTable>d__5.MoveNext () [0x0027e] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at System.Collections.Generic.LargeArrayBuilder`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] items) [0x00046] in <d22af090bceb4be792f53595cf074724>:0
at System.Collections.Generic.EnumerableHelpers.ToArray[T] (System.Collections.Generic.IEnumerable`1[T] source) [0x0003c] in <d22af090bceb4be792f53595cf074724>:0
at System.Linq.Enumerable.ToArray[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0002c] in <d22af090bceb4be792f53595cf074724>:0
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Data.IDbTransaction& transaction, System.Boolean hasVerifiedBackend, System.Boolean forceCompact, Duplicati.Library.Main.BackendManager sharedManager) [0x0018d] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired (Duplicati.Library.Main.BackendManager backend, System.Int64 lastVolumeSize) [0x000a5] in <9cab5b8f1b4f49ec980acae8e278968b>:0
at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x00da9] in <9cab5b8f1b4f49ec980acae8e278968b>:0
I’m not having luck so far reproducing the final error of your backup, but here you got a new dlist uploading, which happens at the end of the backup of file changes. After that are additional things.
Retention policy gets applied, deleting old versions. Here it got confused, maybe by that dlist error.
If it hadn’t died then, it probably would have done a download verification of three files, and ended.
and is probably in /usr/lib/duplicati on Linux Mint.
$ /usr/lib/duplicati/SharpAESCrypt.exe
Usage: SharpAESCrypt e|d[o][1-4] <password> [<fromPath> [<toPath>]]
Use 'e' or 'd' to specify operation: encrypt or decrypt.
Append an 'o' to the operation for optimistic mode. This will skip some tests and leaves partial/invalid files on disk.
Append a single number (up to 4) to the operation to set the number of threads used for crypting. Default is single thread mode (1).
If you ommit the fromPath or toPath, stdin/stdout are used insted, e.g.:
SharpAESCrypt e 1234 < file.jpg > file.jpg.aes
Abnormal exit will return an errorlevel above 0 (zero):
4 - Password invalid
3 - HMAC Mismatch / altered data (also invalid password for version 0 files)
2 - Missing input stream / input file not found
1 - Any other cryptographic or IO exception
$
What pCloud applications are available? talks about pCloud for Web which might be another source. Although they have lots of ways to get files, you want one that’s plenty different from your local store.
i tried open the duplicati-20240202T120002Z.dlist.zip.aes file after copying in other location in aes crypt and i got this error (i’m sure about the password because it’s a copy/paste from 1password:
so this file seems completly dead.
after copied this file in other location i renamed the one in the real location to duplicati-20240202T120002Z.dlist.zip.aes.old like this it’s same as i deleted it but i keep save.
i restarted the “delete and repair” in database and this time duplicati don’t say any error and the icon in taskbar is black.
here the new /home/coco/pCloudDrive/Backups/Lenovo.log: Proton Drive
do i need start the backup job now or i wait the one tomorrow?
Backup any time you like. It will be nice to see one run, however
and we don’t know if any dblock files are. The recreate tested dlist and dindex.
The test all is one way. There’s also a script, utility-scripts/DuplicatiVerify.py
upload-verification-file can upload duplicati-verification.json for its use.
No. That’s a verification, but just one sample of three files, the same as the default after a backup.
Let me quote a bit:
and you already did the specific file test – next (maybe slow) test is to test everything sometime.
backup job today is done and duplicati icon in task bar is black so seems the problem is fixed?
thanks all for your help
We don’t know because you never tested the dblock files. Duplicati does not and can not test all files each time. That would be tremedously slow. It looks at the file sizes, but it does not download the entire backup.
EDIT: Technically, it can be configured to test all the files after any backup, but it wouldn’t make any sense. Your choices are either test, or hope that you have an intact backup to restore from when the need comes.
When the destination corrupts your backup files, you are in a very insecure situation, but it’s your choice…