Error during auto backup

Support
41

ok thanks

i started the purge and actually (still running) i got this:

Listing remote folder …
Uploading file (28,23 MB) …
Deleting file duplicati-20230604T120002Z.dlist.zip.aes …
Uploading file (28,23 MB) …
Deleting file duplicati-20230605T120002Z.dlist.zip.aes …
Uploading file (28,24 MB) …
Deleting file duplicati-20230606T120058Z.dlist.zip.aes …
Uploading file (28,25 MB) …
Deleting file duplicati-20230607T120002Z.dlist.zip.aes …
Uploading file (28,22 MB) …
Deleting file duplicati-20230608T120002Z.dlist.zip.aes …
Uploading file (28,22 MB) …
Deleting file duplicati-20230609T120002Z.dlist.zip.aes …

42

Did you have a particular question? Compared to the example (did you look?), it’s showing less, but example has the lines you quote. Initial investigation shows at least one of the omitted is at log level Information, so you can possibly get more detailed view at About → Show log → Live → Information.

43

was just to know if the command result seems good even if not finished

44

Probably good, but look in the live log as suggested to see if it looks any closer. I can’t give you comment on what’s not there, but except for the missing detail, the updates to the dlist files are expected and in the example. At the rate it’s going, it might update all your versions before we’re done discussing what’s up…

is one missing line I wish we’d seen (as per example), but this is 2017 code. Logging got redone, now

and I see a lot of Information level messages, so I guess you have to work harder for messages now.
console-log-level=information would have been good to have if one knew in advance it’d be helpful…

45

i have nothing in the live log:

image
image1917×447 30.6 KB

take time but infos arrived:

image
image1407×507 41.5 KB

46

Possibly a timing thing. I’m not sure if it can catch up with history if you start a live log late. The question might be why nothing more is coming out. You can certainly change log levels, e.g. to Verbose or even Profiling if you want to look for activity. Your status line seems to indicate there’s still something running.

image
image953×423 28.7 KB

is an example where I turned on live log and then did a purge-broken-files on a good backup. The Commandline window only got Listing remote folder ... and then it was done. Not helpful…

console-log-level defaults to warning. It looks better if I increase it to Information for example

The operation PurgeBrokenFiles has started
No broken filesets found in database, checking for missing remote files
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (11 bytes)
Skipping operation because no files were found to be missing, and no filesets were recorded as broken.

would have left the user less in the dark. For your current darkness, all there is to try is more live log.

EDIT:

Your later live log (after it started having output – another possibility is it had nothing to say for awhile) seems more plausible. It even has 70-some files affected by the lost dblock, sort of as was seen in list.

47

it’s good the log have many lines. i let you know after the purge if the backup job works or not

48

Console output is done a different way. The lines we were getting by default are probably from here:

It’s good to have noticed this. Sorry it had to be you. Maybe faster fix is to get the manual updated, meaning have it talk about console-log-level and explain why it helps, and post new sample output.

Code fix may be TBD. Some other operations managed to put at least basic info onto the console.
It’s sort of redundant with information level log though, so might perhaps be just some highlights…

49

I’m not sure myself if I follow correctly what you mean by that ? That if you start a backup say at 16:00:15, and you activate the live log at 16:00:17, does it display what happened between 16:00:15 and 16:00:17 ?

50

Yes. I know it doesn’t go back forever, but sometimes I’ve been surprised that it went back at all.
Or possibly the time that happens it was just a level change on a live log that was already going.
For example you try Information, get nothing, change to Profiling, and I thought can get old data.

51

To be sure of that, it would be necessary for the live log to display current date more precisely. I wonder if it would be difficult to do that. Not at all it turns out, only change ‘llll’ in AppUtils.js to something like ‘YYYY-MM-DD HH:mm:ss.SS’ after doing that, I don’t see anything going back before the current date after enabling the live log.
However if it has been enabled, then disabled, and enabled again it seems that it is indeed storing somewhere the log data and displaying it back.

52

the purge_broken_files ended and the backup job too and i got the same error :frowning:

53

Presumably not exactly the same, as you deleted first bad file. Did it maybe find a different bad file?
If so, it’s probably time to actually go look for all of them, e.g. with the test command as suggested.
You can look at the latest file name to see if it’s also one of the old files that compact tried to read in.

EDIT:

You can also take the name and verify that it won’t decrypt, using either of the methods given before.

EDIT 2:

AES Crypt GUI is easier but you have to install it. Example of SharpAESCrypt.exe CLI usage is here.
Look in the installation folder for its location. There’s also a utility-scripts folder with a PowerShell tool
which could be used, but I don’t know that it’s a whole lot simpler to run than using the test command.

And of course there’s also the start-fresh idea, but no guarantees that pCloud will work right with that.

54

no the same:
image

and the log:

4 juil. 2023 22:25: L’opération Backup a échoué avec l’erreur : Échec du déchiffrement des données (phrase de passe invalide ?) : Message has been altered, do not trust content
{“ClassName”:“System.Security.Cryptography.CryptographicException”,“Message”:“Échec du déchiffrement des données (phrase de passe invalide ?) : Message has been altered, do not trust content”,“Data”:null,“InnerException”:{“ClassName”:“SharpAESCrypt.SharpAESCrypt+HashMismatchException”,“Message”:“Message has been altered, do not trust content”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" à SharpAESCrypt.SharpAESCrypt.Read(Byte buffer, Int32 offset, Int32 count)\r\n à Duplicati.Library.Utility.Utility.CopyStream(Stream source, Stream target, Boolean tryRewindSource, Byte buf)\r\n à Duplicati.Library.Encryption.EncryptionBase.Decrypt(Stream input, Stream output)“,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nRead\nSharpAESCrypt, Version=1.3.3.0, Culture=neutral, PublicKeyToken=null\nSharpAESCrypt.SharpAESCrypt\nInt32 Read(Byte, Int32, Int32)”,“HResult”:-2146233296,“Source”:“SharpAESCrypt”,“WatsonBuckets”:null},“HelpURL”:null,“StackTraceString”:” à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()\r\n à Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)\r\n à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBackend, Boolean forceCompact, BackendManager sharedManager)\r\n à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)\r\n à Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()\r\n— Fin de la trace de la pile à partir de l’emplacement précédent au niveau duquel l’exception a été levée —\r\n à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n à Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.b__0(BackupResults result)\r\n à Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action1 method)\r\n à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)\r\n à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nget_TempFile\nDuplicati.Library.Main, Version=2.0.7.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile\nDuplicati.Library.Utility.TempFile get_TempFile()","HResult":-2146233296,"Source":"Duplicati.Library.Main","WatsonBuckets":null} 4 juil. 2023 22:25: Fatal error {"ClassName":"System.Security.Cryptography.CryptographicException","Message":"Échec du déchiffrement des données (phrase de passe invalide ?) : Message has been altered, do not trust content","Data":null,"InnerException":{"ClassName":"SharpAESCrypt.SharpAESCrypt+HashMismatchException","Message":"Message has been altered, do not trust content","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" à SharpAESCrypt.SharpAESCrypt.Read(Byte[] buffer, Int32 offset, Int32 count)\r\n à Duplicati.Library.Utility.Utility.CopyStream(Stream source, Stream target, Boolean tryRewindSource, Byte[] buf)\r\n à Duplicati.Library.Encryption.EncryptionBase.Decrypt(Stream input, Stream output)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nRead\nSharpAESCrypt, Version=1.3.3.0, Culture=neutral, PublicKeyToken=null\nSharpAESCrypt.SharpAESCrypt\nInt32 Read(Byte[], Int32, Int32)","HResult":-2146233296,"Source":"SharpAESCrypt","WatsonBuckets":null},"HelpURL":null,"StackTraceString":" à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()\r\n à Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)\r\n à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBackend, Boolean forceCompact, BackendManager sharedManager)\r\n à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)\r\n à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()\r\n--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---\r\n à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n à Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result)\r\n à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)\r\n à Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)\r\n à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)“,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nget_TempFile\nDuplicati.Library.Main, Version=2.0.7.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile\nDuplicati.Library.Utility.TempFile get_TempFile()”,“HResult”:-2146233296,“Source”:“Duplicati.Library.Main”,“WatsonBuckets”:null}
4 juil. 2023 22:25: Backend event: Get - Completed: duplicati-bcb85e7dfad2e4206824db471ea13398c.dblock.zip.aes (100,05 MB)
4 juil. 2023 22:25: Backend event: Get - Started: duplicati-bcb85e7dfad2e4206824db471ea13398c.dblock.zip.aes (100,05 MB)
4 juil. 2023 22:25: Failed to retrieve file duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes
{“ClassName”:“System.Security.Cryptography.CryptographicException”,“Message”:“Échec du déchiffrement des données (phrase de passe invalide ?) : Message has been altered, do not trust content”,“Data”:null,“InnerException”:{“ClassName”:“SharpAESCrypt.SharpAESCrypt+HashMismatchException”,“Message”:“Message has been altered, do not trust content”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:” à SharpAESCrypt.SharpAESCrypt.Read(Byte buffer, Int32 offset, Int32 count)\r\n à Duplicati.Library.Utility.Utility.CopyStream(Stream source, Stream target, Boolean tryRewindSource, Byte buf)\r\n à Duplicati.Library.Encryption.EncryptionBase.Decrypt(Stream input, Stream output)“,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nRead\nSharpAESCrypt, Version=1.3.3.0, Culture=neutral, PublicKeyToken=null\nSharpAESCrypt.SharpAESCrypt\nInt32 Read(Byte, Int32, Int32)”,“HResult”:-2146233296,“Source”:“SharpAESCrypt”,“WatsonBuckets”:null},“HelpURL”:null,“StackTraceString”:” à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()\r\n à Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)\r\n à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBackend, Boolean forceCompact, BackendManager sharedManager)\r\n à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)\r\n à Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()\r\n— Fin de la trace de la pile à partir de l’emplacement précédent au niveau duquel l’exception a été levée —\r\n à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n à Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.b__0(BackupResults result)\r\n à Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method)\r\n à Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)\r\n à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nget_TempFile\nDuplicati.Library.Main, Version=2.0.7.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile\nDuplicati.Library.Utility.TempFile get_TempFile()”,“HResult”:-2146233296,“Source”:“Duplicati.Library.Main”,“WatsonBuckets”:null}
4 juil. 2023 22:25: Backend event: Get - Failed: duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes (100,00 MB)
4 juil. 2023 22:25: Backend event: Get - Started: duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes (100,00 MB)
4 juil. 2023 22:25: Backend event: Get - Retrying: duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes (100,00 MB)
4 juil. 2023 22:25: Backend event: Get - Started: duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes (100,00 MB)
4 juil. 2023 22:25: Backend event: Get - Retrying: duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes (100,00 MB)
4 juil. 2023 22:25: Backend event: Get - Started: duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes (100,00 MB)

55

Different file. duplicati-b5fb72b02fd474b51a5ada216f3e72105.dblock.zip.aes newly seen as bad.
I think. Images are hard to search, but if there’s a previous mention posted, please point to that…
If exactly the same, it’d be about duplicati-b878bdb456f93454ca89844a2dd02f68a.dblock.zip.aes

Rest of the message looks like the same problem (not comparing stack in detail though). Backup presumably finished, decided it needs to compact still – because it didn’t manage to before either.
Compact needs to download some existing file with wasted space which is likely old (please look), discovers it’s corrupted (you can test using either of the two methods that I keep giving), thus fails.

Not much more to say here because I’ve been saying it repeatedly, but I can’t run your test for you. Questions? Need more help with how to do that, or deciding which way you would rather proceed?

56

error (in red) was same but not log.

i think i’ll restart my backup from 0 and lost the history but will be faster and i hope i won’t get the same problem.

thanks all for your help and patience