Release: 2.0.4.5 (beta) 2018-11-28

That makes sense to me - and thanks for the link to the txt file. Maybe we should try to include that (the link) in not-canary release posts.

A “for a complete list of changes see…” link would make it obvious the Release post is just a summary and simplifies letting those who want more info get to the details.

Hi,

I think there could be an issue with the latest release - or it’s me. But I went to do a “system” backup (I have separate jobs for logs, DB, system, and email on a mail server - CentOS 7) and noticed that it was backing up the sqlite files under /root/.config/Duplicati/. Backup of these files did appear to error out:

Log data:
2018-12-03 13:38:26 -06 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-CheckingErrorsForIssue1400]: Checking errors, related to #1400. Unexpected result count: 0, expected 1, hash: elCQ7j+8fWDBZo+E9d6Vml58NRsFml+GZ+fNu4rQt74=, size: 102400, blocksetid: 319219, ix: 18, fullhash: YkKv7TqwdBuLs9dadt6ccZcTymUY3xZjncnybGIl4pc=, fullsize: 1968224
2018-12-03 13:38:26 -06 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-FoundIssue1400Error]: Found block with ID 12318357 and hash elCQ7j+8fWDBZo+E9d6Vml58NRsFml+GZ+fNu4rQt74= and size 37304
2018-12-03 13:38:26 -06 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: /root/.config/Duplicati/VQAOCWDRRT.sqlite-journal
System.Exception: Unexpected result count: 0, expected 1, check log for more messages
  at Duplicati.Library.Main.Database.LocalBackupDatabase.AddBlockset (System.String filehash, System.Int64 size, System.Int32 blocksize, System.Collections.Generic.IEnumerable`1[T] hashes, System.Collections.Generic.IEnumerable`1[T] blocklistHashes, System.Int64& blocksetid, System.Data.IDbTransaction transaction) [0x002fb] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 
  at Duplicati.Library.Main.Operation.Backup.BackupDatabase+<>c__DisplayClass13_0.<AddBlocksetAsync>b__0 () [0x00000] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 
  at Duplicati.Library.Main.Operation.Common.SingleRunner+<>c__DisplayClass4_0`1[T].<RunOnMain>b__0 () [0x00000] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 
  at Duplicati.Library.Main.Operation.Common.SingleRunner.DoRunOnMain[T] (System.Func`1[TResult] method) [0x000b0] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 
  at Duplicati.Library.Main.Operation.Backup.StreamBlockSplitter+<>c__DisplayClass2_0.<Run>b__0 (<>f__AnonymousType13`3[<Input>j__TPar,<ProgressChannel>j__TPar,<BlockOutput>j__TPar] self) [0x00a24] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 
  at Duplicati.Library.Main.Operation.Backup.StreamBlock.ProcessStream (CoCoL.IWriteChannel`1[T] channel, System.String path, System.IO.Stream stream, System.Boolean isMetadata, Duplicati.Library.Interface.CompressionHint hint) [0x0012c] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 
  at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor+<>c__DisplayClass1_0.<Run>b__0 (<>f__AnonymousType8`2[<Input>j__TPar,<StreamBlockChannel>j__TPar] self) [0x002b3] in <c6c6871f516b48f59d88f9d731c3ea4d>:0

Perhaps related, but I have/had a “filter” which should exclude “/root/.pcloud/” but I found that the backup today appeared to include files in this directory in the backup and I don’t believe it did under 2.0.3.3. Now, I’m not 100% sure but in case it was the trailing “/” causing an issue, I’ve changed the filter to be “/root/.pcloud” but have not run another backup to see if this makes any difference. If it doesn’t, it might be something like exclusion filters aren’t working properly under 2.0.4.5 - that would explain why it might be attempting to backup the duplicati sqllite databases.

Now, to complicate things slightly - I also upgraded mono today to 5.16.0.220 (from 5.16.0.187) so the “system” backup will be larger, but this is the first time I’ve ever seen errors regarding the backup of the duplicati databases…

Can you check your restore options and see if the Duplicati databases appear in backups from before the update to 2.0.4.5?

(Note that I edited your post by adding ~~~ before & after your error text to make it easier to read.)

Thanks for the cleanup. Yes, I do in fact see both /root/.config/Duplicati/*.sqllite and /root/.pcloud in my backups from a month ago so the filters were not working beforehand. The filters are configured as this when viewing as text:

-/root/.pcloud/
-/root/.config/Hashbackup/
-/pcloud//

Perhaps this is my bad, but I thought “exclude folder” would exclude the contents of the subdirectories. (I do not see Hashbackup in the list of restore files so that works - however, that path is a symlink to another partition which is excluded.)

Are you able to post your job exported “as command-line” (or at least the filters including the parameter name)?

--backup-name=System_B2 --dbpath=/root/.config/Duplicati/VQAOCWDRRT.sqlite --encryption-module=aes --compression-module=zip --dblock-size=100MB 
--retention-policy="1W:1D,4W:1W,12M:1M" --threshold=65 --small-file-max-count=3000 --disable-module=console-password-input --exclude=/root/.pcloud/ --exclude=/root/.config/Hashbackup/ --exclude="*/pcloud/*/"

Oh, I also ran another backup of the System - it also failed again on the sqllite files, but this time the backup took 17 minutes when normally the backup takes around an hour.

Well everything looks OK to me - and you are correct that EXCLUDE should ignore the folder and all subfolders.

Is the folder actually called .pcloud or pcloud (no dot-prefix)?

under /root/ it’s .pcloud:

[root@voyageur centos]# du -sh /root/.pcloud
5.2G    /root/.pcloud

I have the 3rd exclude there to exclude the mount point, /pcloud and the sources.

Another potential issue/difference I’m seeing in 2.0.4.5 - the space used by /root/.config/Duplicati/ has doubled - I now have twice as many sqlite files:

[root@voyageur Duplicati]# ls -l *sqlite
-rw-r--r-- 1 root root 7761055744 Dec  3 18:02 ATQJGACUOV.sqlite
-rw-r--r-- 1 root root 7761055744 Dec  3 11:00 backup 20181203060001.sqlite
-rw-r--r-- 1 root root      83968 Dec  3 11:36 backup 20181203113621.sqlite
-rw-r--r-- 1 root root   10265600 Dec  3 02:45 backup 20181203123747.sqlite
-rw-r--r-- 1 root root   22885376 Dec  3 00:20 backup 20181203123805.sqlite
-rw-r--r-- 1 root root  787825664 Dec  3 02:06 backup 20181203124427.sqlite
-rw-r--r-- 1 root root   10265600 Dec  3 12:38 CZFNGQMCYW.sqlite
-rw-r--r-- 1 root root      83968 Dec  3 19:10 Duplicati-server.sqlite
-rw-r--r-- 1 root root  806103040 Dec  3 19:10 VQAOCWDRRT.sqlite
-rw-r--r-- 1 root root   22885376 Dec  3 12:38 ZIKDBEFESL.sqlite

Can I delete the “backup YYYYMMDDHHMMSS.sqlite” files if I don’t intend to go back to 2.0.3.3?

Yes, those are safe to delete.

And yes, I think we need to inform users they are being created as they can take up quite a bit of space. :slight_smile:

I’m experiencing a couple of issues with Duplicati - 2.0.4.5_beta_2018-11-28. The first is cosmetic. Previously the UI would display how long the last successful session took to complete. After the update that information is no longer presented. (See attached photo)

The second issue has to do with ERRORS after a backup session. I get two (2) similar errors on nearly every backup, although the backup successfully completes. The errors look like this:

Errors: [
        2018-12-18 06:18:15 -05 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-CheckingErrorsForIssue1400]: Checking errors, related to #1400. Unexpected result count: 0, expected 1, hash: 24eFFzeHvBWhDRzT2+1LyIRFewnDo8EoKmvp4s95hgQ=, size: 88464, blocksetid: 142641, ix: 173, fullhash: cmulyvu4QvHCSDj21c+7G/ov8D0ncdod59yVONl3ReY=, fullsize: 17803664,
        2018-12-18 06:18:15 -05 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-FoundIssue1400Error]: Found block with ID 621173 and hash 24eFFzeHvBWhDRzT2+1LyIRFewnDo8EoKmvp4s95hgQ= and size 63840
    ]

Ideally I’d like for everything to run clean…successful, with no warnings, no errors. I’d like to get some insight on what’s going on with these errors. Any help you can provide is appreciated.

I’m running Duplicati on a MacBookAir with the latest version of macOS Mojave. I use Homebrew to install/update Duplicati’s dependancies.

Hello,

I just updated from
duplicati-2.0.2.1_beta_2017-08-01-x64.msi
->
duplicati-2.0.4.5_beta_2018-11-28-x64.msi

and I can’t start the duplicti application anymore.
All I see is the process popup twice in taskmanager. Then its gone.

this is the crash-log: C:\ProgramData\Duplicati\updates\Duplicati-crashlog.txt

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.FileNotFoundException: Could not load file or assembly 'CoCoL, Version=1.5.0.24305, Culture=neutral, PublicKeyToken=0983de3c914beeaa' or one of its dependencies. The system cannot find the file specified.
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper..ctor(String[] args)
   at Duplicati.GUI.TrayIcon.Program.RealMain(String[] _args)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Duplicati.Library.AutoUpdater.UpdaterManager.RunMethod(MethodInfo method, String[] args)

Any ideas?

I run the setup again. Choose Repair. Apparently the tray gui starts now.
The setup routine must have some strange problems on updates…

Let’s hope it will be doing the backups as it is supposed to do :slight_smile:

Hello @pesch, welcome to the forum!

It sounds like you’re seeing the same thing I did here where an re-install (not an in-app-update) seemed to resolve the issue.

Hi JonMikeIV,

thanks. I tried that, after I saw a similar post.
The in-app update does not work at all. I downloaded the installer and run it manually.

It looks like everything is fine now.

Glad to hear that worked for you!

Yes - the in-app updater seems to have broken with a recent release. We still haven’t identified / fixed the cause so until then manual installs (which will update your “base” version) or manual updates (unzipping the new version into the updates folder) will be necessary. :frowning:

For Windows users who have done the extra manual commandline work to run Duplicati as a Windows service, there are probably two default steps done for Tray Icon in the .msi install that you should change. Details here.

So far, I’ve updated two machines. I was able to download and install via the GUI. Then I clicked activate and it appeared nothing happened. So, I restarted the service, then refreshed the GUI, and it showed the updated version was installed. It’s easier than a manual install.

It appears that every backup set I setup using 2.0.4.5 beta running on macOS 10.13.6 do creates very long filenames of sqlite db.

Backups created in previous versions:
-rw------- 1 root wheel 1615063040 4 Jan 23:37 QAMITZPQQT.sqlite
-rw------- 1 root wheel 366998528 23 Nov 04:08 WEUUNULDGT.sqlite

created with 2.0.4.5_beta_2018-11-28
-rw------- 1 root wheel 2087827456 6 Jan 09:26 73809065877676817180.sqlite
-rw------- 1 root wheel 1240110080 6 Jan 09:33 83657969896684868069.sqlite
-rw------- 1 root wheel 513878016 5 Jan 07:20 86838871726786817187.sqlite

Is this OK by design or should I worry?

Heh - I asked the same thing. As far as I can tell it’s supposed to be this way, but I don’t know why it as changed.