Release: 2.0.9.101 (Canary) 2024-06-27

2.0.9.101_canary_2024-06-27

This release is a canary release intended to be used for testing in preparation of a later stable release.

** Unlike regular canary builds, this one has a major change in the build system, so it now runs on .NET8. **

For that reason, the updater in previous canary builds does not detect this update yet, but this can be activated at a later time.

Please see release notes for v2.0.9.100 for a summary of changes.

Detailed lits of changes:

  • Removed obsolete platform detection code, thanks @Jojo-1000
  • Changed XMPP library to Artalk.Xmpp, thanks @Jojo-1000
  • Fixed an issue with rendering non-ascii
  • Cleaned up some library licenses
  • Fixed SQL quotes and added quote guard to SQL queries, thanks @Jojo-1000
  • Updated AWS region list
  • Fixed compare logic not working on Linux, thanks @Jojo-1000
  • Added logic to set AWS hostname in UI based on selected region
  • Removed deprecated S3 options --s3-european-buckets and --s3-use-rrs
  • Removed support for deprecated spelling of S3 credentials
  • Removed bundled GPG Windows binaries
  • Removed unsupported AlphaFS
  • Changed restore to not use local blocks by default
  • Updated Poli Systems endpoints, thanks @IIPoliII
  • Added Debian Arm7 packages
  • Updated CoCoL to latest .NET 8 version
  • Added support for arguments for scripts
  • Improved speed of the recovery tool, using in-memory lookup and added configurable open archive cache
  • Increasing the default number of archives tested to 0.1%
  • Updated SharpAESCrypt to v2. Note this deprecates threaded encryption, and adds padding validation. Set the environment variable AES_IGNORE_PADDING_BYTES=1 if you re-encrypt archives with an external AES Crypt tool.
2 Likes

Thank-you for this. I started by continuing to get Duplicati running on my RPi-4 still running the older OS, and although the installation was a success without any errors, I cannot get it to run.

The following packages will be upgraded:
  duplicati
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/56.1 MB of archives.
After this operation, 86.4 MB disk space will be freed.
Get:1 /home/pi/duplicati-2.0.9.101_canary_2024-06-27-linux-arm7-cli.deb duplicati armhf 2.0.9.101 [56.1 MB]
Reading changelogs... Done
(Reading database ... 68551 files and directories currently installed.)
Preparing to unpack .../duplicati-2.0.9.101_canary_2024-06-27-linux-arm7-cli.deb ...
Unpacking duplicati (2.0.9.101) over (2.0.8.1-1) ...
Setting up duplicati (2.0.9.101) ...
Processing triggers for mailcap (3.69) ...

Running from the command line I see:

root@OTTO:/home/pi# /usr/bin/duplicati-server --webservice-port=8200 --webservice-interface=any --webservice-sslcertificatefile=/usr/share/Duplicati/otto.mydomain.com.pfx --webservice-sslcertificatepassword=12345
A serious error occurred in Duplicati: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.DllNotFoundException: Unable to load shared library 'SQLite.Interop.dll' or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you're using glibc, consider setting the LD_DEBUG environment variable:
/usr/lib/duplicati/SQLite.Interop.dll.so: cannot open shared object file: No such file or directory
/usr/lib/duplicati/libSQLite.Interop.dll.so: cannot open shared object file: No such file or directory
/lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/duplicati/SQLite.Interop.dll)
SQLite.Interop.dll: cannot open shared object file: No such file or directory
/usr/lib/duplicati/libSQLite.Interop.dll: cannot open shared object file: No such file or directory

   at System.Data.SQLite.UnsafeNativeMethods.sqlite3_libversion()
   at System.Data.SQLite.SQLite3.get_SQLiteVersion()
   at System.Data.SQLite.SQLiteConnection.get_SQLiteVersion()
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimePropertyInfo.GetValue(Object obj, BindingFlags invokeAttr, Binder binder, Object[] index, CultureInfo culture)
   at System.Reflection.RuntimePropertyInfo.GetValue(Object obj, Object[] index)
   at Duplicati.Library.SQLiteHelper.SQLiteLoader.get_SQLiteVersion()
   at Duplicati.Server.Program.GetDatabaseConnection(Dictionary`2 commandlineOptions)
   at Duplicati.Server.Program.Main(String[] _args)

I tried installing sqlite3 and sqlite but it’s not helping. Any ideas?

Hmm, that file is supposed to be there:

How can I install that? Sorry, can’t tell from the link.

Is that why I see these errors in the log?


Jun 27, 2024 5:38 PM: Error in updater
System.Net.WebException: The remote server returned an error: (404) Not Found.
   at System.Net.HttpWebRequest.GetResponse()
   at System.Net.WebClient.GetWebResponse(WebRequest request)
   at System.Net.WebClient.DownloadBits(WebRequest request, Stream writeStream)
   at System.Net.WebClient.DownloadFile(Uri address, String fileName)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)

Although I don’t have the answer, I think “previous canary builds” means before 2.0.9.100.
The release note for that had the same words, but it no longer fits in 2.0.9.101, or at least
my 2.0.9.100 can find 2.0.9.101 when I do “Check for updates”. What release are you on?

image

EDIT:

What’s the Install button supposed to do in this new plan? It doesn’t seem to do anything.

I’m seeing these errors on system(s) that had 2.0.9.100, and it continues with 2.0.9.101. Here’s one example:

For this release the docker tag “canary” is capitalized… was that intentional?

Capture

I’m not sure if the update notifier looks for something similar to currently installed, but mine is duplicati-2.0.9.100_canary_2024-05-30-win-x64-gui.zip. I run several Duplicati independently.
duplicati-2.0.9.101_canary_2024-06-27-win-x64-gui.zip is the other one up in separate folder.

It seems that for the Windows x64 GUI MSI the “FORSERVICE” parameter no longer prevents the desktop icon from being deployed, but also doesn’t deploy the tray icon executable. When I upgraded from 9.100 to 9.101 it wasn’t there so the shortcuts no longer work.

I can work around the desktop icon as I do for many other apps I install by deleting it during deployment, but I still want the tray icon executable.

FYI, these two files were still present though:

Hmm, after extracting it from the ZIP version, when I try to run it I get this and nothing gets downloaded when I try it:

image

Plus, it’s already installed

Sorry for the sporadic reports, kind doing many things at the moment. I just noticed that the service wasn’t running and it too was because the executable was missing like the tray icon. So I reinstalled 9.100 and they came back, however the tray icon will no longer appear but the main service is fine now. When I run the tray icon manually nothing happens:

"C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe" --no-hosted-server --hosturl=https://lisa.mydomain.com:8200

Are both “it” the one file Duplicati.GUI.TrayIcon.exe? What else is in the same folder?

This is not my area of expertise, but when I try to run just that .exe, nothing happens.

Event log Application event says:

Description: A .NET application failed. Application: Duplicati.GUI.TrayIcon.exe Path: C:\tmp\Duplicati.GUI.TrayIcon.exe Message: The application to execute does not exist: ‘C:\tmp\Duplicati.GUI.TrayIcon.dll’.

so I move that into the same folder, try again, and get a popup like you show. Event:

Description: A .NET application failed. Application: Duplicati.GUI.TrayIcon.exe Path: C:\tmp\Duplicati.GUI.TrayIcon.exe Message: Cannot use file stream for [C:\tmp\Duplicati.GUI.TrayIcon.deps.json]: No such file or directory A fatal error was encountered. The library ‘hostpolicy.dll’ required to execute the application was not found in 'C:\Program Files\dotnet'. Failed to run as a self-contained app. - The application was run as a self-contained app because ‘C:\tmp\Duplicati.GUI.TrayIcon.runtimeconfig.json’ was not found. - If this should be a framework-dependent app, add the ‘C:\tmp\Duplicati.GUI.TrayIcon.runtimeconfig.json’ file and specify the appropriate framework.

If I run it in the full folder that the .zip file supplies, it works. A fast test is to ask it for help.

This sounds like the install that you thought wasn’t working had some other problems in it.

I’m not sure if it’s valid currently, but I used to compare .msi install folders against that .zip.
Possibly it’s still valid, although .zip now comes in a lot of flavors. Would it match installer?

Tested duplicati-2.0.9.101_canary_2024-06-27-win-x64-gui.msi by clicking in Explorer.
Got an Apps item for Duplicati (not Duplicati 2. I’m not sure what’s expected now.
Got about 87 missing files compared with the content of the .zip file. Some evidence:

C:\Program Files\Duplicati 2>dir Duplicati.GUI.TrayIcon*
06/27/2024  11:43 AM           202,880 Duplicati.GUI.TrayIcon.deps.json
06/27/2024  11:44 AM           107,672 Duplicati.GUI.TrayIcon.dll
06/27/2024  11:44 AM           333,976 Duplicati.GUI.TrayIcon.Implementation.dll
06/27/2024  11:43 AM               445 Duplicati.GUI.TrayIcon.runtimeconfig.json

C:\Program Files\Duplicati 2>dir *.exe
05/17/2024  05:38 PM            65,872 createdump.exe
06/27/2024  11:44 AM           160,408 Duplicati.CommandLine.AutoUpdater.exe
06/27/2024  11:44 AM           160,408 Duplicati.CommandLine.ConfigurationImporter.Implementation.exe
06/27/2024  11:44 AM           160,408 Duplicati.CommandLine.SharpAESCrypt.exe
06/27/2024  11:44 AM           160,408 Duplicati.CommandLine.Snapshots.exe

C:\Program Files\Duplicati 2>dir Duplicati.Library.*.dll
06/27/2024  11:44 AM            54,936 Duplicati.Library.AutoUpdater.dll
06/27/2024  11:44 AM            34,456 Duplicati.Library.Backend.AliyunOSS.dll
06/27/2024  11:44 AM            32,920 Duplicati.Library.Backend.Idrivee2.dll
06/27/2024  11:44 AM            43,160 Duplicati.Library.Backend.Storj.dll
06/27/2024  11:45 AM            29,336 Duplicati.Library.Backends.dll
06/27/2024  11:44 AM            58,008 Duplicati.Library.Common.dll
06/27/2024  11:45 AM            43,160 Duplicati.Library.Encryption.dll
06/27/2024  11:44 AM           283,800 Duplicati.Library.RestAPI.dll
06/27/2024  11:44 AM            82,072 Duplicati.Library.Snapshots.dll

I did not uninstall the previous Duplicati before the install. I don’t know if that might matter.

EDIT 1:

Clicked the .msi in Explorer again, got some choices, chose Repair. That fixed things up.
There are minimal differences compared to .zip. I still have two old service installer files.
package_id_type.txt files are different, but that seems OK. This one says win-x64-gui.msi

EDIT 2:

TrayIcon comes up now, though it didn’t seem to get a new tab in my open Edge browser.
I kind of expected it to be on port 8400 (third Duplicati at the same time). It was, and said:

image

which is probably just a leftover notification in the server database, and a clue to prior use.
I wonder if these old notifications should be purged, as I think the updater is now different?

I had the same issue on one of my Windows machines. In place upgrade (using MSI) over 2.0.9.100. Many missing files. I ended up uninstalling/installing clean.

Early preview of .Net8 builds, codename 2.0.8.105 has some data, commentary, and cure.
I don’t know if the cure got backed out or wasn’t effective. From my missing files info above,
the appearance to me is that the things that survived are the new files (not already present).

Sorry, I meant to follow up on that, but got sidetracked with the Kestrel update :/.

Anyway, I think this is something related to the RPI causing errors.
I tried with Docker to reproduce (I have the armv7 package in the CWD):

docker run -it --platform linux/arm -p 8200:8200 -v.:/packages  debian:latest bash

Then running inside docker:

apt update -y && apt install -y libicu72 libssl3
dpkg -i /packages/duplicati-2.0.9.101_canary_2024-06-27-linux-arm7-cli.deb
duplicati-server --webservice-port=8200 --webservice-interface=any

This starts up just fine, and creates the SQLite database, indicating no problems with SQLite missing or being broken.
I crashes due to some memory issue, but I suspect this is Docker related.

Since I cannot reproduce your issue, can we perhaps rule out a packaging isssue? I have the following files:

> ls -lah /usr/lib/duplicati/*ite*
-rw-r--r-- 1 root root 215K May 20 04:03 /usr/lib/duplicati/Microsoft.AspNetCore.Rewrite.dll
-rw-r--r-- 1 root root  35K May 20 04:03 /usr/lib/duplicati/Microsoft.Extensions.FileProviders.Composite.dll
-rw-r--r-- 1 root root  91K May 16  2020 /usr/lib/duplicati/SQLite-net.dll
-rw-r--r-- 1 root root 2.6M Apr  8 10:09 /usr/lib/duplicati/SQLite.Interop.dll
-rw-r--r-- 1 root root  22K Jun 27 11:43 /usr/lib/duplicati/SQLiteHelper.dll
-rw-r--r-- 1 root root 1.6K Feb 13 21:38 /usr/lib/duplicati/SQLiteHelper.dll.config
-rw-r--r-- 1 root root 6.0K May  1  2020 /usr/lib/duplicati/SQLitePCLRaw.batteries_v2.dll
-rw-r--r-- 1 root root  46K May  1  2020 /usr/lib/duplicati/SQLitePCLRaw.core.dll
-rw-r--r-- 1 root root 5.5K May  1  2020 /usr/lib/duplicati/SQLitePCLRaw.nativelibrary.dll
-rw-r--r-- 1 root root  56K May  1  2020 /usr/lib/duplicati/SQLitePCLRaw.provider.dynamic_cdecl.dll
-rw-r--r-- 1 root root 369K Aug 25  2021 /usr/lib/duplicati/System.Data.SQLite.dll
-rw-r--r-- 1 root root  71K May 17 17:46 /usr/lib/duplicati/System.Diagnostics.TextWriterTraceListener.dll
-rw-r--r-- 1 root root  55K May 17 17:47 /usr/lib/duplicati/System.Resources.Writer.dll
-rw-r--r-- 1 root root  22K May 17 17:35 /usr/lib/duplicati/System.Xml.ReaderWriter.dll
-rw-r--r-- 1 root root 742K Apr  7  2020 /usr/lib/duplicati/libe_sqlite3.so

I also have an RPI4 to test with, but I need to know what OS version you installed?

No, you should not see those errors anymore. The current canary requests latest-v2.manifest and the beta requests latest.manifest, both files are present and return status code 200.

What urls do you see reported by Duplicati.CommandLine.AutoUpdater.exe / duplicati-autoupdater ?

It is supposed to open the download page:

https://duplicati.com/download-dynamic?channel=canary&from=2.0.9.100

I think the text should be changed to “Download”.
Do you see any errors in the console or logs when clicking “Install” ?

No, I will change it to be backwards compatible with the previous system.

The data is there in the manifest to point to the correct (or at least the same kind), but it does not currently use this. The channel name is embedded in the binaries during build, so it will look for canary builds unless overridden.

Unfortunately, the wixl tool does not currently support toggling the features (read: shortcuts) via conditions, so it will actually install the shortcuts. The missing file sounds like either an MSI quirk or an AV tool has quarantined it. I have not experienced this file as missing in any of my tests.

@Taomyn Is it reproducible for you?

The .NET runtime is bundled with the executables so you do not need to download it. Are you sure you have “unblocked” the zip file before extracting?

I think crashes here are not reported to the commandline, but can be found in the Windows Event Log. Do you see any indications of why it does not start up?

Yes, that part is still the same. The build process now builds the binaries first:

  • win-x64-gui
  • win-x64-cli
  • linux-x64-gui

After the binaries are built, the package tool uses the same source to produce the packages (zip, msi, dmg, deb, rpm, docker, etc). The installed contents of *-win-x64-gui.zip and *-win-x64-gui.msi should be exactly the same and all files should have the same hashes.

Yes, that is how the installation knows what install package you used, and this is intended to be used to guess what updated package you would like (but it is not used just yet).

I think there is something quirky going on with the MSI package. Ideally, it should just remove the old stuff and copy in the new. But for some reason it occasionally seems to install first and then remove afterwards, causing everything to be broken. Sadly, I have not managed to find a way to reproduce it.

Ideally yes, the update messages should be removed.

Yes, I think the error is that the installer somehow removes the files after installation. It seems to have something to do with what version of Duplicati that is being upgraded from.

As mentioned in the 2.0.8.105 thread, I tried several variations of installing 2.0.8.1 and upgrading without being able to replicate the issue. Any hints as to how I can reproduce this are appreciated. My only idea at this point is to try to install a chain of updates and reboot in between the installations.

I compared the file list against mine and they are the same.

The RPi-4 is my Octoprint server, which I install using the official Pi imager then go to “Other Specific Purpose OS” > “3D printing” > “OctoPi” and then the “stable” version. After that and restoring the Octoprint settings, I install Duplicati as normal but I have to add the “mono” repo which is:

deb [signed-by=/usr/share/keyrings/mono-official-archive-keyring.gpg] https://download.mono-project.com/repo/debian stable-raspbianbuster main

It’s just one line that I save to “/etc/apt/sources.list.d/mono-official-stable.list”

Re: MSI, yes I tried several times, there is nothing being blocked by Defender and with the ZIP the file is unblocked, but that’s only a problem if the file is access by an automation, as I use WinRAR it didn’t care. I’ll have to see when I have time and re-try the install to check out the event log - I’ll also try searching for the last time I tried, and report back.

I could only find event logs for the tray icon .exe, nothing for the main server:

Description: A .NET application failed.
Application: Duplicati.GUI.TrayIcon.exe
Path: C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe
Message: A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\Program Files\dotnet\'.
Failed to run as a self-contained app.
  - The application was run as a self-contained app because 'C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.runtimeconfig.json' did not specify a framework.
  - If this should be a framework-dependent app, specify the appropriate framework in 'C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.runtimeconfig.json'.

Application: Duplicati.GUI.TrayIcon.exe
CoreCLR Version: 8.0.123.58001
.NET Version: 8.0.1
Description: The process was terminated due to an unhandled exception.
Exception Info: System.IO.FileNotFoundException: Could not load file or assembly 'Duplicati.Library.Interface, Version=2.0.9.101, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Duplicati.Library.Interface, Version=2.0.9.101, Culture=neutral, PublicKeyToken=null'
   at Duplicati.GUI.TrayIcon.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.Net8.Program.Main(String[] args)

Faulting application name: Duplicati.GUI.TrayIcon.exe, version: 2.0.9.101, time stamp: 0x66470000
Faulting module name: KERNELBASE.dll, version: 10.0.20348.2520, time stamp: 0x72149c73
Exception code: 0xe0434352
Fault offset: 0x000000000003f17c
Faulting process id: 0x2eb0
Faulting application start time: 0x01dac933646eef02
Faulting application path: C:\Program Files\Duplicati 2\Duplicati.GUI.TrayIcon.exe
Faulting module path: C:\WINDOWS\System32\KERNELBASE.dll
Report Id: 4526fbb9-bdf7-4966-92a5-6ada97323d1b
Faulting package full name: 
Faulting package-relative application ID: 


Still seeing the error if I do the Check for Updates from the web UI. Have to go to the Show Log / Stored section to see it.

C:\Program Files\Duplicati 2> Duplicati.CommandLine.AutoUpdater.exe
Usage:
        Duplicati.CommandLine.AutoUpdater.exe [CHECK|DOWNLOAD|HELP]

Environment variables:

AUTOUPDATER_Duplicati_SKIP_UPDATE - Disables updates completely
AUTOUPDATER_Duplicati_URLS - Use alternate updates urls
AUTOUPDATER_Duplicati_CHANNEL - Choose different channel than the default Canary, valid settings: Stable,Beta,Experimental,Canary,Nightly,Debug

Updates are downloaded from: https://updates.duplicati.com/canary/latest-v2.manifest;https://alt.updates.duplicati.com/canary/latest-v2.manifest
Machine settings are installed in: C:\Users\xxx\AppData\Local\Duplicati
This version is "2.0.9.101_canary_2024-06-27" (2.0.9.101) and is installed in: C:\Program Files\Duplicati 2


C:\Program Files\Duplicati 2> Duplicati.CommandLine.AutoUpdater.exe check
No updates found

Unclear if the command line version also got 404 and just spit out “No updates found” since it failed. Would it show a 404 error here?

I can manually download either of those manifests using wget or web browser just fine. Is Duplicati not parsing the multiple URLs correctly? Maybe it’s doing a single web request instead of splitting at the semicolon…