Early preview of .NET8 builds, codename `2.0.8.109`

Releases
1

As announced in the latest canary/experimental/beta, the plan is to move to .NET8 builds going forward.
For those that want to help out or try the very latest, I have prepared a set of builds that use the .NET8 build & package process.

I hope (once again) that this will be the last debug build and that we will be able to release canary builds going forward.

This build has a these fixes reported on the previous build:

  • Improved startup robustness for Avalonia
  • Long-form version name in system info
  • Improved username on Windows in system info
  • Removed Mono fallback support for SSH backend
  • Fixed missing username+password for OpenStack, thanks @Jojo-1000
  • Improved handling of forced locale, thanks @Jojo-1000
  • Improvments to source file picker, thanks @Jojo-1000
  • Options to disable quota checks, thanks @Jojo-1000
  • Fixed recovery tool issues when restoring multiple sources and restoring empty files, thanks @Jojo-1000

The packages are signed (Windows + MacOS) and the GPG signatures are included.

Upgrade notice

Since RC4 encryption is no longer part of the open-source SQLite libraries, the builds will detect and remove the RC4 encryption before connecting to the database. If your security setup previously depended on this feature, make sure to adjust.

The file naming structure:

The filenames are named after operating system, chip architecture and intended use, e.g. linux-x64-gui.
The -gui builds also include the -cli executable, but are generally bigger due to the UI libraries.

The suffix denotes the installer type:

  • .zip: Generally just the files, zipped.
  • .deb: Debian package
  • .rpm: RPM package
  • .pkg, .dmg: MacOS packages
  • .msi: Windows Installer

Package files

Preparation for Canary release

I am building a list of notices and things that are ā€œgood to knowā€ for the users upgrading to the canary.

So far I have this list of notable changes from previous canary build:

  • Changed runtime to .NET8, shipping with dependencies, no more Mono or .NET install required
  • Changed from portable executable to operating system and CPU architecture dependent packages
  • Server database with SQLite is no longer ā€œencryptedā€ with RC4, running a new canary or later will decrypt the database
  • The executable Duplicati.Library.AutoUpdater.exe is renamed to Duplicati.CommandLine.AutoUpdater.exe and has limited functionality
  • The Windows MSI installer has been reduced, it offers essentially no options for now and no UI
  • The updater has been rewritten to rely on installing the packages, instead of having the ā€œin-placeā€ updates
    • There are now longer a spawned secondary process
    • The updates folder can be deleted
    • Installing an update now requires manually downloading and running the installer (you still get a notice)
  • To fit better with non-Windows systems the executables on non-Windows are all lower-case and prefixed with duplicati:
    • Duplicati.GUI.TrayIcon.exe ā†’ duplicati
    • Duplicati.CommandLine.exe ā†’ duplicati-cli
    • Duplicati.RecoveryTool.exe ā†’ duplicati-recovery-tool
    • ā€¦ and similar for others

These are mostly based on discoveries done by @ts678 and if there are more things that should be listed, please add a comment.

A note on the canary release process

The new builds uses an updated manifest file, meaning that the .NET4 canary users will not be immediately notified, even though there is a .NET8 canary build available. I can toggle that later by uploading a fresh manifest file for .NET4 that points to the update page. In other words: releasing a .NET8 canary build will not affect any .NET4 installations, except for those using Docker images to update.

I have created a new board with issues that I am working on. I have not followed up on the switch to Kestrel, but I plan to follow up, and I also think the feedback on MSI was that it was too basic for what people used it for.

1 Like
2

Broken new board link above maybe meant https://github.com/duplicati/duplicati/projects

Windows 10 .zip file install seems fine so far here.

One possible concern in all the recent releases is About ā†’ Libraries lacks the Aliyun OSS information,
however thatā€™s how the pull request was made. Ditto for licenses folder. Source tree thirdparty has info.

3

The link is correct, but the board was marked ā€œPrivateā€; it is now public, thanks for pointing it out.

4

Good catch. I will fix it.
Edit: PR is ready

5

Iā€™m not familiar with this, but it looks like the signatures arenā€™t in that, only the sign-key.txt.
Inside the latter is a URL that doesnā€™t work after a long try. Sometimes it gives proxy error
but on another system it was ā€œNo results foundā€. The link in original post here seems OK:

The new format of the download page finally made me check hashes. I was worried that
mismatches occurred, then I noticed page used Base64. Is that typical for hash listings?

6

OpenSUSE TumbleWeed .rpm install fails in YaST2 with Nothing provides ā€˜libICEā€™
It gives me the choice of continuing without that and breaking Duplicati, then I get:

Error: INVALID:duplicati-2.0.8.109-debug.x86_64 (file-b957739c): Signature verification failed [6-File is unsigned]
Header SHA256 digest: OK
Header SHA1 digest: OK
Payload SHA256 digest: OK
MD5 digest: OK
Package header is not signed!

which seems different than the expected breakage from missing things, however Iā€™m not sure about that.

EDIT 1:

It looks like it should be available. I might be doing something wrong. Iā€™m not on this system very muchā€¦

libICE

and I see some directions to tryā€¦ but the one I tried failed as:

localhost:~ # zypper addrepo https://download.opensuse.org/repositories/openSUSE:Factory/standard/openSUSE:Factory.repo
File '/repositories/openSUSE:Factory/standard/openSUSE:Factory.repo' not found on medium 'https://download.opensuse.org/'
Abort, retry, ignore? [a/r/i/...? shows all options] (a):

EDIT 2:

libICE looks like a new dependency, compared to .NET 4, says rpm --query --requires on .rpm.
I looked in GUI SOFTWARE Management and it looks like only libICE6 is available ā€“ and is installed.

EDIT 3:

Linux Distros in Avalonia FAQ mentions libICE6. I donā€™t know how this differs from libICE though.
Whatā€™s more worrisome on that page is that Wayland support is not out. Wonā€™t this be an issue?

EDIT 4:

SharpAESCrypt.exe is missing, at least from the .zip install. There is a SharpAESCrypt.dll there.

EDIT 5:

Missing on .msi install too, and .deb equivalent is SharpAESCrypt is missing. Will it link from /usr/bin?

As a documentation note, I guess instead of confusing people on non-Windows systems with .exe files sometimes needing to be shell-invoked with mono and sometimes not, executable names are variable.

EDIT 6:

This should have some execute permissions, right? From the .deb install:

$ ls -l Duplicati.Service
-rw-r--r-- 1 root root 72520 May 16 08:11 Duplicati.Service
$ file Duplicati.Service
Duplicati.Service: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=f41ff499f9f18f8c74413c4b50ad412e05168693, stripped
$

EDIT 7:

Seems like it got confused by loss of .exe too:

EDIT 8:

Duplicati.Library.Snapshots.dll ships on .msi,. .deb, and Windows .zip, but the executable is missing.
Because this is a VSS tool (I think), Windows should provide a .exe and others should have nothing?

EDIT 9:

Noncritical issue similar to documentation note above is built in help naming themselves with a .exe:

$ ./Duplicati.CommandLine

See duplicati.commandline.exe help <topic> for more information.

$ duplicati-configuration-importer help
Unhandled exception. System.ArgumentException: Incorrect number of input arguments.  Usage: ConfigurationImporter.exe <configuration-file> --import-metadata=(true | false) --server-datafolder=<folder containing Duplicati-server.sqlite>
   at Duplicati.CommandLine.ConfigurationImporter.ConfigurationImporter.Main(String[] args)
   at Duplicati.CommandLine.ConfigurationImporter.Net8.Program.Main(String[] args)
Aborted (core dumped)

Duplicati.RecoveryTool help canā€™t be tested for output because it crashes. Help works on 2.0.8.1 Beta.

$ ./Duplicati.CommandLine.RecoveryTool help
Program crashed: 
System.ArgumentNullException: Value cannot be null. (Parameter 'stream')
   at System.IO.StreamReader..ctor(Stream stream, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean leaveOpen)
   at Duplicati.CommandLine.RecoveryTool.Help.Run(List`1 args, Dictionary`2 options, IFilter filter)
   at Duplicati.CommandLine.RecoveryTool.Program.Main(String[] _args)
$
7

and in that case, the main point is Library to CommandLine, but for non-Windows .exe files, .exe is dropped, presumably not because it had to be, but to look more like a native executable on that system. This impacts documentation, scripts, and help text, although many languages (such as C) let a program know how it was invoked, so help text could just use that to describe itself. I noticed some help text didnā€™t even name a name.

Program aliases such as /usr/bin symbolic links might also benefit from this. I started this as one name, but got help output with another, rather than using how I invoked it. I donā€™t know if this adds portability issues too.

root@LinuxMint21:/usr/lib/duplicati# ./Duplicati.CommandLine.AutoUpdater help
Usage:
	duplicati-autoupdater [CHECK|DOWNLOAD|HELP]

Environment variables:

AUTOUPDATER_Duplicati_SKIP_UPDATE - Disables updates completely
AUTOUPDATER_Duplicati_URLS - Use alternate updates urls
AUTOUPDATER_Duplicati_CHANNEL - Choose different channel than the default Debug, valid settings: Stable,Beta,Experimental,Canary,Nightly,Debug

Updates are downloaded from: https://updates.duplicati.com/debug/latest-v2.manifest;https://alt.updates.duplicati.com/debug/latest-v2.manifest
Machine settings are installed in: /root/.config/Duplicati
This version is "2.0.8.109_debug_2024-05-16" (2.0.8.109) and is installed in: /usr/lib/duplicati

root@LinuxMint21:/usr/lib/duplicati# ls -l /usr/bin/duplicati-autoupdater
lrwxrwxrwx 1 root root 50 May 16 08:31 /usr/bin/duplicati-autoupdater -> ../lib/duplicati/Duplicati.CommandLine.AutoUpdater
root@LinuxMint21:/usr/lib/duplicati#

EDIT 1:

For the manual, you might be able to just drop the .exe part, as at least some (needs a further look) parts of Windows allow one to run an .exe file without typing the .exe part. That might be usual manual style as well.

EDIT 2:

To bypass the .rpm install failure, I did a .zip install, then went looking for the executables, which I expected would not have a .exe on them, but that plan didnā€™t work. I listed files, and noticed that this form of install is seemingly installing them under what would be symbolic link names in /usr/bin. ./duplicati did come up, but didnā€™t actually make a tray icon that I could find (Iā€™m not that familiar with OpenSUSE Tumbleweed though).

$ ./duplicati
MESA: error: ZINK: vkCreateInstance failed (VK_ERROR_INCOMPATIBLE_DRIVER)
glx: failed to create drisw screen
failed to load driver: zink

Hereā€™s the whatā€™s-it-called problem again:

$ ./duplicati-cli help

See duplicati.commandline.exe help <topic> for more information.
  General: example, changelog
  Commands: backup, find, restore, delete, compact, test, compare, purge, vacuum
  Repair: repair, affected, list-broken-files, purge-broken-files
  Debug: debug, logging, create-report, test-filters, system-info, send-mail
  Targets: aliyunoss, aftp, azure, b2, box, cloudfiles, dropbox, ftp, file, gcs, googledrive, e2, jottacloud, mega, msgroup, onedrivev2, openstack, rclone, s3, ssh, od4b, mssp, sharepoint,
  sia, storj, tahoe, cos, webdav
  Modules: aes, gpg, zip, 7z, console-password-input, http-options, hyperv-options, mssql-options, runscript, sendhttp, sendxmpp, sendmail
  Formats: date, time, size, encryption, compression
  Advanced: mail, advanced, returncodes, filter, filter-groups, <option>

https://duplicati.com/                 Version:  - 2.0.8.109_debug_2024-05-16




$ ./duplicati.commandline.exe help
bash: ./duplicati.commandline.exe: No such file or directory
$ ./duplicati.commandline help
bash: ./duplicati.commandline: No such file or directory
$

EDIT 3:

Because most non-Windows OS are case-sensitive, help text should be too. This might be an old bug, however on a .deb install, the help text shows its problem of not knowing what it should be calling itself:

root@LinuxMint21:/usr/lib/duplicati# ./Duplicati.CommandLine

See duplicati.commandline.exe help <topic> for more information.
  General: example, changelog
  Commands: backup, find, restore, delete, compact, test, compare, purge, vacuum
  Repair: repair, affected, list-broken-files, purge-broken-files
  Debug: debug, logging, create-report, test-filters, system-info, send-mail
  Targets: aliyunoss, aftp, azure, b2, box, cloudfiles, dropbox, ftp, file, gcs, googledrive, e2, jottacloud, mega, msgroup, onedrivev2, openstack, rclone, s3, ssh, od4b, mssp, sharepoint, sia, storj, tahoe, cos, webdav
  Modules: aes, gpg, zip, 7z, console-password-input, http-options, hyperv-options, mssql-options, runscript, sendhttp, sendxmpp, sendmail
  Formats: date, time, size, encryption, compression
  Advanced: mail, advanced, returncodes, filter, filter-groups, <option>

https://duplicati.com/                 Version:  - 2.0.8.109_debug_2024-05-16




root@LinuxMint21:/usr/lib/duplicati# ./duplicati.commandline.exe help example
-bash: ./duplicati.commandline.exe: No such file or directory
root@LinuxMint21:/usr/lib/duplicati#

EDIT 4:

Regarding installability of .rpm, I guess the target would be back to Centos 7 and forward to latest Fedora?
Unfortunately, Iā€™m all out of .rpm based systems now, and I donā€™t know who else has been testing on them.

Regarding the naming confusions, once the eventual design intent is settled, release note can cover it and possibly even talk about future polishings once initial Canary is out ā€“ unless youā€™d rather tweak before then.

Especially for a Canary, some Errata seem reasonable, and wider exposure will likely uncover more things.

8

The recovery tool does not run properly after compiling, it looks like help.txt is not included in the assembly:

System.ArgumentNullException: Value cannot be null. (Parameter 'stream')
   at System.IO.StreamReader..ctor(Stream stream, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean leaveOpen)
   at System.IO.StreamReader..ctor(Stream stream)
   at Duplicati.CommandLine.RecoveryTool.Help.Run(List`1 args, Dictionary`2 options, IFilter filter) in C:\Daten\CS\Projects\duplicati\Duplicati\CommandLine\RecoveryTool\Help.cs:line 40
   at Duplicati.CommandLine.RecoveryTool.Program.Main(String[] _args) in C:\Daten\CS\Projects\duplicati\Duplicati\CommandLine\RecoveryTool\Program.cs:line 100

It could also use a null check for GetManifestResourceStream:

Edit:
The resource is still there, but the assembly name is Duplicati.CommandLine.RecoveryTool.Implementation, while the resource is stored with Duplicati.CommandLine.RecoveryTool. The normal CLI has a hardcoded resource name, so the problem did not occur there.

9

If there are requests for an OpenSUSE desktop RPM, we can look into it. Technically, we should provide a package for each operating system and version, but the maintenance is too much at this point.

It appears to me that libICE6 is the Debian name, and other distros use the much simpler version-free name libICE which prevents all the issues associated with libICU versions.

I assume the Avalonia team has a better track of what the future will look like, so until it is an issue, I would like to stick with Avalonia. There are multiple alternatives that will work for Duplicati, and the design is deliberately to not rely too deeply on Avalonia elements. All the alternatives has tradeoffs as well.

I see, this was skipped as part of the .NET migration.
I have added the missing executables.

Since the name of SharpAESCrypt.exe was inherited from the SharpAESCrypt project, so I have opted to call the version here Duplicati.CommandLine.SharpAESCrypt.exe / duplicati-aescrypt which should avoid name conflicts.

Yes. I had added support for the OS name in the help text and updated the places where I found it.

Mono is no longer used and mentions for this was removed previously as part of the .NET8 merge.

I see the detective work from @Jojo-1000 uncovered the issue.
I have changed the logic to rely on a type from the assembly, instead of using the assembly name (although both should work).

Yes, this used to work with Mono, but it no longer works. I have fixed it as part of the OS dependent name update.

The snapshot util is perhaps most relevant for Windows, but I see no reason not to include the tool for testing on other operating systems. I have added the snapshot tool.

This is a bit of a relic from dotnetā€™s Windows heritage. With Mono it worked the same that you could run any .dll or .exe file with Mono, as it was just bytecode inside (with a small Windows header).

For that reason I opted to keep the .exe suffix, so all operating systems had the same files (single zip file).

With the new dotnet approach having binaries for each operating system, the default is to use the same name without the .exe. On Linux systems, it is uncommon for executables to contain capital letters or punctuation, even though it is allowed. MacOS is a bit more mixed, but at least most commandline tools follow the BSD convention of being all lower-case.

Since I had already started with having convenience launchers (e.g., duplicati-cli ā†’ mono Duplicati.CommandLine.exe) I think it makes sense to continue this. Instead of installing launcher, the package builder just renames the executables, giving less files overall.

It is as simple as using var exename = Environment.GetCommandLineArgs()[0] to get the calling executable name. It worked so-so with Mono where it would sometimes pick up mono as the name and other times not. For now I have tried to remove all mentions of .exe files in favor of the mapped executable names. This also works in cases where the current executable needs to reference another excutable (i.e., recovery tool mentions commandline).

Yes, see the above for the logic of using ā€œplatform nativeā€ names. They are not symlink names, they are the actual executable names.

For better error messages, try with:

DEBUG_AVALONIA=2 ./duplicati

This should now be fixed.

I am testing with 38-1.6 and 39-1.5 for desktop, and rawhide with Docker (no GUI things).
I am open to include other versions on request.

I get no errors on either, but I have not seen it show the tray icon.

Not sure if this is fixable or the desktop does simply no support tray icons.
You can still just re-launch the duplicati binary and it will show the web page.

10

lrwxrwxrwx 1 root root 39 May 16 08:31 /usr/bin/duplicati ā†’ ā€¦/lib/duplicati/Duplicati.GUI.TrayIcon

Is this describing upcoming direction? In this release, they are symlinks. In Beta, they are bash scripts.

Current mixed-case executable names are no longer on the systems, and scripts (etc.) must migrate?

Same messages, but I noticed a tray icon this time. Maybe it was slow on the original try, or I missed it.

11

Which package is this from?

I think I made all packages use /usr/bin/duplicati -> /lib/duplicati/duplicati.
In any case we should not use two different formats.

And yes, Beta used bash scripts to invoke Mono, but this is no longer required, so now they are just symlinks.

Yes. If any scripts etc. relies on the previous name, they must migrate anyway, because the Mono prefix is gone and the name no longer has the .exe suffix.

I realized today that the debug messages work fine in the debugger, but not in release :frowning_face:
I have updated the implementation to actually output the messages.

12

I ran the zip install on Manjaro with XFCE desktop and it worked without further problems. The only thing is that the tray icon disappeared after a few minutes, and didnā€™t come back even when the status changed. So it might be related to that?

13

That sounds like the crash I have seen on MacOS, where the tray crashes, but the service keeps running. We will need the fixed logging output to debug this I think.

14

I think .deb on Lubuntu 20.10 (antique), however I think itā€™s the same on Linux Mint 21.2, looking like

$ ls -l /usr/bin/duplicati
lrwxrwxrwx 1 root root 39 May 16 08:31 /usr/bin/duplicati ā†’ ā€¦/lib/duplicati/Duplicati.GUI.TrayIcon

I just overwrote the old Lubuntu with current AlmaLinux 9.4. Installed, came up, but I get no TrayIcon.
Maybe Iā€™ll wait for the fixed implementation, or maybe youā€™ll figure it out first on your Fedora systems.

Will Windows keep current names? For other systems, I agree going to native naming is a migration.
While it can be covered in the manual, people reading the old forum and Issues will need to translate.
Not insurmountable, and maybe the price of native look, however I hope the user manual describes it.

I was coming from a different direction, which is that if Linux etc. gets native look by dropping the .exe, Windows can also use .exe-free command invocation, because many or all places treat it as optional.

Mostly trying to understand the pain level weā€™re causing, but agree that itā€™s a good window for changes.

15

Windows names stay the same, and all libraries retain their windows-heritage with .dll suffix. I think this is a part of the .NET way and it is not user facing.

The default in .NET is to have executables that just drop the .exe but IMO it does not look native at all.

I think the service is no longer maintained, I will add a new link.
The signatures are actually in the zip file, but they have the full path, instead of the relative, so it looks strange. I will fix that as well.

Good catch! The manifest files only has the hashes in base64 encoded format, but the commandline tools will usually prefer the hex-encoding. I have updated the listing page.

16

On Windows 10 .zip program, it looks non-existent.

image
image992Ɨ105 3.25 KB

Why does Windows Compressed Folders (Zip folders) reject paths that begin with a slash?
explains a possibly worse Windows behavior in 2012. The 2020 format spec still says that itā€™s invalid.

4.4.17.1 The name of the file, with optional relative path.
The path stored MUST NOT contain a drive or
device letter, or a leading slash.

It looks from 7-Zip or a hex viewer that paths begin with /Users. 7-Zip shows excess level as an _ or
duplicati-2.0.8.109_debug_2024-05-16.signatures.zip\\Users\ (showing only the middle).

7-Zip looks like it can get the files. Windows 10 looks like it wonā€™t. Windows 11 has changed some things.

Why is Windows Compressed Folders (Zip folders) support stuck at the turn of the century?

Ahoy, matey! Windows 11 will soon have native RAR support / Tar, 7-zip, rar, gz, and more will all be supported in a Windows 11 update

but I donā€™t know if .zip moves to the new libarchive code. Regardless, Iā€™m glad the file is being fixed.

Thanks!

17

Interesting quirk! On MacOS the leading slash is ignored, but the folder structure is maintained :person_shrugging: