Early preview of .Net8 packages, codename `debug-2.0.8.104`

As announced in the latest canary, the plan is to move to .Net8 builds going forward.
For those that want to help out or try the very latest, I have prepared a set of builds that use the .Net8 build & package process.

This build set is now tested in terms of installer and placements, but are still not advised for using for backups.
The packages are now signed (Windows + MacOS) and the GPG signatures are included.

Upgrade notice

For Windows users, the new versions of SQLite no longer support the deprecated RC4 encryption. Current builds (.Net4 based) can decrypt the Duplicati-server.sqlite file used by the UI, but the new builds cannot. I have prepared a decrypt tool from the .Net4 line:

The file naming structure:

The filenames are named after operating system, chip architecture and intended use, e.g. linux-x64-gui.
The -gui builds also include the cli executable, but are generally bigger due to the UI libraries.

The suffix denotes the installer type:

  • .zip: Generally just the files, zipped.
  • .deb: Debian package
  • .rpm: RPM package
  • .pkg, .dmg: MacOS packages
  • .msi: Windows Installer

Docker images:

Docker images are pushed with the tags 2.0.8.103 and Debug.

The armv7, amd64 and arm64 architectures are included.

List of images

1 Like

image

Linux Mint 20, in support for another year. Based on Ubuntu 20.04 which I think has libicu66, however I also seem to have libicu60 somehow. Duplicati apt show seems to want libicu70 | libicu71 | libicu72.

EDIT:

Installs on Linux Mint 21.2, which seems to have libicu70, however TrayIcon doesn’t seem to show me one.
This used manual start of typing duplicati then confirming that it’s really the TrayIcon executable running.

Web UI is showing 2.0.8.104, but with (maybe remembered from before) “New update found” for 2.0.7.102.
Of course I have to try the buttons. Download doesn’t seem to do anything, and Show’s display is amusing:

image

It was probably a leftover, or at least a Dismiss then Check for updates isn’t bringing it back immediately.

Starting with systemctl uses a different database, and it didn’t come up with the offer of an update-to-lower.
What it did come up with is some new warnings in journaltl -u duplicati if one actually looks there…

Example’s (there are a lot logged):

Apr 11 11:07:11 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: "Uploading"
Apr 11 11:07:11 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: "Temporary"
Apr 11 11:07:13 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: "Deleted"
Apr 11 11:07:13 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: "Deleting"
Apr 11 11:07:13 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: "Deleted"
Apr 11 11:07:13 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: "Deleting"
Apr 11 11:07:13 LinuxMint21 duplicati-server[102472]: SQLite warning (284): automatic index on LatestBlocksetIds-12DBA642AF7FDD45AB12913CC2D4165B(PATH)
Apr 11 11:07:13 LinuxMint21 duplicati-server[102472]: SQLite warning (284): automatic index on BB(VolumeID)
Apr 11 11:07:14 LinuxMint21 duplicati-server[102472]: SQLite warning (28): double-quoted string literal: ""

What’s odd is that the system SQLite should have been new enough to warn on double quotes too, as per
Double-quoted String Literals Are Accepted

As of SQLite 3.27.0 (2019-02-07) the use of a double-quoted string literal causes a warning message to be sent to the error log.

GUI seems to come up on port 3001 as well as where it should be. This interferes with multiple Duplicati as seen in this start when the systemd root Duplicati surprised me by LISTEN on port 3001 in addition to 8200.

$ duplicati-server --webservice-port=8300
info: Duplicati.WebserverCore.ApplicationPartsLogger[0]
      Found the following application parts: 'Duplicati.Server, Duplicati.WebserverCore' with the following controllers: 'RESTHandlerCoreController'
fail: Microsoft.Extensions.Hosting.Internal.Host[11]
      Hosting failed to start
      System.IO.IOException: Failed to bind to address http://127.0.0.1:3001: address already in use.
       ---> Microsoft.AspNetCore.Connections.AddressInUseException: Address already in use
       ---> System.Net.Sockets.SocketException (98): Address already in use
         at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
         at System.Net.Sockets.Socket.Bind(EndPoint localEP)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.LocalhostListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
         at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)

That was a bit of a rabbit hole, but I have added support for many old versions of libicu now, but it will break in the future :'(.

You mean the trayicon does not appear? Or some issue with libicu install?

Looks like it is not bundling the changelog correctly. I will look into it.

Yikes. That is a new check. I think I consistently used double quotes, so that is a fun find-and-replace exercise.

That is the Kestrel server that is starting up in addition to the normal one. I think we can disable it until @npodbielski is ready with it.

I would expect them to use the same database, from ~/.local/duplicati/Duplicati-server.sqlite (or similar).
What are the database paths you see?

I didn’t see anything like the old one. I minimized all windows while looking. It’s possible I still missed it.

This makes for the previous awkward stop where I Control-C, and have to do it twice to actually stop it.

There is also the “automatic index” warning, but there are fewer. I didn’t bother looking up history on that.

It seems to run a lot like the normal one.

Yes, but tilde moves. The systemd one is root, not me. The rest of your path looks different, BTW. It was ~/.config/Duplicati, and I assumed it still was, since I saw some old jobs. I hope it didn’t really move.

Speaking of moves, I tried Windows 10 with the .msi install x64 version this time, and watched it delete C:\Program Files\Duplicati 2 (good thing I didn’t use portable mode) and install below Program Files (x86)…

I figured I’d try it. Task Manager shows it up for about 1 second, and Application event log caught its death.

EDIT:

Sysinternals sigcheck says that TrayIcon in (x86) is 64 bit, so maybe the confusion is in the .msi installer.

SQLiteDecryptTool does not run after I unzip it to folder, although this error message sounds a bit familiar:

C:\Users\Maintenance\Downloads\SQLiteDecryptTool\SQLiteDecryptTool>sqlitedecrypttool.exe
Duplicati Database Decryption Tool

Duplicati version 2.0 used a weak RC4 encryption algorithm to encrypt the database.
The RC4 and other methods are no longer supported for SQLite with the native libraries
The encryption support was primarily available on Windows, but could have been available on other platforms.
This tool will decrypt the database and save a backup before attempting to decrypt it.

Found database in default location: C:\Users\Maintenance\AppData\Local\Duplicati\Duplicati-server.sqlite
Do you want to decrypt this database? (y/n): y
Creating a copy of the database in C:\Users\Maintenance\AppData\Local\Duplicati\Duplicati-server_backup_enc.sqlite...
Decrypting the database...

Unhandled Exception: System.NotSupportedException: An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.
   at System.Reflection.RuntimeAssembly.nLoadFile(String path, Evidence evidence)
   at System.Reflection.Assembly.LoadFile(String path)
   at Duplicati.Library.SQLiteHelper.SQLiteLoader.get_SQLiteConnectionType()
   at Duplicati.Library.SQLiteHelper.SQLiteLoader.LoadConnection()
   at SQLiteDecryptTool.Program.Main(String[] args)

On .zip file, try right click → Properties and check the Unblock box: Finish, extract, try running it again.

image

Seeming success:

C:\Users\Maintenance\Downloads\SQLiteDecryptTool\SQLiteDecryptTool>sqlitedecrypttool.exe
Duplicati Database Decryption Tool

Duplicati version 2.0 used a weak RC4 encryption algorithm to encrypt the database.
The RC4 and other methods are no longer supported for SQLite with the native libraries
The encryption support was primarily available on Windows, but could have been available on other platforms.
This tool will decrypt the database and save a backup before attempting to decrypt it.

Found database in default location: C:\Users\Maintenance\AppData\Local\Duplicati\Duplicati-server.sqlite
Do you want to decrypt this database? (y/n): y
Creating a copy of the database in C:\Users\Maintenance\AppData\Local\Duplicati\Duplicati-server_backup_enc.sqlite...
Decrypting the database...
Database decrypted successfully!
Press enter to close the program

The symptom before the decrypt was:

C:\Program Files (x86)\Duplicati 2>Duplicati.Server.exe
Unhandled exception. System.Exception: A serious error occurred in Duplicati: System.Exception: Failed to create, open or upgrade the database.
Error message: file is not a database
file is not a database
   at Duplicati.Server.Program.GetDatabaseConnection(Dictionary`2 commandlineOptions)
   at Duplicati.Server.Program.Main(String[] _args)
 ---> System.Exception: Failed to create, open or upgrade the database.
Error message: file is not a database
file is not a database
   at Duplicati.Server.Program.GetDatabaseConnection(Dictionary`2 commandlineOptions)
   at Duplicati.Server.Program.Main(String[] _args)
   --- End of inner exception stack trace ---
   at Duplicati.Server.Program.Main(String[] _args)
   at Duplicati.Server.Net8.Program.Main(String[] args)

After decrypt, Duplicati.Server.exe is available (unlike TrayIcon crash) if I browse to it manually. Good.
Spoke too soon. I saw the web UI for a few seconds (complete with an invitation to update to 2.0.6.3).
It lost the connection after that, and the event log shows the crash again. Terminal returned to prompt.

Event 1000, Application Error:

Faulting application name: Duplicati.Server.exe, version: 2.0.8.104, time stamp: 0x65680000
Faulting module name: ntdll.dll, version: 10.0.19041.4239, time stamp: 0xad5435e9
Exception code: 0xc0000409
Fault offset: 0x00000000000a2350
Faulting process id: 0x7d8
Faulting application start time: 0x01da8c47e49ad031
Faulting application path: C:\Program Files (x86)\Duplicati 2\Duplicati.Server.exe
Faulting module path: C:\WINDOWS\SYSTEM32\ntdll.dll
Report Id: c41364bd-f035-4824-8c10-6c2c81e39175
Faulting package full name: 
Faulting package-relative application ID: 

Event 1001, Windows Error Reporting:

Fault bucket 1741720627435224920, type 5
Event Name: BEX64
Response: Not available
Cab Id: 0

Problem signature:
P1: Duplicati.Server.exe
P2: 2.0.8.104
P3: 65680000
P4: ntdll.dll
P5: 10.0.19041.4239
P6: ad5435e9
P7: 00000000000a2350
P8: c0000409
P9: 000000000000000a
P10: 

Attached files:
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER1E29.tmp.mdmp
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER4029.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER4098.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER40D4.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER4171.tmp.txt

These files may be available here:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_Duplicati.Server_392f54b13ca6e7113b7c7fe172189a463a5057_b4d0357b_bf53bffe-27b4-4a49-bb37-4d05e7b842d5

Analysis symbol: 
Rechecking for solution: 0
Report Id: c41364bd-f035-4824-8c10-6c2c81e39175
Report Status: 268435456
Hashed bucket: fb5de0f97a4c6e79982bd5aec03f3f58
Cab Guid: 0

Trying to use F12 Developer Tools in Edge to look at the network, I “think” the first non-response was to

http://localhost:8200/api/v1/serverstate/?lasteventid=-1&longpoll=false&duration=299s

The old web server didn’t have any easy way to log incoming traffic. Does the new one do better there?

Having a hard time getting any CommandLine programs to run on Windows. BackendTester can report Starting run no 0 before silence. BackendTool can give its help, which is more than CommandLine. Application event log is full of the crashes.

I made an attempt to use this debug version and was only partially successful in getting it to work. Windows 11 x64 is my platform, and duplicati-2.0.8.104_debug_2024-04-11-win-x64-gui.msi is the package I selected.

I first tried installing this over an existing install of 2.0.7.1 that is configured to run as a service in portable mode.

It correctly detected that I had a previous install of Duplicati but despite having downloaded the x64 package it installed itself in C:\Program Files (x86) which is traditionally where 32-bit programs are placed instead of in C:\Program Files where 64-bit applications go. Due to this, my existing service definition was incorrect as it was still pointed at the old install in C:\Program Files. I had to fix this manually by running Duplicati.WindowsService.exe uninstall and then reinstalling it.

I then ran into the same set of problems as @ts678 when it came to running the SQLite decryption program, though I believe it was ultimately successful.

Then, when I attempted to start Duplicati as a service, it would not start, giving this error message:

If I look into it further, I get this:

I looked at the service definition, and it seems malformed … it’s trying to call a .dll, which I’m not sure is correct:
image

If I modify the service definition to call Duplicati.WindowsService.exe instead, the service starts normally.

Additionally, for some reason if I start Duplicati via the Tray Icon, it crashes immediately open entering the UI with this error:

image

Lastly, although the service seems to be up and running I am unable to use the UI to do anything. My existing database works and my backups are visible, but when I log into the UI with the password I have about two seconds before it pops up with “Not logged in” and makes me reenter my credentials. This repeats forever whenever this URL is called:
image

I altered the service definition to remove the web service password, which removes the infinite credential loop, but still nothing works. The network requests contain this, which I suspect is a problem:
image

I have tried reloading the page, switching browsers, etc. Nothing works. Trying to start a backup makes it look like it’s going to work, but nothing actually happens.

That’s what I have for now. I’ll report back if I learn anything new.

(Edit: grammar)

Another thing I noted just now:

In the UI, despite not being able to start a backup, I am able to export an existing job to a command line. However, exactly like my problem with the service definition it is slightly malformed pointing at a .dll as before.

One last thing (I promise):

If I attempt to run a backup job from the UI, it appears that it tries to work but suddenly stops. Checking the event log reveals these two items:

and

Edit: Same thing if I try to run a job from the command line:
image

Out of curiosity, I renamed storj_uplink.dll to storj_uplink.dll.old to see how the behavior of the program changed. Various things that used to not work are now operating again, only now I have a new problem. Attempting to run a backup results in this:

Guess that database conversion didn’t go so good after all.

Okay, I’m done for tonight.

This let me get further too.

I think new decrypt tool only works on the Duplicati-server.sqlite database, and also makes a backup file. There doesn’t seem to have been a database version bump (at least yet) for either database for .NET 8.

Was your job database recent and from recent Duplicati before? Mine was from 2020 on an old system.
That means it was updated just due to normal Duplicati progress, and it made a backup file before then.
Specifically, DB browser says my job database went from version 11 to version 12 (change circa 2022).

I was also poking job database slowly with things like Show log and Verify files, but finally I tried a backup. Given that we were advised not to do that, I’m not sure how much further I’ll go with carefully doing them.

Thanks so much for trying it! I have not gotten to Windows yet, but I expected it was working better :cry:

I think the problem is that due to .Net not supporting using a .exe as a library, the executables are now slim wrappers, and the real code is in the *.Implementation.dll files. For some reason this makes the executables think they are in fact the dll files.

The error seems to be that it wants to create the tables, but they are already present. That indicates that it has trouble reading the database.

Good that you mentioned those. The arm64 builds are broken and will crash when any log message is written…

On Windows, the Duplicati tray icon is in the blank space in the middle below. Not enough contrast…

image

If I hover over, it identifies as Duplicati. If I right-click it, I get the usual menu. If I force a backup error:

image

I wonder if my thinking Linux had no tray icon wasn’t quite right. Maybe there was invisible one there?

I’m still not sure why I’m not seeing the job database problem, but FWIW the test here had a new job mainly to see if that worked for me, and it did (but so did the old job).

I used the decrypt tool on the Duplicati-server.sqlite database and it indicated that it was successful. I just tried it again using the backup copy it made, and it indicated success. It definitely appears so too, as all the existing jobs and configuration are present exactly as they were before.

The job database was current as of yesterday on version 2.0.7.1. I am testing it on a machine that although it is backed up regularly by Duplicati, it isn’t the end of the world if something happens to the backup jobs.

Strangely, if I ask the job to do a database delete and recreate, it does so and thereafter works perfectly fine, so it appears that whatever the problem is, is specific to the job?

I should elaborate on “works perfectly fine” - there is one more wrinkle. I normally run my backups with the snapshot-policy set to “Required”, but in this debug version I can’t do that. Snapshots/VSS don’t appear to be working at the moment:

The tag for 2.0.8.104 was 2.0.8.103? Despite the Debug, are you sure these should be on DockerHub?
There might have already been an accident. Do the outputs in that post look like a .NET 8 build to you?

I’m guessing they manually picked the 2.0.8.103 tag.

1

Maybe we’ll hear eventually, but a manual pick followed by a manual complaint on 2.0.7.102 seems odd.

Looks like the new MSI builder tool has a quirk with the install path for x64, I will see if I can get it to behave.

I think we need to have some kind of migration help, so this can happen automatically after upgrade. Issue is that this requires the previous .Net runtime. Maybe we can add a “download-and-run” helper method that is invoked if the database is encrypted.

Not correct, I have fixed it.

Not sure what is going on here, but I see the same. What made you suspect the storj_uplink.dll file?

I could not replicate this in my testing, perhaps I need a older database installed to trigger the update?

Could be. I have also seen that loading the system info takes around 30s, and on Windows it looks like the tray icon is delayed by a similar amount of time?

I don’t know how anyone can easily test them without making them public?

I will look into that.

Only the Duplicati-server.sqlite database is encrypted, because it stores credentials to the remote store. The local database contains only paths (which are also in plain text on the system) so it is not encrypted.

I think it is related to the local database for that particular job. There are some changes in SQLite around quotes (" vs '), and I am guessing that the database was perhaps created in some older version and the table name is now parsed differently? Otherwise it seems strange that Duplicati would create the table that already exists.

On Windows it doesn’t look like a loading problem. It looks like an visibility (or minimal visibility) problem.
The basically-black icon is slightly visible in system tray when expanded by uparrow. No screenshot of it however there are two screenshots of how it looks when system tray isn’t expanded. I can also make it display an almost invisible speck of green, but I only screenshot red. Magnifying it, it “might” be a tiny “X”.

What is the plan with the 3 .docker posted right here in the lots-of-flavors invitation for people to go test? There seem to the same limited architecture choice, but they all look like CLI based on the name of them.

I don’t know if putting the debug builds in Docker Hub will cause a lot of grief, so convenience is a tradeoff. Because I don’t use Docker, I don’t know how hard it is to install without help from Docker Hub. One thing making it nice to have a page like this is that people can be told what’s known bad, and can add to results.

Certainly if Docker Hub convenience is a major plus, the preview pages could probably link to Docker Hub.

I looked a little bit. I don’t know how it managed to downgrade to 1.0.0, and it also looks like 1.4.0 supplied platform specific binaries, but I’m not sure how one was supposed to get those for version 2. Build them?

AlphaFS also had an unexpected version change.

I suspected storj_uplink.dll because upon nearly every failure there would be an entry in the Windows Event log referencing it:
image