Given the work that has gone into the .NET8 branch and the feedback on the debug builds, I think the .NET8 branch is ready to be merged.
I have updated the CI and tests so everything is
This does not mean that a canary for .NET8 is ready. It is more a signal that new development should be on the .NET8 codebase.
I will be adding some new issues to Github that I think are blockers for a .NET8 based canary release.
Feel free to add issues or leave a comment on this topic, and I will try to include as much as possible.
My goal for the initial canary is not to address any of the known issues in Duplicati, but more to get the transition started with as few (other) breaking changes as possible.
The tasks that I have on my short-list so far are:
Updater manifest format
The updater manifest is one of the few places where I forgot to add a version field. Since the updater changes with the .NET8 update, I will remedy this and add a version number and some documentation to the file format.
Storj/uplink
The current binaries makes the whole application crash on loading these (at least for Windows). Also, given the size of the binaries, I think some lazy-down-loading could come into play here. I hope @TopperDEL has some input on this
Upgrade/decrypt database
Since RC4 has been removed from the free SQLite codebase, a special .NET4 based tool is needed to lift the encryption. Currently this is a manual and slightly annoying process. Since it is possible to detect if the database is encrypted, we could have a more automated flow that downloads the binaries and decrypts the database.
Edit: This turned out to be fairly simple, so it can now be done without the need to the .NET4 tool
TrayIcon style
While a cosmetic issue, the current tray-icon has a black outline, and as many OSāes now use black background it becomes virtually invisible
Remove DynamicLoader
One of the first pieces of code I wrote was a dynamic loader that scans for backends. The idea was that a user could drop in a new backend and have it show up without any need for recompilation. While this is somewhat still possible, I have not seen widespread use of it, and it is quite slow and messy as it scan the 500+ files in the install folder.
Fixing stub LegacyHttpRequestShim
The transition to Kestrel has has left a few operations remaining to be implemented, including the deletion of back tasks.
For the current .NET4 release, I have sent out a new experimental release today that will be released as the beta build in appx 2 weeks (unless some new critical issue appears). The idea is that 2.0.8.1 beta will provide the most-recent features & fixes until the .NET8 builds are ready.
System.Data.SQLite: RC4
Supported for compatibility with earlier System.Data.SQLite versions only. Donāt use it in new projects. Since early 2020 the official System.Data.SQLite distribution no longer includes the RC4 encryption extension.
If it has any future value, especially as Duplicati Inc. gets in new areas, maybe it could scan less than all.
It is not directly tied to .NET4, but the last versions that could decrypt are not available for later versions.
The decrypt tool uses the older version of the SQLite packages that has still has RC4 support.
I just found this post. I do not know why I do not get mail notifications anymore.
Can you tell me if you can run backup/restore operation on .net8 version? I am trying right now write selenium tests to do that in an automated way. If this is not possible I would rather merge my changes sooner rather than later.
I.e. I fixed handling of XSFR errors to be done during request in AppService.js instead of in various places on the FE, by i.e. showing annoying modal window.
Looking how LegacyHttpResponseShim is used (with NotImplementexExceptions) I am not sure if .net8 will work at all in that state.
I can fix that I just do not want to double someones other work (that may be done already).
And if this version does not in-fact work correctly I would rather merge my changes now, without finishing Selenium tests.
Yes, the CI tests are passing backup+restore, the Selenium tests are passing and my manual tests are passing. Note that this is with the HttpServer and the RestAPI.
Sound great!
The WebserverCore project is not used by the old HttpServer (using Duplicati.Library.RestAPI).
I am not sure about the LegacyHttpResponseShim intended future role. Maybe @tsuckow has some insights?
I looked again and some properties of this LegacyHttpResponseShim are rarely used i.e. if you try to remove the backup, captcha is shown. If there are no tests for that nobody probably notices that it does not work - like no implementation at all.
But it is used as a wrapper between kestrell and old Webserver interfaces. I am pretty sure it is used everywhere.
I donāt remember exactly where I left things but I was trying to fix things knowing about the webserver that shouldnāt need to know about the webserver. And the legacy response shim was probably a combination of that plus being a middle man for kestrel to avoid needing to port every rest API. Eventually all the rest APIs should be ported to native kestrel and then it would go away.
@npodbielski I just tried to delete a backup with the master branch, and this worked (with HttpServer).
Are you referring to the captcha not showing up when deleting the remote contents as well?
It does not show because it throws NotImplementedException it is possible that it is working differently on another browser - I did not go that much into detail. I just checked instead where if anywhere reading of Query string parameters is done and it is in few places.
I did not touched those places nor I did not touched LegacyHttpResponse/RequestShim classes till I discovered few missing things that should be working (since you were writing on merging .net8 into master) but it was not. āHow come?ā I though, so that asking if I am rebasing on right branch in the first place made more sense for me.
I will check master since .net 5 was merged recently.
I managed to write a small decrypt tool in the .NET8 code, so now it will auto-decrypt
Yes, I will consider if we should allow specifying a folder with plugins.
I think it will be hard to match the assembly versions and dependencies though.
A better approach will likely be to host the backend code in a separate process so it can have its own dependencies, and we can implement the āhard stopā feature that is currently not working.
The PR scans select subfoldes, including one that can be set with environment variables.
By default, all folders are emty, so no dynamic loading is done.
Perhaps this could also help the timeout (actually lack of timeout) situation that bring specific complaints.
Sometimes I wonder if one of the parallel uploaders getting stuck is what holds up the whole show at end, however thereās never been enough logs available to track them one by one to see if all default 4 look OK.
Sometimes I try to get people to check About ā System info upload stats. I donāt know if thatās best test.
Iām not sure if you would have one process for all uploaders. If so, killing all if one times out is pretty ugly, however stuck backups are ugly too ā and this post is inspired by new āWaiting for upload to finishā post.
It would at least allow stopping the process if there is no perceived progress.
Right now the problem is that the connection itself is usually hidden somewhere in the implementation library, so the hosting C# code cannot access it (i.e., S3 has a http connection, but not exposed to the library). No access means no way to monitor or close the connection.
Controlling the input given to the backend process can easily be monitored, and if no data has been sent for say 30s, then assume it has stalled and restart the process (number to be adjusted, may even use EWMA logic).
Another, less drastic, approach would be to fully implement cancellation tokens, which most libraries support.
That would make sense, so one is hanging and eventually times out and is re-tried.
