2023 release planning

Developer
1

This topic is where to follow up on a Releases-category topic that turned into developer proposal.
To keep the topic title somewhat unique (there are similar), I’m calling this 2023 (I certainly hope).

Part of the goal is to get things somewhat restarted, prioritizing around risks, returns, and staffing.
On that note, there is always a need for developers, even if it’s just for limited specific assistance.

There is also a need for brave testers, as we are seemingly heading into pre-Canary-level testing.
There is no lab full of hardware configurations and professionals, so this has to be spread around.

More hands make lighter work, and this project is basically too large and complex for one person.
Huge thanks to @gpatel-fr for showing up with tools, a proposal, and an offer. We need others…

To start off on the maybe-boring technical details, how would people classify the dependabot PRs?
Although I think library authors usually try not to break old code on upgrades, sometimes things do disappear or get deprecated prior to that. I’m just not sure how much manual review is appropriate.

Second obscure comment on release channels and history:

The release channel situation has gotten kind of odd. Canary used to be a don’t use-for-production, however has been forced into more wide usage, as Beta pace slowed. When Canary pace slowed, Canary users wound up with unofficial builds such as .dll drops (great proof of the fix for Canary).

Breakages happen from time to time. Quick respins often followed, then quietness returned for test.
After that, an Experimental to prove that infrequent updates actually work, then a wider Beta rollout.

Or so it has been, but can likely be changed if the team wants otherwise. It’s a lot of test levels, with somewhat defined community for each level, and possibly some people in the wrong level right now.

My first reaction to the proposal of the new way is here in the Releases topic, but can more be here?
Input from Duplicati people wanting to shape this or help is needed. This is still a community project.

Thanks again to all who made it, keep it alive, and maybe have long-term dreams (different topic…).
Please, anybody, volunteer in any capacity you can. It’s not just developers. It takes a whole team…

2

IMO these PRs as relatively safe because they are tested elsewhere, and usually remedy to problems of compatibility, security. Not applying them means to be stuck on olds releases that are not maintained, that’s a risk. Also projects not applying these PR don’t look well, unless there is an explicit note of someone explaining why merging them is breaking things. Finally, if it is breaking things, it will hurt no one’s feeling to revert them, so inclusion in a canary would be my default option.

I have not received any invitation to be part of the project, but this does not concern me at the moment. I have taken a short break to deal with personal matters.

Additional details on my immediate plans: I intend to first complete the task of building the most important binaries, this is the break-up by type:

package type downloads
duplicati_2.0.6.104-1_all.deb 21095
duplicati-2.0.6.104_canary_2022-06-15-x64.msi 7697
duplicati-2.0.6.104_canary_2022-06-15.zip 6188
duplicati-2.0.6.104-2.0.6.104_canary_20220615.noarch.rpm 801
duplicati-2.0.6.104_canary_2022-06-15.dmg 762
duplicati-2.0.6.104_canary_2022-06-15-x86.msi 630
duplicati-2.0.6.104_canary_2022-06-15.pkg 236
duplicati-2.0.6.104_canary_2022-06-15.spk 215

so actually my priority will be to generate a debian package (a surprise for me I was expecting win64 to lead and not at all the zip file - what it is used for I wonder - Docker maybe ?). So .deb and .zip will be first - should be the easiest as well. Also having some kind of suffix for the name would be nice to identify the package.

Well yes of course. Actually getting something out of the gates would be nice to get people to help.

3

It looks like current dependabot PRs are all the same because dependabot isn’t really configured.
This might tone it down to wanting Newtonsoft.Json 12.0.2 to move to 13.0.2 for security reasons.
https://github.com/JamesNK/Newtonsoft.Json/releases isn’t flagging any 13.0.2 concerns to me…

About Dependabot security updates
About Dependabot version updates

Version updates for non-security reasons is when the concerns from me and others kick in further.
Personally, I would prioritize catching up on PRs over doing a maybe-it-helps dependency refresh.

4

Re: dependabot and Newtonsoft.

I’ll not merge these PR if you don’t want to.

However, it will not accelerate PR handling if I don’t merge these PRs. What time I win with not merging these PR, I will use on other stuff. I think I have made clear that I am not keen on merging features, only on clear breakages.

However again, the 4 year lapse between 12 and 13 version has been used by the Newtonsoft project for bug fixes. It’s not a good idea IMO to forgo these bugfixes because they are not explicitly marked ‘security’. In 4 years, it don’t happen that the code has not been made more secure, unless the project is not used at all. That’s hardly the case for this project:

https://www.nuget.org/stats/packages

it’s the most used on Nuget. By a large margin.

And if you browse the downloaded versions (600 K per day - it’s a trusted project if there is one):

take a hard look at this page. I know that you are observant I’m sure that you will not miss the interesting detail :slight_smile:

Enough about Newtonsoft from me, next to plans.

Now that I am a Duplicati developer, I have a small change of mind about the planned release.
In my opinion, the current canary has been tested more than enough, so long indeed that several problems have appeared because some stuff has been obsoleted. The plan in my idea should be to refresh the canary to fix these problems, publish it, then after a short time (one month) go to beta (finally).
There is no need to add new risks to this (very near I hope) beta by merging left and right. The more so because I’m not ready to evaluate complex changes.

So the breakages to handle IMO:

It has been asked for and here I am for the merge since it’s a library :slight_smile: .

What I wanted initially is to add to bug fixes a small improvement as a proof of concept that Duplicati could go forward (even slightly) by merging my perf experimental code.

But having seen the involvement of posters on this forum about the missing tray icon on OSX, I think that it’s better to try to get this in, because first it’s currently broken, so risk of regression is minimal as it can’t be worse than not working at all, second it’s really an improvement involving added code so it’s as good as a proof of concept.
So I plan to review it as an exceptional case - even if I don’t plan to do this in the following months except in case of breakages because I expect to do other things with Duplicati (*), I’m going to test if it regress on other platforms (at least Windows and Linux - done), and if we have a few confirmations that everything is all right on Mac by posters, I’ll commit it instead of my ‘performance’ PRs.

Also (last reason but not least), if I advocate team work I should avoid committing my own changes. In well managed projects, one don’t commit own stuff, another contributor reviews the PRs and commits - even if it’s sometimes formal, it’s better than not caring at all.

Finally for the immediate plans: please give me your final answer about Newtonsoft, I’ll create a synthetic PR regrouping all the planned changes (including or not Newtonsoft), and push it to my test repo, so that binaries could be available to test the next canary (105). As you’ll notice, the test repo now generates binaries for all 3 main platforms, although I can’t test MacOS binaries:

But I have already tested the generated installers for Linux and Windows and they seem to work all right.

(*): getting better at understanding Duplicati code, understanding the OAuth server, preparing for Kestrel…there are quite a lot of things to think of. It’s just the only thing possible to do the minimum to keep the boat afloat, unless there were half a dozen more developers.

5

I have no concerns on the Newtonsoft security fix and bonus improvements. Please merge it.
This actually might take us up to the Beta, but after that decide what to do with other libraries.

Or maybe your “update other libraries” means a general update, with or without dependabot?
While I’d love any fixes in a new System.Data.SQLite, our encryption got removed, so issues.

For PRs from specific authors, I would suggest working with them to check status. I see some
marked draft, and for others a lot of time has passed, so maybe the PR would merit an update.

The one most up to date is mega.nz, as you just had it tested. uplink.NET PR wants 2.9.2710
however that’s now about 24 versions (wow) out of date. Take advantage of contributor advice.
Authors will probably be thrilled that their aged PR is getting love, might have test thoughts, etc.

The extensions change looks safe, the macOS fix seems to be testing well, and seems worth it.
Generally anything which recovers something that’s totally broken seems a good candidate if it
carries little risk of making other things worse. macOS I think still works if one just browses to it.
As discussed, the fix is is a different area. We’re also still working on finding pre-release testers.
Testers can also be lined up to try the Canary when it comes out. This doesn’t need coding skill.

Maybe at some point another volunteer can help with solving logistics around fixing some issue.
They could also work with PR authors, but actually it’s not so hard, especially if they’re on forum.

@albertony and @TopperDEL would you like to comment on PR commit plan here (or GitHub)?

6

I would love to see my PR merged! I may also bring it to the latest uplink.NET before merging. There is no significant change in there, though.

7

No, I will rely on dependabot.

8

If you can take the time, thanks for doing it.

9

Just did it two hours ago. :slight_smile:

10

I’m happy to assist in small areas. I use this tool enough that I want to give back where I can. I’ve got a couple decades of stack development and network management But I’m not terribly proficient in C#. So if you ever run into a situation that feels like it needs some mundane technical work, just shoot me a message.

1 Like
11

The immediate need will be in testing the (soon to come I hope) binaries pre-canary. After that, it depend on what you mean by ‘stack development’. If it’s javascript, there is some in Duplicati and everything needs attention - all software is obsoleted as fast as possible, incompatibilities are creeping everywhere, and users want always better features to boot :slight_smile: