TL;DR There’s performance, but also reliability. For that, stress it some while logging and monitoring, thereby providing a far better debug environment than when a user hits things. Ideally, fix issues fast.
Maybe more than a little, and it could be its own topic. I can split this up later if that seems beneficial.
This is quite a wide open area that could very much use test staff. Volunteers are very much needed. Sometimes equipment is also needed. Other times people might just run whatever they have around. Sometimes they’re using Duplicati anyway as their backup, and testing that better might be desirable.
Performance testing is its own whole area which could help find slow spots and measure fix attempts, following the goal of doing guided optimizations rather than unguided (a worse return on work + risks).
My main wish is to get a better handle on the reliability issues that keep us from being able to release something to the not-yet-used stable channel. Duplicati has made it past its worse time when backup breaks without any obvious provocation. One of those turned out to be from a specific pattern in data.
Next level of bugs after backup itself is compact that runs occasionally after the backup to do cleanup. Because it runs less often, it takes longer to stumble across bugs. Stumbling is a bad test plan when a user does it, because there is rarely any debug level info there to accurately diagnose/fix the problem.
Better information collection is possible by running test cycles until something looks wrong. Check that and preferably find a recovery or avoidance because constant test stops on known bugs get annoying. This is also where a rapid actual fix (instead of bugs lingering for years) may also be used to continue.
If things seem to hold together when unprovoked (and they appear to), then provoke with usual things such as network failures (simulated, or set no retries and let real world do its thing), or system reboots probably simulated by using process kills (because actual reboots are slower and too work-disruptive).
I don’t know the unit tests, so I’m talking system command line tests, because a GUI is harder to drive. duplicat-client can bridge that a bit, and is a Python script so is pretty easy to tweak if the need occurs.
StopNow(), Backup and cancellation tokens has random kill test, and what fixable (or not) bugs it finds.
There’s an unshown script before that which can make random changes to a file to keep backup going.
In my backup on this PC, I’m currently running a profiling log with --profile-all-database-queries which makes the very frequent block operations visible, and I’m keeping a 30 deep history of database. That’s history if it breaks. To prove recreate works, I do it. I used to run test all --full-result, but that accumulates errors over time (but so far they seem harmless Extra entries though), but I stopped.
I’m running the Python checker script I mentioned at the top, and looking directly into destination files…
There are other ways to do this. As a starter effort using Python sqlite3, I did a simple database read. Fancier checks could be done. An idea of what Duplicati already checks would be a helpful guide too…
EDIT:
You might notice that my type of tests are not what one might call functional tests, e.g. can it one time do what the functional spec (there is none) says it should do? Those are easier to just describe in issues, and there are plenty filed awaiting fix. Functional tests are probably good at catching regressions though.
The unexplainable (e.g. based on what a user has seen) reliability breaks are the harder bugs to isolate, which is why my goal is to find and fix those while avoiding regressions by not fixing what’s not broken.
Thanks for asking about tests.