Does “server” mean
If so, the result is the same because there is no use of that server. The directory listing is normally the Duplicati system querying its local database. This has seemingly had some optimizations already, per
Optimizing load speed on the restore page
but maybe they’re not enough for your situation. You could certainly do the same logging work to see if there’s some SQL query that’s especially long, but it could take some looking (as seen in above topic).
I’m not familiar with code internals. If you’re lucky enough to have one query that’s the main time delay, running About → Show log → Live → Profiling might show log pause awhile on “Starting” before end.
For a serious study, you’d want a log-file with at least log-file-log-level=profiling, and tools to study that.
Listing directories for restore very slow #1715 is a very old GitHub issue with last user report in 2018… Discussion may be useful, as comments there also indicate that this is a local Duplicati and DB issue.
Other suggestions given here are good, and will help other slow spots, but I’m not sure about this spot.
The word “server” is ambiguous. Is Duplicati on the same system both runs? If so, expect same result.
If Duplicati itself moved to a system that’s hugely faster, then no change would certainly be unexpected.