Very slow folder browsing during restore

Support
21

OOOPS! Indeed! Really surprising for me! Not a native speaker :slight_smile:

Concur seemed to be like ā€œKonkurrentā€ in my language, which is a competitor. Not something which is in agreement.

1 Like
22

Following up on this, @davegold suggested that we rewrite just the table for browsing files. Since this table is written anyway, and only used within the browse/restore code, it should be possible to split just this table to get faster folder/file listings in the UI.

3 Likes
23

Just proposed a pull request #2897 with a fix that worked for me. I had the same issue, a simple directory listing in restore usually took more than 5 minutes to list the contents. The same for file searching.

Did a bit of study on queries and noticed that subqueries returned way more than needed to join to temporary tables during restore.

After a simple filtering I get the same results but with times in 5 seconds range, which is ok for me (It could get even better, but it would require some rewriting in database).I hope that could help (and sorry for my poor English)!

8 Likes
24

Wow, this is awesome! And your English is great!

2 Likes
25

Did your pull get added? It would be nice to know if this is Coming Soonā„¢.

26

Nevermind. I found a note in .16. Looks like it was added. Great job!

Itā€™s not 5 seconds fast for me, though. WAY faster than before, but still around 30 seconds on a moderate folder.

27

Do you know if you have a lot of versions, files, or long file paths?

Iā€™m guessing you have lots of files and/or long file paths which the implemented fix doesnā€™t improve very much. For those a database rewrite is likely needed. :frowning:

28

Not sure about the paths. Right now I have 27 versions.

29

As another point of reference, Iā€™ve just changed to the experimental channel and am to find that browsing operations have dropped from (many) minutes each, to about 15 seconds. This makes restores of arbitrary files tolerable, although hardly snappy. I run 64 versions.

30

It all happens in the local database, so speed may vary from computer to computer depending on CPU and disk speed.

The canary improved speeds a lot but I figure it should be possible to go down to around 1 second per step from a couple of test queries I made on a test DB.

31

Understood. And Iā€™m not complaining at all. This performance improvement, and the smart backup retention, are very welcome. Weā€™ll be rolling Duplicati out more widely as a result.

1 Like
32

Iā€™m glad to hear that :slight_smile:

I think itā€™s primarily me complaining, but I havenā€™t had time to look more into improving it yet :smile:

1 Like