Very slow folder browsing during restore

kenkendk · October 30, 2017, 12:31pm

I agree with the statements that restoring (and particularly the browsing of data) should be smooth. The slowdown is not easy to fix, as it is caused by a large table of paths needing to be filtered.

I am not aware of an efficient way to write this in SQLite, but anyone who does can look at the query here:

github.com

duplicati/duplicati/blob/master/Duplicati/Library/Main/Database/LocalListDatabase.cs#L217


            var dirsep = Duplicati.Library.Utility.Utility.GuessDirSeparator(s);



            s = s.Substring(ppl);

            var ix = s.IndexOf(dirsep, StringComparison.Ordinal);

            if (ix > 0 && ix != s.Length - 1)

                s = s.Substring(0, ix + 1);

            yield return prefix + s;

        }

}



public IEnumerable<IFileversion> SelectFolderContents(Library.Utility.IFilter filter)

{

    var tbname = "Filenames-" + Library.Utility.Utility.ByteArrayAsHexString(Guid.NewGuid().ToByteArray());

    try

    {

        string pathprefix;

        if (filter == null || filter.Empty)

            pathprefix = "";

        else if (filter as Library.Utility.FilterExpression == null || ((Library.Utility.FilterExpression)filter).Type != Duplicati.Library.Utility.FilterType.Simple || ((Library.Utility.FilterExpression)filter).GetSimpleList().Length != 1)

            throw new ArgumentException("Filter for list-folder-contents must be a path prefix with no wildcards", nameof(filter));

        else

My long-term fix is to rewrite the way the paths are stored in the database, which will make this query really fast.