I have noticed this also. Theoretically, it should be much faster to run queries agains a database, and it is also more crash resistant.
In Duplicati, there are quite a few problems related to the database.
However, many of the issues will not be fixed by simply removing the database.
In some cases there are failures because the remote store reports different contents (for whatever reason). Removing the database here will just make the problem invisible until you really need the files.
Another problem with the database is consistency issues. Again, removing the database will not fix these, just make them show up later on where they are more likely to prevent correct restores. The errors that produce an inconsistent database should really be found and fixed asap.
Then there is the problem with recreating the local database. This is primarily a speed issue (and there are some recreate errors as well). This could be solved by not using a database, or at least using a much simpler database structure. But it could perhaps also be fixed by storing data in the
dindex files that allow a much faster database rebuild. I am more in favor of the latter option here, as the first one is already handled by the
Without implying that there are problems with CY, simply not having a local database means there are a number of checks that CY cannot do. When TI does these checks and they fail, it looks like TI is more fragile than CY.
I really like the simplicity in CY, it contains way less code that needs to be maintained, but I think TI will benefit from the local database in the long run.
The size of the database can also be reduced in the future (path compression will do a great size reduction). But note that CY also stores a large number of files on disk to make a cache of remote blocks.
Actually, it would not require a great deal, if we simply allow the same block to exist twice (well, that is actually already allowed, but not in a useful way).
However, it makes it much harder to detect problems, if we suddenly have two sources that update the remote store. In CY this is less of a problem as they can “see” all the blocks that exist, but because TI hides this inside the zip archives, it needs to download and unpack some files to get the same information.
But, it is not impossible that we can use some of the same strategy as CY uses for concurrently reclaiming old data. It is not on my short-list of things to do