Many empty index files

1

I have a backup with some history. It is a copy of a bigger old backup where several source folders have been removed, leaving two now independent backups. I have also changed volume size from original 25mb to 500mb.

When checking the remote files, there are several (1000ish) dindex files that are empty. It only contains the manifest file. Deleting a index file makes a repair mandatory which recreates the same empty file.

Any ideas? I’m thinking of deleting the files and deleting the db and do a repair. Unless someone has other ideas?

The backup is working fine btw.

Tnx

2

Can you tell if you also have some very empty dblock files (i.e. so empty that their index would be empty)?

You could try comparing file counts. Ordinarily I believe there’s exactly one dindex file for every dblock file.

Do you have unusual settings, especially in the auto-compact settings? Do you dare try manual compact?

No other ideas on recovery, just wondering what this is.

3

The number of index file is 2 or 3 order of magnitudes bigger than blocks (partly because I moved from 25 to 500mb but the empty index files were already present before this move).
Manual compact did not remove index files.

4

Removed some of those files, and doing a db recreate now. It complains about missing dblock files. I’ll let it run to completion but will take a while.

5
  • removed the empty dindex files (and some more of which I was pretty sure they were no longer needed as the everything should have been compacted and thus be moved to new dindex files when I moved from 25MB to 500MB volumes)
  • deleted the local DB
  • recreated the DB succesfully
  • starting a backup complains about missing remote files (this was not flagged as an issue during DB recreated)
  • repairing the DB recreated the deleted files (empty ones mainly)
6

I’m glad things returned to the prior sanity-with-oddity. Manual file deletions in the destination can be risky…

If these index files really bother you, I wonder if The LIST-BROKEN-FILES command can see any problems?

To a previous comment, changing the –dblock-size shouldn’t (AFAIK) change the 1-to-1 ratio with index files.

Compact also doesn’t change everything. It has lots of settings though, and can be nudged in that direction.

I still wonder if you have some empty dblock files, which I think would explain index files with nothing to index.

This could also be examined from a database point of view, but the easy path is to just live with the situation.

7

As it are many thousands files, I really want them to be cleared out. It slows down performance when doing file lists. There are no empty dblock files.

I’ll be checking the db for reference to those files and delete the references… when I find some time…

8

Manual database changing is probably even more risky than manual destination changing… I suggested examination to seek understanding (while reading source code) of what’s going on and how to remedy it.

–list-broken-files looked like it was going near some code relevant to this, and “should” be low-risk to run.

I’m not sure how index files relate to performance. They shouldn’t even have been read (but possibly hurt performance through their making the database more full than it needs to be?). Not my area of expertise. Perhaps you refer to the verification of destination files against database list? That would slow somewhat.

Regardless, I’m probably out of ideas without a lot more code reading, so good luck whatever you decide.

9

They are being generated as part of the repair process after a compact process that had issues (eg. backend related).

If I have some time I’ll try to simulate this to verify.