Hello,
can I exclude files smaller than # bytes from a backup? I specifically refer to files of 0B.
Thanks.
I don’t know the presence of this options, anyway any regular file has a non 0 dimension. In windows the minimum dimension is 1024 byte or the dimension of 1 cluster. How can you see that the file X has the dimension of 0 Bytes?
In Windows Explorer the file shows 0 bytes.
Ok right click on these files → poperty and you can see the actually dimension of these files.
Anyway, files so small could be created temporarily by some programs and shouldn’t backuped. Where are these files? they have some specific exthensions (e.g. *.lock)? in this case you can use filters.
Thanks for taking the time for looking into this.
I see I have quite a lot of these 0B files, with different extensions.
They all show a length of 0 bytes, like this one.
Most of them look to have been non-empty at some time. I’ll have to look for the cause of this.
My concern is: I have read in the forum that files with 0 bytes length may cause a whole lot of problems when backing up or restoring, so that’s what I wanted to omit them.
I understand it’s best to delete them.
Please post the links to those, and ideally be sure you’re looking at reports on your version, not an old one. Current Beta is 2.0.6.3_beta_2021-06-17 and is certainly not perfect, but there have been fixes in the area.
You can also test this for yourself. Have you not yet tried backup or restore? Always test restore of backup.
Sadly, I can’t find a way to do it.
Duplicati has an obvious option to “SkipFilesLargerThan” but you need a “SkipFilesSmallerThan” which as far as I can tell does not exist in the current code. I’m going off the contents of “FilePreFilterProcess.cs” which seems to contain the code for skip files larger than.
Maybe you could run an outside script to do a clean up of the FS and move/remove of all the 0b files.
1 Like
I have found this one Empty source file can make Recreate download all dblock files fruitlessly with huge delay · Issue #3747 · duplicati/duplicati · GitHub, but I remember to have seen a few posts in the forum, at least one of them redirecting here…
Whatever, this is my concern.
Yes, I have done test restores, all went well. However I recently had problems deleting and rebuilding the database and had to restart from scratch. No idea whether 0Byte files were the cause.
They looks like “dummy files”: these type of files aren’t actually stored in file system, or are store temporarily.
Do you have an idea which application create these files?
If I where you I create a filter for exclude these type of files.
Thanks. Well I will sure have a closer look and then delete those files. I hope no new ones will be created.
If you’re using Windows you can make a 0 byte file quite easily.
Open a folder, right-click, New, Text Document and press Enter to save it. You will now have a “New Text Document.txt” in that folder that is 0 bytes in size.
You can probably do the same in most *nix OSes but I don’t have one on hand at the moment to verify that.
I agree most apps won’t make 0 byte files or if they do it usually means something is wrong but I don’t think there is any technical file system reason that a file would have to be greater than 0 bytes.
AFAIK: In most file systems creating a file or folder is “free” so long as is contains no data. By free I mean the space used to store file and folder names is kept in the file allocation table (or equivalent depending on FS), this space is pre-allocated and hidden from you when you partition a drive. This is one of the reasons why a 512GB drive will only have around 476GB of usable space after you create a partition and format it. That “missing” 36GB is what’s used to store file, folder names along with some other important stuff. With all that, we can get 0 byte files or folders.
1 Like
Yes, thanks for elaborating. We’ll see whether there’s still a reason to avoid those 0B-files from a backup. If yes, I’ll have to look for a solution. If not, my only concern is how a legitimate file that is originally greater than 0B turns to be a 0B file - but that’s not a Duplicati problem.
Yeah, that’s mine, but your talking about backing up or restoring problems doesn’t fit. It’s DB recreate.
It was also fixed in 2019 in Canary, and in 2.0.5.1_beta_2020-01-18. I hope you’re not on 2.0.4.x Beta.
Too late to help now. Is there a post describing symptoms? This can unfortunately happen many ways.
I don’t think there is – on a current Beta. You could certainly test more on test backup/restore/recreate.
If you find a way that empty files hurt something, please open a new Issue giving exact steps to repeat.
Duplicati is extremely low on developers (any out there?), but clear steps always make the task easier.
Thank you for all those suggestions.
1 Like