Ignoring remote file listed as Deleted


#1

I see this in my job log. Should I manually delete these files? Or is there a way to have Duplicati try to delete them again?

ignoring remote file listed as Deleted: duplicati-20180203T030000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180204T030000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180427T030000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180427T070000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180427T110000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180427T150000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180427T230000Z.dlist.zip.aes,
ignoring remote file listed as Deleted: duplicati-20180428T030000Z.dlist.zip.aes,

#2

Strange… these 8 files look odd when I browse them in B2. They show 2 versions, the newer one being 0 bytes and hidden. Any ideas? (I don’t know why there are 2 versions, I have B2 bucket configured to only retain the latest version.)


#3

I ended up just deleting the files using the B2 console…


#4

As far as I know Duplicati doesn’t do anything involving making files hidden so I would guess that the hidden / 0 byte ones are something specific to how B2 does file processing.

Of course it could be that incoming files to B2 are hidden by default and start out at 0 bytes until they’re done being uploaded so maybe those are interrupted uploads? :thinking:

Am I correct in assuming the manual deletion of both the 0 b and 15.x MB files resolved your initial problem?


#5

Yes. The warnings stopped once I deleted those files.
I am guessing it was some quirk in B2… Duplicati told B2 to delete the files and it didn’t quite do it right for some reason. Not sure.


#6

Glad it’s working (I’m flagging your post as the “Solution”) but it’s good to know that it’s documented here.

If it happens again we can maybe decide Duplicati needs a method to resolve it other than have users manually delete stuff from their destination.


#7

Found out the root cause. I moved data from S3 to B2 using rclone. I would do an initial (somewhat time consuming) sync using rclone, then temporarily disable backup jobs while I do a final rclone sync. Once that completes I edit the backup job to target B2 and then resume normal backups.

rclone behavior with B2 when synchronizing “deletions” is to not actually delete the file, but rather hide it. This confused Duplicati.

fortunately rclone has an option to do actual deletions when sync to B2:

--b2-hard-delete
Permanently delete files on remote removal, otherwise hide files.

Thought I’d follow up in case anyone else had this issue.


#8

Great work! I went ahead and flagged your post as the solution in case other’s have this issue.