The support of remote locking file makes me want to switch to Duplicati. However, I cannot seem to get it working, following Using remote file locking | Duplicati.
Adding “–remote-file-lock-duration=30D” makes the backup job fail with a SQLLite db lock conflict apparently during the remote locking phase.
Adding “–s3-object-lock-mode=governance” just creates a warning, that this option would not be supported. Adding “–s3-lock-mode=governande” just createa a warning, that the value “governance” would not be supported. Manual inspection of the remote storage location confirms that none of the uploaded files has been locked.
I’m using a fresh, clean install of 2.3.0.1 on a fully patched Windows 11 machine and a B2 bucket with object locking enabled. The same, small backup job w/o locking seems to work flawlessly. Happens independently whether Duplicati is running as a service or as from the tray icon.
Any help appreciated. Has anyone got this to work?
FWIW, I believe the issue with ‘object (file) locking’ has to do with the fact Duplicati creates encrypted ‘blocks of files’ and not individual files (objects) and saves said blocks in the B2 bucket, and locking said blocks at remote storage site changes blocks sent.
Backblaze itself shows “Object Lock” disabled in their Duplicati configuration when creating a bucket.
Duplicati gathers all files to be backed up, deduplicates and compresses them, then sends them to your backup location in blocks or chunks to be stored for maximum efficiency. The first time you run a Duplicati backup job it will perform a full backup of your system, then each backup job after that will send incremental backups of the changed files, further sparing your bandwidth.
You’re correct and that should be fine. Log file shows, that the job has already successfully uploaded all chunks. I suspect that some call to B2 takes longer than expected, such results in the SQLLite db still locked and opened, when another thread tries to access it.
BTW, same thing happens, whether the B2 bucket is accessed via the S3-compatible backend or the B2 specific one. In the latter case, it’s of course --b2-retention-mode instead of --s3-object-lock-mode that’s used and not recognized.
The B2 bucket used as the destination has object locking enabled, of course. Since object locking is a relatively recent capability of Duplicati and likely not many people use it already, it’s not surprising that Backblaze has it off in there config for Duplicati.
I think it’s a bug. The same SQLite locking issue crashes the backup whether it’s
Version 2.3.0.1 or 2.3.0.103,
B2 or S3 or AWS backend,
a dry run or actual upload,
one of the locking mode options - --s3-lock-mode, --s3-object-lock-mode, --b2-retention-mode - is present or not
It’s an initial backup into an empty bucket with no existing database and only a few kb of data in a few files, though.
The crash always happens after the backup’s content has been put into the bucket and apparently right before the remote files would be locked. Not a single API call is made, though, according to Fiddler. Gemini thinks it’s a deadlock between two threads :
That sounds like a bug we have not discovered so far.
That option needs to be set on the B2 connection, not in the backup advanced options, because it changes how the connection to B2 behaves.
There is some very confusing terminology here. On B2 (and similar object storage) you store “objects” which is a poor terminology conceived when S3 started. They chose to not use “file” because S3 is not a filesystem (and for other reasons avoided “blob” or “stream”). But for most purposes you can think of a stored object as a file.
Duplicati splits local files into chunks (called blocks) and combines the blocks into “volumes”. The “volumes” are technically a compressed file. When the file is uploaded to B2 it becomes an “object”.
From Duplicati’s perspective it “locks files” and from B2 it is “locked objects”.
The Backblaze docs predate the introduction of the remote lock support. If you want to use object locks, you need to enable it.
Thanks, I have tested extensively, but maybe there is some special case on Windows that causes the database to be locked.
Let me know how I can help in recreating the db locking issue. It’s currently blocking me from using Duplicati. Guess we’ll be using Arq 7 a little bit longer.
Putting the lock mode option in at the storage provider stage makes sense. Tried it, but it didn’t seem to make a difference. If it should make a difference, maybe state that explicitly in the documentation Using remote file locking | Duplicati . It’s kind of natural and maybe implied, but at least Gemini didn’t get it and told me explicitly to put it into at the Advanced Options stage.
That makes sense. I’ve found that most locking issues come down to an interrupted job or a stale lock that never got cleaned up properly. Checking whether another backup process is still running before removing the lock can save a lot of headaches.
