If the stack trace details look similar to original post, there’s an update you might try:
One technical writeup theorizing about what’s going on is:
Database is locked when performing auto-cleanup #4631
If above map helps any, have at it. Duplicati is very much in need of people willing to help in any area, whether it’s helping on forum, test, documentation, or (all-important to forward progress) development.
Thank you ts678… when the database locks for me it occurs shortly after the “verify files” part of the backup, i.e. at the beginning so I don’t think it’s associated with auto-cleanup for me.
I again set max threads to 1 and managed to rebuild the database. I have changed database backup frequency to 6 hourly (i.e. 4 times a day) and only keep the last 10 days of backups. This should reduce the current database size significantly (and reduce it a little further over time as the 4 hourly backups are replaced with 6 hourly).
The backup looked like it was running well but just after “Deleting unwanted files” it reports “The remote server returned an error: (403) forbidden”… this is another issue and a topic for a different thread. Now I can’t get past the 403 error. I’m stuck, I’ll try again later.
If you mean after it starts that, that’s exactly when it may run – if option asks for that – does yours?
It’s happening after “Verify Files”, it sits on Verify Files for a while so I assume it has completed it. I don’t know what steps in the process display something in the notification bar but nothing else displays as far as I can see before the error occurs.
Just looking through RunAsync which calls PreBackupVerify… PreBackupVerify is called on line 466. On line 482 the progress bar is definitely updated to “Processing Files”. I suspect the code is having issues in UploadSyntheticFileList on 469 or RecreateMissingIndexFiles on 479. Both these areas appear to pass in a new instance of the database (which is suspicious given the error, but I’m not sure because “var” is used on line 431 so I don’t know what the datatype is - it’s a significant problem when using “var”). I haven’t delved into these two areas so I’m not sure what’s in them.
I have seen a potential issue… RunAsync is obviously called asynchronously. In RunAsync variables such as m_result, m_database, etc. are shared but they don’t have locks around them when their properties are assigned. If m_result (for example) is cached then it’s properties are also cached, writing to a property of m_result could write to a cached property and without a lock statement this won’t be flushed to memory immediately. Correct me if I’m wrong, but I would have thought that all these shared object property assignments should have locks around them.
One problem with using status to know where you are is that all you know is you’ve gotten as far as the status seen, but not as far as the next one, and some aren’t specifically seen because of other displays which fill the status bar – one can see phase at About → System info, e.g. for Backup_ProcessingFiles.
Stuck on “Waiting for upload to finish” gave an example of finding phases at the finishing end of backup.
One can sometimes also study log messages (which can be set quite high) to try to figure out locations. Ruling out auto-cleanup, though, may be as easy as your saying it is or is not in your Advanced options.
You would have to ask a C# developer, which I’m not, but sometimes I can follow Duplicati code around. There is unfortunately a huge need for Duplicati developers, as well as for people to help with anything.
Now that I’ve fixed the 403 (forbidden) error I was receiving (that we’ve been discussing in another thread) and reduced the size of my backup I’m not having this issue any longer. I’ll revert my settings to backing up often to try to cause the issue and if it occurs I’ll investigate from there.
At some point I’ll investigate where I think locking is missing… that issue seems to be quite common in the code.
I’m having this issue with version 2.6.3 beta too. It seems that when the database is locked once it persists. Killing the process in task manager does not help and the repair job also fails.
I encountered that issue with Duplicati - version 2.0.7.1_beta_2023-05-25 during one of the three backups on my system.
Using the simple repair option in the database commands resolved the issues. Following that, I was able to proceed with the backup as usual