Our server where i store my duplicati backups crashed the other day. So we reinstalled the server and I created a new clean duplicati backup task on my windows machine that i used before. Now I alwasy get the same error (see topic) aeven when deleteing the database via the GUI and/or deleting the backup folder on the server.
Using Duplicati - 2.0.6.3_beta_2021-06-17 as the lastest on the duplicati site and WEBDAV is storage type (as before the crash)
This error is comparing database records against the server view, meaning you have a database.
So now there are two jobs defined? Which one gives error such as: (notice the top line names job)
Is job scheduled or manually run? You didn’t really need a new job, but with two, don’t confuse them.
EDIT: Also don’t run two jobs to same destination folder, although I don’t know if that’s the case here.
No there is only 1 job and therefor the error message relates to that one
image|528x139
There is no difference if I start the job automatically or manually.
Any other ideas?
Sounds pretty strange. I guess you can open File Explorer to the database folder to see what’s there.
Duplicati-server.sqlite is the server DB with job definitions, but what matters are random-letters.sqlite.
Those are the job databases that would track things like how many files are expected at the remote…
If you want to test a sequence, try
- From home page, open Database screen
- If database exists Delete button will work, otherwise it won’t be blue and so you can’t activate it
- From this and Explorer, make sure that the named database does not exist
- From job menu, try
Verify filesandCreate bug report. Both should error on no database - Run now button should notice lack of database, be fine with that, and do backup to empty folder
Your error message has to be getting information from somewhere. If getting super serious, use Sysinternals Process Explorer to see which .sqlite files anything (including Duplicati) is using.
There are advanced ways to influence where database lives, e.g. dbpath in Advanced options or
–server-datafolder or DUPLICATI_HOME environment variable to influence the server’s storage.
Hi there ts678. thanks for your thoughts however no change. I followed the steps you mentioned and the result is still the same.
From teh database menu I copied this path: C:\Users\Jan\AppData\Local\Duplicati\UFFCPSTQGO.sqlite
and during the backup process I witnessed the database being filled: UFFCPSTQGO.sqlite to size 113.052kB. And the process ended with the same error as in teh beginning.
What else can I do?
???
The error at the beginning would prevent the backup from running.
Are you saying that error of topic title happened before AND after?
???
Which guess? Are you saying that backup found missing files, ran happily, found same count at end?
What’s the destination, and can you estimate its file count and look over dates for new and old ones?
Translation from Dutch: “Sent from Outlook for iOS” and links to a Microsoft login page. What’s there?
Please answer earlier question about what’s in the database folder. Only two files, for server and job?
Please answer earlier question about special setup, especially any odd advanced options e.g. dbpath.
Because you’ve now hit Create bug report and seen a failure from no database, it should now work because you now presumably have DB. If you’re willing, you can put it in some cloud and post link to it, however if you’re concerned about whether the sanitization is sufficient, I can lead through other steps.
Easiest one might be for you to see what you made. New destination dates should be during backup…
After you confirm files, use Verify files button again to see if it gets same error as at end of backup.
Either way, you can go to Show log → Remote to click on list near the top to read what Duplicati saw.
“files that are missing” could be an old or confused database or a failure of destination to list properly.
Ordinarily I’d say get numbers on files and uploads from "BackendStatistics" in Complete job log however the job log reporting results tends not to happen when backup fails. You could look for it there. About → Show log → Stored catches errors that stop job logs, but detail is lower. You can look anyway.
Which guess? Are you saying that backup found missing files, ran happily, found same count at end?
[Sjeppie]: the backup ran happily but ended with the same error as the topictitle
What’s the destination, and can you estimate its file count and look over dates for new and old ones?
[Sjeppie]: The destination is our server and the connection is made via webdav. This setup has worked for a long time until our server crashed. Difficult to estimate file count (total size of the folder on the server = 57GB!). There are a bunch of .aes files dated today and a lot from yesterday’s run. I use the smart retention scheme for backup.
Translation from Dutch: “Sent from Outlook for iOS” and links to a Microsoft login page. What’s there?
[Sjeppie]: nothing interesting there, it’s just a link to a download of outlook for ios
Please answer earlier question about what’s in the database folder. Only two files, for server and job?
[Sjeppie]: 2 files indeed: Duplicati-server.sqlite and UFFCPSTQGO.sqlite
Please answer earlier question about special setup, especially any odd advanced options e.g. dbpath.
[Sjeppie]: nothing like that
Sorry I don’t understand what you mean here.
Yes new destination dats for the run today
Verify files gives exactly the same error as in topic title and show log->remote shows an empty list. See image
Sorry I don’t understand what you mean here.
Good. At least there wasn’t a complaint at the start. At the end is still weird though.
So at default 50 MB dblock (“Remote volume”) size, that should be about 1140 files, but each dblock should have an associated dindex, so I wonder if the destination is losing half of them? Very strange.
What tools does the server have? Windows File Explorer counts files pretty instantly on a folder open. Linux GUI tools vary, but in command line one can usually run something like ls *dindex* | wc -l
The sequence I gave was based on the assumption of an empty destination, which was previous case.
If you started a backup with no database but with destination files, it should have given error on extras.
Things have to look as expected before backup will run. For now, ignore any message advice to repair.
If this is at the time when you know files are there, then maybe WebDAV is faulty. It should show files.
Check logs on these paths for something around the right time:
About → Show log → Stored
and
<job> → Show log
Either way, if something is there and not too private, take a peek. For regular job log at bottom,
Complete log expands to a hard-to-read thing (summarized earlier above that) with detail like
“FilesUploaded”: (upload count is from this backup only)
“KnownFileCount” (as Duplicati sees, but is seeing bad?)
“KnownFileSize”: (total should be about 57 GB currently)
“BackupListCount” (its versions, I hoped 1, but maybe 2)
Given that empty list, maybe it’s time to do a manual test, e.g. from GUI Export As Command-line
get WebDAV URL, In Command Prompt run Duplicati.CommandLine.BackendTool.exe list <URL> possibly altering the folder to see if anything on the server will give a good listing not wrongly empty.
EDIT:
If Options screen has Remote volume size raised from default 50 MB to 100 MB, then math works.
Debugging a consistently empty list is probably easier than if half of the files went badly somehow.
Its a Webdav or related issue if there are remote backup files found but Duplicati gets nothing returned during LIST. LIST is Webdav returning the file list to Duplicati so when its empty or wrong compared to the actual remote Duplicati backup files then it tells you a lot.
If there’s no remote files then that would indeed be an issue and a different Webdav related issue if Duplicati was told that putting them was successful.
This might not be actually Webdav though and could maybe be configuration or permission. Though Duplicati should see failures or retries if things were wrong in most cases. But, there’s certain issues that might not be obvious.
To add to that thought about debug ideas…
You could test file listing with some other tool. Apparently WebDAV does a list using PROPFIND.
I don’t know if it’s especially troublesome, but Duplicati WebDAV has a special debug option for it
--debug-propfind-fileTo aid in debugging issues, it is possible to set a path to a file that will be overwritten with the PROPFIND response
You should be able to set that in Advanced options on screen 5. Ones on screen 2 can get lost.
Apparently the issue indeed has to do with webdav. I followed your advice and ran the Other Command Line Utilities - Duplicati 2 User’s Manual as follows in a CMD-box:
“C:\Program Files\Duplicati 2\Duplicati.CommandLine.BackendTool.exe” LIST “webdavs://freshpcweb.nl/remote.php/webdav//backup-na-cloud-crash?auth-username=xxxxx&auth-password=xxxxxxxxxxxxxxxxx”
And the result was this empty list:
Name Dir/File LastChange Size
I contacted the admin of the server and there will be an investigation on that. Thanks for your help.
1 Like
Did you intend to have a double slash in the path? Seemingly you have Destination like:
Hi ts678
You’re a genius! That’s what I missed. I got rid of the extra slash and everything works like a charm.
I had no suspicion to the url, as the backup itself went fine! It was the verification phase that failed.
Anyhow, I’m happy now.
Thanks a lot for your persistence in trying to help me.
1 Like