Backup configs disappear

Can you create a file in the config directory, with say, something along the lines of

echo “test” >config/testfile

Yep. I ran this from the container. This is what I see:
image

wait, you said that the backups work correctly, yet the database files have a date from 3 weeks ago. Could it be that the database files really used are elsewhere ? Can you check that the database file names displayed in the Web UI (when selecting the job) match the one that you see in this directory ?

I ran fuser against the sqlite database but I’m not sure if I used it correctly - it didn’t report anything
image

I think that the db is not normally open when no Duplicati activity is taking place.

Yeah, these were created when I last successfully saved and ran the backup jobs. The read-only issue is intermittent but out of sheer frustration I didn’t do anything about trying to resolve the problem until a week or so ago. When the backups last worked (ie before they disappeared) these are the databases being used. I have set up, deleted and recreated more backup jobs than I’ve had hot dinners, and they are definitely being stored in this directory. :slight_smile:

Actually I just checked and no, if the server is running, the db is shown as open by fuser:

fuser Duplicati-server.sqlite
/usr/lib/duplicati/data/Duplicati-server.sqlite: 219

My mistake.

After job creation and before backup, the database name is assigned but the database doesn’t exist.
Sliding in the old database can be done on the database screen by typing its path and clicking Save.

Looking is not exactly testing, but actually writing to a database that you care about could destroy it.
Maybe if you ever get one complaining that it’s read-only, and you plan to toss it, try writing to it first.
Your test with testfile reduce chance that permissions mislead, but I don’t know why file is read-only.

Do you see the size shrinking when the configs disappear? Even more of a clue would be if you see:

whose text makes a popup on first run when there is not even a Duplicati-server.sqlite in existence yet.

You can sort of distinguish which config you’re adding from some numbers on GUI URLs that start at 1.
An example Edit of config might use GUI URL http://localhost:8200/ngax/index.html#/edit/1
Maybe this would be another way to tell if old jobs got deleted individually, or entire database got reset.

If relatively simple size and number exam doesn’t help, it’s probably possible to install sqlitebrowser package onto the host, to study DB, but you’d need to start the server with --unencrypted-database.

I have SQLite Database Browser installed, so opened up Duplicati-server.sqlite. What am I looking for?

Hard to say, as this is still a mystery, however you could start with the same ID view in the Backup table that I was trying to suggest as a way to avoid deep dive in the DB. Normal is ID starts at 1 and goes up.

Basically the goal is to distinguish between deletions and a complete start-fresh due to a database loss, neither of which is normal. You might also find something relevant in ErrorLog. Example time converter:

https://www.epochconverter.com/

Hmmm. The backup table appears to be empty, so do most of the tables. The only ones with data are: Notification (1 row), Option (24 rows), and Version (1 row).

I think (not certain) that the idea is to see whether your re-additions get assigned ID 1 or something later.
If you haven’t yet added anything, there’s less you can infer from nothing. Deleted looks like never added except maybe for what you mention. Version is almost automatic, but do Notification or Option help any?

Option BackupID, when positive, goes with that ID in Backup table, and normally I’d expect they’d track. Negative numbers appear to be for special purposes. -2 looks like settings for server itself, not a backup.

This is what I see in the Options table - BackupID = -2

I could blow away the database and try again. Would that help troubleshoot?

I kept the existing database and recreated a new backup. This time it appeared to save (ie I did not get the read-only error).

Browsing the database, there are entries with BackupID = 1

Was this an existing database that had previously lost entries? Testing on mine (you can test yours), deleting a job leaves a gap in the Backup ID sequence. Re-addition doesn’t fill gap, just goes to end.

For example, if a new job gets ID 6, then is deleted, the next new job doesn’t take ID 6 but goes to 7.

If this database had never had a job, then 1 is expected. If it had them then lost them, then it is not…

I notice you’ve opened things up to unrestricted access on any interface. How is Duplicati accessed?

Yes, it was an existing database, I did not delete/recreate it.

I’m not sure what you mean by ‘unrestricted access on any interface’. I run Duplicati on my Raspberry Pi server and access the page via http://[server IP]:8200 from my Windows browser (usually Firefox, but I’ve also tried Edge and Opera)

I recreated 2 existing backups - ‘Documents’ and ‘Pictures’. I reset the random database names to be the original ones, then manually ran each backup. I can see the entries in Duplicati-server.sqlite. Everything appears to be working as expected.

Now I wait for the entries to disappear!!

One other side note. You mentioned I should have enabled --unencrypted-database. I did not do this. I simply opened the sqlite database from within my Windows ‘DB Browser for SQLite’ program. But I can see my passphrases in plain text in the Option table. IS this expected behavour?

The unanswered question was if it had previously lost configs (or not).

Someone set allowed hostnames to * and said to listen on all interfaces.
This would be either in Settings or in the options given on server startup.
The reason the path matters is it might suggest the age of the database.

For a new database, these non-default settings would need to be added.
Unless supplied as options, this means a trip to Settings, so you’d notice.

Another obvious clue that I asked about is that a new database will query

image

so are you getting these unexpectedly, e.g. around when configs disappear?
If so, the problem is the database file disappeared somehow (unknown why).
I would have thought that the is-first-run option is related to above popup,
however mine says False and yours says nothing. I don’t know this table well.

With a blend of things, I forgot that Duplicati is running on Linux, which typically lacks the right SQLite necessary to do the encryption. In that case, it is normal to see it unencrypted without having to ask…