I have missing remote files on a couple of backups. When I try to use the Commandline option to run purge-broken-files, I get this message (sensitive details replaced with X):
Finished!
Found 2 commands but expected 1, commands:
“ftp://192.168.2.65//home/Backup/TB14G6/ON1?auth-username=XXXX&auth-password=XXXXXXXXXXXXX”"C:\ON1"
Return code: 200
What am I doing wrong?
Welcome to the forum @beneix
Got names? The LIST-BROKEN-FILES command and The PURGE-BROKEN-FILES command consider what’s missing. If a dblock is missing, versions of source files it contains are purged.
What’s this trying to do? If it’s just a Commandline arguments box leftover from backup, clear it.
You are most likely using the UI version of the commandline. When it opens up, it will be configured for a backup, which requires the source folders are input as commandline arguments.
When you are running the list-broken-files or purge-broken-files commands, they do not take any arguments besides the url. Simply clear the text field “Commandline arguments” before pressing the button, and it should work fine.
You can use --dry-run=true when you run purge-broken-files, which will show what files it will actually delete, and then re-run without the option to actually remove the files.
Thanks @kenkendk . I am indeed running the “Commandline” command from the GUI (ironic, I know!). Just running the backup as normal gives this error message, which was what made me start my quest to repair the backup:
Found 119 files that are missing from the remote storage, please run repair.
“C:\ON1” refers to the local location; if I remove that, I get another error:
The operation PurgeBrokenFiles has failed with error: Filters are not supported for this operation => Filters are not supported for this operation
ErrorID: FiltersNotAllowedOnPurgeBrokenFiles
Filters are not supported for this operation
Return code: 100
If I remove the “include” and “keep-time” options, I get this error message:
Listing remote folder …
The operation PurgeBrokenFiles has failed with error: No remote volumes were found, refusing purge => No remote volumes were found, refusing purgeErrorID: CannotPurgeWithNoRemoteVolumes
No remote volumes were found, refusing purge
Return code: 100
The remote destination contains hundreds of dblock, dlist and dindex files.
This, compared with the message
Found 119 files that are missing from the remote storage, please run repair.
Seems to indicate that no files are found on the remote destination at all.
What happened prior to the error? Did you upgrade Duplicati?
It looks like you are using an absolute path in the connection, so could you try this URL instead:
ftp://192.168.2.65//home/Backup/TB14G6/ON1?auth-username=XXXX&auth-password=XXXX&ftp-absolute-path=true
I added ftp-absolute-path to the URL to force it to use absolute paths. You can also try with ftp-use-cwd-names=true to toggle a different FTP communication mode.
The ftp communication mode does not seem to be the problem. After trying a lot of different cmd line option combinations, I gave up. I deleted all the files (~360) on the remote server and did (from the WebGUI) a delete of the database. To my astonishment, when I then ran the backup again, Duplicati gave an error message about 119 missing files. How can it determine a certain number of files missing when the database and the remote files are gone? I thought this would re-start the backup process from scratch. Do I have to completely delete the backup definition?
Please look in your job logs. If there are any old ones, you didn’t actually delete the database.
You should have a log from the mentioned new backup. What is run duration? Quick? Awhile?
There are probably Warnings or Errors in the friendlier view, but Complete log has more info.
"Messages": [
"2025-01-18 10:22:01 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started",
"2025-01-18 10:22:02 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2025-01-18 10:22:02 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (7 bytes)",
is where it should fail if it finds missing files at the start. The “bytes” (yours will be different) is the number of files seen (mis-labeled). If it gets past that, it should start Put, but this log can easily become length-limited before it does another List check near end to see if expected files exist.
"BackendStatistics" has the actual "FilesUploaded" count, and also a "KnownFileCount".
Job → Show log → Remote should also have a good view of remote file work during the backup. Clicking on a List will let you see what it saw, so you can see if the file dates seem reasonable, probably meaning (if there’s anything there at all) all the file dates are from the newly run backup.
If you see any older files, that seems wrong, and same goes for any old log records you may see.
That can’t be. The information about which files are supposed to be there are from the database, and only from there.
It can only be that the database is somehow still there. Did you use the GUI to delete the database?
The backup definition is just that: a bunch of settings. The only state it has is the tiny metadata that shows the last backup time, number of versions, size, etc. This metadata is purely for cosmetic reasons, and only shown in the backup list.
OK, I must be having some more fundamental issue. I just did the following:
How can the program report that it is unable to find the files that it has just created?
Just to eliminate FTP issues, I used W11 Terminal to start ftp, logged in as the same user as defined in the Duplicati job, navigated to the NAS directory where the duplicati files are stored, and then I tested putting a file, getting the same file, and also getting one of the duplicati files. All these operations worked in ftp.
I am running 2.1.0.5_stable_2025-03-04.
It might be looking in the FTP root, although I don’t know why it’s inconsistent.
In original post, you had ftp://192.168.2.65//home which has double slash.
The developer knows more, but I think 2.1.0.4 removes one slash before home however 2.1.0.5 may not. Can you try Path on server leading slash removal?
I assumed that what you were suggesting was to change the value of the GUI field “Path on server” to remove the leading slash. I did this, so that the resulting command line Target URL was ftp://192.168.2.65/home/Backup/TB14G6/ON1. This made no difference, I get the exact same behaviour.
If I misunderstood your suggestion, please let me know.
You understood, but it was simply a guess based on recent WebDAV successes.
Do you know what older releases worked or didn’t? All of the 2.1 have a different FTP library, I think maybe because Microsoft was recommending third party FTP.
Duplicati already had that in FTP (Alternative), so FTP is now a mapping to it.
If you are on FTP and want to keep poking, you can do Export As Command-line and take that URL to test BackendTool list variations in search of backup files.
There have been some surprising results with FTP that (IIRC) FTP Alternative fixed. I don’t have the post handy, but can probably dig it up if that helps for you…
Still, if regular FTP can’t list files, that’s a pretty big deal. What OS and what FTP?
EDIT 1:
FWIW a new FTP job worked for me on Windows to Filezilla Server on Windows, after I allowed it to have 3 connections instead of 2 (probably an unrelated issue).
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>Duplicati.CommandLine.BackendTool list "ftp://localhost/test 2?auth-username=duplicati&auth-password=password"
Name Dir/File LastChange Size
duplicati-20250314T141222Z.dlist.zip File 3/14/2025 2:12:23 PM 665 bytes
duplicati-bc197719e4b0443d885ba6c60f555f0d5.dblock.zip File 3/14/2025 2:12:23 PM 768 bytes
duplicati-i5e5eeced25e54ec18f23b0cfdac91836.dindex.zip File 3/14/2025 2:12:23 PM 623 bytes
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>
I can’t recall exactly when things stopped working.
My client is W11. As I wrote above, ftp from the Windows command line works fine both to put and get files. I also use WinSCP against my NAS all the time. The NAS is a QNAP running their implementation of FTP. The SyncBack software works without problems against the same NAS using FTP.
I used the backend command (LIST). It shows none of the files in the target directory:
Name Dir/File LastChange Size
I also tried the backend command with PUT and GET. Both work fine against the same target directory.
Just to be safe, I also tested changing all directories in the path on the NAS to 777. No difference.
“ls” in the W11 ftp client works fine against the target directory, it lists all the duplicati files.
I don’t have one to try. Often with FTP one must experiment.
Several of the developers were involved with doing new FTP.
The somewhat recent topic I remembered looks different than this, but said:
So hopefully the devs can get more of an idea. Meanwhile, try FTP (Alternative)?
Trying out a new option that looks very useful for sorting out weird FTP problems:
--ftp-log-to-console = false
Use this option to log FTP dialog to terminal console for debugging purposes.
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>Duplicati.CommandLine.BackendTool list "ftp://localhost/test 2?auth-username=duplicati&auth-password=password" --ftp-log-to-console
Name Dir/File LastChange Size
# Connect(False)
Status: FluentFTP 50.1.0.0(.NET 6.0) AsyncFtpClient
Status: Connecting(async) AsyncFtpClient.FtpSocketStream(control) IP #1 = ***:21
Status: Waiting for a response
Response: 220-FileZilla Server 1.9.4
Response: 220 Please visit https://filezilla-project.org/ [739323.66d]
Status: Detected FTP server: FileZilla
Command: USER ***
Status: Waiting for response to: USER ***
Response: 331 Please, specify the password. [<1ms]
Command: PASS ***
Status: Waiting for response to: PASS ***
Response: 230 Login successful. [62ms]
Command: FEAT
Status: Waiting for response to: FEAT
Response: 211-Features:
Response: MDTM
Response: REST STREAM
Response: SIZE
Response: MLST type*;size*;modify*;perm*;
Response: MLSD
Response: AUTH SSL
Response: AUTH TLS
Response: PROT
Response: PBSZ
Response: UTF8
Response: TVFS
Response: EPSV
Response: EPRT
Response: MFMT
Response: 211 End [1ms]
Status: Text encoding: System.Text.UTF8Encoding+UTF8EncodingSealed
Command: OPTS UTF8 ON
Status: Waiting for response to: OPTS UTF8 ON
Response: 202 UTF8 mode is always enabled. No need to send this command [<1ms]
Command: SYST
Status: Waiting for response to: SYST
Response: 215 UNIX emulated by FileZilla. [<1ms]
Status: Active ServerHandler is: FileZilla
Status: Listing parser set to: Machine
Command: PWD
Status: Waiting for response to: PWD
Response: 257 "/" is current directory. [<1ms]
Status: Daemon is initialized
# GetListing("/test 2", SizeModify)
Command: TYPE I
Status: Waiting for response to: TYPE I
Response: 200 Type set to I [<1ms]
# OpenDataStreamAsync("MLSD /test 2", 0)
# OpenPassiveDataStreamAsync(AutoPassive, "MLSD /test 2", 0)
Command: EPSV
Status: Waiting for response to: EPSV
Response: 229 Entering Extended Passive Mode (|||55313|) [<1ms]
Status: Connecting(async) AsyncFtpClient.FtpSocketStream(data) IP #1 = ***:55313
Command: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
Response: 150 Starting data transfer. [<1ms]
+---------------------------------------+
Listing: type=file;size=665;modify=20250314141223.998;perms=awrfd; duplicati-20250314T141222Z.dlist.zip
Listing: type=file;size=768;modify=20250314141223.316;perms=awrfd; duplicati-bc197719e4b0443d885ba6c60f555f0d5.dblock.zip
Listing: type=file;size=623;modify=20250314141223.776;perms=awrfd; duplicati-i5e5eeced25e54ec18f23b0cfdac91836.dindex.zip
-----------------------------------------
Status: Disposing(async) AsyncFtpClient.FtpSocketStream(data)
# CloseDataStream()
Status: Waiting for response to: MLSD /test 2
Response: 226 Operation successful [10ms]
duplicati-20250314T141222Z.dlist.zip File 3/14/2025 2:12:23 PM 665 bytes
duplicati-bc197719e4b0443d885ba6c60f555f0d5.dblock.zip File 3/14/2025 2:12:23 PM 768 bytes
duplicati-i5e5eeced25e54ec18f23b0cfdac91836.dindex.zip File 3/14/2025 2:12:23 PM 623 bytes
# Dispose()
Status: Warning: sync dispose called for AsyncFtpClient object
# DisposeAsync()
Status: Disposing(async) AsyncFtpClient
# Disconnect()
Command: QUIT
Status: Waiting for response to: QUIT
Response: 221 Goodbye. [<1ms]
Status: Disposing(async) AsyncFtpClient.FtpSocketStream(control)
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>
Thanks so much for all your help. Unfortunately, FTP (Alternative) makes no difference.
However, your full log option helped me find that somehow the ftp command successfully puts and gets files with the specified path, but with LIST it says that the directory does not exist. Changing the path to the NAS from /home/Backup… to /homes/< user name >/Backup fixed my issue! 
I am not technically clever enough to determine whether this strange difference in how FTP behaves is due to the client or server functionality, but in any case my issue is fixed. THANK YOU!
I’m glad you found a way out. FTP seems to do things a lot of different ways.
To finish my writeup (maybe dev will be interested) an option tries to support
absolute and relative paths in different ways. What worries me is that it isn’t
changing the request that I think is asking for the directory listing. Should it?
A more compact view is available by picking out the listing request.
Now it’s easier to test variations, such as an extra slash or this one:
--ftp-absolute-path = false Use this option to interpret the url path as an absolute path. This option only has an effect if the initial starting folder in the FTP server is not the (virtual) root folder. If not set, the path in the url is treated as relative to the initial login folder.
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>Duplicati.CommandLine.BackendTool list "ftp://localhost/test 2?auth-username=duplicati&auth-password=password" --ftp-log-to-console --ftp-absolute-path=false | findstr MLSD
Response: MLSD
# OpenDataStreamAsync("MLSD /test 2", 0)
# OpenPassiveDataStreamAsync(AutoPassive, "MLSD /test 2", 0)
Command: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>Duplicati.CommandLine.BackendTool list "ftp://localhost/test 2?auth-username=duplicati&auth-password=password" --ftp-log-to-console --ftp-absolute-path=true | findstr MLSD
Response: MLSD
# OpenDataStreamAsync("MLSD /test 2", 0)
# OpenPassiveDataStreamAsync(AutoPassive, "MLSD /test 2", 0)
Command: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>Duplicati.CommandLine.BackendTool list "ftp://localhost//test 2?auth-username=duplicati&auth-password=password" --ftp-log-to-console --ftp-absolute-
path=false | findstr MLSD
Response: MLSD
# OpenDataStreamAsync("MLSD /test 2", 0)
# OpenPassiveDataStreamAsync(AutoPassive, "MLSD /test 2", 0)
Command: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>Duplicati.CommandLine.BackendTool list "ftp://localhost//test 2?auth-username=duplicati&auth-password=password" --ftp-log-to-console --ftp-absolute-path=true | findstr MLSD
Response: MLSD
# OpenDataStreamAsync("MLSD /test 2", 0)
# OpenPassiveDataStreamAsync(AutoPassive, "MLSD /test 2", 0)
Command: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
Status: Waiting for response to: MLSD /test 2
C:\Duplicati\duplicati-2.1.0.5_stable_2025-03-04-win-x64-gui>
EDIT 1:
2.0.8.1 seems to do the same four combinations without a leading slash,
although it did do a PWD to find itself at /. Then a CWD test2, then a list.
EDIT 2:
Regarding the initial starting folder in the FTP server, the lack of effect from the option might be because mine is in fact FTP virtual root, as evidenced by:
PWD
257 "/" is current directory.
and I’m not sure what to conclude about behavior where user Path has initial slash.
EDIT 3:
FileZilla Server screen of the path config has it as follows. Other servers may vary:
