Feature/aftp update #3956 PR discussion follows, expanding on the summary below responding to:
Issues to address to get out of beta
WIP AFTP backend data upload corruption fix #3866 is the PR from earlier in current topic that hit my FTP server SIZE bug. The following comment concerns me in terms of getting the transfer times right.
Currently AFTP will seemingly transfer a file via async ftp and return ‘success’ even though it seems the transfer is still proceeding.
seems pretty rude behavior, but I can’t find any open or closed FluentFTP issues, so I wonder if there’s a chance it’s Duplicati code. There was a PR comment on the 5 second try, which is basically my worry even though the new PR splits the polling into a configurable timed delay and then hardcoded poll time.
I’d note that ignoring _listVerify was new with the first POC PR, and I’m glad to see the option returned.
The WIP POC PR is also where FluentFTP GetFileSize replaced Duplicati List (of everything I guess), followed by a search and size check for the file of interest. It seems better to ask for the file of interest, which FTP SIZE command can do, but with the gotcha that it’s the transfer size, opening a window for broken FTP servers to compute it wrong (e.g. as ASCII transfer). FTP LIST command can get a single file, however the server output format may vary. MLST was defined in 2007 in RFC 3659 to solve that. FluentFTP does know how to handle MLST, but it looks like it’s by GetObjectInfo instead of GetListing, and there’s a chance that an OLD server won’t have it. I don’t know if FluentFTP runs LIST as fallback.
and Duplicai does call GetListing in its List operation. I’m not sure why GetListing can’t do as well as GetFileSize at getting the file size on the remote (which should be OK, regardless of transfer mode).
File Transfer High-level API:
**Upload** () - Uploads a Stream or byte[] to the server. Returns true if succeeded, false if failed or file does not exist.
FluentFTP/FluentFTP.Examples/AsyncConnect.cs unfortunately seems to ignore the return code, yet I can’t spot anything wrong with how Duplicati does it. I’m not real familiar with C# async coding though.
EDIT:
Polling 4 times per second is probably making some poor confused FTP server read through the whole file that often trying to figure out the size after (mis)translation. I think I read that some servers refuse to compute the SIZE unless you’re in BINARY mode so a different confused server might hurt us that way. Listing a single file seems like the fastest poll, but better than polling might be to understand early finish.