TL;DR Revised PR approach looks canary ready. Maybe room for improvement, but lots of unknowns.
and probably unavoidably slow. Perhaps both should someday change to LIST just the file of interest (more below). SIZE is probably slow only if a confused NAS decides to read the whole file to compute transfer size. Proper protocol by my reading would realize it’s BINARY mode and just send file size as returned by the OS, but one never knows what silly things a program is going to do. Fastest answer is asking for and receiving file size as returned by the OS, but using MLST or LIST as described below.
If you look at the 2007 RFC 3659 - Extensions to FTP Table of Contents you can see that it added SIZE, MLST, and other extensions, so it’s not clear why one would be less supported than another. RFC 2389 - Feature negotiation mechanism for the File Transfer Protocol has a way for client to know what’s there.
FTP Commands and Extensions is the official IANA registry of extensions, citing RFCs. SIZE and MLST are listed as “optional”, which is another hole that either scheme could fall into. FluentFTP can fall back MLSD to LIST (which is mandatory). LIST possibly became mandatory in RFC 1123 - Requirements for Internet Hosts – Application and Support - 4.1.2.13 Minimum Implementation in 1989, so may be safest, given an assumption that the server actually implemented the single file capability required by RFC 959:
LIST (LIST)
This command causes a list to be sent from the server to the
passive DTP. If the pathname specifies a directory or other
group of files, the server should transfer a list of files
in the specified directory. If the pathname specifies a
file then the server should send current information on the
file. A null argument implies the user's current working or
default directory. The data transfer is over the data
connection in type ASCII or type EBCDIC. (The user must
ensure that the TYPE is appropriately ASCII or EBCDIC).
Since the information on a file may vary widely from system
to system, this information may be hard to use automatically
in a program, but may be quite useful to a human user.
From above, one can see that the format of LIST is already variable, so one would hope the server can handle the different cases (no guarantees – it might just handle only two of the three possible cases…). There’s also a client-side risk, as FluentFTP source seems to not realize LIST can do a single file, but it possibly doesn’t need to know the three-case processing because it’s somewhat transparent to a client.
What we really want is not MLSD but MLST, and GetObjectInfo can do that if the server has MLST, and internally falls back to LIST otherwise, so we still rely on server’s LIST of non-directory working properly.
Problems after upgrade to 2.0.4.21 was our discovery and discussion of the ASCII failure seen before, and we wound up fixing the library, but maybe now it’s time to try dropping use of its deprecated option despite some concerns you expressed there. My concern would be that I don’t understand the threads that might come through during something like exception processing, and that’s always tough to test…
If I read this correctly, this revision still makes --disable-upload-verify work again (good), while adding --aftp-upload-delay as completely optional and doing nothing unless set to guard against possible but predictable (compared to transfer) delay (good), now turning off EnableThreadSafeDataConnections (we’ll see, and the comment could be misconstrued – we do support parallel uploads in our own way).
There’s no longer a four-times-per-second poll for completion now that early return is resolved (good).
It still uses GetFileSize which was discussed at length above, wondering if a LIST-based way is safer. Good news is that if SIZE breaks, there’s at least one way out by not not doing the upload verify at all. Giving this a try is OK by me even though there might be a better way (never 100% clear what works).
Thanks so much for working on this. I assume it’s tested some, but there’s no substitute for field tests.