FTPS with Indy/Delphi to connect with special Visa FTPS server - Printable Version +- Atozed Forums (https://www.atozed.com/forums) +-- Forum: Indy (https://www.atozed.com/forums/forum-8.html) +--- Forum: Indy General Discussion (https://www.atozed.com/forums/forum-9.html) +--- Thread: FTPS with Indy/Delphi to connect with special Visa FTPS server (/thread-1187.html) |
FTPS with Indy/Delphi to connect with special Visa FTPS server - peak.tibor@gmail.com - 08-15-2019 Hi, any help would be useful. This Visa server is quite specific (I have tried two commercial FTP clients /WS_FTP and TurboFTO/, none of them could connect to this server), two examples: - requires a special sequence of FTP command during connection startup ( I could follow this making some modifications in IdFTP.pas) - behaves a little bit special when using the CCC option (I'm stuck with this) Server documentation states: Quote:The client is expected to initiate the TLS shutdown after receiving the 200 response to the CCC request. When I issue the CCC command, it receives the correct answer from the server, but after this the connection is broken. It seems the Indy Ftp client / OpenSSL solution handles this situation a little bit different as the server requires it, but I do not know how to change this. ANY HELP WOULD BE WELCOME! What I can see in the IdFtp.pas, after issuing the "CCC" command and receiving the acceptence from the server, the following is executed: Code: (IOHandler as TIdSSLIOHandlerSocketBase).PassThrough := True; And that's all!!! I do not know how could any change be applied to modify the client behaviour. Thanks in advance. Tibor Peak RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - rlebeau - 08-16-2019 (08-15-2019, 06:51 AM)peak.tibor@gmail.com Wrote: Server documentation states: TIdFTP automatically issues 'CCC' on the first transfer after FTPS is activated and TIdFTP.UseCCC is true. But, when Indy "clears" a secure command connection to downgrade it back to unsecure, it doesn't actually perform any kind of TLS shutdown at all, it merely flags the SSLIOHandler to stop encrypting/decrypting subsequent data on the socket. The TLS session is still active, just ignored. The only way to shutdown the TLS session without closing the socket is to call OpenSSL's SSL_shutdown() function directly. Per OpenSSL's documentation (emphasis is mine): Quote:SSL_shutdown() tries to send the "close notify" shutdown alert to the peer. Whether the operation succeeds or not, the SSL_SENT_SHUTDOWN flag is set and a currently open session is considered closed and good and will be kept in the session cache for further reuse. (08-15-2019, 06:51 AM)peak.tibor@gmail.com Wrote: What I can see in the IdFtp.pas, after issuing the "CCC" command and receiving the acceptence from the server, the following is executed: Yup. Setting PassThrough to True does not perform an actual TLS shutdown: Code: procedure TIdSSLIOHandlerSocketOpenSSL.SetPassThrough(const Value: Boolean); (08-15-2019, 06:51 AM)peak.tibor@gmail.com Wrote: I do not know how could any change be applied to modify the client behaviour. You would have to modify Indy so that immediately after 'CCC' succeeds, SSL_shutdown() is called until "close notify" events have been sent and received. Doing that in TIdSSLIOHandlerSocketOpenSSL.SetPassThrough() would be best. Only a few places in Indy ever set PassThrough to True after SSL/TLS has been activated, and most of those are in TIdFTP and TIdFTPServer for 'CCC' and 'REIN' handling. So I think it should be safe to handle SSL_shutdown() there. Try this: Code: procedure TIdSSLIOHandlerSocketOpenSSL.SetPassThrough(const Value: Boolean); RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - PeakTibor - 08-18-2019 Hi rlebeau, thanks for advising. I made that modification, using a SVN version 5515 of Indy. It really makes the SSL shutdown now, I can see the 'close notify' TLS messages in the WireShark log. But it fails when I try to get a file from the server with "Access violation at address 00407E8E in module 'IdFtp_Tests_2.exe'. Read of address 00000020" message that appears after a while (destination local file is created, but empty). I cannot really debug the program, before I cannot access the server from my developer pc, I can write only log messages to see at which step the program is executing. It seems to me the problem is in "TIdFTP.InitDataChannel" where "PassThrough" is set to true for the newly cloned IOHandler of the datachannel, and it causes the SSL_shutdown to be called for that IOHandler. Can you advise how to proceed? Thank you in advance. Tibor RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - rlebeau - 08-21-2019 (08-18-2019, 04:08 PM)PeakTibor Wrote: I made that modification, using a SVN version 5515 of Indy. Good. (08-18-2019, 04:08 PM)PeakTibor Wrote: But it fails when I try to get a file from the server with "Access violation at address 00407E8E in module 'IdFtp_Tests_2.exe'. Read of address 00000020" message that appears after a while (destination local file is created, but empty). An AV at a memory address near 0 usually indicates a nil pointer is being accessed. What does the call stack look like when the AV is raised? Which module inside your EXE does memory address $00407E8E reside in? (08-18-2019, 04:08 PM)PeakTibor Wrote: I cannot really debug the program, before I cannot access the server from my developer pc, I can write only log messages to see at which step the program is executing. Well, then you will likely have to install a 3rd party exception logger into your app, like MadExcept, EurekaLog, JclDebug, etc so that you can log those kind of details when the exception happens. (08-18-2019, 04:08 PM)PeakTibor Wrote: It seems to me the problem is in "TIdFTP.InitDataChannel" where "PassThrough" is set to true for the newly cloned IOHandler of the datachannel, and it causes the SSL_shutdown to be called for that IOHandler. Try changing the following line in SetPassThrough() from this: Code: if fSSLSocket <> nil then begin To this: Code: if (fSSLSocket <> nil) and (fSSLSocket.fSSL <> nil) then begin Something else I notice is that the fPassThrough member in IdSSL.pas is initialized to False by default. It probably should be initialized to True instead. Many places inside of Indy force PassThrough=True as soon as they create new instances of SSLIOHandler objects. RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - PeakTibor - 08-26-2019 Hi rlebeau, thanks for all your help. Finally I made some additional changes: 1. I created a separate "ShutDownSSL" procedure, the same was as Indy has a "StartSSL"procedure, and I call explicitly this new procedure after issuing the "CCC" command. 2. I created a separate procedure to issue the CCC ("SendCCC") and removed this part from "SendDataSettings". This specific VISA server does not accept the original sequence of commands in "SendDataSettings": SendPBSZ -> SendPROT -> CCC And I think it may have right. PBSZ and PROT should/could be set before login, CCC can be issued only after login, because sending userid and pwd in clear command channel is usually not acceptable. So, before login I call SendDataSettings, after login I call SendCCC, that sets PassThrough := True and calls ShutDownSSL. Maybe my changes are not generally appliable, but for my specific situation this is what is required. Thanks again Tibor RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - rlebeau - 08-28-2019 I have just now checked in some changes for TIdIOHandlerSocketOpenSSL, more inline with the changes I described earlier rather than the changes you mention above, though. (08-26-2019, 06:40 AM)PeakTibor Wrote: This specific VISA server does not accept the original sequence of commands in "SendDataSettings": SendPBSZ -> SendPROT -> CCC Are you sure about that? According to RFC 2228, PBSZ "must be preceded by a successful security data exchange", same as CCC (PROT "must be preceded by a successful protection buffer size negotiation"). That implies that none of these commands can be sent until after a "successful security data exchange", which is performed using the ADAT command after an AUTH command, but TIdFTP does not currently support/use ADAT. I haven't made any changes to TIdFTP yet, I'll have to look into that later. Feel free to open a ticket at https://github.com/IndySockets/Indy/issues so I don't forget about it. RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - PeakTibor - 08-30-2019 Hi rlebeau, I may missing something, so I try get help. The requested sequnce of FTP commands/activities with this Visa server is as follows: - AUTH TLS - TLS handshake - PBSZ 0 - PROT P (Data channel must be encrypted) - USER, PASS - CCC (if Clear Command Channel have to be used) Is it possible with the current idFTP solution to follow this? Thanks in advance Tibor RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - rlebeau - 08-30-2019 (08-30-2019, 05:50 AM)PeakTibor Wrote: The requested sequnce of FTP commands/activities with this Visa server is as follows: Possible, but not in a straight-forward manner. TIdFTP does have SendPBSZ() and SendPROT() methods, which you can call whenever you want. However, they are protected methods, so you would need to use a descendant/accessor class to reach them, as well as to set TIdFTP.FDataSettingsSent=True so that TIdFTP.SendDataSettings() does not send its own PBSZ/PROT/CCC commands. The TIdFTP.Login() method does not natively allow you to send custom commands after it sends AUTH and performs the TLS handshake, and before it sends USER/PASS/ACCT, so you would have to handle the login sequence manually. You could set TIdFTP.AutoLogin=False and handle the entire login sequence manually. Or, you could set TIdFTP.ProxySettings.ProxyType=fpcmCustomProxy and use the TIdFTP.OnCustomFTPProxy event to send PBSZ, PROT, and USER/PASS after TIdFTP.Login() sends AUTH and performs the TLS handshake for you. That just leaves CCC, which you can disable by setting TIdFTP.UseCCC=False and then send it manually when needed. RE: FTPS with Indy/Delphi to connect with special Visa FTPS server - PeakTibor - 08-31-2019 Hi rlebeau, thanks for advising. I think we have to consider this Visa server requirements as NON-STANDARD, so there is no need to build its support into the idFtp source. As I described above, I could manage to modify idFtp source to support this Visa server, but it should not be considered as general, so I will not open an ISSUE ticket. Thanks again for your support. Tibor |