FTPS with Indy/Delphi to connect with special Visa FTPS server

[rlebeau]

Server documentation states:

The client is expected to initiate the TLS shutdown after receiving the 200 response to the CCC request.

When I issue the CCC command, it receives the correct answer from the server, but after this the connection is broken. It seems the Indy Ftp client / OpenSSL solution handles this situation a little bit different as the server requires it, but I do not know how to change this.

TIdFTP automatically issues 'CCC' on the first transfer after FTPS is activated and TIdFTP.UseCCC is true.  But, when Indy "clears" a secure command connection to downgrade it back to unsecure, it doesn't actually perform any kind of TLS shutdown at all, it merely flags the SSLIOHandler to stop encrypting/decrypting subsequent data on the socket.  The TLS session is still active, just ignored.  The only way to shutdown the TLS session without closing the socket is to call OpenSSL's SSL_shutdown() function directly.

Per OpenSSL's documentation (emphasis is mine):

SSL_shutdown() tries to send the "close notify" shutdown alert to the peer. Whether the operation succeeds or not, the SSL_SENT_SHUTDOWN flag is set and a currently open session is considered closed and good and will be kept in the session cache for further reuse.

Note that SSL_shutdown() must not be called if a previous fatal error has occurred on a connection i.e. if SSL_get_error() has returned SSL_ERROR_SYSCALL or SSL_ERROR_SSL.

The shutdown procedure consists of 2 steps: the sending of the "close notify" shutdown alert and the reception of the peer's "close notify" shutdown alert. According to the TLS standard, it is acceptable for an application to only send its shutdown alert and then close the underlying connection without waiting for the peer's response (this way resources can be saved, as the process can already terminate or serve another connection). When the underlying connection shall be used for more communications, the complete shutdown procedure (bidirectional "close notify" alerts) must be performed, so that the peers stay synchronized.

SSL_shutdown() supports both uni- and bidirectional shutdown by its 2 step behaviour.

When the application is the first party to send the "close notify" alert, SSL_shutdown() will only send the alert and then set the SSL_SENT_SHUTDOWN flag (so that the session is considered good and will be kept in cache). SSL_shutdown() will then return with 0. If a unidirectional shutdown is enough (the underlying connection shall be closed anyway), this first call to SSL_shutdown() is sufficient. In order to complete the bidirectional shutdown handshake, SSL_shutdown() must be called again. The second call will make SSL_shutdown() wait for the peer's "close notify" shutdown alert. On success, the second call to SSL_shutdown() will return with 1.

If the peer already sent the "close notify" alert and it was already processed implicitly inside another function (SSL_read(3)), the SSL_RECEIVED_SHUTDOWN flag is set. SSL_shutdown() will send the "close notify" alert, set the SSL_SENT_SHUTDOWN flag and will immediately return with 1. Whether SSL_RECEIVED_SHUTDOWN is already set can be checked using the SSL_get_shutdown() (see also SSL_set_shutdown(3) call.

It is therefore recommended, to check the return value of SSL_shutdown() and call SSL_shutdown() again, if the bidirectional shutdown is not yet complete (return value of the first call is 0). As the shutdown is not specially handled in the SSLv2 protocol, SSL_shutdown() will succeed on the first call.

What I can see in the IdFtp.pas, after issuing the "CCC" command and receiving the acceptence from the server, the following is executed:

Code:

(IOHandler as TIdSSLIOHandlerSocketBase).PassThrough := True;

And that's all!!!

Yup.  Setting PassThrough to True does not perform an actual TLS shutdown:

Code:

procedure TIdSSLIOHandlerSocketOpenSSL.SetPassThrough(const Value: Boolean);
begin
  if fPassThrough <> Value then begin
    if not Value then begin
      // issues a TLS handshake here ...
    else
      // merely resets the socket buffer sizes here ...
    end;
    fPassThrough := Value;
  end;
end;

I do not know how could any change be applied to modify the client behaviour.

You would have to modify Indy so that immediately after 'CCC' succeeds, SSL_shutdown() is called until "close notify" events have been sent and received.  Doing that in TIdSSLIOHandlerSocketOpenSSL.SetPassThrough() would be best. Only a few places in Indy ever set PassThrough to True after SSL/TLS has been activated, and most of those are in TIdFTP and TIdFTPServer for 'CCC' and 'REIN' handling.  So I think it should be safe to handle SSL_shutdown() there.

Try this:

Code:

procedure TIdSSLIOHandlerSocketOpenSSL.SetPassThrough(const Value: Boolean);
begin
  if fPassThrough <> Value then begin
    if not Value then begin
      if BindingAllocated then begin
        if Assigned(fSSLContext) then begin
          OpenEncodedConnection;
        end else begin
          raise EIdOSSLCouldNotLoadSSLLibrary.Create(RSOSSLCouldNotLoadSSLLibrary);
        end;
      end;
    end
    else begin
      if fSSLSocket <> nil then begin
        if SSL_shutdown(fSSLSocket.fSSL) = 0 then begin
          SSL_shutdown(fSSLSocket.fSSL);
        end;
      end;
      {$IFDEF WIN32_OR_WIN64}
      // begin bug fix
      if BindingAllocated and IndyCheckWindowsVersion(6) then
      begin
        // disables Vista+ SSL_Read and SSL_Write timeout fix
        Binding.SetSockOpt(Id_SOL_SOCKET, Id_SO_RCVTIMEO, 0);
        Binding.SetSockOpt(Id_SOL_SOCKET, Id_SO_SNDTIMEO, 0);
      end;
      // end bug fix
      {$ENDIF}
    end;
    fPassThrough := Value;
  end;
end;