Atozed Forums

I have a bizarre problem with FTP.  I have a small program that has been working for years.  It's purpose is to upload files from our office to our website to be used by a plugin on the site.  It no longer works from the office computer that it runs on.  It works perfectly from my development computer at home.  I can't even figure out how to debug the issue.  Here's the code which does report to me each day on success/failure.  I've tried inserting messages to let me run it
on the office computer remotly but haven't get a good response.  Where can I find a list of commands for the idFTP module?
dbFTP->Passive  = true;
dbFTP->Port    = 21;
dbFTP->Host     = "hostname;
dbFTP->Username = "usernamer";
dbFTP->Password = "password";
String resultstext = "database ftp unknown failure";      // email report text
dbFTP->Connect();
try {
     if (dbFTP->Connected())   {
        dbFTP->ChangeDir("/public_html/Map");
        GetRemoteFilenames();
        Delay(1);
        CopyDBtoFTPServer();       // copies files from aisla folder to ftp server
        GetRemoteFilenames();      // check on results
        resultstext = "Backed up: ";
     }
}
catch(...)  {
     resultstext = "database backup failed";
}
dbFTP->Disconnect();
Main->SendaneMail(resultstext);

(10-31-2022, 07:23 PM)OldBob1938 Wrote: [ -> ]It no longer works from the office computer that it runs on.

Can you be more specific? What is the exact problem you are having with it?

(10-31-2022, 07:23 PM)OldBob1938 Wrote: [ -> ]Here's the code which does report to me each day on success/failure.

You don't need the call to Connected(), as Connect() will raise an exception if it fails - which you are not catching! You should move Connect() inside the try block, eg:


What does GetRemoteFilenames() do, and why is it being called twice?

Why are you using Delay(1)?

Your catch block is discarding all useful error information. If you catch an Exception object instead (or in addition to '...'), then you will have access to the exception's class type, error message, etc, which you can then include in your failure report, eg:

(10-31-2022, 07:23 PM)OldBob1938 Wrote: [ -> ]Where can I find a list of commands for the idFTP module?

What commands are you interested in? If you look at the declaration of the TIdFTP class in the IdFTP.hpp header file, you will see the available commands you can call. If you want to know the low-level FTP commands that TIdFTP implements, you would have to look at its source code in IdFTP.pas.

I have improved my exception handling and discover that on the office system after connection I get the message "entering extended passive mode" and I have no idea as to how to proceed.  The office has 2 data inputs ATT & TMobile.  My code does work as intended when I switch to ATT DSL but we had elected to make TMobile the primary as it's lots faster.  Not my decision.  I'm just plain lost.  

I read that the ftp server expects me to issue an EPSV command and create a data socket and connect it to the IP address of the control connection and port number indicated in the EPSV reply.  (1) I don't understand what I should be doing and (2) why is the server responding differently if I use TMobile or ATT?  The server is the same in both cases and in the case when I use the program from home (again a different ISP).

(11-09-2022, 04:32 PM)OldBob1938 Wrote: [ -> ]on the office system after connection I get the message "entering extended passive mode"

Are you seeing that in a thrown exception? That should not be getting thrown as an exception, since that is not an error condition to begin with. Can you please provide a log showing the actual FTP commands/responses that are being transmitted? You can assign one of Indy's TIdLog... components to the TIdFTP.Intercept property to get that log.

(11-09-2022, 04:32 PM)OldBob1938 Wrote: [ -> ]and I have no idea as to how to proceed.

Ideally, you should not have to do anything. TIdFTP supports "extended passive mode", considering it is the one asking for that mode in the first place (by sending an EPSV command instead of a PASV command).

(11-09-2022, 04:32 PM)OldBob1938 Wrote: [ -> ]The office has 2 data inputs ATT & TMobile.

Are they active at the same time? Are you manually configuring TIdFTP to force it to use one service vs the other? If so, then you may have to use the TIdFTP.OnDataChannelCreate event to manually configure the outbound data connection in a similar way.

(11-09-2022, 04:32 PM)OldBob1938 Wrote: [ -> ]I read that the ftp server expects me to issue an EPSV command and create a data socket and connect it to the IP address of the control connection and port number indicated in the EPSV reply.

That is correct, and TIdFTP should be handling all of that for you.

(11-09-2022, 04:32 PM)OldBob1938 Wrote: [ -> ](1) I don't understand what I should be doing

Ideally, you should not be doing anything. This is TIdFTP's responsibility to manage for you.

(11-09-2022, 04:32 PM)OldBob1938 Wrote: [ -> ](2) why is the server responding differently if I use TMobile or ATT?

It should not be, as it doesn't know which system you are using. What makes you think the server is sending different responses? What do the responses actually look like? It should just be sending you a Port number and an (optional) IP/Host to connect to. The rest is on TIdFTP to manage.

If necessary, you might try setting TIdFTP.PassiveUseControlHost=True, which will force TIdFTP to connect its data connection to the same IP address that the control connection is already connected to, regardless of any IP/Host specified in the EPSV response.

Hi Remy, Thanks for being so patient with me once again. I have tried adding the suggested setting without Joy. I've listed my current code below. This works perfectly on my home system and copies the desired files to the ftp server. I verify that they are on the server with FileZilla. I've copied the executable to the office server and monitored the process. When I run the program from the office the program fails after the line BobMessage("step2" with the message "entering extended passive mode ..." It never reached step 3 from the office. I can copy files to the ftp server using FileZilla with the same login credentials.

void __fastcall TSendFTP::dbConnectClick(TObject *Sender)
{
dbFTP->Passive = true;
dbFTP->Port = 21;
dbFTP->PassiveUseControlHost = true;
dbFTP->Host = myhost;
dbFTP->Username = myusername;
dbFTP->Password = mypassword;
String resultstext = "ftp unknown failure"; // email report text
bool success = false;

try
{
dbFTP->Connect();
try
{
dbFTP->ChangeDir("//public_html//Map");
Main->BobMessage("Step 2 ");
GetRemoteFilenames();
Main->BobMessage("Step 3 ");
CopyDBtoFTPServer(); // copies files from aisla folder to ftp server
GetRemoteFilenames(); // check on results
resultstext = "Backed up: ";
success = true;
}
catch(const Exception &ex) {
resultstext = "database backup failed, error: [" + const_cast<Exception&>(ex).ClassName() + "] " + ex.Message;
}
catch(...) {
resultstext = "database ftp unknown failure"; ;
}
}
catch(const Exception &ex) {
resultstext = "database backup failed, error: [" + const_cast<Exception&>(ex).ClassName() + "] " + ex.Message;
}
catch(...) {
resultstext = "database ftp unknown failure"; ;
}
if (success == true)
dbFTP->Disconnect();
Main->BobMessage(resultstext.c_str());
}

(11-12-2022, 06:00 PM)OldBob1938 Wrote: [ -> ]Hi Remy, Thanks for being so patient with me once again.  I have tried adding the suggested setting without Joy. I've listed my current code below.

I need to see the raw FTP command/response traffic that is being transmitted back and forth, and the complete error that you are seeing.

(11-12-2022, 06:00 PM)OldBob1938 Wrote: [ -> ]When I run the program from the office the program fails after the line BobMessage("step2" with the message "entering extended passive mode ..."  It never reached step 3 from the office.

You have not shown the code for GetRemoteFilenames(), but in any case, I see no possible way for that particular message to be raised as an exception, unless maybe if the FTP traffic is corrupted or out of sync. That is why I need to see the raw FTP command/response traffic. Please provide that log, as I described earlier (just mask out sensitive details, of course).

TIdFTP should not even be raising an exception on a failed EPSV command (and that message you quote does not represent a failure to begin with), it just retries with PASV instead.

Remy, thanks again for your patience.  I struggle with things that are not in my skillset.  I'm attaching the Log file (myerrors.txt)that I received when running my programon the office server.   I've also added the log file when running from my home office (success from home.txt).  The office uses a tp link router that has 2 wan sources ATT & T-Mobile.  The owner wants the T-Mobile source to be dominant because it's way faster and to use ATT for failover.  If I disconnect T-Mobile the program runs just fine.  I've discovered that there are a lot of people who are having trouble with ftp over T-Mobile's modem.  I have no idea as to how to cause the tp-link router to switch to ATT when this program runs.  I've finally included the code for GetRemoteFilenames which seems to be where it fails at the office.

void __fastcall TSendFTP::GetRemoteFilenames()
{
     dbFTP->List(NULL,"",true);
     int lr = dbFTP->ListResult->Count;
     String anItem,filename;
     TLog->Clear();
     for (int i = 0; i < lr; i++ )  
     {
          anItem = dbFTP->ListResult->Strings[i].Trim();
          if (BW("type=file",anItem))
          {
                filename = ExtractFilename(anItem);
                TLog->Add(filename);
          }
     }
}

(11-19-2022, 09:12 PM)OldBob1938 Wrote: [ -> ]I'm attaching the Log file (myerrors.txt)that I received when running my programon the office server.   I've also added the log file when running from my home office (success from home.txt).

The problem is not your faulty, or even TIdFTP's.  The FTP server (or a proxy/router on the network between you and the server) is sending a bad response.

In the scenario that works, TIdFTP sends a PASV command and receives a proper PASV response:


In the scenario that fails, TIdFTP sends a PASV command but receives an EPSV response instead of a PASV response:


The only response code that TIdFTP accepts for PASV is 227, not 229, which is why TIdFTP is throwing this response as an error, as it rightly should.  There is a flaw in the communication.  You need to take that up with the network/server admin.

Note: even though the FTP server is advertising support for EPSV, TIdFTP is not using it because its UseExtensionDataPort property is false by default. You can try setting it to true instead to enable EPSV.

(11-19-2022, 09:12 PM)OldBob1938 Wrote: [ -> ]I've finally included the code for GetRemoteFilenames which seems to be where it fails at the office.

Just an FYI, instead of using the TIdFTP::ListResult property, you should consider using the TIdFTP::DirectoryListing property instead, let TIdFTP parse the results for you, eg: