Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
ProgressBar while FTP retreive
#1
I need users to download large files (up to 800 MB)  using Indy FTP.

I am able to update the progress bar using TTask for some other lengthy processes but as soon as I start the FTP get file the procress bar stops updating.

Does anyone have a soultion for this issue.


TIA
Reply
#2
(05-18-2023, 03:51 AM)zsleo Wrote: I need users to download large files (up to 800 MB)  using Indy FTP.

I am able to update the progress bar using TTask for some other lengthy processes but as soon as I start the FTP get file the procress bar stops updating.

Like most components in Indy, TIdFTP blocks the calling thread until the requested operation is finished.  Are you performing the FTP transfer in the context of the main UI thread or a worker thread?

If the former, are you using TIdAntiFreeze to keep servicing the main UI message queue while the FTP transfer is busy?

If the latter, are you synchronizing with the main UI thread, such as with TThread.Synchronize()/TThread.Notify(), or TIdSync/TIdNotify, etc?

Can you provide your actual FTP code?

Reply
#3
(05-18-2023, 07:54 PM)rlebeau Wrote:
(05-18-2023, 03:51 AM)zsleo Wrote: I need users to download large files (up to 800 MB)  using Indy FTP.

I am able to update the progress bar using TTask for some other lengthy processes but as soon as I start the FTP get file the procress bar stops updating.

Like most components in Indy, TIdFTP blocks the calling thread until the requested operation is finished.  Are you performing the FTP transfer in the context of the main UI thread or a worker thread?

If the former, are you using TIdAntiFreeze to keep servicing the main UI message queue while the FTP transfer is busy?

If the latter, are you synchronizing with the main UI thread, such as with TThread.Synchronize()/TThread.Notify(), or TIdSync/TIdNotify, etc?

Can you provide your actual FTP code?

TidAntiFreeze is in the ServerController.  

I have TidFTPClient in UserSessionUnit with the following events
Code:
procedure TIWUserSession.idftp1Work(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
begin
  gv_FTPFilePercent := round((AWorkCount / gv_FTPFileSize) * 100);
end;

procedure TIWUserSession.idftp1WorkBegin(ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
begin
  gv_FTPDownloading := True;

end;

procedure TIWUserSession.idftp1WorkEnd(ASender: TObject; AWorkMode: TWorkMode);
begin
  gv_FTPDownloading := False;
end;

procedure TIWUserSession.GetFile(ip_FN: string);
begin
  Ret := 0;
  gv_FetchCompleted := False;
  idftp1.Host := 'ftp.mednet.com.au';
  idftp1.Username := [UserName];
  idftp1.Password := [Password];
  idftp1.Port := 21;
  idftp1.Passive := True;
  idftp1.TransferType := ftBinary;

  try
    idftp1.Connect;

    if FileExists(sv_AppDir + ip_FN) then
      Delete_File(sv_AppDir + ip_FN);
    gv_FTPFileSize := idftp1.Size([FTP_Path] + ip_FN);
    idftp1.Get([FTP_Path] + ip_FN, sv_AppDir + ip_FN, True, True);
    idftp1.Disconnect;

    if FileExists(sv_AppDir + ip_FN) then
      gv_FetchCompleted := Z7Extract(sv_AppDir, sv_AppDir + ip_FN);

  except
    on E: Exception do
    begin
      if idftp1.Connected then
        idftp1.Disconnect;
      if sv_Testing then
        doEventNotify(nt_Error, E.Message);
    end;
  end;
end;

In my Main Form I have an TIWTimer and Button with its OnClick calls the following procedure
Code:
procedure TiwfrmMain.OnDoFTP(Sender: TObject; AParams: TStringList);
begin

  UserSession.gv_FTPFilePercent := 0;

  TTask.Run(
    procedure
    begin
      while not gv_ToStop do
      begin
        sleep(200);
      end;
    end);

  iwtmr1.Tag := 1;
  iwtmr1.Enabled := True;

end;

TIWTimer.Interval is set to 200 and the async event as follows
Code:
procedure TiwfrmMain.iwtmr1AsyncTimer(Sender: TObject; EventParams: TStringList);
var
  lv_S: string;
  lv_I: integer;
begin
  if iwtmr1.Tag = 1 then
  begin
    iwtmr1.Tag := 0;
      UserSession.gv_FTPDownloading := True;
      UserSession.GetFile([File_to_get]);

  end
  else
    iwcgjqsyprgrs1.SetProgress(UserSession.gv_FTPFilePercent / 100); { <<< updating the progress bar}

 
  if not UserSession.gv_FTPDownloading then
  begin
    iwtmr1.Enabled := False;
    gv_ToStop := True;

    {Extract the ZIP File}
    
  end;

end;
Reply
#4
(05-19-2023, 03:11 AM)zsleo Wrote: TidAntiFreeze is in the ServerController.  

I'm not familiar with IntraWeb or how it works.  What thread does the ServerController run in?  TIdAntiFreeze does nothing if invoked in a worker thread.

(05-19-2023, 03:11 AM)zsleo Wrote: In my Main Form I have an TIWTimer and Button with its OnClick calls the following procedure

Why are you running a TTask that effectively does nothing?  All it does is Sleep() until gv_ToStop is True.  That is a useless waste of a worker thread.

(05-19-2023, 03:11 AM)zsleo Wrote: TIWTimer.Interval is set to 200 and the async event as follows

You are calling GetFile() (and thus running TIdFTP) in the context of the thread that fires the timer event, so you are blocking your timer while the FTP transfer is in progress, which is why the timer is not able to update your ProgressBar.

I strongly suggest that you move your FTP transfer logic to a separate worker thread. Do not block your timer handler.

Reply
#5
(05-19-2023, 04:25 PM)rlebeau Wrote:
(05-19-2023, 03:11 AM)zsleo Wrote: TidAntiFreeze is in the ServerController.  

I'm not familiar with IntraWeb or how it works.  What thread does the ServerController run in?  TIdAntiFreeze does nothing if invoked in a worker thread.

(05-19-2023, 03:11 AM)zsleo Wrote: In my Main Form I have an TIWTimer and Button with its OnClick calls the following procedure

Why are you running a TTask that effectively does nothing?  All it does is Sleep() until gv_ToStop is True.  That is a useless waste of a worker thread.

(05-19-2023, 03:11 AM)zsleo Wrote: TIWTimer.Interval is set to 200 and the async event as follows

You are calling GetFile() (and thus running TIdFTP) in the context of the thread that fires the timer event, so you are blocking your timer while the FTP transfer is in progress, which is why the timer is not able to update your ProgressBar.

I strongly suggest that you move your FTP transfer logic to a separate worker thread.  Do not block your timer handler.

The ServerController is the main thread hence TidAntiFreeze being placed there.

The TTask was experimental and it has been removed.

Thanks for the advice. I will change the test FTP implementation you suggested.

Thanks again.
Reply
#6
The issue you're experiencing with the progress bar not updating during the FTP file transfer process might be due to blocking the UI thread. The FTP Get operation is likely blocking the UI thread, preventing it from updating the progress bar.

To address this issue, you can perform the FTP file transfer operation in a separate thread or task, allowing the UI thread to continue updating the progress bar. Here's an example of how you can achieve this using TThread:

Code:
procedure TForm1.DownloadFile(const AHost, AUsername, APassword, ARemoteFile, ALocalFile: string);
var
  FTP: TIdFTP;
begin
  FTP := TIdFTP.Create(nil);
  try
    FTP.Host := AHost;
    FTP.Username := AUsername;
    FTP.Password := APassword;
   
    // Set up event handlers to track progress
    FTP.OnWork := FTPWork;
    FTP.OnWorkBegin := FTPWorkBegin;
    FTP.OnWorkEnd := FTPWorkEnd;

    // Connect and download the file in a separate thread
    TThread.CreateAnonymousThread(
      procedure
      begin
        try
          FTP.Connect;
          FTP.Get(ARemoteFile, ALocalFile);
        finally
          FTP.Disconnect;
        end;
      end
    ).Start;

    // Show the progress bar and start updating it
    ProgressBar1.Visible := True;
    ProgressBar1.Position := 0;

    // Wait for the download to complete
    while not FTP.Connected do
      Sleep(100);

    // You can perform any post-download actions here

  finally
    FTP.Free;
  end;
end;

procedure TForm1.FTPWork(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
begin
  // Update the progress bar based on the current work count
  ProgressBar1.Position := AWorkCount;
  Application.ProcessMessages; // Allow UI updates
end;

procedure TForm1.FTPWorkBegin(ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
begin
  // Set the progress bar's maximum value
  ProgressBar1.Max := AWorkCountMax;
end;

procedure TForm1.FTPWorkEnd(ASender: TObject; AWorkMode: TWorkMode);
begin
  // Hide the progress bar when the FTP work ends
  ProgressBar1.Visible := False;
end;

In this example, the DownloadFile procedure sets up an FTP connection and starts the download in a separate thread using TThread.CreateAnonymousThread. The progress bar is updated in the FTPWork event handler, which is called during the FTP operation. The FTPWorkBegin event handler sets the maximum value of the progress bar, and the FTPWorkEnd event handler hides the progress bar when the FTP work is complete.

Make sure to replace the FTP connection details (AHost, AUsername, APassword) and the file paths (ARemoteFile, ALocalFile) with your specific values.

By executing the FTP operation in a separate thread, the UI thread remains responsive and can continue updating the progress bar.
Reply
#7
(06-22-2023, 03:51 PM)Neerarawat Wrote: The issue you're experiencing ...

First off, that felt like a ChatGPT-generated answer to me, but I'm going to address it anyway, because it provided a lot of misinformation that will get you into trouble.

(06-22-2023, 03:51 PM)Neerarawat Wrote: Here's an example of how you can achieve this using TThread

DO NOT call TIdFTP.Connected() from a different thread than the one that is performing the FTP operations.  Connected() determines the socket state by reading from the socket, so you would be reading socket data from 2 threads at the same time, and thus risking the IOHandler.InputBuffer becoming corrupted, which would screw up the FTP operations.

A safer approach would be to wait for the thread itself to terminate after the FTP work is finished.  Note that TThread.CreateAnonymousThread() creates a TThread object whose FreeOnTerminate property is set to True, which means you would not be able to use the TThread.WaitFor() method by default without crashing your code.  But, you can just turn off FreeOnTerminate before starting the anonymous thread.

Also, since the FTP operations are being executed in a worker thread, the FTP.OnWork... event handlers will be called in the context of that thread, not in the context of the main UI thread.  So, you would need to synchronize with the main thread when accessing UI controls.

Try something more like this instead:

Code:
procedure TForm1.DownloadFile(const AHost, AUsername, APassword, ARemoteFile, ALocalFile: string);
var
  LThread: TThread;

  procedure WaitForThread;
  var
    H: THandle;
  begin
    // LThread.WaitFor;
    H := LThread.Handle;
    repeat
      case MsgWaitForMultipleObjects(1, H, False, INFINITE, QS_ALLINPUT) of
        WAIT_OBJECT_0+0: Break;
        WAIT_OBJECT_0+1: Application.ProcessMessages;
      else
        RaiseLastOSError;
      end;
    until False;
  end;

begin
  // Connect and download the file in a separate thread
  LThread := TThread.CreateAnonymousThread(
    procedure
    var
      FTP: TIdFTP;
    begin
      FTP := TIdFTP.Create(nil);
      try
        FTP.Host := AHost;
        FTP.Username := AUsername;
        FTP.Password := APassword;
   
        // Set up event handlers to track progress
        FTP.OnWork := Self.FTPWork;
        FTP.OnWorkBegin := Self.FTPWorkBegin;
        FTP.OnWorkEnd := Self.FTPWorkEnd;

        FTP.Connect;
        try
          FTP.Get(ARemoteFile, ALocalFile);
        finally
          FTP.Disconnect;
        end;
      finally
        FTP.Free;
      end;
    end
  );
  try
    LThread.FreeOnTerminate := False;
    LThread.Start;

    // Wait for the download to complete
    WaitForThread;
  finally
    LThread.Free;
  end;

  // You can perform any post-download actions here
end;

procedure TForm1.FTPWork(ASender: TObject; AWorkMode: TWorkMode; AWorkCount: Int64);
begin
  // Update the progress bar based on the current work count
  TThread.Queue(nil,
    procedure
    begin
      ProgressBar1.Position := AWorkCount;
    end
  );
end;

procedure TForm1.FTPWorkBegin(ASender: TObject; AWorkMode: TWorkMode; AWorkCountMax: Int64);
begin
  // Show the progress bar and set its maximum value
  TThread.Queue(nil,
    procedure
    begin
      ProgressBar1.Visible := True;
      ProgressBar1.Position := 0;
      ProgressBar1.Max := AWorkCountMax;
    end
  );
end;

procedure TForm1.FTPWorkEnd(ASender: TObject; AWorkMode: TWorkMode);
begin
  // Hide the progress bar when the FTP work ends
  TThread.Queue(nil,
    procedure
      ProgressBar1.Visible := False;
    end
  );
end;

Which, to be honest, is really no better than simply eliminating the anonymous thread altogether and just having DownloadFile() use TIdFTP directly in the context of the calling thread, and using TIdAntiFreeze to avoid blockage if DownloadFile() is being called in the context of the main UI thread:

Code:
procedure TForm1.DownloadFile(const AHost, AUsername, APassword, ARemoteFile, ALocalFile: string);
var
  FTP: TIdFTP;
begin
  FTP := TIdFTP.Create(nil);
  try
    FTP.Host := AHost;
    FTP.Username := AUsername;
    FTP.Password := APassword;
   
    // Set up event handlers to track progress
    FTP.OnWork := FTPWork;
    FTP.OnWorkBegin := FTPWorkBegin;
    FTP.OnWorkEnd := FTPWorkEnd;

    FTP.Connect;
    try
      FTP.Get(ARemoteFile, ALocalFile);
    finally
      FTP.Disconnect;
    end;
  finally
    FTP.Free;
  end;

  // You can perform any post-download actions here
end;

That being said, regardless of whichever thread ends up using TIdFTP, do note that for TIdFTP.Get() (but not for TIdFTP.Put()), the AWorkCountMax parameter of the TIdFTP.OnWorkBegin event will always be 0, because the FTP protocol does not advertise data sizes during transfers.  So, you would need to use either TIdFTP.Size(), TIdFTP.List(), or TIdFTP.ExtListItem() to determine a file's size before you then use TIdFTP.Get() to download the file.  I'll leave that as an exercise for you to add that to the above example.

Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)