TIdTCPServer with many concurrent connections

[vince99]

Hello,

I have a TIdTcpServer project with many clients that have permanent connections.  At the moment I am having 2300 concurrent connections (=2300 threads) all the time and I am running into performance problems.  The machine I am running the server application on is a Windows Server 2019 with 8 cores and 24 GB of RAM.  
I suspect the application starts running slow because of the thread scheduling of the operating system.

I have been reading about the IdSchedulerOfThreadPool that would use a thread pool instead of 1 thread per connection, but I also read something that contradicts that a  IdSchedulerOfThreadPool would help.

What is the best I can do to handle that many concurrent connections?

Vincent

[rlebeau]

I have been reading about the IdSchedulerOfThreadPool that would use a thread pool instead of 1 thread per connection

Then you have misread/misunderstood how it works.

TIdTCPServer always uses 1 thread per connection, regardless of the Scheduler. The Scheduler is merely responsible for providing a thread when a new client connects, and cleaning up the thread when the client disconnects, but the Scheduler is otherwise not involved in the workload of the client.

TIdSchedulerOfThreadDefault creates a new thread when a client connects, and terminates the thread when the client disconnects.

TIdSchedulerOfThreadPool, on the other hand, pulls an idle thread out of the pool when a client connects, and then puts the thread back into the pool when the client disconnects. Upon connect, if the pool is empty then a new thread is created. And upon disconnect, if the pool is full then the thread is terminated.

So, the pooling is on idle unconnected threads only. Not on threads waiting for work.

That being said, there was an attempt many years ago (the SuperCore package) to create a 3rd type of Scheduler that used fibers instead of threads, so multiple clients could be serviced on a single thread using overlapped I/O. But that project didn't work very well and was abandoned. But its source code is still available in Indy's repo.

but I also read something that contradicts that a  IdSchedulerOfThreadPool would help.

No, it would not. If you have 2300 concurrent clients, then you will have 2300 concurrent threads running.

What is the best I can do to handle that many concurrent connections?

Hard to answer that without seeing your actual code.

Running 2300 threads is not a problem if they are mostly sleeping, waiting on client input, etc (but if that is the case, then you have to question why those connections are left open).

But, if the threads are actively running full speed, all doing work at the same time, then you will likely need a different server model that is designed for asynchronous socket I/O and more efficient parallelism of the workloads.

Or, at least, run a few TIdTCPServer processes in parallel and put a load balancer in front of them.