After playing a bit with performance tests comparing different memory managers, I decided to go a little further and compare IntraWeb with something else. My first choice was an ASP.NET application. Why ASP.NET? Well, because both Delphi and C# share some common background, more specifically, Anders Hejlsberg, and also the similarities between modern Object Pascal and C# seem to make C# the preferred second (or first?) language for some Delphi developers.
How was the test conceived?
On the Delphi side, I’m using the same application that I created to compare memory managers: A simple web application with 2 forms in it, each one has an element (a button) to make the other form visible. According to my previous test, I’m using an Http.sys applicaiton and MSHeap memory manager, the exact setup that I’d recommend in production. The application will be run as a Stand Alone server (not a service, althought the service has a better throughput), but the SA is simpler to test. Delphi 12 compiler was used and IntraWeb 15.5.6.
On the C# side, I’m using the simplest application ASP.NET web forms application that Visual Studio 2022 is able to create using the wizard. It’s basically a 3 forms application. The ASP.NET application is installed in IIS. No other web application is installed in IIS on the same server. Visual Studio 2022 was used to build the application. The application uses .NET Framework version 4.7.2 (supported on Windows Server 2012 and Windows 7 SP1).
The reason I chose .NET Framework 4.7.2 is to be inclusive enough to allow developers using Windows versions prior to Windows 10 to run the test as well.
However, after publishing the first version of this post, I received comments stating “that is a very old framework, you should really test against net 8“. I understand that anything beyond three or four years in Microsoft time is considered quite old. Perhaps that’s why Microsoft historically tends to drop support for their own software so prematurely…
Anyway, I bit the bullet and re-run the tests, now including a new application built with .NET Core 7.0 (no, I’m not using 8.0 because it is not even installed on my dev machine and I don’t wanted to install it right now). The client machine where JMeter runs is not the same that was used during the first version of this article (the second test uses a faster machine), that’s why the numbers are a little higher than before. The network interface is probably faster too. All tests for all applications were executed again, exactly as the first time.
All 3 applications are build in Release configuration, targeting x64 only. ASP.NET files are deployed from withing Visual Studio IDE, usign the built-in publishing tool.
All the source code and the JMeter test plan can be downloaded here.
Both tests are pretty simple:
- JMeter is used to run the test plan, with 100 threads (simulating 100 users)
- Each thread starts up the user session loading the home page (or the main form) and all files that are required (i.e. external JavaScript and CSS dependencies). These files are requested only once per session/thread, exactly like a browser would do (the browser would use the local cache in the following requests).
- Each thread continuously switches from page A to page B and back (or form, in the Delphi case), completing 500 cycles each. The test runs to completion and no errors should occur. In total, approximately 100,000 requests are processed by each application.
- Client, executing JMeter test plan, and server, executing the ASP.NET or IntraWeb applications, are separate machines in a local network.
- Each test plan is executed 5 times. I decided to eliminate the worse result from the set, for each application, and then I take the average of 4 runs.
- Each application is restared before each test plan execution. In case of the ASP.NET application, the web site is restarted instead.
- Throughput is obtained from JMeter itself, and memory consumption was obtained from task manager. We are aware that Task Manager may not always provide real-time and accurate information about memory usage, but it is good enough for the purpose of this test.
Here are the results:
Some other notes about the test:
- For the ASP.NET application, restarting the Web site, the application pool or the WWW service does not change the throughput results significantly.
- Server machine spec: Intel core i7-11700, 8 cores/16 threads, 4.9 GHz max frequency, 32 Gb RAM
- The server machine runs Windows 11 Professional. Client machine runs Windows 10 Pro.
- Although the .NET application contains 3 pages, only 2 pages are used, just like the IntraWeb application. The number of “unused” pages, of course doesn’t make any difference in the .NET application performance.
- Also, the pages are different and the content is different, however there is very little to none impact on the asp.net application performance.
From the results we can see that:
- IntraWeb throughput is almost 5 times higher than ASP.NET Framework 4.7.2, for a similar application
- IntraWeb throughput is almost 2.5 times higher than ASP.NET Core 7.0, for a similar application
- IntraWeb memory consumption is 22 times lower ASP.NET Framework 4.7.2
- IntraWeb memory consumption is 12 times lower ASP.NET Core 7.0
- ASP.NET Core 7.0 indeed performs much better than .NET Framework 4.7.2 in a server environment
We can also conclude that:
- Using the same resources (Processor power/available memory), an IntraWeb application is able to support 4 to 6 times the number of simultaneous users when compared to a similar ASP.NET application (depending on the .NET flavor being used), using a small fraction (5 to 10%) of the memory
- For the same number of users, the IntraWeb application will consume much less resources from the same server machine.