.NET RESTful Engine Performance and Best Practices

This article outlines some .NET RESTful Engine (v load tests that we ran, as well as some best practices for hosting your own .NET RESTful Engine.

Performance Tests#

For these tests, we used two different "sized" virtual machines on AWS. These are the specifications of the VMs we used:

CPU Credits/ hour1236
Mem (GiB)28
Network PerformanceLow - ModerateModerate

The tests consisted of sending 5 requests per second for 5 minutes (~1500 requests). We measured the time from for first request to the time of the final document becoming ready on the disk. So the time it takes to post and process 1500 requests. Here are the results we found after running the tests.

Performance Test Results#

Run 1 (Document/ second)1.772.96
Run 2 (Document/ second)1.323.80
Run 3 (Document/ second)2.263.12
Average (Document/ second)1.783.293

As you can see, the larger VM was able to process documents almost as twice as fast. This is a result of having double the processing cores in the large VM relative to the small VM. Through our testing we found that memory had little affect on performance, and the number of CPU cores was a lot more impactful.


Please note that the template used for testing was hosted on AWS S3. If you are running a templated saved locally you can expect performance numbers to go up.

Best Practices#

Through our testing we found a couple of "best practices" for hosting the .NET RESTful Engine, especially if you are anticipating a heavy load on your installation:

  1. Processing power matters
    • The number of CPU cores available have a direct impact on the document processing of the RESTful engine. The more CPU cores available, the better the engine will churn through documents.
  2. Long Initialization Time
    • If you realize that the service initialization takes too long and notices that the first requests take longer than the subsequent requests, you may need to make some configurations to your IIS install. Please follow the steps in the following link
  3. Default IIS App Pool Idle Time
    • IIS App Pools have a property called "Idle Time". This property is set to 20 minutes by default. If you are sending huge loads to the RESTful engine in a short period of time, the App Pool will go to sleep in 20 minutes even if the engine is still processing documents in the background. You can "wake" up the engine again by hitting any of its endpoints, and it will pick up where it left off. To avoid this, we recommend changing the app pool Idle Time property . Here are the steps for doing so:
      1. Click on "Application Pools" in the left pane on IIS
      2. Right click on the ".NET v4.5" application pool (your RESTful Engine should be registered under that one)
      3. Click on "Advanced Settings"
      4. In the "Process Model" section you will fine the "Idle Time-out" property, represented in minutes
      5. Change this property to a time period that would work well for you

Please keep in mind that even if the engine is processing documents in the background, the App Pool will still shutdown because IIS describes "A worker process is idle if it is not processing requests and no new requests are received". So if you anticipate a large load please update the property described above.