Albin sent me some more benchmark results; an update to the results I published a couple of weeks ago. The initial results were intriguing because of the low heap usage on IBM z/OS, while we only ran tests with the Rule Execution Server using the J2SE and POJO rule sessions. For the second benchmark run Albin therfore tested WebSphere Application Server running on Windows, to provide a point of comparison with the same application server running on z/OS. He also expanded the benchmark to test ruleset execution using IlrContext, providing a measure of the memory overhead of the Rule Execution Server.

JVM Heap Size by Number of Rules (Windows)

The chart below shows the memory requirements for ruleset execution based on 4 scenarios:

1. Command line application using IlrContext rule engine API
2. Command line application using Rule Execution Server with J2SE rule session provider
3. Servlet deployed to IBM WebSphere Application Server using RES with J2SE rule session
4. Servlet deployed to IBM WebSphere Application Server using RES with POJO rule session

JVM Heap Size by Number of Rules (z/OS)

The four scenarios described above were also run on z/OS.

Comparing Windows vs z/OS Heap Usage

Conclusions

1. JVM heap usage reported on z/OS is consistently less than on Windows. For example, using a Servlet deployed to WAS with Rule Execution Server with POJO provider, the memory usage was 17%-21% less than the same configuration running on Windows.
2. The Rule Execution Server has a heap memory overhead compared to IlrContext of between 3% and 15% depending on configuration and the size of the ruleset.
3. There is very little difference in heap usage between the Rule Execution Server with J2SE provider and with POJO rule session provider on either Windows or z/OS.

The product overall showed predictable memory usage patterns on both Windows and z/OS with modest heap memory requirements. The highest heap memory reported was 32 MB for a Servlet deployed to WebSphere Application Server using the Rule Execution Server J2SE provider, executing a ruleset containing 528 rules.

This week Albin Carpentier and I helped to run JRules memory consumption benchmarks on IBM zSeries. Within R&D we have 24×7 remote access to a machine we rent from IBM in Dallas, Texas. A large bank in Europe is evaluating deploying JRules on the mainframe and Albin was able to quickly deploy the Rule Execution Server to WebSphere and DB2 running on z/OS so we could gather memory consumption statistics to help inform their decision.

The configuration tested was: zOS 1.9, DB2 8.1, WebSphere Application Server 6.1.0.16 and IBM JDK 5.

We used a proof of concept ruleset containing representative rules that were cloned to create rulesets with 33, 66, 132, 264 and 528 rules. Memory consumption was measured using the JDK 5 MemoryMXBean and MemoryPoolMXBean. We ran three scenarios:

1. Invoking the engine within a JVM (no WebSphere) using IlrContext,
2. Rule Execution Server on WAS using J2SE provider,
3. Rule Execution Server on WAS using POJO provider.

The product performed very well and we saw linear scalability in all three scenarios. The highest heap memory usage we saw was a very modest 35 MB on WebSphere with the Rule Execution Server.