Archive for the ‘JRules’ Category

Presentation for October Rules Fest

Thursday, September 18th, 2008

orf-logo.png

I am busy working on my presentation for October Rules Fest this week — assembling the slides and double checking details with UBS. The abstract is below and the presentation is shaping up very nicely. Hope to see you there!

Distributed Data Processing with ILOG JRules

Abstract

UBS Bank operates in over 50 countries and employs more than 80,000 people. Learn how UBS uses ILOG JRules and a distributed grid architecture to generate internal and regulatory reports for all the regions around the world off a single dataset. In order to acheive this UBS is required to process 2 billion records every night, with over 30 million records passing through the rule engine. Performance objectives are in place to ensure the bank meets its regulatory and financial reporting requirements before the trading day starts.

A Picture is Worth a Thousand Words

Tuesday, September 16th, 2008

Have you ever wondered what the Internet, WWW or an individual blog looked like? Having young kids and an interest in visualization — particularly 3D — gets you into these things!

Wordle can create nice “word clouds” from an RSS feed. It is also a good example of a Java Applet (that works!). See below for the cloud I just created for this blog.

blog-words.png

I need to write some code to export an RSS feed from Rule Team Server and then business users can take a unique view of the terms in their rulebase. Luckily the JRules Samples Team has provided a great starting point…

JRules in the Real-World

Friday, September 12th, 2008

800px-Motherhood_and_apple_pie.jpg

I just got back from an on-site visit with one of our customers in the USA, a major insurance company. The customer is using JRules 6.6.x for commercial property insurance underwriting. The trip was very interesting as we were able to get hands-on with their large rule project for 3 days — approximately 40,000 rules. We gained some fascinating insights into their development challenges and in exchange we performed an audit to ensure they were getting the most out of JRules. I came back with a couple of bugs to fix and a headful of ideas for enhancements.

One of their major challenges is that they simultaneously work on 3 versions of their platform — a 3 month major release (2.0 say), a 1 month minor release (1.1) and a weekly “patch” release (1.0.5). The patch release is edited in Rule Team Server by business users while the other two releases are edited in Rule Studio by Java developers with JRules training. Due to this rolling release schedule they version the rules in a source code control system (using branches) and have to perform regular merges of the changes coming from RTS into RS.

The RTS repository does not support branches. This was a conscious limitation when we designed RTS as we believed that branching and merging would be problematic for a business user. The development process we therefore recommended is to treat RTS as a satellite user of the source code control system.

  1. For each branch in the source code control system an RTS instance is created, which could be just a table space in a database rather than a new application server install.
  2. The business users collaboratively edit the rules within RTS. When they are ready they request that the changes be moved into the system of record — the source code control system (SCCS).
  3. A RS user updates their workspace from SCCS and then synchronizes with the appropriate RTS instance. The RS user resolves any conflicting changes and commits everything to SCC.
  4. The RS user then uses the SCCS merge tools to merge the contents of the code branch with the next branch. For example versions 1.0.5 might be merged into the branch for versions 1.1.
  5. At a later checkpoint the 1.1 branch will be merged with the branch for version 2.0

As you can see one of the RS users essentially “proxies” the changes coming from RTS into her local workspace.

Due to the textual nature of BAL rules (if-then-else) merging them is relatively straightforward. Decision Tables and Trees are more problematic as they are persisted as complex XML documents. We spent some time discussing the APIs that we provide that might enable a graphical merge capability for Decision Tables.

We also spent quite a bit of time optimizing the RS build time for these large projects. Here is a checklist:

  • Check RS JVM heap size. For large projects allocate as much heap as you can, typically around 1.2 GB on 32-bit Windows.
  • Use the latest patch level of the JVM and JRules. Both are constantly evolving and usually getting faster!
  • Switch off as many semantic checks as you can to save time during the build. The settings can be found on the Build preferences page. The checks are useful of course but some may be time-consuming, particularly static rule analysis.

    rs-build-prefs.png

  • If you are using Decision Tables switch off the semantic checks for each table. For large decision tables this can result in significant time savings during builds. You may want to leave the checks enabled while you edit the table and switch them off just before committing your changes.
  • rs-dt-prefs.png

  • Keep your vocabulary “lean and mean”, as a bloated vocabulary makes the rules slower to parse. Only verbalize BOM elements that you plan on using in your rules.

I hope these tips will help as your projects get larger. Rest assured that we take real-world feedback sessions such as this one very seriously and we are constantly trying to stay one step ahead in the race for better performance.

JRules Certified Developer

Monday, September 8th, 2008

ILOG just announced a certification program for JRules 6. The program is open to all and you can be certified at the bronze, silver or gold level. For the past 6 months or so Jean Pommier’s team has been collecting and classifying tricky JRules 6 questions for the various exams. The exams are administered by Pearson Vue who have over 4,000 testing centers around the world. There is almost certainly one near you.

The program allows JRules consultants, both internal and external, to objectively demonstrate their deep product knowledge making it easier for JRules customers to select the right consultant for a given job.

Gold certification demonstrates an expert level of ILOG JRules product expertise, typically acquired through extensive experience with ILOG JRules on real life projects. This includes mastery of advanced concepts and comprehensive knowledge of the product APIs.

For the months of September and October 2008, the certification will be offered at an introductory 50 percent inaugural discount. Use promotional code IL4034 to receive the discount.

Words, Words, Words

Tuesday, August 19th, 2008

I have spent an inordinate amount of time lately talking about the benefits of language—something you might appreciate since your are currently reading this blog AND understanding it. Language is a sort of human-level contract we have with each other and by using it we can find our way to understandings—flawed as that may be at times. In fact, some people have developed a discipline of double checking their language and stop to ask questions—feedback to make sure the intended receiver understood the implications of the message (and no, I am not going to talk about TCP/IP).

The requirements process depends upon language to discover critical business features and those that use it will eventually attempt to describe what they learned to developers. However, the understandings (requirements) themselves undergo a transformative process, probably not unlike a Kantian meat grinder in the mind of the business analyst—inputs transforming into logical outputs on the other side. However, even such a rigorous empiricist like Kant expected that new knowledge had to under go the test of common sense. This provided the critical feedback loop to make sure that a proposition was worth its value–so much for a priori knowledge. At this point, one has to ponder the following question: “If the empiricists cannot establish a rational basis for a solid requirement that can stand the test of time then who can?” This is the equivalent of asserting that at any give moment one may be 50% certain about the requirements but if any assumptions change all bets are off—and this includes any assumptions not imagined at the time the requirements were written.

All of this boils down to uncertainty that cannot be eluded in the requirements process because it is conducted by humans and assumptions change. Managing change and embracing it is fundamental to human experience and needs tooling.

Change and ambiguity are closely related.  While working in the financial services space I discovered that for a bank, there is no single balance to an account. The concept of “Account Balance” is understood at all levels of the organization, but this concept often translated into 30 or more concrete variants. All of these had to be understood to support the requirements process. In talking about the fields, the developers did not always share nomenclature. They also used odd-prefixes and suffixes in class names common to their language and they were prone to using synonyms introducing confusion along the way. To avoid this, the business analysts kept a glossary of terms to track the differences and help remove the ambiguities.

Using the vocabulary of a BRMS binds the language of business to the concrete and executable model of the code. This solves several problems:

1. Removes ambiguity by mapping terms to concrete equivalents.
2. Offers levels of validation that non-executable requirements cannot—completeness, regression, versioning, difference and many more.
3. Developers enjoy receiving something that they can use—saving time without waiting to till GUI is available for testing.
4. Concepts or content that don’t belong in a rule can remain affixed to the rule in the form of a description—why is this rule important to the business and what might happen if it did not exist.
5. Requirements expressed as rules can be kept current and eliminate the tendency for them to become stale.
6. Changes can be deployed without shutting down production systems.

It’s true that you can’t express everything as a rule—let’s not fall into myopia. Nor will a BRMS solve all requirements pitfalls; but it is tooling that helps organizations scale by helping us use our language better. In the end, it’s all about words and what they can do…not software.

CCB

When Does Complex Event Processing (CEP) Complement a BRMS?

Tuesday, July 22nd, 2008

It’s my pleasure to introduce Pierre-Henri Clouin, an ILOG colleague, for this post on Complex Event Processing (CEP). CEP has emerged as one of the “hot topics” in the rules space over the past 12 months. Pierre-Henri is based in Sunnyvale, California and has spent several months looking at the technical capabilities of the CEP products as well as how they are positioned. His first post provides a nice introduction to the subject matter.

Perso_pix_web_ILOG.JPG

Pierre-Henri Clouin, ILOG

As interest grows in CEP, we have started receiving inquiries about how CEP and BRMS compete with or complement each other. After discussing with customers, prospects, and vendors, and reviewing a wide range of use cases, a few patterns have emerged.

CEP shines when:

  • event data rates are very high, typically in the 100,000s events per second, and with multiple event streams;
  • latency is low, typically in the millisecond range;
  • flat data model, simple data type;
  • a few stable rules/ statements/ queries (a few dozens at most) are deployed for filtering, joins or aggregate computation.

These core capabilities are well documented. For additional details, Mark Tsimelzon’s CEP Complexity Scorecard summarizes them very effectively.

On the other hand, a BRMS addresses three critical needs:

  1. rich rulesets, typically ranging from a few dozens to tens of thousands of business rules;
  2. a complete lifecycle management environment for business rules, empowering technical and business users to author, manage, simulate and retire business rules;
  3. extreme agility with the ability to update business rules in as little as a few minutes.

ILOG BRMS does not compromise on performance either, as have shown benchmarks and actual deployments with demanding customers, such as some of the largest websites, payment networks, underwriters, and telecom operators.

brms+cep.png

The map above sums it up: a CEP engine complements a BRMS for use cases with large data rates, low latency, and rich decision automation and management. The CEP engine pairs down the volume of events and only passes interesting events on to the BRMS to perform a rich decision process. Examples abound, notably in fraud management and national security.

Conversely, CEP overlaps with BRMS at the low end of data rates, latency requirements and rulesets. This is the area where we’ve seen some confusing accounts and claims and where a CEP engine provides limited value on top of a BRMS.

In upcoming posts, we will continue to explore and discuss best practices surrounding BRMS and CEP. We encourage you to reach out to us with related experience and questions.

Business Object Models

Thursday, July 10th, 2008

Understanding the Business Object Model (or BOM as it is affectionately known) is one of the keys to using JRules effectively. In this entry I hope to lay some groundwork for subsequent more in-depth entries on the Vocabulary, Executable Object Models and B2X mapping.

The BOM defines the “entities”, “classes”, “types”, or “concepts” (”things”) that you want to write rules about. It is a powerful and expressive object model in its own right, heavily influenced by the capabilities of the Java object model, but also with features from XML Schema and with annotations to describe domains.

The BOM is serialized into a .bom file using a plain text serialization that looks very similar to Java-style class definitions. For example, if you create a “Hello World” rule project using the Rule Project Wizard and then open the generated BOM in a text editor you will see:

property uuid "_L9byoE4CEd2KE6Zi_qxKvw"
package helloworld;

public class Clock
{
    public static readonly ilog.rules.brl.Time currentTime;
}

The syntax should be pretty familiar to most developers. Note that ilog.rules.brl.Time is another BOM class, defined in the JRules System BOM. The System BOM defines the base classes and primitives that are automatically imported into a user application BOM.

BOM classes are namespaced using package declarations, and aggregated within a BOM Entry. BOM Entries are referenced from a Rule Project’s BOM Path. All the BOM classes from all the BOM Entries on the BOM Path are available to a Rule Project. The analogy with Java classes, packages, JARs and the classpath should be obvious.

Before the BOM can be used to author Business Action Language if-then-else rules or decision tables/trees it must be verbalized. The process of verbalization generates human-readable phrases for the classes, attributes and methods in the BOM. The verbalized BOM is expressed as a Vocabulary which is serialized into a .voc file. For the Hello World project the .voc file looks like this:

# Vocabulary Properties
uuid = _L9akgE4CEd2KE6Zi_qxKvw
helloworld.Clock.currentTime#phrase.navigation = the current time

I.e. the phrase “the current time” has been associated with the static attribute helloworld.Clock.currentTime.

BOM classes create vocabulary Terms while BOM attributes/methods create vocabulary Phrases. The diagram below shows how the text within a Business Rule is first related to the vocabulary, which is mapped onto a BOM.

businessobjectmodel.png

One key point to keep in mind is that the BOM is locale independent — it is an object model created for rule authoring purposes. The vocabulary on the other-hand is locale specific as it contains human-readable text. You can create different vocabularies for the same BOM, allowing you to define how rules should look in English, French or Chinese for example.

A BOM is typically created though a “bottom-up” design process: Java classes or XML Schemas are created by the development team defining the Executable Object Model (XOM). A BOM Entry is then generated from the classes or complex types in the XOM. It is then a simple matter to verbalize the BOM to create a Vocabulary and start writing rules. Because the BOM was generated from a XOM, the rules will be directly executable, using instances of the types defined within the XOM as the source of data for the rule engine.

An alternative use case is to define the BOM using a “top-down” design process - creating the classes within the BOM independent of the XOM. Additional work is then typically required to map the BOM to a XOM for use during execution. This mapping declaration is defined in a B2X file which I will return to in a later post.

The BOM is represented and manipulated using the classes in the ilog.rules.bom package. For example, using the API it is possible to create a BOM from an external data source, such as database tables, UML diagram or even a structured requirements document.

Conclusions

The BOM serves several important purposes:

  • Defines a locale independent object model describing the “things” you want to write rules about.
  • Often the BOM is a simplified and flattened version of the XOM specifically tuned to the requirements of rule authors.
  • Provides the foundation for defining a vocabulary, the locale specific expression of how you want to verbalize the BOM for a single language.
  • Maps onto an Executable Object Model, which will be used during the execution of the rules. As such it serves as a bridge model between the requirements of rule authoring and rule execution.
  • Allows the XOM to evolve independent of the BOM — the Java classes within the XOM may be subject to a traditional development process, with a controlled impact on the business rules.
  • Enables rule authoring and maintenance to be performed in environments without the XOM accessible, such as within Rule Team Server.
  • During rule testing using Rule Scenario Manager, rules are executed directly upon the BOM, and instances of BOM classes can be created from rows in Excel spreadsheets.

This post has only scraped the surface of the capabilities of the BOM, XOM, Vocabulary and B2X. I aim to drill down into some of the details in subsequent posts. Stay tuned!

JRules Rule Execution Server Memory Usage

Tuesday, July 8th, 2008

Albin sent me some more benchmark results; an update to the results I published a couple of weeks ago. The initial results were intriguing because of the low heap usage on IBM z/OS, while we only ran tests with the Rule Execution Server using the J2SE and POJO rule sessions. For the second benchmark run Albin therfore tested WebSphere Application Server running on Windows, to provide a point of comparison with the same application server running on z/OS. He also expanded the benchmark to test ruleset execution using IlrContext, providing a measure of the memory overhead of the Rule Execution Server.

JVM Heap Size by Number of Rules (Windows)

The chart below shows the memory requirements for ruleset execution based on 4 scenarios:

  1. Command line application using IlrContext rule engine API
  2. Command line application using Rule Execution Server with J2SE rule session provider
  3. Servlet deployed to IBM WebSphere Application Server using RES with J2SE rule session
  4. Servlet deployed to IBM WebSphere Application Server using RES with POJO rule session

win-heap.png

JVM Heap Size by Number of Rules (z/OS)

The four scenarios described above were also run on z/OS.

zos-heap.png

Comparing Windows vs z/OS Heap Usage

win-zos-main-heap.png win-zos-servlet-heap.png

Conclusions

  1. JVM heap usage reported on z/OS is consistently less than on Windows. For example, using a Servlet deployed to WAS with Rule Execution Server with POJO provider, the memory usage was 17%-21% less than the same configuration running on Windows.
  2. The Rule Execution Server has a heap memory overhead compared to IlrContext of between 3% and 15% depending on configuration and the size of the ruleset.
  3. There is very little difference in heap usage between the Rule Execution Server with J2SE provider and with POJO rule session provider on either Windows or z/OS.

The product overall showed predictable memory usage patterns on both Windows and z/OS with modest heap memory requirements. The highest heap memory reported was 32 MB for a Servlet deployed to WebSphere Application Server using the Rule Execution Server J2SE provider, executing a ruleset containing 528 rules.

In Sync?

Monday, June 30th, 2008

syncro.png

Our indefatigable documentation team just published a 52 page documentation addendum for JRules 6.x, covering all things related to synchronizing Rule Studio for Java and Rule Team Server.

Table of Contents

syncro-docs.png

We received feedback from a number of customers stating that the existing documentation for synchronization was insufficient or inaccurate so we are very pleased to make good — and with luck improve your efficiency while using the product.

Please don’t hesitate to send us your comments and criticism!

Memory Requirements for JRules on IBM zSeries

Thursday, June 26th, 2008

This week Albin Carpentier and I helped to run JRules memory consumption benchmarks on IBM zSeries. Within R&D we have 24×7 remote access to a machine we rent from IBM in Dallas, Texas. A large bank in Europe is evaluating deploying JRules on the mainframe and Albin was able to quickly deploy the Rule Execution Server to WebSphere and DB2 running on z/OS so we could gather memory consumption statistics to help inform their decision.

The configuration tested was: zOS 1.9, DB2 8.1, WebSphere Application Server 6.1.0.16 and IBM JDK 5.

We used a proof of concept ruleset containing representative rules that were cloned to create rulesets with 33, 66, 132, 264 and 528 rules. Memory consumption was measured using the JDK 5 MemoryMXBean and MemoryPoolMXBean. We ran three scenarios:

  1. Invoking the engine within a JVM (no WebSphere) using IlrContext,
  2. Rule Execution Server on WAS using J2SE provider,
  3. Rule Execution Server on WAS using POJO provider.

The product performed very well and we saw linear scalability in all three scenarios. The highest heap memory usage we saw was a very modest 35 MB on WebSphere with the Rule Execution Server.

res-zseries-memory.png