A few years ago I recall scouring MSDN looking for a Model View Controller (MVC) framework and found an odd assortment of patterns including one article that was attempting to posit ASP.NET itself as a sort of MVC pattern. At the time my need was great and I decided to implement the “Front Controller”  with its use of the HTTP Handler.   Interestingly, I used the pattern in spite of the verdict that the author of the article thought that the pattern was “cruel and unusual punishment.”  Actually, with a few modifications of my own, I found this pattern to be closer to Struts than anything else I had seen and I modified it slightly so that my command objects operated against a concrete model passed within the ASP.NET context or session. At run-time, the command objects called lower level business or service tiers and at the end of processing determined the next view or redirection URL—a loose MVC if you will.

As the application grew and the command objects increased, I found that there were a lot of rules in both my MVC tier and at the business/service tier.

While much of the time I have been arguing the benefits of a multi-channel approach to SOA and sharing rule execution, I have been ignoring a discussion about all of the integration points that might be at play in a single application. AJAX might even bring rules into play directly from the client—making it possible that just about any part of your n-tier architecture may require access to rules at some point. This brings me to the value of encapsulation as a driving principal with rule management—even for a single application.

Where are your rules? I am interested in your stories of where they are and what finally was your tipping point in using a BRMS? Please post your comments below.

CCB

Laurent Grateau brought this tutorial to my attention.  Its a handy reference for implementing SSL with your WCF services.  This can be used for enforcing encryption with the RES.NET management and persistence services (click here for the tutorial).

CCB

I have spent an inordinate amount of time lately talking about the benefits of language—something you might appreciate since your are currently reading this blog AND understanding it. Language is a sort of human-level contract we have with each other and by using it we can find our way to understandings—flawed as that may be at times. In fact, some people have developed a discipline of double checking their language and stop to ask questions—feedback to make sure the intended receiver understood the implications of the message (and no, I am not going to talk about TCP/IP).

The requirements process depends upon language to discover critical business features and those that use it will eventually attempt to describe what they learned to developers. However, the understandings (requirements) themselves undergo a transformative process, probably not unlike a Kantian meat grinder in the mind of the business analyst—inputs transforming into logical outputs on the other side. However, even such a rigorous empiricist like Kant expected that new knowledge had to under go the test of common sense. This provided the critical feedback loop to make sure that a proposition was worth its value–so much for a priori knowledge. At this point, one has to ponder the following question: “If the empiricists cannot establish a rational basis for a solid requirement that can stand the test of time then who can?” This is the equivalent of asserting that at any give moment one may be 50% certain about the requirements but if any assumptions change all bets are off—and this includes any assumptions not imagined at the time the requirements were written.

All of this boils down to uncertainty that cannot be eluded in the requirements process because it is conducted by humans and assumptions change. Managing change and embracing it is fundamental to human experience and needs tooling.

Change and ambiguity are closely related.  While working in the financial services space I discovered that for a bank, there is no single balance to an account. The concept of “Account Balance” is understood at all levels of the organization, but this concept often translated into 30 or more concrete variants. All of these had to be understood to support the requirements process. In talking about the fields, the developers did not always share nomenclature. They also used odd-prefixes and suffixes in class names common to their language and they were prone to using synonyms introducing confusion along the way. To avoid this, the business analysts kept a glossary of terms to track the differences and help remove the ambiguities.

Using the vocabulary of a BRMS binds the language of business to the concrete and executable model of the code. This solves several problems:

1. Removes ambiguity by mapping terms to concrete equivalents.
2. Offers levels of validation that non-executable requirements cannot—completeness, regression, versioning, difference and many more.
3. Developers enjoy receiving something that they can use—saving time without waiting to till GUI is available for testing.
4. Concepts or content that don’t belong in a rule can remain affixed to the rule in the form of a description—why is this rule important to the business and what might happen if it did not exist.
5. Requirements expressed as rules can be kept current and eliminate the tendency for them to become stale.
6. Changes can be deployed without shutting down production systems.

It’s true that you can’t express everything as a rule—let’s not fall into myopia. Nor will a BRMS solve all requirements pitfalls; but it is tooling that helps organizations scale by helping us use our language better. In the end, it’s all about words and what they can do…not software.

CCB

In the post (I Want To Believe Redux) my goal was to describe the fictitious story about a bright architect that matured a in-house rules technology. Along the way, the architect experienced pressure points and addressed the requirements in motion. At the start, the architect might have reached different conclusions before proceeding with the effort—for we all lack a critical view of something on the horizon from time to time. Below are some of the key pressure points I have experienced while walking in the “wilderness”:

Many rules, many places
Rules are in too many places—across multiple deployed applications (BPM, vertical solutions, and channels), teams, servers and data centers. IT managers and architects eventually find themselves in a position where they are responsible for changes across these boundaries and are now thinking strategically about how to limit the impact of change across these deployed systems.

Rules are constrained by release cycles
When business level needs are translated into software projects, the changes that are required are constrained by the existing software development life cycle (SDLC) of each group or team. Moreover, many changes may have nothing to do with presentation, but simply involve behavior of a production system. Strategically, the IT manager or architect would like the change to be affective in-between formal release cycles.

The collaboration problem
IT and Technical team members need to collaborate with non-IT staff and business stakeholders. Requirements documents are not always up-to-date and the master of the policy must constantly be translated into a business language for audits, simple reviews and other requirements that require visibility. In fact, the process of translating business requirements into executable rule policies is very expensive because of the amount of human interactivity involved. Strategically, organizations should look for technologies that streamline human interactions to avoid waste while at the same time keeping the goal in view. This is the whole point around declarative approaches to software.

Inconsistent terminology
Architects and Developers do not share the same language with Analysts. Analysts often create and maintain glossaries to map technical terms to business terms. They need a way to manage this at the same time they solve their rule visibility problems. Consistent vocabulary reduces (and in cases removes) ambiguity and allows organizations to scale. This is a more abstracted version of a “contract first” approach to design but at the level of language and human behavior.

Shared execution (SOA)
Once a decision is created, architects would like to have the same feature across all of their deployed applications. They don’t want to touch other deployed servers that may be spread across data centers and geography. In turn, they would like to take an SOA approach to decision deployment so that all channels/platforms can reuse a policy.

Code has become brittle
Business logic has reached a tipping point in terms of algorithmic complexity. This complexity is making the rules opaque to the business analysts and complicates communication. It also is increasingly difficult to debug and test. Specific problems are in the area of re-cursive patterns, complex data structures and the amount of data that must be tested at various stages of a decision. The developer needs tooling to simplify rule expression and model how rules are used. Simplifying logic and the expression of rules is critical for adoption by business analysts and other non-technical consumers of a decision.

Tools are too technical
Business analysts need a tool that fits them: Non technical staff cannot use the same tools as developers and need to manage rule artifacts independently of the SDLC used by the IT groups.

Administrators need tools too
Usually administrators do not have tooling to manage how services are created and managed. They need tools that leverage their experiences and practices in the data center. Living with a solution and keeping deployment costs low is as important as up-front costs; however, these long-term costs are more more difficult to address because they require a broader view of an organization.

In short, the pressure points are real and ever present. I recall my first dive into test first programming and unit testing. In the end, I found I was still doing what came naturally, but the tooling added consistency, repeatability, and EVIDENCE that the code worked. As a byproduct of the effort, I discovered many other benefits that I could not have foreseen without using the methodology and tools—like knowing more quickly how a contract or object signature was going to stand the test of time—or make refactoring a lot easier—just watch all the tests break. The same is true for rules. It would be nice if humans could understand the world a priori; but we don’t. Fortunately there are some walking around with a flashlight.

CCB

There is the world as we see it and the world that could be. Imagining how is critical. If you can’t agree on a premise, then it’s hard to move forward in any direction. One is stuck in the question “why are we doing this?” Such has been the case in a blog I have been following entitled “I Want To Believe”. In addition, at TechEd this year I met several industrious architects that implemented their own rules “engines” on the .NET platform. This Blog and these engineers I spoke with seemed to be grappling with similar questions but from slightly different angles. In general, the question is the following: “How does one identify the tipping point for justifying a BRMS?” The answer is not elusive, however it may not be the same for everyone as you would expect. Consider the following evolution of a fictitious solution.

Simple Encapsulation
Most architects can imagine the need to refactor their code and encapsulate their logic into a single location. Imagine that our fictitious architect George has completed a code review of a new but growing code base and has noticed that there were far too many “If/Then” blocks in the presentation tier. So he gathered the troops and offered that these blocks of code should be wrapped by a single library and enforced with a contract using an Interface. This pattern, along with an existing investment in a model, solved the problem of centralized management from a code perspective. The “If/Then” blocks could not be fully removed from the presentation tier, but they did not bloat as the number of rules grew.

Hot Deployment
Now that the rules were in one spot, George realized that the code in the new library was changing faster than he expected and the BAs on the team wanted to see immediate changes. This issue alone was forcing many interim builds. His solution was to place a wrapper around the library and dynamically load it at run-time—problem solved. Now a new library could be deployed at run time by doing a simple file copy.

Services
Success breeds more success. Another project George was active on needed the same solution and rule content as well. He pulled out the latest SOA framework from his toolbox, refactored the existing code and was now able to share the logic. With the service he added a logger and implemented unit testing for decision regressions. It worked as expected and performed really well.

Many Decisions Many Masters
Over time, several more applications were deployed all re-using the decisions from the SOA solution he had implemented. However, now there were 70 different end points, at least 15 different contracts, and several business analysts and developers requiring changes on a standard schedule. Of the 70 end points, many were duplicates. They had to support versioned services for various applications in production and pilot new ones.

A small team had to be spun-up to mange the “Rule Server”, manage regression and work with system administrators to deploy the supporting libraries. Business analysts were supporting the effort by providing requirements documents. Most of the support time was being taken by the requirements process and not deployment.

The Technical Business Analyst
To lighten the load of the developers, George was able to recruit a full-time analyst to take on the task of working with other business analysts. He also invested in some templates for the requirements process that he knew he could later convert to code and compile later. What he really wanted to do was provide a lighter-weight macro language that the BA could use for expressing rules, but in the end had to teach the BA basic 4GL logic and flow constructs along with object references—something he could do once but probably not replicate since its nearly impossible to find a BA with skills that reach this level of technical acumen or tolerance.

Perhaps you can imagine now many other pressure points exist such as the number of rules under management, the algorithmic complexity of the rules themselves, versioning, a simple rule expression that a BA can understand along with an environment for them, tooling for system administrators, monitoring and the like. In fact it’s the pressure points around collaboration and scaling up the dimensions of a solution that often force the tipping point. And frankly, these are not even the business arguments—these are stated from the perspective of the architect. Some wait until the pain is there while others plan ahead for success.

In short, the need for BRMS is real and present—one simply needs to see the horizon.

Here is the URL to the original “I Want To Believe” blog post.
CCB

Steven Núñez has written a review of the latest release of ILOG Rules for .NET. Download a 6-month free trial of Rules for .NET here. You can watch a recorded demo here.

Congratulations to all the Rules for .NET team for another outstanding release!

Microsoft has made some WCF performance data available on MSDN for a while now. The data shows the performance advantages of WCF over ASMX and other SOA frameworks on the platform. In general, WCF is the stronger performer–good news to all the folks who are currently implementing ILOG Rule Execution Server for .NET.

If you are deploying RES.NET using IIS 7.0 with Windows Server 2008, then you are no longer limited to the bassicHttpBinding. Moreover, the new architecture allows one to trim down the modules used by IIS–allowing for minimal features for your deployment. You can read about the new architecture here. If you want to read about the top 10 reasons why you should like IIS 7.0, then here is a link to a blog entry from Microsoft.com operations, “The Tasty Morsels Found in Dogfood…”

CCB

Since the release of ILOG Rules for .NET 3.0 in February of this year I have seen a good number of questions now regarding best practices with RES.NET integration and deployment. In many cases, the question is around what provider to use–Local or Remote. Given the WCF underpinnings, is there anything one should think about that would affect performance? The short answer is yes. Below are some general facts and guidelines that should be understood and followed:

1. The RES.NET execution API wraps WCF by dynamically generating the proxy client (ChannelFactory) and exposes a limited set of options that we must do in our client code. This pattern is used regardless of how you are invoking the API–local or remote.

2. In the case of the local invocation, the execution API spawns a ServiceHost object programmatically in the background. This is a lightweight service running within managed code and shares the same process that launched it. When this is done, the ServiceHost spawns its own thread pool using the .NET library and is built into WCF.

3. If your project is a thick client (.NET managed code) you should cache the session object for reuse—make it a high-level member of the EXE. All threads in the application will in turn point to the ServiceHost that was created.

4. For a managed application or fat client, the most efficient configuration is the local provider. Do not attempt to add your own thread pool or create multiple instances of the session you created on the first invocation of the RES.NET client.

5. If you have an ASP.NET application, you should always deploy RES.NET to IIS and ASP.NET. This is because WCF will share the thread pool with IIS and WCF will reuse the thread for processing the WCF request. This is only the case where both the application and RES.NET share an IIS instance on the same machine. If they are on separate machines, again, you should use the ASP.NET deployment option for RES.NET (this requires the use of the remote provider when using the execution API).

6. Do not use the local provider for ASP.NET applications. This will spawn the ServiceHost and will create an additional thread pool.

7. If you have a class library that is uses RES.NET but need to run in both a fat client and ASP.NET, then you should write a little factory and load the correct provider for the deployment model.

If you would like to start a thread on this topic, post your question or comment to the RES.NET forum (click here for the forum) .

Hope this helps many of you and have fun.

CCB

Two books I have read recently made a surprising connection. The first was the 1972 science fiction novel entitled “Hellstrom’s Hive” by Frank Herbert—the author of the well know Dune series. The premise of the book was that human communities, though extreme in the story, may act collectively in ways that one might call intelligent.

Almost a year later, I found myself reading a book by Steven Johnson titled “Emergence: The connected lives of ants, brains, cities, and software.” I find these two books interesting because they consider human behavior on a macro scale and they address the key point that the collective has its own behavior, tastes, and values.

With the introduction of media, starting with the first news papers, the notion of a feedback loop was introduced to human experience. This established a sort of dialectic between a community as it is and what it might become. Feedback loops also exist in with software. The requirements process needs the feedback loop to ensure features match the market its intended for. Software that fails to have the feedback loop tends to have a short life.

From a Web 2.0 perspective, feedback loops can be used as a tool to solve a larger set of problems that scale beyond the capacity of traditional human organization. Johnson gives several great examples of well-known web sites using feedback to manage everything from fraud to the promotion of solid content. However, all of these examples from the Internet have something in common—they most certainly are using some form of rules technology behind the scenes. Imagine the decentralized behavior of millions of web users on a single site providing, in aggregate, perspective on content, tastes, and enforcement of terms of service. Rules technology provides the right kind of tooling that correlate user behavioral data with policies. In aggregate, it takes the work of the collective and directs it toward something that’s manageable.

There are still companies today that must be organized by traditional means; however, even they can take advantage of emergent behavior. Random remains the most efficient way to load an airplane and so long as the data represents human activity (a loan, a purchase, a claim) rules technology can scale enforcement of policy beyond the performance of most teams. There is still room to mature how best to achieve 100% straight-through-processing in many industries but harnessing the power of the collective remains the only way to solve some business problems at Internet scale.

 (Click here for a link to Johnson’s book)

Recently I have seen some project requirements where customers have requested the use of working memory with ILOG Rule Execution Server for .NET (RES.NET). Even though RES.NET requires the use of parameters for passing in the XOM (see my white paper on parameters vs working memory in the X-Ray Series) you may still want to use Rete and working memory in a stateless manner.

This can be accomplished by creating a virtual method for each object you intend to assert. In the sample below, I have used the business method wizard on a customer object. You can start the wizard from the BOM view of an object and right click on it:

I created a method named “Assert” that took no parameters and used the rule context. You need to check “Use rule context” if you want the wizard to pass in the engine reference to the method:

Next, I added code to the method that calls the assert method on the engine instance:

namespace BusinessObjectModel {
    [ILOG.Rules.BusinessObjectModel.ExtendType(typeof(BusinessObjectModel.Customer))]
    public class CustomerExtender {
        [ILOG.Rules.BusinessObjectModel.Method(UseRuleContext = true)]
        public static void Assert(ILOG.Rules.RuleEngine engine, ILOG.Rules.RuleInstance instance, BusinessObjectModel.Customer customer)
        {
            // TODO: Add Assert implementation
            engine.Assert(customer);
        }
    }
}

The final step requires a new rule that does the assertion:

if
'param customer instance' is not null
then
Assert 'param customer instance'  ;

When using RES.NET, all working memory object instances will be internal to the engine; however, you can still pass out data by assigning instances to your out parameter list. Here is an example:

if
     the age of  the working memory customer instance is more than 21
then
     add the status key "APPROVED" to 'param customer instance' ;
else
     add the status key "DECLINED" to 'param customer instance' ;

Have fun.

CCB