• if you don’t know where you go, you don’t know when you get to your destination, and
• if you don’t track your progress, you don’t know which way you are going.

Early ocean goers had no accurate way of locating their position or destination, or even of tracking their progress. This is quite similar to many current software project managers. Yet these ocean voyages had a much higher rate of success (i.e. reaching their destination with most of their goods sellable and most of the crew alive) than current software projects, where it is said that in fewer than half the cases their results are actually used.

Software Project Managers (PMs) can find some very relevant lessons in the experience of early ocean seafarers. I will develop two what I see as the four most important ones in the next paragraphs:

1. Know where you want to go.
2. Track your progress, estimate where you are.
3. Track your speed, estimate where you are going.
4. Choose the best route.

I’ll skip for now points 1 and 4, leaving them for a later discussion and focus here on the tracking part of a software project.

Track your progress, estimate your position.

Both the time it took you to reach a certain point, and the location of this point count. Project tracking is naturally about time and expenditures, and number of remaining bugs, but also about delivered results.  

As shown in the above screen shot, ISIS encourages the “earned value” tracking method [1] that gives good location estimates by focusing on what is actually delivered to the customer (the green curve above, top graph) , while the ISIS-required properly filled time-sheets tell the project manager where the effort was spent (the red curve above, top graph) .

Tracking of efforts and actual deliveries give Project Managers an approximate snapshot of the project’s position. Yet PMs typically have a difficult time estimating what work is required to finish. They need to get help from the developers who should report the efforts spent, and what they see as the remaining effort, whether it was planned initially or not (“remains to be done”).

Track your speed, estimate where you’re going.

Project Managers can fairly easily tell their approximate position (based on what deliveries have been accepted by the customer on one hand and the time-sheets on the other); but they usually don’t take trends into account. Back to the initial analogy, being next to a shoal and drifting towards it is very different from being next to a shoal and drifting away from it!

ISIS recommends tracking position over time, e.g. through the integration of successive snapshots. This helps getting an accurate estimate of the delivery speed, thus a good estimate of how long the remaining work in the project will take. This is a much better estimate than using the current position snapshot and betting the house that the rest of the project will go according to some initial plan.

Similarly ISIS recommends tracking the “speed” rather than snapshots of bugs and issues, as this helps predict when a test phase finishes.

The following graph (lifted from the ISIS documentation) gives a hint of how such a dynamic tracking can work:

Using the above graph, with snapshots are taken at each vertical line, a project manager could have a good idea of the overall project effort and duration by the 3^rd snapshot, predicting an end of project around the 12^th time interval, rather than either the 5^th as was the initial prediction, or the 6^th as the 3^rd snapshot would have.

A future blog will develop the “Track your speed” aspects, and some of their unexpected consequences on effective mechanics of project management. Please send us your comments and suggestions!

A concept is often better explained by its opposites, or defined by negation: what it is not about. This is a way to put boundaries around an idea and understand it better. Reading Martin’s Fowler’s story about the origins of the agile movement, we see that the term was selected to highlight adaptability and response to change. And what is something neither adaptable nor responding to change? Something rigid, isn’t it? Something lacking flexibility, lacking the ability for all the stakeholders to frequently renegotiate the scope to take into account new elements. Hence the title of this post: agility versus rigidity.

Let’s explore agile software development concept and its relationship to ILOG methodology, ISIS, further.

As the original source of the agile approach, the Manifesto for Agile Software Development establishes the following principles for better software development:

• Value individuals and interactions over processes and tools
• Value working software over comprehensive documentation
• Value customer collaboration over contract negotiation
• Value responding to change over following a plan

Despite their clarity and simplicity, these criteria are quite generic, but they give us some good clues to evaluate if a process is agile or not. If you look into the manifesto pages, you’ll see the principles behind the agile manifesto, which will help you in this understanding. If you keep surfing the Internet, searching for instance for agile software development in Google, you’ll get 1,340,000 results (quite an outstanding score actually projecting a false idea of the real and limited adoption of the agile approach to date).

Fortunately, you do not need to read all these pages to understand agility! Coming back to the source and definition by negation again, the article what’s agile, what’s not by Alistair Cockburn, one of the signatories of the manifesto, is a good reference to delimit agile software development.

Developing decision-support systems is an agile task by nature because humans outsmart machines in terms of ability to deal with change, the ability to come up with new decisions, which is a pure demonstration of agility. In ISIS we promote and support the ideas presented in Cockburn’s article, making our methodology agile in many ways:

• It is iterative. It recommends short iterations (4 to 8 weeks). The outcome of an iteration is an “iteration release”, which is a stable, integrated and tested subset of the complete system, a working piece of software.
• It promotes communication and collaboration. From application assessment to deployment, communication tools such as brainstorming, workshops, facilitation sessions, progress meetings, etc, involving the customer from the very beginning, are used.
• Testing is fundamental, ISIS recommends Test-Driven Development (TDD) and continuous testing from unit to acceptance.
• Continuous integration is accomplished through adequate and recommended tools to streamline and automate the process.
• ISIS provides a set of light, agile templates to produce an effective documentation.

Through these various readings it appears more clearly that, addressing the second question in this post, “not agile” software development is neither simply awkward, nor clumsy, nor stiff software development, it is simply too rigid. Although we understand the value of agility, we also understand that it is not always the best approach, either because of our client’s own methodology requirements or because the special characteristics of the project (see picture bellow, criteria taken from Boehm and Turner’s book Balancing Agility and Discipline: A Guide for the Perplexed).

In practice, while on consulting engagements, we often have to align ourselves on the customer’s existing methodology or process which may not be agile yet. We then tailor our approach to leverage its founding principles and existing artifacts and promote agility where ever we can, to decrease the rigidity of existing processes and replace it with flexibility to increase business adoption of IT solutions.

In software design, a design pattern is a proven and reusable solution to a commonly occurring problem in a determined context.

The idea behind design patterns comes from Christopher Alexander’s book, A Pattern Language (1977), which talks about architectural patterns in buildings and towns. Ten years later, Kent Beck and Ward Cunningham started to apply the patterns to software development. It became popular when the Gang of Four’s book Design Patterns: Elements of Reusable Object-Oriented Software was published in 1994.

Wouldn’t it be great if we were able to apply the appropriate patterns in sequence to obtain the complete design for a software system? Well, we are not there yet… (Are we? See how JUnit was designed in JUnit A Cook’s Tour). Design patterns provide quality, elegance, flexibility and reuse to the systems that use them, but they are intended for small scale, partial solutions, ignoring the big picture. Other concerns such as system complexity, maintainability and performance, architectural issues, have to be taken into account (for this we have architectural patterns, but this could be another blog post). Also, there is no well-defined, systematic approach to apply design patterns, and the possible problem domains in software engineering are so wide that it is difficult to say that we’ve already found all the necessary design patterns to solve them.

Despite of all these limitations, design patterns have evident advantages and it is surprising to see people heavily involved in software development not having heard of them. Currently, design patterns can be found anywhere, even for specialized areas such as Java EE, with its recommended core patterns. Interestingly enough, these patterns, along with those from the Gang of Four, are an important part of the Sun certificaction for enterprise architects, SCEA, which clearly shows their relevance.

Design patterns are perfectly aligned with ISIS principle of “not reinventing the wheel”. They are recommended to ILOG consultants and the methodology contains guidance on them.

The following picture shows the “ubiquitous” singleton pattern, such as it is presented in the ISIS guide for leveraging design patterns.

More generally, there is no good or bad design patterns, they are all useful to some contexts. They are not all relevant to every decision-support system, but most of them are, especially when you consider the other layers and services surrounding a decisioning application. Overall they represent a key concept to leverage during design phases and also to keep in mind to identify new ones which can be reused from one project to another.

What about you? What do you think of design patterns? Do you normally use design patterns in your projects? Have you come with new ones?

Discovering and analyzing your business policy change templates is one of the several key aspects documented in the ISIS methodology, and that must be addressed as part of defining your Rule Governance processes.

The reason why a BRMS component is brought to a company’s IT application mix is because it facilitates the implementation of decision services, but also, and more importantly, because it helps manage their rapid evolution. It is thus surprising that the task of preparing the system and its supporting organization for business policy change management does not always get the attention it deserves.

Note that we are not talking here about preparing for change at the level of the business rule elements, which are the atomic building blocks of a policy. These are usually (or should be) well covered by generic rule governance processes for authoring, validation and deployment. We are talking about change at a macro-level, where it is expressed in terms of raw business policy statements instead of individual business rules. Think change to a paragraph of the underwriting policy used as the reference document by the loan underwriters, for example.

During the application analysis steps, the main questions to the business stakeholders revolve around capturing as accurately as possible the definition of the business policies that will be implemented by the system. However, when analyzed from a snapshot, the policy can appear monolithic or prompt a decomposition along some logical or system-related concepts which are not adapted to the future need to accommodate business changes.

Therefore, another critical task that must be addressed early on in the analysis process is focused on producing an inventory of the probable ways in which the policies may or will change, and with which frequency.

This requires the policy managers to reflect on their experience and come up with as many concrete examples of discrete policy changes that have they have been witnessing regularly in the past. These examples can then be arranged in a taxonomy of business policy change templates.

The most frequent changes are usually the most simple and precisely described ones. The more complex and less frequent ones will often contain some unknowns. For example, the possible templates for a loan pricing application may be:

• Changing the base rate values (may occur weekly).
• Changing the add-on, minimum rate, or fee values (may occur monthly).
• Creating or retiring “specials” for selected regions or channels (may occur quarterly).
• Creating or retiring the pricing structure for a new product (may occur once or twice a year).

Of course, not all changes are predictable. Important and unpredictable ones come, for example, from external regulatory rules or from newly devised company strategies. But for the more banal and recurring ones that can be identified and dissected in advance, the benefits are multiple.

Some of the artifacts that can be prepared for a given business policy change template are:

• A template for change submission, which precisely describes the change and the different parameters involved.
• A process map to implement the change, detailing which rule should be updated, created, or deleted, in which package, whether an update to the rule flow is warranted, etc…, and which specific resource is needed.
• An accurate time and effort estimate to implement this change, from authoring to deployment. The estimate is developed and agreed upon by both IT and business stakeholders, and helps in setting the release schedules and expectations.
• A set of rule templates to facilitate the implementation and reduce the risk of introducing defects.
• A precise test plan and set of test cases. And since the scope of the change is well defined, it is easier to apply techniques such as Delta Testing, which help get extensive test coverage and minimize the updates needed on test cases.

For traditional software, requirement tracking is an important pre-requisite to smooth system maintenance. It allows to analyze the impact of a requirement change to the underlying implementation, and plan the change accordingly.

Business rules management systems take maintenance to the next level and make it a standard activity. It should thus be expected that the efforts on requirement tracking are pushed accordingly to collect and analyze the patterns in requirement change.

To know more about Rule Governance, join me in a 1-hour webinar on May 28th (click here to register). Like our mission states, we are here to help your business handling change and complexity, and rule governance is a key element to make sure you can attain such a goal!

Informs Practice is one of two annual conferences of Informs, the Institute for Operations Research and the Management Sciences. A group extremely vested in building strategic decisioning systems with thousands of practioners around the world, definitely worth spending a couple of days with. What impressed me the most is the ability of this community to address business needs with smart, pragmatic, mathematical and computer-based solutions. A great example of the benefits of Business Analysis and a great lesson or inspiration for IT as a whole.

My goal was to leverage the experience we gained building both optimization and rule-based systems to generalize some fundamental best practices applicable to the implementation of decisioning systems in general, what ever technology is involved. When you look at such systems, here are some common characteristics which come to mind: 

Yes, some decision processes are non deterministic and a decision made through different paths. In other situations, some users change their mind in the way they make decisions and want a computer to assist them in the process. That makes decisioning systems hard to test. In some situations, decisions are made on intangible elements, or intuition, a challenge for our usual way to model things and implement heuristics. Or the human brain is just too powerful and outperform the computer (e.g. visual pattern recognition, optimized way to allocate resources developed over years of experience). Some decisions are not black and white, some are based on fuzzy information, some others are really complex either in terms of number of variables involved or the type of search algorithm. Some users or policies require the decisions to be explained or traced. Some rules are unwritten. And to complicate things even further, decisioning systems are usually involved in processes which are subject to frequent change, for instance a change of policies (internal or external), strategy, organization or business model.

Such qualifiers clearly would not apply to a standard accounting or record-management application such as a CRM system, or an embedded real-time program such as in an ATM or your car ABS.

Although technology can help addressing many of these above challenges, and object-oriented programming in particular, it is not the only solution. Project Management has also to adapt and leverage new paradigms to make sure these projects can cope with so many potential hurdles.

Here are our top-5 picks in ISIS which we will develop further in subsequent posts:

1. Iterative development. Ultimately, you want to put a working system in the hands of your end-users every 4 to 6 weeks. So they can provide early feedback to avoid late discoveries of misunderstood needs or help uncovering mismatch with changing requirements. This is the key concept of the Unified Process, as well as the Agile Manifesto. An approach which is unfortunately not natural in the IT industry yet, but clearly a plus to handle the challenges identified in the above picture.
2. Focus on Time To Profit. It is key to define iterations which each bring some value to the business to anchor the project on the right track, get acceptance and traction from the business and solidify the business case of the solution under development. Business-driven iterations as opposed to purely technical and/or contractual releases or builds like we too often see in classical project life cycle such as the waterfall approach.
3. Intelligent tracking. Most of the IT projects get in trouble because of non realistic tracking. Weeks after weeks, you hear the usual “I’m 80% done” and the percentage keeps the same… Even more so with decisioning systems, estimates need to get revised to account for changes and implementation challenges. Not only you need realistic “pictures” (a project status at a given time), but you need to juxtapose these pictures to make a “movie” and highlight trends which will help you resetting expectations or reevaluating the scope and objectives. In ISIS we have added charts to our weekly status report to explicit potential drifting in workload estimate or track the quality of a deliverable through the evolution of issues found, analyzed and fixed.
4. Project governance. Governance has definitely got a lot of spotlight since the Enron debacle, and a lot of negative connotations for many people since then. However, increased scrutiny surely helped businesses avoid costly mistakes. There are all sorts of governance (e.g. corporate governance, IT governance, SOA governance, rule governance); the project governance should be seen as a positive oversight, an assistance to the project manager to maximize the chance of success. In particular, a steering committee can help the project manager negotiate changes with the business or suggest mitigation actions to address implementation-related issues. ISIS provides checklists and guidelines to make such a governance process work for all the stakeholders.
5. Risk management. We sometimes hear “if we only knew what we know” and this is indeed the title of a great book on knowledge management. Unfortunately, we actually know much more than we want to admit when we start an implementation. For instance, in ISIS, we have a tool to evaluate 193 pre-identified risks and 48 classical project management mistakes. Needless to say, with such a long list, the saying should become “if we had known what we knew then…” Definitely worth spending time in a fair, transparent and upfront risk analysis.

These are only 5 of many foundations of ISIS but key ones to ensure success. As we have seen over the past 20 years and learnt from hundred of projects. They do require quite some discipline though, but the rewards of getting a decisioning system used for many years make them worth the effort!

It was designed 5 years ago and used in ILOG Professional Services worldwide on dozens of projects. Leveraging ABRD mitigates the risk associated with new business rules initiatives by providing a well documented and structured approach for developing rule-based applications. ABRD allows organizations to avoid using ad-hoc processes or having to expend significant time and effort creating their own best practices.

ABRD is an Eclipse plug-in the same way ISIS is. In fact the following diagram outlines the relationships between ISIS, ABRD, and OpenUp.

OpenUp represents the base, the foundation, defining the core principals of software application development such as: collaborate to align interests and share understanding, balance competing priorities to maximize stakeholder value, focus on the architecture early to minimize risks and organize development, evolve to continuously obtain feedback and improve. ABRD is called abrd_openup plugin and extends OpenUP. ISIS has a common plugin to include any elements which are reusable for BRMS, Optimization, Financial Industry or SCM projects. Each plugin in ISIS provides container for knowledge management for specifics industry or practices.

I encourage you to book your agenda May 7^th, this Wednesday, for a web seminar and register at https://iloginc.webex.com/iloginc/onstage/g.php?t=a&d=828310442&SourceId=00005 and go to www.agileitarchitecture.com to have more information on ABRD, BPM, CEP and other agile technologies.

You can also get more information about ABRD by visiting the following web sites:

• ILOG’s BRMS Resource Center
• ABRD’s deployed instance
• my blog on Agile IT Architectures

Let us know how ABRD works for you!

PS: here is a link to the recorded webinar.

What is this blog about?

Since ILOG’s inception in 1987, ILOG consultants have been involved in thousands of projects, helping our customers making the best use of ILOG technology. All these projects share a key characteristic: they are all about decisioning or decision-support. Leveraging computers and smart software to help you, our customers, make better decisions, faster, while managing change and complexity. Not an easy task but we are proud of the unmatched success of our customers in this area, and have learnt a lot in the process, over the past 20 years. We have actually assembled more than 5,000 pages in our PS (Professional Services) methodology and you, ILOG customers, partners, prospects, have asked that we share some of it with you. So, here we are, opening Pandora’s box.

What is ISIS?

ISIS stands for ILOG Solution Implementation Standard. It is a repository of documents gathering and organizing best practices used by our consultants to deliver the best quality in all their engagements based on ILOG technology. It is our Consulting methodology, a process tailored to the implementation of ILOG-based decisioning systems. It is not the dead Egyptian goddess anymore but a very living organism, evolving on a daily basis. It spans from software engineering and program management to product-specific tips and vertical/domain knowledge. It reflects the broad range of decisioning systems that can be built with ILOG products. Technically, it is an extension of the Open Unified Process (OpenUP), but we will come back on this in future posts.

The open-source and Web 2.0 waves

In some way, we follow big trends with this blog, although it is not about source code here but know-how, ideas and concepts. We have actually already made a major step in this area earlier this year with a contribution to the Ecplise Process Framework (EPF) open-source project, in the form of an OpenUP plug-in called Agile Business Rules Development (ABRD). We do hope that the communication will work both ways and you will tell us what you think, share your own experience on a particular topic, what worked or did not work for you and some lessons learnt, useful to anyone implementing decisioning systems in general.

Where can you expect this blog to go?

We plan on navigating across various parts of ISIS and here are some topics which you can expect to be addressed over the coming weeks and months:

• More details on OpenUP and EPF and how we have leveraged them in our implementation process, and our ABRD contribution,
• Benefits of iterative development, the key foundation of ISIS,
• The specific challenges and opportunities in implementing optimization and rule-based decisioning systems,
• Rule governance, a strategic area addressed by BRMS,
• Project governance,
• Improved techniques to track project progress,
• The specific roles involed in the implementation of decisioning systems, on the supplier and customer sides,
• Best practices around testing decisioning systems (e.g. test plan, delta testing),
• Praxeme, an open enterprise modeling method,
• Tips to leverage the BRMS approach and benefits in implementing SOAs,
• Discussion around the right level of documentation to remain agile, how to properly document use cases,
• The benefits of risk analysis and management, and the associated discipline.

There will be more as we comment market news and trends and also as you rise questions and new ideas in your comments. We count on you to keep the communication loop closed and live!

Who are we?

Jean, Pierre, Jerome, Maurice, Julio, we are the team behind ISIS. Leveraging our combined 50 man-years with ILOG implementing decision-support systems based on our four product lines and with the help of 200 consultants on hundreds of engagements, we keep adding new tips and best practices to the ISIS repository, learning from the stimulating and challenging projects our customers come to us with. This is a team blog and you will see contributions from the 5 of us, covering some of the many facets of our methodology.

With this introduction, and in the midst of this Spring of 2008, let the discussion begin and flourish!

The ISIS team.