Event Processing and Process Management

I am of the opinion that we in the software industry as obsessed, yet fickle, about acronyms... We insist on coining acronyms, yet we tire of them easily and introduce new acronyms that (almost) mean what the old ones did.  Must be something in our genetic makeup...

If you are acronym obsessed - today's topic relates to CEP and BPM.  Both relate to Business. The former is all about "Event Processing" and the latter is all about "Process Management".  They are very closely related.

Event Processing is conceptually really simple:  When X happens, do Y.
Process Management is also conceptually really simple: Perform these Activities in this order.

In reality, Process Management is Event Processing:

1. When the Process Starts, do Activity One
2. When Activity One completes, do Activity Two
3. etc. etc. etc.
   

The primary difference between a CEP system and a BPM system lies in defining relationships between events.  In a BPMN diagram we define a flow of events - an order in which we expect those events to occur.  For example, Activity One always comes first in a process instance... It's completion triggers the start of Activity Two.  We're diagramming the expected sequence of events.

Truth be told, all real BPM solutions have to include a bit of CEP because real business processes must always take into account "out of band" events.  For example, an "Order" can be cancelled at any step of the "Order Fulfillment Process". When an Order is canceled, we need to perform some cleanup steps.  Based on where we were in the process, we may need to perform different cleanup steps.

We certainly would not want to "mess up" a nice clean BPMN diagram by adding a "Cancel Order" handler to each Activity, but we do have to deal with this eventuality or our solution isn't worth spit.  In general we will model an "Order Cancelled" sub-process - which is kicked off by a "Cancel Order" event.  Sounds kind of like "Complex Event Processing", doesn't it?

Acronyms and cache phrases are great... but they are highly overrated.  When you are building a Process solution for a Business, you're going to touch on CEP, BAM, Business Rules, Business Performance Monitoring, etc. etc. etc.  These are all aspects of what your Business needs to succeed, and you are going to have to deal with all of these aspects.

The challenge is to integrate all of these concepts in a manner that is comprehensible and maintainable by the Business... Your BPM solution must work (of course), but it can't be a magic trick and it can't be a black box or you haven't really met your goals. 

When it's time to modify the Process (notice that I said "when" and not "if") there will probably be a whole new crop of acronyms to confuse things - Don't let your solution get caught in that trap.

Bonitasoft - BPM Game Changer?

Bonitasoft rolled out their open BPM open solution 5.0 this morning... declaring it as a "game changer" for BPM.

I like Bonitasoft's approach: Keep it simple. Make the common tasks easy and the complex tasks possible.  Don't distract the process developers with any unnecessary "under the hood" details.

They've done a great job building an excellent suite that really feels like seamless solution... and I'm sure that many folks are going to love it.

That said, there's nothing "game changing" in any of Bonitasoft's features.  "Nothing new here" in terms of being able to build anything with Bonitasoft that you couldn't already build with a number of BPM suites, both proprietary and open source.

The "game changing" aspect of Bonitasoft isn't features - it's marketing.

Bonitasoft builds on the rich legacy of the Object Web open source projects -many of which have been in commercial use for years.  The huge difference between Bonitasoft and it's ancestors is packaging... both the packaging of the software components themselves and the packaging of the BPM message.

With most open source projects, if you want to use them you have to engage very skilled developers to install and configure the various packages that you want to use.  You have to spend a lot of time just getting the development infrastructure set up.  You have to spend a lot of money before you can even begin to work on your actual problem.

Contrast that to Bonitasoft where everything that you need is in a nice little bundle - ready to go in a few minutes.  That's a huge win on the Technical front.

A win on the Technical front is great - but that's not where the battle is.  The battle is on the Business front.

Despite what most IT folks would like to believe, it's really the Business that picks the solution in the BPM space.  Business comes first in BPM, so you have to convince Business first.

This is where Bonitasoft's marketing comes in - Their marketing team is transforming a Techno-Geek centric open source tool into a Business-Centric save-your-company tool.  That's what changes the game (if they pull it off).

I work for a company with a great proprietary BPM suite... and our suite is getting better all the time... but I really wish Bonitasoft success.  By lowering the barriers to entry in the BPM space they are helping to spread the message and educate the masses.  They are helping to raise the BPM tide, and a rising tide raises all boats (both large and small).

Good job Bonitasoft, and best wishes.

Why Go? Google's very own Programming Language

In the last few months Google's been releasing information about a new programming language called Go.

I've read a few articles and glanced at a few tutorials, and my impression is that it's a pretty good attempt to get back to basics... it reminds me very much of the RISC attempt to simplify computer CPUs back in the 1980s.

Simple is good... at least in theory.  It seems to me that we always learn a lot when we do a good Spring Cleaning to repair what we need and throw out what we don't - but it also seems to me that we inevitably end up with as much junk as we ever had a few years down the road... Simple has a habit of getting complex very quickly.

I am sure the researchers who are working on Go are sincere, but I can't help thinking that Go might be just another reminder to others in the Software Industry - in this case Oracle - that Google won't be painted into a corner.

Oracle just completed its acquisition of Sun, which means that Oracle just completed its acquisition of Java.  It's quite possible that Go is sending the same message to Oracle about Java as the message that Android is sending to Apple and that Chrome is sending to Microsoft and the other browser manufacturers...

"If you try to lock us out, we'll just build our own... and our's will quite likely be better than yours"

Perhaps I'm wrong... but perhaps I'm not.

Incremental product releases - Mozilla is finally getting it right

 In Randall Kennedy's posting "Why Firefox Will Flame Out", he penned the following:

"Now we hear that Mozilla is abandoning its traditional major release cycle model in favor of smaller, incremental changes that it will slipstream through security patches and other maintenance updates. Basically, Mozilla's developers are admitting that they can no longer deliver a fully baked and tested Firefox release in a timely fashion. So they're switching to an incremental model where they can deliver progress in more manageable chunks, thus bypassing the lengthy external beta/feedback process altogether."

Randall goes on to say:

"releasing lots of small changes without broadly testing their effects on the underlying platform is often a recipe for disaster"

This is where I have to jump in and disagree...

Releasing lots of small changes to the underlying platform can be a disaster because everything that is built on the platform can be effected, but incremental releases are absolutely your best strategy for ending up with a product that does what it needs to do.

Start with just enough to be useful and let people use it.  Get their feedback then add a bit more useful stuff and let people use it. Get their feedback then add a bit more useful stuff.  Continue until "done".

Waiting until everything is perfect is a recipe for never releasing anything... because perfect is a moving target.  You'll find that stuff that you thought was useful when you started wasn't useful after all, and visa-versa.

Personally, I think Firefox has a much better chance of survival with this strategy... Sorry Randall.

Multi-Manager Workflow Solutions

If you've worked with Filenet, Alfresco, or any number of Content Management suites, then you've most likely encountered their embedded workflow capabilities. It's very common to enforce some sort of process around adding, changing, and deleting content from your corporate system of record - so embedding workflow capabilities in these products just plain makes sense.

If you've worked on a BPM project that involves a wider scope than your corporate management system, then you've probably had to answer the dreaded question:

Which product should I use to implement this process?

My first encounter with this conundrum was with an Identity Management product rather than with a Content Management product, but the factors to consider were pretty much the same.

The process that we needed to implement involved employee credentials that were managed by Sun's Identity Management product. The Identity Management product's embedded workflow features weren't enough to handle the whole process,  so we were left with the option of implementing everything in Lombardi's Teamworks or of implementing the "master process" in Teamworks and key sub-processes in the Identity Manager.

Our original approach was to go with a pure Teamworks solution to keep all the process logic in a single system, but during implementation we changed our minds.  Using Identity Manager's embedded features for a few key sub-processes shortened our development cycle, so we went with it.

In an ideal world - at least from my perspective - Process (workflow) Managers would be decoupled from specialized products.  The specialized products would all expose Services (both Human-Powered and Automated), and the external Process Manager would choreograph and orchestrate those Services.

Unfortunately, my perspective is far from ideal if you are the vendor of a specialized product.  Leaving the process and workflow elements of your solutions to an external product just doesn't work.  Your product needs to provide as near an "Out Of The Box" solution as is possible, or people won't buy it.  You could (of course) partner with a Process Manager vendor, but that requires inter-vendor collaboration... and the end result probably won't be as tailored to your specific domain as you'd like.

Consequently, product vendors will embed workflow capabilities into their products, and we'll be left to answer the dreaded question:

Which product should I use to implement this process?

Clearly, it's not unreasonable to assume that you may have to deal with multiple "Process Managers" to implement a single process...  Teamworks may handle most of the process, Sun Identity Manager may handle "identity" aspects, and Filenet may handle "Document" aspects.  What's needed is a standard API for each Process Manager to use when interacting with another Process Manager...  Here's a partial list:

• Start a Process Instance
• Signal that a Process Instance has completed

I could easily come up with many more APIs... but these two are the biggies.  If I can start a process, and get notified when a process completes, then I can incorporate any process that's managed by and external Process Manager into a process that I am managing myself.

So far, so good - but I'm not quite done yet with my wish list...  I'm fine with Process Manager "A" handling the overall process, and with Process Manager "B" handling some key sub-processes... but only if Process Manager "B" keeps Process Manager "A" updated on what's going on.

For example - Let's assume that "A" kicked off a sub-process with a dozen activities that's handled by "B", and for some reason it's taking more time to complete than expected.  What's going on?  Which of "B's" tasks have been completed?  Which of B's tasks are left to complete?  Enquiring minds want to know :-)
 
If users have to switch between process managers to get a clear picture of "who is doing what" (or to claim tasks) then they're not going to be happy.  Multiply process managers are fine "under the hood" - but keep the hood closed and let the users concentrate on driving.

Missing pieces

I came across an article by David Rotman that describes W. Brian Arthur's efforts to understand why it takes so long to commercialize new technologies... and I was drawn to the following statement of why "lab on a chip" technologies weren't commercialized back in the 90s:

"The problem, as Arthur might put it, was that the toolbox was missing key pieces."

It occurs to me that this is the reason the status quo in software development remains the status quo... New and better approaches are all around us, but the incomplete nature of a new approach leads to its rejection.

Case in point, a few years ago I penned my frustration that Java developers resisted the adoption of BPM technologies on my java.net blog.  I got a huge number of mostly angry comments... but they all really boiled down to missing features in the BPM tools.  The BPM tools were more primitive than the "state of the art" development tools, and those missing features invalidated the advantages of BPM.

The BPM toolboxes were missing key features.

A lot has happened in the ensuing years, and the BPM toolboxes are much, much better... but I still wonder if there aren't just a few key features that we're missing that would really make the "Business drives IT" promise of BPM a reality.  It's great that "real" programmers are now comfortable with BPM suites, but I maintain my allegiance to making a BPM suite as natural for a Business person to use as a spreadsheet.

What key pieces are missing?  What are the few nifty tools that will enable those "Power Business User" to keep control of their BPM projects?

I've got my own thoughts on the subject... but I'd love to know what others think.