Seeing is Believing - Business Process Visualization

Time to introduce a new TLA (Three Letter Acronym) to this blog: BPV - Business Process Visualization.

I often sum up basics of BPM (Business Process Management) with the following pitch:

• Model your Process 
  
• Manage your Process 
  
• Monitor your Process 
  
• iMprove your Process
  

For me, the enabling factor for BPM was the Executable Process Diagram.  Use tools to graphically depict the Activities and Flows of a Process, and then auto-generate the software necessary to manage the transitions between those Activities.  

With any good BPM Suite I can rest assured that the Right People will Execute the Right Activities at the Right Time to keep my Processes Flowing Smoothly.

The BPM Industry has more than met the PFS (Processes Flow Smoothly) challenge - The decision for a company to adopt BPM technology and methodology is pretty much a no-brainer (Which suite to adopt may require some thought, but whether or not to adopt isn't even an issue any more).

So what's next?  

In a recent post I attempted to describe a BPMS in terms that might have been familiar to any 19th Century businessman .  One of the comments I received really peaked my interest:

Roeland Loggen said...
Maybe your story reflects one of the key issues with current BPM thinking: if the software provides automated concepts of a year 1890 shopfloor, then not much innovation has been created. Since 1890 many many new ideas have been created on people working in processes. The mechanistic view of typical BPM software has not clue of these more modern concepts. 
The 20th century knowledge worker's reality and productivity is muh different from Taylor based process thinking... not based on orchestration, central coordination, workflow items, inbox/task screen.... but adhoc, collaboration, changing processes on the fly based on new insights and events....

For the sake of argument: If every 1890 shop floor had an exceptional Process Manager on hand, productivity would have soared... but Roland's right: We can't be satisfied to stop with systems that automate 19th century businesses.  It's the 21st Century, and we can do better than that.

The key for doing better is Business Process Visualization (BPV).

In my 19th Century BPMS I discussed someone that I dubbed the "Process Visualizer".  This is that person who took the raw metrics that the Process Manager had gathered and turned them into reports, charts and graphs that the Business Owner could use to understand the past and to plan for the future.  These reports, charts and graphs are an integral part of BPV, but there's a whole lot more to the full picture.

BPV begins with the Process Diagram.  The Process Diagram distills the essence of the problem into a form that Business and IT can share.  It's a crucial part of validating that the process is correct and for communicating the requirements from the process owner to programmers who will implement the process application.  In most of today's BPM Systems the life cycle of the Process Diagram often goes something like this...

The Business decides to improve their operations, so they hire an Analyst to come in and help them.  

The Analyst talks to a lot of people and produces an "As Is" Business Process Diagram.  This diagram is often used by the Analyst to reassure the Business that  the Analyst really understands what's going on.

Now that the Analyst really understands what's going on, a new "To Be" Business Process Diagram is created.  This diagram is used by the Business to capture exactly how they want the new process to operate.

At this point simulations can be run to predict how the new process will behave relative to the old.  Simulation is always tricky, but if the assumptions are accurate (hopefully based on historical data), then it's sometimes possible to tune the "To Be" definition to increase the likelihood of success.

Once the Business and the Analyst are happy with the "To Be" diagram, it can be passed on to the Implementers (and it now becomes known as the "As Planned" diagram).  The Implementers now morph the diagram into an executable diagram, usually adding way more details than the Business wishes to see.

Slight digression:  There's a lot of controversy swirling around BPMN 2.0 over the issue of adding execution semantics to process diagrams.  Implementers (in general) want truly executable diagrams but Business folks (in general) don't want to clutter the diagram with non-Business related details.  I think there's a simple solution... which I will get to later.

The Implementers work with the Analyst and Business until all are happy and the process application is deployed into production... at this point the Implementer's diagram becomes known as the "As Built" diagram.

Note to all: The "As Built" diagram is a clear differentiation from all pre-BPM Systems.  In a BPMS, the "As Built" really is the process that is running in production.  Before BPMS the diagram may have been accurate once, but you never really had confidence that the diagram ever matched reality.

In some BPM Systems, the "As Built" diagram is often the end of the line.  Once the process application is deployed they don't think about it much until the next time a process change is contemplated... However, in many BPM Systems the "As Built" morphs into a living and breathing "As Running" diagram.

Live and historical data breathes life into the "As Running" diagram.  My old friend the Process Visualizer takes the raw data from the Process Manager and animates the "As Running" diagram to give the Business a clear view of what's happening where (and what happened when).  The "As Running" diagram gives you your first glimpse of what BPV can achieve.

Thus far the Process Diagram has be limited to a pretty small audience - those involved in designing the process and those involved in implementing the process.  BPV should expand the Process Diagrams visibility to a much wider audience.

Go into just about any office and ask a typical worker the following question:

"What Processes do you work on?"

Most likely, the response will be a blank stare.

Now ask the worker:

"What Activities do you perform?"

The blank stare will vanish and you'll get a very accurate litany of tasks.

BPV needs to give each worker the answer to that first question.  It's not enough to know what Activities you need to work on - To do your best you really need to know how those Activities fit into a bigger picture.  Process knowledge is a bigger picture, and it goes a long way to helping the workers understand why they are doing what they are doing.

A task that seem stupid in isolation may make sense when seen in a process context - or it may really be stupid.  In the former case the worker may do a better job, and in the latter case BPV may give the worker the context needed to communicate their frustration.

When folks such as Lombardi's Phil Gilbert talk about BPM on every desk in an enterprise, this is what they are talking about.  Process Visualization helps everyone put things in context, and that helps everyone do a better job.

Now for a bit of Devil's advocate:  Most workers couldn't make heads or tails out of a BMPN diagram, and if you add all the execution semantics proposed for BPMN 2.0 that percentage is going to drop even further.

Granted, but there's a solution: Different views for different folks.

Returning to my programming roots... I am a huge fan of Aspect Oriented Programming.

Aspect Oriented Programming evolved from a simple fact: Most worthwhile endeavors can be broken down into distinct concerns, and often you can meet the goals sooner by dealing with these concerns (aspects) in isolation.  Obviously there are limits to this when concerns overlap, but it's a good place to start.

With Aspects as my inspiration it's easy to see where Process Diagrams need to go: Aspect Oriented Diagrams.  If we can identify the aspects of a diagram that the Business Owner is interested in, and those that the Operations Folks are interested in, and those that each Worker is interested in, then we can generate the Views that mean the most to the intended audience. It's all just a matter of Perspective, and BPV enables many, many perspectives.  Some views may look like process diagrams, others may look quite different - doing BPV right will employ a lot of artistic scientists.

Here are just a few process views that come to mind:

• Process Owner's "40,000 foot" view
• Activity Owner's "Escalation logic" view
• Participant's "Where does this fit?" view
• Operation's "What's happening now?" view
• Operation's "What happened?" view
• IT's "Capacity planning" view

 As Roland said, the future is about "ad hoc, collaboration, changing processes on the fly based on new insights and events"...

BPV's going to help us find those insights.

Grilling Business

Mike Hugos offers some interesting thoughts in his article: How To Truly Partner With Business.

Mike points out that the traditional requirements gathering process is:

"about as friendly as being grilled by a police detective"

Not exactly the relationship we've all dreamed of, is it?

== != =

This week I visited a client in the Detroit area, and on the whiteboard in their meeting room I saw the following:

== != =

For those of you who aren't programmers, this is a slam at perhaps one of the worst programming language syntax decisions that has ever been made.  You can read the above expression as "Equal equal is not equal to equals".   Still lost?  

In the C programming language (and derivatives such as C++, Java, JavaScript and C#) the way that you test to see if two variables are equivalent to each other is to use the "==" operator.  For example:

if ( a == b ) { doSomething(); }

If you want to assign a value to a a variable you use the assignment operator "=".  For example:

a = b;

Believe it or not, even folks who have been programming for years occasionally screw up and use the assignment operator "=" when they meant to use the test for equivalence operator ==.   Here's an all too common example:

if ( a = b ) { doSomething;}

The line above sets the value of variable "a" to the value of variable "b".  If "b" is not zero, then the function "doSomething()" will be invoked.  Perhaps that's the behavior that the programmer desired, but I really doubt it.

According to the wikipedia article comparing the Pascal programming language to C: 

"It is common for programmers new to C to accidentally put assignment expressions in conditional statements such as if (a = 10) { ... }, which will always execute because the assignment expression a = 10 has the value 10 ≠ 0 and is therefore considered "true" in C. Also note that a has now got the value 10, which may affect the following code. This kind of mistake cannot happen in Pascal, as assignments are not expressions; using the wrong operator will cause a compile error (and it's not likely that anyone would mistake the := symbol for equality test)." 

I have to take issue with this statement... It's not just programmers who are new to C.  I made this mistake last week :-)

This mistake is so common that some programming editors will actually flag statements like this to warn the programmer.  I've no idea how much time has been lost on this little syntactical monkey wrench, but I think it's probably quite a bit.

Long ago before I learned C I used Pascal.  In Pascal the "=" operator is used to test for equality.  If you want to set a value you used ":=".  When reading this symbol out loud I'd always say "set equal to".  For some reason, I don't ever remember confusing the two or knowing anyone else who ever confused the two.

The BASIC programming language is even simpler... the interpreter uses the context of where "=" appears in an expression to determine what it means.  IF $A = 0 THEN is interpreted as a comparison.  $A = 3 is interpreted as an assignment.

I don't know why the authors of C chose to use "==" and "=" in this way - but unfortunately that decision was inherited by most of the programming languages that are widely used today.  Every  programmer who includes some JavaScript on a web page might screw up and make this mistake... so just remember:

== != =

19th Century BPMS

I sometime find it useful to describe a BPMS (Business Process Management System) in terms of things and people that you probably would have found in an office or factory in the 1890s:

At the center of the BPM system is a Process Manager who makes sure that each Process runs smoothly.  The Process Manager uses Process Definitions to know what all the steps of each process are, the proper sequence in which each step should be performed and which people should perform each step.

The Process Manager hands out assignments to each worker - updating each worker's Task List to insure that each worker knows what to do and when to do it.  The Process Manager also passes along any information from previous Tasks that the workers doing the remaining Tasks may need.

If a worker doesn't finish a Task on schedule, the Process Manager may escalate the Task. Escalation could take many forms - the Task might be reassigned or perhaps the worker's supervisor might be informed - It's all laid out in the Process Definition.

A great Process Manager can manage many instances of many different types of processes simultaneously (that was probably rare in the 19th century).

The workers do the work (just like they did in the 1890s) performing whatever activities are assigned to them by the Process Manager.  When a worker finishes a Task the Process Manager must be informed.

Workers in the 1890s were given detailed instructions that told them exactly what to do to accomplish their current Task along with Forms to record any information that the Process Manager or other workers needed.  In today's world the workers rely on software to guide them through each task (often in the form wizards).  Most BPMS systems run these wizards using Task Engines that can execute any Task Definition, but often custom applications are developed or adapted to perform specific Tasks.

The Process Managers primary job is to keep every process running... But that's not enough.... Are the processes efficient at accomplishing their goals?

• Workers want to know how they are doing relative to their peers.
• Bosses want to know how well their workers are doing and whether or not the company's goals and obligations are being met.
• Owners want to know how today's performance compares with that of last year... and how next year's perfomance might be better.

Everyone needs to be able to Visualize their processes in meaningful ways that can help answer their questions.

The Process Manager records all of the raw data necessary to track each process instance, but often relies on someone else to crunch the numbers and produce the meaningful reports, charts and graphs that help make sense of the processes.  I call this someone else the Process Visualizer for lack of a better title.  I'm sure there were people in 19th century offices who performed a similar function.

Ultimately it's the Process Visualizer that sets a BPM Sytem apart from a Workflow System. Without visibility into a process you can only guess on what changes might improve it. BPM is about enabling Continuous Process Improvement - so that Visualizer is key.

Their are many vendors of BPM Systems in the world, and many might take issue with some aspects of my analogies - but I think that most will agree with my broad characterizations.  

BPM Systems are made up of relatively mundane parts that would sound familiar to most 19th century Businessmen - the concepts are old - it's our technology (and people who know how to use the technology) that is making these BPM concepts transformational. 

Just imagine what could have happened in the 1890s if there had been a great Process Manager and Process Visualizer in every office and factory.

Process Rules - Why BPMN isn't enough

A human-centric process "works" when the right people (aka Participants) perform the right tasks (aka Activities) at the right time and in the right sequence.

From the above statement you can pretty much derive the types of Rules that govern a Process:

• Routing Rules - determine who (which Participant) should perform an Activity
• Flow Rules - determine which Activities should be performed and in what sequence
• Escalation Rules - determine what to do if an Activity is not completed at the right time

On a BPMN diagram the Routing Rules are somewhat indicated by Swim Lanes - Flow Rules are somewhat indicated by Gateways - Escalation Rules are somewhat indicated by attaching Timers (a form of Intermediate Event) to Activities. The challenge for those of us who implement Process Applications is to fill the gaps between the BPMN diagram that somewhat represents these Rules and the nuances of these Rules.

Using BPMN to model a Process is a fairly "Business Person Friendly" method for capturing and validating requirements. With a bit of training most Business People can "read" BPMN diagrams to validate that the diagram is an accurate representation of the Process they want implemented. Many BPM Systems translate BPMN diagrams into executable code - so what the Business Person "sees" is what the Process "does".

Unfortunately there's not a similarly wide-spread "Business Person Friendly" method for capturing and validating Routing, Flow, and Escalation Rules.

I'm intentionally being imprecise in how I use the term "Rules" - I don't want to get bogged down in a discussion about Rules Engines - I'd rather focus on the way most Business People express Rules.

• When a Business Person tells you "Have Frank do this task if the loan amount is over $500K" that's a Routing Rule.
  
• When a Business Person tells you "Do a Credit Check if the Applicant has no Collateral" that's a Flow Rule.
• When a Business Person tells you "If the Credit Check isn't completed in an hour notify Frank's boss" that's an Escalation Rule.

Often you'll get ambiguous or conflicting Rules from Business. It happens all the time. The Process can't (really) be implemented until the ambiguous is made clear and the conflicts are resolved - When you encounter these situations (and you will encounter these situations) you'll immediately understand why there's a lot more to implementing a Managed Business Process than BPMN.