When business process, semantics and search collide

As more and more server and desktop applications migrate to the cloud, there is great opportunity to take old, worn metaphors and functionality and re-invent them in lighter, faster, more usable ways.  A few search vendors are already preparing for this evolution by offering architectures and tool sets to build these applications -- Exalead offers a smart take on what they call search-based applications, or SBAs, and Google just announced a business version of their App Engine. And I am sure that others will soon follow.

But something important is missing:

A business process is usually baked, or hard coded, into the application that supports it (with a slight amount of configuration tolerated), so the challenge has always been how to tie disparate applications together into a unique, business-required flow. For example, a process may flow from the SFA app to a pricing spreadsheet to a CRM app, but there is no common connector other than the process or expertise.

Sure, BPM and workflow software can sometimes do this, but it is expensive, complex, time consuming to implement and ultimately only works for processes and flows that are precise and reoccurring. They do little to support "knowledge" or ad hoc processes. And certainly traditional BPM/workflow systems do not offer the kind of flexibility and usability that can fully leverage the benefits of the cloud.

So who can effectively bring a business process sensibility to the cloud and search-based apps? It's a good question, as process folks are too rigid to accept the ad hoc nature of search, and search folks will not like the structure and rules that process requires to be successful. But whoever figures it out will have a killer platform.

The key to this convergence, by the way, will be semantic technology. Think of it this way --  business process, at its essence, is simply a set of rules around how around how words are displayed and connected. The purpose, of course, is so that individuals within a group can come to agreement as to what it is they do, should do or want to do, and agreement to the methods they will use.

I'd like to walk through a example that may provide some insight on how process could be effectively incorporated into search applications.

Let's begin with a simple, traditional process model. What this says, basically, is since a birthday is occurring (the input), there is a need to bake a cake (the activity).  A baker (or actor) will follow a recipe (a guideline or rule), using available ingredients (material) and an oven (a tool or system) to produce a cake (output). This process could be decomposed into its parts, or sub-activities, such as "Select Recipe," "Purchase Needed Ingredients," "Make Batter," and so on.

Traditionally, this representation would be used for reference (here's how we prefer to do it), or it could be automated into a workflow (you cannot bake a cake unless you have a birthday occurring within the next two days).  But try looking at it like this:

Each component of the process is really just a part of a larger entity or repository of common types, that in the case of baking a cake, happen to fit together like this. Certainly each component can play many different roles at different times. This notion of a distributed process environment is exactly what will work in the cloud. Process definitions become metadata and fuel for rich semantic queries that can tap multiple repositories when needed, retrieving and assembling components on demand into an interface that is meaningful to a user at a precise point in time.

What's cool about this is a business process can be enabled from many places -- from a document, from an email, from a user profile on a social network.  Since any component of any process is aware of its role across many processes, users are able to engage in typical ad hoc behavior but will always be a click away from greater process rigor should they want it or the business require it.

This also means that the ability to "model" or define a process needs to fast, easy and human, something most BPM and workflow tools most definitely are not.

Mediocrity of the crowds

Everyone may have a fundamental right to an opinion, but that doesn't mean they know what the heck they are talking about. And put enough of these people together, and well, welcome to the internet, 2010. Loud, anonymous and opinionated with little substance. Credibility and expertise be damned.

While the touted benefits of the hive mind help rationalize this embracing of mediocrity, it in fact makes it even harder to discover information of substance and value as the outliers, both superior and inferior, get clumped together as they are not of the "norm."

Practically speaking, this makes it harder to know what new books to read, what new music may be of interest, or if a movie or play is worth seeing. Not to mention the utter lack of meaningful serendipity. It is not for lack of opinions, but for lack of quality, substantive opinions. While the internet has made it exceptionally easy to solicit and contribute feedback or post one's grandest or most heart-felt insights to a blog (ouch), there is absolutely no guarantee the thoughts shared are of any value whatsoever. And even if there are postings of substance you may simply never find them, having to first wade thru dozens or hundreds of rants and ramblings.

A real problem with this clumping together of outliers is that it also makes it much more challenging to discover things that may be loosely but not directly of interest. Sure Amazon and Google can tell you what has occurred in the past (people who liked this bought that...), but they are lousy at predicting the future because it's, well, unpredictable.

The answer? Well, for as long as they are around, the best resources are still the traditional media, where people are directly compensated to articulate critical thoughts. Skilled writers and reviewers with years of expertise in specific domains. While one may not always agree with their opinions, there is at least respect for the wisdom applied. And by being trusted sources, they enable readers to make discoveries and connections that software simply cannot know to make.