When business process, semantics and search collide

As more and more server and desktop applications migrate to the cloud, there is great opportunity to take old, worn metaphors and functionality and re-invent them in lighter, faster, more usable ways.  A few search vendors are already preparing for this evolution by offering architectures and tool sets to build these applications -- Exalead offers a smart take on what they call search-based applications, or SBAs, and Google just announced a business version of their App Engine. And I am sure that others will soon follow.

But something important is missing:

A business process is usually baked, or hard coded, into the application that supports it (with a slight amount of configuration tolerated), so the challenge has always been how to tie disparate applications together into a unique, business-required flow. For example, a process may flow from the SFA app to a pricing spreadsheet to a CRM app, but there is no common connector other than the process or expertise.

Sure, BPM and workflow software can sometimes do this, but it is expensive, complex, time consuming to implement and ultimately only works for processes and flows that are precise and reoccurring. They do little to support "knowledge" or ad hoc processes. And certainly traditional BPM/workflow systems do not offer the kind of flexibility and usability that can fully leverage the benefits of the cloud.

So who can effectively bring a business process sensibility to the cloud and search-based apps? It's a good question, as process folks are too rigid to accept the ad hoc nature of search, and search folks will not like the structure and rules that process requires to be successful. But whoever figures it out will have a killer platform.

The key to this convergence, by the way, will be semantic technology. Think of it this way --  business process, at its essence, is simply a set of rules around how around how words are displayed and connected. The purpose, of course, is so that individuals within a group can come to agreement as to what it is they do, should do or want to do, and agreement to the methods they will use.

I'd like to walk through a example that may provide some insight on how process could be effectively incorporated into search applications.

Let's begin with a simple, traditional process model. What this says, basically, is since a birthday is occurring (the input), there is a need to bake a cake (the activity).  A baker (or actor) will follow a recipe (a guideline or rule), using available ingredients (material) and an oven (a tool or system) to produce a cake (output). This process could be decomposed into its parts, or sub-activities, such as "Select Recipe," "Purchase Needed Ingredients," "Make Batter," and so on.

Traditionally, this representation would be used for reference (here's how we prefer to do it), or it could be automated into a workflow (you cannot bake a cake unless you have a birthday occurring within the next two days).  But try looking at it like this:

Each component of the process is really just a part of a larger entity or repository of common types, that in the case of baking a cake, happen to fit together like this. Certainly each component can play many different roles at different times. This notion of a distributed process environment is exactly what will work in the cloud. Process definitions become metadata and fuel for rich semantic queries that can tap multiple repositories when needed, retrieving and assembling components on demand into an interface that is meaningful to a user at a precise point in time.

What's cool about this is a business process can be enabled from many places -- from a document, from an email, from a user profile on a social network.  Since any component of any process is aware of its role across many processes, users are able to engage in typical ad hoc behavior but will always be a click away from greater process rigor should they want it or the business require it.

This also means that the ability to "model" or define a process needs to fast, easy and human, something most BPM and workflow tools most definitely are not.

The big lie in search

The biggest lie in search is also its most touted benefit -- that it is intuitive and easy to use.

Perhaps, as much as a blank piece of paper is intuitive, so too can be a single, empty text box partnered with a single submit button.

But the search results page? Easy and intuitive, it never has been, and it is getting worse and worse as search companies keep adding more options, suggestions, clusters, facets and widgets to try and help you make sense of the endless pages of results. Even the Google with its once pristine layout recently felt compelled to add more clutter to the interface, an explicit acknowledgement that the results they serve up simply are not good enough.

So how could the search experience be improved?

One way is to make the results better by asking just a little more of the searcher up front. If you are able to ask the right questions, you can infer much. For example, if, instead of a single submit button, suppose there were three; "animal," "vegetable," and "mineral." By forcing the user to select, huge amounts of ambiguous results could be eliminated. Of course, this approach impacts the simple, intuitive search box and button combo myth, but if the experience and results are better, does it really matter?

Another approach is to build applications and interfaces that integrate rules to constrain search queries to a specific business purpose or process. To quote from a patent currently pending (um, mine):

"This invention relates generally to software-based search technology, and particularly to rules-based (rules being any standardized set of principles or methods for organizing and structuring information (e.g. The AP Style Manual; IDEF, a business process language; the Marquess of Queensberry rules for boxing) searching via a constrained or guided query structure."

 You can read the full patent here should you be interested.

Bottom line -- attributing the property of "intuitive" to a text box forces a free-for-all environment that must cast too far and too wide, and demands far too much processing on the backend in the attempt to guess at user intent.  Constraining the queries, either by empowering the user or by integrating them into the user experience, can more easily deliver better, more relevant information.