Search Engines Don't Suck, They're Just Limited - Part I

by Carmine Porco — At most organizations the number one intranet user complaint is, “I can’t find anything!"
Typically when people think of retrieving information from their intranet or enterprise portal they think of the ‘corporate search engine.’ And often the frustration and inability to find information is directed towards the search engine itself, leading the user to conclude that search engines ‘suck.’ Search engines, in fact, don’t ‘suck’- they’re just not built to do everything we expect them to do and they are heavily dependent on human input.
 
"Many organizations focus their enterprise search efforts on making it easier for users to find the information they need," states Jeffrey Mann, META Group. "While this is a laudable first step, as they move toward the real goal of implementing contextual collaboration, organizations must try to make content find the user." In a recent article, entitled 'Finding Content and Data, or Making it Find You', Jeffrey continues, "Providing tools to users can make individuals more productive. However, real improvements require that the system understand the task a user is doing and automatically provide the functions, content, and reports needed to complete the task."
 
In order to efficiently retrieve information from the enterprise, we have to use the specialized systems that were designed for specific types of information. The trick is to bring all of these tools together to produce a 'super search' tool: the composite application (we’ll get to this in a moment). First, let’s take a look at some specialized applications in an enterprise environment commonly used for retrieving information:
  • Content management systems (CMS)
  • Document management systems (DMS)
  • Customer relationship management systems (CRM)
  • Accounts Receivables systems
  • Business Intelligence tools
  • Composite applications
 
These systems have all been created with specific and advanced functionality to retrieve, process and analyze data from their respective data stores and feeds. Within this environment, search engines are good at retrieving static documents and data that have no specialized application interface or retrieval mechanism, such as:
  • Static web pages
  • Network file stores
  • Unstructured data repositories
 
Many search engine companies have excellent tools for organizing and working with this sort of information.  However, they are unable to work with the sorts of specialized systems described above to leverage their expertise in finding, retrieving and using information. Consequently, search engines are only doing part of the job required to deliver users accurate and useful results.
 
So how then does a user retrieve business information from these disparate, specialized systems if not by using a search engine?
 
Users can go into each system, retrieve the information they need and then aggregate it to generate some global relevance. The problem: users often cannot find the information because they don’t know where the information resides, how to use the software application that produces it or even if the information exists. In any event, very few employees have the skill, patience or time to do this consistently and reliably. Consequently, they make decisions based on incomplete data.
 

Composite Applications for Information Retrieval

Composite applications offer solutions to these types of problems.  They draw on the capabilities of a collection of other applications in order to satisfy a business need. Simply put, a composite application acts as a conductor, instructing specialized systems to deliver certain views of information that it can either display to a user or, in some cases, pass along to another system for further processing.
 
Composite applications can be manually configured to interact with specific systems. So based on a known business need, the composite application is set to interact with known applications in specified ways. For example, a composite application could be built as a portal page or workplace that shows information about a client from a number of specific systems. This is effective when a company has a limited set of clients and a limited number of systems that hold information about those clients. In such cases, user information needs can be predicted and composite applications can be configured to retrieve the relevant information from different systems to fulfill those needs.
 
These simple composite applications enable business systems to be aligned for a particular need by deploying front-end integration vs. costly back-end reconfiguration or data consolidation.  It’s faster.  It’s cheaper. And in many cases, it’s much easier than training a whole bunch of employees to use entirely new systems.
 
Mark Walker, CEO of Blanketware, creators of Instant Workplace for enterprise portals highlights the value of composite applications to an enterprise. "Creating composite applications to fulfill new business needs not only makes companies more nimble and responsive to change, it enables them to get even greater ROI out of their existing enterprise systems."
 
If, however, a company has many different users, each with different, often unpredictable needs and a host of disparate systems, the administrative burden of configuring composite applications for every possible need becomes prohibitively overwhelming. In cases where enterprise portals are deployed, the inability to deliver users highly personalized and customized workplaces leads to low adoption and use. According to Jupiter Media, “Without end user adoption, portal ROI does not exist. 40% of companies who deployed a portal report adoption rates of less than 50%. Over half of companies report performance problems, even for internal users.
 
To address this particular problem, a new kind of composite application has emerged: dynamic composite applications for information retrieval.
 

Dynamic composite applications

Dynamic composite applications can automatically respond to unknown needs such as a user ‘search’ request. They identify the relevant enterprise systems and applications on the fly and interact with them, effectively instructing them to deliver precise ‘slices’ of information that match each user’s specific needs. Users receive the personalized view of information they need without requiring any additional work on the part of the administrator.