Improving Information Access

Where next for enterprise search? Business Management spoke to Bradley Allen, founder and CTO of Siderean Software, Alain Heurtebise, VP of Sales and Marketing at Exalead, and John Turnbull, General Manager at Thunderstone Software, to find out.

Bradley Allen is founder and CTO of Siderean Software, a company that helps people quickly view and explore the full scope of enterprise and web-based information available for better-informed decision-making. Previously, Allen has held executive positions at Inference, Limbex Corp. and TriVida Corp.

Alain Heurtebise, VP of Sales and Marketing, is responsible for the overall strategy and direction of Exalead's direct and indirect sales channel worldwide. He began his career as a project manager for information technology services company Electronic Data Systems and Générale de Service Informatique (currently known as ADP).

John Turnbull joined Thunderstone Software LLC in 1992 to lead the company’s development efforts in marrying existing full-text search capabilities with structured database-type searches. Thunderstone successfully released the resulting Texis product within a year. He assumed his current role as General Manager in 2001.

BM. The themes and challenges in the enterprise search space aren’t that much different from those in the ECM and business intelligence arenas – at their heart, all are focused on cross-cutting collation and analysis of information found in various web-based or enterprise repositories. How are these different disciplines coming together and what challenges and opportunities does this present to companies large and small in the management of their information?
JT. The ever increasing amount of information produced by and available to companies today provides a growing challenge. To go along with that there are a growing number of products that can help with specific aspects of that information. Each of the disciplines is expanding to try and provide an enterprise-wide view to the available information, and gaining capabilities from the other disciplines, but still maintains it’s core view, whether that be a central repository for all information, analyzing selected information, or an access method to information.

The challenges for companies include knowing what information they have, and deciding how it can be best used within their workflows to provide a competitive advantage. In most organizations there are many opportunities to make better use of the available information; however it can be a challenge to find the right tool with the features and flexibility needed.

AH. At a high-level, information access, business intelligence and enterprise content management solutions all aim to help people across various lines of business to make informed decisions and work more efficiently. Exalead believes that the right information access solution should provide organizations with a single access point to their data repositories, regardless of data format or location, while BI and ECM are more oriented to helping businesses produce the right documentation and reports to discover trends. Traditionally, businesses relied heavily on structured information to make these decisions, whereas more recently businesses have begun to weigh unstructured data more heavily.

As a result, information access technology offers a significant opportunity for businesses to marry BI and ECM for a fully integrated solution that leverages the structured data and unstructured data within an organization. Organizations that integrate information access technology at the infrastructure level will no doubt gain a clear competitive advantage.

BA. We believe that what’s key to managing information for the enterprise is moving the focus from the legacy notions of data and content to metadata. Metadata is the glue that binds different types and sources of information together so that it can be addressed, accessed, navigated and discovered in a unified and coherent manner, irrespective of what silo it came from or the form in which it is stored.

We can add structure to unstructured information in a way that helps improve access and discovery of relevant information by making it possible to mine and slice-and-dice it – operations that were traditionally the province of data warehousing and OLAP. Conversely, we can provide free-text retrieval against chunks of unstructured content embedded in records from relational databases – something that adds tremendous flexibility and usability to the shop-worn paradigm of database query and reporting. The challenge to companies is to find a way that leverages their existing long-term investments in enterprise search, ECM and data-centric technologies while providing a unifying overlay at the metadata level for their users.

BM. The key to achieving the benefits of organization-wide search is effective integration of the search solution with other systems and workflows within the organization. Do you have any recommendations as to how this can best be achieved?
AH. There are several important factors to consider when selecting an information access solution to ensure rapid adoption and ROI. From an adoption perspective, it’s critical that the technology solution offer sophisticated, yet easy-to-use search features as a standard to ensure that both novice and power users can leverage the solution out-of-the-box. Second, organizations should consider the user interface (UI) an important basis for their decision. The design of the UI should be intuitive and allow users to easily refine their search queries – all from a single access point. This will be key in driving user adoption, especially when the solution is extended company-wide, which can include global subsidiaries.

BA. Search solutions have traditionally been treated as entirely user-centric applications, used on an ad-hoc basis at the user’s initiative. The key to effective integration with other systems and workflows is to move to an approach which treats search engines in the same manner as databases – a source of results based on machine-generated queries which are used to generate relevant semi-structured information on demand, changing as new content is ingested and new results are available. In effect, this moves search from a user application to infrastructure that shapes and channels the flow of relevant, actionable information within a broader organizational workflow.

JT. The first thing to consider when looking at integrating a search solution into existing systems and workflows is understanding the existing systems and workflows, and what is required of the search solution. A search solution should have the flexibility to be tailored to existing and future workflows. A well documented interface to the search solution will make it easier to integrate both from the data acquisition side, as well as being able to provide results where they are most needed.

Beyond that is ensuring that the search solution can meet the business requirements of the workflows. Some business requirements to consider there include security models ensuring only authorized people see sensitive information, being able to use all the available information including meta data, structured data from databases along with the unstructured information, how quickly new data should be available to searchers, and what type of searches are possible.

BM. Business processes are increasingly being driven by how people access content and inject that content into business goals. So what will be the key technologies as companies look to convert information into business success?
BA. It’s our belief that the emergence of a feed ecosystem within the enterprise – what most people on the web see as the blogosphere, but increasingly including structured data flows leveraging the existing syndication standards RSS and Atom – will be one key aspect of the coming disruption in the information access market. Companies will soon realize the fact that this ecosystem does for data and content together what the emergence of Web 1.0 in the late 1990s did for content delivery – allowing dramatically easier and cheaper deployment of solutions. Assuming this, key technologies in this upcoming period will include data and content aggregation tools, spanning the full range of sources inside or outside the firewall; semantic technologies such as the W3C standards RDF and OWL, which provide a way of stitching together aggregated pieces of content and data into a unified information architecture; metadata creation, retrieval, update and deletion (CRUD) tools (in particular those focused on creating taxonomies, controlled vocabularies and ontologies) and workflow platforms that add additional value to aggregated metadata; and Web 2.0 solutions that provide ways for people within the corporation to easily contribute content and data relevant to their ability to make effective, informed decisions.

JT. While there are several key technologies today that can help convert information into business success, including search data warehouses which allow users to query all structured and unstructured data in an ad-hoc manner to answer their questions, and collaboration tools to let groups enhance information, the real key technology is a platform that is flexible enough to meet the changing needs of the business as different types of information become available, and the business goals change.

These tools will help users set the line between information overload, and not finding the information they want. Some of the technologies a flexible tool enables are alerting applications that can notify employees of new or changed content they are interested in, filters to hide the content they don’t care about, and passive search embedded into applications that can suggest related information as content is created.

AH. Being able to turn information into business success is predicated on that information being the most recent and most relevant. As such, the critical technologies needed to achieve that will be scalability, specifically the ability to scale and search huge amounts of information quickly and easily; the capability to allow users to search the way they think – what Exalead calls Search by Serendipity, a patented-technology developed by the company to permit someone to zoom in or out on information or topics and navigate across, through and around the information based on their impulses; and real-time indexing, regardless of a repository’s size, to ensure people can gain access to up-to-the-moment information.

BM. With the rise of Web 2.0 tools and technologies, the role of search is expanding fast. What exciting developments is this bringing, and how do you see enterprise search evolving over the next 12-18 months?
JT. The most exciting development is providing users the ability to create their own tools for organizing and finding information. While this can be a challenge for IT staff to set rules and create a framework for the applications, the ability for users to easily create applications will be used, and the challenge will be to make it easy for that content to be reused throughout the entire enterprise, rather than be stuck in an individual’s application.

Enterprise search will evolve to allow more interesting queries using more data from the organization, and be able to consume data from and produce results in more formats to be used throughout the organization, and in individual applications users create. For example a user may create an RSS feed that publishes documents containing statistics about cancer, that another user combines with other feeds to aid their research.

AH. Before Web 2.0, the web was a one-way communication channel, with little human-machine and user-to-user interaction; with Web 2.0, the user is more of a participant in the process of discovering information. What we call Web 2.0 today is largely based on elements of sharing, personalization and social networking. Search has historically been a key driver in the evolution of the web and has continued this trend in helping to shape the Web 2.0 environment.

These key Web 2.0 elements translate well into business environments, which are demanding collaborative tools, demonstrating the infinite potential for Search 2.0 applications to take on new life in the enterprise. Information access will serve as the foundation and access point for sharing data across an organization. The big challenge when Web 2.0 moves inside the firewall is security. Therefore, information access technology must be infrastructure compliant with an organization’s corporate security policies.

BA. The focus of user requirements is moving from search in the traditional sense – typing in a few keywords and getting a list of relevance-ranked results – to the broader notion of information resource discovery. Discovery requires users to easily see the full scope of available information – a task for which search as it is conceived today is insufficient.

With this shift, the role of search moves from the central focus of information access and discovery to an important but supporting role in a broader notion of navigation. Navigation solutions allow people to effectively see information in context, use that context to zero in on relevant information, and then once a relevant resource has been found, treat it not as an end to the search process but as a point of departure to find and follow relationships to get to other relevant information. This means that we can go beyond a notion of searching to find things that we know about (if we know how to ask for it) to a notion of navigating to reliably find things we know about (even if at first we may be unsure how to ask for it), as well as discover that which we didn’t know we needed to know – a very different and more powerful approach.