Searching for the Right Solution

The answers to most business-related queries usually lie within, which makes it imperative that knowledge workers are able to effectively trawl through your company’s information repositories quickly and easily in order to find the answers they need.

Enterprise search can find information stored just about anywhere in your organization – whether you’re looking for data stored on the desktop, tucked away on an intranet site, locked in a line of a business application or kept in a person’s head, an enterprise search tool can help. But what do organizations need to bear in mind before plunging into the murky waters of enterprise search? Business Management asked a number of experts for their views.

Dennis Moore is General Manager for Emerging Solutions at SAP, and heads a team responsible for creating solutions for untapped enterprise markets. With 20 years of experience in the enterprise software market, Moore serves as a member of SAP’s Product Leadership Team and Senior Executive Team.

Kirk Koenigsbauer is the General Manager of Product Management at Microsoft responsible for Office Business Platforms and Enterprise Search. He has held management positions across Microsoft with focus on the development of Microsoft’s own search platform.

Dr Michael Lynch, OBE, is founder and Chief Executive Officer of Autonomy. Dr Lynch founded Autonomy in 1996 and rapidly established the company’s reputation as the world’s leading provider of meaning-based technologies.

Laurent Simoneau is President and CEO of Coveo. He comes to Coveo from Copernic, where he was COO and orchestrated the enterprise product division spin-off into Coveo. His expertise in the search industry stems from more than 10 years of experience in developing and bringing to market innovative search products.

BMUS. It’s estimated that knowledge workers spend up to half their time looking for actionable information in order to bring value to the organization. How can enterprise search solutions help users better leverage information to improve productivity, decision-making and innovation?
DM. One major reason why information workers spend so much time searching for information is that they need to query multiple repositories to find all the information they need. Therefore they need to switch between various tools, with different – often unfamiliar – interfaces. Another is that users often do not know exactly where and how the facts and figures they need are stored. SAP’s enterprise search solution can help solve these problems by providing an easy-to-use single point of access to structured and unstructured content across multiple systems and interfaces.

Unlike the casual web surfer, business users generally want a specific answer to a specific question. For example, only one figure will accurately reflect the actual number of spare parts in inventory. And a purchase order cannot have more than one status. To deliver the right results, SAP Enterprise Search goes beyond broad-brush text-based approaches to focus on relevant data only. In general, search solutions should take into account the overall business context and provide search results based on individual users’ roles, preferences and authorization and yield actionable results. SAP Enterprise Search provides all the above capabilities, enabling users with the right information at the right time for improved productivity and decision-making.

KK. There are a few related issues at play here. First, the basic productivity issue: how can we help our people spend less time looking for the information they need to get their jobs done? Then there’s getting value out of existing assets: how can we tap into all the data and wisdom that’s out there in various systems, documents and in the expertise of our colleagues? The last aspect that goes right to the heart of good enterprise search: how can we make sense of the search results and use and share that information in an easy and productive way? The first two issues are fundamental to enterprise search, but it’s this last issue that can make search a truly strategic asset. In other words, to really help a user, enterprise search has to be easy to use, have a complete view of the information available to that user, and it has to deliver solid, relevant results in the context of tools enabling its use and sharing.

ML. During the last few years, we have seen explosive growth in unstructured information, such as emails, IM, audio and video. In fact, today it’s estimated that 80 percent of all information inside an enterprise is human-friendly and difficult for computers to process. Autonomy automates the processing of this content using unique algorithms that can identify not only the concepts and context of the information, but can also understand patterns and relationships between the data. As a result, highly relevant, actionable information can be delivered to the user in real-time – to support decision-making and drive productivity and innovation. Examples include such capabilities as conceptual search, hyperlinking, automatic taxonomy generation, alerting, collaboration and expertise networks. Today, over 16,000 customers worldwide, including ABN AMRO, BBC, Coca Cola, Ford and the New York Stock Exchange use Autonomy’s enterprise information processing platform.

LS. Enterprise search has both qualitative (quality of the user experience) and quantitative (cost) properties. When a user is looking for information, it’s a qualitative process – the user either finds the information quickly and easily, or they don’t. It’s when the user can’t easily find the information that search turns into a quantitative issue as users waste time trying to access information through other methods, or recreate information that already exists.

Enterprise search solutions improve user productivity if they meet two criteria – deliver high quality results that quickly connect users to the specific information they are looking for, and provide the appropriate context for that information. To improve user productivity, organizations need to ensure that search not only provides the most accurate results with structured and unstructured data, but is also secure, easily administered and readily integrates with existing processes and applications, and matches varying user expectations and requirements. With Coveo, users are better able to access information, make productive and innovative decisions, based on complete and relevant information, faster with a lower overall impact on IT resources.

BMUS. To empower these workers and fully realize the value of search software, organizations need to do more than just point software at existing corporate repositories. Why is a holistic approach to enterprise search important?
DM. One major reason why information workers spend so much time searching for information is that they need to query multiple repositories to find all the information they need. Therefore they need to switch between various tools, with different – often unfamiliar – interfaces. Another problem is that users often do not know exactly where and how the facts and figures they need are stored. SAP’s enterprise search solution can help solve these problems by providing an easy-to-use single point of access to structured and unstructured content across multiple systems and interfaces.

Unlike the casual web surfer, business users generally want a specific answer to a specific question. For example, only one figure will accurately reflect the actual number of spare parts in inventory. And a purchase order cannot have more than one status. To deliver the right results, SAP Enterprise Search goes beyond broad-brush text-based approaches and focus on relevant data only. In general, search solutions should take into account the overall business context and provide search results based on individual users’ roles, preferences and authorization and yield actionable results. SAP Enterprise Search provides all the above capabilities enabling users with right information at the right time for improved productivity and decision-making.

KK. A holistic approach to search is one that acknowledges what a messy place our work environment can be. When I need to find a bit of information, often I don’t know whether it’s in an e-mail, a blog, in a database somewhere, in a presentation or even if it only exists in someone’s head. A patchwork approach to search, where we have to try different tools and system features and still run the risk of missing that critical bit of data, is just not good enough. Not only does that fall far short of helping the user be effective; from a technical and security point of view it’s a nightmare. From a systems perspective, while you can just point a crawler at some of that content information, there’s a lot more of this information you want to be more thoughtful with. The key to effective enterprise search is letting you combine these for the user while maintaining the ease of consumption of the results by the user.

LS. A holistic approach is important because of the role that search plays in the enterprise. Enterprise search is the only application that offers a single access point to corporate knowledge. In this role, search becomes a key lynchpin in the protection, discovery and retrieval of corporate information. Enterprise search affects all users in the organization – from a support engineer to the executive management team – and each consumer has a different expectation, a different need and different technical abilities. By including the users’ role, geography and social influence, search becomes a powerful tool to achieve the corporate balance of protecting content while being able to discover and retrieve it. In addition, these needs must fit into the existing IT infrastructure. From implementation, customization and administration, search must behave holistically. Search is a multifaceted application with the objective of quickly and accurately connecting users with information, and this goal cannot be met without addressing the interdependency of each constituency in the enterprise.

ML. A holistic approach to enterprise search is no longer just important – now it’s the law. On December 1, 2006, the Federal Rules of Civil Procedure (FRCP) were amended for the first time in 73 years, making all electronically stored information (ESI) discoverable for the purposes of litigation. Any company operating in the United States can face significant fines if they fail to discover and produce all electronically stored information relevant to a case within 99 days of receiving the request. This is a huge challenge for enterprises.

If you are a corporate counsel, the likely approach you will take to meeting the FRCP is to outsource the electronic data discovery (EDD) for each case or perhaps to implement a separate EDD search solution. But, running an EDD solution separately from enterprise search is expensive, does not guarantee that all information can be accessed, does not operate in real-time and creates a parallel system problem. The approach from the IT department is to use existing search technology, yet most search engines are not suited for EDD legal search as they often return only partial results for performance reasons and are not able to search all electronic sources such as phone recordings, video, instant messages or custom application formats.

Autonomy offers the only holistic approach to meeting both an enterprise’s search and electronic data discovery needs. We offer a unified solution for pan-enterprise search that can also perform legally acceptable EDD across all sources of information on a common platform. This holistic approach offers IT a way to leverage a single search solution to deliver business value through enterprise search, while simultaneously meeting the rigorous requirements introduced with the changes to the FRCP.

BMUS. This segment is dominated by some fairly major players, with Google, Yahoo and Microsoft all stating their intention to make the enterprise search space their own. What is your unique selling point and how will this enable you to capitalize on the growth of the search market?
DM. Clearly, SAP’s strength is our unparalleled knowledge of business objects, enterprise information workers, and enterprise applications. What differentiates us is that we provide fact-and knowledge-based search, by not only letting users search for documents such as email messages and word files, but also for the vast majority of business-critical data today residing in enterprise applications. Text-based solutions simply cannot deliver the deep search capabilities essential for structured content of this kind. Information workers in an enterprise have very specific questions. For example they need to know “what is the status of a sales order” or “how many do we have in inventory” or “how much quota have I earned”. The information to the questions can only be found by accessing enterprise applications such as SAP. This requires a deep understanding of the semantics of the business architecture. It is therefore very different from a search you do in Google, Yahoo or Microsoft.

ML. Autonomy’s selling point is its unique way of processing unstructured information through a common enterprise-wide platform – the Intelligent Data Operating Layer – which allows for both a conceptual and contextual understanding of all of the information in an enterprise, no matter what the source or format. Other search engines have design or algorithmic flaws when applied to the enterprise. Page ranking methods, for example are merely a popularity contest, not an indication of relevance, while keyword search techniques will only find documents in which a particular word occurs. These approaches fall far short of what is truly required in an enterprise. This is why Autonomy is acknowledged by industry analysts as the clear leader in enterprise search and why it was positioned as the leader in Gartner’s Information Access Magic Quadrant. To maintain this lead, Autonomy continues to invest heavily in R&D – more than twice its nearest competitor in fact – which has resulted in more than one hundred patents. Autonomy provides solutions, built on top of the IDOL platform, for rich media, security and surveillance, electronic data discovery, contact centers and intelligent documents – all focused on helping organizations make sense of unstructured information wherever it resides.

LS. The attention that search is receiving from the top tier platform, web and infrastructure players is simply a validation for Coveo. With an increase in revenue of 400 percent over the past 12 months, and a debut in the ‘visionaries’ quadrant in Gartner’s 2006 Information Access Magic Quadrant, Coveo is a high growth player in the enterprise search space that doesn’t follow the classic search vendor model. As one customer said, “Coveo just gets it right, like the iPod.” Coveo’s applications are easy to use, highly secure, accurate and integrate seamlessly into the fabric of Microsoft-centric IT environments. Coveo Enterprise Search (CES) delivers innovative, powerful and cost-effective value with out-of-the-box document level security, unparalleled accuracy, consumer style ease-of-use, and an implementation cycle of less than 24 hours – a hard model to replicate if a vendor has other ‘hobbies’ like web search, game boxes and patents. In addition, we believe organizations shouldn’t buy the demo but evaluate search in their own environment with their own data, so we offer a free 30-day evaluation.

Coveo Enterprise Search for SharePoint (CESS) offers seamlessly integrated ‘supercharged’ search for content stored in SharePoint 2003 and MOSS 2007, and Windows SharePoint Services (WSS) 2.0 and 3.0. CESS empowers users to quickly make actionable decisions, increases user adoption of SharePoint and enhances the value of a SharePoint investment.

KK. We are one of those big names in consumer search, but I see us standing apart in our degree of experience and expertise in understanding the challenges faced by enterprise customers. We’ve devoted significant energy toward optimizing enterprise search for business information, which is organized very differently than information on the internet. Many search solutions can eventually deliver some information people seek – even if it is from a limited set of sources and they have to spend a lot of time sifting through the results to find it. But to help people not only find more relevant information from all types of sources but to be more productive with that information, Microsoft’s enterprise search solution is integrated within a broader information management infrastructure with our collaboration, portal, content management, and business process platform. We see the information management market in terms of search, collaboration, portals, etc. converging, and we have a great holistic solution including enterprise search to better address actual customer needs than a point solution.

BMUS. A key to achieving the benefits of organization-wide search is effective integration of the search solution with other technology systems within the organization. Do you have any recommendations as to how this can best be achieved?
DM. The approach we are taking with SAP Enterprise Search is that we are technology agnostic – meaning that we provide open interfaces to allow the integration of any information source from SAP as well as non-SAP applications. Our strategy is to federate data from others while also unlocking SAP data to them. Ultimately, our goal is to increase usage of SAP and to make all business intelligence and processes within an organization just one click away for better productivity and decision-making.

SAP Enterprise Search comes with a rich set of connectors, especially to search for data in SAP applications. Of course, delivering search capabilities across all repositories and applications would be a tall order even for a player like us. But this is where our extensive partner ecosystem comes into play. We let partners, development community and customers themselves build connectors to any data source they wish to integrate in SAP Enterprise Search. In addition we made significant investments in our UI technology. To allow for maximum ease-of-use, search should be embedded in the familiar work environment of a user. That is why we are taking a multi-channel approach with SAP Enterprise Search and enabling users to work in their preferred interface – whether this be a portal, desktop productivity tool, RSS, blackberry, mobile, etc.

LS. Coveo architected its application to integrate at three very important levels. One level is to allow users to actually integrate it with a known set of tools, in our case Microsoft technologies like .Net, ASP, Active Directory, XML, VBScript and JScript. Second is to be able to ‘point’ Coveo at the rest of the IT infrastructure and be able to aggregate content into its index and search it securely. Coveo does this via easy to implement connectors, our Open Crawler and highly configurable API. We designed the product for integration and have customers searching Lotus Notes, Oracle, SAP, Symantec Enterprise Vault, audio files, video files, as well as Unix and Linux systems and files. Third and most important is the ability to maintain the administration and scalability. Coveo’s architecture can manage an intranet with mirrored indexes, fault tolerance and geographic synchronization of indexes. Coveo’s core design elegantly handles disparate applications over multiple geographies while maintaining a low total cost of administration.

Coveo offers sophisticated, yet easy-to-use administration to easily optimize the application performance, system performance and results tuning. Coveo’s reporting offers real-time analysis on all services including scheduling, system backups, end-user query history and detailed document properties on content, size and index history.

KK. So many of us are realizing how important the broader information management infrastructure is to our organizations that we are doing a lot to invest in areas like content management, business intelligence and workflow applications. Integrating search within this infrastructure can be as easy as working from the ground up to implement a simple platform solution that encompasses some or all of these capabilities. Because we need to be able to tap into the minds of our experts and collaborate with project teams, a great enterprise search solution integrates with our collaboration platforms. With capabilities like presence notification from the collaboration tools and people search from the enterprise search solution, we can connect with the right people as soon as they’re available, making everyone more productive both individually and collaboratively. That’s why I would recommend having search integrated within collaboration and other solutions earlier rather than later amongst separate point solution systems where the integration costs can be high and never yield a truly integrated, productive user experience.

ML. Enterprise search is so fundamental a requirement these days that we believe it will be embedded in every application in the enterprise – at the user interface layer through application programming interfaces (APIs) and delivered by independent software developers, as well as through connectors. Autonomy’s implicit query function, for example, delivered at the user interface layer, makes search available to all applications. It has a unique ability to form a conceptual understanding of the data displayed on a user’s screen and, based on that understanding, will automatically find and deliver relevant information from other sources direct to the user’s desktop. Thus the user can remain on-task rather than being forced to stop work to search.

Significantly, Autonomy supports over 1000 different file formats. Our KeyView software development kits (SDK), for example, enable applications to interact with the information that matters to end-users in the appropriate format and language. Using our SDKs, OEMs and developers can build solutions that support unstructured information without the onerous task of having to stay current with each new release of the hundreds of native applications and diverse file formats that exist. Autonomy’s technology and performance is unmatched in the market – which is why leading ISVs (including IBM, Lotus, Oracle, Interwoven and Symantec) choose to integrate Autonomy’s proven technology into more than 300 products

BMUS. Enterprise information is a key strategic asset for organizations. How does your solution manage security and prevent unauthorized searches?
DM. Allowing search for business objects in enterprise applications requires a very good understanding of security. Security models for business objects are not maintained by an operating system, but instead are intrinsic to the applications that hold the business objects. This is different from document-based search, which has less stringent security controls. Our advantage is that we know our applications very well, which allows us to safeguard confidential and sensitive business information effectively, and ensure staff cannot access content they are not permitted to view.

In addition, unlike public web search, search in an enterprise needs to support role-based, personalized search with built-in context awareness. For example, even if a sales professional and an IT manager run identical queries, each requires very different results, reflecting their very different jobs. Accordingly, SAP Enterprise Search supports personalized searches based on individual users’ roles, preferences and authorizations. The software takes into account the overall business context – identifying the business process the employee is currently working on, automatically homing in on the systems involved in his or her specific tasks, and excluding all others.

LS. Not a day goes by that you don’t read about a data leak of customer information, credit card information or other such private data. Today’s data must be simultaneously accessible and protected. Secure search must match security on documents, files, e-mails, directories and servers. Access to the search engine collections themselves must also be secured. Secure search is basic functionality for any search engine, and it should not cost extra or take extra effort to implement. In addition, a user should never see search results they are not allowed to see. Coveo is seamlessly integrated with Microsoft-centric IT environments and leverages Active Directory security settings and other security mechanisms. When a user logs into their computer, their security credentials are used in a search to show all results to which they have access. The advantage to this approach is that the response time is prompt, similar to a search of unprotected content. However, the search engine is in sync with the security source. The original content server does not need to be online and the user does not need to take any extra steps to view protected content. Coveo Enterprise Search (CES) can be used as an effective tool to confirm that protected content is indeed protected – IT administrators can use CES to index all information in the organization, and then search for protected content logged on as various users. CES will match your security with unparalleled speed and accuracy at no extra cost, and with no extra services.

ML. The key challenges in offering secure search to customers include being able to deliver sub-second response time, while still controlling user access privileges and document level security – and doing this at scale. Solutions also need to be able to accommodate the fact that enterprises have different security policies, technology environments, authentication methods, as well as devices with sensitive information that could be lost by users. Only Autonomy’s Mapped Security approach guarantees security, while providing the scalability that enterprises require. This high performance model takes advantage of efficient signal handling, whereby the application identifies the user by sending user details along with the user’s query without having to interrogate the repository for each potential hit. This is a huge efficiency advantage.

KK. For any number of reasons, from compliance to protection of intellectual property, it is essential to control access to sensitive business information. Since we can often see at least some of the information contained within displayed results, simple access control may not provide the security our organizations need if searches retrieve and display results from documents and data sources we’re not really supposed to be able to see or perhaps even know exist. Microsoft’s enterprise search solution helps keep information safe by delivering security-filtered results. The search engine takes my user permissions, as defined by your user authentication infrastructure, and delivers only the results that I’m allowed to access. By getting your organization to use a single, security-enhanced, centrally managed search tool, you can close vulnerabilities otherwise faced by using unmanaged search tools and enterprise search solutions that do not filter results based on permissions.

BMUS. First generation enterprise search seems to focus on document search. How do you see search evolving and what will it take for enterprise search to be broadly accepted within organizations?
DM. Clearly, we believe that the value of enterprise search only comes with bringing the worlds of structured and unstructured data together. As mentioned before, the vast majority of business-critical data resides in enterprise applications. In addition, enterprise search must be ultra simple to use to achieve the same experience users have grown to expect on the web. Search solutions will need to provide personalized search results based on individuals users’ roles, preferences and authorizations. This is exactly the approach we are taking with SAP Enterprise Search. The solution reflects our commitment to enable all information workers within an organization to actively participate in business processes.

In fact, our goal is that just as web search is now widely seen as the gateway to the web, so SAP Enterprise Search is intended to become the gateway to processes supported by SAP and non-SAP software. Users will be able to jump directly from the search results to a business transaction or workspace without specialist knowledge of the systems involved.

ML. Being able to access and understand all of the information that resides within an organization – where it is unstructured, structured or semi-structured – has become a strategic imperative for organizations everywhere. In an environment where business decisions are being made in real-time and where all information is now ‘discoverable’ by law, no matter what the format or source, information now represents both a risk and an opportunity. Sub-standard keyword-only search solutions are not enough to address these issues. What is required is the type of pan-enterprise platform provided by Autonomy that not only finds the information, but also makes sense of it and performs sophisticated operations that add real value to the business.

If the first wave of computing successfully brought intelligence to structured data, the second wave of computing will bring meaning to all data, whatever format it comes in. Being able to understand concepts in context and see the relationships that exist between disparate pieces of information is becoming a fundamental requirement for organizations everywhere. Autonomy is leading the way with meaning-based computing.

The US Department of Homeland Security, for example, uses Autonomy’s meaning-based computing technology across 21 agencies to monitor suspected terrorist groups, create a consolidated terrorist watch list and alert authorities in real-time to potential terrorist activity. Similarly, the Ford Motor Company uses meaning-based computing to transform the text, audio and video files in its research libraries into meaningful reference material helping more than 150,000 employees get up to speed on new projects as quickly as possible.

KK. Enterprise search has immense potential, but because so few companies have implemented a truly enterprise-scale solution, not many appreciate what is already possible. Search will continue to evolve to provide richer views of further sources of information, just as our enterprise search solution delivers for subject matter expertise among fellow employees and search of line of business systems for business intelligence purposes.

Search will, I think, take its place as one of those ubiquitous technologies, like e-mail, that people use so much they simply can’t do without it. People will want to see it as part of their collaboration tools, as part of their workflow tools, their communication tools, everywhere they do their work. They don’t want to step out of the thing they’re working on right now so they can go to the search tool. This is why enterprise search is so important – it’s the only approach that can deliver that kind of familiar, integrated, effortless capability across all types of information.

LS. Information in the enterprise takes many forms. Today, documents are the primary currency for search applications. However, search solutions must be able to find information in newly digitized content from scans of paper documents (OCR and PDF files), content in databases and information from enterprise applications (i.e. ERP, CRM). Coveo recognizes these information sources and makes them easily searchable. In addition, as multimedia becomes more prevalent in the enterprise, search applications must provide the same content retrieval standards for rich media that they provide for document content. The ability to search the speech content in rich media files – not just the meta data – is key to meeting strategic initiatives such as compliance and e-discovery, long distance learning and improved customer service. Coveo’s Audio Video Search (AVS) offers a powerful solution optimized to search multimedia content. Coveo Audio Video Search (AVS) accesses rich media content with unparalleled precision and ease, allowing users to search across audio and video content as easily as they search across document content. Coveo AVS’s patent-pending ‘automatic training’ capabilities leverage enterprise content to improve the precision of speech recognition. The result is a searchable, accurate transcript of speech content that is enterprise aware (i.e. proper names, employee names, domain terms).

Coveo’s ‘high value’ navigation features such as; refined search, collaborative ranking, concept summaries, quickview, clustering, expert recommendation, blog and wiki support, and visualization of search results quickly connect knowledge workers with accurate and relevant enterprise content.