Slide 2: OFC326 Enterprise Search Technical Drilldown in Microsoft Office SharePoint Server 2007 Bill English MVP, MCSE, MCT Mindsharp
Slide 3: Agenda Enterprise Search Vision and Strategy Broader Content Aggregation Deployment and Management Wrap Up
Slide 4: Enterprise Search Vision Instantly connect people with the information and people they need to excel at work Great results every time Integrated with familiar applications Across all enterprise repositories from one place Scalable, manageable, extensible and secure
Slide 5: Enterprise Search Strategy Single engine for search across the enterprise Internet Presence Extranet Enterprise Division Team Individual @ Desktop E-mail, Documents, Web Content ERP – CRM – DW – HRS structured un-structured
Slide 6: Strategy Notes Relevant results every time Across all types of business content Scalable and manageable deployments Search as an enterprise service Consistent system across Microsoft Windows SharePoint Services and Microsoft Office SharePoint Server 2007
Slide 7: Consistent Search System Today: Different systems in Windows SharePoint Services v2 and SharePoint Portal Server Different admin and user experiences No simple way to move from Windows SharePoint Services to SharePoint Portal Server search Future: Consistent systems in Windows SharePoint Services v3 and Office SharePoint Server 2007 Search in Windows SharePoint Services v3 scoped to this Web and below Office SharePoint Server 2007 adds aggregation features for portals Windows SharePoint Services v3 experience can be “upsized” to Office SharePoint Server 2007 experience without rebuilding the index
Slide 8: Consistent Search System Deployment models Goal: Only run one search system on a given SharePoint farm Windows SharePoint Services v3 “standalone” farm Windows SharePoint Services search system drives within-site search System scales out as sites are added Windows SharePoint Services/Office SharePoint Server 2007 v3 “big box” farm Office SharePoint Server 2007 search system drives within-site search and portal search over aggregated content Scale out the system to handle more sites and aggregated content
Slide 9: Relevance Challenges and approach Top results on first page, every time Enterprise search different from Internet Lack of rich linking Lots of non-Web content – Microsoft Office docs, line-of-business data, etc. Security is paramount IT runs the service, not us Revamped ranking engine Extensive collaboration with Microsoft Research and MSN Internet search New ingredients tuned for the enterprise
Slide 10: Relevance New ranking ingredients Click Distance – Browsing distance from authoritative sites: shorter tends to be more relevant Anchor Text – Hyperlinks act as annotations on their target URL Depth – URLs higher in the hierarchy tend to be more relevant URL Matching – Direct matches on text in URLs Metadata Extraction – Automatically extract titles and authors from document text Automatic Language Detection – Helps bias toward results in your language File Type Biasing – For example, PPT docs tend to be more relevant than XLS Text Analysis – Traditional text ranking based on matching terms, term frequencies, word variants, etc. Many others in the special sauce
Slide 11: Search Experience Numerous improvements Search Center New UI framework Alerts Get notified on result set changes Integrated with WSS alerts experience Doing queries New syntax Property search Results Hit highlighting Duplicate collapsing Best Bets “Did you mean” Auto definitions Direct matches Customization Style or reuse search controls Add new tabs for custom search
Slide 12: Search Experience Numerous improvements Search Center Doing queries New syntax Alerts Results Customization
Slide 13: Indexing Management Fundamental part of search administration Choose what to index, how, and when Content sources, crawl rules, crawl log Streamlined experience and more control One index per SSP Multiple start addresses per content source Entirely new browsable, filterable index log Explicit SharePoint content source type Content sources decoupled from scopes
Slide 14: Crawl Process Architecture Crawler decides what kind of content will be crawled based on the prefix in the URL Load the following: Protocol handler iFilter (Index Filter) Site Path Rules Crawl Settings (depth and hop) Site Hit Frequency Rules Connect to Content Source Stream out metadata and content
Slide 15: Crawl Process Architecture (Cont’d) Pass content through several plug-ins: Indexer – chunking, work breaking, stemming, crawl history (errors and successes), flushes URL’s out of current crawl queue Archival – Updates the MOSS schema No more Persistent Query Service – Handled by a timer job in the Windows SharePoint Services search engine No more Topic Assistance (Category Assistant) Metadata placed in SQL – No more sps.edb Content placed in file system-based index
Slide 16: Architectural Updates Alerts are now sent daily, based on a timer job that looks at the crawl history and notifies the user of new or updated content Shadow Indexes are continuously propagated from index servers to query servers Problem with network or server down? Automatic pause all propagation and indexing until problem is solved Index reset automatically stops e-mail alerts from firing; administrator manually starts e-mail alerts
Slide 17: New Search Experience Create a new content source and view result set
Slide 18: Agenda Enterprise Search Vision and Strategy Broader Content Aggregation Deployment and Management Wrap Up
Slide 19: Broader Content Aggregation Best “answers” for queries are often not found in documents Colleagues and partners Discussions, blogs, wiki’s Line-of-business systems and reports SharePoint Search offers new ways to index and expose non-document information People search enhancements Business data search
Slide 20: People and Expertise Bring people into the search experience Numerous improvements on SharePoint Portal Server 2003 Index any LDAP v3 directory Dedicated tab for people finding Results grouped by “social distance” to you Result refinement by properties like department Expertise finding with Knowledge Network Free download that compliments MOSS 2007 Skills, Who knows who, external contacts, connections Find people based on their knowledge and contacts
Slide 21: Business Data Search Search data, not just documents Scenario: Find an account manager in Siebel Today Vertical applications lack full-text search Most users can’t locate or access vertical apps Hard to crawl business data with SharePoint Office SharePoint Server 2007: Easily index any structured data source No need to write IFilters or protocol handlers No need to create HTML representations of data Highly customizable results Integrated with scopes and search center
Slide 22: Agenda Enterprise Search Vision and Strategy Broader Content Aggregation Deployment and Management Wrap Up
Slide 23: Deployment and Manageability Administrator is gatekeeper of a great search experience Index coverage, best bets, scopes, etc. Good product + effective administrator = happy users Key concepts and investments Shared services Indexing management Search scopes Schema mapping Query reports Editorial results
Slide 24: Shared Search Services Indexing is resource intensive Network and local I/O, CPU, memory footprint, load on external servers, etc. Therefore want to avoid redundant indexing SharePoint Portal Server 2003 Centralize indexing with “master” portal Specialized configuration with limitations Office SharePoint Server 2007 “Always on” shared services – All sites can be configured to use the same index Resource intensive operations controlled centrally, but experience still manageable per consuming site
Slide 25: Design Notes for Shared Services Individual web applications may or may not be associated with an Shared Services Provider (SSP) Farms can host multiple SSPs Most scenarios: one SSP Extranet/customer portals with high ethical, visual and/or security walls – Ideal for multiple SSPs
Slide 26: Scopes Index subdivisions optimized for fast searching Examples: All Content, People, Specs, Foo Division Key underlying feature for supporting multiple disparate search experiences from one index SharePoint Portal Server 2003 Scopes tied deeply to content sources Inflexible and challenging to manage Office SharePoint Server 2007 Scopes decoupled from content sources Define scopes based on arbitrary content properties such as URL, type, and author Simple or multi-rule scopes For example, “all Marketing Plans on Salesweb” Global scopes and per-site scopes
Slide 27: Scopes Example Contoso Index (SSP) All Content Fabrikam Index (SSP) All Content Support Incidents Specification s Market Research Product Plans Toolsweb Marketing
Slide 28: Schema Management Search system aggregates content from heterogenous repositories Typically thousands of “discovered” properties Need to map foreign properties into search For example, search on “Title” should span file names, Microsoft Office document titles, and discussion subjects Key improvements Revamped UI clearly separates foreign schema from internal search schema Foreign schema automatically bucketed for easier management Flexible mapping Map multiple foreign properties to one search prop Retain all property values or in priority order
Slide 29: Query Reporting Best way to improve search is to understand current usage New out of the box usage reporting in Office SharePoint Server 2007 Query volume trends, top queries, click-through rates, queries with zero results, etc. At both site and service provider levels Export data for extended reporting in Excel Respond to feedback with configuration changes or editorial results
Slide 30: Query Reporting
Slide 31: Editorial Results Keywords and Best Bets Special results controlled by administrator Necessary when Users frequently query for non-indexed items Organization wishes to “promote” certain items to a prominent place in results Authoritative sites and click distance Authoritative pages = excellent directories of good sites Where do they go when they try to find X? Without authoritative sites configured in the relevance settings, the benefits of click-distance are missed
Slide 32: Continuous Propagation Indexes are built on index machines and propagated to query machines Content not available for query until propagation SharePoint Portal Server 2003 Indexes only propagated when crawls complete Large crawls can take days Office SharePoint Server 2007 New “continuous” mode of propagation–only mode Index updates streamed to query server within minutes of content getting indexed Staged propagation also supported for explicit control over propagation timing
Slide 33: Reference Windows SharePoint Services Local SharePoint content Can Index Rick, relevant results Alerts, RSS, DYM, Dup Collap Scopes, Managed Properties Best Bets, Result Removal, Query Reports Tabs People Search, KN BDC Search APIs Provided Office SharePoint Server 2007 SharePoint Web, Exchange Fileshares, Notes, LOB Query Query + Admin
Slide 34: Search Tips Group multiple targets of content into a single content source Spending time planning out the metadata you wish to group into scopes Have sufficient bandwidth to crawl all of your content sources Have a dedicated indexing server in your farm
Slide 35: Resources Technical Chats and Webcasts http://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asp Microsoft Learning and Certification http://www.microsoft.com/learning/default.mspx MSDN & TechNet http://microsoft.com/msdn http://microsoft.com/technet Virtual Labs http://www.microsoft.com/technet/traincert/virtuallab/rms.mspx Newsgroups http://communities2.microsoft.com/ communities/newsgroups/en-us/default.aspx Technical Community Sites http://www.microsoft.com/communities/default.mspx User Groups http://www.microsoft.com/communities/usergroups/default.mspx
Slide 36: Fill out a session evaluation on CommNet for a chance to Win an XBOX 360!
Slide 37: Live from Tech·Ed Webcast Series has Been Brought to You by: www.microsoft.com/hpc
Slide 38: © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.