Difference between revisions of "Blog/Refine search results by term trees"

From DIQA-Homepage
 
(9 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
|Titel=Sharepoint: Improve the search experience with the TermTree Refiner
 
|Titel=Sharepoint: Improve the search experience with the TermTree Refiner
 
|News image=File:EasyTaggingText.png
 
|News image=File:EasyTaggingText.png
|Publication date=2020/09/01
+
|Publication date=2021/05/06
|Published=0
+
|Published=1
 
|Keywords=sharepoint, office 365, o365  
 
|Keywords=sharepoint, office 365, o365  
 
|SEO-Description=Automatic document tagging boosts your document findability? ⇒reade more!
 
|SEO-Description=Automatic document tagging boosts your document findability? ⇒reade more!
|Description=Finding documents in Sharepoint online/Office 365 can be tedious. If you want to restrict your search to certain document types (like: reports, inquiries, CVs, sales orders, invoices) you either end up with too many irrelevant search hits or relevant documents don't show up because they don't include the search term. If you rely on Sharepoint's fulltext search capabilities only, then your users will be frustrated and spend too much time to find the right documents.
+
|Description=Finding documents in Sharepoint can be tedious. If you have a hierarchy of terms, like product groups, series and products, then it is impossible to refine for terms that sit higher in the term tree, e.g. product groups or series. This frustrates users who cannot leverage the structural information of the term store. DIQA's TermTree Refiner Webpart overcomes this problem. Learn in this article how to use it and what it can do for your users.
 
|en_link=
 
|en_link=
 
|de_link=
 
|de_link=
Line 22: Line 22:
 
<div class="row">
 
<div class="row">
 
<!-- --><div class="col-lg-6">
 
<!-- --><div class="col-lg-6">
== Frustrating search experience without meaningful refiners: ==
+
== Better: filter documents by type, language, etc: ==
 
<html>
 
<html>
<div class="center"><div class="floatnone"><a href="/mediawiki_en/images/a/aa/Treerefiner_standardrefiner.PNG" class="image"><img alt="Tagcloud" src="/mediawiki_en/images/a/aa/Treerefiner_standardrefiner.PNG" width="100%" height="100%" style="vertical-align: middle" srcset="/mediawiki_en/images/a/aa/Treerefiner_standardrefiner.PNG 1x" data-file-width="" data-file-height=""></a></div></div>
+
<div class="center"><div class="floatnone"><a href="/mediawiki_en/images/0/06/Treerefiner.PNG" class="image"><img alt="Tagcloud" src="/mediawiki_en/images/0/06/Treerefiner.PNG" width="100%" height="100%" style="vertical-align: middle" srcset="/mediawiki_en/images/0/06/Treerefiner.PNG 1x" data-file-width="" data-file-height=""></a></div></div>
 
</html>
 
</html>
 
<br>
 
<br>
''Entering the search term "order" returns a lot of irrelevant documents (e.g. a CV) and even misses some sales order documents.''
+
''The TermTree Refiner Webpart shows the terms in their tree-context. Users can easily drill-up or drill-down in the tree to broaden or narrow the search.''
 
<!-- --></div>
 
<!-- --></div>
 
<!-- --><div class="col-lg-6">
 
<!-- --><div class="col-lg-6">
== Better: filter documents by type, language, etc: ==
+
== Frustrating search experience without meaningful refiners: ==
 
<html>
 
<html>
<div class="center"><div class="floatnone"><a href="/mediawiki_en/images/0/06/Treerefiner.PNG" class="image"><img alt="Tagcloud" src="/mediawiki_en/images/0/06/Treerefiner.PNG" width="100%" height="100%" style="vertical-align: middle" srcset="/mediawiki_en/images/0/06/Treerefiner.PNG 1x" data-file-width="" data-file-height=""></a></div></div>
+
<div class="center"><div class="floatnone"><a href="/mediawiki_en/images/a/aa/Treerefiner_standardrefiner.PNG" class="image"><img alt="Tagcloud" src="/mediawiki_en/images/a/aa/Treerefiner_standardrefiner.PNG" width="100%" height="100%" style="vertical-align: middle" srcset="/mediawiki_en/images/a/aa/Treerefiner_standardrefiner.PNG 1x" data-file-width="" data-file-height=""></a></div></div>
 
</html>
 
</html>
 
<br>
 
<br>
''If you provide search refiners, then the users will be able to precicely filter for all "sales order" documents, for instance.''
+
''The standard refiner hides important information like the position of a search hit in the term tree. Additionally you cannot filter for parent terms, like product groups.''
 
<!-- --></div>
 
<!-- --></div>
 
</div> <!-- row -->
 
</div> <!-- row -->
  
 
<p>
 
<p>
The following steps assume that you have Sharepoint Online/Office 365 and a library that contains at least dozens of documents or scanned documents (pdf or image formats). If you are familiar with certain configuration aspects of Sharepoint then you will need 30 minutes to go them through in your tenant. If you require assistance then we are happy to help!</p>
+
What do you need to do to achieve this improved search experience? <big> Just follow the steps below! </big>
 
+
</p>
 
+
== See the TreeRefiner Webpart in this video: ==
== Watch the steps in the video: ==
 
 
<div>
 
<div>
 
<p>
 
<p>
 
{{#widget:YouTube
 
{{#widget:YouTube
|id=jvKEF-u5r84&t=97s
+
|id=jvKEF-u5r84
 
}}
 
}}
 
</p>
 
</p>
Line 66: Line 65:
 
=== 1. Create a termset with your document types ===
 
=== 1. Create a termset with your document types ===
 
# Open the termstore manager (from the site settings)  
 
# Open the termstore manager (from the site settings)  
# Create a new termset, named "document types"  
+
# Create a new termset, give it a name like "products"  
# Create a term for each document type that you keep in your document library, e.g. inquiry, invoice, sales order, resume
+
# Create a term for each product of your company, take care to create a meaningful hierarchy of products, like product groups.
 
<!-- --></div>
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Create_termset.png|center|middle]]</div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Create_termset.png|center|middle]]</div>
Line 77: Line 76:
 
# Open the library advanced settings
 
# Open the library advanced settings
 
# Click on "create column"
 
# Click on "create column"
# Enter the name of the new library column, e.g. "document type"
+
# Enter the name of the new library column, e.g. "products"
 
# Select "managed metadata" as type
 
# Select "managed metadata" as type
# In the term set settings: select the termset "document type" (from the step above)
+
# In the term set settings: select the termset "product" (from the step above)
 
# Click on "Save" to close the form  
 
# Click on "Save" to close the form  
 
<!-- --></div>
 
<!-- --></div>
Line 86: Line 85:
 
<div class="row funktionen-row">
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
=== 3. Manually tag a couple of documents as examples (not all!) ===
+
=== 3. Manually tag a couple of documents as examples ===
# Open the library settings and add the new column "Document types" to the default view of the library
+
# Open the library settings and add the new column "products" to the default view of the library
 
# Open the "Quick Edit" view of the library.
 
# Open the "Quick Edit" view of the library.
# Populate the field "Document types" with meaningful values for a couple of documents per document type. It is not necessary to provide each document with a tag, but the more examples you provide, the better the results will be. You should achieve a coverage of 20% of all documents per document type (and at least 10 examples per document type).
+
# Populate the field "products" with meaningful values for a couple of documents. Take care to use leaf terms only, i.e. products, not product groups.
<big>You have now prepared everything required to teach the "Easy Tagging App" about your way to tag. You can now carry on and install the Easy Tagging App and train the "Learn Tags"-predictor.</big>
+
 
 +
<big>Download/install the TreeRefiner Webpart either for Sharepoint 2016, 2019 on premise or Sharepoint online: [[TreeRefiner_Webpart|TreeRefiner Webpart]]</big>
 
<!-- --></div>
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Tagexample.png|center|middle]]</div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Tagexample.png|center|middle]]</div>
Line 96: Line 96:
 
</p>
 
</p>
 
</div>
 
</div>
   <h2>Step 2: Automatically tag all your documents</h2>
+
   <h2>Step 2: Create a classic search center and add the TreeRefiner Webpart</h2>
 
<div>
 
<div>
 
<p>
 
<p>
 
<div class="row funktionen-row">
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
=== 1. Install the Easy Tagging App from the App Source (free evaluation!) ===
+
=== 1. Create a classic Search Center ===
# Open the site settings and click on "Add an App"
+
# Open the contents of your Sharepoint site and click on "subsites".
# Click on "Sharepoint Store"
+
# Create a new sub site by clicking on the "New" action link.
# In the Sharepoint Store: enter "diqa" as search term
+
# A form opens where you specify the new sub site.
# Click on the "Easy Tagging" card to initiate the installation.
+
# In the template field: select "Enterprise/Basic Search Center".
<br><br>
+
# Click on "Create" to create the new seach center.
:Direct link to the Microsoft App Store:
 
<html><a href="https://appsource.microsoft.com/en-us/product/office/WA200000136?tab=Overview" target="_self" title="Go to Microsoft AppSource"> &#9654;&nbsp;Got to Microsoft AppSource</a></html>
 
 
 
 
<!-- --></div>
 
<!-- --></div>
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Appstore.png|center|middle]]</div>
+
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Create_search_center.JPG|center|middle]]</div>
 
</div>
 
</div>
 +
 
<div class="row funktionen-row">
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
=== 2. Launch the app and grant it the requested permissions ===
+
=== 1. Add the TreeRefiner Webpart to the results page ===
# Open the site contents again where you should find the newly installed Easy Tagging App
+
# Open the page "results.aspx" in edit mode.
# Click on the App and grant it the requested permissions.
+
# Add the "TreeRefiner" Webpart into the page.
# The configuration page of the app opens in a new tab.  
+
# Open the Webpart in edit mode.
 +
# In the page edit view: open the property settings of the Refinement webpart
 +
# Click on the button "Choose refiners"
 +
# In the "refinement configuration" dialog: add the refinable managed propery
 +
(from the search schema step), like: RefinableString00 (or 01 etc).
 +
# Click on "Ok" to add RefinableString00 to the refiner
 +
# Save the modified properties of the webpart by clicking on "OK"
 +
# Store the modified page..
 
<!-- --></div>
 
<!-- --></div>
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Launch_the_app_modern.png|center|middle]]</div>
+
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Search_refiner.JPG|center|middle]]</div>
 
</div>
 
</div>
 
<div class="row funktionen-row">
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
=== 3. Connect the Easy Tagging App with the IDAS service (1/2) ===
+
=== 2. Modify the search schema ===
# In the configuration page of the Easy Tagging App: click on the "Predictors & Taggers" tab
+
# Open the Sharepoint admin center for your tenant or the central administration for SP onpremise
# Click on the link "Click here to connect to IDAS"
+
# Click on the "Search" section
<!-- --></div>
+
# In the Search section: click on "Search schema"
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Apptab.png|center|middle]]</div>
+
# Click on the hyperlink “Managed Properties” to retrieve the list of managed properties.
</div>
+
# Enter the name of the refinable managed property that you have selected as refiner (e.g. “RefinableString00”) into the filter and apply the filter to retrieve it in the list of refinable managed properties for strings.
 +
# The edit mode of the RefinableString-property contains the section “Mappings to crawled properties”. Click on the button named “Add a mapping” to open a popup window.
 +
# In the popup window, enter the string “ows_taxid” and klick on “Find” to retrieve the list of crawled properties of type “Managed Metadata”
 +
# Select in the list of crawled properties the property that corresponds to your site column, e.g. "ows_taxid_products". Click on “OK” to save the setting and close the popup window.
 +
 
 +
<big>NOTE: If you cannot find a crawled property that corresponds to your site column, then you should start a full crawl which INCLUDES your library (see next step)!</big>
  
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
=== 4. Connect the Easy Tagging App with the IDAS service (2/2) ===
 
# In the connection dialog: select the "IDAS Evaluation Service" tab
 
# Click on "I accept the terms of use" tick box
 
# If the service status says "IDAS evaluation service is available", then click on the "connect" button. If the status says something different: click on "refresh status".
 
# The dialog closes and you can setup a predictor.
 
 
<!-- --></div>
 
<!-- --></div>
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Connect2.png|center|middle]]</div>
+
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Refinablestring_mapping.PNG|center|middle]]</div>
 
</div>
 
</div>
 
<div class="row funktionen-row">
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
=== 5. Launch the "Learning Wizard" in your library ===
+
=== 3. Important: re-index your document library ===
# After you have created a connection to the IDAS Evaluation service you can launch the "Learning Wizard" which will guide you through the process to learn from your examples and to tag your documents.
 
# Return to your library and open the "Library Settings" ribbon.
 
# Click on the "Learning Wizard" icon which launches the wizard in a dialog.
 
# Go through the three steps:
 
## Click on "Click to start the wizard"
 
## Step 1: select the language that is used in most of the documents of the library. If you documents in other languages: just select english.
 
## Step 1: in the field "Learn to predict tags from this managed metadata column": select the "document types" column that you created in the earlier steps
 
## Step 2: your library is analysed for sufficient examples. If you don't have enough examples: please add further examples before you continue with the wizard.
 
## Step 3: select the column which will receive the document type tags: just keep the selection.
 
## Step 3: click on the link "Start Learning and Tagging".
 
# The learning and tagging processes are running in the background.
 
# You can inspect the progress in the library ribbon "Predictors + Taggers"
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Start_learning_wizard.JPG|center|middle]]</div>
 
</div>
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
=== 6. Important: re-index your document library ===
 
 
# Opent the settings of your library
 
# Opent the settings of your library
 
# Click on "re-index document library"
 
# Click on "re-index document library"
 
# Click on "Ok" to close the form
 
# Click on "Ok" to close the form
<big>Now all your documents in the library have been provided with tags, and the search index is being updated in the background. You can now carry on and setup the search page that uses these tags.</big>
+
<big>Now all your documents in the library have been provided with tags, and the search index is being updated in the background.</big>
 
<!-- --></div>
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Library_settings.JPG|center|middle]]</div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Library_settings.JPG|center|middle]]</div>
Line 171: Line 156:
 
</p>
 
</p>
 
</div>
 
</div>
   <h2>Step 3: Create a search center that includes the search refiners</h2>
+
   <h2>Step 3: Verify the search results</h2>
 
<div>
 
<div>
 
<p>
 
<p>
 
<div class="row funktionen-row">
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
=== 1. Create an Enterprise Search Center ===
+
=== 1. verify the search results ===
# Open the contents of your Sharepoint site and click on "subsites".
 
# Create a new sub site by clicking on the "New" action link.
 
# A form opens where you specify the new sub site.
 
# In the template field: select "Enterprise/Basic Search Center".
 
# Click on "Create" to create the new seach center.
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Create_search_center.JPG|center|middle]]</div>
 
</div>
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
=== 2. Adjust the search schema: map the crawled property to a refinable managed property ===
 
# Open the Sharepoint admin center for your tenant.
 
# Click on the "Search" section
 
# In the Search section: click on "Search schema"
 
# In the "Search schema" settings: activate the tab "Crawled properties"
 
# Search for all crawled properties that start with the string "ows_"
 
# Select the crawled property that relates to your "document type"-managed property, e.g. "ows_Document_x0020_type"
 
# Create a mapping to the managed property "RefinableString01" (or 00, or any other number).
 
# Save the settings.
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Searchschema.JPG|center|middle]]</div>
 
</div>
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
=== 3. Add the new refiner to the search page ===
 
# Open the search center
 
# Open the search results page (named "results.aspx")
 
# Click on "Edit page"
 
# In the page edit view: open the property settings of the Refinement webpart
 
# Click on the button "Choose refiners"
 
# In the "refinement configuration" dialog: add the refinable managed propery
 
(from the search schema step), like: RefinableString00 (or 01 etc).
 
# You should see sample values that contain you document types.
 
# Click on "Ok" to add RefinableString00 to the refiner
 
# Save the modified properties of the webpart by clicking on "OK"
 
# Store the modified page.
 
<!-- --></div>
 
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Search_refiner.JPG|center|middle]]</div>
 
</div>
 
<div class="row funktionen-row">
 
<!-- --><div class="col-md-6 col-md-border funktionen" style="text-align: justify; ">
 
=== 4. verify the search results ===
 
 
# Open the search center
 
# Open the search center
 
# Enter "*" as search string and submit the query
 
# Enter "*" as search string and submit the query
Line 226: Line 169:
 
<big>If the new refiner does not contain values then you have to wait until the Sharepoint online search engine has completed re-indexing your library. If still no values show up then you have to go back to your library settings and click on "re-index library".</big>
 
<big>If the new refiner does not contain values then you have to wait until the Sharepoint online search engine has completed re-indexing your library. If still no values show up then you have to go back to your library settings and click on "re-index library".</big>
 
<!-- --></div>
 
<!-- --></div>
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Searchresults.JPG|center|middle]]</div>
+
<!-- --><div class="col-md-6 col-md-border funktionen imageExpand" style="text-align: justify; >[[File:Tree_view-collapsed.png|center|middle]]</div>
 
</div>
 
</div>
 
</p></div></div>
 
</p></div></div>
 
==What's next?==
 
==What's next?==
* Add refiners for the document language like: english, french, german, spanisch
+
* Get the TermTree Refiner Webpart: [[TreeRefiner_Webpart|TreeRefiner Webpart]]
** The Easy Tagging App has a built-in predictor that detects the language that is used in documents. You can simply use that predictor in a tagger to tag all your documents with the language. Similar to the steps in section 3 you would simply create another refiner that allows users to restrict their search results to certain languages.
 
* Add refiners for important words:
 
** The Easy Tagging App is able to identify important words from documents and store them in the termstore as a new termset. If you let the tagger tag your documents with them, then your users can use them to filter their search results.
 
  
 
==Questions?==
 
==Questions?==
Line 248: Line 188:
 
|width=100%
 
|width=100%
 
|height=100%
 
|height=100%
|id=hMaEIa2DMNs}}
+
|id=jvKEF-u5r84}}
 
}}
 
}}
 
{{Linkbox
 
{{Linkbox
Line 254: Line 194:
 
|heading=related products
 
|heading=related products
 
|links=
 
|links=
* [[Easy_Tagging_for_SharePoint_online|Easy Tagging App]]
+
* [[TreeRefiner_Webpart|TreeRefiner Webpart]]
 
}}
 
}}
 
{{Linkbox
 
{{Linkbox

Latest revision as of 15:32, 6 May 2021

Main Page > Der DIQA Blog > Blog/Refine search results by term trees
Blog

Sharepoint: Improve the search experience with the TermTree Refiner (May 6, 2021)


Finding documents in Sharepoint online/Office 365 can be tedious. If you have a hierarchy of terms, like product groups, series and products, then you can easily refine search results for leaf terms. But it is impossible to refine for terms that sit higher in the term tree, e.g. product groups or series. This is because the standard refinement webpart shows leaf terms, only. This frustrates your users who cannot leverage the structural information of the term store.

You can provide your users with a better search experience if you use DIQA's TermTree Refiner. This Webpart extends the standard refiner webpart and visualises the terms in the hierarchical context of the term tree. Users can select terms in the hierarchy to drill down or drill up in the search results. This webpart makes use of managed metadata. This refinement webpart has these advantages over standard SharePoint-Search:

  • The terms contained in the search results are presented in their tree-context
  • Users can easily drill-up or drill-down in the tree to broaden or narrow the search

Better: filter documents by type, language, etc:

Tagcloud

The TermTree Refiner Webpart shows the terms in their tree-context. Users can easily drill-up or drill-down in the tree to broaden or narrow the search.

Frustrating search experience without meaningful refiners:

Tagcloud

The standard refiner hides important information like the position of a search hit in the term tree. Additionally you cannot filter for parent terms, like product groups.

What do you need to do to achieve this improved search experience? Just follow the steps below!

See the TreeRefiner Webpart in this video:

What's next?

Questions?

EU Dataprotection Terms

* These fields are required.

Watch it here:
related products
more articles