SharePoint 2013 Search Spelling Suggestions Confusion?

 

Over the last three months I have been working on a Search Administration class that has a focus on improving the end user search experience. If you have attended one of my governance classes or consulting sessions then you have heard me preach on the need for a "Search Administrator". If you are interested in this area of administration then check out my new class: Microsoft SharePoint 2013 Search Administration. The next class is 3/11/15!

SharePoint 2013 search is not always well documented. As an example, read the description of Search Spelling Suggestions here: https://technet.microsoft.com/en-us/library/jj591607.aspx While very factual, it just does not tell me enough…

So off to the blogs…

First problem is that it seems a lot of people mix up Search Query Suggestions and Search Spelling Suggestions. They are not the same! Query Suggestions are offered while the user types in the search box. Spelling Suggestions are offered only in the search results pages, after the user executes the search.

A Spelling Suggestion:

(“corydoras” is kind of catfish. )

 

Second problem? It seems due to the lack of extensive documentation that a lot of people are guessing.

• One blog says that "only the first level of Terms is taken into account".
• One even said that spelling suggestions could be used to map between terms. (Must have been thinking about Query Suggestions.)
• Another says that while searching for a term found in the second level of a term set, and no items are found, that the parent level's term will be offered as the suggestion.
• Some call the Query Spelling Inclusions term set the "Static" dictionary.

All of those are wrong.

 

Here's what my trial and error has found:

• There are four sources for spelling suggestions:
  • The default static spelling dictionaries (static out of the box dictionaries).
  • The default dynamic spelling dictionary (dynamically generated from your content). This is also called a "content-aligned spelling dictionary".
  • The Query Spelling Inclusions term set (manually entered).
  • The Query Spelling Exclusions term set (manually entered).
• Spelling suggestions are based on the closest matches in the default spelling dictionaries and the Query Spelling Inclusions list. Only one suggestion will be displayed. It appears that a phonic / sound alike match is being used, so the properly spelled and misspelled words must be similar in length and pattern.
• You can’t edit the default static or dynamic spelling dictionaries.
• Static dictionaries
  • The static dictionary is a canned list of words. (Something like the Word dictionaries.)
  • The user can change the language dropdown and see words from another language.
• Dynamically created dictionaries
  • The dynamic dictionary is created by search as content is indexed. I.e. it's based on words commonly found in your content.
  • The user can change the language dropdown, but this does not change offered words (at least from my testing).
  • Support for dynamically created dictionaries depends on the language. See https://technet.microsoft.com/en-us/library/jj219499.aspx
  • For dynamic spelling correction to work, you should have at least several thousand medium-sized documents. The default settings require that a word occur in at least 1000 documents to be included in the dictionary.
  • The dynamic dictionary is updated once a night and can be a long running process. (The default timeout is 6 hours!)
• Query Spelling Inclusions / Exclusions
  • You must have a configured Managed Metadata Service.
  • The terms must be added to the auto-created Query Spelling Inclusions term set.
  • You manually enter a list of words into an include or exclude list of terms.
  • You can only include single words, not phrases in the term sets.
  • While sub-terms can be entered, they are treated no differently than top level terms. (Some documentation says they are ignored.) Some documentation says terms at the same sublevel can be used as “Do you mean”, but testing does not show this.
  • It may take ten minutes or more for updates to the term sets to show up in search results. (Search Custom Dictionaries Update timer job)

As there are a number of articles about adding words to the Query Spelling Inclusions term set using Managed Metadata Services, code and PowerShell, I will not cover that here.

 

What about the options available in the PowerShell commands?

There are two PowerShell cmdlets that can be used to manage the on premises SharePoint's spelling options. Again the TechNet article is factual, but not too clear on the Static vs. Dynamic topic.

    Get-SPEnterpriseSearchQuerySpellingCorrection
    Set-SPEnterpriseSearchQuerySpellingCorrection

Note: These cmdlets are currently only for on premises SharePoint. It looks like Office 365 is set to use the Dynamic dictionary.

You can see the spelling suggestion options using:

    $ssa = Get-SPEnterpriseSearchServiceApplication
    Get-SPEnterpriseSearchQuerySpellingCorrection -SearchApplication $ssa

   

SpellingDictionary = Static / Dynamic

Many of the blogs state that you can choose either Static or Dynamic as the SpellingDictionary value, and that by selecting one of these you would exclude your manually entered Query Spelling Inclusions . The confusion seems to be around the definitions of the words Static and Dynamic. My testing shows that:

  Dynamic = use the list of words found in your content, plus the Query Spelling Inclusions term set

  Static = use the static built-in / out of the box dictionary, plus the Query Spelling Inclusions term set

As an example, I do not have "SharePoint" or "SharePint" in my Query Spelling Inclusions term set, but I do have "corydoras". When I do a search for "SharePint" I get the following only when SpellingDictionary is set to Static: (this word is in the canned dictionary)

   

When searching for a term in the Query Spelling Inclusions term set like "corydoras" I get results regardless of if SpellingDictionary is set to Static or Dynamic.

   

Why don't I get any help when SpellingDictionary is set to Dynamic and I search for "SharePint"?

   

Take a look at the TermFrequencyThreshold property. Using the defaults SharePoint would need to find at least 1000 documents that contain the word "SharePoint". My testing sample set of documents is not quite that big. If I change it from 1000 to 20 and run the "Spelling dictionary update" timer job then I can start to get useful results from "Did you mean?" for "SharePint". (i.e. "SharePoint" was in at least 20 of my sample documents.)

   

 

 

Note: Using PowerShell to change from Static to Dynamic, or the reverse, immediately impacts user searches.

 

Additional Links of Interest

This article, Search in SharePoint 2013 knowledge articles for Systems Center Operations Manager, hints at more details about the search spelling suggestions feature.

The spelling related timer jobs are listed in this page: Timer job reference (SharePoint 2013)

.