3/10/2015

SharePoint 2013 Items Removed with Search Result Removal Return from the Dead!

 

In SharePoint 2010, Removed was Removed!

In SharePoint 2010 there's a search feature called "Search Result Removal" that made an item immediately disappear from user's search results. All you had to do was enter the URL to the problem item and click Remove Now. SharePoint then remove the item from the index and wrote a Crawl Rule to make sure the content was ignored by the crawler in all future searches.

The 2010 request to remove:

image 

The auto-created Crawl Rule:

image

SharePoint 2010 had a interesting bug here. Uppercase letters in the URL would cause the the removal request to be ignored!


 

In SharePoint 2013 and Office 365, Removed is Just a Temporary Thing! Maybe only a few seconds!

We don't have Crawl Rules in SharePoint 2013 and Office 365. When you request the removal of an item or a URL from the search index, it just deletes it from the index. But… on the next crawl it adds it back! That next crawl could be an hour away, or only seconds, depending on your crawl schedules and the luck of your timing.

Nowhere can I find any documentation that says this!

The screen for removal requests is similar to what we had with 2010, but with no mention about Crawl Rules.

image

 

Removal Straight from the Crawl Logs

Both versions let you browse the crawl logs, find a problem item, click the dropdown on the item and click Remove the Item From The Index. In the Crawl Logs you can remove one item at a time while the Removal pages will let you exclude and entire path.

image

In 2013 we are just told that the item will be removed. In 2010 we are also told that a crawl rule will be created.

image

 

References:

Remove URLs from search results (SharePoint Server 2010
https://technet.microsoft.com/en-us/library/ff621095(v=office.14).aspx

Delete items from the search index or from search results in SharePoint Server 2013
https://technet.microsoft.com/en-us/library/jj219587.aspx

3/06/2015

SharePoint 2013: Missing Search Box

 

You have a search box over your list or library and I don't! Why?

SharePoint 2013 has added a search box to most lists and libraries that is scoped to the current list. When the user performs the search they will stay in the list’s page. They will not be redirected to a search results page. They can continue to sort or filter the results using the column heading dropdowns.

    image

Pretty cool! But it's missing in my library!

 

Four Reasons You Might Not See It

#1 - Turns out that this search box is only available in one View Style. If you have not played with View Styles then take a look at this article http://techtrainingnotes.blogspot.com/2010/05/sharepoint-list-view-styles.html That's an older article, but still applies to SharePoint 2013.

If you pick any style other than Default, you lose the search box. Bummer…

    image

#2 -  It's only available in the Enterprise Edition of SharePoint 2013. I'm guessing that it may only be available in Office 365 / SharePoint Online in the "E" subscriptions.

#3 -  It can be disabled! Edit the page, edit the web part, expand the Miscellaneous section: It is enabled by default in list/library pages. It is disabled by default when you add a list/library web part to a page.
     image

#4 - It is only displayed when server side rendering is disabled.
     image.

.

3/05/2015

SharePoint 2013 Search File Extensions

 

Just a simple list of file extensions that have some kind of default support in the SharePoint 2013 Search Service.

FH = SharePoint 2013 has a File Handler and can index the content of this file.
FT = Is included in the default on premises File Type List.
365 = Is included in the Office 365 / SharePoint Online File Type List. (as of 3/5/15)

Extension

FH

FT

365

Type

MimeType

.ascx

 

x

x

ASP.NET Control

 

.asm

 

x

x

ASP Web Service

 

.asp

 

x

x

ASP web page

 

.aspx

 

x

x

ASP.NET web page

 

.csv

 

x

x

Comma separated values

 

.cxx

 

x

x

   

.def

 

x

x

   

.doc

x

x

x

Microsoft Word

application/msword

.docm

x

x

x

Microsoft Word

application/vnd.ms-word.document.macroEnabled.12

.docx

x

x

x

Microsoft Word

application/vnd.openxmlformats-officedocument.wor...

.dot

x

x

x

Microsoft Word

application/msword

.dotm

x

   

Microsoft Word

application/vnd.ms-word.template.macroEnabledTemp...

.dotx

x

x

x

Microsoft Word

application/vnd.openxmlformats-officedocument.wor...

.eml

x

x

x

Email Message

message/rfc822

.gif

x

x

x

Graphics Interchange Format

image/gif

.html

x

x

x

Web Page

text/html

.infopathml

x

   

Microsoft InfoPath

text/xml

.jpg

x

   

JPEG

image/jpeg

.jsp

 

x

x

Java server page

 

.mht

x

x

x

Web Archive

multipart/related

.mhtml

 

x

x

Web page archive format

 

.msg

x

x

x

Outlook Item

application/vnd.ms-outlook

.nws

 

x

x

   

.obd

x

   

Microsoft Office Binder

application/vnd.ms-binder

.obt

x

   

Microsoft Office Binder

application/vnd.ms-binder

.odc

 

x

x

Office Data Connection File

 

.odp

x

x

x

OpenDocument Presentation

application/vnd.oasis.opendocument.presentation

.ods

x

x

x

OpenDocument Spreadsheet

application/vnd.oasis.opendocument.spreadsheet

.odt

x

x

x

OpenDocument Text

application/vnd.oasis.opendocument.text

.one

x

 

x

OneNote

application/msonenote

.pdf

x

x

x

PDF

application/pdf

.pot

x

   

Microsoft PowerPoint

application/vnd.ms-powerpoint

.potm

x

   

Microsoft PowerPoint

application/vnd.ms-powerpoint.template.macroEnabl...

.potx

x

   

Microsoft PowerPoint

application/vnd.openxmlformats-officedocument.pre...

.ppam

x

   

Microsoft PowerPoint

application/vnd.ms-powerpoint.addin.macroEnabled.12

.pps

x

   

Microsoft PowerPoint

application/vnd.ms-powerpoint

.ppsm

x

   

Microsoft PowerPoint

application/vnd.ms-powerpoint.slideshow.macroEnab...

.ppsx

x

   

Microsoft PowerPoint

application/vnd.openxmlformats-officedocument.pre...

.ppt

x

x

x

Microsoft PowerPoint

application/vnd.ms-powerpoint

.pptm

x

x

x

Microsoft PowerPoint

application/vnd.ms-powerpoint.presentation.macroE...

.pptx

x

x

x

Microsoft PowerPoint

application/vnd.openxmlformats-officedocument.pre...

.pub

x

x

x

Microsoft Publisher

application/x-mspublisher

.rtf

x

   

Rich Text

text/rtf

.txt

x

x

x

Text

text/plain

.vcf

x

   

vCard

text/x-vcard

.vcs

x

   

vCalendar

text/x-vCalendar

.vdw

x

x

x

Visio

application/vnd.visio

.vdx

x

x

x

Visio

application/vnd.visio

.vsd

x

x

x

Visio

application/vnd.visio

.vsdm

x

x

x

Visio

application/vnd.ms-visio.drawing.macroEnabled

.vsdx

x

x

x

Visio

application/vnd.ms-visio.drawing

.vss

x

x

x

Visio

application/vnd.visio

.vssm

x

x

x

Visio

application/vnd.ms-visio.stencil.macroEnabled

.vssx

x

x

x

Visio

application/vnd.ms-visio.stencil

.vst

x

x

x

Visio

application/vnd.visio

.vstm

x

x

x

Visio

application/vnd.ms-visio.template.macroEnabled

.vstx

x

x

x

Visio

application/vnd.ms-visio.template

.vsx

x

x

x

Visio

application/vnd.visio

.vtx

x

x

x

Visio

application/vnd.visio

.xlb

x

 

x

Microsoft Excel

application/vnd.ms-excel

.xlc

x

 

x

Microsoft Excel

application/vnd.ms-excel

.xls

x

x

x

Microsoft Excel

application/vnd.ms-excel

.xlsb

x

x

x

Microsoft Excel

application/vnd.ms-excel.sheet.binary.macroEnable...

.xlsm

x

x

x

Microsoft Excel

application/vnd.ms-excel.sheet.macroEnabled.12

.xlsx

x

x

x

Microsoft Excel

application/vnd.openxmlformats-officedocument.spr...

.xlt

x

 

x

Microsoft Excel

application/vnd.ms-excel

.xml

x

x

x

XML Document

text/xml

.xps

x

   

Microsoft XML Paper Specification

application/vnd.ms-xpsdocument

.zip

x

x

x

ZIP Archive

application/zip

 

Notes:

  • SharePoint indexes all content in a site, even if not in the File Types list. You can search for any of the data in the columns of the library to find the file. For example you can search for "jpg" and find everything that has jpg in the filename, description, or other column. You can't search for the file using "filetype:jpg" though.
  • If the file extension is added to the File Types list then SharePoint treats the item as a "file" and you can then search using "filetype:jpg".
  • If the file extension is in the File Types list AND there is a Format Handler or iFilter then the content of the file will also be indexed.
  • You can dump the list of installed Format Handlers using this PowerShell script:
    • $ssa = Get-SPEnterpriseSearchServiceApplication
      Get-SPEnterpriseSearchFileFormat -SearchApplication $ssa | ft –AutoSize
  • The above list of files has nothing to do with Blocked File Types (files users cannot upload)
    https://technet.microsoft.com/en-us/library/cc262496.aspx
  • Default crawled file name extensions and parsed file types in SharePoint Server 2013
    https://technet.microsoft.com/en-us/library/jj219530.aspx
  • HOW TO: Implement a custom iFilter in SharePoint 2013
    http://blogs.technet.com/b/sharepointdevelopersupport/archive/2013/05/13/how-to-implement-a-custom-ifilter-in-sharepoint-2013.aspx

3/03/2015

SharePoint 2013 Search Spelling Suggestions Confusion?

 

Over the last three months I have been working on a Search Administration class that has a focus on improving the end user search experience. If you have attended one of my governance classes or consulting sessions then you have heard me preach on the need for a "Search Administrator". If you are interested in this area of administration then check out my new class: Microsoft SharePoint 2013 Search Administration. The next class is 3/11/15!

SharePoint 2013 search is not always well documented. As an example, read the description of Search Spelling Suggestions here: https://technet.microsoft.com/en-us/library/jj591607.aspx While very factual, it just does not tell me enough…

So off to the blogs…

First problem is that it seems a lot of people mix up Search Query Suggestions and Search Spelling Suggestions. They are not the same! Query Suggestions are offered while the user types in the search box. Spelling Suggestions are offered only in the search results pages, after the user executes the search.

A Spelling Suggestion:

image

(“corydoras” is kind of catfish. Smile)

 

Second problem? It seems due to the lack of extensive documentation that a lot of people are guessing.

  • One blog says that "only the first level of Terms is taken into account".
  • One even said that spelling suggestions could be used to map between terms. (Must have been thinking about Query Suggestions.)
  • Another says that while searching for a term found in the second level of a term set, and no items are found, that the parent level's term will be offered as the suggestion.
  • Some call the Query Spelling Inclusions term set the "Static" dictionary.

All of those are wrong.

 

Here's what my trial and error has found:

  • There are four sources for spelling suggestions:
    • The default static spelling dictionaries (static out of the box dictionaries).
    • The default dynamic spelling dictionary (dynamically generated from your content). This is also called a "content-aligned spelling dictionary".
    • The Query Spelling Inclusions term set (manually entered).
    • The Query Spelling Exclusions term set (manually entered).
  • Spelling suggestions are based on the closest matches in the default spelling dictionaries and the Query Spelling Inclusions list. Only one suggestion will be displayed. It appears that a phonic / sound alike match is being used, so the properly spelled and misspelled words must be similar in length and pattern.
  • You can’t edit the default static or dynamic spelling dictionaries.
  • Static dictionaries
    • The static dictionary is a canned list of words. (Something like the Word dictionaries.)
    • The user can change the language dropdown and see words from another language.
  • Dynamically created dictionaries
    • The dynamic dictionary is created by search as content is indexed. I.e. it's based on words commonly found in your content.
    • The user can change the language dropdown, but this does not change offered words (at least from my testing).
    • Support for dynamically created dictionaries depends on the language. See https://technet.microsoft.com/en-us/library/jj219499.aspx
    • For dynamic spelling correction to work, you should have at least several thousand medium-sized documents. The default settings require that a word occur in at least 1000 documents to be included in the dictionary.
    • The dynamic dictionary is updated once a night and can be a long running process. (The default timeout is 6 hours!)
  • Query Spelling Inclusions / Exclusions
    • You must have a configured Managed Metadata Service.
    • The terms must be added to the auto-created Query Spelling Inclusions term set.
    • You manually enter a list of words into an include or exclude list of terms.
    • You can only include single words, not phrases in the term sets.
    • While sub-terms can be entered, they are treated no differently than top level terms. (Some documentation says they are ignored.) Some documentation says terms at the same sublevel can be used as “Do you mean”, but testing does not show this.
    • It may take ten minutes or more for updates to the term sets to show up in search results. (Search Custom Dictionaries Update timer job)

As there are a number of articles about adding words to the Query Spelling Inclusions term set using Managed Metadata Services, code and PowerShell, I will not cover that here.


 

What about the options available in the PowerShell commands?

There are two PowerShell cmdlets that can be used to manage the on premises SharePoint's spelling options. Again the TechNet article is factual, but not too clear on the Static vs. Dynamic topic.

    Get-SPEnterpriseSearchQuerySpellingCorrection
    Set-SPEnterpriseSearchQuerySpellingCorrection

Note: These cmdlets are currently only for on premises SharePoint. It looks like Office 365 is set to use the Dynamic dictionary.

You can see the spelling suggestion options using:

    $ssa = Get-SPEnterpriseSearchServiceApplication
    Get-SPEnterpriseSearchQuerySpellingCorrection -SearchApplication $ssa

    image

SpellingDictionary = Static / Dynamic

Many of the blogs state that you can choose either Static or Dynamic as the SpellingDictionary value, and that by selecting one of these you would exclude your manually entered Query Spelling Inclusions . The confusion seems to be around the definitions of the words Static and Dynamic. My testing shows that:

  Dynamic = use the list of words found in your content, plus the Query Spelling Inclusions term set

  Static = use the static built-in / out of the box dictionary, plus the Query Spelling Inclusions term set

As an example, I do not have "SharePoint" or "SharePint" in my Query Spelling Inclusions term set, but I do have "corydoras". When I do a search for "SharePint" I get the following only when SpellingDictionary is set to Static: (this word is in the canned dictionary)

    image

When searching for a term in the Query Spelling Inclusions term set like "corydoras" I get results regardless of if SpellingDictionary is set to Static or Dynamic.

    image

Why don't I get any help when SpellingDictionary is set to Dynamic and I search for "SharePint"?

    image

Take a look at the TermFrequencyThreshold property. Using the defaults SharePoint would need to find at least 1000 documents that contain the word "SharePoint". My testing sample set of documents is not quite that big. If I change it from 1000 to 20 and run the "Spelling dictionary update" timer job then I can start to get useful results from "Did you mean?" for "SharePint". (i.e. "SharePoint" was in at least 20 of my sample documents.)

    image

 

 

Note: Using PowerShell to change from Static to Dynamic, or the reverse, immediately impacts user searches.

 

Additional Links of Interest

This article, Search in SharePoint 2013 knowledge articles for Systems Center Operations Manager, hints at more details about the search spelling suggestions feature.

The spelling related timer jobs are listed in this page: Timer job reference (SharePoint 2013)

.

Note to spammers!

Spammers, don't waste your time... all posts are moderated. If your comment includes unrelated links, is advertising, or just pure spam, it will never be seen.