PowerShell: Collection was modified; enumeration operation may not execute

 

While the following example is for PowerShell, the same applies to C# code.

Collections that are being modified do not like to be processed with FOR EACH.
"Collection was modified; enumeration operation may not execute"

For example:

$web = Get-SPWeb someurl...

ForEach($list in $w.Lists)
{
  if (!$list.Hidden)
  {
    # do something to change the list
    # like add a new view...

    $list.Views.Add(.......)
    # error! 
    # "Collection was modified; enumeration operation may not execute..."
  }
 }
$web.Dispose()

 

The Fix

Change the FOR EACH to a FOR.

change:

  foreach($l in $web.Lists)
     {

to:

   for ($i=0; $i -lt $web.lists.count; $i++)
     {
        $l = $w.lists[$i]

 

And if you are deleting items in the collection…

Walk the collection in reverse order!

   for ($i=$web.lists.count-1; $i –gt 0; $i--)
     {
        $l = $web.lists[$i]

.

Setting a SharePoint Group's Description Using PowerShell

 

The SharePoint Group object has a property named "Description", but it appears to be ignored. When you edit a group in the browser there is no "Description" field, but there is one named "About Me" that is a rich text field. Turns out there's a few secrets you need to know to set this seemly simple little property.

Secret #1: Many of the group's properties are stored in another object, the SPWeb.SiteUserInfoList object.

Secret #2: The internal name of "About Me" is "Notes".

The PowerShell:

#create the group as usual and then retrieve it...
$web = $site.RootWeb;
$group = $web.SiteGroups["Test Group with HTML desc"];

#find the group's SiteUserInfoList info...
$groupInfo = $web.SiteUserInfoList.GetItemById($group.id);

#update the text...
$groupInfo["Notes"] = "<div>This is <strong>bold</strong> and this is <em>italics</em></div>";

#and save it...
$groupInfo.Update();

 

For a C# example see the answer in this discussion: http://stackoverflow.com/questions/968819/change-description-of-a-sharepoint-group

 

.

PowerShell: Finding all SharePoint Lists with Lookup Columns

 

Did you ever want to find all of the columns of a certain type? All Lookups, all Calculated or all Managed Metadata columns? All you have to do is look at the Fields object "TypeDisplayName" property. While the example script below is looking for Lookup columns, you could modify it to look for any kind of column.

 

Find all Lookup Columns

This will find all Lookup columns in the entire farm!

Notes:

• Many of the out of the box lists are "custom" lists and will have fields that look like user added columns. Exclude those with the $TTNExcludeLists variable.
• Field that are Hidden or FromBaseType are not typically user created and are excluded in this example.

$TTNExcludeLists = "Solution Gallery", 
                "Workflow Tasks", 
                "Master Page Gallery"

Get-SPSite -Limit All | Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Where { -Not ($TTNExcludeLists -Contains $_.Title) } | 
  Select -ExpandProperty Fields | 
  Where { $_.TypeDisplayName -eq "Lookup" -and 
          $_.Hidden -eq $false -and 
          $_.FromBaseType -eq $false } | 
  Select {$_.ParentList.ParentWebUrl}, 
         {$_.ParentList}, 
         Title

 

TypeDisplayName

If you would like to see the list of all of the column types used in your site or farm you can run a script like this:

Get-SPWeb "http://yourServer/sites/YourSite" |
  Select -ExpandProperty Lists | 
  Select -ExpandProperty Fields |
  Select TypeDisplayName –Unique |

  Sort

It may take a long time to run (WARNING!) but this will list all of the columns in the farm:

Get-SPSite -Limit All | Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Select -ExpandProperty Fields |
  Select TypeDisplayName -Unique|
  Sort TypeDisplayName

 

Here's the list I got from my test farm:

TypeDisplayName
---------------
All Day Event
Attachments
Audience Targeting
Calculated
Channel Alias
Check Double Booking
Choice
Computed
Content Type Id
Content Type ID
Counter
Cross Project Link
Date and Time
Event Type
File
Free/Busy
Guid
Hold Status
Hyperlink or Picture
Integer
Lookup
Managed Metadata
Moderation Status
Multiple lines of text
Number
Number of Likes
Number of Ratings
Out of Policy
Outcome choice
Page Separator
Permission Level
Person or Group
Publishing HTML
Publishing Image
Publishing Schedule End Date
Publishing Schedule Start Date
Rating (0-5)
Recurrence
Related Items
Resources
Single line of text
Summary Links
ThreadIndex
User Agent Substrings
Variations
Yes/No

 

.

Word Stemming in SharePoint 2013 Search

 

Word Stemming

Word stemming is when a search engine resolves a word to its root form and then returns variations of the word.

For example, search for one of these and return the rest:

• run, ran runner, running, runs
• fish, fishes, fisher, fishing

 

SharePoint 2013 Word Stemming

SharePoint 2013 only does "stemming" on singulars and plurals. So all we get for "run" is "runs" and for "fish" is "fishes". The exact stemming depends on the language.

SharePoint 2007 and 2010 did support an expanded form of word stemming, but you had to enable it in the search results web parts.

 

So what do we do?

Search using "OR".

   fish OR fishing OR fisher

   run OR ran OR running OR runs

Remember that the Boolean operators like "OR" must be typed in upper case and that mixtures of AND and OR usually will need parentheses.

   race AND (run OR ran OR running OR runs)

 

oh well…

 

.

SharePoint 2013 Items Removed with Search Result Removal Return from the Dead!

 

In SharePoint 2010, Removed was Removed!

In SharePoint 2010 there's a search feature called "Search Result Removal" that made an item immediately disappear from user's search results. All you had to do was enter the URL to the problem item and click Remove Now. SharePoint then remove the item from the index and wrote a Crawl Rule to make sure the content was ignored by the crawler in all future searches.

The 2010 request to remove:

 

The auto-created Crawl Rule:

SharePoint 2010 had a interesting bug here. Uppercase letters in the URL would cause the the removal request to be ignored!

 

In SharePoint 2013 and Office 365, Removed is Just a Temporary Thing! Maybe only a few seconds!

We don't have Crawl Rules in SharePoint 2013 and Office 365. When you request the removal of an item or a URL from the search index, it just deletes it from the index. But… on the next crawl it adds it back! That next crawl could be an hour away, or only seconds, depending on your crawl schedules and the luck of your timing.

Nowhere can I find any documentation that says this!

The screen for removal requests is similar to what we had with 2010, but with no mention about Crawl Rules.

 

Removal Straight from the Crawl Logs

Both versions let you browse the crawl logs, find a problem item, click the dropdown on the item and click Remove the Item From The Index. In the Crawl Logs you can remove one item at a time while the Removal pages will let you exclude and entire path.

In 2013 we are just told that the item will be removed. In 2010 we are also told that a crawl rule will be created.

 

References:

Remove URLs from search results (SharePoint Server 2010
https://technet.microsoft.com/en-us/library/ff621095(v=office.14).aspx

Delete items from the search index or from search results in SharePoint Server 2013
https://technet.microsoft.com/en-us/library/jj219587.aspx

SharePoint 2013: Missing Search Box

 

You have a search box over your list or library and I don't! Why?

SharePoint 2013 has added a search box to most lists and libraries that is scoped to the current list. When the user performs the search they will stay in the list’s page. They will not be redirected to a search results page. They can continue to sort or filter the results using the column heading dropdowns.

   

Pretty cool! But it's missing in my library!

 

Four Reasons You Might Not See It

#1 - Turns out that this search box is only available in one View Style. If you have not played with View Styles then take a look at this article http://techtrainingnotes.blogspot.com/2010/05/sharepoint-list-view-styles.html That's an older article, but still applies to SharePoint 2013.

If you pick any style other than Default, you lose the search box. Bummer…

   

#2 -  It's only available in the Enterprise Edition of SharePoint 2013. I'm guessing that it may only be available in Office 365 / SharePoint Online in the "E" subscriptions.

#3 -  It can be disabled! Edit the page, edit the web part, expand the Miscellaneous section: It is enabled by default in list/library pages. It is disabled by default when you add a list/library web part to a page.
    

#4 - It is only displayed when server side rendering is disabled.
     .

.

SharePoint 2013 Search File Extensions

 

Just a simple list of file extensions that have some kind of default support in the SharePoint 2013 Search Service.

FH = SharePoint 2013 has a File Handler and can index the content of this file.
FT = Is included in the default on premises File Type List.
365 = Is included in the Office 365 / SharePoint Online File Type List. (as of 3/5/15)

Extension	FH	FT	365	Type	MimeType
.ascx		x	x	ASP.NET Control
.asm		x	x	ASP Web Service
.asp		x	x	ASP web page
.aspx		x	x	ASP.NET web page
.csv		x	x	Comma separated values
.cxx		x	x
.def		x	x
.doc	x	x	x	Microsoft Word	application/msword
.docm	x	x	x	Microsoft Word	application/vnd.ms-word.document.macroEnabled.12
.docx	x	x	x	Microsoft Word	application/vnd.openxmlformats-officedocument.wor...
.dot	x	x	x	Microsoft Word	application/msword
.dotm	x			Microsoft Word	application/vnd.ms-word.template.macroEnabledTemp...
.dotx	x	x	x	Microsoft Word	application/vnd.openxmlformats-officedocument.wor...
.eml	x	x	x	Email Message	message/rfc822
.gif	x	x	x	Graphics Interchange Format	image/gif
.html	x	x	x	Web Page	text/html
.infopathml	x			Microsoft InfoPath	text/xml
.jpg	x			JPEG	image/jpeg
.jsp		x	x	Java server page
.mht	x	x	x	Web Archive	multipart/related
.mhtml		x	x	Web page archive format
.msg	x	x	x	Outlook Item	application/vnd.ms-outlook
.nws		x	x
.obd	x			Microsoft Office Binder	application/vnd.ms-binder
.obt	x			Microsoft Office Binder	application/vnd.ms-binder
.odc		x	x	Office Data Connection File
.odp	x	x	x	OpenDocument Presentation	application/vnd.oasis.opendocument.presentation
.ods	x	x	x	OpenDocument Spreadsheet	application/vnd.oasis.opendocument.spreadsheet
.odt	x	x	x	OpenDocument Text	application/vnd.oasis.opendocument.text
.one	x		x	OneNote	application/msonenote
.pdf	x	x	x	PDF	application/pdf
.pot	x			Microsoft PowerPoint	application/vnd.ms-powerpoint
.potm	x			Microsoft PowerPoint	application/vnd.ms-powerpoint.template.macroEnabl...
.potx	x			Microsoft PowerPoint	application/vnd.openxmlformats-officedocument.pre...
.ppam	x			Microsoft PowerPoint	application/vnd.ms-powerpoint.addin.macroEnabled.12
.pps	x			Microsoft PowerPoint	application/vnd.ms-powerpoint
.ppsm	x			Microsoft PowerPoint	application/vnd.ms-powerpoint.slideshow.macroEnab...
.ppsx	x			Microsoft PowerPoint	application/vnd.openxmlformats-officedocument.pre...
.ppt	x	x	x	Microsoft PowerPoint	application/vnd.ms-powerpoint
.pptm	x	x	x	Microsoft PowerPoint	application/vnd.ms-powerpoint.presentation.macroE...
.pptx	x	x	x	Microsoft PowerPoint	application/vnd.openxmlformats-officedocument.pre...
.pub	x	x	x	Microsoft Publisher	application/x-mspublisher
.rtf	x			Rich Text	text/rtf
.txt	x	x	x	Text	text/plain
.vcf	x			vCard	text/x-vcard
.vcs	x			vCalendar	text/x-vCalendar
.vdw	x	x	x	Visio	application/vnd.visio
.vdx	x	x	x	Visio	application/vnd.visio
.vsd	x	x	x	Visio	application/vnd.visio
.vsdm	x	x	x	Visio	application/vnd.ms-visio.drawing.macroEnabled
.vsdx	x	x	x	Visio	application/vnd.ms-visio.drawing
.vss	x	x	x	Visio	application/vnd.visio
.vssm	x	x	x	Visio	application/vnd.ms-visio.stencil.macroEnabled
.vssx	x	x	x	Visio	application/vnd.ms-visio.stencil
.vst	x	x	x	Visio	application/vnd.visio
.vstm	x	x	x	Visio	application/vnd.ms-visio.template.macroEnabled
.vstx	x	x	x	Visio	application/vnd.ms-visio.template
.vsx	x	x	x	Visio	application/vnd.visio
.vtx	x	x	x	Visio	application/vnd.visio
.xlb	x		x	Microsoft Excel	application/vnd.ms-excel
.xlc	x		x	Microsoft Excel	application/vnd.ms-excel
.xls	x	x	x	Microsoft Excel	application/vnd.ms-excel
.xlsb	x	x	x	Microsoft Excel	application/vnd.ms-excel.sheet.binary.macroEnable...
.xlsm	x	x	x	Microsoft Excel	application/vnd.ms-excel.sheet.macroEnabled.12
.xlsx	x	x	x	Microsoft Excel	application/vnd.openxmlformats-officedocument.spr...
.xlt	x		x	Microsoft Excel	application/vnd.ms-excel
.xml	x	x	x	XML Document	text/xml
.xps	x			Microsoft XML Paper Specification	application/vnd.ms-xpsdocument
.zip	x	x	x	ZIP Archive	application/zip

 

Notes:

• SharePoint indexes all content in a site, even if not in the File Types list. You can search for any of the data in the columns of the library to find the file. For example you can search for "jpg" and find everything that has jpg in the filename, description, or other column. You can't search for the file using "filetype:jpg" though.
• If the file extension is added to the File Types list then SharePoint treats the item as a "file" and you can then search using "filetype:jpg".
• If the file extension is in the File Types list AND there is a Format Handler or iFilter then the content of the file will also be indexed.
• You can dump the list of installed Format Handlers using this PowerShell script:
• $ssa = Get-SPEnterpriseSearchServiceApplication
  Get-SPEnterpriseSearchFileFormat -SearchApplication $ssa | ft –AutoSize
• The above list of files has nothing to do with Blocked File Types (files users cannot upload)
  https://technet.microsoft.com/en-us/library/cc262496.aspx
• Default crawled file name extensions and parsed file types in SharePoint Server 2013
  https://technet.microsoft.com/en-us/library/jj219530.aspx
• HOW TO: Implement a custom iFilter in SharePoint 2013
  http://blogs.technet.com/b/sharepointdevelopersupport/archive/2013/05/13/how-to-implement-a-custom-ifilter-in-sharepoint-2013.aspx

SharePoint 2013 Search Spelling Suggestions Confusion?

 

Over the last three months I have been working on a Search Administration class that has a focus on improving the end user search experience. If you have attended one of my governance classes or consulting sessions then you have heard me preach on the need for a "Search Administrator". If you are interested in this area of administration then check out my new class: Microsoft SharePoint 2013 Search Administration. The next class is 3/11/15!

SharePoint 2013 search is not always well documented. As an example, read the description of Search Spelling Suggestions here: https://technet.microsoft.com/en-us/library/jj591607.aspx While very factual, it just does not tell me enough…

So off to the blogs…

First problem is that it seems a lot of people mix up Search Query Suggestions and Search Spelling Suggestions. They are not the same! Query Suggestions are offered while the user types in the search box. Spelling Suggestions are offered only in the search results pages, after the user executes the search.

A Spelling Suggestion:

(“corydoras” is kind of catfish. )

 

Second problem? It seems due to the lack of extensive documentation that a lot of people are guessing.

• One blog says that "only the first level of Terms is taken into account".
• One even said that spelling suggestions could be used to map between terms. (Must have been thinking about Query Suggestions.)
• Another says that while searching for a term found in the second level of a term set, and no items are found, that the parent level's term will be offered as the suggestion.
• Some call the Query Spelling Inclusions term set the "Static" dictionary.

All of those are wrong.

 

Here's what my trial and error has found:

• There are four sources for spelling suggestions:
  • The default static spelling dictionaries (static out of the box dictionaries).
  • The default dynamic spelling dictionary (dynamically generated from your content). This is also called a "content-aligned spelling dictionary".
  • The Query Spelling Inclusions term set (manually entered).
  • The Query Spelling Exclusions term set (manually entered).
• Spelling suggestions are based on the closest matches in the default spelling dictionaries and the Query Spelling Inclusions list. Only one suggestion will be displayed. It appears that a phonic / sound alike match is being used, so the properly spelled and misspelled words must be similar in length and pattern.
• You can’t edit the default static or dynamic spelling dictionaries.
• Static dictionaries
  • The static dictionary is a canned list of words. (Something like the Word dictionaries.)
  • The user can change the language dropdown and see words from another language.
• Dynamically created dictionaries
  • The dynamic dictionary is created by search as content is indexed. I.e. it's based on words commonly found in your content.
  • The user can change the language dropdown, but this does not change offered words (at least from my testing).
  • Support for dynamically created dictionaries depends on the language. See https://technet.microsoft.com/en-us/library/jj219499.aspx
  • For dynamic spelling correction to work, you should have at least several thousand medium-sized documents. The default settings require that a word occur in at least 1000 documents to be included in the dictionary.
  • The dynamic dictionary is updated once a night and can be a long running process. (The default timeout is 6 hours!)
• Query Spelling Inclusions / Exclusions
  • You must have a configured Managed Metadata Service.
  • The terms must be added to the auto-created Query Spelling Inclusions term set.
  • You manually enter a list of words into an include or exclude list of terms.
  • You can only include single words, not phrases in the term sets.
  • While sub-terms can be entered, they are treated no differently than top level terms. (Some documentation says they are ignored.) Some documentation says terms at the same sublevel can be used as “Do you mean”, but testing does not show this.
  • It may take ten minutes or more for updates to the term sets to show up in search results. (Search Custom Dictionaries Update timer job)

As there are a number of articles about adding words to the Query Spelling Inclusions term set using Managed Metadata Services, code and PowerShell, I will not cover that here.

 

What about the options available in the PowerShell commands?

There are two PowerShell cmdlets that can be used to manage the on premises SharePoint's spelling options. Again the TechNet article is factual, but not too clear on the Static vs. Dynamic topic.

    Get-SPEnterpriseSearchQuerySpellingCorrection
    Set-SPEnterpriseSearchQuerySpellingCorrection

Note: These cmdlets are currently only for on premises SharePoint. It looks like Office 365 is set to use the Dynamic dictionary.

You can see the spelling suggestion options using:

    $ssa = Get-SPEnterpriseSearchServiceApplication
    Get-SPEnterpriseSearchQuerySpellingCorrection -SearchApplication $ssa

   

SpellingDictionary = Static / Dynamic

Many of the blogs state that you can choose either Static or Dynamic as the SpellingDictionary value, and that by selecting one of these you would exclude your manually entered Query Spelling Inclusions . The confusion seems to be around the definitions of the words Static and Dynamic. My testing shows that:

  Dynamic = use the list of words found in your content, plus the Query Spelling Inclusions term set

  Static = use the static built-in / out of the box dictionary, plus the Query Spelling Inclusions term set

As an example, I do not have "SharePoint" or "SharePint" in my Query Spelling Inclusions term set, but I do have "corydoras". When I do a search for "SharePint" I get the following only when SpellingDictionary is set to Static: (this word is in the canned dictionary)

   

When searching for a term in the Query Spelling Inclusions term set like "corydoras" I get results regardless of if SpellingDictionary is set to Static or Dynamic.

   

Why don't I get any help when SpellingDictionary is set to Dynamic and I search for "SharePint"?

   

Take a look at the TermFrequencyThreshold property. Using the defaults SharePoint would need to find at least 1000 documents that contain the word "SharePoint". My testing sample set of documents is not quite that big. If I change it from 1000 to 20 and run the "Spelling dictionary update" timer job then I can start to get useful results from "Did you mean?" for "SharePint". (i.e. "SharePoint" was in at least 20 of my sample documents.)

   

 

 

Note: Using PowerShell to change from Static to Dynamic, or the reverse, immediately impacts user searches.

 

Additional Links of Interest

This article, Search in SharePoint 2013 knowledge articles for Systems Center Operations Manager, hints at more details about the search spelling suggestions feature.

The spelling related timer jobs are listed in this page: Timer job reference (SharePoint 2013)

.

SharePoint: Create a View for Only Last Month

 

You have to learn some real tricks to get views filtered just the way you want, especially with dates. Dates are fun in that the filter can only compare a date column to an exact date or [Today]. Here's two examples that should get you a start on a wide range of date related filters.

 

Just Last Month Please

What we need: Only the items where some date is during the last full month. Not 30 days ago, but between the first and last day of last month.

 

Create two new columns

These two columns calculate the range of dates to show this item. So to figure this out, stop thinking about this month and last month and think about when do you want an item with a certain date displayed. If the item has a date of 2/14/2015 then we want it displayed from 3/1/2015 to 3/31/2015.

For this example the date column we are testing against is named TheDate. (Creative huh!)

Column 1:  (to calculate the first day of the month after TheDate)

  Name: StartDisplayDateForLastMonth
  Type: Calculated
  Formula: =DATE(YEAR(TheDate),MONTH(TheDate)+1,"1")
  The data type returned from this formula is:  (o) Date & Time
  Date and Time Format: (o) Date Only

Column 2:  (to calculate the last day of the month after TheDate)

  Name: EndDisplayDateForLastMonth
  Type: Calculated
  Formula: =DATE(YEAR(TheDate),MONTH(TheDate)+2,0)
  The data type returned from this formula is:  (o) Date & Time
  Date and Time Format: (o) Date Only

Here's a screen shot of the first column. (click to enlarge)

   

 

Create the View

Create a view with your usual choice of columns, but not the two we just added. You may want to add these columns for test though.

   

Display and test the view!

 

How does it work?

=DATE(YEAR(TheDate),MONTH(TheDate)+1,"1")

DATE needs three values, a year number, a month number and a day number. The first two are easy, if you know a weird fact. YEAR() returns the year number (2015) and MONTH() returns the month number (3). Obviously MONTH()+1 is next month… but what about when it's December plus 1? That's the weird fact. Month 13 bumps up the year value!  So DATE(2014,13,5) is 1/5/2015. The 1 at the end of the formula is for day "1" of the month. Don't believe me? Fire up Excel and play around with the DATE function.

=DATE(YEAR(TheDate),MONTH(TheDate)+2,0)

This one depends on another weird fact… day zero is the last day of the previous month!  DATE(2015,3,0) is actually 2/28/2015. And of course, DATE(2016,3,0) is 2/29/2016! So what we are doing here is calculating a date 2 months minus one day in the future.

Bonus! What's the date of the 250th day of the year?  =DATE(2015,1,250) or 9/7/2015.  (I guess that would also be January 250th, 2015!)

 

What about next month?

Just a slightly different formula.

For the start date: =DATE(YEAR(TheDate),MONTH(TheDate)-1,"1")

For the end date: =DATE(YEAR(TheDate),MONTH(TheDate),0)

 

What an easy way to create these?

Fire up Excel! (And see this article: http://techtrainingnotes.blogspot.com/2010/08/sharepoint-creating-calculated-column.html)

Have fun!

 

.

SharePoint 2013 Query Suggestion Files

 

Over the last three months I have been working on a Search Administration class that has a focus on improving the end user search experience. If you have attended one of my governance classes or consulting sessions then you have heard me preach on the need for a "Search Administrator". If you are interested in this area of administration then check out my new class: Microsoft SharePoint 2013 Search Administration.

I have been experimenting with SharePoint 2013 search Query Suggestions and found that there are some interesting aspects of the import files that don't seem to be documented anywhere.

• One phrase per line.
• The “Always Suggest” file does not need quotes, but they are acceptable.
• The “Never Suggest” file must have quotes around multiword phrases.
• Files can have blank lines.
• Files with duplicates cannot be uploaded.
• The order of the phrases is not important. The phrases will be sorted in the dropdown. For that matter, if you import the file and then export it, the items are in alphabetical order.
• It's a good idea to keep your own backup of the files. Do not depend on the export as the order has been changed. You may also want to keep a "documented" version of the file with some comments. (Remember to remove the comments before uploading!)

I tried most of the ways to add comments, but none of them worked. You may want to keep two copies of your suggestion files, one with comments and one without.

Example of an Always Suggest file:

Example of a Never Suggest file:

In this example we are hiding common misspellings learned by search's automatic query suggestions. We are also dealing with the fact we don’t have a "Cincinnati region". It’s really the South Western Ohio Region.

 

Tip: If you are working with an on premises SharePoint 2013 then you could add a thesaurus entry to expand the user's search for "Cincinnati Region" to include "South Western Ohio Region". 

.