PowerShell Get-Date vs. [DateTime]

 

The Cincinnati PowerShell User Group (@CincyPowershell) opens each meeting with a review of a cmdlet. Tonight is was Get-Date's turn. Many of the examples were done with both Get-Date and [datetime], with one example showing that only [datetime] supports the IsLeapYear() method. This got me to thinking… (sometimes a painful thing…) and the light bulb lit up over my head. Get-Date returns an instance of the System.DateTime class, i.e. an object, while [datetime] is a reference to the class and provides easy access to the class's static methods.

 

System.DateTime

To be precise, [datetime] is really [System.DateTime], but System is the default name space and PowerShell is not case sensitive (most of the time). And to be precise, Get-Date with no options is the same as[System.DateTime]::Now.

What's with the two colons? They are PowerShell's way to call a static method on a class. So going back to the IsLeapYear() question, it is a static method of the DateTime class. To access it we write [System.DateTime]::IsLeapYear(2016).

 

Get-Date

Get-Date is a cmdlet with many options, but the default option is to return the current date and time. I.e. [System.DateTime]::Now.

The class that an object was created from is also called it's type. So what's the "type" of a date?

 

Instances / Objects

If Get-Date returns an instance, how could I do that with [DateTime]? Two ways, use one of the static methods like .Now or .Today, or create a new object using the New-Object cmdlet.

Using the static methods:

$d = [System.DateTime]::Now    (or just [DateTime].Now)

$d = [System.DateTime]::Today  (or just [DateTime].Today)

Using New-Object:

$d = New-Object System.DateTime    (or just DateTime)

The unique aspect of using New-Object with DateTime is that you get back an object with default values for year, month, day, etc. and not today's date. The actual result is "Monday, January 01, 0001 12:00:00 AM".

If you want to create a particular date or time, use one of the class's constructors. (See the DateTime Structure link at the end of the article for details.)

There's a nice write up about classes, instances, statics here from Don Jones.

 

And just for fun… and only in PowerShell 5.0

Who needs [System.DateTime]?  Just call the static methods off of any DateTime object! The following all produce the same result, keeping in mind that Get-Date adds the additional overhead of retrieving the current date, and then ignoring it.

(Get-Date)::IsLeapYear(2016)

$d = Get-Date
$d::IsLeapYear(2016)

$d = [System.DateTime]::Now
$d::IsLeapYear(2016)

 

What about PowerShell 3.0?

The examples for 5.0 do not work with 3.0. We have to ask for the "class", i.e. it's Type.

(Get-Date).GetType()::IsLeapYear(2016)

$d = Get-Date
$d.GetType()::IsLeapYear(2016)

$d = [System.DateTime]::Now
$d.GetType()::IsLeapYear(2016)

 

What about PowerShell 4.0?

Beats me! (Don't have a copy in front of me right now.)

 

Discover the Static Methods and Properties of an Object

Get-Member has a –Static parameter to display the static methods and properties of an object:

Get-Date | Get-Member -Static

 

More Fun With Static Methods

Most .Net classes are not exposed in PowerShell as cmdlets.

For example, you might need a new GUID…

[System.Guid]::NewGuid()   will return a new GUID object.

[System.Guid]::NewGuid().toString()   will return a new GUID as a string.

How about some Math?

Need the value of PI?

[System.,Math]::PI  (or just [Math]::PI)  It's about 3.14159265358979 according to PowerShell and .Net.

How about the square root of 2?

[Math]::Sqrt(2)  (Let's see… I think it's about 1.4142135623731.)

And of course, the Sine of 45 degrees:

[Math]::sin(0.785398163397448)  (in Radians of course) Oh, about 0.707106781186547.

And doing a little converting from degrees to Radians:

[Math]::sin( [Math]::pi/180*45 )

For more of what the Math class can do, see the Math link at the end of the article. Or, wait for it, use PowerShell:

[Math] | Get-Member  -Static

 

Bottom line… if the C# guys can do it, then so can you!

 

So now you know… 2016 is a leap year!  And at least a dozen different ways!

 

(and a quick 5 minute article took over an hour…)

 

References:

Using Static Classes and Methods
https://technet.microsoft.com/en-us/library/dd347632.aspx

How to Create an Object in PowerShell
http://blogs.msdn.com/b/powershell/archive/2009/03/11/how-to-create-an-object-in-powershell.aspx

DateTime Structure (All of the constructors, methods and properties of DateTime)
https://msdn.microsoft.com/en-us/library/system.datetime(v=vs.110).aspx

A nice write up on class, object, instance and static by Don Jones:
http://powershell.org/wp/2014/09/08/powershell-v5-class-support/

System.Math Class
https://msdn.microsoft.com/en-us/library/system.math(v=vs.110).aspx

 

.

Sorting Hashtables and SharePoint Properties Lists with PowerShell

 

Sometimes the obvious just does not work… I was doing some work with SharePoint publishing sites, trying to find some of the data in the SPWeb AllProperties property. AllProperties looks like a collection of Names and Values. Actually it is Hashtable object.

My first attempt looked like this:

    $psite.AllWebs[0].AllProperties | sort -Property name

The output was random, not sorted as expected. After too long messing with trial and error, I did a web search and found a TechNet "Windows PowerShell Tip of the Week" article with the answer. Hashtable objects and sort just don't work together. We need to get an enumerator for the hash table and work with that…

   $psite.AllWebs[0].AllProperties.GetEnumerator() | sort -Property name

That's the output I was looking for!

If you pipe both of the above to Get-Member you will see that the first just returns a Hashtable object that includes a collection of Keys and a collection of Values. The second one, using GetEnumerator, returns a single DictionaryEntry object (one at a time through the pipeline) with a Key and a Value property. Now we have something to sort on!

.

Windows Explorer Search – Learn something new every day…

 

I was searching through around 400 folders with thousands of files for all of the files that contained "_files" in their names. Should be easy… open Windows Explorer and type in the search box *_files* and wait… but I got back every file as if I had typed "*".

Is the underline a wildcard?

Not according to the documentation I can find, and it does not otherwise behave like a wildcard. For example, searching for "f_le" or "*f_le*" did not find anything.

Is the underline ignored?

Seems like it. Except… if you include a file type in the search then the underline is treated as an ordinary character or if you use the FileName property.

Two solutions:

Add a filetype wild card:   *_file*.*   

Search using a property name:   filename:*_file*    (but don't add ".*" if you are also looking for folders!)

 

What about good ole DOS?

Works exactly as expected.   DIR *_files* /s   .

 

What about PowerShell

Also works exactly as expected.   dir -Filter *_files* –Recurse

I should have started with PowerShell anyway as the next step was to rename these files!

 

What creates file names that include "_file"?

Internet Explorer when you use File, Save As, Web Page Complete.

 

.

The specified module 'Microsoft.Online.SharePoint.PowerShell' was not loaded

 

When trying to load the SharePoint Online PowerShell module into the ISE or an existing shell you may get the error below. Simple problem with a non-obvious error message… The fix is easy… after installing the SharePoint Online Management Shell, run PowerShell or the ISE as Administrator.

PS C:\> Import-Module Microsoft.Online.SharePoint.PowerShell -DisableNameChecking
Import-Module : The specified module 'Microsoft.Online.SharePoint.PowerShell' was not loaded because no valid 
module file was found in any module directory.
At line:1 char:1
+ Import-Module Microsoft.Online.SharePoint.PowerShell -DisableNameChecking
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (Microsoft.Online.SharePoint.PowerShell:String) [Import-Mo 
   dule], FileNotFoundException
    + FullyQualifiedErrorId : Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand

SharePoint / PowerShell List Item Update Note

 

Sometimes it's the forests and sometimes it's the trees…

 

I have gotten so used to using shortcuts like chaining of properties in PowerShell that I forget some of my .Net fundamentals.

The Title never gets updated in this little script. What's wrong?

$web = Get-SPWeb "http://maxsp2013wfe/sites/training"
$list = $web.Lists["Announcements"]
$list.items[3]["Title"]="Test Title"
$list.items[3].Update()

Line 3 creates a new SPListItem object in RAM and sets its Title property.
Line 4 creates a new (but different) SPListItem object and calls .Update() on it.

Duh!

 

This works:

$web = Get-SPWeb "http://maxsp2013wfe/sites/training"
$list = $web.Lists["Announcements"]
$item = $list.items[3]
$item["Title"]="Test Title"
$item.Update()

Create the SPListItem object once, change a property and then save it.

And I teach this stuff! 

.

Upload a File to SharePoint Using PowerShell

 

A while back I posted an article on downloading files from SharePoint using PowerShell (see here) that generated a few questions about uploading files. So… here's a few uploading scripts. Don't stop here though, there are many other examples on the web. Google/Bing is your friend!

 

An Upload with Metadata Script

This script not only uploads files, but lets you add metadata.

# Set a few variables
$file = "C:\test\myTestFile.txt"
$TTNwebUrl = "http://server/sites/yoursite"
$library = "Team Documents"
$overWriteExisting = $True #or add new version if versioning enabled

# do a little SharePoint setup
$web = Get-SPWeb $TTNwebUrl
$files = $web.GetFolder($library).Files
$fileNameForLibrary = $file.Substring($file.LastIndexOf("\")+1) 

# read the file
$data = Get-ChildItem $file

# add any needed metadata
$metadata = @{ "Project ID" = "A-200"; "Region" = "North" }

# or if no metadata needed: $metadata = @{}

# do the upload (the following is one line)
$newfile = $files.Add($library + "/" + $fileNameForLibrary, $data.OpenRead(), $metadata, $overWriteExisting)

 

And if you are a "one liner" PowerShell scripter:

(Get-SPWeb "http://server/sites/yoursite").GetFolder("Team Documents").Files.Add("Team Documents/myTestFile.txt", (Get-ChildItem "C:\test\myTestFile.txt").OpenRead(), $True)

And with metadata:
(Get-SPWeb "http://server/sites/yoursite").GetFolder("Team Documents").Files.Add("Team Documents/myTestFile.txt", (Get-ChildItem "C:\test\myTestFile.txt").OpenRead(), @{ "Project ID" = "A-200"; "Region" = "North" }, $True)

 

Upload an Attachment with a New List Item

I generally recommend linking from a list item to a file in a library, but if you want to add attachments to a list item, here's a sample script:

# Set a few variables
  $TTNweb = Get-SPWeb "http://yourServer/sites/yourSite";
  $list = $TTNweb.lists["yourListName"];
  $filePath = "C:\SampleDocs\yourSampleFile.JPG";

# Create a new list item:
  $item = $list.items.Add(); 
  $item["Title"] = "Test Title";
  # set any addition metadata 
  $item.Update();

# Upload the file:
  $bytes = [System.IO.File]::ReadAllBytes($filePath);
  $item.Attachments.Add([System.IO.Path]::GetFileName($filePath), $bytes);
  $item.Update();

 

 

Looking for a PowerShell course for SharePoint administrators and auditors?

SharePoint 2010 and 2013 Auditing and Site Content Administration using PowerShell

https://www.microsoft.com/en-us/learning/course.aspx?cid=55095A

Or you can attend my class in Cincinnati (locally or remotely)!
http://www.maxtrain.com/Classes/ClassInfo.aspx?Id=119394

.

Cincinnati PowerShell User Group Meeting

 

Register here: (free!) http://www.meetup.com/TechLife-Cincinnati/events/224705451/

Cincinnati PowerShell Users group – August Meeting

When: August 27th
Where: Max Technical Training
Mason, OH

PS C:\> Get-PSUG | where {$_.City -eq Cincinnati}

The Cincinnati PowerShell user group is back and under new management!  Last meeting was great with Ed Wilson as our presenter, I hope you all got as fired up about scripting as we did!

Come join us as we dive into….

Version Control?  That's for Developers, not Admins!!

Does your version control look like this?!?!?

We'll cover different ways to approach source control with PowerShell, why we as System Administrators need it, and why the idea of "infrastructure-as-code" is getting so much traction these days.

 

Sponsor!

SAPIEN Technologies will be sponsoring the Cincinnati PowerShell User Group meeting and will be supplying the pizza!

.

Cincinnati PowerShell User Group Meeting this week!

Info. and RSVP here: http://meetu.ps/2JkRsx


The PowerShell User Group is back! (But the future is up to you!)


     PS C:\> Get-Speaker | Plan-Meeting | ShowUp-AndLearn!

Sponsor:  SAPIEN Technologies
is supplying the food!

Ed Wilson "The Scripting Guy" from Microsoft will be here to help us (re)kickoff the PowerShell User Group. If you would like to see future meetings, place your vote by showing up for this meeting!
Wednesday, June 3rd 6:30 PM at MAX Technical Training. (Food and networking starts at 6:00!)



Speaker: Ed Wilson "The Scripting Guy"
Title: Garbage in, Garbage out: Data grooming with Windows PowerShell
Everyone has heard the old adage, "garbage in, garbage out" when talking about databases, or other online data storage / retrieval systems. But do you know that Windows PowerShell can help you with your problem? In this session, Microsoft Scripting Guy Ed Wilson talks about using Windows PowerShell to perform data grooming. He shows how cleaning up names, street addresses, cities, states, and even zip codes by using basic string manipulation techniques. By focusing directly on the data transformation itself, he extracts principles that can be used regards of the database, or other data storage system. After focusing on the individual components of the process, he puts the whole thing into a single script for transforming the sample data. This session is heavy with live demonstration.

Bio
Ed Wilson is the Microsoft Scripting Guy, and writes the daily Hey Scripting Guy blog. He is the co-founder of PowerShell Saturday, and the Charlotte PowerShell User Group. He is a frequent speaker at conferences, user groups and other places where groups of computer people may be found. He has written books about every version of Windows PowerShell, as well as other books related to automation and operating systems.  His most recent book is Windows PowerShell Best Practices.


.

Run SharePoint 2013 Search Reports from PowerShell–Part 2!

 

First… see the original article here: http://techtrainingnotes.blogspot.com/2015/04/run-sharepoint-2013-search-reports-from.html

What if you want reports for each site collection? All we need is another function, a "Get-SPSIte –Limit ALL" and some code to generate unique file names and we can now create hundreds to thousands of Excel files!!! (Please make sure you never fill your server's drives with this!)

 

# This is the path to write the reports to:
$path = "c:\SearchReports\"


function Get-SPSearchReports ($farmurl, $searchreport, $path)
{
  # Report names and IDs
  $Number_of_Queries          = "21be5dff-c853-4259-ab01-ee8b2f6590c7"
  $Top_Queries_by_Day         = "56928342-6e3b-4382-a14d-3f5f4f8b6979"
  $Top_Queries_by_Month       = "a0a26a8c-bf99-48f4-a679-c283de58a0c4"
  $Abandoned_Queries_by_Day   = "e628cb24-27f3-4331-a683-669b5d9b37f0"
  $Abandoned_Queries_by_Month = "fbc9e2c1-49c9-44e7-8b6d-80d21c23f612"
  $No_Result_Queries_by_Day   = "5e97860f-0595-4a07-b6c2-222e784dc3a8"
  $No_Result_Queries_by_Month = "318556b1-cabc-4fad-bbd5-c1bf8ed97ab1"
  $Query_Rule_Usage_by_Day    = "22a16ae2-ded9-499d-934a-d2ddc00d406a"
  $Query_Rule_Usage_by_Month  = "f1d70093-6fa0-4701-909d-c0ed502e3df8"

  # create a unique filename for each site and report
  $sitename = $farmurl.Replace("http://","").Replace("https://","")
  $sitename = $sitename.substring(0,$sitename.IndexOf("/_layouts"))
  $sitename = $sitename.Replace("/","_").Replace(":","_").Replace(".","_")

  $filename = $path + $sitename + " " + 
              (Get-Variable $searchreport).Name + " " + 
              (Get-Date -Format "yyyy-mm-dd") + ".xlsx"
  #Write-Host "Creating $filename"

  $reportid = (Get-Variable $searchreport).Value

  $TTNcontent = "&__EVENTTARGET=__Page&__EVENTARGUMENT=ReportId%3D" + $reportid

  # setup the WebRequest
  $webRequest = [System.Net.WebRequest]::Create($farmurl)
  $webRequest.UseDefaultCredentials = $true
  $webRequest.Accept = "image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, */*"
  $webRequest.ContentType = "application/x-www-form-urlencoded"
  $webRequest.Method = "POST"

  $encodedContent = [System.Text.Encoding]::UTF8.GetBytes($TTNcontent)
    $webRequest.ContentLength = $encodedContent.length
    $requestStream = $webRequest.GetRequestStream()
    $requestStream.Write($encodedContent, 0, $encodedContent.length)
    $requestStream.Close()

  # get the data
  [System.Net.WebResponse] $resp = $webRequest.GetResponse();
    $rs = $resp.GetResponseStream();
    #[System.IO.StreamReader] $sr = New-Object System.IO.StreamReader -argumentList $rs;
    #[byte[]]$results = $sr.ReadToEnd();
    [System.IO.BinaryReader] $sr = New-Object System.IO.BinaryReader -argumentList $rs;
    [byte[]]$results = $sr.ReadBytes(10000000);

  # write the file
  Set-Content $filename $results -enc byte
}


function Get-SPSearchReportsForSite ($url, $path)
{

  Get-SPSearchReports $url "Number_of_Queries" $path
  Get-SPSearchReports $url "Top_Queries_by_Day" $path
  Get-SPSearchReports $url "Top_Queries_by_Month" $path
  Get-SPSearchReports $url "Abandoned_Queries_by_Day" $path
  Get-SPSearchReports $url "Abandoned_Queries_by_Month" $path
  Get-SPSearchReports $url "No_Result_Queries_by_Day" $path
  Get-SPSearchReports $url "No_Result_Queries_by_Month" $path
  Get-SPSearchReports $url "Query_Rule_Usage_by_Day" $path
  Get-SPSearchReports $url "Query_Rule_Usage_by_Month" $path

}


Get-SPSite -Limit All | 
  foreach   {
    $url = $_.Url + "/_layouts/15/Reporting.aspx?Category=AnalyticsSiteCollection";
    #$path = (you might do a custom path for each set of reports)

    Write-Host "Getting files for $url"
    Get-SPSearchReportsForSite $url $path
  }

Run SharePoint 2013 Search Reports from PowerShell

Update: SharePoint 2016 version here: http://techtrainingnotes.blogspot.com/2018/02/run-sharepoint-2013-and-2016-search.html



Update! Need these reports for every site collection in the farm? See Part 2: http://techtrainingnotes.blogspot.com/2015/04/run-sharepoint-2013-search-reports-from_21.html

In my Search Administration class I stress that admins should dump the search reports on a regular basis as the data is only kept in detail for 14 days and in summary form for 35 months. But who wants to both run these reports at least once every 14 days, even they can remember to do so. So, PowerShell to the rescue… Schedule this script to run each weekend and your work is done.
The following script works for on premise SharePoint 2013. To work with Office 365 you will have to figure out how to include your credentials. The example included here works on premises by using "UseDefaultCredentials = $true".
After lots of hacking, detective work (see below) and just plain trial and error, here's the script:

# This is the URL from the Central Admin Search Service Usage Reports page:
$url = "http://yourCentralAdminURL/_layouts/15/reporting.aspx?Category=AnalyticsSearch&appid=ed39c68b%2D7276%2D46f7%2Db94a%2D4ae7125cf567"  

# This is the path to write the reports to (must exist):
$path = "c:\SearchReports\"




function Get-SPSearchReports ($farmurl, $searchreport, $path)
{
  # TechTrainingNotes.blogspot.com

  # Report names and IDs
  $Number_of_Queries          = "21be5dff-c853-4259-ab01-ee8b2f6590c7"
  $Top_Queries_by_Day         = "56928342-6e3b-4382-a14d-3f5f4f8b6979"
  $Top_Queries_by_Month       = "a0a26a8c-bf99-48f4-a679-c283de58a0c4"
  $Abandoned_Queries_by_Day   = "e628cb24-27f3-4331-a683-669b5d9b37f0"
  $Abandoned_Queries_by_Month = "fbc9e2c1-49c9-44e7-8b6d-80d21c23f612"
  $No_Result_Queries_by_Day   = "5e97860f-0595-4a07-b6c2-222e784dc3a8"
  $No_Result_Queries_by_Month = "318556b1-cabc-4fad-bbd5-c1bf8ed97ab1"
  $Query_Rule_Usage_by_Day    = "22a16ae2-ded9-499d-934a-d2ddc00d406a"
  $Query_Rule_Usage_by_Month  = "f1d70093-6fa0-4701-909d-c0ed502e3df8"


  $filename = $path + (Get-Variable $searchreport).Name + " " + (Get-Date -Format "yyyy-mm-dd")  + ".xlsx"
  $reportid = (Get-Variable $searchreport).Value

  $TTNcontent = "&__EVENTTARGET=__Page&__EVENTARGUMENT=ReportId%3D" + $reportid

  # setup the WebRequest
  $webRequest = [System.Net.WebRequest]::Create($farmurl)
  $webRequest.UseDefaultCredentials = $true
  $webRequest.Accept = "image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, */*"
  $webRequest.ContentType = "application/x-www-form-urlencoded"
  $webRequest.Method = "POST"

  $encodedContent = [System.Text.Encoding]::UTF8.GetBytes($TTNcontent)
    $webRequest.ContentLength = $encodedContent.length
    $requestStream = $webRequest.GetRequestStream()
    $requestStream.Write($encodedContent, 0, $encodedContent.length)
    $requestStream.Close()

  # get the data
  [System.Net.WebResponse] $resp = $webRequest.GetResponse();
    $rs = $resp.GetResponseStream();
    #[System.IO.StreamReader] $sr = New-Object System.IO.StreamReader -argumentList $rs;
    #[byte[]]$results = $sr.ReadToEnd();
    [System.IO.BinaryReader] $sr = New-Object System.IO.BinaryReader -argumentList $rs;
    [byte[]]$results = $sr.ReadBytes(10000000);

  # write the file
  Set-Content $filename $results -enc byte
}



Get-SPSearchReports $url "Number_of_Queries" $path
Get-SPSearchReports $url "Top_Queries_by_Day" $path
Get-SPSearchReports $url "Top_Queries_by_Month" $path
Get-SPSearchReports $url "Abandoned_Queries_by_Day" $path
Get-SPSearchReports $url "Abandoned_Queries_by_Month" $path
Get-SPSearchReports $url "No_Result_Queries_by_Day" $path
Get-SPSearchReports $url "No_Result_Queries_by_Month" $path
Get-SPSearchReports $url "Query_Rule_Usage_by_Day" $path
Get-SPSearchReports $url "Query_Rule_Usage_by_Month" $path
 

The Detective Work…

I could not find anything documented on how the reports are called or details on things like the report GUIDs. So here's how I got there:

• Go the search reports page in Central Admin and press F12 to open the Internet Explorer F12 Developer Tools then:
  • Click the Network tab and click the play button to start recording.
  • Click one of the report links.
  • Double-click the link generated for the report in the F12 pane to open up the details.
  • Make note of the URL (It's the same as the report page!)
  • Note the Accept, and Content-Type Request Headers.
  • Click the Request Body tab.
  • Stare at 3000 characters in that string until your head really hurts, or until you recognize most of what is there is the normal page postback stuff like VIEWSTATE. So we need to find what's unique in the string. (It's the Report IDs.)
  • Click on each of the nine reports and copy out the report IDs.
  • With a lot of trial and error figure out what the minimum string needed is to generate the reports. (It's ""&__EVENTTARGET=__Page&__EVENTARGUMENT=ReportId" plus the report id.)
  • Find out how to do an HTTP POST using PowerShell. (Steal most of it from here: http://www.codeproject.com/Articles/846061/PowerShell-Http-Get-Post.)
  • Find some other needed .Net code and convert the C# to PowerShell.
  • Fill in some gaps with PowerShell putty …….

.

PowerShell: Collection was modified; enumeration operation may not execute

 

While the following example is for PowerShell, the same applies to C# code.

Collections that are being modified do not like to be processed with FOR EACH.
"Collection was modified; enumeration operation may not execute"

For example:

$web = Get-SPWeb someurl...

ForEach($list in $w.Lists)
{
  if (!$list.Hidden)
  {
    # do something to change the list
    # like add a new view...

    $list.Views.Add(.......)
    # error! 
    # "Collection was modified; enumeration operation may not execute..."
  }
 }
$web.Dispose()

 

The Fix

Change the FOR EACH to a FOR.

change:

  foreach($l in $web.Lists)
     {

to:

   for ($i=0; $i -lt $web.lists.count; $i++)
     {
        $l = $w.lists[$i]

 

And if you are deleting items in the collection…

Walk the collection in reverse order!

   for ($i=$web.lists.count-1; $i –gt 0; $i--)
     {
        $l = $web.lists[$i]

.

Setting a SharePoint Group's Description Using PowerShell

 

The SharePoint Group object has a property named "Description", but it appears to be ignored. When you edit a group in the browser there is no "Description" field, but there is one named "About Me" that is a rich text field. Turns out there's a few secrets you need to know to set this seemly simple little property.

Secret #1: Many of the group's properties are stored in another object, the SPWeb.SiteUserInfoList object.

Secret #2: The internal name of "About Me" is "Notes".

The PowerShell:

#create the group as usual and then retrieve it...
$web = $site.RootWeb;
$group = $web.SiteGroups["Test Group with HTML desc"];

#find the group's SiteUserInfoList info...
$groupInfo = $web.SiteUserInfoList.GetItemById($group.id);

#update the text...
$groupInfo["Notes"] = "<div>This is <strong>bold</strong> and this is <em>italics</em></div>";

#and save it...
$groupInfo.Update();

 

For a C# example see the answer in this discussion: http://stackoverflow.com/questions/968819/change-description-of-a-sharepoint-group

 

.

PowerShell: Finding all SharePoint Lists with Lookup Columns

 

Did you ever want to find all of the columns of a certain type? All Lookups, all Calculated or all Managed Metadata columns? All you have to do is look at the Fields object "TypeDisplayName" property. While the example script below is looking for Lookup columns, you could modify it to look for any kind of column.

 

Find all Lookup Columns

This will find all Lookup columns in the entire farm!

Notes:

• Many of the out of the box lists are "custom" lists and will have fields that look like user added columns. Exclude those with the $TTNExcludeLists variable.
• Field that are Hidden or FromBaseType are not typically user created and are excluded in this example.

$TTNExcludeLists = "Solution Gallery", 
                "Workflow Tasks", 
                "Master Page Gallery"

Get-SPSite -Limit All | Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Where { -Not ($TTNExcludeLists -Contains $_.Title) } | 
  Select -ExpandProperty Fields | 
  Where { $_.TypeDisplayName -eq "Lookup" -and 
          $_.Hidden -eq $false -and 
          $_.FromBaseType -eq $false } | 
  Select {$_.ParentList.ParentWebUrl}, 
         {$_.ParentList}, 
         Title

 

TypeDisplayName

If you would like to see the list of all of the column types used in your site or farm you can run a script like this:

Get-SPWeb "http://yourServer/sites/YourSite" |
  Select -ExpandProperty Lists | 
  Select -ExpandProperty Fields |
  Select TypeDisplayName –Unique |

  Sort

It may take a long time to run (WARNING!) but this will list all of the columns in the farm:

Get-SPSite -Limit All | Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Select -ExpandProperty Fields |
  Select TypeDisplayName -Unique|
  Sort TypeDisplayName

 

Here's the list I got from my test farm:

TypeDisplayName
---------------
All Day Event
Attachments
Audience Targeting
Calculated
Channel Alias
Check Double Booking
Choice
Computed
Content Type Id
Content Type ID
Counter
Cross Project Link
Date and Time
Event Type
File
Free/Busy
Guid
Hold Status
Hyperlink or Picture
Integer
Lookup
Managed Metadata
Moderation Status
Multiple lines of text
Number
Number of Likes
Number of Ratings
Out of Policy
Outcome choice
Page Separator
Permission Level
Person or Group
Publishing HTML
Publishing Image
Publishing Schedule End Date
Publishing Schedule Start Date
Rating (0-5)
Recurrence
Related Items
Resources
Single line of text
Summary Links
ThreadIndex
User Agent Substrings
Variations
Yes/No

 

.

Working with Quick Launch from PowerShell

 

The following will work with both SharePoint 2010 and 2013. It will also work with SharePoint 2007 if you "manually" create the SPWeb object.

The basic steps to add a new link to Quick Launch:

1. Get the SPWeb object for your site.
2. Get the SPWeb.Navigation.QuickLaunch object.
3. Create a new node .
4. Add the new node to an existing node (like "Lists" or "Libraries") or the root of Quick Launch.
5. Done… No need to call Update() on anything!

Note: This will work with publishing sites as long as you are not using Managed Metadata Navigation.

 

Get your web object and the QuickLaunch object:

$web = Get-SPWeb http://yourServer/sites/yourSite
$quicklaunch = $web.Navigation.QuickLaunch

You can explore your existing Quick Launch from here. Just type $quicklaunch and press enter to see your headings / top level nodes. Type $quicklaunch | where {$_.title -eq "Lists"} to see the links in one of the headings. You could even list all of the headings and their children with this one: $quicklaunch | select -ExpandProperty Children | Select {$_.Parent.Title}, {$_.Title}.

 

Create a new node object:

Each item added to Quick Launch is an SPNavigationNode object. It has three parameters: the text to display, the URL to the linked resource and $true if the link is external to SharePoint or $false if the link is internal to SharePoint. It appears that the $true/$false in the third parameter is used to see if the URL is validated as a real SharePoint URL or not. I could still add SharePoint links by leaving it as $true.

Example for a SharePoint link: (note the $false)

$navnode = New-Object Microsoft.SharePoint.Navigation.SPNavigationNode("Get help!", "/sites/helpsite", $false)

Example for Bing or link external to SharePoint:

$navnode = New-Object Microsoft.SharePoint.Navigation.SPNavigationNode("Bing", http://www.bing.com, $true)

Example for JavaScript:

$navnode = New-Object Microsoft.SharePoint.Navigation.SPNavigationNode("Say Hello!", "javascript:alert('hello')", $true)

Example for JavaScript to open a new dialog box:
(Note the "`" in front of the "$" is needed because "$" means something special in PowerShell. (a variable follows))

$navnode = New-Object Microsoft.SharePoint.Navigation.SPNavigationNode("Add a new task", "JavaScript:var options=SP.UI.`$create_DialogOptions();options.url='/sites/Publishing/Lists/Tasks/NewForm.aspx?RootFolder=&IsDlg=1';options.height = 400;void(SP.UI.ModalDialog.showModalDialog(options))", $true)

For more on JavaScript in Quick Launch see SharePoint: JavaScript in Quick Launch and Top Link Bar! and SharePoint: Opening a 2010 Dialog Box from Quick Launch.

 

Add the node to Quick Launch

To add the new node as a "heading" just add it to the Quick Launch object:

$quicklaunch.AddAsFirst($navnode)             or .AddAsLast

To add the new node as a child of a "heading" then use Where to find that heading:

$heading = $quicklaunch | where {$_.title -eq "Lists"}
$heading.Children.AddAsLast($navnode)

To add the new node after an existing node (i.e. not as First or Last) you will need to retrieve the existing node and then use the .Add method.

$existingLink = $heading.Children | where {$_.title -eq "Search the web with Bing"}

$navnode = New-Object Microsoft.SharePoint.Navigation.SPNavigationNode("Search the web with Google", "http://www.google.com", $true)

$heading.Children.Add($navnode,$existingLink)

 

Delete a link?

Sure… Just retrieve the node and then call Delete.

$heading | Select -ExpandProperty Children | where { $_.Title -eq "Tasks" } | ForEach { $_.Delete() }

If you know that there's exactly one node that matches the Where test then you can shorten it to this:

($heading | Select -ExpandProperty Children | where { $_.Title -eq "Goog
le" }).Delete()

 

Bulk updates?

Sure… with extreme caution!  Danger! Will Robinson! Danger! Do the following at your own risk. No liability assumed, batteries not included!

Here's an example to add a Help link to every site in a site collection or to all site collections in the entire farm. (You way want to filter out the My Sites and the publishing sites!)

First confirm what you are about to change!

Get-SPSite http://sharepoint/sites/training | Get-SPWeb -Limit All | Select URL, Title
   or
Get-SPSite -Limit All | Get-SPWeb -Limit All | Select URL, Title

Create the new link / node.

$navnode = New-Object Microsoft.SharePoint.Navigation.SPNavigationNode("HELP!", "http://yourServer/sites/yourHelpSite", $true)

Now add it to every site in a site collection!

Get-SPSite http://sharepoint/sites/training | Get-SPWeb -Limit All | ForEach { $_.Navigation.QuickLaunch.AddAsFirst($navnode) }

or for all site collections in the farm:

Get-SPSite -Limit All | Get-SPWeb -Limit All | ForEach { $_.Navigation.QuickLaunch.AddAsFirst($navnode) }

 

And if you want to delete all of those helpful links?

Get-SPSite http://sharepoint/sites/training | Get-SPWeb -Limit All | ForEach { ($_.Navigation.QuickLaunch | where {$_.Title -eq "Help!"}).Delete()  }

 

What about a Publishing Site?

Publishing Sites use the "Navigation" editor for links and it does not permit "URLs" that don't begin with "http". This means that you cannot add JavaScript to a Quick Launch link using that editor. For SharePoint 2010 you can use the workaround here, or for 2010 and 2013 you can use the PowerShell above. (PowerShell to the rescue!) For more on JavaScript in Quick Launch see SharePoint: JavaScript in Quick Launch and Top Link Bar! and SharePoint: Opening a 2010 Dialog Box from Quick Launch.

The PowerShell above will work as long as in your Navigation settings you have not selected "Managed Navigation" or "same as parent".

 

References:

MSDN's article showing C# examples: (This is for 2010 but applies to 2007-2013.)

https://msdn.microsoft.com/en-us/library/office/ms427791(v=office.14).aspx

PowerShell Sorting Tip for Enumerations

 

Do you ever have a PowerShell script that just won't sort right?

The problem? PowerShell makes assumptions, and not always the ones you would make. For example let's get a list of SharePoint lists and sort them by their BaseTemplate property:

    $mtg = Get-SPWeb http://maxsp2013wfe/sites/Meetings
    $mtg.Lists | Sort BaseTemplate | Select BaseTemplate, Title

Is the result you would expect?

Not what I first expected… but there's a hint hiding there though… Why is the BaseTemplate column right aligned as if was numeric?

If we take the above script and pipe it to Get-Member we find that BaseTemplate is not a string! It's an object of some kind. It's an SPListTemplateType object!

So let's see what that is… Doing a Bing on Microsoft.SharePoint.SPListTemplateType reveals that it is an enumeration, which internally is a number. If you look at the list in the MSDN article you will see both the text and numeric values of the base types.

What we need is the display text for the enumeration, and we can get that by using the ToString() method. As that would then be an expression, we need to add the annoying curly brackets and the $_. notation.

    $mtg.Lists | Sort { $_.BaseTemplate.ToString() } | Select BaseTemplate, Title

And now we get:

And that's more like it! (except for the right align stuff that going on)

Convert the column in the Select to a string and all's well!

    $mtg.Lists | Sort {$_.BaseTemplate.ToString()} | Select {$_.BaseTemplate.ToString()}, Title

 

So what were the PowerShell assumptions?

1. To display the text value of the enumeration, and align it as a number.
2. To sort on the numeric value of the enumeration.

(And you know what "assume" does, right?) 

 

.

SharePoint Auditing and Site Content Administration using PowerShell

 

If you have attended one of my SharePoint Saturday PowerShell presentations or read my PowerShell series at SharePoint Pro magazine, then you know a bit about what's possible with PowerShell. This Friday. January, 30th, I will be delivering one of the more fun classes I do: "SharePoint 2010 and 2013 Auditing and Site Content Administration using PowerShell". If you are a SharePoint administrator and need to "find and inventory stuff" in SharePoint then you need this class! Whatever is hiding in SharePoint is visible to the PowerShell user, if you know where to look.

Class details are here: http://www.maxtrain.com/Classes/ClassInfo.aspx?Id=119394
You can join us at MAX Technical Training in Cincinnati or attend remotely from your office or home.

This class is available world wide though Microsoft training partners
and the Microsoft Courseware library. Just ask for course 55095.

 

MS-55095 SharePoint 2010 and 2013 Auditing and Site Content Administration using PowerShell

This one day instructor-led class is designed for SharePoint 2010 and 2013 server administrators and auditors who need to query just about anything in SharePoint. The class handout is effectively a take home cheat sheet with over 175 PowerShell scripts plus the general patterns to create your own scripts. These scripts cover:

• getting lists / inventories of servers, services web applications, sites, webs, lists, libraries, items, fields, content types, users and much more
• finding lists by template type, content type and types of content
• finding files by user, content type, file extension, checked out status, size and age
• finding inactive sites
• finding and changing SharePoint Designer settings and finding and resetting customized pages
• inventorying and managing features
• deleting and recycling files and list items
• inventorying users and user permissions and finding out “who can access what”
• creating sites, lists and libraries
• uploading and downloading files
• general tips for counting, reformatting and exporting results;
• drilling up and down the SharePoint object model
• and much more…

 

Prerequisites: You should have good SharePoint skills as and end user and administrator along with some practical experience with PowerShell.

 

.

SharePoint – Use PowerShell to get all Owners, Full Control Users and Site Collection Administrators

Updated 6/12/2015

The following works for both SharePoint 2010 and 2013.

 

So who has control in your SharePoint?

Some users are members of the site's Owners group while others have been directly given Full Control. Some may be Site Collection Administrators or even have "super powers" granted at the Web Application level. How do you find these?

PowerShell to the rescue!

 

Get all users who are members of the "Owners" groups.

Get-SPSite -Limit All | 
  Get-SPWeb -Limit All | 
  where { $_.HasUniquePerm -and $_.AssociatedOwnerGroup -ne $null } | 
  foreach { $TTNweburl = $_.Url; $_ } | 
  Select -ExpandProperty AssociatedOwnerGroup | 
  Select -ExpandProperty Users | 
  Select {$TTNweburl}, UserLogin, DisplayName

 

Get all users directly given Full Control

Get-SPSite -Limit All | 
  Get-SPWeb -Limit All | 
  Where { $_.HasUniquePerm } | 
  foreach { $TTNweb = $_; $_ } | 
  Select -ExpandProperty Users | 
  Where { $TTNweb.DoesUserHavePermissions($_,[Microsoft.SharePoint.SPBasePermissions]::FullMask) } | 
  Select {$TTNweb.Url}, UserLogin, DisplayName

You could also find users with Full Control like roles by testing for "ManageWeb" or "ManagePermissions". For a list of the permission types use:

[System.Enum]::GetNames("Microsoft.SharePoint.SPBasePermissions")

 

Get all users who are Site Collection Administrators:

Get-SPSite -Limit All | 
  Get-SPWeb -Limit All | 
  where { $_.HasUniquePerm } | 
  foreach { $TTNweburl = $_.Url; $_ } | 
  Select -ExpandProperty Users | 
  Where { $_.IsSiteAdmin } | 
  Select {$TTNweburl}, UserLogin, DisplayName

 

Who else can see the content, and might have Full Control?

Some users may have access to site content via Web Application level policies. These are set in Central Administration in the Web Application Management section.

Get-SPWebApplication | 
  foreach { $TTNwebappUrl = $_.Url; $_ } | 
  Select -ExpandProperty Policies |  
  Select {$TTNwebappUrl}, DisplayName, IsSystemUser, PolicyRoleBindings, UserName | FT

 

.

SharePoint PowerShell–Find all Broken Inheritance

The following applies to both SharePoint 2010 and SharePoint 2013 on premises, but not to Office 365.

One of the common SharePoint tasks when you need to do a security audit, document security or cleanup a farm before an upgrade, is to try to figure out where the Site Owners have broken inheritance and created unique permissions. You could visit every site, list, library, folder, list item and document, or you could let PowerShell do the work for you.

The following is one of the many scripts found in SharePoint® 2010 Security for the Site Owner and my PowerShell class "MS-55095 SharePoint 2010 and 2013 Auditing and Site Content Administration using PowerShell". (Sign up for the July class and get a free copy of the book!)

 

First find all of the Webs with broken inheritance:

Get-SPSite http://yourSiteUrl  | 
Get-SPWeb -Limit All | 
Where { $_.HasUniquePerm -AND $_.ParentWeb -NE $Null } | 
Select ServerRelativeUrl, {$_.ParentWeb.ServerRelativeUrl}

 

Then find all of the Lists and Libraries with broken inheritance:

Get-SPSite http://yourSiteUrl  | 
Get-SPWeb -Limit All | 
Select -ExpandProperty Lists |
Where { $_.HasUniqueRoleAssignments -AND -NOT $_.Hidden } | 
Select Title, ParentWebUrl

 

Then find all of the folders with broken inheritance:

Get-SPSite http://yourSiteUrl  | 
Get-SPWeb -Limit All | 
Select -ExpandProperty Lists | 
Select -ExpandProperty Folders | 
Where { $_.HasUniqueRoleAssignments } | 
Select Title, {$_.ParentList.ParentWebUrl + "/" +$_.ParentList.Title}

 

Then find all of the items with broken inheritance:

Get-SPSite http://yourSiteUrl  | 
Get-SPWeb -Limit All | 
Select -ExpandProperty Lists | 
Select -ExpandProperty Items | 
Where { $_.HasUniqueRoleAssignments } | 
Select Name, {$_.ParentList.ParentWebUrl + "/" +$_.ParentList.Title}

 

What if we wanted a nice single list as the output?

Each of the scripts above return different kinds of columns. As PowerShell is a bit picky about what it will merge into a single column we will have a little more work to merge everything into a single list. One solution is to build an array or collection in memory, but this could get quite large. Another solution is to dump everything in to a CSV file and then open the result in Excel.

Note: The following script uses Export-CSV with the –Append parameter, which is not available in PowerShell 2.0.

Changes to the script:

• Add something to the Selects to identify the source.
    Select "List Item", Url, {$_.Web.Url}
• Create custom columns so all of the results have the same column names.
• Output the results to a CSV file.
    | Export-CSV "c:\test\BrokenInheritanceReport.csv" –Append
• Read them back and apply any needed sorting.

The following is all one script!

---

$siteUrl = "http://urlToYourSite"
$savePath = "c:\test\BrokenInheritanceReport.csv"

Get-SPSite $siteUrl  | 
  Get-SPWeb -Limit All | 
  Where { $_.HasUniquePerm -AND $_.ParentWeb -NE $Null } | 
  Select @{Label="Securable"; Expression={"Web"}}, 
         @{Label="Item"; Expression={$_.ServerRelativeUrl}}, 
         @{Label="Parent"; Expression={$_.ParentWeb.ServerRelativeUrl}} |
  Export-CSV $savePath

Get-SPSite $siteUrl  | 
  Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Where { $_.HasUniqueRoleAssignments -AND -NOT $_.Hidden } | 
  Select @{Label="Securable"; Expression={"List"}}, 
         @{Label="Item"; Expression={$_.Title}}, 
         @{Label="Parent"; Expression={$_.ParentWebUrl}} |
  Export-CSV $savePath -Append

Get-SPSite $siteUrl  | 
  Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Where { -NOT $_.Hidden -AND $_.EntityTypeName -NE "PublishedFeedList" } | 
  Select -ExpandProperty Folders | 
  Where { $_.HasUniqueRoleAssignments } | 
  Select @{Label="Securable"; Expression={"Folder"}}, 
         @{Label="Item"; Expression={$_.Title}}, 
         @{Label="Parent"; Expression={$_.ParentList.ParentWebUrl + "/" +$_.ParentList.Title}} | 
  Export-CSV $savePath -Append

Get-SPSite $siteUrl  | 
  Get-SPWeb -Limit All | 
  Select -ExpandProperty Lists | 
  Where { -NOT $_.Hidden -AND $_.EntityTypeName -NE "PublishedFeedList" } | 
  Select -ExpandProperty Items | 
  Where { $_.HasUniqueRoleAssignments } | 
  Select @{Label="Securable"; Expression={"Item"}}, 
         @{Label="Item"; Expression={$_.Name}}, 
         @{Label="Parent"; Expression={$_.ParentList.ParentWebUrl + "/" +$_.ParentList.Title}} | 
  Export-CSV $savePath -Append


Import-CSV  $savePath | Sort Parent | Select *

# or open the CSV file in Excel and sort there.

---

New Book and New Class!

New book: SharePoint^® 2010 Security for the Site Owner

I was always looking for a SharePoint security resource to point people to. I found content for server administrators and for developers, but nothing for site owners. Finally I decided I could quickly put together a little book on the topic. Little did I know just how much I would end up writing, testing and rewriting to get this thing done!

It only took 2½ years to complete! Work on this book started with the creation of my blog site in 2007. Or maybe it started in 2006 with the students in my SharePoint classes when they asked questions about the obvious and not so obvious SharePoint security features. The actual writing started 2½ years ago.

Why a 2010 book in 2014? It took that long to write it. (I'm both slow and busy!) But… give me a few weeks and then you can get the 2013 version of the book.

You can order it now from Amazon.

 

New Class: MS-55095 SharePoint 2010 and 2013 Auditing and Site Content Administration using PowerShell

Starting with the PowerShell chapter from the security book, and adding another 100 or so scripts, I now have a class for on premises SharePoint 2010 and 2013 administrators, auditors and governance teams who need to query just about anything in SharePoint. The class handout is effectively a cheat sheet with over 175 PowerShell scripts plus the general patterns to create your own scripts.

For all of the details of the class see here: http://techtrainingnotes.blogspot.com/2014/06/new-sharepoint-powershell-course.html

For class schedules see here: http://www.maxtrain.com/Classes/ClassInfo.aspx?Id=119394 or call MAX Technical Training at 513-322-8888. This class is available both in Cincinnati, and remotely from anywhere. This class will soon be available to all Microsoft training centers as course 55095AC.

You will need core PowerShell skills for this class, so I'd recommend having attended either of these two classes: MS-10961 Automating Administration with Windows PowerShell or MS-50414 Powershell v2 for Administrators, or equivalent.

Bonus! When you register for the class, tell them that you heard about it from Mike, and we will get you a free copy of the security book!

 

.

Merging Two PowerShell Collections into One Output

 

In SharePoint users and groups are both security principles, and both share some common properties. One property, ID, is interesting as it is unique and never duplicated between the lists of users and group. I.e. if there is a user with an ID of 5 then there is never a group with an ID of 5.

In PowerShell there are two separate properties for users and groups, but I wanted to merge the two into one sorted list. Turns out, as long as both Select statements return columns with the same names, then they can be "added" to get a merged result.

 

Example: Users and Groups

$users = $web.SiteUsers | Select Id, Name
$groups = $web.Groups | Select Id, Name
$users + $groups | Sort Id

If you wanted to do it all in one line, then use a few parentheses:

($web.SiteUsers | Select Id, Name)  +  ($web.Groups | Select Id, Name) | Sort Id

What if the column names don't match (but have similar data types)? You will need to create a PowerShell custom column. In the example below I wanted to use the user's DisplayName property instead of the Name property so I had to create a custom column named "Name" to match the "Name" property in the groups Select.

$users = $web.SiteUsers | Select Id, @{Label="Name"; Expression={$_.DisplayName|}}

$groups = $web.Groups | Select Id, Name

$users + $groups | Sort Id

 

.