Tag Archives: Administration
SPLaunch – new CodePlex for PowerShell automation
Yesterday I published a new CodePlex project to automate PowerShell remoting. Basically I got tired of RDP-ing to 10 machines to type the same command 10 times. Why not use a local PowerShell window to read CSV with server names and execute remotely in bulk? Well, now you can.
Give it a try for an hour to download, setup, and run. You can save that much time in just a day by using this in instead of juggling RDP and typing multiple PowerShell commands. Please leave a comment or CodePlex review to let me know what you think. ![]()
Business Challenge
- Growing list of servers
- Inconsistent configuration (DEV has, TEST lacks, PROD just crazy)
- Need to reduce admin time. Make room for business analysis, teach how to use, evangelize features
Technical Solution
- PowerShell remoting + automation wrapper = rapid fire commands
- PS1 with functions
- Noun CSV with server farms and login user
- Verb CSV with "shortcut" alias PowerShell commands
- Mix and match Noun/Verb CSV with automation wrapper (New-PSSession / Invoke-Command)
Examples
Open all servers in the "DCO" farm (Development Collaboration) and show total RAM size with percent used:
- LaunchFarm DCO
- LaunchShortcut ram | ft -a
Open just the first server across all farms and show the SharePoint build number:
- LaunchFarm ALL 1
- LaunchShortcut spver | ft -a
NOTE – MUST MANUALLY UPDATE "Noun" CSV file with target machine names, AD domain,
and user account before any commands can be run.
Screenshots
Getting Started
- Enable PowerShell remoting and WSMan CredSSP on all target servers. "Enable-PSRemoting -Force"and "Enable-WSManCredSSP -Role Server" http://dustinhatch.tumblr.com/post/24589312635/enable-powershell-remoting-with-credssp-using-group
- Enable PowerShell client with "Enable-PSRemoting -Force" and "Set-Item wsman:\localhost\client\trustedhosts * -Force"
- Update NounCSV with servers and user names
- Run "SPLaunch.ps1 -install" to add to $profile so it starts with new PowerShell windows
- Run LaunchAFarm to see farms
- Run LaunchAFarm DCO to connect to that farm (comma separated allows multiple farms)
- Run LaunchAShortcut to see shortcuts
- Run LaunchAShortcut RAM to execute the "RAM" command (end with " | ft" to format as table)
- Enjoy!
Warnings
- Account lockout – Possible if typing the password wrong and open many sessions at once.
- System update – Verb CSV can modify configuration on many servers quickly. Use carefully for changes.
Please drop me a line via email spjeff@spjeff.com or Twitter @spjeff with ideas. I’m always open to suggestions and improvements.
PowerShell – download all files in a Document Library
I needed to download the full contents of a Document Library to a local folder. Explorer View, SharePoint Designer, and WebDAV simply weren’t fast enough for the large number of items needed. So I wrote a quick script:
PowerShell – SOM (Server Object Model)
$s = Get-SPSite “http://sp2013”
$files = $s.RootWeb.GetFolder("Shared Documents").Files
foreach ($file in $files) {
Write-host $file.Name
$b = $file.OpenBinary()
$fs = New-Object System.IO.FileStream(("d:\temp\"+$file.Name), [System.IO.FileMode]::Create)
$bw = New-Object System.IO.BinaryWriter($fs)
$bw.Write($b)
$bw.Close()
}
PowerShell – Remote HTTP (Client Side / Office 365)
# Download all files from a SharePoint Document Library view to the local folder over HTTP remotely.
# No server side requirement, can run anywhere, including end user desktops.
param (
$url, # "http://portal/sites/test",
$listID, # "41E5FC8E-D174-4F48-AC3C-79F999A045D1",
$viewID # "5E9415B3-3C63-48AD-9DC2-A5A282B61790"
)
# Current folder
$scriptpath = Split-Path $MyInvocation.MyCommand.Path
# Open file list
Write-Host "Opening $url"
$ows = "$url/_vti_bin/owssvr.dll?Cmd=Display&List={$listID}&View={$viewID}&XMLDATA=TRUE"
$owsList = "$url/_vti_bin/owssvr.dll?Cmd=ExportList&List={$listID}&XMLDATA=TRUE"
$r = Invoke-WebRequest $ows -UseDefaultCredentials
$rList = Invoke-WebRequest $owsList -UseDefaultCredentials
$xml = $r.Content
$xmlList = $rList.Content
$folder = $xmlList.List.Url
# Client
$client = New-Object System.Net.WebClient
$client.UseDefaultCredentials = $true
# Loop and download
foreach ($row in $xml.xml.data.row) {
$name = $row.ows_LinkFilename
$from = "$url/$folder/$name"
$client.DownloadFile($from, "$scriptpath\$name")
}
Write-Host "DONE"NOTE – Remote HTTP (Client Side / Office 365) can recurse files from all sub-folders by updating the View to “Show all items (all folders)”
References
PowerShell – Always open default SharePoint site ($profile)
Recently I found myself starting each PowerShell window with the same predictable first command (Get-SPSite http://myportalurl). Knowing that PowerShell supports a start up script ($profile) I thought it may be possible to automate this and save a few seconds each time I need to open a console. Below is the source code with screenshots. Hopefully this helps somebody else to save time also.
Add-PSSnapin Microsoft.SharePoint.PowerShell
function Get-DefaultSite() {
$site = Get-SPSite http://myportalurl/
$web = $site.OpenWeb();
Return $web;
}
Configuring Workflow Manager 1.0 on SharePoint 2013 (screenshots)
During my first attempt to enable Workflow Manager 1.0 on a local VM I ran into the following error:
—> System.InvalidOperationException: Program C:\Program Files\Windows Fabric\bin\Fabric\Fabric.Code.1.0\FabricDeployer.exe exited with error: Windows Fabric deployment failed.DeploymentValidator: warning: The Fabric Data Collection Agent is disabled for this deployment.
None of the declared nodes is for the current machine.
The steps I took to resolve were:
- Start \ Programs \ Workflow Manager 1.0 \ Workflow Manager Configuration \ Remove from Farm (not to be confused with SharePoint “farm”)
- Drop SQL databases for Workflow Manager. (SbGatewayDatabase, SbManagementDB, SBMessageContainer01, WFInstanceManagementDB, WFManagementDB, WFResourceManagementDB)
- Start \ Programs \ Workflow Manager 1.0 \ Workflow Manager Configuration \ Configure Workflow Manager with Default Settings (Recommended)
- Two things I did differently on the next pass:
- I used the SQL alias “SPSQL” instead of the FQDN machine names. I’m using that for SharePoint and prefer SQL aliases for management flexibility.
- I formatted the user name as “DEMO\SPFARM” instead of “SPFARM@DEMO” (default text value). That seems to make more sense for AD login credentials and I didn’t understand why the default was more of an email syntax.
- I commented out all of the HOSTS file entries. While this may not be related, I wanted to simplify the connections as much as possible. I had “127.0.0.1” entries for the local machine to ensure connectivity even as a change VM host network settings, but removed while running the WF configuration wizard.
- Ran the wizard just like before and it completed successfully this time.
Now I can continue the steps outlined at http://technet.microsoft.com/en-us/library/jj658588(v=office.15) and run the PowerShell cmdlet Register-SPWorkflowService
=========================================================================
Configuration for Workflow Manager
Management Database SQL Instance SP15.demo.com
Enable SSL connection with SQL Server instance False
Authentication Windows Authentication
Management Database Name WFManagementDB
Instance Management Database SQL Instance SP15.demo.com
Enable SSL connection with SQL Server instance False
Authentication Windows Authentication
Instance Management Database Name WFInstanceManagementDB
Resource Management Database SQL Instance SP15.demo.com
Enable SSL connection with SQL Server instance False
Authentication Windows Authentication
Resource Management Database Name WFResourceManagementDB
RunAs Account spfarm@DEMO
RunAs Password ***********
Certificate Generation Key ***********
Workflow Manager Outbound Signing Certificate Auto-generated
Service SSL Certificate Auto-generated
Encryption Certificate Auto-generated
Workflow Manager Management Port 12290
HTTP Port Disabled
Enable firewall rules on this computer True
Administrators Group BUILTIN\Administrators
Configuration for Service Bus
Management Database SQL Instance SP15.demo.com
Enable SSL connection with SQL Server instance False
Authentication Windows Authentication
Management Database Name SbManagementDB
Gateway Database SQL Instance SP15.demo.com
Enable SSL connection with SQL Server instance False
Authentication Windows Authentication
Gateway Database Name SbGatewayDatabase
Message Container SQL Instance SP15.demo.com
Enable SSL connection with SQL Server instance False
Authentication Windows Authentication
Message Container Database Name SBMessageContainer01
RunAs Account spfarm@DEMO
RunAs Password ***********
Certificate Generation Key ***********
Farm Certificate Auto-generated
Encryption Certificate Auto-generated
HTTPS Port 9355
TCP Port 9354
Message Broker Port 9356
Internal Communication Port Range 9000 – 9004
Enable firewall rules on this computer True
Administrators Group BUILTIN\Administrators
=========================================================================
# To be run in Workflow Manager PowerShell console that has both Workflow Manager and Service Bus installed.
# Create new SB Farm
$SBCertificateAutoGenerationKey = ConvertTo-SecureString -AsPlainText -Force -String '***** Replace with Service Bus Certificate Auto-generation key ******' -Verbose;
New-SBFarm -SBFarmDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=SbManagementDB;Integrated Security=True;Encrypt=False' -InternalPortRangeStart 9000 -TcpPort 9354 -MessageBrokerPort 9356 -RunAsAccount 'spfarm@DEMO' -AdminGroup 'BUILTIN\Administrators' -GatewayDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=SbGatewayDatabase;Integrated Security=True;Encrypt=False' -CertificateAutoGenerationKey $SBCertificateAutoGenerationKey -MessageContainerDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=SBMessageContainer01;Integrated Security=True;Encrypt=False' -Verbose;
# To be run in Workflow Manager PowerShell console that has both Workflow Manager and Service Bus installed.
# Create new WF Farm
$WFCertAutoGenerationKey = ConvertTo-SecureString -AsPlainText -Force -String '***** Replace with Workflow Manager Certificate Auto-generation key ******' -Verbose;
New-WFFarm -WFFarmDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=WFManagementDB;Integrated Security=True;Encrypt=False' -RunAsAccount 'spfarm@DEMO' -AdminGroup 'BUILTIN\Administrators' -HttpsPort 12290 -HttpPort 12291 -InstanceDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=WFInstanceManagementDB;Integrated Security=True;Encrypt=False' -ResourceDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=WFResourceManagementDB;Integrated Security=True;Encrypt=False' -CertificateAutoGenerationKey $WFCertAutoGenerationKey -Verbose;
# Add SB Host $SBRunAsPassword = ConvertTo-SecureString -AsPlainText -Force -String '***** Replace with RunAs Password for Service Bus ******' -Verbose;
Add-SBHost -SBFarmDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=SbManagementDB;Integrated Security=True;Encrypt=False' -RunAsPassword $SBRunAsPassword -EnableFirewallRules $true -CertificateAutoGenerationKey $SBCertificateAutoGenerationKey -Verbose;
Try {
# Create new SB Namespace New-SBNamespace -Name 'WorkflowDefaultNamespace' -AddressingScheme 'Path' -ManageUsers 'spfarm@DEMO','spfarm@DEMO' -Verbose;
Start-Sleep -s 90
} Catch [system.InvalidOperationException] { }
# Get SB Client Configuration
$SBClientConfiguration = Get-SBClientConfiguration -Namespaces 'WorkflowDefaultNamespace' -Verbose;
# Add WF Host $WFRunAsPassword = ConvertTo-SecureString -AsPlainText -Force -String '***** Replace with RunAs Password for Workflow Manager ******' -Verbose;
Add-WFHost -WFFarmDBConnectionString 'Data Source=SP15.demo.com;Initial Catalog=WFManagementDB;Integrated Security=True;Encrypt=False' -RunAsPassword $WFRunAsPassword -EnableFirewallRules $true -SBClientConfiguration $SBClientConfiguration -CertificateAutoGenerationKey $WFCertAutoGenerationKey -Verbose;
=========================================================================
Configuration operation partially succeeded. Use ‘Join an Existing farm’ to complete the configuration.
[Error] [3/27/2013 9:53:08 AM]: System.Management.Automation.CmdletInvocationException: Program C:\Program Files\Windows Fabric\bin\Fabric\Fabric.Code.1.0\FabricDeployer.exe exited with error: Windows Fabric deployment failed.DeploymentValidator: warning: The Fabric Data Collection Agent is disabled for this deployment.
None of the declared nodes is for the current machine.
—> System.InvalidOperationException: Program C:\Program Files\Windows Fabric\bin\Fabric\Fabric.Code.1.0\FabricDeployer.exe exited with error: Windows Fabric deployment failed.DeploymentValidator: warning: The Fabric Data Collection Agent is disabled for this deployment.
None of the declared nodes is for the current machine.
at Microsoft.ServiceBus.Commands.Common.ProcessHelper.RunCommandInProcess(String exeName, String arguments, String errorString)
at Microsoft.ServiceBus.Commands.AddSBHost.ProcessRecordImplementation()
at Microsoft.ServiceBus.Commands.ServiceBusBaseCmdlet.ProcessRecord()
at System.Management.Automation.CommandProcessor.ProcessRecord()
— End of inner exception stack trace —
at System.Management.Automation.PowerShell.EndInvoke(IAsyncResult asyncResult)
at Microsoft.Workflow.Deployment.ConfigWizard.CommandletHelper.InvokePowershell(Command command, Action`3 updateProgress)
at Microsoft.Workflow.Deployment.ConfigWizard.ProgressPageViewModel.AddSBNode(FarmCreationModel model, Boolean isFirstCommand)
=========================================================================
Starting
Validating input and configuration parameters.
Installing auto-generated certificate.
Granting ‘Log on as Service’ privilege to the run as account.
Windows Fabric configuration started.
Running Windows Fabric deployment.
Program C:\Program Files\Windows Fabric\bin\Fabric\Fabric.Code.1.0\FabricDeployer.exe exited with error: Windows Fabric deployment failed.DeploymentValidator: warning: The Fabric Data Collection Agent is disabled for this deployment.
None of the declared nodes is for the current machine.
WPI Offline Install – Workflow Manager 1.0 for SharePoint 2013
Earlier today I was trying to load Workflow Manager 1.0 to get the new “2013” flavor workflows running on my VM. The Microsoft download links are tiny files which launch WPI (Web Platform Installer). That’s fine for a hobby machine or one time install, but I need an offline install for real data centers. Why? Two reasons: they often block internet access for security and I want to install via script (do many machines faster).
I watched Web Platform Installer download and install the various components I wondered where it was keeping all those nice download files. Not %TEMP% and not C:\WINDOWS\TEMP\. The obvious locations yielded nothing so I launched SysInternals PROCMON.EXE and watched open file handles for the install window (click the target icon to select any active window). That helped me find the right location and now I can copy those MSI files directly for offline installs.
If you need any Web Platform Installer items to run offline, just watch this folder while running on 1 PC and then you can copy to others.
C:\Users\spfarm\AppData\Local\Microsoft\Web Platform Installer\installers\
SharePoint 2013 – Install Notes
Below is a quick brain dump from a one machine VM install for development. Getting the install media folder correctly populated with various patches, offline prerequisites, and correct XML configuration for AutoSPInstaller was key. Once that folder was ready, everything ran very smoothly. ![]()
Hardware
http://technet.microsoft.com/en-us/library/cc262485.aspx#reqOtherCap
- Dynamic expanding VHD up to 96GB with Start > Run > DISKMGMT.MSC > right click Create VHD
- Virtual Box VM (from Windows 8 64bit template)
- 10GB of RAM
- 128MB video RAM
- Bidirectional clipboard enabled
- Shared folder “SPMedia” for install as read only
- Share folder “temp” for read/write
Software
http://technet.microsoft.com/en-us/library/cc262485.aspx#section4
- Windows 2008 R2 (supported by data center team. I plan to make a “2012” flavor also)
- SQL 2008 R2 Service Pack 2 (SP2 is important. SP2013 install requires a minimum of SP1, but may as well get the latest while in there)
- IE 10
- All windows update complete
- Promoted to domain controller as “DEMO.COM”
- Created 1 service account for SQL:
- sqlsrv
- Created 9 service user accounts for SharePoint:
- spfarm
- sppool
- spservice
- spprofile
- spsearch
- spcontent
- spsuperuser
- spsuperreader
- spdata
Install Process
- Downloaded latest AutoSPInstaller (http://autospinstaller.codeplex.com/) build 99077 as of 3/22/2013
- Extracted AutoSPInstaller ZIP, downloaded offline prerequisites, and completed new XML file with service accounts
- Downloaded March 2013 Public Update (PU) and extracted to C:\SPMedia\2013\SharePoint\Updates\
- Run Command Prompt as Administrator
- Run C:\SPMedia\AutoSPInstaller\AutoSPInstallerLaunch.bat
Prerequisite Files
Directory of C:\SPMedia\2013\SharePoint\PrerequisiteInstallerFiles
03/25/2013 10:06 AM <DIR> .
03/25/2013 10:06 AM <DIR> ..
05/28/2011 10:58 PM 4,451,736 427087_intl_x64_zip.exe.zip
06/21/2011 02:26 PM 1,139,960 433385_intl_x64_zip.exe.zip
05/15/2012 03:03 PM 794,144 447698_intl_x64_zip.exe.zip
07/18/2012 12:21 PM 1,399,984 AppFabric1.1-RTM-KB2671763-x64-ENU.exe
03/24/2013 01:08 PM 50,352,408 dotnetfx45_full_x86_x64.exe
07/17/2012 11:59 PM 3,524 DownloadAllSP2013PreReqs.ps1
10/01/2012 08:53 PM <DIR> filterpack
07/18/2012 12:21 PM 249,344 MicrosoftIdentityExtensions-64.msi
09/27/2012 08:45 PM 4,722,704 NDP45-KB2759112-x64.exe
07/30/2012 03:20 PM 0 PutPrerequisiteFilesHere.txt
07/18/2012 12:21 PM 4,943,360 setup_msipc_x64.msi
06/24/2012 01:51 PM 14,214,344 SharePointFoundationLP_en-us_x64.exe
07/18/2012 12:21 PM 8,121,344 sqlncli.msi
07/18/2012 12:21 PM 1,026,048 Synchronization.msi
07/18/2012 12:21 PM 5,735,120 WcfDataServices.exe
01/20/2011 05:40 PM 4,321,793 Windows6.1-KB2472264-v3-x64.msu
07/18/2012 12:21 PM 14,514,662 Windows6.1-KB2506143-x64.msu
05/26/2011 02:59 PM 1,010,821 Windows6.1-KB2554876-v2-x64.msu
07/17/2011 12:52 AM 2,049,178 Windows6.1-KB2567680-x64.msu
05/03/2012 12:56 PM 669,029 Windows6.1-KB2708075-x64.msu
07/18/2012 12:21 PM 1,545,666 Windows6.1-KB974405-x64.msu
10/05/2012 01:48 PM 4,078,084 Windows8-RT-KB2765317-x64.msu
07/18/2012 12:21 PM 33,646,240 WindowsServerAppFabricSetup_x64.exe
22 File(s) 158,989,493 bytes
Directory of C:\SPMedia\2013\SharePoint\PrerequisiteInstallerFiles\filterpack
10/01/2012 08:53 PM <DIR> .
10/01/2012 08:53 PM <DIR> ..
06/23/2012 09:36 PM 450,560 filterpack.msi
06/23/2012 09:37 PM 3,313,863 filterpk.cab
2 File(s) 3,764,423 bytes
Total Files Listed:
24 File(s) 162,753,916 bytes
5 Dir(s) 62,485,356,544 bytes free
Operating Screenshots
- Central Admin
- Team Site
- 15 (and 14 hives)!
- Service Applications
- PowerShell build number
- PowerShell $profile to include “Add-PSSnapin Microsoft.SharePoint.PowerShell”
Document Library Site Collections (DLSC)
Splitting a large site collection into many can make backups, storage scaling, and even development easier. This is a pattern I like to call the DLSC or "Document Library Site Collection.”
A site collection starts our small at first, gets popular, explodes in size seemingly overnight, and the SharePoint admins scramble to support it. Sound familiar?
It can be helpful to peel out a large Document Library into a dedicated site collection. Then you have more options via PowerShell and SQL to manage backups. Have five document libraries with 100GB each? Fine, split them out to 5 site collections in 5 SQL databases. Very manageable.
How To – Split a Document Library into a new Site Collection
1) Make a new SQL content database (optional). If you know LOTS of content will be coming, then give it room to grow now.
2) Make a new blank site collection. The below PowerShell command can help direct the site creation to a new database. I like to use STS#1 for “Blank Site” because it has a minimal simple footprint. Less is more.
New-SPSite http://sharepoint.com/sites/doclib -OwnerAlias “DOMAIN\Admin” -ContentDatabase WSS_Content_DocLib -Name “Document Library Site Collection” -Description “DLSC” -Template “STS#1″
3) Export the source Document Library to CMP format
http://spdeploymentwizard.codeplex.com/ can be used for a friendly safe GUI to export with all the right options. I like to enable checkboxes for ALL security, ALL versions, and ALL user info.
4) Import the source Document Library to CMP format
http://spdeploymentwizard.codeplex.com/ can be used for the import too.
5) Delete the original Document Library.
6) Update and test navigation. The biggest challenge with the DLSC model is navigation. Things must be easy for users so they can’t tell the difference. Be sure to update Top Link bar navigation. I like to add a CEWP (Content Editor Web PArt) with a quick JavaScript to redirect folks back to the main “application” site.
Click here to return to AppSite
Downsides and Caveats
- Manage two sets of security. Yes, you’ll have to grant people permissions in two places. Even using SPGroups won’t help because those are scoped to one site collection. AD groups can help. Those would be a great way to grant security to many site collections at once. All about scale. With only 2 sites AD groups probably aren’t worth the trouble, but with 20 it sure would help.
- Sandbox code solutions. These only run in 1 site collection, so if you’re using them to manage documents and move those document libraries you could see issues. The workaround would probably be to upgrade the code to a Central Admin farm WSP solution.
- Dependency. For some migrations you’ll need to copy ALL site collections. If you copy just 1 then you may have broken links or missing dependencies.
Hope this helps somebody else out there. ![]()
Site Assets – enable Version History to sleep well at night
One thing I like to do on a newly created site is enable Version History for the “Site Assets” document library. With many CSS and JS files supporting front end development, often those code files land here. SharePoint Designer can be used to quickly check this setting.
“Site Pages” will have Version History enabled by default on newly created Team Sites. That covers ASPX pages, but it can be smart to enable both for safety.
FIXED – SharePoint 2013 Setup Error – Appfabric is not correctly configured
When trying to install SharePoint Server 2013, I saw the below error. This was caused by manually double clicking the file “WindowsServerAppFabricSetup_x64.exe” instead of allowing the SharePoint Prerequisite Installer to run this.
The resolution was to run “prerequisiteinstaller.exe” with command line parameters to directly specify the filename for each prerequisite. Just like with SharePoint 2010, when installing offline we must tell the installer where to locate each file. Running the various components manually doesn’t work well. Apparently there is an install sequence or extra parameters which “prerequisiteinstaller.exe” must perform for the prerequisites to install correctly.
Below is the correct command line to use offline. I saved this as “prereq.cmd” on my install folder. Hope that helps! ![]()
prerequisiteinstaller.exe /SQLNCLi:prerequisiteinstallerfiles\sqlncli.msi /PowerShell:prerequisiteinstallerfiles\Windows6.1-KB2506143-x64.msu /NETFX:prerequisiteinstallerfiles\dotNetFx45_Full_x86_x64.exe /IDFX:prerequisiteinstallerfiles\Windows6.1-KB974405-x64.msu /Sync:prerequisiteinstallerfiles\Synchronization.msi /AppFabric:prerequisiteinstallerfiles\WindowsServerAppFabricSetup_x64.exe /IDFX11:prerequisiteinstallerfiles\MicrosoftIdentityExtensions-64.msi /MSIPCClient:prerequisiteinstallerfiles\setup_msipc_x64.msi /WCFDataServices:prerequisiteinstallerfiles\WcfDataServices.exe /quiet /KB2671763:prerequisiteinstallerfiles\AppFabric1.1-RTM-KB2671763-x64-ENU.exe
Download DLSC.vsdx
