PowerShell script to read source SQL table and synchronize into SharePoint Online (SPO) List destination. Great for PowerPlatform, PowerApps, and PowerBI integration scenarios. Optimized with usage of primary key, index columns, [Compare-Object] cmdlet [System.Data.DataRow] type, hashtable, and PNP batch HTTP POST network traffic. Cheers.
Wanted to share trick for how to unlock turbo Crawl search speeds with Microsoft SharePoint Server 2013/2016/2019/SE. Increasing the CPU process priority for “Noderunner.exe” from default “Normal” up to “RealTime” will give increase CPU horsepower to cycle from current document to the next. Given SharePoint and search crawl engine’s linear processing nature (serial document download, parse, save, then repeat) the pipeline performance benefits greatly from low latency.
Basically, with serial (one-at-a-time) processing the faster we pick up next document, the faster overall completion.
Internally, each Microsoft SharePoint search topology green checkmark is one instance of “Noderunner.exe” process doing the background processing. Increasing priority to those gives more O/S allocation of hardware. During “Full Crawl“, for example, we want as much hardware given to this pipeline as possible. Cheers.
CODE
# Turbo speed for Microsoft SharePoint Server 2013/2016/2019/SE crawl engine. Boost CPU process priority to "real-time" for NODERUNNER for full farm search topology and W3WP for specific crawl machines.
Add-PSSnapIn "Microsoft.SharePoint.PowerShell"
$servers = Get-SPServer
$servers | Format-Table -AutoSize
foreach ($s in $servers) {
# Display
$pc = $s.Address
Write-Host $pc -ForegroundColor "Yellow"
# Find Noderunner
$noderunner = (Get-Process -ComputerName $pc -Name "noderunner")
$noderunner | Format-Table -AutoSize
# Increase CPU priority
foreach ($n in $noderunner) {
# https://devblogs.microsoft.com/scripting/hey-scripting-guy-can-i-use-windows-powershell-to-lower-the-priority-of-a-process/
$handle = $n.Handles
# 256 = Realtime
([wmi]"win32_process.handle='$handle'").setPriority(256)
# Increase priority
Get-WmiObject Win32_process -ComputerName $pc -Filter 'name = "noderunner.exe"' | ForEach-Object { $_.SetPriority(256) }
}
}
SEARCH TOPOLOGY
Below example shows traditional SharePoint Search Service Application (SSA) where each green checkmark indicates one instance of “Noderunner.exe”
PROCESS FLOW
Below is the serial pipeline for how search crawlers process one document then next. Low latency (from CPU Realtime) gives faster ability to pick up the next document and repeat.
PERFORMANCE CHART
Document Per Second (DPS) is the key metric for search engine speed to serially complete above steps.
Overall positive sign showing Microsoft investment into PowerPlatform support. Requires more effort to open MS Support Ticket with many form inputs, yet gives better chance at resolution and routing to correct engineer.
Demo how to schedule Azure Automation Runbook PowerShell PS1 on more frequent intervals than default 1 Hour provided by Azure Automation “Schedule” feature.
Leveraging Azure Logic Apps, we can create “Schedule” trigger to execute “Create Job” step and start Runbook every 5 Minutes. Cheers.
