Turbo Crawl – Noderunner.exe CPU (RealTime Priority)

Wanted to share trick for how to unlock turbo Crawl search speeds with Microsoft SharePoint Server 2013/2016/2019/SE. Increasing the CPU process priority for “Noderunner.exe” from default “Normal” up to “RealTime” will give increase CPU horsepower to cycle from current document to the next. Given SharePoint and search crawl engine’s linear processing nature (serial document download, parse, save, then repeat) the pipeline performance benefits greatly from low latency.

Basically, with serial (one-at-a-time) processing the faster we pick up next document, the faster overall completion.

Internally, each Microsoft SharePoint search topology green checkmark is one instance of “Noderunner.exe” process doing the background processing. Increasing priority to those gives more O/S allocation of hardware. During “Full Crawl“, for example, we want as much hardware given to this pipeline as possible. Cheers.

CODE

# Turbo speed for Microsoft SharePoint Server 2013/2016/2019/SE crawl engine.  Boost CPU process priority to "real-time" for NODERUNNER for full farm search topology and W3WP for specific crawl machines.
Add-PSSnapIn "Microsoft.SharePoint.PowerShell"
$servers = Get-SPServer
$servers | Format-Table -AutoSize
foreach ($s in $servers) {
    # Display
    $pc = $s.Address
    Write-Host $pc -ForegroundColor "Yellow"
    # Find Noderunner
    $noderunner = (Get-Process -ComputerName $pc -Name "noderunner")
    $noderunner | Format-Table -AutoSize
    # Increase CPU priority
    foreach ($n in $noderunner) {
        # https://devblogs.microsoft.com/scripting/hey-scripting-guy-can-i-use-windows-powershell-to-lower-the-priority-of-a-process/
        $handle = $n.Handles
        
        # 256 = Realtime
        ([wmi]"win32_process.handle='$handle'").setPriority(256)
        # Increase priority
        Get-WmiObject Win32_process -ComputerName $pc -Filter 'name = "noderunner.exe"' | ForEach-Object { $_.SetPriority(256) }
    }
}

SEARCH TOPOLOGY

Below example shows traditional SharePoint Search Service Application (SSA) where each green checkmark indicates one instance of “Noderunner.exe”

PROCESS FLOW

Below is the serial pipeline for how search crawlers process one document then next. Low latency (from CPU Realtime) gives faster ability to pick up the next document and repeat.

PERFORMANCE CHART

Document Per Second (DPS) is the key metric for search engine speed to serially complete above steps.

REFERENCE

• https://devblogs.microsoft.com/scripting/hey-scripting-guy-can-i-use-windows-powershell-to-lower-the-priority-of-a-process/
• https://www.tenforums.com/tutorials/89548-set-cpu-process-priority-applications-windows-10-a.html
• https://learn.microsoft.com/en-us/sharepoint/overview-of-search
• https://learn.microsoft.com/en-us/sharepoint/search/manage-the-search-topology