Setup a SharePoint Crawl Target Front End Machine Properly in SP2010

  • Article

Hi all-

I wanted to share the steps on setting up a Fast or SharePoint Search Crawl Target in SP2010. The reason for a crawl target is to limit the user impact of the Search Crawl on your Production farm. Crawling will hammer your Front End servers due to massive amount of requests for content. If you set up a crawl target (dedicate a box directly for search crawls) then users will be not impacted by your heavy performance crawl in-progress.

Here are the steps to configure your crawl target.

First you want to disable HTTP throttling on the crawl target so no crawl requests will be queued or delayed on the Front End machine. Open up a SharePoint Management Console (PowerShell) and run this script

$svc=[Microsoft.SharePoint.Administration.SPWebServiceInstance]::LocalContent
$svc.DisableLocalHttpThrottling
$svc.DisableLocalHttpThrottling=$true
$svc.Update() 

Next step is to set the farm to distribute crawl traffic to only the Front End that is specified as your crawl target, in this example : Server1 is my crawl target and https://sharepoint is my farm name. Run this in the same or seperate instance of PowerShell.

$env:farmurl = "https://sharepoint"
$env:crawltarget = "server1"

$listOfUri = new-object System.Collections.Generic.List[System.Uri](1)
$zoneUrl = [Microsoft.SharePoint.Administration.SPUrlZone]'Default'
$webappUrl = $env:farmurl
$webapp = Get-SPWebApplication -Identity $webappUrl
$webApp.SiteDataServers.Remove($zoneUrl)
$listOfUri.Add("https://$env:crawltarget");
$webApp.SiteDataServers.Add($zoneUrl, $listOfUri);
$WebApp.Update()

You have successfully setup your crawl target, now your Search crawls can perform with high performance without impacting users on your farm.