Hello! Hal Zucati here, a writer with Enterprise Search User Assistance. I want to share some information about how to manage crawl deletion policies in SharePoint Server 2010.
SharePoint Server 2010 uses four policies that control accidental deletion of content, when the crawl component (crawler) encounters intermittent errors during a crawl. These policies are controlled by properties on the Search service application and they determine how many times and how long an item will be retried in consecutive crawls.
The four policies and the default values for the properties (in bold) are described below.
When the crawler encounters an access denied or a file not found error, the index item is deleted from the query component search index if the error was encountered in the more than ErrorDeleteCountAllowed consecutive crawls AND the duration since first error is greater than ErrorDeleteIntervalAllowed hours. If both conditions are NOT met, the index item is retried.
The default value for ErrorDeleteCountAllowed is 30 and ErrorDeleteIntervalAllowed is 720 hours (30 days).
When the crawler encounters errors of types other than access denied or file not found, the item is deleted from the search index if the error was encountered in more than ErrorCountAllowed consecutive crawls AND the duration since first error is greater than ErrorIntervalAllowed hours. If both conditions are not met the item is retried.
The default value for ErrorCountAllowed is 100 and ErrorIntervalAllowed is 1440 hours (60 days).
This policy applies only to SharePoint content. If the crawler encounters errors when fetching changes from the SharePoint content database for RecrawlErrorCount consecutive crawls AND the duration since first error is RecrawlIntervalCount hours, the crawler will force a re-crawl on that content database.
The default value for RecrawlErrorCount is 10 and RecrawlIntervalCount is 360 hours (15 days).
In full crawls of a content source, the crawler executes a delete unvisited phase where it deletes items that are in the crawl history but were not found in the current full crawl. This policy is exposed by property DeleteUnvisitedMethod, which determines what items get deleted during this phase. There are three possible values:
When DeleteUnvisitedMethod is 2, none of the unvisited items are deleted.
Each of these property values can ONLY be changed on the Search service application using Windows PowerShell for SharePoint Server 2010.
To change these properties:
To view the current value of a property use the following command:
To change the value of a property use the following command:
For more information, see Get-SPEnterpriseSearchServiceApplication (http://technet.microsoft.com/en-us/library/ff608050.aspx).
Thanks for reading. If you have feedback, leave a comment.