Search Guys

How to throttle a BCS crawl

Matt King - Search — Fri, 19 Apr 2013 19:19:00 GMT

Throttling a BCS crawl is done the same way as other SharePoint crawls, using crawler impact rules.

The main trick is figure out what site name to use. The most precise way is to use a combination of the LobSystem name and the LobSystemInstance name. In my case it was "recipeslob_recipesinst" :

An alternative is to use the LobSystem name with a wildcard (e.g. recipeslob_*).

Now we can set our rule (click on “Crawler Impact Rules” in the Crawling group of the Search Service Application management in Central Admin:

From there click “Add Rule” and fill it in:

To see that it is working, watch the Search Gatherer - SharePointServerSearch / Threads Accessing Network prefmon counter. In this screen shot you can see during the crawl only one thread accessing the network. Also observer that the requests in my console output are going in a nice neat linear order:

Now I’ll change the rule to 4 and run again:

Notice that now there are four threads accessing the network and that the requests are coming in bunches.

SP2013: Getting at the Search Results web part Search Criteria via PowerShell

Matt King - Search — Thu, 04 Apr 2013 18:29:33 GMT

Back in the SharePoint 2010 days to change the source of the Core Results Web Part you would simply set the Scope property on the web part object and be done with it. In SharePoint 2013 we have query builders and result sources and things aren’t quite as straight forward.

First things first, lets get our web part object:

$siteUrl = "http://server/search"  # or whatever your site url is
$resultsPage = "Pages/results.aspx" # or whatever your search results page is
$resultsUrl = "$siteUrl/$resultsPage"

$spweb = Get-SPWeb $siteUrl

# Check out the page so we can make changes
$page = $spweb.GetFile($resultsPage)
$page.CheckOut()

# Get the web part manager for the page
$webPartManager = $spweb.GetLimitedWebPartManager($resultsUrl, [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

# And pull the web part we want to work on
$webpart = $webpartmanager.WebParts | ? { $_.Title -eq 'Search Results' }

So far this is working the same as it did in 2010. The difference is that the 2013 search criteria is now stored inside the DataProviderJSON property:

PS C:\> ConvertFrom-Json $webpart.DataProviderJSON

QueryGroupName                        : Default
QueryPropertiesTemplateUrl            :
IgnoreQueryPropertiesTemplateUrl      : False
SourceID                              :
SourceName                            :
SourceLevel                           :
CollapseSpecification                 :
QueryTemplate                         : {searchboxquery}
FallbackSort                          :
FallbackSortJson                      : null
RankRules                             :
RankRulesJson                         : null
AsynchronousResultRetrieval           : False
SendContentBeforeQuery                : True
BatchClientQuery                      : True
FallbackLanguage                      : -1
FallbackRankingModelID                :
EnableStemming                        : True
EnablePhonetic                        : False
EnableNicknames                       : False
EnableInterleaving                    : True
EnableQueryRules                      : True
EnableOrderingHitHighlightedProperty  : False
HitHighlightedMultivaluePropertyLimit : -1
IgnoreContextualScope                 : False
ScopeResultsToCurrentSite             : False
TrimDuplicates                        : True
Properties                            :
PropertiesJson                        : {}
ClientType                            : AllResultsQuery
UpdateAjaxNavigate                    : True
SummaryLength                         : 180
DesiredSnippetLength                  : 90
PersonalizedQuery                     : False
FallbackRefinementFilters             :
IgnoreStaleServerQuery                : True
RenderTemplateId                      :
AlternateErrorMessage                 :
Title                                 :

Since this is the default search results page you’ll see that there is no Source set and the QueryTemplate is the default “{searchboxquery}”.

To change the source you’ll need the Name, Level and Id and I haven’t yet figured out how to find the source id. So for now, we’ll just modify the QueryTemplate:

# Add a property restriction to the query template
$dataProvider = ConvertFrom-Json $webpart.DataProviderJSON
$dataProvider.QueryTemplate = "{searchboxquery} source:filesystem"
$webpart.DataProviderJSON = ConvertTo-Json $dataProvider -Compress

# Save the changes and check it back in
$webpartmanager.SaveChanges($webpart)
$page.CheckIn("Changed the Query Template")
$page.Publish("Changed the Query Template")

Now when we open up the web part editor and click on the “Change query” button we see our change:

Happy days are here again…

Mucking with search summaries in SP2013

Matt King - Search — Wed, 03 Apr 2013 15:46:54 GMT

For a project I’m working on I needed to add some redaction to the hit highlighting summaries of the search results. If I see anything that looks like a social security number, then replace the first five digits with asterisks.

At first this looked somewhat daunting because looking in the item templates, all the action on rendering the summary is inside of the ctx.RenderBody method:

<div id="_#= $htmlEncode(itemId) =#_" name="Item" data-displaytemplate="DefaultItem" class="ms-srch-item" onmouseover="_#= ctx.currentItem_ShowHoverPanelCallback =#_" onmouseout="_#= ctx.currentItem_HideHoverPanelCallback =#_">
    _#=ctx.RenderBody(ctx)=#_
    <div id="_#= $htmlEncode(hoverId) =#_" class="ms-srch-hover-outerContainer"></div>
</div>

After a little poking and prodding, it turns out all you need to do is modify the HitHighlightedSummary value before the RenderBody is called. Add your code after the $setResultItem as follows:

$setResultItem(itemId, ctx.CurrentItem);

// Redaction of Social Security Numbers
var snippet = ctx.CurrentItem['HitHighlightedSummary'];
if (snippet) {
    var re = /(?:<c0>)?[0-9]{3}(?:<\/c0>)?-(?:<c0>)?[0-9]{2}(?:<\/c0>)?-((?:<c0>)?[0-9]{4}(?:<\/c0>)?)/g;
    snippet = snippet.replace(re, '***-**-$1');
    ctx.CurrentItem['HitHighlightedSummary'] = snippet;
}

The regular expression is somewhat complicated because it needs to account for highlighting tags (). Note that you can add to the HitHighlightedSummary yourself and can use the tags if desired.

SharePoint 2013 and Search Service PerformanceLevel

Matt King - Search — Sun, 24 Feb 2013 17:10:00 GMT

Just a quick note… Set-SPEnterpriseSearchService still has the PerformanceLevel option in SP2013, but apart from the crawler component it doesn’t do anything. And in practice changing from Maximum (the default) to Partly Reduced won’t achieve much throttling anyway. Best just leave it at Maximum.

Content Processing Performance Scaling

Matt King - Search — Tue, 19 Feb 2013 15:56:27 GMT

SharePoint 2013 has added the Content Processing Service to handle preparing documents to be indexed by the search engine. One feature of the CPS is that it will adjust the number of processing flows depending on the number of physical cores present. From the 2013 boundaries page:

If the number of physical cores on the host equals N, then the Content processing component will have N*K feeding sessions. K is a constant coefficient with the initial value 3. A 4-core server will have 12 feeding sessions, which means that the Content processing component can process 12 documents in parallel. You can change the value of K by setting the NumberOfCssFeedersPerCPUForRegularCrawl property of the Search Service Application. SharePoint 2013 limits the value of N upwards to 12, even if a server has more than 12 physical cores. Therefore a 16-core server will have N*K = 12 * 3 = 36 feeding sessions.

Adjusting the K is done via PowerShell, as follows:

PS C:\> $ssa = Get-SPEnterpriseSearchServiceApplication
PS C:\> $ssa.SetProperty("NumberOfCssFeedersPerCPUForRegularCrawl", 4)
PS C:\> $ssa.GetProperty("NumberOfCssFeedersPerCPUForRegularCrawl")
4

Note that this property doesn’t exist in the ssa’s property bag until the default has been changed (so calling GetProperty on the default ssa will return an error)

To validate that your change has taken effect you can use the “Search Content Processing / Flow Instances Active” performance counter. This number will grow or decrease as you change the value of K. It should be equal to the NumberOfCPUOnBox * NumberOfCssFeedersPerCPUForRegularCrawl. There may be times where you’ll see additional flows reported. These most likely will be due to continuous crawl and are expected. This in turn can be confirmed by looking at the “Search Gatherer Content Plugin – SharePointServerSearch / … –Continous Crawl / CTS Sessions” counters.

FS4SP Branding Project

Matt King - Search — Mon, 21 Jan 2013 03:39:54 GMT

In this post I’ll describe a technique for branding a FAST Search for SharePoint Search Center. There are three places where branding changes are often made:

The master page. The FAST Search Center is based on minimal.master and in this example I’ll just take a copy of that page and make a minor tweak to prove that it is my version that is being used. Alternatively you can convert a v4 master page.
The Search Main publishing layout page. This defines the layout of the main search page with just the search box.
The Search Results publishing layout page. This defines the layout of the search results page.

We’ll build this all using Visual Studio 2010 into a site collection feature that can be applied to any search center. When the feature is activated, the search center will be applied and the publishing pages made active. On deactivation, we’ll roll back our changes so the solution can be properly retracted. Lastly for good measure, we’ll add in some logging to the ULS.

This won’t show any actual snazzy branding (I’m no UI designer). I’ll just be adding a custom css file and enough changes to the templates to prove that the customized versions are being used.

1. Open Visual Studio 2010 (as administrator), and create a new project. File > New > Project…, choosing the Empty SharePoint Project template.

Verify your local site and choose "Deploy as farm solution" (we'll be logging to the ULS which you can't do as a sandboxed solution)

2. Add the feature. Right click on MCSBranding\Features in the Solution Explorer and choose "Add Feature". By default it will be called "Feature1", right click and rename to "Branding" (or whatever you want to call it). In the Branding.feature designer, update the title and make sure to change the scope to "Site"

3. Add a module to hold the master page. Right click on the MCSBranding project in the Solution Explorer, Add > New Item…, choosing the "Module" template and naming it "MasterPage"

4. Add our master page. In the Solution Explorer, go to MCSBranding\MasterPage and delete the Sample.txt file that was created by the template. Right click on MasterPage and choose Add > Existing…, navigate to your …14\TEMPLATE\LAYOUTS directory and choose minimal.master. Right click on "minimal.master" and rename to "mcs_minimal.master" (or whatever makes sense for you). The Solution Explorer at this point should look like:

In the MasterPage\Elements.xml (which should have been opened when you added the module), you now need to add some magic bits to make this page available as a master page.

<Module Name="MasterPage" Url="_catalogs/masterPage">
    <File Path="MasterPage\mcs_minimal.master" Url="mcs_minimal.master" Type="GhostableInLibrary">
        <Property Name="Title" Value="MCS Master Page"/>
        <Property Name="MasterPageDescription" Value="MCS Master Page"/>
        <Property Name="ContentType" Value="$Resources:cmscore,contenttype_masterpage_name;" />
    </File>

If you want to make any changes specific to either the search main or results pages you'll need to add those publishing templates as well. THERE MUST BE A BETTER WAY TO DO THIS BUT HERE'S WHAT HAS WORKED FOR ME… Launch SharePoint Designer 2010 and open a site that already has an Enterprise FAST Search Center. Choose "Master Pages" in the Site Objects and you should see SearchMain.aspx and SearchResults.aspx. For each of them do the following: Right click and choose "Edit file in Advanced Mode". Do not check the file out. Switch to "Code" mode and select everything. Copy and paste into your favorite text editor and save to C:\Projects\MCSSolution\MCSBranding\MasterPage adding a "mcs_" prefix.

Next add SearchMain.aspx and SearchResults.aspx for us to customize. Right click on MasterPage again, Add > Existing…, navigate to …\MCSBranding\MasterPage and select both files to add at once. Back to the MasterPage\Elements.xml and these files also need some extra properties:

<File Path="MasterPage\mcs_searchmain.aspx" Url="mcs_searchmain.aspx" Type="GhostableInLibrary">
    <Property Name="Title" Value="MCS Search Main" />
    <Property Name="MasterPageDescription" Value="Search Main for MCS" />
    <Property Name="ContentType" Value="$Resources:cmscore,contenttype_pagelayout_name;"></Property>
    <Property Name="PublishingAssociatedContentType" Value=";#Welcome Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4;#"></Property>
</File>
<File Path="MasterPage\mcs_searchresults.aspx" Url="mcs_searchresults.aspx" Type="GhostableInLibrary">
    <Property Name="Title" Value="MCS Search Main" />
    <Property Name="MasterPageDescription" Value="Search Results for MCS" />
    <Property Name="ContentType" Value="$Resources:cmscore,contenttype_pagelayout_name;"></Property>
    <Property Name="PublishingAssociatedContentType" Value=";#Welcome Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4;#"></Property>
</File>

5. Add a place for your branding elements. Since your branding solution will undoubtedly have custom style sheets and images, we'll next add a module to house them. Click to highlight MCSBranding and press Ctrl-Shift-A and add another Module and name it "mcsBranding" (or whatever you want).

For this example, I'm just going to rename the sample.txt to mcs_branding.css which I'll add a reference to in my master page, inserting the following line just before the </head> tag:

<SharePoint:CssRegistration name="<% $SPUrl:~SiteCollection/mcsBranding/mcs_branding.css %>" runat="server"/>

For customizations in this example all I'm going to add is:

Search main line 153:

<div id="mcs_main">MCS_SEARCHMAIN</div>

Search results line 128:

<div id="mcs_results">MCS_SEARCHRESULTS</div>

Mcs_minimal.master line 136:

<div id="mcs_master">MCS_MASTER_PAGE</div>

And for mcs_branding.css:

#mcs_master 
{
   font-weight: bold;
   color: Red;
}

#mcs_main 
{
   font-weight: bold;
   color: Green;
}

#mcs_results
{
   font-weight: bold;
   color: Blue;
}

6. Feature Activation. Now comes the fun part. Thanks to my colleague Jeff Lyttle for the starter event receiver code that finally got me going on this, here's the setup that worked for me. Right click on your Branding feature and choose "Add Event Reciever".

Right click on the MCSBranding project and choose "Add reference". Select .NET tab and add Microsoft.SharePoint.Publishing. Replace the existing block of using's with:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Publishing;
using Microsoft.SharePoint.Administration;

In the event reciever, select and delete the commented out FeatureActivated and FeatureDeactivating method stubs and replace with

public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
    LoggingService.LogMonitor("DarpaBranding entering FeatureActivated");

    SPSite site = properties.Feature.Parent as SPSite;
    SPList masterPageGallery = site.GetCatalog(SPListTemplateType.MasterPageCatalog);
    foreach (SPListItem li in masterPageGallery.Items)
    {
        if (li.File.Name.ToLower() == "mcs_minimal.master" ||
            li.File.Name.ToLower() == "mcs_searchmain.aspx" ||
            li.File.Name.ToLower() == "mcs_searchresults.aspx" )
        {
            if (!li.HasPublishedVersion)
            {
                LoggingService.LogInfo(String.Format("Publishing {0}", li.File.Name));
                li.File.CheckIn("Automatically checked in by MCS Branding feature",
                SPCheckinType.MajorCheckIn);
                li.File.Update();
                li.File.Approve("Automatically approved by MCS Branding feature");
                li.File.Update();
            }
        }
    }
    SPWeb web = site.RootWeb;
    if (web.CustomMasterUrl.EndsWith("minimal.master"))
    {
        LoggingService.LogInfo("Applying our master page");
        Uri masterUri = new Uri(web.Url + "/_catalogs/masterpage/mcs_minimal.master");
        web.CustomMasterUrl = masterUri.AbsolutePath;
        LoggingService.LogInfo("Applying custom publishing pages");
        // register our search layout pages 
        PageLayout[] allLayouts = AddPageLayout(web, new string[] { "mcs_searchresults.aspx",
                                                                    "mcs_searchmain.aspx" });
        // set them on the search pages
        SetPublishingPageLayout(web, allLayouts, "default.aspx", "mcs_searchmain.aspx");
        SetPublishingPageLayout(web, allLayouts, "results.aspx", "mcs_searchresults.aspx");
        web.Update();
    }
}

public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
{
    SPSite site = properties.Feature.Parent as SPSite;
    SPWeb web = site.RootWeb;
    // roll back master page if needed
    if (web.CustomMasterUrl.EndsWith("mcs_minimal.master"))
    {
        Uri masterUri = new Uri(web.Url + "/_catalogs/masterpage/minimal.master");
        web.CustomMasterUrl = masterUri.AbsolutePath;
    }
    // reset the page layouts so our customized files can be retracted
    PublishingWeb pubWeb = PublishingWeb.GetPublishingWeb(web);
    PageLayout[] layouts = pubWeb.GetAvailablePageLayouts();
    SetPublishingPageLayout(web, layouts, "default.aspx", "SearchMain.aspx");
    SetPublishingPageLayout(web, layouts, "results.aspx", "SearchResults.aspx");
    web.Update();
}

You'll also need these helper methods. Paste at the bottom of the class:

        private static PageLayout[] AddPageLayout(SPWeb currentWeb, string[] layoutNames)
        {
            PublishingWeb pubWeb = PublishingWeb.GetPublishingWeb(currentWeb);

            PageLayout[] layouts = pubWeb.GetAvailablePageLayouts();

            List<string> layoutsToAdd = new List<string>();

            foreach (string currentLayoutName in layoutNames)
            {
                var layoutExists = from PageLayout currentLayout in layouts
                                   where currentLayout.Name == currentLayoutName
                                   select currentLayout;

                if (layoutExists.Count() == 0)
                {
                    layoutsToAdd.Add(currentLayoutName);
                }
            }

            if (layoutsToAdd.Count > 0)
            {
                List<PageLayout> newLayouts = new List<PageLayout>();

                SPList masterPageList = currentWeb.GetCatalog(SPListTemplateType.MasterPageCatalog);

                List<string> j = new List<string>();
                foreach (SPListItem r in masterPageList.Items)
                {
                    j.Add(r.Name);
                }
                j.Sort();
                string[] z = j.ToArray();

                foreach (string currentLayoutToAdd in layoutsToAdd)
                {
                    SPListItem itemLayoutsToAdd = (from SPListItem currentItem in masterPageList.Items
                                                   where (currentItem.Name == currentLayoutToAdd)
                                                   select currentItem).First();

                    newLayouts.Add(new PageLayout(itemLayoutsToAdd));
                }

                List<PageLayout> newLayoutList = layouts.ToList();
                newLayoutList.AddRange(newLayouts);

                PageLayout[] newLayoutSet = newLayoutList.ToArray();

                pubWeb.SetAvailablePageLayouts(newLayoutSet, false);
                pubWeb.Update();
                currentWeb.Update();

                return newLayoutSet.ToArray();

            }
            else
            {
                return layouts.ToArray();
            }

        }

        private static void SetPublishingPageLayout(SPWeb currentWeb, PageLayout[] existingLayouts, string pageName, string layoutName)
        {
            PublishingWeb pubWeb = PublishingWeb.GetPublishingWeb(currentWeb);

            PublishingPageCollection pubPages = pubWeb.GetPublishingPages();

            var pages = from PublishingPage currentPage in pubPages
                                where currentPage.Name == pageName
                                select currentPage;

            if (pages.Count() == 0)
            {
                LoggingService.LogError(String.Format("Failed to find publishing page {0}", pageName));
                return;
            }
            PublishingPage page = pages.First();

            var layouts = from PageLayout currentLayout in existingLayouts
                                        where currentLayout.Name == layoutName
                                        select currentLayout;
            if (layouts.Count() == 0) 
            {
                LoggingService.LogError(String.Format("Failed to find page layout {0}", layoutName));
                return;
            }
            PageLayout layout = layouts.First();

            LoggingService.LogInfo(String.Format("Setting {0} layout to {1}", pageName, layoutName));
            page.CheckOut();
            page.Layout = layout;
            page.Update();
            page.CheckIn("Added in by feature");
            SPFile pageFile = page.ListItem.File;
            pageFile.Publish("Added in by feature");
            LoggingService.LogInfo(String.Format("Publishing {0}", pageName));
        }
    }
        




    class LoggingService : SPDiagnosticsServiceBase
    {
        // This is what shows up in the Product column of the ULS viewer
        public static string LOG_AREA = "MCS Feature";
        private static LoggingService _Current;
        public static LoggingService Current
        {
            get
            {
                if (_Current == null)
                {
                    _Current = new LoggingService();
                }
                return _Current;
            }

        }

        private LoggingService()
            : base("MCS Logging Service", SPFarm.Local)
        {
        }

        protected override IEnumerable<SPDiagnosticsArea> ProvideAreas()
        {
            List<SPDiagnosticsArea> areas = new List<SPDiagnosticsArea>
            {
                new SPDiagnosticsArea(LOG_AREA, new List<SPDiagnosticsCategory>
                {
                    new SPDiagnosticsCategory("ERROR", TraceSeverity.Unexpected, EventSeverity.Error),
                    new SPDiagnosticsCategory("INFO", TraceSeverity.Verbose, EventSeverity.Information),
                    new SPDiagnosticsCategory("monitoring", TraceSeverity.Medium, EventSeverity.Information)
                })
            };
            return areas;
        }

        public static void LogError(string errorMessage)
        {
            LogError(errorMessage, 0);
        }
        public static void LogError(string errorMessage, ushort id)
        {
            string categoryName = "ERROR";
            SPDiagnosticsCategory category = LoggingService.Current.Areas[LOG_AREA].Categories[categoryName];
            LoggingService.Current.WriteTrace(id, category, TraceSeverity.Unexpected, errorMessage);
        }
        public static void LogInfo(string infoMessage)
        {
            LogInfo(infoMessage, 0);
        }
        public static void LogInfo(string infoMessage, ushort id)
        {
            string categoryName = "INFO";
            SPDiagnosticsCategory category = LoggingService.Current.Areas[LOG_AREA].Categories[categoryName];
            LoggingService.Current.WriteTrace(id, category, TraceSeverity.Verbose, infoMessage);
        }
        public static void LogMonitor(string infoMessage)
        {
            LogMonitor(infoMessage, 0);
        }
        public static void LogMonitor(string infoMessage, ushort id)
        {
            string categoryName = "monitoring";
            SPDiagnosticsCategory category = LoggingService.Current.Areas[LOG_AREA].Categories[categoryName];
            LoggingService.Current.WriteTrace(id, category, TraceSeverity.Medium, infoMessage);
        }

    }

7. Deploy it! Right click on the MCSBranding project and choose Deploy. (note: if your test farm has multiple WFE servers this will error out with the error "Error occurred in deployment step 'Activate Features': Feature with Id 'GUID' is not installed in this farm, and cannot be added to this scope."… apparently that's a known issue and you'll have to then go to Central Admin > System Settings > Manage Farm Solutions and deploy the solution manually.)

Now over to your FAST Search Center and to Site Settings -> Site Collection Features and you should see your feature ready to be activated:

Click "Activate" and if everything works you should just see that it has now been activated. If you were running the ULS Viewer you can filter by the Product "MCS Feature" and should be rewarded with:

Now if you go to the search home page you should see your customizations.

And on the results page:

Getting the current user’s alias

Matt King - Search — Mon, 03 Dec 2012 21:21:00 GMT

For a project I’m working on, I needed to know what the logged in user’s account alias was. Looking around I found examples that would make a ajax call out to the profile page and parse the results. This seemed like overkill for what I wanted and on a whim I decided to view-source on the Search Center results page and search for my user name… Sure enough, there it is:

The hidden QLogEnv inputs are used to click event tracking but this is good enough for me. Here’s the JQuery I used to pull it out:

function pullAlias() {
// Pull the alias or return null
var p = /<un>[a-zA-Z]+\\([^<]+)/;
var l = $('input').map(function(index) {
if (this.name.startsWith("QLogEnv") && (this.value.indexOf("<un>") != -1)) {
   var r = p.exec(this.value);
   if (r) {
    return r[1]
   }
}
})
if (!l) { return } // no alias found?
return l[0]
}

Managing the SP Gatherer's temp files

Matt King - Search — Tue, 18 Sep 2012 18:41:00 GMT

While the SharePoint Gatherer is processing files (Office documents, PDFs, etc), it will store them temporarily on local disk. By default this is in the Temp folder of the user running the OSearch14 service. It'll look something like:

C:\Users\SHAREP~1\AppData\Local\Temp\gthrsvc_OSearch14

And during a large crawl, this directory can at times get quite large. (as I found out today when a dev server started complaining abou the C drive being full)

Moving it is easy enough and there are 2 ways.

First, update the service application:

$searchapp= Get-SPEnterpriseSearchServiceApplication -identity "FAST Content SSA"
$searchapp.TempPath = "D:\SearchTemp"

Note that setting the path takes effect immediately. No need to call the $searchapp.Update() method.

UPDATE 5/17/2013
If you use the SSA.TempPath approach and have multiple crawlers, you need to make sure to run the command on each server anyway.

Second approach is to update the registry directly:

$gmpath = 'HKLM:\Software\Microsoft\Office Server\14.0\Search\Global\Gathering Manager'
Set-ItemProperty -Path $gmpath -Name UseSystemTemp -Value 0
Set-ItemProperty -Path $gmpath -Name TempPath -Value "D:\SearchTemp"
Restart-Service OSearch14

And don't forget to exclude the new temp folder from any AV scans.

(Thanks Sreedhar)

Reference

http://technet.microsoft.com/en-us/library/ee513078.aspx

Anti-Virus settings and Pipeline Extensibility

Paul Branson — Tue, 21 Aug 2012 20:26:03 GMT

If you’re using Pipeline Extensibility in Fast Search for SharePoint, you may know that files are written to a local directory for your program to work on before getting dumped back into the pipeline. This directory is:

<Fast user’s home directory>\AppData\LocalLow

Assuming you haven’t changed the location of the home directory, this would be:

C:\Users\<your Fast user>\AppData\LocalLow

Want to see proof that this directory is used?

If you want to see the files accumulate here (dev only of course), edit $FASTSEARCH\etc\pipelineextensibility.xml and add a ‘–logging enable’ parameter to the Run command. Your new command should look something like this:

<Run command="yourbinary.exe" %(input)s %(output)s -logging enable">

Save the file and run ‘psctrl reset’. This will reset all your item processors and read in the new config. Start a new crawl and watch the .txt files start showing up in that directory. When you’re done, change ‘enable’ to ‘disable’ to turn it back off.

My point here is that lots of txt files are writing to this directory. If you follow the anti-virus configuration settings found on TechNet, you won’t see this directory listed. If you’re following the recommendations on that page and still see your AV running high, this could be it. You may want to add your executable to the list of processes to exclude as well.

Getting Refiner Counts Back in SharePoint 2013 Search

Paul Branson — Thu, 16 Aug 2012 15:14:05 GMT

If you’ve taken a look at search in 2013 at all, you might have noticed the beloved counts are now gone from refiner counts. These were first introduced with Fast Search for SharePoint and represent what’s called deep refiners – that is- an accurate count of a refiner value from the entire corpus of data associated with your query. The alternative, just so you know, are shallow refiners which create refiners off the top X number of search results instead of all search results.

** Quick Note ** – this is being done with Beta bits. I’ll update this later if the RTM version ends up being different.

So first off, the counts were removed for a reason. Studies showed that while it may be nice to see them, they weren’t actually used that much by users. For example, if you’re looking for a Word document and use the refiners panel to narrow your search results, does it really matter that you’re going to narrow your results down to 58,245 results?

But… you’re here because you want them back, so here’s how you do it:

Here’s what it looks like by default. No counts:

As you can see I have three refiners, but the top two don’t have count values. For this, I’m going to add the count values to Author.

The first step is understanding what controls the look and feel of the refiners themselves. In Fast Search for SharePoint it was all controlled in XSLT. In 2013, it’s controlled by Display Templates. When you edit the Refinement Panel, you’ll now see this handy refiner chooser:

Click on Choose Refiners and you’ll get the following:

In the picture above, I’ve already selected my DisplayAuthor refiner and highlighted the associated Display Template called Refinement Item. This is what we need to change.

Display Templates are stored in the Master Page Gallery. There’s multiple ways to find this next bit of info, but what I do is start by going to the new Design Manager and selecting Edit Display Templates:

That will get you to the page where you can find the template you’re looking based on the name we saw in the refiner picker page (Refinement Item). From here we can locate the actual home of the template. Select the Open Menu (the 3 dots) and grab the location in the box. For me it’s http://search.contoso.com/_catalogs/masterpage/Display Templates/Filters/Filter_Default.html.

Going to that location, I find my files:

From here you have two options: (1) edit the OOTB template, or (2) create a copy and make a new one. I’m going to select option 2 so I don’t muck with the original. Just download a copy of Filter_Default.html to any location, and use your favorite HTML editor (aka: notepad) to open it up. Once in, you’re looking for this:

To get the counts back, you guessed it, just change the ShowCounts value to true. Once changed, saved the file as a new name. I used Filter_Default_with_counts.html. Now head back to the same page you downloaded the original from, and upload the new file to the same library (Files tab on top –> Upload Document). You’ll get a dialog when you upload to add metadata and give it a pretty name that you’ll see in the UI. I used ‘Refinement Item With Counts’. Once you upload it, you’ll notice it auto generates the associated .js. This is good, and you don’t need to worry about what it’s doing (maybe more on that in a future post). Here’s what you should now see:

Be sure to now check in the new .html file. Just click the little arrow to the right when hovering over it and you’ll see the check in option.

Now it’s just a matter of using this new template. Jump back to your Search Hub, edit the page when you see the refiner, edit the Refinement web part, select the Choose Refiners, pick your refiner (Display Author for me), and select the Display template drop down. Bam! There it is:

Select that, save everything, and check your refiner with another search:

Voila! Counts.