Stefan Goßner

Senior Escalation Engineer for SharePoint (WSS, SPS, MOSS, SP2010) and MCMS

Adjusting the MOSS ROBOTS meta tag for 3rd party search engines - using a Control Adapter (last update: April 11th)

Adjusting the MOSS ROBOTS meta tag for 3rd party search engines - using a Control Adapter (last update: April 11th)

  • Comments 8
  • Likes

When developing a public facing website using the publishing features of MOSS it might be required to emit a ROBOTS meta tag that prevents internet search engines from indexing specific pages.

The standard RobotsMetaTag control included in WSS generates the following tag:

<META NAME="ROBOTS" CONTENT="NOHTMLINDEX">

Unfortunatelly most search engines do not understand the NOHTMLINDEX content value but a value of NOINDEX.

I already posted a solution for this which replaced the RobotsMetaTag control. Here is now a more elegant solution based on ASP.NET Control Adapters. Control Adapters can be used to modify the rendering of a control based on different browsers. But by using the "Default" browser it also allows to modify the default rendering of a control. Thanks to Gael Duhamel for the idea!

Here is the source code of the control adapter which we will use to modify the behavior of the RobotsMetaTag control:

using System;
using System.Collections.Generic;
using System.Text;
using System.Web;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.Adapters;
using Microsoft.SharePoint;

namespace StefanG.ControlAdapters
{
    public class RobotsMetaTagControlAdapter : WebControlAdapter
    {
        protected override void Render(System.Web.UI.HtmlTextWriter writer)
        {
            SPWeb web = SPContext.Current.Web;
            if (web.ASPXPageIndexed == false)
                writer.Write("<META NAME=\"ROBOTS\" CONTENT=\"NOHTMLINDEX,NOINDEX\"/>");
            else 
                writer.Write("");     // updated April 11th to fix a WebControls/Controls cast problem.
        }
    }
}

The code looks very similar to the code I used in my first approach.

To build the adapter you need to create a class library project, add this source code to it, configure the project to sign the assembly and add the resulting DLL to the global assembly cache (GAC).

To bind the control adapter to the RobotsMetaTag control we now need to add a new browser (e.g. SharePointAdapters.browser) file to the App_Browsers directory of your SharePoint web application.

The content of the browser file needs to look similar to this (you need to adjust the assembly name and strong name key based on your specific assembly):

<browsers>
  <browser refID="Default">
    <controlAdapters>
      <adapter 
        controlType="Microsoft.SharePoint.WebControls.RobotsMetaTag"
        adapterType="StefanG.ControlAdapters.RobotsMetaTagControlAdapter, RobotsMetaTagControlAdapter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=dcee90dea8c5fab5
      />
    </controlAdapters>
  </browser>
</browsers>

That's it! After recycling your application pool the RobotsMetaTag control will now render it's content based on the custom control adapter. No modification to your master pages will be required.

Comments
  • Thanks Stefan - have been strugling with this recently.  

  • Mashup Virtual Earth und SharePoint Wunder der Welt Suche Search Custom Security Trimmer for Novell Netware

  • Is there any problem in removing this tag? why is it there in the first place?

  • Hi Seed,

    it is there to allow authors to decide if a specific site/page is allowed to be indexed by search engines or not.

    Cheers,

    Stefan

  • When I add a (second) browser file besides compat.browser. I see no changes in my web application. SharePoint continues using the settings from only the compat.browser file.

    I would like to have the following browser files active:

    - App_Browsers\compat.browser

    - App_Browsers\MyControlAdapters.browser

    Is this possibe?

  • Hi Harmjan,

    that's what I did.

    Cheers,

    Stefan

  • Harmjan (and others), if've just run into the same issue and the fix is to open the compat.browser in Notepad and save it (no need to change anything). This will force reloading the browser files. If you have multiple browser files already present you might need to repeat for each one.

    HTH

  • Hi Steven,

    the RobotsMetaTag controls emits the settings configured in "Site Actions" - "Site Settings" - "Modify all Site Settings" - "Search Visibility".

    Cheers,

    Stefan

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
Raw Html Fix