Stefan Goßner

Senior Escalation Engineer for SharePoint (WSS, SPS, MOSS, SP2010) and MCMS

Adjusting the MOSS ROBOTS meta tag for 3rd party search engines

Adjusting the MOSS ROBOTS meta tag for 3rd party search engines

  • Comments 10
  • Likes

When developing a public facing website using the publishing features of MOSS it might be required to emit a ROBOTS meta tag that prevents internet search engines from indexing specific pages.

The standard RobotsMetaTag control included in WSS generates the following tag:

<META NAME="ROBOTS" CONTENT="NOHTMLINDEX">

Unfortunatelly most search engines do not understand the NOHTMLINDEX content value but a value of NOINDEX. To resolve this it is required to replace the RobotsMetaTag control in the master page with a custom control that emits the correct value.

The following code will do this:

using System;
using System.Collections.Generic;
using System.Text;
using System.Web;
using System.IO;
using Microsoft.SharePoint;
using System.Web.UI;

namespace StefanG.MossControls
{
    public class RobotsMetaTag : Microsoft.SharePoint.WebControls.RobotsMetaTag
    {
        protected override void Render(HtmlTextWriter writer)
        {
            SPWeb web = GetContextWeb(HttpContext.Current);
            if (web.ASPXPageIndexed == false)
            {
                writer.Write("<META NAME=\"ROBOTS\" CONTENT=\"NOHTMLINDEX,NOINDEX\"/>");
            }
        }
    }
}

Here are the required steps to implement this:

  1. compile the above code into a separate DLL (e.g. RobotsMetaTag.dll)
  2. if signed add the DLL to the GAC - otherwise copy it into the bin directory of your web application
  3. add the control to the save controls list in the web.config

          <SafeControl Assembly="RobotsMetaTag" Namespace="StefanG.MossControls" TypeName="*" Safe="True" />

    (if the DLL is in the GAC ensure to add the strong assembly name instead with version and public key token).
  4. Open the master page in SharePoint designer and register your DLL:

          <%@ Register TagPrefix="StefanG" Namespace="StefanG.MossControls" Assembly="RobotsMetaTag" %>

    (if the DLL is in the GAC ensure to add the strong assembly name instead with version and public key token).
  5. replace the following tag:

          <SharePoint:RobotsMetaTag runat="server"/>

    with this:

          <StefanG:RobotsMetaTag runat="server"/>

 

Comments
  • PingBack from http://mikewalsh.bilsimser.com/PermaLink,guid,18881168-9d71-4a58-a8c3-224a8ae4aec0.aspx

  • Is there a way to avoid Sharepoint Designer warning us that we are missing the original       <SharePoint:RobotsMetaTag runat="server"/> tag?

  • HotFix Security Hotfix MS07-059 Artikel zum Lesen Why SharePoint Server is Terrible Dazu der Kommentar

  • Hi Stefan,

    Why do you not use a control adapter instead of custom control? Is it for performance improvment?

    Gael

  • Hi Gael,

    nice idea! You are right: this would be the better solution.

    I will post it in a minute as a new blog entry.

    Cheers,

    Stefan

  • When developing a public facing website using the publishing features of MOSS it might be required to

  • I don't understand.

    If I put this in the Master Page then it means that the WHOLE site isn't indexed, no?

    I need just a few pages to be excluded.

  • Hi Tal,

    then change the logic in the control and look for a page specific property rather than the site property to decide which robot tags to render.

    Cheers,

    Stefan

  • Thanks stefan.

    how do I check a page property?

  • Hi Tal,

    you would need to open the list item for the current URL and then read the property. Similar to the code above.

    Cheers,

    Stefan

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
Raw Html Fix