When developing a public facing website using the publishing features of MOSS it might be required to emit a ROBOTS meta tag that prevents internet search engines from indexing specific pages.
The standard RobotsMetaTag control included in WSS generates the following tag:
<META NAME="ROBOTS" CONTENT="NOHTMLINDEX">
Unfortunatelly most search engines do not understand the NOHTMLINDEX content value but a value of NOINDEX.
I already posted a solution for this which replaced the RobotsMetaTag control. Here is now a more elegant solution based on ASP.NET Control Adapters. Control Adapters can be used to modify the rendering of a control based on different browsers. But by using the "Default" browser it also allows to modify the default rendering of a control. Thanks to Gael Duhamel for the idea!
Here is the source code of the control adapter which we will use to modify the behavior of the RobotsMetaTag control:
The code looks very similar to the code I used in my first approach.
To build the adapter you need to create a class library project, add this source code to it, configure the project to sign the assembly and add the resulting DLL to the global assembly cache (GAC).
To bind the control adapter to the RobotsMetaTag control we now need to add a new browser (e.g. SharePointAdapters.browser) file to the App_Browsers directory of your SharePoint web application.
The content of the browser file needs to look similar to this (you need to adjust the assembly name and strong name key based on your specific assembly):
That's it! After recycling your application pool the RobotsMetaTag control will now render it's content based on the custom control adapter. No modification to your master pages will be required.
Thanks Stefan - have been strugling with this recently.
Mashup Virtual Earth und SharePoint Wunder der Welt Suche Search Custom Security Trimmer for Novell Netware
Is there any problem in removing this tag? why is it there in the first place?
it is there to allow authors to decide if a specific site/page is allowed to be indexed by search engines or not.
When I add a (second) browser file besides compat.browser. I see no changes in my web application. SharePoint continues using the settings from only the compat.browser file.
I would like to have the following browser files active:
Is this possibe?
that's what I did.
Harmjan (and others), if've just run into the same issue and the fix is to open the compat.browser in Notepad and save it (no need to change anything). This will force reloading the browser files. If you have multiple browser files already present you might need to repeat for each one.
the RobotsMetaTag controls emits the settings configured in "Site Actions" - "Site Settings" - "Modify all Site Settings" - "Search Visibility".