Stefan Goßner

Senior Escalation Engineer for SharePoint (WSS, SPS, MOSS, SP2010) and MCMS

Deep Dive into the SharePoint Content Deployment and Migration API - Part 6

Deep Dive into the SharePoint Content Deployment and Migration API - Part 6

  • Comments 62
  • Likes

[Part 1 - Part 2 - Part 3 - Part 4 - Part 5 - Part 6 - Part 7]

Requirements for a successful content deployment

Be aware that this article cannot provide a complete list of all requirements. I will update it from time to time if required. Today I will list all the requirements I have seen in the past which - when missed - can cause content deployment jobs to fail.
 

1) The servers configured as export and import server need to host an instance of the Central Administration website

When configuring the "Content Deployment Settings" for your farm you have the chance to select different servers in your farm to work as export and import servers for the content deployment. This allows to offload this task to a dedictated server to reduce the load on your web frontend servers.

The comments of this configuration options look as follows (here shown for the Import server):

Import Server
Specify the server you want to receive incoming content deployment jobs. This server must have enough available disk space to store the incoming jobs, and it must be running an administration Web application for the farm.

So this requirement is actually listed but a little bit hidden in the second half of a sentence.

In case you configure a server that does not host the Central Admin website then you will not get a warning here!

Impact: The effect you will see is that the content deployment export or import phase will not start.

How to resolve:
1) Provision the Central Administration website on the desired server
2) Change the configuration of the export and/or import server
 

2) Ensure that sufficient disk space is available

This sounds like a simple prerequesit but it isn't. Content deployment uses different places to store the extracted and compressed files. The compressed files are stored in the location you can configure on the "Content Deployment Settings" page in the "Temporary Files" row.

But before export creates the compressed files it first exports everything into a temporary directory. And this directory is placed inside the directory which is configured in the TMP environment variable of the user the Windows SharePoint Services Timer service (OWSTIMER.EXE) runs on (usually refered to as farm credentials). Per default this variable has the following value: "%USERPROFILE%\Local Settings\Temp" which is usually on your system drive.

Impact: So per default MOSS content deployment requires the disk space for the uncompressed exported files on your system drive!

How to resolve: The easiest way to resolve this is to logon to the machine with the farm credentials and adjust the TMP variable to point to a different location. Afterwards you would need to restart the OWSTIMER service.
 

3) Use an empty site collection as the destination of your content deployment job

As already discussed in Part 5 content deployment will fail if the destination database contains conflicting content. To avoid this it is required that the initial deployment is done into an empty site collection.

Be aware that the only way to create an empty site collection is to use the following STSADM command:

STSADM.EXE -o createsite -url <url-to-site-collection> -ownerlogin domain\user -owneremail <email-address>

Using the "Blank Site" template will NOT create an empty site collection! It will actually create a site collection with content. You can see the difference if you create a site collection using both methods and then inspect the content of the created sites using SharePoint designer.

I personally recommed to always use the STSADM command with the syntax above to ensure that you really have an empty site collection as destination.

Impact: If the site collection has been created using a different method or already contains data the content deployment job will fail.

How to resolve: Deploy into an empty site collection
  

4) Install all required features for your site on the destination server

If your site requires custom features ensure that the features are installed on the destination server before running content deployment.

Impact: If the are missing the import phase will fail.

How to identify this: I have written the tool WssAnalyzeFeatures which allows you to identify such problems.

How to resolve: Copy the features to the destination server and install them using STSADM -o installfeature. 


5) Do not activate custom features for your site collection on the destination server manually

You should not activate custom features on the destination server if this activation creates content in the destination database as this can cause conflicts as outlined in "Problem 1" of Part 5 of this article series. Instead you should run content deployment and let the import process activate the features on your destination server as this will ensure that all items get created using the same ids as on the source server which is otherwise not guaranteed.

Impact: If the features have been activated and the content deployment import can fail with similar error messages in as "Problem 1" of Part 5.

How to resolve: Deactivate the feature in the destination site and ensure that all items created by the feature are removed. Alternatively do a full deployment into an empty site collection instead.
 

6) Do not expect that incremental deployment will deactivate features in the destination server site collection

The content deployment and migration API was not designed to deactivate features on the destination server. If a feature needs to be deactivated on the destination server you need to manually perform this deactivation.


7) Ensure that all feature definitions of features activated on the site collection exist on the source server

This is actually a high call generated for Microsoft Support Services: in the development a feature becomes obsolete and is removed or replaced with a different version with a different Guid but on some sites or site collection the old feature is still activated.

Impact: The affected sites can no longer be exported. You will get the errors listed in this article.

How to identify this: I have written the tool WssAnalyzeFeatures which allows you to identify such problems.

How to resolve: Either copy the missing feature files to the required location or uninstall the feature using STSADM -o deactivatefeature/uninstallfeature. In case that STSADM -o deactivatefeature fails to deactivate the feature you can use my tool WssRemoveFeatureFromSite.
 

8) Configure the retention period of the change log to be long enough for incremental deployment

See here for details.
 

9) Ensure that content deployment jobs do not run in parallel

The current implementation of the content deployment and migration API does not allow parallel execution. There are plans to change this behavior in the near future but as is you need to ensure that only one deployment is running at a time.

So if you have multiple deployment paths and jobs for the same site collection you need to ensure to schedule them in a way that they don't overlap.

But this is not the only place to look at! Sometimes it is nearly impossible to prevent parallel execution. Just think that the content deployment and migration API is not only used to deploy content between different webfarms. The same API is used in the copy/move implementation inside site manager and in the variation feature.

With other words: you can experience problems with content deployment if an author copies a page at the same time or creates a new page in the source variation label which is then replicated to the destination. And also vice versa a copy operation can fail because a quick deploy job was running at the same time.

Impact: parallel execution of deployment jobs can lead to failing content deployment.

How to resolve: you need to restart the failed deployment job

10) Ensure that the patch level on source and destination farm is identical

Content deployment is only supported if the WSS and MOSS patch level is identical on source and destination. Some hotfixes have changed the schema of the export packages slightly which can cause deployments between different patch levels to fail.

Impact: content deployment and STSADM -o export/import can fail if the patch level is not identical

How to resolve: ensure that both farms are on the same WSS and MOSS patch level

11) Required Language Packs need to be available

Language Packs used in the source site collection have to be installed on the target farm as well. 

Impact: if a required language is missing content deployment will fail

How to resolve: install the required language packs on the target farm


12) Avoid the problems listed in Part 5 of this article series.

Comments
  • PingBack from http://blogs.technet.com/stefan_gossner/archive/2007/08/30/deep-dive-into-the-sharepoint-content-deployment-and-migration-api-part-1.aspx

  • Thanks for posting this.  I was about to write up something similar but now it will be easier to just point to your blog.

    A few additional items that I would consider adding are:

    1.  Be careful about the frequency of deployment jobs.  Content deployment has a severe impact of performance if your site heavily uses cross list queries, CQWP, or the object cache.  Deployment wipes clean the object cache.

    2.  Be careful about how many jobs you run not just their frequency.  for the same reason as number 1. Sometimes its necessary break down deployment into smaller jobs to get initial deployment.

    3.  Write a script to change the default timeout for transport/import to a larger number.  Default is 10 seconds and on a slow netwrok it will kill deployment.

    4.  Be careful about full deployments.  Your database size will grow with each full deployment.

    5.  Evaluate whether you want versions turned off at the destination.  If so write a script to turn it off.  Full deployment will turn it on and grow the database again.  So be careful.

    I am sure I will think up of a few more :-).  

    Thanks for this series of articles.  They have been very useful.

    Ranjan

  • Hi Ranjan,

    that is great input!

    Maybe you could post the important pieces of the scripts you mentioned as well?

    And keep your input coming!

    :-)

    Cheers,

    Stefan

  • Content Deployment - Getting it Right

  • Content Deployment - Getting it Right

  • Hi Stefan,

    First of all I would like to thank you for the great series of articles, they were useful and direct to the point.

    Now I have successfully exported/imported my SharPoint site from one server to another one, all the contents were moved but I faced 2 points:

    1. I faced error in my custom master page, although I can see it in the designer that it’s there but when I set my custom master page as the site master page it gives this error: File not found.  What’s the reason behind this error although the page was migrated as I saw in the designer but it shows the above error when I select it.  Any clue??

    2. Another point is the security migration, when I migrate the security groups it stars find during the import process then stopped with this error: User or group 23 cannot be resolved.

    Below is part of the log file related to the error:

    [5/11/2008 3:04:18 PM]: Progress: Importing Group NAS Owners.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 1.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 11.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 13.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 14.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 15.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 16.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 30.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group NAS Visitors.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group NAS Members.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Style Resource Readers.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 7.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Designers.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 1.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 10.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 16.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hierarchy Managers.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Approvers.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 1.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 14.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Restricted Readers.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Quick Deploy Users.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Duty Officer.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Staff Agent.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Operational Admin.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Shift Supervisor.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Financial Admin.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Marketing Admin.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Corporate Admin.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Hala Online Customer.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 17.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 18.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 19.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 21.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 22.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 23.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 24.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 25.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 26.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 31.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 32.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 33.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 34.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 35.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 36.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 37.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 38.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 39.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 40.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 41.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 42.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 43.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 44.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 46.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 47.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 48.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 49.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 50.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 55.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 56.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 57.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 60.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 61.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 62.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 63.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 64.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 65.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 66.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 72.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group NAS Survey Subscribers.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 17.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 18.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 19.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 21.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 22.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 34.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 44.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 54.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 55.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 65.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 72.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Editor.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 1.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 27.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 70.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Content Supervisor.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 67.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 73.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 74.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group News–FAQs–Polls Supervisor.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 9.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 54.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group Nassir Group.

    [5/11/2008 3:04:18 PM]: Progress: Importing Group member 1.

    [5/11/2008 3:04:18 PM]: Progress: Starting content import.

    [5/11/2008 3:04:18 PM]: Progress: De-Serializing Objects to Database.

    [5/11/2008 3:04:19 PM]: Warning: User or group 23 cannot be resolved.

    [5/11/2008 3:04:19 PM]: FatalError: User or group 23 cannot be resolved.

      at Microsoft.SharePoint.Deployment.DeploymentLogger.Log(DeploymentLogSeverity severity, SPDeploymentObject deplObject, String message, Boolean throwException)

      at Microsoft.SharePoint.Deployment.DeploymentLogger.Log(DeploymentLogSeverity severity, String message)

      at Microsoft.SharePoint.Deployment.ImportObjectManager.ConvertToDestUserId(Int32 userid, Boolean bAllowFallback, Boolean& bIsGroup)

      at Microsoft.SharePoint.Deployment.WebSerializer.DealWithGroupsOnImport(ImportObjectManager objectManager, DictionaryEntry property)

      at Microsoft.SharePoint.Deployment.WebSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)

      at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)

      at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)

      at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)

      at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)

      at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)

      at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()

      at Microsoft.SharePoint.Deployment.SPImport.Run()

    [5/11/2008 3:04:19 PM]: Progress: Import Completed.

    [5/11/2008 3:04:19 PM]: Finish Time: 5/11/2008 3:04:19 PM.

    [5/11/2008 3:04:19 PM]: Completed with 1 warnings.

    [5/11/2008 3:04:19 PM]: Completed with 1 errors.

    Any clue why this is happening?? For the time being I did import without security until I solve the master page problem.

    Regards,

    Hamzeh Ayesh

  • Hi Hamzeh,

    please open a support case so that your specific problem can be analyzed in more details.

    Cheers,

    Stefan

  • Stefan

    Is there a "best practice" for deleting list items or other pages when it comes to content deployment?  If list times are deleted on the source server will content deployment delete that same item on the destination server?  My experience working with MCMS and Site manager 2002 showed me that it was better to "expire" postings rather than delete them as our replication process could not delete postings or channels on the destination server.

    Thanks

    Franco

  • Hi Franco,

    incremental deployment will take care of deleted items. Full deployment will not take care.

    With MCMS it was the same. Incremental deployment reliably deleted items on the destination.

    Cheers,

    Stefan

  • Hi Stefan

    Thanks for your previous response.

    I read through items 3,4,5,6 and 7 of part 6.  I created a blank site using stsadm, ran a full content deployment successfully and scheduled incremental deployment.  However, a number of features on the destination server are still not activated despite being activated on the source server.

    Is there a way I can activate them without having to re-create the site collection and run a full content deployment?  This did not seem to work this first time.

    Thanks

    Franco

  • Hi Franco,

    are these standard features coming with WSS/MOSS or custom features?

    Cheers,

    Stefan

  • Stefan

    These are custom features developed by the vendor.

    Thanks

    Franco

  • Hi Franco,

    it is hard to tell what's going on here without seeing a repro. I would suggest to open a support case to get this problem analyzed in more detail.

    Cheers,

    Stefan

  • Hi Stefan

    Great articles

    In our system we have an authoring and staging site collection on the same box (different web app).

    The first time we run a full deployment from auth to stage the files are OK, however then next time many files (css, master pages) are unghosted and have been reverted to a previous version of the files - as if they are pulled from some cache somewhere.

    Any advice on how to sort this out?

    Thanks

  • Hi Mark,

    a couple of days two hotfixes have been released which address a similar problem.

    Please open a support call with Microsoft and request the followig two hotfixes:

    952698, 952704

    Cheers,

    Stefan

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
Raw Html Fix