Part 1 - The BasicsPart 2 - The Basics continuedPart 3 - ConfigurationPart 4 - CommunicationPart 5 - Quick DeploymentPart 6 - LoggingPart 7 - Change Token BasicsPart 8 - Job and Change TokenPart 9 - Job and Timerjob
The content deployment configuration in MOSS 2007 differentiates between settings that are unique for each content deployment path or job and general settings that affect the configuration of the whole farm and applies to all paths and jobs.
This section will cover the general settings.
Access to the content deployment configuration information is provided through the following class:
The content deployment general settings are stored in the configuration database of the farm.
It should be noted that only part of the settings can be configured through the Central Administration UI. Some settings have to be done in the web.config of the central administration website others can only be adjusted using custom code.
After installing MOSS 2007 you will find the content deployment general settings on the operations tab in the central administration:
Content Deployment Settings
The following settings can be configured using the Content Deployment Settings page in the UI:
This option allows farm administrators to decide whether or not this farm can be used as the target of content deployment operations. If this option is set to "Reject incoming content deployment jobs" then the current farm can act only as the source for content deployment but other farms cannot deploy content to this farm.
The default setting is "Reject incoming content deployment jobs".
Each farm has to define one server that acts as the import server for incoming content deployment jobs.
Import operations can impact the performance of a server significantly (for example, the memory consumption can be pretty high), therefore it is a good practice to use a dedicated server to perform the import operation.
It is required that the specified server has the central administration Web site provisioned. It should be noted that with older hotfix levels the UI does not verify if the central administration Web site has really been provisioned on the selected server. If a server has been selected that does not have the central administration Web site provisioned, content deployment will fail.
The default setting for this option will be the server that hosts the instance of the Central Administration Web used to render the page.
Each farm has to define one server that acts as the export server for outgoing content deployment jobs.
Export operations can impact the performance of a server significantly (for example, the memory consumption can be pretty high), therefore it is a good practice to use a dedicated server to perform the export operation.
It is required that the specified server has the central administration Web site provisioned. It should be noted that with older hotfix levels the UI does not verify if the central administration Web site has really been provisioned on the selected server. In the case where a server has been selected that does not have the central administration Web site provisioned, content deployment jobs will not start and will remain in the "Preparing" phase.
The default setting for this option will be the server that hosts the Central Administration Web site.
This is a setting that impacts incoming content deployment jobs. If this setting is set to require encryption then SSL encryption is required for the incoming traffic. Any incoming deployment requests without SSL will be denied.
This option is usually required in the case where the upload goes over the internet or another unsecure line.
The default is that SSL encryption is required as this is the more secure setting.
This setting specifies the location of the compressed cabinet files before they get uploaded to the target farm or after they have been received from the source farm. Please note that this setting does not specify the location of the uncompressed content!
Be aware that the UI tries to create the temp directory during saving to verify if the path is accurate. This will happen on the server running the Central Administration Web site used to configure the settings - not on the configured export or import server!
In case where the export and import servers are not identical to the central admin server you are using to configure the settings the export / import process will try to create the directory during export / import if it does not exist.
So be sure to enter a path that is valid on both the export and import server and on the central admin server as no further checks will be done before executing content deployment the first time.
The uncompressed data will be stored in a directory below the directory returned from the Windows OS function, GetTempPath. GetTempPath will use the following logic to determine the location (the first hit wins):
These locations are usually found on the system drive.
Due to the above, it is a good practice to configure the TMP environment variable of the user assigned to the OWSTIMER service and the application pool of the central admin Web site to a location that has sufficient disk space.
Ensure that the TMP environment variable of the farm account points to a location with sufficient disk space on both the source and the target farm.
This setting specifies how many reports are kept for each content deployment job. The default setting is 20.
Some settings which affect content deployment have to be configured in the web.config of the central administration Web application.
Currently the following options are available:
This is a new option that has been added with build 12.0000.6315.5000.
It has been implemented to mitigate the problems that can occur when content deployment jobs are executed in parallel or if authoring activities badly interact with the export phase.
To overcome this limitation the RetryOption has been implemented which allows an export to be retried if the previous export failed. This option can either do a simple retry, which means the same export will be performed again or it can be configured to exclude the sites (SPWeb objects) that caused problems from the next attempt. This allows content deployment to be used in a limited manner where persistent problems in specific sites would cause content deployment jobs to fail.
This setting defines the default value which applies to new jobs. It is possible to specify different values for individual jobs using additional settings which will be covered later.
Possible values for this option are None (=0), SkipFailedWebs (=1), and SimpleRetry (=2).
The default value is None which means that no retry attempt is performed.
See later in the Content Deployment Job configuration settings for an example on how to configure this option in the web.config.
This option is tied to the previous option. If the RetryOption is enabled this setting defines how many times the export will be retried before the job finally ends with a failure. To ensure that the deployment jobs does not run forever if a persistent problem occurs you should avoid configuring this to a number higher than 5.
Possible Values: 1-999. A small number (3-5) is recommended
(See later in the Content Deployment Job configuration settings for an example on how to configure this option in the web.config.)
Some settings are only exposed through the object model because the product group assumed that there is no need to change the default values.
Access to the content deployment configuration information is provided through the following class: Microsoft.SharePoint.Publishing.Administration.ContentDeploymentConfiguration
In addition to the settings listed in the last two sections, this class exposes the following additional properties:
This setting allows the maximum size of each generated cab file to be defined.
The default value for this setting is 10 MB.
Please note that this does not mean that cab files will never exceed 10 MB. Content deployment does not split files over multiple cab files. As a result cab files can exceed this limit. For example, if a file in a document library that needs to be deployed cannot be compressed to a size smaller than 30 MB then the cab file will end up at least 30 MB in size.
After the import job on the target server has been started the content deployment job on the export server of the source farm polls the target farm for status updates in regular intervals. If the status does not change for the timeframe configured through this value the exporting server reports a timeout.
This affects mainly full deployment jobs which deploy a large amount of data. If in such a situation the decompressing phase on the target server takes longer than the configured RemoteTimeout values you will see such a timeout.
The default for this setting is 10 minutes.
This setting is tightly bound to the previous. It defines the interval used by the exporting server to check for status changes on the target server.
Usually it should not be required to change this setting.
The default for this setting is 10 seconds.
The following code sample demonstrates how to adjust the RemoteTimeout value. The other settings can be adjusted in a similar manner:
As discussed earlier a content deployment path defines the content flow from a single source site collection to a single target site collection.
Access to the content deployment path configuration information is provided through the following class:Microsoft.SharePoint.Publishing.Administration.ContentDeploymentPath
This class stores its data in the Content Deployment Paths list which resides in the root web of the central admin website:http://url-to-central-admin/lists/content%20deployment%20paths
Be aware that only part of the settings can be configured through Central Administration. Some additional settings can to be done through STSADM commands.
The following settings can be configured using the Content Deployment Path Settings page in Central Administration:
The Name for a content deployment path has to be unique.
The description for the content deployment path.
This setting defines the URL to the source Web application. The UI provides a drop down with the names of all Web applications on the source server. It should be noted that although the UI presents the Web application name in the drop down only the URL to the site collection is stored in the path settings.
This setting defines the server relative URL for the source site collection which resides inside the selected source web application. As soon as a source Web application has been chosen by a user the UI populates the drop down box with all server relative URLs for the selected Web application.
This setting defines the URL to a server in the target farm that hosts an instance of the central administration Web site. It does not matter if this is the Import server in the target farm or a different server as long as the central administration Web application has been provisioned on this server.
It should be noted that the protocol used in this setting defines whether the future communication to the target farm will use http or SSL.
The UI will present a warning when http is being used:
This setting defines the credentials being used to communicate with the target farm. The account being used has to have farm administrator rights on the target farm. When the same farm account is used in both farms or when the path links two site collections in the same farm the option to connect using the application pool account can be utilized.
Alternatively you can enter the credentials to the target farm and decide whether to use integrated authentication or basic.
This setting defines the URL to the target Web application.
The UI provides a drop down with the names of all Web applications on the target server which is populated as soon as the URL to the target central administration Web application has been entered and the connect button has been pressed.
It should be noted that although the UI presents the Web application name in the drop down only the URL to the site collection is stored in the path settings.
This setting defines the server relative URL for the destination site collection which resides inside the selected destination web application. As soon as a destination Web application has been chosen by a user the UI populates the drop down box with all server relative URLs for the selected Web application.
This setting is the equivalent to the SPImportUserInfoDateTimeOption setting of the underlying content deployment and migration API. It defines whether user information from the source system should be associated with the content after it has been deployed to the target.
This setting defines the level of granularity to use for security information from the source system should be deployed to the target system.
Three different levels are possible:
Some settings are only exposed through STSADM commands.
When performing a content deployment operation the exporting server first exports the content into a temporary directory on the disk and then compresses the data. After the deployment is done, the temporary files are cleaned up - more or less (we will cover this later).
This setting allows you to control in which situations the cleanup will happen. This can be interesting when it comes to troubleshooting content deployment problems. Enabling this option allows you to keep and inspect the generated cab files to identify what has been deployed.
We at Microsoft Support often use this option when working on support cases.
To modify this option please use the following STSADM command:
STSADM -o editcontentdeploymentpath -pathname <pathname> -keeptemporaryfiles Never|Always|Failure (In case your content deployment path name contains blanks please specify it with double quotes.)
Possible values: Never (=0), Always (=1), Failure (=2)
Be aware that Failure means fatal errors. In case that warning or non-fatal errors are reported in the deployment log the job would still report "Success" and the temporary files would not be preserved even if you specified "Failure" for this option.
The default is Never.
This property allows to control whether or not asynchronous event receivers are allowed to fire during the import. The default is that asynchronous event receivers are disabled during import. This configures the SPImportSettings.SuppressAfterEvents setting of the content deployment and migration API when performing the import.
Any modification to the target database that happens in parallel with an import could potentially lead to a deadlock in the SQL database which could cause content deployment import to fail. As asynchronous event receivers get executed in parallel with the import operation such deadlocks could be caused by these event handlers. That's the reason why this option is disabled by default. In general it should not be required to have these event handlers enabled in case that they should modify sharepoint content as the same actions would have fired on the source system and would have performed the modification there already. So the changes done by the event receiver will be deployed to the target system using content deployment. You should only enable after events if you are sure that these events are not modifying anything inside the sharepoint database the import runs in. E.g. if the event receivers only send emails or update content in other databases.
This option can be modified using the following STSADM command:
STSADM -o editcontentdeploymentpath -pathname <pathname> -enableeventreceivers yes | no
Possible values: No (=0), Yes (=1)
Default is No.
This property allows you to control whether the data being deployed is being compressed during the deployment or if the uncompressed data should be sent to the destination server. This setting is enabled by default to provide acceptable performance when uploading the content to a server over a slow connection. In an intranet scenario or when deploying content between two site collections on the same server, the overhead to perform compression and decompression might be higher than the additional time to transport the uncompressed data. Here the compress and uncompress time could cause a significant overhead in the performance of the content deployment jobs.
In such a situation it might make sense to disable the compression.
This setting configures the SPDeploymentSettings.FileCompression property of the content deployment and migration API during export and import.
To modify this option use the following STSADM command:
STSADM -o editcontentdeploymentpath -pathname <pathname> -enablecompression yes | no
Before disabling compression, check the version of the Microsoft.SharePoint.Publishing.dll on your disk. A known problem with disabling this option was fixed in build 12.0000.6315.5000. If your Microsoft.SharePoint.Publishing.dll has a lower build number you cannot disable this option as it would cause your content deployment job to fail.
The default value is Yes.
As discussed earlier a content deployment job defines for which part of a site collection and in which schedule content is transferred to the target site collection.
Access to the content deployment job configuration information is provided through the following class: Microsoft.SharePoint.Publishing.Administration.ContentDeploymentJob
This class stores its data in the Content Deployment Jobs list which resides in the root web of the central admin website: http://url-to-central-admin/lists/content%20deployment%25jobs
Be aware that only part of the settings can be configured through a UI. Some additional settings can to be done through web.config or STSADM commands.
The following settings can be configured using the Content Deployment Job Settings page in the UI:
The Name for a content deployment job has to be unique. Be aware that it has to be unique for all Jobs - not just for all jobs for a specific path. For example, you cannot create a job named "Deploy All" for Path 1 and a job named "Deploy All" for Path 2.
The description for the content deployment job.
This setting identifies the content deployment path that is associated with the current content deployment job. This setting is represented by a drop down in the UI which is populated with all content deployment path definitions in the current farm.
This setting defines whether the whole site collection or only specific sites should be deployed. The UI provides an additional option to show a treeview with all sites in the site collection where a user can pick the sites that should be deployed.
Here he can also decide whether to deploy the whole subtree starting at the selected site or only the selected site.
It is not possible to explicitly exclude specific sites from the deployment. To achieve this it will be required to use a combination of site and subtree inclusions which omit the part of the site collection that should not be deployed.
This setting allows the schedule for the content deployment job to be defined. This option is disabled by default which means that the content deployment job has to be executed manually. The UI provides various different options to schedule the execution of a content deployment job.
It should be noted that content deployment does not work reliably if multiple export or import operations for the same site collection are executed at the same time. That means you should schedule different jobs for the same source or target site collection so that they do not overlap.
It should also be noted that the schedule is not stored in the content deployment job itself. For each Content Deployment Job there is an assigned Timer Job (the timer job is an instance of the class Microsoft.SharePoint.Publishing.Administration.ContentDeploymentJobDefinition) which does the actual work. The schedule will be assigned to the Schedule property of this timer job.
The content deployment job object itself holds a reference to the timer job.
This setting defines if the deployment job will perform a full or an incremental deployment. This setting is equivalent to the SPExportMethodType property passed into the content deployment and migration API.
This setting defines if an email is sent when a content deployment job has succeeded and/or failed. The UI requires that an email address be provided when one of the options is enabled.
Some settings which affect a content deployment job have to be configured in the web.config of the central administration Web application.
To overcome this limitation the retry feature has been implemented which allows the export to be retried if the previous export failed. This option can either do a simple retry – means perform the same export again or it can allow excluding the sites (SPWeb objects) that caused the problem from the next attempt. This allows you to use content deployment in a limited manner in situations where persistent problems in specific sites would cause content deployment jobs to fail.
This setting defines the default value which applies to new jobs. It is possible to specify different values for individual jobs using additional settings. We will cover this later.
Possible values for this option are None (=0), SkipFailedWebs (=1), SimpleRetry (=2)
This option is tied to the previous option. If Retry is enabled this setting defines how many times the content deployment retries the export option before the job finally ends with a Failure.
To ensure that the deployment jobs does not run forever if a persistent problem occurs you should avoid configuring this to a number higher than 5.
The following lines have to be added to the web.config to enable the retry options:
<option> can be one of the following: None, SkipFailedWebs, SimpleRetry
<number-of-retries> can be an integer between 1 and 999
If you don't add any <Job...> elements, then all new jobs will get the values configured as DefaultOption and DefaultRetries. Using the <Job...> element you can configure different jobs with different values.
DefaultOption and DefaultRetries are also the entries that get added to the Content Deployment general settings which we discussed in an earlier section.
After you have added the above listed configuration to the web.config of the Central Administration you have to open the content deployment job definition once in the central administration and save it again. During this save the configured options will be silently added to the content deployment job.
In addition you need to open and save the content deployment general settings page to ensure that the default settings are updated as well. New jobs created after you changed the web.config will automatically get the configured values.
As outlined earlier in this article the settings visible on the content deployment general settings page are not persisted in the database until explicitelly saved using the OK button.
Problems will occur in situations where the settings haven't been saved (default settings used) and later an option is adjusted using object model. E.g. when the RemoteTimeout value is adjusted.
In this situation the settings will be populated in context of the current machine and current user. That means that potentially the machine where the code is executed will become the export and import server of the farm. In addition the path for the temporary files will not be populated with c:\windows\temp but with c:\documents and settings\<username>\local settings\temp.
Often the user adjusting the timeout value is not the same account as the farm account of the sharepoint farm. That means when content deployment later tries to create the temporary files it will try to create them at a location where the farmaccount does not have rights to and the deployment will fail.
If the user configuring the content deployment job does not have read permission in the source site collection he cannot configure the job for selective deployment. The reason is that the treeview which allows a user to select the sites that should be deployed is created by enumerating the site structure in the source database in context of the current user. If this user does not have rights to see these sites or some of these sites they will not be included in the tree view. In case that the user does not have any rights in the source site collection the dialog will show an access denied message.
I always wanted to thank you and to tell how much we appreciate all your work. Your articles are a real teasor for the SharePoint Community.
Ever Since I began using content deployment, a year ago, I always have these warnings in my deployment job report, even if the operation succeeds :
(Webpart or web control is not found or not marked as safe)
Back then, it was not a priority to solve these warnings, but now I want to get rid of them.
When I look a the exported files Manifest.xml, I see lines like :
<SPObject Id="829d0433-f2f7-4f05-8b11-bdee3a7a84f1" ObjectType="SPFile" ParentId="6eaff48f-c5be-408d-8cc2-4ab326803912" ParentWebId="c84e83c2-321f-4081-8741-d874799362e1" ParentWebUrl="...." Url="..../Forms/AllItems.aspx">
<File Url="Documents/Forms/AllItems.aspx" Id="829d0433-f2f7-4f05-8b11-bdee3a7a84f1" ParentWebId="c84e83c2-321f-4081-8741-d874799362e1" ParentWebUrl="...." Name="AllItems.aspx" ListId="62555720-0a33-48ec-b48e-b78817e15675" ParentId="6eaff48f-c5be-408d-8cc2-4ab326803912" TimeCreated="2009-09-08T20:30:55" TimeLastModified="2009-09-08T20:30:55" Version="1.0" IsGhosted="true" SetupPath="pages\viewpage.aspx" SetupPathVersion="3" SetupPathUser="1073741823" FileValue="00000587.dat">
<WebPart Name="e7b34df6-b05f-47db-a659-077980756d62" Flags="25165825" Hidden="false" Threaded="false" ReadOnly="false" RecurrenceRowset="false" FPModified="false" ModerationType="" Personal="false" OrderedView="false" ListId="62555720-0a33-48ec-b48e-b78817e15675" Version="1" DisplayName="Tous les documents" BaseViewID="1" Level="major" WebPartZoneID="Main" WebPartTypeId="2242cce6-491a-657a-c8ee-b10a2a993eda" IsIncluded="true" WebPartOrder="1" Type="0" FrameState="0"><ViewFields><FieldRef Name="DocIcon"/><FieldRef Name="LinkFilename"/><FieldRef Name="Modified"/><FieldRef Name="Editor"/><FieldRef Name="_ModerationStatus"/><FieldRef Name="PublishingStartDate"/><FieldRef Name="PublishingExpirationDate"/><FieldRef Name="CheckoutUser"/></ViewFields><Toolbar Type="Standard"/></WebPart>
All the SPObjects with reference to the missing webparts are OOTB files such as Allitems.aspx, ViewPage.aspx, etc.
Furthermore, the webparts are included in the Requirement.xml file :
<Requirement Type="WebPart" Id="[Unknown: 2242cce6-491a-657a-c8ee-b10a2a993eda]" Name="2242cce6-491a-657a-c8ee-b10a2a993eda" />
<Requirement Type="WebPart" Id="[Unknown: 293e8d0e-486f-e21e-40e3-75bfb77202de]" Name="293e8d0e-486f-e21e-40e3-75bfb77202de" />
I wonder if you have any idea of what these webparts are all about.
during import the requirements are checked. One of the requirements is to check if all the webparts being imported are listed in the SafeControl section of the target web application.
This check is done on the importing server.
That means you have to check if the web.config of the target web application on the server configured as the import server of your target farm contains these web parts.
A common reason for this issue is that the central admin server used as import server is not a web front end server for the farm.
Then this will always happen as the web.config file cannot be found on the import server.
Thank you for your quick response. I understand what you say. The problem is that I cannot find any webpart with the ID [2242cce6-491a-657a-c8ee-b10a2a993eda] or [293e8d0e-486f-e21e-40e3-75bfb77202de] on my farm and to which assembly they belong. The only reference on the net is this link : http://msdn.microsoft.com/en-us/library/ms481179.aspx
You are right, the export server is not a WFE. I compared the WFE web.config with the application server (export server) web.config and I did not find any differnce with regards to Assemblies and SafeControls.
I think I forgot to mention that the warnings I'm talking about show during the export and of course during the import. Sorry!
if the errors occur during export, then please open a support case with Microsoft.
regarding the web.config: how can the web.config be the same? As the server is not a WFE there will be no web.config for the source site collection.
But as indicated: web.config check is only done on import.
During export the system compares the webparts used in the site collection against the webparts installedin the farm.
It seems your farm misses some web parts.
I don't understand you when you say: "As the server is not a WFE there will be no web.config for the source site collection". When we create a web application it's IIS web site (and web.config) is created on the application server and on the WFEs. It's true though, that the web site on the application server is not availabale to users.
Here is the exact warning during the export :
<A Web Part or Web Form Control type could not be found, or is not registered as safe. The Web Part will still be exported. [Unknown: 2242cce6-491a-657a-c8ee-b10a2a993eda]>
Since during export the system compares the webparts used in the site collection against the webparts installed in the farm, the question is : How did SharePoint create a site collection with webparts that are not on the farm? The only thing I can suspect is may be that all happened during the migration from MCMS 2002 to SharePoint.
that means that your application server is also a WFE.
Otherwise you would not get the IIS directories.
Amazing insight on Content Deployment, this is perhaps the most comprehensive post series on Content Deployment that I have read till date.
Stefan, keep up the good work. I am the co-founder of www.SharePointInsights.com.
I welcome you onboard to share your technical insights at our site: www.SharePointInsights.com.
I have a question....guessing I am just having a brain cramp...
With regards to disabling compression: I ran
STSADM -o editcontentdeploymentpath -pathname <sitecollectionurl> -enablecompression no
I get the following error:
Failed to update the content deployment path named...https://sitecollectionurl.
am I using the wrong pathname? I am also using the Admin account.
this needs to be the name of the content deployment Path and not an URL path.
So the content of the name property you used when creating the content deployment path on the content deployment paths and jobs page.
Great thank you....Brain cramp on Wednesday is not a good thing.
Stefan, have you by chance seen this error when running Content Deployment?
Content deployment job 'incremental' failed.The exception thrown was 'System.OutOfMemoryException' : 'Exception of type 'System.OutOfMemoryException' was thrown.'
I assume you have a 32-bit machine. 32-bit environments have a limitation of 2 GB user mode virtual memory per process. As content deployment is a very memory hungry application it is recommended to use 64-bit.
I have read online where I can extend the timeout period for Content Deployment. What I found appears to be VB or C#. I have neither apps available to me, is there another way to extend this timeout period? Sorry I am not a developer, so please forgive my ignorance.