Stefan Goßner

ULS Log

Both Content Deployment and the Content Deployment and Migration API use the ULS log to report problems and progress information using the following categories:

For the WSS Content Deployment and Migration API:

• Category: Upgrade

For the MOSS Content Deployment feature:

• Category: Content Deployment

To troubleshoot, content deployment issues these two categories should be set to Verbose:

This will ensure that the whole progress of content deployment is logged into the ULS logs.

In some situations it might be required to add additional categories (such as "General") as the content deployment and migration API uses the WSS object model during export and import. If a problem occurs inside the WSS object model the error will show up in the related category first.

Export/Import log

An often requested feature is to get the same log files as you will automatically get them using STSADM -o export/import.

There are two possible ways to get this log file for content deployment:

1. Install October 2009 Cumulative Update for WSS and MOSS

   If you have this patch level or later these logfiles are always automatically generated and can be found on the export server of the source farm. The location of these logfiles can be found in the content deployment job report:

   

   The name in front of the path is the machine name where the logfiles can be found. The logfiles are stored on this server in cab file format.

   The directory will be in the ImportExportLogs directory which will be created in the directory configured as temporary path in the content deployment general settings.

   Starting with October 2009 CU these log files will always be created and it is required to manually clean them up to free disk space.

2. The second option is available also with previous builds. If you enable verbose logging for the "Upgrade" category you will find the entries which are usually written to the export/import logs in the ULS log. You only have to filter the logfile for the "Upgrade" category

   The problem with this method is that you will only see the entries AFTER you changed the logging option to verbose. So you will usually not have this information available when the deployment failed the first time.

Quick Deployment Concepts

Quick deployment enables authors to select specific items to be quick deployed as soon as possible. The problem here is that authors are usually not allow to start timer jobs. To overcome this limitation it is required to store the information about which items have to be quick deployed at a place where a normal author of the site collection has write permission. And this is a list inside the source site collection. The quick deploy job that performs the actual deployment has to read the information about the items to be quick deployed from this list when it runs.

As multiple different quick deploy jobs which can run with different schedules can originate at the same source site collection it is required that the items are flagged for each of quick deploy job separatelly. So it is required that the author knows about all the enabled quick deploy jobs. As the author does not have rights to enumerate all quick deploy jobs it is required to store the information about all enabled quick deploy jobs at a place where the author can read. And this is in the property bag of the root site of the source site collection.

Enable Quick Deployment

To enable quick deployment you need to modify the settings of the Quick Deploy Job listed on the Manage Content Deployment Paths and Jobs page.

To enable Quick Deployment you need to check the "Allow Quick Deploy jobs along this path" option.

As soon as a quick deployment job for a path is enabled two things will happen:

1. The job will be marked as enabled by adjusting a field in the list item relate to this quick deploy job in the Content Deployment Jobs list if the root site of the central admin web site.
2. The unique Id of the quick deploy job is added to the quickdeployjobs property in the property bag of the root web of the source site collection which stores the Ids for all enabled quick deploy jobs in XML format.

Allow users to perform Quick Deployment

To be allowed to perform quick deployment, a user needs to have rights to create new items in the hidden Quick Deploy Items list in the root site of the site collection. This list does not inherit permissions by default and only site collection administrators and members of the Quick Deploy Users group have the required permissions.

Be aware that the title of this list is localized. In a german system (e.g.) the title will be Elemente für schnelle Inhaltbereitstellung. But the URL will always be http://url-to-site-collection/Quick%20Deploy%20Items.

The recommended way to give users permissions to perform a quick deployment is to add them to the Quick Deploy Users group.

Perform a quick deployment

The following prerequisites need to be fulfilled to allow a user to perform a quick deploy:

1. The user has contributor rights on the Quick Deploy Items list - means he is a member of the Quick Deploy Users group.
2. At least one quick deploy job is enabled for the site collection - means that at least one unique id for a quick deploy job has been added to the quickdeployjobs key.
3. The page is in a valid state to be quick deployed - means that the latest revision of the page is a major version or an approved minor version that is scheduled to go live in the future.
4. No save conflict has happened.

A user can perform a quick deployment of a publishing page using the console provided by the publishing feature. This action creates an entry in the Quick Deploy Items list (which resides in the root Web of the source site collection) for each of the enabled quick deployment jobs providing the server relative URL to the item that has been selected for quick deployment.

That means if the site collection is the source for three content deployment paths where each has an enabled quick deployment job you will get three list items per quick deployed publishing page.

As soon as the quick deploy job gets executed it will retrieve the list of all the items flagged for this specific quick deploy job and deploy them. After a successful quick deploy the deployed items are removed from the Quick Deploy Items list.

Potential problems related to Quick Deployment

Possible Problems when enabling a quick deploy job

As outlined earlier in this section the quickdeployjobs property in the property bag of the source site collection has to be adjusted to enable a quick deploy job. Problems will occur in a situation that the user trying to enable the quick deploy job does not have rights to adjust the value of this property.

In this situation the user will see an access denied and an option to enter credentials of a different user. Using the "Back" button in the browser will allow the user to go back to the content deployment paths and jobs page and here he can see that the quick deployment job is marked as enabled now.

The problem is that only part of the enabling has happened: the job itself has been flagged as enabled by adjusting the field in the list item in the content deployment jobs list in the central admin web site but the unique Id of the job has not been added to the property bag of the source site collection.

The result of this inconsistency will be that the items being quick deployed will not be added to the quick deploy items list for this specific quick deploy job.

In case that this is the only quick deploy job for this site collection it will also mean that the Quick Deploy option in the authoring toolbar will remain disabled.

The following method is very effective to resolve this issue:

• Disable the quick deploy job
• Ensure that the user has rights to update the property (e.g. by adding him to the site collection administrator group)
• Enable the quick deploy job again

Possible Problems with backup/restore

In case that quick deploy has been enabled for a site collection before taking a backup means that the property bag of the root site of the site collection contains the unique id of a quick deploy job of this farm.

After restoring the backup in a different farm means that the quickdeployjobs property now contains the unique id for a job that does not exist in the current farm.

Problems with this scenario will occur as soon as quick deployment is enabled for this site collection again in the new farm. What happens now is that whenever an author selects a page for quick deployment two entries will be placed in the quick deploy items list: one for the job in the new farm and one for the job that only existed in the original farm.

The items for the job in the new farm will get deployed and afterwards these items will be removed from the quick deploy items list.

But the items that have been added for the job that does not exist in the current farm will remain in the list forever (or at least till they get cleaned up manually).

In the past I have had a couple of support cases where several ten-thousand of these orphaned entries existed in the quick deploy items list.

Even cleaning up such a huge list is challenging and time consuming.

To correct this it would be required to remove the invalid entries from the quickdeployjobs property in the property bag of the source site collection and afterwards it would be required to delete all entries from the quick deploy items list which do not belong to valid quick deploy jobs.

The following method is very effective to resolve this issue:

• Manually start all quick deploy jobs for the site collection to ensure that the quick deploy items list gets cleared from valid entries
• Disable all quick deploy jobs to ensure that the ids of the valid quick deploy jobs are removed from the quickdeployjobs property in the property bag of the source site collection
• Delete the quickdeployjobs property using object model (it will be recreated when enabling the quick deploy jobs later) (see code sample below)
• Delete all entries in the Quick Deploy Items list (only invalid items should be in the list by now) either manually or using object model! Important: do not delete the list itself! Only delete the items in this list!
• Reenable all quick deploy jobs for the source site collection which will recreate the quickdeployjobs property and add the unique ids of the valid quick deploy jobs back to the property.

Code sample

The following code sample demonstrate how to enumerate all enabled quick deploy jobs in the system and dump the unique ID of these quick deploy jobs:

The following code sample demonstrates how to dump the content of the quickdeployjobs key for a given site collection:

The result emitted by the above listed code piece will look similar to this:

Each guid in the property represents a different quick deploy job. 

To delete the quickdeployjobs key (see backup/restore problem above) you can use the following code:

Content Deployment uses the http (or https) protocol for communication between source and target farm. Most of the communication is performed through Web service calls except for the file transfer of the exported files to the target farms which is done using a regular http POST request against an ASPX page to perform the upload operation.

Communication inside the same farm (between worker process and timer service) is implemented by writing to and reading from the configuration database (starting and stopping of timer jobs) and central administration content database (job status information) of the farm.

Communication through a Web Service

This Web service is hosted within the central administration Web application at the following URL:

http://target-central-admin-url/_vti_adm/ContentDeploymentRemoteImport.asmx

It provides the following operations:

GetVirtualServersInformation

This operation is used during the configuration of a content deployment path. It allows enumerating all SharePoint Web applications on the target farm which will then be presented in the UI to allow a user to select it as target for the content deployment path.

GetSiteCollectionNames

This operation is also used during configuration of a content deployment path. It allows enumerating all site collections on the target farm for a given virtual server which will then be presented in the UI to allow a user to select it as a target for the content deployment path.

GetRemoteAdminServerUrl

When configuring content deployment in a farm you have to define one server that has to act as the importing server for content deployment operations. The GetRemoteAdminServerUrl operation returns the information about this server to the caller.

This operation is called as well during execution of a content deployment job to identify which server in the target farm will perform the import.

CreateJob

This operation is called by an executing content deployment job on the source farm. It allows the creation of the shadow job on the target farm that will perform the import operation.

RunJob

This operation is called by an executing content deployment job on the source farm to start an earlier created shadow content deployment job on the target farm. This job will perform the import operation.

GetJobStatus

This operation is called by an executing content deployment job on the source farm in a configurable interval (default: 10 seconds) to get the status of the import operation from the target farm.

It should be noted that refreshing the content deployment status page does not necessarily result in a request for status to the target farm.

It should be noted that if the returned status does not change for a configurable timeframe (default: 10 minutes) the content deployment job on the source farm will report a timeout – independent whether or not the import operation later finishes successfully.

This affects mainly full deployment jobs which deploy a large amount of data. In this situation the decompressing phase on the target server can take longer than 10 minutes which will result in such a timeout.

CancelJob

This operation is called by an executing content deployment job on the source farm to cancel its shadow content deployment job on the target farm when a user requested the cancellation of the content deployment through the UI. Be aware that such a cancel operation can only be performed if the decompressing operation has not yet started.

DeleteJob

This operation is called by an executing content deployment job on the source farm to delete its shadow content deployment job on the target farm after the deployment has been completed.

File transfer through http POST

File transfer of the export packages to the target farm is performed through an http POST request to the following page which is hosted within the central administration Web application:

http://target-central-admin-url/_admin/Content%20Deployment/DeploymentUpload.aspx

The uploaded file is sent as payload of the POST request. The information about the name of the file and the shadow content deployment job the uploaded file belongs to is provided through query string parameters. The actual URL sent with the http request would look similar to this:

http://target-central-admin-url/_admin/Content Deployment/DeploymentUpload.aspx?filename="ExportedFiles.cab"&remoteJobId="49bebd7d-62d0-4d68-a1f0-9118b3ac4416"

Content Deployment Communication Flow

To describe process used during content deployment we will use following diagram. Be aware that communication in the same farm (red arrows) is implemented by writing to/reading from the sharepoint configuration and central administration content database. Communication between farms happens through the web services (dark blue arrows) and the http post request to do the file uploaded (light blue arrow):

When a content deployment job is started through the central admin UI a new one time timer job is created and executed (1). Alternatively the preconfigured timer job for a scheduled deployment job can be started by the timer service manually.

The timer job exports the configured content into a temp directory and compresses it into one or multiple cab files.

When the export is finished the timerjob calls the GetRemoteAdminServerUrl operation of the ContentDeploymentRemoteImport Web service to get the information about the import server to use (2).

The next step is that the timer job calls the CreateJob operation of the ContentDeploymentRemoteImport Web service to create the shadow content deployment job and the associated one time import timer job on the target farm. This job will remain in a stopped state (3/4).

Afterwards the timer job on the source farm uploads the exported cab files to the temp directory on the import server in the target farm using http upload to the DeploymentUpload.aspx (5).

After the upload is completed the timer job on the source farm calls the RunJob operation of the ContentDeploymentRemoteImport Web service to start the import timer job on the target farm (3/4).

While the import is running the import timer job will update the status of the deployment in the associated content deployment job list item.

Every 10 seconds (configurable though object model by setting the RemotePollingInterval setting – see part 2) the timer job on the source farm will call the GetJobStatus operation of the ContentDeploymentRemoteImport Web service to retrieve the status of the import operation (3/6). The timer job on the source farm will update the status in the content deployment job list item with the status received from the target farm. If the update hasn't changed for 600 seconds (configurable though object model by setting the RemoteTimeout setting – see part 2) the timer job on the source farm assumes that something went wrong and will report a timeout.

When the timer job on the source server receives the information that the import operation has completed (either succeeded or failed) it will delete the shadow deployment job and the associated timer job through a call to the DeleteJob operations of the ContentDeploymentRemoteImport Web service (3/4)

The central administration retrieves the status of the timer job by looking at the content deployment job and job report list item which are updated by the timer service on the source farm (7). Every refresh of the page only reads the status in the local content deployment job list item. No call to the target farm to get a status update is done when refreshing this page.

Content Deployment General Settings

The content deployment configuration in MOSS 2007 differentiates between settings that are unique for each content deployment path or job and general settings that affect the configuration of the whole farm and applies to all paths and jobs.

This section will cover the general settings.

Access to the content deployment configuration information is provided through the following class:

Microsoft.SharePoint.Publishing.Administration.ContentDeploymentConfiguration

The content deployment general settings are stored in the configuration database of the farm.

It should be noted that only part of the settings can be configured through the Central Administration UI. Some settings have to be done in the web.config of the central administration website others can only be adjusted using custom code.

Settings configurable through the UI

After installing MOSS 2007 you will find the content deployment general settings on the operations tab in the central administration:

Content Deployment Settings
          

The following settings can be configured using the Content Deployment Settings page in the UI:

Accept Content Deployment Jobs

This option allows farm administrators to decide whether or not this farm can be used as the target of content deployment operations. If this option is set to "Reject incoming content deployment jobs" then the current farm can act only as the source for content deployment but other farms cannot deploy content to this farm.

The default setting is "Reject incoming content deployment jobs".

Import Server

Each farm has to define one server that acts as the import server for incoming content deployment jobs.

Import operations can impact the performance of a server significantly (for example, the memory consumption can be pretty high), therefore it is a good practice to use a dedicated server to perform the import operation.

It is required that the specified server has the central administration Web site provisioned. It should be noted that with older hotfix levels the UI does not verify if the central administration Web site has really been provisioned on the selected server. If a server has been selected that does not have the central administration Web site provisioned, content deployment will fail.

The default setting for this option will be the server that hosts the instance of the Central Administration Web used to render the page.

Export Server

Each farm has to define one server that acts as the export server for outgoing content deployment jobs.

Export operations can impact the performance of a server significantly (for example, the memory consumption can be pretty high), therefore it is a good practice to use a dedicated server to perform the export operation.

It is required that the specified server has the central administration Web site provisioned. It should be noted that with older hotfix levels the UI does not verify if the central administration Web site has really been provisioned on the selected server. In the case where a server has been selected that does not have the central administration Web site provisioned, content deployment jobs will not start and will remain in the "Preparing" phase.

The default setting for this option will be the server that hosts the Central Administration Web site.

Connection Security

This is a setting that impacts incoming content deployment jobs. If this setting is set to require encryption then SSL encryption is required for the incoming traffic. Any incoming deployment requests without SSL will be denied.

This option is usually required in the case where the upload goes over the internet or another unsecure line.

The default is that SSL encryption is required as this is the more secure setting.

Temporary Files

This setting specifies the location of the compressed cabinet files before they get uploaded to the target farm or after they have been received from the source farm. Please note that this setting does not specify the location of the uncompressed content!

The uncompressed data will be stored in a directory below the directory returned from the Windows OS function, GetTempPath. GetTempPath will use the following logic to determine the location (the first hit wins):

• The path specified by the TMP environment variable.
• The path specified by the TEMP environment variable.
• The path specified by the USERPROFILE environment variable.
• The Windows directory.

These locations are usually found on the system drive.

Due to the above, it is a good practice to configure the TMP environment variable of the user assigned to the OWSTIMER service and the application pool of the central admin Web site to a location that has sufficient disk space.

Reporting

This setting specifies how many reports are kept for each content deployment job. The default setting is 20.

Settings configurable through the web.config

Some settings which affect content deployment have to be configured in the web.config of the central administration Web application.

Currently the following options are available:

RetryOption

This is a new option that has been added with build 12.0000.6315.5000.

It has been implemented to mitigate the problems that can occur when content deployment jobs are executed in parallel or if authoring activities badly interact with the export phase.

To overcome this limitation the RetryOption has been implemented which allows an export to be retried if the previous export failed. This option can either do a simple retry, which means the same export will be performed again or it can be configured to exclude the sites (SPWeb objects) that caused problems from the next attempt. This allows content deployment to be used in a limited manner where persistent problems in specific sites would cause content deployment jobs to fail.

This setting defines the default value which applies to new jobs. It is possible to specify different values for individual jobs using additional settings which will be covered later.

Possible values for this option are None (=0), SkipFailedWebs (=1), and SimpleRetry (=2).

The default value is None which means that no retry attempt is performed.

See later in the Content Deployment Job configuration settings for an example on how to configure this option in the web.config.

ExportRetries

This option is tied to the previous option. If the RetryOption is enabled this setting defines how many times the export will be retried before the job finally ends with a failure. To ensure that the deployment jobs does not run forever if a persistent problem occurs you should avoid configuring this to a number higher than 5.

This setting defines the default value which applies to new jobs. It is possible to specify different values for individual jobs using additional settings which will be covered later.

Possible Values: 1-999. A small number (3-5) is recommended

(See later in the Content Deployment Job configuration settings for an example on how to configure this option in the web.config.)

Settings configurable through custom code

Some settings are only exposed through the object model because the product group assumed that there is no need to change the default values.

Access to the content deployment configuration information is provided through the following class:
Microsoft.SharePoint.Publishing.Administration.ContentDeploymentConfiguration

In addition to the settings listed in the last two sections, this class exposes the following additional properties:

FileMaxSize

This setting allows the maximum size of each generated cab file to be defined.

The default value for this setting is 10 MB.

Please note that this does not mean that cab files will never exceed 10 MB. Content deployment does not split files over multiple cab files. As a result cab files can exceed this limit. For example, if a file in a document library that needs to be deployed cannot be compressed to a size smaller than 30 MB then the cab file will end up at least 30 MB in size.

RemoteTimeout

After the import job on the target server has been started the content deployment job on the export server of the source farm polls the target farm for status updates in regular intervals. If the status does not change for the timeframe configured through this value the exporting server reports a timeout.

This affects mainly full deployment jobs which deploy a large amount of data. If in such a situation the decompressing phase on the target server takes longer than the configured RemoteTimeout values you will see such a timeout.

The default for this setting is 10 minutes.

RemotePollingIntervall

This setting is tightly bound to the previous. It defines the interval used by the exporting server to check for status changes on the target server.

Usually it should not be required to change this setting.

The default for this setting is 10 seconds.

The following code sample demonstrates how to adjust the RemoteTimeout value. The other settings can be adjusted in a similar manner:

Content Deployment Path Settings

As discussed earlier a content deployment path defines the content flow from a single source site collection to a single target site collection.

Access to the content deployment path configuration information is provided through the following class:
Microsoft.SharePoint.Publishing.Administration.ContentDeploymentPath

This class stores its data in the Content Deployment Paths list which resides in the root web of the central admin website:
http://url-to-central-admin/lists/content%20deployment%25paths

Be aware that only part of the settings can be configured through Central Administration. Some additional settings can to be done through STSADM commands.

Settings configurable through Central Administration

The following settings can be configured using the Content Deployment Path Settings page in Central Administration:

Name

The Name for a content deployment path has to be unique.

Description

The description for the content deployment path.

Source Web Application

This setting defines the URL to the source Web application. The UI provides a drop down with the names of all Web applications on the source server. It should be noted that although the UI presents the Web application name in the drop down only the URL to the site collection is stored in the path settings.

Source Site Collection

This setting defines the server relative URL for the source site collection which resides inside the selected source web application. As soon as a source Web application has been chosen by a user the UI populates the drop down box with all server relative URLs for the selected Web application.

Destination Central Administration Web Application

This setting defines the URL to a server in the target farm that hosts an instance of the central administration Web site. It does not matter if this is the Import server in the target farm or a different server as long as the central administration Web application has been provisioned on this server.

It should be noted that the protocol used in this setting defines whether the future communication to the target farm will use http or SSL.

The UI will present a warning when http is being used:

Authentication Information

This setting defines the credentials being used to communicate with the target farm. The account being used has to have farm administrator rights on the target farm. When the same farm account is used in both farms or when the path links two site collections in the same farm the option to connect using the application pool account can be utilized.

Alternatively you can enter the credentials to the target farm and decide whether to use integrated authentication or basic.

Destination Web Application

This setting defines the URL to the target Web application.

The UI provides a drop down with the names of all Web applications on the target server which is populated as soon as the URL to the target central administration Web application has been entered and the connect button has been pressed.

It should be noted that although the UI presents the Web application name in the drop down only the URL to the site collection is stored in the path settings.

Destination Site Collection

This setting defines the server relative URL for the destination site collection which resides inside the selected destination web application. As soon as a destination Web application has been chosen by a user the UI populates the drop down box with all server relative URLs for the selected Web application.

User Names

This setting is the equivalent to the SPImportUserInfoDateTimeOption setting of the underlying content deployment and migration API. It defines whether user information from the source system should be associated with the content after it has been deployed to the target.

Security Information

This setting defines the level of granularity to use for security information from the source system should be deployed to the target system.

Three different levels are possible:

• All: all security information should be deployed
• Role definitions only: this option allows to deploy everything except the actual user assignments. That is an interesting option if you would like to have the same sharepoint groups with identical permissions in source and target environment but different users due to different AD infrastructure assigned to the groups.
• None: no security information should be deployed to the target server. That means all security related settings will be inherited from existing content on the target farm.

Settings configurable through STSADM

Some settings are only exposed through STSADM commands.

KeepTemporaryFiles

When performing a content deployment operation the exporting server first exports the content into a temporary directory on the disk and then compresses the data. After the deployment is done, the temporary files are cleaned up - more or less (we will cover this later).

This setting allows you to control in which situations the cleanup will happen. This can be interesting when it comes to troubleshooting content deployment problems. Enabling this option allows you to keep and inspect the generated cab files to identify what has been deployed.

We at Microsoft Support often use this option when working on support cases.

To modify this option please use the following STSADM command:

STSADM -o editcontentdeploymentpath -pathname <pathname> -keeptemporaryfiles Never|Always|Failure
(In case your content deployment path name contains blanks please specify it with double quotes.)

Possible values: Never (=0), Always (=1), Failure (=2)

Be aware that Failure means fatal errors. In case that warning or non-fatal errors are reported in the deployment log the job would still report "Success" and the temporary files would not be preserved even if you specified "Failure" for this option.

The default is Never.

EnableEventReceiver

This property allows to control whether or not asynchronous event receivers are allowed to fire during the import. The default is that asynchronous event receivers are disabled during import. This configures the SPImportSettings.SuppressAfterEvents setting of the content deployment and migration API when performing the import.

Any modification to the target database that happens in parallel with an import could potentially lead to a deadlock in the SQL database which could cause content deployment import to fail. As asynchronous event receivers get executed in parallel with the import operation such deadlocks could be caused by these event handlers. That's the reason why this option is disabled by default. In general it should not be required to have these event handlers enabled in case that they should modify sharepoint content as the same actions would have fired on the source system and would have performed the modification there already. So the changes done by the event receiver will be deployed to the target system using content deployment. You should only enable after events if you are sure that these events are not modifying anything inside the sharepoint database the import runs in. E.g. if the event receivers only send emails or update content in other databases.

This option can be modified using the following STSADM command:

STSADM -o editcontentdeploymentpath -pathname <pathname> -enableeventreceivers yes | no

Possible values: No (=0), Yes (=1)

Default is No.

EnableCompression

This property allows you to control whether the data being deployed is being compressed during the deployment or if the uncompressed data should be sent to the destination server. This setting is enabled by default to provide acceptable performance when uploading the content to a server over a slow connection. In an intranet scenario or when deploying content between two site collections on the same server, the overhead to perform compression and decompression might be higher than the additional time to transport the uncompressed data. Here the compress and uncompress time could cause a significant overhead in the performance of the content deployment jobs.

In such a situation it might make sense to disable the compression.

This setting configures the SPDeploymentSettings.FileCompression property of the content deployment and migration API during export and import.

To modify this option use the following STSADM command:

STSADM -o editcontentdeploymentpath -pathname <pathname> -enablecompression yes | no

Possible values: No (=0), Yes (=1)

The default value is Yes.

Content Deployment Job Settings

As discussed earlier a content deployment job defines for which part of a site collection and in which schedule content is transferred to the target site collection.

Access to the content deployment job configuration information is provided through the following class:
Microsoft.SharePoint.Publishing.Administration.ContentDeploymentJob

This class stores its data in the Content Deployment Jobs list which resides in the root web of the central admin website:
http://url-to-central-admin/lists/content%20deployment%25jobs

Be aware that only part of the settings can be configured through a UI. Some additional settings can to be done through web.config or STSADM commands.  

Settings configurable through the UI

The following settings can be configured using the Content Deployment Job Settings page in the UI:

Name

The Name for a content deployment job has to be unique. Be aware that it has to be unique for all Jobs - not just for all jobs for a specific path. For example, you cannot create a job named "Deploy All" for Path 1 and a job named "Deploy All" for Path 2.

Description

The description for the content deployment job.

Path

This setting identifies the content deployment path that is associated with the current content deployment job. This setting is represented by a drop down in the UI which is populated with all content deployment path definitions in the current farm.

Scope

This setting defines whether the whole site collection or only specific sites should be deployed. The UI provides an additional option to show a treeview with all sites in the site collection where a user can pick the sites that should be deployed.

Here he can also decide whether to deploy the whole subtree starting at the selected site or only the selected site.

It is not possible to explicitly exclude specific sites from the deployment. To achieve this it will be required to use a combination of site and subtree inclusions which omit the part of the site collection that should not be deployed.

Frequency

This setting allows the schedule for the content deployment job to be defined. This option is disabled by default which means that the content deployment job has to be executed manually. The UI provides various different options to schedule the execution of a content deployment job.

It should be noted that content deployment does not work reliably if multiple export or import operations for the same site collection are executed at the same time. That means you should schedule different jobs for the same source or target site collection so that they do not overlap.

It should also be noted that the schedule is not stored in the content deployment job itself. For each Content Deployment Job there is an assigned Timer Job (the timer job is an instance of the class Microsoft.SharePoint.Publishing.Administration.ContentDeploymentJobDefinition) which does the actual work. The schedule will be assigned to the Schedule property of this timer job.

The content deployment job object itself holds a reference to the timer job.

Deployment Options

This setting defines if the deployment job will perform a full or an incremental deployment. This setting is equivalent to the SPExportMethodType property passed into the content deployment and migration API.

Notification

This setting defines if an email is sent when a content deployment job has succeeded and/or failed. The UI requires that an email address be provided when one of the options is enabled.

Settings configurable through the web.config file

Some settings which affect a content deployment job have to be configured in the web.config of the central administration Web application.

Currently the following options are available:

RetryOption

This is a new option that has been added with build 12.0000.6315.5000.

It has been implemented to mitigate the problems that can occur when content deployment jobs are executed in parallel or if authoring activities badly interact with the export phase.

To overcome this limitation the retry feature has been implemented which allows the export to be retried if the previous export failed. This option can either do a simple retry – means perform the same export again or it can allow excluding the sites (SPWeb objects) that caused the problem from the next attempt. This allows you to use content deployment in a limited manner in situations where persistent problems in specific sites would cause content deployment jobs to fail.

This setting defines the default value which applies to new jobs. It is possible to specify different values for individual jobs using additional settings. We will cover this later.

Possible values for this option are None (=0), SkipFailedWebs (=1), SimpleRetry (=2)

The default value is None which means that no retry attempt is performed.

ExportRetries

This option is tied to the previous option. If Retry is enabled this setting defines how many times the content deployment retries the export option before the job finally ends with a Failure.

To ensure that the deployment jobs does not run forever if a persistent problem occurs you should avoid configuring this to a number higher than 5.

This setting defines the default value which applies to new jobs. It is possible to specify different values for individual jobs using additional settings. We will cover this later.

Possible Values: 1-999. A small number (3-5) is recommended

The following lines have to be added to the web.config to enable the retry options:

<option> can be one of the following: None, SkipFailedWebs, SimpleRetry

<number-of-retries> can be an integer between 1 and 999

If you don't add any <Job...> elements, then all new jobs will get the values configured as DefaultOption and DefaultRetries. Using the <Job...> element you can configure different jobs with different values.

DefaultOption and DefaultRetries are also the entries that get added to the Content Deployment general settings which we discussed in an earlier section.

After you have added the above listed configuration to the web.config of the Central Administration you have to open the content deployment job definition once in the central administration and save it again. During this save the configured options will be silently added to the content deployment job.

In addition you need to open and save the content deployment general settings page to ensure that the default settings are updated as well. New jobs created after you changed the web.config will automatically get the configured values.

Potential problems related to Configuration

Problems caused by not saving the content deployment general settings 

As outlined earlier in this article the settings visible on the content deployment general settings page are not persisted in the database until explicitelly saved using the OK button.

Problems will occur in situations where the settings haven't been saved (default settings used) and later an option is adjusted using object model. E.g. when the RemoteTimeout value is adjusted.

In this situation the settings will be populated in context of the current machine and current user. That means that potentially the machine where the code is executed will become the export and import server of the farm. In addition the path for the temporary files will not be populated with c:\windows\temp but with c:\documents and settings\<username>\local settings\temp.

Often the user adjusting the timeout value is not the same account as the farm account of the sharepoint farm. That means when content deployment later tries to create the temporary files it will try to create them at a location where the farmaccount does not have rights to and the deployment will fail.

Problems caused by insufficient permissions in the source site collection

If the user configuring the content deployment job does not have read permission in the source site collection he cannot configure the job for selective deployment. The reason is that the treeview which allows a user to select the sites that should be deployed is created by enumerating the site structure in the source database in context of the current user. If this user does not have rights to see these sites or some of these sites they will not be included in the tree view. In case that the user does not have any rights in the source site collection the dialog will show an access denied message.

Content Deployment Phases

The content deployment process is divided into several different phases which are partly performed on the source and partly on the target server.

Preparing

This phase is unique to jobs that are started through the Central Admin UI. In this phase, the code in the UI of the exporting server creates the one time timer job that will perform the export.

Exporting

This phase is performed on the exporting server of the source farm. It uses the WSS content deployment and migration API to export the relevant content for the specific content deployment job. The content will be exported into a flat file structure in a temporary directory on the exporting server.

Compressing

This phase is performed on the exporting server of the source farm. It is done internally in the WSS content deployment and migration API and compresses the exported files into one or multiple cabinet files (cab files). These files will be stored again in a temporary folder on the exporting server.

Transferring

This phase is performed by the exporting server on the source farm. It uses functionality provided by the content deployment feature coming with MOSS 2007. This phase transfers the cabinet files using http upload to the importing server on the target farm where it ends up in a temporary folder.

Decompressing

This phase is performed by the importing server on the target farm. It is done internally in the WSS content deployment and migration API and decompresses the cabinet files into a temporary folder on the importing server.

Importing

This phase is performed by the importing server on the target farm. The import is performed using the WSS content deployment and migration API which reads the content from the temporary directory where the decompressed files are located and generates the content in the target site collection.

Content Deployment Path and Job

Content Deployment is built on two underlying conceptual objects: content deployment path and job.

Content Deployment Path

Content Deployment allows the deployment of content from exactly one site collection on the source farm to exactly one site collection on the target farm.

The link between the source and target site collection is called a content deployment path. You can have multiple content deployment paths in your farm as long as you don't have multiple paths that reference the same source and target site collection.

A content deployment path is always defined on the source farm.

It is possible to have two content deployment paths linking one site collection on the source farm with two different site collections on the target farm.

It should be noted that the source and target site collection can reside in the same farm. So a path can point from one site collection to a second site collection in the same farm as long as the two site collections reside in different databases.

Content Deployment Job

It is possible to create one or more content deployment jobs for a single content deployment path. Each job can be of a different type (full, incremental or quick deployment) and can be configured with a different schedule.

For all jobs except quick deploy jobs it is possible to configure different content deployment jobs for the same path to deploy different parts of the source site collection. This is done by selecting specific sites that should be deployed with this specific job. Using this method it would be possible to deploy one part of the site daily and a second part which contains more frequently updated information hourly.

Quick Deploy jobs cannot be created manually. They will be created automatically when saving the content deployment path if the source site collection has the publishing feature enabled. There can only be one Quick Deploy job per path.

The content deployment job is always defined on the source farm. During the deployment to the target farm a "shadow" content deployment job will be created on the target farm to perform the import operation. This job will automatically be deleted after the import has been completed.

Also be aware that this "shadow" job will not show up in the content deployment paths and jobs page in the target farm as this job will not be bound to a content deployment path.

A while ago I created a deep dive article series covering the WSS content deployment and migration API which helped many people to develop their own applications to do export and import in a customized manner.

Today I will start a new article series which will discuss all aspects of Content Deployment – with other words the MOSS feature sitting on top of the WSS API.

Most customers see this feature as a monolithic implementation which does not allow any customization – but that is not the case as you will see in future chapters of this article series

But before we can start with the internals we first have to start with the basics.

Content Deployment – The Idea Behind

MOSS 2007 contains Content Deployment as a new feature which has been added to fulfill the requirements of companies which plan to use SharePoint as a Web server to host public facing Web sites.

The main purpose is to allow authors and reviewers to modify and evaluate on a different farm before the content is finally pushed to the public facing server farm – but also to have a single authoring environment and then push the content to multiple different farms of different departments - potentially on different continents.

A similar concept (site deployment) was already available in Microsoft Content Management Server 2002 but required additional programming for automated deployments. With MOSS 2007 this can now be automated and customized easily through a build in UI.

Content Deployment – The Feature

The Content Deployment section shown on the Operations tab of the Central Administration Website is provided by the "Content Deployment" feature which is shipped with MOSS 2007.

You can find this feature in the following directory: …\12\template\features\DeploymentLinks

This feature is activated on the Root Web of the Central Admin site collection. Be aware that this is a hidden feature so you cannot see it in the site feature list in the UI.

Deactivating this feature will remove the Content Deployment section from the central admin Web site and also disable the underlying functionality of Content Deployment as several methods verify internally if this feature has been enabled on the central admin Web site or not.

You can easily try this on your own using the following command:

STSADM -o deactivatefeature -url http://central-admin-url -name DeploymentLinks

After executing this command you will see that the content deployment section is gone from the operations tab. To reenable the functionality use this command:

STSADM -o activatefeature -url http://central-admin-url -name DeploymentLinks

Content Deployment – The different flavors

Content Deployment as it is available in MOSS 2007 comes in three different flavors: full deployment, incremental deployment and quick deployment. Here is a short overview about the three different flavors:

Full Deployment

Full Deployment allows all the content of a site to be deployed to the target site collection independent of when the content was created. This is usually done when deploying a site collection to the target farm for the very first time.

Limitations

Full deployment will only deploy the current state of the site. It will not deploy information about historical actions performed on the site (for example, if an item has been moved or has been deleted).

Benefits

Full deployment does not rely on information about what has been previously deployed to the target. It will always export the current state of the source site collection.

Caveats

Full deployment into a database that has already received content through a content deployment job can cause inconsistencies between the source and the target farm as information about historical actions is not deployed. For example, if a list item is deleted on the source farm between the two content deployment runs you will notice that the item will not get deleted on the target farm.

Incremental Deployment

Incremental Deployment deploys all content that has changed since the last successful run of a content deployment job. Only modified content will be deployed to the target farm. Incremental deployment also transports the information about historical actions such as move, delete and rename to the target farm.

Limitations

The complexity of an incremental deployment is higher than the complexity of a full deployment job. This is due to the additional logic included to deploy only the modified content and items directly linked to the modified content. This increases the chance for problems which can cause incremental content deployment to fail. A second limitation is that an incremental deployment will only look at the source system to determine the data that has to be deployed. It will not analyze source and target and based on the difference decide which content to deploy.

Benefits

Due to the fact that only modified content is deployed, the time for an incremental deployment is usually much shorter than the initial full deployment. In addition, the feature to deploy information about historical actions allows the target system and the source system to keep in sync beyond what is possible with full deployment.

Caveats

If an incremental deployment job fails repeatedly it might be required to perform a new full deployment to the target to re-sync the content on the source farm with the target farm. Since a full deployment does not transport information about historical actions, this will only work reliably if this full deployment is performed into an empty database.

Quick Deployment

Quick Deployment allows authors to pick specific pages for content deployment. This is vital for companies which need to quickly update specific pages on their site which need to be independently updated from other content that would be deployed with an incremental deployment.

Limitations

Quick deployment will deploy only the selected publishing pages and resources bound to these pages. In addition, it is not possible to select items in a regular document library or list for quick deployment.

Benefits

Quick Deployment enables authors to register specific items for quick deploy which will be deployed to the target farm independent from any other changes that have been done on the source farm since the last full or incremental deployment.

Caveats

Quick deployment only deploys publishing pages and resources bound to the selected pages. Sites and lists are not deployed through quick deployment. Due to this, all target folders and libraries for all selected an dependent items must already exist on the target farm – otherwise quick deployment will fail. For example, consider the scenario where a new picture library has been created and a picture from this picture library has been bound to a field in a publishing page that needs to be quick deployed. Quick deployment will fail if the picture library has not been previously deployed to the target site collection.

Myths and Facts about Full, Incremental and Quick Deployment

Myth: I need at least one full and one incremental deployment job per path as I have to run at least on full deployment at the beginning before I can run an incremental deployment

Fact: The above statement is not correct. The very first incremental deployment will always be a full deployment. So there should be no need to create a full deployment job.
 

Myth: Full deployment is more than incremental deployment

Fact: The above statement is not correct. Full deployment only deploys the current state of the site while incremental deployment deploys all changes done to the site since the last deployment. That also includes delete and move operations. An extreme example would be a site collection that contained 100.000 items when it was last deployed. After deleting 90.000 items full deployment would only deploy the 10.000 remaining items and would not deploy the delete operations. So after a full deployment the number of items in the target database would not have changed (assuming that no other operations have been done to the source database as the delete operation). On the other hand an incremental deployment will only deploy the 90.000 deleted items to ensure that these items will also be deleted on the target. With other words: after the full deployment the source and target database will be out of sync while incremental deployment will keep these databases in sync.
 

Myth: It is best practice to run incremental deployments during the week and a full deployment during weekend to ensure that my content database is in sync

Fact: The above statement is not correct. Every single full deployment has the risk that the target site collection runs out of sync with the source as delete and move operations will not be applied the target which also has the risk of breaking future the content deployment jobs which will fail due to the inconsistencies between source and target.
 

Myth: If incremental deployment fails I can do a full deployment to fix the problem

Fact: This statement is not correct. Indeed: full deployment often succeeds when incremental deployment failed and indeed future incremental deployments will often work again afterwards but this is caused by the fact that after the full deployment the next incremental will deploy only content changed after the full deployment. The problem is that in this situation you will usually not really resolve the underlying issue. You are just avoiding it by adding all other content that exists in the source site collection to the package so that inconsistencies in the source database which led to the failing incremental deployment will no longer show up. But often the same problem occurs again a couple of days later when incremental deployment again has to deploy the problematic items. And through the full deployment you made things worse as you now have potentially additional inconsistencies between source and target due to the fact that delete and move operations that would have been deployed with the failing incremental deployment will never be deployed again. So now you added additional inconsistencies to the one that caused the earlier failure which can cause additional problems with future content deployments (as well incremental and full).
 

Myth: Quick deploy is an easy way to ensure that important content can go live easily out of sync with other content.

Fact: The above statement is not fully correct. The quick deployment has to be used very carefully. For every single quick deployed item – and also for all dependent items like referenced images and documents – it has to be guaranteed that the site, the folder the list or library the item is in has been deployed to the target earlier with an incremental deployment. If this is not the case the quick deploy job and all future quick deploy job runs will fail till the next incremental deployment has been performed. That means a single user that quick deploys a page that references a document in a doc lib which has not been previously deployed to the target system can cause a problem that affects the ability of all other users to perform a quick deployment. The main problem is here that quick deployment is done by authors – while incremental deployment is controlled by farm administrators. So it is usually unclear to the authors when the last successful incremental deployment has been done and which sites and lists have been deployed.