Stefan Goßner

Senior Escalation Engineer for SharePoint (WSS, SPS, MOSS, SP2010) and MCMS

Deep Dive into the SharePoint Content Deployment and Migration API - Part 2

Deep Dive into the SharePoint Content Deployment and Migration API - Part 2

  • Comments 112
  • Likes

[Part 1 - Part 2 - Part 3 - Part 4 - Part 5 - Part 6 - Part 7]

Providing some real world samples for export

To demonstrate how powerful the Content Deployment and Migration API is I will now provide some real world examples:

  1. Export a complete Site Collection
  2. Export a specific sub web
  3. Exporting other elements like lists, document libraries, list items or documents from a document library
  4. Exporting using compression
  5. Incremental Export

The examples below assume that the source site collection is on the local server on port 2000.

Lets start with a scenario that can also be covered by using stsadm.exe: 


1) Export a complete Site Collection


Actually there are two different methods to export a complete site collection.

Method 1)

SPExportSettings settings = new SPExportSettings();
settings.SiteUrl = "http://localhost:2000";
settings.ExportMethod = SPExportMethodType.ExportAll; 
settings.FileLocation = @"c:\export";
settings.FileCompression = false;
settings.CommandLineVerbose = true;
 
SPExport export = new SPExport(settings);
export.Run(); 

So what did we actually do here? First we created a SPExportSettings object to define the general configuration settings for the export we would like to perform. As we did not select a specific object to export the configured site collection will be selected for export. Then we created the SPExport object based on the configured settings and started the export by calling the Run method.

Here is a short explanation about the settings being used in the code above:

  • SiteUrl - this property defines which site collection the export should use. All objects being exported always have to be in the same site collection. The Content Deployment and Migration API cannot access items in different site collections in a single operation. 
  • ExportMethod - this property allows to define whether to perform an incremental export (value = ExportChanges) or everything (value = ExportAll). Be aware that ExportChanges would require to provide an Export Change Token in a separate property.
  • FileLocation - this property defines where to store the exported content. The value should point to be an empty directory. If the directory does not exist it will be created during export. If file compression is being used, then only the compressed files will be stored on this location. The uncompressed files will be stored in the directory identified by the value of the system wide TMP environment variable. So you need to ensure that the directory the TMP environment variable points to also needs to have sufficient space available.
  • FileCompression - this property defines whether the content should be compressed into a CAB file. If you need to archive the exported content or need to transfer it to a different machine you should choose to compress. If you only export the content to import it afterwards using code on the same machine and don't need to archive (e.g. a copy or move operation) then you should decide to disable the compression as this is significantly quicker.
  • CommandLineVerbose - this parameter allows to control if the API should provide some verbose output. If you have ever seen the generated output when running STSADM -o export: this is exactly the flag the generates this output. If the value is false no output is generated.

Method 2)

SPSite site = new SPSite("http://localhost:2000");

SPExportObject exportObject = new SPExportObject();
exportObject.Id = site.ID;
exportObject.Type = SPDeploymentObjectType.Site;

SPExportSettings settings = new SPExportSettings(); 
settings.SiteUrl = "http://localhost:2000";
settings.ExportMethod = SPExportMethodType.ExportAll; 
settings.FileLocation = @"c:\export";
settings.FileCompression = false;
settings.ExportObjects.Add(exportObject);

SPExport export = new SPExport(settings);
export.Run();  

site.Dispose();

What did we actually do here? In this method we explicitly add a specific object to the ExportObjects collection - the SPSite object defining the site collection. This leads to the same result as the first method where the SPSite object was implicitly selected as no explicit object was selected for export.

Here is a short explanation about the settings being used in the code above:

  • SPExportObject.Id - GUID of the object that should be exported
  • SPExportObject.Type - The type of the object to be exported. This can be one of the following:
    • Site = SPSite object (the site collection)
    • Web = SPWeb object (often referred to as site)
    • List = SPList object
    • File = SPFile object
    • Folder = SPFolder object
    • ListItem = SPListItem object
  • for the other properties see Method 1


2) Export a specific sub web


The code to export a specific subweb is very similar to the code used in method 2 to export the complete site collection:

SPSite site = new SPSite("http://localhost:2000");

SPWeb web = site.OpenWeb("/SomeWeb"); 

SPExportObject exportObject = new SPExportObject();
exportObject.Id = web.ID;
exportObject.IncludeDescendants = SPIncludeDescendants.All;
exportObject.Type = SPDeploymentObjectType.Web;

SPExportSettings settings = new SPExportSettings(); 
settings.SiteUrl = "http://localhost:2000";
settings.ExportMethod = SPExportMethodType.ExportAll; 
settings.FileLocation = @"c:\export";
settings.FileCompression = false;
settings.ExcludeDependencies = false;
settings.ExportObjects.Add(exportObject);

SPExport export = new SPExport(settings);
export.Run();

web.Dispose();
site.Dispose();

 

Be aware that when exporting a specific sub web using the method above might also export objects outside the specific subweb as not only the objects in the sub web are exported but also dependent objects like images referenced in pages by this sub web and the master page assigned to the sub web.

Whether such dependent objects should be exported or not can be controlled by the ExcludeDependencies property of the SPExportSettings. Whether child objects of an object are exported can be controlled using the IncludeDecendents property of the SPExportObject object:

  • ExcludeDependencies - This property controls whether to export dependent objects like referenced images, master pages or page layouts. If it is unclear if the dependent objects are already in the destination database you should export the dependent objects as well to prevent problems.
  • IncludeDescendants

     

     

    - This property controls whether to export child objects of the selected objects (like sub webs, lists, list items) or only the selected object. Possible values are None (only the object is exported), Content (List and libraries of a web will be exported but not sub webs) and All (all child content is exported)


3) Exporting other elements like lists, document libraries, list items or documents from a document library

Other objects or containers can be exported in the same way by adding them to the ExportObjects collection.

SPSite site = new SPSite("http://localhost:2000");
SPWeb web = site.OpenWeb("/SomeWeb");  
SPList list = web.Lists["MyList"];
SPListItem listItem = list.Items[0]; // select the first list item

SPExportObject exportObject = new SPExportObject();
exportObject.Id = list.ID;
exportObject.Type = SPDeploymentObjectType.List;

SPExportObject exportObject = new SPExportObject();
exportObject.Id = listItem.UniqueId;
exportObject.Type = SPDeploymentObjectType.ListItem;

...

 

Exporting document libraries can be done using the same code as exporting a list as internally a document library is implemented as a list. And a document in a document library can be exported using the same method as exporting any other list item. The SPFile object of the document will automatically be automatically exported together with the SPListItem object.


4) Exporting using compression

When exporting using compression additional settings in the SPExportSettings object need to be adjusted:

SPExportSettings settings = new SPExportSettings();
settings.FileLocation = @"c:\export";
settings.FileCompression = true;
settings.FileMaxSize = 10// defines the maximum filesize as 10 MB
settings.BaseFileName = "ExportedItems.cmp";
... 

Here is an explanation about the two properties:

  • FileCompression - this property defines whether the content should be compressed into a CAB file. If you need to archive the exported content or need to transfer it to a different machine you should choose to compress. If you only export the content to import it afterwards using code on the same machine and don't need to archive (e.g. a copy or move operation) then you should decide to disable the compression as this is significantly quicker.
  • FileLocation - this property defines where to place the export files. If compression is used it defines the location of the generated compressed content migration pack.
  • FileMaxSize - this property defines the maximum size of the created content migration pack.
  • BaseFileName - this property is only used if compression is enabled. It defines the name of the compressed content migration pack file. The default extension (if not specified) is ".cmp". If the size of FileMaxSize is exceeded a new file is created with the BaseFileName and an integer number. E.g. with the above configuration the next file would be ExportedItems1.cmp

 


5) Incremental Export
 
If incremental export should be done it is required to save the Change Token of the last full or incremental export. This change token needs then be provided in the ExportSettings to allow the Content Deployment and Migration API to determine which items have changed.
So after a previous export the change token can be retrieved using the following code:
SPExportSettings settings = new SPExportSettings();
...
 
SPExport export = new SPExport(settings);
export.Run(); 
 
string ChangeToken = settings.CurrentChangeToken;  

Explanation:

  • CurrentChangeToken - this property is a read-only parameter populated during export. It contains the Change Token right after the export. So doing an incremental export providing this change token in the future will export all items that have been created, changed or deleted in the configured scope after the change token was generated.

To start an incremental export using this change token would then be done using the following code:

SPExportSettings settings = new SPExportSettings();
settings.ExportMethod = SPExportMethodType.ExportChanges;  
settings.ExportChangeToken = oldChangeToken;
...  

Explanation:

  • ExportMethod - this property allows to define whether to perform an incremental export (value = ExportChanges) or everything (value = ExportAll).
  • ExportChangeToken - this property defines which items to export when using incremental deployment. the incremental export will only export items that have been created, changed or deleted in the configured scope after the change token was generated.

 

Now after we have seen how to export items the Part 3 of this article series will cover the import part.
Comments
  • Providing some real world samples for import After we managed to export content in part 2 lets now focus

  • Stefan is back! J Stefan is in Redmond this week, and we had a chance to catch up! In fact, I had dinner

  • Stefan Goßner taucht in einer vierteiligen Serie in die Tiefen der SharePoint Deployment und Migration

  • [ Part 1 - Part 2 - Part 3 - Part 4 ] Advanced content deployment scenarios Till now we have covered

  • Fantastic, incredibly detailed, content on content deployment by Stefan

  • [via Stefan Gossner ] This past week I was in Redmond teaching my WCM401 development class in an open

  • Body: Great series of posts by Stefan on how to use the Content Migration API. Very timely for me as

  • Hello,

    We can have full license for MOSS 2007 for intranet / CMS access for content management, though for the public-facing content delivery part we intend on using Windows SharePoint Services 3.0 (which comes w/ Windows Server 2003), and not have to purchase internet connector for MOSS 2007.

    To get a little deep into the scenario, for authoring and content management, MOSS 2007 with SQL Server 2005 as the database would be used. The content authored (we plan to) would eventually be exported on a different box with Win 2003 machine with WSS V3 and SQL Server 2005 installed. Once this is done we can currently think of the below mentioned scenarios to publish our site –

    -         Write a custom engine that pulls out content from the database (using APIs, if provided by WSS 3.0) on every hit to the page.

    -         Use Out of the box features provided by WSS V3 to pull out content from the database on every hit to the page.

    I would really appreciate if you could give us some insight into the feasibility of the approaches that we are intending to take up.

    Regards

    Sachin

    sachsharma@sapient.com

  • Hi Sachin,

    I don't think this method will be practically useful. Without MOSS you will not have the page layout concept of the pages layout. So you only see the fields.

    You will not be able to influence how the fields are arranged in the page and within a master page.

    Cheers,

    Stefan

  • 今天凌晨加班的时候偶然翻到 Stefan Gossner 的这几篇文章,强烈推荐给大家: Deep Dive into the SharePoint Content Deployment and Migration

  • Hi,

    Thank you for these great articles - they gave me a kickstart on content deployment at the right time!

    I have a scenario where we need to move contents from lists on developer servers, to test and further on to production. The site structure is not neccessarily the same on the different environments, so the sample for importing in a different structure is a great starting point.

    A note on incremental import should be said - it is not supported on lists - only sites and webs. I get around this by using an importhandler that checks the imported ListItem, and if it already exists (checked by CAML query on list-webservice) is deleted again. I find this approach easier to code compared to manipulating the manifest.xml file.

    I think the performance is a bit problematic. Exporting 196 ListItems take in average 10 minutes. I fear the time, when we get 1-2000 items in the list. I am currently looking into ways to improve performance and will be investigating the usage of indexed columns. Any hints are welcome..

    Regards

    Flemming

  • Great set of articles by Stefan Goßner : Deep Dive Into the SharePoint Content Deployment...

  • MOSS / SharePoint Content Deployment

  • I' trying to export my "Pages" library with this code:

    SPSite site = new SPSite("http://vm-spwf");

    SPWeb web = site.OpenWeb();

    SPList list = web.Lists["Pages"];

    SPExportObject exportObject = new SPExportObject();

    exportObject.Id = list.ID;

    exportObject.Type = SPDeploymentObjectType.List;

    SPExportSettings settings = new SPExportSettings();

    settings.SiteUrl = "http://vm-spwf";

    settings.ExportMethod = SPExportMethodType.ExportAll;

    settings.FileLocation = @"c:\export.1";

    settings.FileCompression = false;

    settings.ExcludeDependencies = false;

    settings.ExportObjects.Add(exportObject);

    SPExport export = new SPExport(settings);

    export.Run();

    web.Close();

    site.Close();

    I received the following Exception:

    Violation of PRIMARY KEY constraint 'PK__#ExportObjects____73DFBB75'. Cannot insert duplicate key in object 'dbo.#ExportObjects'.

    The statement has been terminated.

    Where is the problem ?

    Regards

    Tihomir Ignatov

  • Hi Tihomir,

    please ensure that the latest security fixes for WSS and MOSS are installed. If this does not fix the problem, please open a support case to get this analyzed.

    Cheers,

    Stefan

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment