Howard Hoy

System Center 2012 Configuration Manager and other thoughts.

An adventure in the sccmcontentlib - single instance store

An adventure in the sccmcontentlib - single instance store

  • Comments 10
  • Likes

Sometimes I have nightmares which are work related. One such nightmare was about Configuration Manager 2012 and it prompted me to write this blog post. In my dream, the telephone rang and it was my Configuration Manager 2012 Distribution point. It kindly told me that one of my packages disappeared for no reason. Geez I need a vacation.

With SCCM 2007, content is stored on a distribution point in package ID folders. If you knew the package ID, you could browse to the file share and install/run manually.

This option is more complicated now in Configuration Manager 2012. The files are not just stored on a drive in a manner you could just copy and execute.

The following blog post is about the new Content Library and my attempts to reconstitute a package from the Single Instance Store.

By default, the content is single storage and browsing to a DP does not store all the files in a package id folder where you could find an install and execute it. You can enable the older method which will create a separate folder share. This will require double the disk space! Obviously, this is a pain point for those who use the distribution points this way as a mitigation step in a crisis.

To understand how the Content Library works in Configuration Manager 2012, I decided to find a way to solve this pain point.

This functional diagram is my review of the single content library:

 

Content Layer

In the Content Layer, individual files are sent and only stored once in a single storage location. For example, there might be three packages that need to be distributed in a Configuration Manager 2012 infrastructure. All three packages contain the same file Psexec.exe with the same md5\sha1\sha2 hash value.

Why send the file and store it in three places?

Great efficiencies are gained with the new shared single storage Content Library in Configuration Manager 2012. The file is hashed, given an ID and sent only once. Each package is stored as pointers to the necessary files. This means that packages share files if they are the same. This saves on network usage and file storage requirements. Sounds awesome; doesn’t it!

Configuration Manager Layer

Here is an example:

I created two packages:

  1. 1.      Filezilla – package id HH00005
  2. 2.      Notepad ++ - Package id HH00006

Each has psexec.exe, one of my favorite sysinternal tools. Note you can get it here at http://live.sysinternals.com

Next I “Distribute Content” in the CM2012 console to my distribution point.

Distribution Point Layer

When content is sent to the distribution point, it is stored by default on the largest available drive in a folder, sccmcontentlib. During the install you can set this location, however, you cannot configure the drive settings in the distribution point properties after installation completes.

The following folders get created:

  • ·        Datalib = Data Library
  • ·        Filelib = File Library
  • ·        Pkglib = Package Library

DataLib

For each package that you distribute a corresponding package+id.version number folder and file will be created. For patches the folder will appear as the patch guid and application folders will appear as Content_guid

Now let’s look at the package HH00005.1 (Filezilla)

Each DataLib folder contains .ini file which includes important information for the actual file.

  • ·        Attributes
  • ·        Size
  • ·        TimeModified
  • ·        Hash

 Viewing the psexec.ini file in the HH00005.1 package folder:

 

A Hash value of F8DB… is found. The first four entries of this hash are important. This is the locater folder name for the actual binary file in the pkgLib folder.

Note

The default hashing algorithm for content is SHA-256

To find the file:

Filelib

The folder names have meaning. Each corresponds to a hash value in the FileLib folder. In the example of HH000005.1, psexec.exe.ini, the value was F8DB

Viewing the F8DB folder:

The hash value .ini contains a list of the type and references of which packages use this file. Notice that there are two packages that are listed. This is single instance storage. Any package/program/content/update that has this exact hash file requirement will use this file and location.

The actual file will be the hash with no extension. Copying and renaming the file to psexec.exe, I verified that is the psexec file:

SHA-256 hash value matches.

Now back to the HH00005 package. It contains the setup for FileZilla and psexec.

Note that the psexec.exe.ini file points to the F8DB filelib folder as I demonstrated above.

To summarize the Content Library, Single Content Storage and this review:

Both HH00005 and HH00006 CM2012 packages contain the file psexec.exe. The file is only sent once to the Content Library and stored as single instance. A SHA-256 hash value is used to track the unique file and the first four digits are used as a locator. When a client requests the download the files are assembled from the Content Library via single instance storage.

Now suppose you want to reconstitute an entire package from the sccmcontentlib folder. I created a basic vb.net application to assist me in this exercise.

Following each PkgLib file to the DataLib, I was able to regenerate all the files from the FileLib for any given package.

I hope my blog posting was helpful in providing an understanding the Content Library and the Single Instance Storage in Configuration Manager 2012.

Comments
  • Howard,

    You are just finishing up that gig in Houston and are back working with me in San Diego. You are having a vacation. We need to make a trip to Hodads to get rid of these CM 2012 nightmares and get you dreaming about food.

  • Between Hodads and those darn breakfast burritos - I'll need more than a vacation. Love the weather in San Diego!! Looking forward to working the summer with you.

  • I can't believe you're not going to share your app with us . C'mon, pretty please!

  • I have some packages were no packageid.ini files is listed on the distribution point. What can be the cause or is it by design?

    Thanks

  • Excellent walk through of how the single-instance store works. Thanks!

  • Do you know if SCCM 2012 provides an API to "recompile" this?

  • Hi Joachim

    Sorry it it's too late to ask about your package issue...

    The package object has a correct Package Source Path? You're searching the missing  packageid.ini files on a DP, if you are searching on DP... Are you sure that the Package content are distributed on the selectect DP, I try to update the content to see it the content library refresh the changes. And the last trick... pay attention on the monitoring - System Status - all message querys (All messages) may be the problen are logged.

    Samu

  • Howard....this is an excelllent post on explaining how the content lib sis functions.  Great Job!

  • AGREE with the previous comments, Howard! THis is the best I've yet to find that explains this quite complex process in detail as you've done. Thans much for this!!!

  • I was really hoping this would be *true* binary / block-level single-instance storage.   I don't see how this really would help much at all.  saving a few hundred kilobytes on a copy of psexec isn't much.  You'd have to have an extraordinarily large SCCM file structure to have enough identical files for it to really make a difference, and I'd still say it's going to be very little percentage. :(

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment