Eric Fleischman's WebLog : Windows

Change visibility in the directory...or lack there of (aka "what's the point of aging?")

efleis — Sun, 29 Oct 2006 01:14:00 GMT

I’m often asked about aging in adamsync so I thought I’d present the more general problem here for people to ponder. Hopefully this gives some context around the problem which aging in adamsync is supposed to address.

Imagine you are writing a tool which sync’s changes out of AD. You (the person running this tool) have some set of permissions…whatever they may be. You are syncing along happily.

One day you get a phone call…”My user was moved from OU=bar in to OU=foo yet the sync target still shows me in OU=bar. What gives?” You begin to investigate only to find out that you don’t have permissions to OU=foo. As a result, you don’t have any of the objects in OU=foo in your target location. The reason is straight forward….you don’t have permissions to the target, so when the object moved from bar to foo you never saw this change. You couldn’t see this change! You didn’t have permissions to OU=foo.

This is one of many such cases. If you don’t have the ability to see some object in the target location, it is hard to say anything about your view of it from the source. You could still have the object in the source location and have no idea that it moved out of your view. The reason is of course straight forward….you can’t see the target so you didn’t see that mod and we don’t have any construct where the source can say “out of your purview but not here anymore.” So you simply don’t realize the object has changed.

Historically, this was not nearly as much of a problem. Most people use DirSync to sync changes out of AD. In Win2K, in order to use DirSync you needed to be a domain admin. So, you could see most things that happen (out of the box anyway). In Win2k3 we built a feature for DirSync that made this problem more common….DirSync object security mode. In this mode anyone can use DirSync to sync out of any partition they so choose, and DirSync only shows changes for objects you have access to see. This is a very useful feature.

So now let’s consider adamsync, a simple DirSync client, and the problem I've mentioned above. When we wrote adamsync we wanted to ensure that we could handle the scenario where you are not an admin and want to sync data out. So, we default the tool to object security mode. This is fairly convenient for non-admins that wish to use the tool.

However, consider a very mainstream case. You are using adamsync to sync objects out of some domain NC. Objects are deleted. You don’t have permissions to see the deletions (remember I said that you “lose changes” when you move an object to a place you don’t have perms to? Well normal users don’t have permissions to the deleted objects container out of the box. So it’s a very common mainstream case for this problem….). As a result, you never reflect the deletion in your target container in ADAM. You’re woefully out of date.

One fix for this problem could be that you just give more perms. In the deleted objects case, just give the user who is syncing permissions to read the deleted objects container. But some people might not find that acceptable, depending upon their scenario.

Enter aging. We wrote aging to be a periodic background thread that goes and checks to make sure objects which we haven’t seen change in a while are actually still there. So you can imagine that every now and then you go back and check to ensure that all objects you have in the target are still in the source. This is the aging approach. While the specifics are configurable, that’s the basic idea.

Aging is just one such mechanism. There are lots of approaches to this problem that one could consider. It’s just the one we chose for adamsync.

One minor point I’ll raise before ending this post. Aging in ADAMSync in R2 is unfortunately not working properly. There is a bug that basically breaks it in some cases. It’s hard to say when but you should assume you’ll hit it at some point….no idea if you will, but you never know. So if you need aging pre-LH (ie, you have a compelling scenario where you want to sync as a non-admin) please open a QFE request with PSS. Or just give perms to deleted objects for now (or whatever the container is which you can't see)…a much easier quick-fix.

(just updated some formatting)

Finding the lost&found container in S.DS.P...or anything that isn't ADSI really

efleis — Fri, 27 Oct 2006 01:46:00 GMT

I found myself writing a piece of C# which would go hunt for objects in lost&found today. This is a pretty straight forward task….find that container, pop in to it and search away. I usually do this by looking at the lost&found well known GUID (which is GUID_LOSTANDFOUND_CONTAINER_W in the platform SDK) then just crafting the search by hand.

Anyway, I was feeling particularly lazy today so I went to take a quick look at MSDN and just use their sample. Much to my surprise the only examples I could find did this via ADSI and talked about “binding to well-known objects using WKGUID.” (I’ve NEVER been a fan of the use of terms there…I don’t like how we say “binding to an object” in ADSI as that is a somewhat unnatural construct given what’s really going on under the hood. Further, it makes moving to wldap32/S.DS.P/etc. harder as the terms are different. If only I ruled the world.....) Given this, I figured I’d paste some sample code here.

First, the LDAP work itself. To do a search for L&F, you want to craft a search as follows:

Base DN: <WKGUID=GUID_LOSTANDFOUND_CONTAINER_W,dc=someNC,dc=com>

Search filter: (objectclass=*)

Search scope: Base

To dissect that baseDN some…..WKGUID means “well known GUID”, instead of the string I pasted in italics (GUID_LOSTANDFOUND_CONTAINER_W) you would of course want to actually use the GUID there, and then after the comma you put the DN of the naming context you wish to search (for example, dc=mydomain,dc=com).

So here’s a quick and dirty piece of C# written against System.DirectoryServices.Protocols APIs that would get this DN for you. Please note that this is sample code so you really want to robustify it some before actually putting this in an application, and clean up the little foreach I have there just because I’m being lazy. ;)

string lAndFBaseDN = String.Format("<WKGUID={0},{1}>", GlobalVars.GUID_LOSTANDFOUND_CONTAINER_W, myNCDN);

string[] attrList = {"dn"};

string searchFilter = “(objectclass=*)”

SearchRequest sr = new SearchRequest(lAndFBaseDN,searchFilter,SearchScope.Base,attrList);

SearchResponse searchResponse = (SearchResponse)myLdapConnection.SendRequest(sr);

if (searchResponse.Entries.Count == 0)

{

// Must be no l&f container in the target dn (like, target not an NC for example)

}

Debug.Assert(searchResponse.Entries.Count == 1, "More than one L&F container found. This is weird.");

// If there is more than one, we just return the first. But there should not be.

foreach (SearchResultEntry sre in searchResponse.Entries)

{

return sre.DistinguishedName;

}

I think it goes w/o saying but you can apply similar logic to other well known GUIDs. I just picked l&f as it was convenient. ;)

Garbage collection & TSL warnings...why now?

efleis — Thu, 19 Oct 2006 23:17:00 GMT

I was recently pinged by a friend who is rolling out LH in their production environment. They were having an interesting issue where the LH DC showed these two events, in this order:

(event log entries snipped some for brevity)

Log Name: Directory Service

Source: NTDS General

Event ID: 1859

Task Category: Garbage Collection

Level: Warning

User: ANONYMOUS LOGON

Description:

Internal event: The current garbage collection interval is larger than the maximum value.

Current garbage collection interval (hours):

40000

Maximum value:

168

New value:

168

Log Name: Directory Service

Source: NTDS General

Event ID: 1088

Task Category: Internal Configuration

Level: Warning

User: ANONYMOUS LOGON

Description:

Internal event: The following tombstone lifetime registry value is too low or incompatible with the following garbage collection interval specified in the Active Directory Domain Services Configuration object. As a result, the following default registry values for the tombstone lifetime and garbage collection will be used.

This is an old domain which has been around for years. The win2k3 boxes are chugging along without a problem…only the LH boxes were throwing this event. So, the question is, what’s going on?

I took a peak at this for this friend and came up with the following. Thought I’d share it in case others see this too.

First event, the GC one. Well, the event told you the problem. J The GC interval is set to 40000 hours. That’s roughly “a long time.” GC interval is huge, we didn’t like it. We enforce in the product that GC is no greater than a week, in hours. So we went ahead and, in memory, set this to a week.

NOTE: We do not change the setting in the DS, we simply ignore it and use our own. So the setting stays wrong in AD.

Next event, the TSL event. This is the more interesting event. Well, we check a few things here. First, we make your TSL is greater than the min….check, this customer had a value which was ok (I forget, but it was somewhere in the teens). Next we check if the TSL is greater than 3 times the GC interval. You failed. Your GC interval is now 7 days (remember before I said it would set it to a week as your GC interval was nonsensically huge) but your TSL is not 3 times that. So we freak and show you the event. We further set TSL to the default in code, which is 60 days. Again, no config change, just in memory change.

Last item…why now? What changed? Why did this customer not get this before on 2k3? No values changed, after all.

This code is largely identical except for a small change. In 2k3 the code for this event firing code throws the event if GC logging is set to 2 or above. In LH, it fires the event if logging is set to 0 or above. Since all of the customer’s machines were set to 0, this explains why 2k3 didn’t throw an event whereas LH did.

(Personally, I’m happy about this change…I think we should have always flagged this condition. It’s weird, we should have always drawn your attention to it.)

Large AD database? Probably not this large...

efleis — Fri, 09 Jun 2006 01:32:00 GMT

Over the last few months there have been a series of threads in regard to max <fill in the item here...there have been many> in a database. These items have ranged from database size to # of objects and other such things. I figured, after the latest thread over on activedir.og, I'd do a little testing and put some numbers behind it so we could say "we have done this" and not "the system should do this."

What should this testing accomplish?

First, raw DB size. Gotta create a big DB or it probably doesn't matter.

Next, # of objects. For my testing, this was the real metric I was interested in. As mentioned over on ActiveDir (I would provide a link to the thread but I can’t seem to get the mail archives to work right now…I’ll try and provide one later), there is a theoretical max # of objects in the lifetime of a database which is, all said and done, 2^31 objects. I wanted to shoot for this. After all, Dean asked what error you would get, and I didn’t know. :)

I wrote a tool which started banging against an ADAM SP1 x64 instance. It was creating pretty small objects as I wanted reduce the amt of time this test took. My objects looked like this:

dn: cn=leafcontX,cn=parentcontY,cn=objectsZ,ou=objdata

changetype: add

objectclass: container

(Of course, sub in values for X, Y and Z as appropriate)

I had it use anywhere from 16 to 40 threads for this work depending upon the phase of import, and I simply wrapped around ldifde for it….I figured, there is a well tested tool for this, why not let it do most of the hard work?

Next, I got my hands on a test box (thx EEC!), put it on a SAN, installed ADAM, and away I went.

Along the way, we did a few other perf tests (looking at increased checkpoint depths and the like) so it added a bit of time to the import. However, after about a month, I had nearly filled my 2TB partition:

06/08/2006 10:41 AM 2,196,927,299,584 adamntds.dit

I created just shy of 2^31 objects. When I went to create that next object (done here by hand in LDP to illustrate the error)…

***Calling Add...

ldap_add_s(ld, "cn=sample1,OU=ObjData", [1] attrs)

Error: Add: Operations Error. <1>

Server error: 000020EF: SvcErr: DSID-0208044C, problem 5012 (DIR_ERROR), data -1076

If you look up -1076, you’ll find it is JET_errOutOfAutoincrementValues (from esent98.h). Woo hoo! I ran out of DNTs.

With this DB in hand, it was time to find out what else works and what else does not…

- Promotion of a replica fails. This makes perfect sense….it tries to create a couple of objects in the config NC, and that fails.

- Create of an NC fails. Again, to be expected, this task consumes DNTs.

- I ran esentutl /ms. It chugged for nearly 30 seconds, but worked perfectly.

- I also ran esentutl /k to make sure the DB did not have any physical corruption, but also to just see how long that took. :)

- Other standard tasks (kicking off garbage collection, online defrag, restarting the service, etc.) all worked perfectly.

- Search works like a champ. Sure it takes a good bit of I/O for most interesting searches, but that’s to be expected, of course.

It is worth noting that anything which failed did so gracefully. There were no nastygrams in my event logs either.

So for those of you who are worrying….you can sleep well at night now. We have tried rolling over DNT, and it works just fine.

A fun stat…..from the esentutl /ms output:

Name Type ObjidFDP PgnoFDP PriExt Owned Available

==============================================================

<EFleis – snip to save some space>

nc_guid_Index Idx 25 43 1-m 10870892 5

That owned number is in pages. That’s right, my NC_GUID index is 82.9GB…bigger than most databases. :)

While there were no major issues, we (Brett was looking at this too) did hit a few bumps along the way, and Brett was kind enough to write a few ESE tools for me to help monitor how we were doing. I’ll outline all of these things over the next few days as I have time to write them up. I’ll also provide more clarity around specific of what we did and saw as we went along.

"VGA"-like drivers for networking

efleis — Thu, 25 May 2006 18:20:00 GMT

One of the things that has always impressed me about keyboard, mouse and monitor support is that it just works. That is, you can plug in almost any keyboard, mouse and monitor, on basically any video card, and it there is some level of support provided by your OS/BIOS/etc. Independent of the OS. Independent of the generation of hardware.

This makes perfect sense. Without at least two of these three (some might argue that the mouse is optional, but I would disagree when you consider non-advanced users), you can’t bootstrap the system. You need to see something in order to load a better video driver. This was a feature born out of necessity.

Times have changed. User expectations around their PC have changed. Networking is mainstream now, not just for corporations. The # of high speed internet connections in the home is growing. Many, many people have and use NICs on a daily basis.

My wish

I think we should have a similar, fallback mechanism to bootstrap NICs to get users online. In the absence of a driver, I should not be required to find a floppy disk (if my PC even has a floppy drive….my new one at home does not to my knowledge) or burn a CD on another machine to get a good driver. I should be able to get online with basic functionality on a basic connection. I further expect my cable modem / dsl / etc. to work in this configuration. I expect this much to work. Then we can let something like WU or downloads from the manufacturers website give us the better driver.

I don’t know anything about writing drivers, so I ask you, the universe. Why does this not exist?

ADAMSync on pre-R2 systems

efleis — Mon, 21 Nov 2005 03:17:00 GMT

From the inbox.....

<quote>
You say the requirements are Win2003R2 with the ADAM installed from the R2 CD.
Will your guides work for us aswell? Running Win2003 SP1 in AD-environment and our developers are running standard Windows XP with the ADAM downloaded from MS download site?
</quote>

Good question! From a technical perspective, pointing ADAMSync at a pre-R2 ADAM is totally ok. And of course, syncing from an AD that has no DCs running R2 is a non-issue. We didn't take any dependency on anything in R2. That said, the following should be noted:
1) To the best of my knowledge, pointing at a pre-R2 ADAM is not an explicitly tested scenario. So perhaps there is some issue lurking here we're not aware of. I would categorize this as "exceptionaly unlikely", but I guess you never know.
2) There might be licensing ramafications to using ADAMSync if you don't use it on an R2 box. I'm not a licensing guy, nor do I play one on TV. I would recommend asking a licensing person this question.

ADAMSync can also transform users in to proxy users

efleis — Fri, 23 Sep 2005 21:04:00 GMT

Now that we have ADAMSync synchronizing our data over, we should probably investigate the most commonly asked for transformation: proxy user transformation.

When we introduced proxy bind in ADAM RTM, customers seemed to really connect with the semantic. If anything, I’d argue we have customers overusing proxy bind! But that’s a conversation for another day.

However, the introduction of proxy bind opened a bit of a management scenario that had not yet been seen. When you use proxy bind, you must first have created the objects to which you will proxy bind. That means one need create and maintain these new objects in the ADAM environment which correspond with the AD users. This seems like a natural scenario for ADAMSync.

In RC0 we enabled what we typically call “user to userProxy transformation.” This transformation is simple….one can take users being synchronized and create proxy users out of them. These proxy users may be of the Microsoft defined proxy user class (called userProxy) which has shipped with ADAM since RTM, they could be of the new class we added to R2 to help with this (userProxyFull), or they could be some custom class you have implemented (any class defined with an aux class of msds-proxybind will have the proxy behavior). We allow you to tweak this in your configuration file.

So let’s give it a try.
Before doing anything, we need to get a class defined that leverages the proxy bind functionality. For the sake of simplicity I’ll use the one that ships with ADAM:

C:\WINDOWS\ADAM>ldifde -i -f MS-UserProxy.LDF -s localhost -t 50000 -c "cn=configuration,dc=x" #configurationNamingContext
Connecting to "localhost"
Logging in as current user using SSPI
Importing directory from file "MS-UserProxy.LDF"
Loading entries....
3 entries modified successfully.

The command has completed successfully

Time for ADAMSync itself….
I’m going to go ahead and change the heart of the sync file as follows (things modified in red):

<?xml version="1.0"?>
<doc>
<configuration>
<description>sample Adamsync configuration file</description>
<security-mode>object</security-mode>
<source-ad-name>erictest.local</source-ad-name>
<source-ad-partition>dc=erictest,dc=local</source-ad-partition>
<source-ad-account></source-ad-account>
<account-domain></account-domain>
<target-dn>ou=SyncTargetOU</target-dn>
<query>
   <base-dn>dc=erictest,dc=local</base-dn>
   <object-filter>(objectCategory=person)</object-filter>
   <attributes>
    <include>objectSID</include>
    <include>sourceObjectGuid</include>
    <include>lastAgedChange</include>
    <exclude></exclude>
   </attributes>
</query>
<user-proxy>
    <source-object-class>user</source-object-class>
    <target-object-class>userProxy</target-object-class>
</user-proxy>
<schedule>
   <aging>
    <frequency>0</frequency>
    <num-objects>0</num-objects>
   </aging>
   <schtasks-cmd></schtasks-cmd>
</schedule>
</configuration>
<synchronizer-state>
<dirsync-cookie></dirsync-cookie>
<status></status>
<authoritative-adam-instance></authoritative-adam-instance>
<configuration-file-guid></configuration-file-guid>
<last-sync-attempt-time></last-sync-attempt-time>
<last-sync-success-time></last-sync-success-time>
<last-sync-error-time></last-sync-error-time>
<last-sync-error-string></last-sync-error-string>
<consecutive-sync-failures></consecutive-sync-failures>
<user-credentials></user-credentials>
<runs-since-last-object-update></runs-since-last-object-update>
<runs-since-last-full-sync></runs-since-last-full-sync>
</synchronizer-state>
</doc>

The new user-proxy section is what defines the transformation.
One can transform….well, anything to anything! :) So long as you are going from some sort of security principal in the source to a proxy user in the target, it’ll fly right along. I’m using userProxy just to keep it simple. Note that I also included objectSid as proxy users require the SID to be specified. Finally, I changed my search filter to look for object with an objectCategory=person just to isolate exactly what I wish to import.

C:\WINDOWS\ADAM>adamsync /install localhost:50000 ADAMSyncDemo.XML
Done.

C:\WINDOWS\ADAM>adamsync /sync localhost:50000 "ou=synctargetou" /log –

<chopped for brevity>

Finished (successful) synchronization run.
Number of entries processed via dirSync: 6
Number of entries processed via ldap: 1
Processing took 0 seconds (0, 1080131584).
Number of object additions: 7
Number of object modifications: 0
Number of object deletions: 0
Number of object renames: 0
Number of references processed / dropped: 0, 0
Maximum number of attributes seen on a single object: 5
Maximum number of values retrieved via range syntax: 0

And sure enough, when I go to look for some of the users I know should be there…..

>> Dn: CN=Administrator,CN=Users,OU=SyncTargetOU
3> objectClass: top; syncEngineAuxObject; userProxy;
1> cn: Administrator;
1> distinguishedName: CN=Administrator,CN=Users,OU=SyncTargetOU;
1> instanceType: 0x4 = ( IT_WRITE );
1> whenCreated: 09/23/2005 10:59:13 Pacific Standard Time Pacific Daylight Time;
1> whenChanged: 09/23/2005 10:59:13 Pacific Standard Time Pacific Daylight Time;
1> uSNCreated: 23644;
1> uSNChanged: 23644;
1> showInAdvancedViewOnly: TRUE;
1> name: Administrator;
1> objectGUID: 613813f4-f8cf-44ba-887b-aae4cb128580;
1> objectSid: S-1-5-21-980059532-776183279-2334900600-500;
1> objectCategory: CN=User-Proxy,CN=Schema,CN=Configuration,CN={B57A6E49-957D-434C-8584-9AA3D3946EF0};
1> sourceObjectGuid: P upy?B$6 ;
1> lastAgedChange: 20050923175913.0Z;

One of the most commonly made mistakes is forgetting to include objectSID so please do include it! If you don't, you'll get the missing attribute error we have seen before.

Synchronizing only the attributes you really want

efleis — Thu, 15 Sep 2005 19:00:00 GMT

In our previous ADAMSync runs we synchronized all attributes except those in the <exclude> tags. This is probably ok for our tinkering, but in a real scenario, you might want to consider picking those you want instead of getting everything but those you say not to.

Why? Well, consider the costs. If you synchronize everything, you’re paying the costs for all of those attributes (cost for lookup in AD, shipping them over the wire, writing them in to ADAM, storage in ADAM, etc.). If you only synchronize what you need you save on those costs while still servicing what you need in your application. And of course, you can always change your mind later. :)

The one tricky thing about this operation is picking the attributes you need. Consider that for some set of classes you’re creating, there is a minimum set of attributes that each class will require in order to be created properly. Should you miss some of them, you will get errors such as this one:

Processing Entry: Page 2, Frame 1, Entry 65, Count 1, USN 0
Processing source entry <guid=09e91eb3653f004fb8f8350d6ef2d577>
Processing in-scope entry 09e91eb3653f004fb8f8350d6ef2d577.
Adding target object CN=Domain System Volume (SYSVOL share),CN=NTFRS Subscriptio
ns,CN=EFLEIS-DF2,OU=Domain Controllers,ou=SyncTargetOU.
Adding attributes: sourceobjectguid, objectClass, instanceType, lastagedchange,

Ldap error occured. ldap_add_sW: Object Class Violation.
Extended Info: 0000207C: UpdErr: DSID-0315116B, problem 6002 (OBJ_CLASS_VIOLATIO
N), data 0

And 207C maps to:

C:\>err 207C
# for hex 0x207c / decimal 8316 :
ERROR_DS_MISSING_REQUIRED_ATT winerror.h
# A required attribute is missing.

So this can be tougher than it first appears. For my test environment (as mentioned previously, a fresh win2k3 forest), the following set of attributes was enough. But perhaps you will need more. If so, note the object that failed, and check out the attributes required on that object. Make sure you include all of them.

With that having been said, let’s go ahead and trim our attribute set down a bit. I’ll go ahead and only retain a subset of the attributes.
I’ll change this section:

<attributes>
    <include></include>
    <exclude>extensionName</exclude>
    <exclude>displayNamePrintable</exclude>
    <exclude>flags</exclude>
    <exclude>isPrivelegeHolder</exclude>
    <exclude>msCom-UserLink</exclude>
    <exclude>msCom-PartitionSetLink</exclude>
    <exclude>reports</exclude>
    <exclude>serviceprincipalname</exclude>
    <exclude>accountExpires</exclude>
    <exclude>adminCount</exclude>
    <exclude>primarygroupid</exclude>
    <exclude>userAccountControl</exclude>
    <exclude>codePage</exclude>
    <exclude>countryCode</exclude>
    <exclude>logonhours</exclude>
    <exclude>lockoutTime</exclude>
   </attributes>

To be:

   <attributes>
    <include>description</include>
    <include>frsstagingpath</include>
    <include>fRSRootPath</include>
    <include>sourceObjectGuid</include>
    <include>lastAgedChange</include>
    <exclude></exclude>
   </attributes>

Here’s where this list came from….
I first just decided I wanted object descriptions.
Then, I gave it a run. It complained with the error previously discussed. So I went to the class definition for the object and included the list of must contain attributes.
The last two attributes (sourceobjectguid and lastagedchange) are ADAMSync attributes themselves. These are used for internal tracking. So I went ahead and included them.

And with a little luck, it’ll work out just as well for you as it did for me.

Finished (successful) synchronization run.
Number of entries processed via dirSync: 169
Number of entries processed via ldap: 3
Processing took 10 seconds (0, 1085404416).
Number of object additions: 168
Number of object modifications: 4
Number of object deletions: 0
Number of object renames: 3
Number of references processed / dropped: 0, 0
Maximum number of attributes seen on a single object: 6
Maximum number of values retrieved via range syntax: 0

Syncing to our OU=SyncTargetOU NC instead

efleis — Wed, 14 Sep 2005 19:56:00 GMT

Earlier in this series of posts I changed our sync target from form “OU=” to “DC=”. This was done to carefully skirt around a small issue. Now with our new found knowledge of logging in ADAMSync, let’s give it another try.

So let’s go ahead in to our previous config file and change this line:
<target-dn>dc=SyncTargetDC</target-dn>

To read:
<target-dn>ou=SyncTargetOU</target-dn>

Of course, this hopefully means you have created one such NC in your ADAM environment.
From there, I went ahead and installed and ran my config:

C:\WINDOWS\ADAM>adamsync /install localhost:50000 ADAMSyncDemo.XML
Done.

C:\WINDOWS\ADAM>adamsync /sync localhost:50000 ou=synctargetou /log OULog1.txt

And when I cracked open the log, there were all sorts of errors.

Processing Entry: Page 2, Frame 1, Entry 22, Count 1, USN 0 Processing source entry <guid=ba50fb2e1bdd53468913b5d023460185> Processing in-scope entry ba50fb2e1bdd53468913b5d023460185. Adding target object CN=Builtin,ou=SyncTargetOU. Adding attributes: sourceobjectguid, objectClass, instanceType, showInAdvancedViewOnly, creationTime, forceLogoff, lockoutDuration, lockOutObservationWindow, lockoutThreshold, maxPwdAge, minPwdAge, minPwdLength, modifiedCountAtLastProm, nextRid, pwdProperties, pwdHistoryLength, uASCompat, lastagedchange, Ldap error occured. ldap_add_sW: Naming Violation. Extended Info: 00002099: NameErr: DSID-03050F78, problem 2005 (NAMING_VIOLATION), data 0, best match of:
'ou=SyncTargetOU'
. Ldap error occured. ldap_add_sW: Naming Violation. Extended Info: 00002099: NameErr: DSID-03050F78, problem 2005 (NAMING_VIOLATION), data 0, best match of:
'ou=SyncTargetOU'
.

First, notice that ADAMSync gives you all of the error text that ADAM returned to it. This is critical data.
So the question is, why did we fail?
Looking at the error in more detail:
Extended Info: 00002099: NameErr: DSID-03050F78, problem 2005 (NAMING_VIOLATION), data 0, best match of:

The most interesting piece of data here is the 2099, which maps to:

C:\>err 2099
# for hex 0x2099 / decimal 8345 :
ERROR_DS_ILLEGAL_SUPERIOR winerror.h
# The object cannot be added because the parent is not on the
# list of possible superiors.

And that’s the problem.
This entry was an attempt to create the object CN=Builtin under the parent object of OU=SyncTargetOU. This makes sense, we asked it to go in to OU=SyncTargetOU.
CN=Builtin is an object with an objectClass of builtinDomain:
>> Dn: CN=Builtin,DC=erictest,DC=local
2> objectClass: top; builtinDomain;
OU=SyncTargetOU is an OrganizationalUnit (specified when we created the NC in dsmgmt). We need to make it such that OrganizationalUnit is a possSuperior of builtinDomain. More info on possSuperiors can be found here.

So anyway, after making that change, I ran sync again…..

Finished (successful) synchronization run.
Number of entries processed via dirSync: 168
Number of entries processed via ldap: 2
Processing took 13 seconds (0, 1085446656).
Number of object additions: 52
Number of object modifications: 118
Number of object deletions: 0
Number of object renames: 112
Number of references processed / dropped: 58, 7
Maximum number of attributes seen on a single object: 18
Maximum number of values retrieved via range syntax: 0

As we said earlier, most ADAMSync failures are schema problems. :)

Update: Corrected a small typo in my before target DN. Thanks!

Getting a log from ADAMSync

efleis — Sat, 10 Sep 2005 23:15:00 GMT

Over the course of the next few posts we’re going to start modifying all sorts of things in the configuration. Depending upon the particulars of your environment this might or might not pan out. :) As such, we should probably take a quick look at the logging available before we break anything too badly.

When you run ADAMSync there’s a switch to give you enhanced logging:
/log [log file] -- Log messages, use "-" option to log to screen

I’m typically a fan of using a filename rather than – as the logs tend to get quite large.

From our run yesterday, here’s some of the output of the log (snipped for brevity):

Adamsync.exe v1.0 (5.2.3790.2021) Establishing connection to target server localhost:50000. Saving Configuration File on DC=SyncTargetDC Saved configuration file. ADAMSync is querying for a writeable replica of erictest.local. Establishing connection to source server efleis-df2.erictest.local:389. Using file .\damF.tmp as a store for deferred dn-references. Populating the schema cache Populating the well known objects cache Starting synchronization run from dc=erictest,dc=local. Starting DirSync Search with object mode security.

<snip>

Processing Entry: Page 1, Frame 1, Entry 4, Count 1, USN 0 Processing source entry <guid=51b2fc571aa27c4e9a488e7b79d1d5e1> Processing in-scope entry 51b2fc571aa27c4e9a488e7b79d1d5e1. Adding target object CN=Users,dc=SyncTargetDC. Adding attributes: sourceobjectguid, objectClass, description, instanceType, showInAdvancedViewOnly, lastagedchange, Previous entry took 0 seconds (31, 31) to process

<snip>

Beginning processing of deferred dn references. Processing deferred modifications for 53d9609b8ee6014c947f57d3fc850aab:ipsecISAKMPReference. + Synchronizing dn-ref to 39b749b601b1f547a2a76a97e5beb0f2.

<snip>

Finished processing of deferred dn references.
Finished (successful) synchronization run.
Number of entries processed via dirSync: 169
Number of entries processed via ldap: 3
Processing took 7 seconds (0, 1082877952).
Number of object additions: 168
Number of object modifications: 4
Number of object deletions: 0
Number of object renames: 3
Number of references processed / dropped: 58, 7
Maximum number of attributes seen on a single object: 18
Maximum number of values retrieved via range syntax: 0
Beginning aging run. Aging requested every 0 runs. We last aged 1 runs ago. Saving Configuration File on DC=SyncTargetDC Saved configuration file.

Alright, so time to slice it up some. This post would go on forever if I sliced it up too much, so I’ll point out some of the highlights.

Of course, we start off with just a bit of overview of what we’re about to do. Version of the tool, host we’re talking to, etc. Nothing out of the ordinary. :)

Next up, object sync. We will have an entry for objects synchronized. Note the GUID that is listed. While this might not look like the sort of GUID you’re used to, it is actually the GUID of the object in the source NC. There are two potentially confusing things about what I just said, so I’d like to call them both out so we’re all on the same page, else things will only get worse from here :):
1) It is the GUID despite looking like it is wrong. Note the form:
51b2fc571aa27c4e9a488e7b79d1d5e1
The GUID of that object when all prettied up is:
57fcb251-a21a-4e7c-9a48-8e7b79d1d5e1
Here’s the magic to the conversion. Let’s look at just the first section:
57fcb251 vs. 51b2fc57
See the pattern yet? Let’s color it in:
57fcb251 vs. 51b2fc57
Yes, there’s some ordering going on. I won’t get in to why here, but that’s how we pretty up GUIDs before we display them. With that example I’m sure you can walk through the rest of it and figure out where it comes from.
2) It is the GUID of the object in the source NC. One of the confusing things about synchronization is that we’re doing logical recreation of some data state in the target environment based upon what we see in the source environment. As a result, some properties aren’t the same. One such property is the GUID. The “copy” of the object in the target will have a different GUID, because all we’re really doing in ADAMSync is telling ADAM to create an object with the following logical properties (name, description, etc.)…namely those properties we care about (you get to pick the list)...and letting the stuff going on at the directory layer do it’s thing. So much like we didn’t tell AD the GUID of the object in the source, we don’t tell ADAM what the GUID should be in the target.
The result of this subtle yet important distinction is that when thinking about tasks ADAMSync is doing it is exceptionally important to consider if the task is relative to the source or the target. Things get awfully confusing awfully fast if you don’t.

We then get in to deferred dn references. ADAMSync processes things like link value attributes later such that all objects are already created when it is time to create the links. We can revisit this later in more detail if people are interested in the subtleties of what and how and why.

And finally, closing statistics with a little note that we succeeded.
Errors will be painstakingly obvious. We’ll probably start seeing some as we start modifying our config file in some crazy ways over the next few posts. I’ll try and include the common errors so you get a sense of what I think is likely that you’ll hit as well as my general methodology in approaching these sorts of errors.

Update: Fixed some formatting issues I didn't notice before.

Configuring and running ADAMSync for the first time

efleis — Fri, 09 Sep 2005 00:50:00 GMT

So now that our ADAM schema is ready, we can go ahead and start configuring ADAMSync.

We need to do three things to get ADAMSync running at this point:
1) Modify the XML file. The XML file is used as the configuration point for ADAMSync, so we’ll tweak this file to have the settings we desire.
2) Install the XML file in to ADAM.
3) Run ADAMSync to perform the synchronization itself.

Step 1: Modifying the XML file
Let’s try and keep it simple to start. The easiest way to get off of the ground is just to pick the proper target, do the fewest modifications required and then run with it.

ADAMSync ships with a sample XML file which has many of the commonly used tags. I typically work from this as my starting point.
I started off by making a quick copy of the default ADAMSync config to use.

C:\WINDOWS\ADAM>copy MS-AdamSyncConf.XML ADAMSyncDemo.XML
1 file(s) copied.

Now with that in hand, we can start making some changes for our first run of the tool.

To get things started, I’d suggest we first make the absolute bare minimum in terms of # of changes. We can explore different parts of the configuration file as we move forward, but it helps to first have a working configuration to work from.
Here’s what my config file looks like (with section I modified from the default in red):

<?xml version="1.0"?>
<doc>
<configuration>
<description>sample Adamsync configuration file</description>
<security-mode>object</security-mode>
<source-ad-name>erictest.local</source-ad-name>
<source-ad-partition>dc=erictest,dc=local</source-ad-partition>
<source-ad-account></source-ad-account>
<account-domain></account-domain>
<target-dn>dc=SyncTargetDC</target-dn>
<query>
   <base-dn>dc=erictest,dc=local</base-dn>
   <object-filter>(objectClass=*)</object-filter>
   <attributes>
    <include></include>
    <exclude>extensionName</exclude>
    <exclude>displayNamePrintable</exclude>
    <exclude>flags</exclude>
    <exclude>isPrivelegeHolder</exclude>
    <exclude>msCom-UserLink</exclude>
    <exclude>msCom-PartitionSetLink</exclude>
    <exclude>reports</exclude>
    <exclude>serviceprincipalname</exclude>
    <exclude>accountExpires</exclude>
    <exclude>adminCount</exclude>
    <exclude>primarygroupid</exclude>
    <exclude>userAccountControl</exclude>
    <exclude>codePage</exclude>
    <exclude>countryCode</exclude>
    <exclude>logonhours</exclude>
    <exclude>lockoutTime</exclude>
   </attributes>
</query>
<schedule>
   <aging>
    <frequency>0</frequency>
    <num-objects>0</num-objects>
   </aging>
   <schtasks-cmd></schtasks-cmd>
</schedule>
</configuration>
<synchronizer-state>
<dirsync-cookie></dirsync-cookie>
<status></status>
<authoritative-adam-instance></authoritative-adam-instance>
<configuration-file-guid></configuration-file-guid>
<last-sync-attempt-time></last-sync-attempt-time>
<last-sync-success-time></last-sync-success-time>
<last-sync-error-time></last-sync-error-time>
<last-sync-error-string></last-sync-error-string>
<consecutive-sync-failures></consecutive-sync-failures>
<user-credentials></user-credentials>
<runs-since-last-object-update></runs-since-last-object-update>
<runs-since-last-full-sync></runs-since-last-full-sync>
</synchronizer-state>
</doc>

Step 2: Installing the XML file
With our XML file in hand, installing it in to the ADAM environment is a piece of cake.
It’s worth noting that the configuration itself is stored in the target environment (ADAM), not in the source. So we’ll focus the install command against our ADAM environment.

C:\WINDOWS\ADAM>adamsync /install localhost:50000 ADAMSyncDemo.XML
Done.

After installing it, I usually ask ADAMSync to list all of the configuration files once so that I can confirm it made it there.

C:\WINDOWS\ADAM>adamsync /list localhost:50000
Listing configuration files:
---------------------------
|-> "OU=SyncTarget": sample Adamsync configuration file
Done.

Looks good!

Step 3: Running ADAMSync for the first time
Our first run….let’s see what happens!

C:\WINDOWS\ADAM>adamsync /sync localhost:50000 dc=synctargetdc

C:\WINDOWS\ADAM>

ADAMSync just returned without error. This is the real measure of success. :) If there was a problem, we would have received some verbiage.
At this point we should be able to look at the target NC and the data should be there.
Did you get an error? If so, hang tight, tomorrow we’ll take a look at some of the logging available and look at one such log.

Gotta get the schema right first

efleis — Thu, 08 Sep 2005 07:15:00 GMT

Before trying to synchronize much of anything, we need to make sure the required schema elements are in place.
NOTE: I would estimate that around a third of all questions I've received on ADAMSync came back to schema problems. It is critically important that the schema be configured properly.

For ADAMSync to properly synchronize your data, there are really two things that need to exist in the target environment:
1) The ADAMSync schema extensions. ADAMSync itself has a series of schema elements which need to be present in the target ADAM environment. These schema elements are used for ADAMSync itself to track configuration information.
2) Whatever schema elements are needed for the data you would like to synchronize. Remember that ADAM, out of the box, has a schema which contains far fewer elements than that of AD. If you’re going to synchronize data from AD in to ADAM which uses schema extensions in AD which are not yet in ADAM, we need to get them in there.

So let’s first prep our ADAM environment by importing the ADAMSync schema extensions.

C:\WINDOWS\ADAM>ldifde -i -f MS-AdamSyncMetadata.LDF -s localhost -t 50000 -c "cn=configuration,dc=x" #configurationNamingContext
Connecting to "localhost"
Logging in as current user using SSPI
Importing directory from file "MS-AdamSyncMetadata.LDF"
Loading entries..........
9 entries modified successfully.

The command has completed successfully

For the required elements for the objects themselves, that can often be quite a bit more tricky.
The real issue is that synchronizing the schema from AD to ADAM is often a tougher task. We heard this loud and clear, and actually built a tool to help with this (included in ADAM as well….ADSchemaAnalyzer). So as to keep things simple, let’s just ignore that for now, and instead import one of the ldif files included with ADAM that has what we’ll need for this first demo. Later on we’ll revisit this and do a more targeted schema move when we look at ADSchemaAnalyzer in more depth (hopefully, after the ADAMSync posts are done, we’ll look at that next).

C:\WINDOWS\ADAM>ldifde -i -f MS-AdamSchemaW2K3.LDF -s localhost -t 50000 -c "cn=configuration,dc=x" #configurationNamingContext
Connecting to "localhost"
Logging in as current user using SSPI
Importing directory from file "MS-AdamSchemaW2K3.LDF"
Loading entries.................................................................
.................................... ............................................
................................................................ ................
................................................................................
........ ........................................................................
.................................... ............................................
................................................................ ................
................................................................................
........ ........................................................................
.................................... ............................................
................................................................ ................
................................................................................
........ .........................................................
1009 entries modified successfully.

The command has completed successfully

Now that our schema is ready, let’s go ahead and make sure that we have the proper NCs. Yesterday I mentioned off-hand that we’ll create one with a given name, but I’d like to just change that name slightly so skirt around some schema issues for now. Never fear, we’ll get back to a target NC of type OrganizationalUnit a bit later, as this is also a very common question.

C:\WINDOWS\ADAM>dsmgmt
dsmgmt: partition management
partition management: connections
server connections: connect to server localhost:50000
Binding to localhost:50000 ...
Connected to localhost:50000 using credentials of locally logged on user.
server connections: q
partition management: list
Note: Directory partition names with International/Unicode characters will only
display correctly if appropriate fonts and language support are loaded
Found 2 Naming Context(s)
0 - CN=Configuration,CN={B57A6E49-957D-434C-8584-9AA3D3946EF0}
1 - CN=Schema,CN=Configuration,CN={B57A6E49-957D-434C-8584-9AA3D3946EF0}

Ah, nothing but schema and configuration. This is because during the install, I didn’t actually specify any NCs that I’d like it to create. Let’s go ahead and create one so we have a place to synchronize our data in to:

partition management: create NC dc=SyncTargetDC DomainDNS NULL
adding object dc=SyncTargetDC

Next stop, our first synchronization task.

Edit: Minor formatting nit

ADAMSync in R2 - a new sync option

efleis — Tue, 06 Sep 2005 19:34:00 GMT

Welcome R2 RC0! As you might have heard, it’s hit the streets. With R2 comes the latest version of ADAM.

ADAM has been a download from Microsoft.com since 2003. Since that time, based in part on customer feedback, the decision was made to bundle ADAM in to the OS as an optional install. For those of us that work on ADAM, we like this idea a lot. :) No longer will we have to point customers at a separate download package. R2 is the first release where ADAM is bundled. One can install ADAM via Add/Remove Programs.

In ADAM R2, we bundled a whole bunch of new features many of which are in the form of new tools. These are a direct result of customer feedback which we wanted to address. I’ll try and touch on many of them over the next few weeks. However, I think ADAMSync needs the first (and perhaps most) coverage because of what it is, what it does, and what you can do with it.

So anyway, let’s get to it. What is ADAMSync?

Since the release of ADAM, we’ve heard loud and clear that people wanted help in the synchronization space. We were told there was a strong desire for a “basic” one-way synchronization of data from AD in to ADAM. The ‘folks asking for this didn’t want complex implementation of business logic but rather just wanted data coalescing in to their ADAM environment with some configuration along the way.

Enter ADAMSync. ADAMSync provides just that: simple data synchronization from AD in to ADAM naming context. There is not complex customization available but rather it’s a very basic “move the data from point A to point B” approach. The upside of this simplicity is that setting up ADAMSync is a much easier operation that takes far less time than other options out there today. To put a number behind it: I recently traveled to a customer site and while there set up ADAMSync with them in under half an hour.

From a technical perspective, ADAMSync leverages DirSync to get the heavy lifting done. For the initial sync of a data set, this means sourcing all of the specified data and writing it in to ADAM. Going forward, when you do periodic synchronization, the cost of this operation is small, which is really where DirSync shines.

Over the next few posts I’ll take you through a tour of what ADAMSync can do, and how to set it up. Particular emphasis will be placed on how to set it up in a few different scenarios.

In preparation for these next few posts, I’d suggest you set up a small test environment in which you can tinker with some of this. The test environment I’m using in writing these posts is:
-         Windows Server 2003 SP1 RTM with R2 RC0 installed
-         Dcpromo’d in to a test domain. (Mine is erictest.local)
-         ADAM is installed with a single naming context having been created. (Mine is OU=SyncTarget)

If all goes according to plan, the configuration done in these examples will be able to be done to your test environment without modification.

And with that, we’re off!

AWE in WOW64 on x64

efleis — Fri, 22 Jul 2005 20:06:00 GMT

Here in the EEC, was asked a good question the other day, thought I’d pass it along.

So the scenario is that the engagement in question was to run SQL 2000 SP4 on a box running Win2k3 x64 Enterprise Edition. The server had a good chunk of RAM (I think 8GB, but it might have been 16GB). The question was, is AWE on the table in this scenario for SQL?

I knew that AWE is not an option in WOW64 on Itanium. I had bumped in to this before. But I had never been asked about WOW64 on x64. I did some digging in the docs (seems like something we’d document) and found that the docs talked about how it isn’t an option on Itanium, but didn’t explicitly call it out in the x64 section either way. Here’s a sample of what I mean.

So with that, I was unsure. I guessed that it would not be an option. Then I asked someone who would know.
I guessed wrong.

It turns out, AWE in WOW mode does in fact work on x64, unlike ia64. Why? Well I had to ask, and got what was a very logical explanation.

It stems from the reason that it _doesn’t_ work on Itanium. X86 binaries use 4K pages. IA64 uses 8K pages. So we couldn’t make the transition work well (don’t ask me why, this is what I was told, and I believe the guy that told me J). However, x64 boxes use 4K pages, just like x86. So making AWE work in that condition wasn’t a problem.

Another random AD change in SP1

efleis — Wed, 13 Jul 2005 08:41:00 GMT

The other day over on activedir it was mentioned that in SP1 we added sidHistory to the list of attributes preserved on tombstones. I thought I’d mention this here, and provide some clarity around what actually changed, since there was some confusion around what we did and how we did it.

In looking at tombstone reanimation, we realized that sidHistory is particularly tricky to restore. For a variety of reasons, we enforce rules on the insertion of values in to sidHistory when using the proper APIs (like DsAddSidHistory()). That said, restoring this attribute to a previous state can be non-trivial.

Therefore, it makes sense to preserve this one on tombstones, such that if you reanimate a tombstone you get it back like it was before.

Normally when such changes are made, they are done in the schema. On each attribute definition, we have an attribute searchFlags, and within searchFlags there is a bit which tells the DS to preserve attributes on tombstones (0x8). So adding an element to this set of preserved attributes is as simple as flipping this bit.

Now I said “normally” above, because in this case, we didn’t do this. In addition to looking at searchFlags, there is actually a second mechanism by which we decide what to preserve on tombstones. We maintain a hard-coded list in the DSA code itself. I can’t authoritatively say why we did this (before my time) but I would speculate it is so that even a schema change which removes something we really need from this attribute set doesn’t cause us to enter a state which could be detrimental to the environment.

So, in SP1, we added sidHistory to that list of always preserved attributes.

Why did we add it to this list rather than changing searchFlags? Well, feedback has been that schema changes are painful, and that they slow adoption. Therefore, we clearly didn’t want to have a schema change associated with SP1. Further, this isolated the change to only SP1 DCs, which was desirable. You don’t have a forest wide change at one moment in time, but rather incrementally introduce this. While we don’t expect a problem (and haven’t seen one to date), it’s safer to always assume one is there and we just don’t know about it yet.

What’s the downside of doing this vs. changing searchFlags? Well, in making this change in the code vs. searchFlags, this means that you could have an inconsistent experience if you have a mix of DCs, some with SP1 and some without. Namely, if you delete an object on an SP1 DC, it will preserve sidHistory. But if you delete it on a non-SP1 DC, it won’t. We understand this isn’t ideal, but if you want to get this feature today w/o waiting for Longhorn and without making a schema change, this is the best we could do. :)

(Edit: fixed weird font thing)