Lookin At Some AD Dumpage

Lookin At Some AD Dumpage

  • Comments 5
  • Likes

Understanding how things work is as much of a tool as anything included in the Adminpack or Support Tools.    If you know how things work, you know what to expect in normal behavior, and what questions to ask when the behavior your seeing is abnormal.

 

For example… J

 

We had an customer who was contacting us because they had something pretty unusual they were seeing in their Active Directory.  What they saw was a partially GUIDed , in the manner you see objects that have been tombstoned, entry appearing in the list of members of a security group.  This entry was recognizable as a particular user—let’s call him Bob—that had been removed many months prior.  There was a dim recollection that this user was one from another forest who had been added to this security group for awhile.

 

It looked like this:

 

CN=Clown Big Bob\0ADEL:6ca094b1-934d-407c-bed8-65b8e2949da1,CN=Deleted Objects,DC=silliness,DC=org

 

An even more unusual part of viewing this entry in the properties of a group in DSA.MSC was that the entry could not be removed from the list.  An error would be given when you select the members tab of the Administrators group “No superior reference has been configured for the directory service. The directory service is therefore unable to issue referrals to the objects outside the forest.”

 

If you clicked OK and then deleted the entry it would not give an error, but the entry (really a back link) would not delete.

 

Quite a puzzler.  We thought it could be database corruption, so we used NTDSUTIL to check the integrity of the database for the DCs but that didn’t help.

 

Before we thought things through we thought that maybe REPADMIN /REHOST would do the trick.  When you think about it, though, the user is not one of that forest, so he won’t be in the global catalog at all (those only host partial attribute sets of naming contexts in their own forest).

 

Finally we thought we should look at this object in the database itself.  To do that we used LDP.EXE to send a DUMPDATABASE control to the AD.  This exports the database to a .DMP file, which is viewable in Wordpad.exe.  Obviously, you have to be an admin to do that, so no worries there.

 

Here’s an article on how to do a dumpdatabase by the way:

 

How to Use the Online Dbdump Feature in Ldp.exe

http://support.microsoft.com/kb/315098

 

Why did we think this would be a good thing to do? Well, there are different types of objects in AD, and different properties of them, some of which are only viewable in this manner.  Even better is a method of tracing an object by a unique identifier (this will ring bells with those of you out there familiar with Exchange or other Jet databases) called a distinguished name tag, or DNT.   Each object also has a parent distinguished name tag, or PDNT.  That’s the child objectàparent container relationship, such as when you have a user object in an OU.

 

But I digress.

 

What we found when we looked at this object was how it looks here, with its parent containers:

 

DNT     PDNT  Ver        IsObject

10013  5224   2      -      false 2005-07-28 14:00:47 -     11     BigTop      BigTop             -      -      255ce266-8bbb-45b0-a5c6-109a44aead7f -  

  -    

10014  10013  4      -      false 2005-07-28 14:00:47 -     11     Circus Clowns OU    Circus Clowns OU    -      -      38123a11-4b10-4b78-a11c-51c0161e58f8 -  

   -    

10015  10014  1      -      false 2005-07-28 14:00:47 -     11     袈઎˜ ”“畷ِ઎袈઎˜ •·ÙàªŽè¢ˆàªŽÂ˜ -Ȇ ” àªŽæ‡ àªŽ ”੼  €í-흥⣪艋î-€çƒ¿æ”˜Â¡ ’ˆà©¼î’ˆCNF:27fd187b-086f-4a18-b3f5-1525e9b06637 袈઎˜ ”“畷ِ઎袈઎˜ •·ÙàªŽè¢ˆàªŽÂ˜ -Ȇ ” àªŽæ‡ àªŽ ”੼  €í-흥⣪艋î-€çƒ¿æ”˜Â¡ ’ˆà©¼î’ˆCNF:27fd187b-086f-4a18-b3f5-1525e9b06637 -      -      27fd187b-086f-4a18-b3f5-1525e9b06637 -    

 -    

10016  10015  1      -      false 2005-07-28 14:00:47 -     11     袈઎´ ”“畷ِ઎袈઎´ •·ÙàªŽè¢ˆàªŽÂ´ -Ȇ  ઎ ”੼  €í-흥⣪艋î-€çƒ¿æ”˜Â¡ ’ˆà©¼î’ˆ

CNF:f2644711-3177-4b05-afad-c17b0b835b0a 袈઎´ ”“畷ِ઎袈઎´ •·ÙàªŽè¢ˆàªŽÂ´ -Ȇ  ઎ ”੼  €í-흥⣪艋î-€çƒ¿æ”˜Â¡ ’ˆà©¼î’ˆ

CNF:f2644711-3177-4b05-afad-c17b0b835b0a -      -      f2644711-3177-4b05-afad-c17b0b835b0a -     

-    

10017  10016  2      -      false 2005-07-28 14:00:47 -     3      bigbob       bigbob            -      -      51d1b8a1-080f-4b11-93aa-6ea1d1dd36de -  

 -    

 

I placed a few of the field names as hints.  When you look at the above there are several things which jump out at you.

 

-Two of the OUs which BigBob lived in are corrupt.

-The “isobject” value says False. 

 

Why are these things important to this drawn out story?  Well, all objects in a domain and forest will show True as objects—even if they are deleted objects (tombstoned).  But phantom objects will show False.  Add to that the fact that two of the parent OUs were showing messed up and we have an interesting hypothesis.

 

What if BigBob from the BigTop domain (in a different forest) was added to Silliness.org security group a long time ago.   When that’s done the AD creates a phantom object and structure there to have some data about this principal onhand.  Then something bad happens in the database and sometime around there BigBob is deleted in his original forest.

 

Here’s a reference on scary phantoms:

 

248047 Phantoms, Tombstones and the Infrastructure Master

http://support.microsoft.com/?id=248047

 

We would expect, in a working scenario, for BigBob and his parent OU phantoms to be garbage collected after the tombstonelifetime.  But in his case, this doesn’t happen because the garbage collector encounters difficulty when it sees these munged parent phantoms which BigBob resides in.

 

In case you’re wondering we tried manually forcing garbage collection and didn’t see what we hoped for.  Here’s the step to do that:

 

1.>In Ldp.exe, when you click Browse on the Modify menu, leave the  Distinguished

name box empty.

2.>In the Edit Entry Attribute box, type  "DoGarbageCollection" (without the

quotation marks),

3.>In the Values box, type  "1" (without the quotation marks).

4.>Set the Operation value set to Add and click the Enter button, and then click

Run.

 

But all we saw when we did so was this event and no removal of the offending back link and phantom

 

Event Type:       Information

Event Source:    NTDS General

Event Category: Garbage Collection

Event ID:           1132

Date:                7/28/2005

Time:                11:00:47 AM

User:                NT AUTHORITY\ANONYMOUS LOGON

Computer:         ClownDC1

Description:

Internal event: The Directory Service removed the expired, deleted object

CN=Clown Big Bob\0ADEL:6ca094b1-934d-407c-bed8-65b8e2949da1,CN=Deleted Objects,DC=silliness,DC=org

from the database.

 

The above was patently untrue.  Without drawing this out too long ultimately we (it was not my idea though I’d love to take credit for it) used LDP.EXE to delete the phantom using it’s objectGUID, and then our back link from the nether regions went away for good.

 

Here’s how to do that:

1) Open LDP.EXE.

2) Connect to the directory and BIND using administrative credentials.

3) Pull down the Modify menu and select Delete.

4) In the DN: area type in the objectGUID in the format of: "<GUID=objectGUID>"

(without the quotes).

 

For example:

<GUID=41019973-cc91-498b-8ffd-77152738b9a9>

 

The returned message in the LDP right hand pane should be similar to this if

succesful:

 

ldap_delete_s(ld, "<GUID=41019973-cc91-498b-8ffd-77152738b9a9>");

Deleted "<GUID=41019973-cc91-498b-8ffd-77152738b9a9>"

 

Am I saying go out and what some objects or phantoms using LDP as soon as possible?  No.  But we’ve gone over some less common stuff in a real world scenario.  This knowledge may save you some grief down the road.  My little gift to you, hopefully you never need it but it’s there if you do.

 

Happy Holidays everyone.  I hope your year ends on a wonderful note.

 

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • With regard to the link http://support.microsoft.com/?id=248047

    I quote: "Certain types of groups in an active directory domain can contain accounts from trusted domains. To make sure the names in the group's membership are accurate, the user object’s GUID is referenced in the membership of the group." I don't think this is true ... LDP and ADSIEdit show that foreign forest memberships are SID references to the ForeignSecurityPrincipals OU, not GUID's, in the "members" attribute of groups.

    Comments?

  • There is a GUID associated with the phantom object which is created as a placeholder for directory operations relating to that foreign principal.  I don't have a list of all operations which instantiate the phantom and it's structure though, before you ask.  

    Keep in mind I'm talking about the database layer here; this is not information you would normally see or need to see.

    Peform a test of your own, dump that database of the trusting domain/forest and take a look around.

  • Hello Tim,

    Very very interseting thread !

    For my understanding, you said "two of the parent OUs were showing messed up.." Could you (re)explain how you found this with the dump ?

    Thanks,

  • Sure!  You can see it more clearly when you look at other objects (try a sample from a database dump in your own environment), but the "garbage" characters like below:

    袈઎˜ ”“畷ِ઎袈઎˜ •·ÙàªŽè¢ˆàªŽÂ˜ -Ȇ ” àªŽæ‡ àªŽ ”੼  €í-

    are a giveaway.  This should be a string of text such as a name of the OU; you can see a parent shows Clown OU for that same field.  We should expect the same legibility in all objects.  It should not be unreadable to you, no matter what unique name you choose for the object or localized version of Windows Server you may be using.

    One point though: a CNF in an object name is not necessarily indicative of corruption, though it could be suggestive of it.

    Tim

  • Ok, i understand where the corruption is.

    Last Q: How do you identify "Two of the OUs which BigBob lived in are corrupt" as ... OUs object type ? That could be any objects in AD...

    I think i misunderstand the meaning of DNT & PDNT and how they interact each other in the dump.

    Thanks Tim for clarification if you have time to... :)

    Yann