This post is a part of the Metadata series. Have a look at the intro to have more information about it:

Metadata #0 - Metadata, what is it and why do we care?

If you are not familiar with querying Active Directory for replication metadata, please read this post:

Metadata #1 - When did the delegation change? How to track security descriptor modifications

It will give you a good overview.

Here is the scenario.

  • Someone has been added to the Domain Admins group, did something and removed himself from the group (or basically any other groups)
  • Audit is not enabled or configured properly

You might think that because the audit didn't catch it it is too late to spot this change. Well it is not. And this is how to look at this information.

Forest Functional Level

I know you tell yourself "why are we talking about functional levels?". Actually it is the foundation of everything we're going to see in this post. If you don't know what is a forest functional level, please consult this article: Understanding Active Directory Domain Services (AD DS) Functional Levels. Before the Windows 2003 FFL native mode (you got it, that is the short for Forest Functional Level) when you add a user into a group, you have to replicate the entire membership with the replication partners. So if the group has thousands of members already, this is a lot of information to replicate. This even caused issues in the past and still does so in old environments. Well - adding a user into a group has nothing to do with the user. It is in fact modifying the attribute member of the group object. This is why in terms of permissions, to be able to add a user into a group, you don't need any permission on the user. You just need to be able to modify the member attribute on the group you want to add the user to.
Everything changed with the Windows 2003 FFL native mode. We have introduced the notion on linked value attributes. In a nutshell when we add a user on the group, we do not replicate all the membership anymore but just the new links we just created. This also means that domain controllers need to keep track of those links. And to do that, you've guessed, domain controllers use replication metadata. And this is very powerful! When you add a user into a group, we need to replicate that. When you remove the user from the group, we need to replicate that too. We need to tell to our replication partners that the user isn't a member of the group anymore, and we are storing that in the metadata. Just reading the metadata will tell us who the current members are, as well as the "removed" members.

msDS-ReplValueMetaData

In the previous post, we just dealt with the attribute msDS-ReplAttributeMetaData. This one contains the replication metadata for the "classic" attributes. For the linked attributes we use another one called msDS-ReplValueMetaData. It follows the same rules as its brother, it is a calculated attribute therefore we need to specifically ask for it and it could be empty if the object does not have any linked value attributes (or an empty group which has always been empty for example).

First look with repadmin

Again, with our out of the box repadmin, we can display the metadata for a given object, including the linked value metadata.

C:\>repadmin /showobjmeta dc2012 CN=Administrators,CN=Builtin,DC=contoso,DC=com

17 entries.

Loc.USN                           Originating DSA  Org.USN  Org.Time/Date        Ver Attribute

=======                           =============== ========= =============        === =========

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 objectClass

   8015            Default-First-Site-Name\DC2012      8015 2013-07-12 15:31:26    1 cn

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 description

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      2791 2000-07-02 20:13:07    4 member

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 instanceType

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 whenCreated

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 showInAdvancedViewOnly

   8015      ecc31b35-05d1-406a-941c-581804e9ad3c     18524 2008-07-21 17:15:53    4 nTSecurityDescriptor

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 name

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 objectSid

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      2814 2000-07-02 20:16:22    1 adminCount

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 sAMAccountName

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 sAMAccountType

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 systemFlags

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 groupType

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 objectCategory

   8015      a6768194-9d5a-43ab-9139-97da4b7fbca1      1416 2000-07-02 19:59:15    1 isCriticalSystemObject

7 entries.

Type    Attribute     Last Mod Time                            Originating DSA  Loc.USN Org.USN Ver

======= ============  =============                           ================= ======= ======= ===

        Distinguished Name

        =============================

LEGACY        member

        CN=Administrator,CN=Users,DC=contoso,DC=com

LEGACY        member

        CN=Enterprise Admins,CN=Users,DC=contoso,DC=com

LEGACY        member

        CN=Domain Admins,CN=Users,DC=contoso,DC=com

LEGACY        member

        CN=Alice,OU=People,DC=contoso,DC=com

PRESENT       member 2014-08-02 13:24:40         Default-First-Site-Name\DC2012   24737   24737   1

        CN=Bob,OU=People,DC=contoso,DC=com

PRESENT       member 2014-08-02 13:25:27         Default-First-Site-Name\DC2012   24743   24743   3

        CN=Charles,OU=People,DC=contoso,DC=com

ABSENT        member 2014-07-14 22:12:55       Default-First-Site-Name\DC2008R2   16400   16434   2

        CN=postmaster,OU=Audit,OU=People,DC=contoso,DC=com

We can see 3 different types of entries:

  • LEGACY means that the member was in the group before you raised the FFL to Windows 2003 native mode. If you want to know when was the last time a member has been added to a group in LEGACY type, you can look at the replication metadata for the attribute member of the group. The version number of the member attribute will also tell you how many operations of add/remove have been performed on this group before the FFL Windows 2003 native. In the example above you can see that the member attribute has been changed 4 times and the last time was 2000-07-02 20:13:07. Then the FFL changed therefore we are not using this attribute directly anymore. Note that it doesn't mean that the FFL changed at that time, it probably indicates that the membership of this group hasn't changed between 2000-07-02 20:13:07 and the moment the FFL has been raised.
  • PRESENT means that the corresponding object is currently a member of the group.
  • ABSENT means that the corresponding object is not a member of the group anymore.

The Ver column is also interesting because it will tell us how many times the member has been added/removed to/from the group. If you add a user, it will be visible with the type PRESENT and the Ver 1. When you remove it, it will be visible with the type ABSENT and the Ver 2. If you re-add the user, it reappears in the membership list, with the type PRESENT and the Ver 3 meaning the user has been bounced around 2 times. This is the case for Charles in the example above.

How long do we keep an ABSENT type in the metadata? This time is the value of the tombstoneLifeTime that you can see in the eponym attribute on the configuration naming context (for example on CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=contoso,DC=com for the contoso.com forest). If the value is not set, it means it is 60 days (short -eh? be you can change it to the new default value since 2003 which is 180 days).

msDS-ReplValueMetaData

Those metadata are accessible via the attribute msDS-ReplValueMetaData. Like its big brother msDS-ReplAttributeMetaData, it is a calculated attribute, therefore you need to explicitly ask for it in your LDAP request if you want to be able to use it:

Get-ADObject `

    -SearchBase "CN=Administrators,CN=Builtin,DC=contoso,DC=com" `

    -Filter * -SearchScope Subtree `

    -Properties msDS-ReplValueMetaData | Select-Object msDS-ReplValueMetaData

msDS-ReplValueMetaData

--------------------------

{<DS_REPL_VALUE_META_DATA>...

 

If you don't ask specifically for it, the output will be empty. Or in the GUI tools with the filter button:

It is an array of some sort of XML structure defined here: DS_REPL_VALUE_META_DATA (or to be exact: DS_REPL_VALUE_META_DATA_2). Let's have a closer look:

  • pszAttributeName is the LDAP display name of the linked value attribute (could be member or manager for example, but in this article we just assume it is group membership therefore we are treating the links as members).
  • pszObjectDn the DN of the member.
  • cbData and pbData are some internal data (binary format) that we do not care about for our script.
  • ftimeDeleted the time the member has been removed (equal 0 if the object is currently still a member).
  • ftimeCreated the time when the member has been added for the first time.
  • dwVersion the version number or how many time the link has been updated.
  • ftimeLastOriginatingChange the time of the latest update of the link.
  • uuidLastOriginatingDsaInvocationID is the invocation ID of the DC from which the latest modification is from (please refer to the article Metadata #0 - Metadata, what is it  and why do we care? for more info about the invocation ID).
  • usnOriginatingChange and usnLocalChange are internal trackers, again look up the article Metadata #0 - Introduction, what are metadata and why do we care? for more info about it.
  • pszLastOriginatingDsaDN is the DN of the NTDS Settings object of the DC from which the latest modification is from. You'll see it in the Sites and Services console for example. It is very useful because the invocation ID isn't very graphic :) just reading this DN will tell you the name of the DC without having to look up the invocation ID.

We can see the power of those metadata, we are able to tell when a user became a member of a group, if it has been removed and when, and even if the user has been added again without the time of the tombstoneLifetime. The deleted information is kept for the lifetime of the TSL to ensure a complete convergence of the information.

Let's collect that information

Ok so let's identify the changes in groups for the last 180 days (well for the TSL time, 180 days in my environment). What groups are we looking at? In my scenario I will look at all the built-in groups but you can basically use the filter you want.
This is the LDAP filter I am going to use:

  • (&(objectClass=group)(adminCount=1)) The objectClass=group is pretty self-explanatory, I want groups, the adminCount=1 though is more tricky, it means that I want the groups which are or have been protected by the adminSDHolder mechanism (see here for more details about this adminSDHolder).

Here is the list of groups in my scope:

  • Account Operators
  • Administrators
  • Backup Operators
  • Domain Admins
  • Domain Controllers
  • Enterprise Admins
  • Print Operators
  • Read-only Domain Controllers
  • Replicator
  • Schema Admins
  • Server Operators

Why do you have a different output in your environment? Totally possible. If you nested groups into protected groups, these nested groups also become protected. So you might have an extended list... And maybe some delegation issues you might want to spend some time on :) Next level, we need to query the group membership modifications for all those groups.

With PowerShell but without the Active Directory Web Service

This is what is done on this script: Track group memberships modifications (available for download). Here is an example of the output of this script:

Now with PowerShell (Windows 7/Windows Server 2008R2)

As I mentioned in the previous post of the metadata series, you can't query metadata in PowerShell with native cmdlet, you have to parse it manually. So let's do that, let's pick a domain controller, and a threshold (the TSL of the forest):

#Set a domain controller to target

$_dc_to_use = "DC01.contoso.com"

#Determine the threshold based on the TSL

$_config_nc = [string] ([ADSI]"LDAP://$_dc_to_use/RootDSE").configurationNamingContext
$_TSL_ADSI = [string] ([ADSI]"LDAP://$_dc_to_use/CN=Directory Service,CN=Windows NT,CN=Services,$_config_nc").tombstoneLifetime
If ( $_TSL_ADSI -eq "" )
{
    $_threshold = 60

} Else { 

    $_threshold = $($_TSL_ADSI)
}

As you can see, we are just using ADSI object. Nothing fancy here, it is just a simple way to call for LDAP stuff, and it works with or without AD Web Service. If the attribute tombstoneLifetime (TSL) does not have a value, the default is 60 days. Then we need to convert this threshold into a date to limit our output

$_threshold_time = (Get-Date).AddDays(-$_threshold)

Why do we care so much about the threshold? Deleted members are just staying for the time of the TSL in the metadata. But active members are in the metadata even if they have been added more than the TSL ago. So if we list the group members without limiting our search in the time, it might mislead us thinking that no one has been removed from the group while it means just that no one has been removed for the last TSL days. Because I want to show all the modifications of the group, I limit my search in time. But nothing forces you to do so, you can totally not limit in the time but be aware of the warning I just mentioned. Besides, LEGACY type will show up with a Version=0, so they do not have a date associated.
Next steps, we query for all the groups we want to scrutinize:

$_myQuery = Get-ADGroup `

   
-LDAPFilter "(&(objectClass=group)(adminCount=1))" `

    -Server $_dc_to_use `

    -Properties msDS-ReplValueMetaData

 

We store in $_myQuery for future re-use. It is not mandatory, you can just pipe it directly to a ForEach-Object, but let's take our time when we script, step by step :)

$_myQuery | ForEach-Object `

{

   Write-Output "Group: $($_.distinguishedName)"

   $_."msDS-ReplValueMetaData" | ForEach-Object `

   {

      $_metadata = [XML] $_.Replace("`0","")

      $_metadata

   }

   Write-Output "--"

}

For each object returned by our query, we display the DN and parse the metadata. As mentioned in this article, we need to escape the trailing null character from the output of the cmdlet to be able to parse it as an XML like object (this is what is performed by $_.Replace("`0","") as you would have guessed). So far we just listed the metadata of those groups:

Group: CN=Administrators,CN=Builtin,DC=contoso,DC=com

DS_REPL_VALUE_META_DATA                                                                                                                                     
-----------------------                                                                                                                                     
DS_REPL_VALUE_META_DATA                                                                                                                                     
DS_REPL_VALUE_META_DATA                                                                                                                                     
DS_REPL_VALUE_META_DATA                                                                                                                                     
--
Group: CN=Print Operators,CN=Builtin,DC=contoso,DC=com
--
Group: CN=Backup Operators,CN=Builtin,DC=contoso,DC=com
--
Group: CN=Replicator,CN=Builtin,DC=contoso,DC=com
DS_REPL_VALUE_META_DATA                                                                                                                                     
--

...

If there is no metadata, it is simply because the group doesn't have any member for longer than the TSL (member or any potentially linked value attributes). So let's filter on two things, the attribute name (we want only the member attribute) and the time of the last update (the threshold that we discussed earlier).

...

$_metadata = [XML] $_.Replace("`0","")

$_metadata.DS_REPL_VALUE_META_DATA | `

Where-Object { $_.pszAttributeName -eq "member" } | ForEach-Object `

{

   If ( (Get-Date $($_.ftimeLastOriginatingChange)) -gt $_threshold_time )

   {

      Write-Output "`tMember: $($_.pszObjectDn)"

   }

}

...

Now we have this output:

Group: CN=Administrators,CN=Builtin,DC=contoso,DC=com

Member: CN=Domain Admins,CN=Users,DC=contoso,DC=com

Member: CN=Enterprise Admins,CN=Users,DC=contoso,DC=com

Member: CN=Administrator,CN=Users,DC=contoso,DC=com

--

Group: CN=Print Operators,CN=Builtin,DC=contoso,DC=com

--

Group: CN=Backup Operators,CN=Builtin,DC=contoso,DC=com

--

Group: CN=Replicator,CN=Builtin,DC=contoso,DC=com

Member: CN=Alice,OU=People,DC=contoso,DC=com

--

Group: CN=Domain Controllers,CN=Users,DC=contoso,DC=com

--

Group: CN=Schema Admins,CN=Users,DC=contoso,DC=com

Member: CN=Administrator,CN=Users,DC=contoso,DC=com

--

Group: CN=Enterprise Admins,CN=Users,DC=contoso,DC=com

Member: CN=Administrator,CN=Users,DC=contoso,DC=com

--

Group: CN=Domain Admins,CN=Users,DC=contoso,DC=com

Member: CN=Administrator,CN=Users,DC=contoso,DC=com

--

Group: CN=Server Operators,CN=Builtin,DC=contoso,DC=com

--

Group: CN=Account Operators,CN=Builtin,DC=contoso,DC=com

Member: CN=Alice,OU=People,DC=contoso,DC=com

--

Group: CN=Read-only Domain Controllers,CN=Users,DC=contoso,DC=com

--

At this point we have all the members for which some updates took place less than the TSL ago. But we don't know yet if it is a deleted member or an active member. So let's dig into the other associated metadata.

If the ftimeDeleted is equal to "1601-01-01T00:00:00Z" it means it is an active member (else it is a deleted member, deleted less than the TSL ago).

If the dwVersion is higher than 1 it means it is not the first time the member has been added. So we can determine:

  • When the member has been added for the first time
  • How many times the object has been a member
  • When was the last time it has been added

...

Write-Output "`tMember: $($_.pszObjectDn)"

If ( $_.ftimeDeleted -eq "1601-01-01T00:00:00Z" )

{

   Write-Output "`t`tCurrently an ACTIVE member"

   If ($_.dwVersion -eq 1)

   {

      Write-Output "`t`t`t-Added: $($_.ftimeCreated)"

   } Else {

      Write-Output "`t`t`t-Added for the first time: $($_.ftimeCreated)"

      Write-Output "`t`t`t-Then removed: $([MATH]::Floor($_.dwVersion/2)) times"

      Write-Output "`t`t`t-Then re-added: $($_.ftimeLastOriginatingChange)"

   }

}

...

As mentioned before if the dwVersion is higher than 1 then it is added and removed, so the simple floored division will tell us how many times the member has been bounced around [MATH]::Floor($_.dwVersion/2). Let's have a look at the output:

Group: CN=Administrators,CN=Builtin,DC=contoso,DC=com

Member: CN=Domain Admins,CN=Users,DC=contoso,DC=com

 Currently an ACTIVE member

  -Added for the first time: 2014-07-21T21:26:49Z

  -Then removed: 1 times

  -Then re-added: 2014-08-13T16:06:55Z

Member: CN=Enterprise Admins,CN=Users,DC=contoso,DC=com

 Currently an ACTIVE member

  -Added: 2014-07-21T21:26:49Z

Member: CN=Administrator,CN=Users,DC=contoso,DC=com

 Currently an ACTIVE member

  -Added: 2014-07-21T21:25:57Z

...

Now let's do the same for the deleted members.

If the ftimeDeleted is not equal to "1601-01-01T00:00:00Z" it means it is an deleted member.

If the dwVersion is higher than 2 it means it is not the first time the member has been removed. So we can determine:

  • When the member has been added for the first time
  • How many times the object has been a member
  • When was the last time it has been removed

Write-Output "`tMember: $($_.pszObjectDn)"

If ( $_.ftimeDeleted -eq "1601-01-01T00:00:00Z" )

{

    ...

} Else {

    Write-Output "`t`tCurrently an DELETED member"

    Write-Output "`t`t`t-Added: $($_.ftimeCreated)"

    Write-Output "`t`t`t-Removed: $($_.ftimeDeleted)"

    If ($_.dwVersion -gt 2)

    {

    Write-Output "`t`t`t-Removed: $([MATH]::Floor( $_.dwVersion / 2 )) times"

        Write-Output "`t`t`t-Last deleted: $($_.ftimeLastOriginatingChange)"

    }

}

And here is an output:

Group: CN=Account Operators,CN=Builtin,DC=contoso,DC=com

Member: CN=Alice,OU=People,DC=contoso,DC=com

 Currently an DELETED member

  -Added: 2014-07-21T21:45:54Z

  -Removed: 2014-08-13T14:36:34Z

  -Removed: 2 times

  -Last deleted: 2014-08-13T14:36:34Z

--

...

Here you go, the complete script will look like this:

$_dc_to_use = "DC01.contoso.com"
$_config_nc = [string] ([ADSI]"LDAP://$_dc_to_use/RootDSE").configurationNamingContext
$_TSL_ADSI = [string] ([ADSI]"LDAP://$_dc_to_use/CN=Directory Service,CN=Windows NT,CN=Services,$_config_nc").tombstoneLifetime
If ( $_TSL_ADSI -eq "" )
{
   $_threshold = 60
} Else {
  
$_threshold = $($_TSL_ADSI)
}
$_threshold_time = (Get-Date).AddDays(-$_threshold)
$_myQuery = Get-ADGroup `
  
-LDAPFilter "(&(objectClass=group)(adminCount=1))" `
  
-Server $_dc_to_use `
  
-Properties msDS-ReplValueMetaData

$_myQuery | ForEach-Object `
{
   Write-Output "Group: $($_.distinguishedName)"
  
$_."msDS-ReplValueMetaData" | ForEach-Object `
  
{
      $_metadata = [XML] $_.Replace("`0","")
     
$_metadata.DS_REPL_VALUE_META_DATA | `
        
Where-Object { $_.pszAttributeName -eq "member" } | ForEach-Object `
        
{
            If ( (Get-Date $($_.ftimeLastOriginatingChange)) -gt $_threshold_time )
           
{
               Write-Output "`tMember: $($_.pszObjectDn)"
              
If ( $_.ftimeDeleted -eq "1601-01-01T00:00:00Z" )
              
{
                  Write-Output "`t`tCurrently an ACTIVE member"
                 
If ($_.dwVersion -eq 1)
                 
{
                     Write-Output "`t`t`t-Added: $($_.ftimeCreated)"
                 
} Else {
                    
Write-Output "`t`t`t-Added for the first time: $($_.ftimeCreated)"
                    
Write-Output "`t`t`t-Then removed: $([MATH]::Floor($_.dwVersion/2)) times" 
                     
Write-Output "`t`t`t-Then re-added: $($_.ftimeLastOriginatingChange)"
                  
}
            } Else {
              
Write-Output "`t`tCurrently an DELETED member"
              
Write-Output "`t`t`t-Added: $($_.ftimeCreated)"
              
Write-Output "`t`t`t-Removed: $($_.ftimeDeleted)"
              
If ($_.dwVersion -gt 2)
               
{
                  Write-Output "`t`t`t-Removed: $([MATH]::Floor( $_.dwVersion / 2 )) times"
                 
Write-Output "`t`t`t-Last deleted: $($_.ftimeLastOriginatingChange)" 
              
}
           
}
        
}
  
   }

   }

   Write-Output "--"
}

In the next example, we'll try to deal with a better output.

Now with PowerShell (Windows 8/Windows Server 2012 and higher)

With the Get-ADReplicationAttributeMetadata everything is easier. Let's go crazy and opt for the one-liner:

Get-ADGroup `

    -LDAPFilter "(&(objectClass=group)(adminCount=1))" `

    -Server $_dc_to_use `

    -Properties msDS-ReplValueMetaData | `

    Get-ADReplicationAttributeMetadata -Server $_dc_to_use -ShowAllLinkedValues | `

        Where-Object { $_.AttributeName -eq "member" } | `

            Select-Object Object, AttributeValue, FirstOriginatingCreateTime,LastOriginatingChangeTime,@{Name="Deleted";Expression={ If ($_.LastOriginatingDeleteTime -eq "1601-01-01T00:00:00Z") { "Active" } Else { $_.LastOriginatingDeleteTime} }},Version | `

                Out-GridView

 

So for each group returned by our Get-ADGroup, we gather the metadata. We do explicitly specify what we want all Linked values with the -ShowAllLinkedValues. We filter on the AttributeName, we could also add the time limitation I wrote about earlier in the post, and then we display everything into a nice dynamic grid with Out-GridView (requires PowerShell_ISE installed). Note that I do not display the value of LastOriginatingDeleteTime unless it is different than "1601-01-01T00:00:00Z". This is the output:

Same way, we could have generated a CSV or an HTML output:

...

    Select-Object Object, AttributeValue, FirstOriginatingCreateTime,LastOriginatingChangeTime,@{Name="Deleted";Expression={ If ($_.LastOriginatingDeleteTime -eq "1601-01-01T00:00:00Z") { "Active" } Else { $_.LastOriginatingDeleteTime} }},Version | `

       ConvertTo-Html | Out-File Report.html

           Invoke-Item Report.html

And here you go:

You can add a title, a body, and if you are a CSS fan, you can even specify one to make beautiful report.