Active Directory Blog : AD Replication

These are the Updates You Are Looking For

Tim Springston — Wed, 23 Sep 2009 22:59:00 GMT

In this blog post we’re going to go over a few techniques that are a bit old school but will come in handy for understanding how things work even if you ultimately use a great monitoring suite like MOM. Now, there are great articles here and here that...(read more)

Bulking Up an ADAM Test Instance

Tim Springston — Fri, 14 Aug 2009 19:40:00 GMT

This week I’ve had the need to do some testing around ADAM (also known by it’s shiny new name of Active Directory Lightweight Directory Services or AD LDS). The tests themselves are not directly relevant to this blog post, but in order for the tests...(read more)

DCs and Network Address Translation

Tim Springston — Wed, 22 Apr 2009 16:11:00 GMT

A lot of planning goes into the features and capabilities of each Windows release. Over the years I’ve noticed that there is not a great deal of awareness out in the general public for just how much work and labor goes into a new version of Windows. We’ll...(read more)

Tabula Rasa

Tim Springston — Wed, 14 Jan 2009 17:15:00 GMT

I was well and truly stumped a few months ago. I joke that once a year I am flat out wrong, and rarely do I have nothing to say on a subject. The 'once a year I may be flat out wrong' statement may be true simply because after 15 years in the IT industry...(read more)

A Complicated Scenario Regarding DNS and the DC Locator SRVs

Tim Springston — Fri, 08 Aug 2008 17:00:00 GMT

When we do initial interviewing of a candidate for a job here in the CSS Directory Services team a question we’ll often start with is “how important is DNS to Active Directory?”. The person’s answer-if the correct answer of very important is given- is...(read more)

Monitor AD Replication Much?

Tim Springston — Fri, 25 Jul 2008 19:21:00 GMT

I want to point out an excellent resource for an administrator out there who is responsible for Active Directory replication in their environment. The resource is a comprehensive and detailed article solely on using Repadmin.exe. The article goes over...(read more)

Conflict Resolution, Lingering Objects, Printers

Tim Springston — Fri, 06 Jun 2008 18:29:00 GMT

I was recently asked to help out on an issue with a similar theme to other cases we have seen over the years. The topic has never been one that has generated a high number of calls to us but the calls we have received are not easy ones to get an initial...(read more)

All The Logging In The World

Tim Springston — Mon, 08 Oct 2007 19:40:00 GMT

There’s normal troubleshooting and then there’s the stuff you do when the basic troubleshooting doesn’t get things resolved. Normal troubleshooting can be things like selecting “last known good” on a reboot after installing a new driver and having a blue screen. Or perhaps uninstalling and then reinstalling an application, or altering settings for the application or operating system to alleviate a problem. Sometimes we have to dig in and find out more. Many admins out there in the world live that every day. Which is why we add methods to find out more into our products. This post is all about listing all of the data gathering methods that a Directory Services person may ever need to know. Since there are so many it will be difficult to organize well in one uber post but I’m going to put out here for you all anyway, disorganized or not....(read more)

Lingering Objects, Like Weekend Guests Who Won't Leave, But for AD

Tim Springston — Fri, 14 Sep 2007 16:01:00 GMT

It struck me the other day what a statistical improbability it was that I haven’t really talked much about lingering object problems in AD yet in this blog. They are one hot topic to support people, even if they are not our most common problem. The hot...(read more)

Adroitly Sidestepping Initial Synchronization Requirements

Tim Springston — Mon, 30 Apr 2007 00:10:00 GMT

One of my pet peeves is when I don’t recall every detail of a troubleshooting step or technique and can’t immediately find a document that explains it. It’s like when I misplace a book I’ve been reading-I wander around the house in circles looking for it until I find it. It’s maddening.

Case in point: Active Directory Initial Synchronization Requirements.

AD Init Sync is normally a great feature. Basically, it places a requirement on a domain controller which holds Flexible Single Master Operations (FSMO) roles, as it comes back up from being rebooted, to have successful AD replication (inbound and outbound) with its known replica partners before it advertises itself as domain controller and starts providing services to clients. Those clients can be users, computers or other domain controllers.

A great deal more about this feature is here: http://support.microsoft.com/kb/305476

This is a good thing since you want a domain controller that holds one or more FSMO roles to know that it may not hold that role any longer. It would be notified via AD replication (if the change happens while this DC is offline or otherwise not in contact) of FSMO holder changes. So the initial sync requirement before advertising is very, very helpful. If you have ever had a Windows 2000 domain controller which believes it hold FMSO roles turned back on following a FSMO seizure to another domain controller then you may have a more poignant understanding of how helpful the init sync is.

But when you test you may not want to run into the init sync. For example, as a support person I am constantly (it feels) creating and tearing down test environments. I typically use Virtual Server or Virtual PCs, but it’s still time intensive and I don’t always recall what the last horrible thing I did to a particular test environment was. Makes life exciting, in a way.

The other day, though, I was resurrecting a test DC only to find that, though it held the FSMO roles, I had not taken the time to remove references to some other test domain controllers from the remaining DCs directory. So the remaining DC thought it needed to sync with the removed DCs before it could advertise the roles and services it offered out to the world.

How would I know such a thing? Well, it was a reasonable assumption based on the DCDIAG.EXE result and KnowsofRoleHolders tests (taken as a DCDIAG /V). Here’s what they looked like with the problem:

Domain Controller Diagnosis

Performing initial setup:

* Verifying that the local machine tspringdc1, is a DC.

* Connecting to directory service on server tspringdc1.

The directory service on tspringdc1 has not finished initializing.

In order for the directory service to consider itself synchronized, it must attempt an initial synchronization with at least one replica of this server's writeable domain. It must also obtain Rid information from the Rid FSMO holder.

The directory service has not signalled the event which lets other services know that it is ready to accept requests. Services such as the Key Distribution Center, Intersite Messaging Service, and NetLogon will not consider this system as an eligible domain controller.

* Collecting site info.

* Identifying all servers.

The directory service on tspringdc1 has not finished initializing. In order for the directory service to consider itself synchronized, it must attempt an initial synchronization with at least one replica of this server's writeable domain. It must also obtain Rid information from the Rid FSMO holder.

* Identifying all NC cross-refs.

* Found 14 DC(s). Testing 1 of them.

Done gathering initial info.

Starting test: FsmoCheck

Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355

A Global Catalog Server could not be located - All GC's are down.

Warning: DcGetDcName(PDC_REQUIRED) call failed, error 1355

A Primary Domain Controller could not be located.

The server holding the PDC role is down.

Warning: DcGetDcName(TIME_SERVER) call failed, error 1355

A Time Server could not be located.

The server holding the PDC role is down.

Warning: DcGetDcName(GOOD_TIME_SERVER_PREFERRED) call failed, error 1355

A Good Time Server could not be located.

Warning: DcGetDcName(KDC_REQUIRED) call failed, error 1355

A KDC could not be located - All the KDCs are down.

......................... tspringtest.com failed test FsmoCheck

…and here’s what a good one looks like:

<no initial error regarding init sync>

Starting test: FsmoCheck

GC Name: \\TSPRINGDC1.tspringtest.com

Locator Flags: 0xe00003fd

PDC Name: \\TSPRINGDC1.tspringtest.com

Locator Flags: 0xe00003fd

Time Server Name: \\TSPRINGDC1.tspringtest.com

Locator Flags: 0xe00003fd

Preferred Time Server Name: \\TSPRINGDC1.tspringtest.com

Locator Flags: 0xe00003fd

KDC Name: \\TSPRINGDC1.tspringtest.com

Locator Flags: 0xe00003fd

......................... tspringtest.com passed test FsmoCheck

So, do you live with it as is? Are you forced to right whatever is wrong? In my test environment case that would have meant that I would need to remove all references to removed DCs from DNS and AD. That’s documented well in the trusty mainstay article:

How to remove data in Active Directory after an unsuccessful domain controller demotion

http://support.microsoft.com/default.aspx/kb/216498

What do you do if you are in a situation like mine- a test environment – and you need the DC online quickly and don’t want to go through the tedium of cleaning up the environment when you may tear it all down soon anyway? Or, in a unique situation where you have lost the WAN link a FSMO role holders replica partners but you need that DC to do its job without waiting for the initial synchronization with those partners?

Well I’m glad you asked.

Add the registry value below and reboot the DC in question. When it comes back up, it will not require the init sync before successfully advertising itself and doing its job.

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters]

"Repl Perform Initial Synchronizations"=dword:00000000

For the above entry, 1 means “do it” while 0 means “do not do it”.

Written from lovely Quebec City, Canada. Until next time folks, take care out there in the wild forest of DCs you wander through.

Active Directory in Longhorn Server

Tim Springston — Wed, 14 Feb 2007 07:29:00 GMT

Recently I posted saying that I’d give a preview of Longhorn Server Active Directory. Well, here it is. I will caution all who read this to take into account that we are still in a beta production phase, so some or all of these items could change (or perhaps not make it in the product if they don’t meet the high bar of testing we have) before Longhorn is released to manufacturing and available for your servers. After hearing about the features it will have, though, I strongly suspect that you’ll really want everything to be in the final product as is and in full force. So, let's get started.

Roles

Services in Longhorn are all about roles, which can be loosely defined as sets of features that provide services. Active Directory Services is one role, as is Directory Lightweight Services or LDS (previously known as ADAM). If you’ve been doing your reading then you about Server Core. ADS, LDS and DNS are all installable on Server Core Longhorn installs. In all except for Server Core, roles will be added via Server Manager as a central snapin interface. Look for wizards taking you on through the addition of the role and the specifics of what will be needed to install as you need.

Read-Only Domain Controllers (RODCs)

Longhorn will add a new type of domain controller: RODC. This is a DC which has a read-only copy of the domain partition it is a DC for. The RODC will host an abbreviated copy of the domain partition and will, by default, not host passwords. It can be set to host those as well if you choose. No changes will be written on the RODC, not for SYSVOL and not for AD. All changes (updates, really) will come from other, writable, Longhorn DCs.

The RODC is designed to be provide a smaller security silhouette as well as less manageability concern for smaller, remote business locations and branch offices. RODCs will need Windows 2003 forest functional level (they use LVR), and require at least one Longhorn DC (not RODC) for their own domain to replicate from. From what I understand, Read-Only Partial Attribute Set (PAS) for RODC will be published to MSDN’s schema repository in the same way GC PAS (isMemberOfPartialAttributeSet) items are for reference purposes, and utilizable in a similar manner for your schema. For now, expect frequently accessed attributes to be the defaults as present objects and attributes on a RODC.

As an additional security enhancement the Krbtgt account, used for issuance of TGTs to domain principals (users, computers and the like), will be unique to each RODC. Group membership, useraccountcontrol value and other unique domain controller details will be different for RODC accounts-no “strong” security group memberships like Enterprise Domain Controllers, for example.

RODC replication will be unidirectional with all changes flowing into the RODC from a full writable Longhorn DC of the same domain.

Admin role separation will allow you to make a “branch office admin” capable of administering all aspects of the RODC without fear of compromising your domain, forest or enterprise.

The ADPREP process is essentially the same as in Server 2003 where the domain and forest must be prepared for the introduction of the first Longhorn DCs. The exception is for RODCs: first addition of an RODC will require a /RODCPREP action-but not necessary if you decide not to add RODCs.

DCPROMO

DCPROMO has been lifting weights. Everyone is familiar with the unattended install options where you can provide an answer file to automate the promotion that we’ve had. Longhorn builds on this. For one thing, when you complete a promotion, you can choose to export the selections you chose in that promotion to an answer file for use with other similar promotions-all with the press of a button.

DCPROMO will also have the ability to use NTDSUTIL to create “seed” media for IFM (install from media) DCPROMOs, sanitized for RODC IFM specifically (if you choose), no passwords, or other sensitive data present.

Prestaging of DC accounts will also be an option. You can run through DCPROMO to add a new DC (from another machine), then later rerun it from the actual machine you mean to promote. This can help for RODC installs or if you’d like to prestage (and thereby limit) other particulars of the new DC’s details (like name or site for example).

Enhanced Reliability

DC promotions and demotions previously required reboots for nearly every step. Efforts have been made to reduce the need for reboots for most actions. In addition, Active Directory is a discrete, restartable or stoppable service on the DC in Longhorn. In prior versions you could attempt to prevent some actions (like authentication or replication) by stopping the netlogon service, but this would not necessarily prevent all services from doing their jobs (like FRS for example). Not with Longhorn-just stop the service for downtime if needed or to apply a fix.

Under the hood there is some added error correcting for some less common database page checksum problems which can occur. Also, instead of FRS replication will be DFSR, with the compression it provides, as the new method for replicating SYSVOL to all replicas, including RODCs.

Disaster Recovery

An extremely cool concept: the Snapshot Browser. This is the idea of using Volume Shadow Copy to do a snapshot of your directory and to store that info in the living AD as a reference for comparison purposes. Though not a backup tool itself, this technique can be used to compare items following the recovery from a disaster to verify that all is as it was before the disaster. The idea would be to take a snapshot using NTDSUTIL (which could be batched and ran periodically as a task) and be able to query it using traditional tools like ADSIEDIT.MSC or via a LDAP querying tool or method. There’s also some discussion of a snapshot browser tool for selecting to “undelete” some objects if you aren’t in a full “restore everything” true disaster situation.

Globalization and Standards

Longhorn ADS will support IPV6 inherently for all roles (DNS Server, ADS, LDS et cetera). For folks in regions it will be helpful in, phonetic sorting in the directory is being added (for language usage in Japanese for example).

DNS Server

AD will continue to rely on DNS and so some focus has been on making that experience better as well. A new “Instant On” feature will prevent advertisement delays and service startup timing issues. Another consideration is single label domain names being supported by use of a new “Global Names Zone”.

Also…teasers…

Improved directory auditing

DC locator site locality enhancements

So there we have a good overview of what you can expect with Longhorn AD. It is not a comprehensive list (there’s more), but I hope it wets your appetite.

Now, let’s all recite the Microsoft blog poster mantra together:

These postings are provided "AS IS" with no warranties, and confer no rights.

I encourage you all to provide feedback on these features, as well as the accidental object deletion feature discussed in my prior post this month. Your opinions matter, and your help is appreciated. You can email me directly from this blog, or post your comment.

On a final note, it’s Valentine’s Day here in the USA, so I hope everyone has a significant other to enjoy the holiday with-Happy Valentine’s Day!

Active Directory in Longhorn...Feature Feedback Requested

Tim Springston — Thu, 08 Feb 2007 02:13:00 GMT

One of the most time consuming, aggravating and emotionally draining issues our customers can go through is a disaster recovery issue. The general gyst of this type of thing is:

-Accidental deletion of <insert terrifyingly large number here> users, groups or other objects in AD

-Mad scramble to immediately hinder AD replication throughout the enterprise only to realize that…

-AD replication is lightning fast and the deletion has already replicated everywhere.

-Simultaneous throbbing headache along with realization that you will likely be up all night recovering these objects and verifying that they are in good state.

Not a pretty picture is it?

Longhorn has planned features to help with that. With a bold statement like that though I must give a caveat: Longhorn Server is currently beta, and beta means incomplete. Unfortunately, my statements are not law, so the end product can change before the “release to manufacturing” or final RTM version.

However, because it is beta the Microsoft Directory Services Community has a unique opportunity to provide feedback. One of our devs has an idea around the above headache-all-nighter-disaster recovery scenario depicted above. Hear me out, then weigh in by either emailing me through the blog or posting a comment on the blog. I guarantee your feedback will be heard by the AD dev team.

The first part of the above scenario is key: accidental. Unintentional mass deletion of objects from AD.

What if, when viewing the properties of an object (like a user) in Active Directory Users and Computers (DSA.MSC) you had a checkbox that would say something like “prevent accidental deletion”? Have the check there, and the object cannot be deleted. Remove the checkbox, delete away.

Having this checked on all of your users, or perhaps a critical segment of them, could certainly save someone from an “oops” moment. On the other hand, maybe it will prevent some utility you need. I don’t know…what do you think?

Does that sound useful to you? Sound out! Let us know.

One other thing. I'll my next post or posts wil be regarding some features you may see (remember it's beta) in Active Directory for Longhorn. Don't miss the posts, I promise it's really good stuff.

Get the Lowdown on your Replication

Tim Springston — Fri, 23 Jun 2006 19:00:00 GMT

There are other utilities out in the wild that can be used to give a comprehensive test of AD replication, but few are as definitive as REPADMIN /SHOWREPL command in the CSV format.

I'm willing to bet there are some folks who just read that statement and are thinking that it sounds pretty silly. I promise you'll be believers after using this tool, particularly those folks out there who are new to the environment they administer or that may work as consultants and so get confronted by new environments all the time.

Here’s the skinny on this from REPADMIN help:

/showrepl [DC_LIST [Source DC object GUID]] [Naming Context] [/verbose]

[/nocache] [/repsto] [/conn] [/all] [/errorsonly] [/intersite]

And here's a synopsis of what this command puts at your fingertips:

· replication status (programmatic error code) for a naming context(s)

· last success and failure times

· number of last success and failures

· network transport used for the replication (nearly always RPC)

· source and destination DCs

· source and destination AD sites

The above doesn’t sound like much until you put it into practice. Let’s talk about doing that.

REPADMIN /SHOWREPL * /CSV >resultfile.csv

The above command will run on all DCs in the forest and gathers the latest replication status codes for all naming contexts (NCs) present. I probably don’t need to say this, but in case I do: NCs are the containers such as your default Configuration and Schema partition, as well as every domain partition in your forest. In my test case we have the Thatsnotright, the Wackiness, and the Silliness domains adding to the default partitions of Config and Schema. The above command will get the replication status for all of these, on every DC.

Key points to keep in mind around this are:

-not every DC will have a copy of every NC; only Global Catalogs will

-Not every site may host a DC for every domain; it just depends on what was deployed there

-This command does not indicate whether a hosted NC is read only (such as in the case of a GC hosting it) or writable (a domain partition).

-This command does not indicate OS version. You could add or get that separately, but that’s important stuff to know in lingering object and other scenarios.

-This command does not display whether a DC is set for strict or loose replication consistency

-The number zero (0) indicates success, no failure present.

So, once you run the command above you need to open and use the file. Here are the steps to do that using Microsoft Excel:

Start Microsoft Excel, and then open the Showrepl.csv file.
Select the A + RPC and SMTPcolumns, and then, on the Edit menu, click Delete.
Select the row that is immediately under the column headers. On the Windows menu, click Freeze Pane.
Select the complete spread sheet. On the Data menu, point to Filter, and then click Auto-Filter.
On the heading of the Last Success column, click the down arrow, and then click Sort Ascending.
On the heading of the source DC column, click the down arrow, and then click Custom.
In the Custom AutoFilter dialog box, select the operand does not contain and type del in the box to the right of the operand. This prevents the display of deleted domain controllers in the results.
On the heading of the Last Failure column, click the down arrow, and then click Custom. In the Custom AutoFilter dialog box, select the operand does not equal, and then type 0 in the box to the right of the operand.
Resolve the replication failures that are displayed.

So now you have a selectable spreadsheet which details your AD replication status as of the moment the report was completed. You can select a single error code, or do a filter for multiple (such as 1722 and 1753). The ways you can reselect this data are infinite, so it is very customizable for any environment, problem and need.

The selected data below is an example of a selection of all DC which show the 8606 error in a test environment:

Destination DC Site	Destination DC	Naming Context	Source DC Site	Source DC	Transport Type	Number of Failures	Last Failure Time	Last Success Time	Last Failure Status
DALLAS	DC111	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC045	RPC	189	4/20/2006 22:06	(never)	8606
DALLAS	DC089	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC030	RPC	3172	4/20/2006 22:06	4/20/2006 4:22	8606
DALLAS	DC122	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC112	RPC	3909	4/20/2006 22:07	4/20/2006 4:22	8606
CHICAGO	DC070	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	CHICAGO	DC345	RPC	216	4/20/2006 22:03	4/20/2006 4:27	8606
CHICAGO	DC070	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	CHICAGO	DC220	RPC	1859	4/20/2006 22:07	4/20/2006 4:31	8606
DALLAS	DC110	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC245	RPC	3175	4/20/2006 22:07	4/20/2006 4:21	8606
DALLAS	DC183	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC045	RPC	216	4/20/2006 22:06	4/20/2006 4:14	8606
DALLAS	DC090	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC067	RPC	215	4/20/2006 22:05	4/20/2006 4:15	8606
DALLAS	DC090	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	DALLAS	DC093	RPC	806	4/20/2006 22:07	(never)	8606
CHICAGO	DC056	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	CHICAGO	DC114	RPC	216	4/20/2006 22:03	4/20/2006 4:27	8606
CHICAGO	DC075	DC=thatsnotright,DC=wackiness,DC=silliness,DC=org	CHICAGO	DC556	RPC	1862	4/20/2006 22:07	4/20/2006 4:31	8606

So, you’re probably saying to yourself “Self, how do I tell what this error number really means?”. Good question, O Reader. These error codes are the programmatic error codes from winerr.h, they are not the same as what you will see in a fully fleshed out event in the event log. As a result, they are lacking in every detail since this code is used to trigger (ultimately) that event to appear in the event log.

But there’s a way to see the comment associated with that error code, and thereby get a better understanding or more info on it. That is the ERR.EXE tool which can be downloaded from this link:

Err.exe

http://www.microsoft.com/downloads/details.aspx?familyid=be596899-7bb8-4208-b7fc-09e02a13696c&displaylang=en

This is command line tool which can take the Windows (not just DS, all Windows error codes) error number as an argument and provide the comment for it. Here’s the example for our “Last Failure Status” above.

C:\ > Err.exe 8606

# for decimal 8606 / hex 0x219e :

ERROR_DS_INSUFFICIENT_ATTR_TO_CREATE_OBJECT winerror.h

# Insufficient attributes were given to create an object.

# This object may not exist because it may have been deleted

# and already garbage collected.

# 1 matches found for "8606"

So, we’ve have a pretty good overview of a REPADMIN.EXE technique that can save some time and ease some general “is this domain replicating or what?” questions. I’m still deciding on the next post (slated for about a week or so from now) so if anyone has a special request just let me know.

Follow up on "The Goodness that is Repadmin.exe"

Tim Springston — Thu, 15 Jun 2006 18:22:00 GMT

I had some really good questions after the previous post and so I wanted to post the replies so all could see. Also, stay tuned for next week's post on the usage of /SHOWREPL before we move on to other topics.

Thanks to Matheeshta and IanCurtis for these questions.

************

1. How do I provide multiple dcnames in the dclist option seperated by spaces.

This is described by the "repadmin /listhelp" switch. Here is the information that gives:

DC_LIST = { <DC_NAME> | * | <part_server_name>* | site:<SITE_NAME> | gc: | fsmo_<FSMO_TYPE>:<FSMO_DN> }

Examples:

"*" = All DCs in the enterprise.

"part_server_name*" = would pick "part_server_name_dc_01" and "part_server_name_dc_02"

but not server "part_server_diff_name".

"site:east_site1" = All DCs in site "east_site1".

"gc:" = All GCs in the enterprise.

"fsmo_pdc:DC=my-corp-dom,DC=com" - repadmin runs against the PDC in the NC "DC=my-corp-dom,DC=com"

"fsmo_istg:east_site1" would pick the ISTG for the east_site1 site.

I get the error - unknown option "secondDC" when I use something like "repadmin /bind firstDC secondDC". Any pointers?

That’s one of the generic errors you can get when it parses the DC_LIST parameter and there is a problem, so the reason is not obvious. Could you try a wildcard, like *DC and see if that works?

2. Is repadmin /checkprop used for checking the properties of an object across all DCs that have a replica of the NC the object is in? I assume I only have to provide one DC for dclist. As I have to know the NC, originating DC's invocation GUID and the Originating USN to create/modify the object all at once, I assume this can be used to check if a newly created/modified object's status across the domain. Like when a new user has been created.

Yes, that is correct.

3. Does the /async option in repadmin /kcc mean the command will return control to the user quicker or will it also initiate an async replication command on the destination?

The “/async” option let’s the replication on the newly added replication connection take place at a later time rather than immediately.

4. Does repadmin /notifyopt only show notification timings for intersite replication or does it show intrasite replication notification defaults?

It looks to the values for msDS-Replication-Notify-First-DSA-Delay and msDS-Replication-Notify-Subsequent-DSA-Delay in the CrossRef object for the partition specified.

5. Does repadmin /replicate's /full switch ask for the whole NC or does it follow the High Watermark vector tables and the Uptodateness vector tables when asking for changes? Does the /async option mean all NCs are replicated simultaneously?

The /replicate /full sync is starting from scratch (i.e., at the first USN) for that NC. The /async option means that the replication does not need to happen immediately.

6. Please explain the repadmin /checkprop option. How is this different from the uptodateness vector table? Why is the originating USN specified? As long as the value I specify is lower than the current local USN for the DC invocation ID supplied, the output is the same.

The difference is that the up-to-dateness vector table is constructed of replica partners if the specified DC, whereas the /checkprop looks to all writables for that NC. The USN is required for comparison purposes, so it makes sense that you see the output the same for USNs which are lower than the current local one.

And finally, IanCurtis' question:

Looking at your blog, can I use this to see where a user account was deleted. I've got doubts about my office in Papeete.

The REPADMIN /SHOWOBJMETA switch may be your best bet unless you have the USN of the deletion, but even this will fall short. What this info will give you is what DCs have not received the deletion update yet-all others (those that have marked the object deleted) will give the generic object not found error mentioned in the post.

The best method to determine who did the deletion and when is to use auditing. This is the by design method-that's why it's there. In the absence of that auditing being enabled beforehand there is one other thing which comes to mind.

You could dump the Active Directory databases from the DCs involved using LDP.EXE, as outlined in the KB article:

http://support.microsoft.com/kb/315098/?sd=RMVP&fr=1

The above technique gives you a .DMP file from each DC you run it on. In that file, you can locate that object. Once found you can look at the delTime field, which will identify the deleted time for that object locally. Given that time must be pretty well in sync to begin with you can make the reasonable assumption that the DC whose .DMP file has the earliest deletion time is the one the original change was done one (or against if it was via an Adminpacked workstation). This doesn't do nearly as well as having auditing in place, but it can be useful.

The Goodness that is Repadmin.exe

Tim Springston — Tue, 13 Jun 2006 05:59:00 GMT

For the first technical post I thought it would be best to lay the groundwork and mention a tool which can be used in a wide variety of AD replication scenarios. We'll talk about one of the useful features in this post, and follow up with one other for the next post before we move onto another topic.

In my personal point of view, there are only two kinds of AD admins: those who love REPLMON, and those who love REPADMIN. As you may be able to infer from the title of this post, I am the latter. Not that there is anything necessarily evil about REPLMON advocates, but let's face it: it's not the cat's meow (or the bee's knees) of AD replication troubleshooting. Not like Repadmin.exe anyhow.

For those who haven't had an opportunity to use this tool before it is the general purpose, command line tool for troubleshooting directory replication. Repadmin.exe is included in the Windows Support Tools, which can be downloaded for free (free is one of my favorite words) here:

http://www.microsoft.com/downloads/details.aspx?familyid=6EC50B78-8BE1-4E81-B3BE-4E7AC4F0912D&displaylang=en

We had the tool available in Windows 2000, but in the Windows 2003 time frame it has more features. Syntax between the versions is slightly different, with a few deprecated commands. One useability thing which was a great improvement, generally speaking, was the addition of code to help with lookup of domain controller names. Windows 2000 would often require the MSDCS GUID to be used in the syntax. General idea is that our development folks (you know who you are) have been improving the tool as needed over the years, and they deserve HUGE kudos for it.

Back to the technical part...

One of the more frequent calls we at Microsoft Directory Services support get is problematic network connections which prevent AD replication. I used the big fancy generic term "problematic" since the actual originating causes of the AD replication failures are many. Could be misconfigured DNS on the DC NIC, wrong binding order of same, closed networking ports anywhere between the DCs in question, missing SRV records-or any of a myriad others. Bottom line is that, once you've fixed that problem, you need to test to make sure you are, in fact, replicating succesfully.

Repadmin's your tool for that.

Let's say that I add a bit of text to a user account in order to see if that data arrives. The Mark One Eye-ball is one method (just open DSA.MSC on the opposite end) but there's a better way, and it can track it as the change travails the forest.

REPADMIN /SHOWOBJMETA * "cn=user,ou=weisenheimers,dc=wacky,dc=org" >usermeta.txt

The asterisk can be replaced by a single domain controller name (like DC1). The asterisk runs against all in forest, so have your EA creds ready for that, and make sure to take into account network topology concerns since the tool queries all DCs for the object metadata.

The distinguishedname in quotes can actually be replaced by the objectGUID. Very cool for when you receive an event citing an objectGUID but no other identifying piece of info. The syntax of that field is “<GUID=objectguid>”.

The command above will output the replication metadata to a text file. This data is the information which the directory replicator uses to track updates for an object's attributes. Since AD replicates changes on a per attribute basis (or in the case of linked value replication, single value update for a multivalued attribute) you will see attribute name, local USN, time and date stamp (good for forensics), and originating DC and USN (also good for forensics).

Here's a sample:

repadmin running command /showobjmeta against server DC1.wacky.org

36 entries.

Loc.USN Originating DC Org.USN Org.Time/Date Ver Attribute

========= =============================================== =========

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 objectClass

80472995 Crazyland\DC1 80472995 2005-10-13 07:49:08 3 cn

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 description

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 instanceType

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 whenCreated

20341575 Crazyland\DC1 20341575 2004-09-10 13:12:48 2 displayName

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 nTSecurityDescriptor

80472995 Crazyland\DC4 33405674 2005-10-13 07:49:06 5 name

80336274 Crazyland\DC5 28321657 2005-10-10 14:35:17 4 userAccountControl

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 codePage

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 countryCode

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 homeDirectory

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 homeDrive

80391546 Crazyland\DC1 80391546 2005-10-11 18:59:09 15 dBCSPwd

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 localPolicyFlags

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 scriptPath

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 logonHours

80566031 Crazyland\DC1 80566031 2006-6-11 18:59:09 15 unicodePwd

80391546 Crazyland\DC1 80391546 2005-10-11 18:59:09 15 ntPwdHistory

80391546 Crazyland\DC1 80391546 2005-10-11 18:59:09 15 pwdLastSet

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 primaryGroupID

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 userParameters

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 profilePath

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 objectSid

33433553 Crazyland\DC2 33433553 2006-06-12 15:16:35 1 comment

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 accountExpires

80391546 Crazyland\DC1 80391546 2005-10-11 18:59:09 15 lmPwdHistory

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 sAMAccountName

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 sAMAccountType

20341645 Crazyland\DC1 20341645 2004-09-10 13:13:00 1 operatingSystem

20341645 Crazyland\DC1 20341645 2004-09-10 13:13:00 1 operatingSystemVersion

20341577 Crazyland\DC1 20341577 2004-09-10 13:12:48 1 dNSHostName

80416919 Crazyland\DC5 28388830 2005-10-12 08:19:49 11 servicePrincipalName

20341573 Crazyland\DC1 20341573 2004-09-10 13:12:48 1 objectCategory

20341574 Crazyland\DC1 20341574 2004-09-10 13:12:48 1 isCriticalSystemObject

80450630 Crazyland\DC1 80450630 2005-10-12 20:51:07 22 lastLogonTimestamp

So, in our replication test to see if things were working, we made a change on DC2-we added a comment to the comment field of a user object in DSA.MSC. Since there is a replication connection where DC1 pulls changes from DC2. Find the “comment” entry above and you’ll see it arrived. Run the SHOWOBJMETA command for this object again against all DCs and you can track it as it successfully replicates throughout the forest.

Final concepts on tracking AD replication using SHOWOBJMETA. First, keep in mind that we have writable and non-writable partitions. In other words, a DC in the WACKY.ORG domain hosts a writable partition for dc=wacky,dc=org, but since it is a GC it will host the non-writable partitions for all other domains in the forest, such as dc=very,dc=wacky,dc=org and dc=notso,dc=wacky,dc=org.

The reason this is mentioned is that if the object and attributes are published as part of the partial attribute set in it’s schema definition, then you may show Global Catalog servers responding in the SHOWOBJMETA that they have (or haven’t) gotten that update.

The other end of that idea is that the non-writable DCs (meaning DCs not in the domain which the user belongs to) which are also not GCs will show the message below:

repadmin running command /showobjmeta against server DC8.very.wacky.org

DsReplicaGetInfo() failed with status 8333 (0x208d):

Directory object not found.

So, we haven’t really gone over any stunningly difficult issues today, but we’ve made an intro to one of our best AD replication troubleshooting tools.

Next post, we’ll going to go over the REPADMIN /SHOWREPL command and how to use it. In the meantime, if anyone has any questions or specific requests then fire away. This blog is for you all, so requests are taken...