Blog - Title

April, 2009

  • Netmon, MPS, RODC's, and that new OS you might have heard about

    Ned here. A few big pieces of news, in case you've been having a busy week:

    • Netmon 3.3 has been released. You can download from here. Read more about the new features (such as autoscroll, frame commenting, experts, WWAN support, and more) right here.
    • MPS Reports. They're back. They work on Vista and 2008, as well as XP and 2003. You don't need a support case to use them. Grab here. Hallelujah.
    • RODC's in DMZ's. Whitepaper on deploying AD Read-Only Domain Controllers into perimeter networks. Jump to it.

    And the mack daddy...

    • Windows 7 and Windows Server 2008 R2 Release Candidate. On schedule for an April 30th MSDN/TechNet release, and a May 5th public release. Wait, you don't believe me, your faithful Beta engineer? Pfffft, believe it.

    - Ned 'Talking Head' Pyle

  • Command line guide for Server Core

    Here is a nice primer on commonly used commands for Server Core.

    http://bsonposh.com/archives/710

    Be sure to check out our previous posts involving Server Core.

    http://blogs.technet.com/askds/archive/tags/Core/default.aspx

  • Other Directory Services Blogs

    There are quite a few people out there blogging about AD-related stuff. Below are some I know about. There is an OPML file attached to this post if you just want to import them all into a feed reader (make sure you click through to this post specifically to see the attachment at the bottom).

    If you know about other blogs that talk about Directory Services, let us know in the comments section.

    http://blogs.technet.com/activedirectoryua
    http://blogs.technet.com/ad
    http://blogs.technet.com/adfs
    http://blogs.technet.com/adfs_documentation
    http://blogs.technet.com/askds
    http://blogs.technet.com/benp/archive/tags/Active+Directory/default.aspx
    http://blogs.technet.com/deds
    http://blogs.technet.com/filecab
    http://blogs.technet.com/glennl
    http://blogs.technet.com/grouppolicy
    http://blogs.technet.com/ipv6
    http://blogs.technet.com/jpntsblog
    http://blogs.technet.com/janelewis
    http://blogs.technet.com/keithcombs
    http://blogs.technet.com/mempson
    http://blogs.technet.com/server_core
    http://blogs.technet.com/uphclean
    http://blogs.technet.com/windowsserver
    http://blogs.msdn.com/adpowershell
    http://blogs.msdn.com/donovanf
    http://blogs.msdn.com/richpec
    http://blogs.msdn.com/spatdsg
    http://blogs.msdn.com/ts
    http://blogs.msdn.com/w32time
    http://msmvps.com/blogs/ad
    http://msmvps.com/blogs/ulfbsimonweidner
    http://blogs.dirteam.com/blogs
    http://blogs.dirteam.com/blogs/tomek
    http://blogs.dirteam.com/blogs/jorge/archive/tags/Active+Directory/default.aspx
    http://theessentialexchange.com/blogs/michael/archive/tags/Active+Directory/default.aspx
    http://imav8n.wordpress.com/category/active-directory
    http://adobsession.blogspot.com
    http://bsonposh.com/archives/tag/active-directory
    http://www.gilkirkpatrick.com/blog
    http://blog.joeware.net
    http://www.open-a-socket.com/index.php/category/active-directory
    http://www.sdmsoftware.com/blog/
    http://jimmytheswede.blogspot.com
    http://blogs.microsoft.co.il/blogs/guyt
    http://trycatch.be/blogs/roggenk
    http://identity-des.com
    http://www.identityblog.com
    http://www.markwilson.co.uk/blog
    http://briandesmond.com/blog

     

     

  • How to configure the Windows Server 2008 CA Web Enrollment Proxy

    Hi all, Rob here again. I had a case recently where the customer wanted to have the Windows Server 2008 Certificate Authority website loaded on another machine.

    For those of you that do not know, you can install the Windows Server 2008 CA web site pages on an alternate server from the CA. One reason why you might deploy this configuration is if you currently have a Windows 2000 / Window Server 2003 Certification Authority and need to be able to deploy certificates to Windows Vista and Windows Server 2008 machines via the CA web site pages. Another reason might be because you want to offer certificate enrollment to Internet-based users but do not want to expose your Certification Authority to the Internet.

    While I was working with the customer I found quite a few different configurations that are possible with IIS7, but each configuration requires a different setup within Active Directory and IIS7.

    Getting started

    The first thing to be decided is which account will be used for the web application pool account.

    • Network Service.
    • A domain user account (also known as custom identity).

    The easier configuration is to leverage Network Service as the application pool account for the CertSrv web site. However, if you plan on bringing up more than one web server and use a network load balancer in front of the web servers your only option is to use a domain user account for the application pool identity.

    The second thing that needs to be decided is what type of delegation you require in the environment.

    I am not going to dig deeply into the differences between open and constrained delegation; you can view the Kerberos section of the AskDS blog. Basically, constrained delegation is more secure because you are limiting to what back end service the application is allowed to impersonate the user account to.

    Installing the Certification Authority website pages

    Alright, so you have made your decision on what type of delegation you want and what account you will be using for the Web Application Pool Identity. Let’s install the IIS components:

    1. Launch Server Manager (servermanager.msc).

    2. Click on the Add Roles link in the right hand pane.

    clip_image002

    Installation Figure 1 - Add Roles

    3. Check Active Directory Certificate Services and click the Next button.

    4. Uncheck Certification Authority and check Certification Authority Web Enrollment. You will see the pictured dialog box stating that IIS roles will need to be added, so click on the Add Required Role Services button, and then click the Next button.

    clip_image004

    Installation Figure 2 - Select Role Services

    5. You will then be asked to select the Enterprise Certification Authority that the web enrollment pages should use.

    6. Click the Browse button, and the Select Certification Authority dialog box will be shown listing all the Certification Authorities in the forest.

    7. Select the Certification Authority and click on the OK button, and click the Next button.

    clip_image006

    Installation Figure 3 - Select Certification Authority

    8. Next you will be taken to the list of Role Services for the Web Server (IIS).

    9. Add any other role services that you might need to support for the CA Web Enrollment web site like maybe enabling Basic Authentication keep in mind that Windows Integrated Authentication is selected by default.

    10. Then click the Next button.

    11. Lastly it will show you a confirmation screen before actually installing the web pages and IIS as shown below.

    clip_image008

    Installation Figure 4 - Confirm Installation Options

    The next sections are broken down first by what the IIS Application Pool Identity runs as and then what type of Kerberos delegation you require. You can skip to the section that is relevant to your environment.

    Setting up with Application Pool Identity set to Network Service

    This section will cover how to configure IIS and the Active Directory accounts to support Kerberos open delegation as well as constrained delegation when the application pool identity is setup for Network Service.

    Configuring for open delegation when using Network Service for AppPool Identity

    1. Open up Active Directory Users and Computers (DSA.MSC) and double click on the IIS Computer account.

    2. Click on the Delegation tab, and select Trust this computer for delegation to any service (Kerberos only).

    clip_image010

    Network Service Delegation 1 - Configure Computer for open delegation

    3. Then click OK.

    4. Next we need to open Internet Information Services (IIS) Manager snapin.

    5. Navigate to the CertSrv web application in the tree view, and double click on Authentication.

    6. You should see all the supported authentication types listed.

    7. Right click on Windows Authentication and select Enable if not already done.

    8. Right click on Windows Authentication and selected Advanced Settings…

    clip_image012

    Network Service Delegation 2 - Windows Authentication Advanced settings

    9. Check Enable Kernel-mode authentication, and click the OK button.

    clip_image014

    Network Service Delegation 3 - Enable Kernel-mode Authentication

    10. Reboot the IIS computer and you are ready to go.

    Configuring for constrained delegation when using Network Service for AppPool Identity

    If you need to support Basic Authentication on the website you will need to make sure that you configure constrained delegation with protocol transition. However, you will need to scroll down to section Configuring the web site to support Basic Authentication below for more steps required to support basic authentication.

    1. Open up Active Directory Users and Computers and find the IIS computer account.

    2. Double click on the IIS Computer account.

    3. Click on the Delegation tab and select Trust this computer for delegation to specified services only

    4. Once the above option selected, you need to make another choice of one of the following:

    a. Use Kerberos only (Kerberos constrained delegation).

    b. Use any authentication protocol (Kerberos constrained delegation with protocol transition).

    5. Click on the Add... button.

    6. You now have the Add Services dialog box, click on the Users or Computers… button.

    7. In the Select Users or Computers dialog, type in the Certification Authority computer account and click OK.

    8. Select the following services HOST and rpcss. Once they have been selected click the OK button.

    9. Once this is done it should look similar to the figure Network Service Delegation 4.

    clip_image016

    Network Service Delegation 4 - Configure Computer for constrained delegation

    10. Next we need to open Internet Information Services (IIS) Manager snapin

    11. Navigate to the CertSrv web application in the tree view, and double click on Authentication.

    12. You should see all the supported authentication types listed.

    13. Right click on Windows Authentication and select Enable if not already done.

    14. Right click on Windows Authentication and selected Advanced Settings…

    clip_image017

    Network Service Delegation 5 - Windows Authentication Advanced Settings

    15. Uncheck Enable Kernel-mode authentication, and click the OK button.

    clip_image019

    Network Service Delegation 6 - Disable Kernel-mode authentication

    16. Reboot the IIS computer and you are ready to go.

    Setting up with Application Pool Identity set to custom account

    This section will cover how to configure IIS and the Active Directory accounts to support Kerberos open delegation as well as constrained delegation when the application pool identity is setup for a custom account.

    Configuring the Application Pool identity custom account

    This section must be done whether you are using open or constrained delegation.

    1. Next we need to open Internet Information Services (IIS) Manager snapin

    2. Navigate to the Application Pools node in the tree view.

    3. Select DefaultAppPool right click and select Advanced Settings…

    clip_image021

    Custom Account Delegation 1 - Application Pool Advanced Settings

    4. Change the Identity on the Advanced Settings dialog box, which then brings up the Application Pool Identity dialog box. Click on the Set… button.

    5. In the Set Credentials dialog box, type in the domain user account to be used and password twice. Do not forget to type in the domain name. For example: FABRIKAM\IISKerbSvc

    6. Click OK

    7. Click OK

    8. Click OK

    clip_image023

    Custom Account Delegation 2 - Configure Custom account

    9. Open Active Directory Users and Computers snapin (dsa.msc) and find the domain account that the application pool was set to from step 5.

    10. Add this account the Windows Authorization Access Group.

    11. Click OK

    12. Exit Active Directory Users and Computers.

    clip_image025

    Custom Account Delegation 3 - Add Custom account to domain group

    13. Open Computer Management snapin (compmgmt.msc) and go to the local groups in the tree view.

    14. Add the account specified for the application pool identity to the IIS_IUSRS group

    clip_image027

    Custom Account Delegation 4 - Add Custom account to IIS_IUSRS

    15. Next we need to open Internet Information Services (IIS) Manager snapin

    16. Navigate to the CertSrv web application in the tree view, and double click on Authentication.

    17. You should see all the supported authentication types listed.

    18. Right click on Windows Authentication and select Enable if not already done.

    19. Right click on Windows Authentication and selected Advanced Settings…

    clip_image028

    Custom Account Delegation 5 - Windows Authentication Advanced Settings

    20. Check Enable Kernel-mode authentication, and click the OK button.

    clip_image029

    Custom Account Delegation 6 - Enable Kernel-mode authentication

    Configuring for open delegation when using custom account for AppPool Identity

    1. Open up Active Directory Users and Computers and double click on the IIS Computer account.

    2. Click on the Delegation tab, and select Trust this computer for delegation to any service (Kerberos only).

    clip_image010[1]

    Custom Account Delegation 7 - Configure Computer for open delegation

    3. Then click OK.

    4. Reboot the IIS computer and you are ready to go.

    Configuring for constrained delegation when using custom account for AppPool Identity

    If you need to support Basic Authentication on the website you will need to make sure that you configure constrained delegation with protocol transition. However, you will need to scroll down to the bottom of this blog for more steps required to support basic authentication.

    1. Open up Active Directory Users and Computers and find the IIS computer account.

    2. Double click on the IIS Computer account.

    3. Click on the Delegation tab and select Trust this computer for delegation to specified services only, and then select Use any authentication protocol.

    4. Click on the Add... button.

    5. You now have the Add Services dialog box, click on the Users or Computers… button.

    6. In the Select Users or Computers dialog, type in the Certification Authority computer account and click OK.

    7. Select the following services HOST and rpcss. Once they have been selected click the OK button.

    8. Once this is done it should look similar to the figure Custom Account Delegation 8.

    clip_image031

    Custom Account Delegation 8 - Configure Computer for constrained delegation

    9. Next we need to add a Service Principal Name to the Application Pool account. This is done by using the SetSPN.exe utility. If my web site name is the IIS computer name (FAB-RT-MEM1) and I am using the account of FABRIKAM\IISKerbSvc the following commands would be ran:

    SetSPN –A http/fab-rt-mem1 FABRIKAM\IISKerbSvc

    SetSPN –A http/fab-rt-mem1.fabrikam.com FABRIKAM\IISKerbSvc

    10. Find the Application Pool Identity account within Active Directory Users and Computers.

    11. Double click on the domain user account.

    12. Click on the Delegation tab and select Trust this computer for delegation to specified services only and then select Use any authentication protocol.

    13. Click on the Add... button.

    14. You now have the Add Services dialog box, click on the Users or Computers… button.

    15. In the Select Users or Computers dialog, type in the Certification Authority computer account and click OK.

    16. Select the following services HOST and rpcss. Once they have been selected click the OK button.

    17. Click on the Add... button.

    18. You now have the Add Services dialog box, click on the Users or Computers… button.

    19. In the Select Users or Computers dialog, type in the IIS Server computer account and click OK.

    20. Select the following services HOST. Once they have been selected click the OK button.

    21. Once this is done it should look similar to the figure Custom Account Delegation 9.

    clip_image033

    Custom Account Delegation 9 - Configure custom account for constrained delegation

    22. Next we need to open Internet Information Services (IIS) Manager snapin

    23. Navigate to the CertSrv web application in the tree view, and double click on Authentication.

    24. You should see all the supported authentication types listed.

    25. Right click on Windows Authentication and select Enable if not already done.

    26. Right click on Windows Authentication and selected Advanced Settings…

    clip_image034

    Custom Account Delegation 10 - Windows Authentication Advanced Settings

    27. Uncheck Enable Kernel-mode authentication, and click the OK button.

    clip_image035

    Custom Account Delegation 11 - Disable Kernel-mode authentication

    28. Reboot the IIS computer and you are ready to go.

    Configuring the web site to support Basic Authentication

    Alright, so you need to support basic authentication huh…. Well, before you do the below steps you will first want to make sure that you have followed one of the set of steps for configuring constrained delegation and selected the Use any authentication protocol when you setup delegation.

    1. Next we need to open Internet Information Services (IIS) Manager snapin

    2. Navigate to the CertSrv web application in the tree view, and double click on Authentication.

    3. You should see all the supported authentication types listed.

    4. Right click on Basic Authentication and select Enable if not already done.

    5. The next part we will need to edit one of the following config files on the file system.

    • %systemroot%\system32\certsrv\en-US\Web.config

    You are looking for the part that is in bold below.


    <authentication>
       <basicAuthentication enabled="true" realm="contoso.com" defaultLogonDomain="CONTOSO" logonMethod="Network" />
       <windowsAuthentication enabled="false" useKernelMode="false" />
    </authentication>

    Change it from logonMethod=”Network” to logonMethod=”ClearText”

    NOTE: You might not have the web.config file located here.

    • %systemroot%\system32\inetsrv\config\applicationHost.config

    You are looking for the part in bold below, and then looking for basicAuthentication.


    <location path="Default Web Site/CertSrv">
          <system.webServer>
              <handlers accessPolicy="Read, Script">
                  <clear />
                  <add name="ISAPI-dll" path="*.dll" verb="*" modules="IsapiModule" resourceType="File" requireAccess="Execute" allowPathInfo="true" />
            …
                  <add name="StaticFile" path="*" verb="*" modules="StaticFileModule,DefaultDocumentModule,DirectoryListingModule" resourceType="Either" requireAccess="Read" />
              </handlers>
              <security>
                  <authentication>
                      <windowsAuthentication enabled="false">
                          <providers>
                              <clear />
                              <add value="Negotiate" />
                              <add value="NTLM" />
                          </providers>
                      </windowsAuthentication>
                      <anonymousAuthentication enabled="false" logonMethod="Network"/>
                      <basicAuthentication enabled="true" logonMethod="Network" />
                  </authentication>
            …

    Change it from logonMethod=”Network” to logonMethod=”ClearText” for basicAuthentication section.

    6. Reboot the IIS computer and you are ready to go.

     

    The steps outlined in this blog are specifically for the CA website being loaded on Windows Server 2008 and will not translate properly to the configuration required for the website on Windows Server 2003.

    The changes made to IIS7 have been drastic, from the GUI to the underlying code, so I hope that you have found this blog interesting and helpful.

    - Rob Greene

  • New Directory Services KB Articles 4/11-4/18

    New KB articles related to Directory Services for the week of 4/11-4/18.

    969299

    A DNS zone transfer between two Windows Server 2003-based DNS servers generates incomplete zone data when the DNS transfer process stops unexpectedly

    968597

    The Tcpipv6.sys driver stops responding to any TCP/IPv6 requests on a Windows Server 2003 SP2-based computer when the driver binds to many network adapters

    969289

    All network share access through the SMB protocol (client-side redirector) may fail on a Windows Server 2003-based computer

    962994

    Windows Server 2003 SP2-based domain controllers return incorrect error code to Kerberos requests during the shutdown process

    969429

    Windows 7 clients cannot locate the Active Directory Management Gateway service that is installed on Windows Server 2003-based domain controllers

    967176

    A Windows Server 2003-based file server may return file identifiers (Fids) that have the 0xffff value under heavy stress

    967357

    Some files are missing on a Windows Server 2003 R2-based computer after a DFSR replication

    969451

    Users cannot perform authentication through ADFS in a Windows Server 2003 R2 environment when the UPN suffixes contain a character that expands to a two-letter pair

    969417

    How do I enable User Account Control in Windows Vista?

  • Conficker causes LSASS to consume CPU Time on Domain Controllers

    Hi Gautam here, I wanted to blog about a high-impact problem we have been seeing recently.

    The problem has to do with LSASS consuming a lot of CPU time on your Domain Controllers (DC's). The cause of this high CPU turns out to be Conficker infected computers throwing bad passwords against the DC's. The rate of bad passwords can be as high as 10,000 per minute from multiple clients.

    Technical information on Conficker can be found here.

    The problem could manifest itself in many ways, some being...

    1. Slow authentication and logons being reported by users,
    2. Slow mail flow
    3. Slow Resource access (resources could be Files shares, printers and more) or even complete failure in Resource access.

    Some of the above problems take time to be narrowed down. You typically will have to go through a few other pieces before you narrow down on the domain controllers being bogged down with high CPU time.

    Background:

    CPU usage on domain controllers continues to be very high (I'm rating high = 70% and above as long as this is not normal for the DC). On looking closer, you find LSASS.EXE eating up all of this CPU. Perfmon reports show the CPU usage stays more or less consistent throughout the day. It doesn't climb down during off-peak hours.

    As you can imagine, this high CPU usage affects other workflows which are AD dependent – including Exchange/SharePoint/Authentication etc.

    If you temporarily pull the network cable from the DC and wait a few minutes, LSASS drops back down to ~1% or whatever value is normal in your setup. Ned Pyle has the logic of pulling out the network cable described in a previous post in detail.

    In this case as well, we saw that pulling out the network cable brings down the LSASS CPU usage to normal limits. Plugging it back in makes LSASS shoot up again to 80%-90% CPU.

    If you follow the steps which Ned has documented in the blog, network traces will show a HUGE number of authentication requests coming into the DCs. Now it's not always easy to differentiate between bad and good traffic when you are looking at 100MB worth of network traffic.

    In this case however, what you are bound to see if something like the below in the network traces – I highly recommend using Netmon 3* - the Conversations feature is ideal to work through a large trace which you are bound to get when collecting network traces from the DC.

    09:54:16.593    192.168.0.1    DC01.CONTOSO.COM    KerberosV5    KerberosV5:AS Request Cname: User1 Realm: CONTOSO.COM Sname: krbtgt/CONTOSO.COM

    09:54:16.625    DC01.CONTOSO.COM    192.168.0.1    KerberosV5    KerberosV5:KRB_ERROR - KDC_ERR_PREAUTH_FAILED (24)

    OR

    09:54:16.531    192.168.0.2    DC01.CONTOSO.COM    TCP    TCP:Flags=......S., SrcPort=4614, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=3314092510, Ack=0, Win=65535 ( ) = 65535

    09:54:16.531    DC01.CONTOSO.COM    192.168.0.2    TCP    TCP:Flags=...A..S., SrcPort=Microsoft-DS(445), DstPort=4614, PayloadLen=0, Seq=1831638666, Ack=3314092511, Win=17520 ( Scale factor not supported ) = 17520

    09:54:16.531    192.168.0.2    DC01.CONTOSO.COM    TCP    TCP:Flags=...A...., SrcPort=4614, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=3314092511, Ack=1831638667, Win=65535 (scale factor 0x0) = 65535

    09:54:16.531    192.168.0.2    DC01.CONTOSO.COM    SMB    SMB:C; Negotiate, Dialect = PC NETWORK PROGRAM 1.0, LANMAN1.0, Windows for Workgroups 3.1a, LM1.2X002, LANMAN2.1, NT LM 0.12

    09:54:16.531    DC01.CONTOSO.COM    192.168.0.2    SMB    SMB:R; Negotiate, Dialect is NT LM 0.12 (#5), SpnegoNegTokenInit

    09:54:16.578    192.168.0.2    DC01.CONTOSO.COM    SMB    SMB:C; Session Setup Andx, NTLM NEGOTIATE MESSAGE

    09:54:16.578    DC01.CONTOSO.COM    192.168.0.2    SMB    SMB:R; Session Setup Andx, NTLM CHALLENGE MESSAGE - NT Status: System - Error, Code = (22) STATUS_MORE_PROCESSING_REQUIRED

    09:54:16.593    192.168.0.2    DC01.CONTOSO.COM    TCP    TCP:Flags=...A...F, SrcPort=4614, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=3314092888, Ack=1831639470, Win=64732 (scale factor 0x0) = 64732

    09:54:16.593    DC01.CONTOSO.COM    192.168.0.2    TCP    TCP:Flags=...A...F, SrcPort=Microsoft-DS(445), DstPort=4614, PayloadLen=0, Seq=1831639470, Ack=3314092889, Win=17143 (scale factor 0x0) = 17143

    09:54:16.593    192.168.0.2    DC01.CONTOSO.COM    TCP    TCP:Flags=...A...., SrcPort=4614, DstPort=Microsoft-DS(445), PayloadLen=0, Seq=3314092889, Ack=1831639471, Win=64732 (scale factor 0x0) = 64732

    Now, in the above three examples of network traffic, the first one with the Kerberos KDC_ERR_PREAUTH_FAILED is a sure shot bad password attempt. The other two traces aren't necessarily always bad authentication attempts, but is data connections to LSARPC which I saw on three of the four recent cases I had with this issue.

    SPA reports will show high number of calls to SAMSRV or LSARPC. Tim Springston, who runs his own excellent AD related blog, has discussed the using of SPA here.

    With TOP users attained from both SPA and from the Network traces, we explored 3 of the top client computers. We pulled MPSReports (an often used PSS Data collection tool) from these client computers. The first thing which stood out in the event logs was all the Audit Failures Logon/Logoff Event Id 529's in the Security Event logs.

    Note: by default, only Success for Logon/Logoff and Account Logon is enabled. And in this case, the Domain Controllers were running with the defaults. The client computers had Failure for Logon/Logoff enabled.

    More..

    This of course led us to...

    1. Checking this customer's account lockout policy –we saw they did not have account lockouts enabled
    2. We enabled Failure for Account Logon on a policy which applied to the Domain Controllers as well.

    No sooner had the failure-audit policy applied to the DC, the Security event logs were filled with Audit Failures Account Logon Event Id 675. Here is an example of a 675 event.

    Event Type:    Failure Audit
    Event Source:    Security
    Event Category:    Account Logon
    Event ID:    675
    Date:        3/23/2009
    Time:        3:03:57 AM
    User:        NT AUTHORITY\SYSTEM
    Computer:    DC01
    Description:
    Pre-authentication failed:

        User Name:    User1
        User ID:        %{S-1-5-21-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxx-xxxxx}
        Service Name:    krbtgt/CONTOSO.COM
        Pre-Authentication Type:    0x2
        Failure Code:    0x18
        Client Address:    192.168.0.100 ß IP of the computer which is throwing the bad credentials

    Using EVENTCOMBMT to pull out the relevant event Ids from various DCs (namely 529, 644, 675, 676, and 681) and a little bit of Office Excel magic, I quickly had a list of ~100 computers sending bad passwords within a 30 minute time frame. The total number of failed logons were enough to drive up the LSASS.EXE CPU usage. LSASS ofcourse was only doing its job of keeping up with the load and failing the bad authentication attempts.

    Putting it all together:

    The kind of (multiple user logons from a single computer) and the rate (100's of attempts per minute per computer) at which they were throwing bad passwords, were a pretty sure sign of malware activity. A few more client computers, which we picked up from the SPA and Netmon reports, revealed traces of Conficker. With the Microsoft PSS Security team and the customers own Antivirus vendor involved, they were able to patch, scan, and clean their computers and this effort showed the LSASS CPU usage on the DCs drop down dramatically.

    So from high LSASS CPU – to network traces leading to TOP client computers – to security events – to DC security events – back to the client computers! As you can imagine, it took some time in nailing down the first time. The 2nd, 3rd and 4th cases were nailed down to unpatched computers infected by Conficker way quicker.

    I hope with this blog post out, someone will save themselves a LOT of time and effort when facing such an issue.

    • Gautam Anand
  • “The LastLogonTimeStamp Attribute” – “What it was designed for and how it works”

    Warren here. In Windows Server 2003 we introduced the lastLogontimeStamp attribute. Administrators can use the lastLogontimeStamp attribute to determine if a user or computer account has recently logged onto the domain. Using this information administrators can then review the accounts identified and determine if they are still needed and take appropriate action.

    Intended Use

    It is important to note that the intended purpose of the lastLogontimeStamp attribute to help identify inactive computer and user accounts. The lastLogon attribute is not designed to provide real time logon information. With default settings in place the lastLogontimeStamp will be 9-14 days behind the current date.

    If you are looking for more “real-time” logon tracking you will need to query the Security Event log on your DC’s for the desired logon events i.e. 528 –Windows XP\2003 and earlier or 4624 Windows Vista\2008 . See this blog post by Eric Fitzgerald for more info. (I think he knows something about auditing)

    IMO your best bet for near real-time data is to use an event log collection service to gather all domain controller security event logs to a centralized database. You can then query a single database for the desired logon events. Microsoft’s solution for security event log collection is Audit Collection Services. There are many 3rd party solutions as well.

    How it worked in Windows 2000

    Prior to Windows Server 2003 administrators had to query the lastLogon attribute to determine the most recent logon of user or computer account. This process was time consuming as the lastLogon attribute is updated only on the DC that validates the logon request. The lastLogon attribute is not replicated. So in the past to determine the most recent logon of a user or computer account the lastLogon attribute had to be queried on all domain controllers (at least in concept) and then the most recent date for lastLogon had to be determined from all the results returned. In Windows 2003 and higher lastLogon is still has the same behavior. It is updated only on the validating DC and is not replicated.

    How it works in Windows Server 2003 and later

    In contrast the lastLogontimeStamp attribute is replicated so all DC's have the same value for the attribute (after replication convergence). Therefore you can query a single DC to find all the users or all the computers that have not logged in within a certain time.

    Requirements

    Your Windows domain must be at Windows 2003 Domain Functional Level for updates to the llastLogontimeStamp to occur.

    Logon types and that will trigger an update to the lastLogontimeStamp attribute.

    The lastLogontimeStamp attribute is not updated with all logon types or at every logon. The good news is that the logon types that admins usually care about will update the attribute and often enough to accomplish its task of identifying inactive accounts.

    Interactive, Network, and Service logons will update the lastLogontimeStamp. So if a user logs on interactively, browses a network share, access the email server, runs an LDAP query etc… the lastLogontimeStamp attribute will updated if the right condition is met. (The conditions are discussed below in the section Update and Replication of lastLogontimeStamp.

    As of Windows 2003 SP1 these logon types will NOT update lastLogontimeStamp

    • Certificate mapping through Microsoft Internet Information Services (IIS).
    • Microsoft .NET Passport mapping through IIS.

    [Update June 19, 2009 - removed one item from the list above that is under debate in a repro currently. Will update when we have more word] 

    ====================================================

    Update and Replication of lastLogontimeStamp

    First become acquainted with the ms-DS-Logon-Time-Sync-Interval attribute. It is an attribute of the domain NC and controls the granularity (in days) with which the lastLogontimeStamp attribute is updated. The default value is 14 and is set in code. Meaning that if you look at this attribute in ADSIEDIT.MSC and you see it as "Not Set" don't be alarmed. This just means the system is using the default value of 14.

    The lastLogontimeStamp attribute is not updated every time a user or computer logs on to the domain. The decision to update the value is based on the current date minus the value of the (ms-DS-Logon-Time-Sync-Interval attribute minus a random percentage of 5). If the result is equal to or greater than lastLogontimeStamp the attribute is updated. There are no special considerations for replication of lastLogontimeStamp. If the attribute is updated it is replicated like any other attribute update. This is not urgent replication

    Walkthrough of a lastLogontimeStampUpdate update

    1. (Assuming the value of the ms-DS-Logon-Time-Sync-Interval is at the default of 14)

    2. User logs on to the domain

    3. The lastLogontimeStamp attribute value of the user is retrieved

    4. 14 - (Random percentage of 5) = X

    5. Current date - value of lastLogontimeStamp = Y

    6. X ≤ Y - update lastLognTimeStamp

    7. X > Y - do not update lastLogontimeStamp

    Why the Randomization?

    This randomization is done to prevent an update of the lastLogontimeStamp attribute from many accounts at the same time causing a high replication load on the DC's. Remember the purpose of the lastLogontimeStamp attribute is locate inactive accounts not provide real-time logon information.

    Controlling the update frequency of lastLogontimeStamp.

    It is possible to change the frequency of updates to the lastLogonTime stamp or turn it off completely if desired. If you need a different time interval you will need to adjust the value of the msDS-LogonTimeSyncInterval attribute to a value between 5-100,000. Yes that’s right: the max value is 100,000 days… Or if you prefer ~280 years... And the max value was set in code not in the schema. (I guess the dev was counting on medical science to solve that pesky aging problem.)

    In my experience the default settings can accommodate almost anyone and there is no need to change the update interval. Most customers I have talked to start considering accounts potentially inactive at the 30 day or higher mark of inactivity.

    Note: If the msDS-LogonTimeSyncInterval is less than 5 days, the randomization is not put into effect.

    How do I turn this thing off?

    If you want to disable the lastLogontimeStamp feature set the msDS-LogonTimeSyncInterval attribute to 0.

    I personally have never spoken with anyone that really had a business need to change how often lastLogontimeStamp needs to be updated. Once it was explained how the update process works and it was proven that the attribute is current and replicated to all DC’s that was all that was needed. If really think you need a more recent timestamp than 9-14 days for inactive account detection I suggest you make small changes and monitor DC workloads. This is especially true in large environments.

    =======================================

    Clearing up the confusion - Verifying that LastLogontimeStamp is in sync across all DCs in the domain.

    Many times customers will be concerned about what their tools are displaying to them (usually a very old date) as the lastLogontimeStamp of a user compared to what they know to be a more accurate date. This is almost always due to the admin using a tool that queries the lastLogon attribute instead of the lastLogontimeStamp attribute.

    For example acctinfo.dll that is included with the Account Lockout tools will display the lastLogon attribute data not the lastLogontimeStamp data. In some cases the date the tool reports may be months or years out of date or display nothing at all. This is because they are querying the lastLogon attribute and the user they are looking up has either never been authenticated by the reference DC (in the case of null) or has not been authenticated by the reference DC in a very long time.

    How to tell if lastLogontimeStamp is in sync

    To verify if the lastLogonTime stamp is being updated and replicated as expected you can use repadmin.exe with the showattr switch. Some examples are given below. These examples are intended to demonstrate that lastLogontimeStamp is being updated within the window of 9-14 days and replicated to all DC’s in the domain. They are not an example of how to manage stale accounts.

    1. Using repadmin to check the value of lastLogontimeStamp on all DC's in a domain for one user:

    repadmin /showattr * (DN of the target user) /attrs:lastLogontimeStamp >lastLogontimeStamp.txt

    Example:

    repadmin /showattr * CN=user1,OU=accounting,DC=domain,dc=com /attrs:lastLogontimeStamp >lastLogontimeStamp.txt

    2. Using repadmin to dump the lastLogontimeStamp for all users in a domain including users that have no data in the lastLogontimeStamp attribute:

    repadmin /showattr * /subtree /filter:"(&(objectCategory=Person)(objectClass=user))" /attrs:lastLogontimeStamp >lastLogontimeStamp.txt

    3. Dump lastLogonTime stamp for users but only ones that have the attribute populated

    repadmin /showattr * dc=domain,dc=com /subtree /filter:"((&(lastLogontimeStamp=*)(objectCategory=Person)(objectClass=user)))" /attrs:lastLogontimeStamp > lastLogontimeStamp-2-22-2009.txt

    - Warren ‘For Once not DFSR’ Williams

  • New Directory Services KB Articles 4/5-4/11

    New KB articles related to Directory Services for the week of 4/5-4/11.

    968730

    Windows Server 2003 and Windows XP clients cannot obtain certificates from a Windows Server 2008-based certification authority (CA) if the CA is configured to use SHA2 256 or higher encryption

    961731

    On a computer that is running Windows Server 2008 or Windows Vista, the certificates and the cryptographic keys are unusable after the user password is changed on another network computer

    961403

    After the Active Directory RMS role is decommissioned on a Windows Server 2008-based server, users cannot open documents that IRM helps protect

    969307

    An error occurs when you run the ADPREP/FORESTPREP command on a Windows Server 2003-based or Windows Server 2008-based computer: "An attribute with the same link identifier already exists"