Blog - Title

May, 2012

  • Come get Windows 8 Release Preview and Windows Server 2012 Release Candidate

    Ned here. Go there:

    While you wait for the downloads, check out Delivering the Windows 8 Release Preview on the B8 blog to learn a few more things about Windows 8 from a non-enterprise standpoint. More Windows 8 here. More Windows Server 2012 here.

    IT Pro content will trickle out; keep your eyes peeled on the Windows Server blog and elsewhere. And know this: we have everything documented to the nth degree and you will learn everything you need, at least after RTM, even if it harelips the governor.

    See you then.

    Ned “Slim Pickens” Pyle

  • New Slow Logon, Slow Boot Troubleshooting Content

    Hi all, Ned here again. We get emailed here all the time about issues involving delays in user logons. Often enough that, a few years back, Bob wrote a multi-part article on the subject.

    Taking it to the next level, some of my esteemed colleagues have created a multi-part TechNet Wiki series on understanding, analyzing, and troubleshooting slow logons and slow boots. These include:

    Before you shrug this off, consider the following example, where we assume for our hypothetical company:

    • Employees work 250 days per year (50 weeks * 5 days per week)
    • Employee labor costs $2 per minute
    • Each employees boots and logs on to a single desktop computer only once per day
    • There are 25 and 30 seconds of removable delay from the boot and logon operations

    That means an annual cost of:

    image
    Benjamin Franklin would not be pleased

    Even if you take just the understated US Bureau of Labor private sector compensation cost numbers (roughly $0.50 average employee total compensation cost per minute), you are still hemorrhaging cash. And those numbers just cover direct compensation and benefit costs, not all the other overhead  that goes into an employee, as well as the fact that they are not producing anything during that time - you are paying them to do nothing. Need I mention that the computer-using employees are probably costing you nearly twice that number?

    Get to reading, people – this is a big deal.

    - Ned “a penny saved is a penny earned” Pyle

  • Friday Mail Sack: Mothers day pfffft… when is son’s day?

    Hi folks, Ned here again. It’s been a little while since the last sack, but I have a good excuse: I just finished writing a poop ton of Windows Server 2012 depth training that our support folks around the world will use to make your lives easier (someday). If I ever open MS Word again it will be too soon, and I’ll probably say the same thing about PowerPoint by June.

    Anyhoo, let’s get to it. This week we talk about:

    Question

    Is it possible to use any ActiveDirectory module cmdlets through invoke-command against a remote non-Windows Server 2012 DC where the module is installed? It always blows up for me as it tries to “locally” (remotely) use the non-existent ADWS with error “Unable to contact the server. This may be because the server does not exist, it is currently down, or it does not have the active directory web services running”

    image

    Answer

    Yes, but you have to ignore that terribly misleading error and put your thinking cap on: the problem is your credentials. When you invoke-command, you make the remote server run the local PowerShell on your behalf. In this case that remote command has to go off-box to yet another remote server – a DC running ADWS. This means a multi-hop credential scenario. Provide –credential (get-credential) to your called cmdlets inside the curly braces and it’ll work fine.

    Question

    We are using a USMT /hardlink migration to preserve disk space and increase performance. However, performance is crazy slow and we’re actually running out of disk space on some machines that have very large files like PSTs. My scanstate log shows:

    Error [0x000000] Write error 112 for C:\users\ned\Desktop [somebig.pst]. Windows error 112 description: There is not enough space on the disk.[gle=0x00000070]

    Error [0x080000] Error 2147942512 while gathering object C:\users\ned\Desktop\somebig.pst. Shell application requested abort![gle=0x00000070]

    Answer

    These files are encrypted and you are using /efs:copyraw instead of /efs:hardlink. Encrypted files are copied into the store whole instead of hardlink'ing, unless you specify /efs:hardlink. If you had not included /efs, this file would have failed with, "File X is encrypted. Use the /efs option to specify a different way to handle this file".

    Yes, I realize that we should probably just require that option. But think of all the billable hours we just gave you!

    Question

    I was using your DFSR pre-seeding post and am finding that robocopy /B is slows down my migration compared to not using it. Is that required for preseeding?

    Answer

    The /B mode, while inherently slower, ensures that files are copied using a backup API regardless of permissions. It is the safest way, so I took the prudent route when I wrote the sample command. It’s definitely expected to be slower – in my semi-scientific repro’s the difference was ~1.75 times slower on average.

    However, /B not required if you are 100% sure you have at least READ permissions to all files.  The downside here is a lot of failures due to permissions might end up making things even slower than just going /B; you will have to test it.

    If you are using Windows Server 2012 and have plenty of hardware to back it up, you can use the following options that really make the robocopy fly, at the cost of memory, CPU, and network utilization (and possibly, some files not copying at all):

    Robocopy <foo> <bar> /e /j /copyall /xd dfsrprivate /log:<sna.foo> /tee /t:128 /r:1

    For those that have used this before, it will look pretty similar – but note:

    • Adds /J option (first introduced in Win8 robocopy) - now performs unbuffered IO, which means gigantic files like ISO and VHD really fly and a 1Gbps network is finally heavily utilized. Adds significant memory overhead, naturally.
    • Add /MT:128 to use 128 simultaneous file copy threads. Adds CPU overhead, naturally.
    • Removes /B and /R:6 in order to guarantee fastest copy method. Make sure you review the log and recopy any failures individually, as you are now skipping any files that failed to copy on the first try.

     

    Question

    Recently I came across an user account that keeps locking out (yes, I've read several of your blogs where you say account lockout policies are bad "Turning on account lockouts is a way to guarantee someone with no credentials can deny service to your entire domain"). We get the Event ID of 4740 saying the account has been locked out, but the calling computer name is blank:

     

    Log Name:     Security

     

    Event ID:     4740

     

    Level:         Information

     

    Description:

     

    A user account was locked out.

     

    Subject:

     

    Security ID: SYSTEM

     

    Account Name: someaccount

     

    Account Domain: somedomain

     

    Logon ID: 0x3e7

     

    Account That Was Locked Out:

     

    Security ID: somesid

     

    Account Name: someguy

     

    Additional Information:

     

    Caller Computer Name:

     

    The 0xC000006A indicates a bad password attempt. This happens every 5 minutes and eventually results in the account being locked out. We can see that the bad password attempts are coming via COMP1 (which is a proxy server) however we can't work out what is sending the requests to COMP1 as the computer is blank again (there should be a computer name).

    Are we missing something here? Is there something else we could be doing to track this down? Is the calling computer name being blank indicative of some other problem or just perhaps means the calling device is a non-Microsoft device?

    Answer

    (I am going to channel my inner Eric here):

    A blank computer name is not unexpected, unfortunately. The audit system relies on the sending computers to provide that information as part of the actual authentication attempt. Kerberos does not have a reliable way to provide the remote computer info in many cases. Name resolution info about a sending computer is also easily spoofed. This is especially true with transitive NTLM logons, where we are relying on one computer to provide info for another computer. NTLM provides names but they are also easily spoofed so even when you see a computer name in auditing, you are mainly asking an honest person to tell you the truth.

    Since it happens very frequently and predictably, I’d configure a network capture on the sending server to run in a circular fashion, then wait for the lock out and stop the event. You’d see all of the traffic and now know exactly who sent it. If the lockout was longer running and less predictable, I’d recommend using a network capture configured to trace in a circular fashion until that 4740 event writes. Then you can see what the sending IP address is and hunt down that machine. Different techniques here:

    [And the customer later noted that since it’s a proxy server, it has lots of logs – and they told him the offender]

    Question

    I am testing USMT 5.0 and finding that if I migrate certain Windows 7 computers to Windows 8 Consumer Preview, Modern Apps won’t start. Some have errors, some just start then go away.

    Answer

    Argh. The problem here is Windows 7’s built-in manifest that implements microsoft-windows-com-base , which then copies this registry key:

    HKEY_LOCAL_MACHINE\Software\Microsoft\OLE

    If the DCOM permissions are modified in that key, they migrate over and interfere with the ones needed by Modern Apps to run. This is a known issue and already fixed so that we don’t copy those values onto Windows 8 anymore. It was never a good idea in the first place, as any applications needing special permissions will just set their own anyways when installed.

    And it’s burned us in the past too…

    Question

    Are there any available PowerShell, WMI, or command-line options for configuring an OCSP responder? I know that I can install the feature with the Add-WindowsFeature, but I'd like to script configuring the responder and creating the array.

    Answer

    [Courtesy of the Jonathan “oh no, feet!” Stephens – Ned]

    There are currently no command line tools or dedicated PowerShell cmdlets available to perform management tasks on the Online Responder. You can, however, use the COM interfaces IOCSPAdmin and IOSCPCAConfiguration to manage the revocation providers on the Online Responder.

    1. Create an IOSCPAdmin object.
    2. The IOSCPAdmin::OCSPCAConfigurationCollection property will return an IOCSPCAConfigurationCollection object.
    3. Use IOCSPCAConfigurationCollection::CreateCAConfiguration to create a new revocation provider.
    4. Make sure you call IOCSPAdmin::SetConfiguration when finished so the online responder gets updated with the new revocation configuration.

    Because these are COM interfaces, you can call them from VBScript or PowerShell, so you have great flexibility in how you write your script.

    Question

    I want to use Windows Desktop Search with DFS Namespaces but according to this TechNet Forum thread it’s not possible to add remote indexes on namespaces. What say you?

    Answer

    There is no DFSN+WDS remote index integration in any OS, including Windows 8 Consumer Preview. At its heart, this comes down to being a massive architectural change in WDS that just hasn’t gotten traction. You can still point to the targets as remote indexes, naturally.

    Question

    Certain files – as pointed out here by AlexSemi – that end with invalid characters like a dot or a space break USMT migration. One way to create these files is to use the echo command into a device path like so:

    image

    These files can’t be opened by anything in Windows, it seems.

    image

    When you try to migrate, you end up with a fatal “windows error 2” “the system cannot find the file specified” error unless you skip the files using /C:

    image

    What gives?

    Answer

    Quit making invalid files! :-)

    USMT didn’t invent CreateFile() so its options here are rather limited… USMT 5.0 handles this case correctly through error control - it skips these files when hardlink’ing because Windows returns that they “don’t exist”. Here is my scanstate log using USMT 5.0 beta, where I used /hardlink and did NOT provide /C:

    image

    In the case of non-hardlink, scanstate copies them without their invalid names and they become non-dotted/non-spaced valid files (even in USMT 4.0). To make it copy these invalid files with the actual invalid name would require a complete re-architecting of USMT or the Win32 file APIs. And why – so that everyone could continue to not open them?

    Other Stuff

    In case you missed it, Windows 8 Enterprise Edition details. With all the new licensing and activation goodness, Enterprise versions are finally within reach of any size customer. Yes, that means you!

    Very solid Mother’s Day TV mash up (a little sweary, but you can’t fight a something that combines The Wire, 30 Rock, and The Cosbys)

    Zombie mall experience. I have to fly to Reading in June to teach… this might be on the agenda

    Well, it’s about time - Congress doesn't "like" it when employers ask for Facebook login details

    Your mother is not this awesome:

    image
    That, my friend, is a Skyrim birthday cake

    SportsCenter wins again (thanks Mark!)

    Don’t miss the latest Between Two Ferns (veeerrrry sweary, but Zach Galifianakis at his best; I just wish they’d add the Tina Fey episode)

    But what happens if you eat it before you read the survival tips, Land Rover?!

     

    Until next time,

    - Ned “demon spawn” Pyle

  • Do not skip the latest B8 boot post

    Hey all, Ned here. The Building Windows 8 blog recently posted a new article from Chris Clark that you might have passed over due to the title, which sounds like another article on boot performance:

    Designing for PCs that boot faster than ever before

    Don’t skip it! A more descriptive title would have been “The F8 and F2 boot menus are gone on Windows 8 and you need to read this post to do your job, IT Pro.

    Windows 8 is designed to run on hardware that boots too fast for a human to react through a keyboard, requiring new methods to get to a boot menu. Note down what the article describes so that when you need to troubleshoot a slow logon or slow boot, you can get into Safe Mode and other diagnostic states (PC Settings, shift+restart, shutdown.exe /o /r, msconfig.exe). All of these apply to Windows 8 Developer Preview and Windows Server “8” Beta, which you can get your hot little hands on right now.

    It is also important to note – and not mentioned in the article – that on Windows Server 2012 only, you can still use F8. The new boot menu system eventually gets you back to the familiar menu with your favorite DSRM option too, so don’t feel like we’re making you relearn everything:

    image 

    Also not mentioned but preemptively answered now: while shutdown /o was updated to support the new boot menus, the restart-computer Windows PowerShell cmdlet was not.

    - Ned “Doc Martens” Pyle

  • Dynamic Access Control intro on Windows Server blog

    Hey all, Ned here with a quick “xerox” post: the Dynamic Access Control developers have released a good intro on their octo-feature through the Windows Server Blog:

    Introduction to Windows Server 2012 Dynamic Access Control

    It’s written by Nir Ben-Zvi, a Program Manager on the Windows Server development team. If you’re unfamiliar with DAC, this is a great first read. Here’s a quote:

    These focus areas were then translated to a set of Windows capabilities that enable data compliance in partner and Windows-based solutions.

    • Add the ability to configure Central Access and Audit Policies in Active Directory. These policies are based on conditional expressions that take into account the following so that organizations can translate business requirements to efficient policy enforcement and considerably reduce the number of security groups needed for access control:
      • Who the user is
      • What device they are using, and
      • What data is being accessed
    • Integrate claims into Windows authentication (Kerberos) so that users and devices can be described not only by the security groups they belong to, but also by claims such as: “User is from the Finance department” and “User’s security clearance is High”
    • Enhance the File Classification Infrastructure to allow business owners and users to identify (tag) their data so that IT administrators are able to target policies based on this tagging. This ability works in parallel with the ability of the File Classification Infrastructure to automatically classify files based on content or any other characteristics
    • Integrate Rights Management Services to automatically protect (encrypt) sensitive information on servers so that even when the information leaves the server, it is still protected.

    Click to the read the rest.

    If you are looking for more depth and “how it works”, check out our very own Mike Stephens’ downloadable whitepaper:

    Understand and Troubleshoot Dynamic Access Control in Windows Server "8" Beta

    Until next time,

    Ned “10 cent copies” Pyle

  • More Central Access Policies blogging happening

    Hi all, Ned here. Our friend Nir has another new DAC-related post up, this time on the File Cab blog: 

    Getting started with Central Access Policies - Reducing security group complexity and achieving data access compliance using Dynamic Access Control

    If you need a reason to go read this, consider the following quote:

    "So, we have 2,000 groups, 2,000 ACLs and many groups that are affected by a person changing a role not to mention the complexity of adding another level (say Branch) or the implications if we want to change the folder structure.

    With Dynamic Access Control, you can cut the number of groups down from 2,000 to 71 (50 for country, 20 for department and 1 for sensitive data access). This is made possible by the ability to use expressions in Windows ACL. For example: You would use MemberOf (Spain_Security_Group) AND MemberOf (Finance_Security_Group) AND MemberOf(Sensitive_Security_Group) to limit access to Spain’s finance department sensitive information."

    Get on over there and give it a read.

    I swear we are going to post some original content here at some point. Just crushed under the load.

    - Ned "sock puppet" Pyle