Friday Mail Sack: Mothers day pfffft… when is son’s day?

Friday Mail Sack: Mothers day pfffft… when is son’s day?

  • Comments 7
  • Likes

Hi folks, Ned here again. It’s been a little while since the last sack, but I have a good excuse: I just finished writing a poop ton of Windows Server 2012 depth training that our support folks around the world will use to make your lives easier (someday). If I ever open MS Word again it will be too soon, and I’ll probably say the same thing about PowerPoint by June.

Anyhoo, let’s get to it. This week we talk about:

Question

Is it possible to use any ActiveDirectory module cmdlets through invoke-command against a remote non-Windows Server 2012 DC where the module is installed? It always blows up for me as it tries to “locally” (remotely) use the non-existent ADWS with error “Unable to contact the server. This may be because the server does not exist, it is currently down, or it does not have the active directory web services running”

image

Answer

Yes, but you have to ignore that terribly misleading error and put your thinking cap on: the problem is your credentials. When you invoke-command, you make the remote server run the local PowerShell on your behalf. In this case that remote command has to go off-box to yet another remote server – a DC running ADWS. This means a multi-hop credential scenario. Provide –credential (get-credential) to your called cmdlets inside the curly braces and it’ll work fine.

Question

We are using a USMT /hardlink migration to preserve disk space and increase performance. However, performance is crazy slow and we’re actually running out of disk space on some machines that have very large files like PSTs. My scanstate log shows:

Error [0x000000] Write error 112 for C:\users\ned\Desktop [somebig.pst]. Windows error 112 description: There is not enough space on the disk.[gle=0x00000070]

Error [0x080000] Error 2147942512 while gathering object C:\users\ned\Desktop\somebig.pst. Shell application requested abort![gle=0x00000070]

Answer

These files are encrypted and you are using /efs:copyraw instead of /efs:hardlink. Encrypted files are copied into the store whole instead of hardlink'ing, unless you specify /efs:hardlink. If you had not included /efs, this file would have failed with, "File X is encrypted. Use the /efs option to specify a different way to handle this file".

Yes, I realize that we should probably just require that option. But think of all the billable hours we just gave you!

Question

I was using your DFSR pre-seeding post and am finding that robocopy /B is slows down my migration compared to not using it. Is that required for preseeding?

Answer

The /B mode, while inherently slower, ensures that files are copied using a backup API regardless of permissions. It is the safest way, so I took the prudent route when I wrote the sample command. It’s definitely expected to be slower – in my semi-scientific repro’s the difference was ~1.75 times slower on average.

However, /B not required if you are 100% sure you have at least READ permissions to all files.  The downside here is a lot of failures due to permissions might end up making things even slower than just going /B; you will have to test it.

If you are using Windows Server 2012 and have plenty of hardware to back it up, you can use the following options that really make the robocopy fly, at the cost of memory, CPU, and network utilization (and possibly, some files not copying at all):

Robocopy <foo> <bar> /e /j /copyall /xd dfsrprivate /log:<sna.foo> /tee /t:128 /r:1

For those that have used this before, it will look pretty similar – but note:

  • Adds /J option (first introduced in Win8 robocopy) - now performs unbuffered IO, which means gigantic files like ISO and VHD really fly and a 1Gbps network is finally heavily utilized. Adds significant memory overhead, naturally.
  • Add /MT:128 to use 128 simultaneous file copy threads. Adds CPU overhead, naturally.
  • Removes /B and /R:6 in order to guarantee fastest copy method. Make sure you review the log and recopy any failures individually, as you are now skipping any files that failed to copy on the first try.

 

Question

Recently I came across an user account that keeps locking out (yes, I've read several of your blogs where you say account lockout policies are bad "Turning on account lockouts is a way to guarantee someone with no credentials can deny service to your entire domain"). We get the Event ID of 4740 saying the account has been locked out, but the calling computer name is blank:

 

Log Name:     Security

 

Event ID:     4740

 

Level:         Information

 

Description:

 

A user account was locked out.

 

Subject:

 

Security ID: SYSTEM

 

Account Name: someaccount

 

Account Domain: somedomain

 

Logon ID: 0x3e7

 

Account That Was Locked Out:

 

Security ID: somesid

 

Account Name: someguy

 

Additional Information:

 

Caller Computer Name:

 

The 0xC000006A indicates a bad password attempt. This happens every 5 minutes and eventually results in the account being locked out. We can see that the bad password attempts are coming via COMP1 (which is a proxy server) however we can't work out what is sending the requests to COMP1 as the computer is blank again (there should be a computer name).

Are we missing something here? Is there something else we could be doing to track this down? Is the calling computer name being blank indicative of some other problem or just perhaps means the calling device is a non-Microsoft device?

Answer

(I am going to channel my inner Eric here):

A blank computer name is not unexpected, unfortunately. The audit system relies on the sending computers to provide that information as part of the actual authentication attempt. Kerberos does not have a reliable way to provide the remote computer info in many cases. Name resolution info about a sending computer is also easily spoofed. This is especially true with transitive NTLM logons, where we are relying on one computer to provide info for another computer. NTLM provides names but they are also easily spoofed so even when you see a computer name in auditing, you are mainly asking an honest person to tell you the truth.

Since it happens very frequently and predictably, I’d configure a network capture on the sending server to run in a circular fashion, then wait for the lock out and stop the event. You’d see all of the traffic and now know exactly who sent it. If the lockout was longer running and less predictable, I’d recommend using a network capture configured to trace in a circular fashion until that 4740 event writes. Then you can see what the sending IP address is and hunt down that machine. Different techniques here:

[And the customer later noted that since it’s a proxy server, it has lots of logs – and they told him the offender]

Question

I am testing USMT 5.0 and finding that if I migrate certain Windows 7 computers to Windows 8 Consumer Preview, Modern Apps won’t start. Some have errors, some just start then go away.

Answer

Argh. The problem here is Windows 7’s built-in manifest that implements microsoft-windows-com-base , which then copies this registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\OLE

If the DCOM permissions are modified in that key, they migrate over and interfere with the ones needed by Modern Apps to run. This is a known issue and already fixed so that we don’t copy those values onto Windows 8 anymore. It was never a good idea in the first place, as any applications needing special permissions will just set their own anyways when installed.

And it’s burned us in the past too…

Question

Are there any available PowerShell, WMI, or command-line options for configuring an OCSP responder? I know that I can install the feature with the Add-WindowsFeature, but I'd like to script configuring the responder and creating the array.

Answer

[Courtesy of the Jonathan “oh no, feet!” Stephens – Ned]

There are currently no command line tools or dedicated PowerShell cmdlets available to perform management tasks on the Online Responder. You can, however, use the COM interfaces IOCSPAdmin and IOSCPCAConfiguration to manage the revocation providers on the Online Responder.

  1. Create an IOSCPAdmin object.
  2. The IOSCPAdmin::OCSPCAConfigurationCollection property will return an IOCSPCAConfigurationCollection object.
  3. Use IOCSPCAConfigurationCollection::CreateCAConfiguration to create a new revocation provider.
  4. Make sure you call IOCSPAdmin::SetConfiguration when finished so the online responder gets updated with the new revocation configuration.

Because these are COM interfaces, you can call them from VBScript or PowerShell, so you have great flexibility in how you write your script.

Question

I want to use Windows Desktop Search with DFS Namespaces but according to this TechNet Forum thread it’s not possible to add remote indexes on namespaces. What say you?

Answer

There is no DFSN+WDS remote index integration in any OS, including Windows 8 Consumer Preview. At its heart, this comes down to being a massive architectural change in WDS that just hasn’t gotten traction. You can still point to the targets as remote indexes, naturally.

Question

Certain files – as pointed out here by AlexSemi – that end with invalid characters like a dot or a space break USMT migration. One way to create these files is to use the echo command into a device path like so:

image

These files can’t be opened by anything in Windows, it seems.

image

When you try to migrate, you end up with a fatal “windows error 2” “the system cannot find the file specified” error unless you skip the files using /C:

image

What gives?

Answer

Quit making invalid files! :-)

USMT didn’t invent CreateFile() so its options here are rather limited… USMT 5.0 handles this case correctly through error control - it skips these files when hardlink’ing because Windows returns that they “don’t exist”. Here is my scanstate log using USMT 5.0 beta, where I used /hardlink and did NOT provide /C:

image

In the case of non-hardlink, scanstate copies them without their invalid names and they become non-dotted/non-spaced valid files (even in USMT 4.0). To make it copy these invalid files with the actual invalid name would require a complete re-architecting of USMT or the Win32 file APIs. And why – so that everyone could continue to not open them?

Other Stuff

In case you missed it, Windows 8 Enterprise Edition details. With all the new licensing and activation goodness, Enterprise versions are finally within reach of any size customer. Yes, that means you!

Very solid Mother’s Day TV mash up (a little sweary, but you can’t fight a something that combines The Wire, 30 Rock, and The Cosbys)

Zombie mall experience. I have to fly to Reading in June to teach… this might be on the agenda

Well, it’s about time - Congress doesn't "like" it when employers ask for Facebook login details

Your mother is not this awesome:

image
That, my friend, is a Skyrim birthday cake

SportsCenter wins again (thanks Mark!)

Don’t miss the latest Between Two Ferns (veeerrrry sweary, but Zach Galifianakis at his best; I just wish they’d add the Tina Fey episode)

But what happens if you eat it before you read the survival tips, Land Rover?!

 

Until next time,

- Ned “demon spawn” Pyle

  • Going to be honest - the CDL/WindowsRT VDA licenses just make no sense to me.  Why is desktop licensing so complex?  What does it matter which device you connect from?

  • Ned,

    Had a question about the robocopy /b, we've come across scenarios where /b didn't work at all, nor did /zb. Turns out while we may have read at the top, on occasion deeper folders we don't have access to. We tried several different scenario's in an attempt to migrate user data, finally we found a method of enabling all the privledges to basically let us run code as SYSTEM.

    msdn.microsoft.com/.../bb530716(VS.85).aspx

    we use the seBackupPriv and this allows us to successfully run a /b migration.

  • That would be expected behavior if your account did not have Backup and Restore privileges, Jeffrey. You can confirm that with the WHOAMI /PRIV command. If they are not members of the Administrators or Backup Operator groups, they won't have those privileges by default. If they are members and still don't have the privileges, someone has changed the defaults (using local or domain-based security policy, most likely).

  • Yes, I believe that is a hole we dug ourselves, Ned. Being the local admin and in backup operators didn't work, using the link I posted earlier was the only way to make it go.

    We figured this out the hard way...but since I think we're not alone in scripting user folders, I thought this may help some folks out there reading your blog.

  • Hi Ned,

    I came across the DFS and WDS issue some years ago and was sorely disappointed.  I hope this is fixed in 2012/Win8?  I work as a tech architect with many large corporates and lots of them are using DFS-N.  Do you guys not use DFS in Microsoft?  It seems you don't use GPO folder redirection either (judging by the assumptions in PowerShell's configuration).

    Please: test stuff with redirected folders to DFS paths!!!

    Thanks

    Chris

  • We use DFS more than anyone else on Earth, by far. Who do you think complains about this lack the most? We do, internally. As I mentioned previously, this capability is not in Windows 8/2012 either.

    You'll have to be more specific with your other point about PowerShell.

  • I also don't know what your last point means, please explain that as well.