Recently I have been gathering a lot of mailbox data for a customer running Exchange 2010 that is in the process of upgrading from Exchange 2007.  One of the more frequent requests has been for mailbox numbers, often broken out by database.  It’s key to know the current state of your environment, especially during an upgrade where mailboxes seem to never stop moving.  For my customer this is how we track which databases are full and keep tabs on the current mailbox load balancing procedure.

Typically you can just run a command similar to ‘Get-Mailbox –ResultSize Unlimited | group database’ to quickly see how the databases look.  However, when you have an environment with tens/hundreds of thousands of mailboxes, the Get-Mailbox command will take a long time to finish.  Sometimes it’s just not fast enough, and I saw this as a great opportunity to see just how fast I could gather a count of mailboxes per database.  After testing a few different methods (including –asjob, start-job, etc) I found the fastest method was LDAP interrogation.

The idea is pretty straight forward as every AD user with an Exchange mailbox will have a homeMDB attribute pointed to the mailbox’s database.  First, let’s pull back a list of databases and retrieve the distinguishedName of the database, then perform an LDAP query for that DN in the homeMDB attribute of the user accounts.  This is a simple example but could be adapted to different environments.  For example, in my lab I tag ‘SLA-1’ into the ‘extensionAttribute1’ attribute for all my Tier 1 SLA users.  I could modify this script to include ‘extensionAttribute1=SLA-1’ in my LDAP filter to make sure these users are all on DB1 which has more database copies than my other databases.

Disclaimer: By no means do I think this is ‘better’ than Get-Mailbox or other native commands.  In some cases, specifically when pulling a small amount of data from a large set of users, this method is simply faster.

Script Execution

Enough talk, let’s see the script run in a lab environment.  Between each test, I reboot the DC to flush any cached LDAP results.  Here are the details of my virtualized lab:
One Domain Controller with dynamic memory up to 3GB, 2 virtual CPU.
One Exchange 2010 SP3 (RU3) server with 4GB RAM, 2 virtual CPU.
All disks share the same set of Raid 0 disks.
~20,000 mailboxes
20,000 mail-enabled users
20 mailbox databases

As a baseline, let’s execute a typical ‘Get-Mailbox’ cmdlet.

Here we see the cmdlet finish in 3.9 minutes – not bad for 20,000 mailboxes.  But we can go faster!  Here’s the script in action:

Once it’s complete, we see that it took 16.9 seconds to run, That’s over 14x faster!

The script can be found on the Script Repository here.

Stay tuned for more PowerShell/automation scripts and tips…

Over the years, I’ve noticed that I develop a general routine depending on the needs of my current customer.  When I get to work in the morning I generally check a few items that I know can develop into hot button issues.  As a general rule, if I am going to repeat a task more than a few times, I write a quick script in PowerShell if possible.  Lately I’ve had to search for and modify several tens of thousands of objects in AD.  The majority of my scripts display a progress bar.  The progress bar is nice, but I wanted to improve the progress bar by adding an active countdown timer.

When I set out to write this, I wanted a function I can just drop into a script if I feel it’s necessary, with minimal code changes.  The function I came up with makes the following assumptions:

• Each processing task takes relatively the same amount of time to execute
• You must know how many items you are processing.  I’m not sure mathematically how to solve this without knowing both how many items we have looped through and how many more we have left to process
• If one or several of the processing tasks takes significantly longer than the other processing tasks (such as a network timeout), the estimated time remaining will be skewed

After adding the function I created to a script, all I need to add is this code to my foreach loop (where $cycles is an integer representing the total objects to process):

$i++
ShowExecutionTime -TotalObjectsToProcess $Cycles

Here is the script in action:

The script in the repository also has a –ShowCalculatedTimes parameter which will document each time the function estimates the script completion time, then will display how many times we guessed each specific completion time.  Below are the results of the script with –ShowCalculatedTimes enabled:

The sample script containing the function can be found here.

This month’s post will be a short one – and there’s not much code to this topic, it’s just an observation.  When I write scripts, I like the idea of adding some output to track what my script is doing, especially if there’s an issue with the script.  It’s so much easier to find bugs or performance issues if I can see the last task that completed successfully – I then know where to start looking for the issue in my code.  However, it is important to understand as more output is added, your script will slow down.  It may not seem like much but when you are calling write-host multiple times for thousands of objects, overall script performance will be impacted.

Let’s take an example script I wrote and assume the script will call a function in a foreach loop processing thousands of objects.  I’ll also add a write-host in my foreach loop each time I enter and exit a function.  That’s a lot of text but it could be meaningful if there is an issue.  Let’s run the following code:

function foo {
    Write-Host "Entering function 'foo'"
    # Some code
    Write-Host "Completed function 'foo'"
}

$startTime = Get-Date
write-host "script started at $starttime"
$Cycles = 10000

1..$Cycles | % {
    foo
}
Write-Host "Done"

$endTime = Get-Date
($endTime - $startTime).totalseconds

This takes about 10 seconds to execute on my computer, which is significant since the only thing running is the write-host command.  After commenting out the write-host commands, the script completes in about 1 second.  In my personal experience, I have seen scripts where over 30 minutes have been saved by removing write-host output.

My suggestion and what I do for most of my scripts: as an alternative to write-host, you can use write-verbose for output, and when you need to see what your script is doing you can call the script with the –verbose parameter.  Your script will execute faster under normal circumstances, and you will still be able to output diagnostic information when you need it - with no code changes.

In the first section of this two part series, I will discuss caching objects to disk in PowerShell, and reading the cache back into memory.  I recently came up with this solution at a customer site where I was running a script as a scheduled task and needed some data to persist after the task was complete.  I use this to track different errors and how many times I’ve encountered those errors; as well as caching data from complex (read: expensive) LDAP queries.

So let’s take a look at something LDAP we could cache.  Let’s say I have an array of objects in memory representing Exchange databases with current number of mailboxes (I like to query LDAP for this data - see this post).  The objects could be anything – but the concept is straightforward as we will cache the data to disk using XML.  For this example, let’s create some fake data with custom PSObjects.  I chose to use different types (string, int, datetime) on purpose here to show how this really shines compared to using something like export/import –csv, which only recognizes strings without doing type casting or conversion.

$DBs = @()
$DBs += New-Object psobject -Property @{DatabaseName="Database-1"; UserCount = 3800; Timestamp=(Get-Date)}
$DBs += New-Object psobject -Property @{DatabaseName="Database-2"; UserCount = 2600; Timestamp=(Get-Date)}

Once the array of custom objects is created, we can simply export it using the built in Export-Clixml cmdlet.

$DBs | Export-Clixml .\DatabaseCache.xml

This will save all of our data to an XML file and retain associated type information, etc.  Here is what the raw XML data looks like.

To use this data again, simply use the Import-Clixml cmdlet and you will have the exact same array of objects you previously exported.  Stay tuned as my next post will show some examples of how to update the cache.

In my previous post about this subject, I demonstrated how to cache objects using an XML file. Let’s take a look at how to load, manipulate and save the data again.

Since the Export-Clixml cmdlet retains the data type in the XML file, we can perform actions on the objects in memory that are type-specific without having to cast the objects.  This means if we have an integer value, we can increment the count by calling the ++ operator.  Likewise, if we have a DateTime object, we can call type-specific methods such as .ToUniversalTime().  I think this is very cool and I wish I’d discovered this sooner. 

Loading the XML back into memory couldn’t be easier:

$DBCache = Import-Clixml C:\DatabaseCache.xml

Using the data created in part 1, let’s assume we just created a new mailbox on Database-1.  With the below code, we can easily take the objects in memory and increment the count to show the new updated mailbox number.  Another example would be adding time to the Timestamp attribute before we save .

($DBCache | where {$_.DatabaseName –eq “Database-1”}).UserCount++

foreach ($DBItem in $DBCache) { $DBItem.Timestamp.AddHours(3) }

Once the objects are modified, simply overwrite the XML file with the updated data:

$DBCache | Export-Clixml “C:\DatabaseCache.xml”

I realize this is a very simple example but the principal applies to any class and any of those class methods.