Move-Mailbox Operations in Exchange 2007 Fail Due to An Attribute "Being Too Long"
Move-Mailbox in Exchange 2007 Fails Due to An Attribute "Being Too Long"
The original scope or reason for this call was that Mailbox-Moves from Exchange 2003 to Exchange 2007
were failing. In this specific case (although it was never performed), mailbox moves would have also
failed if the source mailbox had resided on an Exchange 2007 server and the destination was either
Exchange 2007 or 2003. This is because the Exchange 2007 tools would have to be used to perform the
move.
So, if you are experiencing Move-Mailbox failures when using Exchange 2007 tools (such as the Exchange
Management console or the Exchange Powershell) and the reason being provided is that an "attribute is
too long", I'd like to personally welcome you.
This is definitely your article.
Primary Issue & Symptoms:
The individual I was working with was experiencing problems attempting to migrate Exchange 2003
mailboxes to Exchange 2007. He was receiving sporadic failures during the Move-Mailbox operations.
When the mailboxes failed to move he would receive the following error in Exchange Management Console
and Shell:
_______________________________________________________________________________________________________
(Note: In every example throughout the course of this post, I'm going to focus on Exchange
extensionAttribute2. However, the failure could occur on any attribute that exceeds the
predefined upper limit which will be explained later).
_______________________________________________________________________________________________________
Error:
Custom Attribute2 is too long: maximum length is 1024 and the actual length is "x" (where "x" is
something greater than 1024 characters)
Additionally, running Get-Mailbox in shell "<user name>" for the failed user would result in an error
about the object being corrupted:
Warning: Object domain.com/Users/TestUser has been corrupted and it is in an inconsistent state.
The following validation errors have occurred:
Warning: CustomAttribute2 is too long: maximum length is 1024 and the actual length is 1358 (the
character count on extension attribute2 for my example user).
Troubleshooting:
Prior to calling in the customer had tested whether or not one of these impacted users could be moved to
another Exchange 2003 database and server. The result of that test was that it could be moved successfully.
The issue only manifested itself when moving the mailbox to an Exchange 2007 mail store.
Obviously mailbox moves in Exchange 2007, use an entirely different tool set (EMC, and EMS) than that of
2003/2000.
The first thing I had this fellow do was take an LDP dump of an impacted user and send this to me for review.
A simple review of the LDP dump showed that extensionAttribute2 had a whole lot of characters present on the
user object. To get an exact count, I simply copied the character string in this attribute to Notepad
(disabled Word Wrap) then enabled the Status bar. A single Line in Notepad (with Word Wrap Disabled) can
only hold 1024 characters. So, I simply went to the end of line 2, to get the character count of line 2 and
added that value to 1024 (the character count of line 1).
I did this simply to verify that the Exchange Shell and Management Console were calculating this value
accurately (which they were):
1024 + 334 (character count of line 2) = 1358
Verifying the Limits of Extension Attribute 2 in the Windows Schema
In the Windows Schema there is an object of the attributeSchemaClass for Exchange Extension Attribute 2.
The actual adminDisplayName of this object is: ms-Exch-Extension-Attribute-2.
This object (as well as all other default Exchange related objects and classes) are created during Exchange
2007 /PrepareSchema (or in previousversions of Exchange /ForestPrep).
The minimum and maximum character values for objects of this type are defined by two attributes:
rangeLower (being the minimum)
rangeUpper (being the maximum).
I had this individual provide me with an LDP dump of the ms-Exch-Extension-Attribute-2 object from his
environment for me to review.
This looked like the following (keep in mind this is taken from my own personal lab environment):
Expanding base 'CN=ms-Exch-Extension-Attribute-2,CN=Schema,CN=Configuration,DC=<DOMAIN>,DC=com'...
Result <0>: (null)
Matched DNs:
Getting 1 entries:
>> Dn: CN=ms-Exch-Extension-Attribute-2,CN=Schema,CN=Configuration,DC=<DOMAIN<,DC=com
2> objectClass: top; attributeSchema;
1> cn: ms-Exch-Extension-Attribute-2;
1> distinguishedName: CN=ms-Exch-Extension-Attribute-2,CN=Schema,CN=Configuration,DC=<DOMAIN>,DC=com;
1> instanceType: 0x4 = ( IT_WRITE );
1> whenCreated: 04/03/2002 15:15:43 Pacific Standard Time Pacific Daylight Time;
1> whenChanged: 09/03/2008 17:20:51 Pacific Standard Time Pacific Daylight Time;
1> uSNCreated: 6732;
1> attributeID: 1.2.840.113556.1.2.424;
1> attributeSyntax: 2.5.5.12;
1> isSingleValued: TRUE;
1> rangeLower: 1;
1> rangeUpper: 10240;
1> mAPIID: 32814;
1> uSNChanged: 46036019;
1> showInAdvancedViewOnly: TRUE;
1> adminDisplayName: ms-Exch-Extension-Attribute-2;
1> adminDescription: ms-Exch-Extension-Attribute-2;
1> oMSyntax: 64;
1> searchFlags: 17;
1> lDAPDisplayName: extensionAttribute2;
1> name: ms-Exch-Extension-Attribute-2;
1> objectGUID: bc303dd3-f496-4c60-a5fa-8647a42dd063;
1> schemaIDGUID: bf967969-0de6-11d0-a285-00aa003049e2;
1> attributeSecurityGUID: 1f298a89-de98-47b8-b5cd-572ad53d267e;
1> isMemberOfPartialAttributeSet: TRUE;
1> objectCategory: CN=Attribute-Schema,CN=Schema,CN=Configuration,DC=<DOMAIN>,DC=com;
-----------
So here we see that the rangerLower value is set to 1 (which would be expected as the attribute itself can
only be NOT SET or something > 1), and the rangeUpper value is set to 10240. This rangeUpper value explains
why I can have a character string stored in the AD larger than 1024 (in this case for extensionAttribute2
on a user object) but doesn't explain why the Mailbox Move operation would expect the value to be something
less than or equal to 1024 characters (e.g. 90% less than what is currently defined as the upper bounds on
the schemaClass for the attribute here).
At first glance this looks like a really big bug.
Why would we allow you to store something as large as 10240 characters in the AD and then fail the mailbox
move because we expect this value to be less than 1024 characters?
Time to look at the LDF file responsible for setting this value.
When Exchange setup /prepareschema is run, a series of LDF files are imported to define Classes and valid
attributes for mail enabled objects, etc. Amongst other things it will set valid ranges on some of the
attributes being added or modifed in the Schema. In Exchange 2007 there are 100 of these files within the
installation media named sequentially schema0.ldf - schema 99.ldf (in Exchange 2000/2003 there were 10,
although the file sizes were significantly larger).
That's a lot of files to look through to find the right object. Fortunately I got lucky and
ms-Exch-Extension-Attribute-2 is defined in the first file: schema0.ldf.
If you crack open that file in notepad, and search on the adminDisplayName for the object (in this case:
ms-Exch-Extension-Attribute-2) you find the default values that /PrepareSchema sets for this object:
dn: CN=ms-Exch-Extension-Attribute-2,<SchemaContainerDN>
changetype: add
adminDescription: ms-Exch-Extension-Attribute-2
adminDisplayName: ms-Exch-Extension-Attribute-2
attributeID: 1.2.840.113556.1.2.424
attributeSyntax: 2.5.5.12
isMemberOfPartialAttributeSet: FALSE
isSingleValued: TRUE
lDAPDisplayName: extensionAttribute2
mapiId: 32814
name: ms-Exch-Extension-Attribute-2
oMSyntax: 64
objectCategory: CN=Attribute-Schema,<SchemaContainerDN>
objectClass: attributeSchema
rangeLower: 1
rangeUpper: 1024
schemaIdGuid:: aXmWv+YN0BGihQCqADBJ4g==
searchFlags: 0
Conclusion 1:
Clearly an edit has occured to this object since the time /PrepareSchema was run. The rangeUpper value
/PrepareSchema set was 1024 not 10240.
Customer's rangeUpper Value from LDP Dump: 10240
Schema definition for rangeUpper: 1024
So, somebody or something performed a Schema Edit of this object by appending a 0 to its default value.
____________________________________________________________________________________________________________
Note:
For the record, the only person with permissions to make such a change would be an account with Schema
Admin permissions. So, if you are in a situaton where you need/"have to" figure out who made this change/edit,
you "should have" a relatively short list of people to discuss your concens with. ____________________________________________________________________________________________________________
At this point I was informed by the individual I was working with that there is a certain 3rd Party Password
History/Reset Tool, that uses extensionAttribute2 (although its apparently configurable) to write whatever
values it needs to provide for its additional functionality. Apparently an upper boundary of 1024 characters
was determined to be insufficent. Thus, he had been advised to perform the edit and increase its boundary
to 10240.
To be very, very clear here, Schema Edits/Modifications to the default Exchange and Windows objects have
never been supported. The Reason: It is entirely possible that functionality could be added/removed that
is dependent up the values we set/expect to be present (as in this case).
If you choose to go down this path, your risks include (but aren't necessairly limited to):
1. Your Exchange/Windows implementation not working as expected.
2. Being unsupported.
So at this point I think a summary is in order:
1. Mailbox Moves for specific users destined for Exchange 2007 fail when a value larger than 1024 on
extensionAttribute2 is encountered.
2. These failures only occur when Exchange 2007 Management tools are used.
3. An unsupported Schema Edit has occurred against the SchemaClass object (in this case:
ms-Exch-Extension-Attribute-2).
4. Similar problems are encountered when running Get-Mailbox for a user. The response is that the object
is actually corrupted.
Troubleshooting (Continued):
At this point, I recommended that we perform a test of removing some characters from extensionAttribute2 (to
get under 1024 charaters) for a single user, then re-trying the move. Prior to doing so, I ensured we had a
current and valid LDP dump saved for our user prior to removing any characters off the user. Once done, I
removed 335 characters off extensionAttribute2 for our guinea pig (e.g. "Test User"). This left 1023 characters
remaining on the user object.
It should go without saying that you don't just want to start removing random characters. I personally removed
the first 335 characters (for simplicity), but removing the last 335 would have been just as simple
(so use your creativity here).
I then forced Domain Controller synchronization within the domain where our "Test User" existed.
After verifying the change had replicated (used ADSI Edit and bound to several different DCs within the domain
to verify the change), we retried the move.
The Result: SUCCESS. Guinea Pig moved.
Conclusion 2:
We have our first work around. In this case extensionAttribute2 is expected to be < 1024. Reducing the characters
on extensionAttribute2 to something below the expected rangeUpper value (as defined by the Exchange Schema update)
allows the move to complete successfully.
It does add an additional layer of Administrative complexity (e.g. the users have to be defined and located, their
original values saved, characters removed from the production object to get under the limit, the move performed,
and once completed added back). But once done, the move will complete successfully.
Putting the Pieces Together: New Validation Checks in Exchange 2007 Move-Mailbox:
There are a host of articles and separate blog posts which specifically address this subject, so I'm not going to
go into considerable detail here. However what I will say is that Exchange 2007 Move-Mailbox has added a number
of new pre-move validation checks (these really didn't exist in Exchange 2003/2000/5.5). This was done in effort to
try and reduce situations where partially provisioned accounts/mailboxes would exist as the result of a partial move.
That is where to say from the "Exchange side" a mailbox move might have actually completed (e.g. the items were
copied out of the source mailbox, a new mailbox was created on the target, and said items placed into the target)
but some sort of other failure occurred during the update of the user's attributes (usually near the end of the move).
So, the net result of the failure might be the user having msExchHomeServer set to the original source server, but
homeMDB might be pointed to a database on the target server. The net result is that it is invalid, and now the user
can no longer logon. Anyone that has had to deal with these sorts of problems knows how tedious they can be to
resolve (in my line of work, I've worked way to many of these issues to ever want to have to do it again).
What's important with respect to the scope of this issue is to first know that these checks exist.
The "technical stuff" is that within Move-Mailbox code there is a hard-coded validation lookup table. A full
discussion of this is well outside the scope of this post, but let's just say that there is a global variable
defined for extensionAttribute2 and its rangeUpper value is defined as 1024 characters. Obvisously
extensionAttribute2 is one of many different checks that are performed against the user pre-move, so if you make
a modification to any other object to allow for a larger upper boundary, you run the distinct possibility that
the pre-validation check will fail and the mailbox move cancelled.
The important distinction and thing to remember here is that rangeUpper is hard coded within the validation lookup
table. Move-Mailbox will not make an LDAP call into the Active Directory Schema to determine what current rangeUpper
value has been defined for the attribute class. So even though the Edit in and of itself is technically sound,
the mailbox move will fail if it has more than 1024 characters.
Conclusion 3:
Unfortunately at this point the individual I was assisting was in an unsupported state. Which essentially left 3
hypothetical options (If the first work around was not acceptable):
1. Find a way to circumvent or disable the Move-Mailbox validation check.
2. Request a Development Change Request (DCR), to search the Schema as opposed to hard coding the rangeUpper
value in code.
3. Locate another attribute with sufficent size to store the values currently residing on extensionAttribute2.
Option 1: Not possible. There is no way to disable the validation lookup check.
Option 2: Has been requested before and has been denied by the Product Team because we do not support edits to the
Default Exchange Schema.
Which essentially leaves Option 3.
Exchange Extension Attributes have existed for a long time (I believe they were even present back in the Exchange
4.0 days, but don't quote me on that -- I know they were present in 5.5). They were added to provide for some
general flexibiliy for storing customer specific data (a good example of this might be a repository to store an
IP Phone Number, because no specific IP Phone attribute existed back at that time.). They have also been used
by Microsoft for very specific purposes such as migration tools (anybody remember NTDSNoMatch/NTDSAttrib? -- I
really miss the ADC it kept me busy for years).
Under closer inspection you will find that extension attributes 1-10 have an upper boundary set to 1024 characters.
However, 11-15 are double in size (2048 characters).
Here is a complete list with their corresponding values (taken from the SCHEMA*.LDF files):
ms-Exch-Extension-Attribute-1 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-2 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-3 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-4 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-5 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-6 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-7 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-8 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-9 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-10 -- rangeUpper: 1024
ms-Exch-Extension-Attribute-11 -- rangeUpper: 2048
ms-Exch-Extension-Attribute-12 -- rangeUpper: 2048
ms-Exch-Extension-Attribute-13 -- rangeUpper: 2048
ms-Exch-Extension-Attribute-14 -- rangeUpper: 2048
ms-Exch-Extension-Attribute-15 -- rangeUpper: 2048
In the example I have been using througout this post, the current character size for "Test User' on
extensionAttribute2 has been 1358. That value is considerably less than 2048, so if its value were stored to say
extensionAttribute11, there would be no issue and the mailbox move would complete successfully. This is
potentially valid work-around 2.
However, if you cannot guarantee that values will always be less than 2048 (if using extension11-15) then the use
of an Exchange Extension Attribute is not a wise choice (to be clear you will be able to store them in the AD after
the edit (if done correctly) -- but anytime you need to perform a Mailbox-Move or Get-Mailbox procedure against the
user it will fail validation. This of course means you need to go with work-around 1.
In Summary:
As previously discussed Schema Edits of Default Exchange and Windows objects/attributes have never been supported
and this issue is a perfect example of why. These checks were put into place in an effort to help prevent partially
moved/provisioned mailboxes as well as a host of other scenarios to lengthy to discuss here. At the end of the day
the best overall solution to this problem would be for these 3rd party vendors to create their own unique schema
classes that "extend" (extend being the operative word here) the default User object class provided by Windows.
This is fully supported. Once done they could write whatever values and sizes they want to an "intentional" attribute
for whatever purpose they want. Because these attributes would be unique to their applications, they would never,
(will never say never) be checked in some sort of validation code (in this case move-mailbox).
A general outline for how this might be done:
1. Extend the Schema:
https://technet.microsoft.com/en-us/magazine/2008.05.schema.aspx?pr=blog
2. Extend the User Class in the AD Schema:
https://www.windowsitpro.com/Web/Article/ArticleID/9738/9738.html
3. Contact the IANA to acquire valid OIDs:
https://msdn.microsoft.com/en-us/library/ms677621.aspx
https://pen.iana.org/pen/PenApplication.page
I hope you find this interesting/useful.
Happy Trails.
Eric