Tim McMichael

Navigating the world of high availability...

Exchange 2010 – File Share Witness oddities…

Exchange 2010 – File Share Witness oddities…

  • Comments 15
  • Likes

In Exchange 2010 when a Database Availability Group (DAG) it utilized, and there is an even number of DAG members, the underlying cluster is implemented utilizing the quorum type Node and File Share Majority.  The settings utilized for the File Share Witness are defined on the DAG when the logical DAG object is created and are either set by the administrator or automatically defined.

To verify the quorum type you can use either cluster.exe or cluster powershell extensions (Preferred)

Cluster.exe <cluster> /quorum  (Windows 2008 & Windows 2008 R2)

Cluster.exe cluster.domain.com /quorum

Witness Resource Name Path                                          Type

--------------------- --------------------------------------------- --------

File Share Witness (\\HT-1.DOMAIN.COM\DAG.DOMAIN.COM)               Majority

Get-Cluster <cluster> | Get-ClusterQuorum | FL (Windows 2008 R2 Only)

Cluster        : DAG
QuorumResource : File Share Witness (
\\HT-1.DOMAIN.COM\DAG.DOMAIN.COM)
QuorumType     : NodeAndFileShareMajority

In Failover Cluster Manager, the resources can be viewed by looking at the Cluster Core Resources.

image

It may become necessary to change the server hosting the file share witness.  In Exchange 2010 this is not done utilizing Failover Cluster Manager, but rather utilizing the set-databaseavailabilitygroup commandlet.  It is after the witness server is successfully updated that the oddity occurs.  Here’s an example:

Currently the DAG utilizes the witness server HT-1.  Using the set-databaseavailabilitygroup command the witness server is changed to HT-2.  (set-databaseavailabilitygroupserver –witnessServer HT-2)  The command returns without error.  When running the previous cluster commands the following output is noted:

Cluster.exe cluster.domain.com /quorum (Windows 2008 and Windows 2008 R2)

Witness Resource Name Path                                          Type

--------------------- --------------------------------------------- --------

File Share Witness (\\HT-1.DOMAIN.COM\DAG.DOMAIN.COM)               Majority

Get-Cluster <cluster> | Get-ClusterQuorum | FL (Windows 2008 R2 Only)

Cluster        : DAG
QuorumResource : File Share Witness (
\\HT-1.DOMAIN.COM\DAG.DOMAIN.COM)
QuorumType     : NodeAndFileShareMajority

Also in Failover Cluster Manager the following is noted in the cluster core resources group.

image

After looking at this output the administrator could be lead to believe that the witness server did not successfully update.  After all both cluster.exe and powershell both show the File Share Witness (\\HT-1.DOMAIN.COM\DAG.DOMAIN.COM).  It is only in Failover Cluster Manager, if the windows is fully expanded, that you can see both (\\HT-1.DOMAIN.COM\DAG.DOMAIN.COM) and (\\HT-2.DOMAIN.COM\DAG.DOMAIN.COM).  This leads administrators to believe that two file share witness servers are currently in use.

Thankfully both of these perceived conditions are false.  The command was both successful in changing the witness server and only one file share witness is in use.

Each cluster resource has a display name and a set of public and private properties.  Unfortunately when using set-databaseavailabilitygroup to change the witness server, the File Share Witness resource private property for where the witness is stored is updated but the public property display name, which contains the previous witness server, is not.  Let’s take a look at this further.

Using cluster.exe or powershell I can review the private properties of the File Share Witness resource.  (Command output truncated to show relevant values only.)

Cluster.exe <cluster> res <resource> /priv <or> /prop (Windows 2008 & Windows 2008 R2)

Cluster.exe cluster.domain.com res “File Share Witness (\\HT-1.domain.com\DAG.domain.com)" /prop

Listing properties for 'File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)':

T  Resource             Name                           Value

-- -------------------- ------------------------------ -----------------------

SR File Share Witness (\\HT-1.domain.COM\DAG.domain.COM) Name                           File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)

Cluster.exe cluster.domain.com res “File Share Witness (\\HT-1.domain.com\DAG.domain.com)" /priv

Listing private properties for 'File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)':

T  Resource             Name                           Value

-- -------------------- ------------------------------ -----------------------

S  File Share Witness (\\HT-1.domain.COM\DAG.domain.COM) SharePath                      \\HT-1.domain.com\DAG.domain.com

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | fl (Windows 2008 R2 Only – Public Properties)

Name         : File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)
State        : Online
OwnerGroup   : Cluster Group
ResourceType : File Share Witness

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | Get-ClusterParameter fl (Windows 2008 R2 Only – Private Properties)

Name          : SharePath
IsReadOnly    : False
ParameterType : String
Value         : \\HT-1.domain.com\DAG.domain.com

At this time a set-databaseavailability group is issued to change the witness server.  After the command completes successfully, the previous commands are run.  (Command output truncated to show relevant values only.)

Cluster.exe cluster.domain.com res “File Share Witness (\\HT-1.domain.com\DAG.domain.com)" /prop

Listing properties for 'File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)':

T  Resource             Name                           Value

-- -------------------- ------------------------------ -----------------------

SR File Share Witness (\\HT-1.domain.COM\DAG.domain.COM) Name                           File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)

Cluster.exe cluster.domain.com res “File Share Witness (\\HT-1.domain.com\DAG.domain.com)" /priv

Listing private properties for 'File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)':

T  Resource             Name                           Value

-- -------------------- ------------------------------ -----------------------

S  File Share Witness (\\HT-1.domain.COM\DAG.domain.COM) SharePath                      \\HT-2.domain.com\DAG.domain.com

(Note:  The SharePath in the previous output reflects the new witness server as expected)

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | fl (Windows 2008 R2 Only – Public Properties)

Name         : File Share Witness (\\HT-1.domain.COM\DAG.domain.COM)
State        : Online
OwnerGroup   : Cluster Group
ResourceType : File Share Witness

Get-ClusterResource –Cluster <cluster> –Name <ResourceName> | Get-ClusterParameter fl (Windows 2008 R2 Only – Private Properties)

Name          : SharePath
IsReadOnly    : False
ParameterType : String
Value         : \\HT-2.domain.com\DAG.domain.com

(Note:  The SharePath in the previous output reflects the new witness server as expected)

As you can see the set-databaseavailability group command did complete it’s task successfully by updating the SharePath attribute of the quorum resource to utilize the correct witness server.

Comments
  • Can the name displayed by "Failover Cluster Manager / Cluster Core Resources" be manually updated without issue?

  • @Neil:

    Yes - you can change this via Failover Cluster Manager / shell / command line.

    TIMMCMIC

  • Tim,

    We have a DAG with 7 nodes and the cluster is currently in a Node Majority configuration. We are migrating off of our current witness server and decommissioning it, but we need to change the witness server first so there are no references to the decommissioned server. We understand that in an odd numbered cluster the witness server really isn't in use, but we want to maintain the witness server settings just in case we remove a server from the DAG, which would then activate the witness server. Are there any other necessary steps that we need to take before changing the witness server configuration using the recommended set-databaseavailabilitygroupserver –witnessServer command?

    Thanks for any help you can provide!

  • Nevermind, we ran some tests in our test lab and figured it out. We were able to run the Set-DatabaseAvailabilityGroup -WitnessServer command without a problem while the cluster was in a Node Majority configuration. We then removed one of the members of the DAG and verified that the cluster changed to a Node and File Share Quorum. It worked beautifully!

    Thanks!

  • Excellent - that was the exact solution that I would have provided.

    TIMMCMIC

  • Jason,

    we have a DAG at now with 2 members and we're using NLB for CAS/HUB so i have my FSW in one of my CAS/HUB server, my question is what will happen if this server fail? can i have a FSW in another server in case the first one fail?

  • @Jason:

    My apologies for the delayed reply - I was offline.

    You do not pre-stage a secondary witness for witness failures like this.  In the event the server hosting the witness is offline / down you can simply run the set-databaseavailabilitygroup command and move the witness to a server that is still running.

    TIMMCMIC

  • @JasonK:

    Excellent - I'm glad you found the command you needed.

    TIm

  • What is your recommended way of updating the name property?  The GUI seems like overkill - you have to re-run the quorum wizard.  Can you provide a sample shell or command line?

  • @Graham:

    I recommend configurating hte cluster for node majority and then running set-databaseavailabilitygroup which will convert the DAG back to the correct quroum type.

    TIMMCMIC

  • Tim,

    Is the file share witness not a shared folder? I see in the actual location that it is not a shared folder.

  • @Ayyappan...

    It should be a shared folder.

    TIMMCMIC

  • Hello all,

    Can some on tell me how to change the witness server with the Set-Databaseavailability?

    I tried to change the name but when i use Set-DatabaseAvailabilityGroup -Identity DAG01 -WitnessServer commserve01.corp.loc -WitnessDirector

    y d:\FSW_DAG01 the FSW name stays the same.

    Greats,

    Menno

  • We have an Exchange 2010 (Ent Server 2008 R2) DAG running 16 nodes with two physical data centers containing 8 Exchange servers each. One physical site maintains all of our "1st preference" DAG members and we keep the primary File Share Witness at this same data center on a public folder server. We found recently during a network outage that our servers would not automatically fail over to the other physical site as the FSW was apparently "locked" and could not be accessed by the servers at the other site over the network. To provide some redundancy, is there a way to place the FSW in a third physical location (accessible by both data centers) such that it would provide FSW functionality but not allow one site that disappears from the network to lock it? We were able to manually force the FSW over to the other side after some work but would like this to be automated if possible.   Thanks for any suggestions

  • @Rick:

    Sorry for the delay in getting back to you.  I believe what you experienced is correct.  If the nodes + witness are in the primary and the WAN is lost for any reason the other nodes in the DR datacenter would loose quorum and therefore not be able to mount databases.

    You could place a witness in a third site - but I'm not sure what this would fix for you.  If the witness was in the third site and the primary site was legitimately down then the third site witness would provide the necessary vote for quorum.  Although in the situation you described if the primary was down then you would go through a site level activation leveraging the alternate file share witness.

    The third site witness also has several caveats to it which makes it not reliable in some installations.  

    TIMMCMIC

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment