Doug Deitterick's Blog

Issue with Accessing CMS on Paired Standard Edition Server

2013-04-29T02:54:00Z

I came across this issue while testing paired pool fail-over. The issue is dependent on a couple of things, namely how you configure your base OS server build and when you paired the Standard Edition Servers. For me this became an issue when I performed the following steps in this order:

Created/Installed two Lync Server 2013 Standard Edition Servers that aren't paired together.
Ran the Install-CsDatabase -CentralManagementStore cmdlet on the second Standard Edition Server.
Setup pool pairing in Topology Builder and published the new topology from the second Standard Edition Server.
Ran Step 2 from the Deployment Wizard on both Standard Edition Servers to install the Lync Server Backup Service.
Followed the steps in the Managing Lync Server 2013 Disaster Recovery, High Availability, and Backup Service TechNet article to fail-over to the second Standard Edition Server.
Tied to access the CMS from another Lync Server or bring the first Standard Edition Server back online and try to access the CMS.

With the CMS failed over to the second Standard Edition Server, I was unable to access the CMS from any server except the second Standard Edition Server. Trying to download the topology in Topology Builder on the first Standard Edition Server returned the following:

Running Get-CsBackupServiceStatus on the first Standard Edition Server to check the state of the Lync Server Backup Service returned the following:

However, running the same cmdlets on the second Standard Edition Server returned different results:

Also, the second Standard Edition Server was logging the following in the Lync Server Event Log every 2 minutes:

Log Name:      Lync Server
Source:        LS Backup Service
Date:          4/28/2013 11:21:01 AM
Event ID:      4090
Task Category: (4000)
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      TEST-LS15-SE2.test.deitterick.com
Description:
Microsoft Lync Server 2013, Backup Service central management backup module performs a full sync export.

At the same time the first Standard Edition Server was logging the following errors in the Lync Server Eventlog:

Log Name:      Lync Server
Source:        LS Backup Service
Date:          4/28/2013 11:27:15 AM
Event ID:      4095
Task Category: (4000)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      TEST-LS15-SE1.test.deitterick.com
Description:
Failed to read topology from Master Central Management database Microsoft Lync Server 2013, Backup Service will continuously attempt to retrieve the topology.

While this condition persists, the module will not be able to perform backup.
Exception:
Could not connect to SQL server : [Exception=System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, Boolean withFailover)
   at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)
   at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
   at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at Microsoft.Rtc.Common.Data.DBCore.PerformSprocContextExecution(SprocContext sprocContext)
ClientConnectionId:00000000-0000-0000-0000-000000000000]
Cause: Possible issues with Master Backend database
Resolution:
Ensure that the SQL Server hosting the Master Central Management is running.

Log Name:      Lync Server
Source:        LS Backup Service
Date:          4/28/2013 11:27:15 AM
Event ID:      4082
Task Category: (4000)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      TEST-LS15-SE1.test.deitterick.com
Description:
Microsoft Lync Server 2013, Backup Service central management backup module failed to complete import operation.

Configurations:
Backup Module Identity:CentralMgmt.CMSMaster
Working Directory path:\\TEST-LS15-SE1.test.deitterick.com\share\1-BackupService-4\BackupStore\Temp
Local File Store Unc path:\\TEST-LS15-SE1.test.deitterick.com\share\1-BackupService-4\BackupStore
Remote File Store Unc path:\\TEST-LS15-SE2.test.deitterick.com\share\1-BackupService-3\BackupStore

Additional Message:
Exception: Microsoft.Rtc.BackupService.ModuleUnavailableException: Backup module is temporarily unavailable at this point. Reason: CMS backup module is not initialized yet.
   at Microsoft.Rtc.BackupService.BackupModules.CentralMgmtBackupModule.CheckModuleAvailability(Nullable`1 primaryPool)
   at Microsoft.Rtc.BackupService.BackupModules.CentralMgmtBackupModule.ApplyChanges(Unzipper unzipper, String& newCookie, Boolean& forceSetErrorState)
   at Microsoft.Rtc.BackupService.BackupModuleHandler.ReceiveBackupDataTask.ApplyChanges(Boolean& forceSetErrorState)
   at Microsoft.Rtc.BackupService.BackupModuleHandler.ReceiveBackupDataTask.InternalExecute()
   at Microsoft.Rtc.Common.TaskManager`1.ExecuteTask(Object state)

Cause: Either network or permission issues. Please look through the exception details for more information.
Resolution:
Resolution

Log Name:      Lync Server
Source:        LS Backup Service
Date:          4/28/2013 11:30:45 AM
Event ID:      4080
Task Category: (4000)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      TEST-LS15-SE1.test.deitterick.com
Description:
Microsoft Lync Server 2013, Backup Service central management backup module failed to complete export operation.

Additional Message:
Exception: Microsoft.Rtc.BackupService.ModuleUnavailableException: Backup module is temporarily unavailable at this point. Reason: CMS backup module is not initialized yet.
   at Microsoft.Rtc.BackupService.BackupModules.CentralMgmtBackupModule.CheckModuleAvailability(Nullable`1 primaryPool)
   at Microsoft.Rtc.BackupService.BackupModules.CentralMgmtBackupModule.ConfirmChanges(String cookie)
   at Microsoft.Rtc.BackupService.BackupModuleHandler.SendBackupDataTask.ConfirmChanges(String cookie)
   at Microsoft.Rtc.BackupService.BackupModuleHandler.SendBackupDataTask.ConfirmChangesAndPrepareCookieToSync(Boolean primaryDataExists, Boolean secondDataExists, CookieContainer& oldCookie, Boolean& forPrimaryBatch)
   at Microsoft.Rtc.BackupService.BackupModuleHandler.SendBackupDataTask.InternalExecute()
   at Microsoft.Rtc.Common.TaskManager`1.ExecuteTask(Object state)

Cause: Either network or permission issues. Please look through the exception details for more information.
Resolution:
Resolution

The issue is that the Windows Firewall is blocking access to SQL. If you compare the Windows Firewall rules on the first Standard Edition Server to the rules on the second Standard Edition Server, you'll notice that 2 rules are missing on the second Standard Edition Server:

This is why access to the CMS on the second Standard Edition Server is failing except from the second Standard Edition Server. These two rules were added to the first Standard Edition Server when I ran the "Prepare first Standard Edition server" step in the Deployment Wizard. This is why they're on the first Standard Edition Server, but not the second. The fix is just to manually add these rules to the second Standard Edition Server. Once the rules are added, you'll be able to access the CMS from other Lync Servers. This will also resolve the ErrorState that Get-CsBackupServiceStatus is displaying for the first Standard Edition Server.

If you had decided to pair the Standard Edition Servers when you first authored the topology, you could have just ran the "Prepare first Standard Edition server" step in the Deployment Wizard on both Standard Edition Servers. This would have created the required SQL instance and placed the correct rules in the Windows Firewall. Of course, if you have the Windows Firewall disabled, you wouldn't run into this issue, but it's not recommended to disable the Windows Firewall!

The important thing to remember is to make sure that the Windows Firewall rules on your paired Standard Edition Server are setup correctly. Otherwise you might be in for a surprise when practicing your disaster recovery plan!

Archiving Options in Lync Server 2013

2013-04-25T02:36:00Z

Lync Server 2013 has made a couple of changes to what can be archived and where that archived data can be stored.

What's New?

Collocation of Archiving on Front End Servers
Exchange Server 2013 Integration
SQL Store Mirroring
Archiving of Whiteboard and Polls

With the collocation of the Archiving Server role onto the Front End Servers, we've eliminated the need for a separate Archiving Server in Lync Server 2013. This will help to reduce the server footprint required. The Archiving service uses Unified Data Collection Agents that are responsible for capturing data and are located on all Front End Servers and Standard Edition Servers. This also means that MSMQ is no longer needed! It has been replaces by LySS (Lync Storage Service). Also, you now have the choice to use Lync for the archiving store or Exchange Server 2013. However, you can't archive to both locations at the same time. Archiving to Lync is similar to previous versions. The archived information is stored in the LcsLog database in SQL. If you decide to use Exchange Server 2013 are the archiving store, you will be using OAuth and EWS to deliver the archived data to Exchange Server 2013. Archiving also supports SQL mirroring for high availability and the database can be collocated in the same instance as the Front End Server databases, or it can be in a separate instance.

What Information Can Be Archived?

The following types of content can be archived:

Peer-to-peer instant messages
Conferences (meetings), which are multiparty instant messages
Conference content, including uploaded content (for example, handouts) and event-related content (for example, joining, leaving, uploading sharing, and changes in visibility)
Whiteboards and polls shared during a conference

The following types of content are not archived:

Peer-to-peer file transfers
Audio/video for peer-to-peer instant messages and conferences
Desktop and application sharing for peer-to-peer instant messages and conferences

The Lync Server 2013 Archiving service does not archive Persistent Chat conversations. To archive Persistent Chat conversations, you must enable and configure the compliance service, which is a component that can be deployed with the Lync Server 2013 Persistent Chat Server.

How Archiving to Lync Works

The first thing is to enable archiving. You can do that in the Lync Server 2013 Control Panel or via the Lync Server Management Console:

EnableArchiving
Indicates which items (if any) are saved to the archiving database. Valid values are:

None. Archiving is disabled. This is the default value. If EnableArchiving is set to None then nothing will be archived in either your Lync Server archiving database or in Exchange 2013.
ImOnly. Only instant message transcripts are archived. If Exchange archiving is enabled these transcripts will be archived in Exchange 2013. If Exchange archiving is disabled then these transcripts will be archived to Lync Server.
ImAndWebConf. Both instant message transcripts and Web conferencing transcripts are archived. If Exchange archiving is enabled these transcripts will be archived in Exchange 2013. If Exchange archiving is disabled then these transcripts will be archived to Lync Server.

Once archiving is enabled and an Archiving Policy is applied to your users, the Archiving Data Collection Agent on each of the Front End Servers in the pool will monitor for IMs and/or Web Conferences that need to be archived. It will send those to be archived communications to the LySS. The LySS will then write it to the appropriate repository, either SQL or Exchange Server 2013.

How Archiving to Exchange Server 2013 Works

The process for archiving to Exchange Server 2013 works similarly to archiving to SQL. When configuring the archiving settings for Exchange Server 2013, remember to check the box for Exchange Server integration.

EnableExchangeArchiving
When set to True, Lync Server 2013 instant message and conferencing transcripts are stored in Microsoft Exchange Server 2013 rather than a separate SQL Server database. Note that if you enable Exchange archiving then users will be managed by the Exchange archiving policies instead of Lync Server 2013 archiving policies. The default value is False.

Lync Server 2013 will use OAuth to connect to the user's mailbox in Exchange Server 2013. You will need to configure both Lync Server 2013 and Exchange Server 2013 for Oauth. You can refer to the Configure OAuth Authentication TechNet article for Exchange Server 2013 and the Managing Server-to-Server Authentication (Oauth) and Partner Applications TechNet article for Lync Server 2013. For users with a mailbox on Exchange Server 2013 and on In-Place hold the Exchange Archiving Policies override Lync Server archiving policies. You can also set the archiving policy manually for individual users using the Set-CsUser to select the archiving policy:

ExchangeArchivingPolicy
Indicates where the user's instant messaging sessions are archived. Allowed values are:

Uninitialized - Indicates that archiving will be based on the In-Place Hold settings configured for the user's Exchange mailbox; if In-Place Hold has not been enabled on the user's mailbox then the user will have his or her messaging and Web Conferencing transcripts archived in Lync Server.
UseLyncArchivingPolicy - Indicates that the user's instant messaging and Web conferencing transcripts should be archived in Lync Server rather than in Exchange.
NoArchiving - Indicates that the user's instant messaging and Web conferencing transcripts should not be archived at all. Note that this setting overrides any Lync Server archiving policies assigned to the user.
ArchivingToExchange - Indicates that the user's instant messaging and Web conferencing transcripts should be archived to Exchange regardless of the In-Place Hold settings that have (or have not) been assigned to the user's mailbox.

Where in Exchange Server 2013 are Archived Lync Messages Stored?

When the user is set to archive into Exchange Server 2013, the messages are stored in a hidden folder (Recoverable Items > Purges) in their mailbox. One way to verify this is to use EWSEditor:

Opening up the Purges folder will show you all the conversations that have been archived for that user.

Best Practices

One thing to check when setting up archiving to either SQL or Exchange Server 2013 is to run the Get-CsArchivingConfiguration cmdlet make sure that ArchiveDuplicateMessages is set to True:

From the Set-CsArchivingConfiguration TechNet article:

The ArchiveDuplicateMessages parameter specifies how "cross-pool" instant messages should be archived. Consider a simple example: Ken Myer (with an account in Pool 1) sends an instant message to Pilar Ackerman, who has an account in Pool 2. Pilar, in turn, sends a reply to Ken’s instant message. If ArchiveDuplicateMessages is set to False, then (based on a built-in algorithm) the session transcript will be logged in either Pool 1 or Pool 2, but not both. If ArchiveDuplicateMessages is set to True (the default value), the transcript will be logged in both pools.

Another parameter to look at and possibly adjust depending on your company policies is CachePurgingInterval. From the Set-CsArchivingConfiguration TechNet article:

This parameter indicates how often (in hours) the system is purged of transcripts where none of the participants have been enabled for archiving. By design, all group IM sessions and conferencing sessions are recorded when they take place. At the specified interval, the system determines whether any of the participants in these sessions have been enabled for archiving. If the system finds a session where none of the participants have been enabled for archiving, then that transcript will be deleted from the database. The CachePurgingInterval property can be set to any integer value between 4 and 168, inclusive. The default value is 24.

Issue with Calling Number Translation Rules and E.164 Extensions

2013-04-22T03:42:00Z

I came across this issue while troubleshooting dialing issues at a customer. When a Lync 2013 user placed an outbound call to the PSTN, the call failed. On the gateway we could see that the calling party's number wasn't being translated properly. The interesting thing was that we already had a trunk configuration setup with a calling number translation rule configured:

The above rule should be translating the calling party's number to:

However, looking at the gateway, we can see that this isn't happening:

INVITE sip:94255550105@mp114-red.lab.deitterick.com;user=phone SIP/2.0
FROM: "William Cooper"<sip:+14255550101;ext=0101@lab.deitterick.com;user=phone>;epid=3316DAF438;tag=861f7b5a
TO: <sip:94255550105@mp114-red.lab.deitterick.com;user=phone>

For some reason the full E.164 phone number is being sent to the gateway. The reason this was happening was because when the user's line URI was configured, it was configured with an E.164 extension (;ext=0101). The calling number translation rule we configured is explicitly looking for 12 digits. If you enter the full E.164 phone number including the extension, which is what Outbound Routing is sending, no match is found:

The solution is really simple, you just need to account for the E.164 extension in the calling number translation rule:

The pattern to match is:

^\+1(42555501\d{2})(;ext=\d{4})?$

As you can see I've added the E.164 extension to the pattern to match string. I've also added the ? after it. This means that this rule will also apply if the calling party's number doesn't have an E.164 extension assigned. Now when testing the rule, you can see that it succeeds using both formats:

Now when the gateway receives the call, the calling party's number is properly formatted properly:

INVITE sip:94255550105@mp114-red.lab.deitterick.com;user=phone SIP/2.0
FROM: "William Cooper"<sip:4255550101@lab.deitterick.com;user=phone>;epid=3316DAF438;tag=f9a5772c0
TO: <sip:94255550105@mp114-red.lab.deitterick.com;user=phone>

If you are including the E.164 extension on your Lync 2013 users and you need to apply calling number translation rules, remember that you'll need to account for the extensions when building your rules.

Issue with Automatic Updates Enabled & Office Web Apps Server 2013 Update

2013-03-30T01:33:00Z

Update 4/8/13 - Added information about change to KB2760445 automatically updating.

I came across this issue in my lab the other day. I noticed that in Hyper-V Manager both of my Office Web Apps Servers had really high CPU utilization:

It's pretty rare that one of my VMs would be doing that much to consume that much of the CPU, so I looked at Task Manager on both of the VMs and saw that they were both pegging the processor:

Neither server should be this busy, so I checked out the Application Event Log to find out what was going on and found the following errors:

Log Name:      Application
Source:        .NET Runtime
Date:          3/29/2013 6:11:08 PM
Event ID:      1026
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      LAB-OWA1.lab.deitterick.com
Description:
Application: WordViewerWfeWatchdog.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.TypeInitializationException
Stack:
   at Microsoft.Office.Web.Common.ServiceInstanceFinder.GetLocalAgentInstance(Microsoft.Office.Web.Common.OfficeServiceType)
   at Microsoft.Office.Web.Common.WatchdogHelper.PrepareRegistrations(Microsoft.Office.Web.Common.OfficeServiceType)
   at Microsoft.Office.Web.Common.WatchdogHelper.WatchMachines(Microsoft.Office.Web.Common.OfficeServiceType, CheckServiceInstance, Microsoft.Office.Web.Common.OfficeServiceType, System.String)
   at Microsoft.Office.Web.WordViewerWatchdog.Wfe.Program.Main()

Log Name:      Application
Source:        Application Error
Date:          3/29/2013 6:11:08 PM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      LAB-OWA1.lab.deitterick.com
Description:
Faulting application name: WordViewerWfeWatchdog.exe, version: 15.0.4481.1000, time stamp: 0x50ee5a74
Faulting module name: KERNELBASE.dll, version: 6.2.9200.16451, time stamp: 0x50988aa6
Exception code: 0xe0434352
Fault offset: 0x000000000003811c
Faulting process id: 0x17e8
Faulting application start time: 0x01ce2cca4f9a878f
Faulting application path: C:\Program Files\Microsoft Office Web Apps\WordViewerWfeWatchdog\WordViewerWfeWatchdog.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 8e4fbe96-98bd-11e2-93f0-00155d010f10
Faulting package full name:
Faulting package-relative application ID:

There were tons of these errors in the Application Event Log. Recently an update for Office Web Apps Server 2013 was installed:

Note: You may need to click on the image above in order to read the text.

Since I had Microsoft Update enabled on the servers, the update was installed automatically:

Unfortunately Office Web Apps Server 2013 has specific steps that need to be followed in order to correctly apply updates. Those steps are detailed in the Apply software updates to Office Web Apps Server TechNet article. Since I let Microsoft Update automatically install the updates, the machines weren't removed from the Office Web Apps Server farm before the update was applied, which is what caused the errors listed above. Also unfortunate is that KB2760445 doesn't have an uninstall option. This means that to fix this issue you will need to uninstall Office Web Apps Server 2013, reinstall, and setup the Office Web Apps Server farm again:

Once Office Web Apps Server 2013 is installed again, and before you recreate the Office Web Apps Server farm, download and install KB2760445.

Note: You may need to restart the server in order for the Office Web Apps Server 2013 PowerShell cmdlets to be loaded back into PowerShell by default.

After installing KB2760445 and recreating the Office Web Apps Server farm, CPU utilization on the servers is back to normal:

In the future, to correctly install updates for Office Web Apps Server 2013, follow the steps listed in the Apply software updates to Office Web Apps Server TechNet article.

Note: A change was made to the way Microsoft Update handles KB2760445. The update is no longer installed automatically during the Windows Update maintenance window. An administrator will now need to manually installed the update:

Before applying the update, you will still need to follow the steps listed in the Apply software updates to Office Web Apps Server TechNet article!

Issue Deploying Lync Server 2013 Monitoring Reports to SQL Mirror

2013-03-09T18:53:00Z

I was playing around with the Lync Server 2013 Monitoring Reports and decided to deploy it to both of my SQL Servers in my lab. I have a SQL mirror setup and I installed Reporting Services on both servers. My initial thought was to use a hardware load balancer to distribute the traffic to both Reporting Services Servers. This would also give me high availability within the site for the Monitoring Reports. However, when I went to deploy the Monitoring Reports to the server acting as the SQL mirror, I received the following error:

Cannot grant ReportsReadOnlyRole to user "LAB\srvLyncReports". For details, see the following error message:
Exception calling "Create" with "0" argument(s): "Create failed for User 'LAB\srvLyncReports'."

From the error message it looks like the Deploy Monitoring Reports wizard can't assign rights to the service account in SQL. Opening up the SQL instance in SQL Server Management Studio on the SQL mirror you can see that all of the databases are listed and mirroring is working correctly:

However when I go into the properties for the service account and click on the User Mapping page, I received the following message:

One or more databases are inaccessible and will not be displayed in list.

When I tried the same thing on the primary SQL Server, all of the databases are listed correctly and you can see that the service account is assigned ReportsReadOnlyRole:

It appears that you can only assign permissions to databases on the primary SQL Server in the mirror and the Deploy Monitoring Reports wizard won't continue unless it can access the databases, even though the permissions are already set on the primary. At this point the Monitoring Reports are only deployed to the primary SQL Server. They don't get replicated to the mirror, since Reporting Services doesn't participate in the SQL mirroring that is configured. Looking at the SQL Server Reporting Services (SSRS) website on the SQL mirror, you can see that this is the case:

So in order to publish the Monitoring Reports to the SQL mirror you will need to fail the databases (LcsCDR and QoEMetrics) over to the SQL mirror. You can accomplish this by running the Invoke-CsDatabaseFailover cmdlet:

This will make just the Monitoring Server databases active on the SQL mirror. You can confirm this in SQL Server Management Studio:

With the Monitoring Server databases active on the SQL mirror, you can now run the Deploy Monitoring Reports wizard and deploy the reports:

You can confirm the reports were published by going to the SSRS website on the SQL mirror:

Once you have successfully deployed the Monitoring Reports to the SQL mirror you can fail the Monitoring Server databases back to the primary SQL Server. Now that both SQL Servers have the Monitoring Reports deployed, the Lync Server 2013 Control Panel will display both servers under the View Monitoring reports section:

However, if you try to view the Monitoring Reports from the SQL Server that isn't currently the primary for the Monitoring Server database, you will receive the following error message:

An error has occurred during report processing.
(rsProcessingAborted)
- Cannot create a connection to data source 'CDRDB'.
  (rsErrorOpeningConnection)
  - For more information about this error navigate to
    the report server on the local server machine, or enable remote errors

This is because connection string data sources is pointing to the local SQL Server:

Connection string: data source=(local);initial catalog=LcsCDR

You could change this to point to the active copy of the Monitoring Server databases, but this means that you would need to change the connection string every time you failed over the databases. I prefer to think to the Monitoring Server reports as active/passive. While you won't be able to load balance connections to the reports in this configuration, you will get redundancy in case you need to view the reports while the Monitoring Server databases are failed over to the SQL mirror.

Issue Configuring SQL Mirroring for Lync Server 2013 when SQL Witness is Defined But Not Available

2013-03-05T20:22:00Z

I ran into an interesting issue while trying to setup SQL mirroring with a witness in my lab. When installing the databases and setting up mirroring in Topology Builder:

The wizard kept failing when trying to setup the mirror:

Looking at the log file I saw the following errors listed:

Error: DsRoleGetPrimaryDomainInformation failed with error "6BA".

Error: An error occurred: "Microsoft.Rtc.Management.ADConnect.CannotGetDomainInfoException" "DsRoleGetPrimaryDomainInformation failed with error "6BA"."

Note: You may need to click on the image above in order to read the text.

Unfortunately neither of the error messages provided much useful information on why the error was occurring. I made sure that connectivity to Active Directory was OK and that my Domain Controllers were working properly since the error message made reference to AD. With everything checking out OK and the Install Database wizard still failing in Topology Builder, I decided to run the command manually from the Lync Server Management Shell. Since I was setting up mirroring, Topology Builder was running the Install-CsMirrorDatabase cmdlet. So, I ran that cmdlet from the Lync Server Management Shell using the appropriate parameters as well as including the Verbose parameter. You can find all of the parameters documented in the Install-CsMirrorDatabase TechNet article. Even though the cmdlet failed, this time the error message provided in PowerShell gave me enough information to help me solve the issue:

Install-CsMirrorDatabase : Command execution failed: Cannot setup mirroring because there is an error validating the version of the SQL Server instances. Verify that the SQL Server instances are available. Exception: Microsoft.SqlServer.Management.Common.ConnectionFailureException: Failed to connect to server LAB-SQL-WIT.lab.deitterick.com\SQLEXPRESS. --->
System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)

From the error message above, you can see that the error was actually happening while trying to connect to the SQL witness. In this case I had installed SQL Express in my lab to act as the SQL witness. Once I saw the error message I remembered that SQL Express doesn't allow remote connections by default. In order to allow remote connections I needed to open SQL Server Configuration Manager, expand SQL Server Network Configurations, click on Protocols for SQLEXPRESS, and set TCP/IP to Enabled:

With TCP/IP enabled, this time when running the Install Database wizard from Topology Builder, I was able to successfully complete setting up SQL mirroring with a witness:

So while this time the log files weren't as much help in solving the issue, it is important to remember that when you are troubleshooting an issue in Lync, take advantage of all of the information that you can, between errors in the GUI, log files/tracing, and PowerShell, in order to help you solve any issues that you run into.

How to Apply Lync Server 2013 Cumulative Updates

2013-02-28T03:36:00Z

Update 3/17/13 - Added information about additional update steps from KB2809243.

Just like previous versions of OCS/Lync, there are two methods to update the servers with OCS/Lync cumulative updates. You can use Microsoft Update/WSUS or you can use LyncServerUpdateInstaller.exe. Both ways accomplish the same thing, but what I find is that a lot of customers who use the Microsoft Update/WSUS method forget to then go back and apply any database updates that are needed. This blog post focuses on the LyncServerUpdateInstaller.exe method, as I prefer using it over the Microsoft Update/WSUS method:

Installing Standard Edition Server Cumulative Updates

To apply the latest cumulative updates to a Standard Edition Server, all you need to do is run LyncServerUpdateInstaller.exe and click on Install Updates:

The installer will detect what Lync services have been installed on the server and will display whether or not the latest update has been applied. Once the updates have been installed, restart the server if prompted to do so and then skip down to the "Installing Lync Database Updates" at the bottom of this article.

Installing Enterprise Edition Pool Front End Server Cumulative Updates

With the release of the Lync Server 2013 Cumulative Updates, the process for updating Front End Servers in an Enterprise Edition Pool has changed slightly. Most of the changes have to do with the move to using Windows Fabric to replicate data between the Front End Servers in an Enterprise Edition Pool. With the introduction of Windows Fabric a couple of other concepts were also introduced, User Groups and Upgrade Domains. Both are created and managed by Windows Fabric automatically. You can read more about these new features in the Lync Server 2013: Windows Fabric & User Groups blog post. Because of these new concepts in Lync Server 2013, you will need to update your Front End Servers one at a time, making sure that services are fully restored on that server before you move on to the next Front End Server in the pool in order to make sure that you maintain quorum and at least one copy of all of the user groups. The flow chart below, from the Upgrade or Update Front End Servers TechNet article, walks through the process:

Running Get-CsPoolUpgradeReadinessState should return something similar to the following output:

If the cmdlet returns True for IsReadyForUpgrade for the server you are going to update, you can then run LyncServerUpdateInstaller.exe and apply the necessary updates on the Front End Server:

This process is the same as updating a Standard Edition Server. Once the updates have been installed, restart the server if prompted to do so. Once the server has been restarted, make sure that all of the services have returned to the running state and then run the Get-CsPoolUpgradeReadinessState cmdlet again and repeat the process until all of the Front End Servers in the pool have been updated. You can then skip to the "Installing Lync Database Updates" at the bottom of this article.

Installing Lync Database Updates

Also like previous version of OCS/Lync there Lync database updates that need to be applied. The steps are detailed in the "Step 2: Apply the back end database updates" section in KB2809243. You apply the update by running the following PowerShell cmdlet on one Front End Server in each Enterprise Edition Pool and all Standard Edition Servers in your environment:

Install-CsDatabase -ConfiguredDatabases -SqlServerFqdn FEBE.FQDN -Verbose

New in Lync Server 2013, if you are using SQL Mirroring or you have setup paired pooling for the pool hosting your CMS, you also need to remove and reinstall the mirror using the Uninstall/Install-CsMirrorDatabase cmdlets and the Install-CsDatabase cmdlet.

Additional Update Steps

Also make sure that you follow the steps in sections "Step 3: Enable the Mobility service" and "Step 4: Enable the Unified Communications Web API" in KB2809243.

Installing Updates on Other Lync Server 2013 Servers

To apply the latest cumulative updates on other Lync Server 2013 servers in your environment, just run LyncServerUpdateInstaller.exe and click on Install Updates.

Lync 2013 Mobility

2013-02-27T21:04:00Z

Update 4/2/13 - Updated for release of Android client.
Update 3/12/13 - Updated for release of iOS client.

With the release of the Lync Server 2013: February 2013 Cumulative Updates, additional features are available with Lync mobility. In order to take advantage of these new features, you will need to install the Lync 2013 version of the mobile client for the respective mobile platform. The Lync 2013 mobile client will be released for Windows Phone, iOS, and Andriod through their respective marketplace. The time and availability of the mobile apps will vary between each marketplace.

What's New?

There are a bunch of new features in both the Lync 2013 mobile client and the server. The list below has some of the important changes:

Mobile clients are built using the Unified Communications Web API (UCWA). This is the same API used to build the Lync 2013 Web App. You can find out some more information about UCWA from the Unified Communications Web API website.
Support for Audio/Video over 3G/4G/Wi-Fi.
Support for Application Sharing via the Lync 2013 iPad client.
Layer 7 HLB no longer required. The requirement for cookie-based persistence has been removed and only Layer 4 HLB is now required.
Federation with O365 environment for push notifications on iOS clients is no longer required. When using the Lync 2013 mobile client a VoIP socket is used instead of push notifications. The Lync 2010 mobile client for iOS still uses push notifications, so if you still need to support legacy clients in your environment you will still need to enable federation with O365.
Longer lived registration. You won't be signed out of the mobile client if you haven't launched the application in more than 3 days.
HD photo support from Exchange 2013.

Deployment Considerations

Since A/V is now supported over 3G/4G, or Wi-Fi the Edge Server plays a larger role in mobility deployments. In the diagram below, you can see that the mobile client will still send signaling information via the Reverse Proxy, but it will now send media via the A/V Edge interface on the Edge Server:

When in co-existence with Lync Server 2010, your Lync autodiscover URLs can still resolve to a Lync Server 2010 Front End Server or Director. The Lync autodiscover service will be return the correct external web services FQDN for your user based on your homed pool. The media traffic from the Lync 2013 mobile client can also use a Lync Server 2010 Edge Server.

Windows Phone Client Interoperability

Mobile Client	Install on WP 7.5	Install on WP 8	Connect to Lync Server 2010	Connect to Lync Server 2013
Lync 2010	✔	✔	✔	✔
Lync 2013	✖	✔	✖	✔

From the table above, you can see that the Lync 2013 Windows Phone mobile client can only be installed on Windows Phone 8 and can only connect if the user is homed on a Lync Server 2013 pool that has the cumulative update installed. If you try to sign in to a Lync Server 2010 Front End Server or a Lync Server 2013 Front End Server without the cumulative update using the Lync 2013 mobile client you will see the following error message:

Error: "You can't sign in with this version of Lync. Please install Lync 2010."

iOS and Android Client Interoperability

The Lync 2013 iOS client has been released for iPhone and iPad. It is now available in the iTunes store. You will need to be running iOS 6.0 or later. Keep in mind that for this client to work you will need to be homed on a Lync Server 2013 Standard Edition Server or Enterprise Edition Pool with at least the Lync Server 2013: February 2013 Cumulative Updates installed. Also the Lync 2013 Android client has been released. It is now available in the Google Play store. The list of supported Android phones is listed in KB2829747.

Issue with Exchange UM Voicemails Being Delivered After Applying KB931125

2013-02-11T17:57:00Z

I ran across this today with a customer. We were setting up Exchange UM to connect to Lync and while that was working just fine, users could leave voicemail messages for other users, the email messages weren't being delivered to the user's mailbox. Looking at the Application Event Log on the UM Server we say the following error message:

Log Name:      Application
Source:        MSExchange Unified Messaging
Date:          2/11/2013 11:22:08 AM
Event ID:      1423
Task Category: UMCore
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SERVERNAME
Description:
The Unified Messaging server encountered an error while trying to process the message with header file "C:\Program Files\Microsoft\Exchange Server\V14\UnifiedMessaging\voicemail\8e22c540-bc77-4f72-9a60-758a04a1522b.txt". Error details: "Microsoft.Exchange.Net.ExSmtpClient.TlsApiFailureException: A TLS API failure occurred. Error = 0x80090301

The TLS API failure was being cause because they had applied KB931125 to the server. You can read more about this issue from the Windows Server Blog. Unfortunately this causes the Trusted Root Certificate store to grow rather large and thus causes the TLS API failure. The fix is to either delete any unneeded root certificates or run the Fix it from KB2801679.

Issue with the Lync Basic 2013 Client and A/V Disabled

2013-02-11T04:32:00Z

I ran across this while testing an issue a reader of my blog posted in the comments of an article I wrote about the Lync Basic 2013 client. They had mentioned that when a user using the Lync Basic 2013 client was set to A/V disabled the options for A/V were still presented in the client. In my testing I found this to be true and in fact the options were not only presented to the user, but A/V wasn't blocked if the user tried to initiate an A/V call to another user. For my testing I was using version 15.0.4420.1017 of the Lync Basic 2013 client. You can see in the screen shot below that even though the user is set to A/V disabled in Lync, the Phone tab and the Call Forwarding options still appear and the user is actually in a call with another user:

In addition, the options for starting an A/V call with another user are available:

Also, when looking at the A/V disabled user's presence from another client, you can see that they show as "Video Capable", even though they should be disabled for video:

If you compare this to the same user signed into the Lync 2013 client, you can see that the Phone tab and the Call Forwarding options are removed, as expected:

Also, the options for initiating an A/V call are removed as well:

So how does the Lync client know what features to make available when the user signs in? The answer is through information received by the client via in-band provisioning. The Lync client sends out a SIP SUBSCRIBE requesting provisioning information:

SUBSCRIBE sip:wcooper@test.deitterick.com SIP/2.0

...

Content-Type: application/vnd-microsoft-roaming-self+xml

The response is a SIP/2.0 200 OK and contained in the response is the setting we're interested in:

<telephonyMode>NoAudioVideo</telephonyMode>

The telephonyMode setting can contain a couple of different values, depending on what the user is configured for:

NoAudioVideo = Audio/Video Disabled
None = PC-to-PC only
Rcc = Remote Call Control
Rcc Only = Remote Call Control only
UC = Enterprise Voice

It appears that currently the Lync Basic 2013 client is ignoring this setting and is allowing the user to initiate A/V sessions whether the user is enabled to do so or not.