This posting is provided "AS IS" with no warranties, and confers no rights.The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway. Please use the Microsoft Forums for support requests.
One of the big improvements with the release of Office Communications Server (OCS) 2007 R2 was the new Device Update Service, much more simple than the previous version. Goodbye Windows SharePoint Services on a different server, no more additional complexity in order to update the UC devices deployed in an organization. Now, the Device Update Service is automatically installed on the Web Components Server, which is part of the Front-end server, on a consolidated topology (the only one supported in R2).
Device Update Service supports two types of UC devices: OCS 2007 R2 Communicator Phone Edition (OCPE or Tanjay) and RoundTable (must be manually configured).
Device Update Service is mostly used to upgrade OCPE phones (LG-Nortel IP8540, Polycom CX700 and older Microsoft branded), so one could expect this process to be the most common cause of troubles and frustration. And that is, in fact, the case! Now even more, since there is a new version of the firmware for OCS 2007 R2.
Before we dive into the troubleshooting process, let us know better the Device Update Service Architecture and how it works:
In the default configuration, Communicator Phone Edition connects to Device Update Service in the following manner:
The latest firmware version, (3.5.6907.0, by the time this post was written), can be downloaded from the Microsoft site. The downloaded file is a self-extracting executable that contains a .cab archive with all the supported phones.
In order to upload the update file, we must run the Device Update Service Management Console: open the OCS R2 Management Console, right click the pool and select Device Updater. From the Tools menu, click Upload .cab File, navigate to the .cab file that you want to upload, and then click Open.
Check the Pending tab of the Device Update Service Management Console to verify that the new update is listed.
You can also verify that the upload process went OK by checking the shared updates folder (if you’re using Enterprise edition). For the Standard edition, the default path is %ProgramFiles%\Microsoft Office Communicator 2007 R2\Web Components\DeviceUpdateFiles. There should be a folder hierarchy like the one depicted in the following picture. Note that there are some Logs folders that I’ll cover a little bit ahead.
After the update file is uploaded, the corresponding firmware can be approved for all devices in the organization by selecting it and then clicking Approve. The revision should be more recent than the revision for the last update the UC devices received.
Prior to making the update widely available, it is recommended that you test it on some devices. To add a test device, go to the Test Devices tab, click Add, type a Friendly Name and then fill up the MAC address or serial number of the device (there’s no need to approve the update).
Restart the device and that should trigger the update process, as described before.
If the previous version of the device is less than 1.0.522.98, it will first get updated to an interim build (1.0.522.103) that comes with OCS 2007 R2. This means that, in this case, 2 upgrade cycles are necessary before the device gets the approved/pending build.
You can use the logs in the Logs\Server\Audit\imageUpdates\ folder to audit software update requests from UC devices. There you can find some files named RequestHandlerAuditLog_<server_name>_<date>.log with the information you need to troubleshoot the update service. You can view server log files in a text editor or Microsoft Excel.
In the following example, an LG-Nortel device with version 1.0.522.34 is configured as a test device and will receive the most up-to-date firmware. Since 1.0.522.34 is lower than 1.0.522.98, the device must first be upgraded to the interim version (1.0.522.103), then reboot and finally it receives the most recent version available on the server.
Logging DateTime,User Name,User Host Address,Device Type,Request DateTime,Mac Address,Serial Number,Vendor,Model,Revision,Locale,Requested<FileName;Version;TimeStamp>[# Seperated for Multiple],Response<FileName;Version;TimeStamp>[# Seperated for Multiple] 03-06-2009 17:07:20,ruisilva@demo.local,10.1.1.123,UCPhone,06-03-2009 09:07:20,"0021630F207B","C185H001209","LG-Nortel","IP8540","A","ENU",cpe.nbt;0.0.0.0;01-01-1601 00:00:00, 03-06-2009 17:10:39,ruisilva@demo.local,10.1.1.123,UCPhone,06-03-2009 17:10:39,"0021630F207B","C185H001209","LG-Nortel","IP8540","A","ENU",cpe.nbt;0.0.0.0;01-01-1601 00:00:00,http://ocs.demo.local/DeviceUpdateFiles_Int/OCInterim/ENU/CPE.nbt;1.0.522.103;16-12-2008 04:43:58 03-06-2009 17:18:54,ruisilva@demo.local,10.1.1.123,UCPhone,06-03-2009 17:18:53,"0021630F207B","C185H001209","LG-Nortel","IP8540","A","ENU",cpe.nbt;1.0.522.103;16-12-2008 04:43:58,http://ocs.demo.local/DeviceUpdateFiles_Int/UCPhone/LG-Nortel/IP8540/A/ENU/3.5.6907.0/CPE/CPE.nbt;3.5.6907.0;16-12-2008 04:43:58
In the first line, the device gets in-band provisioning information from the server or Enterprise pool hosting the device user account. After a *manual reboot*, on the second line, it gets the interim version. Finally, on the third line, and after an *automatic reboot* (notice the 8 minute gap) the phone receives the 3.5.6907.0 version.
But besides the logs from the update service,the IIS logs can be extremely valuable to the troubleshooting process. This is the corresponding IIS log from the example above (I removed some unnecessary lines):
#Software: Microsoft Internet Information Services 7.0 #Version: 1.0 #Date: 2009-03-06 16:00:51 #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken 2009-03-06 17:07:20 10.1.1.90 POST /RequestHandler/ucdevice.upx - 443 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 124 2009-03-06 17:10:39 10.1.1.90 POST /RequestHandler/ucdevice.upx - 443 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 171 2009-03-06 17:12:57 10.1.1.90 GET /DeviceUpdateFiles_Int/OCInterim/ENU/CPE.nbt - 80 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 137810 2009-03-06 17:12:57 10.1.1.90 GET /DeviceUpdateFiles_Int/OCInterim/ENU/CPE.cat - 80 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 15 2009-03-06 17:18:54 10.1.1.90 POST /RequestHandler/ucdevice.upx - 443 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 187 2009-03-06 17:21:45 10.1.1.90 GET /DeviceUpdateFiles_Int/UCPhone/LG-Nortel/IP8540/A/ENU/3.5.6907.0/CPE/CPE.nbt - 80 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 170679 2009-03-06 17:21:45 10.1.1.90 GET /DeviceUpdateFiles_Int/UCPhone/LG-Nortel/IP8540/A/ENU/3.5.6907.0/CPE/CPE.cat - 80 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 15
Notice the line where the interim file CPE.nbt is downloaded to the phone. It takes some time (137.810 ms), because the file has a few megs.
2009-03-06 17:12:57 10.192.32.90 GET /DeviceUpdateFiles_Int/OCInterim/ENU/CPE.nbt - 80 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 137810
And then, notice the line where the phone requests the most recent version available. It took 170.679 ms to download the file.
2009-03-06 17:21:45 10.192.32.90 GET /DeviceUpdateFiles_Int/UCPhone/LG-Nortel/IP8540/A/ENU/3.5.6907.0/CPE/CPE.nbt - 80 - 10.1.1.123 Microsoft+UCPhone+Device 200 0 0 170679
Now that we have a pretty good understanding of the Device Update Service, what can we do if something doesn’t go as expected? Let’s look at the troubleshooting process:
2009-03-06 12:35:36 10.1.1.90 GET /DeviceUpdateFiles_Int/UCPhone/LG-Nortel/IP8540/A/ENU/3.5.6907.0/CPE/CPE.nbt - 80 - 10.1.1.1 Microsoft+UCPhone+Device 500 19 5 62
And this pretty much covers the troubleshooting process for the OCPE devices. I would like to hear from you if you have ever encountered a distinct situation that you managed to solve in a different way.
Let me start by saying that, although the title of this post refers Polycom CX700 (because was this device I used), the procedures described here will probably work with other devices (e.g. LG-Nortel or Microsoft).
If you read my previous post Troubleshooting OCS 2007 R2 Device Update Service for Communicator Phone Edition, you probably noticed the comments about upgrading Office Communicator Phone Edition (OCPE) version 1.0.452.0.
When I wrote that post, I hadn’t tested anything lower than 1.0.522.34, so, to tell you the truth, I was not sure it was possible to upgrade these early (Beta) versions. Until today!
When I had the chance to put my hands on one of these early babies (thanks Paulo Silva), I didn’t think twice.
As soon as I plugged the device into my test environment and tried to sign in, I got the following error:
Cannot sign in to Communications Service. Current version does not work with the available server. Contact your system administrator.
I immediately understood what the problem was: the Client Version Filter. Lowering the allowed OCPhone version to 1.0.199 did the trick.
After a quick reboot and still no signs of a successful upgrade, I noticed that I was getting an Update Status (0x0/404) on the phone. The IIS log confirmed the HTTP error 404 – File Not Found. The device was requesting the file /UCDeviceUpdates/ucdevice.upx, which cannot be found because the virtual dir /UCDevicesUpdate doesn’t exist.
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken 2009-04-15 17:47:37 192.168.200.101 POST /UCDeviceUpdates/ucdevice.upx - 80 - 192.168.20.103 Microsoft+UCPhone+Device 404 0 2 0
I’m not sure why the device requests that specific URL. In the tests I made with an even lower version, 1.0.199, the requested URL was /RequestHandler/ucdevice.upx, which is the correct one. Further investigation is needed to determine the cause of this issue.
In order to try to overcome the situation, I decided to create the /UCDevicesUpdate virtual dir, replicating all the settings of the /RequestHandler folder. Here’s how to do it on IIS 7.0 (with IIS 6.0 would probably be easier, since there is an option to redirect a virtual dir):
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken 2009-04-15 18:07:35 192.168.200.101 POST /UCDeviceUpdates/ucdevice.upx - 80 - 192.168.20.103 Microsoft+UCPhone+Device 405 0 64 31
With these changes in place, the upgrade process went as expected: the device gets in-band provisioning about the update URL, downloads and installs the interim version (1.0.522.103), reboots, downloads and installs the approved version (3.5.6907.9), does a final reset and it’s ready to use with Office Communications Server 2007 R2.
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken 2009-04-15 18:36:09 192.168.200.101 POST /UCDeviceUpdates/ucdevice.upx - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 6531 2009-04-15 18:38:28 192.168.200.101 GET /DeviceUpdateFiles_Int/OCInterim/ENU/CPE.nbt - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 139006 2009-04-15 18:38:28 192.168.200.101 GET /DeviceUpdateFiles_Int/OCInterim/ENU/CPE.cat - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 15 2009-04-15 18:46:05 192.168.200.101 POST /requestHandler/ucdevice.upx - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 2093 2009-04-15 18:49:09 192.168.200.101 GET /DeviceUpdateFiles_Int/UCPhone/Polycom/CX700/A/ENU/3.5.6907.9/CPE/CPE.nbt - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 183461 2009-04-15 18:49:09 192.168.200.101 GET /DeviceUpdateFiles_Int/UCPhone/Polycom/CX700/A/ENU/3.5.6907.9/CPE/CPE.cat - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 15 2009-04-15 18:50:29 192.168.200.101 POST /requestHandler/ucdevice.upx - 80 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 140 2009-04-15 18:51:51 192.168.200.101 POST /RequestHandler/ucdevice.upx - 443 - 192.168.20.103 Microsoft+UCPhone+Device 200 0 0 187 2009-04-15 18:56:47 192.168.200.101 GET /Abs/Int/Handler/F-0bd2.dabs - 443 - 192.168.20.103 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+CE) 401 2 5 0 2009-04-15 18:56:47 192.168.200.101 GET /Abs/Int/Handler/F-0bd2.dabs - 443 DEMO\OCPhone 192.168.20.103 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+CE) 200 0 0 125
A final note: I didn’t use the Test Devices option, instead I approved the update (this means that all qualified devices would get the update without first testing it).
So far, the OCS 2007 R2 Device Update Service revealed to be capable of successfully upgrading OCPE version 1.0.452.0 and later. The early beta versions can sometimes be tricky and require some tweaks, like creating a new virtual dir (as explained in this post).
I’ve read some post regarding the 1.0.452.0 version stating that WINS is required. In the tests I made, I didn’t use WINS, only DNS (I had the UCUPDATES and UCUPDATES-R2 defined as A records and pointing to the OCS pool IP address).
I would love to test version 1.0.199. Can someone lend me one? :-)
As much as I would like to catalog the story so far around KB974571 as fiction, the truth is that it resembles more to a horror movie.
Before going any further, I would like to say that Microsoft recommends thorough testing of any software patch/hotfix before applying them in a Production environment. Having said that, I would like to invite you all to read Best Practices for Applying Service Packs, Hotfixes and Security Patches.
The above mentioned TechNet article states the following:
The basic rules are:
"The risk of implementing the service pack, hotfix and security patch should ALWAYS be LESS than the risk of not implementing it."
And,
"You should never be worse off by implementing a service pack, hotfix and security patch. If you are unsure, then take steps to ensure that there is no doubt when moving them to production systems."
Whether you’re superstitious, religious or a devoted fan of Murphy Laws, the naked truth is that sh*t happens! And that’s exactly what recently happened with OCS and the release of the security patch KB974571. For more information, please read the post from the OCS Team Blog: Urgent: Known issue under investigation with KB974571 and LCS/OCS.
But the KB974571 hotfix has another side effect that I recently experienced. Besides affecting the services of LCS/OCS, the very same patch can also prevent the installation of new OCS servers.
Recently, I was building a new demo environment, when, nearly at the end of the installation of a standard edition server, during the activation phase of the server, I got the following error:
The OCS installation logs revealed the following information:
Failure [0xC3EC796C] One or more errors occurred during execution of the wizard; the wizard was unable to complete successfully. Please check the log file for more information.
After expanding the log, the error pointed out to the Activation Standard Edition Server Log.
The error in the Activation Standard Edition Server Log was:
Failure [0xC3EC78D8] Failed to read the Office Communications Server version information. This can happen if the computer clock is not set to correct date and time.
After spending some time troubleshooting the issue, with no solution in sight, I started to doubt the health of the virtual machines I was using (first) and then my sanity (last). But then, when I was on a dead end, a customer asked me to help him installing a new Edge Server. Guess what? Same error, same problem.
Since I was getting lots of noise around KB974571 (I already knew it could affect installed LCS/OCS services), I decided to uninstall that specific hotfix from the list of installed patches. That solved the problem!
Microsoft recommends to postpone installing KB974571 on any LCS 2005 / OCS 2007 /OCS 2007 R2 servers.
“Microsoft is investigating this issue, and will determine the most appropriate way to address it. Customers who are not running OCS or LCS server are not affected by this known issue, and can safely ignore this issue.
Customers who have deployed the OCS or LCS product on a server should assess the risk that is involved to decide whether to install the security update on that server. These customers should revisit this Knowledge Base article often, because this article will be updated as soon as more information and a resolution are available.”
There is now a fix for this issue, available through the KB974571 article. Look under the section “Resolution for the known issue”.
One of the coolest features of Office Communications Server (OCS) 2007 R2 is the Response Group Service (also known as Automatic Call Distribution – ACD – in telephony world), a technology that’s mostly used in Contact Centers, but that can be applied in many different scenarios…
But the purpose of this post is not to describe the Response Group Service, instead is to discuss some problems I faced when configuring custom tabs in Communicator (custom tabs are the preferred method to sign in to formal groups in Response Group Service).
I followed the detailed instruction to configure the Response Group Tab. Basically there are 2 steps involved:
Regarding the first step, my XML file consists of the definition of 2 tabs: one for the Response Group, and other for displaying a Live Search box.
<?xml version="1.0"?> <tabdata> <tab> <image>https://ptstu-ocs01.demo.local/Rgs/Clients/RgsOcTab.png</image> <name>Response Group Tab</name> <tooltip>Response Group Tab</tooltip> <contenturl>https://ptstu-ocs01.demo.local/Rgs/clients/Tab.aspx</contenturl> <siteid>1</siteid> </tab> <tab> <image>https://ptstu-ocs01/images/windows-live.png</image> <name>Microsoft Mobile</name> <tooltip>Microsoft Mobile Tab</tooltip> <contenturl>http://mobile.microsoft.com</contenturl> <siteid>2</siteid> </tab> </tabdata>
The second step is also very simple: importing the Communicator R2 Group Policy extension (communicator.adm), and then configuring the setting “Tab URL” (in my case, the value was https://ptstu-ocs01.demo.local/OCTabs.xml).
The problem was that, although all the necessary configuration was in place, there was no sign at all of the 2 custom tabs I had defined previously!
In order to solve the problem, I had to change 4 settings.
Recently I had the opportunity of participating in a Pilot of Lync Server in a customer that has the internal network segregated by firewalls. So, one of the first questions was about the ports and protocols used by Lync Server.
The requirements of this pilot included:
The ports and protocols used by Lync are pretty well documented in the following TechNet pages:
Additionally, the Microsoft Lync Server 2010 Protocol Workloads Poster does a very good job illustrating not only the ports and protocols used by each UC workload, but also their dependencies and relationships.
Instead of letting the customer compile and aggregate all the provided technical information, we decided to provide some Visio drawings, in order to facilitate the configuration tasks of the security team. In the end we delivered the following schematics:
Please note that these were built for a pilot and not for a Production environment, thus minor errors/inconsistencies may have been depicted. If you find one, please let me know.
If you want the Visio document, it can be downloaded here.