• Run-time error: '438': Object doesn't support this property or method in Microsoft Call Quality Methodology Scorecard for Lync Server (CQM)

    After installing MS14-082 (part of December 2014 Public Updates for Office) you might get the error run-time error '438': Object doesn't support this property or method, when clicking on the enable content button. If you click Debug you will see the reference shown below:

    The root cause of this appears to be related to problems mentioned in KB3025036 and here and performing the steps in the Resolution 1 section of the KB has fixed the issue.. 

  • Practical use of Call Quality Methodology

    This post is authored by Henrik Jørgensen from Microsoft Services in Denmark.

    The following is based upon real life experience with the CQM framework from a Microsoft Consulting Services project.

    One of our customers reported bad experiences with Lync. Specifically the customer reported, that their end-users complained about problems with Audio and Video. The problems reported could be divided into 2 scenarios:

    1. In PC to PC communication, where 2 end-users communicated via the Lync 2010 client.
    2. In Conferences hosted on the Lync 2010 platform, where multiple end-users participated in a conference call.

    The customer is a major global player in their specific area. They host several Lync pools globally and are represented in countries around the globe.

    An approach to analyze the above problems is to use the Call Quality Methodology (CQM) framework as introduced by the Microsoft Lync product group.

    The approach was to establish a baseline, in order to understand the level of the problems but also have a benchmark to compare with after implementation of changes to the Lync environment and related IT infrastructure components.

    We divided the work into the following areas:

    • Client analysis
      • Bios and device drivers
      • Patch level of the OS in use
    • Server Analysis
    • CQM analysis
      • Use the CQM SQL queries to analyze the data in the QoE database
      • Definition of persona profiles for Lync usage
      • Logical map of the Network topology
      • Bandwidth estimate calculation
      • Network Assessment for UC

    The work involved several IT teams at the customer. A key learning was, that the operation of a complex IT-infrastructure as Lync calls for co-operation and communication between the IT teams involved in operations and maintenance of the Lync infrastructure.

    The analysis work revealed several findings:

    • Clients
      • The Bios level needed to upgraded on some pc's
      • It was necessary to deploy newer versions of drivers for Network Interface Cards in the clients
    • Servers
      • The Hardware requirements was met
      • Newer BIOS and firmware was needed at the server level

    The KHI, CQM and Network Analysis revealed other findings. These are presented in more detail in the following.

    Server Key Health Indicators

    We used the KHI collection PowerShell script from the networking guide. We collected data for 5 working days. Afterwards, the data was imported to Microsoft Excel.
    Among other critical findings, we observed packet loss on some of the front-end servers. This called for further analysis of the server problems. A firmware upgrade was part of the solution.


    CQM Analysis

    The CQM SQL queries are divided in 3 areas

    1. Endpoints – the methodology will help to determine whether there are problems with the client pc's or the devices connected to these.
    2. Server to server / gateway traffic – in order to document whether the Lync media servers are healthy or not. Furthermore, the methodology can document whether conditions on the servers are contributing to packet loss and Jitter.
    3. Network – the methodology will document which LAN subnets the Lync poor calls are coming from.

    We used the queries in the Networking guide. A summary of the findings are provided below:

    Endpoints

    The CQM queries revealed that a majority of end-users at given locations did not use Lync certified devices. The customer initiated a process to

    • Provide the end-users with certified devices
    • Learn the end-users to use the devices

    Server traffic

    The CQM queries documented packet loss between the AVMCU and the Mediation Server and from the Mediation Server to the gateway at some sites. Further analysis looked at

    • Non Lync Software on the servers
    • Lync pre-requisites regarding antivirus exclusions
    • Ensure that the network equipment are healthy and it follows Microsoft guidelines from the Open Interoperability List
    • Firmware on the servers network interface card

    Network traffic

    We identified several issues

    • All access to the Lync infrastructure was via VPN for the customers employees when working remotely. Split tunneling was not implemented.
    • Some internal peer-to-peer traffic was relayed via the Edge servers.
    • RTT > 500 Ms was observed on some network connections
    • Packet loss on WiFi networks.

    All above findings called for additional analysis and work in order to solve the problems.

    Network Analysis

    Together with the customer, we defined three persona profiles. These where defined in the bandwidth calculator.
    The customers HR department provided us with a number for each of the persona profiles at the specific locations where the customer is represented.

    The calculations in the bandwidth calculator revealed:

    • Possible overflow of the QoS queue allocated for Lync traffic
    • Locations where more bandwidth was needed to handle the Lync traffic

    Furthermore, the customer initiated a network assessment of the WAN. The assessment documented the predictions from the bandwidth calculator.

    Customer initiated actions to improve the Lync experience

    The customer initiated several actions to improve the Lync experience of their end-users. In summary these are

    • Acquiring more WAN bandwidth for given locations
    • Implementation of Quality of Service on the network
    • Renewal of network equipment at given locations
    • Improvements to existing WiFi implementations at given sites
    • Training of the end-users
    • Proactively use of the CQM methodology in order to monitor improvements

       

    Key learnings

    With the CQM approach, we helped our customer to not only troubleshoot and fix problems with their Lync infrastructure. We also established a methodology that is used pro-actively in their environment to prevent problems in Lync communications internally as well as with external parties.

    A key learning is that CQM is a very good framework, but the value from it can be very limited, if the processes at a customer are not aligned to CQM and a proper Lync service mapping is not in place. Furthermore, the different IT teams at the customer needs to communicate very close about operation and maintenance of the IT infrastructure.

  • Troubleshooting the Microsoft Call Quality Methodology Scorecard for Lync Server

    [Updated August 8, 2014]

    In this post, I will show some of the common problems people run into when using the CQM Scorecard and how to fix them.

    Microsoft Visual Basic for Applications – Compile error: Method or data member not found

    You might see the error message shown below.

    The reason you see this message is most likely that you are using a version of Microsoft Excel prior to 2013. The CQM Scorecard require Microsoft Excel 2013. To solve the problem open the CQM Scorecard in Microsoft Excel 2013.

    Microsoft Visual Basic for Applications – Rune-time error '13': Type mismatch

    You might see the error message shown below.

    The most likely reason for seeing this error is that you have loaded the CQM results using the wrong decimal separator. The Scorecard needs to use the same decimal separator as in the CQM result CSV files. If it is not the same, Excel will load the PoorStreamsRatio as text and not as a number, and that causes the above error.

    You can check if this is the reason for the error by looking at one of the Trend sheets and the PoorStreamsRatio column. If you see formatting being a mixture of left and right aligned values as shown in the figure below, the CQM results were loaded with the wrong decimal separator.

    To discover which decimal separator to use open one of the Trend CQM result CSV files in Notepad and look at one of the lines. An example is shown below.

    "1/23/2014","18874","2380","12.6","Trend_3_Wired"

    The bolded yellow value shows you that the decimal separator is a dot. Therefore, you need to use Dot as the decimal separator, when loading the CSV results.

    You can also get this error when all the devices listed in Endpoint_0_Device sheet has AvgSendListenMOS below the target value set in the Scorecard. There is a bug in logic in this scenario, which will be fixed in the next release. Possible work-arounds are either to add one device with AvgSendListenMOS above target or lower the target.

    Different formats for date labels on trend charts

    You might see different format for date labels on charts. An example of this is shown below. You can see that the date format changes from 1/31/2014 to 02-02-2014.

     

    This happens if the CQM results are loaded with the wrong date format. This leads Excel to load the dates incorrect and that makes the charting incorrect.

    You can see if this is the case by looking in one of the Trend sheets. If you see the dates in the ReportDate column formatted as shown in the figure below the results were loaded with the wrong date format.

    To discover which date format to use open one of the Trend CQM result CSV files in Notepad and look at one of the lines. An example is shown below.

    "1/23/2014","18874","2380","12.6","Trend_3_Wired"

    The bolded yellow value shows you that the date format is MDY, because the date is shown as 1/23/2014. Therefore, you need to use MDY as the date format, when loading the CSV results.

  • Microsoft Call Quality Methodology Scorecard for Lync Server

    Yesterday we made version 2.2 of the Networking Guide available and it now includes the Microsoft Call Quality Methodology Scorecard for Lync Server. The Scorecard is used to implement Call Quality Methodology (CQM) as discussed in Appendix C of the Networking Guide.

    The new features in this version of CQM includes:

    • PowerShell script to run the CQM queries against the QoE database
    • The ability to use different conditions to flag an audio stream as poor
    • The ability to remove off-work days from the results and thereby removing outliers skewing the poor stream ratios
    • The Excel 2013 macro-enabled Scorecard which can load the CSV files generated by running the CQM queries, draw charts, show the Top 10 Issues for various categories and not least update the Scorecard with green/yellow/red status for the different CQM dimensions

    You can read much more about the Scorecard and how to use it in the Microsoft Call Quality Methodology Scorecard for Lync Server document included in the ZIP file located here. Enjoy!

  • Lync Call Quality Methodology sessions from various conferences

    At the Lync Conference 2014, Andrew and I did two sessions on Call Quality Methodology (CQM):

    At TechEd North America 2014 Andrew and Kent did a session on CQM. It includes a deep discussion on how to troubleshoot packet loss reported by CQM and demonstrates the scorecard we are working on.

    Enjoy! J