The Server and Cloud Community Lead Blog

A place to share my thoughts on community and what I am doing inside of Microsoft to drive robust conversations between our Product Development Teams and our customers.

Enabling a broad feedback channel to the Windows Home Server team and SBS team via MS Connect for RTM releases

Enabling a broad feedback channel to the Windows Home Server team and SBS team via MS Connect for RTM releases

  • Comments 9
  • Likes

So, how do you provide feedback to Microsoft today? What I am talking about when I say feedback is a bug or a suggestion like you would on a beta. I know that participating in a Beta like the Windows Home Server Beta is one way to get in your suggestions and bugs to the product team, but how do you do it on RTM product? For this conversation I want to focus completely on simply providing feedback to Microsoft to improve the quality of the product.

Now, I want to immediately remove one scenario from this discussion. The scenario is a bug that is impacting you today that you cannot get around and you need help. In these cases, you should always get help from either Microsoft Customer Support Services (CSS) or another Microsoft Partner or Consultant.

The methods that I know of that exist today for providing feedback to the Microsoft Product teams are the following:

  1. Newsgroup or Web Forum. You can post a suggestion or bug in a newsgroup or Web Forum and hope that the thread attracks enough attention that someone at Microsoft or someone like an MVP or a Microsoft Partner pick it up and do something about it. The problem with this technique is that is unreliable, and is inconsistent. The other problem is that it is not a communication technique that allows for the loop to be closed on the feedback. The customer is left with an empty feeling in their stomach because while they may have gotten the issue off their chest, there is no closure in most cases.
  2. Blog or Personal Web Page. Of course you can always get the issue off your chest by blogging about it. But again, for the same reasons as I mention above this is not a very reliable technique and the there is no closure. The only exception is when your blog or web site has a Big Audience. In these cases, usually issues raised on these very influencial community sites get Microsoft's attention. The funny thing is, usually these people already have an inside track and usually do not blog about it first.They usually just their contacts within Microsoft to get an issue raised. The problem with this technique is it really the voice of the customer or just a vocal minority. In cases where the feedback comes from an MVP or someone who is a very prominent member in the community, the feedback is usually spot on because these individuals have gotten pretty adept at triaginig everything they have read out on the net and been able to raise the issue that they see as the most pressing.
  3. As I mentioned above, you can always engage with CSS for a bug or suggestion even if the intent of the feedback you want to provide is to simply pass along an idea or an observation. The two options you have with CSS is either pay Microsoft for the case or send a letter. With the pay option you will get a full refund if your situation is indeed a bug, but in the case of a suggestion your only option is to send a letter. With this option CSS will research your bug all the way through and the benefit of this technique is that you do get closure most of the time. You will at least get a explanation of what Microsoft plans to do about the bug. The letter option is kind of like option 1 and 2 above. It makes it to the team, but there is no closure. It is kind of like MSWish used to be.
  4. So, last, but not least. In some cases, some teams are using MS Connect for enabling feedback with their customer base. The only two that I know of today is the Visual Studio Team and the Microsoft Operations Manager (MOM) team. I was responsible for the MOM team turning on MS Connect for the general public. What is interesting about this technique is that the two teams each have taken different approaches to this channel of feedback. In the case of MOM, the volume is so low, they simply roll the feedback up into their Sustaining Engineering team. In the case of Visual Studio, from what I understand, they have a team that helps triage and look at the feedback. They give feedback to the customers on what they are doing with the feedback and they also get the issues onto the teams radar.

So, here is what I am thinking. I would like to enable the last option of turning on MS Connect to the general public for Windows Home Server RTM and also Small Business Server R2 RTM this year.  So, why haven't I done it already you ask? I think it all comes down to bacisally 2  to 3 issues/points of discussion that are all related to one another and overlap in a number of ways;

  1. How do you prevent this solution from being a black hole? The fear is about setting expectations with the community. If you provide this channel to the community, how do set and meet the expectations? I think a lot of people on the product team side of the fence believe that if we enable this feedback channel, each and every issue needs to be responded to. I think this is a unfair expectation that the product team puts on itself. I do not think that the community actually realistically believes that we have the ability to simply staff enough people on our team to handle any and all feedback on a one for one basis especially in the case of our mainstream products like Windows Server and Office. I think Windows Home Server and SBS fall into the mainstream bucket. In the case of  the MOM product which I already mentioned is using Connect to capture feedback on their RTM product, they get about 5 pieces of feedback a week which actually scales for them where they can handle and respond to each piece of feedback on a one for one basis.
  2. How do you staff/fund it? This to me is a pretty interesting discussion if you actually manage the expectations of the customers from the first point above. If we told the community at large that we want your feedback and we are providing a channel for you to communicate this feeback, but we have a a few rules for this feedback would the community have an issue with it?
    • What if we said for example that we will only look at a bug feedback item that has a minimum of 10 votes, has an aveage rating of 4 or above and 5 validations at minimum? And that if you want to get the bug to meet this bar, you have to help us get the votes, rating and validations? You all already know how to do this today. You post a link to the bug on Connect on the web forum/newsgroup and/or blog and you ask people to weigh in on the feedback item.
    • Would the community really have an issue with this? We at least are providing a way for feedback to be logged, tracked, triaged, and responded to on a very large scale unlike any of the other techniques mentioned above.  The other important point is that the throttle can be adjusted. If the throttle is too wide open and we get overwhelmed we can always pull back and require more votes and/or a higher average rating, and/or a higher number of validations. Conversely, we could always open the throttle wider if we could handle a larger volume.
  3. What is the process for handling this feedback? Who is going to watch this channel and be on the hook to respond to the issues? I touched upon this before, but if we were to set the expectation of the customer, and if we could throttle the level of feedback that is coming in from the community, what do we do with the feedback? I think it is pretty simple from this point. If we meet the first two points above, the need for additional staff is not necessary. The existing Sustaining Engineering and Product teams should be able to absorb this additional channel with only minimal amount of adjustments to their process because the feedback is being throttled.

 

There are a number of side benefits of doing this:

  1. There is a single place where all community feedback lives.
  2. The loop can be closed for the issues that are looked at by the product team.
  3. The SE, CSS, and Product teams can look at this feedback at any time to gain additional knowledge of the issues that are effecting the community.
  4. In the case of SE, this could potentially shorten the time to resolution on an issue that came in from CSS. They be working on a case that they do not have enough information to go on. If they had a database of bugs that came in via the Connect channel there may find a bug that is very similar if not the same that had the additional information that could help them resolve the case faster.

So, what I need from you is a thumbs up/ thumbs down on this idea. What do you all think about this idea?

Thanks for listening.

Kevin Beares
Community Lead
WSSG (Windows Home Server, SBS, Centro)

 

 

Comments
  • PingBack from http://www.wegotserved.co.uk/2007/08/04/shaping-the-future-of-windows-home-server/

  • Whatever the approach.. every report, whether bug related or a suggestion/wish for some improvement, MUST be given careful thought and consideration. There should be no such criteria as “if it does not get 10 votes it will not be considered”… etc.. Voting should only be used as a guideline for the team to guage customer interest and not a measure of the useability or functionality or adviseability or practicality of a suggestion. An idea may get no votes because of visibility or poor subject wording, or unclear description wording … whatever, but may in fact be a very good idea that needs to be looked at.

    -Art (artfudd)..

  • Agreed, Art!  While I support Kevin's idea, I also want to see all reports for bugs or suggestions looked at, and the voting only used for verification of those who have had the same or similar problems for bugs or support of suggestions.  However, a 10 votes or higher criteria IMHO is very unrealistic for anything, as there are even many suggestions and bug reports in most betas that I have been involved with that have not seen that many votes, and yet are serious problems that have been fixed, so to me, setting the criteria at 10 or more is just not realistic, and sort of casts an less than optimistic view of the overall effort.  

    Jan :)

  • OK, let me clarify the use of 10 votes was a for example. The concern we have is that if the level of participation from the community is tremendously high, there is no way in heck that we would be able to resource this to realistically look at every issue. This is where the community must add it's value.  The rating / voting is key.  If there are not votes or rating, this means it has not been looked at and the community has not weighed in.  We need community help in order to understand the priorities of feedback.

    While I agree in principle that we should read any and all feedback, it is not realistic.

    So, how would you propose setting it up that respects the level of resources that have to be budgeted to look at the feedback.

    Now, because this is an RTM channel for feedback, we should not expect to get the level of feedback that we received during the beta, but given the level of participation that we had from everyone, it is not inconceivable that we could still get a larger that manageble level of feedback.

    Thoughts?

    Kevin

  • I think it is important to point out again that I am not talking about a beta of WHS. I am talking about the RTM version of WHS. It is also important to point out that I am not talking about break fix bugs. I am talking about feedback that you want to pass on to the product team.

    What I am ultimately trying to get at here with you all on this thread is how would you plan for an unknown volume? I completely agree that the Eutopian view of the world, you would staff according to demand and look at, triage, and respond to each item of feedbac that you receive. We do not live in this Eutopian view of the world. I live in the same world you all live in. We staff according to business needs and taking on any additional work load must be shouldered by the team in place. Short term and long term, you

    can make adjustments to your staffing if you can make a compelling enough of an argument and get the headcount that you ask for, but not always.

    Let's look at the current landscape one more time.... How do you, members of the community currently "Officially" submit a suggestion and or bug today against an RTM product? Now that the Vista Beta is over, how do you "Officially" submit a bug or suggestion to the Vista product team?????

    ........ Call CSS. That is it or at least that is all I know right now on

    how we have enabled feedback to the Product team for bugs and suggestions for RTM Products.

    What I am proposing as at least taking one step closer to consolidating feedback to the product team into one location where it will not get lost and it has broad public exposure.  With CSS it is strictly a one for one mapping and it is not visible to the general public. It costs money to staff CSS and this why they charge the going rate for a professional or consumer support incident and if the case is deemed a bug, the refund is given. However, if the case is not deemed to be a bug, the money is not refunded.

    We, the WHS team and SBS teams do not have any data on how hard we would get hit with feedback if we turned on this channel for our RTM product. We could be actually talking about 100 total feedback items in one month. We could be talking about 5 a week or we could be talking about 1k or more in a month. We do not know what level of feedback we could get. This is why I have to

    talk about putting a filter on this channel. How else could I possibly set an expectation with the community if we have no idea on what kind of traffic we are looking at? There is no way to reliably predict what the incoming rate will be. In our betas, we expect a lot of feedback and during the time of a beta we are all heads down working on that code base and treat all imcoming bugs from the public beta basically the same as we do with bugs that are discovered from our testers. Once we hit RTM, a vast majority of

    the product team moves towards developing code for the next version. So, we are not staffed at nearly the level we were when we were during our beta phase.

    Let me throw an analogy at you. What if you started up a coffee or a donut shop in a neighborhood that depserately needed a coffee or a donut shop? Now, you can do all the research in the world to ascertain how many people live in the area, look at how many competitors if there are any in that same area and take a really

    good stab at how much business you might get. In order to get a loan from the bank, you will have to do this anyway. So, you would at least have a really good clue on what the bottom of the barrel would look like.  However, the big thing you probably will not be able to guess is when will the amount of customers be greater than your capacity to service them. In my analogy, if the line for coffee or donuts is too long, you will chose to go elsewhere or come

    back later. We had a Krispy Kreme open up in Issaquah, WA a few years back and it was insane.  There was no way to get a donut unless you knew someone or if you were willing to stand in line for what seemed like an eternity to get service. Eventually, 3 months or so later, the volume dropped down as well as staffing adjustments were made to handle the volume, but initially

    the level of service from this fine establishment was a little bumpy.

    So, I hope you can see what I am trying to get at here. I am trying to plan for the un known and I want to get ideas on how you would do it given the resources I have available to me.

    Would this be better than what you have today or worse? How would you set expectations with the community?

    Thanks,

    Kevin

  • I wonder if it would be possible to enlist the help of MVP's and Insiders to filter the feedback, and pass the "best" suggestions to the product team? Just a thought.

  • I think that is a great idea. What is interesting is some features that are coming down the line with MS Connect that will allow us to enable moderator privileges on feedback. So, active members of the community like MVPs would be able to help triage feedback and maybe even have the ability to escalate certain issues that look important. I think what is important to keep in mind, is we would love the communities help is looking at triaging the feedback much like what already happens in the newsgroups and forums today. No one expects the product team to respond to every post in the newsgroup or forums today so why would the expectation be any different for a feebback channel like this? We do try to look at everything in the newsgroups and forums today, but responding would be an impossible task unless we had a limitless pool of resources.

  • I just want to add, I think the idea of such a channel is fantastic! And I agree that the product team can not be expected to have time to review, or respond to every post, when the potential for ahuge reponse is very real.

  • Nice to see this post. I'm glad someone in Microsoft sees that the organisation has no process in place to capture feedback for bugfixes or feature enhancements in a clear and consistent way.

    The fact that CSS wants to charge me for providing Microsoft with a fix or workaround for a bug is just plain insulting. I find CSS only useful for obtaining limited release hotfixes. Extended turnaround times and slow moving escalation processes are only of use when the problem is systemic and non-urgent - of which there aren't many.

    The use of MS Connect to obtain feedback is a good start, but this is ad-hoc and relies on product manager buy-in. The culture needs to be well and truly set by senior management if Microsoft really wants to gain feedback across all its product range. This may be an issue with resources though, as feedback levels could be very high.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment