While MSVR’s core mission has stayed the same, in the two years since the program’s inception, the program has grown to take on other challenges as they’ve presented themselves. We’ve done so both because those challenges needed to be addressed and because the coordinated nature of MSCR was the best tool for the job. Some examples of those challenges include MSVR’s role in coordinating several large-scale cross-industry vulnerabilities identified by other external security researchers, such as the DNS Rebinding issue disclosed in 2009 by Dan Kaminsky. Microsoft and MSVR also faced a unique situation in coordinating the remediation of a vulnerability in Microsoft’s own code that affected vendors across the ecosystem as a result of the ATL Killbit Bypass vulnerability discovered by Ryan Smith and David Dewey. .
MSVR has also assisted in reporting to vendors instances of zero-day attacks against their products, which were discovered as a result of the telemetry and other threat-detection resources built into the fabric of the MSRC’s response process. MSVR is not intended to serve as a CERT entity; however, we felt leveraging MSVR for these cases was imperative because a vulnerability affected code in both Microsoft and other vendor products.
So what have we learned? For starters, the security maturity of vendors across the ecosystem varies wildly, as you might expect. Sometimes just finding the right organizational entity or person inside a major corporation to report a vulnerability to can be the toughest part of the mission. In other instances, MSVR has had to take on the role not just of vulnerability reporter but also of response educator, as vendors were unaware or unsure of the best ways to communicate the information their customers needed about the vulnerability in their product. On more than one occasion, we encountered statements and questions similar to “what should we put in an Advisory?” -- or, even more challenging, the assertion that “we do not believe this issue warrants an Advisory or public release.” That second response in particular requires MSVR to painstakingly go through the merits of educating the vendor about why notifying customers is important. Touchy work, but worthwhile.
(Interestingly, in the last two years, it has become clear that nothing serves better as the universal vulnerability translator than calc.exe. Sometimes all the man-hours and time spent packaging up what appears to be a cohesive and solid vulnerability report to a vendor devolves to utter exasperation when the response received is “Not a vulnerability”… that is, until the follow-up response from MSVR is the PoC needed to pop calc. Shortly thereafter, it seems like we are suddenly cooking with gas in the MSVR-to-vendor conversation.)
We have also learned that engineering processes across the ecosystem are unique to each company, and each comes with its own challenges that can make engineering a fix complex. Consequently, the timelines vendors require to engineer a fix is unique relative to each vendor and each product in order for them to get it right. To put it more bluntly, a universal time-to-fix mandate from MSVR (or any other vendor) is just not realistic. The speed with which the vendor of a cloud service offering can move to address a cross-site scripting vulnerability doesn’t compare to the time required to address a vulnerability in client-side code in a more traditional boxed product. This statement still holds true even when it is the same vendor who is responsible for both boxed and cloud version of a vulnerable product.
Finally, it has become clear to us that sharing hard data with vendors about the risk posed to our mutual customers can be key to getting traction. Throughout the last year we have leveraged the MSRC’s telemetry-gathering capabilities, both internal to Microsoft and via our MAPP partnerships, to help provide clarity to vendors during situations in which we believed our mutual customers to be at real risk from zero-day vulnerabilities of which they may not have been aware. This has required us to communicate clear and validated data that vendors could use to better prioritize and expedite their engineering processes relative to the trade-offs that they have to make.
It has definitely been a wild ride over the last two years – interesting times indeed. Along the way, we’ve tackled many of same problems that security researchers face when reporting vulnerabilities to vendors, and we’ve learned valuable lessons that will continue to not only make MSVR better, but also that we can channel back to our colleagues in the MSRC. We’ve forged stronger relationships with other security response teams and vendors throughout the ecosystem as well as with the talented security researchers both internal to and outside of Microsoft.
We’re thankful to all of the researchers and MSVR partners, such as MMPC and various MAPP participants, who have chosen to work with us and provided us with needed assistance when called upon. The next two years will hold even bigger challenges and bigger opportunities for MSVR, and it will also come with challenges I am sure I cannot begin to conceive of yet. Interested? I definitely encourage you to check out the whitepaper. And if you’re here at Black Hat drop by the Microsoft booth and let’s talk.
Adrian Stone
Today on the MSRC blog, Matt Thomlinson, General Manager of Trustworthy Computing Security, announced our new philosophy on Coordinated Vulnerability Disclosure. I wanted to provide some context and history on how this came about. This post is about changing the way we at Microsoft talk about some familiar disclosure concepts, and is meant as an introduction to how Microsoft would like to engage with researchers. We’re opening up a dialogue with the community here, and we welcome your feedback.
Responsible Disclosure (RD), Full Disclosure (FD) -- everybody has an opinion, and each believes that their way is the best way to keep users safe. For background, one general definition of RD as most vendors define it is that the issue is reported privately to the vendor *and no one else* until the vendor issues a patch. In contrast, proponents of FD provide all vulnerability details to everyone at the same time, a move designed to make vendors provide updates faster.
Needless to say, most vendors including Microsoft are in favor of RD, while finders fall across the spectrum from FD to RD. Ultimately, we are all part of a virtual security team with the common goal of making the Internet safer and protecting the people using it – it’s good to remind everyone that we’re on the same team, and we should keep the dialogue open, even when we disagree.
The term Coordinated Vulnerability Disclosure was first introduced to me by Jake Kouns of OpenSecurityFoundation.org, when we spoke at great length after I was on a panel at RSA on Responsible Disclosure. WeldPond (AKA Chris Wysopal, CTO of Veracode) recently tweeted: “We need to start calling working with the vendor ‘Coordinated Disclosure.’ I agree that "Responsible" is too loaded.”
The concept of making the name more descriptive makes perfect sense to me, since the term “responsible” can be subjective to so many. Even the ISO draft standard that was originally titled “Responsible Vulnerability Disclosure” is now called “Vulnerability Disclosure,” signaling that researchers, vendors, and (gasp!) even policy makers agree that the old term is more subjective.
The intention of RD was that it was designed to be a fair way to negotiate between researchers and vendors around vulnerability reporting and resolution. However, that has resulted in much debate, between vendors and finders. So, how do we move past this debate towards providing a better solution?
Responsible Disclosure should be deprecated in favor of something focused on getting the job done, which is to improve security and to protect users and systems. As such, Microsoft is asking researchers to work with us under Coordinated Vulnerability Disclosure, and added some coordinated public disclosure possibilities before a vendor-supplied patch is available when active attacks are underway. It uses the trigger of attacks in the wild to switch modes, which is an event that is objectively observable by many independent sources.
Make no mistake about it, CVD is basically founded on the initial premise of Responsible Disclosure, but with a coordinated public disclosure strategy if attacks begin in the wild. That said, what’s critical in the reframing is the heightened role coordination and shared responsibility play in the nature and accepted practice of vulnerability disclosure. This is imperative to understand amidst a changing threat landscape, where we all accept that no longer can one individual, company or technology solve the online crime challenge.
Here are the simple tenets of Coordinated Vulnerability Disclosure as we envision them.
Step 1: Keep it Private, Keep it Safe
● Reporting: Report the issue to the vendor, or to a CERT-CC or some other coordinator you trust who will report to the vendor privately, or sell it to a service that will.
● Communication and timelines: Under CVD, just the same as in RD, finders and vendors should try to agree to a timeframe for fixing the issue. Complex cases may take longer to fix, and Microsoft will be as transparent about our investigation with finders as we can be, to let them know where we are in the investigation and resolution process. We appreciate finders being flexible when we share information with them about why a fix may take longer than the finder thinks it should.
● Status updates: Also as in traditional Responsible Disclosure, under CVD Microsoft will provide timely updates and target dates for resolution so that a finder is aware of the case status.
● Alternative to FD when a vendor is not responding at all: In some circumstances, a vendor may be unwilling or unable to respond to a vulnerability report, which is what advance security advisories are for – advisories published with limited details and no Proof of Concept, plus mitigations and workarounds. Finders can try that before resorting to publishing full details if they can. Some vulns won’t lend themselves easily to this method, but the point is to try.
Step 2: Hurry Up and Wait
Vendors and many finders know there has to be a balance between speed and quality. For Microsoft, even a 1% test failure rate could affect millions of our customers, so we take testing for functionality impact as seriously as we do the testing to make sure the update comprehensively addresses the vulnerability.
Ideally, both vendors and finders should work diligently to find a solution that will keep customers safe. If finders are only interested in working on the attack, that’s ok too, as long as they give the vendor a chance to do their investigation, engineering and testing.
Working together on the update, sharing ideas, and testing each other’s ideas is sensible.
Step 3: Coordinated Public Disclosure
Coordinate public release happens, ideally, when the vendor releases the update. In the case of publicly verifiable active attacks, details may be released prior to an update being released, with emphasis on giving details to protection providers.
For finders who still believe that Full Disclosure is the best way to protect users, we respectfully disagree, but we still want to work with you if you’re willing. We’d encourage folks who support FD to still contact us, as we can then attempt to coordinate release of information with protections that are available. Of course, we still don’t think this is the best method, because the vast majority of customers will only be protected with an update – but we believe that even this level of coordination is definitely better than none at all.
For example, CVD is how we will now handle things when we’re the finders. When Microsoft finders discover issues in third party products, they can use the Microsoft Vulnerability Research Program (MSVR) to report the issues to the vendor. If attacks start in the wild, we may potentially release vulnerability details through the Microsoft Active Protections Program (MAPP) to AV/IDS/IPS providers, or issue a third party killbit in the case of vulnerable Active X controls. We would in all cases coordinate with the affected vendor whenever possible.
So that is Coordinated Vulnerability Disclosure in a nutshell - a renaming of Responsible Disclosure that provides expectations and a process for Microsoft and researchers to work together without either party clouding the discussion with a term that is easily misinterpreted, even in cases where disclosure philosophies may not be entirely in sync. We even want to work with Full Disclosure proponents whenever possible to arm protection providers ahead of attackers.
Not all roles in disclosure have been covered here, so stay tuned for more as we gather feedback from the community. I would like to thank the following people and organizations for their review on this concept, and I welcome further comments on this by the community, including researchers, vendors, coordinators, and users. -Katie Moussouris
Jake Kouns, Open Security Foundation
Steve Christey, CVE Editor, MITRE
Avishai Avivi, Juniper Networks
Bruce Monroe, Intel PSIRT
Pete Allor
Toshio Miyachi, JPCERT Coordination Center
Brian Martin, Tenable Network Security
Art Manion, CERT Coordination Center
Damir Rajnovic (Gaus), Cisco
Dan Kaminsky, Chief Scientist, Recursion Ventures
Mike Caudill, Cisco PSIRT
Jeremiah Grossman, WhiteHat Security
Jayson Jean, iDefense-VeriSign
Ryan Permeh, McAffee
Cassio Goldschmidt, Symantec
Arturo ‘Buanzo’ Busleiman, Buanzo Consulting / ArCERT and ONTI Security Advisor
Andy Steingruebl, PayPal
Dino Dai Zovi, Independent Security Researcher, Trail of Bits
Chris Wysopal, CTO Veracode