In a previous post we noted some stack-based vulnerabilities, such as MS08-067, that GS was not designed to mitigate due to the degree of control available to an attacker. However, other vulnerabilities such as the ANI parsing vulnerability in MS07-017 would have been mitigated if the GS cookie protection had been applied more broadly. The strict_gs_check pragma was mentioned as one way of achieving broader coverage.

One downside of the strict GS heuristic is that it is very aggressive. It will GS protect any function that has an address-taken local variable (even an address-taken integer). MSEC undertook some basic code analysis and identified that a number of functions currently protected with a GS cookie were actually provably safe. The idea was born to give GS an overhaul as part of the Visual Studio 2010 release – Enhanced GS:

1. Enhancing the GS heuristic to consider a greater scope of local variables worthy of protection

2. Implement a new optimization that would remove cookies identified as unnecessary.

Any cookie increase is bound to lead to performance questions due both to the direct overhead of the extra cookie checks and indirect impact on inlining other aspects of code generation and optimization. The goal was for the resulting scheme to be applicable system-wide – as for the original GS work – rather than be a targeted approach such as the strict_gs_check code pragma.

Enhanced GS heuristic in Visual Studio 2010

The highest priority was to update the heuristic to protect data structures such as the ANIHEADER structure in MS07-017. There have been other security bulletins where pure data structures such as these have been overflowed. For example consider the structure involved in the Office Publisher vulnerability in MS06-054:

typedef struct _STTBF
{
  ulong bMax;
  ulong ibstMac;
  ulong reserved3;
  ulong cbExtra;
  uchar fNoShrink;
  uchar wReserved2;
  ushort wReserved;
} STTBF;

Protecting all structures would likely be too expensive in terms of the number of extra cookies that would result. We decided to increase the scope of the structures that GS would treat as potentially vulnerable to only those structures that contained no members of a pointer type. It seems less likely that a component would read in a pointer value to local memory from an untrusted file, compared to it reading document/media or some other kind of data. Thus the ANI structure in the previous post and the STTBF Publisher structure above are both covered by the enhanced GS heuristic in Visual Studio 2010. However a structure defined as follows is not:

typedef struct _MYSTRUCT
{ 
  int bMin;
  int bMax;
  int bAverage;
  void *pSomePointer
} MYSTRUCT;

In a similar vein, whereas the legacy GS heuristic only protects arrays where the element size is 1 or 2 bytes (such as chars and wchars), the Visual Studio 2010 enhanced GS heuristic protects any kind of array as long as the element type is not of pointer type.

char myStringBuffer[256]; // protected
DWORD myIPAddresses[256]; // protected
HANDLE myPointerTable[256]; // not protected

There are also a number of additional criteria: an array needs to have more than 2 elements in order to qualify, any structure including an array of the kind described above would be protected, and so on. However, the two examples above capture the main new cases that the enhanced GS heuristic is designed to cover.

It is also worth mentioning that any variable considered as potentially vulnerable under the legacy GS heuristic is also considered potentially vulnerable under the new heuristic. By carefully designing the heuristic around past security bulletin history and in tandem with performance measurements from internal product teams in Microsoft, we believe this to be truly deployable as an application-wide mitigation.

Enhanced GS optimization in Visual Studio 2010

Just because a variable (due to its type or size) is considered potentially vulnerable does not mean that its use will necessarily lead to a vulnerability! This insight has led to a new compiler optimization that is planned for Visual Studio 2010. Specifically, the new optimization recognizes instances of safe variable usage and suppresses GS cookie emission in such instances.

For example the code snippet below would normally lead to the function being GS protected due to the presence of a character buffer.

STDAPI ConsumeData(BYTE *pbData)
{
  BYTE Temp[MAX];

  if (pbData)
  {
    ...
    memcpy (Temp,
            pbData,
            ARRAYSIZE(Temp));
    ...
}

Analysis of the use of the Temp buffer makes it clear that it can never be overflowed: its only use is in a memcpy instruction, and the number of bytes copied is bounded by the size of the array. In this case, the function does not need to be GS protected and as such can be safely optimized away.

Summary

Visual Studio 2010 introduces an enhanced GS heuristic that provides significant security improvements by increasing the scope of GS protection to a carefully targeted superset of those functions protected by the legacy GS scheme.

Of course the compiler change is just the first step. Any application developed using VS2010 is set to benefit from the improved security provided by enhanced GS. Inside Microsoft, MSEC have already started to work with several product teams so that once Visual Studio 2010 has shipped then they can start migrating their future releases to the updated compiler and take advantage of the enhanced GS capability. In the meantime, try out the upcoming Visual Studio 2010 beta and let us know what you think!

Links to related articles

MS08-067 and the SDL, SDL blog entry, Michael Howard, October 2008

Lessons learned from the Animated Cursor Security Bug, SDL blog entry, Michael Howard, April 2007

Hardening stack-based buffer overrun detection in VC2005 SP1, Michael Howard’s blog, April 2007

#pragma strict_gs_check , MSDN C/C++ pre-processor reference.

*Posting is provided "AS IS" with no warranties, and confers no rights.*