Tom's Blog

Musings on Windows, Graphics, and Technology

Introducing the Microsoft Direct2D API

Introducing the Microsoft Direct2D API

  • Comments 149
  • Likes

One of the challenging aspects of working on a product development team is that you often have to keep the subject of your work secret for various reasons. I’m a Dev Lead in the Windows Desktop Graphics organization, and my team has been working quietly for the past year on an exciting new graphics technology – Direct2D –that was announced recently at the Professional Developer’s Conference (PDC) in Los Angeles.

 

Direct2D (or D2D) is a native immediate-mode 2D graphics rendering API that is built on top of Direct3D, and offers some compelling performance and visual quality improvements over GDI and GDI+:

·         Hardware-acceleration

·         Software fallback when hardware isn’t available

·         Interoperability with GDI, GDI+, and Direct3D

·         ClearType text rendering

·         Per primitive antialiasing

·         Device-independent coordinate system

·         Draw and fill commands for geometries (lines, curves) and bitmaps

·         Solid color, linear, radial, and bitmap brushes.

·         Rendering to intermediate layers

·         Rich geometry operations (eg. unions, intersections, widening, outlining, etc)

·         Server-side rendering

·         Preservation of rendering state

 

Since Direct2D is now public, I can finally talk about it, and this blog is intended to provide a forum for sharing information and tips. I encourage you to send me your feedback and comments, and I will share them with the rest of the development team. While I can’t promise that we’ll act on every piece of feedback, we will take everything that you submit seriously, and we will use the data to better understand your development priorities.

Back to the Future

Do we really need another 2D API? To answer this question, we need to backtrack a little bit and discuss some ancient graphics history. Windows applications continue to rely overwhelmingly on GDI/GDI+ 2D graphics technologies that were designed for hardware that shipped in the 1990’s. Hardware in that era was optimized largely for simple, fixed-function scenarios (eg. BitBlt, StretchBlt, DrawRect , etc). Modern graphics hardware has evolved dramatically since that time. Much of the processing that used to occur in software has now migrated down to the video card’s Graphics Processing Unit (GPU) in the form of programmable pixel and vertex shaders; which increase performance by moving pixel data closer to where it will be manipulated & consumed, in massively parallel (SIMD) fashion. These changes have staggering implications for graphics applications. Expensive pixel-processing operations can be offloaded from the CPU to the GPU which, in turn, frees the CPU to take on other useful tasks, such as creating rich UI, driving animations, doing computations, etc. It also allows applications to scale up primitive counts dramatically with minimal CPU impact.

One of the first proposals on the table was to update GDI/GDI+ to take advantage of these technologies; however, as we surveyed the requirements, we ran into some fundamental limitations that would have made the effort very difficult. For example, GDI/GDI+ are limited to 32 bits-per-pixel, and there’s a ton of stuff – both public and internal – that would need to be revamped or extended (eg. RGBQUAD, COLORREF, PALETTEENTRY) to support high dynamic range pixel formats beyond 32 bits-per-pixel. Similarly, there are quality and performance issues: modulo ClearType text, GDI can only produce aliased graphics (jaggies) and, while GDI+ can render anti-aliased graphics, it can only do so in software. GDI uses an integer coordinate system, so it can't take advantage of improvements related to sub-pixel positioning. Another fundamental problem is that GDI/GDI+, and Direct3D are wholly separate graphics subsystems, with their own formats, memory structures, and synchronization constructs. Getting all of these moving parts to work in unison would have been a huge challenge, to say the least, so we went back to the drawing board to consider an alternative proposal:  to create a new 2D graphics stack on top of Direct3D -- which would interoperate with the latest Direct3D hardware -- while also providing a migration path for GDI and GDI+ applications.

When we compared the development costs of these two proposals, it became clear that creating a new graphics stack would take far less time, would avoid destabilizing the huge number of GDI and GDI+ applications, and would enable us to provide a more cohesive, interoperable, and forward-thinking API. So, we embarked on building the new API, which went through several geeky name changes, until it emerged from the primordial acronym-soup as Direct2D. This isn’t surprising, given that my team is part of the same group that produces Direct3D. It was a natural fit.

Basic Goals

 

We started with some fundamental goals for Direct2D:

 

·         Native Code – Should be accessible to the widest possible community of developers. Minimal footprint for native code callers, and also callable by managed code.

·         Lightweight COM – Should use C++ style interfaces which model Direct3D usage. No support for proxies, cross-process remoting, BSTRs, VARIANTs, COM registration (e.g. the heavyweight stuff).

·         Immediate Mode – Simple BeginDraw/Draw/EndDraw semantic. No expensive retained tree structures.

·        Hardware – Should leverage available hardware for rendering.

·         Scalable – Pay-for-Play. Performance and memory usage should scale as linearly as possible while primitive counts increase. No hidden costs. Hardware improvements should scale performance accordingly.

·         Policy-Free – Should impose as little policy as possible on callers; that is, it should seek to do no more (and no less) than what the caller expects it to do. Minimal amount of abstraction.

·         Software Fallback – Should automatically fall back to software when hardware isn’t available (eg. session 0, server rendering scenarios).

·         High Quality Output – Should produce superb anti-aliased and aliased content. ClearType for text.

·         High Performance – Should leverage GPU hardware as much as possible and minimize CPU usage.

·         Interoperable – Should permit rendering to and from a Direct3D surface, as well as to and from a GDI/GDI+ device context (HDC). Serialization of content to and from surfaces and device contexts must be explicit and predictable. Should work seamlessly with other native Windows technologies (DirectWrite, Windows Imaging Codecs, etc).

·         Direct3D 10.1 – Should utilize D3D10.1 (and/or D3D10Level9) for hardware support.

·         Device-Independent Coordinate System – Should allow applications to be written that automatically handle DPI changes and differences.

·         Rich Geometry – Should provide a rich set of functions for both defining geometries and performing common geometric operations (eg. combining, intersecting)

 

 

 

PerfDemo Chomp

ChartDemo

 

Direct2D Team Members

I want to thank the following folks for their tireless passion and dedication to making D2D possible:

Mark Lawrence, Chris Raubacher, Tom Mulcahy, Anthony Hodsdon, Miles Cohen, Ben Constable, Leonardo Blanco, Alexander Stevenson, Megha Jain, Kam VedBrat, Andy Precious, Brett Bloomquist, Bilgem Cakir, Chris Chui, Bob Brown, Samrach Tun, Sriya Blanco, Jason Hartman

Furthermore, I’d like to acknowledge the following folks for understanding the importance of this new technology and funding the effort:

Anuj Gosalia, Anantha Kacherla, Jeff Norris, Todd Frost

Additional Materials

Watch the Platform Overview PDC session: http://channel9.msdn.com/pdc2008/PC04/

Watch the Direct2D and DirectWrite PDC session: http://channel9.msdn.com/pdc2008/PC18/

Also, I'm posting a Direct2D whitepaper that was distributed at the PDC. Future posts will drilldown into interesting code examples.
 

More Direct2D Blogs

Tom Mulcahy -- Tom is a Software Design Engineer on the Direct2D team, and owns a good swathe of infrastructure (bitmaps and texture management, WIC interop, clipping, etc). Prior to Direct2D, Tom worked on WPF and is widely regarded as our resident video guru.
http://blogs.msdn.com/tmulcahy/default.aspx


Mark Lawrence -- Mark is a Senior Software Design Engineer on the Direct2D team, and driving force behind the design and philosophy of the Direct2D API. He brings a ton of experience from his previous work on WPF, Windows Print architecture, and private industry.
http://blogs.technet.com/MarkLawrence/

 

Ben Constable -- Ben is a Senior Software Design Engineer on the Direct2D team, and implemented all of the Direct3D and GDI Interop functionality within Direct2D. He is a self-described "graphics nerd" who brings  considerable experience in 2D and 3D rendering to the team, and has contributed to numerous Microsoft products (Visual Studio, Windows, Office, SQL, etc).

http://blogs.msdn.com/bencon/

 

Attachment: PDC08_Introducing Direct2D.docx
Comments
  • Questions from a reader:

    How does it relate to the managed code APIs?

    Direct2D is an unmanaged (native) code API. No part of it is managed.

    Will we see a Managed Direct2D, and another transition period during which the unmanaged and managed APIs are not in synch? 

    This is an interesting idea, and we are discussing the possibility of a managed wrapper. Stay tuned for that.

    Or should we regard Direct2D as WPF re-surfacing in the unmanaged world?

    Direct2D and WPF are targeted at different customers. Direct2D is a lightweight native code API which uses an immediate-mode calling pattern (like Direct3D), and will be used primarily by native C/C++ developers. WPF is a managed code framework which uses a retained-mode calling pattern, and will be used primarily by .NET developers.

  • "Server-side rendering"

    Does this mean support for RDP?

    Also, are there any accessibility issues at this point?  

    And, what will it take for broad adoption?  Unfortunately, there seems to be a dearth of WPF apps.

    I don't know D2D would help, but currently many of our apps are quite inconsistent about their ClearType handling--sometimes ClearType is applied to text, but then in another dialog (or even in the same dialog!), you can clearly see that no font smoothing is applied.  Is there any way existing apps can leverage D2D (without being rewritten), or is this just wishful thinking on my part?  Could GDI/GDI+ commands somehow be mapped onto a D2D renderer, gaining of its benefits (or at least some)?

  • "Server-side rendering" Does this mean support for RDP?

    This is a two-part answer. Direct2D is built on top of Direct3D 10.1 and, since D3D10.1 provides both bitmap- and primitive-level remoting via RDP, it does mean support for RDP.

    But there's another point to made here, too. Server-side rendering also means that you can render high quality anti-aliased content using Direct2D in server environments where, for example, you would not be able to use Direct3D. We will automatically fall back to software in those scenarios. And we are considerably faster than GDI+.

    Also, are there any accessibility issues at this point?  

    Can you be more specific? We are a 2D rendering API. Other than text, we have no policy regarding layout.  

    And, what will it take for broad adoption?  Unfortunately, there seems to be a dearth of WPF apps.

    Developers should use whatever tool is appropriate for their needs. If you're writing native C/C++ code, and you need to do high-quality 2D rendering with hardware acceleration on Win7, you should use Direct2D. If you're using managed .NET code, you should use WPF.  The technologies are not in competition.

    I don't know D2D would help, but currently many of our apps are quite inconsistent about their ClearType handling--sometimes ClearType is applied to text, but then in another dialog (or even in the same dialog!), you can clearly see that no font smoothing is applied.  

    Direct2D uses DirectWrite typography APIs to do font management, layout, and glyph coverage. Font smoothing is applied consistently using DWrite; in fact, DWrite goes further than previous ClearType implementations by doing y-axis smoothing, which yields unparalleled text quality.

    Is there any way existing apps can leverage D2D (without being rewritten), or is this just wishful thinking on my part?  Could GDI/GDI+ commands somehow be mapped onto a D2D renderer, gaining of its benefits (or at least some)?

    Yes, I'm glad you asked. Direct2D supports interoperability with both GDI/GDI+ and Direct3D. So, for example, you could do one of a couple things:

     1. You could create a Direct2D render target; then, whenever you need to render GDI content, you QI the render target for IGdiInteropRenderTarget, call GetDC on that interface, BeginDraw, render the GDI stuff to the resulting HDC, and EndDraw.

    2. You create a DC render target, call BindDC to your GDI HDC, BeginDraw, draw the D2D content, and then EndDraw.

     I'll discuss GDI/GDI+ interop in more depth in an upcoming post. But, to answer your question, it's built-in. You don't need to rewrite your app to use Direct2D.

  • Very cool - A lot of familiar names on the "making it possible" list :)

    A few questions:

    - I read the whitepaper and saw the integration with WIC. However, what is unclear is can I pass WIC a ID2D1Bitmap to decode directly into (potentially a HW decode)? If not, is this planned for Windows 7? I am a very interested party :)

    - Any plans on the Windows 7 team introducing a retained mode API?

    - Also curious what "Server side rendering" means?

    - Does the API remote automagically through RDP? If so, what is the compatability story: ie. Will it remote from Windows 7 -> Vista client?

    - Related to the previous question: What version(s) of D3D does this run on? Will it be backwards compatible? ie. Can I run it on Vista?

    Thanks!

  • Very cool - A lot of familiar names on the "making it possible" list :)

    Rajat ... Rajat ... the name sounds very familiar... ;-)

    I read the whitepaper and saw the integration with WIC. However, what is unclear is can I pass WIC a ID2D1Bitmap to decode directly into (potentially a HW decode)? If not, is this planned for Windows 7? I am a very interested party :)

    WIC did a fair bit of prototyping of decoding in hardware, but the metadata, TIFF, and progressive decode work was higher priority. Interesting perf note: We compared doing Map/decode/Unmap on a D3D surface to decoding into a system memory buffer and calling UpdateResource on a D3D surface -- and found that the latter was actually faster.

     - Any plans on the Windows 7 team introducing a retained mode API?

    Not in Windows 7. 

    - Also curious what "Server side rendering" means?

    It means we can run in server environments using software fallback.

    - Does the API remote automagically through RDP? If so, what is the compatability story: ie. Will it remote from Windows 7 -> Vista client?

    There are 2 types of remoting a Direct2D app via RDP: bitmap remoting and Direct3D primitive-level remoting. Look at the D2D factory and render target properties for details on how to specify one versus the other.

    - Related to the previous question: What version(s) of D3D does this run on? Will it be backwards compatible?

    It runs on Direct3D 10.1, which includes so-called 10Level9 functionality -- which allows you to run D3D10.1 apps on a D3D9 driver.

    ie. Can I run it on Vista?

    We are discussing downlevel support now. Keep posted. 

    Thanks!

    You're welcome. :)

  • Hi Tom,

    I looking forward to managed wrapper, make sure it is a good one. And I have a couple of questions, too:

    1. Will there be support for video playback.

    2. Will it run on Windows XP? I assume not...

  • 1. Will there be support for video playback.

    Yes, with certain caveats. Since we interop with Direct3D, you will be able to take frames (D3D surfaces) from a DirectShow filter graph, and render them using D2D. Keep in mind that DirectShow currently uses Direct3D9, while D2D uses Direct3D10.1, so you will need to use a Direct3D9 shared surface. We are currently working up a sample that shows how to do this. Stay tuned.

    2. Will it run on Windows XP? I assume not...

    Not currently. We've had a number of customers asking for XP, and we're still evaluating downlevel support. As above, stay tuned.  ;)

  • 1. If you create a sample that really really works and even better, a managed wrapper - that'd be awesome.

    2. XP is important in gaming industry. Not the one for the PCs but rather the one that delivers games on cabinets (i.e. casino machines).

  • 1. If you create a sample that really really works and even better, a managed wrapper - that'd be awesome.

    Samples will be posted relatively soon. Not sure about the availability of a managed wrapper. We're still discussing that. 

    2. XP is important in gaming industry. Not the one for the PCs but rather the one that delivers games on cabinets (i.e. casino machines).

    I will pass your feedback on to my management. Thank you for letting us know what you need/want. :)

  • Questions posed in email from a reader: 

     Hi, I just read your blog post introducing Direct2D, and it looks really interesting.

    I have a few questions though:

    1. When can we expect to see it? The Nov 2008 DXSDK is probably too soon to hope for.

    The bits will be available in prerelease form to MSDN subscribers and PDC attendees. We tried to hit the November DX SDK date, but there wasn't enough time to get all of our dependencies ready. I'm trying to get clarity on the actual public release date. Will let you know. 

    2. How much control do we have over how it renders the 2-d content?

    It's a fairly comprehensive 2D graphics API. You have a ton of control.

    3. Can we use programmable vertex/pixel shaders with it? In our current graphics engine, we have a number of shaders specifically for 2-d sprite quads that render in various special ways.

    Yes, you can, but you need to do it via Direct3D interop. The API does not directly expose pixel/vertex shaders. I'm making a note to create a sample which shows how to do it.

    4. What is this Direct3D10Level9 that you are talking about?

    D3D10Level9 is a special adapter for D3D10 which allows it to use D3D9 drivers. So, for example, if you only have a D3D9 driver, D3D10 could still run -- which greatly broadens the reach of D3D10. It will be available on Windows 7. 

    5. When Direct2D is released to the public, could the dx sdk include a sample showing how to use Direct2D for rendering a simple gui over a Direct3D app?

    We will have a sample which shows you how to do this. But not in the Nov DX SDK.

  • Hey thanks for posting the answers to my questions. Those samples would be great to have.

     

    I'm curious still about this Direct3D10Level9 adapter. What exactly is the point of an adapter to let D3D10 use a D3D9 driver?

     

    It’s all about reach. There are a fairly broad number of D3D9 drivers available on the market right now.  Xbox360 runs on D3D9, as do most current PC games. D3D10 is gaining adoption, but it will take time. The advantage of providing a way to use D3D9 drivers from D3D10 is that it allows developers to code to a single API – D3D10 – so that they don’t have to write two sets of code.

     

    Since, you certainly wouldn't be able to use any of the new D3D10 features, as the D3D9 driver certainly wouldn't have a clue about how to implement them.

     

    That’s not entirely accurate. There is a reasonably broad subset of the D3D10 features which will run on D3D9; albeit with differing performance and quality characteristics. No, D3D9 doesn’t do everything, but it does a lot. The D3D team has done an excellent job in trying to match feature sets of the two APIs via D3D10Level9.

     

    Any D3D10 level hardware is certainly going to have a D3D10 level driver for it, or its D3D10 features are useless. Particularly, in regards to Windows 7, as I imagine it needs new/modified drivers to be written for it anyways. Unless you are planning to use Direct2D in the Windows 7 gui. But, you still wouldn't be able to use any D3D10 specific features, so, why not write a D3D9 version of Direct2D instead? (which would also enable Xp support and support for old (but still commonly in use) hardware)

     

    Like you, we, too, are customers of D3D10 and D3D10Level9. It allows us to code to a single API, while still getting all the perf benefits of D3D10.

     

    Or unless you are planning on releasing D3D10 for Xp, but, that seems extremely unlikely due to the massive differences between Vista and Xp, and is contrary to everything I've heard from Microsoft with regards to that.

     

    There are no plans at present to port D3D10 to XP. The driver models are very different.

     

    A few more general questions for you (feel free to answer on your blog):

    1. You mentioned support for D3D10.1, but, what about regular D3D10? Since there's no support from Nvidia for D3D10.1, what happens there?

     

    D3D10.1 drivers are a logo requirement for Win7.  

     

    2. Will there be D3D11 support as well? Or will that require Direct2D2?

     

    D3D11 will provide compatibility with prior versions.

     

    3. Will there be support for using Direct2D with Direct3D9?

     

    Yes, to a point, but the support is essentially provided through D3D9 shared surfaces. We had a earlier question about how to render video decoded into D3D surfaces. We’ll have a sample which demonstrates how that works.

     

    4. Compared to rendering screen-aligned quads, how does Direct2D perform?

     

    There’s no quick answer to your question. It depends on a lot of variable factors: what kind of hardware are you using, what is the driver overhead for state changes, how many drawing primitives are you trying to perform, etc. Similarly, since we continue to make perf advancements in the D2D rendering code, what you see today will not be what you see in the final RTM release.

     

    5. Interally, does Direct2D work by generating and rendering screen-aligned quads, or is it more of a true bitblt DirectDraw style rendering?

     

    I can’t discuss internal implementation issues.

     

    6. Is this the end of ID3DXSprite and ID3DXFont?

     

    Not yet. D3DX will continue to live on. But, like everything else, there were undoubtedly be changes to D3DX. Time and technology march onward.

     

  • For those who are wondering how to create a Direct3D10 or Direct3D10Level9 device.

     HRESULT CreateD3DDevice(
       __in IDXGIAdapter *pAdapter,
       D3D10_DRIVER_TYPE DriverType,
       UINT Flags,
       __deref_out ID3D10Device1 **ppDevice
       )
    {
       HRESULT hr = E_FAIL;

       static const D3D10_FEATURE_LEVEL1 levelAttempts[] =
       {
           D3D10_FEATURE_LEVEL_10_0,
           D3D10_FEATURE_LEVEL_9_3,
           D3D10_FEATURE_LEVEL_9_2,
           D3D10_FEATURE_LEVEL_9_1,
       };

       for (UINT level = 0; level < ARRAY_SIZE(levelAttempts); level++)
       {
           hr = D3D10CreateDevice1(
               pAdapter,
               DriverType,
               NULL,
               Flags,
               levelAttempts[level],
               D3D10_1_SDK_VERSION,
               ppDevice
               );

           if (SUCCEEDED(hr))
           {
               break;
           }
       }
       return(hr);
    }

  • hello.

    Great post.I expect also future post.

    I have an unclear thing, does D3D10 GPU run as D3D10Level9 device for D3D10.1?

  • does D3D10 GPU run as D3D10Level9 device for D3D10.1?

     No, D3D10Level9 is only used in conjunction with a D3D9 driver.

  • What about interop with WPF? With .NET 3.5 SP1, it can work with IDirect3D9[Ex] via the D3DImage WPF class.

    Is the software rasterizer optimized for multicore? Or, is it easy (and cheap!) to do things like banded or tiled rendering so that I can multithread it myself? Are there restrictions with using an object (e.g., paths, brushes) from multiple threads, which is a crippling performance issue in GDI/GDI+?

    Also, I bet if a server wants to do a LOT of rendering, then they will want a GPU sitting in there to help out. So, any support for GPU accelerated rendering in session 0? Especially for these: http://www.nvidia.com/page/servers.html

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment