The "U" Word

Making Windows safe for Unix people since 1995


Turning a Unix App into a Web Service using SFU and .NET - Part 1

  • Comments 1
  • Likes

For the last couple of years, the SFU and MSUM teams have shown a demo that takes a Unix application, ports it to Windows using Services For Unix, then turns the app into a COM server and into a Web Service. I've been trying to find the time to write the thing up through formal communications channels at Microsoft for at least the last two years.

Some people have gotten tired of waiting. In particular,  pleloup most recently asked the question.

Let me start by explaining roughly how this works. Later posts will go into the (slightly) gory details.

The most important thing to remember about building apps using the Interix environment in Services For Unix: you cannot mix Unix and Win32 calls in a single process. If the app makes Interix calls (fork, exec, all those other nasty Unix-isms that are hard to emulate through Win32, etc.) then it can't use Win32 calls. This is an architectural design limitation of the Windows “user-mode environment subsystem” model which Win32 and Interix use.

If you think about it for a second, the limit makes a lot of sense. For example, no Win32 API has semantics defined for the fork() operation. Behavior is documented for CreateProcess(); which things are inherited by the new process, which aren't. But fork() is quite different from CreateProcess() and the defined semantics for the latter are insufficient to describe the former. Or try mixing Unix signals with Win32 structured exception handling. When you consider all the tiny little details of the programming models of the two worlds, they're just not compatible. The Windows “subsystem” model puts a high wall between the two API families.

Given all that, there are two obvious ways to expose a Unix app as a web service on Windows. You could port an entire open-source Unix-based web services infrastructure to Windows via SFU; start with apache (already done for you), add a couple dozen libraries and tools, hire a couple consultants, and you can probably make it work. Or, build the web service using VS.NET and “leverage” the functionality of the separate Unix application process to deliver the goods.

That's what we did. We built a small set of C# classes that allow a .NET app to run any Interix application and capture its output (stdout and strerr). Using that, our demo contains a web service which exposes a couple of relevant methods on a target Unix app (which computes tide tables). The implementation of those methods uses the helper classes to invoke the app and capture the output, which the rest of the .NET Framework takes care of translating into XML and returning to the caller.

It's that easy. Of course, the complexity comes into play when you think more about how to communicate from the .NET Web Server to the underlying Unix app. The demo just uses command-line invocation, which is easy but not terribly efficient. If the Unix app exposes an RPC server (e.g. using ONC RPC), then the web method implementation would just make an RPC call to the Unix server. You could build some unsafe C# code that maps a file into memory, and build the Unix app to map the same file; now you have shared memory to move lots of data in the blink of an eye. And so on.

The COM version of the demo follows the same pattern: build a COM server in Win32, but the implementation of a COM method uses a helper function to invoke a Unix app, capture the output, and reformat for consumption. The code is longer, since COM doesn't make this as easy as the .NET Framework, but it gets the job done. One of the demos is an excel spreadsheet with a VBA macro that creates an instance of the demo COM object and stuffs the textual data into a table which it then uses to build a graph. The X11 version of the Unix code has about 4000 lines of code to graph the tide data; replace that with 17 lines of VBA macro and 250 lines of C++ COM code. Voila.

I'll post some code early next week.

  • Today, the hype is all around social software, new more fluid approaches to workflow, and collaboration. Making ourselves as groups and organizations more efficient. The information workers, as we call them now, are not islands. They are linked with each other in vast human communication networks. How are we doing it? By simplifying the interfaces we have with each other to increase the number and effectiveness of our links. I see some similarities between this approach to human collaboration an