Follow Us on Twitter
by MichaelF on October 13, 2006 05:57pm
Today we are introducing Cyril Voisin, Security Advisor for Microsoft in France where he has worked for 9 years. Cyril is a CISSP (Certified Information Security Systems Professional) and along with his work at Microsoft also teaches systems and network security in local schools as time allows. Cyril has started a blog, primarily focused on security (exact blog intent can be seen here) but occasionally dealing with interoperability as it relates to security.
Cyril has given us permission to syndicate his content on Port 25, the first example is below. Please feel free to post any questions or clarifications below or on Cyril’s blog.
We welcome Cyril to Port 25 and look forward to featuring his work and insight in the future.
-michael ------------------------------------------------------------------------------------------------------- How to use Windows Vista’s Boot Manager to boot Linux The Web is full of explanations on how to dual boot Windows and Linux using a Linux boot manager like GRUB or LILO. If you want to dual boot Windows Vista and Linux using Windows Vista’s Boot Manager, please read on. I will assume that you already have installed Linux on your machine using GRUB as your boot loader.
Step 1 – Install GRUB on the Linux partition (outside of MBR) As Windows Vista will replace the Master Boot Record (MBR) with its own, we need to relocate GRUB elsewhere by running grub-install with the Linux partition as a parameter.
•On Linux, launch a Terminal with root privileges •Find the name of the partition Linux is installed on by running fdisk –l (the partition you’re looking for is the one whose system is Linux, can be something like /dev/sda1 or /dev/hda1. For the rest of this post, I’ll use /dev/sda1) •Install GRUB on the Linux partition by running : grub-install /dev/sda1
Step 2 – Get a copy of Linux boot sector We will need to instruct Windows Boot Manager how to boot correctly Linux using Linux boot sector, which we will extract using dd.
•On Linux, launch a Terminal with root privileges •Take a copy of Linux boot sector : dd if=/dev/sda1 of=/tmp/linux.bin bs=512 count=1 •Copy linux.bin on a FAT formatted USB key or any storage accessible from Windows Vista
Step 3 – Install Windows Vista
Step 4 – Configure dual booting in Windows Vista We will create an entry for GRUB in Windows Vista boot configuration data store using bcdedit.
•On Windows Vista, launch a command prompt with administrative privileges (by right clicking on cmd and choosing Run as Administrator) •Copy Linux boot sector on the root of the Windows boot (active) partition, namely the one containing bootmgr. If you don’t know for sure you can use diskpart or diskmgmt.msc to find out which one it is. •Create an entry for GRUB : o bcdedit /create /d “GRUB” /application BOOTSECTOR o Note: bcdedit will return an ID for this entry that we will call {LinuxID} below. You will need to replace {LinuxID} by the returned identifier in this step. An example of {LinuxID} is {81ed7925-47ee-11db-bd26-cbb4e160eb27} •Specify which device hosts a copy of the Linux boot sector o bcdedit /set {LinuxID} device boot •Specify the path to a copy of the Linux boot sector o bcdedit /set {LinuxID} PATH \linux.bin •Add Linux entry to the displayed menu at boot time o bcdedit /displayorder {LinuxID} /addlast •Let the menu be displayed 10 seconds to allow for OS selection o bcdedit /timeout 10
Building a dual boot system with Windows Vista BitLocker protection with TPM support
Many people have wondered if it would be possible to dual boot a TPM-bitlockered instance of Windows Vista with Linux, or another OS. The answer is yes and the following procedure will hopefully help you setup your machine correctly.
Some (simplified) background on Bitlocker: Bitlocker Drive Encryption allows encryption of Windows Vista’s partition and provides a secure startup process when in use with a TPM (a crypto chip on the motherboard). Basically the BIOS, the TPM, the MBR and the boot sector will collaborate to help verify that there was no modification to the boot sequence since Bitlocker was activated. This is done by using a function of the TPM to compute and store a hash of the code before executing it, at each of the initial steps of the boot sequence. Different hashes will be computed and stored in specific registers of the TPM. Then Windows Vista will ask the TPM to unseal its volume encryption key and the TPM will only provide this key if its registers are correctly set. Therefore if you replace Windows Vista’s MBR by a MBR that is not TPM aware, it won’t hash the boot sector before executing it and a register in the TPM won’t be populated. Same with the boot sector. Therefore Bitlocker will simply refuse to be enabled.
The underlying idea here is to have Bitlocker enabled with the original Windows Vista boot files. Another possibility would be to use a TPM-aware version of GRUB. However this would imply using files in the boot sequence that were not tested by Microsoft, which I would not recommend. Moreover, using original Windows Vista files offers you the benefits of code that went through the Security Development Lifecycle, which I personally find very valuable.
Note: I assume that you have a Bitlocker compatible machine (including TPM 1.2, TCG BIOS). See http://www.microsoft.com/technet/windowsvista/library/c61f2a12-8ae6-4957-b031-97b4d762cf31.mspx#BKMK_require
Step 1 – Install Linux Note: be sure to leave enough unpartitioned space for Windows Vista: about 11 GB of free unpartitioned space and slots for 2 partitions are needed
Step 2 – Install GRUB on the Linux partition (outside of MBR) See other post “How to use Windows Vista’s Boot Manager to boot Linux”
Step 3 – Get a copy of Linux boot sector See other post “How to use Windows Vista’s Boot Manager to boot Linux”
Step 4 – Create partitions for Windows Vista We need to create 2 primary NTFS formatted partitions on the disk: one active, 1.5GB size minimum and another larger (all the rest for instance with a minimum of 8.5GB). The former will be used to boot the machine (active partition) and will remain unencrypted while the latter will host Windows Vista and will be encrypted when we activate Bitlocker. You can use diskpart tool to do this (available from Repair options on the Windows Vista DVD). Here is what the instructions may look like :
Step 5 - Install Windows Vista Install Windows Vista on the largest NTFS partition.
Step 6 - Set up Windows Vista Boot Manager to boot Linux See other post “How to use Windows Vista’s Boot Manager to boot Linux”.
Step 7 - Enable BitLocker on Windows Vista See BitLocker documentation, like http://www.microsoft.com/technet/windowsvista/library/c61f2a12-8ae6-4957-b031-97b4d762cf31.mspx#BKMK_S3
by jcannon on October 19, 2006 04:40pm
Paul Moore sits down with Sam in this podcast, our second go around with Centrify. For those who don't know, Centrify builds software that provides access control and centralized identity management in mixed networks (Yes, this can include Windows, Linux, UNIX, Mac, J2EE and web platforms with Active Directory). This interview is a bit different as Sam runs through questions submitted to Port 25 from the first Centrify interview and Paul takes the opportunity to directly answer the community.
Centrify is also a sponsor of TechX, a roadshow being sponsored by IBM, Linux.com, Slashdot, Microsoft and others to help customers better enable interoperabiliy between operating systems, directories, applications (headed to DC, Chicago, Dallas & San Fran - check it out).
by MichaelF on October 16, 2006 08:13pm
Last Thursday we posted an interview with Jeremy Moskowitz in which we mentioned the book he authored: Windows and Linux Integration.
As promised, today we have posted a complete chapter from his book dealing with: Printer Sharing between Windows and Linux
Please respond with any questions or comments, we'll see if we can get Jeremy back on to answer your feedback.
Attachment: winlinchapter.pdf
by billhilf on October 31, 2006 08:30am
Today I had the opportunity to attend and present at the Zend/PHP Conference and Expo in San Jose. I was here to announce a technical collaboration between Zend and Microsoft that will improve the performance of PHP on Windows Server (and down the road Longhorn Server). You can read the specific details of the collaboration here.
As part of my presentation, we performed a demo showing a before and after scenario. We first showed PHP running on Windows Server before the enhancements- then again after. On the latter, I’m pleased to say that we consistently achieved 100% performance gains and on some applications a 150% improvement. All treats, no tricks…that’s right 100-150%.
So what are we doing exactly?
PHP, being the third most popular development language today, is an important addition to the options available to developers who want to leverage the Windows Server platform, and the overall Microsoft ecosystem. According to Zend’s internal statistics, the majority of PHP developers already use Windows as their application development platform – over 70%. Improving their experience with running PHP applications on Windows for production is a natural next step.
In the end, we believe the real winners in this technical collaboration are PHP developers who now have viable options when thinking about platform of choice for their PHP applications. Of course, that means our mutual customers benefit from PHP applications as well as the choice of technologies that best suit their needs. To make sure this not only happens today, but going forward, both Microsoft and Zend will be active participants in the PHP community to ensure open communications and a continually improving experience for PHP developers in Microsoft environments.
As I have said before, interoperability does not happen by accident. This announcement is the result of a lot of hard work by people from both Microsoft and Zend. I personally want to thank Andi Gutmans, CTO and co-founder of Zend Technologies. Andi and I have been in discussions for a long while now and I’m very happy to see the great results from a conversation started long ago. We have been working with Zend over the past 6 months to put all of the pieces in place. From the technical work on FastCGI to helping Zend create an engineering lab to improve PHP as it develops over time. This is the first step in an ongoing relationship.
And for those of you in North America, Happy Halloween!
-Bill
by MichaelF on October 18, 2006 05:00pm
Today we are pleased to introduce Frank Chism. Frank is a Technology Specialist at Microsoft who has worked in the High Performance Computing space for 41 years. He is joining our team of bloggers and will contribute a monthly blog regarding HPC. In this interview Sam and Frank talk about Frank's background as well as some of his insights on HPC both past and present.
Look for Frank's first blog later this week or early next.
by MichaelF on October 03, 2006 06:42pm
About a month ago, I did a presentation on What I Learned from OSCON, based primarily on James Howison's talk on OSS Communities. Since James had such great and interesting feedback on my presentation, we decided to do a follow-up podcast. I'm trying to publically capture as much of this journey as possible of a team at Microsoft learning how to go open. Maybe this journey will make for an interesting OSCON '07 talk, I hope. =)
-sara
Introductions:
James Howison is a doctoral student on Kevin Crowston's NSF-funded research team at the Syracuse University Information School and a regular presenter at O’Reilly events. At OSCON '06, James gave a talk on OSS Communities, where Sara Ford took notes until her fingers went numb.
Sara Ford is the Program Manager for the Power Toys for Visual Studio. Her mission is to explore the various ways Microsoft developers can have conversations with the Visual Studio community at the source code level, while providing great after-market solutions to developer pain-points via Shared Source.
Attachment: http://port25.technet.com/videos/podcasts/jameshowison.mp3
by MichaelF on October 31, 2006 08:31am
Today at Zendcon Bill Hilf and Andi Gutmans announced a new technical collaboration aimed at improving the performance of PHP on Windows both for IIS 6 as well as in the future including IIS 7 on Longhorn. The partnership has multiple components including Microsoft releasing a FastCGI add-on for IIS and Zend will establish a Windows testing lab to ensure high performance on an ongoing basis. In the end the winners are the millions of PHP developers and hosters who want an viable choice in operating systems.
In this interview Sam and Andi discuss how Zend came to be as well as the details of the announcement. Andi provides his perspective on what this collaboration will mean to the PHP community and has a question of his own for Sam.
by MichaelF on October 26, 2006 05:13pm
On September 6, 2006 Microsoft and Cisco announced the details of a technical partnership announced in October of 2004 focused on providing interoperability between the companies' disparate network security technologies: NAC and NAP. In this interview Sam digs into the details with Mark Ashida, General Manager of the Enterprise Networking Group. They also discuss Xorp and why Mark believes his is one of the most "open" groups at Microsoft.
As part of the announcement a whitepaper with details was produced and can be found here.
Attachment: nac-nap.mp3
by MichaelF on October 04, 2006 09:47am
So, as people are finding out, we at OSSL are doing things a little differently. One of those things Sam Ramji’s invite to the Mozilla guys to come up and help them making their products work on Vista better.
This was met with great criticism (Slashdot for example had people make all kinds of wild assumptions that started with suggestions all Mozilla folks flying on different planes to MS offering everybody jobs. And everything in between) and surprise.
Well, I am glad to report that the Mozilla guys are here. I have spent a really cool day in all kinds of meetings with the following guys from Mozilla; Robert Strong, Mike Schroepfer, Doug Turner, Vladimir Vukicevic, Scott Macgregor. (If I slaughtered anybody’s name I apologize for it in advance!!)
They are meeting with all kinds of people here, and it seems that both sides are learning greatly from each other. One thing that I noticed right away is that both the MS side and the Firefox side just love the technology and are hitting it off nicely.
The things we talked about today for example have been;
Cardspace; Mike Jones and Garrett Serack gave an overview of cardspace. Cardspace is a carrier of identity tokens. But you can read about it here as a nice starting point http://blogs.msdn.com/garretts/ . We spend some time talking about what it is providing, and that it is an effort from the industry to provide a way to provide identity management. Vista will include it out of the box, and .Net 3.0 will include the cardspace technology. Which will make it run on XP and 2003.
And we are currently looking to writing a plugin to make cardspace work inside of Firefox. (Like I said, we do things differently here at OSSL )
IE Lower Integrity Marked: IE will be running in a Low Integrity Marked setting, resulting in a greater security environment IE will run in. But it limits a whole bunch of places it can write to. (which is good) We talked with the IE people to the benefits and pitfalls of doing this, Mozilla is interested in checking into doing this for Firefox as well.
MS WPFE: the developers from the plugin had a conversation to make it work inside Firefox better. (I am not an expert on this, but this should give a better understanding of what it is; http://msdn.microsoft.com/winfx/reference/presentation/default.aspx )
MSI: MSI developers talked about the changes in and advantages of using the new MSI way of installing software in Vista. One of the things that came up was that some vendor have problems with their plugins (xpi files) such as Adobe. (Problem relating to global plugins) XPI can only be installed per user in Vista. I will talk more about this in a later blog
Windows Vista Shell: we went through a high level overview of the Windows shell in Vista.
If you want to know what the Mozilla folks thought of yesterday’s meeting, check out Vladimir’s blog http://blog.vlad1.com/
Well, I wanted to keep this short. But still let you all know what is going on. There is another day with them coming up tomorrow.
Btw, the chips have been implemented into the brains of all the Mozilla folks that have shown up here. And it looks like from Firefox 2.0 on it will be released on Windows exclusively.
Hank.
by MichaelF on October 06, 2006 12:57pm
In the IT industry it is axiomatic that whatever is new will be old, and will then be new again! Consider the “Service Bureau” approach that was used in the mainframe days, in which an organization’s computing needs were taken care of by a “Service Bureau” that maintained the infrastructure, served up the applications and provided the support for the users. The Service Bureau typically served many organizations needs, had to keep their customer’s data separate and provide an SLA to each one of their customers. Sound suspiciously like SaaS (Software as a Service) doesn’t it? Service bureaus were not as sexy as SaaS and they never completely went away. With the advent of PCs, computing was available at the individual level – which meant that small teams could manage their own IT infrastructure, applications and support for the users. This freed them from constraints (and some cynics say from discipline!) on their ability to adopt and adapt new software technology. The software on PCs grew capable of handing mission critical applications allied with the raw compute power provided by networked distributed PCs. The small teams began to feel the pain of managing and maintaining such infrastructure, and the PCs were serving not just these small teams but entire enterprises. With the commoditization of hardware and the adoption of common software standards, the “Service Bureau” idea re-emerged as “SaaS”, and was immediately attractive to the customers. Of course, I am oversimplifying, but I think I can use my blog writers license here. The new avatar (SaaS) is better for the users because it doesn’t force the compromises the old avatar (Service Bureau) on its customers. The idea is the same but the implementation is improved immeasurably!
I think I have identified a similar new, old, new cycle in OS technology (ladies and gentlemen, please save the standing ovation for later!). Of course, I have to insert the “don’t-try-this-at-home-kids” warning. I am not an operating systems expert – I just play one on blogs! Heard about microkernels? They were all the rage back in the late 80s/early 90s (that’s in the nineteen hundreds). This is how Wikipedia defines them “A microkernel is a minimal computer operating system kernel providing only basic operating system services (system calls), while other services (commonly provided by kernels) are provided by user-space programs called servers.”
Microkernels were a reaction to the bloat introduced into operating systems, which started out lean but then added all kinds of services as part of the operating system. This meant that operating systems were not as portable as they used to be, because all services had to be ported over – whether to a new processor or to a new board. There were attempts made to make operating systems that were minimal that also had the effect of making them portable – because to port an operating system, all you had to do was port the teensy weensy kernel. I know this because one summer as a mere stripling I was working at the Indian Institute of Technology, Bombay which had just got a bunch of tapes of one of the first microkernel based operating systems called Mach. Some highly talented academic and industry folks were building a Unix workstation (from the hardware up) and wanted to do the least amount of work to do it! To someone using the workstation it was hard to distinguish between that system and a vanilla (think Sun/DEC) Unix workstation. But the speed with which the port could be done was astounding or so the people working there assured me.
Mach, which came out of Carnegie Mellon University, was much more than the portable operating system thing I made it out to be– it was a radical new way to look at OS’s. The idea was to abstract away the non-essentials of an OS and leave a very small microkernel to be dealt with. This had far reaching impact on how portable both the kernel and the services written on it could be from hardware platform to hardware platform. An interesting fact, Rick Rashid who was the Prof leading the MACH team, came to Microsoft and is now the SVP responsible for MS Research. Hmm, worked on Mach, was at Carnegie Mellon and then came to Microsoft to become SVP! Looks like my career is on track here!
Then two things happened:
So the old (microkernel) went out, and the new came in (macrokernel – it’s not a term, just something I invented!).
Ah ha! You say. When is the old going to come back?
I think it has already sneaked in by another name - virtual machines. I don’t mean Virtual Machines as they have been defined by Xen, Microsoft and VMWare. I mean in the computer science understanding of the term virtual machines which implement “virtualization”. According to Wikipedia (what would I do without it!) “In computing, virtualization is the process of presenting a logical grouping or subset of computing resources so that they can be accessed in ways that give benefits over the original configuration”. Hardware abstraction layers (think Xen) and virtualization engines (think network-seen-as-one-computer) are just different aspects of virtualization.
This probably needs some more explanation. A concrete example would be even better. One dropped into my lap as I was writing this blog. My attention was guided towards a company called 3Tera which claimed to be building a “Grid operating system”. They make the claim that they take an existing web application and without changes drop it onto a grid, so that the web application is able to be scaled by the provision of on-demand resources dictated by consumer demand. They are platform independent i.e. it doesn’t matter to them that the application runs on Windows or Linux.
The reason that they are able to do this is because they have redefined a conventional OS using the concept of virtualization. That gave me the idea of how I could diagrammatically show you what this means.
The new microkernel does very few things, it just manages the allocation of today’s OS’s (which are just applications to the new microkernel) on to a grid of commodity hardware. This grid could be a single computer or more likely a widely distributed network of computers. All this is possible because the two dominant OS’s Windows and Linux rely on the same commodity hardware. This allows for the capability of a mainframe (in terms of manageability, security & protection, partitioning) while retaining the advantages of using cheaper distributed computers.
The dream of “a view of computing resources is not restricted by the implementation, geographic location or the physical configuration of underlying resources” will be realized. It is also realized by utilizing in full the investment made in today’s OSs – which do their thang much as they do now!
So what? You ask. Patience, young Jedi!
(Peering into my trusty crystal ball) Operating systems will be sets of services. OS’s will be chosen based on some favorite service, without giving though to “platform lock in”. Operating systems components will be componentized. So that a Linux daemon will (gasp!) be able to use a Windows security component, via the use of standardized protocols. Without it knowing that it is. Clustering and failover will not be high end luxuries, but will be baked into all OS’s – and schedulers will schedule multiple OS’s across Internet scale networks
……….VB programmers will be able to make the equivalent of the salaries they were making during the dot com boom. Ok – so that will never happen!
I think these predictions are probably going to raise some level of discussion. (Understatement, understatement!).
by MichaelF on October 17, 2006 06:16am
Today Microsoft announced the addition of it's Virtual Hard Disk Format (VHD) to the list of specifications covered by the Open Specification Promise (OSP). As a result individuals and organizations can use, redistribute and modify Microsoft's virtualization format for free.
In light of this announcement we thought it would be a good time to catch up with a company that utilizes the VHD format: Xensource. In this podcast Sam and Simon Crosby, CTO of Xensource, discuss the Xensource and Microsoft partnership, Veridian, virtualization approaches, what's on the horizon and more.
Attachment: xensource.mp3
by MichaelF on October 10, 2006 12:10pm
Recently we posted about http://www.devreadiness.com, an online resource for Vista Application Compatibility questions/guidance. After posting we received some feedback that with some changes they could become much more useful for Open Source Developers. Based on this feedback we are going to create a set of resources specifically tailored for the open source developer/ISV.
This is where we would like some help. We can move forward with this based on our understanding and assumptions or we can involve our community and ensure that the “right” changes are being made. To this end we are seeking a small group of volunteers who would be willing to act as a review/guidance team. This endeavor should not be tremendously time consuming. We anticipate a single review/feedback cycle of existing content to get us started. After an initial pass we will request one to two more review cycles to ensure we are on the right path.
While the time required is not tremendous, the impact of your participation will be. Please drop a line to port25@microsoft.com if you are interested.
Thanks!
Sam
by jcannon on October 02, 2006 06:06pm
Last week, I was fortunate to take a day & visit the floor of Interop New York. The Interop conference celebrated its 20th anniversary this year, underscoring the persistence & complexity of interoperability as an industry issue. This year’s mission was no different…to discuss achieving the ideal state of all technology talking to itself, and to others.
OK, so here’s the clever metaphor I came up with on my walk over to the Javitz Center (I live in NYC – anyone around….let me know). Interoperability is a loaded term since it can mean so many different things to different folks …but why? Because interop is really more about many means toward an end. In fact, I would suggest the goal of most interoperability efforts is to enhance the performance of a total system through improved communication and accessibility of the various subsystems – be that protocols, applications, schemas or operating systems (the means). It’s kind of like…..a city! The below pick was snapped on my way over to the show....the total system of a city only works when it’s subcomponents work together successfully – be they water, electric, gas, subway, building specs & zoning, urban development & layout, garbage, sewer, security, transit. All of those systems must talk to themselves and to each other to provide a quality of service to the citizens it serves. I skillfully Photoshop’d the below to illustrate my point….if you live or have visited NYC, you know that it’s amazing that the system works so well, every single day. We grumble that interop in IT is hard to achieve with legacy systems 5, 10, 20 years old. Imagine connecting systems dating back to Gangs of New York. Absolutely nuts.
Pretty Complex Stuff (PCS)
The Show More than anything, it was fascinating to hear how customers & vendors are defining interoperability. The pervasive definition was squarely on technology and the increasing role of (here comes the buzz-train) Web 2.0 in connecting data, information and systems. This was reflected in the keynotes, such as Andrew McAfee and SocialText CEO, Ross Mayfield’s presentation. I’ll start by breaking that notion down into a couple more discrete ideas:
The Browser as Epoxy This was the most interesting theme of all ~ across both IT and Developer tracks, interoperability was a problem best addressed by the browser. I phrase it this way because at its most basic level, the browser, through robust XML frameworks and rich presentation layers, is what is bringing disparate data and systems together in a seamless way for enterprise end-users. That quality of adhesiveness is being enabled by technology like AJAX and DHTML – not surprisingly - and subsequently one arrives at ‘Rich Internet Applications.’ What was interesting to me was that interoperability solutions weren’t being implemented over wires to connect servers. Instead, silo’d data was being fed to powerful client machines for cohesion. In essence, the browser was the epoxy for the data & systems throughout the network. Cool stuff indeed.
The (Dubious) Optimism of Enterprise User-Collaboration The keynote on Thursday was dedicated to examining how social software conventions (Wikis, Tagging, RSS) can & should be extended into the enterprise environment. In contrast to the process orientation of most enterprise-level applications, Andrew McAfee, Harvard Business School, pointed out that there was much to gain through large scale, unstructured data sharing in an enterprise & the subsequent emergent analysis that could be leveraged as a learning system. What makes all this possible? The very cool technologies of Web 2.0: Deep Linking, Search, Tagging, RSS, Easy Authoring.
Why dubious? Only the largest sites on the Internet are proof points of success: Digg, Delicious, Technorati….we have no idea how to bridge legacy systems to a new generation of tools in an enterprise, nor the required tipping point at which critical mass is reached and emergent patterns actually become meaningful. (How useful is this really for a small/medium/large organization?) And control. Users tend to be untrusted, unsupervised, distributed, silo’d and working in environments under great rates of change – how does a manager leverage a system like this?
Can I Get a Customer? There were a ton of great demonstrations and eye candy, but I was hard pressed to find a customer talking about a successful interop implementation using most or all of the above technologies. This indicated – to me- we’re still in the early hype cycle of these technologies and lack a customer base and the watered-down expectations that accompany real-world implementation.
Adult Visual Aides; Server Racks & The Show Entrance
It was absolutely worth the day – although I would have liked to have seen more customers on-site. Can’t blame anybody given it was the last day of the conference though. Did you attend the show, or have any questions about it? Share your thoughts; I would love to hear other perspectives on the themes that were presented.
- Jamie
by anandeep on October 27, 2006 12:27pm
I loved doing development in a research and university environment. You got to write cool code, prove new ideas, break new ground and generally ended up with bragging rights to say “I did an image recognition algorithm on a multi-layer architecture implementing reactive and planning parallelism on an autonomous robot!” The code had to work on your workstation or maybe on a demo machine once. Once you wrote the code, the only people who touched the system were hapless graduate students implementing the next big idea. They had to come to you and you could then dazzle them with your insight! This was “sexy development”!
When I moved to industry and wrote software for day to day use – things changed. Now you had all those people with “manager” titles telling you what to do, and those people called “testers” who told you why your code sucked (you couldn’t logically argue your way out of that because the weasels usually had proof)!. Of course being consummate professionals you adapted. You got the religion of “bullet proof code” and worked on making sure the testers only had “fit and finish” bugs filed against you. Which the intern could work on. That was still fun - a different challenge maybe not as “pure” as designing a neat new algorithm but pretty good nevertheless!
You got past the testers but when they integrated the components that you had bullet-proofed to run end-to-end or user acceptance tests, unexpected stuff happened. Who would have thought that they would configure the machine that way or that another non-surface component could pass you null strings. Now you had to plan not only for the testers – but also for other developers and those pesky sys admin guys. How did they become sys admins? They couldn’t tell a polynomial solution from a log n solution anyway! But being nothing if not adaptable you adapted. You now built bullet proof AND idiot proof code. (My father, a military pilot and flight instructor, when teaching flight safety used to say “Nothing is foolproof because fools are so ingenious!”). It got a little boring at times but you still had the satisfaction of building something that was “engineered”.
I thought I had shipped the product but I found couldn’t sit back and relax. The support guys were making insinuations against my code. It didn’t work they said – and you hadn’t put in the right level of granularity in the logs for them to do a diagnosis. This had nothing to do with Computer Science – any bozo could write stuff to the log. Why didn’t the intern do it? What do you mean he can’t make sense of my code? Yeah, I do know my code best. I guess it’s the right thing to do. Certainly not as fun as designing, bullet proofing and idiot proofing new code but good supportability is “sine qua non” for a well done project!
Is that the end of it? No, further design and coding needs to be done for making software more manageable, to make the logs more systematic, to make sure that the product works when its deployed to multiple configurations, that it performs well and fails gracefully.
Unless you specialize in a certain aspect of manageability, reliability or diagnosis – this is not “sexy” development. I probably wouldn’t get as much satisfaction from designing event logs as I would from designing a new search algorithm.
I was getting paid to do all this (ok, so it was my own startup but I was getting paid in VC money!) and it was still very hard. We did do it but it took lots of coaxing of our developers to pay attention to this. They all preferred to work on the next release that had all the sexy features. Even though they knew that to make the startup successful and still have a job, the unsexy stuff needed to be done and done RIGHT!
When you are working for the “love of the game” and not money, like in Open Source – who coaxes you? Who does the unsexy stuff? Are there enough people who specialize in the esoteric aspects of event logs, that this is not a problem? Or do users who need the feature “just do it” and add the code to the community version? Or are things slipping through the cracks?
I did a sweep of the usual suspect Linux developer mailing lists and found that there is concern about whether unsexy stuff gets done. Here is a typical comment that I saw
“I think that the only issue with Open Source boils down to this:
The things that nobody wants to do, but somebody has to.
Nobody wants to think about documentation. Or user interfaces. These things are hard, tedious, and a hell of a lot more boring than actually coming up with stuff to "make things work".” (from here)
Documentation is famously one of those things that is considered “unsexy” (well, ok in commercial software too). There are efforts like Grokdoc to make documentation of Open Source projects sexy by making it a priority. But the “who does unsexy?” issue is a real concern in Open Source.
We ran into a similar issue with event logs. You know the text stuff you write so that you can find out later what happened. At the lab we just did an investigation of whether we could tell if one of our boxes had crashed from the syslog and from console messages. We were a little taken aback by how many times we couldn’t tell what states the machine had gone through.
On doing some investigation we found that the most influential project that was addressing this issue, the Evlog project (most supported by IBM) has been quiet since 2004. This code is used internally within IBM but was not mainstreamed into the Linux kernel.
How does one get unsexy stuff like this into the Linux kernel so that is comparable to UNIX/VMS/Windows?
I contend that it is critical to Open Source that attention be paid to the event logs. They are critical in making any operating systems reliable. VMS/UNIX/Windows all went through the process of making their event logs more meaningful – and this has helped make them much more reliable.
We will be addressing this further in the next couple of weeks – keep tuned!
by MichaelF on October 23, 2006 12:34pm
Taking a brief detour from the thread about OSS and its similarities (or not) to law to take note of a couple recent publications, both of which discuss the interaction between traditional IT vendors and OSS:
In MIS Quarterly (September) (link) Brian Fitzgerald (University of Limerick—one of the must-read researchers on OSS, IMO) provides a comprehensive survey of what he calls “The Transformation of Open Source” with expectations for “Open Source 2.0.” He expects IT vendors—including Microsoft—to play significant roles in “OSS 2.0.”
In Communications of the ACM (October) (link) Pamela Samuelson (UC Berkeley) discusses “IBM’s Pragmatic Embrace of Open Source.” (The title pretty much speaks for itself as a summary.)
I highlight these because they reflect what seems to me (qualitatively) to reflect a trend in the literature. We’ll work on getting a better sense of what the trend is and researchers’ perspectives on it to bring back to Port25…because (to bring things back to analogy and metaphor), they introduce the question: Is a vendor in my OSS more a fly in the ointment or chocolate in my peanut butter?