Follow Us on Twitter
by admin on June 30, 2006 04:23pm
We've got a quick guest blog before the holiday weekend on a new set of interoperability tools released this week by Microsoft, and in conjunction with some very talented folks in our partner, sales & services group. Welcome Luis Camara Manoel, Program Manager for in Communications & Collaboration...take it away Luis:
------------------
(Luis Camara Manoel)
Hi all, The Interoperability and Migration Solutions team at Microsoft is very happy to have just released the Windows Security and Directory Services for UNIX Guide. This guide details step-by-step instructions for providing security and directory services for mixed Windows and UNIX environments using Active Directory. The approach we followed here describes multiple options to achieve two different system end-states:
1. UNIX clients are enabled to use Windows Active Directory Kerberos for authentication while continuing to use a UNIX-based store for authorization.
2. UNIX clients are enabled to use Windows Active Directory Kerberos for authentication and use Active Directory LDAP for authorization.
We have also added some tools and templates to help plan, develop, deploy, and operate these solutions.
Although this release addresses AD integration only for Solaris 9 and for RedHat 9, it is pretty simple to see how it would apply to other UNIX and UNIX-like systems.
You can download the guide here and please let me know what you think. I am very interested in finding out how the guidance is received.
Have a great holiday weekend, -Luis
by admin on June 28, 2006 06:13pm
Consider this scenario – the CIO’s office just called. They have decided to follow your recommendation and are going to use JBoss (substitute favorite open source software package here) as their application server (substitute database server/web server/whatever server here). They also decided to make it standard on both your Windows servers and your Linux/NetBSD (substitute favorite flavor of open source operating system here). Now, they ask you – will it handle the loads on both operating systems? They don’t want any of the servers to be idle after all that investment they made.
You have a reputation as a “data driven” dude, i.e. you look for empirical evidence in making decisions like this. No hand wavy – “it will work …. maybe” for you. You know your workloads cold. You know you will run test scripts to simulate those workloads.
The million dollar questions are (drum roll please):
Why are these questions important? The objective is to have consistent and comparable tests on each platform – and it is important to make sure that the testing process does not introduce any bias one way or the other.
So which approach - trusting the tools or trusting your methodology allows for consistent and comparable tests on each platform?
Lets examine some pros and cons of each approach:
Damn! The laws of physics strike again. Looks like there is no easy answer since the observer effect will always be present i.e. observing the software using testing software impacts the software being observed so that a precise observation can’t be guaranteed.
So what do you do in such a case?
My friend (and I think he works with me too) Hank Janssen had the following to say (I will not point you again to his handsome visage like I did in my last blog or people will start talking!)
I am not so married to the tool as I am married to a consistent and approved process. There will be many different tools for different languages/applications/infra structures/protocols and the like.
I wanted to stay away from testing methodologies that are so different between different platforms that I would have a hard time interpreting the results. For example, using Loadrunner or Winrunner on Windows and than Grinder on Linux. (In case of Java loads) This example would result in apples to not so apples comparison. Most OSS tools (jMeter and Grinder) are available on both platforms.
Advice from the testers in Microsoft
I spoke to a few people who do testing for a living within Microsoft and here is some advice they had to offer
But enough about us, what approach do YOU take, dear readers? Please do let us know - we are looking for war stories, dispatches from the data centers, tales from the trenches and any other feedback on testing disparate platforms that you have to share!
by admin on June 28, 2006 01:30pm
Spam is a well-known problem for many on the Internet. If you have an email account anywhere, chances are you’ve gotten something you didn’t ask for; a “stock tip”, an adult entertainment solicitation, or possibly a plea from an altruistic member of the “[Random Nation] Royal Family” to assist in some friendly money-laundering.
As the anti-spam movement gets craftier, so do the spammers. Fortunately for the spammers and unfortunately for the internet, there are a wealth of open-relay mail servers should have never been put online. While most common and current-version SMTP software is secure by default, there are plenty of people who still run outdated software, never bothered to upgrade, or configure properly in its present state.
If you are tasked with administering and monitoring a large portion of IP space assigned to people with autonomous control of machines on an externally visible network, this problem can get to be a thorn in your side very quickly - just ask any ISP that allows their customers to run servers.
If you’re not allotted much (or anything) of a software budget to purchase fancy enterprise tools to hunt down open relays on your network, there are some free and lightweight tools for Linux. One such utility is a small application written in C, called “smtprc” (smtp relay check): http://freshmeat.net/projects/smtprc . This simple application takes about 10 minutes to set up. First unzip it into your directory of choice. Next read the README file, and specifically check the Compilation/Installation section to make sure it ends up where you want it to. If not, edit the Makefile and put it where you want it to go. Do a “make” and “make install”, edit your scan configurations and go. It will output results to an html file (location specified in configuration). They will be color-coded by result. The collected data may then be used to notify administrators of vulnerable machines.
Note: Some older versions of NT Mail and Lotus Notes will turn out false positives. The messages smtprc attempts to relay are what I would call “passively rejected”. The SMTP server being tested will accept the inbound messages, but they are never actually delivered. When in doubt, it is best to test manually.
$ telnet mailserver.com 25 ß telnet to the host in question on port 25
Trying 10.197.173.28...
Connected to mailserver.com.
Escape character is '^]'.
220 mailserver.com ESMTP Sendmail 8.13.1/8.13.1; Wed, 14 Jun 2006 15:17:39 -0700
helo bleh ß most mta’s now require a “helo/ehlo”
250 mailserver.com Hello [157.55.209.144], pleased to meet you
mail from:<me@here.com> ß sender address
250 2.1.0 <me@here.com>... Sender ok
rcpt to:someone@wherever.com ß intended recipient address.
250 2.1.5 <someone@wherever.com>... Recipient ok
data ß indicates message is now being written
354 Enter mail, end with "." on a line by itself
Subject: open relay? ß can be anything
Hrrrm…… ß message.
. ß dot on a line by itself indicates end of message, server will queue for delivery
250 2.0.0 k5EMHdHl028091 Message accepted for delivery
quit
221 2.0.0 mailserver.com closing connection
Connection closed by foreign host.
Check your mailbox in about 15-30 minutes. If it doesn’t arrive, chances are this is not an open relay.
by admin on June 26, 2006 11:00am
Every now and again, you read something that sticks in your head like tree sap.
Something that gets you thinking deeply about the future, your life and work… something that keeps you up at night with ‘monkey mind’, as the Zen folks like to say. 2020 Science did just that to me a couple weeks ago. 2020 Science is a look into the future, a “report that contains the initial findings and conclusions of a group of internationally distinguished scientists who met over an intense three days in July 2005 to debate and consider the role and future of science over the next 14 years towards 2020, and in particular the importance and impact of computing and computer science on science towards 2020.”
When I studied computer science I was fascinated by the Grand Challenges in science and mathematics, so this report fascinated me as it approaches many similar grand challenges and – specifically – looks into how computer science can help across these various disciplines.
The paper/roadmap looks across disciplines to understand how the various sciences interact and how they together can propel this 2020 science vision. I particularly enjoyed the coverage on multi-core cpu architectures, data ‘issues’ (association, aggregation, visualization, etc.) and I think the ‘computational thinking’ quote on page 12 (also here) by Professor Jeanette Wing, head of Computer Science at Carnegie Mellon University, is one of the most well articulated statements about effective problem solving that I’ve read.
A 3D model of the human brain © 2005 Malcolm Young, University of Newcastle.
The second part looks into many disciplines and topics, global environment, molecular computing, software platforms and tools, the brain, bio-hybrid chips, genomic research, and communities. Far too much to cover in a blog entry – read this paper, it’s worth the time.
In my role, I often get asked to compare open source software to the software that Microsoft builds. There are many ways to answer as it’s not a simple question. However, on occasion I spend time with folks in Microsoft Research (MSR), and I’m also fortunate to meet many different leading thinkers in academia around the world. As a lifelong computer science ‘student’, I am always surprised at how easy it is for people to focus just on present tense and forget where we as a society want to go in the future, and what it takes to get there. When I read through 2020 Science, when I visit universities around the world, or when I wander across the 520 highway to visit friends in MSR, I’m reminded by the grand challenges that we are challenging on a daily basis. Some problems are hard, very hard and often require very specific skills and new ways of thinking to attempt to solve. Some problems require innovation, some require invention. Some challenges need today’s tools, some require new tools. There are problems that require certain models to solve: by itself, OSS is often not appropriate for some of these problems, nor, at times, is only commercial/proprietary development (read the ‘new communities’ section on page 24). Problem context is crucial to framing the approach. Even IBM’s Linux strategy ‘godfather’, Irving Wladawsky-Berger (Vice president, Technical Strategy and Innovation) commented recently on this community appeal issue as it relates to certain types of problems:
“Generally, proprietary software tends to be higher volume, higher levels of transactions, and higher levels of security, and vendors have very focused teams to do that. And the layers of software that I would associate more with Internet culture are the most likely candidates for open source communities. Remember, in a community, you have to want to work as a community. There are certain kinds of problems that appeal to people to do that, and there are others that don't.”
In addition to having spent my career in both OSS and commercial software environments, I see this ‘model context’ issue in my current work and every day I’m more convinced that choice and evolution of models (and formats, standards, tools, software, etc.) will be the path both science and industry travel on to reach the 2020 vision.
-Bill
by admin on June 23, 2006 06:12pm
Due to popular demand we are going to start posting our interview content with a transcribed text version. Each week, until we catch up, we'll be releasing three transcripts from interviews that are already available on Port 25 in video format starting with the oldest and working our way forward. Comments and feedback on these transcripts are appreciated! Interview: Faces from the Collective: Shared memory anyone? Date: 3/31/06 Description: Sam Ramji sits down with resident UNIX guru and Microsoft Architect Jason Zions to discuss the history of Windows and UNIX interoperability. Download Transcript
Interview: Stuff we think is cool: Improving the Windows experience, it's elementary! Date: 3/31/06 Description: Ben Canning, Group Product Manager from the Office team talks Watson and how this unique solution is helping improve the Windows experience. Download Transcript
Interview: Partner Spotlight: Do your directories play nicely? Date: 3/29/06 Description: Directory specialist Jackson Shaw from Quest Software joins Sam Ramji from the Open Source Software Lab at Microsoft, to talk Active Directory and interoperability. Download Transcript
by admin on June 21, 2006 01:02pm
Last week we announced the formation of an Interoperability Customer Executive Council. This is something many folks here at Microsoft have been working on and thanks go to a wide variety of people. I was involved from an OSS perspective and will continue to be active with both our internal groups and this customer council to help this mission succeed. As you can imagine, interoperability is a very wide and loosely defined word – it can mean many things to many people. Particularly when you produce a lot of software! Rather than try to ‘pick’ certain areas to focus on (believe it or not, Microsoft is constrained by time and money like all other businesses) we decided the most effective and beneficial path would be to have our customers drive this conversation. Thus was born this council, formed by customers of different sizes, business types, and geographic location, in order to provide guidance on interoperability issues that are most important to customers, including connectivity, application integration and data exchange.
Interoperability in a heterogeneous environment doesn’t happen accidentally, nor does having your code open make everything ‘just work’ together. It takes thoughtful design and architecture; it takes time, effort and engineering discipline. It requires working closely with partners, competitors and customers. It also requires mature understanding that all things don’t necessarily require interoperability, and frequently those things that do, sometimes require different types of interoperability. Point being: it takes focus, energy and commitment. This council is a great step and example of this commitment from Microsoft.
I’m very excited by this step forward, below are some further news stories about this:
CNET News.com
Clients to advise Microsoft on software linking
http://news.com.com/Customers+to+advise+Microsoft+on+software+linking/2100-1001_3-6083664.html
eWeek
Microsoft Customer Council to Focus on Interoperability
http://www.eweek.com/article2/0,1895,1976394,00.asp
by admin on June 20, 2006 05:51pm
Not long after I blogged about “disambiguating open” as a research issue, a debate erupted on Slashdot about “How Open Does Open Source Need to Be?” Three different criteria for deciding whether something could legitimately call itself “open source” seemed to me to dominate the discussion: the level and terms of source code availability; whether or not there was a community organized around the code; and compliance (or not) with the Open Source Initiative (OSI) definition.
What struck me about this debate was, first, the fact that it was happening because “open source,” in this case, was regarded as a favorable characterization (and thus in the interest of a software producer to apply to their bits) and that favorability was rooted outside of any producer’s application of the term (in contrast, it’s unlikely such a debate would occur if one of the parties had chosen to label their product “cromulent” for example.) Put another way, the operative assumption is that calling your product “open source” piggy backs on a store of perceived value among potential customers built up by others, which may be in a technical or moral sense be “undeserved” and (at worst) might actually result in dilution of that store of value.
What I thought was missing, however, was some empirical perspective on the ways in which states of the three criteria above might in fact add to or erode that perceived store of value when the so-called (or not-so-called) “open source” product was actually consumed by customers. This reminded me of Perspectives on Open Source Software (2001) from Carnegie-Mellon’s Software Engineering Institute, where the authors do a great job of describing why the answer to “how open” software needs to be will almost always come down to “it depends...”—if we’re thinking of “need” relative to optimizing for the resulting customer experience (as opposed to the relative ease of calling something “open” for marketing purposes or conforming to a check-the-box definition.)
The report describes “open source software” as
“at the most basic level simply[meaning] software for which the source code is open and available.”
Which they define as:
“open--The source code for the software can be read (seen) and written (modified). Further, this term is meant to promote the creation and distribution of derivative works of the software…available--the source code can be acquired either free of charge or for a nominal fee”
And they contrast this with the Open Source Definition (OSD) from the Open Source Initiative (OSI) noting:
“Under OSI (strictly speaking) a software product is in fact open source if and only if it conforms to the OSD…Upon reviewing the complete text of the OSD, it is interesting to point out that the definition does not pertain specifically to the source code itself, but rather to the license under which the source code is distributed. Therefore, in strict conformance to the OSD written by the OSI, a software product that conforms to only eight of the nine criteria is not OSS.”
They raise a few points relevant to distinguishing meaningful customer requirements for the software :
The authors of the report conclude that:
OSS [is] a viable source of components from which to build systems. However, we are not saying that OSS should be chosen over other systems simply because the software is open source. Rather, like COTS [commercial off-the shelf software] and CSS [closed-source software], OSS should be selected and evaluated on its merits…Caveat emptor (let the buyer beware): the [software] product should be chosen based on the mission needs of the system and the needs of the users who will be the ultimate recipients.
To me, reading the discussion thread and then mulling over the SEI’s examples bring me right back to what gets me out of bed every (work) day: there is rich field of inquiry about what makes software successful for its developers and users. Neither any one commercial vendor nor organization has it all figured out and trademarked yet—if caveat emptor is good advice to keep in mind, salus populi suprema lex esto (“the welfare of the people shall be the supreme law “) is a good motto to live by.
(Incidentally, SEI authors Chuck Weinstock and Scott Hissam revisit some of the findings and analysis in the SEI report in a chapter in a great new book Perspectives on Free and Open Source Software (MIT Press) –I’ll blog more about the book soon.)
by admin on June 19, 2006 02:28pm
I’ve been traveling in Brazil recently, one of my favorite countries, and meeting with various customers, developers, IT professionals, government officials and topping it off with a talk at Linux World Brazil in Sao Paulo. On my flight over I was reading a translated version of TEMA magazine which discusses the Brazilian federal government’s IT spending. An interesting statistic from a 2003 study by the Department of Logistics and Information Technology is that Microsoft is just barely on the ‘top spend’ list of commercial software providers to the Brazilian Federal Government. In terms of money spent by the government on software, Microsoft comes in at number eight. Many would believe or would guess that Microsoft is the ‘big gorilla’ in the Brazilian market, which is why the Linux/OSS versus Microsoft debate in Brazil always seems so dramatic in the press. But, alas, we are number eight, in terms of money spent by the state.
According to this study, the biggest software suppliers to the Brazilian federal government (in Brazilian Reals):
Of course, this is just software costs you can imagine what that IBM number looks like if you add in services and hardware.
The reason, I believe, for the misperception that Microsoft is the single ‘foreign’ vendor is because most people ‘see’ Microsoft everyday on their desktops. So Microsoft becomes a singular metaphor (or poster child) for commercial software. The misperception even carries to the academic world, such as Benkler’s Wealth of Networks from Yale Univ. Press, where he discusses the role of free software, giving governments, “freedom from reliance on a single foreign source (read, Microsoft)” – page 333. But if this is true, shouldn’t free software also reduce reliance from the other sources of commercial software in Brazil (players 1-7 above)? Of course it should, but that’s not the general perception because some of these others, such as IBM, have put on the cloak of ‘open systems’ to market themselves as free software loving champions. But if that were really true, I wonder where all that money is coming from and then going to? So the bigger question is: how much do these commercial companies generate for the Brazilian software economy relative to what they are profiting?
This is very important for Brazil, particularly as it relates to exporting to developing nations; for example, from 2002-2005 Brazil has increased total value of exports to Africa, Middle East, and Asian from US$13.4 billion to $28.8 billion (Ricardo Neiva Tavares, Ministry of Foreign Affairs, The Economist, 27/5/2006). Generating revenue to Brazilian companies through software exports will be an important part of future growth in this area.
From a Microsoft perspective, we are seeing real value for Brazil using our software to build their software ecosystem, a few statistics about Brazil and Microsoft:
In my time here in Brazil I will have an ear out for how people are using OSS and how they are interoperating with commercial software – the best lessons are always found ‘in the field’ – and it will be interesting to test how much ‘what you see everyday’ effects the perceptions and how this holds up to reality in the cold light of day (or maybe a little warmer light of day here in Brazil).
P.S. – adding to this on my return home, after I wrote the above. Indeed the veneer of ‘free’ has faded in Brazil like it has in most countries as businesses enter into their 2nd, 3rd, or 4th ‘wave’ of Linux/OSS usage and realize that there are certainly still costs with ‘free’ software, such as in management, integration, security, people, etc.
by admin on June 16, 2006 01:21pm
Infrastructure Management and Strategic Design: Part 2 – Driving Network Efficiencies
Every computing device in existence today lives and breathes on some sort of a Network. Doesn’t matter if your Home PC is connected to a Cable Modem or if you’re office laptop is part of an extended WAN, your device is persistently living and breathing on the Network. From the minute you turn on your device, one of the very first drivers to be loaded are Networking drivers. Everything from DNS, DHCP to the simple sharing of email and office documents depends on the very basic function of the Infrastructure, the Network. So why is it that the same Network which is considered a Category-A asset in an organization is also sometimes at the very root of serious headaches for a Network Administrator. Thoughts about the faith and reliance we put towards Networking Infrastructure is simple staggering. Let’s examine why
If you’re a soccer fan like me, I am sure you have been glued to the World Cup coverage. As I sat with some friends and discussed our loss to the Czech Republic, I realized that in that group, it didn’t matter if you use Firefox or IE to get the game stats, you JUST wanted to know the score. The thought that followed was that maybe “Tools” shouldn’t be as hyped-up as they are sometimes as compared to the stream of “networking” that sustains them. Following that my thoughts turned to the topic of Net Neutrality and how Networks are ideally designed to forward packets regardless of their size, purpose and content. Hmm…this is interesting, I thought. My entire life as an Infrastructure Architect, I tried to come up with creative ways to manage and optimize network / infrastructure performance towards a better outcome for end users.
You see, some of the most perplexing problems for Network Admins is to control and manage “chatty protocols, broadcasts and bandwidth hogs”. How can we do that effectively? Let’s look at this closely: When I read a blog about Web 2.0 or something similar, some of the questions that pop in my head are “Have they considered what effect the implementation would have on their Network Performance” ? How far does the implementation of a new model go towards Application Bandwidth Testing and how inherently and intrinsically dependent we are on Networking.
Networking and the simple availability of bandwidth, wired or wireless, has become as much of an expectation as running water. A few months ago I remember someone saying “Network is like electricity – you don’t call up and thank the power company when you turn on the light, do you, nor do you ask them how much power you can use for your house”. Intriguing thought, nevertheless, for those Layer 2 and 3 warriors, the term “Port Saturation” should definitely ring a bell. Key contributors to reaching port density are instances when there just isn’t any more bandwidth to go around. To avoid choking up the network, I found some of the following tips helpful if you’re managing a small to medium sized network:
Alright, that’s it. I am logging off and heading to the movies to watch the Da Vinci Code. Will be back here next week and Thank You for tuning into Port25.
by admin on June 14, 2006 05:46pm
HOW-TO: CONNECTING SQL SERVER 2005 JDBC to JBOSS We have been working in the labs regarding testing and integration of jBoss talking to a SQL server 2005 backend.
We have seen some questions on how to connect a SQL Server in the Windows environment to jBoss. So I wrote a little how-to that describes just that.
It turns out that it is actually very simple.
Essentially, all that’s involved with installing the driver is:
<local-tx-datasource>
<jndi-name>DATASOURCENAME</jndi-name>
<connection-url>jdbc:sqlserver://DATABASE URL;databaseName=jboss;</connection-url>
<driver-class>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class>
<user-name>DATABASE USERNAME</user-name>
<password>DATABASE PASSWORD</password>
</local-tx-datasource>
NOTE: Substitute DATASOURCE NAME, DATABASE-URL, DATABASE USERNAME and DATABASE PASSWORD for your local values.
And that is all there is to it. jBoss now should be able to connect to the SQL Server.
Special thanks to Dan Simonton one of our penguins who did a bunch of the testing on this.
by admin on June 13, 2006 03:29pm
FreedomHEC trip report On Friday the 26th of April I attended the FreedomHEC Un-Conference (Yeah I am late with posting it). This was a two day conference which was held on the 26th and 27th. I only attended the first day. The FreedomHEC Unconference was billed as:
The hardware unconference where you'll learn how easy it is to make your hardware compatible with free, open source operating systems such as Linux, and available to new markets such as servers, next-generation entertainment devices, and more. Get answers on everything from kernel data structures to the fine points of licensing. Discover how participating in the Linux process is fast and simple, how the development process works, and where to get started.
The hardware unconference where you'll learn how easy it is to make your hardware compatible with free, open source operating systems such as Linux, and available to new markets such as servers, next-generation entertainment devices, and more.
Get answers on everything from kernel data structures to the fine points of licensing. Discover how participating in the Linux process is fast and simple, how the development process works, and where to get started.
It did not completely achieve this goal, but was very helpful to people who have never done device driver development for the Linux Kernel.
It was set up by Don Marti, and was attended by approximately 35 people of which 8 or so were current kernel maintainers. I think the enthusiasts outnumbered those people who would be writing company device drivers by 2 to 1.
The conference started with a suggestion session as to what people wanted to see so that a calendar could be created. As a result the first day calendar became this;
1. SYSFS OVERVIEW
2. LINUX SOCIAL ENGINEERING
3. SCSI Q AND A
4. ROCKETS
5. GETTING DRIVERS INTO THE KERNEL Q&A
Below are more details on the conference. My apologies if this at times is confusing, but I am working off my notes here...
This was an overview given by Greg Kroah-Hartman, who is the current Linux PCI tree maintainer (Among others he also does sysfs, kobject, debugfs and kref code). He works for SuSE Labs at Novell.
He presented a high level walkthrough of the sysfs subsystem (the /sys area) This is only available in the 2.6 kernels and is still evolving.
Sysfs (/sys) is a RAM file system, so anything created in there by humans after boot will be lost after any subsequent boots.
-sysfs shows all of the devices (virtual and real) and their inter-connectedness
In the past /proc has been used but /proc should be used for processes and not device drivers. /proc is the older method of creating device driver configuration files.
One of the reasons that /sys exists is to standardize configurations of various device drivers. /proc has been used/misused in the past for configuration representations/files/atributes, the data under this file system are different from programmer to programmer and device to device and are very hard to interpret unless you know the format.
/sys changes all that. It provides a standard method of defining and using device driver attributes. It is based off the principle of one value per file. And this value can only be a simple value, no histograms or large binary configurations (debugfs is specifically for this purpose!). Still people seem to break these rules at times.
Do a tree under /sys to see what the structure is, and cat the files for the values.
Power management used to turn things on or off, is not fully working yet. USB is getting there for power management; the rest, not yet. Put in a USB Pen drive for example and see /sys/block change real time (/sys/block/sda). If you unplug a device, udev will take care of cleaning up. You do not have to un-mount or do anything else. (If you do that to a drive you are writing to, of course you are on your own) You can also mount things by label easily.
If a change is made to a device attribute in /sys, an event is triggered that programs like hal can get real time. General guideline, if you write a user space program/driver, tie it into hal.
Also, /dev is now a RAM file system in 2.6 (Some distro’s might not have implemented this)
If you want to add proprietary drivers, they will live in /lib/usdev/devices, this is when they are not sysfs aware. This location is persistent.
This presentation was given by Randy Dunlop, A past USB and Kernel Janitor and maintainer.
The presentation was given to give people an overview of do’s and don’ts when they start submitting code to the kernel. A lot of the information given was common sense but a lot of people do not follow the rules. (e.g Rules so Obvious that they are not followed)
The presentation was very much in Bullet form and so are my notes on this, so they might not flow as well as they could.
SOCIAL ENGINEERING
(Meritocracy) Development.
Massive amounts of open communication via email etc.
LINUX CULTURE
LINUX DEVELOPMENT VALUES
THINGS TO AVOID
WHEN NEW INFRASTRUCUTRE IS NEEDED
DRIVERS FOR NEW HARDWARE
NEW DRIVER DEV
COMMUNICATIONS
MAILING LIST etiquette
GENERAL
In all the presentation was a crash course for people who have never done collaborative development, and tried to prepare people what is ahead and what kind of commitment will be needed to be part of the Linux kernel development.
Led by James Bottomley SCSI Subsystem maintainer.
I do not have many notes on this part because not much was said to keep notes on, a few things that I did write are as follows (Second item is of great interest)
If you want to get an idea of how SCSI drivers work, check out the 53c700 which is an excellent driver to learn from. It is older but continuously maintained.
There is kind of a new concept called a Target driver. Target drivers make it appear to the outside world as if it is a driver (A virtual driver more or less), but instead connects to the actual physical device drivers. Allowing, for example, to turn a Linux box into a RAID device. Then you can talk to the target driver as if it is one device. Not many of them out there yet, but there is one from IBM which James did not think was a very good. And there are a few on the way. There are two people (names escape me currently) that are actively developing them. In my opinion this is a pretty interesting concept that allows you to do a whole bunch of cool things. (e.g Network routers, Cheap raid devices etc)
You can (should?) write SCSI drivers in user space.
A lighter presentation was given by two students from Portland state. They are using Open source hardware and software to build rockets. Some interesting reads can be found on that project here:
http://psas.pdx.edu
This was pretty much done by Greg.
Some source code control software of choice of the Kernel Maintainers
Testing Kernel code is hard, but crashme is used by the maintainers be used for (stress) testing kernel stuff.
Greg was very adamant that they take any device driver. No matter what HW it uses, as long as you or somebody else is willing to maintain it they will take it. Old/New it does not matter. Even if there is only one user for it they will take it.
That’s all folks.
by admin on June 12, 2006 07:12pm
Peer Review, Port 25 and Good Things to Come The peer review system is a much revered and reviled system within academic circles. Tim O’Reilly’s recent comment on Nature magazine’s fresh look at peer review piqued my interest. Interest piquing is always bad, because its sets off the oft dangerous process of “thinking”!
There is a great description of peer review on Wikipedia. I encourage you to read the complete description, if only to get a sense for the issues involved in peer reviews.
In short, the peer review process:
ensures work submitted is commented on by experts in the field exposes the possibilities or the mistakes in an intellectual approach improves research quality in most cases even where the experts misunderstand the research! Pre-history of Peer Review
I did all my research in the Internet/Arpanet era, but I am old enough to be able to see the tail end of the paper based peer review process. Talk about snail-mail!
First you send off a request for re-prints for all the papers that you think you will need for your research. Each of them costs about $$ in reprint costs. Of course you order a lot of them since you don’t know which ones you will find useful. Then you write your paper (it takes a couple of months for you to get all those re-prints btw!) and send them to the editors of journals, referees in conferences etc. in those nice brown paper envelopes. The reviewers send you their comments (this is a couple of months later). You modify your manuscript and repeat the process. I am surprised the world made any progress at all!
The predecessors to the World Wide Web include FTP, WAIS and Gopher. The first wide use of all these technologies was for distribution of research papers. (That may be just my bias speaking, since I was a computer science graduate student in the 1990s and that’s all I could use these technologies for!). Tim Berners-Lee invented the World Wide Web based on the vision of peer collaboration between scientists after all!
With the advent of the Internet peer review was now potentially much more efficient. Research papers and reviews could now be retrieved in near real time. However it has taken a little bit of time for the academic greybeards to catch on. My personal opinion is that the way forward on peer review was shown by the Open Source community.
Here comes Open Source
The first thing that comes to mind when talking about peer review in the OSS world, is in the realm of development. You know the usual story, person develops code, throws code for peer review, corrections and comments come back, code is unstoppable! My friend and colleague Hank Janssen does say (see his handsome visage) “OSS world peer review delivery can be very brutal. The social aspect of peer review is not very becoming many times”. But that is what it is!
However there is another peer review system within the OSS framework and that is called OSS Intelligence. OSS Intelligence means the application of collaborative principles developed by the Open Source Software movement to the gathering and analysis of information. Wikipedia is an outstanding example of this, but any of the major discussion boards supporting OSS users are also examples of this.
Now academia is catching up – and Nature magazine is trying a new peer review process in which a manuscript is thrown open for public peer review and the authors are invited by the editors to reply to comments from the public.
Peer Review and You
That got me thinking (remember the dangerous activity I alluded to before?). Aren’t our Port 25 readers our peers in ensuring that we have great, interoperable, easy to manage, easy to install, easy to maintain software that changes the world? Why not use this forum for peer review of what we do in our lab?
The proposal is that we open the Open Source Lab project planning to the Port 25 community. The project research methodology would be proposed on the site with methods, experiments, hypotheses and data sets all laid out for “peer review”. Please be gentle - on second thoughts this is dealing with the Open Source community, so just be natural! We will open this for a period that makes sense. Then we will carry out the research based on what we hear back from our peers right here on port 25. The Open Source Lab will carry out the research and put the conclusions and feedback back to the Port 25 community for review. We will then publish this “peer reviewed” content in its final form.
Of course you are always welcome to suggest what those projects should be!
As an example of the kind of information you will see, let us take a project that the Open Source Software Lab recently executed. This involved simulating RedHat and SUSE in a production environment and evaluating the patch management experience.
The research plan will describe:
The peer review feedback could let us know (just a sample):
Now in the best traditions of peer review – what do you think of this idea?
by admin on June 07, 2006 06:55pm
What is a guy like me doing in a place like this...?
Well, by now some of you probably saw my video interview on Port25, so I wanted to take this opportunity to tell you more about my background and what made me decide to work for Microsoft. For those that know me in real life, it came as a shock to find out that I decided to work for Microsoft. But more about that later…
I started in the industry about 20 years ago, more specifically working at AT&T and Philips who where in a joint venture at the time. I started working for them initially as a contractor and that is where I had my first exposure to Unix operating systems. Very quickly I started doing System Administration and tools development. Very early on this turned into full time development for Unix.
After a year or so, I was invited to transfer to the AT&T UK division and there I developed a database system that tied all AT&T field support offices together. This allowed any office in the world to contact any other office in a matter of minutes with any support issues that they had. Once transferred, these tickets stayed in sync no matter what organization(s) they were sent to. This system was 100% custom written. It had its own database system that I wrote from scratch. It is probably still one of the fastest custom database systems around. This whole system was written before the days of Internet, and high speed data links available to all. It was the time of uucp, rje and vt100’s for those who still remember that J…and Linux did not exist. 1200 Baud Rixon modems (sometimes even 300 baud) were a lot of the ways this system communicated. It was finally retired about 3 years ago.
AT&T in the US asked me to work in the US and modify and train other people to extend that product. That offered me a whole bunch of new opportunities in the form of area’s to work on. In that capacity I worked a lot on the modified Unix systems that AT&T used in their 5ESS telephone switches. As such I did part of the process handler that that version of Unix used for System V. I also worked inside AT&T on the customization of the X windows system. I met and got to work a little with Stroustrup and Ritchie.
I will fast forward a little here before people fall asleep. I ended up staying in the US. And after 6 or so years I left AT&T. After this I worked for Cincinnati Bell Information Systems. They, at the time, were the largest Cellular billing system service provider in the US and in the early 90s, I was the lead designer and developer on a real time Point Of Sale cellular phone activation system for Sprint. Now that is not such a big deal anymore. People find it normal that you walk into a store and walk out with an activated cell phone. At the time, this was the first and only system that did that. What was even cooler is that it was 100% based on an OSS operating system and development tools. It was deployed on one of the very first Red Hat releases. It took a lot of persuasion but we where able to convince the customer that this was the way to go.
Anyway, after this I worked on several other systems, a real time fraud management system that was based on OpenBSD, and I did a bunch of Device driver work for that as well as a lot of work in cellular billing systems. These are systems that process 15+ million records a day.
For the last 5 or so years I have been a systems architect working on a variety of systems mainly used in the telecommunications world. For those who have worked in that area before, it is a world that has many different systems and interoperability is a very big concern.
All the systems were Unix based. And throughout my career I primarily worked with Unix, Linux and OSS products.
Now how does somebody with a background like this end up working for Microsoft??? Well, that is something that I am still shocked about every day.
In the past I stayed away from MS products because of various reasons. They where not fast enough, secure enough, did not talk to things other than MS products well enough, etc, etc.
But working in the industries I have been in I found that you have to be pragmatic, fanaticism about one product line or company or another product line and company does not work very well. People want to work with fast, secure and easy systems and every company I have ever worked for has a large variety in products and OS systems. Yet, having said that, I love working with Linux and OSS products.
So, when a call came a few months ago from a company that presented this job to me I was very much torn. A few weeks went by, and more agonizing took place. I kept on coming back to one thing; it is very easy to make comments and go home at night, it does not take a lot of effort. But what do you do if somebody gives you an opportunity to, in some way, change the things you have always complained about? It sure isn’t easy, but it puts me in a place to put my money where my mouth is.
So after I got to talk with people at Microsoft (Bill Hilf and Sam Ramji to name a few) I was very interested to find what they thought this department should be. And it very much fit with what I wanted to do if I ever ended up working at Microsoft.
So what will I be doing and focusing on primarily inside Microsoft??? Well my primary area is research into Linux and OSS Applications. We figure out what makes them tick, and we figure out how we can better interoperate with them. There are some interesting things coming down the line from us in the future that might surprise people.
So, I am very much looking forward to constructive criticisms, suggestions from you all and I am very excited to work you in future. Drop me a line and give me your suggestions as to what you want us to look into or help you with.
by admin on June 06, 2006 05:49pm
New friends from Linux World Brazil I’m recently returning from Linux World Brazil where I presented on ‘OSS and Microsoft.’ One night in Sao Paulo I had the opportunity to chat with two of the leading OSS technologists in Brazil – Cesar Brod and Helio Chessini de Castro. Cesar has an interesting background, working at Tandem from 1992-1998 and at various companies throughout Brazil, including his own consulting company. Cesar is involved with Linux International and was also one of three finalists for the Free Software Community Award in 2004. Helio is well known in Brazil as one of the key developers of Conectiva and a prolific KDE and Linux developer and instructor. He currently works for Mandriva (who acquired Conectiva to form the Mandrake+Conectiva distribution). If you’ve spent time in the Conectiva or KDE developer world, you certainly have heard of Helio. We had a great conversation about the Linux/OSS environment in Brazil, particularly the history of this community. Cesar echoed a statement I’ve heard from quite a few other OSS developers about those who ‘do’ actual technical work in the OSS community and those who ‘talk’ about it. His point was that there has always been a developer community in Brazil – before OSS hit the scene – and Cesar has tried to keep this community focused on the real work, not simply the rhetoric and politics. Cesar is a well regarded OSS participant and has some great stories about his early days at Tandem, how he discovered Linux (an alternative from flying back and forth from city to city to use a Unix machine) and how the OSS community has evolved. Both Cesar’s and Helio’s pragmatism, honesty, and open mindedness show their experience and wisdom – I hope to continue our discussions in the future. Cesar was also kind enough to introduce me to his friend John “Maddog” Hall. Maddog is well known in OSS circles and although we’ve heard about each other, this was the first time we had the opportunity to meet face to face.
(From left: Maddog, Me) Maddog’s keynote was right before mine, so I had the chance to see him present on ‘Total Value’ of software. It was interesting, and although I disagreed with some of his points (and the ‘blue screen of death’ jabs – come on, used any Microsoft software since 2003?), Maddog is a good presenter and hit some important points about standards, competition and choice that I agree strongly with. Shortly after our presentations I had the chance to talk with Maddog. We had a good conversation about change at large corporations (Maddog is a former Digital guy), and software trends. Despite the name, Maddog is a balanced industry veteran who I could talk to easily for hours, his perspectives and insight are valuable and although I think we may disagree on some points, there are many others where we find harmony. I have visited Brazil many times, and I was honored to speak at Linux World Brazil, but this may be one of the most useful of my trips as I was able to meet customers, partners, Microsoft teams, government officials and developers and thinkers in the OSS world. New friendships are always important to me, but it’s the cascade of new thoughts from all of these various discussions that keeps me awake on my flight back to Seattle. -Bill
by admin on June 06, 2006 03:30pm
Research Strategy Corner: Disambiguating “Open”
Disambiguate (transitive verb): to establish the true meaning of an expression, regulation, or ruling that is confusing or that could be interpreted in more than one way
I’m a Research Strategist with the Open Source Lab here at Microsoft. When folks ask what that means , I usually tell them the second-best definition of “strategist” I’ve ever head is “a researcher who gets to make stuff up”—but the first-best is “someone who establishes a series of steps to achieve a goal.” The latter is what my job is all about—in the process, because it involves synthesizing both technical information and insights from computer science, organizational science, and sociology research, I sorta get to make stuff up—as long as the math works (which is probably a little bit different from what the average civilian bystander thinks of as “making stuff up.”)
What’s the goal? The title of this post summarizes it concisely: disambiguating “open.” When the phrase “open source” is used for example, it could represent a simple descriptive statement of fact about code visibility (read any good mash-ups lately?); it could also be referring to software artifacts available under a fairly wide range of license types…or it could be intended to refer to something compliant with a very specific set of criteria like the Open Source Initiative’s ten-point definition (http://opensource.org/docs/definition.php). It could be referring to one of tens of thousands of single-developer projects on SourceForge—or to highly coordinated efforts like FreeBSD; and on the other hand altogether, it could be in marketing materials from big corporations like IBM or Novell (if this animation is still up on Novell’s site you can experience a dizzying array of suggestions for what “open means to your enterprise:” http://www.novell.com/solutions/?sourceidint=hdr_productsandsolutions ).
“Open” is one of those words that today in the software domain is increasingly becoming probabilistically uninformative…the word as applied to an endeavor (like a software development project) or an artifact (like a piece of software) less and less enables you to more accurately predict attributes of the endeavor or artifact—because attributes of the endeavor (like who built it and how) and the artifact (like architecture, coupling, interaction paradigm) may actually help you better predict what will happen now and further on down the road if you choose to take a dependency on something.
I don’t care how the characterization “open” makes you feel, whether it is nervous or giddy with excitement: my objective function is the ability to use one bit of information to reliably predict other bits of information. In this space I’ll share our efforts to go about doing that with regard to “open” and what we find in the lab and in the world of academic research—but first I’ll give you some visibility into how I start out structuring lines of inquiry.
There are a few different approaches you can take to things: for our purposes here, an analytic approach is like an argument from first principles—the position of Free/Libre software advocates is essentially analytic, as their argument is software should be a certain way given the set of principles they start from, and there really isn’t any evidence you could collectout in the world that would change their minds. This discussion isn’t particularly relevant to my day job. An empirical approach is all about data and probability: if you know foo you predict bar better. This is entirely relevant but is exactly what we don’t know enough about yet. A phenomenological approach (http://plato.stanford.edu/entries/phenomenology/ if you really want to know) starts with experience as it is experienced--and this is useful for starting to disambiguate open source. Here’s why: I don’t want to argue about what’s open and what’s not or about whether things should be a certain way—I want to build an informative base of data that lets me characterize and analyze endeavors and artifacts underneath this fuzzy term “open”. So we can start by asking: what would I—and, if there’s enough established shared meaning, other people—experience as a phenomenon that is certainly “like open source” versus certainly “not like open source.” I suggest two sets of statements along an axis shown in figure 1, below. Once we have this down, we have a starting for the collection of empirical data that ( if our starting point is right) will position endeavors and artifacts somewhere along a continuum between the two extremes.
Figure 1: Phenomenological approach to characterizing endeavors
I won’t start into operationalized definitions here, because to some extent that would defeat the purpose of capturing a top of mind response to the statements themselves. So what do you think: top of mind, as you experience these collections of statements, is the essence of like and not like open source represented? Do they raise questions? Controversy? Let me know and we can dive in to where some of these come from (yes, I said my experience-as-experienced, but remember, when I make stuff up the math has to work…there’s lot of great research out there that can help tune these characterizations).
Disambiguation: Because what you don’t know you don’t know probably will hurt you