This is a guest post by Sriram Rao, Principal Scientist Lead in the Cloud and Information Services Lab (CISL) at Microsoft. His current research interests are in building storage and compute infrastructure for processing big data. Some of the results of his research have been released as part of two open source projects that he started: (1) Sailfish, a compute infrastructure for big data, and (2) Kosmos file system (KFS), a distributed file system for storing massive amounts of data. KFS is currently deployed in a production setting, on the scale of 1000's of nodes, where it is used to manage multiple petabytes of data.
See the remainder of the post at http://blogs.msdn.com/b/silverlining/archive/2012/09/27/quantcast-open-sources-their-file-system-based-on-kfs.aspx.