Follow Us on Twitter
I’m really excited to be able to give you an update on our strategy and product roadmap for Big Data, especially around our embrace of Apache Hadoop as part of our data platform.
As you may remember, at the PASS Summit last October we laid out our roadmap for Big Data, with Microsoft Corporate Vice President Ted Kummert announcing plans to deliver enterprise class Apache Hadoop based distributions on both Windows Server and Windows Azure.
Even more importantly, he announced that Microsoft will be working with the community to offer contributions for inclusion into the Apache Hadoop project and its ecosystem of tools and technologies.
Now, this week at the O’Reilly Strata Conference, Dave Campbell, a Microsoft Technical Fellow, will give a keynote address on Wednesday morning where he will talk about how we are demonstrating our progress on this front as we strive to help organizations derive new insights from Big Data.
In a blog post today, Campbell notes that Microsoft has been working hard to bring the simplicity and manageability of Windows to Hadoop based solutions, and we are expanding the reach with a Hadoop based service on Windows Azure.
“Hadoop is a great tool but, to fully realize the vision of the modern data platform, we also need a marketplace to search, share and use 1st and 3rd party data and services. And, to bring the power to everyone in the business, we need to connect the new big data ecosystem to business intelligence tools like PowerPivot and Power View,” he says.
Microsoft is working closely with the community and ecosystem – including partners such as Karmasphere, Datameer and HStreaming – to deliver an open and flexible platform that is compatible with Hadoop and works well with leading 3rd party tools and technologies.
“We have recently reached a significant milestone in this journey, with our first series of contributions to the Apache Hadoop projects. Working with Hortonworks, we have submitted a proposal to the Apache Software Foundation for enhancements to Hadoop to run on Windows Server and are also in the process of submitting further proposals for a JavaScript framework and a Hive ODBC Driver,” Campbell says.
As Gianugo Rabellino, Microsoft’s Senior Director for Open Source Communities said last October, these moves benefit not only the broader Open Source community by enabling them to take their existing skill sets and assets use them on Windows Azure and Windows Server, but also developers, our customers and partners.
“It is also another example of our ongoing commitment to providing Interoperability, compatibility and flexibility,” he said at that time.
You can read Campbell’s blog here and learn more about what we are doing for Big Data here.
As you know, educational institutions across the globe face the challenge of trying to meet accelerating technology demands with limited budgets. That is why I am really pleased to report that, today, Moodlerooms has released an open source plug-in that integrates Microsoft Live@edu with Moodle, the world's leading open source Learning Management System with 55 million users.
With this integration, Moodlerooms will make the rich functionality of Live@edu directly accessible within the Moodle 2.0 and 1.9 environments via single sign-on, further enabling teachers and students to access the quality, enterprise-level tools they need to effectively teach and learn online.
Microsoft Live@edu makes hosted email, communications and collaboration services freely available to educational institutions and currently benefits more than 22 million people worldwide.
Moodlerooms’ Microsoft Live@edu services plug-in for Moodle will bring Microsoft’s Live@edu email, calendar, instant messaging and Bing services right into the Moodle classroom environment.
The plug-in is freely available to download from the moodle.org plug-in directory here.
You can read more about this on the Moodlerooms release here, as well as on the Microsoft in Education blog and the Microsoft Openness blog.
The success of the recent Screen Actors Guild (SAG) Awards ceremony was buoyed by the move of its Drupal-based website hosted on internal Linux servers to one hosted on Windows Azure.
The SAG Awards site is a highly visible, high-traffic website running on Drupal. Hosting it on Azure provides a scalable, public cloud environment for SAG team. They can tune up or down the compute and storage requirements according to expected website loads, thereby getting a more scalable, manageable and cost-effective solution for running their site.
SAG also gets the benefits of PaaS – no need to manage the operating system patches, virtual machine images, network topology etc. This is particularly useful for SAG as the site has stable traffic for nine months, but which spikes for the three months from when award nominations open to the night of the event itself.
The SAG Awards site was previously hosted on internal Linux boxes. In previous years, performance was negatively impacted by site outages and slow performance during peak-usage days, with SAG having to consistently upgrade their hardware to meet demand for those days. That upgraded hardware was then not optimally used during the rest of the year.
The usage pattern for the SAG Awards site fluctuates, but spikes between November and February when the site is used for SAG award nominations in early November to the actual announcement of nominations in in mid-December. Peak usage is on the night of the awards ceremony where multiple uploads of pictures, news articles, and site visits happen.
What is even more impressive is that both visits and page views almost doubled on the night of the event. In 2011, some 222,816 people visited the site and 434,743 pages were viewed, while this year there were some 325,303 site visits and 789,310 page views, reflecting the stability and performance of the site on Windows Azure.
Microsoft started working with the SAG Awards team in May 2011, when their CIO Erin Griffin joined the Interoperability Executive Council (IEC) - founded by Microsoft in 2006 with a goal of identifying the industry’s greatest areas of need and to work together to create solutions - and attended a council meeting.
In September Mike Story, SAG’s chief architect, attended an IEC work stream meeting and asked for Microsoft’s support in porting the site to Azure. The Business Platform Division’s Customer Experience (CAT) team, the Interoperability group and Windows Azure all started working with SAG in early October and, on December 20, 2011, the site went live on Windows Azure.
“We moved to Windows Azure after looking at the services it offered,” said Erin Griffin, CIO at SAG. “Understanding the best usage scenario for us took time and effort, but with help from Microsoft, we successfully moved our site to Windows Azure and the biggest traffic day for us went off with flying colors.”
This is just one real world outcome from the IEC, which has counseled Microsoft on many interoperability topics and introduced a number of real world scenarios for discussion. The IEC, working together with Microsoft, has developed a number of solutions for these scenarios, with this one for the SAG Awards being the latest.
Curt Peterson, Microsoft’s Principal Group Program Manager, BPD Customer Experience, notes that the success of Sunday’s SAG Awards ceremony underscores how Windows Azure is a scalable, open Cloud platform ready for production use. “We are committed to making it easier for all our customers to use cloud computing on their terms with Windows Azure,” he says.
My colleague Grigori Melnik has a great blog post about the CQRS Journey project, which aims to deliver the source code for a complete, working reference implementation (an end-to-end sample app) that illustrates the key concepts, patterns, and approaches related to Command Query Responsibility Segregation and Event Sourcing (CQRS & ES), and a written guide to accompany the code, provide explanations, context, and references to other relevant material.
CQRS is an architectural pattern in which you separate your read model from the write model. The benefits include almost infinite scalability, adaptability to changing business requirements, resolution of concurrency conflicts, and overall simplification of the design.
As Grigori notes in his blog, they have positioned the CQRS guidance project as a learning journey. An advisory board has been established and some public consultation has taken place that helped scope the project initially.
"Unfortunately, there’s no solid guidance around implementing it and so we are embarking on a development project where we will implement a real world, non-trivial system using variety of CQRS approaches and share those learnings with the community," he told me. "The good news now is that we can have members of the community to participate in this journey together with us."
He goes on to say that while this is good news, "we recognize that for this project to be successful, we need to be not only open and transparent, but we also need to collaborate with the community (in its way a global village) more closely. That’s why I am extremely happy and proud to announce that for the first time in the history of the Microsoft Patterns & Practices team, the following: In the true spirit of open source, we will be taking community contributions on the CQRS Journey project,” he said.
This means:
The project will be hosted on GitHub and the contribution guidelines will be published soon. You can read his full blog here and find the project site here.