SharePoint’s Founding Father, Jeff Teper, Talks About the SharePoint History, Vision, and Lessons Learned.

Hey everyone, I just wanted to share a post from SharePoint's founding father and Information Week's 2009 Top 10 Innovator and Influencer, Corporate Vice-President Jeff Teper, where he talks about the SharePoint History, Vision, and Lessons Learned.

SharePoint History

Hello!

As we are just two weeks away from disclosing SharePoint 2010 at the SharePoint Conference starting October 19^th, I wanted to write three posts to provide context on the upcoming release. This first post will cover the history of SharePoint. I hope it will provide some useful perspective behind our vision and what we have learned as well as a few fun anecdotes. The second post will cover the engineering process for SharePoint 2010 - how we design and develop SharePoint in the Office team, what new approaches we have taken during the 2010 development cycle and my take on a few frequent questions I hear from customers and partners. The third post will coincide with the opening of SharePoint Conference and cover the major feature investments. After that, our team will start blogging in depth about the new SharePoint capabilities. For folks who cannot wait, we have highlighted a few of the new features on the SharePoint 2010 Preview Site as commented on several Office 2010 client capabilities including a few points of SharePoint integration on the Office Engineering Blog.

Before we jump in to this post, I want to thank everyone who has been with us on the journey so far and is supporting us in the release of SharePoint 2010 and feedback for releases beyond. While I will talk about the efforts that evolved into SharePoint, its origin dates to around March 1998 when we first started planning the projects that led to SharePoint. Since the first beta of the first release, we have been extremely fortunate to have your support and hear your feedback. It has been extremely rewarding to see all things you are doing with SharePoint. We have posters of many of your sites on the walls of Building 16 on the Microsoft Redmond Campus and these are a source of great motivation and pride for our team to take things to the next level. We always appreciated your ideas and feedback friendly or otherwise about where the product should go or how we could help you better. We think SharePoint 2010 will be another big step forward but we know there is more to be done and are thankful you care enough to keep pushing us. Finally, we want to thank the Microsoft teams around the world who has worked so hard to build and support the product with special thanks to a few dozen people who were with us from the very beginning in 1998. We are very fortunate to have a great team, customers and partners and that is what keeps us fired up to come to work every day.

Ok, on to the post.

Pre-History

As long-time Microsoft followers know, in the 1990s we had several efforts targeting information access and sharing. The rise of web on the internet and intranet was a great catalyst to focus these plans. I will highlight three threads that I consider part of the SharePoint pre-history in the late 1990s. The first thread is focused on end users via the FrontPage and then Office Server Extensions. These extensions installed on web servers to let users create and edit web sites, post Office documents, participate in discussions and get subscriptions. Many ISPs hosted the Server Extensions helping millions of consumers and business users collaborate on the web. There was strong integration with Office 97 and Office 2000 so you might say these were the first cloud-based Office web collaboration releases! The second thread focused on more sophisticated web site developers and was called Site Server which was a grab bag of tools and services to help create and manage sites. Finally, we had a set of internal incubations. One was called "TeamPages" which was developed inside the Access team enabling users to create and edit simple web-based lists. Under the covers, this was one of the first uses of XML and was the genesis of the "CAML" markup inside SharePoint that pre-dated XSLT. The other was called the "Digital Dashboard" which was initially an effort between our marketing, Excel and SQL teams to make it easy to create dashboards in Outlook but quickly moved to a web-centric design. We learned a lot about the needs of end users, developers and IT from these efforts and the strengths and weaknesses of projects that helped us design the first release of SharePoint and make a strategic bet for Microsoft in the second release. This process continues at Microsoft as we have both top-down planned efforts as well as organic prototyping in our product and research teams. For example, our Office Labs Group created a social networking project called "Townsquare" that we used a bit inside Microsoft before refining the ideas in SharePoint 2010's new social networking features.

Vision

As part of planning of what became SharePoint(s) V1 and Office XP, we did lots of research around the world. We visiting different size and type organizations and met with a wide range of users, developers and administrators. We talked to lots of people and looked at lots of intranets, document and knowledge management deployments. It was immediately clear there was an opportunity to help both end users and IT be more efficient and effective. Many customers told us about intranets or ECM systems they had rolled out with lots of fanfare and then failed to get used. The web site got stale. Users shared documents via e-mail vs. the official repository. Many customers told us about their challenges with lots of different silos that had grown up to meet the needs of a range of different business units. One Fortune 50 company told us they had counted and had 600 different content management systems! Imagine the difficulty for users trying to collaborate, developers trying to link business processes and administrators trying to keep costs down and reliability up. Users told us they wanted to go around IT. IT told us when they did this they create flakey custom web sites. But as with many latent needs, there was no single definition of the ideal solution in the industry. So we worked on a plan to address both usability and manageability needs. As I was writing the blog, I went through some of our early presentations and I was struck by a one slide PowerPoint deck we gave to Bill Gates early in the planning. We knew reviews with Bill were always intense so we decided to come in with a single slide explaining our vision with lot of screenshots of what we thought the product might look like. We have lots of fancy slides now to explain SharePoint but I thought it would be fun to show what we shared with Bill exactly as is below. And yes it was an intense meeting!

SharePoint V1 and Office XP

As some of you recall, there were two SharePoints at the beginning. These evolved from projects codenamed "Office Server" and "Tahoe" around the Office XP development cycle. "Office Server" evolved out of the FrontPage and Office Server Extensions and "Team Pages" and targeted simple, bottoms-up collaboration. "Tahoe" was built on shared technology with Exchange and the "Digital Dashboard" and was targeted at top-down portals, search and document management. They shared Office client integration and "Tahoe" had web part and search links so top-down portals could contain integrate with bottoms-up sites. Coming up with the name "SharePoint" was a fun part of the process. We were in a giant conference room with dozens of proposed names on the wall. Everything from meaningless words we would imbue with meaning in the market to "Office Document Server" which was descriptive but boring and limiting. We agreed on "SharePoint" in a single meeting but had many discussions then and in the years about what were the right words around "SharePoint". We eventually did elect to use the same name for both projects to convey the integration and future strategy. We decided to ground "Tahoe" in the "Portal" category because it was an area of a lot of buzz ("Portal Power" had was a big cover story for InformationWeek, there were portal conferences, reviews, budgets, etc.) and we had seen Lotus Notes and Exchange had struggled to define a new groupware category. So "SharePoint Team Services" and "SharePoint Portal Server" were the first names. The most memorable part of this release cycle was the end game. I remember hours being in what we then called the "war room" (a hyperbolic, if not insensitive engineering term that we generally avoid now). Projected on the wall was an Excel spreadsheet with the status of 20 customers we wanted to know were live with their deployment before we shipped the software. Program managers would come into room with updates from daily calls with these early adopters about issues, how much content was stored in the system and number of active users. When everyone was green, we released and had a fairly memorable ship party. As we still do today, we had screenshots of each of these customers' portals papered to the walls so everyone in the team would know we weren't writing code for ourselves but to help these customers solve business problems. One of the most gratifying aspects of this first release was the reviews. EWeek's headline was "SharePoint's a Hit" and Computer Reseller News called SharePoint a "Whopper of a product". But we knew there was a lot more to do to build a truly integrated product and experience between the two SharePoint and we could do a better job of holistic planning across scenarios and technologies. Since not many people see a server software box anymore vs. just getting the CDs or downloads, I thought it would be fun to show the SharePoint 2001 package:

SharePoint 2003 and Office 2003

While it was great to see the rapid adoption of the first release of STS and SPS, it was clear customers wanted a more integrated and comprehensive solution from us. As just one example, they told us like they liked the WYSWIG HTML editing of SharePoint Team Services and the Web Part declarative and reusable editing of SharePoint Portal but wanted to use both models on the same site? Inside Microsoft, we were deciding how much to bet on SharePoint. Would it be a good product in a broad family of servers or an even more fundamental bet for the company? It was exciting period because we were at an inflection point on both our platform and application strategies. On the platform side, we were making a very fundamental bet on Web Services including collaborating with vendors across the industry on the evolution of XML, SOAP and much more. We saw this and the increased maturation and scalability of SQL Server would give SharePoint a powerful foundation for highly scalable web-based solutions that eased developer integration and customization. With growing computing power, we could host even the largest companies on a few sever farms (Microsoft has 3 for its worldwide SharePoint deployment). On the application side, we were hearing customers wanted Office to go beyond personal productivity to organizational productivity and we had to decide whether Microsoft would invest in content management, portals, unified communications, business intelligence and many other new scenarios. As you can guess now, in classic Microsoft fashion we decided to bet big. However, we were pretty quiet about until we were close to beta of the 2003 wave. The entire Office team had SharePoint as a vision pillar and we designed the user experience, architecture, extensibility and in a more holistic way than we had done in the previous release. We evolved STS into a more scalable and flexible platform and build SPS on top of it. One of the many heated discussions during this time was between the WYSWIG and data-driven camps on web user interface framework and we ultimately reconciled these in the SharePoint page model where pages have zones where the web site owner can decide who can add web parts. At this point, we also knew that customers, partners and Microsoft would be hosting SharePoint on the internet and we made a lot of fundamental investments in role-based delegation, partitioning, stateless front-ends, etc. to enable this. These key architecture bets enabled us to ultimately offering SharePoint Online to both dedicated and multi-tenant customers which we ultimately did in the 2007 release. While this was a long release, we could not have gotten the integrated experience and platform as lots of different skunk works efforts. It really needed teams across the organization to work every day to research, plan, develop and test together. At the same time, it was a very aggressive project and we had to make a lot of tradeoffs about what to defer to the following release. One of the most controversial was item level security. We knew this was a very critical feature for some projects and a big checkbox even for people who wouldn't use it. However, it was going to be huge investment for us on top of everything else we were doing and was not realistically going to make the 2003 schedule. So we decided to defer it until the next release when we could do it right. We still use this as an example in the team of the tough tradeoffs needed for software projects. One of the best bets we made in this release was MySites. This was a classic bottoms-up feature from the development team. In this era, most portal products allowed users to have personalized pages ala MSN, Yahoo, etc. The team proposed we give every user a personalized site and is still very proud they shipped it in 2003 before there was a MySpace or Facebook. It is a great example we talk about in product development about not always doing what the customer asks for explicitly but what we think they might eventually find most useful. When we first released MySites in 2003, many IT people first asked us how to turn it off. By 2007, they were asking us how to turn it back on as their employees and even their CEOs were asking them for enterprise social networking. Below is a picture we used many of times inside and outside the team to focus the 2003 team on what I call "The Scalable SharePoint Release". We launched it as SharePoint Portal Server 2003 built on top of Windows SharePoint Services V2. There were many important lessons from this project about the power of shared vision and a collaboration culture. However, we learned while we provided IT a central infrastructure users where users could go create their own sites, we had not yet provided all the management tools and governance guidelines people needed. That and depth category investments were the focus on SharePoint 2007.

SharePoint and Office 2007

With the integrated experience and framework in place, we began to add depth in the various modules of SharePoint. Hence I think of 2007 as "The Pie Release" because of the picture we used below to describe it. Content Management was our biggest area of feedback and, therefore, investment. As customers put more content in SharePoint and built more sites with it, we got a lot of requests to take this to the next level - more formal processes, more oversight and more sophisticated web publishing. Building on the new Microsoft platform technologies such as Windows Workflow and Windows Right Management, we invested document and records management across the server and client. We were very aware customers had a wide variety of ECM applications in place. So while we invested to be a leader, particularly breaking down many of the user experience and programmability barriers, we made sure SharePoint was an open platform and worked with vendors across the industry on a variety of integration approaches including published APIs and protocols. One of the biggest sources of feedback internally and externally in 2003 was the relationship between SPS and our Content Management Server (CMS) 2002 web publishing product which was used by a lot of internet sites. The teams came together and proposed a shared experience and architecture that was one of the highlights of the 2007 release. No longer do the pictures of the SharePoint sites just sit on the walls of Building 16, you can see themselves many of them live on the internet by navigating from a growing number of 3^rd-party blogs and sites on the internet likes www.topsharepoint.com. This was another good example of the power of a shared long-term vision. To be frank, we had a lot of pressure to do something more incremental. Complementing content management, we increased our search depth with a focus on new relevance algorithms and innovative people and business data search. Finally, to enable customers to build business process integration and business intelligence portals, we added Excel Services and InfoPath Forms Services. Besides being exciting features, we gained invaluable learning for the team how to have an architecture that worked in the rich Office client and on the server with web access with high fidelity, round tripping, etc. We have expanded on this and our experience with Outlook + Outlook Web Access in the Office team for the new set of read-write Office Web Apps as part of the 2010 wave. While customers appreciated the depth and management knobs, from this release we learned we need to invest more in readiness (training our employees, customers and partners) and frankly it took us a while to catch-up with the demand of 2007. This blog was a big part of our investments that I will discuss below. We are very focused on addressing technical readiness for the 2010 release and the depth you will see around the SharePoint Conference this month will be a big step towards this. However, there are many other activities in progress to train 10s of thousands of people before we release. We want to hear your feedback on what else you need so we can have the content, training and industry ready by the release of SharePoint and Office in the first half of next year.

Supporting SharePoint 2007

If you have been a frequent reader of this blog, you know notice we have spent very few of the 100s of posts talking about SharePoint 2010. This is for two reasons. First, we did not want to start talking about the next release until it was solid and we were near something actionable (e.g. getting educated on the product, try out the beta). Second, we want to keep the focus of Microsoft and industry on great SharePoint 2007 deployments. Since the release we have made a number of investments our team has talked about in this blog. I will highlight just a few things because many have been posted about already:

- Enhancing Technical Content and Guidance. Since the release of 2007, we doubled the size of the SharePoint documentation team and increased the content on TechNet, MSDN and this blog. We created a virtual team, called the SharePoint Content Partnership Council, spanning R&D, documentation, consulting, support, marketing, etc. that reviews your feedback and prioritizes the content to be written or improved. We have focused on the key areas you have told us about such as interoperability and governance. We also evangelized a lot of consultants and partners to post "real world" experience to the blog since know that is very important to you.

- Increasing Microsoft and Partner Staffing and Training. We were bullish on the adoption of SharePoint 2007 but frankly we underestimated the demand and didn't have enough trained Microsoft staff and partners at launch. Since then we expanded our internal support and consultant staff significantly and introduced training programs reaching thousands of people outside of Microsoft. We introduced a new multi-level certification including a "Mmaster" designation and a packaged offering of 5, 10 and 15 planning for customers called SharePoint Deployment Services that over a thousand organizations have used. This has been a great source of feedback. We have already started the process of training Microsoft employees on SharePoint 2010 and the SharePoint Conference starts the process for 2010 for the industry. Much more to come that you will hear about in this blog very shortly.

- Creating a Customer Assistance Team (CAT) - In the past, Program Managers in our development worked with our consultants, customers and partners to get feedback and publish best practices. In 2003, we created an elite team of experienced consultants called the SharePoint Center of Excellence to drive this within our consulting organization and partners. After the 2007 release, we went a step further creating a CAT team within R&D that lived and breathed the most demanding deployments to make sure our content on topics like capacity planning or disaster recovery was deep and we had even richer feedback loop into the 2010 development team.

Enhancing SharePoint 2007

One topic some people ask us about is responding to a wide range of requests for new features and solutions. I will spare you the software laws of physics discussion about why we do not update 1% of SharePoint every month. We spend a lot of time thinking about this and think the model we have gets the great amount of capabilities out to customers in a reliable and predictable way for a product of the scope and mission criticality of SharePoint. However, we know there are places people need more solutions with agility and I expect some people would be surprised by the investments we have made so I thought I would walk through a few of them below.

- Cumulative Updates - We heard your feedback that you want something more convenient than hot fixes and more frequent than Service Packs so we introduced Cumulative Updates for SharePoint and Office. This was a big change for our development, testing and release process. It wasn't all smooth at the beginning and we heard your feedback on this blog about simplifying these and my sense is the feedback is now very positive.

- SharePoint Administration Toolkit - We created a team to address feedback on IT operations requests and release supported tools that we have posted on this blog. Unlike core features, these are more straightforward to do because they don't invalidate the underlying code nor create a wide mix of configurations for us to test.

- Toolkits and Codeplex - As SharePoint is mission critical, you asked us to make sure the base product is robust, performant, secure, localized, upgradable, documented, supported, etc. Not surprisingly, this is expensive and causes us not to do as many features as if we had a lower release bar. Some of you are fine with that. Others have told us you would like some starter solutions to save you from writing value-added code yourselves. Hence, we have invested in a broad range of kits for SharePoint (Silverlight, Community, Site Templates, Patterns and Practices, etc.) and fostered a vibrant open source community on Codeplex while setting expectations these are not packaged products. I just checked and there are now 876 projects for SharePoint. We know these are in various stages of quality and adoption and unfortunately we can't bless them but do think it is important to foster the community and listen to which ones you find valuable.

- SharePoint Partner Solutions and Services - While we aspire to provide the most comprehensive solution in the industry, customers have asked for a growing range of solutions and services and we have worked to foster a growing community of thousands of business around the world building on top of SharePoint. They have the expertise, focus and reach to meet a large and growing range of needs. We also strive to make SharePoint a great business opportunity to foster the investment and innovation in our partners.

- Search Investments - We heard that great search is a critical part of making a successful SharePoint deployment. We also see highly scalable and flexible search technology as a key source of innovation across internet and intranet sites, content management, social networking, business and intelligence and more in the future. Therefore we did two things following a "good, better, best" approach that people have liked with SharePoint. First, we released a free version of SharePoint Search named Search Server Express. Second, we acquired the leading enterprise search firm, FAST. FAST's ESP is the most sophisticated and flexible technology serving the world's largest publishing, media, commerce and telecommunications sites on the internet as well as many of the most sophisticated internal search applications. The FAST engineering team is outstanding balancing extreme technical depth with great passion for the most sophisticated customer scenarios. We continue to invest in both standalone products and products optimized for SharePoint (ESP for SharePoint). Over the last year and a half the combined Microsoft Enterprise Search Group has come together and we will share a lot of innovation in user experience, relevance, connectivity and more as part of the 2010 disclosure. We will continue to invest in standalone search technology and have some exciting work coming in the next year there as well.

- SharePoint Online - Based on the SharePoint 2007 codebase, we released both a Dedicated (single tenant) version of SharePoint Online as well as a Standard (multi-tenant) version of SharePoint Online. This is all part of our Business Productivity Online Suite including SharePoint Online, Exchange Online, Office Communications Online and Live Meeting. The response to both the dedicated and multi-tenant offering has been outstanding. Customers have told us they like getting access to the most comprehensive and flexible set of collaboration and communication tools with the reliability, security and manageability they need. Large organizations such as Coca Cola Enterprise have adopted SharePoint Online to give them more agility and free up their IT staff to partner with their business users on more critical activities than configuring server farms. It also lets us reach new business who previously were not able to deploy collaboration technologies because of lack of internal IT expertise. SharePoint Online also lets our partners work with us on a recurring revenue stream and focus their business on higher value and profitable services. Finally, because Microsoft does the operations and incurs the cost of running SharePoint Online it is an incredible feedback loop at high scale that helps drive product and documentation improvements that we will share with our customers running their own servers. As I will share in the next blog post, the development of SharePoint 2010 was very much informed by our experience during the last few years on SharePoint Online.

So as you can see the team has been very busy enhancing SharePoint 2007 and not just working on 2010.

Lessons Learned

To sum up our lessons learned from the last decade on SharePoint that we'll apply to our second decade of investments:

- Customers come first. There is a lot of exciting technology in SharePoint but the objective is not a building a computer science project. What matters is solving business problems. We love to see your sites so keep the case studies coming!

- Long-term vision and commitment. Business is faster paced than ever and we and you need to continue to adapt. At the same time, success usually only comes from a clear long-term vision, commitment and feedback loop. I think that has been a key to your response to SharePoint as we discussed above.

- Balance of both innovation and execution. This is something we've prided ourselves on in Office. Sometimes it means we incubate new technologies like we have done in the history of SharePoint. Sometimes in means we take bets where we believe we can improve the customer experience like MySites in SharePoint 2003 and the Ribbon in Office 2007. At the same time, shipping a product and service as comprehensive as SharePoint Server and SharePoint Online is a critical engineering project and you are depending on us to do it with rigor and continuous improvement.

- Viral and top down adoption. As hopefully you have seen our approach for SharePoint dating back to the first release is trying to strike the balance between empowerment and governance so people can be more productive while the organization can manage its knowledge, security and costs.

- Feedback loop. We need to continue to listen to your feedback about what we've done well and where we can improve. Even if we can't address the feedback immediately, we must always listen, analyze and decide how we can best address it. We have a number of approaches for this in Office and SharePoint that I will cover in the next post but we always welcome your feedback in this blog and at the upcoming events.

Thanks for your support and for reading this long. Hope it was interesting. I will post in a week about the development process we used for SharePoint 2010.

Jeff

Jeff Teper - Corporate Vice President, SharePoint Server, Microsoft