With SQL Azure, you're no longer in charge of the Instance of SQL Server that you're running on. You're dropped into your environment at a database level. Also, the size of those databases are more restricted than you might be used to for your on-premise system (Keep in mind that SQL Azure has a set of use-cases, and those aren't always the same as the on-premise installation of SQL Server or other RDBMS systems). Because of these two reasons, you want to start "thinking at the database level". In one case, this means a shift in thinking at a lower level, and in another, at a higher level.
First, because you're at the database level when you enter the system, you don't need to control the Instance-level settings, or work with security and so on at the higher level. That measn that you should focus "down-level" to the database settings you do control.
Second, because of those size limits you need to think differently about the strategies you use for dealing with "big" data - in this case, as of this writing, the 1-50GB databases you can create on SQL Azure. In SQL Server on-premise installations, you can "partition" large sets of data by breaking them out using a Partition Scheme and a Partition Function - more on that here, with a great explanation of previous versions of SQL Server Partitioning. You also have access to FileGroups, which point to files that can be placed on different physical devices. In SQL Azure, you can think about the database as a container like the tables are in on-premise systems, in effect, "thinking up" to the database level.
This actually has some advantages - by placing data sets that you might partition by date, customer ID and so on in different databases, you gain the advantage of a different logical system running that database, gaining you CPU, memory and so on. My friend Wayne Berry has an excellent series of articles dealing with this starting here. He develops a strategy on Partitioning in SQL Azure in that series.
The point is that using SQL Azure requires understanding the way it holds and processes data. While it's ideally suited to data in the sub-50GB range, that doesn't mean that you don't have options to work with larger sets.
There has been a lot of buzz lately about "the cloud" and what it means for IT, and the organizations they serve. I just finished watching Steve Ballmer deliver Microsoft's vision for the cloud, wondering if it would be different than the offerings of other firms. It feels like it is different. So what is that strategy, and more importantly, what does that mean to the data professional?
First, let's recap what Steve mentioned for Microsoft's strategy. The word "cloud" can mean a lot of things, from hosting data (often called "somebody else's hard drives) to running software where the user logs into an application hosted on a web server somewhere (called Software as a Service, or SaaS). Microsoft actually has both. In fact, we've had things like Hotmail (which is a service) and the various Live offerings for quite some time. And we've had XBox Live, which is a hosted environment that is kind of a hybrid - there's a "fat" or hardware client that talks to the main server out on the web. There's also an offering of a complete Microsoft Office, SharePoint services and even LiveMeeting in the cloud - nothing to install, runs on lots of platforms, and more (details here).
But there's something new. Microsoft has two new offerings, called Azure, and SQL Azure. Azure is more of a programming platform that can host data, and SQL Azure is SQL Server in the cloud. But SQL Azure isn't just a server you rent - we actually maintain the systems, handle the optimizations and so on. You create databases and database objects. You can take the data from SQL Azure and send it to a local SQL Server, and vice-versa. There's also the ability (through something called Sync Services) to replicate data between the two.
So how does the Data Professional meld in the "cloud" to our day-to-day systems? How do we use Microsoft's strategy in our own strategies? I've seen a couple of interesting uses so far from those that are trying it out:
Front-End Start, Back-End Archive: In this mode, companies spin up an application quickly (with no server build) into Azure, backed by SQL Azure. They are able to quickly deploy the app, and if it grows they can bring that data in-house or even bring a subset of that data locally for reporting, keeping a smaller data-set up in SQL Azure.
Start There, Stay There: Some of the companies I've seen are taking the application ideas and starting them in SQL Azure or Azure (or both). No capital expense, no hardware purchases, no installs, nothing to deploy. When the app is developed, you just point your clients there and off they go. It's a pay-as-you-use system, so the costs mimic the profit.
Start There, Come Here: In other cases, organizations want to start projects quicker than they can get hardware and software installed. So they follow the previous process, and then bring the code and the database in-house when they are ready.
There are other strategies, such as using the cloud as part of High Availability/Disaster Recovery or a remote-office access system and so on, but whatever your reasons, you need to spend a little time getting familiar with the cloud - and Azure and SQL Azure should be high on your research list. Here's some places to get started:
Azure Overview: http://tinyurl.com/y8s52vh Windows Azure Training Kit: http://tinyurl.com/5vrt7q Writing a Azure Program in 5 steps: http://tinyurl.com/ye6chog Azure Data Sync: http://tinyurl.com/ylsykfb Future of programming with Azure Video: http://tinyurl.com/ykqsbsf Reference list for Windows Azure: http://tinyurl.com/yze9azr
I follow the Database Trends and Applications magazine because it is pretty platform independent, with a section for Oracle, SQL Server and DB/2. I found it interesting that when I turned there yesterday and today, among the top stories are "Cloud" - front and center. It's easy to see why. When we think about technology today, we've got three big "circles" of components like a Venn diagram. In one circle is the "Platform", which includes hardware and the software that runs on it. Developers live here, as do System Administrators and even Data Professionals. In another circle is the "Connectivity", which includes network and security. In yet another circle is "Data", which includes data professionals of course, developers, SAN admins and Sysadmins from file shares to SharePoint.
The first circle - Platform - has all of the hardware in it. That's a capital expense that also has operational expenses, like people that maintain, manage and monitor them. Companies really want to tone that down wherever they can. That's where this whole "Cloud" thing comes into play. The devs are still there, writing apps for the organization. The data specialists are still there, managing and designing data. The network will always be there. But hardware and software? Does your CEO really want to pay to have that laying around? And what about capacity growth and shrink - not easy to manage when you own everything. So many of them are asking you about the Cloud. And you're probably wondering where it fits in.
Well, Microsoft is certainly one of the main players in this area. From Windows Azure, the Application Fabric and SQL Azure, we have real-live offerings you can leverage right now. But is SQL Azure ready? For many applications, yes. No, I don't recommend that you grab your 1 Terrabyte database that has a 500GB ETL process running on it and just toss that unchanged into SQL Azure. I think there's a place for it - and it keeps growing every day. There's a great article over on Softpedia where you can read more about the changes, improvements and enhancements to SQL Azure that we've just released. But here's a side of that you might not have considered: you don't have to do anything to get those improvements. There's no install. There's no patching. To be sure, there is still testing and so on that you need to do, but it just changes and gets better. Certainly this is one argument for the Cloud. Again, it's another tool in the toolbox - not meant to replace every on-premise environment today, but something you should learn about to see where it does fit.
So check out that article, and post your comments here and there. I'm curious about what you think.
OK, sort of.
I've been a fan of the Economist magazine since a friend of mine in Florida introduced me to it years ago. Imagine my shock when I check out the latest issue and find a story about - Big Data! This is the kind of thing we think about all the time in the SQL Server group here at Microsoft. Then when I read the article, I find this stunning bit of information: did you know that Wal-Mart handles more than 1m customer transactions every hour, feeding databases at more than 2.5 petabytes – the equivalent of 167 times the books in Americ’s Library of Congress?
And Wal-Mart uses SQL Server. And SQL Server 2008, no less - with Policy Based Management at the forefront of helping them track this all. Also, if you read the article, you'll see that Craig Mundie (Microsoft) and Eric Schmidt (yeah, the boss of Google) both sit on a presidential task force dealing with health care data.
This is heady stuff. Sometimes I get asked if SQL Server can scale, if it's really different than it used to be - and since I live and breathe the database world, I'm surprised by the question. But when I think how quickly we've moved into the upper tiers of the enterprise, I'm not as surprised.
Definitely check out the article here: http://www.economist.com/specialreports/displayStory.cfm?story_id=15557443. For more about a REALLY big SQL Server, check this out: http://msdn.microsoft.com/en-us/library/aa226316%28SQL.70%29.aspx. and Gizmodo (a must-follow blog) has a good story on Pivot (hit the link if you don't know what that is) here: http://gizmodo.com/5488641/, Awesome.
I've mentioned elsewhere that you should be thinking about a "data roadmap". A way of thinking about how you intend to leverage all of that data you're collecting.
And in that post I mentioned that Microsoft has a "roadmap" for SQL Server. We think about how to evolve the product all the time, with one overriding goal - to be the "Information Platform" for your organization. It's more than just storing and securing your data - it's about giving you tools that let you access that data from anywhere, quickly and safely. In SQL Server 2008 R2 you'll see big investments in this kind of thinking, from Master Data Services to Stream Insight and PowerPivot and even interaction with SQL Azure. We're giving you new ways to do the basic inputs and outputs, but we're also thinking beyond those operations to new ways of allowing you to provide that data to your organization. Bloomberg has a review on some of these features that you can check out here.
So make your data roadmap with our product roadmap in mind - it's not just data, it's information.
In early 2001 I worked @ a start-up company with about 150 people. I call it a start-up because it had no revenue but did have free breakfast, lunch and dinner. @ the company there were five people named Michelle and six named Dan (including yours truly). One day the VP of operations got confused about which Michelle and Dan he was talking about; he followed it with the following comment – you can’t swing a dead cat around here without hitting a Dan or a Michelle. Ok, I apologize to all cat lovers – I’m one myself and I for sure don’t condone the swinging of dead animals, or live ones for that matter. But I feel the same way about virtualization and consolidation. Hardly a days goes past when I’m not in a conversation about these topics. It’s important to remember that while virtualization has a coolness factor I recommend you approach it as an enabler to meeting business objectives and not the silver bullet that’ll solve all your woes. Also, while virtualization and consolidation are often mentioned in the same breath they’re not the same thing. A couple of guys on my team put together the following graphic to discuss SQL Server consolidation options:
As you move left to right across the picture you move from Higher Isolation (which equates to higher cost) to Higher Density (which equates to lower costs). High isolation refers to resource and security isolation. Obviously you could take this to the extreme and have each application (instance of SQL Server) reside on its own hardware sitting in its own data center residing in its own building on its own power grid. But that’s crazy, right? Higher density means the greatest sharing of resources.
Here’s a brief explanation of each lane:
The main point here is there is no one size fits all solution or technology and virtualization is just one of the ways to meet consolidation needs. You will likely employ multiple solutions within your environment with the key being to chose the right technology to meet the business requirements of the application. As an IT professional it’s your job to understand the technology and how to apply that technology to solve business problems.
There are several different models out there for approaching consolidation. The high-level steps that resonate with me are:
Finally, I couldn’t end this article without mentioning my favorite virtualization related features in Win7 and Win2K8 R2:
Virtualization and consolidation are concepts that have been around for at least 30 years. With the ever increasing pressure on IT to do more with less they are realities that can no longer be ignored. As I mentioned in my opening blog post, one of the challenging aspects of my job is I’m always thinking one to two releases out and I’m talking these days a lot about consolidation and virtualization… The first of this wave of capabilities sees the light of day with SQL Server 2008 R2 Application and Multi-Server Management.
Cheers,Dan
Most data professionals are familiar with the term "planned downtime" and "unplanned downtime". The first is painful to ask for, and the second is painful to explain. We strive not to have either. SQL Server 2008 and SQL Server 2008 R2 have introduced features, such as better recovery from corrupt pages in a Database Mirroring and so forth that attempt to keep the problems down. But "planned" and "unplanned" can also be used to describe our daily work - and we don't have a choice most of the time for how we deal with either kind.
Planned work is the task we do because it has a schedule, or at least *could* be scheduled. Backups, building a server, applying a service pack, reviewing the logs - all of these could be things that we can schedule. Looking at my "Tasks" in Microsoft Outlook, I have a lot of things that I have scheduled for today, this week, and this month. I never really close or complete some of them, I just change the due date to the next period of time when I need to deal with that task again.
Other work is very "unplanned". This kind of work can come from anywhere - from a co-worker who needs help, a manager with an emergency request, and most of the time from a server that has a problem, with anything from issues with Replication to a failed backup.
It's kind of difficult to meld these two together. When you're in the middle of building a server, it's hard to leave the server room, run downstairs to talk with an irate manager and then fix the issue with the system's database so that her application can still run. Even worse, for data professional it's often a case of having to prove it *isn't* the database that is causing the performance problem - but that's another post.
There are, however, tools and processes that can help you deal with both planned and unplanned work. As I mentioned, I use Outlook for just about everything, since I can access it from many locations (even my Windows Mobile phone) and it combines my calendar, tasks, contacts and of course e-mail in one place.
Another tool I've come to rely on is OneNote. Of course you can just use notepad or a Word-processor to take notes, but OneNote is integrated with Outlook (and just about every other Microsoft program), it can "share" notebooks between teams and has a rich set of tags to help qualify what I need to know visually and quickly.
But tools aren't the whole story. First, I try to keep a level head during the interruptions. I've been a data professional for a really long time, so I've gotten over the panic stage. It also helped that at one point in my career I volunteered as an Emergency Medical Technician (EMT) on an ambulance, which of course *really* puts you in life and death situations. After that, a server crash isn't cause for complete panic.
I have developed a process to deal with both of these kinds of work. I plan what I can - trying to look out as far as possible, creating checklists, and coordinating with the rest of my team and my organization. I try to get the most important planned work done as soon as possible - first thing in the week, first thing in the morning. That way, if I get an unplanned event, as much as possible of the planned work is complete. In a way, I'm planning for unplanned work!
For the truly unplanned work, such as an emergency, I keep a OneNote page nearby with links that are categorized by the type of issues I think I might face. I document each step I follow to correct the issue, even if I have to wait until later. I try and keep the energy from all of the emotions low, and work on the problem as systematically as it will allow. Above all, I communicate constantly, letting the right people know what has happened, what is happening now, and what I'm doing about it. That OneNote document comes in really handy here.
So how are you doing it? How do you handle the work that comes at you from all sides?
I’ve always loved getting new things. And when it comes to software it’s no exception. I enjoy exploring a new release of my favorite app – see how things have changed and advanced and discovering new features. There are two new things that I want to call your attention to.
First, this week we released SQL Server 2008 R2 November CTP. This CTP is a huge milestone because it is our feature complete CTP. You’ll find some great stuff in there for the Control Point and the Data-tier Application.
You can download the SQL Server 2008 R2 English language versions using these links:
SQL Server 2008 R2 Eval (including Master Data Services and StreamInsight): http://www.microsoft.com/downloads/details.aspx?familyid=FE0C6A31-5AD6-4EEA-A865-73BBE2608BD1&displaylang=en
SQL Server 2008 R2 Express: http://www.microsoft.com/downloads/details.aspx?FamilyID=c772467d-e45b-43e1-9208-2c7b663d7ad1&displaylang=en
SQL Server 2008 R2 Feature Pack: http://www.microsoft.com/downloads/details.aspx?familyid=020EE0D5-BCE4-4A45-9D64-B0C49C8831E5&displaylang=en
Don’t delay, pull these down, install them and put them through your own personal gauntlet. Send us your feedback using Microsoft Connect: http://connect.microsoft.com/sqlserver.
The second new thing is a new book: SQL Server MVP Deep Dives. There are two special things about this book: 1) it’s written by experts. The content is top notch and I think this is a must have book for every SQL Server DBA. Second, it was an all-volunteer book and all of the author proceeds are going to WarChild.org. You can find the book using the link above or also on Amazon. From the sidelines I watched the MVPs come together and get the book off the ground. It is a prime example of the strength and dedication of the SQL Server community.
Just after a new release of SQL Server, I often get e-mails and calls from folks with this question: “Can I upgrade from Customer Technical Preview (CTP) x or Beta #x or Release Candidate (RC) to the “Released to Manufacturing” (RTM) version?”
Unfortunately, no. Right up until the last minute, things are changing in the code – and you want that to happen. Our internal testing runs right up until the second we lock down for release, and we watch the CTP/RC/Beta reports to make sure there are no show-stoppers, and fix what we find. And it’s not just “big” changes you need to worry about – a simple change in one line of code can have a massive effect.
Even if you've done this before and things seemed to go well, you may be in a difficult situation because of it.I’ve dealt with someone who faced this exact situation in SQL Server 2008. They upgraded (which is clearly prohibited in the documentation) from a CTP to the RTM version over a year ago. Everything was working fine.
But then…one day they had an issue. Couldn’t fix it themselves, we took a look, days went by, and we finally had to call in the big guns for support. Turns out, the upgrade was the problem. So we had to come up with some elaborate schemes to get the system migrated over while they were in production. This was painful for everyone involved. So in general it's just a really not a good idea.
There is one caveat to this story – if you are a “TAP” customer (you’ll know if you are), we help you move from the CTP products to RTM, but that’s a special case that we track carefully and send along special instructions and tools to help you along. That level of effort isn’t possible on a large scale, so it’s not just a magic tool that we run to upgrade from CTP to RTM. So again, unless you’re a TAP customer, it’s a no-no.
Bar Bosch, in Palma de Mallorca, is one of my favourite places to hang out. It's hectic and noisy, but still leaves space and time for endless discussions or reflection. It is the perfect place to pretentiously pore over Žižek or Derrida. Yet in the autumn of '98 I found myself sitting there, ordering carajillas and reading up as much as I could about Oracle's Business Intelligence tools.
It was before I joined Microsoft, and I had just started a new job with a BI team focused on Oracle products. I had to get up to speed quickly, but, fortunately, I had some vacation already booked. So, one week after I joined, my wife and son headed to the beaches of Cala Ferrera, and I headed to Bar Bosch with my Oracle books under my arm. It was a long week of reading, and mostly in vain. When we returned home, my CEO had news for me. "All that Oracle stuff," he said. "Forget it. We're focussing 100% on SQL Server now." The rest, as far as my personal story is concerned, is history. I've never looked back.
What happened? Some of the C-level team had attended the SQL Server 7 technical preview at Microsoft's invitation in Redmond. On day one, during a break, they phoned back to our office from the lobby and told the development team to stop Oracle development. They found the Microsoft BI story, told then by Bill Baker and Amir Netz, so compelling that they immediately changed direction. Microsoft presented three irresistible messages: a vision of what BI is about, a commitment to BI customers and partners, and a compelling pricing model.
I could wish all Oracle migrations were so easy. Some are, but I always maintain, as in a recent discussion with Ted Cuzillo, that migrating technologies is easier than migrating people. When one has established a way of working, it's difficult to adapt to a new way, even if it is technically equivalent or superior. It's just easier to carry on with one's current practices. Software manufacturers know this, of course. We all want our products to be, as the marketers say, "sticky" - and all of us in the software business like features and methodologies that build loyalty. That loyalty, once gained, is naturally a most valuable asset.
Yet, recently, Oracle, appear to have taken a different approach: one that customers increasingly resent. They are tying companies into the Oracle ecosystem financially, especially by acquiring other strategic suppliers: writing checks, not code, as Larry Ellison himself once said. Their customers are complaining in stark terms, as they find themselves not only tied in, but subject to ever-increasing prices and intransigent demands. Business Week headlined that "Oracle has Customers Over a Barrel" quoting one customer as saying "Once you've made a deal with the devil, it's hard to get away."
This approach from Oracle is surely a deeply alienating move, and it tastes to me of a certain desperation. Personally, I find it baffling. Here is a company with a truly great database product, a broad range of applications and excellent engineers, but they are driving their own customers to despair with licensing practices that feel like extortion according to some. So, while there have always been Oracle customers willing to make the effort to change, at Microsoft we are expecting to see that number increase.
Naturally, we're encouraging them, and why not? SQL Server is growing rapidly: over 11% in 2008 according to IDC. In the past, much of our growth came from net new customers: we have always been aware that we win many businesses choosing their first enterprise database or BI implementation. Now, we are seeing a large number of switchers: customers willing to make that migration from another technology. To win more of these customers, we need to show that they need not be tied into the Oracle universe, even when their existing commitments are substantial. This case study of the Turkish appliance manufacturer Arçelik describes them effectively moving a 5Tb SAP implementation from Oracle to SQL Server, and gaining substantially in performance and lower costs. We also have a neat little tool that TCO calculator that has proved very popular. To be sure, you don't need an animated tool to find significant TCO savings persuasive, but it's fascinating and fun to play with the various combinations of servers and staff and requirements.
Of course, I would not deny that Oracle are still growing. There's a lot to admire in the efficiency with which they target and execute on acquisitions. Frankly, they sometimes take the rest of us in the industry by surprise; which is partly a sign that they are imaginative and bold, yet partly suggests that they are chasing growth by acquisition at all costs, often seemingly at random. This week they acquired Hyperroll. That gives Oracle four OLAP products. It's little surprise that their customers, and from what I hear, their internal teams, are puzzled and often anxious about their direction. On a larger scale, their acquisition of Sun sent many of the same confusing signals.
How does all this affect someone like me, making suggestions and decisions about our own BI futures in Microsoft? In some ways, it affects my work remarkably little. When Oracle buy a company like Hyperion, it may mean that there's a new flag on the island, but it takes a much longer time to build bridges and integration. As a result, we in Microsoft, especially in the product teams (rather than, say, in marketing) may look at the Oracle applications and see relatively few feature-for-feature challenges. In other cases, such as Oracle's own in-database-OLAP features, we largely had to ignore their feature set when considering their future direction because it seems so unclear.
Oracle integration remains an important priority for us, for several reasons. For one thing, it helps our customers who are coming off Oracle to do so in a planned, progressive manner. Supporting Oracle systems in our tools - such as Analysis Services, Reporting Services and Integration Services also helps us to provide BI features to those customers who do find themselves on the wrong side of the devil's bargain and unable to extract themselves for it. For example, we see many customers using Analysis Services, or Reporting Services or Integration Services with Oracle systems. In fact, demand for Oracle support was so great, that in SQL Server 2008 we introduced an Integration Services high-performance data loader for Oracle. Yes, a "data loader" not an extractor: we released a feature that makes it easier to get data into an Oracle database!
Seeing these customer satisfaction issues at another major and successful vendor, and at the risk of sounding too much like a motivational call, I am convinced that we, in the SQL Server team, must keep to our own core value propositions: a persuasive vision, a commitment to our customers' success, and a compelling pricing model. After all, these are the same key points that were so signifcant to my old team over 10 years ago at that technical preview in Redmond. For their part, our friends at Redwood Shores appear to be doing their best to send customers our way. Is that what they call synergy? I'll ponder that, although this autumn it will be over a latte in Third Place Books rather than a carajilla at the Bar Bosch.
"Five" seems to be a big number these days - is it compression to have a "top-5" instead of a "top-ten"? :) Over at Network world they've posted a quick slide deck of "Five things we love about SQL Server R2". They even include...the Start Menu? OK - I guess everyone has different likes and dislikes. I think they're happy that we gave SQL Server 2008 R2 it's own menu area, rather than lumping it in with SQL Server 2008 (you're welcome).
I agree with their first four for sure - and I'll throw in there things that you might not see as often, like being able to connect to SQL Azure from SQL Server Management Studio, increased database sizes in SQL Server Express, just to name a couple. By the way - if you haven't rushed over to check out the resources we have for you to learn the tech for SQL Server 2008 R2, what are you waiting for? They're free! Enjoy.
In the middle of May, just about 6 weeks ago, the SQL Server BI team launched PowerPivot. This weekend, we have we passed an important milestone: 100,000 downloads of the Excel add-in. For a BI application, this really is a huge number. I guess it truly underscores something we have said all along: that PowerPivot is aimed at a much wider market than traditional BI. (As Amir Netz says, those who can spell Excel, but can't spell BI.) I blogged about this very topic recently when I asked Who is PowerPivot For?
For me, it simply reminds me of one of the reasons I love working at Microsoft; we are the only company who can bring analytics of this power to this size of audience. It's not that other companies don't do great work; I love what many of our partners and rivals are able to accomplish. Yet only at Microsoft can we impact so many people. It's a huge privilege, and I know from my own experience that none of us take it lightly.Of course, it is early days yet. There is a lot more to come. Some of you may be wondering if these 100,000 users already represent the bulk of those who may be interested? I can tell you that the download rate is increasing, and increasing rapidly.
For now, having passed this landmark, I think the team deserves to feel quite satisfied with the early reaction. It promises to be an exciting year!
Since my team started work on Project Houston hardly a day goes by I’m not in some discussion about the Cloud. It also hits me when I go through my RSS feeds each day: cloud this and cloud that. There is obviously a significant new trend here, it may even qualify as a wave; as in the Client/Server Wave or the Internet Wave or the SOA Wave, to name a few – ok, to name all the ones I can name. Almost 100% of what I read is focused on how the Cloud Wave impacts IT and the “business” in general. Don’t get me wrong, I completely agree it’s real and it’s going to have a profound impact. I think this posting by my friend Buck Woody (and fellow BIEB blogger) sums it up pretty succinctly. The primary point being it doesn’t matter what size your business is today or tomorrow, the Cloud will impact you in a very meaningful way.
What I don’t see much discussion about is how the Cloud Wave (is it just me or does that sound like a break dancing move?) changes the way software vendors (ISVs) develop software. In addition to Project Houston my team is responsible for keeping SQL Server Management Studio (SSMS) up with the SQL Azure feature set. If we step back for a second and look at what we have, we have a product that runs as a service on Windows Azure (running in multiple data centers across the globe) and the traditional thick client application that is downloaded, installed, and run locally. These are very different beasts but they are both challenged with keeping pace with SQL Azure.
SSMS has traditionally been on the release rhythm of the boxed product. This means a release every three years. The engineering system we use to develop software is finely tuned to the three year release cycle. The way we package up and distribute the software is also tuned to the three year release cycle. It’s a pretty good machine and by and large it works. When I went to the team who manages our release cycle and explained to them that I needed to release SSMS more frequently, as in at least once a quarter if not more often, they didn’t know what to say. This isn’t to say these aren’t smart people, they are. But they had never thought about how to adjust the engineering system to release a capability like SSMS more often than the boxed product, let alone every quarter. I hate to admit it but it took a couple of months of discussion just to figure how we could do this. It challenged almost every assumption made about how we develop, package and release software. But the team came through and now we’re able to release SSMS almost any time we want. There are still challenges but at least we have the backing of the engineering system. I’m pretty confident we would have eventually arrived at this solution even without SQL Azure. But given the rapid pace of innovation in the Cloud we were forced to arrive at it sooner.
Project Houston is an entirely different. There is no download for Project Houston, it only runs in the Cloud. The SQL Azure team runs a very different engineering system (although it is a derivation) than what we run for the boxed product. It’s still pretty young and not as tuned but it’s tailored to suit the needs of a service offering. When we first started Project Houston we tried to use our current engineering system. During development it worked pretty well. However, when we got to our first deployment it was a complete mess. We had no idea what we were doing. We worked with an Azure deployment team and we spoke completely different languages. It took a few months of constant discussion and troubleshooting to figure out what we were doing wrong and how we needed to operate to be successful. Today we snap more closely with the SQL Azure engineering system and we leverage release resources on their side to bridge the gap between our dev team and the SQL Azure operations team. It used to take us weeks to get a deployment completed. Now we can do it, should we have to, in the matter of hours. That’s a huge accomplishment by my dev team, the Azure ops team, and our release managers.
There’s another aspect to this as well. Releasing a product that runs as a service introduces an additional set of requirements. One in particular completely blindsided us. Sure when I tell you it’ll be obvious but it caught my team completely off guard. As we get close to releasing of any software (pre-release or GA)we do a formal security review. We have a dedicate team that leads this. It’s a very thorough investigation of the design in an attempt to identify problems. And it works – let me just leave it at that. In the case of Project Houston we hit a situation no one anticipated. The SQL Azure gateway has built-in functionality to guard against DOS (Denial of Service) Attacks. Project Houston is running on the opposite side of the gateway from SQL Azure, it runs on the Windows Azure platform. Since Project Houston handles multiple users connecting to different servers & databases there’s an opportunity for a DOS. During the security review the security team asked how we were guarding against a DOS. As you can image our response was a blank stair and the words D-O-what were uttered a few times.
We had been heads down for 10 months with never a mention of handling a DOS. We were getting really close to releasing the first CTP. We could barely spell D-O-S much less design and implement a solution in the matter of a few weeks. The team jumped on it calling on experts from across the company. We reviewed 5 or 6 different designs each with their own set of pros and cons. The team finally landed on a design and got it implemented. We did miss the original CTP date but not by much.
You’re probably an IT person wondering why this is relevant to you. The point in all this is simple. When you’re dealing with a vendor who claims their product is optimized for the Cloud or designed for the Cloud do they really know what they’re talking about or did the marketing team simply change the product name, redesign the logo and graphics to make it appear Cloud enabled. Moving from traditional boxed software to Cloud is easy. Do it right, is hard – I know, I’m living through it every day.
This past week we released SQL Server 2008 R2 to manufacturing. This is a huge accomplishment for the team and our customers are anxious to get their hands on it. I came across one blog post that expressed disappointment that the only thing they could download was the evaluation edition – they couldn’t wait to get their hands on a fully licensed edition, which will be available shortly.
Rather than go into a laundry list of what’s in the release here are links to a few of the RTM stories:
Even though I’m a Manageability Guy and there are some terrific manageability features in R2 the most important feature, in my opinion, is PowerPivot. PowerPivot is going to change everything about business intelligence for IT and information workers. Early in my career as an IT Pro I designed a system that used Excel Pivot Tables that were loaded with massive volumes of sales data. Unfortunately I had to have tens of Pivot Tables spread out across an almost equal number of Excel Workbooks. Since there were so many files and tables I had to build a monthly process for refreshing the data. Plus if one of the users wanted a new view of the data I had to craft it by hand for them. It’s an understatement to say this was a pain. If I had PowerPivot back then it would have greatly simplified my life and better supported the needs of my users. As you read up on PowerPivot you’re going to think it’s too good to be true, take it for a test drive to convince yourself how truly remarkable this technology is and how it’ll transform the way you think about BI.
Though short, this article in PC World makes an important point: the PowerPivot feature in SQL Server 2008 R2 is incredibly valuable to small businesses. I mentioned in a previous post that in another “life”, in enterprise IT for a Fortune 10 company, I built a “BI” app using a SQL Server backend and pivot tables in Excel. Given all the complexities back then the end-users relied on my expertise to deliver their precious data each month.The operative word in there is “enterprise”. Back then this sort of environment was out of reach of most small businesses and required certain expertise.
But isn’t BI important to small business too? Absolutely! And until now it has, for he most part, been too complex for small businesses. With the introduction of PowerPivot small business have access to an intensely powerful set of capabilities that are extremely friendly. If you own or work for a small business I encourage you to check it out for yourself.
I like it when *other* people review Microsoft software. I still volunteer as a DBA, so I still use multiple versions of our own products (and comptetitors, by the way) in the "real world". That's how I normally present, that's how I teach at the University of Washington, and how I work with my Microsoft clients. When I talk about the product, it's something I use - but of course I work here, so folks always like finding other people that have tried the new versions of our software and read what they say. Sometimes that's good, sometimes the reviewer has issues that they talk about. In any case, I like when they show detail about what they have tried and how it worked. And here is just such a review - it's by Jason Brooks over at e-Week.com - enjoy: http://www.eweek.com/c/a/IT-Infrastructure/SQL-Server-2008-R2-Offers-Enhancements-New-Management-Capabilities-518969/
Most of us have some help we can provide back to the community. Even if you're new, you can write down some of the things you've learned. And there are several easy ways to to that - you can certainly jump in on the forums (http://social.msdn.microsoft.com/Forums/en-US/category/sqlserver/) and answer any questions you know the answer for and you can even participate by helping to write a form of documentation - did you know we run a Wiki, where you can edit the documentation? Check it out here: http://social.technet.microsoft.com/wiki/
So take some time, peruse those resources, and write something up. We learn best when we teach others, so don't think that you can't give back - you can!
I first met Sean McCown when I worked on the team developing SQL Server Integration Services. Sean was simultaneously one of our most active supporters and one of our most trenchant critics. He can very – what’s the word? – very forthright. You know, blunt, outspoken and generally on the money. So, I keep a look-out for Sean’s articles and certainly his reviews of our latest efforts.
So, frankly, I was at least relieved to see the headling of Sean’s latest article - Seven reasons to care about SQL Server 2008 R2 – and glad to read it. Sean calls out what I would agree are the important reasons to upgrade: Managed Self Service Business Intelligence, Report Components, Master Data Services, StreamInsight, Multi-Server management, DACPACs and Sysprep. But don’t just make do with the list: Sean’s comments are insightful and critical where he needs to be, and he calls out great use cases for the features he likes.
For an example of his criticism, Sean takes us to task for the ways in which we have factored Enterprise (now supporting 8 CPUs) and DataCenter. Time and customers will tell – especially new customers. Mark Beyer, of Gartner, had a different view of our changes: “It’s about time. The amount is reasonable, the per socket is still better than any percentage per core that other vendors use, and SQL 2008 R2 remains a solid value for the price.” SKU factoring is always a difficult balance, particularly where existing customers find themselves right on the edge of two editions. We’ll have customers in that situation and I trust our field teams will help them make a choice based on our need to build a long-term relationship, rather than overselling an edition for the box price. Traditionally, this is exactly how we have worked – thus the very high numbers of SQL Server Standard Edition that we see.
Enjoy Sean’s post here and please do add your comments. Would love to see what you think is compelling in SQL Server 2008 R2 too.
Donald Farmer
Some things just really work well together. Peanut butter and chocolate. Abbot and Costello. Coffee and...well...anything. You get the idea. But there are some real advantages in using SQL Server 2008 (and of course the upcoming SQL Server 2008 R2 release) with Windows Server 2008 and higher. It's not just that they are both better than their predecessors, SQL Server actually takes advantage of the improvements in Windows Server 2008.
One practical example is in how Windows Server 2008 handles the infamous "drive offset". This is a small block size movement from the first part of the hard drive sectors - it's an internal thing - but it causes real issues with software that exercises the I/O subsystem, and makes its own calls there. Like SQL Server. In the past, the data professional had to follow a process called "Partition Alignment", and this had to be done when the system was set up. That's all now a thing of the past - with SQL Server 2008 and Windows 2008 Server, this just happens.
Another example is in how Windows 2008 Server deals with the "sliding TCP/IP window". This enhancement directly affects how fast SQL Server can send large frames of data - especially with Replication and large binary objects. At Microsoft we noticed tremendous speed gains just by moving to Windows 2008 Server.
There are lots of other examples - from new virtualization and consolidation changes in both products to clustering enhancements, and now in SQL Server 2008 R2 the ability to run the "sysprep" utility after SQL Server has been installed.
You can read more about this "pairing effect" in this White Paper.
And be sure to check out John Kelbley's Post on the Windows Server blog where he also talks about ways that SQL Server and Windows Server work "Better Together".
Well, just look again. $250 million is a helluva partnership investment, even for giants in the industry. It's always the case that press releases tend to be somewhat "glossy" - frankly, they all sound the same to me - so again, it's easy to overlook the details. Better than the press release, look here, to some of the applications that are already available, some of the case studies that already prove the value of the collaboration, and some of the details of hardware, software and services that are being integrated: http://bit.ly/5VTD4k
From a purely personal viewpoint, I have to say that the people I work with in Microsoft and HP are genuinely excited about the extended partnership. My friend at HP, John Santaferraro has been tweeting like crazy! For us it's not just marketing, and that's particularly true in the database and business intelligence fields. SQL Server is the most rapidly growing database platform, our BI is making strides in the market, and our partners in HP have also made some very compelling business intelligence investments.
So what is it, that we are offering?
For one thing, pre-configured, packaged solutions for OLTP, BI and DW workloads for different sizes of business. For SQL Server, this is great. Today, our customers see the cost-effectiveness, ease-of-use and power of the SQL Server and Microsoft BI platforms, but they often see only that. We are after all a platform company (having, I suspect, more platforms than the Jackson Five.) The new solutions build on the platform with HPs services and hardware experience which are, of course, first class. If this was a just a case of packaging up a services, software and hardware offering to make it easy to market, I think I would be unimpressed myself, but there is more to it. We are developing tools and services specifically for these offerings - from Microsoft, tools to make virtualization easier; from HP, customized BI professional services in information governance, master data management and so on. These professional services from HP are critical - they have over 11,000 certified Microsoft Professionals, so building out a portfolio of service offerings specifically for them, greatly increases our clout.
For the mid-market, I can see some of our other partners being concerned at first reading of the announcement. However, I see good news for them, too. Naturally, the noise has been about the big things HP and Microsoft can do together, even if somewhat exclusively, but the channel remains very important to us. ("Super-important" as Microsoft execs are wont to say.) After all, we have 32000 HP and Microsoft channel partners.
They will see much larger investment in our marketing programs - something like ten times the current spend, I believe. The investment will go into bundled software and hardware packages that should reduce sales cycles, new financing options to make integrated solutions easier to acquire, and - hugely important and hopefully worthy of a cheer - integrated support from dedicated field engineers.
I hope this gives some impression of why we are excited by the announcement. Of course, the proof of the pudding will be in the eating, and we'll be watching over the next year and more for the restaurant reviews of this particular dessert to come rolling in. I'm looking forward to them.
Meanwhile, it's time for me to get out of my hotel room, away from this laptop, and out into the Arizona sun. Even after so many visits, I'm not going to take it for granted. Look at the HP / MSFT announcement in that light.
Mary-Jo Foley recently wrote a blog post about being better together. At Microsoft this is something we take seriously. In her blog post she includes some snippets of an interview with Donald Famer. Donald discussed how we exchanged some team members with the Excel team in order to have a great integration story. Anyone in management knows how important people are and will understand that moving people between teams is serious business.
So why do we do this? We do it because it creates the greatest value for our customers. Creating seamless experiences between products – built on open APIs - ensures customers get the greatest benefit out of their investments. it also means that end users (information workers as we like to call them) end up with the most natural level of interaction with the products without having to think much about it.
I’ll offer one proof point: I was recently talking with a friend of mine who’s at a large packaged food company. Mind you he’s in sales not in IT so he offers more of a business rather than technology perspective. They just started a pilot of SharePoint, Excel and PowerPivot. He couldn’t agree more with the value of better together. They initially started out looking at nonMicrosoft products and quickly realized the cost of employees learning another tool which didn’t naturally integrate into their daily route or with other tools would be too disruptive and costly to the business. This really hit home for me that the price on the box is only a small part of the overall equation. The productivity of your employees is far more important.
I monitor a number of different database discussion forums and you can always find a healthy debate on the best database platform. The problem with this question is the answer is always, it depends. It depends on the business and technical requirements today and over the next few years. We just published a case study on a company called Arabah. Arabah is is the official U.S. distributor of the Jericho by Paloma line of beauty products made from Dead Sea minerals. Arabah started down the path with MySQL but as they began to compare the business and technical requirements to what MySQL offered they realized it wasn’t the correct choice. I’m sure you know how the story ends. The morale of this story isn’t that SQL Server 2008 is a better database platform than MySQL (even though it is), it’s that you must first define your requirements before you select your solution. Now does anyone know what’s the best car?
We discussed consolidation and virtualization - one of the hottest topics customers raise with us these days. CTOs are typically looking to control costs, but cost savings are hardly automatic: you must choose the right strategy. In some (notorious) cases, I have seen CPU utilization actually drop on consolidated boxes; typically because the servers were IO-bound.
Given these potential pitfalls, how are you to go about setting a strategy? A great starting point is our server consolidation guidance, including a flowchart to walk you through the right choices. For example, for many folks considering virtualization, security is a real concern - a flowchart helps you choose between virtualization, and instance or database consolidation. Similarly, for high availability, manageability and other potential concerns, we can walk you through the options.
Other resources include our all-up Virtualization Brief - at two pages it's certainly brief, but it will give you a pretty good high-level overview of the options and benefits for SQL Server, Microsoft SharePoint and Microsoft Exchange. We also have a rather good customer case study with Avanade, where you can see how the right choices enabled them to cut physical servers from 136 to 20, while increasing their database performance by 50%. (I can tell you now, they were not IO bound!)
In our video conversation, Ted also makes it clear that partners will be important in guiding customers. Our generic guidance and tools are invaluable for understanding our architectures and capabilities, but partners are especially well placed to get into specifics, whether hardware or integrated systems.
This conversation about virtualization and consolidation is one we often have directly with customers: it was great to be able to have it with Ted before the cameras for a wider audience.
I was really in Vegas, for a couple of days. We stayed downtown on Fremont Street, visiting with Alison's vacationing parents, just before the conference, then shuffled up the strip to ritzier lodgings for the main event. It's not much of a boundary to cross, perhaps a formality in some ways, but nevertheless significant. Fremont street has its colourful fun, but it isn't the Strip: and The Four Seasons is surely not the Lady Luck.
I have recently been crossing other boundaries too. For one thing, I have a new role within Microsoft. Just like driving up from Las Vegas to the Strip, to the casual observer you may not notice much difference, but difference there is. Here's the change ...
I have always enjoyed being very close with customers, partners and the wider Business Intelligence community. Since I first joined Microsoft, back in 2001, I have been working in engineering teams, striving sometimes to keep up with the deeply technical side, but still trying to keep in touch with what our customers and partners needed in the real world. It has been a great role, but a difficult one to balance. So now, rather than being in a single vertical product team - Analysis Services, or Integration Services, for example - I'll be working in a cross-team role. My focus will be to help Microsoft articulate a vision for business intelligence, and to improving our technical engagement with analysts, partners and other teams. As part of this role, I'll still need to keep technically close to the development teams: still contributing to engineering execution and vision as I can. (I still have some patents up my sleeve!) Nevertheless, the role does change. I'll miss leading my group of Program Managers - but I'll still be working with them daily. Maybe I'll even miss the hassle of integrating trees of code, fighting bug fires, and juggling development and test resources. However, I have much to look forward to, especially as we enter a whole new area with our "managed self-service" approach to business intelligence.
If you want to keep up with my new role, and the various technical and community initiatives in which I'll be involved, please do read my blog here. It's a Microsoft blog, for sure: in the sense that it is focused on our products, and our issues. For example, my next post will cover our new PowerPivot product, explaining some of the thinking behind our direction, and behind the name. You can also read my other blog where I'll be rather more expansive on issues of broader interest in the BI community. And you can follow me on Twitter: donalddotfarmer.
It's going to be fun. For me, better than Vegas
You've probably seen some press lately about how pervasive Microsoft is in lots of areas (here, here and here). We've also seen SQL Server show up in more and more enterprises - you can read lots of case studies here. There are lots of reasons that you're seeing SQL Server numbers grow - we're less expensive than other offerings, have a pretty good track record and a great toolset to monitor and manage the system. We've also got good support, a well-established repuation in large enterprises and all the way down to small businesses, but there's one factor that I'm seeing make a difference. It's our community.
I've used lots of software over the years (I'm really old) and I've seen lots of community efforts. Open-Source does a great job of keeping people in contact with each other, as do a few other vendors. But I see a LOT of community for SQL Server - people of all levels of skill, all around the world, in multiple industries helping each other out, freely and willingly. Thomas LaRock (SQL Server MVP) recently blogged about this phenomena as well.
I think community makes a difference - if we (Microsoft as a company) listen. And I think we do. I know the folks I deal with constantly monitor social networking sites, blogs, press reports and other sources for a mention of their product area. When they see someone struggling with a feature, they really take that to heart. No, we don't base every product decision on a sampling of people talking, but we sure do listen to what they are talking about.
If you're not connecting with the SQL Server community through social networks, blogs, user groups, SQL Saturdays, PASS or any number of other sources, you're missing out. There's lots of folks out there willing to help - and happy to do it.