Ross Mistry, a noted author on
many Microsoft topics, has his latest book out, and it's on SQL Server 2008 R2.
We're able to make that available to you for free in electronic form - just go
here to pick up your copy: http://blogs.msdn.com/microsoft_press/archive/2010/04/14/free-ebook-introducing-microsoft-sql-server-2008-r2.aspx
One of the most exciting times in a product release cycle is when the launch festivities begin. In the product development team it means that we get to start the celebration! That is until we start work on the next release, which for us has already begun. The SQL PASS European conference in Neuss, Germany will feature a special full-day SQL Server 2008 R2 Launch Event. Donald Farmer is the keynote speaker! You can find more information about the event here.
I saw a post the other day that you should definitely go check out. It’s a cost/benefit decision, and although the author gives it a quick treatment and doesn’t take all points in the decision into account, you should focus on the process he follows. It’s a quick and simple example of the kind of thought process we should have as data professionals when we pick a server, a process, or application and even platform software.
The key is to include more than just the price of a piece of software or hardware. You need to think about the “other” costs in the decision, and then make the right one. Sometimes the cheapest option is the cheapest, and other times, well, it isn’t. I’ve seen this played out not only in the decision to go with a certain selection, but in the options or editions it comes in. You have to put all of the decision points in the analysis to come up with the right answer, and you have to be able to explain your logic to your team and your company. This is the way you become a data professional, not just a DBA.
You can check out the post here – it deals with Azure, but the point is the process, not Azure itself: http://blogs.msdn.com/eugeniop/archive/2010/03/19/windows-azure-guidance-a-simplistic-economic-analysis-of-a-expense-migration.aspx
OK, sort of.
I've been a fan of the Economist magazine since a friend of mine in Florida introduced me to it years ago. Imagine my shock when I check out the latest issue and find a story about - Big Data! This is the kind of thing we think about all the time in the SQL Server group here at Microsoft. Then when I read the article, I find this stunning bit of information: did you know that Wal-Mart handles more than 1m customer transactions every hour, feeding databases at more than 2.5 petabytes – the equivalent of 167 times the books in Americ’s Library of Congress?
And Wal-Mart uses SQL Server. And SQL Server 2008, no less - with Policy Based Management at the forefront of helping them track this all. Also, if you read the article, you'll see that Craig Mundie (Microsoft) and Eric Schmidt (yeah, the boss of Google) both sit on a presidential task force dealing with health care data.
This is heady stuff. Sometimes I get asked if SQL Server can scale, if it's really different than it used to be - and since I live and breathe the database world, I'm surprised by the question. But when I think how quickly we've moved into the upper tiers of the enterprise, I'm not as surprised.
Definitely check out the article here: http://www.economist.com/specialreports/displayStory.cfm?story_id=15557443. For more about a REALLY big SQL Server, check this out: http://msdn.microsoft.com/en-us/library/aa226316%28SQL.70%29.aspx. and Gizmodo (a must-follow blog) has a good story on Pivot (hit the link if you don't know what that is) here: http://gizmodo.com/5488641/, Awesome.
There has been a lot of buzz lately about "the
cloud" and what it means for IT, and the organizations they serve. I
just finished watching Steve Ballmer deliver Microsoft's vision for the cloud,
wondering if it would be different than the offerings of other firms. It feels
like it is different. So what is that
strategy, and more importantly, what does that mean to the data professional?
First, let's recap what Steve mentioned for
Microsoft's strategy. The word "cloud" can mean a lot of things, from
hosting data (often called "somebody else's hard drives) to running
software where the user logs into an application hosted on a web server somewhere
(called Software as a Service, or SaaS). Microsoft actually has both. In fact,
we've had things like Hotmail (which is a service) and the various Live
offerings for quite some time. And we've had XBox Live, which is a hosted
environment that is kind of a hybrid - there's a "fat" or hardware
client that talks to the main server out on the web. There's also an offering
of a complete Microsoft Office, SharePoint services and even LiveMeeting in the
cloud - nothing to install, runs on lots of platforms, and more (details here).
But there's something new. Microsoft has two new
offerings, called Azure, and SQL Azure. Azure is more of a programming platform
that can host data, and SQL Azure is SQL Server in the cloud. But SQL Azure
isn't just a server you rent - we actually maintain the systems, handle the
optimizations and so on. You create databases and database objects. You can
take the data from SQL Azure and send it to a local SQL Server, and vice-versa.
There's also the ability (through something called Sync Services) to replicate
data between the two.
So how does the Data Professional meld in the
"cloud" to our day-to-day systems? How do we use Microsoft's strategy
in our own strategies? I've seen a couple of interesting uses so far from those
that are trying it out:
Start, Back-End Archive: In this
mode, companies spin up an application quickly (with no server build) into
Azure, backed by SQL Azure. They are able to quickly deploy the app, and if it
grows they can bring that data in-house or even bring a subset of that data
locally for reporting, keeping a smaller data-set up in SQL Azure.
There, Stay There: Some of the
companies I've seen are taking the application ideas and starting them in SQL
Azure or Azure (or both). No capital expense, no hardware purchases, no
installs, nothing to deploy. When the app is developed, you just point your
clients there and off they go. It's a pay-as-you-use system, so the costs mimic
There, Come Here: In other cases,
organizations want to start projects quicker than they can get hardware and
software installed. So they follow the previous process, and then bring the
code and the database in-house when they are ready.
There are other strategies, such as using the cloud
as part of High Availability/Disaster Recovery or a remote-office access system
and so on, but whatever your reasons, you need to spend a little time getting
familiar with the cloud - and Azure and SQL Azure should be high on your
research list. Here's some places to get started:
Azure Overview: http://tinyurl.com/y8s52vh
Windows Azure Training Kit: http://tinyurl.com/5vrt7q
Writing a Azure Program in 5 steps: http://tinyurl.com/ye6chog
Azure Data Sync: http://tinyurl.com/ylsykfb
Future of programming with Azure Video: http://tinyurl.com/ykqsbsf
Reference list for Windows Azure: http://tinyurl.com/yze9azr
In my last posting I spoke about changing databases. In this posting I want to jump into a little more detail on some things to improve the success of your migration project. I’ll spare you the lecture on planning – but remember the best way to improve the success of any migration project is ensure you spend time planning. So here are my tips for what I’ve seen work:
First, as part of the planning phase be sure to inventory all upstream and downstream dependencies. What systems feed into the one being replaced? What assumptions are these systems making about the system being replaced? This shouldn’t be too difficult but it’ll take a little time. Downstream dependencies are far harder and are going to require significant detective work. The main challenge here is degrees of separation. You should be able to find those systems that are one degree away by analyzing connections and SSIS/DTS packages. This won’t be straight forward but you should be able to account for the majority of cases. The farther you move down the dependency chain the harder it’ll be and the longer it’ll take.
Second, you’ll need robust test data. You won’t be able to account for every scenario so you need to think through the interesting cases. Hopefully you have a starting point, what you currently use to validate new releases of the existing system. You may need to expand this set to be more encompassing given you’re changing the entire system. When you create your test data think through any special processing windows, end of month, end of quarter, end of year, that may have different requirements. You may need different data sets to test each of these special processing windows. To create the test data you may want to use existing tools that capture data/transactions, some of which can obfuscate data.
Third, running test data through the system isn’t enough. You will need tools to compare the results from each system to ensure the new system yields the same results. Depending on your situation you may have to build custom tools. This may not seem like a good investment but it’ll pay dividends.
Fourth, running test data through each system is necessary and once you’ve achieved parity an even better validation is to run the systems in parallel. The old system should stay as primary but each transaction should secondarily be routed to the new system. For a batch processing system this will be easy. For on-line systems you’ll need a capture/replay tool. Though usually minimal, these tools do add overhead to the system so be sure to plan accordingly.
Fifth, create a bridge between the new and old system. This means, once the new system is the primary for upstream systems you'll want to load the old system to lessen the urgency of migrating downstream dependencies. This is a temporary solution until all downstream dependencies are migrated to the new system.
Lastly, once all dependencies have been migrated to the new system you can unplug the old. The team has worked extremely hard to reach this milestone and you don’t to let it pass by without celebration.
Every migration will be a little bit different. Some will be easy, taking as little as a weekend, while others will be extremely complex taking a year or more. The most important advice I can offer is break it into multiple phases and validate at the end of each phase before moving to the next!
I’m not sure why but there seems to be a lot of chatter lately about changing DB platforms. Information Technology Intelligence Corp (ITIC) recently ran a survey about DB migrations. You can find the result here (search the page for “ITIC Sunbelt 2010 SQL Server Survey Results”). At the risk of sounding like an advertisement Microsoft has provided a SQL Server Migration Tool Kit for some time. It aids the migration to SQL Server from Oracle, Sybase, MySQL and even Access. But this is never a slam dunk. It takes careful planning and testing to ensure the applications that use the data source continue to function properly.
I’ve never heard of a DBA waking up one morning and deciding to migrate from one DB platform to another. DB platforms are pretty sticky and there is almost always some catalyst for the change: license/contract renewal, change in supported platforms by a packaged application, vendor consolidation, etc. Let’s be honest migrations are scary projects and you want to be sure the drivers are well understood by the project team and the stakeholders. Then you want to be sure you take your time to plan and analyze the situation.
Over the course of my career I’ve worked on a few very large migration projects that involved more than just the database platform. These had large budgets and extremely stressful. It would have been very easy to become complacent and ignore the need to migrate but opportunity would have been lost. Most of the projects were successful (even the greatest baseball player of all time doesn’t bat 1,000). They resulted in cost savings, staying in a supported environment, or creating more flexibility for the business. The reward was worth the risk. I’d say the two key things that contributed to the success were: breaking the project into meaningful stages and a comprehensive set of regression tests (running against old and new and comparing the results).
I’ll have more to share about migrations in my next posting. Bottom line, don’t let FUD keep you from doing what you need to do to achieve your business goals.
Donald Farmer, he of the BI fame, has been on a roll lately. He and I have something of a competition going most of the time, but it's a "friendly" competition. See, my audience is usually more administration and programming focused; his more BI focused. And no one can deny that BI has a lot more eye-candy. Of course, I explain to Donald that I actually look better than he does, so we're even on the eye-candy part.
OK, enough of that. BI is a hot topic, and people are asking me about it quite a bit, much to Donald's amusement. Since he held a great chat on BI a while back, I thought I would share the link: The Business Intelligence Agenda for 2010. It's a free listen, but you do have to register.
OK, Donald. You owe me one.
Some things just
really work well together. Peanut butter and chocolate. Abbot and Costello.
Coffee and...well...anything. You get the idea. But there are some real advantages
in using SQL Server 2008 (and of course the upcoming SQL Server 2008 R2
release) with Windows Server 2008 and higher. It's not just that they are both
better than their predecessors, SQL Server actually takes advantage of the
improvements in Windows Server 2008.
example is in how Windows Server 2008 handles the infamous "drive offset". This
is a small block size movement from the first part of the hard drive sectors -
it's an internal thing - but it causes real issues with software that exercises
the I/O subsystem, and makes its own calls there. Like SQL Server. In the past,
the data professional had to follow a process called "Partition Alignment", and
this had to be done when the system was set up. That's all now a thing of the
past - with SQL Server 2008 and Windows 2008 Server, this just happens.
Another example is
in how Windows 2008 Server deals with the "sliding TCP/IP window". This
enhancement directly affects how fast SQL Server can send large frames of data
- especially with Replication and large binary objects. At Microsoft we noticed
tremendous speed gains just by moving to Windows 2008 Server.
There are lots of
other examples - from new virtualization and consolidation changes in both
products to clustering enhancements, and now in SQL Server 2008 R2 the ability
to run the "sysprep" utility after SQL Server has been
You can read more about this "pairing effect" in this White
And be sure to check out John
Kelbley's Post on the Windows Server blog where he also talks about ways
that SQL Server and Windows Server work "Better Together".
professionals are familiar with the term "planned downtime" and
"unplanned downtime". The first is painful to ask for, and the second
is painful to explain. We strive not to have either. SQL Server 2008 and SQL
Server 2008 R2 have introduced features, such as better recovery from corrupt
pages in a Database Mirroring and so forth that attempt to keep the problems
down. But "planned" and "unplanned" can also be used to
describe our daily work - and we don't have a choice most of the time for how
we deal with either kind.
Planned work is the
task we do because it has a schedule, or at least *could* be scheduled.
Backups, building a server, applying a service pack, reviewing the logs - all
of these could be things that we can schedule. Looking at my "Tasks"
in Microsoft Outlook, I have a lot of things that I have scheduled for today,
this week, and this month. I never really close or complete some of them, I
just change the due date to the next period of time when I need to deal with
that task again.
Other work is very
"unplanned". This kind of work can come from anywhere - from a
co-worker who needs help, a manager with an emergency request, and most of the
time from a server that has a problem, with anything from issues with
Replication to a failed backup.
It's kind of
difficult to meld these two together. When you're in the middle of building a
server, it's hard to leave the server room, run downstairs to talk with an
irate manager and then fix the issue with the system's database so that her
application can still run. Even worse, for data professional it's often a case
of having to prove it *isn't* the database that is causing the performance
problem - but that's another post.
There are, however,
tools and processes that can help you deal with both planned and unplanned
work. As I mentioned, I use Outlook for just about everything, since I can
access it from many locations (even my Windows Mobile phone) and it
combines my calendar, tasks, contacts and of course e-mail in one place.
Another tool I've
come to rely on is OneNote. Of course you can just use notepad or a
Word-processor to take notes, but OneNote is integrated with Outlook (and just
about every other Microsoft program), it can "share" notebooks
between teams and has a rich set of tags to help qualify what I need to know
visually and quickly.
But tools aren't
the whole story. First, I try to keep a level head during the interruptions.
I've been a data professional for a really long time, so I've gotten over the
panic stage. It also helped that at one point in my career I volunteered as an
Emergency Medical Technician (EMT) on an ambulance, which of course *really*
puts you in life and death situations. After that, a server crash isn't cause
for complete panic.
I have developed a
process to deal with both of these kinds of work. I plan what I can - trying to
look out as far as possible, creating checklists, and coordinating with the
rest of my team and my organization. I try to get the most important planned
work done as soon as possible - first thing in the week, first thing in the
morning. That way, if I get an unplanned event, as much as possible of the
planned work is complete. In a way, I'm planning for unplanned work!
For the truly
unplanned work, such as an emergency, I keep a OneNote page nearby with links
that are categorized by the type of issues I think I might face. I document
each step I follow to correct the issue, even if I have to wait until later. I
try and keep the energy from all of the emotions low, and work on the problem
as systematically as it will allow. Above all, I communicate constantly,
letting the right people know what has happened, what is happening now,
and what I'm doing about it. That OneNote document comes in really handy
So how are you
doing it? How do you handle the work that comes at you from all sides?