Evolving the Data Warehouse

Demo Series on MSDN Channel 9

Joe Longtin — Tue, 30 Jun 2009 21:15:00 GMT

Recently Eric Hanson generated more than a dozen short demo screencasts highlighting the best new features in SQL Server 2008 specific to data warehousing. A couple of them are pertinent to earlier versions of SQL Server as well. If you can only view a few, try to watch the configuration ones and the pair on compression. Compression has been a favorite topic of DW customers. They all range from three to eight minutes. I'm posting them over the next couple of days as I encode them.

Channel 9 videos tagged with Data Warehousing

What other demos would you like to see? Email me, or post a comment here or on Channel 9.

SQL Server Magazine articles related to DW

Joe Longtin — Fri, 29 May 2009 02:36:00 GMT

In case you haven't already seen these, here are a few recent articles from SQL Server Magazine. The first one is a short but sweet tip on reviewing partitions. Find Out What's Going on Inside Your Partitions. I'm going to ask one of our PMs to download the code snippet and test it out.

The second item, Configuring SQL Server 2008's Resource Governor, is a lengthier lesson on using Resource Governor in SQL Server 2008 to allocate CPU and memory to different resource pools and workload groups. Greg Low, the author, explains the clear advantages of Resource Governor for data warehousing while pointing out a few of the situations where it is not useful, such as OLTP apps. (Unless, heaven forbid, you're running your OLTP database and DW on the same box.)

7 Steps for Successful Data Warehouse Projects is a high-level DW 101 that emphasizes taking the long view from a project management perspective. The author, Michelle A. Poolet, breaks the task into three big chunks - a track each for Technology, Data, and Application Layer. Some good caveats included.

If you want to go to the next level - beyond the Gantt chart, actually diving into code - I recommend looking at The Microsoft Data Warehouse Toolkit (ISBN-10: 0471267155), which goes another level deeper and is actually used as a textbook at some schools. Can't wait for the next edition focusing on SQL Server 2008 DW enhancements.

New Project Madison datasheet and related webinar June 25

Joe Longtin — Thu, 14 May 2009 19:53:00 GMT

Since it has been a while, some items have accumulated...

Marketing stuff:

We recently refreshed the Project Madison home page and tweaked the other data warehousing pages to help people navigate through them. Check out the brand-new Project "Madison" data sheet. I would be interested to get your feedback and questions.
A related webinar is coming up: TechNet Webcast: SQL Server Data Warehousing Road Map: Build, Manage, Deliver, and Grow (Level 300). We hope that this answers some questions around moving your existing environment onto SQL Server 2008 Enterprise and a Fast Track reference config.

Product stuff:

We are on track to release the first private technology preview for Project “Madison” in the second half of this year. A select group of customers will deploy a prototype MPP data warehouse on Madison. These previews are different from typical CTPs due to the unique nature of the solution.
I rediscovered a couple of short videos on data warehousing.

One was specific to the enhancements in SQL Server 2008, with Eric Hanson, our principal program manager. You can find it on MSDN Channel9. (opens in new window) Eric also recently put together a series of brief (3-5 minute) demos that we will post to Channel9 soon. I will post a note when they're available.
A second video from a third party was an introduction to DW on SQL Server and Analysis Services on TechTarget's SearchSQLServer site. This is a 100-level intro to DW that delves into Analysis Services. (opens in a new window)

Finally, thanks to the customers who dedicated people and $$ to attend Tech Ed in Los Angeles last week. We were elated to talk to some customers who are really taking advantage of the major DW features in Enterprise, particularly Data Compression and Resource Governor. One happy VP of IT had discovered that data compression not only reduced disk space, but in many cases it resulted in huge performance improvements because working data sets could often fit entirely in memory - reducing disk IO.

Upcoming webcast on Kilimanjaro and Madison

Joe Longtin — Thu, 23 Apr 2009 02:31:00 GMT

On April 27 and April 30, we're offering a "level 100" webcast presentation on both releases. The content will be the same for each date, so please register for the one that suits your schedule. If neither time works, you can register and view it on demand later.

This session will provide you with an sneak preview into the next release of SQL Server codenamed ‘Kilimanjaro’ as well as project ‘Madison’ both of which will ship in the first half of CY 2010. This session will introduce at a high level the set of upcoming technologies in the areas of Self Service Business Intelligence, Application & Multi-Server Management and Scale-out Data Warehousing as well as several other key capabilities planned for the next release of SQL Server.

April 27, 2009
11:00am-12:00pm Pacific
http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032413070&EventCategory=4&culture=en-US&CountryCode=US

April 30, 2009
9:00am-10:00am Pacific
http://www2.eventsvc.com/academylive/register/95712536-abbb-4621-a484-33cb28bce69c

Fast Track Data Warehouse - Bridge to Madison

Joe Longtin — Wed, 25 Mar 2009 20:39:00 GMT

About our new reference architectures. When DATAllegro joined Microsoft last fall, we saw an opportunity to help SQL Server customers leap forward in scalability and performance in a short time frame, by creating new balanced configurations specifically for data warehouses.

We initiated the SQL Server Fast Track Data Warehouse offering and within the space of just a few months, we had tested and proven several new reference configurations on HP and Dell servers, as well as EMC storage. In addition our team drafted a pair of white papers – an introduction and an implementation guide.

Several things make Fast Track unique and compelling. At first glance, these reference architectures provide orders of magnitude improvements in speed, with more predictable performance. The cost per terabyte is lower than most other data warehousing reference architectures for SQL Server, and as of this date, much lower than many DW appliances on the market. Customers that have EA, APA or other licensing agreements in place enjoy even lower pricing.

Going one level deeper, there are a few critical principles that guide us in building a new reference configuration. These are the first reference configurations developed and tested by Microsoft, and further, they are designed with a laser focus on optimizing for data warehousing, not all-purpose. Sequential IO results in far better performance than random IO, so we avoid random IO as much as possible.

We’ve strived to balance the utilization across all components, from the CPU to the SAN storage disk drive, which is also unique. For example, there’s often a tendency to max out spending on processors, but in practice the processors are underutilized when other components are running at 100%. With a Fast Track DW system, every component operates at or near its maximum or optimal rate, thus eliminating hardware performance bottlenecks.

Check out the TechNet portal for Data Warehousing where you can download the Introduction and Implementation white papers.

First post: Project Madison Questions

Joe Longtin — Wed, 25 Mar 2009 19:27:00 GMT

Welcome! To kick this blog off I’d like to share some of the most frequently asked questions regarding Project Madison, the new MPP data warehousing solution based on SQL Server.

Is Madison a Data Warehousing appliance? Is it the same thing as DATAllegro?

Madison will follow many of the same principles that DATAllegro employed in designing its DW appliance. Strictly speaking, Madison will be delivered as an appliance-like solution. Madison is different from the current DATAllegro product in terms of the hardware configuration, OS and database. However, the vast majority of the “secret sauce” that enabled all of the hardware components and the software layer to work together will be carried forward in the Madison solution. Like DATAllegro, Madison relies on using industry standard hardware to keep costs low and facilitate scale-up and scale-out. The plan is to also keep the sales and support model very streamlined for the customer, so that they do not have to work with mulitple vendors to purchase or to log a support ticket.

When will it ship?

Our plans to release Madison in early 2010 are on track. Of course there is a lot of work to do between now and then. MTP and TAP beta programs will kick off this June and will run through early 2010.

What’s the feature parity with the DATAllegro v3 appliance?

Madison is planned to have functional parity with the last major release of DATAllegro, with a few critical enhancements to support some unique capabilities in SQL Server 2008. As we draw closer to MTP1 and the TAP programs, we will be able to share more details about the features.

Will Madison support SQL Server 2000, 2005?

Previous releases of SQL Server will not be supported. Madison is based on SQL Server 2008 Enterprise because this release has unique enhancements around data warehousing that result in a much more comprehensive and easy to manage data warehouse.

Learn more about Project Madison on the home page and check out the Enterprise DW Architectures white papers on the TechNet portal for DW.

To those who have tuned in to the Data Beat blog, thanks! In the future, please join us here.