Rob's SQL Server Blog : SQL Server

Scottish SQL Server User Group Meeting - Oct 8th

robcarrol — Thu, 24 Sep 2009 18:38:00 GMT

I have the great pleasure of speaking at the next Scottish SQL Server User Group meeting being held in Mocrosoft's Edinburgh ofice on Thursday the 8th of October. Full details can be found on the UK User Group site. My talk will focus on some of the great tools we have for collecting and analysing SQL Server performance data. Martin Bell MVP will also be talking about using Powershell with SQL Server, which should be excellent as always.

Hope to see you there !

Troubleshooting SQL Server 2008 Performance Problems

robcarrol — Sun, 12 Apr 2009 23:26:00 GMT

A new whitepaper covering the topic of performance troubleshooting in SQL Server 2008 has now been released. You can download the paper from the following link:

http://msdn.microsoft.com/en-us/library/dd672789.aspx

SQL Server 2005 Multi-Site Clustering with Windows Server 2008

robcarrol — Mon, 16 Mar 2009 01:12:00 GMT

I was working recently with a customer who was looking to deploy a SQL Server 2005 cluster across 2 geographically dispersed sites using Windows Server 2008. They were looking to utilise the new clustering improvements in Windows Server 2008 to build a highly available SQL Server solution. The customer required automatic failover between the sites in the event of a disaster, but their current solution required manual intervention by an administrator in order to failover to the disaster recovery site. Automatic failover would increase application availability, and reduce the complexity of the solution. Each site has it's own SAN storage and the customer planned to replicate data between each site using SRDF replication.

This led me to do further research into clustering SQL Server in this type of environment. Windows Server 2008 introduces greater flexibility in the choice of Quorum configuration. The concept of quorum moves away from the requirement of a shared storage resource, but now refers to the number of votes needed to establish a majority. All nodes and a witness resource can get a vote, which removes the disk as the single point of failure as in previous clustering models. The 4 Quorum Models available are:

Node and Disk Majority
Disk Only
Node Majority
Node and File Share Majority

As there is no shared storage between the nodes in a multi-site cluster, 2 of these Quorum models are suitable for multi-site clustering: Node Majority and Node and File Share Majority. Node and Disk Majority and Disk Only should only be used in a multi-site cluster if specifically directed by your storage vendor as your disk replication software needs to support these configurations.

Node and File Share Majority:

This allows the creation of up to 16 nodes with no shared disks. A file share acts as a witness, meaning that a 2 node cluster will have have 3 votes, so connectivity can be lost by either one of the nodes or the witness and the cluster can still continue to function.

A cluster quorum configured to use a node-and-file-share majority is a great solution for multi-site clusters. The file share witness can reside at a third site independent of either site hosting a cluster node for high disaster resilience. A single file server can serve as a witness to multiple clusters (with each cluster using a separate file share witness on the file server).

This configuration gives the highest resilience as the cluster can automatically recover from the loss of any one site without manual intervention.

The File Share Witness (FSW) needs to be in the sane forest as the nodes and be running Windows Server 2003 or Windows Server 2008. For maximum resilience, it is best to locate the FSW at a 3rd site separate from the cluster nodes. The FSW does not need to be attached to shared storage and should NOT be a node in the same cluster.

Node Majority:

A node-majority cluster consists of 3 or more nodes without shared storage. Each of the nodes has a vote and there is no shared disk vote. A majority of votes are necessary to operate the cluster if 2 nodes fail in a 3 node cluster, then the remaining node drops out of the cluster. An administrator can manually over-ride this and force the remaining node to start. When the other nodes come back, majority quorum is achieved again and the cluster comes back online seamlessly.

This configuration works best with an odd number of cluster nodes as it is not enough to have half the cluster nodes functioning in this model. If four nodes were set up in a node-majority configuration, the cluster would continue to operate with the loss of one node but not with the loss of two nodes. You should use an odd number of nodes with Node Majority as 4 nodes can only survive 1 failure, which is the same as 3 nodes.

The node-majority quorum configuration can work when there is more than one cluster node at each site. Consider a multi-site cluster consisting of five nodes, three of which reside at Site A and the remaining two at Site B. With a break in connectivity between the two sites, Site A can still communicate with three nodes (which is greater than 50 percent of the total), so all of the nodes at Site A stay up. The nodes in Site B are able to communicate with each other, but no one else. Since the two nodes at Site B cannot communicate with the majority, they drop out of cluster membership. (Were Site A is to go down in this case, in order to bring up the cluster at Site B, it would require manual intervention to override the non-majority.)

As a result, the Node Majority configuration does not give automatic failover between sites as nodes 4 and 5 cannot achieve quorum. In this situation, you would need to manually force a failover.

SQL Server Networking Considerations:

Windows Server 2008 now allows nodes in the same cluster to reside in different network subnets and communicate across network routers. However, be aware that SQL Server 2005 and 2008 still require all cluster nodes to reside in the same network subnet, so you will still need to set up virtual local area networks (VLANs) to connect geographically separated cluster nodes. This can have some benefits with regard to client response times though, as DNS replication may impact client re-connection times in the event of a failover from one site to another. VLAN's allow DNS names to stay the same, so can increase availability.

Storage Considerations:

As there is no shared storage between the nodes in a multi-site cluster, the main consideration is how to keep the data replicated between the sites. The choice of 3rd-party replication solution is important and can have a major effect on how you deploy your cluster. As such, you should work closely with your storage vendor from an early stage in the design process.

Synchronous replication results in no data loss, but requires shorter distances between nodes and higher bandwidth to avoid write latency from impacting performance. Asynchronous allows you to stretch cluster nodes across longer distances, however there is a potential for data loss in the event of a failure. Asynchronous data replication also assumes a large enough network bandwidth to keep up with data changes and does not significantly impact application performance.

Conclusion:

As data replication is key in a multi-site SQL Server cluster, work with your storage vendor from an early stage to ensure they support your cluster configuration. Multi-site clustering allows you to achieve high availability and disaster recovery, however it can be a costly and complex solution. You should evaluate your business requirements first and then decide on the best technology to meet these. It could be the case that Database Mirroring, for example, could be used to give you the required level of resilience across geographical sites.

In this case, the customer chose to implement a 2-node, 3-site solution using the Node and File Share Majority quorum model, with a File Share Witness located in the 3rd site. This gives site-level resilience in the event of a disaster and also allows automatic failover between the cluster nodes without having to re-write client applications, meeting the business requirements.

Additional Resources:

Windows Server 2008 Multi-Site Clustering Whitepaper

TechNet Webcast: Geographically Dispersed Failover Clustering in Windows Server 2008

TechNet Webcast: Failover Clustering and Quorum in Windows Server 2008 Enterprise

Support Webcast: Microsoft SQL Server 2005 Failover Clustering on Windows Server 2008

How to: Create a New SQL Server 2005 Failover Cluster (Setup)

Performance Dashboard Reports for SQL Server 2008

robcarrol — Wed, 18 Feb 2009 17:20:00 GMT

During a recent visit to Seattle for a Microsoft conference, I learned from my colleague Michael Thomassy that it's possible to run the SQL Server 2005 Performance Dashboard reports on SQL Server 2008, with a couple of slight modifications. There is a great new feature in SQL Server 2008 called Performance Data Collection, which I have blogged about in the past, and this is excellent for tracking SQL Server performance over time across your 2008 estate. There is also the excellent revamped Activity Monitor in SQL 2008. However, if you want to continue to use the Performance Dashboard reports, which many DBA's have found invaluable, they are not supported in SQL Server 2008. If you try to install the Performance Dashboard reports, you get the following error:

Msg 207, Level 16, State 1, Procedure usp_Main_GetCPUHistory, Line 6
Invalid column name 'cpu_ticks_in_ms'.
Msg 15151, Level 16, State 1, Line 1
Cannot find the object 'usp_Main_GetCPUHistory', because it does not exist or you do not have permission.

The reason for this is due to a change in the sys.dm_os_sys_info DMV from SQL Server 2005 to 2008 (the cpu_ticks_in_ms column has been removed in 2008 http://msdn.microsoft.com/en-us/library/ms175048.aspx). I have attached PerfDashboardReports.zip, which contains modifications to the setup.sql and performance_dashboard_main.rdl files. Download and install the performance dashboard reports as normal (but save the files in the Program Files\Microsoft SQL Server\100\Tools\PerformanceDashboard directory) and then over-write the original files with the modified ones attached, then run the setup.sql file against your SQL Server 2008 instance.

Please note that SQL Server 2008 has introduced new wait types that the Performance Dashboard reports currently don't handle. I would strongly recommend using the new Management Data Warehouse reports in SQL Server 2008 in order to get the best user experience. However, this workaround will help you get the Performance Dashboard Reports up and running on your SQL Server 2008 instances (see the screenshot below).

CU #3 for SQL Server 2008 Released

robcarrol — Tue, 20 Jan 2009 09:39:00 GMT

On January 19, 2009 Microsoft shipped the Third Cumulative Update for SQL Server 2008 RTM.

This CU represents 36 Resolved Issues and 30 Unique Customer Requests. More details and download details can be found here:

http://support.microsoft.com/kb/960484/en-us

SQL Server 2008 Upgrade Technical Reference Guide Released

robcarrol — Wed, 26 Nov 2008 13:11:00 GMT

The SQL Server 2008 Technical Reference Guide has just been released, covering the essential steps and best practices to upgrade existing 2000 and 2005 instances to 2008. There is also a list of other upgrade resources available at http://msdn.microsoft.com/en-us/library/cc936623.aspx

These guides are intended to suppliment the information already available in SQL Server 2008 Books Online.

SQL Server 2008 Cumulative Update #2 Released

robcarrol — Tue, 18 Nov 2008 14:09:00 GMT

Cumulative Update package 2 has now been released for SQL Server 2008. See the following KB article for full details:

http://support.microsoft.com/kb/958186/en-us

Scottish Area SQL Server User Group

robcarrol — Fri, 07 Nov 2008 02:53:00 GMT

I'm happy to announce that the next Scottish SQL Server User Group will be hosted at Microsoft Edinburgh on the evening of the 13th November. Full details of the agenda and how to register can be found on the UK SQL Server User Group site here:

http://sqlserverfaq.com/events/144/Scottish-Area-SQL-Server-User-Group-Jammin-SQL-Server-2008.aspx

The theme of the meeting is "Jammin' with SQL Server 2008", so I'm looking forward to some freestyle SQL Server 2008 demos and discussions !

Updated: Support Policy for SQL Server on Virtualised Platforms

robcarrol — Wed, 08 Oct 2008 18:48:00 GMT

The support policy for SQL Server running on virtualised platforms has now been updated to include support for SQL Server 2005 running on Hyper-V

http://support.microsoft.com/?id=956893

Microsoft provides technical support for SQL Server 2005 and for SQL Server 2008 that are running in the following hardware virtualization environments:

Windows Server 2008 with Hyper-V
Microsoft Hyper-V Server 2008
Configurations that are certified through the Server Virtualization Validation Program (SVVP) http://windowsservercatalog.com/svvp.aspx?svvppage=svvp.htm

Any non-Microsoft virtualization software that is not a validated configuration through the SVVP program will be handled under the policy in the following KB article:

http://support.microsoft.com/kb/897615/

The following restrictions apply to support for SQL Server products in any hardware virtualization environment:

Guest Failover Clustering is not supported in a virtual machine.
The virtual machine must be running an operating system that is supported for the specific SQL Server version and edition and that is supported for virtualization
The SQL Server product must be a supported version under its current Microsoft Support Lifecycle policy http://support.microsoft.com/?pr=lifecycle

Support Policy for SQL Server on Hyper-V

robcarrol — Tue, 19 Aug 2008 21:10:00 GMT

Microsoft have now published the support policy for running SQL Server on Hyper-V. There is support for SQL Server 2008 initially, with plans to include SQL Server 2005 at a later date.

http://support.microsoft.com/?id=956893

SQL Server 2008 - Performance Studio

robcarrol — Fri, 20 Jun 2008 19:50:00 GMT

I've been looking at this new feature of SQL Server 2008 in order to demo it to customers, and I have to say it's pretty cool ! Along with Resource Governor and compression, it's one of my favourite new features and a good reason to consider an upgrade. Performance Studio builds on the concept of the Database Reports in SQL Server 2005 and the Performance Dashboard introduced in SQL Server 2005 SP2. Like it's predecessors, it's built on top of standard DMV's but with one crucial advantage... data is historical and is persisted across service restarts. Previously, you had to roll your own code in order to persist DMV data, now SQL Server 2008 gives you it straight out the box.

Setting it up is easy... expand the 'Management' folder and then right-click 'Data Collection' then 'Configure Management Data Warehouse'. This opens a wizard which guides you through the configuration of the Management Data Warehouse (MDW). Select the 'Create or Upgrade a Management Data Warehouse' option and enter your server details. You can either configure a new database or use an existing one in order to collect the performance data.

Once the MDW has been configured, run through the wizard again and select the 'Set up Data Collection' option. At this point you need to also ensure that SQL Server Agent is running or else the performance counters will not be uploaded to the database. Also, SQL Server Integration Services is required to manage collections. There are 3 new database roles in the MDW database: MDW Admin, MDW Writer, MDW Reader. It's a good idea to restrict access to this database, especially if you are collecting sensitive data. Performance Studio will only collect data against SQL Server 2008 databases, so unfortunately you can't use it to monitor older versions of SQL Server. MSDB is used to store the log entries, Agent jobs and SSIS packages.

When creating the MDW, plan for data growth of up to 250 - 500 MB a day, depending on your query plans and consider running an archive job to aggregate summary data before SQL Server runs it's purge job after 14 days. Regarding performance overhead, Microsoft detected approximately 3 - 4% increase in CPU performance on it's TPC-C tests, which is fairly low overhead.

Performance Studio comes with 3 built-in Collection Sets: Disk Usage Collection Set, Query Activity Collection Set and Server Activity Collection Set. The Disk Usage Collection Set collects data every 6 hours and retains it for a default of 730 days. It gathers data and log disk usage and plots them over time. This gives a nice visual view of data file growth over time.

The Query Activity Collection Set uploads query activity every 15 minutes and retains it for 14 days. It caches active sessions and requests from DMV's every 10 seconds. It uses dm_exec_query_stats and uploads the 3 most "interesting" queries and any queries where the query plan has changed. What constitutes an interesting query, I have no idea :-) These can then be viewed graphically based on CPU, Duration, Total I/O, Physical Reads or Logical Writes. You can drill-down into the reports to show the query text, query plan, showplan and any missing indexes identified that could improve the query execution.

The Server Activity Collection Set may turn out to be the most useful performance tuning weapon. Data is uploaded every 15 minutes to the MDW and is collected every 10/60 seconds depending on the particular counter. This data is retained for 14 days before being purged. It collects data on Server CPU usage, Memory, Waitstats, Disk I/O and Network Usage, amongst others. Again, you can click through these reports for detailed information.

In addition to the built-in functionality, you can also create your own collection sets, however be careful doing this or you may end up collecting huge amounts of data, particularly if you run a SQL Trace collection. If you do want to create a custom Profiler collection, set up a trace in Profiler then select the "Save as Trace Collection" option. This will then script out the XML trace definition for you which can then be executed against your SQL Server system... simple as that !

For more information, see the following Webcast. It's well worth taking a look !

https://msevents.microsoft.com/cui/WebCastEventDetails.aspx?culture=en-US&EventID=1032349947&CountryCode=US

Indexing Strategies

robcarrol — Tue, 17 Jun 2008 10:02:00 GMT

I attended a SQL Server User Group meeting earlier this year and heard a presentation about favourite DMV's. Mentioned in the discussion was sys.dm_db_index_usage_stats and that got me thinking about indexing strategies. I have been involved in performance troubleshooting databases that have used a variety of indexing stategies, ranging from none to lots of narrow indexes on practically every column ! However, there is no right and wrong indexing strategy, it depends entirely on your application and the type and frequency of queries being executed against the database.

So how do you know if you have a problem? For me, it's usually when users tell me that the application is "running slow" or they get timeouts. However, poor perfomance can be open to interpretation. There could be a whole host of factors to take into account when you are dealing with web applications, such as network connections and web servers problems. However, this shows that you need to have good benchmarks in place in order to compare performance over a period of time. Another sign could be high CPU, high memory usage or increased disk IO activity. The SQLCAT team has a post detailing the top OLTP performance issues on 2005 and some basic performance counters to monitor can be found here.

So once you've determined that there is a problem, what do you do? If you know the specific query that is causing the issue, then you can verify the query execution plan. You do this by chosing the 'Display Estimated Execution Plan' option under the 'Query' menu in Management Studio. This does not execute the query, but returns the estimated execution plan based on the statistics the server has. As a result, you need to ensure that the statistics are up to date, or you may get the wrong results. It's also a good idea to turn on statistics IO. Things to look out for in the execution plan are table or index scans and hash aggregates. Scans imply that there are no indexes for SQL Server to use or the indexes are not selective enough and SQL Server has decided it's less expensive to run a scan. Bear in mind that a clustered index scan is exactly the same as a table scan. Hash aggregates have to create a worktable i.e. a temp table in TempDB. Watch out for Hash aggregates as Statistics IO does not show the cost of the worktables, which can often be very expensive. As a rule of thumb, anytime you see "Hash" in your plan it means temp tables and this can be done better ! Another cool thing about showplan is that you can force the queries to use different indexes and run them side by side. This will show the execution plan for both queries and the cost of each relative to the batch. This is a quick way to see which index choices are most expensive.

However, on shared systems with multiple applications running against the SQL Server instance, chances are you will not know the queries that are causing the performance issues and you will need to do some digging. You can use DMV's in 2005, but as I work in a mixed environment (2000 and 2005) I prefer to use profiler to give me an idea what is going on. I run a server side trace and remotely log the results to a file as oppossed to a SQL Server table to minimise overhead on the system. I will typically run this for short periods (10 - 15 mins) just to get a feel for the queries executing against the server. This is a quick way to see if there are any expensive queries running and how frequently. This gives me some clues as to which database is causing the problems. Once I know this, I can really focus in on that database application. More information on running Profiler traces is available here.

Once I have identified the database, the hardest part is deciding which queries to index. Lots of frequently run queries can give you a far bigger overall performance gain than a large query run once a month. It is important to use Profiler and run traces over longer periods to get a feel for the mix of reads and writes. Never build an index in isolation, always consider the workload over the course of time. You can then combine these traces with DTA in order to fully analyse your workload and come up with the proper indexing strategy based on your application's specific needs. It is also worth restoring a backup of your database on a test system and use this for running your DTA analysis on. I cannot stress highly enough, do not create indexes for indexes sake. I often ask the question when interviewing, "when would droping an index actually help to increase performance"? A lot of people are conditioned to believe that you need to build lots of indexes to increase performance, but this is not true. If you have lots of inserts, updates and deletes then this is a significant overhead to keep all these indexes updated if they are not required in the first place.

Before going off and building indexes it is worth looking at alternative options. It is a good idea to update your statistics to ensure that SQL Server has most up to date information to work out the optimal execution plan. If you are working with stored procedures, you can consider recompiling them. You can also consider re-writting the code, especially if you are using cursors!

Generally, SQL Server does better with wider indexes (covering several columns) and fewer of them than it does with narrower ones. Narrow indexes will require a bookmark lookup to get the rest of the data if you don't cover the query. Bookmark lookups are expensive and SQL Server may even decide not to use the index at all and perform a table scan. In this situation, the narrow indexes will not be used and are an unecesary overhead. You can use the DMV sys.dm_db_missing_index_details to see which multi-column indexes could give better performance. Another word of warning here if you are using the 2005 DMV's, such as sys.dm_db_index_usage_stats. Remember that the data in these DMV's is not persisted across server shutdowns, or database restores. Do not assume that because the DMVs show that an index has not been used you can safely drop it. This only means that it hasn't been used since the cache was cleared out. If you are going to rely on the DMV's, you need to persist the data over a period of time and then analyse it. This is easy to do as you can select directly from the DMV's into tables, which can then persist the data. A good time period would be to persist the data every 30 mins. Paul Randal has a post here with information how to persist the index usage data from DMVs.

The final thing I'd like to say about indexing (for the time being !) is that you can't just create indexes and walk away. You need to regularly maintain them in order to reduce fragmentation. Fragmentation in indexes can be caused by inserts, updates and deletes and can be a problem if you have a busy system. By default, indexes are created with 100% fill factor, which means they are densely packed as soon as you create them. If you then need to insert rows or update data, SQL Server will need to carry out page splits in order to do so. To avoid this happening, it is advisable to create indexes with a fill factor of around 80 - 90%, however this may vary depending on the frequency of data updates. Not only will this increase the performance of inserts and updates, it will also keep fragmentation at a minimum. You can check for fragementation using DBCC SHOWCONTIG (2000) and sys.dm_db_index_physical_stats (2005) and remove fragmentation using DBCC INDEXDEFRAG or DBCC DBREINDEX in 2000, and ALTER INDEX REORGANIZE or REBUILD in 2005. There is further information and advce here for rebuilding indexes and updating statistics.

This is a huge topic and I could go on all day, but the importance of good indexing cannot be stressed highly enough ! In summary, you need to ensure that you understand your application's workload, you need to check that the indexes you have are actually being used and that you are not missing any indexes. Finally, you need to maintain your indexes so they are kept in optimal shape. Good indexing means good performance which means happy users !

[This was originally posted on http://sqlblogcasts.com/ in February 2008]

SQL Server 2005 Performance Dashboard

robcarrol — Thu, 12 Jun 2008 14:25:00 GMT

I've had one or two requests recently to set up the Performance Dashboard reports on SQL Server so I've created this blog post to step through the process. Firstly, the Performance Dashboard was created to allow customers and support engineers to monitor the general performance characteristics of a server and investigate the cause of any performance problems that may occur. The goal of the Performance Dashboard is to reduce the time spent discovering the problem so effort can be focused on actually resolving it.

The SQL Server 2005 Performance Dashboard Reports are available as a link off the SP2 Feature Pack (Feb 2007). The Performance Dashboard reports are built on the SQL Server 2005 dynamic management views, accessing data that is already captured by SQL Server 2005. Consequently there is no performance impact of using the dashboard except when you actually open/refresh a report. This allows engineers and customers to go much further on a performance problem while the problem is happening. The Performance Dashboard requires SQL Server 2005 SP2 or later.

Install the Dashboard by running the msi from the link above, which will install to a default location of Program Files\Microsoft SQL Server\90\Tools\PerformanceDashboard. After setup finishes, open Management Studio and connect to the server and run the SETUP.SQL script (once for each SQL instance you want to monitor). Then from Object Explorer select the server, right mouse click and choose Reports – Custom Reports and browse to find the PERFORMANCE_DASHBOARD_MAIN.RDL file. This report is the only report intended to be directly loaded from SSMS; all other reports are accessed as a drill through off of the main report. The help file, PERFDASH.CHM has details about setup and permission requirements, how each report is accessed, details about the general methodology used in the dashboard and how you can use/interpret the information on each individual report.

SQL Server 2008 RC0 Available

robcarrol — Tue, 10 Jun 2008 11:26:00 GMT

The release of SQL Server 2008 has hit another milestone with the news that Release Candidate 0 (RC0) has been made available for public download this morning. This was initially released to Technet Plus and MSDN subscribers last Friday. It is expected that this will be the final release prior to SQL Server 2008 RTM in Q3 this calendar year. This release includes support for upgrading failover clusters from previous versions of SQL Server, as well as several bug fixes and minor improvements that have been made in response to the customer feedback Microsoft has received from previous CTPs.

Interest and excitement is continuing to build for the release of 2008, with 100,000 people downloading the last CTP (CTP6 Feb), and 250,000 total downloads of SQL Server 2008 CTP's. I am also seeing huge amounts of interest from customers who are looking to upgrade straight to 2008 to get the full benefit of features such as Policy Based Management, Resource Governor, Filestream data, Business Intelligence and Compression.

June SQL Technical Rollup Mail

robcarrol — Wed, 04 Jun 2008 00:49:00 GMT

The June SQL Server Technical Rollup Mail has now been released. Check it out for some great SQL Server information and resources...

http://blogs.technet.com/trm/archive/2008/06/01/june-2008-technical-rollup-mail-sql.aspx