Mat Stephen's SQL Server WebLog : High Availability and Disaster Recovery

SQL 2005: Making sense of the new Optimistic Concurrency Controls and the readers that block writers

Mat_Stephen — Tue, 06 Sep 2005 20:11:00 GMT

For a long time now Oracle enthusiasts have bashed SQL Server for not having Optimistic Concurrency Controls. They would demonstrate a SQL Server reader blocking a writer and possibly a writer blocking a reader. Thus, having appeared to demonstrate that SQL can't handle these two operations concurrently, they would therefore conclude that SQL can't scale - certainly not to enterprise level workloads; workloads that involve lots of people writing to a database at the same time as lots of people wanting to read from it.

This, of course, is a rather dirty trick; you only have to look at http://www.microsoft.com/sql/evaluation/compare/benchmarks.mspx to see SQL Server 2000 performance against various typical 3rd party LOB application workloads that involve reading and writing. So how did/do they perform these mischievous tricks - here's how:

Open connection 1

USE AdventureWorks

BEGIN TRANSACTION

UPDATE Production.Product

SET StandardCost = 5.0,

ListPrice = 11.50

WHERE Name LIKE '%sock%'

Open connection 2

USE AdventureWorks

SELECT Name,

StandardCost,

ListPrice

FROM Production.Product

WHERE Name LIKE '%Sock%'

You will now see that connection 2 will not return anything - instead it just sits there doing nothing. Here the writer on connection 1 blocks the reader on connection 2. This blocking will continue until connection 1 either commits or rollbacks the transaction it started with BEGIN TRANSACTION.

Well it would be very easy, at this stage, to get into a religious slanging match - and you know I don't do religion very easily. Luckily, with SQL 2005, I don't even have to think about get religious about this situation, because SQL Server 2005 now supports the necessary isolation levels to stop this happening.

At this point I'm reminded of an old joke told by Tommy Cooper: Bloke goes to see a Doctor, he lifts his arm in the air and says, "Doctor, every time I do this it hurts." And the Doctor says, "Well don't do it then."

I'm reminded of this joke because if you begin a transaction, update a value and then you find it causes other people problems - I would suggest you don't do it. Why begin a transaction, do some work and then walk off without committing it?

Okay - so you might have a 'really big update' that takes ages to run and it can't be batched up into smaller transactions and you need to run some very long running reports that have to have a transactionally consistant view of the data. Indeed; you might also be very unlucky and get struck by lightening!

In SQL 2000 there are various ways to obviate the encumbrance of this 'really big update' and nasty report – but I don't want to go down that road, like I said I want to avoid a religious debate. Now, in SQL 2005, we have exactly the same capability as Oracle to make such an operation very easy for the developer. Like Oracle, this capability employs the concept of 'versioning', which means each connection in the above scenario will work with its own version of the data, thus avoiding contention.

However as with Oracle, versioning takes up both processing and i/o at the server, resources that are potentially very expensive and not to be wasted if you don't have to. Does Oracle do any benchmarks with this feature turned on? I don't think so, and nor would SQL for that matter.

I will argue that versioning is frequently used as a way to make developers jobs easier and not because it’s actually necessary. By that I also mean it makes it easier for developers to write sloppy code - and by that you can happily infer I believe it makes it easier to employ cheaper developers.

So how do we go about ~~employing these cheaper developers~~, sorry, reducing contention on the database? To do this we now have two new isolation levels:

1. SNAPSHOT ISOLATION (Transaction-level Snapshot)

Administrators must set the new ALLOW_SNAPSHOT_ISOLATION database option to allow Snapshot Isolation.

In order to start a transaction that uses Snapshot Isolation a developer must SET TRANSACTION ISOLATION LEVEL SNAPSHOT

RESULT: Every statement within a Snapshot Isolation Transaction sees the same version of data comprised only of committed changes which occurred before the start

of the transaction. Other statements inside other transactions do not see the changes made inside this Snapshot Isolation Transaction.

2. READ COMMITTED WITH SNAPSHOT ISOLATION (Statement-level Snapshot)

Administrators must set the new READ_COMMITTED_SNAPSHOT database option to allow Read Committed Snapshot Isolation

No further application level changes are required to have statements use Read-Committed Isolation.

RESULT: Each statement sees a version of the data that was committed just before the statement began, instead of when the resource is read. This is merely a new implementation of read committed that is non-locking and non-blocking; the data is accurate only as at the start of the statement.

Follow up on Database snapshot performance question from SQL Server™ 2005 High Availability TechNet evening 21/6/05

Mat_Stephen — Wed, 22 Jun 2005 22:31:00 GMT

Follow up on Database snapshot performance question from SQL Server™ 2005 High Availability TechNet evening 21/6/05

Question; how does a snapshot database perform compared to the original database when it comes to querying – can it make use of the indexes on the original database?

Thanks to my colleague in the States, Lewis Bruck, I can confirm that all pages in the database (including index pages) are available to the snapshot database.

Query performance on the snapshot compared to the original database depends on a number of factors: the snapshot is read-only, so fewer locks are obtained, but copied data can introduce disk fragmentation requiring more read costs.

Follow up on Witnesses question from SQL Server™ 2005 High Availability TechNet evening 21/6/05

Mat_Stephen — Wed, 22 Jun 2005 15:29:00 GMT

Database Mirroring:

How many witness can you have in quorum? - One

How many witnesses can you have on one box? - Many

Two SQL 2005 TechNet Evenings I'll be delivering in Reading, UK

Mat_Stephen — Tue, 07 Jun 2005 13:15:00 GMT

For those of you in the area, I'll be presenting these two TechNet evenings in Reading UK - it would be good to see you there!

14 th June Session Abstract: Microsoft SQL Server 2005 is the next version of the successful SQL Server product family. This new edition offers a number of enhancements to provide a product for today’s environment. This session looks at the key areas that concern IT professionals. We look at various security enhancements, including the authentication changes and the granularity of permissions that the product allows. We look at availability and the changes to improve service availability and replication of data. We also cover the improvements in the manageability of the product.

Booking URL: https://msevents-eu.microsoft.com/cui/EventDetail.aspx?culture=en-GB&eventid=118753366

21st June Session Abstract: There are a number of barriers to availability, and only some of these are addressable by DBMS technology. This session looks at some of the features of SQL Server 2005, the next release of the SQL Server product, that can help address some of these barriers. We look at three specific areas: database server failure or disaster, user or application error, and data access concurrency limitations, and also discuss database mirroring and clustering.

Booking URL: https://msevents-eu.microsoft.com/cui/EventDetail.aspx?culture=en-GB&eventid=118753378

Processing XML Showplans Using SQLCLR in SQL Server 2005 - Make Use of the New XML Showplans

Mat_Stephen — Mon, 23 May 2005 18:23:00 GMT

SQL 2005 now provides Showplan (Query plan) output in XML - why? To make it easier for a dba/developer to write TSQL, or other code, that can determine the cost of a query and thus dynamically determine if the query should be prevented from running based on the load it will create on the server.

To make it easy to capitalise on this new feature this article http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsql90/html/xmlshowplans.asp provides all the supporting code for two example solutions. The difference between the two methods is due to the fact that XQuery queries can not be used for in-process data access - so one solution uses purely XPath and the other with a mix of both XPath and XQuery.

TechNet Roadshow Example code for Microsoft SQL Server 2005 Table partitioning

Mat_Stephen — Mon, 18 Apr 2005 17:49:00 GMT

For those who want to give this a try, this is the code I've been using on the 2K5 H1 roadshow for my SQL 2k5 Table Partitioning demo.

USE AdventureWorks
GO

ALTER DATABASE AdventureWorks ADD FILEGROUP FG1
ALTER DATABASE AdventureWorks ADD FILEGROUP FG2
GO

ALTER DATABASE AdventureWorks
ADD FILE
( NAME = dat1,
FILENAME = 'c:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\dat1.ndf',
SIZE = 1MB,
MAXSIZE = 100MB,
FILEGROWTH = 1MB)
TO FILEGROUP FG1

ALTER DATABASE AdventureWorks
ADD FILE
( NAME = dat2,
FILENAME = 'c:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\dat2.ndf',
SIZE = 1MB,
MAXSIZE = 100MB,
FILEGROWTH = 1MB)
TO FILEGROUP FG2
GO

--Step 1
USE AdventureWorks
GO
--Create the partition function
CREATE PARTITION FUNCTION StorePF (int)
AS RANGE RIGHT FOR VALUES (350)

--Step 2
--Create the partition scheme
CREATE PARTITION SCHEME StorePS AS
PARTITION StorePF TO (FG1, FG2)
GO

--Step 3
--Create the partitioned table
CREATE TABLE Sales.PartitionedStore(
StoreID int NOT NULL PRIMARY KEY,
Name nvarchar(50))
ON StorePS (StoreID)

--Add data into the partitioned table
--from the existing Store table
INSERT INTO Sales.PartitionedStore SELECT CustomerID, Name FROM Sales.store
GO

--Step 4
--Retrieve your partition information
--Displays each row in the table and the partition it is stored on.
SELECT StoreID, Name, $partition.StorePF(StoreID)
FROM Sales.PartitionedStore

--Step 5
--You can also display partition information for a row.
--In this case for the row with storeID of 349.
SELECT $partition.StorePF (349)

--In this case for the row with storeID of 350.
SELECT $partition.StorePF (350)
GO

The Final Word in Microsoft SQL Server 2000 Backup, Restore and recovery

Mat_Stephen — Wed, 30 Mar 2005 16:28:00 GMT

I am pleased to be able to point you to the last word in SQL Server 2000 Backup and Restore methods.

At last all the answers in one place!

SQL Server 2000 Backup and Restore

Published: March 1, 2005

Author: Pankaj Agarwal

Published: February, 2005

Summary: This paper discusses the types of backups that are available in SQL Server 2000 and how backup functionality can be used in a disaster recovery plan. It includes general recommendations on how to improve backup and restore throughput and several case scenarios.

http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/sqlbackuprest.mspx

Microsoft SQL Server Performance Top Tip: Degrees of Parallelism and a Degree of Uncertainty

Mat_Stephen — Tue, 08 Feb 2005 17:50:00 GMT

Max degree of parallelism is an advanced configuration option in SQL Server about which many question have been asked and much written. In this blog entry I hope to offer some insight and certainty as to what this option does and how it should be used.

Firstly, I must dispel any idea you might have, that this setting has anything to do with how many processors SQL Server can use when trying to service more than one connection (or user) – it hasn’t. If SQL Server has access to four idle processors and it has four processes to run, it will use all four processors – regardless of the max degree of Parallelism.

So what does this option do? This option sets the maximum number of processors SQL Server can use for one query. If SQL Server has to bring back a lot of data (lots of rows) for a query it sometimes makes sense to break the query down into a number of smaller queries, each query returning a subset of the total rows. This way SQL Server can make use of more than processor and hence on multi processor boxes it can potentially return a lot of rows more quickly, for a given query, than it can on a single processor box.

There are a number of criteria that have to be met before SQL Server will invoke ‘Intra Query Parallelism’ (breaking the query down into more than one stream), there’s no point in detailing them here you can find them in BOL under ‘Degree of parallelism’. Suffice to say the decision is based on the available memory on the box and most importantly the availability of the processors.

So why do we need to think about this option – well because leaving it on the default (letting SQL Server decide) can sometimes give you undesirable effects. These effects are:

Queries may go slower with parallelism.
Query time for a particular query may become non-deterministic and this can annoy users. The time may vary because:
1. The query may sometimes go parallel and sometimes it may not.
2. The query may get blocked by a parallel query in front of it hogging the processors.

Before we go on, I don’t think there’s any real benefit in diving into the internals of parallelism, if you’re interested you can look at ‘Parallel Query Processing’ in Books on Line for more detailed information. I think there are only two important things to know about the internals of parallelism:

1. Parallel queries may spawn many more threads than specified in the Max degree of parallelism option. DOP 4 could allow for 12+ threads, four for the query and the extra threads being used for sorts, threads, aggregates and gathering etc.

2. Parallel queries can cause spids to wait with a waittype of CXPACKET or 0X0200

So armed you can now identify spids waiting parallel operations as they will have a waittype of CXPACKET in sysprocesses. To help you here you can use the stored procs in my blog CREATE proc track_waitstats & CREATE proc get_waitstats - useful code for diagnosing SQL 2000 performance

So ‘Queries may go slower with parallelism’ how come?

If the system has a very poor data throughput on the disk subsystem(s) then query decomposition is unlikely to go faster.
Possible skew of data, or possible lock in a data range for a cpu, meaning one parallel process is behind, etc.
If there is a missing an index for a predicate that results in a table scan. A parallel query operation may hide the fact that the query would run much faster as a serial plan with an appropriate index.

Consequently it is recommend you test without parallelism (DOP=1) so these issues can be identified.

The other effects of parallelism mentioned above should be self explanatory and should lead you to the idea that intra query parallelism is not really appropriate for OLTP applications. These are apps where varying query time can annoy users and where a server servicing a number of concurrent users is unlikely to choose a parallel plan because of the cpu workload profile.

So if you are going to employ parallelism it is most likely going to be in a data warehouse, decision support or reporting scenario, where there are a low number of high workload queries on a big fat box with plenty of memory.

So you have decided to employ parallelism – what should you set DOP to. Well one guideline that crops up internally is that if you have 8 processors then DOP = 4 is likely to be the optimal setting. However there is no mast and no flag pinned to it here – testing is the only way to be sure. Further to this I have not seen any other figures, but having said all that, my advice would be to never set this figure to anything more than half the number of processors I have. If I have less than 6 processors I would set DOP to 1 - which doesn't allow any parallelism to take place. I might make an exception if I had a database that was only going to support a single user process (some sort of data warehousing or reporting task) – here the exception would be to set DOP to 0 (default) which lets SQL decide.

Before I finish my ramble – beware that parallel index creation will depend on the figure you set for DOP. This means you may want to change it at times to suit what you’re doing and of course you can use the MAXDOP query hint to override the configuration option for those out of hours jobs.

Finally your query may go slower using parallelism due to a bug – make sure you have the latest service pack.

I'm trying to find some detail on Patch management / deployment on a 4 SQL Server node cluster?

Mat_Stephen — Wed, 02 Feb 2005 00:28:00 GMT

Stephen Daborn asked this question on my High Availability blog entry and I didn't get back before the post closed - sorry Stephen - been doing a lot of snow boarding recently.

Anyway its a good question and it deserves an answer.

The best thing to do is to read the readme that comes with any service pack, hotfix, patch or whatever. For instance if you look in the readme for service pack 3a @ http://support.microsoft.com/default.aspx?scid=/support/servicepacks/SQL/2000/SP3ReadMe.asp#_installing_on_a_failover_cluster you'll see all the steps you need. For additional information on installing SP3a on a failover cluster, see Knowledge Base article 811168 - this is also referenced in the readme.

Always read the readme as the instructions may change between patches.

It should be possible to keep the downtime to a minimum - most of the time you should find this no more than the time taken to failover to a patched node and not the time it takes to actually install the patch.

Oracle RAC, Grid, IBM, and Microsoft: A Corporate Altercation or Religious Divergence

Mat_Stephen — Mon, 10 Jan 2005 15:50:00 GMT

People often ask me, “When Microsoft is going to get into the type of clustering as defined by Oracle’s RAC (Real Application Clusters)?” Generally, I find these are people who believe the technical concepts behind RAC are the way of the future, so naturally conclude that Microsoft will want to follow.

I have a simple answer to the question, “Microsoft doesn’t intend to implement such a technology”. Naturally this is followed by a pause that demands I say why, so I oblige with the simplest explanation I can muster. “Microsoft believes RAC’s promise of scalability is flawed, flawed by bottle necks in the interconnects between cluster nodes.”

Beyond this I don’t go any further, because I don’t think I need to and because I fear I might step wide of the DOJ’s antitrust consent decree that Microsoft had to sign up to.

Taking the later point; compliance with the DOJ’s antitrust consent decree requires Microsoft does not abuse its monopoly position. This means, as a Microsoft employee, I am not allowed to say anything about our competitors that might be deemed combative or demeaning. For instance, I learn I can no longer refer to a company that runs Oracle as an ‘Oracle shop’. So, knowing that I can demonstrate a combative steak, I feel very reluctant about commenting on our competitors products, for fear of contravening the decree.

Taking the former point, I don’t need to tell you what I think about RAC because there’s so much out there that’s already been said. All I need to do is point to it. In the first instance, I’m going to point you to a document from IBM that describes what IBM thinks about RAC. Notice how ‘gloves off’ this document is; solid combative stuff from a company, I can only imagine, is not inhibited by court rulings – lucky them!

IBM’s document: A Technical Discussion of High Availability and Crash Recovery:
http://www-306.ibm.com/software/data/highlights/rac.pdf

In the second instance, I’m going to point you to a document from Microsoft that describes what Microsoft cautiously thinks about RAC.

Microsoft’s document: The Reality behind Oracle Real Application Clusters (RAC) Marketing Messages:
http://www.microsoft.com/sql/evaluation/compare/oracle-rac.asp

Spot the difference in style – amusing isn’t it?

First and foremost I’m a technologist, I am keen to see a balanced discussion and consequently I am all too happy to provide links to Oracle’s rebuttal to both these articles.

Oracle’s rebuttal to IBM’s paper:

http://www.oracle.com/technology/deploy/availability/pdf/DB2_HaCrash_Response.pdf

Oracle’s rebuttal to Microsoft’s paper:

http://www.oracle.com/technology/products/database/clustering/pdf/MSSQL_Rebuttal.pdf

Again in the interests of fair play, I should point out that Microsoft’s paper has been updated since Oracle wrote their rebuttal.

Go read and form your own opinion.

PS - Thanks to Chris's comment we have some more debate, in his words:

"I thought you might like to know that there has been another paper published on this subject around RAC, availability and suchlike .. very detailed and definitely a (and fair) read:" This white paper from SQL Server Architect James Hamilton discusses the problems that Oracle Real Application Cluster (RAC) is designed to solve, and presents alternative technologies and trends that often increase application availability and scalability in a more effective way.
http://www.microsoft.com/sql/evaluation/compare/oracle_rac2.asp"

SQL Server Health and History Tool (SQLH2)

Mat_Stephen — Fri, 07 Jan 2005 19:23:00 GMT

This tool has been available for a couple of months but I only found out about it yesterday; I was reading some stuff about SQL Server 2005 and found a reference to say this tool will be supported in SQL 2k5. Realising such downloads have sometimes escaped me I’ve decided to subscribe to Download Notifications

http://www.microsoft.com/downloads/render.aspx?displaylang=en&content=notifications

From the website SQLH2:

http://www.microsoft.com/downloads/details.aspx?FamilyID=eedd10d6-75f7-4763-86de-d2347b8b5f89&DisplayLang=en

The Microsoft SQL Server Health and History Tool (SQLH2) allows you to collect information from instances of SQL Server, store this information, and run reports against the data in order to determine how SQL Server is being used.

Overview

SQLH2 collects four main types of information:

1. Feature Usage – What services/features are installed, running and level of workload on the service.

2. Configuration Settings – Machine, OS and SQL configuration settings, SQL instance and database metadata.

3. Uptime of the SQL Server service

4. Performance Counters (optional) – Used to determine performance trends

TechNet Webcast: A Technical Overview of SQL 2005 High Availability Features (Part 2 of 2) - Level: 200

Mat_Stephen — Mon, 20 Dec 2004 21:15:00 GMT

Friday, January 14, 2005 - 9:30 - 10:30 AM Pacific Time

Matthew Hester, TechNet Presenter, Microsoft Corporation

There are a number of barriers to availability, and only some of these are addressable by DBMS technology. This session looks at some of the features of SQL Server 2005, the next release of the SQL Server product that can help address some of these barriers. This second part of the series addresses two specific areas: user or application error, and data access concurrency limitations. User or application error and concurrency limitations can cause problems for availability. Join this webcast to learn about the SQL Server 2005 features that can help minimize this type of situation.

http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032266465&Culture=en-US

TechNet Webcast: A Technical Overview of SQL 2005 High Availability Features (Part 1 of 2) - Level: 200

Mat_Stephen — Mon, 20 Dec 2004 21:14:00 GMT

Friday, January 07, 2005 - 9:30 - 10:30 AM Pacific Time

Bryan Von Axelson, TechNet Presenter, Microsoft Corporation

There are a number of barriers to availability, and only some of these are addressable by DBMS technology. This first part of a two-part series looks at some of the features of SQL Server 2005, the next release of the SQL Server product that can help address some of these barriers. This first part concentrates on database server failure or disaster. We will talk about database mirroring, what it is and how to configure it. We will also talk about replication as a method of creating a highly available database solution.

http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032266436&Culture=en-US

SQL Server Disaster Recovery (DR) and SANs, Remote Mirrors and Geo-Clusters

Mat_Stephen — Sat, 18 Dec 2004 04:26:00 GMT

After some positive comments arising from my High Availability post I’ve decided to Blog on DR earlier than I had intended. I hadn’t intended to blog on DR for a while because it’s a difficult subject and I was hoping it might get easier if I left it to stew a little longer.

Okay so let’s start by having a clear definition of Disaster Recovery. (Disaster Recovery sounds like an oxymoron; if you can recover from a disaster, was it really a disaster in the first place?) A disaster is the loss of a data centre. This could be a loss of power, flood, fire, earthquake or train/plane crash. This is to separate DR from High availability (HA). High availability refers to the recovery from a probable hardware failure or even more probable human moment of lunacy. (It’s a sad fact that non-failover clustered systems have a better up time than failover clustered systems, the biggest influence here being the ignorance of those people with access to the systems.)

DR is all about getting backups of your data offsite. The only real questions you have to ask are: how much data are you prepared to loose in the event of a disaster, how long can you afford to be ‘down’ and how much money do you have to spend? (Similar to the questions you ask when planning your HA).

So simple DR, used by many, is to simply take a backup every so often and take it off site. Obviously there’s plenty of scope for loosing data and being down for a good while, but hey a disaster’s a disaster isn’t it. Yeh – you all know about this easy stuff, what you all want to know about is the zero data loss, dual data centre strategy, don’t you.

There are two dual data centre strategies, Remote Mirrored data and Geo-Cluster (Stretch Cluster). Both of these strategies currently depend on a SAN infrastructure to replicate the data. The Mirrored data strategy relies on human intervention to invoke the failover while, as the name suggests, a Geo-Cluster automates the failover like a normal failover cluster.

Remote Mirroring:

Independent SQL Server at each site

SAN to SAN data copy

User databases (including active log) copied in real time

System Catalog not copied. User procedures required to keep them in sync

For setup & fail-over

Third party & user procedures for automation

Mount volumes at secondary site host & attach to SQL Server

The independent SQL Server at each site can be MSCS clusters

For detail refer http://www.microsoft.com/technet/prodtechnol/sql/2000/deploy/hasog05.mspx

Geo-Cluster

Third party extensions to MSCS cluster offering

Listed in Windows Server Catalog
http://www.microsoft.com/windows/catalog/server/default.aspx?xslt=categoryproduct&subid=22&pgn=b55095f4-71f3-4b26-98b1-05f3a9506d0d

Virtual SQL Server offered to clients

SAN to SAN data copy including system catalog

Requires synchronous copy

For setup & fail-over

Third party & user procedures to augment Microsoft provided procedures

For detail refer http://www.microsoft.com/technet/prodtechnol/sql/2000/deploy/hasog05.mspx

Majority Node Clusters and ‘split brains’

Big problem with geo-clusters has traditionally been with the concept of the ‘split brain’. Split brain occurs when the heart beat between the cluster nodes is broken by a failure in the network (man drives JCB through cable). If this occurs the failover node will try to come up whilst the primary is still functioning – ooops. Majority node clusters overcome this problem by maintaining an odd number of servers in the cluster such that if the heart beat network fails the data center with the majority of the nodes assumes responsibility for the database. Please see

http://www.microsoft.com/technet/prodtechnol/windowsserver2003/technologies/clustering/qsvclust.mspx

Kinda sounds simple so far? Well it all starts to get more complicated when you start to look at how the data is mirrored. There several ways to remotely copy (mirror) your data:

Synchronous

Data pages are written synchronously at either end of the mirrored set and SQL Server isn’t told of a successful write until the writes are completed at both ends. This is the only mode to use with Geo-Clusters.

Semi-synchronous

Some kind of weird scenario that doesn’t seem to curry much favour

Asynchronous

Acknowledgments for writes are given to SQL Server as soon as the data is written at the primary site. Data is written at the secondary site as and when.

Software-based Remote Copy

Self explanatory – Veritas etc

Adaptive, Remote, Peer-to-Peer

No inherent D/R capability on its own – lets not go here.

Multi-Hop, Three Data-Center

Designed for long distance DR. A synchronous copy to nearby storage and then an asynchronous copy to a distant storage. Means the source system doesn’t have to incur the latency to the distant storage.

Each method is usually called something different by the big four SAN Vendors (HP, Hitachi, EMC and IBM) and the implementation can vary too, so you can see this is now getting complicated.

But hang on which methods are supported by Microsoft? Remote mirroring is supported with the caveat that any data integrity issues related to data replication should be addressed by the customer to the hardware vendor and not to Microsoft.

SQL Server requires that ‘write ahead logging’ be honoured, there be no torn pages and pages arrive in the correct order. The mirroring configuration needs to support this, but don’t take my word as a definitive statement of the requirements – you must discuss this with the hardware vendor and make sure they’re prepared to support you. I suggest you properly test the configuration by pulling the plug on the primary site.

Key considerations and comparison for Remote Mirroring & Geo-Clustering

Both

Dependent on 3rd. party enhancements

Data Integrity requires that 3rd. party enhancements must ensure “Write Sequence Dependency” & protection from “Torn Pages”

Geo-Clustering

Offers automatic failure detection & fail-over automation

Requires less work – no extra logins & schema synchronization – single image due to virtual SQL Server

Offers an additional option for client redirection – use of virtual name

Requires down time for SQL Server 2000 maintenance

Remote Mirroring

Requires user developed procedures for failure detection & fail-over

Requires more work for logins & schema synchronization – independent SQL Servers

Offers an additional option for data copy – Asynchronous (with less adverse performance impact, but with potential data loss) besides synchronous

Offers potential continuous availability during maintenance with planned fail-over. Stretch Clustering along with Log Shipping offers similar functionality

Latency

This has to get a mention as a concern. Naturally you’ll be looking to do a commit at the remote site and apparently you can expect this to take 3ms or so I’ve heard. Then you’ve got to add the amount of time it takes to get to the remote sight and back at the speed of light. At 3*10^8 ms light would take 6ms seconds to go to a remote site 100km away. Shouldn’t be too bad, but I gather its SQL flushing the cache that puts the system under stress, merely because of the volume data that suddenly decides to flow down the pipe.

Examples:

Danfoss in Denmark are using data mirroring between two towns – this case study shows their SAP architecture:

http://www.microsoft-sap.com/docs/Danfoss.doc

Akzo Nobel, great Geo-Cluster example with two unmanned data centres using IBM kit shark storage (ESS). This cases study is amusing: we found it on the IBM website, I gave them a ring to find out more about it, they took it off their website within 24 hours. I guess they might have decided it was a better case study for Microsoft than it was for IBM, after I had come sniffing around.

http://www.microsoft-sap.com/docs/AKZO%20Nobel.pdf

For more information about EMC SRDF solutions, visit the following EMC Web site:

http://www.emc.com/products/networking/srdf.jsp

EMC: Symmetrix Remote Data Facility (SRDF) Connectivity Solutions
https://powerlink.emc.com/HighFreq/P_C852.2_srdf_connectivity_sol.wp_ldv.pdf

Hitachi: Disaster Recovery Issues and Solutions http://www.hds.com/products_services/storage_mgmt_software/business_continuity/

HP: StorageWorks Data Replication Manager http://h18004.www1.hp.com/products/sanworks/drm/index.html

I’d be very interested in your experiences with this type of technology, if you’d care to share them.

SQL Server High Availability – I have to talk about this so much I feel like a scratched record.

Mat_Stephen — Wed, 15 Dec 2004 16:12:00 GMT

Rather than write endless amounts here I’m just going to describe some useful pointers that you can reference. The first reference is to failover clusters, I mention this one first as it dives head on into that word ‘Cluster’. This word is a pain as it means different things to different people and if its not properly qualified it just causes confusion. Unfortunately in Microsoft we often just use the word ‘clustering’ – when we do we usually mean failover clustering.

However, in the Reporting Services documentation (enterprise deployments), I found my first instance of Microsoft referring to Network Load Balanced (NLB) Clusters with the word cluster, and boy has it caused confusion – Reporting Services is not failover cluster aware!

So apart from ‘failover’ and ‘NLB’ clusters there are also ‘scale out’ clusters, sometimes called (by Microsoft) ‘federated servers’. Oracle also has their Real Application Clusters (RAC) which can provide both ‘scale out’ and ‘failover’ capabilities.

Ref:- SQL Server 2000 Failover Clustering

Published: June 18, 2002 | Updated: October 14, 2004

Abstract: Users demand not only performance and scalability, but availability of the applications they use. Achieving high availability requires a whole solution, part of which includes the SQL Server 2000 Server failover clustering. This white paper will explain what failover clustering is: how it works, considerations to take into account when designing your solution, how to implement and administer your solution, and how to troubleshoot a failover cluster. Also included are some useful worksheets and checklists to assist you during the installation, as well as links to additional information where necessary

http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/failclus.mspx

The next reference is the most comprehensive discussion on High Availability (HA) I know – unfortunately it’s a little out of date but it is still valuable. It also deals with stretch clusters (sometimes referred to a geo clusters) but I consider this to fall more into the camp of Disaster Recovery (DR) – I’ll probably blog on DR another day.

This reference deals with both planning and deploying. Helpfully it discusses what high availability really means, because this can lead to mine field of arguments like ‘do you really need 5 9s? I find it useful to show customers as it put a number of technologies, like log shipping and NLB clusters into one document.

Ref: Microsoft SQL Server 2000 High Availability Series

Updated: November 19, 2002

The Patterns & Practices team has decided to archive this content to streamline content offerings and keep focused on the newest, most relevant content. However, we will continue to make this content available because it is still of interest to some users. We offer this content as-is, without warranty that it is still technically accurate as some of the material may be outdated. Note that the content may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

http://www.microsoft.com/technet/prodtechnol/sql/2000/deploy/sqlhalp.mspx

The following reference is actually a chapter pulled out of the SQL Server High Availability Series referenced above. I highlight it because it deals with backup and recovery strategies going all the way through to the two most advanced topics of ‘Split-mirror’ and ‘Copy-on-write’ snapshots.

Ref:-SQL Server 2000 High Availability Series: Recovering a Data Center by Using Database Backups

Recovering a Data Center with Database Backups

Updated: November 14, 2002

Despite the best-laid plans, you cannot prevent every disaster that might befall a data center. You must therefore ensure that you can recover the data center from backups after a catastrophic failure, such as a user error or a hardware-induced database corruption. This chapter discusses using database backups to ensure a complete recovery of the data center in the event of a disaster. It focuses on techniques to optimize the Microsoft SQL Server 2000 recovery process and reduce the total recovery time to meet data center availability goals. A basic understanding of SQL Server backup and recovery procedures is assumed. For more information about SQL Server backup and recovery procedures, see "Backing Up and Restoring Databases" on the MSDN Web site at http://msdn.microsoft.com/library/en-us/adminsql/ad_bkprst_9zcj.asp.

http://www.microsoft.com/technet/prodtechnol/sql/2000/deploy/harag03.mspx

If you feel the content of this blog misses something out please let me know.