Collective blog for SQLOS team
While Resource Governor spans multiple components of SQL Server Engine, it seems logical to discuss the feature in this blog as the resource management in general is so close to the scope of SQLOS.
Without further delay, let's get to point. As we have recently released a public CTP (Community Technology Preview) of SQL Server 2008 (aka SQL Server 2008 November CTP), which has Resource Governor functionality, and a number of presentations with Resource Governor demo on various conferences, I started getting a lot of questions about the demo itself which made me write this post.
We created this demo when Resource Governor was only a prototype to illustrate the concept, but probably its simplicity made it so attractive that it was included in a number of talks and demos by different people all over the world.
If you are looking for general concepts about Resource Governor, check out the Books Online which comes as a separate download or read it directly on MSDN Library.
In the demo we will create two workload groups which share a common resource pool and one workload group which has its own dedicated resource pool. Each group gets a CPU intensive workload. Using the Resource Governor you will be able to affect CPU distribution between these competing workloads. The effects of the resource management are observed using performance counters specific to Resource Governor.
On a newly installed server (i.e. no prior Resource Governor configuration) you may need to run these setup steps. I normally doing my demo on a dual core laptop and for the sake of simplicity I'm using a single CPU for SQL Server. To do so, I adjust CPU affinity mask as follows:
-- first enable advanced options in sp_configure
sp_configure 'show advanced', 1
-- use only 1 CPU on demo machine
sp_configure 'affinity mask', 1
Using 1 CPU for SQL Server on a dual proc machine has an interesting side-effect: we normalize "CPU usage %" counter to number of CPUs on the box and thus, the values will hover around 50% as maximum and not 100% as you might expect. I will illustrate this below.
Of course you can use both CPUs but this complicates the demo. This is a subject for whole another post - what happens when you have multiple CPUs.
In addition to that for demo purposes I will set min/max server memory to a fixed value, since it will improve predictability of the demo on the laptop.
-- 512 Mb is suitable for laptop demo
sp_configure 'min server', 512
sp_configure 'max server', 512
Now we will be setting up the following hierarchy of workload groups and resource pools:
Each corresponding workload group contains queries of the corresponding class or department (i.e. Marketing, Adhoc, and VP). Note that, Marketing and Adhoc queries share the same resource pool, while workload group VP has its own similarly named pool. The reason of such separation will become clear as we see how we adjust Resource Governor controls.
You will start building the above configuration in a bottom up manner (i.e. starting from pools and going up)
To do so, we execute the following T-SQL:
-- create user pools
-- note that we are using all default parameters
CREATE RESOURCE POOL PoolMarketingAdhoc
CREATE RESOURCE POOL PoolVP
-- create user groups
-- also note that all groups created with default parameters
-- only pointing to the corresponding pools (and not 'default')
CREATE WORKLOAD GROUP GroupMarketing
CREATE WORKLOAD GROUP GroupAdhoc
CREATE WORKLOAD GROUP GroupVP
Now, what you have just done is created hierarchy of the groups and pools, however, how does the server know about which query goes where? This is where classification comes in. The above picture becomes:
There is a couple of things:
To implement the above 2 steps we will run the following:
-- classifier function should be created in master database
-- switch to master unless you are there already
-- create logins to separate users into different groups
-- note that we disabled strong password checking for demo purposes
-- but this is against any best practice
CREATE LOGIN UserMarketing WITH PASSWORD = 'UserMarketingPwd', CHECK_POLICY = OFF
CREATE LOGIN UserAdhoc WITH PASSWORD = 'UserAdhocPwd', CHECK_POLICY = OFF
CREATE LOGIN UserVP WITH PASSWORD = 'UserVPPwd', CHECK_POLICY = OFF
-- now create the classifier function
IF OBJECT_ID('DBO.CLASSIFIER_V1','FN') IS NOT NULL
DROP FUNCTION DBO.CLASSIFIER_V1
-- note that this is just a regular function
CREATE FUNCTION CLASSIFIER_V1 ()
RETURNS SYSNAME WITH SCHEMABINDING
DECLARE @val varchar(32)
SET @val = 'default';
if 'UserVP' = SUSER_SNAME()
SET @val = 'GroupVP';
else if 'UserMarketing' = SUSER_SNAME()
SET @val = 'GroupMarketing';
else if 'UserAdhoc' = SUSER_SNAME()
SET @val = 'GroupAdhoc';
After all this work, can we start workloads and see what happens? The answer, as you have guessed by the question is - no. What's left? Again, a couple of steps:
First step is done by
ALTER RESOURCE GOVERNOR
WITH (CLASSIFIER_FUNCTION = dbo.CLASSIFIER_V1)
For the second step, let's compare output of catalog views with in-memory information (note difference in names of catalog views and dynamic management views (DMVs) which are prefixed with dm_:
-- metadata information
SELECT * FROM sys.resource_governor_workload_groups
SELECT * FROM sys.resource_governor_resource_pools
SELECT * FROM sys.resource_governor_configuration
-- in-memory information
SELECT * FROM sys.dm_resource_governor_workload_groups
SELECT * FROM sys.dm_resource_governor_resource_pools
SELECT * FROM sys.dm_resource_governor_configuration
Now transfer changes from metadata to memory by running the following statement. Also, do not confuse it with already existing RECONFIGURE command:
-- make the changes effective
ALTER RESOURCE GOVERNOR RECONFIGURE
And rerun the above query on metadata and DMVs and you should see that new groups, pools and classifier function ID are present in corresponding DMVs.
The easiest way to simulate a CPU intensive workload is to run the following in a loop:
set nocount on
declare @i int
declare @s varchar(100)
set @i = 100000000
while @i > 0
select @s = @@version;
set @i = @i - 1;
Also, instead of running this query from the Management Studio, consider saving it in a file and running from a command prompt by using a script similar to the below. Note that we are using 3 different user names to connect to the server.
echo "Press any key to start Marketing workload"
pausestart sqlcmd -S <your_server_name> -U UserMarketing -P UserMarketingPwd -i "CPU intensive loop.sql"
echo "Press any key to start VP workload"
start sqlcmd -S <your_server_name> -U UserVP -P UserVPPwd -i "CPU intensive loop.sql"
echo "Press any key to start Adhoc workload"
start sqlcmd -S <your_server_name>" -U UserAdhoc -P UserAdhocPwd -i "CPU intensive loop.sql"
To observe the effects of the load, add the following performance counters in the perfmon:
Before you start the next workload, observe the counters for pools and groups for a number of seconds, you should see approximately the following:
For groups (click on the image to open in a new window):
Few things to note:
Now we came to the point where we want to apply action to change the above picture. Specifically, we want our VP workload to proceed faster and thus, limit CPU usage by Marketing and Adhoc workloads to 50% of the CPU.
To do this, we alter the PoolMarketingAdhoc using the following syntax (remember, we created the pool using all default parameters):
-- adjust PoolMarketingAdhoc to not consume more than 50% of CPU
ALTER RESOURCE POOL PoolMarketingAdhoc
WITH (MAX_CPU_PERCENT = 50)
Remember to make changes effective:
Now, let's look at the counters:
What you will see is what happened at point D: Pool usage by PoolMarketingAdhoc went back to half of the CPU while PoolVP took the remaining part of it. Note that, on group side GroupVP usage went up to use half of the CPU (it has a single workload) while groups Marketing and Adhoc divide another half equally (12.5% each).
Further step is to alter IMPORTANCE parameters of the Marketing and Adhoc groups. IMPORTANCE affects CPU distribution when two groups share the same pool. Value of importance has a numeric meaning for CPU bandwidth distribution. Ratio of Low:Medium:High IMPORTANCE is equal to 1:3:9.
To illustrate this, let's run the following:
-- alter importance of adhoc group
ALTER WORKLOAD GROUP GroupAdhoc
WITH (IMPORTANCE = Low)
-- alter importance of marketing group
ALTER WORKLOAD GROUP GroupMarketing
WITH (IMPORTANCE = High)
When changes become effective corresponds to the point E on the above graphs. Note that distribution between pools does not change, amount of CPU available to GroupVP also does not change (there are no other groups in this pool), but GroupMarketing takes about 10 times of the bandwidth available to GroupAdhoc (remember the ratio of Low:High = 1:9, but together they stay within PoolMarketingAdhoc limits.
That's enough for the first post. In the upcoming post I will try to go over the cases when the demo does not seem to work or shows "unusual" behavior and possible explanations of it.
If you are looking for some demos to play and learn about Resource Governor, you should have a look to
Борис! Большое тебе спасибо! Ждём от тебя всего того, что мы обсуждали на сминаре в Москве.
Это были темы сочетания Soft NUMA и Resource Governor, слчетание виртуализации с Soft NUMA и Resource Governor... да и вообще, интерсно узнать "кухню" виртуализации, т.е. как вилияеи на работу планировщиков виртуализация процессоров, особенности рабты оптимизатора в виртуальной среде, какие возможности открывает IO Affinity и т.п.
Кстати, может быть тебе завести ещё и русскую верси этого блога? ... Так уже делали Людмила Фокина, Алексей Халяко, Моша Пасумански и др. ;)
I need a complete example described step by step through the creation
and use of
SQLServer 2008 Resource Governor to MAXDOP = 1 in one or more database.
Example of the database name: XYZ.
Has something to send as described above?
José Júlio Duarte
Can you tell me how to make the script? Is it a batch file, a vbs? Do I need I need to run it in the same folder that sqlcmd exists? Just copy and pasting that into a bat file in my documents does not work. I put the sql file in the same location.