I've run into a rather odd issue. We've building out a new SCOM 2012 sp1 system and all was fine until we added around 400 new clients in a week. Were now at 715 total. The next week our Data warehouse size exploded. The strange part about it is the space is being used up by the Configuration Dataset. Its currently taking 85% of the space at nearly 300g. Any idea on what might be causing this or if we can tame it down?

Great article Kevin. A couple of things from me...

1) how many clients were you basing your sizes on?

2) Is this purely for servers? i.e. any network devices included? If not, how would you expect the network devices to effect the overall sizing required?

Kevin, thanks for the very quick reply. Sorry, I have probably taken this article out of context with my questions. Basically I have followed the MS sizing tool and worked out the overall total but its a very general vague total. What I am far more interested in is real world examples of DBs that are groomed and the sizes of them dependant upon whats being monitored, datasets used, number of devices you are collecting data from etc. You published a table which shows actual real world values of current sizes. I was curious as to how many agents have collected that data in your environment.

My second question was more a hopeful punt as to if you had experience of the kind of increase in data size that would occur if you were also monitoring x amounts of network devices.

Thanks Kevin! Got my DW at a reasonable level to continue my deployment of OpsMgr Clients. Also, the help you gave me @ the Hyper-V training was tremendous. Once I ran Remove-SCOMDisabledClassInstance all my print server client issues went away!

Hope to see you at other MS Events!

How soon after making changes to the data sets do the DB sizes actually shrink?  Is there a DW Maintenance job in SCOM that runs on a schedule that needs to happen first before you see any changes?

Hi Kevin.. is there any documentation which throws light on how data aggregation happens.. what algorithm exactly it follows in different aggregation type (raw, hourly, daily)?

Thanks Kevin. Great Article. The problem I am facing is the DW was about to get full. So I created a .ndf file in a different drive. But did not stop autogrowth in the original .mdf file. Now the drive space where the .mdf file lies has 0MB free space left. I have stopped the 'Auto Growth' now. Also set the retention policy for Dataset that was set to 400 to 90 days. My question is: now that I have reduced the dataset retention days, will DW delete the records which are older than 90 days? Regards, Praveen

Based upon the queries you have provided, I created a small DatawareHouse Query Report MP that lists some important values.
http://dynamicdatacenter.wordpress.com/2014/02/14/scom-data-warehouse-query-reports-management-pack/

Hi Kevin, my client want to store raw performance data for one year, no matter in DB ops or DB warehouse, so that there would have one year raw data showing on dashboard and report. Is that possible? Would there be any limitation and retrictions?

Kevin I must Thank you for such really nice blog. you covered pretty much everything in SQL. I have been doing some research on SCOM DW database grooming. I have configured the grooming setting for different datasets retentions. I am looking for a query to check when did grooming run last time and what was the status.

I run the query select * from maintenancesetting and I get 3 value. Instance optimization, instance grooming and Config optimization time. Could you please throw some lights what each of this type means and which one I should look for to check for grooming of datasets.

Thanks for your help again !!