Sharepoint Tips and Tricks : Sharepoint Reporting

SharePoint IIS Traffic Analysis

corybu — Tue, 26 Feb 2008 23:46:00 GMT

SharePoint IIS Traffic Analysis

Internally we have experienced situations where internal users have built tools or solutions which have negatively affected performance for our SharePoint users. This can be done in different ways such as Setting up multiple MOSS Farms to crawl our customer facing front ends at high performance setting or creating a custom application to execute against various MOSS web services without throttling the application in any way. From that need spawned a little tool we created with the help of log parser J The following is an example of how you could setup a scheduled task to notify you on number of Hits against your moss environments and how much bandwidth each user is consuming.

Please remember to never perform any of the following on production without first trying it out in a test environment, all items here are merely examples of how you ‘could’ perform such an action. J

Skills and Items required / assumed to perform this task.

· A SQL 2005 server with XP_CMDSHELL enabled

· Read Permissions for SQL Server Account Granted to the IIS Log directory of your SharePoint Servers

· Access to create and modify databases on your SQL Server

· Logparser 2.2 Installed on your SQL Server

· Good Understanding of SQL Server and TSQL

· Good Understanding of IIS

· Access to all customer Facing SharePoint Front Ends

· Good Understanding of SharePoint J

Step 1: Create a reporting database on your SQL Server, for the benefit of this blog entry we will call this database IISReports. Set that reporting database to autogrow for the data file and log file.

Step 2: Grant read permissions to all IIS log directories where your Web application Logs are stored on all servers you wish to report on.

Example: %WINDIR%\System32\Logfiles\ex080218.log For better performance internally we place our IIS Log files on a serperate drive

Step 3: Install Logparser 2.2 to a directory on your sql server(make note of this directory) For the purpose of this entry we will call this c:\temp\logparser.exe

Logparser 2.2 can be downloaded @ http://www.microsoft.com/technet/scriptcenter/tools/logparser/default.mspx

Step 4: Create Tables that we will use to pull in and manipulate the data

Click Here to open example Script

Step 5: execute the Create script for the stored procedure that will be used to pull the data.

Click Here to Open example Script – Make sure to read the /**notes**/ as they direct you to make changes to the script to cater to your environment.

Step 6: If everything has been followed and updated accordingly you can now run the following in SQL.. shortly there after you will receive an email similar to the following J

use [IISReports]

exec IISLogtracking

Example email received

Top 20 for BW used

Bandwidth Used(MB)	Total Hits	Logfilename	Url Host		Username	IP
20563	114502	\\servername\e$\w3svc\ex080226.log	sharepoint	Domain\username		127.0.0.1
5317	438216	\\servername\e$\w3svc\ex080226.log	sharepoint		Domain\username	127.0.0.1
4074	9055	\\servername\e$\w3svc\ex080226.log	sharepoint		Domain\username	127.0.0.1
2854	7720	\\servername\e$\w3svc\ex080226.log	sharepoint		Domain\username	127.0.0.1
2231	42288	\\servername\e$\w3svc\ex080226.log	my		Domain\username	127.0.0.1

Top 20 for Total Hits

Total Hits	Logfilename	Url Host	UserName	IP	Bandwidth Used (MB)
438216	\\servername\e$\w3svc\ex080226.log	sharepoint	Domain\username	127.0.0.1	5317
114502	\\servername\e$\w3svc\ex080226.log	sharepoint	Domain\username	127.0.0.1	20563
92346	\\servername\e$\w3svc\ex080226.log	sharepoint	Domain\username	127.0.0.1	532
56829	\\servername\e$\w3svc\ex080226.log	my	Domain\username	127.0.0.1	122
42288	\\servername\e$\w3svc\ex080226.log	my	Domain\username	127.0.0.1	2231

Cory

MSIT SharePoint Reporting Framework 2.0 Released!

corybu — Tue, 09 Oct 2007 20:41:00 GMT

I am pleased to announce I have finished version 2.0 of the MSIT SharePoint Reporting Framework. Versions 1.0 was used heavily internally for our service metrics and operational tracking. With version 2.0 many great improvements have come out of it, additionally with the release to the public :) I encourage everyone to head over and check it out. Thanks!!

http://www.codeplex.com/MSITSRF

This is one of many very great tools that will be available on the CD included in our upcoming book 'SharePoint for Architects and Engineers'

Sharepoint Reporting Methods

corybu — Fri, 01 Jun 2007 18:34:00 GMT

Reporting in SharePoint can be very helpful for all types of scenarios, storage growth projection, weekly service reviews, operational tracking for backups / restores and more. Below are a few scripts that I have wrote for MOSS 2007 / WSS3.0, These can be used to pull all kinds of good information that might not be available from the Front End(OM).

Please keep in mind that any direct database queries or modifications are not supported by any means. A suggestion might be to perform these against a test / backed up copy of your production data.

Content Script 1

The following script grabs all kinds of great data from your moss 2007 / WSS3.0 Content Database such as:

· SiteURL – Path for a site collection

· SiteAdmin – Site Owner / Admin for the site Collection

· RecycleBin – Amount of disk space RecycleBin is currently using

· BandwidthUsed – Amount of traffic the site has generated

· SiteSize – Amount of space the site takes up in the database

· SiteMaxQuota – Specific Max Quota the site has specified

· SiteID – Site GUID assigned for specific collection

· Content_DB – Database that site resides in

· ServerName – Server that Database Resides on

· LastContentChange – Last time a user modified any part of the site

· DaysSinceLastChange – Days Since last modification to site

Use <ContentDatabase>

select distinct a.fullurl as [SiteUrl],

b.tp_login as [SiteAdmin],

sum(cast(c.size as decimal))/1024/1024 as [recyclebin],

cast(d.bwused as decimal)/1024/1024 as [BandwidthUsed],

cast(d.diskused as decimal)/1024/1024 as [SiteSize],

cast(d.diskquota as decimal)/1024/1024 as [SiteMaxQuota],

d.id as [SiteID],(select db_name(dbid) from master..sysprocesses where spid=@@SPID) as [Content_DB],

(select @@servername) as [ServerName],

d.lastcontentchange as [LastContentChange],

(select datediff(day,d.lastcontentchange,current_timestamp)) as [DaysSinceLastChange]

from webs as a inner join

sites as d on a.siteid=d.id inner join

userinfo as b on a.siteid=b.tp_siteid left join

recyclebin as c on a.siteid=c.siteid where b.tp_siteadmin = '1' and a.parentwebid is null

group by a.fullurl, b.tp_login, d.diskused, d.id, d.bwused, d.diskquota, d.lastcontentchange

Order by a.fullurl

Configuration Script 1

The following script will create a high level snapshot of how your configuration database looks from a SQL standpoint. It would be run against your configuration database. It includes the following set of columns

· ConfigServer – Server your configuration database lives on

· Config_DB – Name of your configuration database

· Content_DB – Name of the content database that specific site lives in

· SiteID – Site GUID assigned for specific collection

· ServerName – Server where content database lives

· SiteURL – Full SiteUrl for each site collection in the Farm

Use <Configuration Database>

SELECT (Select @@servername) as [ConfigServer],

(select db_name(dbid) from master..sysprocesses where spid=@@SPID) AS [config_db], a.name as [content_db],c.id as [SiteID], b.name as [servername], 'http://'+d.name+c.path as [Siteurl]

from objects as a inner join

sitemap as c on a.id=c.databaseid inner join

objects as d on c.applicationid=d.id inner join

objects as F on a.parentid=f.id inner join

objects as b on f.parentid=b.id

where a.id in (select databaseid from sitecounts)

order by a.name

Content Script 2

The following script grabs lower level web information for your site collections including the following columns:

· ServerName - ServerName content database is located on

· Content_DB – Content Database web lives in

· SiteID – Site GUID assigned for specific collection

· WebID – Web GUID assigned for specific Web

· WebURL – URL for web

· ParentwebID – Relative GUID in which web is a child of ( Null = Site Collection )

· WebTemplate – Template Web was provisioned as

· Language – Language web was provisioned as

· ProductVersion – Version of site

· DocumentCount – Number of documents that live within web

select distinct(select @@servername) as [servername], (select db_name(dbid) from master..sysprocesses where spid=@@SPID) as [Content_db], a.siteid, a.id as [WebID], a.fullurl as [WebURL], a.parentwebid, a.webtemplate, a.language, a.productversion, count(b.id) as [DocumentCount]

from webs as a inner join

alldocs as b on a.siteid=b.siteid and a.id=b.webid

group by a.siteid, a.id, a.fullurl, a.parentwebid, a.webtemplate, a.language, a.productversion

order by a.fullurl

As you can see all 3 queries contain the site GUID, a content_db and a Servername… With that being said if you were to pull this data into a central repository you could then write SQL joins on those 3 columns and create all kinds of helpful views on your infrastructure. Additionally you can write SQL cursors around these scripts to pick up on table SCHEMA and loop through all of your content databases rather than manually executing on just one. The sky really is the limit. I will have an update to this with some example cursors and joins in the coming weeks.

Cory