• Performance Analysis–Excel (2010) automation : how to create graphs from a Perfmon-generated CSV file–Essay#1

     

    Hi all,

    Today I’m concentrating my efforts into generating graphs as painless as possible for many data collected on Exchange servers. I’ll start by generating graphs from the two following counters which give an idea of the load of the servers:

    - Active User Count

    - RPC Operations/sec

    The first step (Essay#1) is to generate as quickly as possible a nice graph to display the trend for these counters.

    The second step (Essay#2) will be to generate quickly also graphs from a bunch of files located on a folder.

    The third step (Essay#3) will be to quickly generate these graphs on a separate Excel spreadsheet, or even better on a Word document to start a report

    the fourth step (Essay #4) will then be to generate the most significant graphs to generate a report that will enable a good graphical performance analysis of many servers at a time, simply using Excel and Word.

     

    First you need to collect Perfmon data and configure the Perfmon data collector to dump statistics on .CSV files. You can also chose to convert existing BLG files to .CSV files using RELOG for example … or loading .BLG files onto a Perfmon console, and export the data on .CSV files … many ways to do this, but it’s not the purpose of this post.

     

    Second, we will then generate our graph with the above mentioned two counters (as a start of my live project).

    You have to open Excel 2010, show the “Developer” tab, and copy the following code :

    Sub Macro_Search_Active_User()


    'NAME the first column which is the timeline column
    Range("A:A").Name = "Time_Line"

    'FIND the column showing the number of active user count
        Cells.Find(What:="Active User Count", After:=ActiveCell, LookIn:=xlFormulas, _
            LookAt:=xlPart, SearchOrder:=xlByRows, SearchDirection:=xlNext, _
            MatchCase:=False, SearchFormat:=False).Select
    'NAME the column just found
    Range(Selection, Selection.End(xlDown)).Name = "Active_User_Count"

    'FIND the column showing the RPC activity
        Cells.Find(What:="RPC Operations/sec", After:=ActiveCell, LookIn:=xlFormulas, _
            LookAt:=xlPart, SearchOrder:=xlByRows, SearchDirection:=xlNext, _
            MatchCase:=False, SearchFormat:=False).Select
    'NAME the column just found
    Range(Selection, Selection.End(xlDown)).Name = "RPC_Ops_Per_Sec"

    'SELECT then all these 3 colums
    Range("Time_Line, Active_User_Count, RPC_Ops_Per_Sec").Select
    Range("A1").Activate

    'GENERATE the Excel graph
    ActiveSheet.Shapes.AddChart2(227, xlLine).Select
    ActiveChart.SetSourceData Source:=Range("Time_Line, Active_User_Count, RPC_Ops_Per_Sec")
    'NAME the graph to easily retrieve it on other code lines
    ActiveChart.Parent.Name = "ActiveUsersAndRPCOps"

    'This step is optional : DELETE the X axe label - "xlCategory" (or find an equivalent function to deactivate it) – because you may want to keep the X-axe time data
    ActiveChart.Axes(xlCategory).Select
    Selection.Delete

    ' Add a secondary Axe for one of the data series(no matter which one)
    ' .. Select series nb 1
    ActiveChart.FullSeriesCollection(1).Select
    ' .. add data as a secondart axe
    ActiveChart.FullSeriesCollection(1).AxisGroup = 2
    ' .. Color selected collection to in RGB(xxx,xxx,xxx) with xxx btw 0 and 255
        With Selection.Format.Line
            .Visible = msoTrue
            .ForeColor.RGB = RGB(255, 0, 0)
            .Transparency = 0
        End With
    ' .. Color the axis linked to the secondary collection into the same color
    ' ... Select the secondary axe first
    ActiveChart.Axes(xlValue, xlSecondary).Select
    ' ... then set the selection properties : line visible, color and not transparent ...
        With Selection.Format.Line
            .Visible = msoTrue
            .ForeColor.RGB = RGB(255, 0, 0)
            .Transparency = 0
        End With
        ActiveChart.Axes(xlValue, xlSecondary).Select
        With Selection.TickLabels.Font
            .Color = RGB(255, 0, 0)
        End With


    '. Same SELECTing and FORMATting the second data collection and axe
    ActiveChart.FullSeriesCollection(2).Select
        With Selection.Format.Line
            .Visible = msoTrue
            .ForeColor.RGB = RGB(0, 130, 0)
            .Transparency = 0
        End With
    ActiveChart.Axes(xlValue).Select
        With Selection.Format.Line
            .Visible = msoTrue
            .ForeColor.RGB = RGB(0, 130, 0)
            .Transparency = 0
        End With


    'Finally, delete the title
        ActiveChart.ChartTitle.Select
        Selection.Delete

    End Sub

     

    Execute the macro and you’ll instantly have the following type of graph:

    image

     

    Next I’ll try (and succeed hopefully) to generate the above graph type for many CSV Perfmon files stored in a directory.

  • Exchange 2010–Dumpster V2.0 questions and answers

     

    Ø Does the archive database has a separate dumpster with a separate retention ?

    Remember in my presentation, I talked about the dumpster v2.0 and the purposes of the “Deletions”, “Versions”, and “Purges” subfolders. Note that these will be used only if you activate the “Single Item Recovery” or the “Litigation Hold” feature on your mailboxes :

    clip_image001

    clip_image002

    The archive mailbox, which can be either in the same database as the “live” mailbox or in a separate, dedicated database that you can chose anytime (when creating the live mailbox or you can also move the archive after a while).

    So each archive mailbox is always associated with the “live” mailbox. And each archive mailbox has exactly the same Dumpster v2.0 structure than the “live” mailbox:

    clip_image003

    clip_image004

    Ø What happens when I move an email from my inbox to the archive and then delete it.

    If you move a mail from the “live” mailbox to the archive mailbox, the mail will reside in the archive mailbox. Then, if you delete it, it will go on the Dumpster V2.0 from the archive mailbox, on the “Deletions” sub-folder of the archive mailbox’ dumpster v2.0. If you chose to purge it using the “Recover deleted items” menu from Outlook, e.g :

    clip_image005

    … then the mails will go in the “Purge” items, still on the dumpster of the archive mailbox.

    Ø How long would the deleted copy be kept?

    - The time period by which the deleted data is maintained is based on the deleted item retention window. The default time period is 14 days in Exchange 2010 and is configurable per database or per mailbox. The following cmdlets let you alter this behavior:

    o For the mailbox database: Set-MailboxDatabase -DeletedItemRetention

    o For the mailbox: Set-Mailbox -RetainDeletedItemsFor

    - Also, to avoid denial of service attacks by placing lot of data into the dumpster (lots of spams for example, that the user deletes), you can configure the dumpster to have a quota. By default, the database-level limits are configured for the dumpster. You can use the RecoverableItemsWarningQuota and/or the RecoverableItemsQuota parameters from the mailbox settings to set the dumpster’s quota and the behavior of the dumpster in case the quota is reached.

    o Recoverable Items Warning Quota : lets the dumpster purge the oldest mails to permit new mails to be placed in the dumpster

    o Recoverable Items Quota : prevents any deletions on the “live” mailbox when the dumpster’s quota is reached.

    More information about all the answers given in the mail can be found on the following links :

    Understanding Recoverable Items

    http://technet.microsoft.com/en-us/library/ee364755(v=exchg.141).aspx

    Recoverable Items Folder

    http://blogs.technet.com/b/exchange/archive/2009/09/25/3408389.aspx

  • Exchange 2007–Event ID 9786 potentially followed by Event ID 623 possible causes

     

    Authors : Levon Kohoutek (for the question) and Nagesh Mahadev (for the answer)

    Question :

    Exchange 2007 SP3 RU6 - Server 2008 SP2 environment.

    Quite frequently we’re receiving ExchangeIS Event 9786’s potentially followed by ESE Event ID: 623 “The version store for this instance (x) has reached its maximum size of xxxMb.” errors. At this point we’re unsure as to what action is precipitating this (e.g. performing a search in Outlook). Also we may have high item counts, and we know we have to do a procdump of Store.

    The question is as follows:

    When viewing the 623 Event in Event Viewer, we can see that it cites a possible long-running transaction as the cause. It identifies this possible transaction by a SessionID, Session-context, and Session-context ThreadID.

    What the SessionID is referring to ? Is there a way to derive anything identifiable (user/mailbox/etc…) from that number and if so, how?

    Answer :

    The SessionID represents a particular connection between the store process and ESE. It has no bearing on the client at the other end of the request. It is only helpful when looking at dump of store process. While ESE.DLL is tightly coupled with the Store.exe, ESE is a database engine and serves clients. Similar to how SQL would handle multiple clients issuing requests against a server, except in this case, the clients are different threads within the store. Within a procdump, you *may* be able to locate a session (represented by a JSES object) and tie it back to a particular user depending upon context – again typically within a procdump only.

     

    About event 9786 :

    Event Type:        Warning
    Event Source:    MSExchangeIS
    Event Category:                Database Storage Engine
    Event ID:              9786
    Date:                     10/18/2007
    Time:                     1:34:04 PM
    User:                     N/A
    Computer:          BORO
    Description:
    The database engine has consumed 90% of the "b-trees" resource (60820 used out of a maximum of 67096) for storage group 'GLDSTORE1'.

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
    Data:
    0000: 46 43 42                  FCB    

    ==> This can indicate that we have a user that sends a huge attachment to a large amount of people for example, or there is a large message that is looping between two servers, anyways, there may be a leak in the ESE process due to an other database usage pattern.

    - To see what happens exactly, we have to get a dump and use the userdmp3.pl script on the Dump to see if there was a message leak and or if someone send a large message to a ton of people.