Clint Huffman's Windows Troubleshooting in the Field Blog

PAL processing, processors, and threads

2013-01-14T19:32:52Z

I commonly get questioned on what response should be given to the NumberOfProcessors question variable in the PAL tool, so I thought I might try to explain it a bit…

PAL is designed to be a stand-alone tool where the analysis of a performance counter log can be analyzed on a workstation where an administrator can analyze counter logs of other computers with no connectivity. For example, customers regularly send me counter logs (*.blg) files to me and I analyze them from my home office or when I am at a hotel. I use PAL on my laptop to do the analysis.

When you reach the Questions tab in the PAL Wizard, these questions are in regards to the computer(s) in which the performance counter log was captured. The number of processors refers to the number of logical processors that would be seen in Task Manager of that computer. With this in mind, I will change the wording of the question to be more specific.

At the end of the PAL Wizard on the Execute tab, it asks how many threads to use during analysis. This is asking how many threads can the PAL tool use on the local computer (workstation) to use for analysis. PAL is very processor intensive, so I recommend 1 minus the number of processors of the local workstation. For example, if you have a 4 processor workstation, then use no more than 3 threads. Otherwise, your workstation might become sluggish and hot due to the long-term, high processor usage.

PAL collector script – PalCollector.ps1

2013-01-09T07:45:32Z

One of the top questions I get with the PAL tool is what data to collect. As many of you know, the PAL tool only analyze existing counter logs. It is up to you to create the counter log. To help with this problem, I created a PowerShell script called PalCollector.ps1. This script will query your local computer’s performance counters and will find a PAL threshold file that best matches your computer, then it creates a data collector set called, “PalCollector” using the counter paths from the best matching threshold file. It creates the data collector as a 200 MB binary circular log which means that it will continuously collect data every 15 seconds and will never get over 200 MB in size – roughly 24 hours of data. Once you are done collecting data, analyze the counter log (*.blg) using the PAL tool as you normally would.

Instructions

Download PalCollector.zip from http://sdrv.ms/10dZBNb.
Extract the zip file to a folder such as your Desktop or somewhere under your “My Documents” folder.
In Windows Explorer, find PalCollector.ps1, go to Properties of the file and click the Unblock button. This will allow the script to run on your system.
With administrator rights (required) open an elevated PowerShell session.
If your execution policy is not set to RemoteSigned or Unrestricted, then do so now by running:
Get-ExecutionPolicy
Set-ExecutionPolicy RemoteSigned
Change directory to the location where you extracted the zip file.
Run “.\PalCollector.ps1”

Please let me know your thoughts on how it is working for you and any recommendations on improvement.

How to create a threshold file for the PAL tool

2013-01-08T08:49:59Z

The Performance Analysis of Logs (PAL) tool is an open source project at http://pal.codeplex.com that analyzes performance counter logs. It has thresholds for performance counters written by experts in their respective areas spanning nearly all of the major Microsoft Server products, some Citrix XenApp,, VMware. and others. The number of products covered is limited to only those willing to create a threshold file to define it. It’s my hope that this article will help enable you to create your own threshold file and together we can create prescriptive guidance on all of the performance counters.

First, open the PAL Wizard as you normally would to analyze a counter log and navigate to the Threshold File tab and click “Edit…”. It doesn’t matter which threshold file is selected at this time. The PAL Editor will show.

Figure 1: The Threshold File tab in the PAL Wizard.

In the PAL Editor, go to the upper left and click File, New.

Figure 2: The PAL editor after clearing it with File, New.

This clears the editor, but hasn’t created a new threshold file just yet. We will get to that. For now, lets create a new analysis by clicking the New button at the lower left of the editor. This will show the counter New Analysis dialog box. An analysis is the primary container for one or more data source counters that you want to analyze, the thresholds that are applied to the “counter to analyze”, and the charts generated for the data source counters.

Figure 3: The New Analysis and Add Counters dialog boxes showing the buttons to click to add new counters to a new analysis.

Click the Browse Counters button and this will open the Add Counters dialog box. This interface allows you to connect to a computer either local or remotely that has the performance counter that you want to add. It’s important to the PAL tool connects to computer that has the counter to ensure that the counter path is exact. Click Close or OK to all of the open dialog boxes to continue.

Once at the main editor again, notice that many of the fields have been populated with data from the selected counter. This is a good opportunity to update the description of the analysis to tell the end users the purpose of this analysis, what is being checked and why, and what do to if the thresholds are exceeded. The description field supports the use of HTML tags and the rendered HTML can be previewed on the right pane. The text in the description will always show in the PAL report with this analysis. Once finished with editing the description, click the Update Analysis button to set the change and click File, Save to permanently save the changes to the threshold file.

In this case, I am adding all of the instances of the \Power Meter(*)\Power performance counter. Once I click OK, you are returned to the main PAL editor and should now see the performance counter that you added on the left pane.

Excluding counter instances

In some cases, it is necessary to exclude specific instances of a performance counter. For example, the _Total counter instance of the LogicalDisk counter object is commonly excluded because the _Total instance sums all of the logical disk counter values together which is typically not helpful when trying to analyze each disk. To exclude a counter instance, select the data source counter to edit, then click the Edit button.

Figure 4: The Edit DataSource Counter form in the PAL editor.

The Edit DataSource Counter form will show. Click the Add button and specify the counter instance to exclude. Repeat as many times as necessary to define all of the counter instances to exclue. In this case, I excluded the _Total instance. Click OK when finished.

Figure 5: The PAL editor with a new analysis.

Now is the time to save the work as a new threshold file. At the top left, click File, Save As. This opens the Save As dialog box that will allow you to save the work you’ve done so far to an XML file that is the new threshold file. Navigate to a folder that you have write permissions to such as your Desktop or a location under your My Document folder. In the File Name field, type in a file name that you want the threshold file to have and click Save. Once saved, move the file to the folder where PAL is installed. This typically requires elevated privileges. The default installation folder for PAL is C:\Program Files\PAL\PAL. Saving files directly to this folder is typically prevented by Windows unless you use elevated privileges. In this case, I saved the threshold file as PowerStates.xml and I saved it to the PAL installation folder.

Next, let’s give the threshold file a name and other information. Click the Edit Threshold File Properties… button. In the Threshold File Properties dialog, change the Title field to a more presentable name. In this case, I named mine, “Windows power states”.

Title: This is the name of your threshold file that will be shown. This must be a unique name relative to the other threshold file names. A title with the name of the manufacturer, product, and product version that the threshold file focuses on such as Microsoft IIS8 is recommended.

Version: This starts off as 1.0. Increment the major and/or minor version numbers when significant changes are made to your threshold file.

Content owner(s): You are the content owner… the one who’s reputation is behind this threshold file. Put your name and names of other contributors in this field.

Feedback email addresses: Put your email address or addresses separated by semicolons (;) that you would like users to contact you for support or questions.

Threshold file description: This is a sentence or two describing purpose of the threshold file.

Threshold file inheritance order: There is no need to recreate all of the thresholds of the other threshold files. Simply inherit from all of the threshold files that you want. I generally recommend inheriting from at least the Quick System Overview threshold file because it contains all of the threshold for the core operating system. For example, Microsoft BizTalk Server depends on SQL Server and IIS, so it inherits from the SQL Server and IIS threshold files. When a change is made to any of the inherited threshold files, your threshold file automatically gets those changes allowing your threshold file to evolve with the other threshold files.

The order that the threshold files are listed is used to resolve conflicts where the two or more threshold files have an analysis with the same name or same identifier (in the XML only – not exposed by the editor). Your threshold file is always applied last meaning it will always win conflicts. This means that if you don’t agree with the logical disk latency thresholds defined in the Quick System Overview, then all you have to do is defined an analysis with the same name and create your own thresholds which will override the inheritance. This is what the Microsoft Exchange Server threshold files do because they have more restrictive threshold for disk latency than what the Quick System Overview threshold has defined. With that said, the Exchange Server threshold file still gets all of the other thresholds defined in the Quick System Overview threshold file.

To add a threshold file to inherit from, click the Add button, browse to the PAL installation folder, and select one of the threshold files listed there, then click Open. You should see the threshold file name listed in the inheritance order. If necessary use the Move up and Move down buttons to change the order in which the threshold files are applied. Remember, your threshold file will be applied last allowing it to win any conflicts in analysis names.

Figure 6: The Threshold File Properties dialog box.

In my case, I added the QuickSystemOverview.xml file. Click OK when finished. For good measure, save your work so far by clicking File, Save in the main PAL editor.

At this point, the threshold file is usable and you should find it in the drop down menu on the PAL Wizard, but when no thresholds are defined, the counter will only show a chart and statistics only.

Next, let’s add question variables to the threshold files.

Question variables (optional)

Question variables allow you to ask the end user more information about the computer system(s) where the counter log was captured that cannot be retrieved by any other means. The answer provided by the user can by used by thresholds in your threshold file for a more thorough analysis. For example, you could ask the user what phase of the moon it was when the counter log was captured.

To add a question variable, click the Edit Questions button on the main PAL editor. Edit Questions will show. Click Add and “-Needs Updated-“ will show. Click “-Needs Updated-“ and replace the default data on the right as appropriate.

Question Variable Name: This is the variable name that will be used in the threshold code. Ensure that the name meets the variable naming requirements of PowerShell such as no spaces in the name.

Question: This is the question that is presented to the end user.

DataType: Choose Boolean or String. Boolean provides a True or False value type for the variable. String provides a text value type for the variable.

Default Answer: If no answer is provided by the end user, then this is the default response to the question.

Figure 7: The Edit Questions form

Click Update, then OK when finished and do another File, Save for good measure.

Adding a threshold (optional)

To add a new threshold to an analysis, click the Add button in the Thresholds group. This will open the Add Threshold Properties.

Name: This is the name of the threshold and is the text that shows with all alerts generated by this threshold. Make this a concise description.

Condition: Choose Warning or Critical. Use Warning to alert the user that a critical threshold is near, there might be an ambiguous condition that could lead to a larger problem, or when the threshold is experimental. Use Critical when it is clear that there is a problem or a condition that the user must be made aware of. Notice that when the condition is changed, the priority changes. This is because Critical conditions are more important than Warning conditions.

Color: This will always be yellow for Warning or red for Critical conditions.

Priority: You can add as many thresholds as you want to an analysis, but if more than one threshold is broken, then only one threshold will win to produce an alert. When multiple thresholds in an analysis are broken, the threshold with the highest priority will win – meaning the name, condition, and color of the “winning” threshold will be used in the alert generated from the broken threshold(s).

Variables: This is a list of variables and descriptions of those variables that can be used in the PowerShell Threshold Code. These could be question variables such as the $PhaseOfTheMoon variable that I created earlier.

PowerShell Threshold Code: This is where nearly any PowerShell code can be added towards analyzing the “counter to analyze” data source counter. It can be as simple or as advanced as you prefer. By default, PAL provides a “ready to use” threshold by automatically adding the appropriate arguments to the StaticThreshold function. It defaults with a threshold of greater than 10. All of the lines that precede with “#//” are comments and can be removed. They are there only to provide as help.

Note: Please keep in mind that the threshold code can be much more advanced than the standard StaticThreshold. For examples of advanced threshold code, explore the Process Private Bytes analysis of the System Overview threshold file.

StaticThreshold: This is a function inside of PAL.ps1 that will automatically compare the operator and threshold arguments to the values of the “counter to analyze” counter and will generate an alert each time the threshold is exceeded.

CollectionOfCounterInstances: This value must be the variable that contains all of the instances of the “counter to analyze” counter which is automatically named and provided.

Operator: This is a string type that accepts less than ‘lt’, less than or equal to ‘le’, greater than ‘gt’, or greater than or equal to ‘ge’.

Threshold: This must be an integer or double type that will be compared against the values of the “counter to analyze” counter.

Figure 8: The Add Threshold Properties form.

Click OK when finished and the threshold should appear in the Thresholds section of the main PAL editor. Click the Update Analysis button on the lower right of the PAL editor, then click File and Save to save your changes.

At this point, the threshold file can be used and if any of the thresholds are exceeded, then they will throw an alert with the corresponding conditions of the threshold.

Adding visual thresholds to the chart (optional)

If you are adding thresholds to an analysis, then it is highly recommended to add corresponding visual thresholds into the chart or charts generated for the analysis. On the main PAL editor form, click the Edit Chart button on the upper right.

Figure 9: The main PAL editor form highlighting the Edit Chart button.

This will show a new form that allows you to create a Warning threshold and/or a Critical threshold into the chart or charts generated for this analysis. These thresholds will show as yellow and red gradients with the ranges specified in this form. Like the counter thresholds, by default, the visual chrart thresholds of Warning and Critical are automatically generated and usable. You can enable one or both of them by clicking the Enabled combo box next to the respective threshold.

Figure 10: The Edit Chart form in the PAL editor.

StaticChartThreshold: This is a function in PAL.ps1 that can be called to create the visual thresholds seen as gradients on the analysis charts.

CollectionOfCounterInstances: This argument requires the variable that contains all of the counter instances of the “counter to analyze”. It is recommended to only use the variable already provided.

MinThreshold: This is the lowest value of the respective Warning and/or Critical chart threshold.

MaxThreshold: This is the highest value of the respective Warning and/or Critical chart threshold. If the maximum value of Critical or Warning (if Warning is the only threshold) is 30 and if none of the counter values reach 30, then the chart will automatically expand to 35 which makes the placement of the gradient seem off. Therefore, consider using a value ending in .999 such as 29.999 to represent 30.

IsOperatorGreaterThan: This is a Boolean (true|false) argument. If True, then it is assumed that the greater the counter value, the worse the condition leading from yellow [Warning] to red [Critical] as the value increases. If False, then the effect is inverted – meaning lower values are considered a worse condition leading from yellow to red in a downward view.

UseMaxValue: This is a Boolean (true|false) argument. If True, then if this chart threshold is exceeded by the counter value, then this chart threshold is increased automatically to match the counter value. If False, then the chart threshold values will not change on the chart. When using both Warning and Critical chart thresholds, it is recommended to set the Warning chart threshold to False and set the Critical chart threshold to True allowing the Warning threshold to stay in place and the Critical threshold to continue to increase matching the counter value if it had exceeded the MaxThreshold value for Critical.

Once finished, click OK to return to the main PAL editor, click Update Analysis on the lower right, then File, Save to permanently save your changes to the threshold file.

At this point, the analysis should be relatively complete and should be tested. When working with many analyses within a threshold file, consider using the Enabled combo box near the top of the analysis to enable or disable the analysis. This is helpful when needing to test some, but not all of the analyses in your threshold file.

Generated counters (optional)

The PAL tool has the unique ability to create fake counters that don’t normally exist in a performance counter log, but can be analyzed, charted, and processed with thresholds exactly like normal performance counters. Unfortunately, the PAL editor does not provide a way to create a generated performance counter. It must be created by manually editing the XML code of the threshold file using a text or XML editor.

The Network Interface % Network Utilization analysis is an example of a generated counter. In this example, the values of the counters \Network Interface(*)\Bytes Total/sec and \Network Interface(*)\Current Bandwidth are put through a formula that produces a percentage value of the amount of network bandwidth used based on the amount of data passing through compared to the current bandwidth of the network interface. In the PAL report, the % Network Utilization performance counter appears as if it was a real performance counter.

The technique of creating generated counters based on other counters was also used in the SQL Server threshold file to compare full scans/sec to batch requests/sec in a ratio. Once the generated ratio counter was created, it is easy to add thresholds and chart thresholds for it.

Examine the XML code of the analyses mentioned above as examples of creating your own generated performance counters.

Conclusion

I know this guide on creating your own threshold files for PAL is very much over due, but I hope you find it useful. If you create a threshold file, then I am happy to include it in the next release of the PAL tool. Just ping me on Twitter @ClintH or post your questions on the PAL forums at http://pal.codeplex.com.

Page Frame Number (PFN) database

2013-01-07T07:33:38Z

I just finished writing an article on the public, wiki, PFE PerfGuide on the subject of the Page Frame Number database in Microsoft Windows and Windows Server. This little known database is used by the operating system to keep track of the physical memory of the system. Please check it out and update it if necessary.

http://social.technet.microsoft.com/wiki/contents/articles/15259.page-frame-number-pfn-database.aspx

Memory combining in Windows 8 and Windows Server 2012

2012-11-29T22:23:35Z

I’ve spent that last few weeks studying the memory architecture of Windows 8 and Windows Server 2012.

Windows and Windows Server has always had sharable memory where portions of DLLs and EXEs will have a single copy in physical memory (synonymous with RAM) and all of the applications that need them will simply reference the page already in physical memory. This is still counted in their working sets, but overall the operating system saves on physical memory usage.

One of the interesting features of Windows 8 and Windows Server 2012 is how process private page-able memory is periodically combined further saving on physical memory usage. This was mentioned in Bill Karagounis’s blog, Reducing runtime memory in Windows 8. The savings on physical memory usage could be dramatic, but possibly have some overhead from the system needing to search for duplicate pages of memory. I can only speculate at best right now.

An important distinction is that Windows 8 has memory [page] combining enabled by default, but Windows Server 2012 does not.

To check if your computer has page combining enabled or not, open an elevated Powershell session and type the following command:

Get-MMAgent

You should see output similar to this:

To enable page combining on Windows Server 2012, run the following command:

Enable-MMAgent –PageCombining

Get-MMAgent

You should see output similar to this:

I did not get a prompt to reboot, so I assume this is working now.

If you enable this feature, then please let me know what kind of impact it has on your solution. I am particularly interested in the value of the counter \Memory\Available MBytes before and after testing or real world load. Keep in mind that I suspect that the \Process(*)\Working Set sizes will remain the same simply because the physical page in RAM will be counted in the working sets normally, but physically have only one real page in RAM.

Can a process be limited on how much physical memory it uses?

2012-10-11T16:43:00Z

I've been asked a lot of great questions lately and thought I'd post some of them.

As you might know, I am one of the instructors of the popular workshop "Vital Signs" which teaches students Windows architecture and how to identify performance bottlenecks. One of the instructors recently had a student who asked if a process (application) can be limited on the amount of physical memory (RAM) that it can use. The answer is it largely depends on the overall physical memory usage, but it can be limited.

The amount of physical memory that a process uses is called Working Set. The operating system's memory manager (referring to Microsoft Windows and Microsoft Windows Server) controls that amount of physical memory that a process uses by expanding and trimming the working set size of the process. The process itself has little control over this, but this model allows the operating system to manage physical resources more efficiently by only allowing memory that is most actively touched (read or written to) in physical memory.

If you use a tool like Process Explorer (http://live.sysinternals.com/procexp.exe), you can have it show the field Max Working Set. This is the amount of physical memory that the process believes it will need and it provides a suggestion to the operating system as to how large or small the working set of the process should be. For the most part though, maximum working set is largely ignored. The reason for this is that if the operating system has plenty of available physical memory, then it will allows frequently accessed pages of a working set to stay in physical memory simply because it is more efficient to keep it there.

According to the Sysinternals Administrators Reference book, here are the definitions of the working set fields:

Minimum Working Set. The amount of physical memory reserved for the process; the operating system guarantees that the process’ working set can always be assigned at least this amount. The process can also lock pages in the working set up to that amount minus eight pages. This minimum does not guarantee that the process’ working set will always be at least that large, unless a hard limit has been set by a resource management application.
Maximum Working Set. Indicates the maximum amount of working set assigned to the process. However, this number is ignored by Windows unless a hard limit has been configured for the process by a resource management application.
Working Set Size. The amount of physical memory assigned to the process by the memory manager.

Ref: Russinovich, Mark E.; Aaron Margosis (2011-06-29). Windows® Sysinternals Administrator’s Reference (p. 59).

Also, per David Solomon (one of the authors of the Windows Internals series of book), Process Explorer does not show if the process has a hard or soft max or min [working set] set. Also, he suggests using the Windows API, SetProcessWorkingSetSizeEx, to set a hard working set size.

With that said, there is a tool called the Windows System Resource Manager which can limit the amount of working set that a process uses. This tool is installable (not installed by default) through the Add Features console on Windows Server 2008 R2.

For more information on this subject, I recommend watching Mark Russinovich's, "Mysteries of Memory Management Revealed".

My IIS7 PowerShell Scripts

2011-12-04T10:08:02Z

I regularly go onsite with enterprise customers of Microsoft and do Microsoft Internet Information Services (IIS) health checks. Recently, I have been rewriting many of my VBScripts into PowerShell scripts to help make the health check easier. I will be writing more as I go. In the meantime, I hope that you will find these scripts useful.

General features

Alternate credentials: They permit the use of different credentials against remote IIS7 servers.
Encrypted data transfers: Use WMI remote-ing and encryption: Most of the scripts use remote WMI calls which use DCOM. I set them to always use encryption on these connections to protect sensitive data collection over the wire. Warning: Log-EphemeralPortStats.ps1 uses PsExec for remote-ing and sends the password over the network in the clear only when changing credentials. Using a domain account with admin rights will not send the credentials in the clear.
Useful output: Many of the scripts output to a comma separated value (csv) file or an XML file. This allows easy post analysis using Microsoft Excel or Microsoft Internet Explorer to view the collected data.

Please understand that I am publishing these scripts to the open community so that you might be able to self-help yourself and have an even better experience with Microsoft products. These script are provided “as-is” as sample code and are not supported. For more information on Microsoft Internet Information Services and how to automate it, go to http://learn.iis.net.

Get-NtfsPermissionsOfIisContentToCsv.ps1

This script gets the discretionary access control lists (DACLs) from the physical paths of IIS7 web sites and virtual directories and writes them to a comma separated file (CSV) for post-analysis such as auto-filter in Microsoft Excel. This script requires remote WMI connectivity to all of the servers specified. WMI uses Remote Procedure Calls (RPC) which uses random network ports. The WMI connections are encrypted when possible.

Get-NtfsPermissionsOfIisFoldersToCsv.ps1

Gets the discretionary access control lists (DACLs) from the physical paths of IIS7 operational folders and writes them to a comma separated file (CSV) for post-analysis such as auto-filter in Microsoft Excel. This script requires remote WMI connectivity to all of the servers specified. WMI uses Remote Procedure Calls (RPC) which uses random network ports. The WMI connections are encrypted when possible.

Get-WebAdministrationToXml.ps1

Gets all of the WMI data of one or more IIS7 servers and writes it to a single XML document. This data comes from the root\WebAdministration WMI namespace. Requires administrator rights on the target server(s). The WMI connections are encrypted when possible. This is probably the most powerful of all of the scripts simply because it gets everything related to IIS from each IIS server.

Get-ParentPaths.ps1

Gets the ASP Parent Path setting for all web sites for all IIS7 servers. This scripts uses the root\WebAdministration WMI namespace. Requires administrator rights on the target server(s). The WMI connections are encrypted when possible.

Get-SecurityGroupMembership.ps1

This script is not specific to IIS7, but helps with checking operating system security health. This script gets the membership of security groups from one or more computers and writes them to a comma separated file (CSV) for post-analysis such as auto-filter in Microsoft Excel. This script requires remote WMI connectivity to all of the servers specified. WMI uses Remote Procedure Calls (RPC) which uses random network ports. The WMI connections are encrypted when possible.

Set-EnableAllW3cFields.ps1

Enables all of the W3C logging fields on all web sites of one or more IIS7 servers. This scripts uses the root\WebAdministration WMI namespace. Requires administrator rights on the target server(s). The WMI connections are encrypted when possible. This script is helpful to prepare IIS servers for intensive W3C log analysis.

Log-EphemeralPortStats.ps1

This script is not specific to IIS7, but helps with checking operating system network health. Runs in an infinite loop getting the TCP ephemeral port and listening port statistics for each local IP address and outputs the data to a text file log. The script writes the ephemeral port stats every 60 seconds by default. To get data from remote computers, this script requires PsExec.exe (SysInternals) to be in the same directory as this script. WARNING: Credentials passed into PSExec are sent over the network in clear text! Prevent this by logging in interactively with a domain account that has administrator rights on the target computers and not specifying credentials to this script. PsExec is a Sysinternals tool owned by Microsoft Corporation. PsExec can be downloaded for free at http://live.sysinternals.com/psexec.exe.

Download

These scripts are available on my SkyDrive at:
https://skydrive.live.com/?cid=e6360c54b48a891b&sc=documents&id=E6360C54B48A891B%21428#cid=E6360C54B48A891B&id=E6360C54B48A891B%21964&sc=documents

Enjoy!

Clint Huffman | Senior Premier Field Engineer | Microsoft Services | clinth@microsoft.com | TS:Windows Internals

Microsoft Tag: Download my contact information to your phone.

Get the free app for your phone at http://gettag.mobi

How to Speak SAN-ish

2011-05-14T00:16:06Z

I recently signed a contract with MCP Magazine to publish articles. This is effectively a syndication of this blog.

Check out my first article called, “How to Speak SAN-ish” at
http://mcpmag.com/articles/2011/05/12/how-to-speak-san-ish.aspx

The Microsoft TechNet Wiki Performance Guide (PerfGuide)

2011-03-28T20:06:28Z

One of the major reasons why I haven’t been blogging much is because I have spent most of my “free” time working on the Microsoft TechNet Wiki writing what I call the “PerfGuide”. We have a lot of great content up there for Windows performance analysis.

The Microsoft PFE Performance Guide (PerfGuide): Start Here
http://social.technet.microsoft.com/wiki/contents/articles/the-microsoft-pfe-performance-guide-perfguide-start-here.aspx

The PAL Tool on Memory Leaks

2011-03-28T19:01:52Z

To prevent a process from crashing due to a System.OutOfMemory condition, the .NET garbage collector (GC) automatically defragments virtual memory. It can only do this for Gen 0 and Gen 1 memory allocations. Any allocations at 64 KB or larger will go to the large object heap. The large object heap and any non-managed objects (COM, C++, etc.) cannot be defragmented by the GC.

I recently published an article on the Microsoft Technet Wiki PerfGuide on diagnosing process virtual memory issues.
PerfGuide: Out of Process Virtual Memory
http://social.technet.microsoft.com/wiki/contents/articles/perfguide-out-of-process-virtual-memory.aspx

In any case, PAL is looking for a gradual and significant increase in process committed memory for which the operating system must provide system committed resources (physical RAM and/or page file). The committed memory of each process can be measured using the “\Process(*)\Private Bytes” counter. The amount of committed memory of a process is dictated by the process’s usage of memory – coded by the developer of the application. The .NET GC also removes/deallocates variables that are out of scope (such as local variables in a function that is no longer executing), but ultimately, the application developer determines which variables are in use. Therefore, we diagnose the memory usage of the application threw debugging or profiling to determine where the memory usage goes. My point is that, yes, you can have a memory leak even if you are using pure .NET. The GC just makes it less likely to happen.

With all of that said, memory leaks need to be looked at over a long period of time because when an application is busy, it will naturally use more memory. A memory leak is when the memory accumulates unnecessarily.

If a process has plenty of virtual memory (x64 has 8 TB of virtual memory per process) and if the leak continues unchecked, then the system might eventually run out of system commit memory which is the total amount of RAM and all of the page files combined. For more information on this topic, see the following PerfGuide post.

PerfGuide: Out of System Committed Memory
http://social.technet.microsoft.com/wiki/contents/articles/perfguide-out-of-system-committed-memory.aspx

We [Microsoft] typically work with applications in production, so we commonly use DebugDiag or WinDBG to analyze memory leaks by gathering several dumps (*.dmp) from the target process.

DebugDiag
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&displaylang=en