Clint Huffman's Windows Troubleshooting in the Field Bloghttp://blogs.technet.com/b/clinth/default.aspxClint Huffman is a Microsoft Premier Field Engineer (PFE) who has been with Microsoft for over 10 years. This blog documents the challenges he faces week to week in hopes that these experiences will help others.en-US7.x ProductionAvailable for pre-order: Windows Performance Analysis Field Guidehttp://blogs.technet.com/b/clinth/archive/2014/07/10/available-for-pre-order-windows-performance-analysis-field-guide.aspxFri, 11 Jul 2014 00:43:12 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:41735157-1874-467a-9130-d67bc269aea8Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3634422http://blogs.technet.com/b/clinth/archive/2014/07/10/available-for-pre-order-windows-performance-analysis-field-guide.aspx#comments<p>Yay! After over a year in development, my book, the “Windows Performance Analysis Field Guide”, is available for pre-order! <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/clip_5F00_image002_5F00_57E34034.jpg"><img title="clip_image002" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="clip_image002" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/clip_5F00_image002_5F00_thumb_5F00_30A8F6FF.jpg" width="178" height="219" /></a> <p>$37.46 at <a href="http://store.elsevier.com/product.jsp?isbn=9780124167018&amp;_requestid=465338">http://store.elsevier.com/product.jsp?isbn=9780124167018&amp;_requestid=465338</a> <p>$45.42 (Prime discount) at <a href="http://www.amazon.com/dp/0124167012/ref=wl_it_dp_o_pC_nS_ttl?_encoding=UTF8&amp;colid=12JDCG3UP69SD&amp;coliid=I2TOVTYHI6HDHC">http://www.amazon.com/dp/0124167012/ref=wl_it_dp_o_pC_nS_ttl?_encoding=UTF8&amp;colid=12JDCG3UP69SD&amp;coliid=I2TOVTYHI6HDHC</a> <p>I certainly don’t know everything about Windows performance analysis (no one ever will), so *<b>many</b>* of my friends and colleagues are named in the book as people who have helped me along the way. <p>A special thanks to Microsoft Premier Field Engineering (PFE), Jeff Stokes, Yong Rhee, Ed Wilson, and Mario Hewardt for helping to make this book possible.</p></p></p></p></p></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3634422&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">How to create a “black box” performance counter data collectorhttp://blogs.technet.com/b/clinth/archive/2014/05/23/how-to-create-a-blackbox-performance-counter-data-collector.aspxFri, 23 May 2014 23:15:52 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:0368b0b9-3e71-48e2-baa0-9dfaa243358cClint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3631363http://blogs.technet.com/b/clinth/archive/2014/05/23/how-to-create-a-blackbox-performance-counter-data-collector.aspx#comments<p>I highly encourage my customers to run a “black box” performance counter data collector set on their Windows Servers. The purpose of it is to continuously collect a detailed amount of data about the system in a circular file buffer of adjustable size so what if/when the system has a performance problem, we have roughly 24 hours of data leading up to the problem. <p>This article is a walk through of how to create a performance counter data collector set template with all of the performance counters used by industry experts, how to import the template onto a local or remote system, and how to automatically start the data collector set at Windows start up – Performance Monitor does not have a feature to automatically start a data collector set after a reboot. <p>This procedure was tested on Windows 8.1 and Windows Server 2012 R2, but should work fine on Windows Vista, Windows Server 2008, and later operating systems. <p>&nbsp; <h2>Create a “black box” performance counter data collector template</h2> <p>First, determine the Microsoft products and services installed on the target system. In this example, I have a customer who needs a data collector set template for a Microsoft Windows Server 2012 R2 system running a named instance of Microsoft SQL Server 2012. Therefore, I want to target performance counters for both Windows Server 2012 R2 as well as SQL Server 2012. <p>Arguably the easiest way to do this is to install the <a href="http://pal.codeplex.com/" target="_blank">Performance Analysis of Logs (PAL)</a> tool – an open source project/tool of mine. It requires Powershell 2.0 and the <a href="http://www.microsoft.com/en-us/download/details.aspx?id=14422" target="_blank">Microsoft Chart Controls for the .NET Framework 3.5</a> – both are free products from Microsoft. Once installed, run the PAL Wizard from the Windows Start menu and navigate directly to the Threshold File tab. Select a threshold file or combination of threshold files (they can be mixed and matched through the inheritance pane, and click the Export to Perfmon template file button. <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_77CD022A.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_3E49F233.png" width="244" height="196" /></a> <p>In this case, I selected Microsoft SQL Server 2012 which already has an inheritance of System Overview and Quick System Overview which are good for all Microsoft Windows operating systems. <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_52CF41B1.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_6754912F.png" width="244" height="188" /></a> <p>Since SQL Server might have named instances, the PAL tool detected SQL counters and prompts me to name any SQL named instances. <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_4DEC8DF5.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_028CEA31.png" width="244" height="104" /></a> <p>In this case, I specified “SQL2” and saved it as an XML file which is the format that Windows Server 2008 and later uses for counter log templates. Once the XML file is saved. Optionally, open it in Notepad to verify that it has the counters that you expect to see. <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_6924E6F6.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_4180FB7D.png" width="230" height="244" /></a> <p>This template is a good start, but will require additional modifications to make it a circular log. <p>&nbsp; <h2>Create a performance counter data collector using the template</h2> <p>Copy the XML template that you created in the previous step to a file system or network share location that is accessible to your domain account. This could be the a local disk drive on your workstation, a local disk drive on the target system, or a network share that your workstation or the target server has access to. <p>On your workstation or at the console of the target system, open an Administrator command prompt. This procedure requires administrator rights on the target system. <p>Run the following command to create a data collector using the template: <p><font face="Courier New">logman import &lt;NameOfDataCollectorSet&gt; -xml &lt;PathToXmlTemplate&gt; -s &lt;NameOfTargetSystem&gt;</font> <p>Here is an example of me creating the data collector set named, “SQL2_Blackbox_darksteel” (the %computername% resolves to the local computer name using the template I created earlier, “Sql2Template.xml”, on a remote system called, “darksteel”. <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_4ADCC0B1.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_5F62102F.png" width="244" height="44" /></a> <p>Run the following command to check if the data collector set was created: <p><font face="Courier New">logman query</font> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_2CFE3CB0.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_7A9A6930.png" width="244" height="61" /></a> <p>Note that the data collector set has not been started yet. Do not start it just yet. <p>Update the data collector set to be in binary circular format with a log file maximum size of 300 MB or a size that you prefer. <p><font face="Courier New">logman update &lt;NameOfDataCollectorSet&gt; –f bincirc –max &lt;SizeOfLogInMB&gt;</font> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_3345133E.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_75B7B574.png" width="244" height="53" /></a> <p>Now, the data collector set can be started and left to run forever without ever being larger than the size specified. <p>Optionally, the output location can be changed using the following sample command:<br /><font face="Courier New">logman update SQL2_Blackbox_DARKSTEEL -o D:\Perflogs\SQL2_Blackbox_DARKSTEEL.blg</font> <p>This file path is in respect to the file system on the target system – not the workstation from which the command might be running. <p><strong><font color="#ff0000">Important:</font></strong> Ensure that the output directory location does not already contain data. Performance Monitor’s Data Management features might delete data at that location. <p>To start the data collector, run the following command: <p><font face="Courier New">logman start &lt;NameOfDataCollectorSet&gt;</font> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_5C4FB23A.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_50B9F4FB.png" width="244" height="72" /></a> <p>&nbsp; <h2>Setup the data collector set to start on the startup of Windows</h2> <p>Unfortunately, Performance Monitor does not provide a way to automatically start the data collector set after a reboot. This means all of your hard work of creating a “black box” data recorder will be for nothing after someone reboots the system. In this step, we will setup the data collector set to automatically start when Windows starts by using Task Scheduler. <p><strong>Run the following command in order to have the data collector set automatically start after a reboot:</strong></p> <p><font face="Courier New">schtasks /create /tn &lt;NameOfDataCollectorSetOrAUniqueName&gt; /sc onstart /tr &quot;logman start &lt;NameOfDataCollectorSet&gt;&quot; /ru system /S &lt;NameOfTargetSystem&gt;</font></p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_3751F1C1.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_32DB70FA.png" width="244" height="34" /></a> <p>Note: The /S parameter of schtasks can be used to create this task on a target, remote, system. <p>Now the data collector set will automatically be started when the system starts. <p>&nbsp; <h2>Setup the data collector set to delete the oldest log file when low on disk space</h2> <p>By default, data collector sets will create a unique counter log each time it is started. This is to prevent the deletion of the existing logs. This is a nice feature, but this can result in several log files that might just be taking up too much disk space. <p>To setup the data collector set to delete the oldest log file when the disk space is low, open Performance Monitor (Start, Run, Perfmon.exe). Expand Data Collector Sets, expand User Defined and select the data collector set you wish to target for this. Right click on the data collector set and click Data Manager. <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_2745B3BB.png"><img title="image" style="border-top:0px;border-right:0px;border-bottom:0px;border-left:0px;display:inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/image_5F00_thumb_5F00_74E1E03B.png" width="219" height="244" /></a> <p>In this case, I setup the data collector set to delete the oldest counter log file. <p><strong><font color="#ff0000">Important:</font></strong> This setting can potentially delete existing data in the output directory location. Ensure that the output directory location *only* contains counter log files for this data collector. Avoid paths such as the root of a disk drive. <p>&nbsp; <h2>Conclusion</h2> <p>After following these steps you will have a data collector set that can run indefinitely without running the system out of disk space – it stays at the maximum size or smaller. Just stop the data collector set (the data collector set must be stopped before moving the counter log file otherwise it can result in file corruption) and analyze the counter log with the <a href="http://pal.codeplex.com/" target="_blank">Performance Analysis of Logs (PAL)</a> tool. <p>Want more? These procedures and *much* more is covered in my book, “The Windows Performance Analysis Field Guide” due out in August 2014.</p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3631363&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Using Autoruns to validate system drivershttp://blogs.technet.com/b/clinth/archive/2013/11/21/using-autoruns-to-validate-system-drivers.aspxThu, 21 Nov 2013 23:05:00 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:91ccaaa0-4d72-40bb-a20f-dfa4cf4a77d9Clint Huffman2http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3612930http://blogs.technet.com/b/clinth/archive/2013/11/21/using-autoruns-to-validate-system-drivers.aspx#comments<p>Recently, one of my enterprise customers had a system crash popularly known as a &ldquo;blue screen of death&rdquo; and this reminded me of the importance to validate drivers. Validating drivers is something I commonly do with my non-techie friends and family as well, simply because poorly written drivers are the most common cause of system crashes.</p>
<p>A relatively quick and easy way of validating the drivers of a system is to use the Sysinternals (Microsoft owned) tool, <a href="http://live.sysinternals.com/autoruns.exe" target="_blank">Autoruns</a>. Autoruns is a free tool that does not require installation. Just download and run. For more information on Autoruns go to <a title="http://technet.microsoft.com/en-us/sysinternals/bb963902" href="http://technet.microsoft.com/en-us/sysinternals/bb963902">http://technet.microsoft.com/en-us/sysinternals/bb963902</a> and consider reading the book, &ldquo;<a href="http://technet.microsoft.com/en-us/sysinternals/hh290819" target="_blank">Windows Sysinternals Administrator&rsquo;s Reference</a>&rdquo;.</p>
<ol>
<li><strong>Download and launch Autoruns.exe</strong></li>
</ol><ol>
<li>Download Autoruns using one of the links above or click <a href="http://live.sysinternals.com/autoruns.exe" target="_blank">here</a> and save it to your desktop.</li>
<li>Double-click on it to run it.&nbsp; <br /><strong>Note:</strong> Administrator rights is not required unless you intend to make changes to the system. Also, you might be prompted for Administrator credentials or other User Access Control (UAC) prompts depending on your settings.</li>
<li>Autoruns will begin to gather system information. For now, press the ESC key to stop the gathering of data since we will be rescanning the system shortly. Otherwise, wait for the Status bar to reports, &ldquo;Ready&rdquo;.</li>
</ol>
<ul><li><strong>Filter Autoruns:</strong> This step will remove common entries that are relatively safe and enable verification of the signatures of the drivers, EXEs and DLLs of software targeted for startup on the system. <br /><strong>Note:</strong> Microsoft drivers are certainly not immune to problems, but they are generally written well enough to not be on the list of usual suspects.</li>
</ul><ol>
<li>In the menu, click <strong>Options</strong>, <strong>Filter Options</strong>.</li>
<li>Enable/check <strong>Verify code signatures</strong>, <strong>Hide Microsoft entries</strong>, and <strong>Hide Windows entries. <br /> <br /><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8562.image_5F00_290F9296.png"><img width="244" height="162" title="image" style="display:inline;" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5751.image_5F00_thumb_5F00_3675A59C.png" border="0" /></a> <br /></strong></li>
<li>Click the <strong>Rescan</strong> button and wait for the hour glasses to go away. Again, this might take a few minutes depending on the resources of the system.</li>
</ol>
<ul><li><strong>Validate drivers:</strong> This step checks non-Microsoft drivers. According to the Windows Sysinternals Administrator&rsquo;s Reference, &ldquo;Verifying a digital signature associated with that file gives a much higher degree of assurance of the file&rsquo;s authenticity and integrity.&rdquo; <br /><strong>Note:</strong> When a driver is verified, the Publisher field changes from the company name to the name on the signed certificate.</li>
</ul><ol>
<li>Click the <strong>Drivers</strong> tab and look for drivers that are &ldquo;Not Verified&rdquo;. This will show up in the Publisher field.</li>
<li>In my case, I found one driver that is not Microsoft, but it was verified to be a virtual networking vendor. <br /> <br /><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0068.image_5F00_3D28AF1F.png"><img width="244" height="132" title="image" style="display:inline;" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3223.image_5F00_thumb_5F00_71C90B5A.png" border="0" /></a>&nbsp;</li>
<li>If any of the drivers are highlighted and come up as &ldquo;Not Verified&rdquo; in the Publisher field, then the driver does not have a digital signature.</li>
</ol>
<p>Drivers that are &ldquo;Not Verified&rdquo; do not have a digital signature or&nbsp;have been modified. It doesn&rsquo;t mean that the driver is malware. It just means that the driver cannot be verified to be from the publisher that is claims to be from. For example, the driver might have been written by a small company that didn&rsquo;t bother to have their driver signed.</p>
<p><strong>Note:</strong> 64-bit versions of Windows and Windows Server require all drivers to be signed when loaded, but this policy can be&nbsp;by-passed.</p>
<p>With that said, malware commonly installs a driver to gain access to the system and it could be exposed through this procedure. My advice is if the system is not functioning properly, then I would uninstall the unverified drivers.</p>
<p>Beyond driver validation, Autoruns is great for fixing performance issues as well. I simply go to all of the other tabs and uncheck/disable all of the software that I don&rsquo;t recognize. Be careful when doing this and disable only the entries that you recognize and don&rsquo;t want. If you accidentally disable something that you need, then its okay because Autoruns makes a backup of the entry and it can be re-enabled by running the tool again.</p>
<p>I hope you find this helpful.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3612930&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Convert a performance counter data collector template into a PAL threshold filehttp://blogs.technet.com/b/clinth/archive/2013/11/18/convert-a-performance-counter-data-collector-template-into-a-pal-threshold-file.aspxTue, 19 Nov 2013 06:19:48 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:02ada2fc-3258-48be-a9a5-1564a7b07b6fClint Huffman3http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3611963http://blogs.technet.com/b/clinth/archive/2013/11/18/convert-a-performance-counter-data-collector-template-into-a-pal-threshold-file.aspx#comments<p>If you want to have all of the counters in a counter log be represented in a PAL report, then use the AllCounterStats feature in the PAL Wizard. This will use all of the thresholds in the PAL threshold files as well as ensure that all of the counters are in the report at Stats Only.</p> <p>With that said, if you deal with many unique counter and don’t want to create your own threshold file yet, then consider using the script discussed in this blog entry as a starting point.</p> <p>Due to popular demand, I created a Powershell script that will convert a performance counter data collector template into a <a href="http://pal.codeplex.com" target="_blank">PAL</a> threshold file. The script is called a non-imaginative PerfmonTemplateToPalThresholdFile.ps1. Be forewarned that the threshold file produced from this will not have any thresholds in it.</p> <p>For now, I consider it to be beta, so it can be downloaded from:</p> <p><a href="http://aka.ms/clinth">http://aka.ms/clinth</a>, then go to PAL/BetaTesting</p> <p>or</p> <p><a title="https://skydrive.live.com/?cid=e6360c54b48a891b&amp;id=E6360C54B48A891B%21428&amp;authkey=!APQbXtD8U-HExzE#cid=E6360C54B48A891B&amp;id=E6360C54B48A891B%215119&amp;authkey=%21APQbXtD8U-HExzE" href="https://skydrive.live.com/?cid=e6360c54b48a891b&amp;id=E6360C54B48A891B%21428&amp;authkey=!APQbXtD8U-HExzE#cid=E6360C54B48A891B&amp;id=E6360C54B48A891B%215119&amp;authkey=%21APQbXtD8U-HExzE">https://skydrive.live.com/?cid=e6360c54b48a891b&amp;id=E6360C54B48A891B%21428&amp;authkey=!APQbXtD8U-HExzE#cid=E6360C54B48A891B&amp;id=E6360C54B48A891B%215119&amp;authkey=%21APQbXtD8U-HExzE</a>&#160;</p> <p>In a few weeks or so, I will include it with the PAL tool – likely v2.4.1.</p> <h3>How to use it</h3> <ol> <li>Ensure Powershell is set to unrestricted – WARNING this can be a potential security risk</li> <ol> <li>At an administrator Powershell session run: Set-ExecutionPolicy unrestricted</li> </ol> <li>Place a performance counter data collector template file (it must be in XML format) in the same directory as the script.</li> <li>Unblock the script file. Files downloaded from the internet are considered high risk, so go to the properties of the script file and click the Unblock button. Otherwise, the script will not be permitted to run.</li> <li>Run the script</li> <ol> <li>Start a Powershell session and change directory to the script location.</li> <li>.\PerfmonTemplateToPalThresholdFile –PerfMonTemplate .\SysTemplate.xml</li> <li>A PAL threshold file will be created in the same directory. It will use the DataCollector Name attribute in the template file as the file name.</li> </ol> <li>Copy the PAL threshold file to the PAL installation directory</li> <ol> <li>Once the PAL threshold file is created, copy it to the PAL installation directory.</li> <li>Run the PAL Wizard and you should see the file in the list of threshold files.</li> </ol> </ol> <h3></h3> <h3>Considerations</h3> <ul> <li>This script is designed for PAL v2.0 and later.</li> <li>The PAL threshold file produced will only create statistics only – no thresholds.</li> <li>If you goal is to have all of the counters in a counter log to be represented in the PAL report, then use the AllCounterStats feature.</li> </ul><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3611963&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Tracking page file reads and writeshttp://blogs.technet.com/b/clinth/archive/2013/10/16/tracking-page-file-reads-and-writes.aspxThu, 17 Oct 2013 02:35:22 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:cae14dba-4d7d-4162-9d80-f0ecf2cfb4dfClint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3603039http://blogs.technet.com/b/clinth/archive/2013/10/16/tracking-page-file-reads-and-writes.aspx#comments<p>&#160;</p> <p>The only real way of knowing if a page file is actually being “read from” is to get a file IO trace. This can be collected and/or viewed with tools such as the Microsoft Performance Recorder/Analyzer, Microsoft Resource Monitor, or Sysinternals Process Monitor.</p> <p></p> <h3>Using Resource Monitor</h3> <p>Resource Monitor is built into the operating system and can be launched from the Performance tab of Task Manager. </p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3463.image_5F00_27D6F156.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7266.image_5F00_thumb_5F00_75731DD6.png" width="244" height="101" /></a> </p> <p><strong>The Disk Activity pane in Microsoft Resource showing reads and writes to pagefile.sys.</strong></p> <p>The Disk tab shows the processes and files involved in live disk activity. This data comes from Event Tracing for Windows (ETW) and shows much more data than what performance counters can provide. In this case, we can see the file C:\pagefile.sys being written to by the System process. </p> <p></p> <h3>Using Windows Performance Recorder/Analyzer</h3> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3060.image_5F00_353D045C.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6082.image_5F00_thumb_5F00_7506EAE1.png" width="244" height="193" /></a> </p> <p>Microsoft Windows Performance Recorder/Analyzer is part of the free Windows Assessment and Deployment Kit (ADK) and can capture activity related to hard page faults and the processes and files associated with them. In this case, I used WPR to record Disk IO and File IO activity while forcing the system to use the page file. In the screenshot I am showing the Hard Faults chart within Windows Performance Analyzer (WPA) and I aggregated the data to show the files most involved with hard page faults and in this case it was C:\pagefile.sys. This is proof that most of the hard page fault activity was related to the page file in this case.</p> <p></p> <h3>Using Sysinternals Process Monitor</h3> <p>Sysinternals Process Monitor (Procmon) can also show page file reads and writes. This can be done by enabling the <strong>Advanced Output</strong> option under Filter, Enable Advanced Output. Once enabled, set the filter to only show events where “Path is C:\pagefile.sys” or similar for other page files.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8171.image_5F00_62BE241F.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3884.image_5F00_thumb_5F00_37798D18.png" width="244" height="151" /></a> </p> <p>After capturing data for a few minutes, click Tools, File Summary to get more details of the number of reads and writes to the page file.</p> <p></p> <h3>Hard page fault related performance counters</h3> <p>The performance counters <strong>\Memory\Page/sec</strong>, <strong>\Memory\Page Reads/sec</strong>, and <strong>\Memory\Page Inputs/sec</strong> measure hard page faults (faults that must be resolved by disk) which <strong>*may*</strong> or <strong>*may not*</strong> be related to a page file or a low physical memory condition. Hard page faults are a normal function of the operating system and happen when reading the portions of image files (DLLs and EXEs) as they are needed, when reading memory mapped files, or when reading from a page file. High values for these counters (excessive paging) indicate disk access (generally 4 KB per page fault on x86 and x64 versions of Windows and Windows Server), but again, they may or may not be related to page file activity, but they contribute towards disk usage which can increase the likelihood of system-wide delays assuming the related disk(s) are overwhelmed. Therefore, it is recommended to monitor the disk performance of the logical disks hosting a page file in correlation with these counters. </p> <p><strong>Note: \Memory\Page Writes/sec</strong> and <strong>\Memory\Page Output/sec</strong> do measure only page file writes.</p> <p>In addition, the counter <strong>\Paging File(*)\% Usage</strong> provides us the usage, but not how often a page file is actually being accessed. In addition, hard page faults just mean disk access. </p> <p>This is the reason why the Performance Analysis of Logs (<a href="http://pal.codeplex.com/" target="_blank">PAL</a>) tool measures a low memory condition that could cause system wide delays by taking many performance counters into consideration. The PAL tool has an analysis called Physical Memory Overwhelmed and creates a factious counter called <strong>\Memory\Physical Memory Overwhelmed</strong>. In this analysis, it analyzes for low physical memory (<strong>\Memory\Available MBytes</strong>), page file usage (<strong>\Paging File(*)\% Usage</strong>), and disk counters related to the page file(s) to determine if they are overwhelmed when the system is low on physical memory.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3603039&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Full debugging of VBScripts using Visual Studio 2005http://blogs.technet.com/b/clinth/archive/2013/09/30/full-debugging-of-vbscripts-using-visual-studio-2005.aspxMon, 30 Sep 2013 19:06:22 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:79df1a4b-dcdb-4335-aa21-e91e719e5c09Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3599628http://blogs.technet.com/b/clinth/archive/2013/09/30/full-debugging-of-vbscripts-using-visual-studio-2005.aspx#comments<p>Want to do *full* debugging of a VBScripts, then use this procedure.</p> <p>One of the hardest parts about scripting is getting to know the properties and methods of objects and state of a script during execution.&#160; In this procedure, I show you how to modify Microsoft Visual Studio 2005 for full debugging of VBScripts. I have used this procedure for many years starting with Windows XP. This time, I recently did this on Windows 8.1.</p> <h2>Install and run Visual Studio 2005</h2> <p>Yes, it must be a full version of Microsoft Visual Studio 2005 (Express editions do not work) and it can’t be older or a newer version of Visual Studio, so break out that old archive of software and install it.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5460.image_5F00_545FF95A.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/1263.image_5F00_thumb_5F00_6F98525B.png" width="244" height="187" /></a> </p> <p>During installation, the only option that is needed is “Visual Basic”. In my case, I cleared all other checkboxes and only checked Visual Basic. Then finish the installation wizard.</p> <p>On Windows 8.1, I had to install an older version of the Microsoft .NET Framework and run Visual Studio 2005 with Administrator rights in order for it to create some starting folders and files.</p> <h2>Configure Visual Studio 2005</h2> <p>Once Visual Studio 2005 is installed, you can immediately run your scripts through the Visual Studio 2005 debugger by using the “//X” argument in CScript.exe or WScript.exe. This prompts the scripting engine for a debugger. I will show how to do this later. In the meantime, let’s configure Visual Studio 2005 a bit more to make it better suited for VBScript execution and editing.</p> <p>Visual Studio will look like this initially.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/1856.image_5F00_1666689C.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4503.image_5F00_thumb_5F00_1D19721F.png" width="244" height="204" /></a> </p> <p>In the menu, click Tools, External Tools.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0702.image_5F00_43E7885F.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3438.image_5F00_thumb_5F00_51B9CE5A.png" width="244" height="239" /></a> </p> <p>Optionally, delete the “Dot&amp;fuscator Community Edition” entry. In my case, I removed it.</p> <p></p> <h3>For CScript execution</h3> <p>Click Add and set the fields to these values:</p> <table cellspacing="0" cellpadding="2" width="400" border="1"><tbody> <tr> <td valign="top" width="201"><font size="3"><strong>Field</strong></font></td> <td valign="top" width="197"><font size="3"><strong>Comment</strong></font></td> </tr> <tr> <td valign="top" width="201"><strong>Title:</strong> CScript</td> <td valign="top" width="197">This can be any title that you want.</td> </tr> <tr> <td valign="top" width="201"><strong>Command:</strong> C:\Windows\System32\cscript.exe</td> <td valign="top" width="197">This must be the path to the Windows installation directory and point to cscript.exe.</td> </tr> <tr> <td valign="top" width="201"><strong>Arguments:</strong> //nologo $(ItemPath)</td> <td valign="top" width="197">Click the right-arrow button and select “Item Path”. Optionally use the “//nologo” argument so that the CScript logo information will not be part of the output during script execution.</td> </tr> <tr> <td valign="top" width="201"><strong>Initial directory:</strong> $(ItemDir)</td> <td valign="top" width="197">Click the right-arrow button and click “Item Directory”.</td> </tr> <tr> <td valign="top" width="201"><strong>Use Output Window:</strong> [checked]</td> <td valign="top" width="197">Enable Use Output window. This will have the VBScript output go to the output window in Visual Studio</td> </tr> </tbody></table> <p>Leave all other entries at default settings and Click Apply.</p> <p><strong>Note:</strong> Optionally, use WScript in the command path if you want to have WScript.exe as the VBScript execution engine.</p> <p></p> <h3>For CScript debugging</h3> <p>Click Add and set the fields to these values:</p> <table cellspacing="0" cellpadding="2" width="441" border="1"><tbody> <tr> <td valign="top" width="235"><font size="3"><strong>Field</strong></font></td> <td valign="top" width="204"><font size="3"><strong>Comment</strong></font></td> </tr> <tr> <td valign="top" width="235"><strong>Title:</strong> CScript Debug</td> <td valign="top" width="204">This can be any title that you want.</td> </tr> <tr> <td valign="top" width="235"><strong>Command:</strong> C:\Windows\System32\cscript.exe</td> <td valign="top" width="204">This must be the path to the Windows installation directory and point to cscript.exe.</td> </tr> <tr> <td valign="top" width="235"><strong>Arguments:</strong> //nologo //X $(ItemPath)</td> <td valign="top" width="204">Click the right-arrow button and select “Item Path”. The “//X” argument is a CScript argument that tells the engine to prompt for a debugger. Optionally use the “//nologo” argument so that the CScript logo information will not be part of the output during script execution.</td> </tr> <tr> <td valign="top" width="235"><strong>Initial directory:</strong> $(ItemDir)</td> <td valign="top" width="204">Click the right-arrow button and click “Item Directory”.</td> </tr> <tr> <td valign="top" width="235"><strong>Use Output Window:</strong> [checked]</td> <td valign="top" width="204">Enable Use Output window. This will have the VBScript output go to the output window in Visual Studio</td> </tr> </tbody></table> <p>Leave all other entries at default settings and Click Apply.</p> <p><strong>Note:</strong> If you want to prompt for arguments to the script before execution, then check the “Prompt for arguments” check box.</p> <p>Click OK on the “External Tools” dialog box when finished adding entries.</p> <p></p> <h3>Add the scripting buttons to the toolbar</h3> <p>Right-click the Visual Studio 2005 toolbar (the bar with File, Edit, View, and so on) and then click Customize. Click the Commands tab and select Tools from the Categories menu. In the Commands menu, scroll down until you find “External Command 1”. Click, hold, and drag “External Command 1” to the menu bar of Visual Studio 2005. You can put it anywhere you want.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2451.image_5F00_7887E49A.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8712.image_5F00_thumb_5F00_7168A822.png" width="244" height="190" /></a> </p> <p><strong>Note:</strong> The button will initially show up as “External Command 1”, but will change to the external command title we provided earlier once the toolbar Customize dialog box is closed.</p> <p>Repeat this for all of the entries in the External Command window that you created earlier such as “External Command 2” and “External Command 3” and so on. Click Close on the Customize dialog box when finished.</p> <p>In my case, CScript and CScript Debug show up on my toolbar.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/1200.image_5F00_4624111B.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2388.image_5F00_thumb_5F00_45B7DE26.png" width="244" height="144" /></a> </p> <p></p> <h3>Show line numbers</h3> <p>When a VBScript throws an error, it provides the line number and character number of where the error occurred. For this reason, it is convenient to have the line numbers shown when editing the scripts.</p> <p>Click Tools and select Options. The Options dialog box will show. Check “Show all settings”.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2308.image_5F00_1A73471F.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4035.image_5F00_thumb_5F00_60F03727.png" width="244" height="142" /></a> </p> <p>In the tree view control on the left, navigate to Text Editor and select All Languages. Check “Line numbers” on the right and then click OK.</p> <p></p> <h2>Executing VBScripts in Visual Studio 2005</h2> <p>To open a script within Visual Studio 2005, simply right-click on the script, and Open with Visual Studio 2005. In this case, I opened up test.vbs.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7510.image_5F00_07BE4D68.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7026.image_5F00_thumb_5F00_51502C16.png" width="244" height="224" /></a> </p> <p>To execute the script, I just click the CScript button. In my case, my script output automatically showed the output in the Output window below.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2728.image_5F00_22F6A669.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8611.image_5F00_thumb_5F00_77B20F61.png" width="244" height="224" /></a> </p> <p></p> <h2>Debugging VBScript in Visual Studio 2005</h2> <p>Now for the best part! Debugging the script. When a script loaded into Visual Studio 2005, click the CScript Debug button (or whatever you named it). If it throws an error of not able to find the script, then click anywhere in the script editor pane and try it again. The script has to have the focus.</p> <p>When the debugger starts, it prompts if you want to debug the script from within the same instance of Visual Studio or a new instance of Visual Studio. Choose the existing instance of Visual Studio especially if you already have break points set. Then click the Yes button.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/1185.image_5F00_0C375EE0.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7446.image_5F00_thumb_5F00_40D7BB1B.png" width="224" height="244" /></a> </p> <p>This starts off the script in the debugger and it stops at the first line of code. The yellow arrow and highlight indicate the line of execution.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0118.image_5F00_67A5D15B.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3755.image_5F00_thumb_5F00_00356EAC.png" width="244" height="224" /></a> </p> <p>At this point, you can do debugging as you normally would in Visual Studio such as setting break points, run to cursor, show locals, have a watch window and so on. Use the commands in the Debug menu and/or Debug icons to debug the script.</p> <h3>Showing the output when debugging</h3> <p>A quirk of the debugger is that you have to add the output window again by clicking on down arrow of the icon of a window with a red sphere, then selecting Output. </p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4834.image_5F00_54F0D7A4.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7571.image_5F00_thumb_5F00_7BBEEDE4.png" width="156" height="244" /></a> </p> <p>Change the output window to the debug output.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2287.image_5F00_507A56DD.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0552.image_5F00_thumb_5F00_16F746E6.png" width="244" height="223" /></a> </p> <p>Now, you should see the WScript.echo output from the script when debugging.</p> <p></p> <h3>Using Locals and/or the Watch window</h3> <p>I like looking at the variables of my scripts using the Locals and Watch windows. While in the script is paused in the debugger, click the “Watch 1” icon in the Visual Studio tool bar. This will replace the Output window with the Watch 1 window.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5355.image_5F00_6BB2AFDE.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7002.image_5F00_thumb_5F00_2B7C9664.png" width="244" height="224" /></a> </p> <p>In this case, I can select variable “b” and drag it to the Watch Window.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3704.image_5F00_245D59EC.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5751.image_5F00_thumb_5F00_1214932A.png" width="244" height="224" /></a> </p> <p>I can do this with any variable or object within a script. This technique is especially helpful with objects and arrays because the Watch window allows you to explore the it and see the values.</p> <p><strong>Note:</strong> You cannot edit the script while it is running in the debugger and the script must be saved in order to run what you are seeing in the editor.</p> <p></p> <h2>Intellisense and quirks</h2> <p>Visual Studio tries to provide “intellisense” to objects, so that you can just type an object instance name and type, “.” to have it provide you with the properties and methods of the object. Unfortunately, Visual Studio 2005 only provides intellisense for a few of the VBScript objects. Also, please understand that Visual Studio 2005 actually thinks that the VBScript is VB.NET code, so there will be a few inconsistencies. </p> <p>Also, remember that the debug output has to be enabled. See “Showing the output when debugging” discussed earlier.</p> <p>For a richer intellisense experience with VBScript editing, consider third party tools such as Sapien Technologies <a href="http://www.sapien.com/software/primalscript" target="_blank">Primal Script</a>.</p> <p></p> <h2>Conclusion</h2> <p>There are certainly other editors out there for debugging VBScripts, but this one seems to be the easiest and richest experience in regards to VBScript debugging. I’ve used this technique for years for my own VBScripting needs and to teach students the basics of VBScripting. I hope you find this useful.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3599628&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Out of Pool Paged memory on 32-bit Windows Server 2003http://blogs.technet.com/b/clinth/archive/2013/09/23/out-of-pool-paged-memory-on-32-bit-windows-server-2003.aspxMon, 23 Sep 2013 18:59:56 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:c02c20bc-28c6-4aad-823f-462decb120efClint Huffman1http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3598310http://blogs.technet.com/b/clinth/archive/2013/09/23/out-of-pool-paged-memory-on-32-bit-windows-server-2003.aspx#comments<p>Lately, I have been assisting customers who are still using 32-bit Windows Server 2003 and inevitably running out of kernel pool memory. When one of the kernel pools (Pool Paged and Pool Nonpaged) are full (meaning a memory allocation to one of these pools fails due to a lack of free space), then applications or even the system might hang or malfunction, so this is a serious consideration. In any case, the primary problem here is 32-bits.</p> <p>The largest number that can be represented by 32-bit is 4 GB. That 4 GB is divided into user mode (process memory) and kernel mode. By default, this gives the kernel 2 GB to work with. The kernel memory is then divided further into a Page Frame Number (PFN) database, Pool Paged, Pool Nonpaged, and the rest of the free memory is given to Free System Page Table Entries (PTEs). In Windows Server 2003, Pool Paged and Pool Nonpaged have a specific size. This means that the system can run out of either pool and hang when there is still plenty of free kernel virtual memory elsewhere. This is why 32-bit versions of Windows Vista and Windows Server 2008 and later allow the full kernel virtual address space to be used. </p> <p>Instead of me going into all of the complexities of how to detect this problem and how to troubleshoot it, I will just go into these recommendations if you suspect the system it out of kernel pool memory.</p> <ol> <li><strong>Migrate to 64-bit Windows Server:</strong> 64-bit has 8 TB of virtual address space for kernel memory which will effectively solve this issue, but I realize that it might be difficult to go to 64-bit Windows Server. All drivers on a 64-bit version of Windows Server must be 64-bit, but 32-bit applications can run on 64-bit Windows Server.</li> <li><strong>Upgrade to 32-bit Windows Server 2008:</strong> If it is impossible to go to a 64-bit version of Windows Server, then at the very least consider upgrading to 32-bit Windows Server 2008. This version of Windows Server allow Pool Paged and Pool NonPaged to expand to up to just less than 2 GB assuming there is more than 2 GB of physical memory installed – Pool NonPaged’s maximum is 75% of physical memory (RAM).</li> <li><strong>Disable Dynamic Memory (hot-add):</strong> If the physical system has the capability to add more physical memory (RAM) to the system, then the Page Frame Number (PFN) database in Windows is likely larger than needed. To reduce the PFN database to a smaller size, set the DynamicMemory registry key to 1 which means 1 GB. The system realizes it is too small and resets it to the actual memory installed. This will likely increase the size of Pool Paged Bytes by 100 MB which could make the Pool Paged size to 169 MB. <br /> How to Configure the Paged Address Pool and System Page Table Entry Memory Areas <br /><a href="http://support.microsoft.com/kb/247904">http://support.microsoft.com/kb/247904</a> <br /> The DynamicMemory registry key is located under the following registry key: <br /> HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management</li> <li><strong>View the actual amount of Pool Paged maximum size:</strong> Use Process Explorer with the Debugging Tools for Windows (point Process Explorer to the dbghelp.dll file that comes with the Debugging Tools for Windows) and a symbol path of “SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols” (internet access required) to see the maximum size of the kernel pool paged.</li> <li><strong>Look for kernel insufficient memory events:</strong> In the System event logs, look for Event ID 2019 and 2020 messages. If the system has these events, then it indicates that the system ran out of kernel virtual memory.</li> <li><strong>Consider adjusting PoolPagedSize with CAUTION:</strong> Be *very* careful with this!!! The <a href="http://technet.microsoft.com/en-us/library/cc976157.aspx">PoolPagedSize</a> registry key (see the link for more information) can be set to a larger amount than what it is, *but* this will take away from the System PTEs and your system is borderline for running out of those as well. Only do this if a Microsoft Support Professional has recommended this.</li> <li><strong>Use Poolmon.exe:</strong> If the pool paged memory usage is too much, then use Poolmon.exe to identify which drivers are consuming the most. The download and usage of Poolmon is described in this blog entry by Mark Russinovich (author of the Windows Internals book and creator of the Sysinternals tools). <br /> Mark Russinovich’s blog entry on Windows pool memory troubleshooting: <br /><a href="http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx?wa=wsignin1.0">http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx?wa=wsignin1.0</a>. <br /></li> </ol><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3598310&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">My personal reminder of common debugging commandshttp://blogs.technet.com/b/clinth/archive/2013/08/15/my-personal-reminder-of-common-debugging-commands.aspxThu, 15 Aug 2013 21:20:47 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:67d4db9f-f2d5-48f3-ba9f-726ec1f444ebClint Huffman1http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3590769http://blogs.technet.com/b/clinth/archive/2013/08/15/my-personal-reminder-of-common-debugging-commands.aspx#comments<p>I don’t get as many opportunities to debug as I would like, but when I do, I always forget the command that I like to use, so this is my personal document to remind me of those commands.</p> <p><strong>!sym noisy</strong></p> <p>This gives me details of symbol resolution.</p> <p><strong>.reload /f</strong></p> <p>This forces all of the symbols to be downloaded. Great for packaging the symbols to take to a location that doesn’t have internet access.</p> <p><strong>~*kb</strong></p> <p>This enumerates all of the native thread stacks.</p> <p><strong>!threads</strong></p> <p>A summary of all of the threads.</p> <p><strong>.load C:\Users\clinth\Desktop\psscor2\psscor2\amd64\psscor2.dll</strong></p> <p>This loads the PSSCOR2 debugger extension for debugging .NET applications.</p> <p><strong>~*e!clrstack</strong></p> <p>Once the PSSCOR2 extension is loaded, this command enumerates the managed .NET call stack of all of the threads.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3590769&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Detecting ephemeral port exhaustionhttp://blogs.technet.com/b/clinth/archive/2013/08/09/detecting-ephemeral-port-exhaustion.aspxFri, 09 Aug 2013 20:19:37 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:6d6e8620-ef3c-46bb-a7ad-df220d0cfdfbClint Huffman2http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3589545http://blogs.technet.com/b/clinth/archive/2013/08/09/detecting-ephemeral-port-exhaustion.aspx#comments<p>Ephemeral ports are range of ports that Windows and Windows Server use for outbound communications over the TCP/IP network protocol. When an outbound connection is finished, the port associated to the connection is put into a TIMED_WAIT state for two minutes by default. This allows any lingering packets on the network to be ignored. Windows Server 2008 and later use the IANA range which uses the ports between 49152 and 65535 providing 16,383 ports.</p> <p>Some applications and services such as Microsoft Exchange Server CAS servers can be very “chatty” and might actually use all 16,383 ports within a two minute time period. The result is connection failures similar to “Couldn’t connect to X, due to no ports available from the end point mapper”.</p> <p>&#160;</p> <p>If you suspect ephemeral port exhaustion, then consider running the following Powershell script called “Log-EphemeralPortStats.ps1”: <br /><a title="https://skydrive.live.com/redir?resid=E6360C54B48A891B!5328" href="https://skydrive.live.com/redir?resid=E6360C54B48A891B!5328">https://skydrive.live.com/redir?resid=E6360C54B48A891B!5328</a></p> <p>Warning: This script is provided as sample code only. Please review it and use at your own risk.</p> <p>This script is designed to run in an infinite loop of 1 minute sleep intervals and write to a log file called “EphemeralPortStats.log”. Here is an example of the output it produces:</p> <p><font size="1" face="Courier New">Computer&#160;&#160;&#160;&#160;&#160;&#160; DateTime&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; LocalAddress&#160; #OfEPortsInUse Max#OfEPorts %EPortUsage #OfTcpListeningPorts #OfPids <br /> --------&#160;&#160;&#160;&#160;&#160;&#160; --------&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; ------------&#160; -------------- ------------ ----------- -------------------- ------- <br /> ETCHEDCHAMPION 8/9/2013 12:37:42 PM 127.0.0.1&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 6&#160;&#160;&#160;&#160;&#160;&#160;&#160; 16384&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 0&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 15&#160;&#160;&#160;&#160;&#160; 11 <br /> ETCHEDCHAMPION 8/9/2013 12:37:42 PM 172.18.96.192&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 3&#160;&#160;&#160;&#160;&#160;&#160;&#160; 16384&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 0&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 15&#160;&#160;&#160;&#160;&#160; 10 <br /> ETCHEDCHAMPION 8/9/2013 12:37:42 PM 192.168.1.2&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 69&#160;&#160;&#160;&#160;&#160;&#160;&#160; 16384&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 0.4&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160; 15&#160;&#160;&#160;&#160;&#160; 17</font></p> <p>This script is intended to be ran from the console of the computer suspected to be running low on ephemeral ports and to leave it running. Periodically review the log to see if there was any ephemeral port exhaustion detected.</p> <p>This script gets the port range from:</p> <p>netsh int ipv4 show dynamicportrange tcp</p> <p>Then, correlates this information with the output of:</p> <p>netstat –ano –p tcp</p> <p>PsExec can be potentially used to get this information from remote computers, but keep in mind that passwords used in PsExec are sent in the clear over the network.</p> <p>My PFE colleagues and customers have used this script quit a bit and I hope it will help you as well.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3589545&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Physical memory overwhelmed PAL analysis - holy grail found!http://blogs.technet.com/b/clinth/archive/2013/06/11/physical-memory-overwhelmed-pal-analysis-holy-grail-found.aspxTue, 11 Jun 2013 21:54:00 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:8a89828c-4e82-4276-bfe2-f68159098920Clint Huffman4http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3578250http://blogs.technet.com/b/clinth/archive/2013/06/11/physical-memory-overwhelmed-pal-analysis-holy-grail-found.aspx#comments<p>I just wrote a very complicated PAL analysis that determines if physical memory is overwhelmed. This analysis takes into consideration the amount of available physical memory and the disk queue length, IO size, and response times of the logical disks hosting the paging files.</p> <p>Also, if no paging files are configured, then it simply has a warning (1) for less than 10% available physical memory and critical (2) for less than 5% available physical memory.</p> <p>This analysis (once tested) will be in PAL v2.3.6.</p> <p>I am making an effort to make this analysis as perfect as I can, so I am open to discussion on this. For example, I might add in if pages/sec is greater than 1 MB, but we have to assume that the disks hosting the paging files are likely servicing non-paging related IO as well. I’m just trying to make it identify that there is *<b>some</b>* paging going on that may or may not be related to the paging files. Also, I am considering adding an increasing trend analysis to this analysis for \Paging File(*)\% Usage, but catching it increasing for a relatively short amount of time is difficult.</p> <p>Here is a screenshot of Perfmon with the counters that PAL is analyzing:</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7674.clip_5F00_image001_5F00_5D4190DB.png"><img title="clip_image001" style="margin: 0px; display: inline; background-image: none;" border="0" alt="clip_image001" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/1665.clip_5F00_image001_5F00_thumb_5F00_359B14B1.png" width="234" height="244" /></a></p> <p>Here is PAL’s simplified analysis…</p> <h4><a name="MemoryPhysicalMemoryOverwhelmed">Memory Physical Memory Overwhelmed</a></h4> <p><b>Description:</b> It's complicated.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5468.clip_5F00_image002_5F00_352EE1BC.png"><img title="clip_image002" style="display: inline; background-image: none;" border="0" alt="clip_image002" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8105.clip_5F00_image002_5F00_thumb_5F00_2E0FA544.png" width="244" height="190" /></a></p> <p>The physical memory overwhelmed analysis explained…</p> <p>When the system is low on available physical memory (available refers to the amount of physical memory that can be reused without incurring disk IO), the system will write modified pages (modified pages contain data that is not backed by disk) to disk. The rate at which it writes depends on the pressure on physical memory and the performance of the disk drives.</p> <p>To determine if a system is incurring system-wide delays due to a low physical memory condition:</p> <ol> <li>Is <strong>\Memory\Available MBytes</strong> less than 5% of physical memory? A “yes” does not indicate system-wide delays. If yes, then go to the next step.</li> <li>Identify the logical disks hosting paging files by looking at the counter instances of <strong>\Paging File(*)\% Usage</strong>. Is the usage of paging files increasing? If yes, go to the next step.</li> <li>Is there significant hard page faults using <strong>\Memory\Pages/sec</strong>? Hard page faults might or might not be related to paging files, so this counter alone is not an indicator of a memory problem. A page is 4 KB in size on x86 and x64 Windows and Windows Server, so 1000 hard page faults is 4 MB per second. Most disk drives can handle about 10 MB per second, but we can’t assume that paging is the only consumer of disk IO.</li> <li>Are the logical disks hosting the paging files overwhelmed? If the logical disk constantly has outstanding IO requests determined by <strong>\LogicalDisk(*)\% Idle Time</strong> of less than 10 and if the response time are greater than 15 ms (measured by <strong>\LogicalDisk(*)\Avg. Disk sec/Transfer</strong>) and if the IO sizes are greater than 64 KB (measured by<strong> \LogicalDisk(*)\Avg. Disk Bytes/Transfer</strong>), then add 10 ms to the response time threshold.</li> <li>As a supplemental indicator, if <strong>\Process(_Total)\Working Set</strong> is going down in size, then it might indicate that global working set trims are occurring.</li> <li>If all of the above is true, then the system’s physical memory is overwhelmed.</li> </ol> <p>I know this is complicated and this is why I created the analysis in PAL (<a href="http://pal.codeplex.com">http://pal.codeplex.com</a>) called \Memory\Physical Memory Overwhelmed that takes all of this into consideration and turns it into a simply red, yellow, or green indicator.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3578250&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">paging fileWindows performance analysispage faultsMemoryPagesShould the paging file be moved from C: drive?http://blogs.technet.com/b/clinth/archive/2013/06/07/should-the-paging-file-be-moved-from-c-drive.aspxFri, 07 Jun 2013 18:53:13 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:c82b6c7d-a983-4357-98f9-d162e7d5ea46Clint Huffman7http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3577494http://blogs.technet.com/b/clinth/archive/2013/06/07/should-the-paging-file-be-moved-from-c-drive.aspx#comments<p>Should the paging file be moved from C: drive to another drive? This was the question I received today and thought I’d share my response to this.</p> <p>There is no general answer for all situations, so this question needs more information about the environment. This is why you will not (and should not) find any official articles answering this question in any generalized form.</p> <p>First, the paging file must be able to accommodate the crash dump settings. Windows Server 2003 requires the paging file to be on the system partition and be large enough to accommodate the crash dump setting. Windows Server 2008 and later allows the paging file to be on other direct attached drives when accommodating a crash dump. Complete memory dumps requires the paging file to be 1xRAM + 1 MB, Kernel memory dumps vary based on the amount of kernel memory usage such as pool paged and pool non-paged sizes estimating roughly 100 MB for every 1 GB of RAM. Small dumps require about 1 MB of a paging file.</p> <p>Second, it depends on the amount of system committed memory that has been promised. The system commit limit is the size of RAM + paging files. It must always be larger than the system commit charge. The system commit charge will vary based on actual usage. With that said, committed memory doesn’t *not* mean that the paging file is actually being used. If the system commit charge is very large, but little of it is “touched”, then having a paging file on a slow disk drive is fine because the paging file is not really being used – just there to accommodate the commit charge *if* it happens to become “touched” memory.</p> <p>Third, assuming the C: drive is a slow drive and that the system commit charge is greater than RAM, and the committed memory is actually “touched”, then the performance of the disk where the paging file is at is important. A page is 4 KB, so a sustained 1000 pages/sec (hard page faults) is 4 MB per second. Most 7200 RPM disk drives can handle 10 MB per second for reference. </p> <p>If paging files are defined across multiple disks, then the paging file that is available first is used – meaning this is a load balanced situation which can help. Ultimately, if the paging file is *really* being used this much, then just add more RAM or place the paging file on a SSD.</p> <p>In short, moving the paging file really depends on how much the paging file(s) are really being used (touched memory), the crash dump settings, version of the operating system, the system commit charge at peak, and disk drive performance. </p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3577494&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">paging fileWindows performance analysisMemoryPAL processing, processors, and threadshttp://blogs.technet.com/b/clinth/archive/2013/01/14/pal-processing-processors-and-threads.aspxMon, 14 Jan 2013 19:32:52 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:cb66cfee-c85a-409b-b1bb-0cfc44d4f6f1Clint Huffman1http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3545765http://blogs.technet.com/b/clinth/archive/2013/01/14/pal-processing-processors-and-threads.aspx#comments<p>I commonly get questioned on what response should be given to the NumberOfProcessors question variable in the PAL tool, so I thought I might try to explain it a bit…</p> <p>PAL is designed to be a stand-alone tool where the analysis of a performance counter log can be analyzed on a workstation where an administrator can analyze counter logs of other computers with no connectivity. For example, customers regularly send me counter logs (*.blg) files to me and I analyze them from my home office or when I am at a hotel. I use PAL on my laptop to do the analysis. </p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7433.image_5F00_502A606A.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0160.image_5F00_thumb_5F00_73E38804.png" width="244" height="196" /></a> </p> <p>When you reach the Questions tab in the PAL Wizard, these questions are in regards to the computer(s) in which the performance counter log was captured. The number of processors refers to the number of logical processors that would be seen in Task Manager of that computer. With this in mind, I will change the wording of the question to be more specific. </p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4300.image_5F00_1AB19E45.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8103.image_5F00_thumb_5F00_4832BE08.png" width="244" height="196" /></a> </p> <p>At the end of the PAL Wizard on the Execute tab, it asks how many threads to use during analysis. This is asking how many threads can the PAL tool use on the local computer (workstation) to use for analysis. PAL is very processor intensive, so I recommend 1 minus the number of processors of the local workstation. For example, if you have a 4 processor workstation, then use no more than 3 threads. Otherwise, your workstation might become sluggish and hot due to the long-term, high processor usage.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3545765&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">PAL collector script – PalCollector.ps1http://blogs.technet.com/b/clinth/archive/2013/01/08/pal-collector-script-palcollector-ps1.aspxWed, 09 Jan 2013 07:45:32 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:6edd035b-5af1-4e5b-909e-97ca12a84d7bClint Huffman5http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3544849http://blogs.technet.com/b/clinth/archive/2013/01/08/pal-collector-script-palcollector-ps1.aspx#comments<p>One of the top questions I get with the <a href="http://pal.codeplex.com/">PAL tool</a> is what data to collect. As many of you know, the PAL tool only analyze existing counter logs. It is up to you to create the counter log. To help with this problem, I created a PowerShell script called PalCollector.ps1. This script will query your local computer’s performance counters and will find a PAL threshold file that best matches your computer, then it creates a data collector set called, “PalCollector” using the counter paths from the best matching threshold file. It creates the data collector as a 200 MB binary circular log which means that it will continuously collect data every 15 seconds and will never get over 200 MB in size – roughly 24 hours of data. Once you are done collecting data, analyze the counter log (*.blg) using the PAL tool as you normally would.</p> <h2>Instructions</h2> <ol> <li>Download PalCollector.zip from <a title="http://sdrv.ms/10dZBNb" href="http://sdrv.ms/10dZBNb">http://sdrv.ms/10dZBNb</a>.</li> <li>Extract the zip file to a folder such as your Desktop or somewhere under your “My Documents” folder.</li> <li>In Windows Explorer, find PalCollector.ps1, go to Properties of the file and click the Unblock button. This will allow the script to run on your system.</li> <li>With administrator rights (required) open an elevated PowerShell session.</li> <li>If your execution policy is not set to RemoteSigned or Unrestricted, then do so now by running: <br />Get-ExecutionPolicy <br />Set-ExecutionPolicy RemoteSigned</li> <li>Change directory to the location where you extracted the zip file.</li> <li>Run “.\PalCollector.ps1”</li> </ol> <p>Please let me know your thoughts on how it is working for you and any recommendations on improvement.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3544849&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">How to create a threshold file for the PAL toolhttp://blogs.technet.com/b/clinth/archive/2013/01/08/how-to-create-a-threshold-file-for-the-pal-tool.aspxTue, 08 Jan 2013 08:49:59 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:0144f206-b1e4-4501-afb9-db91c79af8e1Clint Huffman3http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3544625http://blogs.technet.com/b/clinth/archive/2013/01/08/how-to-create-a-threshold-file-for-the-pal-tool.aspx#comments<p>The Performance Analysis of Logs (PAL) tool is an open source project at <a href="http://pal.codeplex.com">http://pal.codeplex.com</a> that analyzes performance counter logs. It has thresholds for performance counters written by experts in their respective areas spanning nearly all of the major Microsoft Server products, some Citrix XenApp,, VMware. and others. The number of products covered is limited to only those willing to create a threshold file to define it. It’s my hope that this article will help enable you to create your own threshold file and together we can create prescriptive guidance on all of the performance counters.</p> <p>First, open the PAL Wizard as you normally would to analyze a counter log and navigate to the Threshold File tab and click “Edit…”. It doesn’t matter which threshold file is selected at this time. The PAL Editor will show.</p> <p>&#160;</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6052.image_5F00_3665369B.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6457.image_5F00_thumb_5F00_6E377ABE.png" width="603" height="484" /></a></p> <p align="center"><strong>Figure 1: The Threshold File tab in the PAL Wizard.</strong></p> <p>&#160;</p> <p>In the PAL Editor, go to the upper left and click File, New. </p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2251.image_5F00_4DB03B0C.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8182.image_5F00_thumb_5F00_3B67744A.png" width="644" height="401" /></a></p> <p align="center"><strong>Figure 2: The PAL editor after clearing it with File, New.</strong></p> <p>This clears the editor, but hasn’t created a new threshold file just yet. We will get to that. For now, lets create a new analysis by clicking the New button at the lower left of the editor. This will show the counter New Analysis dialog box. An analysis is the primary container for one or more data source counters that you want to analyze, the thresholds that are applied to the “counter to analyze”, and the charts generated for the data source counters.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8105.image_5F00_5B164E12.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2742.image_5F00_thumb_5F00_53F7119A.png" width="220" height="244" /></a> <a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6136.image_5F00_7AC527DA.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0272.image_5F00_thumb_5F00_73A5EB62.png" width="244" height="221" /></a> <a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0172.image_5F00_336FD1E8.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5875.image_5F00_thumb_5F00_5A3DE828.png" width="222" height="244" /></a> </p> <p><strong>Figure 3: The New Analysis and Add Counters dialog boxes showing the buttons to click to add new counters to a new analysis.</strong></p> <p>Click the Browse Counters button and this will open the Add Counters dialog box. This interface allows you to connect to a computer either local or remotely that has the performance counter that you want to add. It’s important to the PAL tool connects to computer that has the counter to ensure that the counter path is exact. Click Close or OK to all of the open dialog boxes to continue.</p> <p>Once at the main editor again, notice that many of the fields have been populated with data from the selected counter. This is a good opportunity to update the description of the analysis to tell the end users the purpose of this analysis, what is being checked and why, and what do to if the thresholds are exceeded. The description field supports the use of HTML tags and the rendered HTML can be previewed on the right pane. The text in the description will always show in the PAL report with this analysis. Once finished with editing the description, click the Update Analysis button to set the change and click File, Save to permanently save the changes to the threshold file.</p> <p>In this case, I am adding all of the instances of the <strong>\Power Meter(*)\Power </strong>performance counter. Once I click OK, you are returned to the main PAL editor and should now see the performance counter that you added on the left pane.</p> <h2>Excluding counter instances</h2> <p>In some cases, it is necessary to exclude specific instances of a performance counter. For example, the _Total counter instance of the LogicalDisk counter object is commonly excluded because the _Total instance sums all of the logical disk counter values together which is typically not helpful when trying to analyze each disk. To exclude a counter instance, select the data source counter to edit, then click the Edit button.</p> <p align="center"><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6445.image_5F00_1A07CEAE.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7534.image_5F00_thumb_5F00_79ECC1F0.png" width="244" height="218" /></a><strong>Figure 4: The Edit DataSource Counter form in the PAL editor.</strong> </p> <p>The Edit DataSource Counter form will show. Click the Add button and specify the counter instance to exclude. Repeat as many times as necessary to define all of the counter instances to exclue. In this case, I excluded the _Total instance. Click OK when finished.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8270.image_5F00_39B6A876.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/0763.image_5F00_thumb_5F00_5965823E.png" width="644" height="399" /></a></p> <p align="center"><strong>Figure 5: The PAL editor with a new analysis.</strong></p> <p>Now is the time to save the work as a new threshold file. At the top left, click File, Save As. This opens the Save As dialog box that will allow you to save the work you’ve done so far to an XML file that is the new threshold file. Navigate to a folder that you have write permissions to such as your Desktop or a location under your My Document folder. In the File Name field, type in a file name that you want the threshold file to have and click Save. Once saved, move the file to the folder where PAL is installed. This typically requires elevated privileges. The default installation folder for PAL is <strong>C:\Program Files\PAL\PAL</strong>. Saving files directly to this folder is typically prevented by Windows unless you use elevated privileges. In this case, I saved the threshold file as PowerStates.xml and I saved it to the PAL installation folder.</p> <p>Next, let’s give the threshold file a name and other information. Click the <strong>Edit Threshold File Properties…</strong> button. In the <strong>Threshold File Properties</strong> dialog, change the Title field to a more presentable name. In this case, I named mine, “Windows power states”. </p> <blockquote> <p><strong>Title:</strong> This is the name of your threshold file that will be shown. This must be a unique name relative to the other threshold file names. A title with the name of the manufacturer, product, and product version that the threshold file focuses on such as <strong>Microsoft IIS8</strong> is recommended.</p> <p><strong>Version:</strong> This starts off as 1.0. Increment the major and/or minor version numbers when significant changes are made to your threshold file.</p> <p><strong>Content owner(s):</strong> You are the content owner… the one who’s reputation is behind this threshold file. Put your name and names of other contributors in this field.</p> <p><strong>Feedback email addresses:</strong> Put your email address or addresses separated by semicolons (;) that you would like users to contact you for support or questions.</p> <p><strong>Threshold file description:</strong> This is a sentence or two describing purpose of the threshold file.</p> <p><strong>Threshold file inheritance order:</strong> There is no need to recreate all of the thresholds of the other threshold files. Simply inherit from all of the threshold files that you want. I generally recommend inheriting from at least the Quick System Overview threshold file because it contains all of the threshold for the core operating system. For example, Microsoft BizTalk Server depends on SQL Server and IIS, so it inherits from the SQL Server and IIS threshold files. When a change is made to any of the inherited threshold files, your threshold file automatically gets those changes allowing your threshold file to evolve with the other threshold files. <br /> <br />The order that the threshold files are listed is used to resolve conflicts where the two or more threshold files have an analysis with the same name or same identifier (in the XML only – not exposed by the editor). Your threshold file is always applied last meaning it will always win conflicts. This means that if you don’t agree with the logical disk latency thresholds defined in the Quick System Overview, then all you have to do is defined an analysis with the same name and create your own thresholds which will override the inheritance. This is what the Microsoft Exchange Server threshold files do because they have more restrictive threshold for disk latency than what the Quick System Overview threshold has defined. With that said, the Exchange Server threshold file still gets all of the other thresholds defined in the Quick System Overview threshold file. <br /> <br />To add a threshold file to inherit from, click the Add button, browse to the PAL installation folder, and select one of the threshold files listed there, then click Open. You should see the threshold file name listed in the inheritance order. If necessary use the Move up and Move down buttons to change the order in which the threshold files are applied. Remember, your threshold file will be applied last allowing it to win any conflicts in analysis names.</p> </blockquote> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6433.image_5F00_79145C06.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4863.image_5F00_thumb_5F00_31BF0614.png" width="360" height="484" /></a></p> <p align="center"><strong>Figure 6: The Threshold File Properties dialog box.</strong></p> <p>In my case, I added the QuickSystemOverview.xml file. Click OK when finished. For good measure, save your work so far by clicking File, Save in the main PAL editor.</p> <p>At this point, the threshold file is usable and you should find it in the drop down menu on the PAL Wizard, but when no thresholds are defined, the counter will only show a chart and statistics only.</p> <p>Next, let’s add question variables to the threshold files.</p> <h2>Question variables (optional)</h2> <p>Question variables allow you to ask the end user more information about the computer system(s) where the counter log was captured that cannot be retrieved by any other means. The answer provided by the user can by used by thresholds in your threshold file for a more thorough analysis. For example, you could ask the user what phase of the moon it was when the counter log was captured.</p> <p>To add a question variable, click the Edit Questions button on the main PAL editor. Edit Questions will show. Click Add and “-Needs Updated-“ will show. Click “-Needs Updated-“ and replace the default data on the right as appropriate.</p> <blockquote> <p><strong>Question Variable Name:</strong> This is the variable name that will be used in the threshold code. Ensure that the name meets the variable naming requirements of PowerShell such as no spaces in the name.</p> <p><strong>Question:</strong> This is the question that is presented to the end user.</p> <p><strong>DataType:</strong> Choose Boolean or String. Boolean provides a True or False value type for the variable. String provides a text value type for the variable.</p> <p><strong>Default Answer:</strong> If no answer is provided by the end user, then this is the default response to the question.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7610.image_5F00_03658067.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8688.image_5F00_thumb_5F00_2A3396A7.png" width="220" height="244" /></a></p> <p align="center"><strong>Figure 7: The Edit Questions form</strong> </p> </blockquote> <p>Click Update, then OK when finished and do another File, Save for good measure.</p> <h2>Adding a threshold (optional)</h2> <p>To add a new threshold to an analysis, click the Add button in the Thresholds group. This will open the Add Threshold Properties.</p> <blockquote> <p><strong>Name:</strong> This is the name of the threshold and is the text that shows with all alerts generated by this threshold. Make this a concise description.</p> <p><strong>Condition:</strong> Choose Warning or Critical. Use Warning to alert the user that a critical threshold is near, there might be an ambiguous condition that could lead to a larger problem, or when the threshold is experimental. Use Critical when it is clear that there is a problem or a condition that the user must be made aware of. Notice that when the condition is changed, the priority changes. This is because Critical conditions are more important than Warning conditions.</p> <p><strong>Color:</strong> This will always be yellow for Warning or red for Critical conditions.</p> <p><strong>Priority:</strong> You can add as many thresholds as you want to an analysis, but if more than one threshold is broken, then only one threshold will win to produce an alert. When multiple thresholds in an analysis are broken, the threshold with the highest priority will win – meaning the name, condition, and color of the “winning” threshold will be used in the alert generated from the broken threshold(s).</p> <p><strong>Variables:</strong> This is a list of variables and descriptions of those variables that can be used in the PowerShell Threshold Code. These could be question variables such as the $PhaseOfTheMoon variable that I created earlier.</p> <p><strong>PowerShell Threshold Code:</strong> This is where nearly any PowerShell code can be added towards analyzing the “counter to analyze” data source counter. It can be as simple or as advanced as you prefer. By default, PAL provides a “ready to use” threshold by automatically adding the appropriate arguments to the StaticThreshold function. It defaults with a threshold of greater than 10. All of the lines that precede with “#//” are comments and can be removed. They are there only to provide as help. <br /> <br /><em>Note: Please keep in mind that the threshold code can be much more advanced than the standard StaticThreshold. For examples of advanced threshold code, explore the Process Private Bytes analysis of the System Overview threshold file.</em></p> <p><strong><em>StaticThreshold: </em></strong>This is a function inside of PAL.ps1 that will automatically compare the operator and threshold arguments to the values of the “counter to analyze” counter and will generate an alert each time the threshold is exceeded.</p> <p><strong><em>CollectionOfCounterInstances:</em></strong> This value must be the variable that contains all of the instances of the “counter to analyze” counter which is automatically named and provided.</p> <p><strong><em>Operator:</em></strong> This is a string type that accepts less than ‘lt’, less than or equal to ‘le’, greater than ‘gt’, or greater than or equal to ‘ge’.</p> <p><strong><em>Threshold:</em></strong> This must be an integer or double type that will be compared against the values of the “counter to analyze” counter.</p> </blockquote> <p>&#160;</p> <p align="center"><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3731.image_5F00_3C102A74.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/2816.image_5F00_thumb_5F00_5B52D147.png" width="644" height="428" /></a><strong>Figure 8: The Add Threshold Properties form.</strong></p> <p>Click OK when finished and the threshold should appear in the Thresholds section of the main PAL editor. Click the Update Analysis button on the lower right of the PAL editor, then click File and Save to save your changes.</p> <p>At this point, the threshold file can be used and if any of the thresholds are exceeded, then they will throw an alert with the corresponding conditions of the threshold.</p> <h2>Adding visual thresholds to the chart (optional)</h2> <p>If you are adding thresholds to an analysis, then it is highly recommended to add corresponding visual thresholds into the chart or charts generated for the analysis. On the main PAL editor form, click the Edit Chart button on the upper right.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4786.image_5F00_0F86FA8E.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3731.image_5F00_thumb_5F00_1A4451E3.png" width="644" height="400" /></a></p> <p align="center"><strong>Figure 9: The main PAL editor form highlighting the Edit Chart button.</strong></p> <p>&#160;</p> <p>This will show a new form that allows you to create a Warning threshold and/or a Critical threshold into the chart or charts generated for this analysis. These thresholds will show as yellow and red gradients with the ranges specified in this form. Like the counter thresholds, by default, the visual chrart thresholds of Warning and Critical are automatically generated and usable. You can enable one or both of them by clicking the Enabled combo box next to the respective threshold.</p> <p align="center"><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/1323.image_5F00_20F75B66.png"><img title="image" style="margin-right: auto; margin-left: auto; float: none; display: block;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4382.image_5F00_thumb_5F00_4E787B29.png" width="623" height="484" /></a><strong>Figure 10: The Edit Chart form in the PAL editor.</strong>&#160;</p> <blockquote> <p><strong>StaticChartThreshold:</strong> This is a function in PAL.ps1 that can be called to create the visual thresholds seen as gradients on the analysis charts.</p> <p><strong>CollectionOfCounterInstances:</strong> This argument requires the variable that contains all of the counter instances of the “counter to analyze”. It is recommended to only use the variable already provided.</p> <p><strong>MinThreshold:</strong> This is the lowest value of the respective Warning and/or Critical chart threshold.</p> <p><strong>MaxThreshold:</strong> This is the highest value of the respective Warning and/or Critical chart threshold. If the maximum value of Critical or Warning (if Warning is the only threshold) is 30 and if none of the counter values reach 30, then the chart will automatically expand to 35 which makes the placement of the gradient seem off. Therefore, consider using a value ending in .999 such as 29.999 to represent 30.</p> <p><strong>IsOperatorGreaterThan:</strong> This is a Boolean (true|false) argument. If True, then it is assumed that the greater the counter value, the worse the condition leading from yellow [Warning] to red [Critical] as the value increases. If False, then the effect is inverted – meaning lower values are considered a worse condition leading from yellow to red in a downward view.</p> <p><strong>UseMaxValue:</strong> This is a Boolean (true|false) argument. If True, then if this chart threshold is exceeded by the counter value, then this chart threshold is increased automatically to match the counter value. If False, then the chart threshold values will not change on the chart. When using both Warning and Critical chart thresholds, it is recommended to set the Warning chart threshold to False and set the Critical chart threshold to True allowing the Warning threshold to stay in place and the Critical threshold to continue to increase matching the counter value if it had exceeded the MaxThreshold value for Critical.</p> </blockquote> <p>&#160;</p> <p>Once finished, click OK to return to the main PAL editor, click Update Analysis on the lower right, then File, Save to permanently save your changes to the threshold file.</p> <p>At this point, the analysis should be relatively complete and should be tested. When working with many analyses within a threshold file, consider using the Enabled combo box near the top of the analysis to enable or disable the analysis. This is helpful when needing to test some, but not all of the analyses in your threshold file.</p> <h2>Generated counters (optional)</h2> <p>The PAL tool has the unique ability to create fake counters that don’t normally exist in a performance counter log, but can be analyzed, charted, and processed with thresholds exactly like normal performance counters. Unfortunately, the PAL editor does not provide a way to create a generated performance counter. It must be created by manually editing the XML code of the threshold file using a text or XML editor.</p> <p>The Network Interface % Network Utilization analysis is an example of a generated counter. In this example, the values of the counters \Network Interface(*)\Bytes Total/sec and \Network Interface(*)\Current Bandwidth are put through a formula that produces a percentage value of the amount of network bandwidth used based on the amount of data passing through compared to the current bandwidth of the network interface. In the PAL report, the % Network Utilization performance counter appears as if it was a real performance counter.</p> <p>The technique of creating generated counters based on other counters was also used in the SQL Server threshold file to compare full scans/sec to batch requests/sec in a ratio. Once the generated ratio counter was created, it is easy to add thresholds and chart thresholds for it.</p> <p>Examine the XML code of the analyses mentioned above as examples of creating your own generated performance counters.</p> <h2>PAL version is incompatible</h2> <p>If you receive the following error, “CheckPalXmlThresholdFileVersion : The threshold file specified is not compatible with PAL v2.0”, then the threshold file is missing the PALVERSION attribute. This is a bug with the editor. Open the threshold file in an XML or text editor and add the XML attribute PALVERSION to the PAL XML node with a value of “2.0”. It should look similar to this:</p> <p>&lt;?xml version=&quot;1.0&quot;?&gt; <br />&lt;PAL <strong>PALVERSION=&quot;2.0&quot;</strong> NAME=&quot;Quick System Overview&quot; …</p> <p>Save the threshold file and try again. This was discussed on the PAL forum at: <br /><a title="https://pal.codeplex.com/discussions/468305" href="https://pal.codeplex.com/discussions/468305">https://pal.codeplex.com/discussions/468305</a></p> <h2>Conclusion</h2> <p>I know this guide on creating your own threshold files for PAL is very much over due, but I hope you find it useful. If you create a threshold file, then I am happy to include it in the next release of the PAL tool. Just ping me on Twitter @ClintH or post your questions on the PAL forums at <a href="http://pal.codeplex.com">http://pal.codeplex.com</a>.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3544625&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Page Frame Number (PFN) databasehttp://blogs.technet.com/b/clinth/archive/2013/01/06/page-frame-number-pfn-database.aspxMon, 07 Jan 2013 07:33:38 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:32edc8d0-5103-4acd-92b5-d92fcfe8d59fClint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3544398http://blogs.technet.com/b/clinth/archive/2013/01/06/page-frame-number-pfn-database.aspx#comments<p>I just finished writing an article on the public, wiki, PFE PerfGuide on the subject of the Page Frame Number database in Microsoft Windows and Windows Server. This little known database is used by the operating system to keep track of the physical memory of the system. Please check it out and update it if necessary. </p> <p><a title="http://social.technet.microsoft.com/wiki/contents/articles/15259.page-frame-number-pfn-database.aspx" href="http://social.technet.microsoft.com/wiki/contents/articles/15259.page-frame-number-pfn-database.aspx">http://social.technet.microsoft.com/wiki/contents/articles/15259.page-frame-number-pfn-database.aspx</a></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3544398&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Memory combining in Windows 8 and Windows Server 2012http://blogs.technet.com/b/clinth/archive/2012/11/29/memory-combining-in-windows-8-and-windows-server-2012.aspxThu, 29 Nov 2012 22:23:35 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:a5684e7c-5388-4f7e-be8e-81764f407670Clint Huffman2http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3535579http://blogs.technet.com/b/clinth/archive/2012/11/29/memory-combining-in-windows-8-and-windows-server-2012.aspx#comments<p>I’ve spent that last few weeks studying the memory architecture of Windows 8 and Windows Server 2012. </p> <p>Windows and Windows Server has always had sharable memory where portions of DLLs and EXEs will have a single copy in physical memory (synonymous with RAM) and all of the applications that need them will simply reference the page already in physical memory. This is still counted in their working sets, but overall the operating system saves on physical memory usage.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8585.image_5F00_0930403C.png"><img title="image" style="border: 0px currentcolor; display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3113.image_5F00_thumb_5F00_48FA26C1.png" width="314" height="405" /></a> </p> <p>One of the interesting features of Windows 8 and Windows Server 2012 is how process private page-able memory is periodically combined further saving on physical memory usage. This was mentioned in Bill Karagounis’s blog, <a href="http://blogs.msdn.com/b/b8/archive/2011/10/07/reducing-runtime-memory-in-windows-8.aspx">Reducing runtime memory in Windows 8</a>. The savings on physical memory usage could be dramatic, but possibly have some overhead from the system needing to search for duplicate pages of memory. I can only speculate at best right now.</p> <p>An important distinction is that Windows 8 has memory [page] combining enabled by default, but Windows Server 2012 does not.</p> <p>To check if your computer has page combining enabled or not, open an elevated Powershell session and type the following command:</p> <p>Get-MMAgent</p> <p>You should see output similar to this:</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/3125.image_5F00_41DAEA49.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7411.image_5F00_thumb_5F00_3ABBADD1.png" width="291" height="113" /></a> </p> <p>To enable page combining on Windows Server 2012, run the following command:</p> <p>Enable-MMAgent –PageCombining</p> <p>Get-MMAgent</p> <p>You should see output similar to this:</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5367.image_5F00_339C7159.png"><img title="image" style="display: inline;" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6837.image_5F00_thumb_5F00_1381649C.png" width="415" height="135" /></a></p> <p>I did not get a prompt to reboot, so I assume this is working now.</p> <p>If you enable this feature, then please let me know what kind of impact it has on your solution. I am particularly interested in the value of the counter <strong>\Memory\Available MBytes</strong> before and after testing or real world load. Keep in mind that I suspect that the <strong>\Process(*)\Working Set</strong> sizes will remain the same simply because the physical page in RAM will be counted in the working sets normally, but physically have only one real page in RAM.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3535579&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Can a process be limited on how much physical memory it uses?http://blogs.technet.com/b/clinth/archive/2012/10/11/can-a-process-be-limited-on-how-much-physical-memory-it-uses.aspxThu, 11 Oct 2012 16:43:00 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:270a0ab7-e12f-4125-9c33-025ffbb0cc2bClint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3525537http://blogs.technet.com/b/clinth/archive/2012/10/11/can-a-process-be-limited-on-how-much-physical-memory-it-uses.aspx#comments<p>I've been asked a lot of great questions lately and thought I'd post some of them.</p>
<p>As you might know, I am one of the instructors of the popular workshop "Vital Signs" which teaches students Windows architecture and how to identify performance bottlenecks. One of the instructors recently had a student who asked if a process (application) can be limited on the amount of physical memory (RAM) that it can use. The answer is it largely&nbsp;depends on the overall physical memory usage, but it can be limited.</p>
<p>The amount of physical memory that a process uses is called Working Set. The operating system's memory manager (referring to Microsoft Windows and Microsoft Windows Server) controls that amount of physical memory that a process uses by expanding and trimming the working set size of the process. The process itself has little control over this, but this model allows the operating system to manage physical resources more efficiently by only allowing memory that is most actively touched (read or written to) in physical memory.</p>
<p>If you use a tool like Process Explorer (<a href="http://live.sysinternals.com/procexp.exe">http://live.sysinternals.com/procexp.exe</a>), you can have it show the field Max Working Set. This is the amount of physical memory that the process believes it will need and it provides a suggestion to the operating system as to how large or small the working set of the process should be. For the most part though, maximum working set is largely ignored. The reason for this is that if the operating system has plenty of available physical memory, then it will allows frequently accessed pages of a working set to stay in physical memory simply because it is more efficient to keep it there.</p>
<p>According to the <a title="Sysinternals Administrators Reference" href="http://technet.microsoft.com/en-us/sysinternals/hh290819.aspx">Sysinternals Administrators Reference</a> book, here&nbsp;are the definitions of the working set fields:</p>
<ul>
<li><strong>Minimum Working Set.</strong>&nbsp;The amount of physical memory reserved for the process; the operating system guarantees that the process&rsquo; working set can always be assigned at least this amount. The process can also lock pages in the working set up to that amount minus eight pages. This minimum does not guarantee that the process&rsquo; working set will always be at least that large, unless a hard limit has been set by a resource management application.</li>
<li><strong>Maximum Working Set.</strong>&nbsp;Indicates the maximum amount of working set assigned to the process. However, this number is ignored by Windows unless a hard limit has been configured for the process by a resource management application.</li>
<li><strong>Working Set Size.</strong>&nbsp;The amount of physical memory assigned to the process by the memory manager.</li>
</ul>
<p>Ref: Russinovich, Mark E.; Aaron Margosis (2011-06-29). Windows&reg; Sysinternals Administrator&rsquo;s Reference (p. 59).</p>
<p>Also, per David Solomon (one of the authors of the <a title="Windows Internals" href="http://technet.microsoft.com/en-us/sysinternals/bb963901.aspx">Windows Internals</a> series of book), Process Explorer does not show if the process has a hard or soft max or min [working set] set. Also, he suggests using the Windows API, <a title="SetProcessWorkingSetSizeEx" href="http://msdn.microsoft.com/en-us/library/windows/desktop/ms686237(v=vs.85).aspx">SetProcessWorkingSetSizeEx</a>, to set a hard working set size.</p>
<p>With that said, there is a tool called the Windows System Resource Manager which can limit the amount of working set that a process uses. This tool is installable (not installed by default) through the Add Features console on Windows Server 2008 R2.</p>
<p>For more information on this subject, I recommend watching Mark Russinovich's, "<a title="Mysteries of Memory Management Revealed" href="http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/WCL405">Mysteries of Memory Management Revealed</a>".</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3525537&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Windows performance analysisMemoryMy IIS7 PowerShell Scriptshttp://blogs.technet.com/b/clinth/archive/2011/12/04/my-iis7-powershell-scripts.aspxSun, 04 Dec 2011 10:08:02 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:f0b6bb0d-ecbc-4c99-9531-3479a2032bb9Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3468833http://blogs.technet.com/b/clinth/archive/2011/12/04/my-iis7-powershell-scripts.aspx#comments<p>I regularly go onsite with enterprise customers of Microsoft and do Microsoft Internet Information Services (IIS) health checks. Recently, I have been rewriting many of my VBScripts into PowerShell scripts to help make the health check easier. I will be writing more as I go. In the meantime, I hope that you will find these scripts useful.</p> <h3>General features</h3> <ul> <li><strong>Alternate credentials:</strong> They permit the use of different credentials against remote IIS7 servers.</li> <li><strong>Encrypted data transfers:</strong> Use WMI remote-ing and encryption: Most of the scripts use remote WMI calls which use DCOM. I set them to always use encryption on these connections to protect sensitive data collection over the wire. Warning: Log-EphemeralPortStats.ps1 uses PsExec for remote-ing and sends the password over the network in the clear only when changing credentials. Using a domain account with admin rights will not send the credentials in the clear.</li> <li><strong>Useful output:</strong> Many of the scripts output to a comma separated value (csv) file or an XML file. This allows easy post analysis using Microsoft Excel or Microsoft Internet Explorer to view the collected data.</li> </ul> <p>Please understand that I am publishing these scripts to the open community so that you might be able to self-help yourself and have an even better experience with Microsoft products. These script are provided “as-is” as sample code and are not supported. For more information on Microsoft Internet Information Services and how to automate it, go to <a href="http://learn.iis.net">http://learn.iis.net</a>.</p> <h5>Get-NtfsPermissionsOfIisContentToCsv.ps1</h5> <p>This script gets the discretionary access control lists (DACLs) from the physical paths of IIS7 web sites and virtual directories and writes them to a comma separated file (CSV) for post-analysis such as auto-filter in Microsoft Excel. This script requires remote WMI connectivity to all of the servers specified. WMI uses Remote Procedure Calls (RPC) which uses random network ports. The WMI connections are encrypted when possible.</p> <h5>Get-NtfsPermissionsOfIisFoldersToCsv.ps1</h5> <p>Gets the discretionary access control lists (DACLs) from the physical paths of IIS7 operational folders and writes them to a comma separated file (CSV) for post-analysis such as auto-filter in Microsoft Excel. This script requires remote WMI connectivity to all of the servers specified. WMI uses Remote Procedure Calls (RPC) which uses random network ports. The WMI connections are encrypted when possible.</p> <h5>Get-WebAdministrationToXml.ps1</h5> <p>Gets all of the WMI data of one or more IIS7 servers and writes it to a single XML document. This data comes from the root\WebAdministration WMI namespace. Requires administrator rights on the target server(s). The WMI connections are encrypted when possible. This is probably the most powerful of all of the scripts simply because it gets everything related to IIS from each IIS server.</p> <h5>Get-ParentPaths.ps1</h5> <p>Gets the ASP Parent Path setting for all web sites for all IIS7 servers. This scripts uses the root\WebAdministration WMI namespace. Requires administrator rights on the target server(s). The WMI connections are encrypted when possible.</p> <h5>Get-SecurityGroupMembership.ps1</h5> <p>This script is not specific to IIS7, but helps with checking operating system security health. This script gets the membership of security groups from one or more computers and writes them to a comma separated file (CSV) for post-analysis such as auto-filter in Microsoft Excel. This script requires remote WMI connectivity to all of the servers specified. WMI uses Remote Procedure Calls (RPC) which uses random network ports. The WMI connections are encrypted when possible.</p> <h5>Set-EnableAllW3cFields.ps1</h5> <p>Enables all of the W3C logging fields on all web sites of one or more IIS7 servers. This scripts uses the root\WebAdministration WMI namespace. Requires administrator rights on the target server(s). The WMI connections are encrypted when possible. This script is helpful to prepare IIS servers for intensive W3C log analysis.</p> <h5>Log-EphemeralPortStats.ps1</h5> <p>This script is not specific to IIS7, but helps with checking operating system network health. Runs in an infinite loop getting the TCP ephemeral port and listening port statistics for each local IP address and outputs the data to a text file log. The script writes the ephemeral port stats every 60 seconds by default. To get data from remote computers, this script requires PsExec.exe (SysInternals) to be in the same directory as this script. WARNING: Credentials passed into PSExec are sent over the network in clear text! Prevent this by logging in interactively with a domain account that has administrator rights on the target computers and not specifying credentials to this script. PsExec is a Sysinternals tool owned by Microsoft Corporation. PsExec can be downloaded for free at <a href="http://live.sysinternals.com/psexec.exe">http://live.sysinternals.com/psexec.exe</a>. </p> <h4>Download</h4> <p>These scripts are available on my SkyDrive at: <br /><a title="https://skydrive.live.com/?cid=e6360c54b48a891b&amp;sc=documents&amp;id=E6360C54B48A891B%21428#cid=E6360C54B48A891B&amp;id=E6360C54B48A891B%21964&amp;sc=documents" href="https://skydrive.live.com/?cid=e6360c54b48a891b&amp;sc=documents&amp;id=E6360C54B48A891B%21428#cid=E6360C54B48A891B&amp;id=E6360C54B48A891B%21964&amp;sc=documents">https://skydrive.live.com/?cid=e6360c54b48a891b&amp;sc=documents&amp;id=E6360C54B48A891B%21428#cid=E6360C54B48A891B&amp;id=E6360C54B48A891B%21964&amp;sc=documents</a></p> <p>Enjoy!</p> <p><b>Clint Huffman</b> |<b> </b><b>Senior Premier Field Engineer </b>| <b>Microsoft Services |</b> <a href="mailto:clinth@microsoft.com">clinth@microsoft.com</a> | TS:Windows Internals <table border="0" cellspacing="0" cellpadding="0"><tbody> <tr> <td valign="top" width="154"> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4621.clip_5F00_image002_5F00_26294553.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/6371.clip_5F00_image002_5F00_thumb_5F00_6CA6355B.jpg" width="144" height="105" /></a></p> </td> <td valign="top" width="194"> <p><b>Microsoft Tag:</b> Download my contact information to your phone. </p> <p>Get the free app for your phone at <a href="http://gettag.mobi/">http://gettag.mobi</a></p> </td> <td valign="top" width="145"> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/8422.clip_5F00_image004_5F00_21469197.gif"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image004" border="0" alt="clip_image004" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7457.clip_5F00_image004_5F00_thumb_5F00_7A0C4861.gif" width="68" height="68" /></a><a href="http://twitter.com/clinth"><b><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image006" border="0" alt="clip_image006" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/4718.clip_5F00_image006_5F00_6BCDCF71.gif" width="21" height="21" /></b></a><b> </b><a href="http://www.linkedin.com/pub/clint-huffman/1/606/485"><b><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image008" border="0" alt="clip_image008" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/5126.clip_5F00_image008_5F00_4089386A.gif" width="21" height="21" /></b></a> <a href="http://www.facebook.com/clint.huffman"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="clip_image010" border="0" alt="clip_image010" src="http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-75-73-metablogapi/7853.clip_5F00_image010_5F00_5265CC37.gif" width="21" height="21" /></a><b> </b></p> </td> </tr> </tbody></table></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3468833&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">How to Speak SAN-ishhttp://blogs.technet.com/b/clinth/archive/2011/05/13/how-to-speak-san-ish.aspxSat, 14 May 2011 00:16:06 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:81cf179e-f515-48b3-8ba6-47860f6032f9Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3429148http://blogs.technet.com/b/clinth/archive/2011/05/13/how-to-speak-san-ish.aspx#comments<p>I recently signed a contract with MCP Magazine to publish articles. This is effectively a syndication of this blog.</p> <p>Check out my first article called, “How to Speak SAN-ish” at <br /><a title="http://mcpmag.com/articles/2011/05/12/how-to-speak-san-ish.aspx" href="http://mcpmag.com/articles/2011/05/12/how-to-speak-san-ish.aspx">http://mcpmag.com/articles/2011/05/12/how-to-speak-san-ish.aspx</a></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3429148&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">The Microsoft TechNet Wiki Performance Guide (PerfGuide)http://blogs.technet.com/b/clinth/archive/2011/03/28/the-microsoft-technet-wiki-performance-guide-perfguide.aspxMon, 28 Mar 2011 20:06:28 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:86474c69-8e6d-4363-b88a-5221a350f6dcClint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3416606http://blogs.technet.com/b/clinth/archive/2011/03/28/the-microsoft-technet-wiki-performance-guide-perfguide.aspx#comments<p>One of the major reasons why I haven’t been blogging much is because I have spent most of my “free” time working on the Microsoft TechNet Wiki writing what I call the “PerfGuide”. We have a lot of great content up there for Windows performance analysis.</p> <p>The Microsoft PFE Performance Guide (PerfGuide): Start Here <br /><a title="http://social.technet.microsoft.com/wiki/contents/articles/the-microsoft-pfe-performance-guide-perfguide-start-here.aspx" href="http://social.technet.microsoft.com/wiki/contents/articles/the-microsoft-pfe-performance-guide-perfguide-start-here.aspx">http://social.technet.microsoft.com/wiki/contents/articles/the-microsoft-pfe-performance-guide-perfguide-start-here.aspx</a></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3416606&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">The PAL Tool on Memory Leakshttp://blogs.technet.com/b/clinth/archive/2011/03/28/the-pal-tool-on-memory-leaks.aspxMon, 28 Mar 2011 19:01:52 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:a53a50c2-4d1a-475f-a4ef-915daea46cf3Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3416585http://blogs.technet.com/b/clinth/archive/2011/03/28/the-pal-tool-on-memory-leaks.aspx#comments<p>To prevent a process from crashing due to a System.OutOfMemory condition, the .NET garbage collector (GC) automatically defragments virtual memory. It can only do this for Gen 0 and Gen 1 memory allocations. Any allocations at 64 KB or larger will go to the large object heap. The large object heap and any non-managed objects (COM, C++, etc.) cannot be defragmented by the GC.</p> <p>I recently published an article on the Microsoft Technet Wiki PerfGuide on diagnosing process virtual memory issues. <br /><strong>PerfGuide: Out of Process Virtual Memory</strong> <br /><a href="http://social.technet.microsoft.com/wiki/contents/articles/perfguide-out-of-process-virtual-memory.aspx">http://social.technet.microsoft.com/wiki/contents/articles/perfguide-out-of-process-virtual-memory.aspx</a></p> <p>In any case, PAL is looking for a gradual and significant increase in process committed memory for which the operating system must provide system committed resources (physical RAM and/or page file). The committed memory of each process can be measured using the “\Process(*)\Private Bytes” counter. The amount of committed memory of a process is dictated by the process’s usage of memory – coded by the developer of the application. The .NET GC also removes/deallocates variables that are out of scope (such as local variables in a function that is no longer executing), but ultimately, the application developer determines which variables are in use. Therefore, we diagnose the memory usage of the application threw debugging or profiling to determine where the memory usage goes. My point is that, yes, you can have a memory leak even if you are using pure .NET. The GC just makes it less likely to happen.</p> <p>With all of that said, memory leaks need to be looked at over a long period of time because when an application is busy, it will naturally use more memory. A memory leak is when the memory accumulates unnecessarily. </p> <p>If a process has plenty of virtual memory (x64 has 8 TB of virtual memory per process) and if the leak continues unchecked, then the system might eventually run out of system commit memory which is the total amount of RAM and all of the page files combined. For more information on this topic, see the following PerfGuide post.</p> <p>PerfGuide: Out of System Committed Memory <br /><a href="http://social.technet.microsoft.com/wiki/contents/articles/perfguide-out-of-system-committed-memory.aspx">http://social.technet.microsoft.com/wiki/contents/articles/perfguide-out-of-system-committed-memory.aspx</a> </p> <p>We [Microsoft] typically work with applications in production, so we commonly use DebugDiag or WinDBG to analyze memory leaks by gathering several dumps (*.dmp) from the target process.</p> <p>DebugDiag <br /><a href="http://www.microsoft.com/downloads/en/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&amp;displaylang=en">http://www.microsoft.com/downloads/en/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&amp;displaylang=en</a></p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3416585&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Choose Your Own Adventure: High Deferred Procedure Calls (DPCs) or High Interruptshttp://blogs.technet.com/b/clinth/archive/2010/09/23/choose-your-own-adventure-high-deferred-procedure-calls.aspxFri, 24 Sep 2010 00:58:58 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:c33096ee-9389-44bf-816a-c15c27b8aaabClint Huffman1http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3357515http://blogs.technet.com/b/clinth/archive/2010/09/23/choose-your-own-adventure-high-deferred-procedure-calls.aspx#comments<p>You have arrived here because you have identified a high amount (greater than 50%) of processor time (\Processor(*)\% Processor Time), a high amount (greater than 30%) of privileged time (kernel time) (\Processor(*)\% Privileged Time) and a high amount of deferred procedure calls (greater than 20%) (\Processor(*)\% DPC Time) or a high number of interrupts (greater than 20%) (\Processor(*)\% Interrupt Time) on your Windows computer. If this is not correct, then return to the <a href="http://blogs.technet.com/clinth/archive/2010/02/20/choose-your-own-adventure-start-here.aspx">Start of the Adventure</a>.</p> <p>% DPC Time is the percentage of time that the processor spent receiving and servicing deferred procedure calls (DPCs) during the sample interval. DPCs are interrupts that run at a lower priority than standard interrupts. % DPC Time is a component of % Privileged Time because DPCs are executed in privileged mode. They are counted separately and are not a component of the interrupt counters.</p> <p> <p><a href="http://www.windowsitpro.com/print/performance/examining-xperf.aspx"></a></p> % Interrupt Time is the time the processor spends receiving and servicing hardware interrupts during sample intervals. This value is an indirect indicator of the activity of devices that generate interrupts, such as the system clock, the mouse, disk drivers, data communication lines, network interface cards and other peripheral devices. These devices normally interrupt the processor when they have completed a task or require attention. Normal thread execution is suspended during interrupts. Most system clocks interrupt the processor every 10 milliseconds, creating a background of interrupt activity. suspends normal thread execution during interrupts. This counter displays the average busy time as a percentage of the sample time.</p> <p>High Deferred Procedure Calls (DPCs) and high interrupts are typically caused by very busy or poorly written device drivers. Consider using a tool such as Microsoft xPerf to assist with diagnosis of this problem.</p> <p>Here is an article by Windows IT Pro Magazine that goes into diagnosing DPC issues using Microsoft xPerf:</p> <p>Windows IT Pro Magazine: Examining xPerf <br /><a href="http://www.windowsitpro.com/print/performance/examining-xperf.aspx">http://www.windowsitpro.com/print/performance/examining-xperf.aspx</a></p> <p><strong>Tip: </strong>If the computer hangs often, then while it is hung, press the Cap Lock key on the keyboard to see if the light on the keyboard toggles on and off when you press it. If the Cap Lock key light does not respond, then the computer is hung due to a hardware problem. Otherwise, it is most likely a software problem.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3357515&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">My Incredible Week at TechEd 2010http://blogs.technet.com/b/clinth/archive/2010/06/11/my-incredible-week-at-teched-2010.aspxSat, 12 Jun 2010 00:24:04 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:70ddb91b-ee76-4760-9bba-d9c30593e1b2Clint Huffman1http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3337727http://blogs.technet.com/b/clinth/archive/2010/06/11/my-incredible-week-at-teched-2010.aspx#comments<p>I attended TechEd 2010 this week and it was great! I met a lot of great people and had a great time. TechEd was *huge* this year. They originally expected about 6,000 attendees, but about 8,000 actually attended. To say the least it was very crowded and the sessions were over flowing with people. If you didn’t get to the session early enough, then you would likely be denied. Believe me I missed a few sessions to due this. Overflow rooms were provided, so attendees could watch some of the sessions remotely. I’m actually impressed with the TechEd team in how they handled the overflow.</p> <h3>Met the Hyper-V Product Team</h3> <p>At TechEd, I met Vijay Tewari who owns Hyper-V. He and I worked the booth a lot and I learned quite a bit about the new features in Hyper-V R2 SP1 (currently in beta). I was also honored to meet Taylor Brown (Hyper-V Tester). He and I “geeked” out on many of the Hyper-V features such as Virtual Memory Queues (VMQ) networking.</p> <h3>Hyper-V RemoteFX</h3> <p>Windows Server 2008 R2 SP1 Hyper-V (aka Hyper-V R2) has two significant features that I am aware of. First, it has the RemoteFX feature which allows 3D graphics to work inside of virtual guests. Normally, any game or application that requires 3D video hardware acceleration would not work inside of a virtual guest, but RemoteFX with a special video card that can support remote video acceleration can run 3D graphics now. We had a simple example of a 3D chess game in our demo, but I’m curious to see how other graphics intensive games will do. If this works well, then I might have to consider running Hyper-V R2 SP1 on my laptop – I have to have my games after all.</p> <h3>Hyper-V Dynamic Memory</h3> <p>Dynamic Memory is another feature of Hyper-V R2 that will make a *huge* impact on the industry. It allows virtual guests to expand and contract the amount of RAM that is allocated to them. This allows the root partition (host computer) to use it’s RAM much more effectively and allows it to run far more virtual guests. It is implemented by setting a minimum and a maximum RAM size. Once the virtual guest is rebooted, it starts off at it’s minimum RAM size, then the root partition will increase the virtual guest’s RAM size as memory demands increase. Once the memory demands have decreased, then the RAM size is reduced allowing the root partition to give the RAM to other needy virtual guests. </p> <p>If the root partition is completely out of RAM, then it will take away RAM from other VMs. As an administrator, you can prioritize your VM’s so that higher priority VMs get first dibs on the RAM.</p> <p>In Ben Armstrong’s (a member of the Hyper-V product team) presentation on Hyper-V R2 SP1 Dynamic Memory presentation, he had 12 virtual guests (VMs) all with 1GB of RAM allocated. The host partition only has 8GBs of RAM, so he can only run 7 of the VM’s – the 8th VM refuses to run due to a lack of RAM on the host. He enabled Dynamic Memory setting each VM to a minimum of 512MBs and a maximum of 2GBs. All of the VMs are restarted (this is required) and they start off at 512MBs of RAM. Since the VMs are all using about half the RAM they originally had, he can now run all 12VMs at the same time. This gives Hyper-V R2 SP1 far more VM density. He also showed how the VM priority takes effect when the root partition is out of memory. Great stuff!</p> <p>VM’s can be set to a percentage of available RAM which the root partition will try to maintain. If the available RAM of the VM is lower than the setting, then RAM is added – if there is too much available RAM, then it gives RAM back to the host.</p> <p>It is important to allow Windows to have available RAM. Available RAM is the sum of zeroized memory (memory that has all zeroes written to it, free memory (memory that was used by a process and hasn’t been zeroized yet) and standby memory. Standby memory is memory that was in use by a process at one point, but contains data that is already on the disk – meaning Standby memory is disk cache. This cache can prevent disk reads and time by having in RAM still. This is why it is always good to give a system lots of RAM. By giving the system about 10% RAM, you are allowing the system to maintain a health disk cache. </p> <p>With Dynamic Memory comes more performance counters, so I plan in adding these counters and threshold to my Performance Analysis of Logs (PAL) tool within the next few weeks. As a matter of fact, I will be meeting with the Hyper-V team next week to start on this. I am very excited.</p> <p>I ran into Richard Campbell who is the host of the RunAs Radio podcast and known for web performance analysis. I introduced him to Vijay Tewari (owner of Hyper-V) and it is likely that Vijay will be a guest on the show soon. Watch for it at <a href="http://www.runasradio.com">http://www.runasradio.com</a>. </p> <h3>Performance Counter Book Idea</h3> <p>Richard mentioned that he is interested in writing a book on performance counters. I’ve been itching to do one as well, so we are thinking about writing the book together. Book writing is extremely time consuming, so its just and idea right now.</p> <h3>.NET MultiThreaded By Default</h3> <p>Richard and I also talked about the need for .NET objects to be natively multi-threaded to take advantage of more than one processor. My idea was that the .NET functions should spawn off multiple threads by default if/when resource locking it not an issue.</p> <h3>Meeting Jeffery Snover (PowerShell Architect)</h3> <p>Another highlight for me was meeting Jeffrey Snover face to face. He is the original architect of PowerShell. Him and I talked quite a bit about my <a href="http://pal.codeplex.com" target="_blank">PAL</a> v2.0 tool which is a 5,000 line PowerShell script. The PAL tool analyzes performance counter logs and PowerShell makes it easy to implement. Jeffrey contributed to the tool back in January 2010 by providing a few functions to force PowerShell v2.0. He is very approachable and even remembered my name. :-) His most notable feature was his “minimum” shoes (shoes that cover the feet, but have minimum protection – better for your feet in the long run) while dressed in business casual clothes. It was truly an honor to meet the guy who started it all.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-75-73-metablogapi/0486.0609101649a_5F00_2.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="0609101649a" border="0" alt="0609101649a" src="http://blogs.technet.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-75-73-metablogapi/8322.0609101649a_5F00_thumb.jpg" width="244" height="184" /></a> </p> <p>Here is a picture of Jeffrey Snover (left) and I (right).</p> <h3>PowerWF</h3> <p>One of the vendors at the show that impressed me was <a href="http://www.powerwf.com" target="_blank">PowerWF</a>. Their PowerWF tool converts Work Flow (WF) diagrams to PowerShell code and converts code to WF. This is really handy for documenting your PowerShell code to non-developers and it allows non-developers the power to create PowerShell scripts without knowing code. Jeffrey Snover stopped by PowerWF booth several times and the “geeked” out a lot.</p> <h3>21 Tips to a Better Performing Hyper-V Solution</h3> <p>Kenon Owens (a member of the Hyper-V product team) and I presented a breakout session on Hyper-V performance analysis and the performance differences between Hyper-V R1 and Hyper-V R2. We had a great turnout for it being an 8am session and we received some great evaluations. I have to credit Tony Voellm as the source for most of my content. I will write my own blog entry soon on the same subject to get the content out to everyone.</p> <p><a href="http://blogs.technet.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-75-73-metablogapi/3443.image_5F00_2.png"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="image" border="0" alt="image" src="http://blogs.technet.com/cfs-file.ashx/__key/CommunityServer-Blogs-Components-WeblogFiles/00-00-00-75-73-metablogapi/3036.image_5F00_thumb.png" width="244" height="139" /></a> </p> <h3>The BizTalk Booth</h3> <p>When I attended TechEd 2008, I helped with the BizTalk booth. I inevitably stopped by the booth and caught up with my friends there. They showed me a new feature called the Financial Message Services Bus (FMSB). This is an add-on to the Enterprise Services Bus (ESB) toolkit and Business Activity Monitor (BAM). It uses Silverlight to make BAM look good… I mean *really* good. It allows tracking information to be presented in high quality charts and graphs that can be manipulated. I was very impressed. Better yet, it is not directly tied to Financial data, meaning - it can work with any BAM data and it will even configure BAM for you which is a *huge* hurdle for those who have tried it.</p> <h3>Windows Phone 7</h3> <p>I love my Zune and I love my Windows Mobile Phone. Put them together and what to do you got… Windows Phone 7! While we were not permitted to “touch” the prototypes, we got to see the phone close up and see for ourselves how responsive it is and how pretty it is. This thing is a powerful business machine with a slick consumer friendly interface. There was quite a few booths dedicated to business app development for it. It has great Microsoft Office integration allowing what it seems to be full editing features and powerful zooming capabilities for most of the Office suite. I asked about InfoPath support, but it doesn’t look like it is in there. My team Microsoft Premier Field Engineering (PFE) uses InfoPath forms to create our daily status reports when onsite with customers, so having the ability to send updates from my Windows Phone 7 would be very helpful. I am also hoping it will have VPN and Direct Access support. Now that would rock!</p> <h3>Windows Server R2 Kernel Changes</h3> <p>Mark Russinovich did a session on the Windows Server R2 kernel changes. The new kernel has a lot of great features such as interrupt consolidation, much higher scalable resource locking, and my favorite is the core parking feature. Core parking allows some of the processor cores to go into a very low power state saving energy. There are new performance counters that indicate this, so I can’t wait to add those counter to PAL.</p> <h3>Met the Server Ninja, Jacob Beneke</h3> <p>Jacob and I finally met at the TechEd Keynote. It was great to put a name to a face. He created and maintains the ServerNinjas.com web site which talks about tips and tricks for IT Professionals.</p> <h3>PowerShell Advanced Functions and Forms</h3> <p>I attended a great session on PowerShell Advanced functions in PowerShell v2.0 which allows criteria and default values to be added to parameters passed into fuctions. The PowerShell product team showed us how to create Windows forms easily in native PowerShell using Sapien Technologies <a href="http://www.primaltools.com/products/info.asp?p=PrimalForms" target="_blank">PrimalForms</a> tool. I might just have to use this to rewrite the <a href="http://pal.codeplex.com" target="_blank">PAL</a> tools interface.</p> <p>Well, I’m sure I am forgetting something, but these are the highlights of what I thought was cool at TechEd 2010.</p><div style="clear:both;"></div><img src="http://blogs.technet.com/aggbug.aspx?PostID=3337727&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">PAL v2.0’s New Counter Generation Feature is Powerfulhttp://blogs.technet.com/b/clinth/archive/2010/03/02/pal-v2-0-s-new-counter-generation-feature-is-powerful.aspxWed, 03 Mar 2010 00:04:03 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:ebf43508-6441-4116-8e7a-693a9a008079Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3316411http://blogs.technet.com/b/clinth/archive/2010/03/02/pal-v2-0-s-new-counter-generation-feature-is-powerful.aspx#comments<p>So, I’ve been playing with generating counters within PAL to compare ratios of SQL Server Full Scans to Index Searches performance counters. This is an effort to get PAL v2.0 updated and more accurate than PAL v1.x. It works out quite nicely! I feel like an evil genius right now. Mhahahahahaha!!! :-)</p> <p>By the way, PAL v1.x cannot do this. ;-) The counter “\SQLServer:Access Methods\Full Scans to Index Searches Ratio” is a fake counter that is PAL created during analysis.</p> <p><strong>Disclaimer:</strong> The purpose of this posting is to demonstrate PAL’s ability to generate and display counter data. Do not use this posting for SQL performance analysis. As always, be skeptical about blog postings.</p> <p>Here is the analysis I have been playing with.</p> <h4><a name="FullScanstoIndexSearchesRatio">Full Scans to Index Searches Ratio</a></h4> <p><b>Description:</b> This counter monitors the number of full scans on base tables or indexes. Values greater than 1 or 2 indicate that we are having table / Index page scans. If we see high CPU then we need to investigate this counter, otherwise if the full scans are on small tables we can ignore this counter. A few of the main causes of high Full Scans/sec are missing indexes, too many rows requested, queries with missing indexes, or too many rows requested will have a large number of logical reads and an increased CPU time. <br />This analysis throws a Warning alert if the ratio of Index Searches/sec to Full Scans/sec is less than 1000 to 1 and if there are more than 1000 Index Searches/sec. <br />Formula: (AvgSQLServerAccessMethodsIndexSearchessecAll / AvgSQLServerAccessMethods_FullScanssec) &lt; 1000 </p> <p><a href="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/PAL.0sNewCounterGenerationFeatureisPower_E1ED/clip_image001_2.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="clip_image001" border="0" alt="clip_image001" src="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/PAL.0sNewCounterGenerationFeatureisPower_E1ED/clip_image001_thumb.png" width="244" height="184" /></a></p> <p><a href="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/PAL.0sNewCounterGenerationFeatureisPower_E1ED/clip_image002_2.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="clip_image002" border="0" alt="clip_image002" src="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/PAL.0sNewCounterGenerationFeatureisPower_E1ED/clip_image002_thumb.png" width="244" height="184" /></a></p> <p><a href="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/PAL.0sNewCounterGenerationFeatureisPower_E1ED/clip_image003_2.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="clip_image003" border="0" alt="clip_image003" src="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/PAL.0sNewCounterGenerationFeatureisPower_E1ED/clip_image003_thumb.png" width="244" height="184" /></a></p> <h5>Overall Counter Instance Statistics</h5> <table border="0" cellpadding="0"><tbody> <tr> <td> <p>Overall statistics of each of the counter instances. Min, Avg, and Max are the minimum, average, and Maximum values in the entire log. Hourly Trend is the calculated hourly slope of the entire log. 10%, 20%, and 30% of Outliers Removed is the average of the values after the percentage of outliers furthest away from the average have been removed. This is to help determine if a small percentage of the values are extreme which can skew the average. </p> </td> </tr> </tbody></table> <table border="1" cellpadding="0"><tbody> <tr> <td> <p><b>Condition</b><b></b></p> </td> <td> <p><b>\SQLServer:Access Methods\Full Scans/sec</b><b></b></p> </td> <td> <p><b>Min</b><b></b></p> </td> <td> <p><b>Avg</b><b></b></p> </td> <td> <p><b>Max</b><b></b></p> </td> <td> <p><b>Hourly Trend</b><b></b></p> </td> <td> <p><b>Std Deviation</b><b></b></p> </td> <td> <p><b>10% of Outliers Removed</b><b></b></p> </td> <td> <p><b>20% of Outliers Removed</b><b></b></p> </td> <td> <p><b>30% of Outliers Removed</b><b></b></p> </td> </tr> <tr> <td> <p>OK</p> </td> <td> <p>VSTP24/TP24PRD</p> </td> <td> <p>0</p> </td> <td> <p>7</p> </td> <td> <p>377</p> </td> <td> <p>-11</p> </td> <td> <p>45</p> </td> <td> <p>0</p> </td> <td> <p>0</p> </td> <td> <p>0</p> </td> </tr> </tbody></table> <table border="1" cellpadding="0"><tbody> <tr> <td> <p><b>Condition</b><b></b></p> </td> <td> <p><b>\SQLServer:Access Methods\Index Searches/sec</b><b></b></p> </td> <td> <p><b>Min</b><b></b></p> </td> <td> <p><b>Avg</b><b></b></p> </td> <td> <p><b>Max</b><b></b></p> </td> <td> <p><b>Hourly Trend</b><b></b></p> </td> <td> <p><b>Std Deviation</b><b></b></p> </td> <td> <p><b>10% of Outliers Removed</b><b></b></p> </td> <td> <p><b>20% of Outliers Removed</b><b></b></p> </td> <td> <p><b>30% of Outliers Removed</b><b></b></p> </td> </tr> <tr> <td> <p>OK</p> </td> <td> <p>VSTP24/TP24PRD</p> </td> <td> <p>27</p> </td> <td> <p>3,478</p> </td> <td> <p>54,437</p> </td> <td> <p>-6,486</p> </td> <td> <p>8,188</p> </td> <td> <p>1,076</p> </td> <td> <p>411</p> </td> <td> <p>325</p> </td> </tr> </tbody></table> <table border="1" cellpadding="0"><tbody> <tr> <td> <p><b>Condition</b><b></b></p> </td> <td> <p><b>\SQLServer:Access Methods\Full Scans to Index Searches Ratio</b><b></b></p> </td> <td> <p><b>Min</b><b></b></p> </td> <td> <p><b>Avg</b><b></b></p> </td> <td> <p><b>Max</b><b></b></p> </td> <td> <p><b>Hourly Trend</b><b></b></p> </td> <td> <p><b>Std Deviation</b><b></b></p> </td> <td> <p><b>10% of Outliers Removed</b><b></b></p> </td> <td> <p><b>20% of Outliers Removed</b><b></b></p> </td> <td> <p><b>30% of Outliers Removed</b><b></b></p> </td> </tr> <tr> <td> <p>Ratio of Index searches/sec to Full scan/sec less than 1000 to 1.</p> </td> <td> <p>VSTP24/TP24PRD</p> </td> <td> <p>0</p> </td> <td> <p>247</p> </td> <td> <p>28,487</p> </td> <td> <p>-155</p> </td> <td> <p>2,265</p> </td> <td> <p>0</p> </td> <td> <p>0</p> </td> <td> <p>0</p> </td> </tr> </tbody></table> <h5>Alerts</h5> <table border="0" cellpadding="0"><tbody> <tr> <td> <p>An alert is generated if any of the thresholds were broken during one of the time ranges analyzed. The background of each of the values represents the highest priority threshold that the value broke. See each of the counter's respective analysis section for more details about what the threshold means. </p> </td> </tr> </tbody></table> <table border="1" cellpadding="0"><tbody> <tr> <td> <p><b>Time Range</b><b></b></p> </td> <td>&#160;</td> <td>&#160;</td> <td>&#160;</td> <td>&#160;</td> <td>&#160;</td> <td>&#160;</td> </tr> <tr> <td> <p><b><a href="#TimeRange_12182007110718AM1218200711084">12/18/2007 11:07:18 AM - 12/18/2007 11:08:49 AM</a></b><b></b></p> </td> <td> <p><b>Condition</b><b></b></p> </td> <td> <p><b>Counter</b><b></b></p> </td> <td> <p><b>Min</b><b></b></p> </td> <td> <p><b>Avg</b><b></b></p> </td> <td> <p><b>Max</b><b></b></p> </td> <td> <p><b>Hourly Trend</b><b></b></p> </td> </tr> <tr> <td>&#160;</td> <td> <p>Ratio of Index searches/sec to Full scan/sec less than 1000 to 1.</p> </td> <td> <p><a href="file:///\\VSTP24\MSSQL$TP24PRD:Access">\\VSTP24\MSSQL$TP24PRD:Access</a> Methods\Full Scans to Index Searches Ratio</p> </td> <td> <p>0</p> </td> <td> <p>928</p> </td> <td> <p>5,565</p> </td> <td> <p>17,957</p> </td> </tr> <tr> <td> <p><b><a href="#TimeRange_12182007112112AM1218200711224">12/18/2007 11:21:12 AM - 12/18/2007 11:22:43 AM</a></b><b></b></p> </td> <td> <p><b>Condition</b><b></b></p> </td> <td> <p><b>Counter</b><b></b></p> </td> <td> <p><b>Min</b><b></b></p> </td> <td> <p><b>Avg</b><b></b></p> </td> <td> <p><b>Max</b><b></b></p> </td> <td> <p><b>Hourly Trend</b><b></b></p> </td> </tr> <tr> <td>&#160;</td> <td> <p>Ratio of Index searches/sec to Full scan/sec less than 1000 to 1.</p> </td> <td> <p><a href="file:///\\VSTP24\MSSQL$TP24PRD:Access">\\VSTP24\MSSQL$TP24PRD:Access</a> Methods\Full Scans to Index Searches Ratio</p> </td> <td> <p>0</p> </td> <td> <p>589</p> </td> <td> <p>3,532</p> </td> <td> <p>1,767</p> </td> </tr> <tr> <td> <p><b><a href="#TimeRange_12182007113908AM1218200711403">12/18/2007 11:39:08 AM - 12/18/2007 11:40:39 AM</a></b><b></b></p> </td> <td> <p><b>Condition</b><b></b></p> </td> <td> <p><b>Counter</b><b></b></p> </td> <td> <p><b>Min</b><b></b></p> </td> <td> <p><b>Avg</b><b></b></p> </td> <td> <p><b>Max</b><b></b></p> </td> <td> <p><b>Hourly Trend</b><b></b></p> </td> </tr> <tr> <td>&#160;</td> <td> <p>Ratio of Index searches/sec to Full scan/sec less than 1000 to 1.</p> </td> <td> <p><a href="file:///\\VSTP24\MSSQL$TP24PRD:Access">\\VSTP24\MSSQL$TP24PRD:Access</a> Methods\Full Scans to Index Searches Ratio</p> </td> <td> <p>0</p> </td> <td> <p>4,835</p> </td> <td> <p>28,487</p> </td> <td> <p>8,215</p> </td> </tr> </tbody></table> <p>PAL v2.0 is still in Alpha right now, but give it a try and send me your feedback. <a href="http://pal.codeplex.com">http://pal.codeplex.com</a>. </p> <p>The thresholds used in PAL analysis is based on the SQL performance experience of David Pless. David and I are both in the Microsoft Premier Field Engineering (PFE) organization.</p><img src="http://blogs.technet.com/aggbug.aspx?PostID=3316411&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">Choose Your Own Adventure: User Mode Versus Privileged Mode Processor Usagehttp://blogs.technet.com/b/clinth/archive/2010/02/20/choose-your-own-adventure-user-mode-versus-privileged-mode-processor-usage.aspxSat, 20 Feb 2010 09:20:58 GMTd5e57398-b9ef-4490-9955-07cbb4e4a80d:80a60dbe-9545-4409-9448-bb948f602ca4Clint Huffman0http://blogs.technet.com/b/clinth/rsscomments.aspx?WeblogPostID=3314207http://blogs.technet.com/b/clinth/archive/2010/02/20/choose-your-own-adventure-user-mode-versus-privileged-mode-processor-usage.aspx#comments<p>You have arrived at this point in the adventure because you have identified high processor usage on your Windows computer. If this is not correct, then return to the <a href="http://blogs.technet.com/clinth/archive/2010/02/20/choose-your-own-adventure-start-here.aspx">Start of the Adventure</a>.</p> <p>Threads (the worker bees of a process) can execute in one of two modes: User Mode or Privileged Mode</p> <p>This article will help you determine what kind of processor mode your computer or server is spending most of it’s time in. This is important because this is a major decision path in the adventure and change change the troubleshooting steps and the outcome dramatically.</p> <p>“\Processor(_Total)\% Processor Time” is the sum of “\Processor(*)\% User Time” and “\Processor(*)\% Privileged Time”. </p> <h3>Privileged (Kernel) Mode</h3> <p>Privileged Time is the amount of time being spent in the Windows kernel executing system calls such as drivers, IRPs (I/O Request Packets), context switching, etc. If the operating system is spending more than 30% of it’s time in privileged mode, then it means that it is likely doing a high amount of I/O and one or more of the drivers are executing to manage that I/O. </p> <p>You can identify this by using Task Manager, clicking on the Performance tab, then go to View, Show Kernel Times or by using the “\Processor(*)\% Privileged Time” performance counter in Performance Monitor.</p> <p>The following screenshot shows a high amount of privileged time processor usage.</p> <p><a href="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/ChooseYourOwnAdventureUserModeVersusPriv_B44/image_2.png"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="image" border="0" alt="image" src="http://blogs.technet.com/blogfiles/clinth/WindowsLiveWriter/ChooseYourOwnAdventureUserModeVersusPriv_B44/image_thumb.png" width="190" height="244" /></a> </p> <p>If your computer or server is spending more than 30% of it’s time in privileged mode, then look at a % DPC Time, % Interrupt Time, and Context Switches/sec in performance monitor.</p> <p>If “\Processor(*)\% DPC Time” is greater than 20%, then go <a href="http://blogs.technet.com/b/clinth/archive/2010/09/23/choose-your-own-adventure-high-deferred-procedure-calls.aspx">here</a>.</p> <p>If “\Processor(*)\% Interrupt Time” is greater then 20%, then go <a href="http://blogs.technet.com/b/clinth/archive/2010/09/23/choose-your-own-adventure-high-deferred-procedure-calls.aspx">here</a>.</p> <p>If both of the counters above show very low values (less than 20%), then it could also be caused by high amounts of context switches per second.</p> <p>Here is a case study on how to solve high context switching due to high disk I/O.</p> <p>The Case of the 2 Million Context Switches <br /><a title="http://blogs.technet.com/b/clinth/archive/2009/10/28/the-case-of-the-2-million-context-switches.aspx" href="http://blogs.technet.com/b/clinth/archive/2009/10/28/the-case-of-the-2-million-context-switches.aspx">http://blogs.technet.com/b/clinth/archive/2009/10/28/the-case-of-the-2-million-context-switches.aspx</a></p> <p>With that said, high context switching can also be caused by a high number of ready threads on the system, so reducing the number of threads can help as well.</p> <p>Here is a good article by Mark Russinovich on how to use the driver SDK tool, KernRate, to profile the kernel. KernRate only works on Windows XP/2003 computers. <br /><a href="http://blogs.technet.com/markrussinovich/archive/2008/04/07/3031251.aspx">Mark Russinovich’s The Case of the System Process CPU Spikes</a></p> <h3>User Mode</h3> <p>User Time is the amount of time the processor spends executing application code therefore we need to determine what processes are consuming the most time and the function calls they are executing the most.</p> <p>You can identify this by using Task Manager, clicking on the Performance tab, then go to View, Show Kernel Times or by using the “\Processor(*)\% User Time” performance counter in Performance Monitor.</p> <p>The following screenshot shows a high amount of user time processor usage.</p> <p><img title="HighCPU4.GIF" alt="HighCPU4.GIF" src="http://i3.codeplex.com/Project/Download/FileDownload.aspx?ProjectName=PerfTesting&amp;DownloadId=6400" /> <br /><img title="HighCPU5.GIF" alt="HighCPU5.GIF" src="http://i3.codeplex.com/Project/Download/FileDownload.aspx?ProjectName=PerfTesting&amp;DownloadId=6401" /></p> <p>If your computer or server is spending most of it’s time in user mode, then follow this link on how to troubleshoot this issue: <br /><a href="http://perftesting.codeplex.com/wikipage?title=How%20To%3a%20Identify%20a%20Disk%20Performance%20Bottleneck%20Using%20SPA&amp;version=6">How To: Identify Functions causing a High User-mode CPU Bottleneck for Server Applications in a Production Environment</a></p><img src="http://blogs.technet.com/aggbug.aspx?PostID=3314207&AppID=7573&AppType=Weblog&ContentType=0" width="1" height="1">