Just wanted to list out some of the tools I use on a daily basis and some of the more common parameters I use with each. I've been doing enterprise wide administrative support for a while so these should come in handy to someone in the same role.
EventcombMT.exeDownload eventcomb here. This tool has some built in queries which can help you out finding issues on your domain controllers. The main use of this tool is to scan multiple machines for a particular event with a multithreaded app. For instance, last week I wanted to see all the DC's where LSASS had crashed. When LSASS crashes on a machine, the server reboots gracefully, there is no bugcheck so sometimes if your not watching DCs and other servers might be rebooting without you knowing.So what can we do? Well I can add all the DCs into eventcomb and then scan for event 1074 in the system log and then only return those events that have the string LSASS in them. Using eventcomb this process only takes about 20 minutes to scan every DC worldwide remotely and give me the list back.
IFEO for debugging or virus controlImage File Execution Options, located here "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options" or under the Wow6432Node for an x86 apps and exes.
This ties in well with LSASS crashing for me personally. Now that I know all the DCs where LSASS is crashing I can create an IFEO for the LSASS.exe process and run it under debugger so next time it crashes it will fall into debugger so we can find out why instead of just rebooting the server. Using IFEO is great for ANY app that is crashing and you want to catchthe break in debugger. I've used this numerous times for exes and apps that crash on initialization or during a certain repro.
Another nice thing you can do with IFEO is when you have a machine that is infected with a virus. Marcelo has an awesome article here that describes how you can use IFEO for the virus to stop it!
NLTEST /dbflag:2080FFFFHave a machine that is having authentication issues? Then set this dbflag to create a netlogon debug log that falls under %Windir%\debug\netlogon.log. This is useful for web servers that are denying access to certain clients for example.
Tail.exeNow that you have netlogon logging running, how bout watching it in real time to see as entries are added? You can use tail -f c:\windows\debug\netlogon.log to do just that.Looks like it is part of the Windows 2003 resource kit tools which you can get here.
Sc.exeFor remote service control. Want to bounce a service remotely? Easiest way is to use: SC \\Servername stop Service then start service. Or you can query services by running SC \\Servername query service:C:\>sc \\jam_rock query netlogon SERVICE_NAME: netlogonTYPE : 20 WIN32_SHARE_PROCESSSTATE : 4 RUNNING(STOPPABLE, PAUSABLE, IGNORES_SHUTDOWN)WIN32_EXIT_CODE : 0 (0x0)SERVICE_EXIT_CODE : 0 (0x0)CHECKPOINT : 0x0WAIT_HINT : 0x0
Reg.exe Gotta love reg.exe for remote registry manipulation or for querying certaion keys like our IFEO key mentioned above.
C:\localbinx64>reg query "\\Jam_rock\HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options" HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\apitrap.dllHKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\ASSTE.dllHKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\AVSTE.dllHKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\Cleanup.dll
Srvinfo.exe
Good tool to get a understanding of what a server looks like in regards to hardware and software.
C:\localbinx64>srvinfo -ns Server Name: MachineNameSecurity: UsersRegistered Owner: YouRegistered Organization:ProductID: xxxxxxxxxxxxxxxxxxxxxxxxxxOriginal Install Date: Sat Dec 15 20:01:18 2142Base Source Path:Version: 6.0Build: 6000.vista_rtm.061101-2205Current Type: Multiprocessor FreeProduct Name: Windows Vista (TM) UltimatePhysical Address Extensions: enabledProduct Options: Professional, Terminal ServerHAL.DLL is 6.0.6000.16386 - Microsoft Corporation - 6.0:6000.2PDC: \\PDC_OF_DomainDomain: your_domainDomain Guid: {FEFKLJEELDJED:LKE:LDKE:LKE:LKLF:KELF:K}DNS Forest Name: forest.microsoft.comPDC Site Name: REDComputer Site Name: (null)Manufacturer: Hewlett-PackardModel: HP xw9300 WorkstationTotal Physical Memory: 4094 MBCPU[0]: AMD64 Family 15 Model 5 Stepping 10: 2393 MHzCPU[1]: AMD64 Family 15 Model 5 Stepping 10: 2393 MHzSystem BIOS Date: 01/26/06System BIOS Version: HP - 2006012632-bit Hotfixes:MSXML4SP2:Q827429Microsoft Firewall Client:4.0.3441[Update KB919491]: Installed on ??/??/?? byDrive: [FileSys] [ Size ] [ Free ] [ Used ]C$ NTFS 180000 145882 34118Y$ NTFS 23459 13629 9830Q$ NTFS 59370 14016 45354X$ NTFS 47450 13415 34035Network Card [0]: NVIDIA nForce Networking ControllerIP Address(es): xxxxxxMAC Address: xxxxxxProtocols:Link-Layer Topology Discovery ResponderNDIS Usermode I/O ProtocolRemote Access IPv6 ARP DriverRemote Access IP ARP DriverMicrosoft NetbiosSmbMessage-oriented TCP/IP Protocol (SMB session)Link-Layer Topology Discovery Mapper I/O DriverRemote Access NDIS WAN DriverMicrosoft TCP/IP version 6 - TunnelsInternet Protocol Version 6 (TCP/IPv6)Point to Point Protocol Over EthernetInternet Protocol (TCP/IP) - TunnelsPoint to Point Tunneling ProtocolLayer 2 Tunneling ProtocolInternet Protocol Version 4 (TCP/IPv4)WINS Client(TCP/IP) ProtocolSystem Up Time: 1 Days, 16 Hr, 33 Min, 29 Sec
Uptime.exe Another useful tool to see a report on uptime for your servers. It will also list abnormal shutdowns and a bugcheck code. Download it here.
C:\Debuggers>uptime /sUptime Report for: \\My_PC Current OS: Windows (TM) Vista Ultimate Multiprocessor Free.Time Zone: @tzres.dll,-212 System Events as of 11/11/2006 5:07:08 PM: Date: Time: Event: Comment:---------- ----------- ------------------- -----------------------------------11/8/2006 7:24:56 PM Boot11/8/2006 7:30:24 PM Shutdown Prior uptime:0d 0h:5m:28s11/8/2006 7:35:56 PM Boot Prior downtime:0d 0h:5m:32s11/8/2006 8:29:40 PM Shutdown Prior uptime:0d 0h:53m:44s11/8/2006 8:31:03 PM Boot Prior downtime:0d 0h:1m:23s11/8/2006 9:55:24 PM Shutdown Prior uptime:0d 1h:24m:21s11/9/2006 4:20:17 AM Boot Prior downtime:0d 6h:24m:53s Current System Uptime: 0 day(s), 16 hour(s), 13 minute(s), 31 second(s) -------------------------------------------------------------------------------- Since 11/8/2006: System Availability: 90.6317%Total Uptime: 2d 15h:10m:24sTotal Downtime: 0d 6h:31m:48sTotal Reboots: 4Mean Time Between Reboots: 0.73 daysTotal Bluescreens: 0
Portqry.exe Great tool to see if ports are listening on a server. I use this a lot to see if 389, 3268, 88, 445, 139 are listening on a DC but you can supply any UDP or TCP port you'd like. Info and download here.
C:\localbinx64>portqry -n my_dc-01-e 88 Querying target system called: my_dc-01 Attempting to resolve name to IP address... Name resolved to 145.121.31.698 TCP port 88 (kerberos service): LISTENING
KrView Last one that I am going to mention today but another really useful tool to figure out why a certain process is running hot. If you have an in-house app that is running 80% CPU usage what can you do to look into it? Well you could use perfmon and a debugger to help but to get an overview of the process and the functions that are consuming the most CPU cycles you can use KRView. Info and download here. This tool is a bit more complex so I can really talk about it in this blog segment perhaps I'll make another article later on this subject.
Till next time. Seacrest Out.