E-Mail Protection feature in TMG was a feature that I used to work a lot when I was in CSS. This year I delivered a session internally about Troubleshooting E-Mail Protection in TMG and today I’m sharing the slide deck (the public version) with you. Feel free to download it:
Note: most of the troubleshooting tips that I added in this slide deck were also included in the Forefront Threat Management Gateway (TMG) 2010 Troubleshooting Survival Guide, E-mail Protection Troubleshooting section.
Enjoy it
Introduction
Almost three years ago I wrote this post about using Netmon to identify unexpected traffic. Although Netmon is a great tool and the advices that were written on that post are still valid, there are some scenarios that you need to go beyond that and identify if the process that is generating this traffic is suspicious or not. This post will describe how to use TCPView and Process Monitor to identify suspicious activity on your local system.
Symptoms
Investigation Process
When investigating issues of this nature and mainly when you’ve done the basics of scanning the computer the next step will be to understand if this computer is sending anything (or receiving) through the network. One great test that should be done right in the beginning is to ask/test: does the performance issue still happen if you remove the network cable from the computer? Many times a potential compromised computer will behave differently when you unplug it from the network. As this step was not done in this case, we will start the investigation using TCPView to understand the current footprint for this computer.
This is a great tool because it allows you to view many aspects of the process from the socket perspective to the TCP state of the connection. You can argue that this is similar to run the command netstat -naob and I’m okay using netstat tool, but the nice thing TCPView is that you have live experience with what’s going on. In other words, you don’t need to refresh it, if a connection gets closed, it will update the UI. With netstat, you will need to keep running to see if the state changed or not, this is the main reason I prefer using this tool for live investigation on issues of this nature. Apparently there is no suspicious activity; however there is a process that doesn’t seems to be a valid process, it is called msupdate.exe. Notice that this process is listening on port 3349, which is also not a usual port. Now is time to use Process Explorer to better understand what this process is trying to do:
There are some suspicious signs around this process:
Note: I’m saying suspicious signs because this could indicate a malicious process, but at this point we don’t have enough information to confirm that.
To investigate further this process, right click on the suspicious process and choose Properties, the window below will appear:
As you can see this process is located in the %windir%\system32 folder and it is starting a command line with some additional parameters. As I’m not sure which process is that, one way to obtain more information about what this process loaded in memory is to verity the Strings tab (choose memory radio button on the bottom of this window). Notice that in this case the Strings tab shows some interesting information about the process, such as the one that I pointed out below:
This line is used by NC (NetCat) tool and it matches with the parameters that we saw in the Image tab for this executable file. This means that this executable file is actually the NC tool, renamed to msupdate.exe, it is listening on port 3349 and once someone access this port this tool will execute Command Prompt (cmd.exe).
Conclusion
The findings here showed that this workstation was compromised with an implementation of NC that could be used as a backdoor. A simple implementation but for the purpose of this investigation was enough to understand how we can leverage Process Explorer to identify suspicious activities that sometimes are not caught by antivirus.
The ISSA Journal May 2011 issue was just released and this issue brings an article that I wrote with Deb Shinder about some considerations while migrating to the cloud and on-premise resources. The subject is very interesting and we tried to cover some core points on why it is important to keep the defense in depth approach while migrating to the cloud. ISSA members will be able to download the full Journal from here. However, if you want to read only our article you can access from here.
I would like thanks Deb Shinder for partnering with me in this article, it was indeed a great experience writing and brainstorming ideas with you !!
Before joining the Windows iX IT PRO Security team I spent my last 11 years working in the enterprise support field, where 5 were at Microsoft CSS (former PSS). During the Conficker outbreak I was in Oklahoma for New Years Eve 2008/2009 (which BTW is pretty cool) and while I was there I wrote this post about blocking Conficker proliferation via ISA Server. Six months later I kept hearing comments from the IR (Incident Response) folks about getting new cases related to Conficker. I remember talking to one of those guys and hearing from him that some companies were without patches for years. We are not talking about small offices; I’m talking about enterprise level type of company with thousands of workstations and hundreds of servers - unpatched.
While it is hard to believe that this type of practice still happening, today reading this article I got the confirmation that Conficker didn’t teach the full lesson to everybody. Unfortunately the reality is that there are still many servers and workstations (regardless of the OS) unpatched out there. In other hand, it is also very good to see that people are warning about that in many ways, such as with an article like this: “Patch Management Crucial to Defend Against Cyber-Attacks: Report” that explains how important it is to patch and beyond that, how important it is to make sure all platforms are patched. The article has a great statement, that says:
“While Windows vulnerabilities receive wide attention, Norman security experts also warned that IT administrators in enterprises, government and small to midsize businesses (SMBs) should focus on patch management involving all major operating systems, including Microsoft Windows, Linux, Mac OS, Sun Solaris and HP.”
If you don’t know where to start on patch management subject for your Microsoft platform or if you want to review if you patch management strategy is correct, go ahead and download the Microsoft Security Update Guide, Second Edition – this is a great source of information about this subject.
Stay safe!
Last May 2nd me and Tom started this project as he outlined in this post. We currently have two episodes live: Episode 1 with hosting Jim Harrison as guest and Episode 2 hosting Kevin Saye as Guest. We already recorded Episode 3, where will not have a guest, but we will discuss general security topics and demonstrate an attempt to exploit a vulnerability in a Windows system. Episode 3 will be live next week (first week of June).
But I’m here today to invite you Forefront MVP and Enterprise Security MVP to participate on the show. Even if you can’t be here in Texas (where we record the Episodes), we have plans to host you as a guest using Live Meeting. Sounds interesting? It is…I think it is a good way that you can reach out the community and discuss security topics with us, for example: demonstrate a technology that you feel it is important to assist customers using Microsoft products with their security needs.
Start planning now what you want to present and make sure to reach me and Tom with your proposal. Access the Security Talk show blog and send an e-mail to us with your plan. We already have a solid agenda for Episodes 4 and 5 (July), but we are open for Episode 6 (August). Think about and let’s talk about security with us.
Have a great weekend and if you are in US, have a great Memorial Day.
Two years ago I was announcing in this blog the registration for a presentation called MS Security Day. Today I’m going to announce that MS Security Day V2 is going to happen again in Fortaleza/CE and the registration is now open at https://msevents.microsoft.com/cui/EventDetail.aspx?EventID=1032486309&culture=pt-BR . My presentation is called “Migration to the Cloud and On-Premise Security”, which is based on the article that I wrote for ISSA Journal - May 2011 issue.
Hope to see you there!!
Another event that I will be presenting as speaker was confirmed for June, this one is called TechPE Day and it will happen in Recife/PE/Brazil. The agenda and registration for this event can be found at https://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032487195&culture=pt-br . I will be delivering two presentations:
If you are in Recife in June make sure to register and I hope to see you there.