Mark Russinovich’s technical blog covering topics such as Windows troubleshooting, technologies and security.
This post continues in the malware hunting theme of the last couple of posts as Zero Day availability draws near (it’s available tomorrow!). It began when a friend of mine at Microsoft told me that a neighbor of hers had a laptop that malware had rendered unusable and asked if as a favor I’d be willing to take a look. Her friend was desperate because she had important files, including documents and pictures, on the laptop and had no backup.
Unlike most people in the computer industry that view the requests of friends and family for troubleshooting help as a burden to be avoided, I embrace the challenge. When fixing a system or application problem, it’s me against the computer and success is satisfying and always a learning experience. But that success also has an academic feel. With malware, it becomes personal, pitting me against the minds of criminal hackers. Defeating malware is a victory of good over evil. I should print a t-shirt that says “Yes, I will fix your computer!”. I immediately agreed and we made arrangements to get the laptop dropped off at my office.
When I had a few free minutes the next day I powered on the laptop, logged in, and within seconds was greeted with a torrent of warning dialogs announcing that the computer was infested with malware and that it was under attack from the Internet:
I also saw a barrage of warnings that various applications had been stopped from launching because they were infected:
I hadn’t seen scareware this aggressive. After a minute the appearance of new warnings ceased and I began my investigation. Starting with the insertion of a USB key containing the Sysinternals tools, I tried launching Process Explorer. However, I found that trying to run anything - whether part of Windows or third-party - resulted not in the execution of the application, but in the display of the same “Security Warning” dialog reporting that the application was infected. This system was truly unusable.
The infected account was the only one configured, so that ruled out trying to clean from a different account in the hope that it might not be infected. I was afraid that cleaning the malware might require off-line access to the system via a boot CD installed with the Microsoft Diagnostic and Repair Toolset (the Microsoft product that’s the descendent of ERD Commander, the product I created at Winternals Software). My MSDaRT CD was at home and I’d have to burn a new one. But I had noticed when I logged on that it was 5-10 seconds before the first popups started appearing. If the malware didn’t block running applications during that time window, either because it was initializing or just letting the first few logon applications run so that the Explorer could fully start, I might be able to sneak Process Explorer and Autoruns in before the lock down. That would save me the time and trouble of burning a CD. It was worth a try.
Before logging off, I copied Process Explorer and Autoruns to the desktop for easy access. I logged on and double-clicked the icons in quick succession. There was a short pause and then both applications appeared. It had worked! I had to wait for the avalanche of warning dialogs to stop and then turned my attention to Process Explorer. Sure enough, one process stood out, hgobsysguard.exe:
I explain the common characteristics of malware in my Advanced Malware Cleaning presentation and this sample had all the telltale signs:
I carefully studied the other running executables, including the services running within the various Svchost.exe hosting processes, but I didn’t see anything else that looked suspicious. Sometimes malware employs the “buddy system”, where it uses two processes, each watching the other so that if either terminates, the other restarts it, making it virtually impossible to terminate them. When I see that I use Process Explorer’s suspend feature to put both to sleep and then kill them (which is also arguably more humane). Here all I had was one malicious process, so I just terminated it. It didn’t reappear, which was a good sign that there wasn’t a buddy lurking within another process as a DLL. I then navigated to the malware’s install directory and deleted its files.
With the process and executables out of the way, the next step was to determine how the malware activated and delete its autostart entries. I switched to Autoruns, which had finished its scan in the meantime, and spotted two entries pointing at the malware’s executable. Both entries had names that appeared to have been randomly generated, consistent with typical malware:
I deleted the entries, studied the rest in case there was some other component that wasn’t so obvious, and did some standard crapware cleanup while I was there. I rebooted the system and logged back on to confirm it was clean. This time there were no popups, I was able to run software as normal, and neither Process Explorer nor Autoruns showed any sign of more infection. I had spent a total of five minutes and had some fun outwitting the malware to avoid offline cleaning. Case closed.
Cleaning malware is fine - but unless the user learns to exercise common sense and caution, and to run regular backups, the same issue will repeat itself and the next time it might not be a 5-minute job.
Family? Yes. But then they get demoted to a limited user account, I reset the administrator password, and it goes on my Live Mesh so that I can login remotely and keep it updated from time to time.
I'm also a "once it's infected, you can't trust it ever again" kind of guy. I've seen more than one malware infection simultaneously. Could be turtles on top of turtles -- you just don't know for sure. Once a system is compromised, safer just to blow everything away and start over.
Agreed, it can not only take more time but the cleaning process itself can uproot the degree of damage and eventually cause the system to crash.
Imaging the typical user's machines is not practical and getting them to do backup is like pulling teeth w/o Novocaine.
This is all dependent on what damage has occurred in the first place.
Casually passing this cleaning process off is ill advise to a user.
Unfortunately most users don't want to learn the basics of the issues around Malware and too openly accept the false view that a computer must be replaced every few years because it has gotten slow and is now old.
Sysinternals utilities have become more popular and drawn more attention from malware creators. There will be one day dangerous malware can successfully block all useful Sysinternals utilities (including Desktops which was mentioned in the previous post), how can we defeat malware then?
Another method of protection would be something like Deepfreeze, which resets everything after a reboot. But be sure to optimize Readyboot first. i/o in win7 seems to be slowwww compared to xp.
Thanks for the writeup, Mark. Pretty interesting.
The only question I have is: Did you end the cleanup there, or did you look deeper before declaring the machine clean? Anymore when I see an infected machine I can't help but wonder what else might be there, but unseen. There's a pretty good argument for using a rootkit-based system where "obvious" malware is installed by the invisible rootkit every 10-20 days. If somebody cleans it off things look okay for another couple weeks and then BAM, malware is back even though the user hasn't done anything.
Do you personally suggest reinstallation of the operating system after a computer is compromised by malware like this? When you help a family member clean their system, do you feel okay with them accessing their online banking from the "cleaned" system, or do you perform a more in-depth check for a rootkit?
Nice work! My preferred method for dealing with something like this is to boot to Safe Mode. That blocks the malware from ever loading in the first place (almost no startup entries get processed in Safe Mode).
@Seth: That day was a few years ago. Modern rootkits (for example) don't block the sysinternals tools -- they just make themselves invisible inside said tools.
an idea, you can implement a thin hypervisor and a infrastructure to use it with your sysinternals utilities like modern hvm rootkits. So, when somebody wants to analyze a normal user computer (whicih is modern but no use of any hypervisor thing) your utilities run and analyze safely while the OS switches VM mode and your hypervisor handles the rest.
What about the image hijack registry entries that allowed the malware to stop anything else from running? I had one system where I failed to remove these keys and applications still wouldn't run. If there aren't image hijacks, how did the malware stop other processes from starting? Just curious.
Your process has really helped me streamline and improve my own scareware removal procedures.
"Installed somewhere other than the \Program Files directory" Don't forget that Win7 added the per user programs known folder. (And Chrome etc install in appdata)
@Jon True - this could have been defeated in safe mode as well. I've posted this before, but obviously best practice is to wipe and restart when you have an infection. But security is about risk analysis and management and wiping/reload has a high cost. Given the purpose and characteristics of this malware, and my previous experience, the likelihood that I had successfully cleaned it was very high.
@WndSks - true, you'll find legitmate software in the \windows and \window\system32 directories as well. However, places other than Program Files are the exception for legitimate software, but the rule for malware.
How is malware able to stop all applications from launching? I support many computers (running as restricted user, Win 7, Vista and XP) that get malware like this. It seems like a flaw in Windows to allow this behavior without admin privilege. Please don’t give away any knowledge that will help malware, but I would like know your thoughts about why Windows would allow this. It is very easy to clean from an Admin account, but it is very inconvenient for the users. Symantec Endpoint doesn’t seem to help with the problem.
Keep the blog posts coming! I love reading your "Case of the Unexplained" blog posts!