The Case of the Slooooow System

re: The Case of the Slooooow System

Michael Dragone — Wed, 24 Sep 2008 22:15:56 GMT

I experience the same behavior with Flash frequently.

re: The Case of the Slooooow System

Sarat — Wed, 24 Sep 2008 22:16:56 GMT

You're Insanely smart as Larry Osterman said and dear hubby :D.

I'm also digging into a sluggish performance problem with my nVdia 8800 card. It's using the latest Display Driver available for Windows XP. The hardware seems to be working fine with Vista (with it's default driver).

Anyway thanks a lot for your posts on trouble shooting and debugging.

re: The Case of the Slooooow System

Jonathan — Wed, 24 Sep 2008 22:24:07 GMT

Re. Flash: It could be because some lame Flash programmer wrote a polling loop in ActiveScript. No amount of version updates will get you out of it.

Re. sonicMP4Demux.dll: I presume you could just run "regsvr32 /u sonicMP4Demux.dll", and have it deregistered.

re: The Case of the Slooooow System

fds — Wed, 24 Sep 2008 22:29:53 GMT

You should be able to unregister any DirectShow filter using regsvr32 /u.

regsvr32 /u Sonicmp4demux.ax

The name sounds like it's a demuxer for .MP4 format files, which is basically the same as Apple's QuickTime .mov format.

re: The Case of the Slooooow System

JaredPar — Wed, 24 Sep 2008 22:36:08 GMT

"I Windows-Live-searched (that just rolls off the tongue, doesn't it?)"

I prefer to call in learching. Since the live team didn't choose a name which could be used as a verb (aka google-ing) we need to get creative. Thus comes learching: Live sEARCH-ing. I prenounce (probably incorrectly) as lurching.

re: The Case of the Slooooow System

Ehsan Samani — Wed, 24 Sep 2008 22:58:53 GMT

Mark Why is it that this flash thing does not suprise me?

Flash always hangs my machine. causes my IE to crash and etc. thanks god IE8 has tab recovery feature.

re: The Case of the Slooooow System

Joe Butler — Wed, 24 Sep 2008 22:59:48 GMT

Given the importance of the customer... Ha!

Anyway, did anyone else do a double-take during the opening credits of the new Smallville.

Does Mark have just a passing resemblance to the Green Arrow?

http://thetvaddict.com/2006/09/22/smallville-scoop-2/

Will read your article now...

re: The Case of the Slooooow System

Lionel — Wed, 24 Sep 2008 23:02:46 GMT

Would it be possible to change the thread scheduling algorithm in order to improve the system's responsiveness in such circumstances (high cpu load)?

re: The Case of the Slooooow System

Gerard Leenhouts — Wed, 24 Sep 2008 23:04:55 GMT

I was waiting for one of these articles to appear again! It was as entertaining and educational as always. Thanks a lot, Mark.

re: The Case of the Slooooow System

Kevin — Thu, 25 Sep 2008 00:15:33 GMT

I was having alot of issues with high cpu loads on my core2duo system, and whenever i would look deeper into the problem that was causing it, it was almost always related to either a audio/video application hanging, or related to the "Multimedia Class Scheduler" service, thats new with vista, and came with SP1 i believe.

The biggest issues i seem to have are all related to the fact that i typically have ALOT of Directshow filters and plugins installed at any one time, because i do alot of video production, and have to take video from alot of different formats, and hack them to pieces and put them back together.

Its mostly due to the fact that i have to import alot of video that is not native to the program i am using, or I have to use a media splitter/demuxer to fool the application into importing the video at all.

I pretty much solved all my CPU hogging issues by doing this -

uninstall quicktime and anything that relies on quicktime, such as itunes

install K-lite codec package

install Quicktime Alternative

install the Quicktime-itunes addon, so that you can still run itunes and any other cranky quicktime dependant application without having quicktime actually installed.

you can get all those programs from www.codecguide.com

it is an absolute godsend for anyone who is needing to edit, process, or format video for many different devices, and seems to help alot with keeping media apps that are dependant on media splitting, demuxing, or importing through directshow from hanging and just burning up cpu cycles

re: The Case of the Slooooow System

Phileosophos — Thu, 25 Sep 2008 00:19:21 GMT

So the real question, or at least the one I find most interesting, is this: when will the various bits of Windows/IE that so glibly host third-party code detect and reign in such excesses?

re: The Case of the Slooooow System

Walt — Thu, 25 Sep 2008 00:22:38 GMT

Great article Mark! What's the average customer to do however? Inevitably they will take their system back to where they bought it, the technicians won't find the problem but will reinstall the operating system. Everything will be fine until the customer re-installs the offending software and then the whole thing starts all over again.

re: The Case of the Slooooow System

Tyler — Thu, 25 Sep 2008 00:24:52 GMT

Mark,

Thanks for some insight on how to utilize Process Explorer better. I was always frustrated with finding out what addin causes IE to crash.

re: The Case of the Slooooow System

Nick — Thu, 25 Sep 2008 00:51:00 GMT

Mark, I think this case is interesting because would really really like to understand why those 2 processes were able to make the entire system almost unusable. Can you shed any light on the following questions:

Why does it take 30 seconds to open the start menu ? If there are 2 processes running, each of them taking up almost 100% of the CPU cycles of one core, why is the Windows scheduler not able to allocate some reasonable fraction of the CPU cycles to other processes, especially Explorer.exe ?

It seems utterly bizarre to me that one process running in a tight loop can make the whole machine almost unusable. Is it not reasonable to expect the scheduler to evenly split the CPU between processes or threads ?

I remember using a Sun-3 machine (Motorola 68020, running some version of Sun's Unix) back in 1989 and if there was a runaway process (or two), everything would slow down proportionally. So if you had 2 processes spinning in a tight loop, and you wanted to do something else, that "something else" would get 33% of the CPU cycles. That seems much better than giving almost all CPU cycles to those 2 processes and almost none to Explorer, so that the start menu takes 30 seconds to respond to a click.

I would really love to find out why Windows allocates the CPU cycles the way it does.

re: The Case of the Slooooow System

Stephen Clarke-Willson — Thu, 25 Sep 2008 01:28:04 GMT

My wife had the same problem (the Flash problem). I switched her to Chrome because the Chrome task manager makes it very easy for her to identify a page with a wacky out of control Flash widget and kill it, or, alternatively, kill Flash.

This kind of problem is only going to increase as random people write poorly tested scripts that go into infinite loops. *Sigh*.

re: The Case of the Slooooow System

Doc O. Backbush — Thu, 25 Sep 2008 02:13:41 GMT

As an IT tech, I've encountered this behavior from windows quite frequently. Third party applications acting haywire. That was half-expected on 98, ME, even XP. But...

Considering how long Vista was in development...

Considering the cost of the OS...

Considering the intended audience that this flash-bang gee-whizz OS was created for...

Considering all that? I'm sorry Mark. Vista should be good enough to prevent this from occurring. Vista should be able to manage this so the end-user...a home customer...could manage any such problems without being married to one of the leading software debugging techs in the world.

Thank you for a very informative blog entry. I'm sure it'll be shared with anyone coming to be for recommendations on Windows Vista.

re: The Case of the Slooooow System

Steven — Thu, 25 Sep 2008 04:18:56 GMT

I have experienced similar slowness for the past year or more. However, my diagnostics skills do not match yours.

I finally gave up and got a Mac. After 18 years of Windows, I was ready for a change. I'm glad I did. It's been a long time since I was this happy with a computer. Sure I've had a few issues with my Mac, but far fewer in the months since I've bought it than I get in one week on Windows.

Sorry to sound like a Mac fanboy. I had no intention of leaving Windows...until it left me (with a crappy system).

re: The Case of the Slooooow System

Vektor — Thu, 25 Sep 2008 05:51:41 GMT

> So if you had 2 processes spinning in a tight loop, and you wanted to do something else, that "something else" would get 33% of the CPU cycles. That seems much better than giving almost all CPU cycles to those 2 processes and almost none to Explorer, so that the start menu takes 30 seconds to respond to a click.

TimeSlice of Windows 9X was reasonable. Also Explorer.exe was faster - it didn't do all the useless stuff the newest versions do.

re: The Case of the Slooooow System

gary — Thu, 25 Sep 2008 07:14:50 GMT

i used process explorer to track a client's dual core performance problem, too. it was taking 2 minutes to run an excel app i worte that runs in less than 40 seconds on any other pc they had. fired up process explorer, ran the code and what showed up thousands of times? google desktop search. uninstalled it and the code ran in about 30 seconds. thanks.

and as an aside, i hate flash, refuse to install it on my pc's, because it's the most obtrusive piece of software ever written. well, maybe it's not the software, but the web developers, that think everything has to flash, play a movie and blare sound as soon as a website is visted, when all i wanted to do was to read an article.

no thanks.

re: The Case of the Slooooow System

Gabriel — Thu, 25 Sep 2008 08:39:36 GMT

Thanks for the interesting blog entry that helps give some more ideas about how to use Process Explorer and Process Monitor for troubleshooting problems with software running on Windows.

As soon as I started reading the problem the first thing that came to my mind was that it was a problem caused by Adobe Flash player. I'm not surprise that you were not able to determine the root cause of that problem beyond tracing it to Adobe.

However, I am curious about why the Windows UI was unresponsive simply because of high CPU utilization. As some of the other readers have noted, the Windows scheduling algorithm should allow the user's actions to receive enough CPU time, to do something like open the Start menu, even if there is another task running that is using all available CPU time.

Usually when I see a system where Flash is using all the CPU time it will cause my browser (Firefox) to be unresponsive, but the Windows UI itself will continue to respond to user input. I would think that in Vista using Windows Desktop Manager there should be even less reason for a misbehaving application to make the Windows UI itself unresponsive. Also I've run various compute intensive programs that use 100% CPU time without causing the Windows UI to become unresponsive the way you described (30 seconds for the Start menu to open).

I don't know if you can comment on this at all or not, but I think it is a question that several people have posed and it would be interesting to know if you have any idea why the Windows UI was unresponsive in the problem you were troubleshooting.

re: The Case of the Slooooow System

Ailurophobe — Thu, 25 Sep 2008 10:49:30 GMT

Personally I disable video thumbnailers everywhere it is possible. They are commonly broken since they rely on being able to use third party code for things it wasn't actually designed for. And many DirectShow filters don't really support the full feature set of whatever they are supposed to handle to begin with turning thumbnailing into an automated crash generator. And apart from giving nice pictures for marketing, thumbnails are invariably totally useless. I mean really, what are the odds of a randomly picked image on a video file being anywhere near as informative as, a totally random example (not), the filename. Somewhere below 1% unless you have really bad naming conventions, I'd guess.

As a Firefox user, I have the AdBlock extension installed for the express purpose of being able to disable broken Flash on web pages. Mark is probably stuck using IE but maybe there is something similar available. Not much that can be done about the web pages that refuse to work without broken Flash apart from avoiding them, though.

Sorry if this ended up as a double post, I had a little technical problem.

re: The Case of the Slooooow System

Rune — Thu, 25 Sep 2008 14:12:28 GMT

Doc wrote:

"Considering all that? I'm sorry Mark. Vista should be good enough to prevent this from occurring."

_How_?

Until you can answer that simple question, you are just a troll.

Rune

re: The Case of the Slooooow System

Rune — Thu, 25 Sep 2008 14:15:53 GMT

Gabriel wrote:

"As some of the other readers have noted, the Windows scheduling algorithm should allow the user's actions to receive enough CPU time"

How? By increasing the priority of explorer.exe? What if some plugin in explorer.exe runs away with the CPU? Instead of running at the same priority as other processes, you now have a high priority process eating CPU cycles...

Possible Solution: Always run Task Manager (because TM is usually stable) at a higher priority, thus allowing the user to quickly shut down normal priority processes. (this still does nothing for the average Joe)

Rune

re: The Case of the Slooooow System

Keff — Thu, 25 Sep 2008 14:21:06 GMT

Everyone blaming the scheduler: There's no way modern scheduler could have hardcoded process names. If you want to give a process a priority - well, that's what priority is for!

Explorer.exe can't run with elevated priority, because then OS would focus on running the shell more than on running your applications, which is what you buy PC for. If a process requests higher priority and then hogs CPU, it is application author's responsibility, and he is the one to blame - call Adobe and Roxio, instead of blaming MS.

On the other side, if the processes have same priorities, then a reasonable scheduler should give time to both of them even when other app goes berserk.

roxio

Rune — Thu, 25 Sep 2008 15:23:56 GMT

I have heard so many bad things about roxio's product, and this for me is just another nail in the coffin. I just don't trust them.

Mark, look at the filter drivers they install for your CD/DVD devices. I think they might contain one or two surprises as well... (that you may or may not notice as time goes by)

re: The Case of the Slooooow System

o.s. — Thu, 25 Sep 2008 20:32:07 GMT

'A few weeks ago my wife complained that her Vista desktop was not responding to her typing or mouse clicks. Given the importance of the customer, I immediately sat down at the system to troubleshoot. ' Hilarious Mark just hilarious!

I had a question though. So since 'I had seen from the traces that the thumbnail cache generator had come across an AVI file that caused it to load the Sonic demultiplexer' is true does this mean that the component went into an infinite loop trying to create a thumbnail for an avi file?

re: The Case of the Slooooow System

Bob — Thu, 25 Sep 2008 22:01:34 GMT

Interesting. Thx for sharing the troubleshooting process, I learned a lot. But I wouldn't submit that for an "I'm a PC" video :-)

re: The Case of the Slooooow System

fsr — Thu, 25 Sep 2008 22:01:44 GMT

This blog is excellent to assess current real-world support: I doubt Microsoft Support would be able to troubleshoot that problem, and it would be impossible for a 'poor' OEM supporter, like Dell or HP, to troubleshoot it through popular email, chat, phone or VNC support; and if they were able to get machine on workbench, seems to me they would simply format HDD to factory settings and close ticket.

Also, are you wondering why some costumers have VIP support and other's don't ? Get the answer from THE Windows expert !

re: The Case of the Slooooow System

JF — Thu, 25 Sep 2008 23:36:32 GMT

Thanks, Mark, very interesting...

re: The Case of the Slooooow System

rhagman — Fri, 26 Sep 2008 00:13:14 GMT

Doc wrote:

"Considering all that? I'm sorry Mark. Vista should be good enough to prevent this from occurring."

Rune wrote:

"_How_?

Until you can answer that simple question, you are just a troll."

Build in to the OS that what a few 3th Party tools already are doing: Automatically and temporally lower the priority of any user process that is causing a High CPU load for more than x seconds, where the time the priority is lowered and the thresholds (% CPU load and duration) are both configurable,

and there also should be a way to exempt processes from this.

There are even tools that do this also for memory , where you can assign a maximum memory limit to each user (in a multi user environment)

It is quite common to use tools like this on multi user environments like Citrix XenApp servers to prevent the users perceived performance to become unacceptable when some user starts a high CPU or memory demanding application, you think flash is bad, you should see what some Business reporting software can do to a XenApp server with 30 - 50 users if you don’t use tools like this

- Rob

re: The Case of the Slooooow System

Guy Leech — Fri, 26 Sep 2008 00:34:11 GMT

Great, thorough and amusing article,as ever, but what I find can work better with a CPU hogging process is to set its base priority to "Low" in Task Manager, or "Idle" in Process Explorer, so that its threads keep running but if any other thread becomes ready to run, it should get on a processor in preference to the Low priority threads of the wayward task since they will almost certainly be at a higher base priority (typically "Normal").

re: The Case of the Slooooow System

markrussinovich — Fri, 26 Sep 2008 00:40:39 GMT

The reason that the system became unusable in this situation is due to the fundamental way that the NT scheduler has worked since NT4, for good or for bad. You can read the gory details in my book "Windows Internals", but I'll give a brief explaination here.

By default, threads run at priority 8, but get priority boosts for short periods when they wake up from sleeping or waiting and when they get a windows message and they are their window is the foreground.

What happened on my system was that the flash and dllhost threads had priorities set to "above normal", which is 9 for threads of processes running in the default process priority class.

When I unlocked the screen, clicked on the start menu, and interacted with other processes running at the default priority of 8, their main window thread got a short boost to 12 and then got dropped back to 8. At that point it was below the priorities of the active threads. Windows includes an anti-starvation priority boost that kicks in every several seconds and temporarily boosts the starved thread's priority to 15.

The long delay to see a response to input was therefore the result of the window's thread finally getting enough of those short priority 15 boosts to display it fully. In the case of the start menu, that was around 30 seconds.

Yes, it's not great, but it's been that way for 13 years across many OS releases - NT4, Win2K, XP, Server 2003, Vista and Server 2008 - and you'll only see this kind of problem in special circumstances - like I did. That said, we're always looking at things like this to see how we can improve it while keeping in mind the problems of application compatibility and avoiding unintendend side effects.

re: The Case of the Slooooow System

Robert 'Groby' Blum — Fri, 26 Sep 2008 01:12:53 GMT

I wish it was always that easy :)

I'm still trying to figure out why my machine gets slower over time, without any obvious culprits in process explorer. (I.e. total CPU usage < 10%, plenty of phys. mem. available, and yet, sluggishness)

So I'll just go ahead and ask if you've got any ideas what to look for. (I mean, if *you* don't know, who would? ;)

re: The Case of the Slooooow System

Scott Johnson — Fri, 26 Sep 2008 05:27:22 GMT

Mark, you state that "only see this kind of problem in special circumstances", but with Flash and 3rd party codecs becoming so prevalent, this is something that I see several times per month on different systems. Both on XP and on Vista. For servers I agree: it generally doesn't happen all that often. But for the less controlled environment of a personal workstation, it happens very often.

re: The Case of the Slooooow System

Paul Renaud — Fri, 26 Sep 2008 06:39:39 GMT

This is a great example of how Vista has failed to address the sheer and growing complexity of the Windows environment.

Most of us geeks would have failed to get as far as you did, so what is the non-geek down the street to do?

Rebooting and encountering the same problems over and over again is hardly a customer experience that will engender loyalty to the platform.

re: The Case of the Slooooow System

Cristian — Fri, 26 Sep 2008 07:14:27 GMT

I love your posts, man! Process Explorer is a wonderful tool!

re: The Case of the Slooooow System

bcthanks — Fri, 26 Sep 2008 09:11:56 GMT

Mark, you're too kind. I would have kicked Roxio off my systems for good soon as I realized I need to hack the registry to fix things.

re: The Case of the Slooooow System

Gabriel — Fri, 26 Sep 2008 09:23:50 GMT

@Mark

Thanks for the additional explanation that makes it clear why you experienced the behavior you described. Do you know why these threads were running with priority "Above Normal." I tried running a webpage with Flash (youtube) on my Win XP system and the base priority for the IE process was 8. Is there something different in Vista that raises the base priority for multimedia applications? Also when I was checking this I noticed that Process Explorer had a base priority of 13. Is this something you added to help keep Process Explorer responsive when troubleshooting these kinds of problems?

re: The Case of the Slooooow System

markrussinovich — Fri, 26 Sep 2008 12:35:12 GMT

@Gabriel

>Do you know why these threads were running with priority "Above Normal."

No, but it's not uncommon for programmers to program their threads with higher priorities with the intention of making their operation more responsive. In most cases programmers should leave their threads at the default priority.

Yes, Process Explorer sets its priority to "High" so that it can operate even when threads like the ones in this case are in CPU-bound loops.

re: The Case of the Slooooow System

Johnny Sutherland — Fri, 26 Sep 2008 13:23:17 GMT

Thanks for taking the time to explain this with the screenshots. Much appreciated. A great post.

re: The Case of the Slooooow System

Pentti Hirvo — Fri, 26 Sep 2008 15:09:48 GMT

So Vista was slow. No news there.

Q: Doctor doctor, my Vista is slow.

A: Don't use it then.

The take home point here would be that if you really insist suffering Vista, you'd better marry someone bright bloke like Russinovich to trouble-shoot it every other day.

re: The Case of the Slooooow System

Wayne Robinson — Fri, 26 Sep 2008 15:32:03 GMT

Really good and detailed investigation as always Mark.

Thank you so much for sharing your experience with everyone.

Cheers

re: The Case of the Slooooow System

Lars Jeppesen — Fri, 26 Sep 2008 16:20:08 GMT

@Mark

>No, but it's not uncommon for programmers to >program their threads with higher priorities >with the intention of making their operation >more responsive. In most cases programmers >should leave their threads at the default >priority.

So this particully problem (dllhost.exe) was actually cause by two different bugs.

1. The 3. party codec could hang in an endless loop.

2. The priority was incorrectly "above normal", especially for a background thumbnail updater.

But is bug 2 actully a problem in windows ?

re: The Case of the Slooooow System

markrussinovich — Fri, 26 Sep 2008 16:39:08 GMT

@Pentti Hirvo

Nothing in this case was Vista-specific.

re: The Case of the Slooooow System

fsr — Fri, 26 Sep 2008 17:17:54 GMT

Vista is great if you have the hardware, but troubleshooting it is as difficult as XP. I am just a Windows experienced user and I doubt I would be capable of using PE like Mark. If I would have found such a problem and if nothing was logged in event viewer, I would start excluding third-party entries with msconfig (or with msconfig in combination with autoruns), and once I have found the problematic add-in, I would have updated/downgraded the program or simply discarded it, replacing it for another.

re: The Case of the Slooooow System

Angus — Fri, 26 Sep 2008 17:26:38 GMT

ProcessMonitor really is a wonderful piece of software, thanks for it. I just used it yesterday to discover what was preventing me from installing Visual Studio .NET 2003 on my laptop. The laptop has had multiple install/deinstall cycles, and one of them had left incorrect product information in the Installer's registry settings. ProcessMonitor made it easy to look at what setup.exe was doing just before it reported a failure to find vs_setup.msi, I found the offending registry information, moved it out the way, and the install worked correctly the next time.

re: The Case of the Slooooow System

Doc O. Backbush — Sat, 27 Sep 2008 00:40:45 GMT

Rune wrote:

"Until you can answer that simple question, you are just a troll."

Sure Rune, take the easy way out label me a complainer. Just someone who knows a little bit more than the average user, but not in a position to do something about it.

I don't work at Microsoft. I don't write OSes for a living. Maybe what I proposed is absolutely impossible to do. But that still doesn't change the fact that the end user is not going to be married to a Debugging Genius if they insist on using Vista.

Check out my blog. I say the things that need to be said, especially if too few are saying them. Vista should NOT behave like this. That needs to be said. What will be done about it? Who knows.

But the first step to dealing with a problem is acknowledging it. And Rune? Putting your head in the sand with such demeaning terms like "troll" doesn't solve the problem, it only exasperates it.

re: The Case of the Slooooow System

Scott — Sat, 27 Sep 2008 07:04:25 GMT

Mark,

Thanks as always for an informative how-to...

I poked around on my system (XP Home SP3) and I know I have the latest Flash, and it's Flash09f.ocx from March 2008 (yours appears to be Flash09d.ocx from the screen caps).

I suspect yours is NOT the latest version.

On XP, it's stored in C:\WINDOWS\system32\Macromed\Flash and I've had problems with newer versions leaving older ones present in that directory and I've had to do manual cleanup several times.

re: The Case of the Slooooow System

Srikanth — Sat, 27 Sep 2008 15:57:56 GMT

Mark, this is a fantastic narration of a great troubleshooting experience. Invaluable for those reading this.

And I understood more about ProcessExplorer, you're awesome :)

re: The Case of the Slooooow System

Jason — Sat, 27 Sep 2008 16:48:30 GMT

"That’s because the computer was a dual-core system and so to consume 100% of the available CPU cycles a process would have to have two threads, each hogging one of the cores."

Of course, with browsers moving toward creating individual processes per tab/plugin, a couple of tabs with malformed flash could easily consume 100% CPU, even on a multi-core system.

re: The Case of the Slooooow System

Monk E. Boy — Sun, 28 Sep 2008 07:32:22 GMT

Just to follow up on what Scott said, Adobe likes to issue silent updates to Flash. I would recommend uninstalling Flash, downloading the latest version, then installing Flash. In all likelihood you will end up with a newer version that was installed. Whether the new version will solve the problem or not is anybody's guess - the wife probably just visited a website with a malicious Flash widget.

re: The Case of the Slooooow System

Robert Lacroix — Sun, 28 Sep 2008 13:57:21 GMT

Interesting article, but non-IE browsers seem to have problems with the SSL certificate of blogs.technet.com an so some of the pictures don't show up.

re: The Case of the Slooooow System

Dimitris — Mon, 29 Sep 2008 00:27:39 GMT

Wonderful work Mark, thanks for sharing this info with us.

I wonder why people keep saying Vista is slow and it is Vista's fault and so on.

1st, that is not at all constructive, hardly even a complaint. especially when, as mark said, nothing here is vista specific.

2nd, OS's have different uses, and Vista is very good for what it offers. Mac is completely different, Linux as well. (I currently have Vista, XP, MacOS x, open suse, and ubuntu.)

3rd, Runaway processes happen in every system, and taking into account that windows has by far the most 3rd party programs availiable, because of its popularity, i think it handles that really well. I have seen my mac hang cause of flash.

4th, having a better scheduler does not sound easy. its one of the things that if you make too complicated, would make everything slower.

re: The Case of the Slooooow System

Dr. House — Mon, 29 Sep 2008 09:09:10 GMT

Beyond trolls, super-geeks, and, non-geeks,all of you people read like the goober picking, sorry...eating, no life, no dates, and certainly, no clue buggers whom beg to be beaten simply for being. Members of those E-C High School clubs lasting long enough for the real people to leave, so the janitors didn't have to retrieve your high-water pants or long concealing coat from the flagpole? Now that I have all of you angry, ask yourselves this; "at whom, and, why, am I mad?" The answer, of course is moot because I don't care. I see brilliance, all of you are intelligent people however, what good becomes instruction when real-people who really need your help, can't find these smart pages. Mark? Are you there, Mark? Oh' let us not punish Mark,let us pretend the English language hasn't become castrated to electronic acronyms which, have very little meaning for the end user."Exasperates," the word is "exacerbates" my computer nerdy pals yet, it's Mark's Blog. Exacerbating the problems of the scared end-user, hacking registries, dll's,I-frames, HDD's, adding Marks' very nice detailed diagrams, deary me? Whatever would I do without something to complain about? Hmm' to be, or not to be heard and understood by those whom would...still, run your pants up the flagpole, but say thank you, afterwards. Hughe

re: The Case of the Slooooow System

DC — Mon, 29 Sep 2008 13:22:37 GMT

Always a pleasure to read your articles Mark, you make my job easier

re: The Case of the Slooooow System

Gis Bun — Mon, 29 Sep 2008 15:29:30 GMT

Isn't Mark's wife lucky that she has an expert to help her out. :-)

re: The Case of the Slooooow System

Bobw111 — Mon, 29 Sep 2008 15:44:33 GMT

Amazing. 30+ years in computers and I am still reading articles on how to trouble shoot run away processes.

Maybe the ancient round robin technique of so many ticks per process wasn't such a bad thing after all.

Mark, Thanks for a very informative article.

re: The Case of the Slooooow System

Richard A. Steele — Mon, 29 Sep 2008 16:49:07 GMT

Re Flash hogging CPU:

Please vote for the bug at https://bugs.adobe.com/jira/browse/FP-417. Though I've seen it specifically with Firefox, I'm not surprised it also shows up when using IE. I, too, started using Flashblock because of this very defect.

re: The Case of the Slooooow System

David — Mon, 29 Sep 2008 18:05:46 GMT

I still hate how Vista updates the thumbnail cache even when I'm not browsing around in Windows Explorer. Comes out to a couple consecutive minutes of disk activity each day that I can't easily disable.

re: The Case of the Slooooow System

MH — Wed, 01 Oct 2008 17:51:19 GMT

Mark, did you read through this post properly before publishing? You surely made an obvious mistake here? ;)

'At this point the only options I had were to uninstall Flash and leave my wife with a degraded web experience, or terminate IE to stop the current CPU usage and hope that it wouldn’t happen again.'

When you wrote 'degraded' you surely meant 'improved'?

Seriously though, nice work as usual. Just one request - more frequent posts please!

re: The Case of the Slooooow System

CS — Wed, 01 Oct 2008 21:33:42 GMT

After all these years of evolution, computers, operating systems, and software overall, are all still merciless with the average user, the ones who should be using computers, not being afraid of doing it…

What if Mark’s wife wasn’t married to him? Would she have been lucky and stumbled over an excellent post like this one?

It’s not only the PC… not only Windows. It’s the concept: does the industry really expect average users that are able to deal with process managers, threads, and the like?

But wait: things get worse: I am pretty sure that the average support staff does not even have half of Mark’s knowledge

Let’s face it: we’re still light-years away from people smoothly and naturally using computing devices for their daily chores… at work, at home, and on the street.

re: The Case of the Slooooow System

Patrick J. Burwell — Wed, 01 Oct 2008 21:33:59 GMT

We DISABLE flash, and any attempt to install/update flash, on our Terminal Servers for this VERY reason. Flash component programming is too easy to foul your system and we certainly didn't want our client performance ratings to be affected because of an advertisement.

Patrick J. Burwell

VP, Technology

iQor, Inc.

re: The Case of the Slooooow System

Stephen Walker — Thu, 02 Oct 2008 14:00:09 GMT

Video thumbnailers, including the one built in to XP, tend to have great problems when thumbnailing an AVI file with a broken or missing Index Block at the end of the file. This could be the cause of the second problem.

re: The Case of the Slooooow System

John — Fri, 03 Oct 2008 01:03:32 GMT

A few weeks ago my wife complained that her Vista desktop was not responding to her typing or mouse clicks. Given the importance of the customer, I immediately sat down at the system to troubleshoot. It wasn’t completely hung, but extremely sluggish. For example, the mouse moved and when I clicked on the start button the start menu opened after about 30 seconds.

</blockquote>

I thought that was the standard Windows behavior!

Obvious

Ryan — Fri, 03 Oct 2008 10:23:46 GMT

Perhaps XP with a Nero install would reduce your time spent troubleshooting.

Jokes aside -- is your wife using the Roxio functionality?

Or does she just burn CDs/DVDs like most of us?

Do you really need 640mb of bloat for unused functionality?

Is there a well developed application with a smaller footprint which performs whatever functions she needs?

Mark -- I presume from your excellent applications that you understand such an argument. I beg of you to use your influence to echo such a concept throughout Microsoft.

re: The Case of the Slooooow System

Adrian Wilkins — Fri, 03 Oct 2008 12:34:32 GMT

I find that I have to turn off thumbnailing on Vista completely, or Explorer just isn't stable ; many has been the time where just viewing a folder, or even the parent of a folder, containing media files, makes Explorer drop dead.

My take on it is that you shouldn't be allowing a task like thumbnailing to be killing a whole process, especially one as critical to your productivity as Explorer ; why isn't the process insulated from it?

re: The Case of the Slooooow System

blagoev — Fri, 03 Oct 2008 14:12:55 GMT

There is an easy way to disable shell addons. Check this tool ShellExView (from NirSoft)

re: The Case of the Slooooow System

TG — Fri, 03 Oct 2008 15:15:52 GMT

Very interesting article.

But I wanted to troubleshoot some slowness on a company laptop and it seems the new 2.0 version of Process Explorer doesn't work on Windows 2000.

I get the following error:

The procedure entry point getnameinfo could not be located in the dynamic link library WS2_32.dll

Ugh

GagMe — Fri, 03 Oct 2008 19:28:22 GMT

Good troubleshooting efforts, Mark.

But Windoze-Live-searching? I sincerely hope you were attempting humor.

re: The Case of the Slooooow System

Jeff W — Fri, 03 Oct 2008 22:05:01 GMT

Very good sample of how to use Process Explorer. I consider myself a pretty solid troubleshooter and I learned a few things here.

I'll follow your example except my searches will probbaly be run on a site named for a very large number.

re: The Case of the Slooooow System

Jeff W — Fri, 03 Oct 2008 22:05:42 GMT

Very good sample of how to use Process Explorer. I consider myself a pretty solid troubleshooter and I learned a few things here.

I'll follow your example except my searches will probbaly be run on a site named for a very large number.

FYI - I have seen these sorts of issues with Flash and 3rd Party Codex forever. I usually assume it's sloppy Flash code but it would be nice if Flash trapped it better.

re: The Case of the Slooooow System

molotov — Sat, 04 Oct 2008 00:41:45 GMT

@TG:

See http://forum.sysinternals.com/forum_posts.asp?TID=16360&PN=2

re: The Case of the Slooooow System

Yuhong Bao — Sat, 04 Oct 2008 11:18:30 GMT

On one Duron 850/Win2000 system, I often had to reduce the priority of Firefox after the system became unresponsive for seconds due to it consuming 100% CPU time.

Anyway, thanks for all of "The Case of..." articles. Too many people blame every problem they had when running Windows on Windows itself, while more often the problems were really caused by 3rd party software.

re: The Case of the Slooooow System

Martin — Sat, 04 Oct 2008 19:14:25 GMT

Could be a good idea to add a Refresh button to that "Stack for thread xxxxx" dialog box to eliminate the need to close and re-open it. Thanks for great blogging!

re: The Case of the Slooooow System

Adam — Sat, 04 Oct 2008 23:14:11 GMT

Awesome to see a come back of The Case of... Thanks Mark!

re: The Case of the Slooooow System

Lorraine — Mon, 06 Oct 2008 01:43:20 GMT

Oh woe is me!!! Why couldn't I have come across your most excellent explanation BEFORE I spent two days attempting to fix a friend's (first mistake) slooooow and Trojanized machine. :-(

I refuse to even look at a machine unless I can install Process Explorer but realize that until now, I have only touched the tip of the iceberg in what it can do and what it could have shown me. This article gets printed on drilled paper.

Thanks from long time fan.

re: The Case of the Slooooow System

Shahbaz — Mon, 06 Oct 2008 19:31:08 GMT

Great article Mark. It's got to be simpler than this & handles by the OS itself as several million of us all have similar customers running Vista OS :) .

Fixed in the next Desktop OS release ?

re: The Case of the Slooooow System

Intentsly — Mon, 06 Oct 2008 21:37:22 GMT

I hope you forwarded your findings to Adobe and/or Roxio. That may actually be enough grease to have either come up with a solution. Perhaps even provide solutions to support problems they've had go unanswered for a long time. Good Job Mark!

re: The Case of the Slooooow System

Jeroen — Tue, 07 Oct 2008 01:31:50 GMT

suspending a process helped me today to get a system free from malware, saved a reinstall. remembered that from this post, after the normal struggle with an agressive malware got me almost stuck. thanks again.

re: The Case of the Slooooow System

David De Roxio — Wed, 08 Oct 2008 00:35:01 GMT

Hello,

I am a Roxio support agent.

I see that this issue was resolved. However, if you ever want to optimize your computer's performance, this article should help adjust Creator:

http://kb.roxio.com/content/kb/Creator/000103CR?set-locale=en

Regards,

David

Roxio Support

re: The Case of the Slooooow System

SRS — Wed, 08 Oct 2008 15:12:13 GMT

From Roxio support: "Locate RoxWatchTray10 (or whichever version you are using) under the Startup Item column and uncheck it. " - i.e. "To make your computer run faster, don't run our software". Roxio should be commended for honesty.

re: The Case of the Slooooow System

Vinh — Wed, 08 Oct 2008 23:12:44 GMT

I'm very like the processexplorer and process monitor. I'm learning how to use them professionlly like Mark by check out his blog every month, but everytime he about process monitor, I'm getting confuse. Well, how can I master using process monitor? or I appreciate any suggestion.

re: The Case of the Slooooow System

molotov — Thu, 09 Oct 2008 04:38:54 GMT

@Vinh:

You might have a look here:

http://forum.sysinternals.com/forum_posts.asp?TID=15520&PID=75140#75140

re: The Case of the Slooooow System

Miksa — Thu, 09 Oct 2008 13:00:42 GMT

Obvious wrote:

> Perhaps XP with a Nero install would reduce

> your time spent troubleshooting.

Nero is just as bad. It has installed it's own MPEG4 demuxers and codecs since at least version 6 and this has caused no end of problems playing videos. I usually reduced the position of the Nero codecs on directshow filterchain using Zoomplayer's filter management, so Dshow would use other codecs that I had installed, like DivX/XviD or ffdshow.

re: The Case of the Slooooow System

Chris Hills — Fri, 10 Oct 2008 13:54:45 GMT

Might I suggest for altering DirectShow filters, a program called "DirectShow Filter Manager" (I wonder why sysinternals didn't come up with this!) available from http://www.free-codecs.com/download/DirectShow_Filter_Manager.htm. It makes it trivial to register, unregister, and alter the merit (a DS term - with values ranging from "preferred" down to "do not use") of each filter.

re: The Case of the Slooooow System

Mat — Fri, 10 Oct 2008 17:37:34 GMT

I agree with Rune. Troll.

Anyway, thanks Mark for an inspiring, informative post.

It never ceases to amaze me how resistant people are to new operating systems that are supposedly 'forced' upon them!

Mark has clearly and concisely illustrated how it's purely an application issue, and how the operating system is built by design not to behave differently for applications that demand the CPU's attention.

Whilst it may affect user experience, we must remember that it's the user who installed Adobe Flash. They clicked the little button that says "I accept".

Well - maybe not the user, maybe the administrator - in which case it is him or her who should take responsibility.

Either way, it's not the duty of an OS to make decisions about what gets priority. I use my computer for some vast graphics and audio number crunching sometimes, I would get pretty pissy if Vista didn't respect my needs as a user of that computer to do the work as quickly as possible!

re: The Case of the Slooooow System

Marc — Fri, 10 Oct 2008 19:11:40 GMT

The comments about efficiency, scheduling, and round robin algorithms are interesting. Having worked as a systems programmer, I agree that it's not easy to code these things so they'll work for everyone without unintended side effects. However, the algorithm chosen has to be appropriate for the user base and the problem set. QNX (a real-time O.S.) was completely different from the various Unixes I used for this very reason.

What I'm suggesting here is that the scheduler for Windows desktop users needs to be changed to reduce the damage runaway apps can cause, rather than trying to produce the highest possible performance. Requirements can conflict, and as Steven Wright says, "You can't have everything. Where would you put it?"

Sometimes you have to make compromises. With raw iron cheaper and faster, maybe it's time to change our mindset about performance and think more about usability under a variety of circumstances, including runaway apps.

re: The Case of the Slooooow System

Warren — Sat, 11 Oct 2008 17:44:31 GMT

My Sloooow Vista System is at startup right after I log in. I can't do anything for a good minute and a half. Is there any way to get Process explorer to log what is happening at this time?

re: The Case of the Slooooow System

John H — Sun, 12 Oct 2008 06:15:49 GMT

Interesting insight from everyone which helps explain what is "hosing" the CPU. I am running XP, IE6, Roxio 6. Knowing I need a new dual core laptop with 4 meg memory, probably this Christmas, I don't like the reactions from Vista users and this will be my only OS choice offerred. I RUN msconfig and uncheck alot of processes and this seems to help, but not all are listed, so thanks Mark for P.E.

I constantly clean the registry with RegScrub XP, scan with Lavasoft AdAware 4 times a day, clean my TEMP folders, and run Spybot Search/Destroy. I am sure we all get sick of doctoring up the health of our PC's. It is still a priviledge to have all this iNFO at our fingertips EVEN IF IT TAKES TIME TO GET IT!

re: The Case of the Slooooow System

Kevin P. Rice — Sun, 12 Oct 2008 15:37:08 GMT

"my wife hadn't noticed it because the remaining CPU capacity was enough to mask it (another good reason to buy multi-core processors!)"

-- I have a program with this statement because if 50% of my processing power (or 25% on a quad-core) is unknowingly being stolen... this could go on for weeks without being noticed. I didn't pay extra $$$ for a quad-core just to cede one or two cores to rogue threads!

Perhaps a monitor is needed that warns when a long-running thread is hogging CPU cycles? Of course, one more info balloon in Vista would just about send me off the deep end!

Ideas?

re: The Case of the Slooooow System

David De Roxio — Tue, 14 Oct 2008 01:32:35 GMT

Hello,

As an amendment to my post above:

Roxio wants users to get the most out of their software. There are a few features and configurations that may not be suited for every machine. That’s why Roxio supplies articles to help adjust and optimize your system.

Imagine a car that didn’t have an adjustable seat or mirrors. Software, and the environments they run on, need to be just as flexible.

Regards

David

Roxio Support

re: The Case of the Slooooow System

Chris Kreukniet — Tue, 14 Oct 2008 10:13:33 GMT

Thanks for sharing your problem and solution. It's nice to read how you troubleshoot your event.

re: The Case of the Slooooow System

SRS — Thu, 16 Oct 2008 15:05:06 GMT

David De Roxio: Nice sentiments, but you did say earlier "if you ever want to optimize your computer's performance..." and the link you supplied suggesting "optimiz(ing) your computer's performance" by disabling bits of Roxio's software - so Roxio believes and documents that computers run faster without Roxio software installed.

Your analogy of adjustable car seats and mirrors doesn't really apply, as adjustment of these components (or even the removal to stretch the point) doesn't change the speed of the car - but the safety of the journey.

re: The Case of the Slooooow System

Richard — Thu, 16 Oct 2008 20:59:12 GMT

From the screen-shot, it looks like Flash had just been updated, and was probably trying to display a notification message. "FlashUtil9d.exe" is the Flash ActiveX updater utility in C:\Windows\System32\Macromed\Flash\.

re: The Case of the Slooooow System

jcg — Thu, 16 Oct 2008 22:05:43 GMT

John H - you are not stuck with Vista on your new PC this Christmas. most vendors should be able to preinstall a XP "downgrade" with your chosen Vista OS, particularly if you visit their web site as a "small business user" (very small = one employee). i just went to the Dell web site, and they will do this for you, even for home users (on some models).

re: The Case of the Slooooow System

jcg — Thu, 16 Oct 2008 22:16:35 GMT

scheduling algorithms don't seem that hard to figure out. there is already a setting in windows that lets you give priority to foreground processes (vs background). it seems that windows may not be interpreting foreground the same way that users do. from a users perspective, the priority of any OS would probably be: 1. mouse and keyboard (if you can't override what's going on or start something new, you're staring at a vista-colored brick), 2. things that have a non-minimized window, 3. things that have a minimized (taskbar) window, 4. things in the tray, 5. stuff i can't see running (background).

re: The Case of the Slooooow System

SPowell — Fri, 17 Oct 2008 16:22:21 GMT

Wow, my wife would never let me delve into her PC that deeply to find a bug, she would have restarted it herself before I would ever have been able to get to it, she's a bit more impatient than your wife apparently. I have to say, these blogs of yours are certainly helpful in creating new troubleshooting steps for similar issues we see as Network Engineers when we're out at a clients location. Your tools are immensely useful and have helped us in more situations than we can count. Keep up the good work, and keep that sense of humor going.

re: The Case of the Slooooow System

Victor C — Mon, 20 Oct 2008 04:04:01 GMT

This was very helpful. I am always being educated as well as entertained.

re: The Case of the Slooooow System

Anonymous — Mon, 20 Oct 2008 15:25:14 GMT

Good article, but I bet you actually googled the CLSID :)

re: The Case of the Slooooow System

gki610 — Tue, 21 Oct 2008 15:45:55 GMT

Hey, folks, Flash has a new one: 10a.ocx. I had software issues since a tech ruined a memory stick, and I found it, removed it, and now nothing crashes. I thought Flash was going to kill me.

re: The Case of the Slooooow System

Tue, 21 Oct 2008 19:03:09 GMT

I've noticed IE spinning when I load lots of tabs (typically from browsing through RSS feeds of blogs and clicking links for later review).

My analysis is that so many sites feature Flash animated ads that there just aren't enough cycles to service them all. As I close tabs the usage drops down again.

ProcessExplorer is a great tool for tracking these things. I also used it looking at slow startup on an old XP Tablet to determine that Windows Update was thrashing reevaluating hundreds of old $NT... artifacts over several years of system updates.

Thanks Mark!

re: The Case of the Slooooow System

Dave Minnick — Wed, 22 Oct 2008 03:45:17 GMT

A lot of information that needs to be carefully digested before attempting.

How about a printable version without the comments, some of which are well intentioned but not necessary. After building a new system I was about to yell at Intel because the system would hang up if I tried to use it before it got done running a 100 or so items. As I have both Flash and Roxio installed I need to look deeper as to the primary cause. I installed Process Explorer and it is not yet telling me what is using up the dual processor.

re: The Case of the Slooooow System

Sam — Fri, 24 Oct 2008 18:59:10 GMT

I have 2 things I want to share too.

1) Thumbnail generation is working for windows explorer. So if that demultiplexer is hogging CPU, you can't just boost the user input by putting the explorer.exe to higher priority because you've just boosted that high consumption process even higher.

It is already much better in Vista SP1 than pre-Vista SP1, which if you have an error in your video file the explorer most often would just hang and get you a COM Surrogate error.

2) people just have a misconception of boosting user interaction process would solve everything. System resource is a scarced thing. The OS is under the presumption that if an application is running on "above normal" priority, it must have a good reason for it. It's a reasonable presumption because, say, if you're burning a DVD, you're most like don't want to disrupt the burning process even the user interaction is a little bit sluggish at the time. FYI, when I burn disc on my Mac OSX, guess what? The system slows down too.

Multicores CPU could helps sometimes, but most programs nowadays are multi-threaded so it wouldn't help by much. The burden of judgement for which process should run higher than the other shouldn't lay on the OS. The user and the application writer should take responsible for that matter... though I have yet to see a solution that would let user manage process priorty that would be easily understand by average user. (i.e. my grandma)

Adobe, out of the good intention, probably wants their Flash player plays as smooth as possible, thus the "above normal" priority, but when it failed it failed very, very hard.

re: The Case of the Slooooow System

Mon, 27 Oct 2008 18:49:01 GMT

I have recently gone through a very similar debugging process using Process Explorer, for my partner, and found that the culprit for a slow unusable machine which has crashed many times and lost hours of work is again buggy Roxio software: the "media watch tray".

Looking this up, I found many sites saying never to uninstall Roxio software, as it uninstalls some components which are written by Roxio, but part of the Microsoft OS and used for CD burning.

I think it is astonishing that someone from Roxio support can claim here that the problem "is resolved". There must be thousands of people who have installed this software and haven't got a hope of finding why their machine is now broken. My partner teaches IT, but using Process Explorer is a bit advanced even for a software egineer like me.

I'm also astonished that they can suggest hacking the registry using msconfig to disable their software, instead of providing a tool that removes it safely.

No software should install components that aren't necessary for the basic functioning of the product without giving the user the option to opt out at install time.

re: The Case of the Slooooow System

Maureen Sampson — Mon, 27 Oct 2008 22:19:20 GMT

It has taken me forever to read all the comments relating to this matter and to be honest it is all too technical for me to understand (I am a silver surfer). I could throw my computer through the window because of its slowness and also the freezing of the screen and the only way I can close it down is manually which I understand is not a good thing for the computer system. I get so frustrated and my blood pressure soars!!! I try to get help and information through the internet but in the majority of cases my computer freezes before I have gone anywhere. The problem seems to start after I have logged on. The arrow moves but nothing happens. Even in the middle of typing a works document the printing stops. I do have Roxio installed on my machine (I think) because a Roxio pop-up comes on when I start my machine. Is there anyone I could speak to or email who could help me with this problem please.I am afraid my usual advisor is at uni until beginning of December. Someone please help me.

re: The Case of the Slooooow System

Patrick Julien — Tue, 28 Oct 2008 05:26:40 GMT

Hi Mark,

What would you recommend to track down this problem:

I have many vista machines and on one machine, just one, where the transition between sessions take several seconds. This happens on login, logout, ctrl+alt+del, UAC prompt or any other session switch, for example, the infor card manager.

The reason I'm stumped with this problem is that the session switch is a very akward spot to get a problem. The session is in the process of being switched so it's hard to actually monitor the system in that state.

re: The Case of the Slooooow System

JV — Tue, 28 Oct 2008 09:39:38 GMT

I love process explorer and process monitor; I used to use other shareware apps which did the same thing before the free sysinternals tools came along.

I'm more minimalist than most users; but I find uninstalling Macromedia Flash and seek-and-destroying any auto-updating or daily version-checking utility (Roxio, I'm talking about YOU) goes a LONG WAY to making my systems more stable, responsive, and overall leaner and meaner and more pleasant to use.

I'm here to seek out the content the google search results told me I was interested in, I have no time for your stupid animated advertising that is just AN OBSTACLE in my way.

So like one would run "top" continuously on a linux computer; I also run process explorer continuously in WinXP to see "what's stealing my cpu cycles?!?"

re: The Case of the Slooooow System

Erik Itland — Wed, 29 Oct 2008 22:44:51 GMT

Somebody mentioned ad block for IE. I think it's included in ie7pro, but I don't know if it's good, since I'm using Linux most of the time. Think it has features like getures etc also.

Mark will maybe just make something smarter than that for what i know, it's just in case anyone else reading this wonders :)

re: The Case of the Slooooow System

a.k.a. — Sat, 01 Nov 2008 05:34:26 GMT

Here's another problem: What is it that's preventing a reader from saving this blog post in IE7 through anything but the Print-as-XPS file option? I've noticed that with the NYT blogs as well, but here there's no Flash ad that's crashing the Save function.

This is a big usability annoyance. Mark - You have the pull with MS to get their CMS programmers to work this out, right?

Thanks!

re: The Case of the Slooooow System

kman — Mon, 03 Nov 2008 19:27:26 GMT

Go to add or remove programs and delete Roxio and Flash. Or, get rid of Vista and go back to XP or better yet buy a Mac. Case solved.

re: The Case of the Slooooow System

eep — Thu, 06 Nov 2008 22:52:52 GMT

I actually read the whole article because this kind of thing bugs me also. The Flash issue would appear to be specific to the content of a particular page. On one popular news blogging sites I used to leave the IE tab loaded all day and the auto-refresh feature of the web page would eventually cause lots of memory and CPU to be consumed after a day of refreshes. The site frequently had active Flash content that was used for displaying advertisements. So, perhaps usage patterns have something to do with the Flash CPU phenomenon, particularly if there is JavaScript or some such other scripting language that is attempting to control the embedded object.

On a lighter note, your quips about Windows Live Search made me think of a nickname that could supplant "google" as a search verb or gesture: wiggle. Thrive Ventures has wiggle.com registered: http://www.thrive.com/

Sorry, but I rarely wiggle, unless I think there's a chance that some Microsoft-specific information might be more readily available there. I use an iMac now and run Windows in VMWare Fusion 2.0, if I ever need to do anything in Windows. I could have purchased a touch screen HP that runs Windows Vista, but I have an iPhone and I decided to immerse myself in the alternative geek lifestyle. I'm loving it so far.

Now, if only I'd bother to turn on my XBox 360...

re: The Case of the Slooooow System

technogeist2k6 — Fri, 07 Nov 2008 03:12:30 GMT

I had a similar problem with Roxio on my laptop. Flash is a recurring nightmare. If it isn't bugs in Flash Player to begin with, it's some badly coded flash content. (deliberate with some sites)

Mark seeing as you're now working at MSFT, couldn't you push these issues up the food chain?

We've been suffering this type of issue for many years, and still your employer insists 'It's the best Windows yet!' Really?

I'm a big fan of virtual server which helps to limit CPU resources, and so prevents the guest OS from achieving 100% utilization when things go awry. Perhaps you should virtualize your wife's PC.

re: The Case of the Slooooow System

MrPete — Sun, 09 Nov 2008 15:34:37 GMT

Mark, there's another situation where the scheduling algorithm makes systems fall apart.

Any time an interaction throws off huge numbers of new processes, the scheduler becomes "priority fragile"...

Right now I'm tracking down a similar system-killer. In this case, a script is cleaning up some unused ACL's by using subinacl. That generates a ton of new processes.

But the real problem is an interaction with Logitech's LVprcsrv service. It hit 1.3 million (!!!) handles before I got the distress cry from my wife (family tech support: the ultimate NMI :) ) and figured out what was going on.

Not surprisingly, Vista has pretty crappy performance when a process has eaten a million handles. But it sure would be nice if the UI had higher priority than scripts, etc.

re: The Case of the Slooooow System

Danno — Thu, 13 Nov 2008 19:38:25 GMT

I remember back in the day when I was programming in a mini-mainframe environment (yes, I'm dating myself ;-). Back then, the OS's scheduler (Primos) was very robust. There were 3 primary components that were admin-controllable: Priority, timeslice, and keyboard burst.

It seems that the NT4 scheduler model (still being used in Vista, according to Mark) is quite obsolete and needs to be redesigned from scratch. While moving priorities up and down can help with sluggish response, it does address timeslice. A process that is assigned a reasonable default, configurable timeslice by the scheduler won't have nearly the impact on the system, since when it's timeslice is up, another process can at least get a shot at the CPU. Also, allowing for mouse/keyboard burst time will allow a foreground, interactive process (typing, moving the mouse, dragging/dropping) to get immediate response. This should be default scheduler behavior, unless the computer has some unusual usage requirement (server, real-time data acquisition, etc.).

IMO, there's difficult to imagine a legitimate reason for there to ever have a 30-sec delay for mouse, keyboard response, unless a process/thread has been assigned real-time priority or an unlimited timeslice. I wonder if any of the 'nix OS schedulers have these features built in. Perhaps someone more familiar with 'nix internals can fill me in on this.

re: The Case of the Slooooow System

Tim Reeves — Fri, 14 Nov 2008 22:00:15 GMT

I'm sure you're not allowed to say so, but another option might be use another browser besides IE!

re: The Case of the Slooooow System

Kropotkins_anarchy — Mon, 17 Nov 2008 00:48:31 GMT

>>>

I'm sure you're not allowed to say so, but another option might be use another browser besides IE!

>>>.

This is exactly that I did. Firefox + addons solved MANY of my problems.

re: The Case of the Slooooow System

Jeff — Mon, 17 Nov 2008 03:57:41 GMT

I have observed the same behavior when using the Live Writer application. The dual CPU usage goes to 100% and typing response slows to a crawl. After 15-20 seconds the CPU load goes back to normal and all is good for a few more minutes before this cycle repeats.

re: The Case of the Slooooow System

ken — Mon, 17 Nov 2008 23:08:10 GMT

how odd. I did a technet search for

http://technet.microsoft.com/en-us/default.aspx?query=badbeef

badbeef.

nothing about bad beef. But the reason I'm looking for this is a SLOW system. And in the process of attempting to restore services to default settings.

maybe I'll find out what badbeef means.

re: The Case of the Slooooow System

Tim Donovan — Fri, 21 Nov 2008 04:06:59 GMT

Mark great post. I too was having some issues with the wife complaining of slowness and your post made me think of checking for this and found the same problem with flash in IEFRAME. I had an old version and upgraded. I didn't have the other problem your wife did but upgrading flash has seems to fix my problem. If I hadn't read this I would have continued to blame Vista for this (and I'm no vista alarmist). Obviously Microsoft can't check for every 3rd party but flash is fairly ubiquitous, I would think that there would be some sort of Windows update sort of alert to advise of things like this. But I suppose legal issues intervene. I make my living as a c# developer and have been on software projects long enough to see that there are trade offs for a successful program that lasts over years (ie. supporting backward compatibility and dealing with hindsight 20/20 complaints) Anyway now when I see similar issues with Vista I will use this technique and act accordingly but it's stuff like this that makes a power user consider Mac and linux just to see what we're supposedly missing. I think they will find it's lonely at the top too if they get there.

Anyways you're tools have saved my bacon many a day and I salute you sir and am glad you are a part of the solution!

Cheers!

Tim

re: The Case of the Slooooow System

ochiru — Sun, 23 Nov 2008 01:48:30 GMT

Hi Mark,

is it possible to trace and undo registry/file changes with Process Monitor or other Sysinternals tools when installing/removing software?.

If it isn't possible yet, I think that it would be easy to implement to your Process Explorer/Monitor. I also think that it would be "A Killer Feature" :D

btw. what's with Adobe installing flash to %windir%\SysWOW64\Macromed? Who would want to install a browser plugin to a system folder?!

re: The Case of the Slooooow System

Nelson — Mon, 01 Dec 2008 15:58:43 GMT

Danno hit the nail on the head. I was going to submit a similar post but he pretty much covered the multi-thread OS concept. No "robust" OS should let ANY process run away with the CPU the way, at least, Microsoft OSs have permitted and apparently still do (due to a faulty and obsolete scheduler implementation) so that no thread gets starved of the CPU for 30 seconds. If a thread hasn't finished its work within its timeslice, it must wait its turn for another timeslice to resume work, barring some reasonable priority scheme (not 1/2 minute).

With the speed of modern CPUs, the user would typically not notice the CPU hog since they would get a timely response from the OS. These concepts have been around for years (before Windows & even DOS) so Windows has no excuse for not having better task management.

I hate it when I click on something like the Start Menu and nothing happens. So you click it again, and wait...and wait, and then, the Start Menu pops-up and then disappears as it executes both clicks, although I've never had to wait 30 seconds. And yes, I have even learned to click it a third time so that the menu will be visible when the OS finally gets around to me.

@Dr. House, Hughe, maybe Denise Miller, or more likely J@CKA$$:

Before ripping into somebody for making a mistake, make sure you know what you are talking about. Exasperates and exacerbates have practically the same definitions and are synonyms, so Yahoo! (I had to throw them in there for fairness/balance) synonyms and apologize.

Also, don't lie and say you don't care. You cared enough to post didn't you?

BTW, (<--acronym around before electronics so I'll assume it somehow isn't considered English language castration by you) there is no such thing as "high-water pants" anymore. The kids now call these shorts:)

re: The Case of the Slooooow System

Walter — Tue, 02 Dec 2008 00:19:18 GMT

Nelson comments:

"Exasperates and exacerbates have practically the same definitions and are synonyms"

This is exactly the sort of exasperating mis-"information" that exacerbates the growing problem with careless and imprecise use of language. :)

Exacerbate: to make a situation worse; more violent, bitter, or severe

Exasperate: to annoy or provoke anger

Agree with you, though, about "Hughe". His diatribe is now immortalized in my collection.

...Great article, Mark.

re: The Case of the Slooooow System

RU — Sat, 06 Dec 2008 21:11:00 GMT

I've been a big fan of Mark's since I discovered Winterals years ago. I love the logical organization of his apps in areas which MS has always needed major help with. (though I've never been able to get your access group dumping app to work in my domains <cringe in frustration>).

After reading a number of these comments after Mark's story I think we have to say that Windows's memory management needs help. Not only should we be able to run a memory gatekeeper app which has a set of rules that looks for issues like runaway CPU hogging processes which has self identification (loops of 1000's of cycles that never end), you can drill through various categories like the stack flow, dll cpu process levels, etc, when they're happening. But also, a way to identify memory allocation to components to identify excess, a 100% detailed and accurate breakdown of how much memory Windows is allocating at start and after apps are run, so you can see why windows needs 800 MB on a fresh startup, but after an hour why is it at 1400 MB, or whatever the issue.

I wish there were companies that would produce memory management subsystems which we could swap in as a superset to correct these problems or help granularly diagnose them.

As far as I'm concerned Mark has done a great job at unveiling some of the mystery with Windows subsystems with his apps. Maybe someone at MS will get smart and make him a project manager of some of these subsystems so the next version of Windows can finally improve it's efficiency rather than just cosmetically.

re: The Case of the Slooooow System

Hieronymous Cheese — Thu, 11 Dec 2008 01:21:30 GMT

My computer has been horribly slow for the past few days. The mouse was sluggish and, as noted above, pressing buttons resulted in an extended delay before anything happened. The keyboard kept missing letters when I typed.

I usually keep the Task Manager open, so I can keep an eye on things, and am therefore familiar with the processes which "should" be running under normal circumstances. Over the past few days I noticed the appearance of the aforementioned "dllhost", which I don't ever remember seeing previously.

Noticing the high CPU usage of Firefox, I decided to change the priority to Below Normal. However, I was surprised to find the priority already set to High. I changed it to Low. Checked again, and it was back at High. Tried several more times, with the same result. WTF??? Nothing like this happened before.

So what changed a few days ago?

I installed Silverlight.

Could this be the answer? Bet yer house it could!

After uninstalling Silverlight, everything is back to its normal speed. Firefox no longer insists on running at Above Normal priority, the mouse zooms around the screen as it should, and the keyboard is functioning correctly.

What the hell are MS playing at with this nonsense?

re: The Case of the Slooooow System

Sanjay Melinamani — Tue, 16 Dec 2008 00:09:01 GMT

This is really a great info for professionals. Applications like Flash, Roxio, AOL etc are the culprits for Windows bad behavior. Unfortunately that is not what others see it. So, I wish Windows team would properly handle such scenario and a threshold alert say 70% CPU. If it goes beyond 70%, let Windows report the info you collected using all your knowledge and the tools to the end user. This helps a lot.

re: The Case of the Slooooow System

Sankaran — Thu, 18 Dec 2008 18:58:23 GMT

Even in linux (Ubuntu, Fedora ...) I am seeing flash taking 100% cpu time.

Why blame windows for it?

re: The Case of the Slooooow System

Ilya Barskiy — Sun, 21 Dec 2008 01:31:30 GMT

Hey, Mark,

Hope you're still monitoring this. This is kind of an appropriate place to post it. There is some kind of a bug or "feature" (clearly not working right) that makes CPU usage on my laptop (and other people's machines) to spike heavily thanks to explorer.exe (windows explorer process) when there is My Web Sites on MSN shortcut present in the system. I don't know where it's from, and once moved to recycle bin the usage of explorer.exe goes from 50% on my new Vista x64 installation to something like 1-3%.

Call stack shows all MS dlls

re: The Case of the Slooooow System

GH — Sat, 27 Dec 2008 01:38:10 GMT

I am sorry for english.

after all years, A giant like microsoft is developing an OS which is Windows and it is getting slower or crashing because of stupid web pages those high loaded with flash files, or the registry is getting corrupted because of bad un/installations,etc.

What the hack thousands of Microsoft OS developer are doing ? are they know thinking ? or designing ? or microsoft management is sleeping ?

It is shame , because that BIG VISTA is getting responceless because of stupid Flash files in the web pages ?

Shame.

I am not anti windowser or lover of mac or lover of linux.

I just want please, drink a black coffee and think and design a working OS, not the Kinder Garden OS.

re: The Case of the Slooooow System

Flatliner DOA — Mon, 19 Jan 2009 17:43:36 GMT

Interesting, I'm now running the Windows 7 Beta and i'm experiencing a similar perf hit in explorer when viewing folders with AVI's in them. Seems like there is a rogue filter, which seems like it is also affecting Media Player and Media Centres performance enumerating files.

re: The Case of the Slooooow System

PuddinPants — Thu, 22 Jan 2009 03:23:16 GMT

If you are running Flash 10 and still experiencing the problem, then drop back to Flash player 9.0.151.0.

re: The Case of the Slooooow System

skeptikal — Fri, 23 Jan 2009 03:52:50 GMT

Regarding the 'Sonic' application in the article, this is not actually part of the Roxio software package.

I believe Roxio bought them out or something, I dunno exactly the deal there.

You got the Sonic software most likely as part of the software with the computer, I'm guessing you had a HP or Compaq.

If you look for updates to Sonic it just kicks you to the Roxio page, my assumption is that Sonic is legacy product & they'd prefer you just remove Sonic & install Roxio.

You might be able to get an update by going to HP/Compaq and doing a software update instead of trying to do the app by itself.

re: The Case of the Slooooow System

Sam — Fri, 23 Jan 2009 04:16:42 GMT

Surely the correct way to fix IE is to install Firefox or its variants, and flashblock? Then you can ONLY allow flash you WANT to run, saving your DL as well.

re: The Case of the Slooooow System

Ben U — Fri, 23 Jan 2009 07:19:20 GMT

Wow. Slow Vista System? Let's go through twenty steps no normal human could ever accomplish, then decide that simply solving half of the issue is good enough because you've got enough raw hardware that the user won't notice.

Awesome.

Why not just get a quad core? Then you can go another two years without fixing a Windows issue.

re: The Case of the Slooooow System

mojo — Fri, 23 Jan 2009 08:06:29 GMT

i run flash in firefox on linux and periodically have seen similar problems stalling the browser. in a fit of youtube click-itis i may see several tabs with flash. this has not been so much a problem since getting the 64bit native flash plugin that they finally released. previously i had been using nspluginwrapper to run the 32bit version. the usage still maxes some times and the browser becomes unresponisve, but the other issues i had are gone. with the 32bit plug-in, some times after a while the tabs will not load the flash without a few reloads, and other times all the flash content dies leaving grey boxes in their place on every tab that had some.

re: The Case of the Slooooow System

mojo — Fri, 23 Jan 2009 08:07:21 GMT

sure wish flash were not closed source ;)

Cut it out

Fran Taylor — Fri, 23 Jan 2009 08:49:22 GMT

Anyone who knows me will know that I'm the last person to defend Windows, but these sorts of problems happen with EVERY operating system.

For example, on Unix/Linux, an application can pass junk to the X server and totally hose the server. You can move the mouse but otherwise it's just dead. You can kill the X server or restart the system, both are bad. I don't know OSX well enough to develop a scenario, but haven't we all seen more than our share of that spinning beach ball.

Schedulers are black voodoo magic on every operating system. Even the most minor of 'tweaks' can have grave consequences. Look at the developer's mailing list archive for any open source operating system, and you will see LOTS of back and forth about schedulers.

Where Windows falls down for me is that you have to be Mark Russinovich to diagnose these problems. With Linux, everything is out there in the open, you have symbols and source for EVERYTHING (except Flash!), and you can track down the problem readily.

re: The Case of the Slooooow System

Baughn — Fri, 23 Jan 2009 12:00:55 GMT

I don't really use windows much, so maybe I'm missing something obvious here, but wasn't the real issue that programs are *allowed* to run at above normal priority?

On linux, everything defaults to priority 0. You can lower the priority if you want, but only the superuser is allowed to *increase* it; thus, this sort of problem can't happen. (There's also the usual bunch of features like boosting the priority of inactive programs, etc.)

Also, having an above-normal priority program hogging the CPU means normal-priority ones don't get to run at all? Shouldn't it be more of a proportional boost? Getting unquestioned access to the CPU is what realtime priority is for.

re: The Case of the Slooooow System

Tom — Thu, 29 Jan 2009 06:45:28 GMT

I've been frustrated with the Windows scheduler since NT. You can explain priorities to me all you want but I don't see that as an excuse for something that normally takes a fraction of a second to take 30 seconds instead.

I write multithreaded CPU-intensive scientific code. If I'm in Windows and run with 4 threads on my quad, the system becomes unusable. It takes 5-20 seconds to check my email, bring up a new browser window, etc. Ridiculous. My only solution is to run with 3 threads and wait longer for my job to finish--i.e., I'm less productive.

If I run the same code with 4 threads under Linux or OS X, it's as fast as it should be and I can barely tell it's running. In fact, sometimes I literally forget that it's running and hours later I'll notice the louder CPU fan speed or _slightly_ sluggish browser performance and realize what's been going on.

I actually like WinXP quite a bit and use it for writing e-mail, surfing the web, etc., but because of the scheduler and certain other deficiencies it seems that Windows is unsuited to what I would call "real work."

re: The Case of the Slooooow System, ROXIO

Huub van Dijk — Wed, 04 Feb 2009 04:35:37 GMT

I had problems with ROXIO similar to those you encountered.

- incompatiblility with AVG Free edition

- AVG will refuse to update or install

until ROXIO is updated

- I did not manage the hugely complicated and

possibly faulty registration process

for my OEM installed ROXIO

- I could not update ROXIO

- ROXIO has a legion of autorunners

re: The Case of the Slooooow System

Alexandre Grigoriev — Fri, 13 Feb 2009 03:36:33 GMT

Rune,

Unfortunately, Windows scheduler is no good against the runaway threads. And it's very easy to detect those threads - they always use up their time slice. It's very easy for the scheduler to give them dynamic priority bust (as opposed to boost).

Even though Windows is supposed to give a priority boost to a foreground window, it doesn't seem to help.

And another strange thing of Windows scheduler: explorer file copy operation may take MUCH longer if you put its progress window to background. Why?

re: The Case of the Slooooow System

B.Good — Tue, 10 Mar 2009 20:44:55 GMT

The thought did cross my mind, "I wonder if he would have tried Firefox if he wasn't now paid by Microsoft?" That said, I realize this is a scheduler problem. And possibly one that afflicts other operating systems based on some posts here. However, can't some of the best programmers on the planet figure out SOME way to keep the entire OS from becoming unresponsive? If only to pop up a box that says "Program XYZ is consuming an unusually large amount of resources, would you like to reduce its priority to make your computer more responsive? (Yes|No) Note that clicking 'Yes' may make Program XYZ run more slowly or take longer to complete."

re: The Case of the Slooooow System

Neel — Thu, 12 Mar 2009 16:25:25 GMT

Hi,

Thanks for the article. I reached here after searching for defects that mentioned high cpu usage and I found https://bugs.adobe.com/jira/browse/FP-417

Someone mentioned that flash programmer wrote an infinite loop.. absolutely.. I don't see ANY other way this can happen. It has to be while (1).. grrrrrrr... I try to download multiple videos and watch later (because it's much better than embedded ones) and when I start two IE instances, because of this flash problem, all I can do is watch the scrollbar until it completes the video downloading..

This is a perfect oppurtunity for some millionaire to show his kindness and announce reward for any Flash programmer to find root cause of this.. I don’t see any other way this defect getting any attention otherwise.. (search for cpu usage related defects on adobe flash bugs page and see how many of them are happily unresolved..)

Thanks,

-Neel.

re: The Case of the Slooooow System

Jason — Thu, 12 Mar 2009 18:29:56 GMT

All of the pc's I use at work and at home run XP, so I don't really know about the problems with flash player on Vista..I've never used it!

I have seen this CPU hogging problem with flash on XP, but only a few times.

At my last job, I worked for a few years at a UK based e-learning company, developing (amongst other things) flash based elearning content which was delivered both online and offline. (I'm working as a C++ analyst/programmer nowadays at a CAD/CAM software company!)

When our content was running in Internet Explorer or FireFox, (online or offline) the CPU usage wasn't adversely affected.

However, We did encounter the problem a few times during the initial development of the templates for our flash interface and content.

But this was usually down to some poor logic or a bug in the actionscript (usually something stupid causing an infinite loop, or some complex, convoluted logic that was just generally screwing things up and causing hangups)..But we managed to fix the problems and the final flash software turned out to be brilliant!

However, whilst browsing the web; I've visited a few sites recently which contained flash content which caused the CPU usage to jump alarmingly high and stay there until I navigated away from the offending page.

However on the flip-side of that, I've also been on a lot of sites where the flash content has ran smoothly and without a glitch!

So personally, I don't believe it to be a problem with Adobe's Flash browser plugins, I think it's purely related to the people producing the flash content.

It seems to be down to the flash developers Quality control processes (or lack thereof!).

There is one flash related issue that's been annoying me recently though:

I've noticed that a lot of advertisers on various sites have posted debug builds of their flash adverts.

The reason that's annoying (other than the fact that they're bloody adverts!) is because I use the debug version of flash player (as I still do a bit of flash development here and there myself) so whenever I encounter one of these debug builds in a web page, I keep getting annoying popups asking where my local debugger is..Grrrr...I don't want to debug their stupid advert!

Again, proof of the point that some flash developers have no quality control systems in place...

I will stress again, they seem to be in a minority as most flash content on the web is ok! (in my experience)

So to my mind, Flash developers need to do a few simple things....

1. Properly plan and design their projects..

2. Code their projects more carefully to minimise/eliminate the risk of hangs/infinite loops/bugs.

3. Properly plan the Debugging and testing of their projects, again to minimise the risk of bugs in their final release.

4. Ensure that the final uploaded content is a Release build and not a bloody Debug build!

All pretty fundamental stuff..Fundamental to programmers anyway!

I suspect that most of the offenders in these cases are more artistic types with no formal programming background...That or just sloppy programmers!

Anyway, that's my 2 pence!

Jas.

re: The Case of the Slooooow System

Jason — Thu, 12 Mar 2009 18:33:46 GMT

P.s. Great article by the way Mark.

I've been using the sysinternals tools on and off for several years now and they've proved invaluable!

re: The Case of the Slooooow System

we — Fri, 13 Mar 2009 00:59:30 GMT

What you are all missing out on here is that it is not Flash fault:

It is an OS DESIGN PROBLEM.

Try this:

Launch a browser with Flash or any media player on XP or Vista. You will see your other app using OpenGL or DirectX massively impacted.

What a crappy OS Windows is..

Why slow at NORMAL priority. Yielding?

JMV — Sun, 12 Apr 2009 21:16:26 GMT

I regularly do serious calculations on my system. If such a process runs in normal priority, like mathematica, such a system slows down a UI process by more than 50% on a single core system. The slow down can be by a factor of more than 20.

WHY???????????????????

Is it because a UI process switches threads constantly, and ends up effectively yielding all the time?

Or is it because it makes kernel calls all the time?

The Case of the not so Slooooow System

JMV — Sun, 12 Apr 2009 21:20:56 GMT

To TOM:

Knock down the priority of your scientific code on your quad!

You'll be relieved.

what about BeOS

Cliff — Sun, 28 Jun 2009 05:12:33 GMT

When I was playing around with and reading about BeOS, one of the things that impressed me is that the UI thread *always* had priority; no matter what happened the UI *always* stayed responsive.

Am I misremembering this?

re: The Case of the Slooooow System

TIm — Tue, 22 Sep 2009 16:48:53 GMT

A wonderful article with excellent explanation of your trouble shooting approach. I already knew a little about Process Explorer, threads, dlls, etc. but this article puts it all together.

Thank you Mark.

-Tim

re: The Case of the Slooooow System

Nathan Schultz — Wed, 25 Nov 2009 18:36:03 GMT

When will process explorer have the ability to save the session. E.G. I would like process explorer run in the background and record the information to file for later playback. Is this possible? If not is that on the drawing board?

re: The Case of the Slooooow System

molotov — Fri, 27 Nov 2009 10:51:17 GMT

@Nathan Schultz

Not quite the same as you appear to be inquiring about, but have you checked out Process Monitor?

http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx