Mark Russinovich’s technical blog covering topics such as Windows troubleshooting, technologies and security.
A few weeks ago a poster with the handle dloneranger reported in the 2CPU forums that he experienced reduced network throughput on his Vista system when he played audio or video. Other posters chimed in with similar results, and in the last week attention has been drawn to the behavior by other sites, including Slashdot and Zdnet blogger Adrian Kingsley-Hughes.
Many people have correctly surmised that the degradation in network performance during multimedia playback is directly connected with mechanisms employed by the Multimedia Class Scheduler Service (MMCSS), a feature new to Windows Vista that I covered in my three-part TechNet Magazine article series on Windows Vista kernel changes. Multimedia playback requires a constant rate of media streaming, and playback will glitch or sputter if its requirements aren’t met. The MMCSS service runs in the generic service hosting process Svchost.exe, where it automatically prioritizes the playback of video and audio in order to prevent other tasks from interfering with the CPU usage of the playback software:
When a multimedia application begins playback, the multimedia APIs it uses call the MMCSS service to boost the priority of the playback thread into the realtime range, which covers priorities 16-31, for up to 8ms of every 10ms interval of the time, depending on how much CPU the playback thread requires. Because other threads run at priorities in the dynamic priority range below 15, even very CPU intensive applications won’t interfere with the playback.
You can see the boost by playing an audio or video clip in Windows Media Player (WMP), running the Reliability and Performance Monitor (Start->Run->Perfmon), selecting the Performance Monitor item, and adding the Priority Current value for all the Wmplayer threads in the Thread object. Set the graph scale to 31 (the highest priority value on Windows) and you’ll easily spot the boosted thread, shown here running at priority 21:
Besides activity by other threads, media playback can also be affected by network activity. When a network packet arrives at system, it triggers a CPU interrupt, which causes the device driver for the device at which the packet arrived to execute an Interrupt Service Routine (ISR). Other device interrupts are blocked while ISRs run, so ISRs typically do some device book-keeping and then perform the more lengthy transfer of data to or from their device in a Deferred Procedure Call (DPC) that runs with device interrupts enabled. While DPCs execute with interrupts enabled, they take precedence over all thread execution, regardless of priority, on the processor on which they run, and can therefore impede media playback threads.
Network DPC receive processing is among the most expensive, because it includes handing packets to the TCP/IP driver, which can result in lengthy computation. The TCP/IP driver verifies each packet, determines the packet’s protocol, updates the connection state, finds the receiving application, and copies the received data into the application’s buffers. This Process Explorer screenshot shows how CPU usage for DPCs rose dramatically when I copied a large file from another system:
Tests of MMCSS during Vista development showed that, even with thread-priority boosting, heavy network traffic can cause enough long-running DPCs to prevent playback threads from keeping up with their media streaming requirements, resulting in glitching. MMCSS’ glitch-resistant mechanisms were therefore extended to include throttling of network activity. It does so by issuing a command to the NDIS device driver, which is the driver that gives packets received by network adapter drivers to the TCP/IP driver, that causes NDIS to “indicate”, or pass along, at most 10 packets per millisecond (10,000 packets per second).
Because the standard Ethernet frame size is about 1500 bytes, a limit of 10,000 packets per second equals a maximum throughput of roughly 15MB/s. 100Mb networks can handle at most 12MB/s, so if your system is on a 100Mb network, you typically won’t see any slowdown. However, if you have a 1Gb network infrastructure and both the sending system and your Vista receiving system have 1Gb network adapters, you’ll see throughput drop to roughly 15%.
Further, there’s an unfortunate bug in the NDIS throttling code that magnifies throttling if you have multiple NICs. If you have a system with both wireless and wired adapters, for instance, NDIS will process at most 8000 packets per second, and with three adapters it will process a maximum of 6000 packets per second. 6000 packets per second equals 9MB/s, a limit that’s visible even on 100Mb networks.
I caused throttling to be visible on my laptop, which has three adapters, by copying a large file to it from another system and then starting WMP and playing a song. The Task Manager screenshot below shows how the copy achieves a throughput of about 20%, but drops to around 6% on my 1Gb network after I start playing a song:
You can monitor the number of receive packets NDIS processes by adding the “packets received per second” counter in the Network object to the Performance Monitor view. Below, you can see the packet receive rate change as I ran the experiment. The number of packets NDIS processed didn’t realize the theoretical throttling maximum of 6,000, probably due to handshaking with the remote system.
Despite even this level of throttling, Internet traffic, even on the best broadband connection, won’t be affected. That’s because the multiplicity of intermediate connections between your system and another one on the Internet fragments packets and slows down packet travel, and therefore reduces the rate at which systems transfer data.
The throttling rate Vista uses was derived from experiments that reliably achieved glitch-resistant playback on systems with one CPU on 100Mb networks with high packet receive rates. The hard-coded limit was short-sighted with respect to today’s systems that have faster CPUs, multiple cores and Gigabit networks, and in addition to fixing the bug that affects throttling on multi-adapter systems, the networking team is actively working with the MMCSS team on a fix that allows for not so dramatically penalizing network traffic, while still delivering a glitch-resistant experience.
Stay tuned to my blog for more information.
FWIW on the subject of media playback, video, audio etc. I was having problems with music tracks and video (audio) skipping in Vista Ult sp1, Core 2 duo, 2Gb ram.
Music playback skipping in iTunes and WinAmp, Video (audio) skipping in Win Media Player and Quicktime. So I read through this thread (thxs Mark for the article and insight), tried a few things with no success, still audio playback skipping). Then, I disabled my 'wireless NIC' and the audio skipping problem disappeared, re-enabled it and the audio skipping came back, disabled it and the skipping went.
So at the moment the audio playback skipping has ceased. Not looked into the network throughput issue yet.
Does anyone else get similar behaviour to this? The audio playback skipping was driving me to distraction and after browsing for solutions, I couldn't find any that worked for me.
Ahh, I think you solved my mystery below:
I have a Vista Ultimate based HTPC (which used to run MCE 2005 prior to my doing a clean Vista install) and a Win XP Pro desktop on the network. Both machines have Gigabit ethernet, and are connected via Cat 6 cable across a D-Link DGL-4300 Gigabit router. Jumbo packets are disabled on both ends. Both have latest chipset drivers, NIC drivers, Vista patches, etc installed.
Prior to clean-installing Vista, I was getting transfer speeds of 35 MBytes/sec between the HTPC and the WinXP Pro desktop. After clean-installing Vista, I noticed that this dropped to a measley 11 MBytes/sec!
I've tried several solutions, including turning off Vista's firewall (transfer rates went 11 MB/s -> 16 MB/s). I also tried turning off autotuning (no difference) and turning off remote differential compression (minimal difference).
Finally, I realized something important:
If I had VMC running in the background (idling mind you, ie. not playing video, not accessing network, etc), then my transfer rates were limited to 16 MB/sec with Vista's firewall off. However, if I exited completely out of VMC, then my transfer speeds shot back up to 35 MB/sec to the WinXP Pro desktop!
I think this is the reason why loading ISO files from within MyMovie's VMC interface (auto-mounted through Daemon Tools) takes FOREVER with Vista (45-60 secs), but was relatively fast in MCE 2005 (~10 secs).
Anyone have any ideas why VMC running in the background would cause a > 50% drop in network speeds, even though VMC isn't doing anything with the network?
Leo Davidson commented on August 28, 2007: "MS (or at least two MS employees in their blogs) are being quite open about the fact there is a problem that needs fixing."
Yet here I am in late June 2008, with Vista SP1+Hotfixes and the problem is as bad as it has always been.
Firing the guys who invented this would be too good for them. I'm not sure if "hanged, drawn and quartered" would be appropriate, but it would be a start - considering what they have done.
I use a sat receiver to watch HD streams.
That constantly has CPU usage between 50% - 70%
In xp I just gave the sat software priority of high (13) that provided a glitch free experience.
Even if a antivirus scanner was running at the same time
But in vista there were glitches.
Have tried many different things but nothing worked.
until i disabled MMCSS.
Now I have a glitch free experience once again
This is one of the silly features, why vista has such a bad reputation.
It seems that Vista is designed for a PIII with 100mbit network. This cant be serious !!!
I have a problem that is the opposite to this. When ever there is network activity video and music playback is affected it is 'jumpy' and 'laggy' even when playing flash videos in the firefox, I have to wait until the video has buffered fully before I attempt to watch it. This doesn't happen as soon as I turn on my laptop but after about an hour it starts.
Exact same problem, after a clean reboot, my system can play vids and download 500kb/s easy, but after an hour or two, any network activity causes media to jump, and also my mouse to jump and basically my system becomes unusuable at around 500 kb/s activity. Basically my system is unusable to download torrents or any file sharing. Pretty rediculous.
My Laptop have the similar problem: Most of MP3 or CDs play properly in XP or earlier windows PCs, but they always have some disruption problem with my new Vista PC, either "glitch, sputter or something else". It's very annoying as I always enjoy listening musics from my PC (and seldom care about network speeds).
When the wireless network was switched off, music playback seems getting smoother.
But if I chose to disable the MMSC, the laptop can not even play any musics.
Can you or anyone help on this? Thanks in advance.
Ever since i've had Vista i've had this issue in varying degrees. When I used to use a USB adapter (Wireless N) I would experience music slowdown every minute on the dot.... very annoying. I assume it was polling the AP.
I went back to a wired connection (with a bastard long cable) and everything was fine up until my network traffic became high, and then the music would once again become stuttery.
Very very glad to have finally come across an answer, by luck of all things. Will have a play around with disabling MMSC. Its funny, now I have read this, even keeping an eye on NetMeter during playback of music proves the point...
Hmmm, it's quite funny ... I have Vista SP1 with the same problem. I lost many hours with tuning stupid speaker fill with no success. Rear speakers just doesn't work anymore in Vista. Also I spend many hours with this stupid LAN problem ... and the solution is just f***ing easy! Disable MMCSS! It should be by the default in SP1 but it isn't. Then it should be at least on Microsoft homepage but it isn't. Vista is out about 2 years and nobody fixed this? Well I'm quite dissappointed ... Aero is cool, but it cannot be altered that suxx. Taskbar colors are absolutely flat and hard to distinguish active application window ... alttab is absolutely awkward, hopefully there is undocumented AltTabSettings which disables that "cool & stupid" thumbnails ... if I would decide now if I spend tens of hours solving Vista stupidity (like missing standard desktop themes which could cost 10kb or less on installation DVD?), I would stay on XP ... maybe with Windows 7 I will be so dissatisfied that I will finally "upgrade" to XP again ...
I have tried this and every other tweak I could find. nothing has fixed this problem for me. I hate vista
I have tried it on vista x64. I noticed that the network throughput tripled!! (from 12MB/s to 37MB/s) when media player was running. I have Intel Quad Penryn processor and video playback performance is not a problem at all. Whoever programed this stupid prioritizing in vista deserves the worst programmer of the year title. his manager too.
I've been having horribly corrupted / hiccuping / crackling / popping audio under Windows Vista on Quad Core laptop.
So far, disabling processor speed scaling seems to have solved the problem.
I've tried tweaking both the SystemResponsiveness and NetworkThrottlingIndex register variables that were added to Vista SP1's MMCSS.
The MMCSS service seems to be a badly designed nasty hack that was thrown in at the last minute rather than a real attempt to fix the various problems with audio scheduling in Vista.
I have got the stuttering audio problem in Vista 64, sp1. Turning off MMCSS didn't work at all.
How do I tweak Systemresponsiveness and NetworkThrottlingIndex register variables(what is the registry hack)?
This is making me hate Vista once again! Will probably have to return to xp or even xp64 and cross my fingers that these things will work in Windows 7.
Vista has been out for more than two years now, and still this incredibly simple thing like playing back some audio (or video) doesn't work!
It's beyond belief how bad it is, no wonder Vista tanked so much that even Ballmer admitted it was a failure!
I found how to tweak the Systemresponsiveness and NetworkThrottlingIndex, but that didn't work either.
Nothing works! Sound still stutters terribly whenever I do some torrenting or something else involving networking activity.
I have a quadcore(penryn) cpu, 4 gigs of ram, a (disabled for testing now) wireless network usb-adapter, two network cards ,one for wired internet via the router and one attached to a network printer(which i will attach to my router, but it has stayed directly attached for a while due to my laziness)
And I don't use an integrated audio card, instead I use a professional sound card, the ESI Juli@.
Strange thing is that it has worked flawlessly for half a year (since I started using vista 64) up until this week.