Blog du Tristank : x64 Early Adoption

Test Signing X64 Drivers

tristank — Tue, 30 Jan 2007 06:16:00 GMT

Well, I did learn a lot today!

You may be familiar with the restrictions that Windows Vista X64 places on unsigned kernel-mode drivers (I was only vaguely familiar with it). This is new - XP64 (which I upgraded from) would warn but allow installation, from memory - WV64, the driver won't load.

There was a driver for a USB device that I absolutely, positively had to have working, so I looked around for ideas on how to approach it.

Here's the step-by-step guide. Useful for testing while developing a driver, or in situations of dire need.

ISA Server Firewall Client for Windows Vista

tristank — Mon, 22 Jan 2007 03:37:31 GMT

Over here.

Works with all ISA Server versions to date, and with Windows Vista client, importantly! x64 and x86 versions included. Safe when taken as directed.

via Josh and KB article 929556.

DebugDiag 1.1 Released!

tristank — Wed, 17 Jan 2007 10:36:31 GMT

IIS Troubleshooters rejoice!

If you've ever had an application on an IIS server do one of the following, this tool is for you:

- hang (or respond slowly)
- crash (W3WP.exe terminated unexpectedly)
- chew memory in the lead-up to a crash or need a restart on a regular basis (memory climbs, until it falls over, then memory starts climbing again)

DebugDiag automates much of the routine debugging work that we perform in PSS. We're still useful for something (I'm told; my manager is supportive that way) but I'd guess that a good 50% of problems we see can be understood and resolved through the use of this tool.

And it's not just for IIS, I should add - it's a generally useful debugging tool, that just happens to be focused on IIS. But you could use it against pretty much any process in ad-hoc mode (where you just point and shoot a process to get a memory dump, though with Windows Vista, a dump file is always just a Task Manager click away!).

Once you have a dump file, the Hang/Crash and Memory Leak analyzers in DebugDiag can be run to provide analysis of that file, hopefully work out what's gone wrong, and produce some recommendations about what to do next. Brilliant, no?

New in 1.1 (this is a quote, so please don't shoot me if there's something inaccurate here):

· IE7, Vista, and 64-bit Windows compatibility (currently targets only 32-bit processes, but works on 64-bit platforms)
· Leak rule improvements (dump on memory thresholds)
· Leak monitoring improvements (‘fasttrack’ feature, injection fixes)
· Analysis improvements (Data access components and socket support)
· Custom actions for crash rules (DebugDiag scripts)
· Managed call stack resolution
· Support for managed exceptions (such as System.DivideByZeroException)
· Support for debugger events (such as Module Load and Unload)
· Sample scripts included

I'm itching to give it a try! It's available here.

Netmon 3, Now With Added Blogging!*

tristank — Tue, 24 Oct 2006 08:08:00 GMT

Though it might technically be possible, it's probably quite difficult ("quite" in the "very" sense) to blog from Netmon.

Doesn't stop us rolling out the most significant new version in, oh, years, and the Netmon team have started a blog with tips and things for the new version.

It's a ground-up rewrite, it's all new, and it's the first version (er, of the third version), so it's still rough in places, but getting better all the time. And live capture viewing FTW!

So, go check them out. Netmon3 can be beta'd using connect.microsoft.com. (Note that you're looking for Network Monitor 3.0" rather than "Netmon") There's an X64 flavour as well, making life easier than the Netcap + Ethereal combination I've been using...

PAE and VMM... For Parky

tristank — Fri, 26 May 2006 16:50:00 GMT

Well Parky, you asked, so I'm going to try to answer!

The way I think about PAE is that it kinda works a bit like a stonking great in-memory pagefile might. It doesn't change the game for 32-bit applications, but it does give the OS more headroom to manage them.

Without PAE, any memory over 4GB can't be "seen" by the OS itself, so it can't be used.

With PAE, the memory manager can see all the installed memory, but it doesn't change the per-process or kernel limits.

So if, for example, you ran 3 database programs at once, each of which used their entire 2GB user address space, with a PAE box and 6+GB, the whole lot would potentially fit into memory (assuming your kernel didn't mind getting squeezy).

So, in really short form: almost the same architecure, but more RAM!

And you're right on the other front - after a certain point on a 32-bit Terminal Server, the limiting factor is likely to be kernel address space, so if you're eyeing PAE as a possible answer and you haven't yet deployed the box, consider going x64 instead.

Now, here's the other attempt I was working on, but un-fact-checked and likely subtly (or grossly) misleading - think of it as a work-in-progress lie. For the actual story, hit "Windows Internals, 4th Edition", by Mark Russinovich and David Solomon.

I'm unlikely ever to finish this, so I figured I might as well post it for... um, well, fun, and to show I care enough to try* (for a while) :)

--------------

Trial #1: Intro to PAE using Sheep as an accessible metaphor

32 Bit Addressing = 386

The CPUs we know and love today are all descendants of the i386. The '386 was a chip that had 32 address lines, which is a techy way of saying that it could talk to up to 4GB RAM. 32 bits = 4 billion possible individual memory locations.

36 Bit Addressing = PAE

Physical Address Extension is a 36-bit addressing thingamabob that got tacked onto Pentium Pro and later CPUs.

PAE is cool, because the extra 4 bits mean that the processor can talk to a whopping 64GB of RAM, instead of the paltry 4GB that seemed so cool just ten short years ago. Heck, in ten years' time, my phone will probably have 2GB onboard!

Enter The Sheep Metaphor

In short: each process gets a 4GB address space, and the lower 2GB is a play area unique to that process. All processes share their address space with the same kernel, which is the upper 2GB.

Let's say that Sheep is our 32-bit Windows program. When Sheep is started by the OS, it'll be plonked into a 2GB field in which it can play, with a small amount of nasty barbed wire near the very bottom, and a big wall with a tiny window near the top.

The Kernel memory area is another 2GB field beyond the wall, with a sign tacked to the front "All that 2GB field is yours, except this 2GB here. Attempt no grazing here."

Any other programs you run get put in their own totally separate 2GB field (say, SkyscraperBuilder.exe) but they all see the same Kernel field. It's a bit like the dead people in the Sixth Sense (honestly, if you haven't seen it yet, you need to stay in more) - they can't see each other, but the kid (Kernel Kid™?) can see them.

With me so far?

All This Could One Day Be Yours (but you have to allocate it)

Windows doesn't just hand each process a fully allocated 2GB field of memory - (count the number of processes running on your computer at startup; now imagine having to install 2GB of RAM for each process to run!) - it gives it just enough to get it loaded, and then the process has to actually ask for what it needs.

A Sheep might only use less than 1% of its field while it's wandering in a small area and grazing, whereas the SkyscraperBuilder is likely to try to use all the space it has available, and subsequently harangue, harass and attempt to blackmail the planning authority for more.

But at the beginning, they both believe that the field is empty, and they just start asking for memory.

Virtual Memory

Virtual memory - to cut a very, very involved story of lies and deception rather short - is how the OS manages to allow each application to request and use memory that all seems nice and contiguous to the application, but is actually "backed" by memory in a physical location elsewhere - and that "elsewhere" can be somewhere else in RAM, or on the hard disk, in the pagefile.

The Kernel address space itself is virtualized, though k-mode components are able to "look behind the curtain" if really necessary.

In a situation where you've got less memory than 4GB, VM means that everything gets to actually run, while having this wonderfully seemingly neat memory area to play in, and room to grow.

If you've got the whole 4GB (which is our theoretical maximum at this point in the discussion) and a bunch of tiny programs, everything's going to go swimmingly.

But just flip that on its head for a second - just say your requirements were greater than 4GB. Say that the amount of memory actively used by all the programs on your computer (called the "Working Set") exceeds 4GB. Say that all up, you really need 6GB in RAM at one time, across a bunch of processes.

The CPU can only use 4GB total… so even if you somehow drop 8GB into the machine, you're in for some paging (hitting the hard disk to swap memory in and out of physical RAM) without PAE; the OS can only keep track of so much memory.

But Enable PAE, and whop! The CPU can now use however much RAM you've got in the box (up to 64GB), so less paging happens. The kernel/user split is still the same - we're still talking 2GB user space per application and 2GB kernel space, so it's business as usual to each process on the machine - it's just that the virtual memory manager can now use all the RAM in the box to satisfy demand before having to go to the page file.

Microsoft.com Operations on x64

tristank — Tue, 27 Sep 2005 02:50:00 GMT

The Microsoft.Com Operations blog just posted about their experiences moving from x86 to x64, most notably the increase in performance they achieved by doing this.

The numbers are compelling:

To give you a quick comparison:

X86 ASP req/sec 7.85, Response time 244ms
X86 ISAPI req/sec 110.85, Response time 248ms
X86 Static req/sec 41.9, Response time 135ms
X86 Static (cached) req/sec 47.11 Response time 1ms

X64 ASP req/sec 7.41, Response time 53ms
X64 ISAPI req/sec 125.43, Response time 18ms
X64 Static req/sec 31.01, Response time 3ms
X64 Static (cached) req/sec 54.51 Response time 1ms

And this is performance gained just by running a 32 bit application (ASP.Net 1.1) on x64, via the WOW64 emulation layer (WOW indeed!)

Worth a read.

x64-friendly Smartcard Reader

tristank — Thu, 22 Sep 2005 11:51:00 GMT

I've had a Gemplus 430 USB card reader since we started using Smart Cards for RAS, all those years ago.

Since the upgrade to X64 at home, I've been essentially VPNless. No X64 drivers.

But finally, as if by magic, I came across (and successfully bartered for) a 433. I know X64 drivers exist for this. This makes me happy: I prefer Remote Desktop to web-based protocols by a longshot (especially as half my email-borne life needs to end up in a PST file - if Outlook isn't running for a day, I bust my mailbox allowance).

Ctrl+C In A Message Box Does What!?

tristank — Tue, 23 Aug 2005 09:54:00 GMT

OK, so now searching for this reveals it as a widely-known and somewhat ancient tip, but I didn't know until this week. I can't remember where I saw it.

I can't tell you how many hours this might have saved me over the years, because I DIDN'T KNOW ABOUT IT YEARS AGO. Sigh.

Ever faced with a dialog box that you desperately want to keep the text from?

Have you been resorting to OneNote, PrintScreen (ok, PrtScn if you're that way inclined) or screen captures for message boxes, like this?

Well, as it turns out, pressing Ctrl-C while the little messagebox has focus results in this being copied to the clipboard:

---------------------------
\\bwahahahaha
---------------------------
\\bwahahahaha

The network path was not found.

---------------------------
OK
---------------------------

Man, if only I'd known. I will make it my personal mission to evangelize this until there's nobody left that feels like they have to send a screenshot of a dialog box by email any more*.

Note that it doesn't work for all message boxes, and I haven't really put a ton of time into working out which ones it does or doesn't. I guess just the non-custom Win32 sort - the message from Word telling me I need at least a "to:" entry in an email isn't capturable, for instance. But anyway, if it saves eight clicks just once, it's a good tip.

Another set everyone knows already:

Windows Key + R = Run dialog, straight away (though I find myself using the MSN Deskbar more as a pseudo-batch-file-CLI-type-thing these days). Especially useful if you've converted to the New-Fangled Start Menu and have more than one item beginning with R.

Windows Key + L = Lock Workstation.

64 Bit: It's When, Not If

tristank — Wed, 10 Aug 2005 09:44:00 GMT

Harold raised the question, Clive riffed on it, and now I'm going back to the original question with this:

At the moment, is it even possible to buy a performance chip that isn't 64-bit capable?

Any AMD Athlon64 or Opteron is AMD64-capable; Intel Xeons have been for a while, and EM64T is moving into their desktop offerings too, with notebook variants on the horizon.

So the question isn't about whether or not to buy a 64 bit chip - chances are you will be buying an x64-capable chip anyway when you next purchase one* - the question is really about when to actually throw the switch and move to a 64 bit OS.

For most consumers, that's probably not just yet. I think it's important to acknowledge that, because bluntly, it's not like we're going to be selling a copy of X64 to my parents, and right now, we shouldn't be trying - while it's not a whole new architecture (unlike, say, Itanium), there are still 32 new and previously unseen bits, Drivers Will Need To Be Recompiled, Recompilation Takes Time And Is Not Always Feasible, and as a result, Not Every Driver Is Available In An X64 Version At The Moment.

The ability of users to exceed the capacity of the 32 bit architecture isn't yet widespread outside of niche environments, but we're not too far off that time: games regularly run best with at least a gig of memory (and let's face it, games are the real driver of the leading edge of consumer adoption, right?), and more is usually better. We're not yet at the tipping point of the old 640K mark - where all the band-aids had been applied, but the patient was still very much at death's door - and that lessens the impetus for the consumer space.

So IMHO, right now, the cost/benefit doesn't typically work for consumers. Yes, I run Windows X64 Edition at home, but I don't have any super-concrete reasons for doing so that require bits 33-64 to be present (at least that I'm aware of at the moment). There are a few games taking advantage of the AMD64 extensions, but that's about it for now, for me.

Servers, on the other hand, can already eat all the memory we throw at them, and don't typically have the handicap of having to run, say, my Dad's aging scanner, so for high-performance large-memory workloads, moving to 64-bit is a no-brainer, especially if you can do so on hardware you've already purchased.

On x64, being able to run a 32-bit Large Address Aware process with a full 4GB of user address space is often a compelling enough benefit on its own, but using a 64-bit native image lets you use... um, well, it's a really, really big number, something like eight thousand gigabytes, if you can jam that much in your machine.

And multiuser scaling on Terminal Server - yes, for 32 bit applications - is from all indications Where It's At on X64 at the moment. In the Terminal Server space, we've been constrained for a number of years by the 32-bit architecture (and whether to enable PAE, and whether we end up constrained by System PTEs or Paged Pool or NonPaged Pool, and so on), and the increased kernel headroom buys increased scalability. Nice.

So there's my $0.02 x 2.

* - Intel's processor chart seems to show gaps in the EM64T capable lineup, but it's hard to read on a single screen, and I've given up. Anyway, the point is, likelihood of a given chip being 64-bit capable increases as time goes on.

** (don't bother looking for a **, there wasn't one in the body) And I plan on going X2 as soon as the prices drop to something a little more palatable. Like, 300-400 bucks palatable, for something that "doubles" my 3500+.

No Scanner and Camera Wizard? (And a little on Remote Assistance under x64)

tristank — Sat, 30 Jul 2005 05:41:00 GMT

I despise digital cameras. My girlfriend constantly asks me to charge mine, although I have no intention of ever using it. It breaks down like this:

Fun photo-taking duties: her.
Charging and general maintenance - including downloading, storing, publishing and otherwise doing things with photos: me.

I pray daily for the convergence of all portable digital devices, so that I can lose the vines of charger cables for phones, cameras, MP3 players and so on that frond their way across my living room.

Back To The Point, Son

My parents called up with a minor emergency: when they plugged their new or old digital cameras in, "nothing happened". Closer investigation revealed that "nothing" was actually "something": the folder view appeared, but not the Scanner and Camera Wizard.

They had attempted all the usual remedies before calling: installing the Windows 98 software that came with the old camera (they're on XPSP2, natch); clicking things in various control panels; hanging out the washing; getting upset with each other; cursing the "bloody thing".

I assumed that they'd simply picked a default action in the Scanner and Camera Wizard, and that they weren't going to be prompted ever again, and set about looking for a way to undo this. All the search engines suggested that uninstalling Realplayer would resolve the problem, but I wasn't prepared to install RealPlayer, just to remove it, just to get the wizard back.

Remote Assistance, MSN Messenger 7.0, Windows X64 Edition

Their Remote Assistance invites (er, actually about 50% were RA Invites, 50% were "You have been added to this conversation") failed with a message that indicated Remote Assistance is not installed or available on my computer, or is disabled. Funny, I thought X64 did include RA. Quick web search didn't show any "of course, we had to cut that feature" announcements.

As a last ditch attempt, I killed MSN Messenger and started Windows Messenger. This time, the invitation actually worked, and I got as far as the Accept stage before Messenger went to 100% CPU. Closing everything that might have a vested interest in Messenger components (Windows Messenger, MSN Messenger, Outlook, Maxthon, everything right-click and Exit-ed), I fired up Windows Messenger again, and this time got the RA session working.

The Problem(s)

Now I could see their screen, so I got them to plug in their Sony camera. Up popped a folder view, with a couple of weirdly named folders. Browsing through them, I couldn't find any JPGs or RAWs or similar.

I asked if it would be a fair assessment that they had no photos actually taken on the device? No. Can you take one? OK.

One photo taken later, the camera is plugged back in, and whop - there's the Scanner and Camera Wizard, eager to dispose of the photo.

The other camera (an old Olympus D520 Zoom) turned out not to be able to take photos any more under pretty much any circumstances, so might be of limited use as a camera.

ADD-friendly Summary

If you came here just looking for a quick fix of information, here are the key learnings in easy-to-consume form:

The Scanner and Camera Wizard might not appear if you haven't got any photos on the device you're plugging in.
Receiving Remote Assistance Invites under Windows XP Pro X64 (at least on my system) needs Windows Messenger to receive the invitation - MSN Messenger 7.0 doesn't cut it. Remember to close any open programs that might consume Messenger features if trying to switch between them.
I'm a rotten relative to ask for tech support.

I Make Things Happen*.

tristank — Fri, 24 Jun 2005 02:59:00 GMT

I am a genius (and humble to boot).

Quickie: The latest twist on the RSS Feeds Du Jour is that EvenBalance just released a new version of PunkBuster for Joint Operations (caught during the morning catchup via Newsgator). Haven't had a chance to try it yet, but hopefully this one will support X64, and I can again play online. Yay!

This brings my "bizarre predictive RSS feed" score to 100% within ~24 hours of publication. Wait, no it's 66%, still no sign of... you know, that one.

Now, after playing Battlefield 2 to warm up, I can get back into JOPS, with any luck...

New Nvidia Drivers for x64 (W00t)

tristank — Thu, 23 Jun 2005 02:29:00 GMT

Nvidia released new drivers for x64 (and x86) today.
The dodgy feeds worked!

Smug? Yeah, a little.

B-Day (no, not bidet) for Battlefield 2

tristank — Wed, 22 Jun 2005 06:57:00 GMT

Well, Battlefield 2 is in my hot little hands right now. I had to get the CD version (3 CDs) because I don't believe in preordering (it violates rule #1 of impulse buying, which is that if the goods aren't miraculously present on the store shelf, I don't pick them up and purchase them).

Now I just need those Nvidia X64 drivers to show up... c'mon! Pleeease!

[Update, 2:20am] Well... it's pretty good, but I find myself wanting to play Joint Operations after I get annoyed at the infantry game. It's all about the vehicles in BF2. Opinions may change with experience, but I'll give it a 7/10 at the moment.

Aussie MSN Maps: I'm Ashamed I Never Knew

tristank — Wed, 22 Jun 2005 06:16:00 GMT

I've never really been interested in maps, as such. They're in the "nice to have when driving" category for me, 98% of the time.

On the subject, I've been a user of Whereis for many years, not knowing that there was an alternative for Australia. I knew about MapPoint as an API, but knowing of an API and having a usable interface is something quite different.

With the recent mapping hoo-hah, I'd just resigned myself to thinking that Australia will lag the other regions (the tyranny of being a small economy with a huge geography).

Well, thanks to Raymond, I know there's an alternative, and it's fast, pretty good, and the maps are resizable (the smaller streets become visible when the maps are zoomed). It's MSN Maps. Give it a try!

Can't wait for MSN's Local Search beta to cover regions outside the US too...

RSS Feeds Ain't Yet Prevalent Enough (so I made my own)

tristank — Tue, 21 Jun 2005 15:49:00 GMT

[Note: the new master article for the feeds is here. Check there first!]

And there's nothing like the tiny dose of repetitive strain injury caused by having to click through a few sites daily - just looking for updates - to spur you into action.

Built from the tattered skeleton of my last foray into the dark and wretched existence of late night Internet scripting (see: experiments with Xmlhttp and Winhttp), I dissected a page or two, and present for my personal amusement my own little driver feeds:

The Nvidia X64 Drivers RSS Feed (one item for Graphics, one for Nforce4 AMD Platform Drivers) - just in time for an imagined-and-really-really-hoped-for Battlefield 2 driver drop.

Because it was easy to fiddle the existing script: the x86 32-bit version (all the interesting stuff I could spot - the filename convention changes at one point, so the NF4AMD one might not work reliably).

And the only faintly ironic Nvidia DVD Decoder Store Feed .

Neither of which has been updated in a while, though the deranged statistician in me wants to say that that makes each passing minute more likely to contain an update.

At the end of this week, I'm going on holiday to Canadia, to play with some crazy Canadans, so don't expect 24/7-nine-nines-gleaming-teeth-and-smiles-from-execs-in-grey-pressed-suits uptime from the feeds, but feel free to use them as a convenience, by all means. The script runs every 4 hours, and the RSS TTL is 6 hours, so, well, generally, it should be OK.

It is entirely likely that the dodgy scripts behind them (hint: it uses a lot of InStr()) will produce utterly meaningless content once the URLs they gather information from change, so in the case of these particular feeds, please consider a break a sign that something new and different may has been published, and go and look for it.

[Update, June 23, 2am] Added PunkBuster clients to the list; Battlefield 2 makes me yearn for some good solid Joint Operations action, for which the PB client update for X64 is due "soon".

[Update, June 24, 2am] Moved most of this into a new master article (it takes a while for me to work out whether something's a good idea). Check there for any updates In The Future (er, I know, there are these things called "RSS Feeds" that provide updates without you manually checking, but, oh, it's late).