Tablet PC Team Blog : Handwriting Recognition

Handwriting personalization, sentence samples, and data migration

Tablet PC Team — Tue, 27 Jun 2006 23:55:00 GMT

For personalization of handwriting recognition, it's true that 50 sentences can seem like a lot to ask - but it turns out that that set represents just the right amount of training data to ensure that users across the board will be able to get a noticeable benefit for their investment. The threshold was set based on test datasets from many users.

Meanwhile a lot of research has gone into how to get that number down, to lower the 'entry level' for reaping the benefits of personalization. As a result of this, the handwriting recognition personalization system can actually give an accuracy improvement after submitting just over 10 sentences via the Handwriting recognition personalization tool. This is the secret feature Ian referred to -- it kicks in for users of the Input Panel with several hundred successfully recognized passages under their belts and Automatic learning enabled. (Btw if any of you experience this feature in action, I'd be happy to hear your feedback!)

However to guarantee that personalization is giving you the most benefit, you should submit at least the initial 50 sentence samples. Remember that the more you submit, the better handwriting recognition works.

One bright side of submitting a large set of samples (besides the world's best handwriting reco experience!) is that you never have to resubmit the samples, even when you upgrade your build. The samples migrate using the Windows Easy Transfer wizard (found in Accessories->SystemTools), along with the rest of your personal files and settings. When I'm waiting for a meeting to start (or in one that doesn't require a lot of participation... ahem), I ink a few sentences or troubleshoot some words using the personalization wizard. Then before upgrading, I use Easy Transfer to save my personal data to a USB stick. Over time I've accumulated significant personalization data that I can easily load onto any post-Beta2 Tablet, and the effect continues to amaze me.

Why "post-Beta2"? Because migration of this data unfortunately doesn't work well from the Beta2 build to a post-Beta2 build. However I've used it many times post-Beta2 and it's been working admirably.

Fyi Easy Transfer is also useful for avoiding repetitive setup tasks for each upgrade, such as copying desktop files, or tweaking various settings such as folder options or optin to Automatic learning.

Bradley Music - Mobile PC Team

Why can't my Tablet write small?

Tablet PC Team — Wed, 24 Aug 2005 23:26:00 GMT

When comparing pen & paper, with the Tablet PC, you will quickly realize they are not the same.

There's issues such as tactile feel, the distance between your stylus and the ink dot (the cursor) on inking surfaces, and other such differences observable at first site. But the difference that sticks out to me personally is that for some reason I end up writing bigger on a Tablet PC as compared to pen & paper. There are many factors feeding into this, of these being the fact that writing on plastic or glass just does not have the same tactile feel as writing on paper, but this blog is a tad of explanation on other factors that feed into this, notably NOISE.

The Tablet PC has the following characteristcs:

It is an electronic device which has many sub components such as the LCD backlight, hard drive, as well as some having optical drives
It has an electro magnetic digitizer built into them today, which creates an EM field on the surface of the LCD
The electro magnetic digitizer in general has really really high resolution, far more higher than the LCD display capabilities (DPI)
The electro magnetic digitizer in general has really high sampling rates, far more than a mouse
The software subsystem from the digitizer driver up is structured in such a way as to not be bogged down by the latency that exists in the Windows message pump and application threads blocking

Now all the above is good and dandy, but then why do I end up writing big?

The factors that feed into this phenomenon as far as my understanding goes are the first 2 bullets. Essentially there is circuity noise on any Tablet PC no matter how hard we try to mitigate these thru great engineering of our OEMs. As an end result you have jittery input where even when the pen is held still, the cursor keeps shaking - although there is a range in this problem where it is human perceivable and where not.

As a result, filters are applied to smooth out the input from the digitizer, potentially at the EM digitizer firmware level as well as at the Tablet PC input software level. This happens to remove small features in your handwriting to the point that you can't write really small.

This is just one of the many problems we continue to challenge going forward in order to bring the true intuitiveness of pen & paper to computers. Stay tuned for our future releases where we will continue to improve.

Koji Kato - Platform Lead Program Manager - Tablet PC

Do You Print?

Tablet PC Team — Fri, 19 Aug 2005 01:17:00 GMT

If you’re like me, the first time you picked up a Tablet PC you began writing in your favorite block print handwriting style. Perhaps like me, you were disappointed with the results. Personally, I found myself adjusting my handwriting to fit the computer. Three years after the Tablet PC launch (and a couple more playing with the developing technology), I’ve been broken of my block print. I still use print in note-taking, but I’ve even adopted lower-case letters (which tend to yield better recognition results).

Clearly, this is not the way to bring “natural computing” to every user. That is why I am happy to tell you about just one of several improvements on the way for Windows Vista. Some of these improvements are available in an early (and by no means final) stage in the Beta 1 TIP Download.

The single-character recognizer has been completely revamped. Internally, we call this the “Basic Print Recognizer”. It not only helps recognize printed words (written in the lined mode of the TIP, for example); but it also helps when correcting (in the TIP “correction comb”), inputting special characters, and trying to recognize words that are not in the dictionary. It also makes it much easier to enter URLs and Web addresses.

You may have been especially frustrated with “confusable” characters such as “0” and “O”, “I” and “l”, and “Z” and “2”. Another aggravation is lower-case versus upper-case. Because the neural networks behind BPR have been re-trained with specific data to better disambiguate these confusable characters, we have significantly improved the experience.

Perhaps it’s been too long for me to revive my block print habit, but I for one have noticed a difference. Around the office here, several people have remarked on how much easier it was for them to recognize print or quickly correct free-form recognition errors. With any luck, you’ll finally be able to input your business acronyms or use your characteristic abbreviations in instant messages!

Kurt Geisel - Director of Development - Tablet PC