James O'Neill's blog

So long, farewell Auf Wiedersehen, good bye

2010-11-16T15:37:00Z

Today is my last day at Microsoft after 10 years, it’s sad to be going, but as someone said once “Regrets. I’ve had a few, but then again too few to mention.” I’ve met some great people while I’ve been here and I look back on my time here with more pleasure than regret.

That’s the end for this blog. My new blogs is at http://jamesOne111.wordpress.com. If you’ve found what I had to say interesting, update your bookmarks.

James.

Thinking about the cloud - part 2, Office 365

2010-11-01T15:03:12Z

In my previous post I was talking in general terms about why BPOS was a sound idea. The recent announcement of Ray Ozzie’s retirement set people quoting his mantra “Three screens and a cloud” – the three screens being Computer, Mobile device, and TV. The unwritten part of “Three screens” is recognising their diversity: people should interact with the best possible client – which means adapting to the specifics of each “screen”; it’s not “any browser and a cloud”: many phone apps do something which PCs do in the browser, they only exist because of the need to give a different experience on a different kind of screen. Instead of seeing a monolithic website (which in reality probably wasn’t monolithic) we see an app which consumes a service (probably the same service which was behind the web site).

But there was more than publishing stuff using services instead of HTML pages; more even than the likes of Groove or Live Meeting which used the cloud to enable new things. From Ozzie’s vision, famously expressed in 2005, came a realization that services already used by business PCs and devices would increasingly be in the cloud, instead of on an organizations own servers. That was the cue to provide Exchange as a service, SharePoint as a service and so on. We’ve tried to make a distinction between “Software as a Service” – which in some people’s minds is “Any browser and a cloud” and “Software PLUS Services” – which covers a plethora of client software: from multi-player games on Xbox to iTunes to Outlook talking to an Exchange server. But when Office Outlook on a PC accesses Exchange-Online , Exchange is software and it is provided as a service –it just isn’t accessed using a browser: I haven’t yet seen a successful way to make the distinction between the two kinds of “Software as a service” just understand it has different meanings depending on who is speaking.

I don’t know if it was planned but it seemed fitting that we should announce the next generation of BPOS on the day after Ray’s announcement. I prefer the new name Office 365. Mary Jo Foley posted something headed “This is not Office in the cloud” – in which she says “this was not some out-of-the-blue change in Microsoft’s business model. Microsoft is still pushing Office first and foremost as a PC-based software package.” Which is spot on: if you need office in a browser, Office Web App is there but it is not a replacement. I wrote in the previous post about the challenges of providing SharePoint, Exchange and so on, it is not Office but the services behind Office which are in the cloud. The key points of Office 365 are these:

At it’s core are the latest versions of the Server Software (Lync replaces Office Communications Server and provides Live Meeting functionality, and both Exchange and SharePoint are updated). The FAQ page has a link to explain what happens to existing BPOS customers (and there are plenty of them – sending 167 million e-mails a day).
The ability to create a Public website (previously part of Office Live Small Business) has moved into Office 365 (Again the FAQ page explains what will happen to Office Live Small Business)
The update to SharePoint 2010 enables us to offer Office Web Apps – so documents can be viewed in high fidelity and edited from the browser.
Despite the the presence of Office Web Apps the main client will be Office on Desktop computers : Office Professional Plus for the desktop is now available as a part of the package on the same monthly subscription basis
There is a-la-carte pricing for individual parts of the suite and bundles known as plans targeted at different market segments.

I think the a-la-carte pricing option is a good thing – though some are bound to say “Microsoft are offering too many options”. The plans are just the combinations of cloud services we think will be popular; services can be added to a plan or bought standalone – for example “Kiosk” workers can get on the company e-mail system with Outlook web access from $2. We’ve announced that the plans will cost between $4 to $27 per month, that one of the enterprise plans closely mirrors the current BPOS at the same $10/user/month, and that there will be $6 plan with the features we think small business will need. In the run up to the launch I did see some details of different plans and options but I haven’t seen all of these in the announcements and it is not impossible that they will be fine tuned before the system goes fully live. When will that be? The launch has a beta programme (sign-up is at http://office365.microsoft.com) , Mary-Jo said back in July that the plan was for full launch was early 2011 which sounds about right – it’s also necessarily vague, because a beta might reveal a lot of unexpected work to be done: if you want a more precise date I always say in these cases those who know won’t talk, and those who talk don’t know.

We’ve positioned Office 365 as helping small businesses to think big and big business to act fast – the link gives examples which range from the Starwood hotel chain to a single independent restaurant – it’s worth taking time to work out what it might mean to the organization(s) you work in/with: the cloud might be right for you, it might not - but if it isn’t I’d want to be able to explain why not and not have people think an opportunity was being missed through inertia.

This post now appears on my new blog (http://jamesone111.wordpress.com ) - James O’Neill’s blog Thinking about the cloud Part 2 office 365

Thinking about the cloud (part 1).

2010-10-19T16:49:19Z

I was telling someone recently that before I joined Microsoft I spent the late 1990s running a small training company. The number of employees varied, averaging out at a dozen or so. I delivered training, did the business management, helped the win over customers and I looked after the IT. It was like doing two or three jobs.

I’ve been quite reticent about our “Business Productivity Online Service"partly because it takes a long and closely argued post to cover why, from an IT professional’s point of view, getting rid of your servers isn’t abdicating. (This is not going to be that post). But as chance would have it I was looking at BPOS again with my old job in my thoughts. B-POS sounds like it should be something… ”points of sale”, but it is Exchange,Communications server and Sharepoint provided as Pay-monthly “Cloud services”

In the training company we ran all our own IT services, but there’s no way I’d host my own web-server today: the sense of using a hosting company was clear before I left for Microsoft. The launch of BPOS gave businesses a way to get hosted Mail (Exchange), Presence & IM (OCS) and Collaboration & Document management (Sharepoint) for $10 US per month – or in round numbers £80 annually - per user. Comparing that with the cost of server hardware and software and especially the time that in-house systems took up, if I were running that business today, my head would say get rid of the servers. You can mix in-house and in-cloud servers; users keep the same desktop software which is crucial: you don’t give up Outlook to move your mailboxes to the cloud.

It needs a change of attitude to give up the server. If my head argued costs and figures, my heart might have come back with benefits like “You are master of your own destiny with the servers in-house”. But are you ? Back then we couldn’t justify clustering our servers, so if hardware failed – work would stop until it was repaired. Paying for a service in a Microsoft datacentre means it runs on clustered hardware, which someone else maintains. Microsoft’s datacentre is a bigger target for attack, but the sheer scale of the operation allows investment in tiers of defence. Small businesses tend not to worry about these things until something goes wrong, and you can always tell yourself that the risk is OK if you’re getting a better service in-house. But the truth is you’re probably not getting better service. As a Microsoft employee I’m used to having access to my mail and calendar from anything that connect to the internet – laptop at home, or on the move, any PC with web access, or Sync’d to a phone. I doubt if I would have set that up for the training company but it’s part of BPOS – even to the extent of supporting iPhones and Blackberries. Getting rid of servers could not only save money but give users a better set of tools to use in their jobs – an easier thing to accept now that I don’t run servers for a business.

Now if you’ve come across the idea of the HypeCycle (see Wikipedia if not) – I agree with Gartner that cloud technologies somewhere near “peak of inflated expectations” - in other words people are talking up “the cloud” beyond it’s true capabilities, and if things follow a normal course there will be a “trough of disillusionment” before things find their true level. I don’t buy into the idea that in the future scarcely any business will bother with keeping their own server, any more than they would generate their own electricity. Nor do I buy into the polar opposite - that very few organisations, and none with any sense, will keep critical services in the cloud – that idea seems just as implausible to me. So the truth must lie in between: the method of delivering services to users won’t change from one foregone conclusion (the in-house server) to another foregone conclusion (the service in the cloud), like so many things it will be a question of businesses asking “does it make sense to do this in-house”, and I think IT professionals will want to avoid depending on that question being answered one way.

This post appears on my new blog (http://jamesone111.wordpress.com )- James O'Neill's blog Thinking about the cloud (part 1).

An unexpected call from a help desk? Hang up.

2010-10-18T13:04:51Z

My phone rang: it was my dad. Father/son combinations don’t ring to chat like mother/daughter ones do, and Dad had been having computer problems. Specifically, Excel had been crashing but managing to recover his work. Each time it had offered to send data to Microsoft and each time he had declined. Then his phone had rung and the caller said it was about the problems he was having problems with his computer.

To me this was immediately suspicious, there is nothing in the Microsoft reporting process which sends personal information like phone-numbers. In fact when you register Windows you don’t put a phone number in, and it is not stored anywhere in the configuration of the machine. Dad doesn’t have a support contract with anyone so even if personal information were being sent I wouldn’t expect a phone call. It would need quite some call centre to manage a courtesy call every time an app crashed. The only way the caller could know that there was a problem and have his details was if something malign on the machine was telling them.
Dad assumed the caller was legitimate: he assumed they’d been given his details by Microsoft, we only give your personal information to a 3rd party if you have requested a specific service which needs us to do that, or said you were happy to be called about something specific by a partner (which is Opt-in, not opt-out).
They had his confidence and things now went from bad to worse, the caller got Dad to give him remote access to the PC for 50 minutes. There’s no telling what went on in that time, but at this stage I had to assume his machine could be doing anything and everything on it machine was potentially compromised. Changing passwords would do no good if a key-logger had been installed.
After 50 minutes they called back and told Dad they’d removed 300 viruses from his machine (A bit of a dent for the Anti-virus software he was using, and almost certainly untrue) and signed him up for a £180 support contract which he paid by credit card. When he went to use the card… as if you couldn’t guess, it bounced.

I told him to turn everything off and quarantine the PC. Having realised he’d been taken in, he took steps to get his credit card re-issued, and he set about changing all the passwords which might have been exposed on this machine -using a different one. He’d heard about someone whose stock portfolio had been compromised: the crook had changed address and bank details and it was only when they tried to sell everything, that the broker’s system spotted something might be wrong. I had to visit, and

Remove the hard disk and connect it to another machine.
Copy all the data off, for safety,
Scan for Malware (nothing found)
Roll windows back to a checkpoint before all this occurred.
Re-apply updates from Windows updates since that check point
Replace the anti-virus he was using with Microsoft Security essentials and let it scan the machine (nothing found).
Re-run the Malicious Software Removal tool (MRT.EXE) – again nothing was found.

My father was a smart man – smarter than me if I’m honest – and although he has been retired for nearly 20 years I don’t think he has lost his wits. I spent the afternoon of that phone call moving between rage and incomprehension – how could he be so stupid. (Many readers will know the famous “Word Perfect support call ” story – put that into Bing or Google if you don’t: it ends with "unplug your system and pack it up just like it was when you got it. Then take it back to the store you bought it from." “What do I tell them?" "Tell them you're too stupid to own a computer." ) . The problem is – of course – that confidence tricksters are plausible, and anyone can fall for social engineering if it is well enough done.

It turns out that this is a scam being used by a couple of firms in India – they don’t get malware onto the machine and then call to fix it; they randomly call people and tell them they have a problem. It was a company using the name of OnlinePcCare.com who scammed my Dad . One interesting thing was they put the credit card transaction through a third party G2S.com, and it may be that which triggered a fraud alert, the credit card company wouldn’t say. A search on Bing for “onlinePcCare +scam” finds plenty of other victims or near misses. This one from Ireland was immediately familiar “can you start event viewer… are there any errors in the application log ?”. If the event log is empty, logging itself is broken. Some people recorded the scammers and have posted the call to you-tube – these come near the top in a search and Digital toast has a selection – together with a list of other names used by these people.

Charles Arthur at The Guardian has been covering this story for a little while, see Police crack down on computer support phone scam, and Virus phone scam being run from call centres in India. His blog post Those 'PC virus' phone call scams: the unanswered questions is a worth reading too; it confirms my finding that no malware seems to get installed, and shares my opinion that this fits the definition of obtaining money by deception. (Dad’s call to the police got a response of “here’s a crime number – we’ve got a big file on this and we’ll add you to it”)

If you act as family tech support, do yourselves a favour. As well as pointing out that none of those nice men in Nigeria will really have a fortune which they will share, and that it is statistically nearly impossible to have a large lottery win and at the same time be unaware of entering the draw, now you need to add “no one really knows if you have a problem with your computer and calls to fix it” (you may be yelling at the computer but in cyberspace no-one can hear you scream.)

You might add “If you don’t like unsolicited calls register with the telephone preference service, no reputable company will call you once you’ve registered and any company which does call you is, by definition, not reputable

Postscript. While I was working on this post I got a call from OnlinePcCare. It may be one of those random things or they may be stepping up their activities.

This post appears on my new blog ( http://jamesone111.wordpress.com/ ) - James O'Neill's blog An unexpected call from a help desk? Hang up.

Working with the image module for PowerShell; part 3, GPS and other data

2010-07-07T06:59:00Z

In Part one I showed how my downloadable PowerShell module can tag photos using related data – like GPS position – which was logged as they were being taken, and in part two I showed how I’d extended the module in James Brundage’s PowerPack for Windows 7. Now I want to explain the extensions which automate the processes of:

Getting the data logged by GPS units and similar devices
Reading each image file from the memory card and matching it to an entry in the log made at around the same time
Building up the set of EXIF filters filters based on the log entry.

The data and pictures are connected by the time stamp on each, but to connect properly the scripts must cope with any time difference between the camera’s clock and time on the logging device – whether that’s a GPS unit or my wrist mounted scuba computer. A few seconds won’t introduce much error, but the devices might be in different time zones –for example GPS works on Universal time (GMT) - so the offset is often hours, not seconds. My quick and dirty way of making a note of the difference is to photograph whatever is doing the logging (assuming it can display its time). The camera will record the time its own clock was set to in the EXIF “Date and time taken” field and subtracting that from the time displayed on the logger in the picture gives an offset to apply to all data points. The following is the core of a function named Set-Offset which could be seen in part one;

   $RefDate = ([datetime]( Read-Host ("Please enter the Date & time " +      
                                   "in the reference picture, formatted as" + [char]13 +       
                        [Char]10 + "Either MM/DD/yyyy HH:MM:SS ±Z or " +       
                                   "dd MMMM yyyy HH:mm:ss ±Z"))   
            ).touniversalTime()
    $ReferenceImagePath = Read-Host "Please enter the path to the picture"      
if ($ReferenceImagePath -and (test-path $ReferenceImagePath) -and $RefDate) {       
     $picTime = (get-exif -image $ReferenceImagePath).dateTaken  
     $Global:offset = ($picTime - $refdate).totalSeconds       
}

The real Set-Offset can take –refDate and –ReferenceImagePath parameters so the user doesn’t need to be prompted for them. Most of code you can see is concerned with getting the user to enter the time (in a format that PowerShell can use) and the path to the file. The only part which uses the image module is
(get-exif -image $ReferenceImagePath).dateTakenGet-Exif is a command I added, and it returns an object which contains all the interesting EXIF data from the image file. Only the value in the DateTaken property is of interest here; it is used to calculate the number of seconds between the camera time and logger time and the result is stored in a global variable named $offset.

The next step is to read the data and applying the offset to it; depending on the how it was logged the next step will be either:
$Points = Get-NMEAData -Path $Logpath -offset $offset Or $Points = Get-GPXData -Path $Logpath -offset $offset Or $Points = Get-CSVGPSData -Path $Logpath -offset $offset Or $Points = Get-SuuntoData -Path $Logpath -offset $offset

The last one handles the comma separated data exported from the Suunto Dive Manager program which downloads the data from my dive watch. The other 3 deal with different formats of GPS data, it may be in the form of NMEA sentences (comma separated again) or the CSV format used by Efficasoft GPS utilities on my phone or the XML-based GPX format. (GPS data formats are worth another post of their own). You may need to make slight alterations to these functions to work with your own logger, but they are easy to change. All of them except Get-GPXdata import from a CSV file – and use a feature which is new in PowerShell V2 to specify the CSV column headings when using the import-csv command. Get-GPXData uses XML documents looking for a hierarchy which goes <gpx> <trk> <trkseg><trkpt><trkpt><trkpt><trkpt>… All the functions use select-object to remove fields which aren’t needed and insert calculated data (for example converting the native speed in knots from GPS to MPH and KM/H )

After running one of these commands there will be a collection of data points stored in the variable $points. Each data point has a time – adjusted by the offset value, so it is time as the camera would have seen it. The Suunto dive computer points have a Description (the name of the dive site and water temperature) and depth, while the GPS points have Speed (GPS works in knots and the script calculates Miles per Hour and Kilometres per hour); bearing, latitude as Degrees, Minutes, Seconds, North or South, Longitude as Degrees, Minutes, Seconds East or West, Latitude & Longitude in their original form from the logger and Altitude in both meters and Feet (NMEA data needs extra processing to get the attitude data and Get-NMEAdata has a –NoAltitude switch to allow processing to be speeded up if it only Latitude and Longitude are needed )

Armed with a collection of points the next step find the one nearest to the time the picture was taken; a function named Get-NearestPoint does this. Given the time stamped on the photo the function returns the data point logged closest to that time. It isn’t very sophisticated, taking 3 parameters: a time, a time-sorted array of points and the name of field on the data points to check for the time, and working through the points until the point being looked at is further away from the target time than the previous point; the core of the function looks like this:

      $variance = [math]::Abs(($dataPoints[0].$columnName - $MatchingTime).totalseconds)      
   $i = 1       
   do {       
        $v = [math]::Abs(($dataPoints[$i].$columnName - $MatchingTime).totalseconds)       
        if ($v -le $variance) {$i ++ ; $variance = $v }       
      } while (($v -eq $variance) -and ($i -lt $datapoints.count))       
   $datapoints[($i -1)]

In use it looks something like this.

   $image = Get-Image        –Path "MyPicture.Jpg"      
$dt    = Get-ExifItem     -image $image  -ExifID $ExifIDDateTimeTaken       
$point = Get-nearestPoint –Data  $points -Column "DateTime" -MatchingTime $dt

$point contains the data used to set the EXIF properties of the picture, a process which requires a series of Exif filters to be created – and I explained EXIF Filters in Part 2. As well as data retrieved from a log, there are times when I want to tag a picture manually. For example I took some photos in London’s Trafalgar Square without a GPS logger that I want to tag with 51°30’30” N, 0° 7’40” W . To make this easier I created a function named Convert-GPStoEXIFFilter which can be invoked like this:

$filter = Convert-GPStoEXIFFilter 51,30,30 "N" 0,7,40 "W"

If you’re not used to PowerShell I should say that in some places 51,30,30 would be the way to write 3 parameters. In PowerShell it is one array parameter with 3 members. (Even old hands at Powershell occasionally get confused and put in a comma which turns two parameters into a single array parameter) I could have explicitly named the parameters and made it clear that these 3 were an array by writing
-LatDMS @(51,30,30) -NS "N"

Convert-GPStoEXIFFilter returns a chain of up to 7 EXIF Filters for GPS version, Latitude reference (North or South) Longitude reference (East or West), Altitude reference (above or below Sea Level), the Latitude & Longitude (as degrees, Minutes, Second and Decimals) and Altitude in meters (altitude is optional). If $point holds the data logged at the time the picture was taken Convert-GPStoEXIFFilter can be invoked like this:

$filter = Convert-GPStoEXIFFilter -LatDMS $point.Latdms -NS $point.NS ` -LONDMS $point.londms -EW $point.ew -AltM $point.altM

At the end of part 2 I showed the Copy-Image command that handles renaming, rotating, and setting keywords & title EXIF fields and mentioned it could be handed a set of filters. All the parameters that Copy-image uses are available to Copy-GPSImage, which takes the the set of points as well . Internally it performs the $image= , $dt= , $point= and $filter= commands seen above before calling Copy-image with the image, the filter chain and the other parameters it was passed. The full set of parameters for Copy-GPSImage is as follows

Image	The image to work on – this can be an image object, a file object or a file name can come from the Pipeline.
Points	The array of GPS data points from Get-NMEAdata, get-GPXData or Get-CSVGPSData
Keywords	Keywords to go into the EXIF Keyword Tags field
Title	Text to go into the EXIF Title field
Rotate	If specified, adds whatever rotate filter is indicated by the EXIF Orientation field
NoClobber	The conventional PowerShell switch to say “Don’t overwrite the file if it already exists”
Destination	The FOLDER to which the file should be saved.
Replace	Two values separated by a comma specifying a replacement in the file NAME
ReturnInfo	If specified returns the point(s) matched with the pictures

So now it is possible to use three commands to geotag the images, the first two get time offset , and get the data points, applying that offset in the process.

set-offset "D:\dcim \100Pentx\IMG43272.JPG" –Verbose $points= Get-CSVGPSData 'F:\My Documents\My GPS\Track Log\20100425115503.log' ‑offset $offset

and the third gets the files on a memory card and push them into Copy-GPSImage

$photoPoints = Dir E:\dcim –include *.jpg –recurse | Copy-GpsImage -Points $Points ` -verbose -DestPath "C:\users\jamesone\pictures\oxford" ` -Keywords "Oxfordshire" -replace "IMG","OX-" -returnInfo

This is much as it appeared in Part 1 although third command has changed slightly.Copy-GPSImage now has a –returnInfo switch which returns the points where a photo was taken; to link the point to the image file(s) which patched it an extra property Paths is added to the points.

I mention this because I wanted to show the functions I put added almost for fun at the end. Out-MapPoint and ConvertTo-GPX got brief mentions in part 1: with the data in $photopoints I can push camera symbols through to a map like this : (note the sort –unique to remove duplicate points, 79 is the camera symbol)
$photopoints | sort dateTime -Unique | Out-MapPoint -symbol {79}

Alternatively I can create a GPX file which can be imported into MapPoint, Google Earth and lots of other tools. GPX files need to be UTF8 text, PowerShell wants to write output files as Unicode – thwarting it isn’t hard but is ugly.
$photopoints | sort dateTime -Unique | convertto-gpx | out-file photoPoints.gpx -Encoding utf8

With the photo points logged it would be nice to show the path I walked but that will have too many points so I wrote Merge-GPSPoint which combines all the points for each minute so I can do
Merge-GPSPoints $points | Out-MapPointor
Merge-GPSPoints $points | convertto-gpx | out-file WalkPoints.gpx -Encoding utf8

One thing I should point out here is that the GPX format which I convert to is a series of Waypoints (i.e places that will be navigated to in future), not track points (places which have been visited in the past). The import routine processes the latter.

The last detail of the module for now is that I also gave it a function to find out where an image was taken, like this

PS > resolve-imageplace 'C:\users\Jamesone\Pictures\Oxford\OX-43624.JPG' Summertown, Oxford, Oxford, Oxfordshire, England, United Kingdom, Europe, Earth

That’s not a data error when it says Oxford, Oxford. The Geoplaces web service I use returns

ToponymName :	name	fcode	Desctiption for fcode
Earth	Earth	AREA	a tract of land without homogeneous character or boundaries
Europe	Europe	CONT	Continent
United Kingdom of Great Britain and Northern Ireland	United Kingdom	PCLI	Independent political entity
England	England	ADM1	First-order administrative division (US States, England, Scotland etc)
County of Oxfordshire	Oxfordshire	ADM2	A sub-division of an ADM1 (Counties in the UK)
Oxford District	Oxford	ADM3	A sub-division of an ADM2 (District level councils in the UK)
Oxford	Oxford	PPL	Populated Place (Cities, Towns, Villages)
Summertown	Summertown	PPLX	Section of populated place

I haven’t done much to introduce intelligence into processing this. I used Trafalagar square in one part 2 and this returns Charing Cross, London, City of Westminster, Greater London, England, United Kingdom, Europe, Earth which is correct but difficult to allow for. To make matters worse all sorts of strange geo-political questions come up as well if you say UK is the country, and England is the topmost Administrative division: English people might well think counties are the tier below parliament adminstratively but since the Scottish parliament and Welsh assembly opened, you might find a different view if you step over the border. Software which works to the American model of displaying the Populated place and First admin Division – for example Seattle, Washington; is easily thrown giving Reading, Berkshire it gives Reading, England. Those are questions to look at another time

Exploring the IMAGE PowerShell Module

2010-07-05T11:57:44Z

In part one of this series I showed the finished version of photo-tagging script I’ve been using. I based my work (which is available for download) on James Brundage’s PSImageTools module for PowerShell which is part of the PowerPack included with the Windows 7 Resource kit (and downloadable independently). In this post I want to show the building blocks that were in the original library provide and the ones I added.
Producing a modified image using this module usually means working to the following pattern:

Read an image
Create a set of filters
Apply the filters to the image
Save the modified image

If you are wondering what a filter is, that will become clear in a moment. James B’s original module had these commands.

`Get-Image`	Loads an image from a file
`Add-CropFilter`	Creates a filter to crop the image to a given size
`Add-OverlayFilter`	Creates a filter to an overlay such as a watermark or copyright notice
`Add-RotateFlipFilter`	Creates a filter to rotate the image in multiples of 90 degrees or to mirror it vertically or horizontally
`Add-ScaleFilter`	Creates a filter to resize the image
`Set-ImageFilter`	Applies a set of filters to one or more images
`Get-ImageProperty`	Gets Items of EXIF data from an image
`ConvertTo-Bitmap`	Loads a file, applies a conversion filter to it, and saves it as a BMP
`ConvertTo-Jpeg`	Loads a file, applies a conversion filter to it, and saves it as a JPG
`Copy-ImageIntoOrganizedFolder`	Organizes pictures into folders based on EXIF data

You can see there are 4 kinds of filter with their own commands in the list and each one makes some modification to the image: cropping, scaling, rotating, or adding an overlay. Inside the two ConvertTo commands, a 5th kind of filter, conversion, is used and I added a function to create filters to do that. I made some changes to the existing functions to give better flexibility with how they can be called, and added some further functions, mostly to work with EXIF data embedded in the image file. The full list of functions I added is as follows:

`Save-Image`	Not strictly required but it is a logical command to have at the end of a pipe line, instead of calling a method of the image object
`New-ImageFilter`	Not strictly required either but it makes the syntax of adding filters more logical
`New-Overlay`	Takes text and font information and creates a bitmap with the text in that font
`Add-ConversionFilter`	Creates a conversion filter for JPG, GIF, TIF, BMP or PNG format (as used in ConvertTo-Jpeg / Bitmap without applying it to an image or saving it)
`Add-ExifFilter`	Adds a filter to set EXIF data
`Copy-Image`	Copies one or more images, renaming, rotating and setting title keyword tags in the process.
`Get-EXIF`	Returns an object representing the EXIF data of the image
`Get-EXIFItem`	Returns a single item of EXIF data using its EXIF ID (the common IDs are defined as constants
`Get-PentaxMakerNoteProperty`	Decodes information from the Maker-Note Exif field, I have only implemented this for Pentax data
`Get-PentaxExif`	Similar to Get-Exif but with Maker-Note fields for Pentax

The image below was resized and labelled using these commands. The first step is to create an image to act as an overlay: I’m going to a copyright notice in Red red text, in 32 point Arial

PS> $Overlay = New-overlay -text "© James O'Neill 2008" -size 32 -TypeFace "Arial" ` -color "red" -filename "$Pwd\overLay.jpg"

I’m using a picture I took in 2008: and I could have used a more complex command to build the text from the date taken field in the EXIF data. Next I’m going to create a chain of filters to:

Resize my image to be 800 pixels high (the aspect ratio is preserved by default),
Add my overlay
Set the EXIF fields for the keyword-tags, title and Copyright information
Save the image as a JPEG with a 70/100 quality rating

Despite the multi-line formatting here, this is a single PowerShell command: $filter = new-Filter | add | add | add...

PS> $filter = new-Imagefilter | Add-ScaleFilter -passThru -height 800 -width 65535 | Add-OverlayFilter -passThru –top 750 –left 0 –image $Overlay | Add-ExifFilter -passThru -ExifID $ExifIDKeywords -typeName "vectorofbyte" -string "Ocean" | Add-ExifFilter -passThru -ExifID $ExifIDTitle -typeName "vectorofbyte" -string "StingRay" | Add-ExifFilter -passThru -ExifID $ExifidCopyright-typeName "String" -value "© James O'Neill 2008" | Add-ConversionFilter -passThru –typeName jpg -quality 70

Given a set of filters, a script can get an image, apply the filters to it and save it. Originally these 3 steps needed 3 commands to be piped together like this
PS> Get-Image C:\Users\Jamesone\Pictures\IMG_3333.JPG | Set-ImageFilter -filter $filter | Save-image -fileName {$_.FullName -replace ".jpg$","-small.jpg"}

I streamlined this first by changing James B’s Set-ImageFilter so that if it is given something other than an image object, it hands it to Get-Image. In other words Get-Image X | Set-Image is reduced to Set-Image X (and I made sure X could be path, including one with wild cards or one or more file objects) . After processing I added a -savepath parameter so that set-image –SavePath P is the same as Set-Image | Save-Image P . P can be a path, or script block which becomes a path, or empty to over-write the image. Get an image, apply the filters to it and save it becomes a singe command.
PS> Set-ImageFilter –Image ".\IMG_3333.JPG" -filter $filter ` –SaveName {$_.FullName -replace ".jpg$","-small.jpg"}

The workflow for my photos typically begins with copying files from a memory card, replacing the start of the filename - like the “IMG_” in the example above - with text like “DIVE” (I try to keep the sequential numbers the camera stamps on the pictures as a basis for a unique ID). Next, I rotate any which were shot in portrait format so they display correctly and finally I add descriptive information to the EXIF data: keyword tags like “Ocean” and titles like “Stingray”. So it made sense to create a copy-image function which would handle all of that in one command. The only part of this which hasn’t already appeared is rotation. The Orientation EXIF field contains 8 to show the image has been rotated 90 degrees, 6 indicates 270 degrees of rotation, and 1 to show the image is correctly rotated, so it is a question of read the data, and depending on what we find add filters to rotate and reset the orientation data.

$orient = Get-ExifItem -image $image -ExifID $ExifIDOrientation if ($orient -eq 8) {Add-RotateFlipFilter -filter $filter -angle 270 Add-exifFilter -filter $filter -ExifID $ExifIDOrientation` -value 1 -typeid $ExifUnsignedInteger }

There is similar code to deal with rotation in the opposite direction, and rotation is just another filter like adding the EXIF data for keywords or title, all job of Copy-Image does is to build a chain of filters to add Title and Keyword tags and rotate the image, determine the full path the new copy should be saved to and invoke Set-ImageFilter. To make it more flexible, gave Copy-Image the ability to add filters to an existing filter chain: in the part one you could see Copy-GPSImage which finds the GPS data to apply to a picture and produces a series of filters from it: these filters are passed on to Copy-Image which does the rest.

The last aspect of Copy-Image to look at is renaming: -Replace has become one of my favourite PowerShell operators. It takes a regular expression and a block of text, and replaces all instances of expression found in a string with the text. Regular expressions can be complex but “IMG” is perfectly valid so if I have a lot of pictures to name as “OX-” for “Oxford” I can call the function with a replace parameter of "IMG","OX-" . Inside Copy-Image, the parameter $replace is used with the -replace operator (using PowerShell’s ability to treat “img”,”ox” as one parameter in two parts). $savePath is worked out as follows:

if ($replace) {$SavePath= join-path -Path $Destination ` -ChildPath ((Split-Path $image.FullName -Leaf) -Replace $replace)} else {$SavePath= join-path -Path $Destination ` -ChildPath (Split-Path $image.FullName -Leaf) }

As mentioned above I went to some trouble to make sure the functions can accept image objects or names of image files or file objects – because at different times, different ones will suit me. So all of the following are valid ways to copy multiple files from my memory card to the current directory ($pwd), renaming, rotating and applying the keyword tag “oxfordshire”

PS[1]> Copy-Image E:\DCIM\100PENTX\img4422*.jpg -Destination $pwd ` -Rotate -keywords "oxfordshire" -replace "IMG","OX-" PS[2]> dir E:\DCIM\100PENTX\img4422*.jpg | Copy-Image -Destination $pwd ` -Rotate -keywords "oxfordshire" -replace "IMG","OX-" PS[3]> get-image E:\DCIM\100PENTX\img4422*.jpg | Copy-Image -Destination $pwd ` -Rotate -keywords "oxfordshire" -replace "IMG","OX-" PS[4]> $i = get-image E:\DCIM\100PENTX\img4422*.jpg; Copy-Image $i -Destination $pwd ` -Rotate -keywords "oxfordshire" -replace "IMG","OX-" PS[5]> dir E:\DCIM\100PENTX\img4422*.jpg | get-image | Copy-Image -Destination $pwd ` -Rotate -keywords "oxfordshire" -replace "IMG","OX-"

Of course if I have the GPS data from taking the logger with me on a walk I can use Copy-GPSImage to geotag the files as they are copied, and in the next part I’ll look at how the GPS data is processed.

GPS, and other kinds of Picture tagging with PowerShell

2010-07-01T15:38:27Z

Well… I have been off for a bit and you can have a read of the previous post for some background on that. During that time I’ve done a lot of walking, taking photos as I go. Having raved about my HTC Touch pro 2 and its GPS I’ve been using it to GeoTag those photos. Naturally (for me) PowerShell figures in here somewhere. I’ve added to the image module in the Windows Powerpack that James Brundage wrote. It now takes data from a log and applies it to pictures. It supports several log formats and incidental things I found I wanted to do with GPS data. And it is available for download

I log my walks using Efficasoft’s GPS utilities (on the right). I have a picture of the logger to help with the first of the three PowerShell commands I need to use

Set-Offset works out the time difference between the time on the logger and time on the camera and stores it in $offset
Get-CSVGPSData reads the GPS log from a CSV file, using $offset to adjust the time so it matches the camera. I store the result in $points and a successful import can be verified by looking at $points.count.
Copy-GpsImage copies pictures from a memory card to my computer renaming them, rotating them if need be, and tagging them using the GPS data I stored in $points (if copying many files, it is useful to use -verbose switch to see progress)

Steps 2 and 3 might be repeated to tag more than one set of photos provided the camera clock is a consistent interval away from “GPS time”. Here’s what a session looks like in PowerShell

PS > set-offset "D:\dcim \100Pentx\IMG43272.JPG" –Verbose     Please enter the Date & time in the reference picture, formatted as       
Either MM/DD/yyyy HH:MM:SS ±Z or dd MMMM yyyy HH:mm:ss ±Z: 04/04/2010 16:02:17 +1        
VERBOSE: Loading file D:\dcim \100Pentx\IMG43272.JPG        
VERBOSE: OffSet = 3607
    PS> $points= Get-CSVGPSData 'F:\My Documents\My GPS\Track Log\20100425115503.log' ‑offset $offset        
PS> $points.count         
593
    PS> Dir E:\dcim –include *.jpg –recurse |        
      Copy-GpsImage -Points $Points `         
          -Keywords "Oxfordshire" `         
          -DestPath "C:\users\jamesone\pictures\oxford" `         
          -replace "IMG","OX-" `         
          -verbose
    VERBOSE: Loading file E:\dcim\100PENTX\IMG43624.JPG        
VERBOSE: Checking 593 points for one where DateTime is closest to 04/25/2010 12:36:41         
VERBOSE: Point 229 matched with variance of 2 seconds         
VERBOSE: Performing operation "Write image" on Target " C:\users\jamesone\pictures\oxford\OX-43624".

In my case this will go through a stack of files, ending
VERBOSE: Performing operation "Write image" on Target " C:\users\jamesone\pictures\oxford\OX-43757"

The picture shows a detail from the Radcliffe Observatory building in Oxford (featured in a recent episode of Lewis) with its GPS co-ordinates visible through File/Properties. I also wrote a little bit of code to push the data from the log though to MapPoint – this ended up as a function Out-MapPoint although I later added a function named convertTo-GPX which does the same job more quickly and also works with programs like GoogleEarth.

$MPApp = New-Object -ComObject "Mappoint.Application" $MPApp.Visible = $true $map = $mpapp.ActiveMap ForEach ($point in $points) { $location=$map.GetLocation($point.Lat, $point.lon) $Pin =$map.AddPushpin( $location, $point.datetime) if ( $point.datetime -lt [datetime]"04/25/2010 12:12:00"){$pin.symbol= 6 } else {$pin.symbol = 7} }The details behind this this take some explaining, so this will be the first of several parts: if you would like to know more have a look at the next few posts where I will drill into how it all works, but if you want to dive in and play the code is available for download now.

How to make and share Panoramas more easily – Microsoft ICE + Photosynth

2010-03-22T11:10:39Z

I mentioned in yesterday’s post that I was working on a post about scanned images – in fact it’s the same set of scanned images I was using in this earlier post. In that post I talked about putting Panoramas into SilverLight Deep Zoom

The teams behind Microsoft ICE* – (the Image Composite and Editing tool) and Photosynth have worked together so the newest version of ICE can upload its results into Photosynth: Photosynth is driven by deep zoom so I don’t have to any work to get a shared, zoomable panorama, with Geotagging thrown in. This is not the only great new addition to ICE** it has support for mechanised capture devices which produce an ordered grid or as ICE terms it a structured panorama. You can read more on the HDview blog and on the Photosynth blog (Matt mentions that the ability to see Geotagged panoramas on the same map as Geotagged synths isn’t there yet, but is coming – and I’ve got another post in the pipeline about Geotagging photos) .

I put together a little video to show just how easy the process is (if the embedded video doesn’t work properly, you can view it in its original location) - at 1280x720 resolution it doesn’t really do Photosyth’s viewer justice. You can click the version below and hit the button on the right for the full screen view to see just why I like it so much. I have a couple more panoramas which I will rebuild and upload.

* Should I proud of myself for not making any contrived “ICEmen” jokes ?
** I could have said cool new thing in ICE .

The 50th birthday that would have been

2010-03-21T13:42:35Z

If you believe in the parallel universes then there are ones where Ayrton Senna is celebrating his 50th birthday today, having won 6, 7 or 8 world championships.

In this one, the 50th anniversary of his birth is marked a more sombrely. It’s not quite 16 years since he died with 3 championships to his name and last week when his nephew flipped open the visor on a very similarly patterned helmet to reveal very similar looking eyes I felt like I had seen a ghost – and heard Martin Brundle articulate the same thought on his TV commentary. I said pretty much all I wanted to about Ayrton on fifteenth anniversary of his death. But since I’ve posted recently about scanning pictures and I have another post in draft about that , I thought I’d share a couple of successful scans.

In 1991 Senna looked like he was going successfully defend the title he won in 1990 – he won the first 3 races before Nigel Mansell had got a finish, and he wasn’t even the leading Williams driver until 7th race. The 8th was the British Grand Prix, and I was there. To cap a perfect day for Mansell and a partisan crowd, Senna ran out of fuel on the last lap. Mansell on his victory lap stopped and gave his adversary a lift back to the pits. In this picture you can see how much more exposed the drivers heads were in those days – which was to be the death of Senna in another Williams, 3 years later. Riding back like that is forbidden now, and that too speaks of the attitudes to safety. But it says something to me about the nature of sport that drivers could fight for everything on the track, yet offer help and not be humiliated by accepting it.

The cheap film, second-rate lenses and my own technique limit how good the scan of the photo can be – but I hope the story explains why I treasure it. Far better from a technical point of view is this second picture – but sometimes the technical quality isn’t what matters.

To mark the anniversary, Autosport have a page Ayrton Senna: A life in pictures – they are arranged with the oldest at the bottom – you can see his first F1 test (in a Williams), another shot of the ride home with Mansell in the middle and on the left of the top row is quite a poor shot – you can’t tell what it is from the thumbnail. It’s the back of the car, and I generally don’t keep those. The caption reads “Ayrton Senna, Williams FW16 Renault; 1994 San Marino Grand Prix at Imola.” And then slowly it dawns that the wall and trees in the background mean the car is going into the corner named Tamburello and there’s a big gap to car behind, so this must be the sixth lap – a few seconds after this shot was taken the right front wheel of FW16 hit the wall a little further down than we can see in the shot, and parted company from the car. On another day or in another universe it would have passed harmlessly by, but it didn’t and that picture – like the lift home one – captures what we now know to be a decisive moment.

Virtualization announcements today.

2010-03-18T16:23:22Z

In another window I am listening to the desktop Virtualization hour which I blogged about yesterday. A couple of hours ahead of the broadcast we posted the press release on Press pass which contained the following detail of what we are announcing today.

• New VDI promotions available for qualified customers to choose from today. Microsoft and Citrix Systems are offering the “Rescue for VMware VDI” promotion, which allows VMware View customers to trade in up to 500 licenses at no additional cost, and the “VDI Kick Start” promotion, which offers new customers a more than 50 percent discount off the estimated retail price. Eligibility and other details on the two promotions can be found at http://www.citrixandmicrosoft.com.

• Improved licensing model for virtual Windows desktop. Beginning July 1, 2010, Windows Client Software Assurance customers will no longer have to buy a separate license to access their Windows operating system in a VDI environment, as virtual desktop access rights now will be a Software Assurance benefit. [Note the new name VDA is what we used to call VECD.]

• New roaming use rights improve flexibility. Beginning July 1, 2010, Windows Client Software Assurance and new Virtual Desktop Access license customers will have the right to access their virtual Windows desktop and their Microsoft Office applications hosted on VDI technology on secondary, non-corporate network devices, such as home PCs and kiosks.

• Windows XP Mode no longer requires hardware virtualization technology. This change simplifies the experience by making virtualization more accessible to many more PCs for small and midsize businesses wanting to migrate to Windows 7 Professional or higher editions, while still running Windows XP-based productivity applications.

• Two new features coming in Windows Server 2008 R2 service pack 1. Microsoft Dynamic Memory will allow customers to adjust memory of a guest virtual machine on demand to maximize server hardware use. Microsoft RemoteFX will enable users of virtual desktops and applications to receive a rich 3-D, multimedia experience while accessing information remotely. [Note the new name RemoteFx is the technology we acquired with the purchase of Calista.]

• New technology agreement with Citrix Systems. The companies will work together to enable the high-definition HDX technology in Citrix XenDesktop to enhance and extend the capabilities of the Microsoft RemoteFX platform.

Good stuff all round, but from a technical viewpoint it’s the new bits in SP1 which will get the attention. I’ll post a little more on what Dynamic memory is and is not in the next day or two.