Do you click on 77 spam emails every day or have you spent $47,000 on a lost laptop? Me neither, but Google thinks you do and these are exactly the sort of numbers used to calculate your potential cost savings for "Gone Google".
In today's macroeconomic climate, businesses need thorough and defensible cost frameworks to justify a new investment. In this first (of a two) part blog post, I will compare the 'Gone Google' calculator (this post) to Microsoft's methodology with Forrester Research for evaluating new solutions.
'Gone Google' Calculator
Google Apps latest marketing gimmick is an online savings calculator that customizes potential cost savings for companies who 'Go Google'. First, I'll give credit to the web team at Google who have built a nice digital experience that is more visually compelling than their original calculator.
Despite the new facelift, this calculator is oversimplified to a fault and brings into question whether Google really understands businesses. I argue it asserts Google's belief that business customers are gullible, which in our experience they are not.
Google might retort the calculator is simply a bit of 'puffery' that is intended for marketing effect. Regardless, the tool is grossly misleading, arguably irresponsible and businesses of all sizes should exercise caution when relying on the tool for any evaluation. Here's why:
Data 'Mash Up'
The tool starts with two primary flaws at a macro level. First, the assumptions used to generate results are a witch's brew of research that can't responsibly be melted into one defensible methodology. This is NOT a statement about the individual merits of any of them. It's simply irresponsible to take 5 disparate sets of quantitative research and mash them up with qualitative and anecdotal studies conducted by two customers into one decision engine. I only found one of the research reports readily available for review.
Every good marketer knows you have to rationalize research bias, statistical relevance, extrapolation, etc. Google appears to have ignored these principles and disappointingly the tool doesn't attempt to right size these gaps by also factoring in questions about a customer's existing topology, performance, complexity of user roles based on size, much less any inputs for geography, etc.. You simply can't do such numerical 'cold fusion' of so many variables to build a 'customized' report.
Second, the tool has glaring omissions of direct costs related to actually 'Going Google'. Many such costs are promoted by Google on their own marketplace. For example:
Case Study - Acme, Inc.
Below, I've gone through step by step how this tool works and its hokey calculations that are all just sleight of hand. At the end, I've included a video demonstration but I urge you to read each point below as this is where I spend the time to evaluate the claims offered in each in sequence.
For my demo, I'll use Acme, Inc. with 7,000 employees well in the neighborhood of Google's claimed references: City of LA, Genentech, Jaguar Land Rover, et al. For your orientation, the tool uses two buckets of claims: hard cost savings (related to licenses, software, etc) and soft costs of 'hours saved' which I've included in each step to show how the tool progresses it's calculations.
The Magic of 3,001 & 6,516 & 56 hours
After 'Step one' of providing your company name and employee count, you are presented with the neat graphic of a printer in 'Step 2' (looks like a HP Deskjet 710c I once had) with your first hard cost savings claim. You will either get a dollar ($) savings, or in my case a percentage (%) savings claim? Why the difference?
Customer inputs between 1-3,000 get dollar estimates while customers larger than 3,001 get percentages. As you dig deeper, you notice why Google switches their claims because the tool shows you that larger customers save less money. In fact, I plotted increments such as 1, 10, 100, 1,000, 2,000 etc…and noticed below the extremely steep decay curve (more like ski jump) that exists on the estimated hard costs savings. Interestingly I found that at 6,516 employees, according to Google, you save ZERO on hard license savings.
Soft Cost savings will accrue through the remaining steps (3-12) but no matter how many employees you have, the tool will always say employees will save 56hours of work. That's a bold claim with no substantiation and seems to conflict with these customers experience.
Errant Mobile Claims
Step three makes a claim that mobile workers will be 2.8 times as productive. Never mind that this assumes an always on connection. Will anyone fly? Are any of my employees in countries with slower bandwidth? Also since Google nixed Offline for Docs, can they really justify this number? Mary Jo Foley might weigh in here. Also if Google would follow the rest of the guidance in the Forrester report they like to cite, they'd factor in the segmentation which argues for a majority of users needing a rich client app like…Outlook. Given all of their issues with Outlook sync, that would more accurately lead to a productivity number of….well in the words of Jake Harris, an IT Admin using Google Apps 'a headache'.
Step 4 just makes a claim of how large your inboxes will become with Google Apps. It takes your employee count and multiplies by 24.5 GB to spit out a figure. Do you want all employees saving every single email? Do you need archiving? Where is the added cost of Postini archiving? Regardless, it does not factor into these calculations. My savings are still showing zero.
The worst SPAM filter…ever!
Step 5 states workers will avoid the 28hrs they currently spend dealing with all that SPAM. Google provides nifty per second, per click calculations based on users getting 77 spam emails a day they must triage. Do these people have no spam filter in place now? Where do they work, Swiss Cheese Inc? Even free services like Hotmail, Yahoo, AOL have SPAM filters. Outlook even has a built in junk email filter. I've never clicked on 77 SPAM messages in a day, ever. Regardless, the tool assumes it's all upside and introduces the first soft costs in a new graphic by multiplying 28hr * 7,000.
Revisions and team work.
Step 6 communicates a fact about document revisions. Problem is it doesn't even scale with the inputs. Put in a 1 person company, it assumes you work in a 10 member team, put in 10,000, same calculations. This is why using a linear scale is faulty. It also avoids the #1 issue of collaboration that Google is yet to fix, Fidelity. When you work with anyone who isn't on Google, whether they use OpenOffice.org, Microsoft Office, et al, Google Docs can destroy your content. Avoiding this fact is sneaky. The tool should offer assumptions such as average number of existing documents per employee to import when 'Going Google' and how many documents have macros, graphs or styles that will outright break. Take that, apply a salary to fix and subtract from time saved and money lost. The tool does alter it's estimates.
36% more efficient – trust us
Step 7 uses an internal, qualitative study done by one customer in the newspaper industry. It's not provided and I couldn't find it on the internet. The customer found 36% greater communication with Google Apps. What does that even mean? 36% more IM's? 36% more information created? Employees left work 36% earlier? Still no changes to my calculations so why is Google even offering these steps?
Those terrible operating systems
Step 8 is truly out of left field. It uses the fact that operating systems may require 1 hr updates and patches. Fine, but how will all of my employees access Google? Here's a hint, with computer running any OS that must be updated. So what does this have to do with 'Going Google'? For fun here's an old article about Apple, Windows and Linux on updates. It also doesn't factor in Google's own 1hr of planned downtime per MONTH, which is the same impact? Regardless, the tool added 7,000 hours of savings. (1hr * 7,000) New Totals appear!
Ignore the Google Apps 450+minutes of downtime since mid-April
Step 9 is a great one. It uses a study by Radicati, not available without subscription, to show Exchange, Notes and Groupwise downtime per month. It defaults to Exchange with 150mins /month. I won't even bite on this number. It calculates every employee will gain an additional 27hrs of productivity by 'Going Google'.
This again ignores Google's own 12hours of planned downtime per year and Google's free 10min downtime rule. Out of curiosity, I looked at the Google Apps dashboard that shows Google Apps services have accrued over 450 minutes of outages in the last 60days! Lost money? Lost Hours? The tool updates by taking 27hr * 7,000 for my new totals.
What if 10% of all employees lost their laptop? Tack it on.
By Step 10 you are getting how this shell game is working, but you click on. It takes the % of employees that use a laptop (set at 60%) and assumes 10% of them will lose their laptop with 10GB's ea. For my example, that equals 420 employees losing 4,200GB. Well, according to Google that means $47,246 per employee in costs. This is getting fun now as I am awarded potential savings jackpot of $19,843,320. (420 * $47,246)
The reasoning here is so faulty but the tool takes this money and puts it into the hard cost saving bucket, so I now have new totals.
Now we're talking…pile it on $$
Step 11 takes an estimate of $150/user cost for data recovery services and multiplies it by my 7,000 employees. Is this money I already pay or can entirely avoid? Do people back anything up today? After my previous $19m jackpot, I am disappointed it's only $1,050,000 but I'll take it. Did you know Google Apps won't tell their customers where their data is located? What about the costs associated with risk and exposure of your data being held in regions where you don't even conduct business? Or most cloud providers have a force majeure clause which means you should back up data locally as well. Regardless, the hard costs increase to new totals.
Close it out with free posters
Steps 12-13 finish up with some marketing and a nice touch of taking the results with you in the form of Excel (for real calculations) or a poster with which you can tell your CEO you just found a way to save her over $20Million dollars! Good luck trying to defend how you came up with that number.
Instead, I recommend you spend your time showing your CEO how Office, SharePoint and Exchange 2010 can save them money and make people more productive just like other companies such as Sony, Telus, Football Association, General Mills and Subaru.
Oh wait, one last number. . . 11 minutes. That's the time you'll save not wasting your time filling out the so-called Google "calculator".
Great analysis, Andrew! I'm glad to see Microsoft rebuffing these ridiculous claims. I wish this sort of information had been out last year when we were considering the switch. BPOS is better in every way over Google Apps, and, combined with the fact that we could ditch WebEx, we're saving real dollars (not just made-up ones!) every month thanks to BPOS.
Could this be any more one sided? Sure the calculator is obviously stupid but you can't just point out the negatives in the scenario. Your biggest concern, the fidelity of using docs with outside customers. What about the potential increase in productivity for the internal company users of docs. Or a mid size organization where you want to complain about the costs of adding software to desktops. How much cost is associated with the backup, management, upgrade, and maintenance costs of running exchange, sql server, sharepoint, ocs. Those specialists don't have small salaries. What about the increased data center size and cooling required for hosting your own email? Google absolutely should fix this calculator, no question. But my goodness, when attacking something for being inaccurate and flawed you can't just ignore the potential data that doesn't fit your agenda.
@All negative? - Thanks for the comment. First, this is about their calculator not our own. My next post will be discussing how we include such costs holistically. Second, your point about adding software to desktops is something Google does miss. There are several scenarios where you must deploy software to clients for Google scenarios to work fully. Like a new browser, or add ins for offline or outlook connectors if you still use what's likely on the machine, or GTalk rich client or video add in's etc. That is missing in their calculator. So we agree there. Finally seems like we agree their calculator is bogus and needs great improvement but this is my point. If you see, I see it, why dont they and why would they promote this thing? Is that something you think speaks of business ready, trustworthy type of stuff?
Andrew - do you have an article on the Windows Azure TCO calculator? www.microsoft.com/.../tco
...or the frankly laughable Hyper-V vs VMware calculator? Seems Google don't have a monopoly on using dumb calculators with little basis in reality as marketing tools.