Do you click on 77 spam emails every day or have you spent $47,000 on a lost laptop? Me neither, but Google thinks you do and these are exactly the sort of numbers used to calculate your potential cost savings for "Gone Google".
In today's macroeconomic climate, businesses need thorough and defensible cost frameworks to justify a new investment. In this first (of a two) part blog post, I will compare the 'Gone Google' calculator (this post) to Microsoft's methodology with Forrester Research for evaluating new solutions.
'Gone Google' Calculator
Google Apps latest marketing gimmick is an online savings calculator that customizes potential cost savings for companies who 'Go Google'. First, I'll give credit to the web team at Google who have built a nice digital experience that is more visually compelling than their original calculator.
Despite the new facelift, this calculator is oversimplified to a fault and brings into question whether Google really understands businesses. I argue it asserts Google's belief that business customers are gullible, which in our experience they are not.
Google might retort the calculator is simply a bit of 'puffery' that is intended for marketing effect. Regardless, the tool is grossly misleading, arguably irresponsible and businesses of all sizes should exercise caution when relying on the tool for any evaluation. Here's why:
Data 'Mash Up'
The tool starts with two primary flaws at a macro level. First, the assumptions used to generate results are a witch's brew of research that can't responsibly be melted into one defensible methodology. This is NOT a statement about the individual merits of any of them. It's simply irresponsible to take 5 disparate sets of quantitative research and mash them up with qualitative and anecdotal studies conducted by two customers into one decision engine. I only found one of the research reports readily available for review.
Every good marketer knows you have to rationalize research bias, statistical relevance, extrapolation, etc. Google appears to have ignored these principles and disappointingly the tool doesn't attempt to right size these gaps by also factoring in questions about a customer's existing topology, performance, complexity of user roles based on size, much less any inputs for geography, etc.. You simply can't do such numerical 'cold fusion' of so many variables to build a 'customized' report.
Second, the tool has glaring omissions of direct costs related to actually 'Going Google'. Many such costs are promoted by Google on their own marketplace. For example:
Case Study - Acme, Inc.
Below, I've gone through step by step how this tool works and its hokey calculations that are all just sleight of hand. At the end, I've included a video demonstration but I urge you to read each point below as this is where I spend the time to evaluate the claims offered in each in sequence.
For my demo, I'll use Acme, Inc. with 7,000 employees well in the neighborhood of Google's claimed references: City of LA, Genentech, Jaguar Land Rover, et al. For your orientation, the tool uses two buckets of claims: hard cost savings (related to licenses, software, etc) and soft costs of 'hours saved' which I've included in each step to show how the tool progresses it's calculations.
The Magic of 3,001 & 6,516 & 56 hours
After 'Step one' of providing your company name and employee count, you are presented with the neat graphic of a printer in 'Step 2' (looks like a HP Deskjet 710c I once had) with your first hard cost savings claim. You will either get a dollar ($) savings, or in my case a percentage (%) savings claim? Why the difference?
Customer inputs between 1-3,000 get dollar estimates while customers larger than 3,001 get percentages. As you dig deeper, you notice why Google switches their claims because the tool shows you that larger customers save less money. In fact, I plotted increments such as 1, 10, 100, 1,000, 2,000 etc…and noticed below the extremely steep decay curve (more like ski jump) that exists on the estimated hard costs savings. Interestingly I found that at 6,516 employees, according to Google, you save ZERO on hard license savings.
Soft Cost savings will accrue through the remaining steps (3-12) but no matter how many employees you have, the tool will always say employees will save 56hours of work. That's a bold claim with no substantiation and seems to conflict with these customers experience.
Errant Mobile Claims
Step three makes a claim that mobile workers will be 2.8 times as productive. Never mind that this assumes an always on connection. Will anyone fly? Are any of my employees in countries with slower bandwidth? Also since Google nixed Offline for Docs, can they really justify this number? Mary Jo Foley might weigh in here. Also if Google would follow the rest of the guidance in the Forrester report they like to cite, they'd factor in the segmentation which argues for a majority of users needing a rich client app like…Outlook. Given all of their issues with Outlook sync, that would more accurately lead to a productivity number of….well in the words of Jake Harris, an IT Admin using Google Apps 'a headache'.
Step 4 just makes a claim of how large your inboxes will become with Google Apps. It takes your employee count and multiplies by 24.5 GB to spit out a figure. Do you want all employees saving every single email? Do you need archiving? Where is the added cost of Postini archiving? Regardless, it does not factor into these calculations. My savings are still showing zero.
The worst SPAM filter…ever!
Step 5 states workers will avoid the 28hrs they currently spend dealing with all that SPAM. Google provides nifty per second, per click calculations based on users getting 77 spam emails a day they must triage. Do these people have no spam filter in place now? Where do they work, Swiss Cheese Inc? Even free services like Hotmail, Yahoo, AOL have SPAM filters. Outlook even has a built in junk email filter. I've never clicked on 77 SPAM messages in a day, ever. Regardless, the tool assumes it's all upside and introduces the first soft costs in a new graphic by multiplying 28hr * 7,000.
Revisions and team work.
Step 6 communicates a fact about document revisions. Problem is it doesn't even scale with the inputs. Put in a 1 person company, it assumes you work in a 10 member team, put in 10,000, same calculations. This is why using a linear scale is faulty. It also avoids the #1 issue of collaboration that Google is yet to fix, Fidelity. When you work with anyone who isn't on Google, whether they use OpenOffice.org, Microsoft Office, et al, Google Docs can destroy your content. Avoiding this fact is sneaky. The tool should offer assumptions such as average number of existing documents per employee to import when 'Going Google' and how many documents have macros, graphs or styles that will outright break. Take that, apply a salary to fix and subtract from time saved and money lost. The tool does alter it's estimates.
36% more efficient – trust us
Step 7 uses an internal, qualitative study done by one customer in the newspaper industry. It's not provided and I couldn't find it on the internet. The customer found 36% greater communication with Google Apps. What does that even mean? 36% more IM's? 36% more information created? Employees left work 36% earlier? Still no changes to my calculations so why is Google even offering these steps?
Those terrible operating systems
Step 8 is truly out of left field. It uses the fact that operating systems may require 1 hr updates and patches. Fine, but how will all of my employees access Google? Here's a hint, with computer running any OS that must be updated. So what does this have to do with 'Going Google'? For fun here's an old article about Apple, Windows and Linux on updates. It also doesn't factor in Google's own 1hr of planned downtime per MONTH, which is the same impact? Regardless, the tool added 7,000 hours of savings. (1hr * 7,000) New Totals appear!
Ignore the Google Apps 450+minutes of downtime since mid-April
Step 9 is a great one. It uses a study by Radicati, not available without subscription, to show Exchange, Notes and Groupwise downtime per month. It defaults to Exchange with 150mins /month. I won't even bite on this number. It calculates every employee will gain an additional 27hrs of productivity by 'Going Google'.
This again ignores Google's own 12hours of planned downtime per year and Google's free 10min downtime rule. Out of curiosity, I looked at the Google Apps dashboard that shows Google Apps services have accrued over 450 minutes of outages in the last 60days! Lost money? Lost Hours? The tool updates by taking 27hr * 7,000 for my new totals.
What if 10% of all employees lost their laptop? Tack it on.
By Step 10 you are getting how this shell game is working, but you click on. It takes the % of employees that use a laptop (set at 60%) and assumes 10% of them will lose their laptop with 10GB's ea. For my example, that equals 420 employees losing 4,200GB. Well, according to Google that means $47,246 per employee in costs. This is getting fun now as I am awarded potential savings jackpot of $19,843,320. (420 * $47,246)
The reasoning here is so faulty but the tool takes this money and puts it into the hard cost saving bucket, so I now have new totals.
Now we're talking…pile it on $$
Step 11 takes an estimate of $150/user cost for data recovery services and multiplies it by my 7,000 employees. Is this money I already pay or can entirely avoid? Do people back anything up today? After my previous $19m jackpot, I am disappointed it's only $1,050,000 but I'll take it. Did you know Google Apps won't tell their customers where their data is located? What about the costs associated with risk and exposure of your data being held in regions where you don't even conduct business? Or most cloud providers have a force majeure clause which means you should back up data locally as well. Regardless, the hard costs increase to new totals.
Close it out with free posters
Steps 12-13 finish up with some marketing and a nice touch of taking the results with you in the form of Excel (for real calculations) or a poster with which you can tell your CEO you just found a way to save her over $20Million dollars! Good luck trying to defend how you came up with that number.
Instead, I recommend you spend your time showing your CEO how Office, SharePoint and Exchange 2010 can save them money and make people more productive just like other companies such as Sony, Telus, Football Association, General Mills and Subaru.
Oh wait, one last number. . . 11 minutes. That's the time you'll save not wasting your time filling out the so-called Google "calculator".
This is great. Why would google even put something like this up?
Tom - Thanks for reading. It's a great question, maybe someone from Google will read and respond? :)
Great post Andrew! I guess Google is redefining what it means to do no evil on a daily basis? I know this is marketing, but clearly Google is riding one their "we're cheap, and good enough" message to customers.
I love the way their "2 min calculator" turned into an 11min one...just an example of the crpa they spit out!
Awesome analysis of their tool. Very few users do an objective comparison of what Google offers (or does not offer) as compared to Microsoft.
I like it and I'm impressed by the detailed professional analysis ... and I have the same question around my head: "Why would they do something stupid like this?"
Fantastic work Andrew! Companies that truly take the time to evaluate the costs and benefits of any technology would easily come to the same conclusion, but it's great that you've stepped through this so completely.
Google actually recommends half the density for BES.
500 users per BES server for Exchange (MS Recommendation)
250 users per BES server for Google (Google Recommendation)
I tried that calculator yesterday. For 10 emplyoees, Google Apps would offer me -750GB of storage space, enough for - 1.8M emails: http://bit.ly/b1CkOf
@rADo unfortunately you are incorrect. I just confirmed. 750GB would be 75GB per employee? The tool shows 245GB for 10employees. 24.5GBx10 per the calculations. Maybe test again.
@Andrew - my mailbox is unlimited (Hosted Exchange, US$ 10 / month), but I entered 100GB / mailbox via "Edit Assumptions" link. I could enter also 1000GB or anything else.
10*25GB (Gmail) - 10*100GB (Hosted Exchange) = -750GB
Great job! It never ceases to amaze me that Google can get away with false analysis like this and it not get caught! I like the breakdown and explanations. Thanks for doing this work!
This is awesome Andrew. I had a couple of clients migrate to Google, and even before their free 30 days had expired they came running to me like puppies. The loss of productivity, the loss of offline support, and not to mention the loss of calendar sharing capabilities (or at least decent ones). And the biggest pain was the migration back, all their mail from before uploading to Google was synced back with the same date flag.
Damn Google (not to mention their privacy issues).
Fantastic read, amazed at the effort put in going through bungle calculator
The 10-12minutes I spent reading this and watching the video is obviously a better use of my time than doing the actual calculator for an IT department....I am an ex-Gartner TCO analyst and this is hysterical. Google seems to have confused "Don't be evil" with "Don't have a clue". I pity any poor CIO that has been conned into the switch and now has to defend his/her job because of the crap the field and the board are now piling on.....
Check out Exchange 2010 and leave it at that....