Google’s Addition Problem is Significant
Google estimated their effect on the US economy in a recent report called ‘Google’s Economic Impact’ (PDF). In one part they assume that for each click on a paid ad there are about 5 clicks on the ‘natural’ listings in their search results. They also assume that each natural click is worth about 70% of a paid click. Combined, all these clicks were worth about $54 Billion dollars in 2009, they claim.
There’s a tragic mistake they make in their calculations, but I need to explain the concept first, so please bear with me for a minute:
Let’s say that you want to manufacture an invention of yours and your brother has agreed to help you fund it. (“Whatever money I have is yours to use,” he said.) You’ve just graduated from engineering school and when you scrape together every single penny you have, you find that it’s $1,618.03. You’ve counted three times and that’s how much money you have, to the very last cent.
Your brother was a world-famous snowboarder who’s now retired at age 24. You call and ask how much money he has. “I don’t know” he says, “about $10 million.” About? No, seriously, how much money? “Look, little buddy, most of my money is in stocks. My accountant told me they were worth about $10 million last year, but the market’s been bumpier than a Jamaican half-pipe. So, maybe I only have $8 million. Maybe it’s up to $12 million. I just don’t know, dude.”
OK, so how much money do you have to work with? Would you say that the most conservative estimate is $8,001,618.03, to the penny? No, of course not. Your brother’s worst-case estimate was about $8 million. Another thousand or two does not change that estimate at all. “About $8 million” plus $1618.03 is still “about $8 million”, isn’t it?
In science and engineering, the concept is called ‘significant digits’. You only know how much money your brother has to the nearest few million, so it doesn’t matter how many pennies or dollars you have. Your contribution is not significant to the overall total.
But look at this snippet from Google’s report. (They show similar figures on a state-by-state basis):
Google estimates that 1,600 advertisers and website publishers received $24,140,000 in economic value from Google’s services.
First, this number actually isn’t known to the nearest $10,000. It’s a rough estimate. If every natural click was worth 60% of a paid click, then their estimate would have been about $21 million. If each was worth 80%, then their estimate would have been $27 million. (Even the 5 natural clicks per paid click was rounded down from an estimate of 5.3. Had they used 5.3 instead, it would have added about another $1 million to their estimate.)
You see, Google’s calculations are only significant to the first digit – it would have been better to write the estimate as $2?,???,??? dollars. Everything after that first digit is really just a guess.
But then Google says that they donated $1,000 in free advertising to 10 non-profit recipients through their Google Grants program. So, they add that $1,000 to their estimate of $24,140,000 in economic value to get a total impact of $24,141,000.
This is terrible math. High-school students have lost points on their physics homework for ignoring the concept of significant digits when doing calculations like this.
Google bungled the launch of Buzz. (Even one their founders, Sergey Brin said, “We screwed up.”) They might get sued for collecting wi-fi data. (Google claims that the collection of personal data was “inadvertent”, which is another word for “we screwed up”.) Now, they can’t even add two numbers together correctly.
Look, if your stated mission is “to organize all the world’s information”, then you need to be competent enough to do so. And if the engineers at Google haven’t yet mastered the concept of significant digits, then perhaps they should not be the ones in charge of that.