I’ve seen this post bandied about Twitter a lot today with the claim that IBM realized $4.6 million in savings thanks to tagging documents in their electronic document system. I’ve set up systems like that for companies in the past, and for a company like IBM, where knowledge is everything, making a knowledge base more useful is a big, important task. I was all set to trumpet these results as still further proof that social tactics, in this case something as simple as bookmarking, are smart. Since ROI numbers are sometimes difficult to come by, I thought this was a great find.
Unfortunately, I followed the link to the original article and found the numbers to be a little suspect. Don’t get me wrong, I’m not saying anyone is providing intentionally false numbers, but I will say the original post makes a couple of assumptions that most people wouldn’t.
Here’s the problem: The $4.6 million in savings is calculated based on a survey of users of a knowledge repository. The users claim an average of 12 seconds saved per search. The post then multiplies those 12 seconds times the huge number of searches done every week (Over 286K–remember, this is IBM so they’re probably hitting this database all the time) to come up with 955 hours saved. That’s the first problem. Those 955 hours are made up of 12 second bits of work that it is assumed will be spent doing something else instead of watching a computer churn through a search. That’s not the way people work.
On Twitter I compared this to a company forcing employees to wear zippers on their coats instead of buttons because it saves 12 seconds to take a coat off and on with a zipper instead of buttons. A company with 1000 employees could then claim they saved $552,000 a year based on the aggregated time savings from not buttoning during the six months when coats are required. (See below for the hourly rate I used to figure that out.) That’s not even really an accurate comparison since you really could use 12 seconds not buttoning a coat to do something else. Waiting for a computer to churn through a search still requires your computer, i.e. the tool you’re using, to be occupied. But you get the point.
The second issue is that the 955 hours per week are then said to equal “roughly” $4.6 million in savings. By my calculation, that means that every person making a search is earning over $191,000. ($4.6 million divided by 955 hours x 52 weeks a year.) That’s a pretty high median income. I’m betting it’s based on the average billable hour for a consultant from IBM since it comes close to $92/hour. But that’s the problem: I have to make some assumptions to get to $4.6 million.
What is really interesting about the original post, and the savings claim they make on behalf of bookmarking, is that they mention the system they instituted the program on was not very well liked. By adding tags, standard taxonomy, etc. to it they made it more efficient and more user friendly. As I said, I’ve built things like this before and the changes they put in place are usually considered just good organization of a document system. So it seems to me they’re making the claim that doing the work right saved time over the way it had been done. To me that seems as though they are trying to use work done the wrong way as a benchmark for work done the right way. That’s like claiming you saved time by not getting lost.
As I said, I would love nothing more than the $4.6 million claim to be true, and I’m afraid it will now be a part of social media lore that gets quoted over and over. That’s too bad and won’t help with the perception that social media is not serious about numbers.
PLEASE NOTE: If I’m reading the blog posts wrong, or if the assumptions are clarified/explained somewhere, please let me know. This post has been hastily written and I’m not trying to be combative. I also do not want to imply that blogging and social bookmarking are not great knowledge-sharing tools. They are. I just don’t think they should be oversold with numbers that might not meet the sniff test. I would love to discuss this with anyone and figure out what the numbers mean. And if no one disputes my numbers, and you use the $4.6 million in a presentation where I’m in the audience, expect to see me during the Q&A asking how they arrived at the numbers.