Who really is measuring?

Some time ago I saw some Melcrum research that suggested that although everyone in IC said you should measure and evaluate, very few people were actually doing very much about it.  In fact when I ask people on the Black Belt Programme, most people admit to feeling guilty that they don't do enough.

I think that there are basically three levels of measurement:

• Process - did we do what we set out to do?  Were ten newsletters published?  Did we hold the town hall meetings?  And if not, what can we learn from our failures?
• Understanding - did people get the message from the communications and can they play it back to us as we hoped?  Are we having a sensible conversation about this issue?
• Impact - did anything change as a result of the communications?

Clearly measurement gets more difficult as the usefulness of the information gathered increases.  Before you can measure Impact properly you have to isolate out the other factors apart from communication that might drive behaviours in a workplace - no mean feat.

But wherever I hear about the paucity of measurement I shake my head and sigh.  My theory is that other functions are so much better than us (with the exception of PR of course!) and I assume that if the day ever comes when IC people routinely spend big bucks then effective measurement will be the norm.

This little snippet from IT analysts Forrester has rather shaken my faith!

And this week I received an excellent e-newsletter from measurement writer Angela Sinikas in which she tells a story about being challenged on the ethics of measuring at all.  Apparently someone wrote once about one of her articles:

"I'm sure the slavish focus on the organization's bottom-line concerns makes for effective consultancy, but it disrespects the humans who invest so much of their lives therein."

Angela uses this story to make the point that failing to listen is even more unethical than being worried about business performance - but it does make me wonder whether there is a silent majority out there that is in denial about the need to measure at all.

Liam

Comments

[Adam Hibbert]

For my part, a reluctance to get into measurement is due to the tenuous quality of the facts at the "impact" end of the measurement scale, which is the end we need to reach in order to demonstrate real added value.

In financial services, there are at least seven people in any Exec meeting who are better at numerical analysis than you are, and just one vulnerability in your data will trash your credibility.

Perhaps worse, people might take your data seriously when they shouldn't ... when your methodology actually conceals a flaw that makes it impossible to budge the metric.

Say, for example, you identify in a climate survey that comms are felt to be a weak area. So you go to your exec to ask for 20% more cash to raise the comms game, and tell the employees you've heard their view and you're investing hard. Next year, that climate score is worse ... because the expectations you accidentally set have adjusted upwards by 30%.

So it comes down to a profound scepticism that what we do (to be bald, a type of politics) is susceptible to science. If there's a scientific method, it's at the very least interesting end of the behavioural range, where you give very functional information to people, whose behaviours then readily and measurably change.

Any attempt to measure the less tangible stuff, the engagement and discretionary effort part, has to measure behaviours that are subject to a huge range of influences. Short of keeping some employees in a darkened cupboard as a control group, I'm yet to encounter a convincing methodology for isolating the comms input to that.

Having vented, I must say I'd *love* for there to be a science, here. But much of what passes for metrics that I've seen so far falls well short of that quality.