Wednesday, November 14, 2007

New metrics: or some things I wish I could measure


Just sat through an engaging presentation from an online market research company (OTX) which has some new brand-experience measurement tricks up their sleeve (modeling store environments, distracted exposures, etc). But the presentation also reminded me of the other things I wish someone knew how to measure in a cost-effective way.

1) Brand utility: of all the newly coined phrases describing the attention to what brands actually do for consumers (vs. what they merely show and tell), “brand utility” is my personal favorite. I first heard it in an interview on PSFK with Benjamin Palmer though I think he co-credits Johnny Vulcan of Anomaly with the coinage. Wouldn’t it be great if we could develop a metric that would measure a brand’s utility factor (or whatever we call it) in relation to the competition? If what we are all saying is true, then “utility” should be just as, if not more, important than “relevance” and “uniqueness” and all the traditional metrics.

Imagine if we could compare the relative utility factors of all the travel sites out there: Travelocity, Kayak, Expedia, Orbitz, etc. This wouldn’t be a strict usability measure which is easy enough to generate but rather some combination of ease of use and ability of the brand to anticipate and facilitate your desires behavior, both on and offline. It would be a more comprehensive measure than one you could generate through simple online user satisfaction survey.

Amazon might be considered the gold-standard of a certain kind of ecommerce transactional utility, against which other ecommerce brands could aspire.

Once we established a baseline, then we could--theoretically of course--measure how much a given usability, service improvement or new application impacted the score, and then (praise be the intelligent designer!) we might even determine if the cost investment in the improvment was worth the result? Maybe someone can already do this; But I haven’t come across it.

2) Sharability/Network/Viral effects: Another thing almost everyone wants these days is for their work to go all viral: have consumers talk about their brands and products and advertisements for free, for fun. It’s not easy but not impossible to measure how well content (or a conversation, as we say today) travels around the web once it’s produced, but wouldn’t it be cool if we could pre-test the power of a piece of a content in some kind of enclosed network? You might call it something like a Network Communication Test. The goal would be to see both how sharable some particular chunk of content was and to understand why it was so sharable. Which might even help us develop even more viral-icity.

3) Participation: Everyone not yet involved in social media that wants to be involved in social media looks at the rise of MySpace and Facebook and thinks it’s easy to get consumers to participate in their sites, rating and reviewing and uploading video of themselves doing stupid things with their friends when they're drunk. Those of us working on sites of various kinds know it’s not as easy as it looks. That we often have to prod consumers (with incentives of various kinds) to get them going and optimize the mechanisms for involvement before the machine really starts humming. Would love to find some way to measure the interest in and barriers to participation on a competitive basis at a significant scale and then compare the effectiveness of various models of inciting involvement.

Any takers?

2 comments:

Brad Noble said...

How bout ongoingness? I always want to know how well an experience suggests repeat exposure, use elsewhere, printing for later, reference.

sk said...

Great idea, Brad. Obviously, you can measure repeat visits through site analytics, but the broader definition of "ongoingness" you are describing certainly eludes our abilities. We'd need to create another composite score involving bookmarking/tagging, reference in online conversations and other re-use (printing coupons to redeem)... Ultimately, we'd need some magical virtual rfid chip that would move from virtual to real objects which I don't think exists.
But I almost think we should make a list of these measurement gaps and pose a challenge to all the research co's out here.