Blog

The Recall Based vs Observed Behaviour Debate

By Wayne Clouten, BPR

First Part of a Two-Part Series

Over the last years there has been some debate about the relative merits of recall-based research vs observed behaviour data. Sadly, the debate often degenerates into a chest thumping exercise about which is superior depending on where your vested interest lies.  The observed behaviour tribe like to portray recall -based research as old fashioned and expensive, while the sample-based tribe tend to highlight the obscure sample integrity of digital and the inability of observed behaviour analytics to see outside of the box or into someone’s mind.

The reality is that both forms of research have their place and the issue is not whether one is “old, new, better or bad” but rather which tool most accurately serves what you are trying to achieve.

Before we go too far down the rabbit hole it is worth defining the difference between the two methodologies for the purposes of this article.

Recall based research is where questions are asked of a stratified representative sample of the marketplace.  The responses to those questions will generally cover things such as what people remember of their listening behaviour across the day/week, what they think about your radio station and your competitors and what their listening preference are (likes and dislikes).

Observed behaviour data in the case of radio generally means transactional data siphoned from digital listening streams, particularly your station stream.  It normally measures such things as when users log in and log out and how long they listener for. The amount of demographic profile information available varies depending on the sophistication of your streaming providers analytics and log-in mechanics.  Observed behaviour data for podcast downloads can also be part of this.

Observed behaviour research based on transactional data has been around for a very long time.  In the case of the retail industry since the invention of the cash register.  I recall pouring over millions of transactional records in various retail and FMCG projects prior to joining BPR.  The analysis involved identifying sales trends, optimal price points, credit cards used, customer flow charts and the relationship between sales and display locations or staffing levels. Recall based research in the retail context was about what could not be drawn from the transactional data such as brand insights, advertising recall, the degree of satisfaction with the shopping experience, what needed to be done better and things the customer wanted in the future.

When the ability to capture digital transactional data on media and entertainment consumption came along, I was an enthusiast, even serving on one of the early committees involved in developing passive people meters.  Observed behaviour data is very sexy to play with however there is one extremely significant issue with using observed behaviour data in a strategic context for media, particularly radio.  The issue is that radio listening is mostly an emotional transaction.  The essence of radio programming is understanding the emotional transaction between your content and your listeners whether that transaction be good or bad.  The best indicator of that emotional transaction is what your listeners remember of their listening experience, what a listener remembers about your radio station reflects the emotional imprint you made on the listener.  Critically, what a listener remembers about your radio station frames their opinion of your radio station and this is reflected in two fundamental behaviours:

1. What they subsequently tell other people about (I don’t have to tell you how critical talk of town is)
2. The likelihood they will tune into your station again at their next radio listening opportunity.

It follows that the best and most effective way to measure the emotional imprint you make with a listener is what that listener recalls about your station.  Recall is the ultimate litmus test of how effective your station has been in triggering an emotional transaction.

You may have a radio turned on (or an app open in the background) but that does not mean you are actually listening to that radio station or actively engaged or remembering the experience.  This is what a lot of people miss when arguing the relative merits of recall-based vs observed behaviour data when it comes to media, particularly radio.

In the next instalment we will discuss in more granular detail the relative merits of both forms of research and outline the most appropriate situations to use either or both.