The implication of the word ‘research,’ is that you are setting out to understand something. All too often over the years, I have observed situations where that is not the case.
Market research can be a money trap, consuming resources with little or no payback. It can also be a huge capability to be leveraged for great benefit when done well.
The challenge is that it is a set of interrelated disciplines from statistics, psychology, behavioural economics, and science, and therefore requires a wide breadth of skill and acquired wisdom to be useful.
Doing commercially productive market research is a bit like learning to swim. No matter how much you read about it, study wise texts, and observe others, until you get into the pool, immerse yourself, you will never really understand it.
Some things are relatively easy to research. Usually they are adverse outcomes that have happened, and have been quantified. The research is aimed at understanding the drivers of those adverse outcomes. More challenging is research that seeks to put a shape around the future. If this happens, what then?
Most material published on the topic is about the techniques, the templates to use. They are very useful, but fail to accommodate the realities that intrude in real commercial situations, that impact on a research outcome.
Following are some of the hard won lessons from doing marketing and market research over the last 40 years. The tools have changed dramatically in the last decade, the principals remain unchanged.
Not understanding the ‘scientific method’.
Most are familiar with ‘the scientific method’. Identify a problem, form a hypothesis, test that hypothesis, adjust the hypothesis based on the results, rinse and repeat. However, most do not recognise the foundation of the scientific method is to set out to disprove an idea. This objective to disprove a proposition, ensures that all relevant information is made available. All contrary data, opinions and untested ideas are brought to the table for examination. It often happens that information that may be relevant is not considered. Ego, confirmation bias, existing standard procedures, and just lack of critical thinking clouds the process. Over the years I have seen piles of research that is setting out to prove a theory, and does so by, usually unconsciously, excluding data that might not confirm the proposition.
Failure to identify the problem/opportunity.
Useful research depends on providing answers to problems, or offering insight into the scale and location of opportunities. In the absence of clarity of the objective of the research, you cannot reasonably expect there to be any value delivered.
Asking poor questions.
Not all questions are created equal. Asking good questions implies that there has been enough work done to identify what is, and what is not, a good question. Also important is the manner in which the questions are asked. It is easy to generate different responses by seemingly subtle variations in the way questions are asked. Eg. ‘How big do you like the fruit pieces to be in your brand of yoghurt? This implies that the fruit in yogurt is in pieces, and ignores the possibility that the fruit may be added as a puree. Those who may prefer a homogeneous product using puree are thus precluded from giving an accurate response. Such a question would be relevant to the marketer of fruited yogurt seeking a point of differentiation, which would influence the product ingredients and choice of processing equipment.
Less than rigorous & neutral data collection & analysis.
We all know numbers can lie, we see it every day. Numbers can be used to support any proposition you choose, when managed to that end. The absence of rigor from research methodology and analysis, will lead to flawed conclusions every time.
Not knowing what you will do with the outcomes
In the absence of a clear use for the research, why do it? The answer to this is usually found amongst ego, seeking validation of a currently expressed position, or as a crutch to avoid making a decision. How often have we heard the phrase: ‘More research is required’
Selective use of results.
Selective use of research outcomes is standard practice in many places. Parts of the research that supports a proposition are used in the presentation of a position, and any parts that do not support the position are ignored. You see this all the time in the political discourse in this country. Politicians of differing parties, taking the same research reports and claiming opposite conclusions is common. Exactly the same process exists in corporate bureaucracies.
Lack of understanding of the techniques
You do not have to be a statistician to be able to understand the outcomes of data. However, you do need to understand what the terms mean, and the implications they carry. This applies from sampling techniques, to the tools of statistical analysis and results presentation. You must understand the principals sufficiently well to be able to ask informed questions, and recognise gobbledy gook when it comes back to you.
Not considering Anthropology & Context
Anthropology might seem a bit misplaced in market research, as it is the study of behaviour in varying cultural settings. However, consider how different the answer to a question about your work might be if asked while sitting at your desk absorbed by a task, to when asked the same question while on a holiday. Same question, different context.
These days we are often allocated to teams at work that are set up to solve problems, generate ideas, or just manage work flow. How different are our reactions inside those groups, to those to which we choose to belong outside the work context, and how differently do we behave?
Conducting research in the absence of such considerations can generate misleading outcomes. E.g. Conducting research on a new piece of packaging around a group discussion table will evoke responses, and a conclusion. How different might the reactions of those same people be when confronted by the new pack while shopping in a supermarket.
Failure to understand the drivers of Behaviour
Psychology plays a huge role in the development and reporting of research. Our brains are hard wired to reduce cognitive load, so can be easily tempted to accept a conclusion not supported by research. It is relatively easy to persuade others of the veracity of a conclusion, simply by the manner in which they are presented. E.g. Which milk is better for you: one that contains 3% fat, or one that is 97% fat free? They are identical products, but in research, a significant majority will answer ‘B’: 97% fat free.
Similarly in a qualitative group discussion, a proposition seemingly supported by most around the table can gather overwhelming support, irrespective of the accuracy of the proposition. This outcome has been repeated endlessly in first year psychology experiments, based on Solomon Asch’s 1951 experiment seeking to examine the power of a group to influence the expressed opinion of an individual.
What people say they do and what they actually do can be very different.
When you ask questions, they are answered from within the existing frame of reference of those being questioned. Their ‘mental Models’ dominate how they see things. Henry Ford was right when he quipped: ‘ He would not consult customers on what they wanted because he already knew, a faster horse. Steve Jobs expressed exactly the same opinion, in different words on several occasions, and was again proven correct.
Too much research is aimed at connecting the future dots to give a sense of certainty about the future, just to make people feel more comfortable. If we could tell the future accurately, we would all be at the local casino for a few nights until we got banned for winning too much.
Respecting the status quo too much
We humans are keen to retain the status quo, simply because it has been proven to work, and change involves risk. We are hard wired to avoid risk, a function of evolutionary psychology, when taking risks often meant you became breakfast for something nasty. The promise of a reward must be many times stronger than the downside of a behaviour before most of us are prepared to entertain the risk.
Presenting a research finding that is inconsistent with the well known view of the Managing Director is a risky undertaking that is often avoided. This is commonly called a HiPPO (Highest Paid Persons Opinion) and is pervasive. It is particularly challenging when the person concerned (often a bloke) is repeating the opinion of someone else. In consumer products, this is often his partner.
Poor presentation of results & Conclusions.
The errors I have seen in presentations are myriad. However, the worst are:
- Lack of clarity and simplicity in the conclusions, which limits useability.
- They do not answer the question. Generally this is because the question was ambiguous, unnecessary or stated a proposition someone wanted verified.
- Death by Powerpoint.
Every research project can be placed somewhere on the matrix in the header. The more right hand side and higher you go, the greater the degree of uncertainty is involved. In the bottom left quadrant, you are seeking answers that are quantifiable, things that have happened, that you are seeking to understand. Top right quadrant is the future, and contains things we do not know much, if anything, about. Often we do not even see them. Research that puts numbers against hypotheses that fall into this quadrant should not be believed. At best they are an estimate of a probability, at worst, just a WAG. (Wild Arsed Guess). What is important in these circumstances is that you understand the risks. Remember that old cliché, ‘plan for the worst, hope for the best’
