{{title}}
{{textarea}}
{{task}}
{{select}}
{{team}}
{{input}}
{{solution}}
{{result}}
ICDS Insights Behind the Method
A Null Result Is Not a Failed Study
A note on what to do when the first test of a hypothesis comes out near zero.
VM
Vadim Matyushkin Behavioral Scientist & Sociologist · ICDS
The first six articles of the series worked at the level of single events: a media peak, then search activity in the following months. Article 7 asked the next question. Does the whole-year social media presence of a brand relate to what consumers think of it and to what they buy? The test used three sources: annual Share of Voice from the ICDS corpus (71 brands, six markets, more than 900 million mentions), the YouGov Consideration Score for attitudes, and Kantar Consumer Reach Points for actual purchases.
The First Result Was Close to Zero
Share of Voice against Consideration gave a correlation of 0.12 to 0.15 on 51 brands, with p above 0.3. Six specifications were tried and only one reached p = 0.05: a one-year lag on 22 brands (r = 0.42, p = 0.05). The article reports it as a preliminary signal and does not use it as confirmation. Within Beverages the sign was negative (r = −0.33, N = 26).
0.12–0.15Share of Voice vs. Consideration, 51 brands (p > 0.3)
1 of 6Specifications reaching p = 0.05 (one-year lag, 22 brands, r = 0.42)
ρ = −0.33Beverages: the sign ran negative (N = 26)
Three Possible Moves
My first reaction to that result was a flick on the nose, a trip after which you get up, brush yourself off and look, this time, at your feet. One option I was offered was to accept the result as it stood and record that the hypothesis had not been confirmed. That did not satisfy me. I believe regularities exist, and I kept searching.
At that point there were three possible moves. The hypothesis could be dropped. More specifications could be tried until one passed. Or the instrument could be examined to see whether it was able to register the effect at all. Article 7 took the third route, and the result of that examination is the main content of the article.
What the Consideration Score Actually Measures
The Consideration question asks which brands a person would consider in their next purchase. Three groups of people answer it in the same way. New buyers, whose attitudes media could shift. Habitual buyers, who answer yes because they will keep buying the brand. Aspirational respondents, who name brands they are unlikely to purchase. In FMCG, where households keep a stable set of brands and buy them repeatedly, the second group dominates. The published score then reflects established loyalty. It has little room to react to a media event.
The data fit this reading. Of the 22 brands with Consideration data for both 2023 and 2024, all 22 rose. The mean increase was +2.18 percentage points, from +0.3 (Sprite) to +12.7 (Dove). The article links this to the general recovery in consumer confidence after the cost-of-living peak. A score that moves up for every brand leaves little brand-specific variation for a media measure to explain.
A Second Problem: Global Presence, Local Consideration
A second problem was scale. Share of Voice was available as a global figure, and Consideration is surveyed inside individual markets. In Beverages, Coca-Cola had a 43% Share of Voice and a Consideration of 32 in the US. Almarai had a Share of Voice near zero and a Consideration of 57 in the UAE and Saudi Arabia. Global presence says little about local consideration, and the correct test needs Share of Voice by market.
Questioning terms and metrics that business uses automatically, out of habit, is part of my experience and skills, and it is what led me to examine Consideration itself. The gap between global presence and local consideration took little effort to see. The data practically shouted.
The purchase side gave a weak positive signal. Share of Voice and Consumer Reach Points correlated at r = 0.55 (p = 0.16) on eight brands, and six of the eight moved in the same direction. The article reports this as a direction and makes no claim of significance.
The Decision Taken
The decision taken was therefore a specific one. Article 7 does not say that media has no effect on attitudes, and it does not say that it has one. It says that this test, with these data, could not have detected the effect, and it lists what a test would need. YouGov publishes Purchase Intent, and it is absent from open-access reports. The article names that absence as the most consequential data gap for the attitudinal stage of the Signal Chain.
What a test would need
- Share of Voice by market, not a single global figure
- A Purchase Intent measure in place of Consideration
- Monthly granularity
- A panel design
What a null result carries
This changes what a null result carries. A test that cannot detect an effect produces a null result whether or not the effect exists, so the number alone settles nothing. What the null result does carry is information about the measurement: which public metrics respond to media over a short horizon and which do not. That information tells the next study where to look, and it tells a client which published indicators are unsuited to the question they are asking.
The annual level remains an open question. The monthly peak-to-search chain, tested in earlier articles, is not affected by this result.
VM
About the author
Behavioral Scientist & Sociologist · ICDS
Vadim Matyushkin is a psychologist and sociologist with close to twenty years of research into digital behavior, trust, and information dynamics, and a researcher behind the Institute of Communication and Data Science (ICDS). His work has supported organizations including Coca-Cola, PepsiCo, Mars, Danone, Nestlé, and Bayer in moving from self-reported survey data toward direct behavioral evidence.
ICDS — Institute of Communication and Data Science is an independent research institute focused on understanding how trust, behavior, and reputation are formed in digital environments. ICDS operates as an intellectual institution — not an agency, not a platform, and not an educational provider.