At AdExperiments our focus on digital creative testing is to understand the online impact of ads through A/B optimisation. We work with panel providers who have shown an aptitude for balancing sample effectively. But their methods for sampling can be complemented with simple first party data. This can be easily captured for mobile and desktop respondents and can be used to verify audience composition or balance it, ensuring that our sample truly represents a digital audience.

The modern day panel

Good panel providers have long adhered to best practices of balancing their sample across age, gender, income and household to try and capture the elusive "nationally representative" sample. The best providers have worked hard to avoid the low value incentivised recruitment techniques which eroded trust and prompted clients to dismiss panel respondents as consisting solely of unemployed, low income, bored people at home.

Many providers now use a new variety of modern recruitment methods to capture different types of respondents such as: airmile rewards for the high income traveler, shopping vouchers for families and gaming rewards for a younger generation. But beyond these cleansing factors it is really up to the researcher to identify attributes of the respondents that can be predictive of the behaviour or behaviour change they would like to capture from the research.

I should caveat that whilst this range of recruitment methods supports a good online sample, if you are interested in the population as a whole you should consider a panel that employs both online and traditional methods of recruitment (door-to-door and phone) to capture older demographics with a lower online presence.

The modern day digital researcher

Being in the business of digital creative measurement we wanted to look at how some digital specific factors affect our audience. When a user visits a website the web browser passes along their user agent to the website. The user agent identifies their browser, operating system (OS), relevant versions, and other information such as device model. We explored how the OS and browser captured through our benchmark studies indicated brand knowledge and ad preference.

When looking at hardware product ads the starkest differences were observed. Respondents identified as Safari users (the Apple default browser) had less baseline knowledge of non-Apple branded phones and smartwatches than other respondents and a greater level of irritation with these ads. Respondents identified as Edge users (Microsoft's newest browser) had better knowledge of a new Windows compatible HP laptop than other respondents.

Media consumption habits varied in particular for IE (Microsoft's older browser). IE respondents were less likely to regularly visit entertainment, online video and tech websites than other respondents. Firefox users were less likely to visit family related websites. ^[1]

Media consumption habits also varied by OS, with Android users more likely to spend time on gaming apps and news content and iOS users more likely to spend their time on social media.

This simple analysis shows respondent brand knowledge and media consumption habits do vary by browser and OS. But was this naturally catered for by a standard demo balanced sample send out? We tested several panel providers across multiple markets and compared browser and OS distribution to actual market share.^[1:1] For desktop, browser share of the respondents was incredibly consistent with true market share. Where we did observe a significant variance from true market share was the mobile operating system, with some panels in particular markets skewing towards Android or iOS. Potentially this is because the panels' recruitment methods on mobile tend to be app based, which can have a heavy OS bias depending on their availability and popularity in either the iOS App Store or Android's Google Play. A desktop, web based recruitment would be far less likely to encounter browser compatibility issues at scale, potentially accounting for the natural consistency.

Based on our findings AdExperiments have released a mobile operating system based quota check, specific to the target market, as a default setting for all our mobile video and display studies to complement the balanced demographic controls already in place.

---

---

1. Source OS share: kantar; Source browser share: gs.statscounter.com ↩︎ ↩︎