Preview Changes will not be visible to customers until published.

What is Sig-Testing? [WIP]

If your table has filters in the rows or columns, then you can use the significance testing feature to test if a particular audience (e.g. Males) is significantly different from another audience (e.g. Males vs. Females, or Males vs. Everyone).

There are 4 types of Sig-Testing in AV2:

I: Everyone vs. Anyone (e.g. Completes vs. Males)

[Subset% sig-test] - test the "Completes" or "All Responses" filter against all other filters in your table. For example:

  • Completes vs. Males (true)
  • Completes vs. Females (true)
  • Completes vs 18-34 (false)
  • Completes vs 35-54 (false)
  • Completes vs 55+ (false)
  • Completes vs. UK
  • Completes vs. USA
  • Completes vs. AUS

[insert image]


II: Anyone vs. Anyone Else (e.g. Males vs. Females)

[Independent% sig-test]

Test any filter against all other non-overlapping (independent) filters in your table. For example:

  • Males vs. Females is fine because they're not the same, thus different.
  • 18-34 y.o vs 35-54 y.o is fine because they're not the same people, thus mutually exclusive.
  • 18-34 y.o. vs 55-99 y.o. is fine because again, they're not the same people, thus independent.
  • But 55-99 y.o. vs. UK is not fine because they're overlapping respondents
  • And Males vs. 18-34 y.o. is not fine because again, they could be the same people.
    • There can be 18-34 year olds who are Males, and conversely
    • There can be Males who are 18-34 years old
    • They're not different from each other.
    • They could be the same as each other.

[insert image]


III: Average vs Average (e.g. Males vs. Females)

[Average sig-test]

Test two


[insert image]



IV: Biggest vs. Smallest (e.g. Males vs. Females)

[Max vs. Min sig-test]

Within a group or span of mutually exclusive filters, e.g. Genders: Males, Females OR Ages: 18-34s, 35-54s, 55+ OR Markets: CAN, AUS, NZ, UK, US

  • First, we find the filter in each group with the BIGGEST % or score
  • Then, we find the filter with the smallest % or score in each group.
  • Lastly, we auto-magically test them for statistically significant differences using the independent % sig-test above in method II.

Et voila, see your output below!

Green: the difference between the highest% and the lowest% in each filter group is statistically significant.

Red: the difference between the lowest% and the highest% in each filter span is statistically significant.

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.