How we (mis)conceptualise p-values (and what we can do about it)

This research presentation was given at the inaugural Annual Research Conference of the AAAPC (Australasian Association for Academic Primary Care) in Adelaide, on 12 July 2019.

Tam CWM, Khan A, Knight A, Rhee J, Price K, McLean K. How we (mis)conceptualise p-values (and what we can do about it). Annual Research Conference, Australasian Association for Academic Primary Care, University of Adelaide, Adelaide Australia. 2019 July 12.

Slides (PDF)

YouTube video:

Abstract

Context

Researchers and clinicians have been criticised for frequently misinterpreting and misusing p-values. P-values are often misinterpreted as providing far stronger evidence than is actually the case, which not only has harmful impacts on the understanding of medical research, but potentially on the delivery of patient care.

Objective

This study sought to describe and categorise what and how experienced clinicians (in this case, Australian and New Zealand GPs) conceptualised p-values presented in the manner that it is typically encountered in a medical publication. These results may help inform how to provide targeted statistics education to clinicians.

Design

This mixed methods study used quantitative and qualitative questions embedded in an online questionnaire. It included questions that elaborated the participant’s conceptualisation of “p = 0.05” within a scenario, and tested their p-value interpretation ability and confidence.

Setting

The online questionnaire was promoted in a Facebook group of authenticated Australian and New Zealand GPs and GP trainees (GPs Down Under) in 2017.

Participants

There were 247 respondents who completed the questionnaire. In brief, a preponderance of the respondents were female (66%), Australian residents(97%), Fellows of The Royal Australian College of General Practitioners (71%), and reported some research (82%) and teaching experience (72%).

Findings

Participant conceptualisations of p-values were described by six thematic categories. The most common (and erroneous) conceptualisation was that p-values numerically indicated a “real-world probability”. No demographic factor, including research experience, seemed associated with better interpretation ability. A confidence-ability gap was detected.

Implication(s) for practice

P-value misunderstanding is pervasive and might be influenced by a few central misconceptions. Statistics education for clinicians should consider explicitly addressing the most common misconceptions.

Leave a Reply

%d bloggers like this: