This research presentation was given at the Wonca Asia-Pacific 2019 Conference in Kyoto, on 16 May 2019.
Tam CWM, Khan A, Knight A, Rhee J, Price K, McLean K. How we (mis)conceptualise p-values (and what we can do about it). Wonca Asia-Pacific 2019, Kyoto International Conference Centre, Kyoto Japan. 2019 May 16.
YouTube video:
Abstract
Objectives
Researchers and clinicians have been criticised for frequently misinterpreting and misusing p-values. P-values are often misinterpreted as providing far stronger evidence than is actually the case, which not only has harmful impacts on the understanding of medical research, but potentially on the delivery of patient care.
This study sought to describe and categorise what and how experienced clinicians (in this case, Australian and New Zealand GPs) conceptualised p-values presented in the manner that it is typically encountered in a medical publication. These results may help inform how to provide targeted statistics education to clinicians.
Method
This mixed methods study used quantitative and qualitative questions embedded in an online questionnaire, delivered through an Australian and New Zealand Facebook group (GPs Down Under) in 2017. It included questions that elaborated the participant’s conceptualisation of “p = 0.05” within a scenario, and tested their p-value interpretation ability and confidence.
Results
Participant conceptualisations of p-values were described by six thematic categories. The most common (and erroneous) conceptualisation was that p-values numerically indicated a “real-world probability”. No demographic factor, including research experience, seemed associated with better interpretation ability. A confidence-ability gap was detected.
Conclusion
P-value misunderstanding is pervasive and might be influenced by a few central misconceptions. Statistics education for clinicians should consider explicitly addressing the most common misconceptions.
