The Phrase "Random Chance" for Explaining P-values

In the ASA p-value statement in 2016 (Wasserstein and Lazar, 2016), the following explanation is written in #2.

2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.

 

However, in many cook books for beginners of statistics, I find some other variants of expressions using the phrase "random chance". Because there are so many textbooks using the phrase "random chance", I had been afraid that some expressions might be correct or reasonable. So, I asked the following question to the first author, Ron.

Sorry for this sudden e-mail.

I am Yusuke Ono in Japan. I am not an academic person, but work for a statistical software company.

Although I don't contribute anything to ASA, I am a member of ASA.

 In your p-value statement 2016, you wrote as below.

 "2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone."

I find there are several variants of expressions with "random chance" when people explain p-values.

Do you think that all following expressions are false for expressing or explaining p-values in English?

 

a) P-values measure the probability that the data were produced by random chance.

b) P-values measure the probability that the data were produced by random chance alone under a proposed model.

c) P-values measure the probability that the data were produced by random chance under a proposed model.

d) P-values measure the probability that the test statistic is larger than or equal to the observed value by random chance alone under a proposed model.

e) P-values measure the probability that the test statistic is larger than or equal to the observed value by random chance under a proposed model. 

 

Best Regards,

 

Yusuke Ono

JMP Japan Group

SAS Institute Japan

 

The following is the reply from Ron.

Please call me Ron. Everyone does.

I agree that all those statements are false. I myself would not try to craft a statement about the p-value that involved the words “random chance.” Is it really clear to the ordinary person what those words mean, anyway?

Beyond that, p-values are influenced by many things. Is it clear that all those things are contained in the words “proposed model?” I am not sure that is always well understood or agreed upon.

 Anyway, I wouldn’t be happy with any of the definitions you listed, and I don’t think the panel that wrote the ASA statement would, either.

 Is this answer helpful? I hope so.

 Ron

 

Please don't think the above reply is an easy how-to rule for expressing p-values. How to understand p-values must be much more important than how to express p-values. I hope this information becomes a seed of your thought for understanding p-values.

 

Note: Although Ron allowed me to open the above reply, my query is just an informal one. 

 

[Reference]

Ronald L. Wasserstein & Nicole A. Lazar (2016) The ASA Statement on p-Values: Context, Process, and Purpose, The American Statistician, 70:2, 129-133, DOI:10.1080/00031305.2016.1154108

 

Yusuke Ono

JMP Japan Group

SAS Institute Japan