Misunderestimating the significance of p-values

In (biological) science there is an expectation that any quantitative evidence should be accompanied with a statistical significance metric such as a p-value or error bars. These values are then deemed to be significant if they fall within certain confidence intervals. e.g. 99.9% confidence interval for a p-value of 0.01. I personally believe p-values should only be considered significant at extremely high confidence intervals with 99.9% a bare minimum, ideally 99.999% or higher. But this obviously depends on the experiment and sample size.

Recently there have been two excellent papers addressing the issue of p-value misuse. One late last year by Regina Nuzzo in PNAS [DOI] and the other this week by Valen Johnson in Nature [DOI]. Both paper discuss the affect the p-value has on data (and conclusion) reproducibility.

This is a difficult issue to address as the current p-value usage is so ingrained in the scientific community. I see the only effective way of addressing this is through journals requiring p-values be used correctly accompanied with all the data provided open access.

To preprint of not to preprint

There has been a lot of discussion around the openness of scientific publishing. Papers should be freely available to everyone upon publication so they reach the largest possible audience, disseminating science often funded by charitable bodies. But, as I am in the process of putting together my next paper, I am weighing up the pros and cons of uploading my finished manuscript to a preprint server bioRxiv before submitting to my journal of choice.

Pros: My paper gets out into the world and is citable immediately and I get feedback from comments to improve the paper (see cons below)

Cons: Preprint servers for biological papers are pretty new, so there is a developing community for commenting on the paper. So one of the main pros for preprinting, might not be currently viable. i.e. I don’t get any comments. I read a lot of papers on bioRxiv and very few have attracted comments so far. Not all journals accept papers previously submitted to a preprint server. This could reduce my choice of journals to submit to (although a lot of my journals of choice do accept preprinted papers, and more journals are updating their policies)

Conclusion: I like the principal of using preprint servers, and I will be using bioRxiv for my next paper. However, I may not immediately benefit from doing so. By using bioRxiv now, I hope to do my bit for open publishing and contribute to the bioRxiv community. In particular I think it’s important to contribute comments to the papers I read. What would be good to see is scientists being invited by bioRxiv to add comments to a paper.