Now that the Tour de France is wrapping up, I found this paper on comparing the Tour de France and the Vuelta in terms of (how the authors define) difficulty. I like these types of studies. Here is one arguing the Dawn Wall in Yosemite is the hardest rock climb in the world. I need…
Category: Statistics
Is p-hacking acceptable for exploratory data analysis? (Part 1)
The problem of multiple comparison is well established in statistics. The multiple comparison problem occurs when many(!) hypothesis tests are performed on the same dataset. By doing so, the chance of a false positive result (e.g., finding a statistically significant result that is actually not) increases. For example, if performing 100 hypothesis tests each at…
Sample statistics vs population parameters
This post could be lifted from an introductory statistics text. But in my experience working with researchers the important distinction between a sample statistic and a population parameter gets a little muddled. A population parameter is an value that describes a population, such as the mean BMI of all American’s or the mean rent in…
Interaction effects require massive increase in sample size
I gave a quick internet-read to this blogpost by Andrew Gelman. The core of the idea is that to estimate an interaction effect a huge increase in sample size is needed compared to estimating a main effect. That is, if you power a study on main effects you could be very underpowered if you want…
P values are not transferable
Of the many challenges I encounter in communicating statistical concepts to collaborators, a common one is that p-values are seen as “transferable.” Here is a simple example to illustrate what I mean. Consider the following table, where the rows are sex (Male and Female) and the columns are smoking status (Current, Former, Never). We are…
Another improvement in reporting statistics in science
To add a little more to this post, I’ve read several papers where the authors add some details regarding the statistical methods/tests used with the results rather than just reporting a p-value. I think this approach really adds clarity. Here is an example. First, what I see as a typical approach: We found that the…
Brief example of interaction effects in regression
Statistical interactions can be challenging to understand and interpret. This simple example illustrates a regression model with an interaction effect. First I generate some fake data, which allows us to know the underlying structure of the data. Let y be some generic response variable (e.g., weight). Corresponding to each y is a sex variable (M/F)…
Report more than just the p-value
It is common in some academic fields to only report the p-value from a hypothesis test. For example, “We found a significant difference between the treatment group and control group (p=0.002)”. Some academic fields require including the test statistic value and other information, e.g., https://my.ilstu.edu/~jhkahn/apastats.html. As long as the hypothesis test was performed correctly reporting…
Box plots are not confidence intervals
The following plot is from page 622 from a paper by Philip, et al., 2019. I’m pretty sure that this plot is of standard box plots, which show the median (line inside of the box), the first and third quartiles (the lower and upper edges of the box) and some measure of outliers (the “whiskers”)….
Should medical students be doing research?
I posted here some thoughts on biomedical research from a newbie’s perspective. In my current role I work with many medical students and residents with varying levels of research (and statistics) experience. They are expected to produce publishable research products as part of their program while also training to be good physicians. That seems like…