{"id":209,"date":"2021-07-04T17:08:26","date_gmt":"2021-07-04T17:08:26","guid":{"rendered":"https:\/\/darrengemoets.me\/wp\/?p=209"},"modified":"2021-07-04T17:11:39","modified_gmt":"2021-07-04T17:11:39","slug":"p-values-are-not-transferable","status":"publish","type":"post","link":"https:\/\/darrengemoets.me\/wp\/2021\/07\/04\/p-values-are-not-transferable\/","title":{"rendered":"P values are not transferable"},"content":{"rendered":"

Of the many challenges I encounter in communicating statistical concepts to collaborators, a common one is that p-values are seen as \u201ctransferable.\u201d Here is a simple example to illustrate what I mean.<\/p>\n

Consider the following \"2 \times 3\" table, where the rows are sex (Male and Female) and the columns are smoking status (Current, Former, Never). We are interested in determining if the sex variable is independent of the smoking status variable. The counts are the number of observations in each category followed by relative frequencies.<\/p>\n

dat.synth1 <- matrix(c(55,45,70,50,50,70),nrow=2)\nrownames(dat.synth1) <- c(\"M\",\"F\")\ncolnames(dat.synth1) <- c(\"Current\",\"Former\",\"Never\")\ndat.synth1\n<\/code><\/pre>\n
##   Current Former Never\n## M      55     70    50\n## F      45     50    70\n<\/pre>\n
dat.synth1\/sum(dat.synth1)\n<\/code><\/pre>\n
##     Current    Former     Never\n## M 0.1617647 0.2058824 0.1470588\n## F 0.1323529 0.1470588 0.2058824\n<\/pre>\n
I perform a \"\chi^2\" Test of Independence, with p-value given below. I would conclude that the sex variable is not independent of the smoking variable.<\/p>\n
chisq.test(dat.synth1)\n<\/code><\/pre>\n
## \n##  Pearson's Chi-squared test\n## \n## data:  dat.synth1\n## X-squared = 7.3789, df = 2, p-value = 0.02499\n<\/pre>\n
This test for association does not \u201ctransfer\u201d to other hypotheses, e.g., marginal quantities. For example, in the sample 55 out of 175 are currently smokers and male (\"\approx\" 31.4 % and 45 out of 165 (\"\approx\" 27.3%) are current female smokers. The following would be incorrect to report: \u201cWe find that smoking status is not independent of sex (p=0.025), thus the proportion of males that are current smokers is different from the proportion of females that are current smokers.\u201d<\/p>\n
We can’t use the p value from the \"\chi^2\" Test of Independence for any \u201cpost-hoc\u201d type test on each proportion as done in the second part of the sentence. To assess the second claim we test if the proportion male current smokers is different from the proportion of females that are current smokers.<\/p>\n
prop.test(c(55,45),c(175,165))\n<\/code><\/pre>\n
## \n##  2-sample test for equality of proportions with continuity correction\n## \n## data:  c(55, 45) out of c(175, 165)\n## X-squared = 0.5205, df = 1, p-value = 0.4706\n## alternative hypothesis: two.sided\n## 95 percent confidence interval:\n##  -0.06101684  0.14413372\n## sample estimates:\n##    prop 1    prop 2 \n## 0.3142857 0.2727273\n<\/pre>\n
So we can’t conclude a difference in the two proportions in the population. The main issue is that hypotheses should be clearly established a priori, and not generated and tested using that data. Or put another way, performing many tests after seeing the data is p-hacking.<\/p>\n","protected":false},"excerpt":{"rendered":"
Of the many challenges I encounter in communicating statistical concepts to collaborators, a common one is that p-values are seen as \u201ctransferable.\u201d Here is a simple example to illustrate what I mean. Consider the following table, where the rows are sex (Male and Female) and the columns are smoking status (Current, Former, Never). We are…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[28,26],"class_list":["post-209","post","type-post","status-publish","format-standard","hentry","category-statistics","tag-chi-square","tag-pvalues"],"_links":{"self":[{"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/posts\/209","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/comments?post=209"}],"version-history":[{"count":4,"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/posts\/209\/revisions"}],"predecessor-version":[{"id":214,"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/posts\/209\/revisions\/214"}],"wp:attachment":[{"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/media?parent=209"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/categories?post=209"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/darrengemoets.me\/wp\/wp-json\/wp\/v2\/tags?post=209"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}